Merge dkimpy-milter:dkg/socket-activation into dkimpy-milter:master

Proposed by dkg
Status: Superseded
Proposed branch: dkimpy-milter:dkg/socket-activation
Merge into: dkimpy-milter:master
Prerequisite: dkimpy-milter:dkg/python3
Diff against target: 189 lines (+86/-14) (has conflicts)
7 files modified
dkimpy_milter/ (+12/-1)
dkimpy_milter/ (+2/-2)
dkimpy_milter/ (+16/-10)
man/dkimpy-milter.conf.5 (+1/-1)
system/socket-activation/ (+32/-0)
system/socket-activation/dkimpy-milter.service (+11/-0)
system/socket-activation/dkimpy-milter.socket (+12/-0)
Conflict in README
Conflict in TODO
Conflict in man/dkimpy-milter.conf.5
Reviewer Review Type Date Requested Status
Scott Kitterman Pending
Review via email:

This proposal has been superseded by a proposal from 2019-02-25.

Commit message

This series makes it possible to do socket activation. It is written on top of the earlier patch series to make it easier to consider merging in sequence (it is also quite close to several bugfixes)

To post a comment you must log in.

Unmerged commits

7092874... by dkg

Enable sd_listen_fds(3)-style socket-activation support

I've added straightforward systemd unit files in
system/socket-activation/ that make use of this approach, and a in the same location that describes the tradeoffs.

25fdd3b... by dkg

Do not create PidFile by default

By default, avoid creating a PIDFile.

PIDFiles are racy and potentially dangerous. Modern system
supervision systems don't need them, because they manage the process
groups directly.

If the configuration file doesn't specify a PidFile, dkimpy-milter
shouldn't try to create one.

9d5316c... by dkg

Handle defaults for Socket differently

We want to be able to select the default for Socket differently in the

This change augments the API for dkimpy_milter.util.own_socketfile()
by adding an optional sockname argument. This is a
backward-compatible change. If we aren't committed to API stability
for this function, we could make a more invasive change that would
probably be a more reasonable API going forward, but this is probably
good enough.

ea09bab... by dkg

Convert to python3

The main work here is about bytes vs. strings. This work was
confusing for several reasons:

 * pymilter thinks that headers are all strings, but body is bytes

 * dkimpy wants to deal with bytes objects generally (though it
   accepts a string object as an ed25519 secret key for some reason,
   despite requiring bytes as an RSA secret key)

 * authres.AuthenticationResultsHeader object converts easily to a
   string, but has no direct bytes conversion. meanwhile, it wants
   its arguments as strings, but will accept them if they are bytes
   and convert them with something like str(), which leaves weird
   cruft like "header.a=b'ed25519-sha256'"

 * dkimpy_milter/ contains fold() which expects bytes

 * self.fp needs to accumulate the on-the-wire version of the message
   as a whole (so it needs to be bytes). That means converting the
   headers. Header names and values are US-ASCII, per ยง2.2 of RFC
   5322, so they should be convertible cleanly, but we still have to
   convert them explicitly so that python knows the right thing to do.

At any rate, tests/runtests all passes with these changes, and the
output for both Authentication-Results: and DKIM-Signature headers
looks the same.

391b535... by dkg

Convert mostly to python3 (still need strings/bytes conversions)

This covers conversion of the whole project to python3, *except* for
the strings/bytes distinction in, which i'm leaving for a
second commit.

The changes in this commit are intended to be relatively
uncontroversial, so that the following commit contains the tricky

ad8f396... by dkg

Expand test suite to cover RSA as well as ed25519

479820a... by dkg

tests: test DKIM signing and verification

This test makes use of DNSOverride and the new verifying milter to
ensure that signatures can be verified properly.

It doesn't test the actual interaction with the public DNS, but
getting that kind of test to work on arbitrary platforms might be more
trouble than it's worth.

I note that the DNSOverride only works as long as testkey.dns is a
single line, which is fine for ed25519, but maybe not for RSA.

7bfb87f... by dkg

Set up, use it in tests

This allows us to invoke dkimpy-milter as "python -m dkimpy_milter
dkimpy-milter.conf", which makes running the test suite easier.

5c1d5d6... by dkg

tests: Run a verifying milter as well as a signing milter

Having a verifying milter will come in handy when we want to test both
sides of the DKIM process.

ae31730... by dkg

check for actions claimed by the filter

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/dkimpy_milter/ b/dkimpy_milter/
2index bcdf2a7..4c64891 100644
3--- a/dkimpy_milter/
4+++ b/dkimpy_milter/
5@@ -354,7 +354,18 @@ def main():
6 Milter.set_flags(Milter.CHGHDRS + Milter.ADDHDRS)
7 miltername = 'dkimpy-filter'
8 socketname = milterconfig.get('Socket')
9- own_socketfile(milterconfig)
10+ if socketname is None:
11+ if int(os.environ.get('LISTEN_PID', '0')) == os.getpid():
12+ lfds = os.environ.get('LISTEN_FDS')
13+ if lfds is not None:
14+ if lfds != '1':
15+ syslog.syslog('LISTEN_FDS is set to "{0}", but we only know how to deal with "1", ignoring it'.
16+ format(lfds))
17+ else:
18+ socketname = 'fd:3'
19+ if socketname is None:
20+ socketname = 'local:/var/run/dkimpy-milter/dkimpy-milter.sock'
21+ own_socketfile(milterconfig, socketname)
22 drop_privileges(milterconfig)
23 sys.stdout.flush()
24 Milter.runmilter(miltername, socketname, 240)
25diff --git a/dkimpy_milter/ b/dkimpy_milter/
26index bf6551a..3e2c736 100644
27--- a/dkimpy_milter/
28+++ b/dkimpy_milter/
29@@ -39,8 +39,8 @@ defaultConfigData = {
30 'SyslogFacility': 'mail',
31 'UMask': 0o07,
32 'Mode': 'sv',
33- 'Socket': 'local:/var/run/dkimpy-milter/dkimpy-milter.sock',
34- 'PidFile': '/var/run/dkimpy-milter/',
35+ 'Socket': None,
36+ 'PidFile': None,
37 'UserID': 'dkimpy-milter',
38 'Canonicalization': 'relaxed/simple',
39 'InternalHosts': '',
40diff --git a/dkimpy_milter/ b/dkimpy_milter/
41index 53d4c0d..bcdd11b 100644
42--- a/dkimpy_milter/
43+++ b/dkimpy_milter/
44@@ -115,43 +115,49 @@ def write_pid(milterconfig):
45 """Write PID in pidfile. Will not overwrite an existing file."""
46 import os
47 import syslog
48- if not os.path.isfile(milterconfig.get('PidFile')):
49+ pidfile = milterconfig.get('PidFile')
50+ if pidfile is None:
51+ return
52+ if not os.path.isfile(pidfile):
53 pid = str(os.getpid())
54 try:
55- f = open(milterconfig.get('PidFile'), 'w')
56+ f = open(pidfile, 'w')
57 except IOError as e:
58 if str(e)[:35] == '[Errno 2] No such file or directory':
59- piddir = milterconfig.get('PidFile').rsplit('/', 1)[0]
60+ piddir = pidfile.rsplit('/', 1)[0]
61 os.mkdir(piddir)
62 user, group = user_group(milterconfig.get('UserID'))
63 os.chown(piddir, user, group)
64- f = open(milterconfig.get('PidFile'), 'w')
65+ f = open(pidfile, 'w')
66 if milterconfig.get('Syslog'):
67 syslog.syslog('PID dir created: {0}'.format(piddir))
68 else:
69 if milterconfig.get('Syslog'):
70 syslog.syslog('Unable to write pidfle {0}. IOError: {1}'
71- .format(milterconfig.get('PidFile'), e))
72+ .format(pidfile, e))
73 raise
74 f.write(pid)
75 f.close()
76 user, group = user_group(milterconfig.get('UserID'))
77- os.chown(milterconfig.get('PidFile'), user, group)
78+ os.chown(pidfile, user, group)
79 else:
80 if milterconfig.get('Syslog'):
81 syslog.syslog('Unable to write pidfle {0}. File exists.'
82- .format(milterconfig.get('PidFile')))
83+ .format(pidfile))
84 raise RuntimeError('Unable to write pidfle {0}. File exists.'
85- .format(milterconfig.get('PidFile')))
86+ .format(pidfile))
87 return pid
90-def own_socketfile(milterconfig):
91+def own_socketfile(milterconfig, sockname=None):
92 """If socket is Unix socket, chown to UserID before dropping privileges"""
93 import os
94 user, group = user_group(milterconfig.get('UserID'))
95 offset = None
96- sockname = milterconfig.get('Socket')
97+ if sockname is None:
98+ sockname = milterconfig.get('Socket')
99+ if sockname is None:
100+ return
101 if sockname[:1] == '/':
102 offset = 0
103 elif sockname[:6] == "local:":
104diff --git a/man/dkimpy-milter.conf.5 b/man/dkimpy-milter.conf.5
105index d0662c4..ea07593 100644
106--- a/man/dkimpy-milter.conf.5
107+++ b/man/dkimpy-milter.conf.5
108@@ -355,7 +355,7 @@ will be checked. [PeerList NOT IMPLEMENTED - included for reference only]
109 .TP
110 .I PidFile (string)
111 Specifies the path to a file that should be created at process start
112-containing the process ID.
113+containing the process ID. If not specified, no such file will be created.
115 .TP
116 .I Selector (string)
117diff --git a/system/socket-activation/ b/system/socket-activation/
118new file mode 100644
119index 0000000..b4410ac
120--- /dev/null
121+++ b/system/socket-activation/
122@@ -0,0 +1,32 @@
123+This directory contains example systemd unit files for running a
124+supervised, socket-activated instance of dkimpy-milter.
126+There are several advantages of using socket activation:
128+- dkimpy-milter never runs with elevated privileges, they are dropped
129+ before any dkimpy-milter code is executed.
131+- The socket is opened before dkimpy-milter runs. This means that
132+ clients can connect() to the socket immediately. So even if there
133+ is a delay in dkimpy-milter startup, or in libmilter itself, the
134+ connection will not fail.
136+- You can set the privileges of a listening Unix-domain socket by an
137+ override of ListenGroup= in dkimpy-milter.socket (see
138+ systemd.unit(5) for how to override). This lets you control who has
139+ access to the daemon with finer granularity than is available with
140+ dkimpy-milter on its own.
142+- dkimpy-milter will not consume system resources if it is not used.
144+- A fully-supervised dkimpy-milter needs no PIDFile, UMask, UserID, or
145+ Socket configuation. This eliminates common race conditions and
146+ startup failures, and simplifies the resulting configuration file.
148+There is one downside to using socket activation:
150+- it will only work on systems where libmilter can support connection
151+ strings like "fd:3". This has been supported on Debian and derived
152+ systems since sendmail 8.14.4-6 (before Debian Jessie, in early
153+ 2014), see for example:
155diff --git a/system/socket-activation/dkimpy-milter.service b/system/socket-activation/dkimpy-milter.service
156new file mode 100644
157index 0000000..2116bb7
158--- /dev/null
159+++ b/system/socket-activation/dkimpy-milter.service
160@@ -0,0 +1,11 @@
162+Description=DKIMpy Milter
163+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
167+ExecStart=/usr/bin/dkimpy-milter /etc/dkimpy-milter.conf
172diff --git a/system/socket-activation/dkimpy-milter.socket b/system/socket-activation/dkimpy-milter.socket
173new file mode 100644
174index 0000000..d4338a3
175--- /dev/null
176+++ b/system/socket-activation/dkimpy-milter.socket
177@@ -0,0 +1,12 @@
179+Description=DKIMpy Milter socket
180+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
185+# override SocketGroup to grant access to members of another system group:


People subscribed via source and target branches

to all changes: