Currently unusable under Debian with sysv

Bug #1839487 reported by Sindarina
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
dkimpy-milter
Fix Released
High
Scott Kitterman

Bug Description

Both the 'dkimpy-milter' package as shipping with Debian Buster as well as the newer Python 3 version installed via pip are currently unusable on a sysv system, but also when started directly.

As in, '/usr/bin/dkimpy-milter /etc/dkimpy-milter.conf' starts, logs absolutely nothing, and just never backgrounds or anything. If killed via Ctrl-C it finally logs a 'started' message, which suggests that it gets stuck somewhere before it reaches that line;

https://git.launchpad.net/dkimpy-milter/tree/dkimpy_milter/__init__.py#n389

The sysv init script also seems woefully out of sync with the expectations of both the packaged version as well as the version that ships via pip/pypi, and there seems to be a general lack of error handling, leading to tracebacks left and right while trying to debug this problem, instead of properly caught errors that are returned via both syslog as well as stderr.

Is this actually used in production, or still in development?

Revision history for this message
Scott Kitterman (kitterman) wrote :

It is used in production, but not as far as I know with the sysv init script. Patches welcome for improving the sysv init as I don't have a good way to test it.

There is a bug here, but not what you'd expect. The line before that:

Milter.runmilter(miltername, socketname, 240)

starts the milter, so it's running, waiting for input via the milter protocol. The 'started' message is actually pointless.

It's not designed to background itselt, but to have the init system do it, so the fact that it remains in foreground is expected. For testing, it works just fine that way. You can configure your sendmail or postfix to talk to it and see if it works. If it does, then it's almost certainly an init script issue and I'd be glad to update that if you have specific feedback.

Revision history for this message
Sindarina (sindarina) wrote :

I don't mind improving the init script, but right now the biggest problem is that it just doesn't do the job; it simply doesn't work as a milter when integrated with Postfix, doesn't verify signatures, etc.

The fact that it doesn't log anything makes it rather difficult to discern whether it's actually in a working state, and attaching a strace to it shows nothing but polling and futex timeouts in what looks to be the communication between the process and its threads.

It's basically a black box, right now, and not at all behaving like you would expect from a service that's supposed to run in the background.

Revision history for this message
Sindarina (sindarina) wrote :

Right, so with 'debugLevel 3' in the configuration file, which isn't in there by default and was gleaned from the code, we are indeed getting some results, but not the desired ones.

As in, email sent by Google causes dkimpy-milter to crap out and throw tracebacks into syslog again. This is purely for incoming mail, in verify mode, no signing.

==
TypeError: a bytes-like object is required, not 'NoneType'
#012The above exception was the direct cause of the following exception:#012
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/Milter/__init__.py", line 772, in <lambda>#012
milter.set_eom_callback(lambda ctx: ctx.getpriv().eom())
File "/usr/local/lib/python3.7/dist-packages/dkimpy_milter/__init__.py", line 196, in eom#012 self.check_dkim(txt)
File "/usr/local/lib/python3.7/dist-packages/dkimpy_milter/__init__.py", line 292, in check_dkim#012 self.header_i = codecs.decode(d.signature_fields.get(b'i'), 'ascii')
TypeError: decoding with 'ascii' codec failed (TypeError: a bytes-like object is required, not 'NoneType')
==

Which results in Postfix throwing a transient error with '4.3.0 pymilter: untrapped exception in dkimpy-filter', and mail not being delivered.

That doesn't really say "ready for production", to me :-/

Revision history for this message
Scott Kitterman (kitterman) wrote :

OK, that's definitely a bug. i= is optional and the milter is crapping out because it's not there. Does this help:

diff --git a/dkimpy_milter/__init__.py b/dkimpy_milter/__init__.py
index 0440967..6052b85 100644
--- a/dkimpy_milter/__init__.py
+++ b/dkimpy_milter/__init__.py
@@ -289,7 +289,11 @@ class dkimMilter(Milter.Base):
                 self.dkim_comment = str(x)
                 if milterconfig.get('Syslog'):
                     syslog.syslog("check_dkim: {0}".format(x))
- self.header_i = codecs.decode(d.signature_fields.get(b'i'), 'ascii')
+ try:
+ # i= is optional and dkimpy is fine if it's not provided
+ self.header_i = codecs.decode(d.signature_fields.get(b'i'), 'ascii')
+ except TypeError as x:
+ pass
             self.header_d = codecs.decode(d.signature_fields.get(b'd'), 'ascii')
             self.header_a = codecs.decode(d.signature_fields.get(b'a'), 'ascii')
             if res:

I mostly use it for signing, so that's definitely better tested. I have tested verification with signatures from multiple implementations, so it works at least in general. Mostly I see i= included in the signature (even though it's optional).

Thanks for the feedback and being willing to work on this. Once we get things sorted, I'll be glad to cut a new release (and get updates into Debian Stable).

Revision history for this message
Scott Kitterman (kitterman) wrote :

That didn't work very well (indentation lost). I'll attach it as a file.

Revision history for this message
Scott Kitterman (kitterman) wrote :

This should work better.

Changed in dkimpy-milter:
status: New → Confirmed
importance: Undecided → High
assignee: nobody → Scott Kitterman (kitterman)
Revision history for this message
Sindarina (sindarina) wrote :

I'll have a look once I get around to DKIM verification and signing on our test MX, where we can break the configuration without any consequences. Have reverted to OpenDKIM for production, for now.

Revision history for this message
Scott Kitterman (kitterman) wrote : Re: [Bug 1839487] Re: Currently unusable under Debian with sysv

On Friday, August 9, 2019 11:12:52 AM EDT you wrote:
> I'll have a look once I get around to DKIM verification and signing on
> our test MX, where we can break the configuration without any
> consequences. Have reverted to OpenDKIM for production, for now.

Thanks,

FYI, for the long run, opendkim is unmaintained in Debian (and only minimally
so upstream).

Scott K

Revision history for this message
Sindarina (sindarina) wrote :

I got that impression, yes, which is part of why we wanted to move on to something else while we were rebuilding our relay infrastructure anyway. I had hoped that dkimpy-milter would indeed be a drop-in replacement, but it turns out that a bit more testing is required.

We'll move forward with the existing OpenDKIM setup, and then return to dkimpy-milter for 'out-of-band' testing elsewhere, without it blocking progress now.

Revision history for this message
Scott Kitterman (kitterman) wrote :

The missing i= part of this was reported as Bug #1842250 and I'm tracking it there. I'll publish a new release shortly. As far as the init script goes, I don't have a good way to test it, so any feedback you can give on what needs fixing would be helpful.

Revision history for this message
Scott Kitterman (kitterman) wrote :

I've tested in docker and can confirm your problems with the init. A fixed version will be in the next release (including 1.0 and 1.1 stable updates).

Changed in dkimpy-milter:
milestone: none → 1.2.0
status: Confirmed → Fix Committed
Revision history for this message
Scott Kitterman (kitterman) wrote :

Fixed in 1.1.3 and for the 1.0 series will be fixed in 1.0.2.

Changed in dkimpy-milter:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.