workaround for 8 bit headers containing invalid UTF-8

Bug #1980821 reported by Patrick Ben Koetter
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
dkimpy-milter
Confirmed
Medium
Unassigned

Bug Description

Pasting from an email conversation with Scott, so he won't forget this bug:

Scott: I think the correct answer is to use bytes, but the work-around you propose is reasonable and I am pretty sure I know how to implement it. Please file a bug on launchpad asking for the work-around so I don't forget.

Florian and Patrick:
In a nutshell, the problem is that python3 strings are utf-8, while email (technically) is 8 bit anything. Of course in headers not RFC compliant, but those mails are out there and they get delivered usually.

So the problem occurs, if there is an 8 bit header which is NOT valid UTF-8. In Germany for example ISO-85591 "Umlauts". Usually in the To or Subject header. See for example comment by " scandox commented on Jul 25, 2019" in issue https://github.com/sdgathman/pymilter/issues/12.

For our milters we uses the fix proposed by "william6502 commented on Jun 27, 2019" which changes the type to bytes in the lib, which of course implies you have to change the python code also to these types, but that was pretty straight forward and worked for us. We could change dkimpy-milter the same way I guess, but then it only works with a patched libmilter and therefore is nothing for general use.

The hotfix alternative, and thats what Patrick suggested I think, would be to catch the exception and just not process the mail, but not return an error. Then invalid mails would not be signed but still transported.

"sdgathman commented on Jun 19, 2020" suggests an other approach, but still seems to be uncertain if to proceed this way. As "our" problem was solved we did not reall look/push deeper there so far.

> I'd like to do some experimentation
Simply create a mail with an non-utf-8 8bit subject, feed it through your milter with swaks or so, and you should get the effect. For example this Subject header (base64 here...):U3ViamVjdDog3G1s5HV0cyBjcmFzaAo= should "work".

Revision history for this message
Scott Kitterman (kitterman) wrote :

With pymilter 1.0.5 this is much more tractable. Things that were crashing in the milter bindings now don't. With a few changes in the logging, we can get to "don't crash". Figuring out making signing/validation work is more complicated.

Revision history for this message
Scott Kitterman (kitterman) wrote :

I'm going to leave this open, since we're only up to not having a traceback for non-UTF-8 content in the headers.

Changed in dkimpy-milter:
importance: Undecided → Medium
milestone: none → future
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.