Syslog spammed with "domain is utf8 flagged"

Bug #1862154 reported by Paul
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
spamassassin (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Fix Released
Medium
Unassigned
Bionic
Fix Released
Medium
Unassigned
Eoan
Fix Released
Medium
Unassigned

Bug Description

[Impact]

 * An issue fixed upstream in later releases can cause issues with UTF
   emails

 * backport the upstream fix to stable supported releases to avoid log
   flooding with related error messages.

[Test Case]

 * using a test mail attached on this bug:
   $ apt install spamassassin
   $ systemctl start spamassassin
   $ wget https://bugs.launchpad.net/ubuntu/+source/spamassassin/+bug/1862154/+attachment/5326196/+files/bug1862154.eml
   $ spamc -R < bug1862154.eml
   $ grep "domain is utf8 flagged" /var/log/mail.log

If still affected this will add some lines to the log, otherwise it won't.

[Regression Potential]

 * The change is fortunately minimal (one line) and only changes the
   output severity to not pop up in the logs.
   The one risk I can think of is people expecting this output, but I
   can't think of a useful real case for this.

[Other Info]

 * n/a

---

Seems that https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7632 has been re-introduced.

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.3 LTS
Release: 18.04
Codename: bionic

# apt-cache policy spamassassin
spamassassin:
  Installed: 3.4.2-0ubuntu0.18.04.3
  Candidate: 3.4.2-0ubuntu0.18.04.3

I am seeing *lots* of info logs for every email:

Feb 6 07:30:26 spam02a spamd[11370]: dns: new_dns_packet: domain is utf8 flagged: grace.ns.cloudflare.com
Feb 6 07:30:26 spam02a spamd[11370]: dns: new_dns_packet: domain is utf8 flagged: sid.ns.cloudflare.com
Feb 6 07:30:26 spam02a spamd[11370]: dns: new_dns_packet: domain is utf8 flagged: ns1.silverpop.com

SpamAssassin/DnsResolver.pm - Line 549

---8<---
    if (utf8::is_utf8($domain)) { # since Perl 5.8.1
      info("dns: new_dns_packet: domain is utf8 flagged: %s", $domain);
    }
---8<---

This should be a dbg flag - this is fixed in Spamassassin 3.4.4 upstream - could we have a backport for this current version of 3.4.2?

Related branches

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

$ git show pkg/ubuntu/bionic-devel:lib/Mail/SpamAssassin/DnsResolver.pm | grep "domain is utf8 flagged"
      info("dns: new_dns_packet: domain is utf8 flagged: %s", $domain);

$ git show pkg/ubuntu/eoan-devel:lib/Mail/SpamAssassin/DnsResolver.pm | grep "domain is utf8 flagged"
      info("dns: new_dns_packet: domain is utf8 flagged: %s", $domain);

$ git show pkg/ubuntu/focal-devel:lib/Mail/SpamAssassin/DnsResolver.pm | grep "domain is utf8 flagged"
      dbg("dns: new_dns_packet: domain is utf8 flagged: %s", $domain);

Upstream change is:
https://github.com/apache/spamassassin/commit/bdacc776ccbfdedee7f1e8175a333d9e419d4426

Changed in spamassassin (Ubuntu):
status: New → Fix Released
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Introduced in pkg/import/3.4.0-1 as part of the upstream code.

@Paul I'm not seeing this was ever fixed before since you called it "re-introduced".
Did we have a version in Ubuntu that was affected, then a fix and then re-introduced the error?
If so please let me know.

For example we have Bionic starting with 3.4.1-8build1 up to todays 3.4.2-0ubuntu0.18.04.3.
Nothing in between changes any line that has "domain is utf8 flagged".

Never the less I think the upstream fix can be trivially applied and SRUed.
The one piece that is missing for an SRU [1] are minimal steps to reproduce on a clean system.

@Paul - since you have a setup that is affected, would you be able to provide steps here for install/config/commands to trigger this issue? That would help a lot getting this processed.

[1]: https://wiki.ubuntu.com/StableReleaseUpdates

Changed in spamassassin (Ubuntu Xenial):
status: New → Triaged
Changed in spamassassin (Ubuntu Bionic):
status: New → Triaged
Changed in spamassassin (Ubuntu Eoan):
status: New → Triaged
tags: added: bitesize
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Triage Note: this becomes server-next (to drive the SRU) once we have steps to reproduce

Changed in spamassassin (Ubuntu Xenial):
importance: Undecided → Medium
Changed in spamassassin (Ubuntu Bionic):
importance: Undecided → Medium
Changed in spamassassin (Ubuntu Eoan):
importance: Undecided → Medium
Revision history for this message
Paul (steadramon) wrote :

Sorry - mistyped when I said "reintroduced" - can confirm this is present in previous versions, as per upstream releases.

Steps to reproduce on a clean system -

# lsb_release

# sudo apt-get install spamassassin
# sudo service spamassassin start

Locate the attached email example on the system, then run spamc against it -

# spamc -R < /tmp/bug1862154.eml
# grep "domain is utf8 flagged" /var/log/mail.log
Feb 7 11:12:19 ubuntu spamd[31432]: dns: new_dns_packet: domain is utf8 flagged: ns1.canonical.com
Feb 7 11:12:19 ubuntu spamd[31432]: dns: new_dns_packet: domain is utf8 flagged: ns2.canonical.com
Feb 7 11:12:19 ubuntu spamd[31432]: dns: new_dns_packet: domain is utf8 flagged: ns3.canonical.com

Revision history for this message
Paul (steadramon) wrote :

lsb_release output -

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.3 LTS
Release: 18.04
Codename: bionic

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thanks Paul, added an SRU template based on this.
Will prepare PPAs and MPs now ...

description: updated
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

MPs approved - Uploaded for acceptance by the SRU Team

Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Paul, or anyone else affected,

Accepted spamassassin into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/spamassassin/3.4.2-1ubuntu0.19.10.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in spamassassin (Ubuntu Eoan):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-eoan
Changed in spamassassin (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Paul, or anyone else affected,

Accepted spamassassin into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/spamassassin/3.4.2-0ubuntu0.18.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Paul, or anyone else affected,

Accepted spamassassin into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/spamassassin/3.4.2-0ubuntu0.16.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in spamassassin (Ubuntu Xenial):
status: Triaged → Fix Committed
tags: added: verification-needed-xenial
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Installed 3.4.2-0ubuntu0.16.04.4 3.4.2-0ubuntu0.18.04.4 and 3.4.2-1ubuntu0.19.10.3 without issues.

Ran them through the test and they all no more expose the behavior to report "domain is utf8 flagged:".
Info for others that re-test this, you have to re-start the spamassassin service to pick up the new code.

Setting verification-done

tags: added: verification-done verification-done-bionic verification-done-eoan verification-done-xenial
removed: verification-needed verification-needed-bionic verification-needed-eoan verification-needed-xenial
Revision history for this message
Paul (steadramon) wrote :

Installed 3.4.2-0ubuntu0.18.04.4 via apt-get using the proposed repository.

Can confirm that the update has removed the info based "domain is utf8 flagged" messages.

Thanks for your time and help with this.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package spamassassin - 3.4.2-1ubuntu0.19.10.3

---------------
spamassassin (3.4.2-1ubuntu0.19.10.3) eoan; urgency=medium

  * d/p/lp-1862154-*: avoid 'domain is utf8 flagged' flooding the logs
    (LP: #1862154)

 -- Christian Ehrhardt <email address hidden> Mon, 10 Feb 2020 16:17:02 +0100

Changed in spamassassin (Ubuntu Eoan):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for spamassassin has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package spamassassin - 3.4.2-0ubuntu0.18.04.4

---------------
spamassassin (3.4.2-0ubuntu0.18.04.4) bionic; urgency=medium

  * d/p/lp-1862154-*: avoid 'domain is utf8 flagged' flooding the logs
    (LP: #1862154)

 -- Christian Ehrhardt <email address hidden> Mon, 10 Feb 2020 16:17:02 +0100

Changed in spamassassin (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package spamassassin - 3.4.2-0ubuntu0.16.04.4

---------------
spamassassin (3.4.2-0ubuntu0.16.04.4) xenial; urgency=medium

  * d/p/lp-1862154-*: avoid 'domain is utf8 flagged' flooding the logs
    (LP: #1862154)

 -- Christian Ehrhardt <email address hidden> Mon, 10 Feb 2020 16:17:02 +0100

Changed in spamassassin (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.