Merge ~lucaskanashiro/ubuntu/+source/clamav:focal-merge into ubuntu/+source/clamav:debian/sid

[Lucas Kanashiro]

Merge version 0.102.1+dfsg-2 from Debian. In this version, the maintainer did the following:

  * Add the clamonacc binary to the clamav-daemon package.
  * Drop ScanOnAccess option. The clamonacc provides this functionality.

Our delta kept the same:

    - clamav-daemon may fail to start due to options removed in new version
      and manually edited configuration file. (LP #1783632)
      + debian/patches/Deprecate-unused-options-instead-of-removing-it.patch:
        add patch from Debian stretch to simply warn about removed options.

It built fine in all architectures in my PPA:

https://launchpad.net/~lucaskanashiro/+archive/ubuntu/focal-clamav-merge-3.49.1-1

And the DEP-8 tests are also passing:

autopkgtest [14:03:53]: @@@@@@@@@@@@@@@@@@@@ summary
clamd PASS
client PASS
milter PASS

[Andreas Hasenack (ahasenack)]

Taking a look

[Andreas Hasenack (ahasenack)]

Can you find out what was the last ubuntu release which shipped an upstream clamav that still supported those options we are deprecating (instead of removing) via Deprecate-unused-options-instead-of-removing-it.patch? Just so we know when we can drop it.

review: Needs Information

[Andreas Hasenack (ahasenack)]

Also please try this clamav version with those options set, see what happens. If they work with a deprecation warning, or if things explode.

[Andreas Hasenack (ahasenack)]

The merge machinery itself is correct, so let's just see for how long we have to carry that patch still (or if we could even drop it already).

[Lucas Kanashiro (lucaskanashiro)]

> Can you find out what was the last ubuntu release which shipped an upstream
> clamav that still supported those options we are deprecating (instead of
> removing) via Deprecate-unused-options-instead-of-removing-it.patch? Just so
> we know when we can drop it.

According to the information in the bug report (https://bugs.launchpad.net/ubuntu/+source/clamav/+bug/1783632) those options were dropped by upstream in version 0.100.0. As you can see below, the version 0.99.4 was shipped in bionic but later it was fixed via security/updates. So if for some reason an user doesn't have security/updates enabled in their system, they were not warned about the removal of those features.

$ rmadison clamav
...
 clamav | 0.99.4+addedllvm-0ubuntu1 | bionic | source, amd64, arm64, armhf, i386, ppc64el, s390x
 clamav | 0.100.3+dfsg-0ubuntu0.14.04.1 | trusty-security | source, amd64, arm64, armhf, i386, powerpc, ppc64el
 clamav | 0.100.3+dfsg-0ubuntu0.14.04.1 | trusty-updates | source, amd64, arm64, armhf, i386, powerpc, ppc64el
 clamav | 0.100.3+dfsg-0ubuntu1 | disco | source, amd64, arm64, armhf, i386, ppc64el, s390x
 clamav | 0.101.4+dfsg-1ubuntu1 | eoan | source, amd64, arm64, armhf, i386, ppc64el, s390x
 clamav | 0.102.1+dfsg-0ubuntu0.16.04.2 | xenial-security | source, amd64, arm64, armhf, i386, powerpc, ppc64el, s390x
 clamav | 0.102.1+dfsg-0ubuntu0.16.04.2 | xenial-updates | source, amd64, arm64, armhf, i386, powerpc, ppc64el, s390x
 clamav | 0.102.1+dfsg-0ubuntu0.18.04.2 | bionic-security | source, amd64, arm64, armhf, i386, ppc64el, s390x
 clamav | 0.102.1+dfsg-0ubuntu0.18.04.2 | bionic-updates | source, amd64, arm64, armhf, i386, ppc64el, s390x
 clamav | 0.102.1+dfsg-0ubuntu0.19.04.2 | disco-security | source, amd64, arm64, armhf, i386, ppc64el, s390x
 clamav | 0.102.1+dfsg-0ubuntu0.19.04.2 | disco-updates | source, amd64, arm64, armhf, i386, ppc64el, s390x
 clamav | 0.102.1+dfsg-0ubuntu0.19.10.2 | eoan-security | source, amd64, arm64, armhf, i386, ppc64el, s390x
 clamav | 0.102.1+dfsg-0ubuntu0.19.10.2 | eoan-updates | source, amd64, arm64, armhf, i386, ppc64el, s390x
 clamav | 0.102.1+dfsg-1ubuntu1 | focal | source, amd64, arm64, armhf, ppc64el, s390x

[Andreas Hasenack (ahasenack)]

And what does it look like in the logs when you use one of these options?

[Lucas Kanashiro (lucaskanashiro)]

It looks like as expected. I tested only one option and it works and the user is warned about the deprecation. I added "StatsEnabled true" at the of /etc/clamav/clamd.conf and when the clamd is executed the user get this message:

WARNING: Ignoring deprecated option StatsEnabled at /etc/clamav/clamd.conf:88

[Andreas Hasenack (ahasenack)]

So this patch is already present in the bionic updates. I'm wondering if we can just drop it now.

@paelzer, you always have me check for when patches can be dropped, what do you think? It's debian/patches/Deprecate-unused-options-instead-of-removing-it.patch and we have it in bionic updates and security already, just not in bionic release. That should have been enough warning? Or should we keep it in focal?

[Andreas Hasenack (ahasenack)]

Sorry, added the comment before adding a review slot for Christian.

@paelzer, could you please take a look as to whether we can drop this delta, given my comment just above?

The impact is, if the user still has one of these options, clamd will fail to start, i.e., package upgrades would likely fail with a postinst error.

[Andreas Hasenack (ahasenack)]

Thanks, +1. Sponsoring fce6a19e0f2b19043381b6ca2cb8946b7a298d17

review: Approve

[Andreas Hasenack (ahasenack)]

$ git push pkg upload/0.102.1+dfsg-2ubuntu1
Enumerating objects: 25, done.
Counting objects: 100% (25/25), done.
Delta compression using up to 4 threads
Compressing objects: 100% (18/18), done.
Writing objects: 100% (19/19), 4.45 KiB | 227.00 KiB/s, done.
Total 19 (delta 13), reused 1 (delta 1)
To ssh://git.launchpad.net/~usd-import-team/ubuntu/+source/clamav
 * [new tag] upload/0.102.1+dfsg-2ubuntu1 -> upload/0.102.1+dfsg-2ubuntu1

$ dput ubuntu ../clamav_0.102.1+dfsg-2ubuntu1_source.changes
Checking signature on .changes
gpg: ../clamav_0.102.1+dfsg-2ubuntu1_source.changes: Valid signature from AC983EB5BF6BCBA9
Checking signature on .dsc
gpg: ../clamav_0.102.1+dfsg-2ubuntu1.dsc: Valid signature from AC983EB5BF6BCBA9
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading clamav_0.102.1+dfsg-2ubuntu1.dsc: done.
  Uploading clamav_0.102.1+dfsg.orig.tar.xz: done.
  Uploading clamav_0.102.1+dfsg-2ubuntu1.debian.tar.xz: done.
  Uploading clamav_0.102.1+dfsg-2ubuntu1_source.buildinfo: done.
  Uploading clamav_0.102.1+dfsg-2ubuntu1_source.changes: done.
Successfully uploaded packages.

[Andreas Hasenack (ahasenack)]

This migrated into focal release.

diff --git a/debian/changelog b/debian/changelog
index 85b6b14..edb7a61 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,13 @@
+clamav (0.102.1+dfsg-2ubuntu1) focal; urgency=medium
+
+  * Merge with Debian unstable. Remaining changes:
+    - clamav-daemon may fail to start due to options removed in new version
+      and manually edited configuration file. (LP #1783632)
+      + debian/patches/Deprecate-unused-options-instead-of-removing-it.patch:
+        add patch from Debian stretch to simply warn about removed options.
+
+ -- Lucas Kanashiro <[e-mail]>  Wed, 22 Jan 2020 09:37:47 -0300
+
 clamav (0.102.1+dfsg-2) unstable; urgency=medium
 
   * Add the clamonacc binary to the clamav-daemon package.
@@ -5,6 +15,16 @@ clamav (0.102.1+dfsg-2) unstable; urgency=medium
 
  -- Sebastian Andrzej Siewior <[e-mail]>  Mon, 23 Dec 2019 20:54:21 +0100
 
+clamav (0.102.1+dfsg-1ubuntu1) focal; urgency=medium
+
+  * Merge with Debian unstable. Remaining changes:
+    - clamav-daemon may fail to start due to options removed in new version
+      and manually edited configuration file. (LP #1783632)
+      + debian/patches/Deprecate-unused-options-instead-of-removing-it.patch:
+        add patch from Debian stretch to simply warn about removed options.
+
+ -- Lucas Kanashiro <[e-mail]>  Thu, 05 Dec 2019 18:01:14 -0300
+
 clamav (0.102.1+dfsg-1) unstable; urgency=medium
 
   * Import 0.102.1 (Closes: #945265)
@@ -16,6 +36,16 @@ clamav (0.102.1+dfsg-1) unstable; urgency=medium
 
  -- Sebastian Andrzej Siewior <[e-mail]>  Sat, 30 Nov 2019 19:22:15 +0100
 
+clamav (0.101.4+dfsg-1ubuntu1) eoan; urgency=low
+
+  * Merge from Debian unstable.  Remaining changes:
+    - clamav-daemon may fail to start due to options removed in new version
+      and manually edited configuration file. (LP #1783632)
+      + debian/patches/Deprecate-unused-options-instead-of-removing-it.patch:
+        add patch from Debian stretch to simply warn about removed options.
+
+ -- Gianfranco Costamagna <[e-mail]>  Sun, 25 Aug 2019 23:25:27 +0200
+
 clamav (0.101.4+dfsg-1) unstable; urgency=medium
 
   * Import 0.101.4
@@ -27,6 +57,16 @@ clamav (0.101.4+dfsg-1) unstable; urgency=medium
 
  -- Sebastian Andrzej Siewior <[e-mail]>  Sun, 25 Aug 2019 12:38:25 +0200
 
+clamav (0.101.2+dfsg-3ubuntu1) eoan; urgency=low
+
+  * Merge from Debian unstable.  Remaining changes:
+    - clamav-daemon may fail to start due to options removed in new version
+      and manually edited configuration file. (LP #1783632)
+      + debian/patches/Deprecate-unused-options-instead-of-removing-it.patch:
+        add patch from Debian stretch to simply warn about removed options.
+
+ -- Gianfranco Costamagna <[e-mail]>  Wed, 07 Aug 2019 08:54:47 +0200
+
 clamav (0.101.2+dfsg-3) unstable; urgency=medium
 
   * Cherry-pick a fix from 0.101.3 to address a vulnerability to
@@ -34,6 +74,16 @@ clamav (0.101.2+dfsg-3) unstable; urgency=medium
 
  -- Sebastian Andrzej Siewior <[e-mail]>  Tue, 06 Aug 2019 21:42:06 +0200
 
+clamav (0.101.2+dfsg-2ubuntu1) eoan; urgency=low
+
+  * Merge from Debian unstable.  Remaining changes:
+    - clamav-daemon may fail to start due to options removed in new version
+      and manually edited configuration file. (LP #1783632)
+      + debian/patches/Deprecate-unused-options-instead-of-removing-it.patch:
+        add patch from Debian stretch to simply warn about removed options.
+
+ -- Gianfranco Costamagna <[e-mail]>  Sat, 03 Aug 2019 10:20:31 +0200
+
 clamav (0.101.2+dfsg-2) unstable; urgency=medium
 
   * Remove python from build-depends:
@@ -43,6 +93,22 @@ clamav (0.101.2+dfsg-2) unstable; urgency=medium
 
  -- Scott Kitterman <[e-mail]>  Fri, 02 Aug 2019 09:20:43 -0400
 
+clamav (0.101.2+dfsg-1ubuntu2) eoan; urgency=medium
+
+  * Rebuild against new libjson-c4.
+
+ -- Gianfranco Costamagna <[e-mail]>  Sat, 29 Jun 2019 13:48:24 +0200
+
+clamav (0.101.2+dfsg-1ubuntu1) eoan; urgency=medium
+
+  * Sync with Debian. Remaining change:
+    - clamav-daemon may fail to start due to options removed in new version
+      and manually edited configuration file. (LP #1783632)
+      + debian/patches/Deprecate-unused-options-instead-of-removing-it.patch:
+        add patch from Debian stretch to simply warn about removed options.
+
+ -- Marc Deslauriers <[e-mail]>  Tue, 23 Apr 2019 11:40:41 -0400
+
 clamav (0.101.2+dfsg-1) unstable; urgency=high
 
   * Import 0.101.2
diff --git a/debian/control b/debian/control
index a635cf6..1e1cd53 100644
--- a/debian/control
+++ b/debian/control
@@ -1,7 +1,8 @@
 Source: clamav
 Section: utils
 Priority: optional
-Maintainer: ClamAV Team <[e-mail]>
+Maintainer: Ubuntu Developers <[e-mail]>
+XSBC-Original-Maintainer: ClamAV Team <[e-mail]>
 Uploaders: Michael Meskes <[e-mail]>,
            Michael Tautschnig <[e-mail]>,
            Scott Kitterman <[e-mail]>,
diff --git a/debian/patches/Deprecate-unused-options-instead-of-removing-it.patch b/debian/patches/Deprecate-unused-options-instead-of-removing-it.patch
new file mode 100644
index 0000000..8303c07
--- /dev/null
+++ b/debian/patches/Deprecate-unused-options-instead-of-removing-it.patch
@@ -0,0 +1,56 @@
+Description: Deprecate unused options instead of removing it
+ Refresh Debian patch from Sebastian Andrzej Siewior <[e-mail]>,
+ original description below:
+ .
+ The following options were removed:
+ - StatsHostID
+ - StatsEnabled
+ - StatsPEDisabled
+ - StatsTimeout
+ - SubmitDetectionStats
+ - DetectionStatsCountry
+ - DetectionStatsHostID
+ .
+ and if they remain in the config file (during automatic upgrade without
+ user action which would be required by ucf) then clamav will refuse the
+ start. By marking them as deprecated clamav will point it out and
+ continue. This patch will be dropped by moving to next Debian stable
+ version (the user is expected to edit the config manually at this
+ point if it did not happen earlier).
+ .
+ NOTE: this patch should be droppped when Ubuntu 18.04 is EOL.
+Author: Sebastian Andrzej Siewior <[e-mail]>
+Origin: vendor, https://salsa.debian.org/clamav-team/clamav/commit/66e2eb1b356939b83369dd8cf21f06f2f3abb234
+Bug: https://bugs.debian.org/902290
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1783632
+Reviewed-by: Lucas Kanashiro <[e-mail]>
+Last-Update: 2019-12-05
+---
+This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
+
+--- a/shared/optparser.c
++++ b/shared/optparser.c
+@@ -587,6 +587,23 @@
+     {"TCPSocket", NULL, 0, CLOPT_TYPE_NUMBER, MATCH_NUMBER, -1, NULL, 0, OPT_MILTER | OPT_DEPRECATED, "", ""},
+     {"TemporaryDirectory", NULL, 0, CLOPT_TYPE_STRING, NULL, -1, NULL, 0, OPT_MILTER | OPT_DEPRECATED, "", ""},
+ 
++    /* Deprecated stats options - avoid breakage during upgrades from old Ubuntu releases */
++
++    { "StatsHostID", "stats-host-id", 0, CLOPT_TYPE_STRING, NULL, -1, NULL, 0, OPT_FRESHCLAM | OPT_CLAMD | OPT_CLAMSCAN | OPT_DEPRECATED, "HostID in the form of an UUID to use when submitting statistical information. See the clamscan manpage for more information.", "default" },
++
++    { "StatsEnabled", "enable-stats", 0, CLOPT_TYPE_BOOL, MATCH_BOOL, 0, NULL, 0, OPT_CLAMD | OPT_FRESHCLAM | OPT_CLAMSCAN | OPT_DEPRECATED, "Enable submission of statistical data", "yes" },
++
++    { "StatsPEDisabled", "disable-pe-stats", 0, CLOPT_TYPE_BOOL, MATCH_BOOL, 0, NULL, 0, OPT_CLAMD | OPT_CLAMSCAN | OPT_DEPRECATED, "Disable submission of PE section statistical data", "no" },
++
++    { "StatsTimeout", "stats-timeout", 0, CLOPT_TYPE_NUMBER, MATCH_NUMBER, -1, NULL, 0, OPT_CLAMD | OPT_CLAMSCAN | OPT_FRESHCLAM | OPT_DEPRECATED, "Timeout in seconds to timeout communication with the stats server.", "10" },
++
++    { "SubmitDetectionStats", NULL, 0, CLOPT_TYPE_STRING, NULL, -1, NULL, 0, OPT_FRESHCLAM | OPT_DEPRECATED, "When enabled freshclam will submit statistics to the ClamAV Project about\nthe latest virus detections in your environment. The ClamAV maintainers\nwill then use this data to determine what types of malware are the most\ndetected in the field and in what geographic area they are.\nFreshclam will connect to clamd in order to get recent statistics.", "/path/to/clamd.conf" },
++
++    { "DetectionStatsCountry", NULL, 0, CLOPT_TYPE_STRING, NULL, -1, NULL, 0, OPT_FRESHCLAM | OPT_DEPRECATED, "Country of origin of malware/detection statistics (for statistical\npurposes only). The statistics collector at ClamAV.net will look up\nyour IP address to determine the geographical origin of the malware\nreported by your installation. If this installation is mainly used to\nscan data which comes from a different location, please enable this\noption and enter a two-letter code (see http://www.iana.org/domains/root/db/)\nof the country of origin.", "country-code" },
++
++    { "DetectionStatsHostID", NULL, 0, CLOPT_TYPE_STRING, NULL, -1, NULL, 0, OPT_FRESHCLAM | OPT_DEPRECATED, "This option enables support for our \"Personal Statistics\" service.\nWhen this option is enabled, the information on malware detected by\nyour clamd installation is made available to you through our website.\nTo get your HostID, log on http://www.stats.clamav.net and add a new\nhost to your host list. Once you have the HostID, uncomment this option\nand paste the HostID here. As soon as your freshclam starts submitting\ninformation to our stats collecting service, you will be able to view\nthe statistics of this clamd installation by logging into\nhttp://www.stats.clamav.net with the same credentials you used to\ngenerate the HostID. For more information refer to:\nhttp://www.clamav.net/doc/cctts.html\nThis feature requires SubmitDetectionStats to be enabled.", "unique-id" },
++
++
+     {NULL, NULL, 0, 0, NULL, 0, NULL, 0, 0, NULL, NULL}};
+ const struct clam_option *clam_options = __clam_options;
+ 
diff --git a/debian/patches/series b/debian/patches/series
index 28004d0..1e4a08e 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -4,3 +4,4 @@ clamd_dont_depend_on_clamav_demon_socket.patch
 Add-support-for-LLVM-3.7.patch
 Add-support-for-LLVM-3.8.patch
 Add-support-for-LLVM-3.9.patch
+Deprecate-unused-options-instead-of-removing-it.patch