Merge ~ahasenack/ubuntu/+source/clamav:bionic-clamav-merge-0.99.3beta2-2u1 into ~usd-import-team/ubuntu/+source/clamav:debian/sid

[Andreas Hasenack]

Merge from debian's 0.99.3~beta1+dfsg-2.

Dropped security patches (already applied) and updated the DEP3 header of debian/patches/fix_newer_zlib.patch to indicate that it was merged upstream, so some future merge can likely drop it. I added this node to d/changelog, but I wasn't sure I should have. It's not something we would want to keep repeating in d/changelog in the case this patch isn't dropped in the next merge.

I tested in a bionic container running clam[d]scan in all files in /usr/share/clamav-testfiles from the clamav-testfiles package.

There is a small issue that is coming from debian and that we have been carrying already in the extend.conf systemd extra file we create in postinst:
# cat /etc/systemd/system/clamav-daemon.service.d/extend.conf
[Service]
ExecStartPre=-/bin/mkdir /run/clamav
ExecStartPre=/bin/chown clamav /run/clamav

That mkdir may fail if the directory already exists, that's why that command is prefixed with "-". I think a cleaner approach would have been to also add -p to mkdir. Then we wouldn't see that FAILURE bit in one of the ExecStartPre lines:

root@bionic-clamav-test:~# systemctl restart clamav-daemon
root@bionic-clamav-test:~# systemctl status clamav-daemon
● clamav-daemon.service - Clam AntiVirus userspace daemon
   Loaded: loaded (/lib/systemd/system/clamav-daemon.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/clamav-daemon.service.d
           └─extend.conf
   Active: active (running) since Wed 2017-11-22 21:53:20 UTC; 3s ago
     Docs: man:clamd(8)
           man:clamd.conf(5)
           http://www.clamav.net/lang/en/doc/
  Process: 21515 ExecStartPre=/bin/chown clamav /run/clamav (code=exited, status=0/SUCCESS)
  Process: 21514 ExecStartPre=/bin/mkdir /run/clamav (code=exited, status=1/FAILURE)
 Main PID: 21516 (clamd)
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/clamav-daemon.service
           └─21516 /usr/sbin/clamd --foreground=true

Nov 22 21:53:20 bionic-clamav-test systemd[1]: Starting Clam AntiVirus userspace daemon...
Nov 22 21:53:20 bionic-clamav-test systemd[1]: Started Clam AntiVirus userspace daemon.

f301d06... by Andreas Hasenack on 2017-11-22

merge-changelogs

5281f64... by Andreas Hasenack on 2017-11-22

reconstruct-changelog

bba0cd6... by Andreas Hasenack on 2017-11-22

update-maintainer

[ChristianEhrhardt (paelzer)]

Hi,
FYI I found this by checking from the Trello card.
Since it is lacking a review request from ~canonical-server it is missing on [1] where I think we would usually look for it.

Putting it onto my TODO List.
I have a few things I need to prep and submit to run tests on then I hope to get to it.
If you get online today and this is not done yet please remind me.

[1]: https://code.launchpad.net/~canonical-server/+activereviews

[ChristianEhrhardt (paelzer)]

Arrr well all my tasks suck atm - damn you bugs :-)
So lets unblock you at least.

General linting via git ubuntu lint - ok
Building .changes - ok
dput linter - only stuff you didn't cause (in Debian and nto too bad)

Test from ppa: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3050
Build cross arches - ok

Carried changes: ok - thanks for the header update
 - the llvm one is a bit odd afaik upstream only goes to 3.6, but for dependencies we want 3.9
   I had related issues 1717574, but I think it is more for dependencies than for testcases
   Except you have better info that points you to testcases?
Dropped changes: ok - all in upstream already

Test - TBD

[ChristianEhrhardt (paelzer)]

Build cross arch (I had formerly built the wrong changes file): Ok
Test: unsure

Note: ppa at https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3050

Out of some older bugs I new that some (black and very dark) magic happens when freshclam is complete and clamav needs to compile the bytecode. But even that worked for me on the tests.

It successfully kicked of the service and other things.
Also on the box without network it correctly waited on the condition to have a virus database.

But - I happened to see an error I'm not sure it is new with the new version.
Nov 23 09:22:37 bionic-test freshclam[3708]: bytecode.cvd updated (version: 318, sigs: 75, f-level: 63, builder: raynman)
Nov 23 09:22:41 bionic-test freshclam[3708]: Database updated (6353848 signatures) from db.local.clamav.net (IP: 5.9.38.226)
Nov 23 09:22:41 bionic-test freshclam[3708]: ERROR: NotifyClamd: Can't find or parse configuration file /etc/clamav/clamd.conf

That file existed later and a restart of the service no more complained.

I found the file really does not exist if you only install freshclam.
And on any parsing of new files we get the error (which means it goes on with defaults).

I found the file is installed by clamav-daemon (generated in postinst according to the comment in the file).

It does not seem to break anything - except freshclam running on defaults unless clamav-daemon is installed.
Is that a BUG - I'm not sure, but one should take a look at it.

Steps to reproduce:
$ apt install clamav-freshclam
# wait until the files are downloaded and compiled (some random wait + download time)
$ systemctl status clamav-freshclam
# Instead you can stop the service and just call sudo freshclam (no config)

I checked the pre-merge version and the issue is present there as well.
I found the bug on Arch, RH, Fedora, Ubuntu, Debian e.g. [1]
ALso keep in mind that this config is meant to make freshclam NOTIFY clamd (which is not installed) so all is ok.
Maybe the message shold be no error, but ok for now.

Much noise and time spent, but the merge is good - +1

Waiting for your ping-ack to actually sponsor it into Debian.

[1]: https://askubuntu.com/questions/589318/freshclam-error-clamd-conf-file-not-found/632911

review: Approve

[Andreas Hasenack (ahasenack)]

Yeah, the daemon can only run once the databases are downloaded. And after they are downloaded by freshclam, it tries to notify the daemon, but fails because it's not running. So there is a bit of hand holding that has to happen for fresh installations. It works, but is not super friendly on a fresh install.

Please go ahead with the sponsoring, thanks!

diff --git a/debian/changelog b/debian/changelog
index 7d7970c..cd85c2e 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,30 @@
+clamav (0.99.3~beta1+dfsg-2ubuntu1) bionic; urgency=medium
+
+  * Merge with Debian unstable (LP: #1732439). Remaining changes:
+    - Fix build by forcing llvm 3.9
+      (testsuite seems to be failing)
+    - debian/patches/fix_newer_zlib.patch: fix compatibility with zlib
+      1.2.9 and newer (LP #1692073).
+      [DEP3 header updated to indicate it was incorporated upstream]
+  * Drop:
+    * debian/patches/zlib-check.patch:
+      + cherry-pick upstream fix for wrong zlib version check
+        [Fixed upstream]
+    - SECURITY UPDATE: DoS via crafted e-mail message
+      + debian/patches/CVE-2017-6418.patch: fix invalid read in
+        libclamav/message.c.
+      + CVE-2017-6418
+        [Fixed upstream]
+    - SECURITY UPDATE: DoS via WWPack compression
+      + debian/patches/CVE-2017-6420.patch: add bounds checks to
+        libclamav/wwunpack.c.
+      + debian/patches/CVE-2017-6420-2.patch: fix unit tests in
+        libclamav/wwunpack.c, unit_tests/check_jsnorm.c.
+      + CVE-2017-6420
+        [Fixed upstream]
+
+ -- Andreas Hasenack <[e-mail]>  Wed, 22 Nov 2017 19:56:26 -0200
+
 clamav (0.99.3~beta1+dfsg-2) unstable; urgency=medium
 
   * Build again against system's libmspack (dropped by accident)
@@ -40,6 +67,38 @@ clamav (0.99.3~snapshot20170704+dfsg-1) experimental; urgency=medium
 
  -- Sebastian Andrzej Siewior <[e-mail]>  Mon, 10 Jul 2017 00:06:32 +0200
 
+clamav (0.99.2+dfsg-6ubuntu2) artful; urgency=medium
+
+  * SECURITY UPDATE: DoS via crafted e-mail message
+    - debian/patches/CVE-2017-6418.patch: fix invalid read in
+      libclamav/message.c.
+    - CVE-2017-6418
+  * SECURITY UPDATE: DoS via WWPack compression
+    - debian/patches/CVE-2017-6420.patch: add bounds checks to
+      libclamav/wwunpack.c.
+    - debian/patches/CVE-2017-6420-2.patch: fix unit tests in
+      libclamav/wwunpack.c, unit_tests/check_jsnorm.c.
+    - CVE-2017-6420
+  * debian/patches/fix_newer_zlib.patch: fix compatibility with zlib
+    1.2.9 and newer (LP: #1692073).
+
+ -- Marc Deslauriers <[e-mail]>  Tue, 15 Aug 2017 16:04:46 -0400
+
+clamav (0.99.2+dfsg-6ubuntu1) artful; urgency=medium
+
+  * Fix build by forcing llvm 3.9
+    (testsuite seems to be failing)
+  * debian/patches/zlib-check.patch:
+    - cherry-pick upstream fix for wrong zlib version check
+
+ -- Gianfranco Costamagna <[e-mail]>  Tue, 02 May 2017 11:18:30 +0200
+
+clamav (0.99.2+dfsg-6build1) artful; urgency=medium
+
+  * Rebuild against new llvm versioned symbols.
+
+ -- Gianfranco Costamagna <[e-mail]>  Mon, 01 May 2017 22:33:23 +0200
+
 clamav (0.99.2+dfsg-6) unstable; urgency=medium
 
   * Fix detection of curl. Patch by Reiner Herrmann <[e-mail]>
diff --git a/debian/control b/debian/control
index 481b0b2..863febc 100644
--- a/debian/control
+++ b/debian/control
@@ -1,7 +1,8 @@
 Source: clamav
 Section: utils
 Priority: optional
-Maintainer: ClamAV Team <[e-mail]>
+Maintainer: Ubuntu Developers <[e-mail]>
+XSBC-Original-Maintainer: ClamAV Team <[e-mail]>
 Uploaders: Michael Meskes <[e-mail]>,
            Michael Tautschnig <[e-mail]>,
            Scott Kitterman <[e-mail]>,
@@ -26,7 +27,7 @@ Build-Depends: automake,
                libsystemd-dev [linux-any],
                libtfm-dev,
                libxml2-dev,
-               llvm-dev [i386 amd64 kfreebsd-amd64 kfreebsd-i386],
+               llvm-3.9-dev [i386 amd64 kfreebsd-amd64 kfreebsd-i386],
                perl:native,
                pkg-config,
                po-debconf,
diff --git a/debian/patches/fix_newer_zlib.patch b/debian/patches/fix_newer_zlib.patch
new file mode 100644
index 0000000..75b483b
--- /dev/null
+++ b/debian/patches/fix_newer_zlib.patch
@@ -0,0 +1,53 @@
+Description: fix compatibility with zlib 1.2.9 and newer
+Author: Marc Deslauriers <[e-mail]>
+Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/clamav/+bug/1692073
+Applied-Upstream: https://github.com/vrtadmin/clamav-devel/commit/48fef7b8ec06dbfbf899986cdcbe05071a633eb4
+Last-Update: 2017-11-22
+Index: clamav-0.99.2+dfsg/libclamav/bytecode_api.c
+===================================================================
+--- clamav-0.99.2+dfsg.orig/libclamav/bytecode_api.c	2017-08-08 15:20:06.651685637 -0400
++++ clamav-0.99.2+dfsg/libclamav/bytecode_api.c	2017-08-15 15:45:14.645714766 -0400
+@@ -811,8 +811,20 @@ int32_t cli_bcapi_inflate_init(struct cl
+         cli_dbgmsg("bytecode api: inflate_init: invalid buffers!\n");
+         return -1;
+     }
+-    memset(&stream, 0, sizeof(stream));
+-    ret = inflateInit2(&stream, windowBits);
++
++    b = cli_realloc(ctx->inflates, sizeof(*ctx->inflates)*n);
++    if (!b) {
++        return -1;
++    }
++    ctx->inflates = b;
++    ctx->ninflates = n;
++    b = &b[n-1];
++
++    b->from = from;
++    b->to = to;
++    b->needSync = 0;
++    memset(&b->stream, 0, sizeof(stream));
++    ret = inflateInit2(&b->stream, windowBits);
+     switch (ret) {
+         case Z_MEM_ERROR:
+             cli_dbgmsg("bytecode api: inflateInit2: out of memory!\n");
+@@ -829,20 +841,6 @@ int32_t cli_bcapi_inflate_init(struct cl
+             cli_dbgmsg("bytecode api: inflateInit2: unknown error %d\n", ret);
+             return -1;
+     }
+-
+-    b = cli_realloc(ctx->inflates, sizeof(*ctx->inflates)*n);
+-    if (!b) {
+-        inflateEnd(&stream);
+-        return -1;
+-    }
+-    ctx->inflates = b;
+-    ctx->ninflates = n;
+-    b = &b[n-1];
+-
+-    b->from = from;
+-    b->to = to;
+-    b->needSync = 0;
+-    memcpy(&b->stream, &stream, sizeof(stream));
+     return n-1;
+ }
+ 
diff --git a/debian/patches/series b/debian/patches/series
index e77765b..6a69a7c 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -6,3 +6,4 @@ Add-support-for-LLVM-3.8.patch
 Add-support-for-LLVM-3.9.patch
 Fix_detection_of_libcurl.patch
 clamsubmit-add-JSON-libs-to-clamsubmit.patch
+fix_newer_zlib.patch
diff --git a/debian/rules b/debian/rules
index 7d12c7a..84e78c6 100755
--- a/debian/rules
+++ b/debian/rules
@@ -30,7 +30,7 @@ LDFLAGS  := $(shell dpkg-buildflags --get LDFLAGS) -Wl,--as-needed
 ifeq (,$(filter $(DEB_HOST_ARCH), i386 amd64 kfreebsd-amd64 kfreebsd-i386))
   export enable_llvm=no
 else
-    SYSTEM_LLVM = -with-system-llvm=/usr/bin/llvm-config --with-llvm-linking=dynamic
+    SYSTEM_LLVM = -with-system-llvm=/usr/bin/llvm-config-3.9 --with-llvm-linking=dynamic
 endif
 
 # Set the configure options: