USN-3867-1: Partially applies to MariaDB too

The 5.5 series update for 14.04 is now available.

Please use git-buildpackage to fetch and build from the ubuntu-14.04 branch at https://salsa.debian.org/mariadb-team/mariadb-5.5/tree/ubuntu-14.04

The repository uses pristine-tar, so there is no need to separately download the sources. You can just check the signature/SHA1SUM directly from the git-buildpackage generated tarball.

Test builds and testsuite passed on all platforms at
https://launchpad.net/~mysql-ubuntu/+archive/ubuntu/mariadb/+builds?build_text=&build_state=all

Debdiffs can be created directly from the repo like in a local clone with 'git diff <tag1>..<tag2> debian/'

Security sponsor note these: https://wiki.ubuntu.com/SecurityTeam/PublicationNotes#Sponsoring_MariaDB_Security_Updates

The 10.0 series update for 16.04 is now available.

Please use git-buildpackage to fetch and build from the ubuntu-16.04 branch at https://salsa.debian.org/mariadb-team/mariadb-10.0/tree/ubuntu-16.04

The repository uses pristine-tar, so there is no need to separately download the sources. You can just check the signature/SHA1SUM directly from the git-buildpackage generated tarball.

Test builds and testsuite passed on all platforms at
https://launchpad.net/~mysql-ubuntu/+archive/ubuntu/mariadb-10.0/+builds?build_text=&build_state=all

Hey Otto, thanks for preparing these. I'm on community next week, so I'll work on sponsoring these.

The 10.1 series update for 18.04 is now available.

Please use git-buildpackage to fetch and build from the ubuntu-18.04 branch at https://salsa.debian.org/mariadb-team/mariadb-10.1/tree/ubuntu-18.04

The repository uses pristine-tar, so there is no need to separately download the sources. You can just check the signature/SHA1SUM directly from the git-buildpackage generated tarball.

Test builds and testsuite passed on all platforms at
https://launchpad.net/~mysql-ubuntu/+archive/ubuntu/mariadb-10.1/+builds?build_text=&build_state=all

Hey Otto,

The mariadb-5.5 for trusty git tree worked fine. For xenial, gbp could not generate the orig tarball; it failed with the following error:
  $ gbp buildpackage --git-builder="umt source"
  gbp:info: Creating /srv/work/mariadb-10.0/tmp/mariadb-10.0_10.0.38.orig.tar.gz
  gbp:error: Error creating mariadb-10.0_10.0.38.orig.tar.gz: Pristine-tar couldn't checkout "mariadb-10.0_10.0.38.orig.tar.gz": xdelta3: target window checksum mismatch: XD3_INVALID_INPUT
  xdelta3: normally this indicates that the source file is incorrect
  xdelta3: please verify the source file with sha1sum or equivalent
  xdelta3: target window checksum mismatch: XD3_INVALID_INPUT
  xdelta3: normally this indicates that the source file is incorrect
  xdelta3: please verify the source file with sha1sum or equivalent
  xdelta3: target window checksum mismatch: XD3_INVALID_INPUT
  xdelta3: normally this indicates that the source file is incorrect
  xdelta3: please verify the source file with sha1sum or equivalent
  xdelta3: target window checksum mismatch: XD3_INVALID_INPUT
  xdelta3: normally this indicates that the source file is incorrect
  xdelta3: please verify the source file with sha1sum or equivalent
  pristine-tar: Failed to reproduce original tarball. Please file a bug report.
  pristine-tar: failed to generate tarball

I ended up pulling the tarball from the mariadb site and using it (after verifying the signature).

Also, the changelog for xenial was not indented enough, I ended up making the following minor change to the changelog (patch attached).

Thanks.

Hi Otto,

For 10.0/xenial, did you manually run the testsuite to capture results? The testsuite is disabled in the build with the following comment:

  # Run testsuite
  # Disable test suite because of too many false positives that we don't have
  # time to fix in time of the Ubuntu 16.04 release, and as actual fixes must
  # get in, the test suite with false positives is an unnecessary obstacle.

Also, the security team's security-proposed ppa now has autopkgtests enabled on it; see e.g. the bionic results at http://people.canonical.com/~platform/security-britney/current/security_bionic_excuses.html

For mariadb-10.1/bionic, autopkgtests attempted to run, but because the 10.1 packaging doesn't produce the mariadb-test package, the upstream tests failed. I'm not sure that they're any different than the tests that get run at build time, so there may not actually be any gain by fixing this.

Thanks.

This bug was fixed in the package mariadb-5.5 - 5.5.63-1ubuntu0.14.04.1

---------------
mariadb-5.5 (5.5.63-1ubuntu0.14.04.1) trusty-security; urgency=high

  * SECURITY UPDATE: New upstream release 5.5.63. Includes fixes for
    the following security vulnerabilities (LP: #1814258):
    - CVE-2019-2529
  * Previous release 5.5.62 included fixes for the following security
    vulnerabilities:
    - CVE-2019-2503
    - CVE-2018-3282
    - CVE-2018-3174
    - CVE-2016-9843

 -- Otto Kekäläinen <email address hidden> Fri, 01 Feb 2019 14:11:30 +0100

This bug was fixed in the package mariadb-10.1 - 1:10.1.38-0ubuntu0.18.04.1

---------------
mariadb-10.1 (1:10.1.38-0ubuntu0.18.04.1) bionic-security; urgency=medium

  * SECURITY UPDATE: New upstream release 10.1.38. Includes fixes for
    the following security vulnerabilities (LP: #1814258):
    - CVE-2019-2537
    - CVE-2019-2529
  * Remove non-applying Hurd patch as Ubuntu does not ship Hurd anyway
  * Use list-missing instead of fail in d/rules so builds pass
  * Add (and rename) new man pages
  * Previous upstream version 10.1.37 included fixes for the following
    security vulnerabilities:
    - CVE-2018-3282
    - CVE-2018-3251
    - CVE-2018-3174
    - CVE-2018-3156
    - CVE-2018-3143
    - CVE-2016-9843
  * Previous upstream version 10.1.36 included fixes for the following
    security vulnerabilities:
    - CVE-2019-2503
  * Previous upstream version 10.1.35 included fixes for the following
    security vulnerabilities:
    - CVE-2018-3066
    - CVE-2018-3064
    - CVE-2018-3063
    - CVE-2018-3058

 -- Otto Kekäläinen <email address hidden> Wed, 06 Feb 2019 07:53:10 +0200

I have a bit of fever now, will try to check reported 10.0 issues tomorrow.

I can repeat the pristine-tar issue you had:

(jessie) otto@XPS-13-9370:/tmp/mariadb-10.0$ gbp buildpackage --git-pristine-tar
gbp:info: Creating /tmp/mariadb-10.0_10.0.38.orig.tar.gz
gbp:error: Error creating mariadb-10.0_10.0.38.orig.tar.gz: Pristine-tar couldn't checkout "mariadb-10.0_10.0.38.orig.tar.gz": xdelta3: target window checksum mismatch: XD3_INVALID_INPUT
xdelta3: normally this indicates that the source file is incorrect
xdelta3: please verify the source file with sha1sum or equivalent
xdelta3: target window checksum mismatch: XD3_INVALID_INPUT
xdelta3: normally this indicates that the source file is incorrect
xdelta3: please verify the source file with sha1sum or equivalent
xdelta3: target window checksum mismatch: XD3_INVALID_INPUT
xdelta3: normally this indicates that the source file is incorrect
xdelta3: please verify the source file with sha1sum or equivalent
xdelta3: target window checksum mismatch: XD3_INVALID_INPUT
xdelta3: normally this indicates that the source file is incorrect
xdelta3: please verify the source file with sha1sum or equivalent
pristine-tar: Failed to reproduce original tarball. Please file a bug report.
pristine-tar: failed to generate tarball

Something must have gone wrong in the commit https://salsa.debian.org/mariadb-team/mariadb-10.0/commit/962a0a6be4512d699a7e130db9609613d2cd2ca8 - I will ask Emilio what tooling he used. I heard from some that pristine-tar might be broken in certain situations, maybe this is it.

The test failures were due to upstream issues, and most likely are fixed by now, so I will experiment with reverting https://salsa.debian.org/mariadb-team/mariadb-10.0/commit/62562ed38dce0bbab3bdb666f3acc01a23a33a51

I noticed a "hole" in my pipeline, the test suite is actually not forced to run anywhere, not https://salsa.debian.org/mariadb-team/mariadb-10.0/pipelines/34858, not https://launchpad.net/~mysql-ubuntu/+archive/ubuntu/mariadb-10.0/+builds?build_text=&build_state=all and there was no autopkgtest here either https://ci.debian.net/packages/m/mariadb-10.0/

I will address this.

By the way I don't see any MariaDB here either: http://people.canonical.com/~platform/security-britney/current/security_bionic_excuses.html

Next time you encounter issues, please report them, and give me 1-2 days time to fix, and only upload once you have fixed stuff. Anyways, I'll fix it now and they should be in place for 10.0.X upload some day.

Hi Otto,

Okay, I'm going to hold off on publishing the mariadb-10.0 updates until some confirmation that they have been tested.

For mariadb-10.1, I checked the build time tests, and they looked okay, so I went ahead and published that package, which is why it's not showing up in the excuses page. The autopkgtest failure log is at https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic-ubuntu-security-proposed-ppa/bionic/amd64/m/mariadb-10.1/20190207_122932_ab116@/log.gz (at least for the time being) if you want to look at it directly.

If it does get reaped, the relevant bits of the log are as follows:

  Removing autopkgtest-satdep (0) ...
  autopkgtest: WARNING: Test dependencies are unsatisfiable - calling apt install on test deps directly for further data about failing dependencies in test logs
  Reading package lists...
  Building dependency tree...
  Reading state information...
  E: Unable to locate package mariadb-test
  upstream FAIL badpkg
  blame: mariadb-10.1
  badpkg: Test dependencies are unsatisfiable. A common reason is that your testbed is out of date with respect to the archive, and you need to use a current testbed or run apt-get update or use -U.
  autopkgtest [12:29:22]: @@@@@@@@@@@@@@@@@@@@ summary
  smoke PASS
  upstream FAIL badpkg
  blame: mariadb-10.1

In particular, note the "E: Unable to locate package mariadb-test" message. The mariadb-10.1 control file does not build a mariadb-test package.

Thanks.

There are indeed still multiple test failures in 10.0 and I doubt upstream will fix them. I did however fix the indentation myself.

Missing mariadb-test in Bionic is weird. The package exists in all Debian releases: https://packages.debian.org/search?keywords=mariadb-test&searchon=names&suite=all&section=all
..but it is missing from Bionic and even bionic and cosmic: https://packages.ubuntu.com/search?suite=all&section=all&arch=any&keywords=mariadb-test&searchon=names

Digging in git blame turns out this is part of the fall-out of Ondrej when he messed up the packages: https://salsa.debian.org/mariadb-team/mariadb-10.1/commit/27202e3286910482a35bc48cfd7cd16defb82cd3

I guess we cannot re-introduce the test packages in Bionic anymore, so the only option is to remove the 'upstream' test from autopkgtests and only keep the 'control'. They haven't worked since 10.1.25-1, which was never in the actual Bionic release: https://autopkgtest.ubuntu.com/packages/mariadb-10.1/bionic/amd64

The Ubuntu 18.04 version however runs the testsuite during the build and is fully passes: https://launchpadlibrarian.net/410048810/buildlog_ubuntu-bionic-amd64.mariadb-10.1_1%3A10.1.38-0ubuntu0.18.04.1~ubuntu18.04.1~1549486053.a4bd40485f_BUILDING.txt.gz

Going forward, I don't have a problem reintroducing the mariadb-test package back for mariadb-10.0 in a future update cycle, since nothing else should depend on it. But as you point out, the tests are run at build time as well. Are there any differences in which tests are run at build time versus via autopkgtests, possibly based on what's available in the autopkgtest environment?

Regarding your question on differences:

In MariaDB the build time and autopkg tests (part 'upstream') are running the same mtr test suite 'main'. Depending on the environment some tests might bail out, but there is no intentional difference by design.

https://launchpadlibrarian.net/407954652/buildlog_ubuntu-disco-amd64.mariadb-10.3_1%3A10.3.12-2~ubuntu19.04.1~1548328652.011399006_BUILDING.txt.gz
=> Completed: All 739 tests were successful. 97 tests were skipped, 42 by the test itself.

https://ci.debian.net/data/autopkgtest/unstable/amd64/m/mariadb-10.3/1892278/log.gz
=> Completed: All 702 tests were successful. 128 tests were skipped, 63 by the test itself.

By the way, all of this is team maintained in modern Gitlab with extensive Gitlab-CI integration etc, so if you dig deeper into something and find flaws, it is very easy to fire a merge request to get it fixed: https://salsa.debian.org/mariadb-team/

Regarding mariadb-10.0:

Upstream has not fixed the test suite issues and they are not that relevant, so let's keep having the test suite disabled. The mariadb-10.0 10.0.38-0+deb8u1 was uploaded to Debian Jessie almost 2 weeks ago and no regressions are reported, so it should be safe to upload this to Xenial now.

This bug was fixed in the package mariadb-10.0 - 10.0.38-0ubuntu0.16.04.1

---------------
mariadb-10.0 (10.0.38-0ubuntu0.16.04.1) xenial-security; urgency=high

  * SECURITY UPDATE: New upstream release 10.0.38. Includes fixes for
    the following security vulnerabilities (LP: #1814258):
    - CVE-2019-2537
    - CVE-2019-2529
  * Previous release 10.0.37 included fixes for the following security
    vulnerabilities:
    - CVE-2019-2503
    - CVE-2018-3282
    - CVE-2018-3251
    - CVE-2018-3174
    - CVE-2018-3156
    - CVE-2018-3143
    - CVE-2016-9843

 -- Otto Kekäläinen <email address hidden> Fri, 01 Feb 2019 14:51:00 +0100