Merge lp:~paelzer/britney/hints-ubuntu-bump-openvswitch into lp:britney

[Christian Ehrhardt ]

Testing openvswitch is still broken by the kernel, this bumps the force-badtest and also limits it to i386 as this is the only arch affected.

[Christian Ehrhardt  (paelzer)]

Note that James Page had a similar bug open - we duped them together to coordinate better, but despite work by Kernel team and on OVS the fixes are complete yet.
Therefore asking for the bump to the badtest for now.

Unmerged revisions

2732. By Christian Ehrhardt  on 2018-01-03

duped bugs for openvswitch issue, update comment

2731. By Christian Ehrhardt  on 2018-01-03

bump version of openvswitch as infra is still broken

2730. By Stefano Rivera on 2017-12-29

dulwich is repeatedly failing on s390x due to some FS race

2729. By Steve Langasek on 2017-12-19

badtest pinentry/arm64 also

2728. By Andy Whitcroft on 2017-12-19

snapd: kick hint down the road

Signed-off-by: Andy Whitcroft <email address hidden>

2727. By Andy Whitcroft on 2017-12-18

sphinx is a victim of unrelated test failures

Signed-off-by: Andy Whitcroft <email address hidden>

2726. By Steve Langasek on 2017-12-15

badtest freeipa

2725. By Steve Langasek on 2017-12-13

badtest systemd/amd64 for now

2724. By Steve Langasek on 2017-12-12

badtest dogtag-pki

2723. By Steve Langasek on 2017-12-08

badtest kdepim-runtime/{ppc64el,s390x}

=== added file 'adconrad'
--- adconrad	1970-01-01 00:00:00 +0000
+++ adconrad	2018-01-03 17:16:24 +0000
@@ -0,0 +1,73 @@
+# This is regressed with s390x moving to kvm, xnox working on fixes:
+force-badtest systemd/235-2ubuntu3/s390x systemd/235-3ubuntu2/s390x
+# Ran out of patience and ran i386/amd64 coyote tests manually:
+force-skiptest plplot/5.13.0+dfsg-6ubuntu2 gnudatalanguage/0.9.7-6
+# No code changes, breaking update-manager upgrades:
+force-skiptest gnupg1/1.4.22-3ubuntu2
+# Consistently times out on armhf:
+force-badtest snapcraft/2.34+17.10/armhf
+# Regressed in release pocket:
+force-badtest mash/1.1.1-2build2/armhf mash/2.0-2/armhf
+force-badtest cinder/2:11.0.0-0ubuntu2/s390x
+force-badtest node-chokidar/1.7.0-1/s390x
+force-badtest node-platform/1.3.4-3/s390x
+force-badtest node-yargs/6.4.0-2/armhf
+# These tests fail now that s390x provides isolation-machine:
+force-badtest dahdi-linux/all/s390x
+force-badtest gsequencer/1.1.4-1/s390x
+force-badtest libdata-uuid-libuuid-perl/0.05-3build1/s390x
+# This test randomly times out only on i386 and has done so for eons:
+force-badtest kwin/4:5.10.5-0ubuntu1/i386
+# This test is flaky as all get out:
+force-badtest golang-gopkg-lxc-go-lxc.v2/0.0~git20161126.1.82a07a6-0ubuntu7/amd64
+# Click should never have had users on ppc64el, so carefactor zero:
+force-badtest click/0.4.46+17.10.20170607.3-0ubuntu1/ppc64el click-apparmor/0.3.18/ppc64el
+# Test is flaky on all arches, it's a miracle it passed once:
+force-badtest pglogical/2.0.0-1
+force-badtest pglogical/2.1.0-1
+# It's a mystery that this ever passed:
+force-badtest libffi-platypus-perl/0.45-1/amd64 libffi-platypus-perl/0.45-1build1/amd64
+# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=855092
+force-badtest beets/1.3.19-2.1/i386
+# The only remaining failure is a chicken-and-egg with lxc images not existing yet:
+force-skiptest debootstrap/1.0.91ubuntu2
+# Regressed in release pocket:
+force-badtest twisted/16.6.0-2ubuntu3 twisted/17.9.0-1
+# Depends on x86-only libngs-java
+force-badtest beagle/all/armhf beagle/all/ppc64el beagle/all/s390x
+force-badtest fastqc/all/armhf fastqc/all/ppc64el fastqc/all/s390x
+# Depends on stuff no longer built:
+force-badtest libcatmandu-store-mongodb-perl/all/i386
+force-badtest roary/all/armhf roary/all/s390x
+force-badtest pbgenomicconsensus/all/i386
+# Not built on i386, the test might not work:
+force-badtest pbh5tools/all/i386
+# Not built on ppc64el or s390x, so whee:
+force-badtest phyml/all/ppc64el
+force-badtest phyml/all/s390x
+# Mysteriously passed once, and never again:
+force-badtest vagrant-libvirt/0.0.40-1/amd64
+# Racy test, debugging in progress, but shouldn't prevent dep migration:
+force-badtest libpam-alreadyloggedin/0.3-6
+# Regression exists in release and proposed:
+force-badtest r-bioc-annotationhub/2.8.1-1build1 r-bioc-annotationhub/2.10.1-1
+force-badtest r-bioc-biocparallel/1.10.1-1build1/s390x
+# Alignment error exposed by no longer doing fixups on test infra:
+force-badtest ruby-ferret/0.11.8.6-2
+# Infra-related, not the package:
+force-badtest libdbd-mysql-perl/all/s390x
+# debootstrap's tests are a flaming heap in containers:
+force-badtest debootstrap/all/s390x
+# ACC Test broken by new GCC: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=837162
+force-badtest purpose/1.1-5ubuntu2
+force-badtest gpgmepp/16.04.3-2
+force-badtest libkf5kdgantt2/4:16.04.3-0ubuntu1
+force-badtest telepathy-logger-qt/15.04.0-2
+# fails only on lxc hosts
+force-badtest sbuild/all/armhf sbuild/all/s390x
+# Not a regression from the previous version:
+force-badtest kdepim-runtime/all/armhf
+# profnet-snapfun test is regressed in both release and proposed:
+force-badtest profnet/1.0.22-4build2
+# needs to be taught about bionic, queued for 2.29:
+force-badtest snapd/2.28.5+17.10

=== added file 'apw'
--- apw	1970-01-01 00:00:00 +0000
+++ apw	2018-01-03 17:16:24 +0000
@@ -0,0 +1,62 @@
+# While copying kernels forward from xenial it rather too easy for linux
+# to slip out without the testing being fully reviewed.  As britney is not
+# able to track previous failures correctly for kernels.
+block linux
+block linux-raspi2
+block linux-snapdragon
+
+# Development kernel unblocks:
+unblock linux/4.13.0-17.20
+unblock linux-raspi2/4.13.0-1008.8
+unblock linux-snapdragon/4.4.0-1081.86
+
+#
+# arm* removed in zesty-proposed hint in-progress tests against them.
+force-badtest sunpy/all/armhf
+
+# dkms failures here are actually kernel 4.9 related
+force-badtest kpatch/0.3.2-3ubuntu1
+
+# broken by python 2.7
+force-badtest freecad/0.16+dfsg2-3build3/ppc64el freecad/0.16+dfsg2-3build3/s390x
+force-badtest freecad/0.16.6712+dfsg1-1/ppc64el freecad/0.16.6712+dfsg1-1/s390x
+force-badtest freecad/0.16.6712+dfsg1-1build1/ppc64el freecad/0.16.6712+dfsg1-1build1/s390x
+force-badtest freecad/0.16.6712+dfsg1-1ubuntu1/ppc64el freecad/0.16.6712+dfsg1-1ubuntu1/s390x
+
+# python-pysam does not (and never has) built on armhf and i386
+force-badtest python-pysam/all/armhf python-pysam/all/i386
+
+# ginggs -- monkeysign ... unexpected pass
+force-badtest monkeysign/2.2.3/i386 monkeysign/2.2.3/ppc64el
+
+# kde is splittins apps out of this package, it is not relevant to those split apps.
+force-badtest kde-baseapps/4:16.04.3-0ubuntu1
+
+# 14-feb-2017: armhf test non-installable
+force-badtest python-cobra/all/armhf
+
+# wireguard should never have got into a stable release (LP: #1685522)
+block wireguard
+
+# ginggs: this is an old existing error on armhf, masked by another error in all arches.
+force-badtest r-cran-dplyr/0.5.0-1ubuntu4/armhf r-cran-dplyr/0.7.4-2/armhf
+
+# samtools got removed on armhf and s390x
+force-badtest winff/1.5.5-4/armhf
+
+# lttgng-modules failure on ppc64el is an internal compiler error
+force-badtest lttng-modules/2.9.0-1ubuntu3/ppc64el
+force-badtest lttng-modules/2.9.0-1ubuntu4/ppc64el
+
+# python-scipy/i386 is failing due a bug in the tests, do not want to trigger
+# a whole new set of tests to fix it.
+force-badtest python-scipy/0.19.1-1ubuntu1/i386
+
+# snapd/s390x testing is failing becuase it is now in nova
+force-badtest snapd/2.29.4.1+18.04/s390x snapd/2.29.4.2+18.04/s390x
+
+# openssl/1.0.2g-1ubuntu14 being held by an unrelated ruby test failure
+force-skiptest openssl/1.0.2g-1ubuntu14
+
+# two failing tests are not related to sphinx
+force-skiptest sphinx/1.6.5-3

=== added file 'arges'
=== added file 'brian-murray'
=== added file 'cjwatson'
=== added file 'freeze'
=== added file 'jriddell'
=== added file 'kitterman'
=== added file 'laney'
--- laney	1970-01-01 00:00:00 +0000
+++ laney	2018-01-03 17:16:24 +0000
@@ -0,0 +1,52 @@
+# isolation-machine, needs thunderbird which isn't available
+force-badtest firejail/all/s390x
+
+# 21/11 17:37:35 <ginggs> would some kind soul please 'force-badtest r-bioc-ensembldb/2.2.0-1' ?  all of the autopkgtests now need https://bioconductor.org/packages/release/data/annotation/html/EnsDb.Hsapiens.v75.html which is 75MB in size and no plans to package it in debian
+# (testsuite to be dropped in debian with the next upload)
+force-badtest r-bioc-ensembldb/2.2.0-1
+
+# someone manually triggered these, unfortunately logs don't go back far enough
+force-badtest akonadi-calendar/4:17.04.3-0ubuntu1/s390x akonadi-calendar/4:17.04.3-0ubuntu1/ppc64el
+force-badtest libkf5eventviews/4:17.04.3-0ubuntu1/s390x libkf5eventviews/4:17.04.3-0ubuntu1/ppc64el
+
+# acheronuk asked for this to be ignored, will report upstream (LP: https://bugs.launchpad.net/ubuntu/+source/konqueror/+bug/1732680)
+force-badtest konqueror/4:17.04.3-0ubuntu1
+
+# passed once by fluke on armhf it seems
+
+force-badtest ostree/all/armhf
+
+# according to xnox, autopkgtest-build-lxc doesn't work in nplan-only images
+# any more we're switching to cloud VMs soon for s390x; don't invest time in
+# fixing the LXC backend up right now. Drop this when we're switched over.
+
+force-badtest nplan/all/s390x
+
+# https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1699772
+force-badtest libreoffice/1:5.4.2-0ubuntu5/i386 libreoffice/1:5.4.3-0ubuntu1/i386
+
+# mongo isn't built for armhf/i386 any more
+force-badtest flask-mongoengine/all/i386
+force-badtest mongodb/all/i386
+force-badtest flask-mongoengine/all/armhf
+force-badtest mongodb/all/armhf
+
+# FTBFS, no package on this arch, unclear how it passed(!)
+force-badtest python-fitsio/all/armhf
+
+# Field3D only supports Ogawa-backed files on little-endian systems.
+force-badtest field3d/all/s390x
+
+# @INC problem; not perl 5.24
+force-badtest eekboek/2.02.05+dfsg-2
+force-badtest slice/1.3.8-13
+
+# pre-existing, skipped last time
+force-badtest parcimonie/all/s390x
+
+# bad test which fails if the test runner can't reach the public Internet.
+# LP: #1427982
+force-badtest libtest-requiresinternet-perl/0.05-2
+
+# kubuntu doesn't care about s390x
+force-badtest okular/all/s390x

=== added file 'pitti'
--- pitti	1970-01-01 00:00:00 +0000
+++ pitti	2018-01-03 17:16:24 +0000
@@ -0,0 +1,82 @@
+# new PEP-8 errors with zesty's newer pep8 tool; don't bump!
+force-badtest rpmlint/1.9-6
+
+# segfaults on 32 bit
+force-badtest baloo-kf5/all/i386 baloo-kf5/all/armhf
+
+# LP: #1510060
+force-badtest apt-clone/0.4.1ubuntu2
+
+# started failing with move to lxd on armhf
+force-badtest pbuilder/all/armhf
+
+#
+# blacklisted for test runs as they kill testbeds
+#
+#
+force-badtest stress-ng/all/ppc64el
+
+#
+# flaky, universe, nobody cares in Ubuntu
+#
+#
+force-badtest beets/all/s390x
+# times out
+force-badtest c2esp/all/armhf
+force-badtest debci/1.7 debci/1.7.1
+force-badtest elasticsearch-curator/4.2.5-1 elasticsearch-curator/5.2.0-1
+# takes painfully long, times out on armhf, testing on x86 is enough
+force-badtest epson-inkjet-printer-escpr/all/armhf
+# works in manual run
+force-badtest eso-midas/17.02pl1.2-2build1/s390x
+# got broken with new python2.7
+force-badtest flufl.lock/2.4.1-1
+force-badtest gjs/all/s390x
+force-badtest gtml/3.5.4-21/s390x gtml/3.5.4-22/s390x
+force-badtest jsurf-alggeo/0.3.0+ds-1
+force-badtest julia/all/i386
+force-badtest libcgi-application-plugin-captcha-perl/0.04-2
+# broken test if git is installed
+force-badtest libdist-zilla-plugin-githubmeta-perl/0.52-1
+force-badtest mercurial/all/armhf
+# SIGBUS
+force-badtest ostree/all/i386
+# flaky on s390x, alwaysfail on others
+force-badtest palabos/s390x/all
+force-badtest pdns/all
+# fails to install in yakkety
+force-badtest pdns-recursor/4.0.6-1/ppc64el
+force-badtest pbsuite/all/all
+force-badtest php-horde-http/2.1.7-1
+force-badtest php-horde-icalendar/all/armhf php-horde-icalendar/all/i386
+force-badtest postgresql-multicorn/1.3.3-2ubuntu2
+force-badtest puppet-module-asciiduck-sssd/0.3.0-1
+force-badtest python-publicsuffix/1.1.0-2
+force-badtest python-ruffus/2.6.3+dfsg-4/amd64
+force-badtest pyzmq/all/s390x pyzmq/all/armhf
+force-badtest rexical/1.0.5-2build1
+force-badtest ruby-amq-protocol/all/s390x
+force-badtest ruby-augeas/1:0.5.0-3build4
+# Debian #816161
+force-badtest ruby-awesome-print/1.6.1-2
+force-badtest ruby-fakeweb/1.3.0+git20131202.48208f9+dfsg-3
+force-badtest ruby-kaminari/0.17.0-3
+force-badtest ruby-minitest-excludes/2.0.0-1
+force-badtest ruby-mixlib-shellout/2.2.6-1
+force-badtest ruby-rack/1.6.4-4
+force-badtest ruby-roadie-rails/1.1.0-1
+force-badtest ruby-sshkit/all/s390x
+force-badtest ruby-test-unit/3.2.5-1/amd64
+force-badtest ruby-typhoeus/all/armhf
+force-badtest ruby-webmock/3.0.1-1
+force-badtest s3ql/2.23+dfsg-1 s3ql/2.24+dfsg-1
+# samtools got removed on armhf and s390x
+force-badtest fastaq/all/armhf fastaq/all/s390x
+force-badtest smalt/all/armhf smalt/all/s390x
+force-badtest zodb/1:3.10.5+1.gb28a24c-1
+
+#
+# ignored stuff for CI train (vivid)
+#
+
+force-badtest nvidia-graphics-drivers-340/all/armhf

=== added file 'raof'
=== added file 'stefanor'
--- stefanor	1970-01-01 00:00:00 +0000
+++ stefanor	2018-01-03 17:16:24 +0000
@@ -0,0 +1,8 @@
+# autodep8 bug #832567
+force-badtest ruby-cucumber-wire/0.0.1-2
+
+# pbbam is currently FTBFS on 32bit Debian: #829741
+force-badtest pbsuite/15.8.24+dfsg-2/i386
+
+# https://github.com/jelmer/dulwich/issues/541
+force-badtest dulwich/0.18.5-1/s390x

=== added file 'stgraber'
=== added file 'tjaalton'
=== added file 'vorlon'
--- vorlon	1970-01-01 00:00:00 +0000
+++ vorlon	2018-01-03 17:16:24 +0000
@@ -0,0 +1,207 @@
+# regressed in release due to changed output of 'ip route get'
+force-badtest freeipa/4.4.4-3ubuntu1
+
+# unfortunately unreliable
+force-badtest systemd/235-3ubuntu2
+
+# regressed in release
+force-badtest dogtag-pki/10.3.5+12-5
+
+# how does someone write a tty-related test so bad that it passes in a container and fails in a VM?
+force-badtest pinentry/1.0.0-3/s390x pinentry/1.0.0-3/arm64
+
+# unclear how this test ever passed on Ubuntu, with its misuse of nc and general non-functioning
+force-badtest zorp/6.0.10-3
+
+# Was skipped in lxc, never passed in a vm, should be "always failed"
+force-badtest lxcfs/2.0.8-1ubuntu2/s390x
+
+# Was skipped in lxc, never passed in a vm, not supported on big-endian
+# https://github.com/markfasheh/ocfs2-tools/issues/22
+force-badtest ocfs2-tools/1.8.5-2/s390x
+
+# Was skipped in lxc, never passed in a vm, questionable usefulness on s390x/ppc64el given it tests usb camera...
+force-badtest gvfs/1.34.1-1ubuntu3/s390x gvfs/1.34.1-1ubuntu3/ppc64el gvfs/1.34.1-1ubuntu4/s390x gvfs/1.34.1-1ubuntu4/ppc64el
+
+# Did previously pass, but upstream (pitti) say the failing avocado test is bogus and will be removed in the next upload
+force-badtest cockpit/156-1/s390x
+
+# bad test, assumes that architecture has WiFi, urfkill, dhcp.  previously it would SKIP on s390x, now it is run in nova-kvm and is treated as a regression rather than an always-failed
+force-badtest network-manager/all/s390x
+
+# Skip tests that fail now that s390x has full VMs
+force-badtest dahdi-linux/1:2.11.1~dfsg-1ubuntu3/s390x
+force-badtest firejail/0.9.50-3/s390x
+force-badtest vagrant-lxc/1.2.1-3ubuntu1/s390x
+force-badtest c-icap/1:0.4.4-1/s390x
+
+# regresses with perl/5.26.1-2ubuntu1 but doesn't look like a perl bug, so don't block perl migration
+force-badtest libtest-valgrind-perl/1.19-1/ppc64el
+
+# postgresql-common hard-codes the list of supported Ubuntu releases and throws a warning if not recognized.
+force-badtest slony1-2/2.2.6-1
+
+# nonsensical error with new perl; allow it to regress to let perl migrate
+force-badtest libdata-dpath-perl/0.55-1 libdata-dpath-perl/0.57-1
+
+# broken with newer glibc
+force-badtest snapcraft/2.34+17.10/amd64 snapcraft/2.34+17.10/i386
+
+# testing triggered from autodep-8: the kernel contains matching modules which causes an installation
+# failure when used with a kernel which contains them.
+force-badtest virtualbox/5.1.28-dfsg-3
+
+# dependencies removed on these archs
+force-badtest astroml/all/s390x astroml/all/ppc64el
+
+# passed once as a fluke
+force-badtest libanyevent-httpd-perl/0.93-5/ppc64el libanyevent-httpd-perl/all/s390x libanyevent-httpd-perl/all/armhf
+
+# assumes /etc/init.d/networking
+force-badtest pcs/0.9.159-3ubuntu2 pcs/0.9.161-1
+
+# passed once as a fluke
+force-badtest mbuffer/20161115-1/ppc64el mbuffer/20171011-1/ppc64el
+
+# regressed in release
+force-badtest nut/2.7.4-5.1ubuntu1/s390x nut/2.7.4-5.1ubuntu2/s390x
+force-badtest mailman3-core/3.1.0-1
+force-badtest libpoe-component-client-dns-perl/1:1.054-1ubuntu1
+force-badtest libanyevent-yubico-perl/0.9.3-1/ppc64el
+force-badtest grr/3.1.0.2+dfsg-4
+force-badtest cacti/1.1.27+ds1-3/armhf
+force-badtest xandikos/0.0.6-2/amd64 xandikos/0.0.6-2/i386
+force-badtest kio/5.38.0-0ubuntu1/amd64 kio/5.38.0-0ubuntu1/i386
+force-badtest python-py/1.4.34-3
+force-badtest bubblewrap/0.2.0-2/i386
+# hicolor-icon-theme
+force-badtest pyxdg/0.25-4
+
+# tests that hit the network and are proxy-aware but not passing in scalingstack.  Infrastructure bug, don't block.
+force-badtest python-mechanicalsoup/0.7.0-1
+
+# regressed in release; maybe related to resolved/netplan.
+force-badtest ifupdown/0.8.16ubuntu2/ppc64el
+
+# regressed in release due to demotion of resolvconf
+force-badtest open-iscsi/2.0.874-4ubuntu2/amd64
+
+# broken by infra; remove badtest asap - still broken, see 1736390
+force-badtest openvswitch/2.8.1-0ubuntu2/i386
+
+# The only test that is run is a rebuild test because the integration test doesn't work in a container; a kernel build takes 7h on armhf, the max a test is allowed to run on autopkgtest is 2h47m.  Blacklist this until further notice.
+force-badtest linux-raspi2/all linux/all/armhf
+
+# regressed in release
+force-badtest udisks2/2.6.5-2ubuntu2/ppc64el
+force-badtest htslib/1.5-1/i386
+
+# flaky
+force-badtest mysql-5.7/5.7.19-0ubuntu1/i386
+
+# passed once as a fluke
+force-badtest casync/2-1ubuntu1/ppc64el
+# and flaky on other archs
+force-badtest casync/2-1ubuntu1/amd64 casync/2-1ubuntu1/i386 casync/2-1ubuntu1/s390x
+
+# regressed in release
+force-badtest pdb2pqr/2.1.1+dfsg-2
+force-badtest gap-guava/3.13+ds-2/i386 gap-guava/3.13+ds-2/s390x gap-guava/3.13+ds-2/amd64
+
+# regressed in release due to compiler strictness (stderr)
+force-badtest globjects/1.1.0-2/armhf
+force-badtest mathicgb/1.0~git20170606-1/armhf
+force-badtest primesieve/5.7.2+ds-2/armhf
+
+# testdep removed from archive
+force-badtest mercurial/4.3.1-2 mercurial/4.4.1-1
+
+# now failing with timeouts
+force-badtest haveged/1.9.1-6/armhf
+
+# regressed in release
+force-badtest yorick/2.2.04+dfsg1-6build1
+
+# pandas no longer available on armhf
+force-badtest lmfit-py/armhf/all
+
+# regressed in release (udev/container related)
+force-badtest pgpool2/armhf/3.6.6-2
+
+# still bad
+force-badtest bareos/14.2.6-3 bareos/16.2.6-1
+
+# flaky test
+force-badtest tracker/all/s390x
+
+# flaky / timeouts
+force-badtest node-tap/8.0.0-4ubuntu5/armhf
+
+# regressed in release due to gdata
+force-badtest r-cran-surveillance/1.13.0-1build1
+
+# regressed in release; possibly all due to the same cause of something updating in the perl auto test infrastructure
+force-badtest license-reconcile/0.14
+force-badtest libsoap-wsdl-perl/3.003-2
+force-badtest libjson-validator-perl/0.92+dfsg-1
+force-badtest libdist-zilla-util-configdumper-perl/0.003007-1
+force-badtest bioperl/1.7.1-2 bioperl/1.7.1-3
+force-badtest bio-tradis/1.3.3+dfsg-1 bio-tradis/1.3.3+dfsg-2
+
+# mongodb-server is not installable on armhf/i386; this tells us nothing of whether the client works
+force-badtest python-motor/all/armhf python-motor/all/i386
+
+# already broken in release
+force-badtest ddcci-driver-linux/0.3.1-1/s390x
+
+# test deps now unsatisfiable because of kde removals
+force-badtest kopete/all/ppc64el kopete/all/s390x
+force-badtest libkf5ksieve/all/ppc64el libkf5ksieve/all/s390x
+force-badtest kf5-kdepim-apps-libs/all/ppc64el kf5-kdepim-apps-libs/all/s390x
+force-badtest libkf5calendarsupport/all/ppc64el libkf5calendarsupport/all/s390x
+force-badtest libkf5incidenceeditor/all/ppc64el libkf5incidenceeditor/all/s390x
+force-badtest libkf5gravatar/all/ppc64el libkf5gravatar/all/s390x
+force-badtest libkf5mailimporter/all/ppc64el libkf5mailimporter/all/s390x
+force-badtest akonadi-calendar/all/ppc64el akonadi-calendar/all/s390x
+force-badtest libkf5eventviews/all/ppc64el libkf5eventviews/all/s390x
+force-badtest kdepim-addons/all/s390x
+force-badtest kdepim-runtime/all/ppc64el kdepim-runtime/all/s390x
+
+# non-empty stderr with new g++
+force-badtest glbinding/2.1.1-1/armhf
+
+# passes once in a blue moon, just often enough to gum things up.
+force-badtest pyzmq/16.0.2-2build2
+
+# was already broken but silently passing; failure now exposed thanks to the maintainer
+force-badtest python-asdf/1.2.1-2/s390x python-asdf/1.3.1-1/s390x
+
+# regressed in release by unknown means
+force-badtest qbzr/0.23.2-2ubuntu2/ppc64el qbzr/0.23.2-2ubuntu2/s390x
+
+# regressed due to new warnings from gcc
+force-badtest fast5/0.5.8-1build3/armhf ciftilib/1.5.1-1 ciftilib/1.5.1-1build1
+
+# regressed in release pocket
+force-badtest octave-io/2.4.7-1build1/armhf octavio-/2.4.8-1/armhf
+
+force-badtest aptdaemon/1.1.1+bzr982-0ubuntu17
+
+# "blacklisted", self-explanatory.
+force-badtest upstart/blacklisted libnih/blacklisted linux/blacklisted/i386 linux/blacklisted/amd64 linux/blacklisted/ppc64el
+force-badtest stress-ng/blacklisted
+
+# regressed in release in some way related to gphoto2 tests
+force-badtest umockdev/0.9.2-1
+
+# regression of unknown origin
+force-badtest kpatch/0.3.2-3ubuntu1/amd64
+
+# As requested by yofel: testing needed to allow a xenial SRU.
+force-badtest plasma-framework/all/s390x
+
+# breaks with firefox 49, which (with this hint) we are allowing into the release
+force-badtest y-u-no-validate/2013052407-3
+
+# bad test, but better to have this package in xenial and unblock things than leave it on the list
+force-badtest guessit/0.11.0-2