Merge ~juliank/autopkgtest/+git/development:master into ~ubuntu-release/autopkgtest/+git/development:master

[Julian Andres Klode]

Should force push instead of merge

[Julian Andres Klode (juliank)]

The following commit has been dropped:

commit b442efb577be63f4f03fcc5d821a68dc553628a3 (ubuntu/master)
Author: Steve Langasek <email address hidden>
Date: Fri Aug 31 21:53:49 2018 -0700

    Don't pick uids between 500 and 999 for our container user

    these are system users under Debian Policy, and we now have a common user
    in this uid range, systemd-coredump, with a non-writable homedir. That
    messes up a lot of tests.

It was no longer necessary, upstream took it a bit further and made it pick in [1000,59999] rather than any >= 1000.

[Iain Lane (laney)]

Done, thanks (to pings on #debian-release).

I re-rebased, so this probably won't get marked as merged automatically.

review: Approve

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
new file mode 100644
index 0000000..f28fa86
--- /dev/null
+++ b/.gitlab-ci.yml
@@ -0,0 +1,26 @@
+quicktests:
+  stage: test
+  image: debian:sid
+  script:
+    - apt-get update
+    - apt-get install -y autodep8 libdpkg-perl pycodestyle pyflakes3 python3-debian
+    - tests/autopkgtest_args
+    - tests/pycodestyle
+    - tests/pyflakes
+    - tests/testdesc
+
+tests-sid:
+  stage: test
+  image: debian:sid
+  script:
+    - apt-get update
+    - apt-get install -y apt-utils autodep8 build-essential debhelper libdpkg-perl procps python3 python3-debian
+    - tests/autopkgtest NullRunner NullRunnerRoot ChrootRunner
+
+tests-stretch:
+  stage: test
+  image: debian:stretch
+  script:
+    - apt-get update
+    - apt-get install -y apt-utils autodep8 build-essential debhelper libdpkg-perl procps python3 python3-debian
+    - tests/autopkgtest NullRunner NullRunnerRoot ChrootRunner
diff --git a/debian/changelog b/debian/changelog
index 212d827..ec56090 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,107 @@
+autopkgtest (5.5) unstable; urgency=medium
+
+  [ Antonio Terceiro ]
+  * Different implementation to make sure to never pass an empty string as
+    username to `su`
+  * When using systemd, reboot via systemd-run
+  * test_breaks_testbed: allow output on stderr
+
+  [ Paul Gevers ]
+  * Fix whitespace in old parts of d/changelog
+  * Add gitlab pipelines to autopkgtest
+  * Fixes to be conform pycodestyle
+  * tests: replace encoding='utf-8' by universal_newlines=True for
+    python3.5 (stretch)
+  * Fix missing empty line in commit 30470603
+  * Add option to disable the apt fallback when dependencies can't be
+    installed due to pinning
+  * Make autopkgtest Build-Depends-Arch aware (Closes: #903398)
+  * Add skip-not-installable restriction (Closes: #905311)
+  * Drop ancient X-Python3-Version (thanks lintian)
+
+  [ Ian Jackson ]
+  * doc/README.package-tests.rst: document hint-testsuite-triggers
+    (Closes: #905310)
+
+  [ Simon McVittie ]
+  * runner: Document exit status 14 as possible
+  * tests: Don't rely on being able to install B-D as non-root (Closes:
+    #905677)
+  * tests: Skip test_tree_build_needed_success if necessary
+  * runner: Exit 8 if every test was skipped or otherwise ignored
+    (Closes: #901804)
+
+ -- Paul Gevers <[e-mail]>  Fri, 10 Aug 2018 20:27:40 +0200
+
+autopkgtest (5.4.2) unstable; urgency=medium
+
+  [ Martin Pitt ]
+  * doc: Document "test-name" feature
+  * doc/README.package-tests.rst: Fix formatting of previous commit
+
+  [ Niko Tyni ]
+  * Fix versioned provides test dependency handling
+  * Refactor _synthesize_deps() parameters
+  * Properly handle virtual packages in alternative test dependencies
+    (Closes: #903975)
+  * Add test cases for alternative test dependencies improvements
+
+  [ Antonio Terceiro ]
+  * Always pass a non-empty user string to `su` (Closes: #904870)
+
+ -- Antonio Terceiro <[e-mail]>  Sun, 29 Jul 2018 12:13:06 -0300
+
+autopkgtest (5.4.1) unstable; urgency=medium
+
+  * README.package-tests.rst: explain that architecture qualifiers are
+    allowed in Test-Depends
+  * SchrootRunner: add testcases for arch qualifiers in Test-Depends
+  * Fix regression on arch-specific packages with '@' in Test-Depends
+    (Closes: #903000)
+
+ -- Paul Gevers <[e-mail]>  Fri, 06 Jul 2018 20:05:39 +0200
+
+autopkgtest (5.4) unstable; urgency=medium
+
+  [ Simon McVittie ]
+  * README.package-tests.rst: Document AUTOPKGTEST_NORMAL_USER
+  * d/tests/lxd: Don't assume all test-runners set AUTOPKGTEST_NORMAL_USER
+  * qemu: Only set up base image device if requested (Closes: #892023, #862362)
+  * qemu: Document --baseimage
+  * qemu: Update test for --baseimage no longer being the default
+  * qemu, lxc, lxd: Look for a user account in the 1000-59999 range
+    (Closes: #897170)
+  * qemu: Add a shortcut for running tests on an EFI-booted image (Closes:
+    #898340)
+  * doc: Describe how to parse Features, Restrictions, Classes
+  * Add support for flaky tests (Closes: #851558)
+  * Add support for tests declaring themselves to have been skipped
+  * autopkgtest(1): Document FLAKY as a possible summary status
+
+  [ Balint Reczey ]
+  * Fix bashism in retrying apt update
+
+  [ Paul Gevers ]
+  * Enable testing to continue after badpkg (Closes: #832751)
+  * adt_testbed.py fix missed piece of regular expression in commit a7e1dad
+  * d/tests/autopkgtest Update SchrootRunner
+  * runner/autopkgtest: Drop Ubuntu 12.04 fallback
+  * manpage: make ordering consistent with --help (Closes: #901643)
+
+  [ Julian Andres Klode ]
+  * ssh-setup/nova: Add support for keystone v3 auth (LP: #1767433)
+
+  [ Rafael Laboissiere ]
+  * Set Maintainer email address to [e-mail]
+
+  [ Niko Tyni ]
+  * Add a couple of testcases for versioned provides support
+  * Ensure synthesized test dependencies are not satisfied by versioned Provides
+    (Closes: #867081)
+  * Remove the old '(>= 0)' hack for ensuring '@' pulls in real packages
+
+ -- Paul Gevers <[e-mail]>  Mon, 02 Jul 2018 11:50:21 +0200
+
 autopkgtest (5.3.1) unstable; urgency=medium
 
   [ Iain Lane ]
@@ -3398,11 +3502,11 @@ autopkgtest (1.0.4) gutsy; urgency=low
   * adt-run:
     - Options for setting timeouts.
     - Increase default timeouts 100s, 3ks, 10ks, 100ks.
-  
+
   * hosts/chinstrap/*:
      Screen-scrape Launchpad to get existing bugs and generate a suitable
-     suppression file so we file a bug iff there isn't one already. 
-  
+     suppression file so we file a bug iff there isn't one already.
+
   * hosts/cadmium/*:
      New directory for convenience scripts etc on Canonical buildd.
 
@@ -3541,7 +3645,7 @@ autopkgtest (0.8.2) gutsy unstable; urgency=low
   * Fix handling of pre-built source trees.
   * Fix cleanup handling not to delete tmpdir before resetting testbed's
     apt.
-  
+
  -- Ian Jackson <[e-mail]>  Fri, 27 Apr 2007 16:06:15 +0100
 
 autopkgtest (0.8.1) feisty; urgency=low
@@ -3571,7 +3675,7 @@ autopkgtest (0.8.0) feisty; urgency=low
   * Longer timeouts by default.
   * Print `adt-run: trace' for trace output.
   * Show all apt stdout (including dpkg stdout) in contemporaneous trace.
-  
+
  -- Ian Jackson <[e-mail]>  Tue,  3 Apr 2007 20:08:13 +0100
 
 autopkgtest (0.7.2) feisty; urgency=low
diff --git a/debian/control b/debian/control
index eed730e..d1a838b 100644
--- a/debian/control
+++ b/debian/control
@@ -1,5 +1,5 @@
 Source: autopkgtest
-Maintainer: Debian CI team <[e-mail]>
+Maintainer: Debian CI team <[e-mail]>
 Uploaders: Ian Jackson <[e-mail]>, Martin Pitt <[e-mail]>, Antonio Terceiro <[e-mail]>, Paul Gevers <[e-mail]>
 Section: devel
 Priority: optional
@@ -14,7 +14,6 @@ Build-Depends: debhelper (>= 9),
  pycodestyle | pep8,
 Vcs-Git: https://salsa.debian.org/ci-team/autopkgtest.git
 Vcs-Browser: https://salsa.debian.org/ci-team/autopkgtest
-X-Python3-Version: >= 3.1
 
 Package: autopkgtest
 Architecture: all
@@ -25,7 +24,15 @@ Depends: python3,
  procps,
  ${misc:Depends}
 Recommends: autodep8
-Suggests: schroot, lxc, lxd-client, qemu-system, qemu-utils
+Suggests:
+ lxc,
+ lxd-client,
+ ovmf,
+ qemu-efi-aarch64,
+ qemu-efi-arm,
+ qemu-system,
+ qemu-utils,
+ schroot,
 Breaks: debci (<< 1.7~)
 Description: automatic as-installed testing for Debian packages
  autopkgtest runs tests on binary packages.  The tests are run on the
diff --git a/debian/copyright b/debian/copyright
index d592b1c..7b35827 100644
--- a/debian/copyright
+++ b/debian/copyright
@@ -1,4 +1,4 @@
-Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
 
 Files: *
 Copyright: Copyright (C) 2006-2014 Canonical Ltd.
diff --git a/doc/README.package-tests.rst b/doc/README.package-tests.rst
index b212597..4cb60ba 100644
--- a/doc/README.package-tests.rst
+++ b/doc/README.package-tests.rst
@@ -107,14 +107,19 @@ Restrictions: restriction-name [, another-restriction-name ...]
     declare unknown restrictions will be skipped. See below for the
     defined restrictions.
 
+    Restrictions are separated by commas and/or whitespace.
+
 Features: feature-name [, another-feature-name ...]
     Declares some additional capabilities or good properties of the
     tests defined in this stanza. Any unknown features declared will be
     completely ignored. See below for the defined features.
 
+    Features are separated by commas and/or whitespace.
+
 Depends: dpkg dependency field syntax
     Declares that the specified packages must be installed for the test
-    to go ahead. This supports all features of dpkg dependencies (see
+    to go ahead. This supports all features of dpkg dependencies, including
+    the architecture qualifiers (see
     https://www.debian.org/doc/debian-policy/ch-relationships.html),
     plus the following extensions:
 
@@ -126,7 +131,7 @@ Depends: dpkg dependency field syntax
     once and without a version restriction).
 
     ``@builddeps@`` will be replaced by the package's
-    ``Build-Depends:``, ``Build-Depends-Indep:``, and
+    ``Build-Depends:``, ``Build-Depends-Indep:``, ``Build-Depends-Arch:``, and
     ``build-essential``. This is useful if you have many build
     dependencies which are only necessary for running the test suite and
     you don't want to replicate them in the test ``Depends:``. However,
@@ -164,6 +169,8 @@ Classes: class-1 [, class-2 ...]
     This is purely an informational field for autopkgtest itself and
     will be ignored.
 
+    Classes are separated by commas and/or whitespace.
+
 Any unknown fields will cause the whole stanza to be skipped.
 
 Defined restrictions
@@ -241,10 +248,63 @@ needs-recommends
     Enable installation of recommended packages in apt for the test
     dependencies. This does not affect build dependencies.
 
+flaky
+    The test is expected to fail intermittently, and is not suitable for
+    gating continuous integration. This indicates a bug in either the
+    package under test, a dependency or the test itself, but such bugs
+    can be difficult to fix, and it is often difficult to know when the
+    bug has been fixed without running the test for a while. If a
+    ``flaky`` test succeeds, it will be treated like any other
+    successful test, but if it fails it will be treated as though it
+    had been skipped.
+
+skippable
+    The test might need to be skipped for reasons that cannot be
+    described by an existing restriction such as isolation-machine or
+    breaks-testbed, but must instead be detected at runtime. If the
+    test exits with status 77 (a convention borrowed from Automake), it
+    will be treated as though it had been skipped. If it exits with any
+    other status, its success or failure will be derived from the exit
+    status and stderr as usual. Test authors must be careful to ensure
+    that ``skippable`` tests never exit with status 77 for reasons that
+    should be treated as a failure.
+
+skip-not-installable
+    This test might have test dependencies that can't be fulfilled on all
+    architectures. Therefore, when apt-get installs the dependencies, it will
+    fail. Don't treat this as a test failure, but instead treat it as if the
+    test was skipped.
+
+hint-testsuite-triggers
+    This test exists purely as a hint to suggest when rerunning the
+    tests is likely to be useful.  Specifically, it exists to
+    influence the way dpkg-source generates the Testsuite-Triggers
+    .dsc header from test metadata: the Depends for this test are
+    to be added to Testsuite-Triggers.  (Just as they are for any other
+    test.)
+
+    The test with the hint-testsuite-triggers restriction should not
+    actually be run.  Future systems which understand per-test update
+    triggering should treat the Depends of the test with this
+    restriction, as triggering packages for all tests in this
+    debian/tests/control.
+
+    The packages listed as Depends for this test are usually indirect
+    dependencies, updates to which are considered to pose a risk of
+    regressions in other tests defined in this package.
+
+    There is currently no way to specify this hint on a per-test
+    basis; but in any case the debian.org machinery is not able to
+    think about triggering individual tests.
+
 Defined features
 ----------------
 
-There are no currently defined Features.
+test-name
+    Set an explicit test name for the log heading and the ``summary`` file
+    for a ``Test-Command:`` inline test. When not given, these are
+    enumerated like ``command1``.
+
 
 Source package header
 ---------------------
diff --git a/lib/VirtSubproc.py b/lib/VirtSubproc.py
index c12d0c6..671a21d 100644
--- a/lib/VirtSubproc.py
+++ b/lib/VirtSubproc.py
@@ -337,8 +337,17 @@ def cmd_reboot(c, ce):
     if len(c) > 1 and c[1] == 'prepare-only':
         adtlog.info('state saved, waiting for testbed to reboot...')
     else:
-        execute_timeout(None, 30, auxverb +
-                        ['sh', '-c', '(sleep 3; reboot) >/dev/null 2>&1 &'])
+        (systemd_check, out, err) = execute_timeout(
+            None, 10, auxverb + ['test', '-d', '/run/systemd/system'],
+            stdout=subprocess.PIPE, stderr=subprocess.PIPE
+        )
+        if systemd_check == 0:
+            reboot_cmd = ['systemd-run', '--no-block', '--quiet', 'sh', '-c', 'sleep 3; reboot']
+        else:
+            reboot_cmd = ['sh', '-c', '(sleep 3; reboot) >/dev/null 2>&1 &']
+
+        adtlog.debug('rebooting with command: %s' % reboot_cmd)
+        execute_timeout(None, 30, auxverb + reboot_cmd)
     caller.hook_wait_reboot()
 
     # restore downtmp
diff --git a/lib/adt_testbed.py b/lib/adt_testbed.py
index b52f5b8..6f228a4 100644
--- a/lib/adt_testbed.py
+++ b/lib/adt_testbed.py
@@ -47,7 +47,8 @@ class Testbed:
     def __init__(self, vserver_argv, output_dir, user,
                  setup_commands=[], setup_commands_boot=[], add_apt_pockets=[],
                  copy_files=[], pin_packages=[], add_apt_sources=[],
-                 add_apt_releases=[], apt_default_release=None):
+                 add_apt_releases=[], apt_default_release=None,
+                 enable_apt_fallback=True):
         self.sp = None
         self.lastsend = None
         self.scratch = None
@@ -77,6 +78,7 @@ class Testbed:
         self.last_reboot_marker = ''
         self.eatmydata_prefix = []
         self.apt_pin_for_releases = []
+        self.enable_apt_fallback = enable_apt_fallback
         self.nproc = None
         self.cpu_model = None
         self.cpu_flags = None
@@ -88,6 +90,10 @@ class Testbed:
 
         adtlog.debug('testbed init')
 
+    @property
+    def su_user(self):
+        return self.user or 'root'
+
     def start(self):
         # are we running from a checkout?
         root_dir = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
@@ -389,7 +395,7 @@ class Testbed:
             self._opened(pl)
         self.modified = False
 
-    def install_deps(self, deps_new, recommends, shell_on_failure=False):
+    def install_deps(self, deps_new, recommends, shell_on_failure=False, synth_deps=[]):
         '''Install dependencies into testbed'''
         adtlog.debug('install_deps: deps_new=%s, recommends=%s' % (deps_new, recommends))
 
@@ -397,7 +403,7 @@ class Testbed:
         self.recommends_installed = recommends
         if not deps_new:
             return
-        self.satisfy_dependencies_string(', '.join(deps_new), 'install-deps', recommends, shell_on_failure=shell_on_failure)
+        self.satisfy_dependencies_string(', '.join(deps_new), 'install-deps', recommends, shell_on_failure=shell_on_failure, synth_deps=synth_deps)
 
     def needs_reset(self):
         # show what caused a reset
@@ -412,7 +418,9 @@ class Testbed:
         raise _type(m)
 
     def badpkg(self, m):
-        self.bomb(m, adtlog.BadPackageError)
+        _type = adtlog.BadPackageError
+        adtlog.debug('%s %s' % (_type.__name__, m))
+        raise _type(m)
 
     def send(self, string):
         try:
@@ -541,7 +549,60 @@ class Testbed:
                       adtlog.AutopkgtestError)
         return out
 
-    def install_apt(self, deps, recommends=False, shell_on_failure=False):
+    def is_real_package_installed(self, package):
+        '''Check if a non-virtual package is installed in the testbed'''
+        (rc, out, err) = self.execute(['dpkg-query', '--show', '-f', '${Status}', package],
+                                      stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        if rc != 0:
+            if 'no packages found' in err:
+                return False
+            self.badpkg('Failed to run dpkg-query: %s (exit code %d)' % (err, rc))
+        if out == 'install ok installed':
+            return True
+        return False
+
+    def is_virtual_package_installed(self, package):
+        '''Check if a package is installed in the testbed'''
+        (rc, out, err) = self.execute(['dpkg-query', '--show', '-f', '${Status} ${Provides}\n', '*'],
+                                      stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        if rc != 0:
+            self.badpkg('Failed to run dpkg-query: %s (exit code %d)' % (err, rc))
+            return False
+        if not out:
+            return False
+        for line in out.splitlines():
+            prefix = 'install ok installed '
+            if not line.startswith(prefix):
+                continue
+            line = line[len(prefix):]
+            for p in line.split(','):
+                (p, _, _) = p.lstrip().partition(' ')  # ' foo (== 1.0)' => 'foo'
+                if p == package:
+                    return True
+        return False
+
+    def _run_apt_install(self, what, prefix, recommends):
+        '''actually run apt-get install'''
+
+        # capture status-fd to stderr
+        (rc, _, serr) = self.execute(['/bin/sh', '-ec', '%s apt-get install '
+                                      '--assume-yes %s '
+                                      '-o APT::Status-Fd=3 '
+                                      '-o APT::Install-Recommends=%s '
+                                      '-o Dpkg::Options::=--force-confnew '
+                                      '-o Debug::pkgProblemResolver=true 3>&2 2>&1' %
+                                      (prefix, what, recommends)],
+                                     kind='install', stderr=subprocess.PIPE)
+        if rc != 0:
+            adtlog.debug('apt-get install %s failed; status-fd:\n%s' % (what, serr))
+            # check if apt failed during package download, which might be a
+            # transient error, so retry
+            if 'dlstatus:' in serr and 'pmstatus:' not in serr:
+                raise adtlog.AptDownloadError
+            else:
+                raise adtlog.AptPermanentError
+
+    def install_apt(self, deps, recommends=False, shell_on_failure=False, synth_deps=[]):
         '''Install dependencies with apt-get into testbed
 
         This requires root privileges and a writable file system.
@@ -575,28 +636,60 @@ Description: satisfy autopkgtest test dependencies
         download_fail_retries = 3
         while True:
             self.check_exec(['dpkg', '--unpack', deb.tb], stdout=subprocess.PIPE)
-            # capture status-fd to stderr
-            (rc, _, serr) = self.execute(['/bin/sh', '-ec', '%s apt-get install '
-                                          '--assume-yes --fix-broken '
-                                          '-o APT::Status-Fd=3 '
-                                          '-o APT::Install-Recommends=%s '
-                                          '-o Dpkg::Options::=--force-confnew '
-                                          '-o Debug::pkgProblemResolver=true 3>&2 2>&1' %
-                                          (' '.join(self.eatmydata_prefix), recommends)],
-                                         kind='install', stderr=subprocess.PIPE)
-            if rc != 0:
-                adtlog.debug('apt-get install failed; status-fd:\n%s' % serr)
-                # check if apt failed during package download, which might be a
-                # transient error, so retry
-                if 'dlstatus:' in serr and 'pmstatus:' not in serr:
-                    download_fail_retries -= 1
-                    if download_fail_retries > 0:
-                        adtlog.warning('apt failed to download packages, retrying in 10s...')
-                        time.sleep(10)
-                        continue
+
+            # if '@' was provided as a test dependency (as seen in synth_deps),
+            # we explicitly install the corresponding binary packages in case
+            # the dependencies on those were satisfied by versioned Provides
+
+            rc = 0
+            try:
+                self._run_apt_install('--fix-broken', ' '.join(self.eatmydata_prefix), recommends)
+                need_explicit_install = []
+                for dep in synth_deps:
+                    if dep != list(dep):
+                        # simple test dependency (no alternatives)
+                        if self.is_real_package_installed(dep):
+                            continue
+                        if self.is_virtual_package_installed(dep):
+                            need_explicit_install.append(dep)
+                            continue
+                        adtlog.warning('package %s is not installed though it should be' % dep)
                     else:
-                        self.bomb('apt repeatedly failed to download packages')
+                        # figure out which test dependency alternative got installed
+                        installed_virtual_packages = []
+                        found_a_real_package = False
+                        for pkg in dep:
+                            if self.is_real_package_installed(pkg):
+                                # no need to install anything from this set of alternatives
+                                found_a_real_package = True
+                                break
+                            if self.is_virtual_package_installed(pkg):
+                                installed_virtual_packages.append(pkg)
+                        if found_a_real_package:
+                            continue
+                        if not installed_virtual_packages:
+                            adtlog.warning('no alternative in %s is installed though one should be' % dep)
+                        if len(installed_virtual_packages) > 1:
+                            adtlog.warning('more than one test dependency alternative in %s installed as a virtual package, installing the first one (%s) as the real package' % (dep, installed_virtual_packages[0]))
+                        need_explicit_install.append(installed_virtual_packages[0])
+
+                if need_explicit_install:
+                    adtlog.debug('installing real packages of test dependencies: %s' % need_explicit_install)
+                    self._run_apt_install(' '.join(need_explicit_install), ' '.join(self.eatmydata_prefix), recommends)
+
+            # check if apt failed during package download, which might be a
+            # transient error, so retry
+            except adtlog.AptDownloadError:
+                download_fail_retries -= 1
+                if download_fail_retries > 0:
+                    adtlog.warning('apt failed to download packages, retrying in 10s...')
+                    time.sleep(10)
+                    continue
+                else:
+                    self.bomb('apt repeatedly failed to download packages')
 
+            except adtlog.AptPermanentError:
+                rc = -1
                 if shell_on_failure:
                     self.run_shell()
             else:
@@ -607,7 +700,7 @@ Description: satisfy autopkgtest test dependencies
                                   stderr=subprocess.PIPE)[0]
 
             if rc != 0:
-                if self.apt_pin_for_releases:
+                if self.apt_pin_for_releases and self.enable_apt_fallback:
                     release = self.apt_pin_for_releases.pop()
                     adtlog.warning('Test dependencies are unsatisfiable with using apt pinning. '
                                    'Retrying with using all packages from %s' % release)
@@ -618,10 +711,17 @@ Description: satisfy autopkgtest test dependencies
 
                 if shell_on_failure:
                     self.run_shell()
-                self.badpkg('Test dependencies are unsatisfiable. A common reason is '
-                            'that your testbed is out of date with respect to the '
-                            'archive, and you need to use a current testbed or run '
-                            'apt-get update or use -U.')
+                if self.enable_apt_fallback:
+                    self.badpkg('Test dependencies are unsatisfiable. A common reason is '
+                                'that your testbed is out of date with respect to the '
+                                'archive, and you need to use a current testbed or run '
+                                'apt-get update or use -U.')
+                else:
+                    self.badpkg('Test dependencies are unsatisfiable. A common reason is '
+                                'that the requested apt pinning prevented dependencies '
+                                'from the non-default suite to be installed. In that '
+                                'case you need to add those dependencies to the pinning '
+                                'list.')
             break
 
         # remove autopkgtest-satdep to avoid confusing tests, but avoid marking our
@@ -842,7 +942,7 @@ fi
         # we don't want su -l here which resets the environment from
         # self.execute(); so emulate the parts that we want
         # FIXME: move "run as user" as an argument of execute()/check_exec() and run with -l
-        self.check_exec(['su', '--shell=/bin/sh', self.user, '-c',
+        self.check_exec(['su', '--shell=/bin/sh', self.su_user, '-c',
                          ('export USER=%s;' % self.user) +
                          '. /etc/profile >/dev/null 2>&1 || true; '
                          ' . ~/.profile >/dev/null 2>&1 || true; '
@@ -915,7 +1015,7 @@ fi
             self.bomb('Failed to update click AppArmor rules')
 
     def satisfy_dependencies_string(self, deps, what, recommends=False,
-                                    build_dep=False, shell_on_failure=False):
+                                    build_dep=False, shell_on_failure=False, synth_deps=[]):
         '''Install dependencies from a string into the testbed'''
 
         adtlog.debug('%s: satisfying %s' % (what, deps))
@@ -957,7 +1057,7 @@ fi
         adtlog.debug('can use apt-get on testbed: %s' % can_apt_get)
 
         if can_apt_get:
-            self.install_apt(deps, recommends, shell_on_failure)
+            self.install_apt(deps, recommends, shell_on_failure, synth_deps)
         else:
             self.install_tmp(deps, recommends)
 
@@ -1069,7 +1169,7 @@ fi
             # we don't want -l here which resets the environment from
             # self.execute(); so emulate the parts that we want
             # FIXME: move "run as user" as an argument of execute()/check_exec() and run with -l
-            test_argv = ['su', '-s', '/bin/bash', self.user, '-c']
+            test_argv = ['su', '-s', '/bin/bash', self.su_user, '-c']
 
             if 'rw-build-tree' in test.restrictions:
                 self.check_exec(['chown', '-R', self.user, tree.tb])
@@ -1160,6 +1260,8 @@ fi
 
         if timeout:
             test.failed('timed out')
+        elif rc == 77 and 'skippable' in test.restrictions:
+            test.set_skipped('exit status 77 and marked as skippable')
         elif rc != 0:
             test.failed('non-zero exit status %d' % rc)
         elif se_size != 0 and 'allow-stderr' not in test.restrictions:
@@ -1265,7 +1367,7 @@ fi
         if self.default_release is None:
 
             script = 'SRCS=$(ls /etc/apt/sources.list /etc/apt/sources.list.d/*.list 2>/dev/null|| true); '
-            script += '''sed -rn '/^(deb|deb-src) (http|https|file):/ { s/\[.*\] +//; s/^[^ ]+ +[^ ]* +([^ -]+) +.*$/\\1/p }' $SRCS | head -n1'''
+            script += '''sed -rn '/^(deb|deb-src) +(\[.*\] *)?(http|https|file):/ { s/\[.*\] +//; s/^[^ ]+ +[^ ]* +([^ -]+) +.*$/\\1/p }' $SRCS | head -n1'''
             self.default_release = self.check_exec(['sh', '-ec', script], stdout=True).strip()
 
         return self.default_release
diff --git a/lib/adtlog.py b/lib/adtlog.py
index 8f18386..4405c76 100644
--- a/lib/adtlog.py
+++ b/lib/adtlog.py
@@ -125,3 +125,11 @@ def bomb(m):
 
 def badpkg(m):
     raise BadPackageError(m)
+
+
+class AptPermanentError(Exception):
+    pass
+
+
+class AptDownloadError(Exception):
+    pass
diff --git a/lib/autopkgtest_args.py b/lib/autopkgtest_args.py
index e681bc0..0e8e0f3 100644
--- a/lib/autopkgtest_args.py
+++ b/lib/autopkgtest_args.py
@@ -339,6 +339,14 @@ for details.'''
                         help='Set "parallel=N" DEB_BUILD_OPTION for building '
                         'packages (default: number of available processors)')
     g_misc.add_argument(
+        '--no-apt-fallback',
+        dest='enable_apt_fallback',
+        action='store_false',
+        default=True,
+        help='Disable the apt fallback which is used with --apt-pocket or '
+             '--pin-packages in case installation of dependencies fails due to '
+             'strict pinning')
+    g_misc.add_argument(
         '-h', '--help', action='help', default=argparse.SUPPRESS,
         help='show this help message and exit')
 
diff --git a/lib/testdesc.py b/lib/testdesc.py
index c6bb3ed..dfc9ee3 100644
--- a/lib/testdesc.py
+++ b/lib/testdesc.py
@@ -42,7 +42,9 @@ import adtlog
 
 known_restrictions = ['rw-build-tree', 'breaks-testbed', 'needs-root',
                       'build-needed', 'allow-stderr', 'isolation-container',
-                      'isolation-machine', 'needs-recommends', 'needs-reboot']
+                      'isolation-machine', 'needs-recommends', 'needs-reboot',
+                      'flaky', 'skippable',
+                      'skip-not-installable']
 
 
 class Unsupported(Exception):
@@ -80,7 +82,7 @@ class Test:
     actions.
     '''
     def __init__(self, name, path, command, restrictions, features, depends,
-                 clicks, installed_clicks):
+                 clicks, installed_clicks, synth_depends):
         '''Create new test description
 
         A test must have either "path" or "command", the respective other value
@@ -93,6 +95,7 @@ class Test:
         @depends: string list of test dependencies (packages)
         @clicks: path list of click packages to install for this test
         @installed_clicks: names of already installed clicks for this test
+        @synth_depends: string list of synthesized test dependencies (packages)
         '''
         if '/' in name:
             raise Unsupported(name, 'test name may not contain / character')
@@ -111,8 +114,10 @@ class Test:
         self.depends = depends
         self.clicks = clicks
         self.installed_clicks = installed_clicks
+        self.synth_depends = synth_depends
         # None while test hasn't run yet; True: pass, False: fail
         self.result = None
+        self.skipped = False
         adtlog.debug('Test defined: name %s path %s command "%s" '
                      'restrictions %s features %s depends %s clicks %s '
                      'installed clicks %s' %
@@ -125,11 +130,23 @@ class Test:
         self.result = True
         adtlog.report(self.name, 'PASS')
 
+    def set_skipped(self, reason):
+        '''Mark test as skipped'''
+        # This isn't called skipped() to avoid clashing with the boolean
+        # attribute.
+
+        self.skipped = True
+        self.result = True
+        adtlog.report(self.name, 'SKIP ' + reason)
+
     def failed(self, reason):
         '''Mark test as failed'''
 
         self.result = False
-        adtlog.report(self.name, 'FAIL ' + reason)
+        if 'flaky' in self.restrictions:
+            adtlog.report(self.name, 'FLAKY ' + reason)
+        else:
+            adtlog.report(self.name, 'FAIL ' + reason)
 
     def check_testbed_compat(self, caps):
         '''Check for restrictions incompatible with test bed capabilities.
@@ -221,6 +238,7 @@ def _debian_check_unknown_fields(name, record):
 
 def _debian_packages_from_source(srcdir):
     packages = []
+    packages_no_arch = []
 
     for st in parse_rfc822(os.path.join(srcdir, 'debian/control')):
         if 'Package' not in st:
@@ -232,11 +250,12 @@ def _debian_packages_from_source(srcdir):
             continue
         arch = st['Architecture']
         if arch in ('all', 'any'):
-            packages.append('%s (>= 0~)' % st['Package'])
+            packages.append(st['Package'])
         else:
-            packages.append('%s (>= 0~) [%s]' % (st['Package'], arch))
+            packages.append('%s [%s]' % (st['Package'], arch))
+        packages_no_arch.append(st['Package'])
 
-    return packages
+    return (packages, packages_no_arch)
 
 
 def _debian_build_deps_from_source(srcdir, testbed_arch):
@@ -246,6 +265,8 @@ def _debian_build_deps_from_source(srcdir, testbed_arch):
             deps += st['Build-depends']
         if 'Build-depends-indep' in st:
             deps += ', ' + st['Build-depends-indep']
+        if 'Build-depends-arch' in st:
+            deps += ', ' + st['Build-depends-arch']
 
     # resolve arch specific dependencies and build profiles
     perl = subprocess.Popen(['perl', '-'], stdin=subprocess.PIPE,
@@ -274,7 +295,7 @@ def _debian_build_deps_from_source(srcdir, testbed_arch):
 dep_re = re.compile(
     r'(?P<package>[a-z0-9+-.]+)(?::native)?\s*'
     r'(\((?P<relation><<|<=|>=|=|>>)\s*(?P<version>[^\)]*)\))?'
-    r'(\s*\[[[a-z0-9+-.! ]+\])?$')
+    r'(\s*\[(?P<arch>[[a-z0-9+-.! ]+)\])?$')
 
 
 def _debian_check_dep(testname, dep):
@@ -296,6 +317,38 @@ def _debian_check_dep(testname, dep):
             # too old python-debian, skip the check
             pass
 
+    return (m.group('package'), m.group('version'))
+
+
+def _synthesize_deps(dep, testbed_arch):
+    '''Ensure that apt can install synthesized Depends
+
+    Test Depends may have architecture qualifiers we need to check,
+    because apt command-line can't handle those and can't do the check.
+    (Policy says the architecture qualifier should not be in the binary
+    control file). We'll ignore the version here.
+    '''
+
+    dep = dep.strip()
+    m = dep_re.match(dep)
+
+    arch_matches = 'y'
+    if m.group('arch') is not None:
+        arch_matches = False
+        perl = subprocess.Popen(['perl', '-'], stdin=subprocess.PIPE,
+                                stdout=subprocess.PIPE)
+        code = '''use Dpkg::Deps;
+                  print deps_parse('foo [%s]')->arch_is_concerned('%s') ? 'y' : 'n';
+                  ''' % (m.group('arch'), testbed_arch)
+        arch_matches = perl.communicate(code.encode('UTF-8'))[0].decode('UTF-8').strip()
+        if perl.returncode != 0:
+            raise InvalidControl('source', 'Invalid (Test-)Depends architecture qualifiers')
+
+    if arch_matches == 'y':
+        return m.group('package')
+    else:
+        return None
+
 
 def _parse_debian_depends(testname, dep_str, srcdir, testbed_arch):
     '''Parse Depends: line in a Debian package
@@ -307,6 +360,8 @@ def _parse_debian_depends(testname, dep_str, srcdir, testbed_arch):
     dependencies.
     '''
     deps = []
+    synthdeps = []
+    (my_packages, my_packages_no_arch) = _debian_packages_from_source(srcdir)
     for alt_group_str in dep_str.split(','):
         alt_group_str = alt_group_str.strip()
         if not alt_group_str:
@@ -314,19 +369,35 @@ def _parse_debian_depends(testname, dep_str, srcdir, testbed_arch):
             continue
         adtlog.debug('processing dependency %s' % alt_group_str)
         if alt_group_str == '@':
-            for d in _debian_packages_from_source(srcdir):
+            for d in my_packages:
                 adtlog.debug('synthesised dependency %s' % d)
                 deps.append(d)
+                s = _synthesize_deps(d, testbed_arch)
+                if s:
+                    synthdeps.append(s)
         elif alt_group_str == '@builddeps@':
             for d in _debian_build_deps_from_source(srcdir, testbed_arch):
                 adtlog.debug('synthesised dependency %s' % d)
                 deps.append(d)
         else:
+            synthdep_alternatives = []
             for dep in alt_group_str.split('|'):
-                _debian_check_dep(testname, dep)
+                (pkg, version) = _debian_check_dep(testname, dep)
+                if pkg not in my_packages_no_arch:
+                    synthdep_alternatives = []
+                    break
+                s = _synthesize_deps(dep, testbed_arch)
+                if s:
+                    synthdep_alternatives.append(s)
+            if synthdep_alternatives:
+                    adtlog.debug('marked alternatives %s as a synthesised dependency' % synthdep_alternatives)
+                    if len(synthdep_alternatives) > 1:
+                        synthdeps.append(synthdep_alternatives)
+                    else:
+                        synthdeps.append(synthdep_alternatives[0])
             deps.append(alt_group_str)
 
-    return deps
+    return (deps, synthdeps)
 
 
 def _autodep8(srcdir):
@@ -411,10 +482,11 @@ def parse_debian_source(srcdir, testbed_caps, testbed_arch, control_path=None,
 
             if 'Tests' in record:
                 test_names = record['Tests'].replace(',', ' ').split()
-                depends = _parse_debian_depends(test_names[0],
-                                                record.get('Depends', '@'),
-                                                srcdir,
-                                                testbed_arch)
+                (depends, synth_depends) = _parse_debian_depends(
+                    test_names[0],
+                    record.get('Depends', '@'),
+                    srcdir,
+                    testbed_arch)
                 if 'Test-command' in record:
                     raise InvalidControl('*', 'Only one of "Tests" or '
                                          '"Test-Command" may be given')
@@ -426,15 +498,16 @@ def parse_debian_source(srcdir, testbed_caps, testbed_arch, control_path=None,
                 _debian_check_unknown_fields(test_names[0], record)
                 for n in test_names:
                     test = Test(n, os.path.join(test_dir, n), None,
-                                restrictions, features, depends, [], [])
+                                restrictions, features, depends, [], [], synth_depends)
                     test.check_testbed_compat(testbed_caps)
                     tests.append(test)
             elif 'Test-command' in record:
                 command = record['Test-command']
-                depends = _parse_debian_depends(command,
-                                                record.get('Depends', '@'),
-                                                srcdir,
-                                                testbed_arch)
+                (depends, synth_depends) = _parse_debian_depends(
+                    command,
+                    record.get('Depends', '@'),
+                    srcdir,
+                    testbed_arch)
                 if feature_test_name is None:
                     command_counter += 1
                     name = 'command%i' % command_counter
@@ -442,7 +515,7 @@ def parse_debian_source(srcdir, testbed_caps, testbed_arch, control_path=None,
                     name = feature_test_name
                 _debian_check_unknown_fields(name, record)
                 test = Test(name, None, command, restrictions, features,
-                            depends, [], [])
+                            depends, [], [], synth_depends)
                 test.check_testbed_compat(testbed_caps)
                 tests.append(test)
             else:
@@ -532,7 +605,7 @@ def parse_click_manifest(manifest, testbed_caps, clickdeps, use_installed,
         try:
             test = Test(name, desc.get('path'), desc.get('command'),
                         desc.get('restrictions', []), desc.get('features', []),
-                        desc.get('depends', []), clickdeps, installed_clicks)
+                        desc.get('depends', []), clickdeps, installed_clicks, [])
             test.check_testbed_compat(testbed_caps)
             tests.append(test)
         except Unsupported as u:
diff --git a/runner/autopkgtest b/runner/autopkgtest
index 90a3fd9..24fd84a 100755
--- a/runner/autopkgtest
+++ b/runner/autopkgtest
@@ -138,27 +138,60 @@ def setup_trace():
 def run_tests(tests, tree):
     global errorcode, testbed
 
+    # We should not get here if we have had an error other than skipping
+    # tests
+    assert errorcode in (0, 2), errorcode
+
     if not tests:
         # if we have skipped tests, don't claim that we don't have any
-        if not errorcode & 2:
+        if errorcode == 0:
             adtlog.report('*', 'SKIP no tests in this package')
-            errorcode |= 8
+
+        errorcode = 8
         return
 
+    any_passed = False
+
     for t in tests:
         # Set up clean test bed with given dependencies
         adtlog.info('test %s: preparing testbed' % t.name)
         testbed.reset(t.depends, 'needs-recommends' in t.restrictions)
         binaries.publish()
-        testbed.install_deps(t.depends, 'needs-recommends' in t.restrictions, opts.shell_fail)
+        doTest = True
+        try:
+            testbed.install_deps(t.depends, 'needs-recommends' in t.restrictions, opts.shell_fail, t.synth_depends)
+        except adtlog.BadPackageError as e:
+            if 'skip-not-installable' in t.restrictions:
+                errorcode |= 2
+                adtlog.report(t.name, 'SKIP installation fails and skip-not-installable set')
+            else:
+                errorcode |= 12
+                adtlog.report(t.name, 'FAIL badpkg')
+                adtlog.preport('blame: ' + ' '.join(blamed))
+                adtlog.preport('badpkg: ' + str(e))
+            doTest = False
+
+        if doTest:
+            testbed.run_test(tree, t, opts.env, opts.shell_fail, opts.shell,
+                             opts.build_parallel)
+            if t.skipped:
+                errorcode |= 2
+            elif not t.result:
+                if 'flaky' in t.restrictions:
+                    errorcode |= 2
+                else:
+                    errorcode |= 4
+            else:
+                any_passed = True
 
-        testbed.run_test(tree, t, opts.env, opts.shell_fail, opts.shell,
-                         opts.build_parallel)
-        if not t.result:
-            errorcode |= 4
         if 'breaks-testbed' in t.restrictions:
             testbed.needs_reset()
 
+    if errorcode in (0, 2) and not any_passed:
+        # If we have skipped or ignored every test, set the same exit
+        # status as if we didn't have any tests
+        errorcode = 8
+
     testbed.needs_reset()
 
 
@@ -362,11 +395,9 @@ def build_source(kind, arg, built_binaries):
         #
         # apt-get source is terribly noisy; only show what gets downloaded
         #
-        # apt in Ubuntu 12.04 does not yet understand --only-source, so add a fallback
-        #
         # very old source packages don't have Package-List: yet, fall back to Binary:
         # (Binary: is generally not sufficient as it gets truncated for long lists)
-        create_command = ('pkgs=$(apt-cache showsrc --only-source %(src)s || [ $? != 100 ] || apt-cache showsrc %(src)s); '
+        create_command = ('pkgs=$(apt-cache showsrc --only-source %(src)s); '
                           'pkgs=$(echo "$pkgs\n" | awk "/^Package: / { if (\$2 != \\"%(src)s\\") { skippar=1; next; } else { skippar=0}} { if (skippar) next; }    /^Binary:/ { sub(/^Binary:/, \\"\\"); gsub(/,/, \\"\\"); split(\$0,oldpkgs)}; /^Package-List:/ { inlist=1; have_pl=1; delete thissrc; if (\$2) thissrc[\$2] = 1; next } (/^ / && inlist == 1) { thissrc[\$1] = 1; next } { if (!inlist) next; inlist=0; if (intersect) {for (p in pkgs) { if (!(p in thissrc)) delete pkgs[p]; else remaining=1}; if (!remaining) {for (p in thissrc) {pkgs[p] = 1}} } else { for (p in thissrc) { pkgs[p] = 1}; intersect=1 } } END {if (have_pl) { for (p in pkgs) print p } else {for (p in oldpkgs) print oldpkgs[p]} }");'
                           ' [ -n "$pkgs" ] || exit 1; '
                           ' for pkg in $pkgs; do'
@@ -460,6 +491,7 @@ def build_source(kind, arg, built_binaries):
             d = deb822.Deb822(sequence=f)
             bd = d.get('Build-Depends', '')
             bdi = d.get('Build-Depends-Indep', '')
+            bda = d.get('Build-Depends-Arch', '')
 
         # determine build command and build-essential packages
         build_essential = ['build-essential']
@@ -472,7 +504,7 @@ def build_source(kind, arg, built_binaries):
             if testbed.user or 'root-on-testbed' not in testbed.caps:
                 build_essential += ['fakeroot']
 
-        testbed.satisfy_dependencies_string(bd + ', ' + bdi + ', ' + ', '.join(build_essential), arg,
+        testbed.satisfy_dependencies_string(bd + ', ' + bdi + ', ' + bda + ', ' + ', '.join(build_essential), arg,
                                             build_dep=True, shell_on_failure=opts.shell_fail)
 
         # keep patches applied for tests
@@ -650,6 +682,7 @@ def main():
                                       setup_commands_boot=opts.setup_commands_boot,
                                       add_apt_pockets=opts.apt_pocket,
                                       copy_files=opts.copy,
+                                      enable_apt_fallback=opts.enable_apt_fallback,
                                       add_apt_sources=getattr(opts, 'add_apt_sources', []),
                                       add_apt_releases=getattr(opts, 'add_apt_releases', []),
                                       pin_packages=opts.pin_packages,
diff --git a/runner/autopkgtest.1 b/runner/autopkgtest.1
index 168a7a0..2a98c32 100644
--- a/runner/autopkgtest.1
+++ b/runner/autopkgtest.1
@@ -3,8 +3,9 @@
 autopkgtest \- test an installed binary package using the source package's tests
 .SH SYNOPSIS
 .B autopkgtest
-.IR [ options ...]
-.IR "testsrc " [ testbinary "...]"
+.RI [ options ...]
+.RI [ testbinary ...]
+.I testsrc
 .B \-\-
 .I virt\-server
 .RI [ virt\-server\-arg ...]
@@ -442,12 +443,15 @@ individual servers for how to use them.
 .SH OUTPUT FORMAT
 During a normal test run, one line is printed for each test.  This
 consists of a short string identifying the test, some horizontal
-whitespace, and either
-.B PASS
-or
-.BR FAIL " reason"
+whitespace, and one of
+.BR PASS ,
+.B FAIL
+.IR reason ,
+.B SKIP
+.IR reason ,
 or
-.BR SKIP " reason"
+.B FLAKY
+.I reason
 where the pass/fail indication is separated by any reason by some
 horizontal whitespace.
 
@@ -455,6 +459,15 @@ The string to identify the test consists of a short alphanumeric
 string invented by \fBautopkgtest\fR to distinguish different command-line
 arguments, the \fIargid\fR, followed by a hyphen and the test name.
 
+.B SKIP
+indicates that a test was not run, or that the test code was started
+but detected that the test could not complete, for instance because a
+required resource was not available.
+
+.B FLAKY
+indicates that a test would ordinarily have failed, but because this
+particular test is known to be unreliable, the failure was ignored.
+
 Sometimes a
 .B SKIP
 will be reported when the name of the test is not known or not
@@ -520,7 +533,7 @@ argument value.
 .SH EXIT STATUS
 0	all tests passed
 .br
-2	at least one test skipped
+2	at least one test was skipped (or at least one flaky test failed)
 .br
 4	at least one test failed
 .br
@@ -530,6 +543,8 @@ argument value.
 .br
 12	erroneous package
 .br
+14	erroneous package and at least one test skipped
+.br
 16	testbed failure
 .br
 20	other unexpected failures including bad usage
diff --git a/tests/autopkgtest b/tests/autopkgtest
index aa6e431..ceb0ee6 100755
--- a/tests/autopkgtest
+++ b/tests/autopkgtest
@@ -64,6 +64,12 @@ have_ubuntu_device_flash = subprocess.call(['which', 'ubuntu-device-flash'],
 have_autodep8 = subprocess.call(['which', 'autodep8'], stdout=subprocess.PIPE) == 0
 have_git = subprocess.call(['which', 'git'], stdout=subprocess.PIPE) == 0
 
+host_arch = subprocess.run(['dpkg-architecture', '-q', 'DEB_HOST_ARCH'],
+                           stdout=subprocess.PIPE, universal_newlines=True).stdout.strip()
+
+host_os = subprocess.run(['dpkg-architecture', '-q', 'DEB_HOST_ARCH_OS'],
+                         stdout=subprocess.PIPE, universal_newlines=True).stdout.strip()
+
 
 class AdtTestCase(unittest.TestCase):
     '''Base class with common test setup'''
@@ -158,6 +164,18 @@ class AdtTestCase(unittest.TestCase):
         (out, err) = p.communicate()
         return (p.returncode, out.decode('UTF-8', 'replace'), err.decode('UTF-8', 'replace'))
 
+    @property
+    def has_isolation_machine(self):
+        return self.virt_args[0] in ['qemu', 'null']
+
+    @property
+    def has_isolation_container(self):
+        return self.virt_args[0] not in ['chroot', 'schroot']
+
+    @property
+    def can_revert_full_system(self):
+        return self.virt_args[0] not in ['null', 'chroot', 'schroot']
+
 
 class DebTestsAll:
     '''Common deb tests for all runners'''
@@ -189,21 +207,35 @@ class DebTestsAll:
         # should log package version
         self.assertIn('testing package testpkg version 1\n', err)
 
+    @unittest.skipIf((os.getuid() == 0 and
+                      os.path.exists('/usr/share/doc/aspell-doc')),
+                     'needs aspell-doc uninstalled if run as root')
     def test_tree_build_needed_success(self):
         '''source tree, build-needed restriction, test success'''
 
         p = self.build_src('Tests: pass\nDepends: coreutils\nRestrictions: build-needed\n',
                            {'pass': '#!/bin/sh -e\n./test_built | grep -q "built script OK"\n'
                             './test_abspath | grep -q "built script OK"\necho GOOD'})
-        # add Build-Depends-Indep: and a build profile package
-        subprocess.check_call(['sed', '-i', '/^Build-Depends:/ a\Build-Depends-Indep: dpkg-dev, nonexisting <cross>',
-                               os.path.join(p, 'debian', 'control')])
+
+        if os.getuid() == 0:
+            # Add Build-Depends-Indep:, Build-Depends-Arch and a build
+            # profile package. We can only do this if we're root, because
+            # packages that we unpack non-system-wide as non-root are
+            # invisible to dpkg-checkbuilddeps so the build would fail.
+            subprocess.check_call(['sed', '-i', '-e',
+                                   '/^Build-Depends:/ a\Build-Depends-Indep: dpkg-dev, nonexisting <cross>',
+                                   '-e',
+                                   '/^Build-Depends:/ a\Build-Depends-Arch: aspell-doc',
+                                   os.path.join(p, 'debian', 'control')])
 
         (code, out, err) = self.runtest(['--no-built-binaries', p])
         # test should succeed
         self.assertEqual(code, 0, out + err)
         self.assertRegex(out, 'pass\s+PASS', out)
 
+        if os.getuid() == 0:
+            self.assertIn('Unpacking aspell-doc', out)
+
         # should build package
         self.assertIn('dh build', err)
 
@@ -222,26 +254,34 @@ class DebTestsAll:
 
         (code, out, err) = self.runtest(['-B', p])
 
-        if self.virt_args[0] in ['qemu', 'null']:
-            self.assertEqual(code, 0, out + err)
+        if self.has_isolation_machine:
             self.assertRegex(out, 'im\s+PASS', out)
             self.assertIn('\nmachine ok\n', out)
         else:
-            self.assertEqual(code, 2, out + err)
             self.assertRegex(out, 'im\s+SKIP .*machine', out)
             self.assertNotIn('machine ok', out)
 
-        if self.virt_args[0] in ['chroot', 'schroot']:
+        if not self.has_isolation_container:
             self.assertRegex(out, 'ic\s+SKIP .*container', out)
             self.assertNotIn('container ok', out)
         else:
             self.assertRegex(out, 'ic\s+PASS', out)
             self.assertIn('container ok\n', out)
 
+        if self.has_isolation_machine and self.has_isolation_container:
+            # all tests run
+            self.assertEqual(code, 0, out + err)
+        elif self.has_isolation_machine or self.has_isolation_container:
+            # one test run, one skipped
+            self.assertEqual(code, 2, out + err)
+        else:
+            # all skipped
+            self.assertEqual(code, 8, out + err)
+
     def test_breaks_testbed(self):
         '''breaks-testbed restriction'''
 
-        p = self.build_src('Tests: zap boom\nDepends:\nRestrictions: needs-root breaks-testbed',
+        p = self.build_src('Tests: zap boom\nDepends:\nRestrictions: allow-stderr needs-root breaks-testbed',
                            {'zap': '#!/bin/sh\ntouch /zap\n'
                             '[ -d "$HOME" ] || echo "HOME not set" >&2\n'
                             'apt-get install -y aspell-doc\n',
@@ -249,9 +289,9 @@ class DebTestsAll:
 
         (code, out, err) = self.runtest(['--no-built-binaries', p, '-d'])
 
-        if self.virt_args[0] in ['null', 'chroot', 'schroot']:
+        if not self.can_revert_full_system:
             # test should be skipped as these runners doesn't provide revert-full-system
-            self.assertEqual(code, 2, err)
+            self.assertEqual(code, 8, err)
             self.assertRegex(out, 'zap\s+SKIP Test breaks testbed')
             # FIXME: we should see this too!
             # self.assertRegex(out, 'boom\s+SKIP Test breaks testbed')
@@ -264,6 +304,22 @@ class DebTestsAll:
             self.assertRegex(out, 'boom\s+PASS')
             self.assertIn('Unpacking aspell-doc', out)
 
+    def test_flaky_success(self):
+        '''A flaky test succeeds'''
+        p = self.build_src('Tests: unreproducible\nRestrictions: flaky\nDepends: coreutils\n',
+                           {'unreproducible': '#!/bin/sh\necho I am fine\n'})
+        (code, out, err) = self.runtest(['-d', '--no-built-binaries', p])
+        self.assertEqual(code, 0, err)
+        self.assertRegex(out, 'unreproducible\s+PASS', out)
+
+    def test_skippable_success(self):
+        '''A skippable test succeeds'''
+        p = self.build_src('Tests: needs-magic\nRestrictions: skippable\nDepends: coreutils\n',
+                           {'needs-magic': '#!/bin/sh\necho I am fine\n'})
+        (code, out, err) = self.runtest(['-d', '--no-built-binaries', p])
+        self.assertEqual(code, 0, err)
+        self.assertRegex(out, 'needs-magic\s+PASS', out)
+
 
 class DebTestsFailureModes:
     '''Common deb tests for handling various failure modes
@@ -359,6 +415,59 @@ class DebTestsFailureModes:
         # no restricted dependencies functionality as they are already installed
         self.assertNotIn('will only work for some packages', err)
 
+    def test_flaky_fail(self):
+        '''A flaky test fails'''
+        p = self.build_src('Tests: unreproducible\nRestrictions: flaky\nDepends: coreutils\n',
+                           {'unreproducible': '#!/bin/sh\nexit 1\n'})
+        (code, out, err) = self.runtest(['-d', '--no-built-binaries', p])
+        self.assertEqual(code, 8, err)
+        self.assertRegex(out, 'unreproducible\s+FLAKY', out)
+
+    def test_flaky_fail_mixed(self):
+        '''A flaky test fails, but another test passes'''
+        p = self.build_src('Tests: unreproducible ok\nRestrictions: flaky\nDepends: coreutils\n',
+                           {'unreproducible': '#!/bin/sh\nexit 1\n',
+                            'ok': '#!/bin/sh\n'})
+        (code, out, err) = self.runtest(['-d', '--no-built-binaries', p])
+        self.assertEqual(code, 2, err)
+        self.assertRegex(out, 'unreproducible\s+FLAKY', out)
+        self.assertRegex(out, 'ok\s+PASS', out)
+
+    def test_skippable_skipped(self):
+        '''A skippable test skips itself'''
+        p = self.build_src('Tests: needs-magic\nRestrictions: skippable\nDepends: coreutils\n',
+                           {'needs-magic': '#!/bin/sh\necho Cannot run >&2\nexit 77\n'})
+        (code, out, err) = self.runtest(['-d', '--no-built-binaries', p])
+        self.assertEqual(code, 8, err)
+        self.assertRegex(out, 'needs-magic\s+SKIP', out)
+
+    def test_skippable_skipped_mixed(self):
+        '''A skippable test skips itself, but another test passes'''
+        p = self.build_src('Tests: needs-magic ok\nRestrictions: skippable\nDepends: coreutils\n',
+                           {'needs-magic': '#!/bin/sh\necho Cannot run >&2\nexit 77\n',
+                            'ok': '#!/bin/sh\n'})
+        (code, out, err) = self.runtest(['-d', '--no-built-binaries', p])
+        self.assertEqual(code, 2, err)
+        self.assertRegex(out, 'needs-magic\s+SKIP', out)
+        self.assertRegex(out, 'ok\s+PASS', out)
+
+    def test_skippable_fail(self):
+        '''A skippable test fails'''
+        p = self.build_src('Tests: needs-magic\nRestrictions: skippable\nDepends: coreutils\n',
+                           {'needs-magic': '#!/bin/sh\nexit 1\n'})
+        (code, out, err) = self.runtest(['-d', '--no-built-binaries', p])
+        self.assertEqual(code, 4, err)
+        self.assertRegex(out, 'needs-magic\s+FAIL non-zero exit status 1', out)
+
+    def test_unskippable(self):
+        '''A non-skippable test fails with an exit status that happens
+        to match the skippable API.'''
+        p = self.build_src('Tests: no-magic\nDepends: coreutils\n',
+                           {'no-magic': '#!/bin/sh\nexit 77\n'})
+        (code, out, err) = self.runtest(['-d', '--no-built-binaries', p])
+        self.assertEqual(code, 4, err)
+        self.assertRegex(out, 'no-magic\s+FAIL non-zero exit status 77', out)
+
 
 class NullRunner(AdtTestCase, DebTestsAll, DebTestsFailureModes):
     def __init__(self, *args, **kwargs):
@@ -1399,8 +1508,69 @@ if ($pid) { # parent
 
         p = self.build_src('Test-Command: true\nDepends:\nRestrictions: needs-reboot', {})
         (code, out, err) = self.runtest(['-d', '-B', p])
+        self.assertEqual(code, 8, err)
+        self.assertRegex(out, 'command1\s+SKIP Test needs to reboot testbed but testbed does not provide reboot capability')
+
+    def test_reboot_mixed(self):
+        '''needs-reboot test gets skipped but another test is still run'''
+
+        p = self.build_src('Test-Command: true 1\nDepends:\nRestrictions: needs-reboot\n'
+                           '\n'
+                           'Test-Command: true 2\nDepends:\n',
+                           {})
+        (code, out, err) = self.runtest(['-d', '-B', p])
         self.assertEqual(code, 2, err)
         self.assertRegex(out, 'command1\s+SKIP Test needs to reboot testbed but testbed does not provide reboot capability')
+        self.assertRegex(out, 'command2\s+PASS')
+
+    def test_broken_test_deps(self):
+        '''unsatisfiable test dependencies'''
+
+        p = self.build_src('Tests: p\nDepends: unknown, libc6 (>= 99:99)\n',
+                           {'p': '#!/bin/sh -e\ntrue'})
+
+        (code, out, err) = self.runtest(['--no-built-binaries', p])
+        self.assertEqual(code, 12, err)
+        self.assertIn('Test dependencies are unsatisfiable', err)
+        self.assertIn('Test dependencies are unsatisfiable', out)
+
+    def test_continue_with_other_tests(self):
+        '''Install failure should continue with next test'''
+
+        p = self.build_src('Test-Command: false\nDepends:\n\nTest-Command: true\nDepends:unknown\n\nTest-Command: true\nDepends:\n', {})
+
+        summary = os.path.join(self.workdir, 'summary.log')
+
+        (code, out, err) = self.runtest(['--debug', '--no-built-binaries', p, '--summary=' + summary])
+        self.assertEqual(code, 12, err)
+        self.assertRegex(out, 'command1\s+FAIL')
+        self.assertRegex(out, 'command2\s+FAIL')
+        self.assertRegex(out, 'command3\s+PASS')
+
+    def test_skip_not_installable(self):
+        '''Test skip-not-installable restriction'''
+        p = self.build_src('Tests: notinstallable\n'
+                           'Restrictions: skip-not-installable\n'
+                           'Depends: nonexisting\n',
+                           {'notinstallable': '#!/bin/sh\necho But I am fine\n'})
+        (code, out, err) = self.runtest(['-d', '--no-built-binaries', p])
+        self.assertEqual(code, 8, err)
+        self.assertRegex(out, 'notinstallable\s+SKIP installation fails', out)
+
+    def test_skip_not_installable_mixed(self):
+        '''Test skip-not-installable restriction combined with a passing test'''
+        p = self.build_src('Tests: notinstallable\n'
+                           'Restrictions: skip-not-installable\n'
+                           'Depends: nonexisting\n'
+                           '\n'
+                           'Tests: ok\n'
+                           'Depends:\n',
+                           {'notinstallable': '#!/bin/sh\necho But I am fine\n',
+                            'ok': '#!/bin/sh\nexit 0\n'})
+        (code, out, err) = self.runtest(['-d', '--no-built-binaries', p])
+        self.assertEqual(code, 2, err)
+        self.assertRegex(out, 'notinstallable\s+SKIP installation fails', out)
+        self.assertRegex(out, 'ok\s+PASS', out)
 
 
 @unittest.skipIf(os.getuid() > 0,
@@ -1629,7 +1799,8 @@ Restrictions: needs-root
 
         # add extra build dependencies to testpkg
         subprocess.check_call(['sed', '-i', '/^Build-Depends:/ s/:.*/: bdep1 , bdep2,\\n bdep3, # moo\\n#comment\\n bdep4\\n'
-                                            'Build-Depends-Indep: bdep5/',
+                                            'Build-Depends-Indep: bdep5\\n'
+                                            'Build-Depends-Arch: bdep6/',
                                os.path.join(p, 'debian', 'control')])
 
         # run this under C locale to test that UTF-8 debian/control is still
@@ -1639,13 +1810,14 @@ Restrictions: needs-root
         self.assertEqual(code, 0, err)
         self.assertRegex(out, 'pass\s+PASS')
 
-        self.assertIn('synthesised dependency testpkg (>= 0~)\n', err)
+        self.assertIn('synthesised dependency testpkg\n', err)
         self.assertIn('processing dependency testdep1\n', err)
         self.assertIn('synthesised dependency bdep1\n', err)
         self.assertIn('synthesised dependency bdep2\n', err)
         self.assertIn('synthesised dependency bdep3\n', err)
         self.assertIn('synthesised dependency bdep4\n', err)
         self.assertIn('synthesised dependency bdep5\n', err)
+        self.assertIn('synthesised dependency bdep6\n', err)
         self.assertIn('synthesised dependency build-essential\n', err)
         self.assertIn('processing dependency testdep2\n', err)
 
@@ -3094,9 +3266,9 @@ class SchrootRunner(AdtTestCase, DebTestsVirtFS, DebTestsFailureModes):
         (code, out, err) = self.runtest(['--no-built-binaries', p])
         self.assertEqual(code, 12, err)
         self.assertRegex(out, '[dD]epends.*unknown', err)
-        self.assertRegex(err, 'Test dependencies are unsatisfiable', err)
+        self.assertIn('Test dependencies are unsatisfiable', err)
         self.assertRegex(out, 'blame: .*/testpkg', out)
-        self.assertRegex(out, 'Test dependencies are unsatisfiable', out)
+        self.assertIn('Test dependencies are unsatisfiable', out)
 
     def test_install_failure(self):
         '''test dependency install failure'''
@@ -3110,7 +3282,7 @@ class SchrootRunner(AdtTestCase, DebTestsVirtFS, DebTestsFailureModes):
         self.assertEqual(code, 12, err)
         self.assertIn('BROKEN POSTINST', out)
         self.assertRegex(out, 'blame: .*/testpkg', out)
-        self.assertRegex(out, 'badpkg: Test dependencies are unsatisfiable', out)
+        self.assertIn('badpkg: Test dependencies are unsatisfiable', out)
 
     def do_apt_pocket_test(self, pocket_args, ver_checks,
                            add_deb_src=False, expect_success=True,
@@ -3197,7 +3369,7 @@ class SchrootRunner(AdtTestCase, DebTestsVirtFS, DebTestsFailureModes):
         # FIXME: our current apt pinning does not resolve this properly, so
         # this will re-try with the entirety of -proposed
         self.assertIn('WARNING: Test dependencies are unsatisfiable with using apt pinning.', err)
-        self.assertIn('Retrying with using all packages from proposed\n', err)
+        self.assertIn('Retrying with using all packages from testy-proposed\n', err)
 
     def test_apt_pocket_multi_nonexisting_dep(self):
         '''--apt-pocket with multiple tests and a nonexisting dependency'''
@@ -3219,7 +3391,16 @@ Depends: nonexisting'''
         self.assertRegex(out, 'command1\s+PASS', out)
         self.assertNotIn(out, 'command2')
         self.assertIn('WARNING: Test dependencies are unsatisfiable with using apt pinning.', err)
-        self.assertIn('ERROR: erroneous package: Test dependencies are unsatisfiable', err)
+        self.assertIn('badpkg: Test dependencies are unsatisfiable', err)
+
+    def test_apt_pocket_no_fallback_fail(self):
+        '''--apt-pocket selecting package from -proposed with dep from -proposed with --no-apt-fallback'''
+
+        (code, out, err) = self.do_apt_pocket_test(
+            ['--apt-pocket=proposed=vanilla', '--no-apt-fallback'],
+            '[ "$V" = 2 ]; [ "$C" = 2 ]; [ "$M" = 2 ]; [ "$H" = 2 ]; [ "$L" = 2 ]',
+            expect_success=False)
+        self.assertIn('Test dependencies are unsatisfiable.', err)
 
     def test_apt_pocket_nbs(self):
         '''--apt-pocket with only NBS binaries in unstable'''
@@ -3273,7 +3454,7 @@ Depends: nonexisting'''
                                          '--apt-upgrade',
                                          '--debug', 'flavor'])
         # we cannot actually download the package, but it should get as far as trying
-        self.assertTrue('apt-get source -q --only-source flavor=3' in err, err)
+        self.assertTrue('apt-get source -d -q --only-source flavor=3' in err, err)
         self.assertEqual(code, 12, err)
 
     def test_git_source_build(self):
@@ -3281,7 +3462,7 @@ Depends: nonexisting'''
 
         (code, out, err) = self.runtest(
             ['--env=DEB_BUILD_OPTIONS=nocheck',
-             'git://anonscm.debian.org/autopkgtest/autopkgtest.git'])
+             'https://salsa.debian.org/ci-team/autopkgtest.git'])
         # test should succeed, but skip the Lxd test (if present)
         self.assertIn(code, [0, 2], err)
         self.assertRegex(err, 'autopkgtest\s+PASS')
@@ -3359,6 +3540,174 @@ Depends: nonexisting'''
         # should build package
         self.assertIn('dh build', err)
 
+    def _build_testprovider(self, provision, arch='all'):
+        '''mangle a 'testprovider' package from the testpkg sources'''
+
+        pr = self.build_src('Tests: pass',
+                            {},
+                            'testprovider')
+
+        subprocess.check_call(['sed', '-i', 's/testpkg/testprovider/g'] +
+                              [os.path.join(pr, 'debian', f) for f in ('control', 'changelog', 'links')])
+
+        # fix conflicting files
+        subprocess.check_call(['sed', '-i', 's,/usr/bin,/usr/share/testprovider,'] +
+                              [os.path.join(pr, 'Makefile')])
+
+        # add a (possibly versioned) Provides
+        subprocess.check_call(['sed', '-i', '/^Depends:/ a\Provides: %s' % provision,
+                               os.path.join(pr, 'debian', 'control')])
+
+        if arch != 'all':
+            subprocess.check_call(['sed', '-i', 's/Architecture: all/Architecture: %s/' % arch,
+                                   os.path.join(pr, 'debian', 'control')])
+        subprocess.check_call(['dpkg-buildpackage', '-b', '-us', '-uc', '-tc'],
+                              stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+                              cwd=pr)
+
+        shutil.rmtree(pr)
+
+        if arch == 'all':
+            deb = os.path.join(os.path.dirname(pr), 'testprovider_1_all.deb')
+        else:
+            deb = os.path.join(os.path.dirname(pr), 'testprovider_1_' + host_arch + '.deb')
+        return deb
+
+    def _test_provides(self, provision, test_dependency, arch='all', control_extra='', expect_failure=False):
+        '''internal function to test different combinations of provides support'''
+
+        testprovider_deb = self._build_testprovider(provision, arch)
+
+        # we check that the testprovider package isn't enough to satisfy
+        # testpkg's test-dependency on itself via 'test_dependency'
+        # even though testprovider Provides: a versioned testpkg
+        p = self.build_src('Tests: pass\nDepends: testprovider, %s' % test_dependency,
+                           {'pass': '#!/bin/sh -e\ndpkg-query -W -f\${Status} testpkg | grep -q ^install'})
+
+        if arch != 'all':
+            subprocess.check_call(['sed', '-i', 's/Architecture: all/Architecture: %s/' % arch,
+                                   os.path.join(p, 'debian', 'control')])
+
+        extradebs = []
+        if control_extra:
+            with open(os.path.join(p, 'debian', 'control'), 'a') as f:
+                f.write('\n' + control_extra)
+            extradebs = [os.path.join(os.path.dirname(p), pkg + '_1_all.deb')
+                         for pkg in re.findall('^Package: (.+)', control_extra, re.MULTILINE)]
+
+        subprocess.check_call(['dpkg-buildpackage', '-b', '-us', '-uc', '-tc'],
+                              stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+                              cwd=p)
+
+        if arch == 'all':
+            testpkg_deb = os.path.join(os.path.dirname(p), 'testpkg_1_all.deb')
+        else:
+            testpkg_deb = os.path.join(os.path.dirname(p), 'testpkg_1_' + host_arch + '.deb')
+
+        failed = False
+        (code, out, err) = self.runtest([p, testpkg_deb, testprovider_deb] + extradebs)
+        try:
+            self.assertEqual(code, 0, out + err)
+            self.assertRegex(out, 'pass\s+PASS', out)
+        except BaseException:
+            if not expect_failure:
+                raise
+            failed = True
+
+        shutil.rmtree(p)
+
+        # now check that testpkg gets installed even if testprovider already is
+        p = self.build_src('Tests: pass\nDepends: %s' % test_dependency,
+                           {'pass': '#!/bin/sh -e\ndpkg-query -W -f\${Status} testpkg | grep -q ^install'})
+        if control_extra:
+            with open(os.path.join(p, 'debian', 'control'), 'a') as f:
+                f.write('\n' + control_extra)
+
+        tb_deb = '/tmp/provider.deb'
+
+        (code, out, err) = self.runtest(['--copy', '%s:%s' % (testprovider_deb, tb_deb),
+                                         '--setup-commands', 'dpkg -i %s' % tb_deb, p])
+        try:
+            self.assertEqual(code, 0, out + err)
+            self.assertRegex(out, 'pass\s+PASS', out)
+        except BaseException:
+            if not expect_failure:
+                raise
+            failed = True
+
+        self.assertTrue(failed or not expect_failure)
+
+        shutil.rmtree(p)
+
+    def test_unversioned_provides_with_at(self):
+        '''test that unversioned provides do not satisfy test dependencies on "@"'''
+        self._test_provides('testpkg', '@')
+
+    def test_unversioned_provides_with_unversioned_explicit(self):
+        '''test that unversioned provides do not satisfy unversioned test dependencies on binary package from the same source'''
+        self._test_provides('testpkg', 'testpkg')
+
+    # this is somewhat silly but included for completeness
+    def test_unversioned_provides_with_versioned_explicit(self):
+        '''test that unversioned provides do not satisfy versioned test dependencies on binary package from the same source'''
+        self._test_provides('testpkg', 'testpkg (>= 0.5)')
+
+    def test_versioned_provides_with_at(self):
+        '''test that versioned provides do not satisfy test dependencies on "@"'''
+        self._test_provides('testpkg (= 0.5)', '@')
+
+    def test_versioned_provides_with_unversioned_explicit(self):
+        '''test that versioned provides do not satisfy unversioned test dependencies on binary package from the same source'''
+        self._test_provides('testpkg (= 0.5)', 'testpkg')
+
+    def test_versioned_provides_with_versioned_explicit(self):
+        '''test that versioned provides do not satisfy versioned test dependencies on binary package from the same source'''
+        self._test_provides('testpkg (= 0.5)', 'testpkg (>= 0.5)')
+
+    def test_versioned_provides_with_at_arch_any(self):
+        '''test that versioned provides do not satisfy test dependencies on "@" with arch:any binary packages'''
+        self._test_provides('testpkg (= 0.5)', '@', arch='any')
+
+    def test_versioned_provides_with_at_arch_multiple(self):
+        '''test that versioned provides do not satisfy test dependencies on "@" with <os>-any + hurd-any binary packages'''
+        self._test_provides('testpkg (= 0.5)', '@', arch=host_os + '-any hurd-any')
+
+    def test_versioned_provides_with_unversioned_explicit_on_not_hurd(self):
+        '''test that versioned provides do not satisfy test dependencies on !hurd-any'''
+        self._test_provides('testpkg (= 0.5)', 'testpkg [!hurd-any]')
+
+    def test_versioned_provides_with_unversioned_explicit_on_host_arch(self):
+        '''test that versioned provides do not satisfy test dependencies on <host_arch>'''
+        self._test_provides('testpkg (= 0.5)', 'testpkg [' + host_arch + ']')
+
+    def test_unversioned_provides_with_conflicting_alternatives(self):
+        '''test that unversioned provides do not satisfy test dependencies on conflicting alternatives from the same source'''
+        testpkg2 = '\nPackage: testpkg2\nArchitecture: all\nDescription: test package 2\nConflicts: testpkg'
+        self._test_provides('testpkg', 'testpkg | testpkg2', control_extra=testpkg2)
+
+    def test_versioned_provides_with_conflicting_alternatives(self):
+        '''test that versioned provides do not satisfy test dependencies on conflicting alternatives from the same source'''
+        testpkg2 = '\nPackage: testpkg2\nArchitecture: all\nDescription: test package 2\nConflicts: testpkg'
+        self._test_provides('testpkg (= 0.5)', 'testpkg | testpkg2', control_extra=testpkg2)
+
+    def test_unversioned_provides_with_nonconflicting_alternatives(self):
+        '''test that unversioned provides do not satisfy test dependencies on not conflicting alternatives from the same source'''
+        testpkg2 = '\nPackage: testpkg2\nArchitecture: all\nDescription: test package 2'
+        self._test_provides('testpkg', 'testpkg | testpkg2', control_extra=testpkg2)
+
+    def test_versioned_provides_with_nonconflicting_alternatives(self):
+        '''test that versioned provides do not satisfy test dependencies on not conflicting alternatives from the same source'''
+        testpkg2 = '\nPackage: testpkg2\nArchitecture: all\nDescription: test package 2'
+        self._test_provides('testpkg (= 0.5)', 'testpkg | testpkg2', control_extra=testpkg2)
+
+    def test_unversioned_provides_with_unrelated_alternatives(self):
+        '''test that unversioned provides satisfy test dependencies on alternatives not all from the same source'''
+        self._test_provides('testpkg', 'testpkg | dpkg', expect_failure=True)
+
+    def test_versioned_provides_with_unrelated_alternatives(self):
+        '''test that versioned provides satisfy test dependencies on alternatives not all from the same source'''
+        self._test_provides('testpkg (= 0.5)', 'testpkg | dpkg', expect_failure=True)
+
 
 @unittest.skipUnless('AUTOPKGTEST_TEST_SCHROOT_CLICK' in os.environ,
                      'Set $AUTOPKGTEST_TEST_SCHROOT_CLICK to an existing schroot')
diff --git a/tests/testdesc b/tests/testdesc
index b749018..4a51f44 100755
--- a/tests/testdesc
+++ b/tests/testdesc
@@ -108,7 +108,7 @@ class Test(unittest.TestCase):
         '''valid Test instantiation with path'''
 
         t = testdesc.Test('foo', 'tests/do_foo', None, ['needs-root'],
-                          ['unknown_feature'], ['coreutils >= 7'], [], [])
+                          ['unknown_feature'], ['coreutils >= 7'], [], [], [])
         self.assertEqual(t.name, 'foo')
         self.assertEqual(t.path, 'tests/do_foo')
         self.assertEqual(t.command, None)
@@ -120,7 +120,7 @@ class Test(unittest.TestCase):
         '''valid Test instantiation with command'''
 
         t = testdesc.Test('foo', None, 'echo hi', ['needs-root'],
-                          ['unknown_feature'], ['coreutils >= 7'], [], [])
+                          ['unknown_feature'], ['coreutils >= 7'], [], [], [])
         self.assertEqual(t.name, 'foo')
         self.assertEqual(t.path, None)
         self.assertEqual(t.command, 'echo hi')
@@ -130,7 +130,7 @@ class Test(unittest.TestCase):
         '''Test with invalid name'''
 
         with self.assertRaises(testdesc.Unsupported) as cm:
-            testdesc.Test('foo/bar', 'do_foo', None, [], [], [], [], [])
+            testdesc.Test('foo/bar', 'do_foo', None, [], [], [], [], [], [])
         self.assertIn('may not contain /', str(cm.exception))
 
     def test_unknown_restriction(self):
@@ -138,28 +138,28 @@ class Test(unittest.TestCase):
 
         with self.assertRaises(testdesc.Unsupported) as cm:
             testdesc.Test('foo', 'tests/do_foo', None, ['needs-red'], [], [],
-                          [], [])
+                          [], [], [])
         self.assertIn('unknown restriction needs-red', str(cm.exception))
 
     def test_neither_path_nor_command(self):
         '''Test without path nor command'''
 
         with self.assertRaises(testdesc.InvalidControl) as cm:
-            testdesc.Test('foo', None, None, [], [], [], [], [])
+            testdesc.Test('foo', None, None, [], [], [], [], [], [])
         self.assertIn('either path or command', str(cm.exception))
 
     def test_both_path_and_command(self):
         '''Test with path and command'''
 
         with self.assertRaises(testdesc.InvalidControl) as cm:
-            testdesc.Test('foo', 'do_foo', 'echo hi', [], [], [], [], [])
+            testdesc.Test('foo', 'do_foo', 'echo hi', [], [], [], [], [], [])
         self.assertIn('either path or command', str(cm.exception))
 
     def test_capabilities_compat(self):
         '''Test compatibility with testbed capabilities'''
 
         t = testdesc.Test('foo', 'tests/do_foo', None,
-                          ['needs-root', 'isolation-container'], [], [], [], [])
+                          ['needs-root', 'isolation-container'], [], [], [], [], [])
 
         self.assertRaises(testdesc.Unsupported,
                           t.check_testbed_compat, ['isolation-container'])
@@ -221,7 +221,7 @@ class Debian(unittest.TestCase):
         for t in ts:
             self.assertEqual(t.restrictions, [])
             self.assertEqual(t.features, [])
-            self.assertEqual(t.depends, ['one (>= 0~)', 'two (>= 0~)'])
+            self.assertEqual(t.depends, ['one', 'two'])
         self.assertFalse(skipped)
 
     def test_arch_specific(self):
@@ -238,8 +238,8 @@ class Debian(unittest.TestCase):
         self.assertEqual(ts[0].restrictions, [])
         self.assertEqual(ts[0].features, [])
         self.assertEqual(ts[0].depends,
-                         ['one (>= 0~)', 'two (>= 0~) [linux-any]',
-                          'three (>= 0~) [c64 vax]'])
+                         ['one', 'two [linux-any]',
+                          'three [c64 vax]'])
         self.assertFalse(skipped)
 
     def test_test_name_feature(self):
@@ -375,10 +375,12 @@ class Debian(unittest.TestCase):
             'Tests: t\nDepends: @, @builddeps@, foo (>= 7)',
             'Source: nums\nBuild-Depends: bd1, bd2 [armhf], bd3:native (>= 7) | bd4 [linux-any]\n'
             'Build-Depends-Indep: bdi1, bdi2 [amd64]\n'
+            'Build-Depends-Arch: bda1, bda2 [amd64]\n'
             '\n'
             'Package: one\nArchitecture: any')
-        self.assertEqual(ts[0].depends, ['one (>= 0~)', 'bd1', 'bd3:native (>= 7) | bd4',
-                                         'bdi1', 'bdi2', 'build-essential', 'foo (>= 7)'])
+        self.assertEqual(ts[0].depends, ['one', 'bd1', 'bd3:native (>= 7) | bd4',
+                                         'bdi1', 'bdi2', 'bda1', 'bda2',
+                                         'build-essential', 'foo (>= 7)'])
         self.assertFalse(skipped)
 
     @unittest.skipUnless(have_dpkg_build_profiles,
@@ -391,7 +393,7 @@ class Debian(unittest.TestCase):
             'Source: nums\nBuild-Depends: bd1, bd2 <!check>, bd3 <!cross>, bdnotme <stage1> <cross>\n'
             '\n'
             'Package: one\nArchitecture: any')
-        self.assertEqual(ts[0].depends, ['one (>= 0~)', 'bd1', 'bd2', 'bd3', 'build-essential'])
+        self.assertEqual(ts[0].depends, ['one', 'bd1', 'bd2', 'bd3', 'build-essential'])
         self.assertFalse(skipped)
 
     def test_complex_deps(self):
@@ -401,7 +403,7 @@ class Debian(unittest.TestCase):
             'Tests: t\nDepends: @,\n foo (>= 7) [linux-any],\n'
             ' bd3:native (>= 4) | bd4 [armhf megacpu],\n',
             'Source: nums\n\nPackage: one\nArchitecture: any')
-        self.assertEqual(ts[0].depends, ['one (>= 0~)', 'foo (>= 7) [linux-any]',
+        self.assertEqual(ts[0].depends, ['one', 'foo (>= 7) [linux-any]',
                                          'bd3:native (>= 4) | bd4 [armhf megacpu]'])
         self.assertFalse(skipped)
 
@@ -433,7 +435,7 @@ class Debian(unittest.TestCase):
             ' , bd2\n'
             '\n'
             'Package: one\nArchitecture: any')
-        self.assertEqual(ts[0].depends, ['one (>= 0~)', 'bd1', 'bd2', 'build-essential'])
+        self.assertEqual(ts[0].depends, ['one', 'bd1', 'bd2', 'build-essential'])
         self.assertFalse(skipped)
 
     @patch('adtlog.report')
@@ -565,7 +567,7 @@ class Debian(unittest.TestCase):
             self.assertGreaterEqual(len(ts), 1)
             self.assertIn('pkg-perl-autopkgtest', ts[0].command)
             self.assertIn('pkg-perl-autopkgtest', ts[0].depends)
-            self.assertIn('libfoo-perl (>= 0~)', ts[0].depends)
+            self.assertIn('libfoo-perl', ts[0].depends)
         else:
             self.assertEqual(len(ts), 0)
 
diff --git a/virt/autopkgtest-virt-lxc b/virt/autopkgtest-virt-lxc
index 7a9a682..a2ecd76 100755
--- a/virt/autopkgtest-virt-lxc
+++ b/virt/autopkgtest-virt-lxc
@@ -139,10 +139,18 @@ def determine_normal_user(lxc_name):
 
     global capabilities, normal_user
 
+<<<<<<< virt/autopkgtest-virt-lxc
     # get the first UID >= 1000
     cmd = ['lxc-attach', '--name', lxc_name, '--', 'sh', '-c',
            'getent passwd | sort -t: -nk3 | '
            "awk -F: '{if ($3 >= 1000) { print $1; exit } }'"]
+=======
+    # get the first UID in the Debian Policy §9.2.2 "dynamically allocated
+    # user account" range
+    cmd = ['lxc-attach', '--name', lxc_name, '--', 'sh', '-c',
+           'getent passwd | sort -t: -nk3 | '
+           "awk -F: '{if ($3 >= 1000 && $3 <= 59999) { print $1; exit } }'"]
+>>>>>>> virt/autopkgtest-virt-lxc
     out = VirtSubproc.execute_timeout(None, 10, sudoify(cmd),
                                       stdout=subprocess.PIPE)[1].strip()
     if out:
@@ -150,7 +158,11 @@ def determine_normal_user(lxc_name):
         capabilities.append('suggested-normal-user=' + normal_user)
         adtlog.debug('determine_normal_user: got user "%s"' % normal_user)
     else:
+<<<<<<< virt/autopkgtest-virt-lxc
         adtlog.debug('determine_normal_user: no uid >= 1000 available')
+=======
+        adtlog.debug('determine_normal_user: no uid in [1000,59999] available')
+>>>>>>> virt/autopkgtest-virt-lxc
 
 
 def start_lxc1():
diff --git a/virt/autopkgtest-virt-lxd b/virt/autopkgtest-virt-lxd
index 46394da..f51ca34 100755
--- a/virt/autopkgtest-virt-lxd
+++ b/virt/autopkgtest-virt-lxd
@@ -115,10 +115,18 @@ def determine_normal_user():
 
     global capabilities, normal_user
 
+<<<<<<< virt/autopkgtest-virt-lxd
     # get the first UID >= 1000
     cmd = ['lxc', 'exec', container_name, '--', 'sh', '-c',
            'getent passwd | sort -t: -nk3 | '
            "awk -F: '{if ($3 >= 1000) { print $1; exit } }'"]
+=======
+    # get the first UID in the Debian Policy §9.2.2 "dynamically allocated
+    # user account" range
+    cmd = ['lxc', 'exec', container_name, '--', 'sh', '-c',
+           'getent passwd | sort -t: -nk3 | '
+           "awk -F: '{if ($3 >= 1000 && $3 <= 59999) { print $1; exit } }'"]
+>>>>>>> virt/autopkgtest-virt-lxd
     out = VirtSubproc.execute_timeout(None, 10, cmd,
                                       stdout=subprocess.PIPE)[1].strip()
     if out:
@@ -126,7 +134,11 @@ def determine_normal_user():
         capabilities.append('suggested-normal-user=' + normal_user)
         adtlog.debug('determine_normal_user: got user "%s"' % normal_user)
     else:
+<<<<<<< virt/autopkgtest-virt-lxd
         adtlog.debug('determine_normal_user: no uid >= 1000 available')
+=======
+        adtlog.debug('determine_normal_user: no uid in [1000,59999] available')
+>>>>>>> virt/autopkgtest-virt-lxd
 
 
 def hook_open():
diff --git a/virt/autopkgtest-virt-qemu b/virt/autopkgtest-virt-qemu
index afb82e7..416c4c8 100755
--- a/virt/autopkgtest-virt-qemu
+++ b/virt/autopkgtest-virt-qemu
@@ -90,6 +90,8 @@ def parse_args():
                         help='Pass through arguments to QEMU command.')
     parser.add_argument('--baseimage', action='store_true', default=False,
                         help='Provide a read-only copy of the base image at /dev/baseimage')
+    parser.add_argument('--efi', action='store_true', default=False,
+                        help='Use OVMF or AAVMF to boot virtual machine using EFI (default: BIOS)')
     parser.add_argument('image', nargs='+',
                         help='disk image to add to the VM (in order)')
 
@@ -494,10 +496,11 @@ def determine_normal_user(shared_dir):
         normal_user = args.user
         return
 
-    # get the first UID >= 500
+    # get the first UID in the Debian Policy §9.2.2 "dynamically allocated
+    # user account" range
     term = VirtSubproc.get_unix_socket(os.path.join(workdir, 'ttyS1'))
     term.send(b"getent passwd | sort -t: -nk3 | "
-              b"awk -F: '{if ($3 >= 500) { print $1; exit } }'"
+              b"awk -F: '{if ($3 >= 1000 && $3 <= 59999) { print $1; exit } }'"
               b"> /run/autopkgtest/shared/normal_user\n")
     with VirtSubproc.timeout(5, 'timed out on determining normal user'):
         outfile = os.path.join(shared_dir, 'normal_user')
@@ -509,7 +512,7 @@ def determine_normal_user(shared_dir):
             normal_user = out.strip()
             adtlog.debug('determine_normal_user: got user "%s"' % normal_user)
         else:
-            adtlog.debug('determine_normal_user: no uid >= 500 available')
+            adtlog.debug('determine_normal_user: no uid in [1000,59999] available')
 
 
 def hook_open():
@@ -544,6 +547,30 @@ def hook_open():
             '-virtfs',
             'local,id=autopkgtest,path=%s,security_model=none,mount_tag=autopkgtest' % shareddir,
             '-drive', 'file=%s,cache=unsafe,if=virtio,index=0' % overlay]
+
+    if args.efi:
+        code = None
+        data = None
+
+        if 'qemu-system-x86_64' in args.qemu_command or \
+                'qemu-system-i386' in args.qemu_command:
+            code = '/usr/share/OVMF/OVMF_CODE.fd'
+            data = '/usr/share/OVMF/OVMF_VARS.fd'
+        elif 'qemu-system-aarch64' in args.qemu_command:
+            code = '/usr/share/AAVMF/AAVMF_CODE.fd'
+            data = '/usr/share/AAVMF/AAVMF_VARS.fd'
+        elif 'qemu-system-arm' in args.qemu_command:
+            code = '/usr/share/AAVMF/AAVMF32_CODE.fd'
+            data = '/usr/share/AAVMF/AAVMF32_VARS.fd'
+        else:
+            VirtSubproc.bomb('Unknown architecture for EFI boot')
+
+        shutil.copy(data, '%s/efivars.fd' % workdir)
+        argv.append('-drive')
+        argv.append('if=pflash,format=raw,read-only,file=' + code)
+        argv.append('-drive')
+        argv.append('if=pflash,format=raw,file=%s/efivars.fd' % workdir)
+
     for i, image in enumerate(args.image[1:]):
         argv.append('-drive')
         argv.append('file=%s,if=virtio,index=%i,readonly' % (image, i + 1))
diff --git a/virt/autopkgtest-virt-qemu.1 b/virt/autopkgtest-virt-qemu.1
index cdfe190..2137725 100644
--- a/virt/autopkgtest-virt-qemu.1
+++ b/virt/autopkgtest-virt-qemu.1
@@ -110,6 +110,14 @@ will not be accessible between calling
 and the next boot, thus make sure to stop accessing it before.
 
 .TP
+.B  \-\-efi
+Configure QEMU with OVMF or AAVMF firmware images suitable for booting
+an x86, ARM or ARM64 virtual machine using EFI. This requires the
+ovmf, qemu-efi-arm or qemu-efi-aarch64 package. The correct firmware
+image is guessed from the
+.BR --qemu-command .
+
+.TP
 .BI "--qemu-options=" arguments
 Pass through arguments to QEMU command; e. g. --qemu-options='-readconfig qemu.cfg'