Merge lp:~robru/cupstream2distro/fix-statuses into lp:cupstream2distro

[Robert Bruce Park]

Improve reliability of status reporting for individual packages.

This branch is a little experimental, I'm trying to make packages a little more self-aware about what state they're in, rather than relying on local train state which often becomes inconsistent (for example the current issue with not being able to un-dirty silos after deleting commits that made them dirty in the first place).

[lp:~robru/cupstream2distro/fix-statuses]

1257. By Robert Bruce Park on 2015-11-18

Better test isolation.

1258. By Robert Bruce Park on 2015-11-18

More publication blocker words.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:1257
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/901/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/901/rebuild

review: Approve (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:1258
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/902/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/902/rebuild

review: Approve (continuous-integration)

[lp:~robru/cupstream2distro/fix-statuses]

1259. By Robert Bruce Park on 2015-11-18

EVEN FASTER Package.get_package_version().

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:1259
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/903/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/903/rebuild

review: Approve (continuous-integration)

[lp:~robru/cupstream2distro/fix-statuses]

1260. By Robert Bruce Park on 2015-11-18

Speed up BuildBase._check_build_status()

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:1260
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/904/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/904/rebuild

review: Approve (continuous-integration)

[lp:~robru/cupstream2distro/fix-statuses]

1261. By Robert Bruce Park on 2015-11-18

Drop migration frequency to 30 minutes due to slowness.

1262. By Robert Bruce Park on 2015-11-18

Debug logging.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:1262
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/905/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/905/rebuild

review: Approve (continuous-integration)

[lp:~robru/cupstream2distro/fix-statuses]

1263. By Robert Bruce Park on 2015-11-18

Experimental speedup for Archive.find_archive_arches().

1264. By Robert Bruce Park on 2015-11-18

More experiments in speed.

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:1264
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/906/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/906/rebuild

review: Needs Fixing (continuous-integration)

[lp:~robru/cupstream2distro/fix-statuses]

1265. By Robert Bruce Park on 2015-11-18

Logging tweak.

1266. By Robert Bruce Park on 2015-11-18

Fix tests.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:1266
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/907/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/907/rebuild

review: Approve (continuous-integration)

[lp:~robru/cupstream2distro/fix-statuses]

1267. By Robert Bruce Park on 2015-11-18

Push more slow stuff out of check-publication-migration.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:1267
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/908/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/908/rebuild

review: Approve (continuous-integration)

[Robert Bruce Park (robru)]

Alright, I'm really happy with this & tested it well in staging. Will roll out first thing tomorrow.

review: Approve

=== modified file 'citrain/autopkgtests.py'
--- citrain/autopkgtests.py	2015-11-10 17:50:43 +0000
+++ citrain/autopkgtests.py	2015-11-18 06:50:36 +0000
@@ -36,7 +36,9 @@
 
 def autopkgtest(silo_state):
     """Submit $SILONAME to debci for testing."""
-    Manager(silo_state).do('validate', 'amqp')
+    manager = Manager(silo_state)
+    manager.do('validate', 'amqp')
+    silo_state.status = manager.get_states()
 
 
 run_script(__name__, __doc__, lambda: stock_main(autopkgtest))

=== modified file 'citrain/build.py'
--- citrain/build.py	2015-11-06 21:45:41 +0000
+++ citrain/build.py	2015-11-18 06:50:36 +0000
@@ -46,7 +46,7 @@
 from citrain.recipes.base import missing
 from citrain.recipes.merge import check_for_unbuilt_revids
 from citrain.recipes.manager import Manager
-from cupstream2distro.silomanager import SiloState, get_dirty, splitter
+from cupstream2distro.silomanager import splitter, stock_main
 from cupstream2distro.errors import BuildError, CITrainError
 from cupstream2distro.utils import env, run_script
 
@@ -67,7 +67,6 @@
                 if check_for_unbuilt_revids(source, merges) == []:
                     logging.info(source + ' has no new commits, skipping.')
                     self.names.discard(source)
-            self.names.update(set(get_dirty(env.SILONAME)) & all_projects)
         self.names.update(missing(self.names))
         if not self.names:
             raise BuildError(
@@ -75,27 +74,18 @@
                 'PACKAGES_TO_REBUILD, FORCE_REBUILD, or WATCH_ONLY.')
 
 
-def main(buildmanager=BuildManager):
-    """Execute the build, logging & saving any errors.
-
-    :returns: 0 on success, 1 on any failure.
-    """
-    try:
-        silo_state = SiloState(env.SILONAME, primary=True)
+def buildfactory(buildmanager=BuildManager):
+    """Create a build function."""
+    def build(silo_state):
+        """Execute the build, logging & saving any errors."""
         manager = buildmanager(silo_state)
         manager.do('validate')
         if env.WATCH_ONLY != 'true':
             manager.do('clean', 'collect', 'build', 'upload', 'expose')
-        Manager(silo_state).do('watch', 'diff')
-        silo_state.status = 'Packages built.'
-    except CITrainError as err:
-        silo_state.status = err
-        silo_state.mark_dirty()
-        silo_state.save_config()
-        return 1
-    silo_state.mark_dirty()
-    silo_state.save_config()
-    return 0
-
-
-run_script(__name__, __doc__, main)
+        manager = Manager(silo_state)
+        manager.do('watch', 'diff')
+        silo_state.status = manager.get_states()
+    return build
+
+
+run_script(__name__, __doc__, lambda: stock_main(buildfactory(BuildManager)))

=== modified file 'citrain/jenkins-templates/check-publication-migration.xml.tmpl'
--- citrain/jenkins-templates/check-publication-migration.xml.tmpl	2015-10-27 19:46:01 +0000
+++ citrain/jenkins-templates/check-publication-migration.xml.tmpl	2015-11-18 06:50:36 +0000
@@ -38,7 +38,7 @@
   <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
   <triggers>
     <hudson.triggers.TimerTrigger>
-      <spec>H/15 * * * *</spec>
+      <spec>H/30 * * * *</spec>
     </hudson.triggers.TimerTrigger>
   </triggers>
   <concurrentBuild>false</concurrentBuild>

=== modified file 'citrain/merge_clean.py'
--- citrain/merge_clean.py	2015-11-06 21:45:41 +0000
+++ citrain/merge_clean.py	2015-11-18 06:50:36 +0000
@@ -42,9 +42,9 @@
 
 def merge(silo_state):
     """Do the steps specific to merging & cleaning."""
-    Manager(silo_state).do(
-        'validate', 'enumeration', 'merge', 'push', 'nuke')
+    manager = Manager(silo_state)
+    manager.do('validate', 'enumeration', 'merge', 'push', 'nuke')
     silo_state.status = 'Landed'
 
 
-run_script(__name__, __doc__, lambda: stock_main(merge, mark_dirty=True))
+run_script(__name__, __doc__, lambda: stock_main(merge))

=== modified file 'citrain/migration.py'
--- citrain/migration.py	2015-11-09 17:06:20 +0000
+++ citrain/migration.py	2015-11-18 06:50:36 +0000
@@ -25,17 +25,16 @@
     Enable debug infos
 """
 
+import re
 import logging
 
 from citrain.recipes.base import BuildBase
 from citrain.recipes.manager import Manager
-from cupstream2distro.errors import CITrainError
 from cupstream2distro.utils import env, run_script
 from cupstream2distro.silomanager import SiloState, stock_main
 from citrain.merge_clean import merge
 
-
-CITrainError.prefix = 'Migration: '
+SUCCESS = re.compile(r'^((Release|Updates) pocket \([^()]+\). ?)+$')
 
 
 def main():
@@ -46,27 +45,20 @@
     for silo_state in SiloState.iterate():
         env.SILONAME = silo_state
         BuildBase.failures.clear()
+        logging.info('\n\nInspecting silo %s:', silo_state.ppa.web_link)
         try:
             silo_state.enforce_lock()
             silo_state.lock_fd.close()
         except BlockingIOError:
-            logging.info('Silo %s is in use, skipping.\n', env.SILONAME)
+            logging.info('Silo %s is in use, skipping.', env.SILONAME)
             continue
 
-        if 'dirty' not in silo_state.tokenize():
-            logging.info('Inspecting silo %s:', silo_state.ppa.web_link)
-            manager = Manager(silo_state)
-            try:
-                manager.do('unbuilt')
-                if silo_state.is_autopkgtesting:
-                    manager.do('autopkgtest')
-                if silo_state.is_published:
-                    manager.do('migration')
-                    stock_main(merge, mark_dirty=True)
-            except CITrainError as err:
-                silo_state.status = err
-                silo_state.mark_dirty()
-                silo_state.save_config()
+        status = silo_state.status = Manager(silo_state).get_states()
+        if SUCCESS.match(status):
+            logging.info('Looks good, proceeding with merge & clean.')
+            stock_main(merge)
+            continue
+        silo_state.save_config()
     return 0
 
 

=== modified file 'citrain/prepare_silo.py'
--- citrain/prepare_silo.py	2015-11-06 21:45:41 +0000
+++ citrain/prepare_silo.py	2015-11-18 06:50:36 +0000
@@ -48,7 +48,7 @@
         SiloState().assign(env.REQUEST_ID)
     except CITrainError as err:
         logging.error(str(err))
-        return 1
+        return err.code
     return 0
 
 

=== modified file 'citrain/publisher.py'
--- citrain/publisher.py	2015-11-09 18:09:39 +0000
+++ citrain/publisher.py	2015-11-18 06:50:36 +0000
@@ -51,9 +51,9 @@
             'This silo must be transitioned to xenial before publishing.')
     mgr = Manager(silo_state)
     mgr.do('checkstatus', 'validate', 'diff', 'checkupload', 'ackaging')
-    silo_state.status = 'Publishing.'
+    silo_state.status = 'Beginning publication...'
     silo_state.save_config()
     mgr.do('dest_version_check', 'unapproved', 'unbuilt', 'publish')
 
 
-run_script(__name__, __doc__, lambda: stock_main(publish, mark_dirty=True))
+run_script(__name__, __doc__, lambda: stock_main(publish))

=== modified file 'citrain/recipes/base.py'
--- citrain/recipes/base.py	2015-11-10 18:16:52 +0000
+++ citrain/recipes/base.py	2015-11-18 06:50:36 +0000
@@ -25,6 +25,7 @@
 import time
 
 from urllib.request import urlretrieve
+from collections import defaultdict
 from os.path import exists, join
 from os import remove, mkdir
 from glob import glob
@@ -41,7 +42,6 @@
     AutopkgError,
     BuildError,
     PublishError,
-    MigrationError,
     NoStatusError,
 )
 from cupstream2distro.settings import (
@@ -100,21 +100,22 @@
     return [TWINS[n] for n in names if n in TWINS and not TWINS[n] in names]
 
 
-DO_NOT_PUBLISH = {'failed', 'error', 'dirty'}
-BUILD_FAILURES = (
+DO_NOT_PUBLISH = {'failed', 'error', 'rebuild', 'problem'}
+BUILD_FAILURES = {
     'Build for superseded Source',
     'Cancelled build',
     'Chroot problem',
     'Failed to build',
     'Failed to upload',
-)
-
-
+}
+
+
+ARCH_FMT = '{} [{}]'
 MISSING_VERSION = """\
 {} {} is missing from the changelog, which has up to {}. \
 Please sync destination version back to trunk."""
-VERSION_MISMATCH = """\
-Version of {} at dest changed from {} to {} since packages built."""
+DEST_MISMATCH = """\
+Destination version changed from {} to {} since packages built"""
 
 
 AUTOPKGTEST_ARCHES = {'armhf', 'amd64', 'i386', 'ppc64el'}
@@ -158,6 +159,13 @@
     return response
 
 
+def pretty_print_states(states, fmt='{} ({}).'):
+    """Print a dict of lists in a nice way."""
+    return ' '.join(
+        fmt.format(stat, ', '.join(arch for arch in sorted(arches)))
+        for stat, arches in sorted(states.items()))
+
+
 class BuildBase(Package, Branch, Archive):
     """Outlines build steps common to all types of package builds."""
     blame = lp.people[env.BUILD_USER_ID] if env.BUILD_USER_ID else None
@@ -165,7 +173,6 @@
     packagelist = None
     authorized = None
     failures = set()
-    states = dict()
 
     def __init__(self, source_name, series, dest, silo_ppa):
         """Set instance variables needed for building a debian package.
@@ -183,6 +190,107 @@
         self.target = None
         self.dest = dest
 
+    def get_version(self):
+        """Identify what version this package is."""
+        for get_ver in (self.get_package_version, self.get_archive_version):
+            version = get_ver()
+            if version:
+                return version
+
+    def check_status(self):
+        """Identify this package's progress through the train."""
+        checks = (
+            self.check_silo_version,
+            self.check_new_commits,
+            self.check_destination_version,
+            self.archive_pocket,
+            self.archive_queue,
+            self.check_autopkgtest_status,
+            self.check_build_status,
+        )
+        version = self.get_version()
+        for check in checks:
+            logging.debug('Checking %s for %s', check, self.name)
+            message = check(version)
+            if message:
+                return message
+        return 'Unknown (this is a bug)'
+
+    def check_silo_version(self, version):
+        """Check that local version matches silo version."""
+        version = self.get_package_version()
+        silo_version = self.get_archive_version()
+        if not silo_version:
+            return 'Not in PPA'
+        if version and version != silo_version:
+            return '{} not in PPA'.format(version)
+
+    def check_new_commits(self, version):
+        """Implemented by subclasses."""
+        pass
+
+    def check_destination_version(self, version=None):
+        """Check that destination version unchanged since package built."""
+        dest_version = self.get_archive_version(self.dest, self.series)
+        supposed = self.dest_current_version
+        log_value_of.dest_version('Dest archive has')
+        log_value_of.supposed('At build time, dest had')
+        if supposed and dest_version != supposed:
+            return DEST_MISMATCH.format(supposed, dest_version)
+
+    def check_autopkgtest_status(self, version=None):
+        """Check current status of autopkgtests, if any."""
+        found = set()
+        for arch in AUTOPKGTEST_ARCHES:
+            stamp_key = self.timestamp_key(arch)
+            our_stamp = self.get_file(stamp_key)
+            if our_stamp is None:
+                return
+            local_path = self.result_tar_path(arch)
+            log_value_of.local_path('Checking')
+            response = check_swift(
+                self.silo_ppa, self.series.name, arch, self.name, our_stamp)
+            latest = our_stamp
+            if response.status_code == 200:
+                for line in reversed(sorted(response.iter_lines())):
+                    line = line.decode('utf-8')
+                    stamp = line.split('@')[0].split('/')[-1]
+                    latest = max(stamp, our_stamp, latest)
+                    if stamp < latest:
+                        break
+                    full_url = response.url.split('?')[0] + line
+                    logging.info(full_url)
+                    if line.endswith('result.tar') and stamp > our_stamp:
+                        self.set_file(stamp_key, latest)
+                        urlretrieve(full_url, local_path)
+                        log_value_of.local_path('Downloaded')
+            if exists(local_path):
+                found.add(arch)
+        if AUTOPKGTEST_ARCHES - found:
+            return 'Autopkgtests running'
+        else:
+            return self.inspect_tar_files()
+
+    def _check_build_status(self, version=None):
+        """Report the build pass/fail status in the PPA."""
+        watcher = ArchWatcher(self.arches)
+        source = self.get_source_from_archive(
+            version=version or self.get_version())
+        states = defaultdict(list)
+        if source:
+            for build in source.getBuilds():
+                if watcher(build):
+                    state = build.buildstate
+                    if state in BUILD_FAILURES:
+                        logging.error(build.build_log_url)
+                    states[state].append(build.arch_tag)
+        return states
+
+    def check_build_status(self, version):
+        """Check that build in PPA has succeeded."""
+        return pretty_print_states(
+            self._check_build_status(version), fmt=ARCH_FMT)
+
     def find_file(self, key):
         """Identify where keyfiles are located."""
         return '{}_{}'.format(self.path, key)
@@ -217,6 +325,7 @@
 
     def __getattr__(self, attr):
         """Fetch some special properties."""
+        log_value_of.attr('Getattr')
         if attr in self.magic:
             return self.get_file(attr)
         raise AttributeError
@@ -293,7 +402,7 @@
         """Upload this locally built package to the silo PPA."""
         logging.info('Uploading %s.', self.name)
         ours = self.get_package_version()
-        silo = self.get_archive_version(self.silo_ppa)
+        silo = self.get_archive_version()
         log_value_of.ours()
         log_value_of.silo()
         if V(ours) > V(silo):
@@ -312,7 +421,7 @@
         silo_state.status = 'Building.'
         silo_state.save_config()
 
-    def timed_out(self, started, timeout, message, condition=True):
+    def timed_out(self, started, timeout):
         """Determine whether or not a timeout has been reached.
 
         :param started: The start time as returned by time.time().
@@ -321,8 +430,8 @@
         :param condition: A boolean that can disable the timeout if False.
         :returns: True if the timeout has been reached, otherwise None.
         """
-        if (time.time() - started) > timeout and condition:
-            self.failures.add(message)
+        if (time.time() - started) > timeout:
+            self.failures.add(self.name + ' timed out')
             return True
 
     def watch_phase(self):
@@ -330,41 +439,23 @@
 
         :returns: One of 'build failed', 'depwait', 'success', or 'timeout'
         """
-        logging.info('Watching %s...', self.name)
-        arches = set(self.find_archive_arches(self.main_archive))
-        watcher = ArchWatcher(arches)
-        version = self.get_package_version()
-        kwargs = dict(source_name=self.name, exact_match=True)
-        if version:
-            # Expected to be '' for manual sources and binary syncs.
-            kwargs['version'] = version
-        log_value_of.kwargs('self.silo_ppa.getPublishedSources')
+        version = self.get_version() or ''
+        logging.info('Watching %s %s...', self.name, version)
         started = time.time()
         while True:
-            source = newest(self.silo_ppa, **kwargs)
-            if source is not None:
-                logging.info('Checking %s silo:', source.display_name)
-                builds = [b for b in source.getBuilds() if watcher(b)]
-                success = 0
-                for build in builds:
-                    state = build.buildstate
-                    logging.info('%s: %s', build.arch_tag, state)
-                    if state in BUILD_FAILURES:
-                        self.failures.add(
-                            '{} failed to build.'.format(self.name))
-                        logging.error(build.build_log_url)
-                        return 'build failed'
-                    if state == 'Successfully built':
-                        success += 1
-                    if state == 'Dependency wait':
-                        message = '{} timeout for {}.'.format(state, self.name)
-                        if self.timed_out(started, DEPWAIT_TIMEOUT, message):
-                            return 'depwait'
-                if success == len(builds):
-                    return 'success'
-            message = '{} {} not found in Silo PPA.'.format(self.name, version)
-            if self.timed_out(started, PPA_TIMEOUT, message, not source):
+            states = self._check_build_status()
+            if not states and self.timed_out(started, PPA_TIMEOUT):
                 return 'timeout'
+            state_keys = states.keys()
+            logging.info(pretty_print_states(states, fmt=ARCH_FMT))
+            if 'Dependency wait' in states:
+                if self.timed_out(started, DEPWAIT_TIMEOUT):
+                    return 'depwait'
+            if state_keys & BUILD_FAILURES:
+                self.failures.add('{} failed to build.'.format(self.name))
+                return 'build failed'
+            if state_keys == {'Successfully built'}:
+                return 'success'
             time.sleep(TIME_BETWEEN_PPA_CHECKS)
 
     @staticmethod
@@ -441,37 +532,6 @@
             if process.returncode != 0:
                 raise AutopkgError('amqp-publish failed.')
 
-    def autopkgtest_phase(self):
-        """Poll swift for autopkgtest results."""
-        found = set()
-        for arch in AUTOPKGTEST_ARCHES:
-            stamp_key = self.timestamp_key(arch)
-            our_stamp = self.get_file(stamp_key)
-            local_path = self.result_tar_path(arch)
-            log_value_of.local_path('Checking')
-            response = check_swift(
-                self.silo_ppa, self.series.name, arch, self.name, our_stamp)
-            latest = our_stamp
-            if response.status_code == 200:
-                for line in reversed(sorted(response.iter_lines())):
-                    line = line.decode('utf-8')
-                    stamp = line.split('@')[0].split('/')[-1]
-                    latest = max(stamp, our_stamp, latest)
-                    if stamp < latest:
-                        break
-                    full_url = response.url.split('?')[0] + line
-                    logging.info(full_url)
-                    if line.endswith('result.tar') and stamp > our_stamp:
-                        self.set_file(stamp_key, latest)
-                        urlretrieve(full_url, local_path)
-                        log_value_of.local_path('Downloaded')
-            if exists(local_path):
-                found.add(arch)
-        if AUTOPKGTEST_ARCHES - found:
-            raise AutopkgError('In progress.')
-        else:
-            self.inspect_tar_files()
-
     def inspect_tar_files(self):
         """Check exitcodes in all tar files."""
         codes = set()
@@ -481,8 +541,9 @@
                 with tarfile.open(None, 'r', data) as tar:
                     codes.add(tar.extractfile('exitcode').read().strip())
         if codes - {b'0', b'2'}:
-            raise AutopkgError('Failed (see log).')
-        raise AutopkgError('Success!')
+            return 'Autopkgtests failed'
+        else:
+            return 'Autopkgtests succeeded'
 
     @staticmethod
     def post_checkstatus_phase(silo_state):
@@ -490,9 +551,6 @@
         if 'Publish failed' not in silo_state.status:
             if DO_NOT_PUBLISH & silo_state.tokenize():
                 raise NoStatusError('Silo has bad status.')
-        if glob(SILO_DIR('*_is_dirty')):
-            silo_state.mark_dirty()
-            raise NoStatusError(silo_state.status)
 
     def checkupload_phase(self):
         """Confirm that the silo publisher has upload rights."""
@@ -544,13 +602,8 @@
         if env.IGNORE_VERSIONDESTINATION != 'true':
             if lp.is_distro_archive(self.dest):
                 logging.info('Checking for new %s uploads at dest.', self.name)
-                version = self.get_archive_version(self.dest, self.series)
-                supposed = self.dest_current_version
-                log_value_of.version('Dest archive has')
-                log_value_of.supposed('At build time, dest had')
-                if supposed and version != supposed:
-                    self.failures.add(
-                        VERSION_MISMATCH.format(self.name, supposed, version))
+                if self.check_destination_version():
+                    raise PublishError('Unexpected upload at destination.')
 
     @classmethod
     def pre_publish_phase(cls, silo_state):
@@ -580,29 +633,6 @@
         if cls.packagelist.lines:
             cls.packagelist.write()
 
-    def migration_phase(self):
-        """Identify the location of this package in its migration."""
-        version = self.published_version
-        log_value_of.version('Checking for')
-        self.states[self.name] = self.check_archive_migration(version)
-        logging.info('%s: %s', self.name, self.states[self.name])
-
-    @staticmethod
-    def post_migration_phase(silo_state):
-        """Prevent migration from proceeding to merge if not all migrated.
-
-        :raises: MigrationError if not everything has fully migrated.
-        """
-        success = set(['Release pocket', 'Updates pocket'])
-        migrating = []
-        for source, location in BuildBase.states.items():
-            if location not in success:
-                migrating.append('{} is in the {}.'.format(source, location))
-        BuildBase.states.clear()
-        if migrating:
-            raise MigrationError(' '.join(sorted(migrating)))
-        logging.info('Looks good, proceeding with merge & clean.')
-
     def enumeration_phase(self):
         """Halt merge&clean job if packages not in destination.
 

=== modified file 'citrain/recipes/manager.py'
--- citrain/recipes/manager.py	2015-11-04 15:43:33 +0000
+++ citrain/recipes/manager.py	2015-11-18 06:50:36 +0000
@@ -20,7 +20,9 @@
 
 import logging
 
-from citrain.recipes.base import BuildBase
+from collections import defaultdict
+
+from citrain.recipes.base import BuildBase, pretty_print_states
 from citrain.recipes.merge import Merge
 from citrain.recipes.manual import Manual
 from citrain.recipes.secondary import Secondary
@@ -105,3 +107,13 @@
             raise CITrainError('. '.join(
                 fail.strip('. ') for fail in sorted(BuildBase.failures)))
         getattr(BuildBase, 'post_{}_phase'.format(phase), nop)(self.silo_state)
+
+    def get_states(self):
+        """Report the statuses of all packages in this silo."""
+        self.do()  # Instantiates build objects if necessary.
+        states = defaultdict(list)
+        for name, build in sorted(self.builds.items()):
+            states[build.check_status()].append(
+                '{}/{}'.format(build.name, build.series.name))
+        log_value_of.states()
+        return pretty_print_states(states)

=== modified file 'citrain/recipes/merge.py'
--- citrain/recipes/merge.py	2015-11-04 15:43:33 +0000
+++ citrain/recipes/merge.py	2015-11-18 06:50:36 +0000
@@ -24,7 +24,6 @@
 from cupstream2distro.version import V
 from cupstream2distro.errors import BuildError, PrepError
 from cupstream2distro.settings import BOT_DEBFULLNAME
-from cupstream2distro.silomanager import mark_packages_dirty
 from cupstream2distro.utils import (
     SILO_DIR,
     env,
@@ -95,14 +94,18 @@
     """Return a list of MPs that have new commits since last build."""
     unbuilt = []
     logging.info('Checking %s for new commits...', source)
-    # TODO: check also the merge target revids, which only just started being
-    # recorded and need time to appear in production.
+    target = merges[0].target_branch
+    url = target.web_link
+    revid = read_revid(source, url)
+    if revid and revid != target.last_scanned_id:
+        unbuilt.append(url)
     for merge in merges:
         url = merge.web_link
         revid = read_revid(source, url)
         if merge.source_branch.last_scanned_id != revid:
             unbuilt.append(url)
-            logging.info('New commits: ' + url)
+    for url in unbuilt:
+        logging.info('New commits: ' + url)
     return unbuilt
 
 
@@ -275,10 +278,14 @@
         if env.ALLOW_UNAPPROVED != 'true':
             self.enforce_merge_states(PUBLISHABLE_STATES)
 
+    def check_new_commits(self, version=None):
+        """Report if this package needs a rebuild, or not."""
+        if check_for_unbuilt_revids(self.name, self.merges):
+            return 'Needs rebuild due to new commits'
+
     def unbuilt_phase(self):
         """Block publication of merges that have new commits."""
         if check_for_unbuilt_revids(self.name, self.merges):
-            mark_packages_dirty(env.SILONAME, [self.name])
             self.failures.add(self.name + ' has new, unbuilt commits.')
 
     def merge_phase(self):

=== modified file 'citrain/revert.py'
--- citrain/revert.py	2015-11-06 21:45:41 +0000
+++ citrain/revert.py	2015-11-18 06:50:36 +0000
@@ -44,7 +44,8 @@
 from citrain.recipes.sourcesync import SourceSync
 from cupstream2distro.version import V
 from cupstream2distro.archive import sort_by_date
-from citrain.build import main as build_main
+from citrain.build import buildfactory
+from cupstream2distro.silomanager import stock_main
 from cupstream2distro.errors import CITrainError, RevertError
 from cupstream2distro.utils import (
     SILO_DIR,
@@ -110,9 +111,4 @@
             Revert.versions[source_name] = version
 
 
-def main():
-    """Execute the revert."""
-    return build_main(RevertManager)
-
-
-run_script(__name__, __doc__, main)
+run_script(__name__, __doc__, lambda: stock_main(buildfactory(RevertManager)))

=== modified file 'cupstream2distro/archive.py'
--- cupstream2distro/archive.py	2015-11-04 15:43:33 +0000
+++ cupstream2distro/archive.py	2015-11-18 06:50:36 +0000
@@ -30,6 +30,7 @@
 from cupstream2distro.utils import (
     log_call,
     log_value_of,
+    memoize,
     suppress,
 )
 
@@ -61,6 +62,12 @@
     """A mixin providing launchpad-archive related functions."""
     dest = None
 
+    @property
+    @memoize
+    def arches(self):
+        """Return the set of arches that we care to watch."""
+        return sorted(self.find_archive_arches(self.main_archive))
+
     def find_archive_arches(self, archive):
         """Determine what arches we should watch by checking the destination.
 
@@ -78,15 +85,15 @@
             status='Published',
             distro_series=self.series,
             source_name=self.name)
-        if not source:
-            return
-        for binary in source.getPublishedBinaries():
-            # architecture_specific field filters out Arch:all packages showing
-            # up even on arches for which builds have failed.
-            if binary.architecture_specific:
-                yield binary.distro_arch_series.architecture_tag
+        log_value_of.source('Inspecting this for arches')
+        if source:
+            return {
+                binary.distro_arch_series_link.split('/')[-1]
+                for binary in source.getPublishedBinaries()
+                if binary.architecture_specific}
+        return set()
 
-    def get_archive_version(self, archive, series=None):
+    def get_archive_version(self, archive=None, series=None):
         """Get current version for a package name in that series.
 
         :param archive: An lp api archive object.
@@ -94,20 +101,21 @@
         :returns: The version of this package in the archive, or '0'.
         """
         series = series or self.series
+        archive = archive or self.silo_ppa
         assert hasattr(series, 'previous_series_link')
         source = newest(archive, source_name=self.name, distro_series=series)
-        return source.source_package_version if source else '0'
+        return source.source_package_version if source else None
 
-    def get_source_from_archive(self, archive=None):
+    def get_source_from_archive(self, archive=None, version=None):
         """Return the LP source package object representing this package.
 
         :param archive: What archive to get from, defaults to self.silo_ppa.
+        :param version: What version of the package to request.
         """
-        return newest(
-            archive or self.silo_ppa,
-            status='Published',
-            distro_series=self.series,
-            source_name=self.name)
+        kwargs = dict(distro_series=self.series, source_name=self.name)
+        if version:
+            kwargs['version'] = version
+        return newest(archive or self.silo_ppa, **kwargs)
 
     def copy_to_archive(self, source, archive=None):
         """Copy this source package to a given archive.

=== modified file 'cupstream2distro/errors.py'
--- cupstream2distro/errors.py	2015-11-09 23:47:01 +0000
+++ cupstream2distro/errors.py	2015-11-18 06:50:36 +0000
@@ -22,7 +22,7 @@
 
 class CITrainError(Exception):
     """Base Exception class for all CI Train error conditions."""
-    prefix = 'Unknown error: '
+    prefix = 'Failed: '
     code = next(COUNTER)
     loud = True
     error = True
@@ -36,66 +36,48 @@
     """Special Exception that when raised won't set status."""
     code = next(COUNTER)
     loud = False
-    prefix = ''
 
 
 class ArchiveError(CITrainError):
     """Exception raised when there's a problem with a Launchpad archive."""
-    prefix = 'Launchpad error: '
     code = next(COUNTER)
 
 
 class BranchError(CITrainError):
     """Exception raised when there's a problem with a Bzr branch."""
-    prefix = 'Branch error: '
     code = next(COUNTER)
 
 
 class PackageError(CITrainError):
     """Exception raised when there's a problem with a debian package."""
-    prefix = 'Package error: '
     code = next(COUNTER)
 
 
 class PrepError(CITrainError):
     """Exception raised when there's a problem during preparation."""
-    prefix = 'Prepare failed: '
     code = next(COUNTER)
 
 
 class PublishError(CITrainError):
     """Exception raised when there's a problem during publishing."""
-    prefix = 'Publish failed: '
     code = next(COUNTER)
 
 
 class BuildError(CITrainError):
     """Exception raised when there's a problem with building."""
-    prefix = 'Build failed: '
     code = next(COUNTER)
 
 
 class MergeError(CITrainError):
     """Exception raised when there's a problem with merging."""
-    prefix = 'Merging failed: '
-    code = next(COUNTER)
-
-
-class MigrationError(CITrainError):
-    """Exception raised to report on the silo migration status."""
-    prefix = 'Migration: '
-    code = next(COUNTER)
-    error = False
+    code = next(COUNTER)
 
 
 class RevertError(CITrainError):
     """Exception raised when there's a problem with reverting."""
-    prefix = 'Revert failed: '
     code = next(COUNTER)
 
 
 class AutopkgError(CITrainError):
     """Exception raised when there's a problem with autopkgtests."""
-    prefix = 'Autopkgtest: '
     code = next(COUNTER)
-    error = False

=== modified file 'cupstream2distro/packagemanager.py'
--- cupstream2distro/packagemanager.py	2015-11-04 16:04:14 +0000
+++ cupstream2distro/packagemanager.py	2015-11-18 06:50:36 +0000
@@ -38,6 +38,7 @@
     env,
     log_call,
     log_value_of,
+    suppress,
     utf8_inplace,
     utf8_open,
 )
@@ -109,18 +110,17 @@
 
     def get_package_version(self):
         """Get newest version from debian changelog."""
-        key = 'Version: '
-        try:
-            inst, stdout, stderr = call(['dpkg-parsechangelog'], cwd=self.path)
-        except FileNotFoundError:
-            return None
-        if inst.returncode != 0:
-            raise PackageError(stderr)
-        for line in stdout.splitlines():
-            if line.startswith(key):
-                return line.partition(key)[-1]
-        raise PackageError(
-            'dpkg-parsechangelog failure in {}.'.format(self.path))
+        # Performance summary of `python3 -mtimeit`:
+        # -s 'import re; x=re.compile("[()]")' 'x.split("foo (bar) baz")[1]'
+        # 1000000 loops, best of 3: 0.811 usec per loop
+        # '"foo (bar) baz".split("(")[-1].split(")")[0]'
+        # 1000000 loops, best of 3: 0.446 usec per loop
+        # 'x="foo (bar) baz"; x[x.find("(") + 1:x.find(")")]'
+        # 1000000 loops, best of 3: 0.382 usec per loop
+        with suppress(FileNotFoundError):
+            with utf8_open(self.debian_changelog) as change:
+                line = next(change)
+                return line[line.index('(') + 1:line.index(')')]
 
     def get_package_component(self):
         """Get the selected package component (main, universe) for series"""

=== modified file 'cupstream2distro/silomanager.py'
--- cupstream2distro/silomanager.py	2015-11-06 15:40:26 +0000
+++ cupstream2distro/silomanager.py	2015-11-18 06:50:36 +0000
@@ -30,7 +30,7 @@
 
 from glob import glob
 from random import shuffle
-from os.path import basename, dirname, join
+from os.path import dirname, join
 from collections import defaultdict
 
 from lazr.restfulclient.errors import PreconditionFailed, NotFound
@@ -91,7 +91,6 @@
 
 
 DEFAULT_EXCEPTHOOK = sys.excepthook
-DIRTY_FILE = join(SILOS_DIR, '{siloname}', '{package}_is_dirty')
 JSON = {'content-type': 'application/json'}
 MANDATORY = (
     'description',
@@ -108,19 +107,6 @@
     return string.replace(',', ' ').strip().split()
 
 
-def mark_packages_dirty(siloname, packages):
-    """Create {package}_is_dirty files in the silo dir."""
-    for package in packages:
-        with utf8_open(DIRTY_FILE.format(**locals()), 'w') as dirty_file:
-            dirty_file.write('We can rebuild it. We have the technology.')
-
-
-def get_dirty(siloname, package='*'):
-    """Identify all dirty packages in a given silo."""
-    for path in glob(DIRTY_FILE.format(**locals())):
-        yield basename(path).split('_')[0]
-
-
 def _bileto(api, **kwargs):
     """Send details to bileto."""
     try:
@@ -144,29 +130,16 @@
     _bileto(COMMENT_API, **kwargs)
 
 
-def stock_main(body, mark_dirty=False):
+def stock_main(body, returncode=0):
     """Generic error handling for CI Train scripts."""
     silo_state = SiloState(env.SILONAME, primary=True)
     try:
         body(silo_state)
     except CITrainError as err:
         silo_state.status = err
-        silo_state.mark_dirty()
-        silo_state.save_config()
-        return err.code
-    silo_state.mark_dirty()
+        returncode = err.code
     silo_state.save_config()
-    # This bit needs to be in a separate block because it's so slow that it
-    # makes the last call to save_config() end up losing races with
-    # check-publication-migration. Then you get logs that say
-    # 'Publishing. Migration:... Publishing. Migration:...'
-    if mark_dirty:
-        try:
-            silo_state.mark_others_dirty()
-        except Exception as err:
-            logging.warning('Had trouble marking other silos dirty:')
-            logging.warning(err)
-    return 0
+    return returncode
 
 
 class SiloState(object):
@@ -312,33 +285,6 @@
             with utf8_open(self.request_id_file.format(value), 'w') as rid:
                 rid.write(value)
 
-    def mark_others_dirty(self):
-        """Indicate that other silos need to be rebuilt."""
-        ours = set(self.all_projects)
-        log_value_of.ours('Looking for conflicts with')
-        for other in SiloState.iterate(ignore=self.siloname):
-            if self.series == other.series:
-                theirs = other.ppa.getPublishedSources(
-                    distro_series=self.series, status='Published')
-                theirs = [pack.source_package_name for pack in theirs]
-                log_value_of.theirs('Checking conflicts')
-                conflicting = sorted(ours & set(theirs))
-                if conflicting:
-                    self.info('Silo {} is now dirty.'.format(other))
-                    other.mark_dirty(conflicting)
-
-    def mark_dirty(self, packages=None):
-        """Indicate that this silo has packages needing a rebuild."""
-        msg = 'SILO DIRTY: You must rebuild: {}'
-        if packages:
-            mark_packages_dirty(self.siloname, packages)
-        # There may already be dirty packages from before, so we check that.
-        packages = sorted(get_dirty(self.siloname))
-        if packages:
-            self.status = msg.format(', '.join(packages))
-            self.push_to_bileto()
-        return packages
-
     def load_bileto(self):
         """Fetch request from Bileto."""
         rid = self.requestid
@@ -414,7 +360,7 @@
             siloname=self.siloname or self._siloname,
         )
         if self.strict:
-            # This is slow so don't do it when marking the entire world dirty.
+            # This is slow so don't do it when iterating over all silos.
             kwargs['sources'] = ' '.join(sorted(self.all_projects))
         update_bileto(**kwargs)
         bileto_comment(

=== modified file 'cupstream2distro/version.py'
--- cupstream2distro/version.py	2015-11-04 02:23:10 +0000
+++ cupstream2distro/version.py	2015-11-18 06:50:36 +0000
@@ -53,7 +53,7 @@
         self.v = version
 
     def __str__(self):
-        return self.v
+        return self.v or ''
 
     def get_upstream(self, epoch=False):
         """Remove ubuntu & epoch data from source package version.

=== modified file 'tests/unit/test_archive.py'
--- tests/unit/test_archive.py	2015-11-04 03:23:41 +0000
+++ tests/unit/test_archive.py	2015-11-18 06:50:36 +0000
@@ -21,38 +21,24 @@
 from mock import Mock, patch, call
 
 from tests.unit import BaseTestCase
-from tests.unit.test_recipe_base import BuildFake
+from tests.unit.test_recipe_base import BuildFake, Listish
 
 from cupstream2distro import archive
 from cupstream2distro.archive import Archive, ArchWatcher
 from cupstream2distro.errors import ArchiveError
 
 
-class Listish(list):
-    """Pretend to be a lazr collection."""
-    @property
-    def total_size(self):
-        """Report the length of this object."""
-        return len(self)
-
-
-class DistroArchSeries(object):
-    """Mimic a launchpadlib distro_arch_series object."""
-    def __init__(self):
-        self._arches = ['arm64', 'ppc64el', 'i386', 'amd64', 'powerpc']
-
-    @property
-    def architecture_tag(self):
-        """Return and remove an arch from the list of arches."""
-        return self._arches.pop()
-
-
 class BinaryFake(object):
     """Mimic a launchpadlib binary object."""
     def __init__(self, specific=True):
-        self.distro_arch_series = DistroArchSeries()
+        self._arches = ['arm64', 'ppc64el', 'i386', 'amd64', 'powerpc']
         self.architecture_specific = specific
 
+    @property
+    def distro_arch_series_link(self):
+        """Fake a URL pointing at a distro_arch_series."""
+        return 'http://example.com/long/url/pointing/to/' + self._arches.pop()
+
 
 class ArchiveTests(BaseTestCase):
     """Test that we can handle a Branch!"""
@@ -163,12 +149,10 @@
 
     @patch('cupstream2distro.archive.newest')
     def test_get_archive_version_none(self, new):
-        """Return '0' when there is no archive version."""
+        """Return None when there is no archive version."""
         new.return_value = None
         dest = Mock()
-        self.assertEqual(
-            self.archive.get_archive_version(dest),
-            '0')
+        self.assertIsNone(self.archive.get_archive_version(dest))
         new.assert_called_once_with(
             dest, source_name=self.archive.name,
             distro_series=self.archive.series)
@@ -201,7 +185,6 @@
         self.archive.silo_ppa.getPublishedSources.assert_called_once_with(
             distro_series=self.archive.series,
             source_name='archie',
-            status='Published',
             exact_match=True)
 
     def test_copy_to_archive(self):

=== modified file 'tests/unit/test_packagemanager.py'
--- tests/unit/test_packagemanager.py	2015-10-05 22:42:23 +0000
+++ tests/unit/test_packagemanager.py	2015-11-18 06:50:36 +0000
@@ -92,16 +92,7 @@
     @patch('cupstream2distro.packagemanager.call')
     def test_get_package_version_failed(self, call_mock):
         """Raise PackageError when dpkg-parsechangelog errors out."""
-        call_mock.return_value = (Mock(returncode=1), '', 'No such file')
-        with self.assertRaisesRegexp(PackageError, 'No such file'):
-            self.package.get_package_version()
-
-    @patch('cupstream2distro.packagemanager.call')
-    def test_get_package_version_failed2(self, call_mock):
-        """Raise PackageError when no Version: field found."""
-        call_mock.return_value = (Mock(returncode=0), '', '')
-        with self.assertRaisesRegexp(PackageError, 'parsechangelog failure'):
-            self.package.get_package_version()
+        self.assertIsNone(self.package.get_package_version())
 
     def test_preserve_package_version(self):
         """Check debian/control for no-rewrite-version flag."""

=== modified file 'tests/unit/test_recipe_base.py'
--- tests/unit/test_recipe_base.py	2015-11-10 18:16:52 +0000
+++ tests/unit/test_recipe_base.py	2015-11-18 06:50:36 +0000
@@ -29,8 +29,8 @@
 from citrain.recipes.base import (
     BuildBase,
     ClientError,
-    DEPWAIT_TIMEOUT as DW,
-    PPA_TIMEOUT as PT,
+    DEPWAIT_TIMEOUT,
+    PPA_TIMEOUT,
     check_swift,
 )
 from cupstream2distro.utils import env, utf8_open
@@ -38,7 +38,6 @@
     CITrainError,
     AutopkgError,
     PublishError,
-    MigrationError,
     NoStatusError,
 )
 
@@ -46,6 +45,14 @@
 N = '\n'
 
 
+class Listish(list):
+    """Pretend to be a lazr collection."""
+    @property
+    def total_size(self):
+        """Report the length of this object."""
+        return len(self)
+
+
 class BuildFake(object):
     """Mimic a launchpadlib build object."""
     def __init__(self, arch, status=''):
@@ -104,17 +111,88 @@
         for bit in bits:
             self.assertIn(bit, arg)
 
+    @patch('cupstream2distro.archive.newest')
+    def test_check_status_missing(self, new):
+        """Correctly identify when package missing from PPA."""
+        new.return_value = None
+        build = BuildBase('state', self.series, self.dest, self.ppa)
+        build.get_package_version = Mock(return_value='42-0ubuntu1')
+        self.assertEqual(build.check_status(), 'Not in PPA')
+
+    @patch('cupstream2distro.archive.newest')
+    def test_check_status_mismatch(self, new):
+        """Correctly identify when PPA has wrong version."""
+        new.return_value.source_package_version = '41-0ubuntu1'
+        build = BuildBase('state', self.series, self.dest, self.ppa)
+        build.get_package_version = Mock(return_value='42-0ubuntu1')
+        self.assertEqual(build.check_status(), '42-0ubuntu1 not in PPA')
+
+    @patch('cupstream2distro.archive.newest')
+    def test_check_status_pocket(self, new):
+        """Correctly identify when PPA has wrong version."""
+        new.return_value.source_package_version = '42-0ubuntu1'
+        new.return_value.pocket = 'Proposed'
+        build = BuildBase('state', self.series, self.dest, self.ppa)
+        build.get_package_version = Mock(return_value='42-0ubuntu1')
+        self.assertEqual(build.check_status(), 'Proposed pocket')
+
+    @patch('cupstream2distro.archive.newest')
+    def test_check_status_queue(self, new):
+        """Correctly identify when PPA has wrong version."""
+        sources = {
+            self.dest: None,
+            self.ppa: Mock(source_package_version='42-0ubuntu1'),
+        }
+        new.side_effect = lambda archive, *ign, **ignore: sources.get(archive)
+        self.series.getPackageUploads.return_value = Listish(
+            [Mock(status='new')])
+        build = BuildBase('state', self.series, self.dest, self.ppa)
+        build.get_package_version = Mock(return_value='42-0ubuntu1')
+        self.assertEqual(build.check_status(), 'NEW queue')
+
+    @patch('cupstream2distro.archive.newest')
+    def test_check_status_build_state(self, new):
+        """Correctly identify when PPA has wrong version."""
+        build = Mock(buildstate='Successfully built', arch_tag='i386')
+        source = Mock(
+            source_package_version='42-0ubuntu1',
+            getBuilds=Mock(return_value=[build])
+        )
+        sources = {
+            self.dest: None,
+            self.ppa: source,
+        }
+        new.side_effect = lambda archive, *ign, **ignore: sources.get(archive)
+        self.series.getPackageUploads.return_value = Listish()
+        build = BuildBase('state', self.series, self.dest, self.ppa)
+        build.get_package_version = Mock(return_value='42-0ubuntu1')
+        self.assertEqual(build.check_status(), 'Successfully built [i386]')
+
+    @patch('cupstream2distro.archive.newest')
+    def test_check_status_unknown(self, new):
+        """Just give up"""
+        new.return_value = None
+        build = BuildBase('state', self.series, self.dest, self.ppa)
+        build.check_silo_version = Mock(return_value=None)
+        build.check_new_commits = Mock(return_value=None)
+        build.check_destination_version = Mock(return_value=None)
+        build.archive_pocket = Mock(return_value=None)
+        build.archive_queue = Mock(return_value=None)
+        build.check_autopkgtest_status = Mock(return_value=None)
+        build.check_build_status = Mock(return_value=None)
+        self.assertEqual(build.check_status(), 'Unknown (this is a bug)')
+
     def test_getattr(self):
         """Ensure we can read files from disk."""
         path = join(self.tempdir, 'getattr_published_version')
         with utf8_open(path, 'w') as data:
             data.write('hello')
-        build = BuildBase('getattr', self.dest, self.series, self.ppa)
+        build = BuildBase('getattr', self.series, self.dest, self.ppa)
         self.assertEqual(build.published_version, 'hello')
 
     def test_getattr_failed(self):
         """Raise AttributeError for wrong attributes."""
-        build = BuildBase('wrong', self.dest, self.series, self.ppa)
+        build = BuildBase('wrong', self.series, self.dest, self.ppa)
         with self.assertRaises(AttributeError):
             build.zipblorp()
 
@@ -125,7 +203,7 @@
         self.assertFalse(isfile(new_path))
         with utf8_open(path, 'w') as data:
             data.write('watever\nglorp\npublished_version = hiya\n')
-        build = BuildBase('transition', self.dest, self.series, self.ppa)
+        build = BuildBase('transition', self.series, self.dest, self.ppa)
         self.assertEqual(build.published_version, 'hiya')
         with utf8_open(new_path) as data:
             self.assertEqual(data.read().strip(), 'hiya')
@@ -135,7 +213,7 @@
         path = join(self.tempdir, 'transition.project')
         with utf8_open(path, 'w') as data:
             data.write('published_version =\nha\n')
-        build = BuildBase('transition', self.dest, self.series, self.ppa)
+        build = BuildBase('transition', self.series, self.dest, self.ppa)
         self.assertEqual(build.published_version, '')
 
     def test_pre_validate(self):
@@ -265,21 +343,12 @@
         time_mock.time.return_value = 100
         BuildBase.failures = set()
         build = BuildBase('foo', self.series, self.dest, self.ppa)
-        self.assertTrue(
-            build.timed_out(started=50, timeout=10, message='failed at 10'))
-        self.assertTrue(
-            build.timed_out(started=0, timeout=99, message='failed at 99'))
-        self.assertIsNone(
-            build.timed_out(started=90, timeout=110, message='passed'))
-        self.assertIsNone(
-            build.timed_out(started=0, timeout=9, message='', condition=False))
-        self.assertEqual(BuildBase.failures, set([
-            'failed at 10',
-            'failed at 99',
-        ]))
+        self.assertTrue(build.timed_out(started=50, timeout=10))
+        self.assertTrue(build.timed_out(started=0, timeout=99))
+        self.assertIsNone(build.timed_out(started=90, timeout=110))
 
     @patch('citrain.recipes.base.time')
-    @patch('citrain.recipes.base.newest')
+    @patch('cupstream2distro.archive.newest')
     def test_buildbase_watch_phase(self, new_mock, time_mock):
         """A single build success is discovered immediately."""
         source = SourceFake('foo', '1.0-0ubuntu1')
@@ -294,15 +363,15 @@
         self.assertEqual(build.watch_phase(), 'success')
         new_mock.assert_called_once_with(
             self.ppa, source_name='foo', version='1.0-0ubuntu1',
-            exact_match=True)
+            distro_series=build.series)
         build.find_archive_arches.assert_called_once_with(build.main_archive)
-        build.get_package_version.assert_called_once_with()
+        self.assertEqual(build.get_package_version.mock_calls, [call()] * 2)
         time_mock.time.assert_called_once_with()
         self.assertEqual(time_mock.sleep.mock_calls, [])
         self.assertEqual(BuildBase.failures, set())
 
     @patch('citrain.recipes.base.time')
-    @patch('citrain.recipes.base.newest')
+    @patch('cupstream2distro.archive.newest')
     def test_buildbase_watch_phase_build_failure(self, new_mock, time_mock):
         """A single build success is discovered immediately."""
         source = SourceFake('foo', '1.0-0ubuntu1')
@@ -316,23 +385,23 @@
         build.timed_out = Mock(return_value=False)
         self.assertEqual(build.watch_phase(), 'build failed')
         new_mock.assert_called_once_with(
-            self.ppa, source_name='foo', exact_match=True,
+            self.ppa, source_name='foo', distro_series=build.series,
             version='1.0-0ubuntu1')
         build.find_archive_arches.assert_called_once_with(build.main_archive)
-        build.get_package_version.assert_called_once_with()
+        self.assertEqual(build.get_package_version.mock_calls, [call()] * 2)
         time_mock.time.assert_called_once_with()
         self.assertEqual(time_mock.sleep.mock_calls, [])
         self.assertEqual(
             BuildBase.failures, set(['foo failed to build.']))
 
     @patch('citrain.recipes.base.time')
-    @patch('citrain.recipes.base.newest')
+    @patch('cupstream2distro.archive.newest')
     def test_buildbase_watch_phase_dependency_wait(self, new_mock, time_mock):
         """Dependency wait times out after 1.5 hours."""
         t = time_mock.time.return_value
-        timeouts = [True, False, False, False, False, False, False]
+        timeouts = [True, False, False, False]
         source = SourceFake('foo', '1.0-0ubuntu1')
-        source._builds = [[BuildFake('amd64', 'Dependency wait')]] * 4
+        source._builds = [[BuildFake('amd64', 'Dependency wait')]] * 40
         self.ppa.getPublishedSources.return_value = source
         new_mock.return_value = (
             self.ppa.getPublishedSources.return_value)
@@ -341,30 +410,26 @@
         build.find_archive_arches = Mock(return_value=['amd64'])
         build.timed_out = Mock(side_effect=lambda *ignore: timeouts.pop())
         self.assertEqual(build.watch_phase(), 'depwait')
-        pprint(new_mock.mock_calls)
         self.assertEqual(new_mock.mock_calls, [
-            call(self.ppa, exact_match=True, source_name='foo',
-                 version='1.0-0ubuntu1'),
+            call(self.ppa, source_name='foo',
+                 version='1.0-0ubuntu1', distro_series=build.series),
         ] * 4)
         build.find_archive_arches.assert_called_once_with(build.main_archive)
-        build.get_package_version.assert_called_once_with()
+        self.assertEqual(build.get_package_version.mock_calls, [call()] * 5)
         time_mock.time.assert_called_once_with()
         self.assertEqual(time_mock.sleep.mock_calls, [
             call(300), call(300), call(300),
         ])
         self.assertEqual(
             build.timed_out.mock_calls, [
-                call(t, DW, 'Dependency wait timeout for foo.'),
-                call(t, PT, 'foo 1.0-0ubuntu1 not found in Silo PPA.', False),
-                call(t, DW, 'Dependency wait timeout for foo.'),
-                call(t, PT, 'foo 1.0-0ubuntu1 not found in Silo PPA.', False),
-                call(t, DW, 'Dependency wait timeout for foo.'),
-                call(t, PT, 'foo 1.0-0ubuntu1 not found in Silo PPA.', False),
-                call(t, DW, 'Dependency wait timeout for foo.'),
+                call(t, DEPWAIT_TIMEOUT),
+                call(t, DEPWAIT_TIMEOUT),
+                call(t, DEPWAIT_TIMEOUT),
+                call(t, DEPWAIT_TIMEOUT),
             ])
 
     @patch('citrain.recipes.base.time')
-    @patch('citrain.recipes.base.newest')
+    @patch('cupstream2distro.archive.newest')
     def test_buildbase_watch_phase_upload_failed(self, new_mock, time_mock):
         """Sometimes dput claims success but the PPA does not accept it."""
         t = time_mock.time.return_value
@@ -380,18 +445,18 @@
         pprint(new_mock.mock_calls)
         self.assertEqual(new_mock.mock_calls, [
             call(self.ppa, version='1.0-0ubuntu1',
-                 source_name='foo', exact_match=True),
+                 source_name='foo', distro_series=build.series),
         ] * 4)
         self.assertEqual(time_mock.sleep.mock_calls, [
             call(300), call(300), call(300),
         ])
         self.assertEqual(
             build.timed_out.mock_calls, [
-                call(t, PT, 'foo 1.0-0ubuntu1 not found in Silo PPA.', True),
+                call(t, PPA_TIMEOUT),
             ] * 4)
 
     @patch('citrain.recipes.base.time')
-    @patch('citrain.recipes.base.newest')
+    @patch('cupstream2distro.archive.newest')
     def test_buildbase_watch_phase_upload_failed_2(self, new_mock, time_mock):
         """PPA contains old, Deleted builds, but nothing current."""
         t = time_mock.time.return_value
@@ -399,20 +464,21 @@
         self.ppa.getPublishedSources.return_value = 'Deleted'
         new_mock.return_value = None
         build = BuildBase('foo', self.series, self.dest, self.ppa)
+        build.get_version = Mock(return_value='')
         build.get_package_version = Mock(return_value='')
         build.find_archive_arches = Mock(return_value=['amd64'])
         build.timed_out = Mock(side_effect=lambda *ignore: timeouts.pop())
         self.assertEqual(build.watch_phase(), 'timeout')
         pprint(new_mock.mock_calls)
         self.assertEqual(new_mock.mock_calls, [
-            call(self.ppa, source_name='foo', exact_match=True),
+            call(self.ppa, source_name='foo', distro_series=build.series),
         ] * 4)
         self.assertEqual(time_mock.sleep.mock_calls, [
             call(300), call(300), call(300),
         ])
         self.assertEqual(
             build.timed_out.mock_calls, [
-                call(t, PT, 'foo  not found in Silo PPA.', True),
+                call(t, PPA_TIMEOUT),
             ] * 4)
 
     def test_buildbase_pre_diff_phase(self):
@@ -626,7 +692,7 @@
     @patch('citrain.recipes.base.requests')
     @patch('citrain.recipes.base.urlretrieve')
     @patch('citrain.recipes.base.AUTOPKGTEST_ARCHES', {'armhf'})
-    def test_buildbase_autopkgtest_phase(self, url_mock, req_mock):
+    def test_buildbase_check_autopkgtest_status(self, url_mock, req_mock):
         """Ensure autopkgtests are polled correctly."""
         req_mock.get.return_value.status_code = 200
         req_mock.get.return_value.iter_lines.return_value = [
@@ -641,8 +707,8 @@
         build.silo_ppa.owner.name = 'polling'
         build.inspect_tar_files = Mock()
         build.get_file = Mock(return_value='20151106_141604')
-        with self.assertRaisesRegexp(AutopkgError, 'Autopkgtest: In prog'):
-            build.autopkgtest_phase()
+        self.assertEqual(
+            build.check_autopkgtest_status(), 'Autopkgtests running')
         url = req_mock.mock_calls[0][1][0]
         self.assertIn('autopkgtest-wily-polling-landing-543/', url)
         self.assertIn('marker=wily%2Farmhf%2Fw%2Fwhip%2F20151106_141604', url)
@@ -660,9 +726,9 @@
     @patch('citrain.recipes.base.requests')
     @patch('citrain.recipes.base.urlretrieve')
     @patch('citrain.recipes.base.AUTOPKGTEST_ARCHES', {'armhf'})
-    def test_buildbase_autopkgtest_phase_success(self, url_mock, req, exists):
+    def test_buildbase_check_autopkgtest_status_success(self, url, req, exist):
         """Inspect tarballs if all are found."""
-        exists.return_value = True
+        exist.return_value = True
         req.get.return_value.status_code = 200
         req.get.return_value.iter_lines.return_value = [
             b'foo/[e-mail]',
@@ -676,35 +742,37 @@
         build.silo_ppa.owner.name = 'polling'
         build.inspect_tar_files = Mock()
         build.get_file = Mock(return_value='20151106_141604')
-        build.autopkgtest_phase()
+        self.assertEqual(
+            build.check_autopkgtest_status(),
+            build.inspect_tar_files.return_value)
         req.get.assert_called_once_with(req.mock_calls[0][1][0])
         req.get.return_value.iter_lines.assert_called_once_with()
-        self.assertEqual(url_mock.mock_calls, [])
+        self.assertEqual(url.mock_calls, [])
         self.assertEqual(build.inspect_tar_files.mock_calls, [call()])
 
     @patch('citrain.recipes.base.requests')
     @patch('citrain.recipes.base.urlretrieve')
     @patch('citrain.recipes.base.AUTOPKGTEST_ARCHES', {'armhf'})
-    def test_buildbase_autopkgtest_phase_stale(self, url_mock, req_mock):
+    def test_buildbase_check_autopkgtest_status_stale(self, url, req):
         """Skip over stale results from swift."""
-        req_mock.get.return_value.status_code = 200
-        req_mock.get.return_value.iter_lines.return_value = [
+        req.get.return_value.status_code = 200
+        req.get.return_value.iter_lines.return_value = [
             b'foo/[e-mail]',
             b'foo/[e-mail]',
             b'foo/20151106_100104@whatever',
         ]
-        req_mock.get.return_value.url = 'http://example.com/?bar'
+        req.get.return_value.url = 'http://example.com/?bar'
         build = BuildBase('whip', self.series, self.dest, self.ppa)
         build.series.name = 'wily'
         build.silo_ppa.name = 'landing-543'
         build.silo_ppa.owner.name = 'polling'
         build.inspect_tar_files = Mock()
         build.get_file = Mock(return_value='20151106_141604')
-        with self.assertRaisesRegexp(AutopkgError, 'Autopkgtest: In prog'):
-            build.autopkgtest_phase()
-        req_mock.get.assert_called_once_with(req_mock.mock_calls[0][1][0])
-        req_mock.get.return_value.iter_lines.assert_called_once_with()
-        self.assertEqual(url_mock.mock_calls, [])
+        self.assertEqual(
+            build.check_autopkgtest_status(), 'Autopkgtests running')
+        req.get.assert_called_once_with(req.mock_calls[0][1][0])
+        req.get.return_value.iter_lines.assert_called_once_with()
+        self.assertEqual(url.mock_calls, [])
         self.assertEqual(build.inspect_tar_files.mock_calls, [])
 
     def test_buildbase_inspect_tar_files_pass(self):
@@ -718,8 +786,7 @@
         for inn, out in cases:
             shutil.copy(join(self.tarballs_dir, inn), join(self.tempdir, out))
         build = BuildBase('pass', self.series, self.dest, self.ppa)
-        with self.assertRaisesRegexp(AutopkgError, 'Autopkgtest: Success!'):
-            build.inspect_tar_files()
+        self.assertEqual(build.inspect_tar_files(), 'Autopkgtests succeeded')
 
     def test_buildbase_inspect_tar_files_fail(self):
         """Ensure failing tar files are reported correctly."""
@@ -732,8 +799,7 @@
         for inn, out in cases:
             shutil.copy(join(self.tarballs_dir, inn), join(self.tempdir, out))
         build = BuildBase('fail', self.series, self.dest, self.ppa)
-        with self.assertRaisesRegexp(AutopkgError, 'Autopkgtest: Failed'):
-            build.inspect_tar_files()
+        self.assertEqual(build.inspect_tar_files(), 'Autopkgtests failed')
 
     def test_buildbase_post_checkstatus_phase(self):
         """Silently allow publication when no problems found."""
@@ -752,23 +818,12 @@
     def test_buildbase_post_checkstatus_phase_fail(self):
         """Ensure that publish is halted if silo in bad state."""
         silo_state = Mock()
-        for problem in ('failed', 'dirty', 'error'):
+        for problem in ('failed', 'error'):
             silo_state.tokenize.return_value = {'everything', 'is', problem}
             silo_state.status = 'Everything is ' + problem
             with self.assertRaisesRegexp(NoStatusError, 'Silo has bad status'):
                 BuildBase.post_checkstatus_phase(silo_state)
 
-    @patch('citrain.recipes.base.glob')
-    def test_buildbase_post_checkstatus_phase_dirty(self, glob_mock):
-        """Ensure that publish is halted if silo in bad state."""
-        silo_state = Mock()
-        silo_state.tokenize.return_value = {'everything', 'is', 'awesome'}
-        silo_state.status = 'Soooo dirty'
-        glob_mock.return_value = ['foo_is_dirty', 'bar_is_dirty']
-        with self.assertRaisesRegexp(NoStatusError, 'Soooo dirty'):
-            BuildBase.post_checkstatus_phase(silo_state)
-        silo_state.mark_dirty.assert_called_once_with()
-
     @patch('citrain.recipes.base.lp')
     def test_buildbase_checkupload_phase(self, lp_mock):
         """Ensure that we call checkupload correctly."""
@@ -882,11 +937,9 @@
         build = BuildBase('grue', self.series, self.dest, self.ppa)
         build.dest_current_version = '1.0'
         build.get_archive_version.return_value = '1.1'
-        build.dest_version_check_phase()
-        self.assertEqual(build.failures, {
-            'Version of grue at dest changed from 1.0 to 1.1 '
-            'since packages built.',
-        })
+        with self.assertRaisesRegexp(PublishError, 'Unexpected upload'):
+            build.dest_version_check_phase()
+        self.assertEqual(build.failures, set())
 
     @patch('citrain.recipes.base.lp')
     @patch('citrain.recipes.base.BuildBase.get_archive_version', Mock())
@@ -979,34 +1032,6 @@
         self.assertEqual(silo_state.mock_calls, [])
         BuildBase.packagelist.write.assert_called_once_with()  # ONCE
 
-    def test_buildbase_migration_phase(self):
-        """Report the location status of this package."""
-        build = BuildBase('foo', self.series, self.dest, self.ppa)
-        build.published_version = '2.0'
-        build.check_archive_migration = Mock(return_value='Release pocket')
-        build.migration_phase()
-        build.check_archive_migration.assert_called_once_with('2.0')
-        self.assertEqual(
-            BuildBase.states,
-            dict(foo='Release pocket'))
-
-    def test_buildbase_post_migration_phase(self):
-        """Silently allow merging to proceed if everything is migrated."""
-        silo_state = Mock()
-        BuildBase.states = dict(
-            foo='Release pocket', bar='Updates pocket')
-        BuildBase.post_migration_phase(silo_state)
-        self.assertEqual(BuildBase.states, {})
-
-    def test_buildbase_post_migration_phase_incomplete(self):
-        """Raise exception if not everything has migrated."""
-        silo_state = Mock()
-        BuildBase.states = dict(
-            foo='UNAPPROVED queue', bar='void')
-        with self.assertRaisesRegexp(MigrationError, 'bar is in the void'):
-            BuildBase.post_migration_phase(silo_state)
-        self.assertEqual(BuildBase.states, {})
-
     def test_buildbase_enumeration_phase(self):
         """Don't raise an exception when versions match."""
         build = BuildBase('foo', self.series, self.dest, self.ppa)

=== modified file 'tests/unit/test_recipe_manager.py'
--- tests/unit/test_recipe_manager.py	2015-10-07 20:47:51 +0000
+++ tests/unit/test_recipe_manager.py	2015-11-18 06:50:36 +0000
@@ -159,3 +159,32 @@
         merge.liquid_phase.assert_called_once_with()
         sync.liquid_phase.assert_called_once_with()
         self.assertEqual(bbase.post_liquid_phase.mock_calls, [])
+
+    def test_manager_get_states(self):
+        """Fetch all the states from all the builds."""
+        silo_state = Mock()
+        one = Mock(check_status=Mock(return_value='Successfully built'))
+        one.name = 'one'
+        two = Mock(check_status=Mock(return_value='Successfully built'))
+        two.name = 'two'
+        one.series.name = two.series.name = 'xenial'
+        bman = Manager(silo_state)
+        bman.types = True
+        bman.builds = dict(one=one, two=two)
+        self.assertEqual(
+            bman.get_states(), 'Successfully built (one/xenial, two/xenial).')
+
+    def test_manager_get_states_multi(self):
+        """Fetch all the states from all the builds."""
+        silo_state = Mock()
+        one = Mock(check_status=Mock(return_value='Successfully built'))
+        one.name = 'one'
+        two = Mock(check_status=Mock(return_value='Failed to build'))
+        two.name = 'two'
+        one.series.name = two.series.name = 'xenial'
+        bman = Manager(silo_state)
+        bman.types = True
+        bman.builds = dict(one=one, two=two)
+        self.assertEqual(
+            bman.get_states(),
+            'Failed to build (two/xenial). Successfully built (one/xenial).')

=== modified file 'tests/unit/test_recipe_merge.py'
--- tests/unit/test_recipe_merge.py	2015-11-04 15:43:33 +0000
+++ tests/unit/test_recipe_merge.py	2015-11-18 06:50:36 +0000
@@ -442,28 +442,57 @@
         self.assertFalse(merge.failures)
 
     @patch('citrain.recipes.merge.read_revid')
-    @patch('citrain.recipes.merge.mark_packages_dirty')
-    def test_merge_unbuilt_phase(self, dirty, rev_mock):
+    def test_merge_check_new_commits(self, rev_mock):
+        """Report if this package has new commits."""
+        rev_mock.return_value = None
+        Merge.all_mps = dict(checkered=[Mock(
+            web_link='foo',
+            source_branch=Mock(last_scanned_id='zip'),
+            target_branch=Mock(last_scanned_id='zip', web_link='trunk'),
+        )])
+        merge = Merge('checkered', self.series, self.dest, self.ppa)
+        self.assertIn('Needs rebuild', merge.check_new_commits())
+        rev_mock.return_value = 'zip'
+        self.assertIsNone(merge.check_new_commits())
+
+    @patch('citrain.recipes.merge.read_revid')
+    def test_merge_unbuilt_phase(self, rev_mock):
         """Block publication if merges have new commits."""
         rev_mock.return_value = None
         Merge.all_mps = dict(foobaz=[Mock(
             web_link='foo',
             source_branch=Mock(last_scanned_id='zip'),
+            target_branch=Mock(last_scanned_id='zip', web_link='trunk'),
         )])
         merge = Merge('foobaz', self.series, self.dest, self.ppa)
         merge.unbuilt_phase()
-        dirty.assert_called_once_with(self.tempdir, ['foobaz'])
         self.assertEqual(merge.failures, {
             'foobaz has new, unbuilt commits.'
         })
 
     @patch('citrain.recipes.merge.read_revid')
+    def test_merge_unbuilt_phase_trunk(self, rev_mock):
+        """Block publication if trunk has new commits."""
+        rev_mock.return_value = 'match'
+        Merge.all_mps = dict(trunky=[Mock(
+            web_link='foo',
+            source_branch=Mock(last_scanned_id='match'),
+            target_branch=Mock(last_scanned_id='newness', web_link='trunk'),
+        )])
+        merge = Merge('trunky', self.series, self.dest, self.ppa)
+        merge.unbuilt_phase()
+        self.assertEqual(merge.failures, {
+            'trunky has new, unbuilt commits.'
+        })
+
+    @patch('citrain.recipes.merge.read_revid')
     def test_merge_unbuilt_phase_none(self, rev_mock):
         """Allow publication if there are no new commits."""
         rev_mock.return_value = 'zip'
         Merge.all_mps = dict(foobaz=[Mock(
             web_link='foo',
             source_branch=Mock(last_scanned_id='zip'),
+            target_branch=Mock(last_scanned_id='zip', web_link='trunk'),
         )])
         merge = Merge('foobaz', self.series, self.dest, self.ppa)
         merge.unbuilt_phase()

=== modified file 'tests/unit/test_script_autopkgtests.py'
--- tests/unit/test_script_autopkgtests.py	2015-11-06 18:20:52 +0000
+++ tests/unit/test_script_autopkgtests.py	2015-11-18 06:50:36 +0000
@@ -33,13 +33,14 @@
         self.assertEqual(self.script.Manager.mock_calls, [
             call(silo_state),
             call().do('validate', 'amqp'),
+            call().get_states(),
         ])
 
     def test_main_fail(self):
         """Ensure main() does error reporting correctly."""
         silo_state = Mock()
         self.script.Manager.return_value.do.side_effect = AutopkgError('no')
-        with self.assertRaisesRegexp(AutopkgError, 'Autopkgtest: no'):
+        with self.assertRaisesRegexp(AutopkgError, 'Autopkgtest failed: no'):
             self.script.autopkgtest(silo_state)
         self.assertEqual(self.script.Manager.mock_calls, [
             call(silo_state),

=== modified file 'tests/unit/test_script_build.py'
--- tests/unit/test_script_build.py	2015-11-06 06:35:12 +0000
+++ tests/unit/test_script_build.py	2015-11-18 06:50:36 +0000
@@ -46,34 +46,22 @@
             mps=dict(
                 c=[Mock(
                     source_branch=Mock(last_scanned_id=None),
+                    target_branch=Mock(
+                        last_scanned_id='foo', web_link='foo/trunk'),
                     web_link='foo/123')],
                 d=[Mock(
                     source_branch=Mock(last_scanned_id='stale'),
+                    target_branch=Mock(
+                        last_scanned_id='foo', web_link='foo/trunk'),
                     web_link='foo/789')]),
             sources=list('abcd'))
         self.script.env.PACKAGES_TO_REBUILD = ''
         self.script.env.FORCE_REBUILD = 'false'
+        self.script.env.WATCH_ONLY = 'false'
         bman = self.script.BuildManager(silo_state)
         bman.choose_packages()
         self.assertEqual(bman.names, {'a', 'b', 'd'})
 
-    def test_buildmanager_choose_packages_dirty(self):
-        """Include dirty packages by default."""
-        self.script.get_dirty = Mock(return_value=list('abcde'))
-        silo_state = Mock(
-            all_projects=list('abcd'),
-            mps=dict(
-                c=[Mock(
-                    source_branch=Mock(last_scanned_id=None),
-                    web_link='foo/123')]),
-            sources=list('abcd'))
-        self.script.env.PACKAGES_TO_REBUILD = ''
-        self.script.env.FORCE_REBUILD = 'false'
-        bman = self.script.BuildManager(silo_state)
-        bman.choose_packages()
-        self.script.get_dirty.assert_called_once_with(self.tempdir)
-        self.assertEqual(bman.names, set('abcd'))
-
     def test_buildmanager_choose_packages_twins(self):
         """Always add twins for all builds."""
         sources = ['qtmir', 'qtmir-gles', 'foo']
@@ -91,9 +79,13 @@
             mps={
                 'qtmir': [Mock(
                     source_branch=Mock(last_scanned_id=None),
+                    target_branch=Mock(
+                        last_scanned_id='foo', web_link='foo/trunk'),
                     web_link='qtmir/123')],
                 'qtmir-gles': [Mock(
                     source_branch=Mock(last_scanned_id='stale'),
+                    target_branch=Mock(
+                        last_scanned_id='foo', web_link='foo/trunk'),
                     web_link='qtmir-gles/789')]},
             sources=['qtmir', 'qtmir-gles'])
         self.script.env.PACKAGES_TO_REBUILD = ''
@@ -108,6 +100,8 @@
             all_projects=['a'],
             mps=dict(a=[Mock(
                 source_branch=Mock(last_scanned_id=None),
+                target_branch=Mock(
+                    last_scanned_id='foo', web_link='foo/trunk'),
                 web_link='foo/123')]),
             sources=[])
         self.script.env.PACKAGES_TO_REBUILD = ''
@@ -118,55 +112,36 @@
             bman.choose_packages()
         self.assertEqual(bman.names, set())
 
-    def test_main(self):
+    def test_build(self):
         """Run things without issue."""
         self.script.env.WATCH_ONLY = 'false'
-        silo_state = self.script.SiloState.return_value
+        silo_state = Mock()
         bman = self.script.BuildManager = Mock()
-        self.assertEqual(self.script.main(bman), 0)
+        self.assertIsNone(self.script.buildfactory(bman)(silo_state))
         bman.assert_called_once_with(silo_state)
         self.assertEqual(bman.return_value.mock_calls, [
             call.do('validate'),
             call.do('clean', 'collect', 'build', 'upload', 'expose'),
         ])
-        self.assertEqual(self.script.Manager.return_value.mock_calls, [
+        manager = self.script.Manager.return_value
+        self.assertEqual(manager.mock_calls, [
             call.do('watch', 'diff'),
+            call.get_states(),
         ])
-        self.assertEqual(silo_state.status, 'Packages built.')
-        silo_state.mark_dirty.assert_called_once_with()
-        silo_state.save_config.assert_called_once_with()
+        self.assertEqual(silo_state.status, manager.get_states())
 
-    def test_main_watch_only(self):
+    def test_build_watch_only(self):
         """Fewer phases during WATCH_ONLY."""
         self.script.env.WATCH_ONLY = 'true'
-        silo_state = self.script.SiloState.return_value
+        silo_state = Mock()
         bman = self.script.BuildManager = Mock()
-        self.assertEqual(self.script.main(bman), 0)
+        self.assertIsNone(self.script.buildfactory(bman)(silo_state))
         bman.assert_called_once_with(silo_state)
         self.assertEqual(bman.return_value.mock_calls, [
             call.do('validate'),
         ])
-        self.assertEqual(self.script.Manager.return_value.mock_calls, [
+        manager = self.script.Manager.return_value
+        self.assertEqual(manager.mock_calls, [
             call.do('watch', 'diff'),
+            call.get_states(),
         ])
-        silo_state.save_config.assert_called_once_with()
-
-    def test_main_error(self):
-        """Log raised exceptions."""
-        errs = [BuildError('It asploded!')]
-
-        def raise_errs(*ignore):
-            """Raise an error if there's one to be raised."""
-            if errs:
-                raise errs.pop()
-        silo_state = self.script.SiloState.return_value
-        bman = self.script.BuildManager = Mock()
-        bman.return_value.do = Mock(side_effect=raise_errs)
-        self.assertEqual(self.script.main(bman), 1)
-        bman.assert_called_once_with(silo_state)
-        self.assertEqual(
-            bman.return_value.mock_calls, [
-                call.do('validate'),
-            ])
-        self.assertEqual(str(silo_state.status), 'Build failed: It asploded!')
-        silo_state.save_config.assert_called_once_with()

=== modified file 'tests/unit/test_script_merge_clean.py'
--- tests/unit/test_script_merge_clean.py	2015-11-10 17:50:43 +0000
+++ tests/unit/test_script_merge_clean.py	2015-11-18 06:50:36 +0000
@@ -40,6 +40,7 @@
         self.script.Manager.assert_called_once_with(silo_state)
         mergemanager.do.assert_called_once_with(
             'validate', 'enumeration', 'merge', 'push', 'nuke')
+        self.assertEqual(silo_state.status, 'Landed')
 
     def test_merge_failed(self):
         """merge() doesn't catch exceptions."""
@@ -48,8 +49,9 @@
         mergemanager = self.script.Manager.return_value
         mergemanager.do.side_effect = MergeError('whoa buddy!')
         silo_state = self.script.SiloState.return_value
-        with self.assertRaisesRegexp(MergeError, 'Merging failed: whoa'):
+        with self.assertRaisesRegexp(MergeError, 'Merge&Clean failed: whoa'):
             self.script.merge(silo_state)
         self.script.Manager.assert_called_once_with(silo_state)
         mergemanager.do.assert_called_once_with(
             'validate', 'enumeration', 'merge', 'push', 'nuke')
+        assert silo_state.status != 'Landed'

=== modified file 'tests/unit/test_script_migration.py'
--- tests/unit/test_script_migration.py	2015-11-06 15:40:26 +0000
+++ tests/unit/test_script_migration.py	2015-11-18 06:50:36 +0000
@@ -22,7 +22,6 @@
 from tests.unit import CITrainScriptTestCase
 
 from cupstream2distro.settings import SILOS_DIR
-from cupstream2distro.errors import MigrationError
 
 # Unused import necessary for code coverage reporting
 from citrain import migration
@@ -41,6 +40,24 @@
             join(SILOS_DIR, 'ubuntu', 'landing-00{}'.format(x))
             for x in range(3)]
 
+    def test_success_regex(self):
+        """Ensure we can properly recognize silos ready to auto-merge."""
+        good = (
+            'Release pocket (foo).',
+            'Updates pocket (foo, bar).',
+            'Release pocket (foo, bar). Updates pocket (baz, qux).',
+            'Updates pocket (foo, bar). Release pocket (baz, qux).',
+        )
+        results = [bool(self.script.SUCCESS.match(stat)) for stat in good]
+        self.assertEqual(results, [True] * len(good))
+        bad = (
+            'Proposed pocket (foo).',
+            'NEW queue (foo, bar).',
+            'Dependency wait (foo, bar). Successfully built (baz, qux).',
+        )
+        results = [bool(self.script.SUCCESS.match(stat)) for stat in bad]
+        self.assertEqual(results, [False] * len(bad))
+
     def test_main(self):
         """Trigger merges after successful migrations."""
         tokenize = lambda: Mock(return_value={'hi'})
@@ -48,26 +65,24 @@
             tokenize=tokenize(),
             is_published=True,
             is_autopkgtesting=False) for rid in range(3)]
+        mgr = self.script.Manager.return_value
+        mgr.get_states.return_value = 'Release pocket (foo).'
         self.assertEqual(self.script.main(), 0)
         for state in states:
             self.assertEqual(state.mock_calls, [
                 call.enforce_lock(),
                 call.lock_fd.close(),
-                call.tokenize(),
             ])
         self.assertEqual(self.script.Manager.mock_calls, [
             call(states[0]),
-            call().do('unbuilt'),
-            call().do('migration'),
+            call().get_states(),
             call(states[1]),
-            call().do('unbuilt'),
-            call().do('migration'),
+            call().get_states(),
             call(states[2]),
-            call().do('unbuilt'),
-            call().do('migration'),
+            call().get_states(),
         ])
         self.assertEqual(self.script.stock_main.mock_calls, [
-            call(self.script.merge, mark_dirty=True)
+            call(self.script.merge)
         ] * 3)
 
     def test_main_skip_empty_silos(self):
@@ -91,22 +106,21 @@
         tokenize = Mock(return_value={'hi'})
         states = self.script.SiloState.iterate.return_value = [
             Mock(tokenize=tokenize, is_published=False) for rid in range(3)]
+        mgr = self.script.Manager.return_value
+        mgr.get_states.return_value = 'Proposed pocket (foo).'
         self.assertEqual(self.script.main(), 0)
         self.assertEqual(self.script.Manager.mock_calls, [
             call(states[0]),
-            call().do('unbuilt'),
-            call().do('autopkgtest'),
+            call().get_states(),
             call(states[1]),
-            call().do('unbuilt'),
-            call().do('autopkgtest'),
+            call().get_states(),
             call(states[2]),
-            call().do('unbuilt'),
-            call().do('autopkgtest'),
+            call().get_states(),
         ])
         self.assertEqual(self.script.stock_main.mock_calls, [])
         for state in states:
             self.assertEqual(state.set_migrating.mock_calls, [])
-            self.assertEqual(state.save_config.mock_calls, [])
+            self.assertEqual(state.save_config.mock_calls, [call()])
 
     def test_main_dont_merge_migrating_silos(self):
         """Don't merge silos that are still migrating."""
@@ -114,29 +128,19 @@
         states = self.script.SiloState.iterate.return_value = [Mock(
             tokenize=tokenize,
             is_published=True,
-            is_autopkgtesting=False,
             requestid=str(1),
         )]
-        states[0].mark_dirty.return_value = False
-
-        def set_side_effect(*ignore):
-            """Raise an exception second time mock is called."""
-            self.script.Manager.return_value.do.side_effect = MigrationError(
-                'foo is in the Proposed pocket.')
-        self.script.Manager.return_value.do.side_effect = set_side_effect
+        mgr = self.script.Manager.return_value
+        mgr.get_states.return_value = 'Proposed pocket (foo).'
         self.assertEqual(self.script.main(), 0)
-        self.assertEqual(
-            str(states[0].status), 'Migration: foo is in the Proposed pocket.')
+        self.assertEqual(states[0].status, mgr.get_states.return_value)
         self.assertEqual(states[0].mock_calls, [
             call.enforce_lock(),
             call.lock_fd.close(),
-            call.tokenize(),
-            call.mark_dirty(),
             call.save_config(),
         ])
         self.assertEqual(self.script.Manager.mock_calls, [
             call(states[0]),
-            call().do('unbuilt'),
-            call().do('migration'),
+            call().get_states(),
         ])
         self.assertEqual(self.script.stock_main.mock_calls, [])

=== modified file 'tests/unit/test_script_prepare_silo.py'
--- tests/unit/test_script_prepare_silo.py	2015-11-04 02:19:56 +0000
+++ tests/unit/test_script_prepare_silo.py	2015-11-18 06:50:36 +0000
@@ -48,4 +48,4 @@
     def test_main_fail(self):
         """Explode."""
         self.script.SiloState.return_value.assign.side_effect = PrepError
-        self.assertEqual(self.script.main(), 1)
+        self.assertEqual(self.script.main(), 6)

=== modified file 'tests/unit/test_script_publisher.py'
--- tests/unit/test_script_publisher.py	2015-11-09 18:09:39 +0000
+++ tests/unit/test_script_publisher.py	2015-11-18 06:50:36 +0000
@@ -37,14 +37,14 @@
     def test_publish_success(self):
         """publisher.publish() should call the right functions."""
         silo_state = Mock()
-        self.script.Manager = Mock()
+        mgr = self.script.Manager = Mock()
         self.assertIsNone(self.script.publish(silo_state))
-        self.script.Manager.assert_called_once_with(silo_state)
-        self.assertEqual(self.script.Manager.return_value.do.mock_calls, [
+        mgr.assert_called_once_with(silo_state)
+        self.assertEqual(mgr.return_value.do.mock_calls, [
             call('checkstatus', 'validate', 'diff', 'checkupload', 'ackaging'),
             call('dest_version_check', 'unapproved', 'unbuilt', 'publish'),
         ])
-        self.assertEqual(silo_state.status, 'Publishing.')
+        self.assertEqual(silo_state.status, 'Beginning publication...')
         self.assertEqual(silo_state.mock_calls, [
             call.save_config(),
         ])

=== modified file 'tests/unit/test_script_revert.py'
--- tests/unit/test_script_revert.py	2015-11-04 02:19:56 +0000
+++ tests/unit/test_script_revert.py	2015-11-18 06:50:36 +0000
@@ -126,11 +126,3 @@
             'foo': '1.0',
             'bar': '2.0',
         })
-
-    def test_main(self):
-        """Ensure main() triggers everything."""
-        self.script.build_main = Mock()
-        self.assertEqual(
-            self.script.main(), self.script.build_main.return_value)
-        self.script.build_main.assert_called_once_with(
-            self.script.RevertManager)

=== modified file 'tests/unit/test_silomanager.py'
--- tests/unit/test_silomanager.py	2015-11-06 15:40:26 +0000
+++ tests/unit/test_silomanager.py	2015-11-18 06:50:36 +0000
@@ -23,7 +23,7 @@
 import logging
 
 from mock import Mock, patch, call
-from os.path import join, exists
+from os.path import join
 
 from tests.unit import DirectoryAwareTestCase
 
@@ -63,7 +63,6 @@
         class Other:
             """Fake SiloState instance."""
             series = None
-            mark_dirty = Mock()
             ppa = Mock()
         self.other = Other
         super().setUp()
@@ -89,7 +88,6 @@
         silomanager.requests = Mock()
         silomanager.lp = Mock()
         silomanager.lp.load.side_effect = lambda x: Mock(self_link=x)
-        silomanager.DIRTY_FILE = join(self.tempdir, '{siloname}', '{package}')
         silomanager.SILO_NAME_LIST = dict(
             ubuntu=['ubuntu/landing-00{}'.format(x) for x in range(0, 10)])
 
@@ -117,12 +115,10 @@
     @patch('cupstream2distro.silomanager.SiloState')
     def test_stock_main(self, state_mock):
         """Ensure that the stock main() function works."""
-        self.assertEqual(stock_main(lambda state: state, True), 0)
+        self.assertEqual(stock_main(lambda state: state), 0)
         self.assertEqual(state_mock.mock_calls, [
             call(self.tempdir, primary=True),
-            call().mark_dirty(),
             call().save_config(),
-            call().mark_others_dirty(),
         ])
 
     @patch('cupstream2distro.silomanager.SiloState')
@@ -131,23 +127,10 @@
         def body(silo_state):
             """Raise a little hell."""
             raise PrepError('Boo!')
-        self.assertEqual(stock_main(body, True), PrepError.code)
-        self.assertEqual(state_mock.mock_calls, [
-            call(self.tempdir, primary=True),
-            call().mark_dirty(),
-            call().save_config(),
-        ])
-
-    @patch('cupstream2distro.silomanager.SiloState')
-    def test_stock_main_dirty_fail(self, state_mock):
-        """Ensure that the stock main() function handles dirty exceptions."""
-        state_mock.return_value.mark_others_dirty.side_effect = Exception('Oh')
-        self.assertEqual(stock_main(lambda state: state, True), 0)
-        self.assertEqual(state_mock.mock_calls, [
-            call(self.tempdir, primary=True),
-            call().mark_dirty(),
-            call().save_config(),
-            call().mark_others_dirty(),
+        self.assertEqual(stock_main(body), PrepError.code)
+        self.assertEqual(state_mock.mock_calls, [
+            call(self.tempdir, primary=True),
+            call().save_config(),
         ])
 
     def test_silostate_init(self):
@@ -278,24 +261,6 @@
         self.assertIsNone(self.state.source_archive)
         self.assertIsNone(self.state.source_series)
 
-    def test_mark_packages_dirty(self):
-        """Create dirty package marker files."""
-        names = ['bleep', 'blorp']
-        os.makedirs(join(self.tempdir, 'foo'))
-        silomanager.mark_packages_dirty('foo', names)
-        for name in names:
-            self.assertTrue(exists(join(self.tempdir, 'foo', name)))
-
-    def test_get_dirty(self):
-        """List packages that have been marked dirty."""
-        names = ['whiskey', 'tango', 'foxtrot']
-        os.makedirs(join(self.tempdir, 'mud'))
-        for name in names:
-            path = join(self.tempdir, 'mud', name)
-            with utf8_open(path, 'w') as dirt:
-                dirt.write('boo')
-        self.assertEqual(sorted(silomanager.get_dirty('mud')), sorted(names))
-
     def test_scrub(self):
         """Scrub API tokens out of messages."""
         cases = [
@@ -418,62 +383,6 @@
         self.state.status = NoStatusError('No')
         self.assertEquals(self.state.status, 'Preserved')
 
-    @patch('cupstream2distro.silomanager.SiloState.series')
-    @patch('cupstream2distro.silomanager.SiloState.iterate')
-    @patch('cupstream2distro.silomanager.SiloState.all_projects', ['qtmir'])
-    def test_silostate_mark_others_dirty(self, iterate, series):
-        """SiloState marks other silos dirty when told to."""
-        self.other.series = self.state.series
-        self.other.ppa.getPublishedSources.return_value = [
-            Mock(source_package_name='qtmir'),
-        ]
-        iterate.return_value = [self.other]
-        self.state.mark_others_dirty()
-        iterate.assert_called_once_with(ignore='ubuntu/landing-123')
-        self.other.mark_dirty.assert_called_once_with(['qtmir'])
-
-    @patch('cupstream2distro.silomanager.SiloState.series')
-    @patch('cupstream2distro.silomanager.SiloState.iterate')
-    @patch('cupstream2distro.silomanager.SiloState.all_projects', ['qtmir'])
-    def test_silostate_mark_others_dirty_none(self, iterate, series):
-        """SiloState skips non-conflicting silos."""
-        self.other.series = self.state.series
-        self.other.ppa.getPublishedSources.return_value = [
-            Mock(source_package_name='foobar'),
-        ]
-        iterate.return_value = [self.other]
-        self.state.mark_others_dirty()
-        iterate.assert_called_once_with(ignore='ubuntu/landing-123')
-        self.assertEqual(self.other.mark_dirty.mock_calls, [])
-
-    @patch('cupstream2distro.silomanager.SiloState.series')
-    @patch('cupstream2distro.silomanager.SiloState.iterate')
-    @patch('cupstream2distro.silomanager.SiloState.all_projects', ['qtmir'])
-    def test_silostate_mark_others_dirty_series(self, iterate, series):
-        """SiloState only marks same-series dirty."""
-        self.other.series = 'some other series'
-        self.other.ppa.getPublishedSources.return_value = [
-            Mock(source_package_name='qtmir'),
-        ]
-        iterate.return_value = [self.other]
-        self.state.mark_others_dirty()
-        iterate.assert_called_once_with(ignore='ubuntu/landing-123')
-        self.assertEqual(self.other.mark_dirty.mock_calls, [])
-
-    @patch('cupstream2distro.silomanager.mark_packages_dirty')
-    @patch('cupstream2distro.silomanager.get_dirty')
-    def test_silostate_mark_dirty(self, get, mark):
-        """SiloState correctly marks itself dirty."""
-        self.state.set_build_failed = Mock()
-        self.state.push_to_bileto = Mock()
-        get.return_value = list('dbca')
-        self.state.mark_dirty(list('abc'))
-        mark.assert_called_once_with(self.state.siloname, ['a', 'b', 'c'])
-        get.assert_called_once_with(self.state.siloname)
-        self.assertEqual(
-            self.state.status, 'SILO DIRTY: You must rebuild: a, b, c, d')
-        self.state.push_to_bileto.assert_called_once_with()
-
     @patch('cupstream2distro.silomanager.lp')
     @patch('cupstream2distro.silomanager.requests')
     @patch('cupstream2distro.silomanager.SiloState.requestid', '121')