Merge ~nacc/git-ubuntu:lp1731554-importer-rework into git-ubuntu:master

[Nish Aravamudan]

Make jenkins happy.

0fd899e... by Nish Aravamudan on 2017-11-17

importer: more consolidation

Unmerged commits

0fd899e... by Nish Aravamudan on 2017-11-17

importer: more consolidation

ff6d378... by Nish Aravamudan on 2017-11-11

importer: massive rework

I'm sorry, Robie :)

Basically, our current main importer loop looks like:

for applied in unapplied, applied:
    for dist in debian, ubuntu:
        for new publishes in $dist relative to $applied $dist branches
            import $applied publish
                this updates the branch pointer for the pocket + series
                if dist == 'ubuntu': this also updates the devel pointers

That is a fair amount of branch manipulation that will be discarded (the
series might have multiple publishes). Esp. on the first/reimport import
(worst case).

So instead:

for dist in debian, ubuntu:
    for applied in unapplied, applied:
        for new publishes in $dist relative to $applied $dist branches
            if publish has already been imported: continue
            import $applied publish

update all affected branch pointers,
    where affected is defined by running track of refs we *would* have
    updated in the original algorithm, and we also store the 'last'
    commit that it would have pointed to. This consolidates the branch
    and devel updates into one place as well.
    This also makes the importer algorithm match the spec on some level
    -- update the commit graph by importing only new publishes (which
    would create new import tags and new applied tags, while verifying
    any repeated publish data matches exactly) and then forcibly moving
    branch pointers to where they are "now" in Launchpad. The branch
    pointers are not part of the commit graph, so they change in a
    distinct step.

In my testing, this drops the reimport time for ipsec-tools from ~65
minutes to ~42 minutes consistently. We could probably speed it up more
starting from this base, if need be -- e.g., I'm not sure it makes sense
to to do the import action itself in the loop. Perhaps it's better to
accumulate the set of unique publishes in a set and then iterate that
set in a second loop.

Given that we are going to be doing a lot of reimporting soon, speeding
it up is a net win. This also tries to clean up code for readability,
and add comment as we go.

Note this does not resolve the fundamental issue I am sure exists:
orphan tags. It simply removes support for them.

LP: #1731554

e138c26... by Nish Aravamudan on 2017-11-10

source_information: drop unused parent_{applied,}_head_name

These are no longer used after the importer publishing parent drop.

2bbeba2... by Nish Aravamudan on 2017-11-10

importer: LP: #1731555

diff --git a/gitubuntu/dsc.py b/gitubuntu/dsc.py
index 1fc3129..91a78ea 100644
--- a/gitubuntu/dsc.py
+++ b/gitubuntu/dsc.py
@@ -84,3 +84,41 @@ class GitUbuntuDsc(Dsc):
     @property
     def all_tarball_paths(self):
         return [self.orig_tarball_path] + list(self.component_tarball_paths.values())
+
+    def _compare_dsc(self, other, field, key):
+        if field not in self or field not in other:
+            return False
+        our_checksums = dict(
+            (entry['name'], (int(entry['size']), entry[key]))
+            for entry in self[field]
+        )
+        their_checksums = dict(
+            (entry['name'], (int(entry['size']), entry[key]))
+            for entry in other[field]
+        )
+        # check that every file in ours is in theirs
+        for name, (size, checksum) in our_checksums.items():
+            if name not in their_checksums:
+                return False
+            if size != their_checksums[name][0] or checksum != their_checksums[name][1]:
+                return False
+        # check that every file in theirs is in ours
+        for name, (size, checksum) in their_checksums.items():
+            if name not in our_checksums:
+                return False
+            if size != our_checksums[name][0] or checksum != our_checksums[name][1]:
+                return False
+        # all files in ours and theirs for this hash match
+        return True
+
+    def compare_dsc(self, other):
+        """Check whether some set of hashes in these two dscs are
+        identical
+        """
+        return any(
+            self._compare_dsc(other, field, key) for field, key in (
+                ('Checksums-Sha256', 'sha256'),
+                ('Checksums-Sha1', 'sha1'),
+                ('Files', 'md5sum'),
+            )
+        )
diff --git a/gitubuntu/git_repository.py b/gitubuntu/git_repository.py
index c598c10..45d5166 100644
--- a/gitubuntu/git_repository.py
+++ b/gitubuntu/git_repository.py
@@ -537,6 +537,20 @@ class GitUbuntuRepository:
                                  )
 
     @contextmanager
+    def dsc_branch(self, dist, namespace='pkg'):
+        """Context manager wrapping dsc branch manipulation
+
+        In this context, HEAD will point to the dsc branch for
+        @namespace.
+        """
+        dsc_branch = '%s/importer/%s/dsc' % (namespace, dist)
+        with self.temporary_worktree(dsc_branch):
+            try:
+                yield
+            except:
+                raise
+
+    @contextmanager
     def pristine_tar_branches(self, dist, namespace='pkg'):
         """Context manager wrapping pristine-tar branch manipulation
 
@@ -1106,16 +1120,15 @@ class GitUbuntuRepository:
                 'Cannot get changelog source package name'
             )
 
-    def get_heads_and_versions(self, head_prefix, namespace):
+    def get_heads_and_versions(self, head_ref_namespace):
         """Extract the last version in debian/changelog of all
-        '<namespace>/<head_prefix>/debian/*' and
-        '<namespace>/<head_prefix>/ubuntu/*' branches.
+        '<namespace>/debian/*' and
+        '<namespace>/ubuntu/*' branches.
         """
         versions = dict()
         errors = False
         for head in self.local_branches:
-            prefix = '%s/%s' % (namespace, head_prefix)
-            if not head.branch_name.startswith(prefix):
+            if not head.branch_name.startswith(head_ref_namespace):
                 continue
             if 'ubuntu/devel' in head.branch_name:
                 continue
diff --git a/gitubuntu/importer.py b/gitubuntu/importer.py
index 2063f83..bea993d 100644
--- a/gitubuntu/importer.py
+++ b/gitubuntu/importer.py
@@ -26,8 +26,6 @@
 
 import argparse
 import atexit
-import functools
-import getpass
 import logging
 import os
 import re
@@ -45,14 +43,13 @@ from gitubuntu.dsc import (
 from gitubuntu.git_repository import (
     GitUbuntuRepository,
     GitUbuntuRepositoryFetchError,
-    orphan_tag,
     applied_tag,
     import_tag,
-    upstream_tag,
     PristineTarError,
     is_dir_3_0_quilt,
 )
-from gitubuntu.run import decode_binary, run, runq
+import gitubuntu.importerutils
+from gitubuntu.run import run, runq
 from gitubuntu.source_information import (
     GitUbuntuSourceInformation,
     NoPublicationHistoryException,
@@ -130,6 +127,14 @@ def cleanup(no_clean, local_dir):
         logging.info('Leaving %s as directed' % local_dir)
 
 
+def debug_log_head_versions(head_versions):
+    for head_name in sorted(head_versions):
+        logging.debug('Last imported %s version is %s',
+            head_name,
+            head_versions[head_name]['version']
+        )
+
+
 # XXX: need a namedtuple to hold common arguments
 def main(
     pkgname,
@@ -195,6 +200,9 @@ def main(
     else:
         namespace = owner
 
+    # now sets a global _PARENT_OVERRIDES
+    parse_parentfile(parentfile, pkgname)
+
     logging.info('Ubuntu Server Team importer v%s' % VERSION)
 
     repo = GitUbuntuRepository(directory, user, proto)
@@ -203,6 +211,13 @@ def main(
 
     atexit.register(cleanup, no_clean, repo.local_dir)
 
+    if dl_cache is None:
+        workdir = os.path.join(repo.git_dir, CACHE_PATH)
+    else:
+        workdir = dl_cache
+
+    os.makedirs(workdir, exist_ok=True)
+
     repo.add_base_remotes(pkgname, repo_owner=owner)
     if not no_fetch and not reimport:
         try:
@@ -241,97 +256,21 @@ def main(
 
     repo.ensure_importer_branches_exist(namespace)
 
-    debian_sinfo = GitUbuntuSourceInformation(
-        'debian',
-        pkgname,
-        os.path.abspath(pullfile),
-        retries,
-        retry_backoffs,
-    )
-
-    ubuntu_sinfo = GitUbuntuSourceInformation(
-        'ubuntu',
-        pkgname,
-        os.path.abspath(pullfile),
-        retries,
-        retry_backoffs,
-    )
-
-    debian_head_versions = (
-        repo.get_heads_and_versions('debian', namespace)
-    )
-    for head_name in sorted(debian_head_versions):
-        _, _, pretty_head_name = head_name.partition('%s/' % namespace)
-        logging.debug('Last imported %s version is %s',
-            pretty_head_name,
-            debian_head_versions[head_name]['version']
-        )
-
-    ubuntu_head_versions = (
-        repo.get_heads_and_versions('ubuntu', namespace)
-    )
-    for head_name in sorted(ubuntu_head_versions):
-        _, _, pretty_head_name = head_name.partition('%s/' % namespace)
-        logging.debug('Last imported %s version is %s',
-            pretty_head_name,
-            ubuntu_head_versions[head_name]['version']
-        )
-
-    if not skip_applied:
-        applied_debian_head_versions = (
-            repo.get_heads_and_versions(
-                'applied/debian', namespace
-            )
-        )
-        for head_name in sorted(applied_debian_head_versions):
-            _, _, pretty_head_name = head_name.partition(
-                '%s/' % namespace
-            )
-            logging.debug('Last applied %s version is %s',
-                pretty_head_name,
-                applied_debian_head_versions[head_name]['version']
-            )
-
-        applied_ubuntu_head_versions = (
-            repo.get_heads_and_versions(
-                'applied/ubuntu', namespace
-            )
-        )
-        for head_name in sorted(applied_ubuntu_head_versions):
-            _, _, pretty_head_name = head_name.partition(
-                '%s/' % namespace
-            )
-            logging.debug('Last applied %s version is %s',
-                pretty_head_name,
-                applied_ubuntu_head_versions[head_name]['version']
-            )
-
     oldcwd = os.getcwd()
     os.chdir(repo.local_dir)
 
-    if dl_cache is None:
-        workdir = os.path.join(repo.git_dir, CACHE_PATH)
-    else:
-        workdir = dl_cache
-
-    os.makedirs(workdir, exist_ok=True)
-
-    # now sets a global _PARENT_OVERRIDES
-    parse_parentfile(parentfile, pkgname)
-
     only_debian, history_found = import_publishes(
         repo=repo,
         pkgname=pkgname,
-        namespace=namespace,
-        patches_applied=False,
-        debian_head_versions=debian_head_versions,
-        ubuntu_head_versions=ubuntu_head_versions,
-        debian_sinfo=debian_sinfo,
-        ubuntu_sinfo=ubuntu_sinfo,
+        base_ref_namespace=namespace,
         active_series_only=active_series_only,
         workdir=workdir,
         skip_orig=skip_orig,
+        skip_applied=skip_applied,
         allow_applied_failures=allow_applied_failures,
+        pullfile=pullfile,
+        retries=retries,
+        retry_backoffs=retry_backoffs,
     )
 
     if not history_found:
@@ -339,29 +278,6 @@ def main(
                       "Wrong source package name?", pkgname)
         return 1
 
-    if not skip_applied:
-        import_publishes(
-            repo=repo,
-            pkgname=pkgname,
-            namespace=namespace,
-            patches_applied=True,
-            debian_head_versions=applied_debian_head_versions,
-            ubuntu_head_versions=applied_ubuntu_head_versions,
-            debian_sinfo=debian_sinfo,
-            ubuntu_sinfo=ubuntu_sinfo,
-            active_series_only=active_series_only,
-            workdir=workdir,
-            skip_orig=skip_orig,
-            allow_applied_failures=allow_applied_failures,
-        )
-
-    update_devel_branches(
-        repo=repo,
-        namespace=namespace,
-        pkgname=pkgname,
-        ubuntu_sinfo=ubuntu_sinfo,
-    )
-
     os.chdir(oldcwd)
 
     repo.garbage_collect()
@@ -476,49 +392,48 @@ def _commit_import(
     debian/changelog.
 
     Arguments:
-    spi - A SourcePackageInformation instance for this publish
-    tree_hash - SHA1 from git-dsc-commit --tree-only of this publish
-    changelog_parent_commit = SHA1 of changelog parent
-    upload_parent_commit = SHA1 of upload parent
-    unapplied_parent_commit = SHA1 of unapplied-patches import parent
+    repo - a gitubuntu.git_repository.GitUbuntuRepositoory object
+    spi - a
+    gitubuntu.source_information.GitUbuntuSourcePackageInformation
+    object for the publish in @tree_hash
+    tree_hash - string SHA1 from git-dsc-commit --tree-only of this publish
+    changelog_parent_commit - string SHA1 of changelog parent
+    upload_parent_commit - string SHA1 of upload parent
+    unapplied_parent_commit - string SHA1 of unapplied-patches import
+    parent
     """
     tag = None
 
     if unapplied_parent_commit:
         import_type = 'patches-applied'
-        target_head_name = spi.applied_head_name(namespace)
         if repo.get_applied_tag(spi.version, namespace) is None:
             # Not imported before
             tag = applied_tag(spi.version, namespace)
     else:
         import_type = 'patches-unapplied'
-        target_head_name = spi.head_name(namespace)
         if repo.get_import_tag(spi.version, namespace) is None:
             # Not imported before
             tag = import_tag(spi.version, namespace)
 
-    # Do not show importer/ namespace to user
-    _, _, pretty_head_name = target_head_name.partition('%s/' % namespace)
-
     changelog_entry = get_changelog_for_commit(
         repo,
         tree_hash,
         changelog_parent_commit
     )
-    msg = (b'Import %s version %b to %b\n\nImported using git-ubuntu import.' %
-           (import_type.encode(), spi.version.encode(), pretty_head_name.encode())
-          )
+    msg = b'Import %s version %b\n\nImported using git-ubuntu import.' % (
+        import_type.encode(),
+        spi.version.encode(),
+    )
 
     parents = []
 
-    if changelog_parent_commit is None and \
-        upload_parent_commit is None and \
-        unapplied_parent_commit is None and \
-        target_head_name in repo.local_branch_names:
-        # No parents and not creating a new branch
-        tag = orphan_tag(spi.version, namespace)
-    else:
-        msg += b'\n'
+    #if changelog_parent_commit is None and \
+    #    upload_parent_commit is None and \
+    #    unapplied_parent_commit is None:
+    #    # XXX: No parents and not creating a new branch
+    #    tag = orphan_tag(spi.version, namespace)
+    #else:
+    msg += b'\n'
 
     if changelog_parent_commit is not None:
         parents.append(changelog_parent_commit)
@@ -539,19 +454,26 @@ def _commit_import(
         environment_spi=spi
     )
 
-    repo.update_head_to_commit(target_head_name, commit_hash)
-
-    logging.debug('Committed %s import of %s as %s in %s',
-                 import_type, spi.version, commit_hash, pretty_head_name
-                )
+    logging.debug(
+        "Committed %s import of %s as %s",
+        import_type,
+        spi.version,
+        commit_hash,
+    )
 
     if tag is not None:
         # should be annotated to use create_tag API
         logging.debug('Creating tag %s pointing to %s', tag, commit_hash)
-        repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION,
-                      tag, commit_hash
-                     ]
-                    )
+        repo.git_run(
+            [
+                'tag',
+                '-a',
+                '-m',
+                'git-ubuntu import v%s' % VERSION,
+                tag,
+                commit_hash,
+            ],
+        )
 
 def commit_unapplied_patches_import(
     repo,
@@ -592,9 +514,63 @@ def commit_applied_patches_import(
         unapplied_parent_commit,
     )
 
+def dsc_already_imported(
+    repo,
+    spi,
+    ref_namespace,
+    distname,
+):
+    # always check if the same dist has a matching dsc
+    with repo.dsc_branch(
+        namespace=ref_namespace,
+        dist=distname,
+    ):
+        cached_dsc_path = os.path.join(
+            os.getcwd(),
+            spi.version,
+            os.path.basename(spi.dsc_pathname),
+        )
+
+        if os.path.exists(cached_dsc_path):
+            # If we have imported this version before, compare the
+            # DSC in the DSC branch to this one
+            imported_dsc = GitUbuntuDsc(cached_dsc_path)
+            return GitUbuntuDsc(spi.dsc_pathname).compare_dsc(imported_dsc)
+
+    # if a debian dsc exists and matches, but an ubuntu dsc does not
+    # exist, import the ubuntu dsc, but signal that a full import is
+    # not necessary
+    if distname == 'ubuntu':
+        with repo.dsc_branch(
+            namespace=ref_namespace,
+            dist='debian',
+        ):
+            imported_dsc_path = os.path.join(
+                os.getcwd(),
+                spi.version,
+                os.path.basename(spi.dsc_pathname),
+            )
+
+            if os.path.exists(imported_dsc_path):
+                # If we have imported this version before, compare the
+                # DSC in the DSC branch to this one
+                imported_dsc = GitUbuntuDsc(imported_dsc_path)
+                if GitUbuntuDsc(spi.dsc_pathname).compare_dsc(imported_dsc):
+                    import_dsc(
+                        repo,
+                        GitUbuntuDsc(spi.dsc_pathname),
+                        ref_namespace,
+                        str(spi.version),
+                        spi.distribution.name.lower(),
+                    )
+                    return True
+    return False
+
 def import_dsc(repo, dsc, namespace, version, dist):
     """Imports the dsc file to importer/{debian,ubuntu}/dsc
 
+    XXX use a context manager?
+
     Arguments:
     spi - A GitUbuntuSourcePackageInformation instance
     """
@@ -937,7 +913,7 @@ def _devel_branch_updates(
 
     devel_hash = None
     devel_name = None
-    devel_branch_name = ref_prefix + 'ubuntu/devel'
+    devel_branch_name = ref_prefix + '/devel'
 
     for series_name in series_name_list:
         series_devel_hash = None
@@ -946,7 +922,7 @@ def _devel_branch_updates(
         # Find highest version publish in these pockets, favoring this
         # order of pockets
         for suff in ['-proposed', '-updates', '-security', '']:
-            name = "%subuntu/%s%s" % (ref_prefix, series_name, suff)
+            name = "%s/%s%s" % (ref_prefix, series_name, suff)
             if name in head_versions:
                 head_commit, head_version = head_versions[name]
                 if version_compare(head_version, series_devel_version) > 0:
@@ -954,7 +930,7 @@ def _devel_branch_updates(
                     series_devel_version = head_version
                     series_devel_hash = head_commit
         if series_devel_hash:
-            series_devel_branch_name = '%subuntu/%s-devel' % (
+            series_devel_branch_name = '%s/%s-devel' % (
                 ref_prefix,
                 series_name,
             )
@@ -969,87 +945,213 @@ def _devel_branch_updates(
                 devel_name = series_devel_name
                 devel_hash = series_devel_hash
 
-    yield ref_prefix + 'ubuntu/devel', devel_hash
+    yield devel_branch_name, devel_hash
 
 
-def update_devel_branches(
+def update_branches(
+    distname,
+    dist_sinfo,
     repo,
-    namespace,
+    head_ref_namespace,
     pkgname,
-    ubuntu_sinfo,
+    updated_head_refs,
 ):
-    """update_devel_branches - move the 'meta' -devel branches to the
-    latest publication in each series
+    """update_branches - move the series+pocket branches based upon Launchpad imports
 
-    For all known series in @ubuntu_sinfo, update the
-    ubuntu/series-devel branch to the latest version in the series'
-    pocket branches.
-
-    Update the ubuntu/devel branch to the latest ubuntu/series-devel
-    branch.
-
-    Do this for both the unapplied and applied branches.
+    If distname is 'ubuntu', ubuntu/series-devel branches and update the
+    ubuntu/devel branch to the latest ubuntu/series-devel branch.
 
     Arguments:
+    distname - string distribution name (one of 'debian', 'ubuntu')
+    dist_sinfo - gitubuntu.source_information.GitUbuntuSourceInformation object
     repo - gitubuntu.git_repository.GitUbuntuRepository object
+    head_ref_namespace - string namespace under which the relevant
+    branch refs can be found
     pkgname - string source package name
-    namespace - string namespace under which the relevant branch refs
-    can be found
-    ubuntu_sinfo - GitUbuntuSourceInformation object for the ubuntu
-    archive
     """
-    for applied_prefix in ['', 'applied/']:
-        head_versions = {
-            ref: (str(target['head'].peel().id), target['version'])
-            for ref, target in repo.get_heads_and_versions(
-                    '%subuntu' % applied_prefix,
-                    namespace=namespace,
-                ).items()
-        }
-        for ref, commit in _devel_branch_updates(
-            ref_prefix=('%s/%s' % (namespace, applied_prefix)),
-            head_versions=head_versions,
-            series_name_list=ubuntu_sinfo.all_series_name_list,
-        ):
-            if commit is None:
-                logging.warn(
-                    "Source package '%s' does not appear to have been published "
-                    "in Ubuntu. No %s created.",
-                    pkgname,
-                    ref,
+    for ref, commit_hash in updated_head_refs.items():
+        repo.update_head_to_commit(
+            ref,
+            commit_hash,
+        )
+
+    if distname == 'debian':
+        return
+
+    head_versions = {
+        ref: (str(target['head'].peel().id), target['version'])
+        for ref, target in repo.get_heads_and_versions(
+                head_ref_namespace,
+            ).items()
+    }
+    for ref, commit in _devel_branch_updates(
+        ref_prefix=head_ref_namespace,
+        head_versions=head_versions,
+        series_name_list=dist_sinfo.all_series_name_list,
+    ):
+        if commit is None:
+            logging.warn(
+                "Source package '%s' does not appear to have been published "
+                "in Ubuntu. No %s created.",
+                pkgname,
+                ref,
+            )
+        else:
+            logging.debug(
+                "Setting %s to '%s'",
+                ref,
+                commit,
+            )
+            repo.update_head_to_commit(ref, commit)
+
+
+def get_changelog_parent_commit(
+    repo,
+    spi,
+    ref_namespace,
+    import_tree_versions,
+    patches_applied,
+):
+    if spi.version in _PARENT_OVERRIDES:
+        logging.debug(
+            '%s is specified in the parent override file.',
+            spi.version
+        )
+
+        try:
+            (
+                unapplied_changelog_parent_commit,
+                applied_changelog_parent_commit, 
+            ) = override_parents(repo, spi, ref_namespace)
+            if patches_applied:
+                return applied_changelog_parent_commit
+            else:
+                return unapplied_changelog_parent_commit
+        except ParentOverrideError as e:
+            logging.error("%s" % e)
+            return None
+    else:
+        # Walk changelog backwards until we find an imported version
+        for version in import_tree_versions:
+            if patches_applied:
+                changelog_parent_tag = repo.get_applied_tag(
+                    version,
+                    ref_namespace,
                 )
             else:
-                logging.debug(
-                    "Setting %s to '%s'",
-                    ref,
-                    commit,
+                changelog_parent_tag = repo.get_import_tag(
+                    version,
+                    ref_namespace,
                 )
-                repo.update_head_to_commit(ref, commit)
+            if changelog_parent_tag:
+                # sanity check that version from d/changelog of the
+                # tagged parent matches ours
+                parent_changelog_version, _ = \
+                    repo.get_changelog_versions_from_treeish(
+                        str(changelog_parent_tag.peel(pygit2.Tree).id),
+                    )
+                # if the parent was imported via a parent override
+                # itself, assume it is valid without checking its
+                # tree's debian/changelog, as it wasn't used
+                if (version not in _PARENT_OVERRIDES and
+                    parent_changelog_version != version
+                ):
+                    logging.error(
+                        "Found a tag corresponding to parent version "
+                        "%s, but d/changelog version (%s) differs. Will "
+                        "orphan commit.",
+                        version,
+                        parent_changelog_version,
+                    )
+                    return None
+                else:
+                    logging.debug("Changelog parent (tag) is %s",
+                        repo.tag_to_pretty_name(changelog_parent_tag)
+                    )
+                    return str(changelog_parent_tag.peel().id)
+    return None
+
+
+def get_unapplied_changelog_parent_commit(
+    repo,
+    spi,
+    ref_namespace,
+    import_tree_versions,
+):
+    return get_changelog_parent_commit(
+        repo,
+        spi,
+        ref_namespace,
+        import_tree_versions,
+        patches_applied=False,
+    )
+
+
+def get_applied_changelog_parent_commit(
+    repo,
+    spi,
+    ref_namespace,
+    import_tree_versions,
+):
+    return get_changelog_parent_commit(
+        repo,
+        spi,
+        ref_namespace,
+        import_tree_versions,
+        patches_applied=True,
+    )
+
+
+def get_upload_parent_commit(
+    repo,
+    spi,
+    ref_namespace,
+    unapplied_import_tree_hash,
+):
+    upload_tag = repo.get_upload_tag(spi.version, ref_namespace)
+    if upload_tag:
+        if str(upload_tag.peel().tree.id) != unapplied_import_tree_hash:
+            logging.warn(
+                "Found upload tag %s, but the corresponding tree "
+                "does not match the published version. Will import %s as "
+                "normal, ignoring the upload tag.",
+                repo.tag_to_pretty_name(upload_tag),
+                spi.version,
+            )
+            return None
+        else:
+            upload_parent_commit = str(upload_tag.peel().id)
+            return upload_parent_commit
+
+    return None
 
 
 # imports a package based upon source package information
-def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo):
+def import_unapplied_spi(repo, spi, ref_namespace, skip_orig):
     """Imports a source package from Launchpad into the git
     repository
 
+    Only called if the version has not been imported before
+
     Arguments:
-    spi - a SourcePackageInformation instance
+    repo - a gitubutu.git_repository.GitUbuntuRepository object
+    spi - a
+    gitubuntu.source_information.GitUbuntuSourcePackageInformation object
+    ref_namespace - a string Git-ref namespace this import will live in
+    skip_orig - a boolean indicating the orig tarball should not
+    be imported
     """
-    pretty_head_name = spi.pretty_head_name
 
-    logging.info('Importing patches-unapplied %s to %s',
-        spi.version, pretty_head_name
-    )
-    unapplied_tip_head = repo.get_head_by_name(spi.head_name(namespace))
+    logging.info("Importing patches-unapplied %s", spi.version)
 
-    spi.pull()
+    assert not repo.get_import_tag(spi.version, ref_namespace)
 
     repo.clean_repository_state()
 
     import_dsc(
         repo,
         GitUbuntuDsc(spi.dsc_pathname),
-        namespace,
+        ref_namespace,
         str(spi.version),
         spi.distribution.name.lower(),
     )
@@ -1058,7 +1160,7 @@ def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo):
             import_orig(
                 repo,
                 GitUbuntuDsc(spi.dsc_pathname),
-                namespace,
+                ref_namespace,
                 str(spi.version),
                 spi.distribution.name.lower(),
             )
@@ -1094,132 +1196,46 @@ def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo):
             'source pkg version: {} != changelog version: {}'.format(
                 spi.version, changelog_version))
 
-    unapplied_tip_version = None
-    # check if the version to import has already been imported to
-    # this head
-    if unapplied_tip_head is not None:
-        if repo.treeishs_identical(
-            unapplied_import_tree_hash, str(unapplied_tip_head.peel().id)
-        ):
-            logging.warn('%s is identical to %s',
-                         pretty_head_name, spi.version
-                        )
-            return
-        unapplied_tip_version, _ = repo.get_changelog_versions_from_treeish(
-                             str(unapplied_tip_head.peel().id),
-                                                                 )
-
-    logging.debug('Tip version is %s', unapplied_tip_version)
-
-    unapplied_changelog_parent_commit = None
-    upload_parent_commit = None
-    unapplied_parent_commit = None
-
-    if spi.version in _PARENT_OVERRIDES:
-        logging.debug(
-            '%s is specified in the parent override file.',
-            spi.version
-        )
-
-        try:
-            (
-                unapplied_changelog_parent_commit,
-                _
-            ) = override_parents(repo, spi, namespace)
-        except ParentOverrideError as e:
-            logging.error("%s" % e)
-            return
-    else:
-        if version_compare(str(spi.version), unapplied_tip_version) <= 0:
-            logging.warn(
-                 "Version to import (%s) is not after %s tip (%s)",
-                 spi.version,
-                 pretty_head_name,
-                 unapplied_tip_version,
-            )
-
-        # Walk changelog backwards until we find an imported version
-        for version in import_tree_versions:
-            unapplied_changelog_parent_tag = repo.get_import_tag(version, namespace)
-            if unapplied_changelog_parent_tag is not None:
-                # sanity check that version from d/changelog of the
-                # tagged parent matches ours
-                parent_changelog_version, _ = \
-                    repo.get_changelog_versions_from_treeish(
-                        str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id),
-                                                            )
-                # if the parent was imported via a parent override
-                # itself, assume it is valid without checking its
-                # tree's debian/changelog, as it wasn't used
-                if (version not in _PARENT_OVERRIDES and
-                    parent_changelog_version != version
-                ):
-                    logging.error(
-                        "Found a tag corresponding to parent version "
-                        "%s, but d/changelog version (%s) differs. Will "
-                        "orphan commit.",
-                        version,
-                        parent_changelog_version,
-                    )
-                else:
-                    unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id)
-                    logging.debug("Changelog parent (tag) is %s",
-                        repo.tag_to_pretty_name(unapplied_changelog_parent_tag)
-                    )
-                break
+    unapplied_changelog_parent_commit = get_unapplied_changelog_parent_commit(
+        repo,
+        spi,
+        ref_namespace,
+        import_tree_versions,
+    )
 
+    # XXX: if we are here, the spi has not been imported -- but that
+    # does not mean the the contents of the imported spi are identical
+    # (rework orphan tags)
     # check if the version to import has already been uploaded to
     # this head -- we leverage the above code to perform a 'dry-run'
     # of the import tree, to obtain the changelog parent
-    upload_tag = repo.get_upload_tag(spi.version, namespace)
-    import_tag = repo.get_import_tag(spi.version, namespace)
-    if import_tag:
-        if str(import_tag.peel().tree.id) != unapplied_import_tree_hash:
-            logging.error(
-                "Found import tag %s, but the corresponding tree "
-                "does not match the published version. Will orphan commit.",
-                repo.tag_to_pretty_name(import_tag),
+    upload_parent_commit = get_upload_parent_commit(
+        repo,
+        spi,
+        ref_namespace,
+        unapplied_import_tree_hash,
+    )
+    if upload_parent_commit and unapplied_changelog_parent_commit:
+        try:
+            repo.git_run(
+                [
+                    'merge-base',
+                    '--is-ancestor',
+                    unapplied_changelog_parent_commit,
+                    upload_parent_commit,
+                ],
+                verbose_on_failure=False,
             )
             unapplied_changelog_parent_commit = None
-        else:
-            repo.update_head_to_commit(
-                spi.head_name(namespace),
-                str(import_tag.peel().id),
-            )
-            return
-    elif upload_tag:
-        if str(upload_tag.peel().tree.id) != unapplied_import_tree_hash:
-            logging.warn(
-                "Found upload tag %s, but the corresponding tree "
-                "does not match the published version. Will import %s as "
-                "normal, ignoring the upload tag.",
-                repo.tag_to_pretty_name(upload_tag),
-                spi.version,
-            )
-        else:
-            upload_parent_commit = str(upload_tag.peel().id)
-
-            if unapplied_changelog_parent_commit is not None:
-                try:
-                    repo.git_run(
-                        [
-                            'merge-base',
-                            '--is-ancestor',
-                            unapplied_changelog_parent_commit,
-                            upload_parent_commit,
-                        ],
-                        verbose_on_failure=False,
-                    )
-                    unapplied_changelog_parent_commit = None
-                except CalledProcessError as e:
-                    if e.returncode != 1:
-                        raise
+        except CalledProcessError as e:
+            if e.returncode != 1:
+                raise
 
     commit_unapplied_patches_import(
         repo,
         spi,
         unapplied_import_tree_hash,
-        namespace,
+        ref_namespace,
         unapplied_changelog_parent_commit,
         upload_parent_commit,
     )
@@ -1227,242 +1243,300 @@ def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo):
 def import_applied_spi(
     repo,
     spi,
-    namespace,
-    ubuntu_sinfo,
-    allow_applied_failures
+    ref_namespace,
 ):
     """Imports a source package from Launchpad into the git
     repository
 
     Arguments:
-    spi - a SourcePackageInformation instance
+    repo - a gitubutu.git_repository.GitUbuntuRepository object
+    spi - a
+    gitubuntu.source_information.GitUbuntuSourcePackageInformation object
+    ref_namespace - a string Git-ref namespace this import will live in
     """
-    pretty_head_name = spi.pretty_head_name
-
-    logging.info('Importing patches-applied %s to %s' % (spi.version, pretty_head_name))
-    applied_tip_head = repo.get_head_by_name(spi.applied_head_name(namespace))
-
-    spi.pull()
+    logging.info('Importing patches-applied %s', spi.version)
 
     repo.clean_repository_state()
 
-    unapplied_parent_tag = repo.get_import_tag(spi.version, namespace)
+    unapplied_parent_tag = repo.get_import_tag(spi.version, ref_namespace)
     unapplied_parent_commit = str(unapplied_parent_tag.peel().id)
     unapplied_import_tree_hash = str(unapplied_parent_tag.peel(pygit2.Tree).id)
     logging.debug('Found patches-unapplied version %s as commit %s',
                   str(spi.version), unapplied_parent_commit)
 
     import_tree_versions = repo.get_all_changelog_versions_from_treeish(
-                               unapplied_import_tree_hash
-                                                                       )
-    changelog_version = import_tree_versions[0]
-
-    applied_tip_version = None
-    # check if the version to import has already been imported to
-    # this head
-    if applied_tip_head is not None:
-        if repo.treeishs_identical(
-                unapplied_import_tree_hash, str(applied_tip_head.peel().id)
-            ):
-            logging.warn('%s is identical to %s',
-                         pretty_head_name, spi.version
-                        )
-            return
-        applied_tip_version, _ = repo.get_changelog_versions_from_treeish(
-                             str(applied_tip_head.peel().id),
-                                                                 )
-
-    logging.debug('Tip version is %s', applied_tip_version)
-
-    applied_changelog_parent_commit = None
-
-    if spi.version in _PARENT_OVERRIDES:
-        logging.debug('%s is specified in the parent override file.' %
-                     spi.version
-                    )
-
-        (
-            _,
-            applied_changelog_parent_commit,
-        ) = override_parents(
-            repo,
-            spi,
-            namespace,
-         )
-    else:
-        if version_compare(str(spi.version), applied_tip_version) <= 0:
-            logging.warn('Version to import (%s) is not after %s tip (%s)',
-                         spi.version, pretty_head_name, applied_tip_version
-                        )
-
-        # Walk changelog backwards until we find an imported version
-        for version in import_tree_versions:
-            applied_changelog_parent_tag = repo.get_applied_tag(version, namespace)
-            if applied_changelog_parent_tag is not None:
-                # sanity check that version from d/changelog of the
-                # tagged parent matches ours
-                parent_changelog_version, _ = \
-                    repo.get_changelog_versions_from_treeish(
-                        str(applied_changelog_parent_tag.peel(pygit2.Tree).id),
-                                                            )
-                # if the parent was imported via a parent override
-                # itself, assume it is valid without checking its
-                # tree's debian/changelog, as it wasn't used
-                if (version not in _PARENT_OVERRIDES and
-                    parent_changelog_version != version
-                   ):
-                    logging.error('Found a tag corresponding to '
-                                  'parent version %s, but d/changelog '
-                                  'version (%s) differs. Will '
-                                  'orphan commit.' % (
-                                  version,
-                                  parent_changelog_version
-                                  )
-                                 )
-                else:
-                    applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id)
-                    logging.debug('Changelog parent (tag) is %s',
-                        repo.tag_to_pretty_name(applied_changelog_parent_tag)
-                                )
-                break
+        unapplied_import_tree_hash
+    )
 
-    applied_tag = repo.get_applied_tag(version, namespace)
-    if applied_tag:
-        # XXX: What should be checked here? The result of pushing all
-        # the patches? How to do it most non-destructively? (Might be
-        # able to use our new helper to get treeish after running a
-        # command, in this case, `quilt push -a`)
-        # if str(applied_tag.peel().tree.id) != applied_import_tree_hash:
-        #     logging.error(
-        #         "Found patches-applied import tag %s, but the "
-        #         "corresponding tree does not match the published "
-        #         "version. Will orphan commit.",
-        #        repo.tag_to_pretty_name(applied_tag),
-        #    )
-        #    applied_changelog_parent_commit = None
-        #else:
-        repo.update_head_to_commit(
-            spi.applied_head_name(namespace),
-            str(applied_tag.peel().id),
-        )
-        return
+    applied_changelog_parent_commit = get_applied_changelog_parent_commit(
+        repo,
+        spi,
+        ref_namespace,
+        import_tree_versions,
+    )
 
     # Assume no patches to apply
     applied_import_tree_hash = unapplied_import_tree_hash
     # get tree id from above commit
-    try:
-        for (
-           applied_import_tree_hash,
-           patch_name,
-           patch_desc
-        ) in import_patches_applied_tree(repo, spi.dsc_pathname):
-            # special case for .pc removal
-            if patch_name is None:
-                msg = b'%b.' % (patch_desc.encode())
-            else:
-                if patch_desc is None:
-                    patch_desc = (
-                        "%s\n\nNo DEP3 Subject or Description header found" %
-                         patch_name
-                    )
-                msg = b'%b\n\nGbp-Pq: %b.' % (
-                    patch_desc.encode(),
-                    patch_name.encode()
+    for (
+       applied_import_tree_hash,
+       patch_name,
+       patch_desc
+    ) in import_patches_applied_tree(repo, spi.dsc_pathname):
+        # special case for .pc removal
+        if not patch_name is None:
+            msg = b'%b.' % (patch_desc.encode())
+        else:
+            if patch_desc is None:
+                patch_desc = (
+                    "%s\n\nNo DEP3 Subject or Description header found" %
+                     patch_name
                 )
-            unapplied_parent_commit = repo.commit_tree_hash(
-                applied_import_tree_hash,
-                [unapplied_parent_commit],
-                msg,
-                spi
+            msg = b'%b\n\nGbp-Pq: %b.' % (
+                patch_desc.encode(),
+                patch_name.encode()
             )
+        unapplied_parent_commit = repo.commit_tree_hash(
+            applied_import_tree_hash,
+            [unapplied_parent_commit],
+            msg,
+            spi
+        )
 
-            logging.debug(
-                "Committed patch-application of %s as %s",
-                patch_name, unapplied_parent_commit
-            )
-    except CalledProcessError as e:
-        if allow_applied_failures:
-            return
-        raise
+        logging.debug(
+            "Committed patch-application of %s as %s",
+            patch_name, unapplied_parent_commit
+        )
 
     commit_applied_patches_import(
         repo,
         spi,
         applied_import_tree_hash,
-        namespace,
+        ref_namespace,
         applied_changelog_parent_commit,
         unapplied_parent_commit,
     )
 
-def import_publishes(
+
+def import_new_publishes(
     repo,
     pkgname,
-    namespace,
+    distname,
+    new_spi,
+    ref_namespace,
+    head_ref_namespace,
+    skip_orig,
+    allow_applied_failures,
     patches_applied,
-    debian_head_versions,
-    ubuntu_head_versions,
-    debian_sinfo,
-    ubuntu_sinfo,
+):
+    history_found = False
+    spi = None
+
+    updated_head_refs = dict()
+    try:
+        for spi in new_spi:
+            history_found = True
+            spi.pull()
+            # DSC can only be already imported if the import tag
+            # exists
+
+            if patches_applied:
+                existing_tag = repo.get_applied_tag(spi.version, ref_namespace)
+                head_ref_name = gitubuntu.importerutils.applied_head_ref_name(
+                    spi,
+                    ref_namespace,
+                )
+            else:
+                existing_tag = repo.get_import_tag(spi.version, ref_namespace)
+                head_ref_name = gitubuntu.importerutils.head_ref_name(
+                    spi,
+                    ref_namespace,
+                )
+
+            if existing_tag:
+                # if an identical DSC has already been imported,
+                # then we only need to update our branch tracker
+                if dsc_already_imported(
+                    repo,
+                    spi,
+                    ref_namespace,
+                    distname,
+                ):
+                    updated_head_refs[head_ref_name] = str(
+                        existing_tag.peel(pygit2.Commit).id
+                    )
+                    continue
+
+            try:
+                if patches_applied:
+                    import_applied_spi(
+                        repo=repo,
+                        spi=spi,
+                        ref_namespace=ref_namespace,
+                    )
+                else:
+                    import_unapplied_spi(
+                        repo=repo,
+                        spi=spi,
+                        ref_namespace=ref_namespace,
+                        skip_orig=skip_orig,
+                    )
+            except CalledProcessError:
+                if patches_applied and allow_applied_failures:
+                    continue
+                raise
+
+            if patches_applied:
+                existing_tag = repo.get_applied_tag(spi.version, ref_namespace)
+            else:
+                existing_tag = repo.get_import_tag(spi.version, ref_namespace)
+
+            updated_head_refs[head_ref_name] = str(
+                existing_tag.peel(pygit2.Commit).id
+            )
+    except Exception as e:
+        if patches_applied:
+            if not spi:
+                msg = "Unable to import patches-applied to %s" % distname
+            else:
+                msg = "Unable to import patches-applied %s to %s" % (
+                    str(spi.version),
+                    distname,
+                )
+            logging.error(msg)
+        else:
+            if not spi:
+                msg = "Unable to import patches-unapplied to %s" % distname
+            else:
+                msg = "Unable to import patches-unapplied %s to %s" % (
+                    str(spi.version),
+                    distname,
+                )
+            raise GitUbuntuImportError(msg) from e
+    else:
+        history_found = True
+
+    return history_found, updated_head_refs
+
+
+def import_publishes(
+    repo,
+    pkgname,
+    base_ref_namespace,
     active_series_only,
     workdir,
     skip_orig,
+    skip_applied,
     allow_applied_failures,
+    pullfile,
+    retries,
+    retry_backoffs,
 ):
-    history_found = False
     only_debian = False
-    srcpkg_information = None
-    if patches_applied:
-        _namespace = namespace
-        namespace = '%s/applied' % namespace
-        import_type = 'patches-applied'
-        import_func =  functools.partial(
-            import_applied_spi,
-            allow_applied_failures=allow_applied_failures,
+    history_found = False
+
+    debian_sinfo = GitUbuntuSourceInformation(
+        'debian',
+        pkgname,
+        os.path.abspath(pullfile),
+        retries,
+        retry_backoffs,
+    )
+
+    ubuntu_sinfo = GitUbuntuSourceInformation(
+        'ubuntu',
+        pkgname,
+        os.path.abspath(pullfile),
+        retries,
+        retry_backoffs,
+    )
+
+    for distname, dist_sinfo in (
+        ('debian', debian_sinfo),
+        ('ubuntu', ubuntu_sinfo),
+    ):
+        unapplied_head_ref_namespace = '%s/%s' % (base_ref_namespace, distname)
+        applied_head_ref_namespace='%s/applied/%s' % (base_ref_namespace, distname)
+
+        unapplied_versions = repo.get_heads_and_versions(
+            head_ref_namespace=unapplied_head_ref_namespace,
         )
-    else:
-        _namespace = namespace
-        import_type = 'patches-unapplied'
-        import_func =  functools.partial(
-            import_unapplied_spi,
-            skip_orig=skip_orig,
+        applied_versions = repo.get_heads_and_versions(
+            head_ref_namespace=applied_head_ref_namespace
         )
-    for distname, versions, dist_sinfo in (
-            ("debian", debian_head_versions, debian_sinfo),
-            ("ubuntu", ubuntu_head_versions, ubuntu_sinfo)):
+        debug_log_head_versions(unapplied_versions)
+        debug_log_head_versions(applied_versions)
         if active_series_only and distname == "debian":
             continue
+
+        try:
+            history_found, updated_head_refs = import_new_publishes(
+                repo,
+                pkgname,
+                distname,
+                dist_sinfo.launchpad_versions_published_after(
+                    unapplied_versions,
+                    unapplied_head_ref_namespace,
+                    workdir,
+                    active_series_only,
+                ),
+                base_ref_namespace,
+                unapplied_head_ref_namespace,
+                skip_orig,
+                allow_applied_failures=None,
+                patches_applied=False,
+            ) 
+            update_branches(
+                distname,
+                dist_sinfo,
+                repo,
+                unapplied_head_ref_namespace,
+                pkgname,
+                updated_head_refs,
+            )
+        except NoPublicationHistoryException:
+            logging.warning("No publication history found for %s in %s.",
+                pkgname, distname
+            )
+            if distname == 'ubuntu':
+                only_debian = True
+            continue
+
+        if skip_applied:
+            continue
+
         try:
-            for srcpkg_information in dist_sinfo.launchpad_versions_published_after(
-                versions,
-                namespace,
-                workdir=workdir,
-                active_series_only=active_series_only
-            ):
-                history_found = True
-                import_func(
-                    repo=repo,
-                    spi=srcpkg_information,
-                    namespace=_namespace,
-                    ubuntu_sinfo=ubuntu_sinfo,
-                )
+            _, updated_head_refs = import_new_publishes(
+                repo,
+                pkgname,
+                distname,
+                dist_sinfo.launchpad_versions_published_after(
+                    applied_versions,
+                    applied_head_ref_namespace,
+                    workdir,
+                    active_series_only,
+                ),
+                base_ref_namespace,
+                applied_head_ref_namespace,
+                skip_orig=None,
+                allow_applied_failures=allow_applied_failures,
+                patches_applied=True,
+            )
+            update_branches(
+                distname,
+                dist_sinfo,
+                repo,
+                applied_head_ref_namespace,
+                pkgname,
+                updated_head_refs,
+            )
         except NoPublicationHistoryException:
             logging.warning("No publication history found for %s in %s.",
                 pkgname, distname
             )
             if distname == 'ubuntu':
                 only_debian = True
-        except Exception as e:
-            if srcpkg_information is None:
-                msg = 'Unable to import %s to %s' % (import_type, distname)
-            else:
-                msg = 'Unable to import %s %s to %s' % (import_type,
-                     str(srcpkg_information.version), distname)
-            if not patches_applied:
-                raise GitUbuntuImportError(msg) from e
-            else:
-                logging.error(msg)
-        else:
-            history_found = True
+            continue
 
     return (only_debian, history_found)
 
diff --git a/gitubuntu/importerutils.py b/gitubuntu/importerutils.py
 new file mode 100644
index 0000000..6035c45
--- /dev/null
+++ b/gitubuntu/importerutils.py
@@ -0,0 +1,26 @@
+def head_ref_name(spi, ref_namespace):
+    """head_ref_name - get the patches-unapplied Git ref name a spi would import to
+
+    spi - a gitubuntu.source_information.GitUbuntuSourcePackageInformation object
+    ref_namespace - string namespace the resulting Git ref should be in
+    """
+    if spi.pocket.lower() == 'release':
+        pocket_suffix = ''
+    else:
+        pocket_suffix = '-' + spi.pocket.lower()
+
+    return '%s/%s/%s%s' % (
+        ref_namespace,
+        spi.distribution.name.lower(),
+        spi.series.name.lower(),
+        pocket_suffix,
+    )
+
+def applied_head_ref_name(spi, ref_namespace):
+    """applied_head_ref_name - get the patches-applied Git ref name a spi would import to
+
+    Arguments:
+    spi - a gitubuntu.source_information.GitUbuntuSourcePackageInformation object
+    ref_namespace - string namespace the resulting Git ref should be in
+    """
+    return head_ref_name(spi, '%s/applied' % ref_namespace)
diff --git a/gitubuntu/importppa.py b/gitubuntu/importppa.py
index 1f3283c..3bfde5e 100644
--- a/gitubuntu/importppa.py
+++ b/gitubuntu/importppa.py
@@ -83,9 +83,8 @@ def main(
         namespace,
     )
     for head_name in sorted(ubuntu_head_versions):
-        _, _, pretty_head_name = head_name.partition('%s/', namespace)
         logging.debug('Last imported %s version is %s',
-            pretty_head_name,
+            head_name,
             ubuntu_head_versions[head_name]['version']
         )
 
@@ -94,9 +93,8 @@ def main(
         namespace,
     )
     for head_name in sorted(applied_ubuntu_head_versions):
-        _, _, pretty_head_name = head_name.partition('%s/', namespace)
         logging.debug('Last applied %s version is %s',
-            pretty_head_name,
+            head_name,
             applied_ubuntu_head_versions[head_name]['version']
         )
 
diff --git a/gitubuntu/source_information.py b/gitubuntu/source_information.py
index b97bc8a..527e2f0 100644
--- a/gitubuntu/source_information.py
+++ b/gitubuntu/source_information.py
@@ -21,6 +21,8 @@ except ImportError:
     logging.error('Is %s installed?', pkg)
     sys.exit(1)
 
+import gitubuntu.importerutils
+
 _ddi = DebianDistroInfo()
 _udi = UbuntuDistroInfo()
 
@@ -131,8 +133,18 @@ class GitUbuntuPPASourcePackage(UbuntuSourcePackage):
 
 
 class GitUbuntuSourcePackageInformation:
-    def __init__(self, spphr, dist_name, retries=0, retry_backoffs=[],
-                 workdir=None, dsc=None, files=list()):
+    def __init__(
+        self,
+        spphr,
+        dist_name,
+        retries=0,
+        retry_backoffs=[],
+        workdir=None,
+        dsc=None,
+        files=None,
+    ):
+        if not files:
+            files = list()
         self._spphr = spphr
         self._dist_name = dist_name
         self._pkgname = self.spphr.source_package_name
@@ -151,13 +163,15 @@ class GitUbuntuSourcePackageInformation:
 
         # do this here, in case files is passed
         if workdir and not os.path.isdir(workdir):
-             os.makedirs(workdir, exist_ok=True)
-
-        self._archive_srcpkg = func(package=self._pkgname,
-                                    version=self._version,
-                                    workdir=workdir,
-                                    quiet=True,
-                                    dscfile=dsc)
+            os.makedirs(workdir, exist_ok=True)
+
+        self._archive_srcpkg = func(
+            package=self._pkgname,
+            version=self._version,
+            workdir=workdir,
+            quiet=True,
+            dscfile=dsc,
+        )
         for f in files:
             self._archive_srcpkg._download_file(f, f.split('/')[-1])
 
@@ -202,62 +216,6 @@ class GitUbuntuSourcePackageInformation:
         return self._spphr.pocket
 
     @property
-    def pretty_head_name(self):
-        if self._spphr.pocket.lower() == 'release':
-            head_name = '%s/%s' % (
-            self.distribution.name.lower(),
-            self.series.name.lower(),
-            )
-        else:
-            head_name = '%s/%s-%s' % (
-            self.distribution.name.lower(),
-            self.series.name.lower(),
-            self.pocket.lower()
-            )
-        return head_name
-
-    def head_name(self, prefix):
-        if self._spphr.pocket.lower() == 'release':
-            head_name = '%s/%s/%s' % (
-            prefix,
-            self.distribution.name.lower(),
-            self.series.name.lower(),
-            )
-        else:
-            head_name = '%s/%s/%s-%s' % (
-            prefix,
-            self.distribution.name.lower(),
-            self.series.name.lower(),
-            self.pocket.lower()
-            )
-        return head_name
-
-    def applied_head_name(self, prefix):
-       return self.head_name('%s/applied' % prefix)
-
-    def parent_head_name(self, prefix):
-        if self.parent_series == None:
-            return None
-        if self.pocket.lower() == 'release':
-            # release pockets descend from prior series
-            head_name = '%s/%s/%s' % (
-            prefix,
-            self.distribution.name.lower(),
-            self.parent_series.name.lower()
-            )
-        else:
-            # non-release pockets descend from release
-            head_name = '%s/%s/%s' % (
-            prefix,
-            self.distribution.name.lower(),
-            self.series.name.lower()
-            )
-        return head_name
-
-    def parent_applied_head_name(self, prefix):
-        return self.parent_head_name('%s/applied' % prefix)
-
-    @property
     def dsc(self):
         return self._archive_srcpkg.dsc
 
@@ -295,11 +253,14 @@ class GitUbuntuSourceInformation(object):
     _stable_series_list = None
     _current_series = None
 
-    def __init__(self, dist_name, pkgname=None,
-                 pull_overrides_filename='/dev/null',
-                 retries=0,
-                 retry_backoffs=[]
-                ):
+    def __init__(
+        self,
+        dist_name,
+        pkgname=None,
+        pull_overrides_filename='/dev/null',
+        retries=0,
+        retry_backoffs=[]
+    ):
         self.launchpad = launchpad_login()
         self.dist_name = dist_name
         if self.dist_name.startswith('ppa:'):
@@ -324,18 +285,6 @@ class GitUbuntuSourceInformation(object):
         self.retries = retries
         self.retry_backoffs = retry_backoffs
 
-    @staticmethod
-    def _head_version_is_equal(head_versions, namespace, spi):
-        try:
-            if (head_versions[spi.head_name(namespace)]['version'] == spi.version and
-                (spi.date_published is None or
-                 int(spi.date_published.timestamp()) == head_versions[spi.head_name(namespace)]['head'].peel().commit_time)
-                ):
-                 return True
-            return False
-        except KeyError:
-            return False
-
     @property
     def current_series(self):
         if self.dist_name.startswith('ppa:'):
@@ -402,7 +351,7 @@ class GitUbuntuSourceInformation(object):
     def all_series_name_list(self):
         return [r.name for r in self.all_series]
 
-    def get_corrected_spi(self, srcpkg, workdir=None):
+    def get_corrected_spi(self, srcpkg, workdir):
        try:
            pull_override = self.pull_overrides[srcpkg.source_package_version]
            dsc = pull_override['dsc']
@@ -410,11 +359,17 @@ class GitUbuntuSourceInformation(object):
        except KeyError:
            dsc = None
            files = list()
-       return GitUbuntuSourcePackageInformation(srcpkg, self.dist_name,
-           self.retries, self.retry_backoffs, workdir=workdir,
-           dsc=dsc, files=files)
-
-    def launchpad_version_is_published(self, version, workdir=None):
+       return GitUbuntuSourcePackageInformation(
+           srcpkg,
+           self.dist_name,
+           self.retries,
+           self.retry_backoffs,
+           workdir=workdir,
+           dsc=dsc,
+           files=files,
+       )
+
+    def launchpad_version_is_published(self, version):
         spph = self.archive.getPublishedSources(
             exact_match=True,
             source_name=self.pkgname,
@@ -423,13 +378,16 @@ class GitUbuntuSourceInformation(object):
         )
         return len(spph) != 0
 
-    def launchpad_versions_published(self, workdir=None,
-        sorted_by_version=False, series=None
+    def launchpad_versions_published(
+        self,
+        workdir,
+        sorted_by_version=False,
+        series=None,
     ):
         args = {
-                'exact_match':True,
-                'source_name':self.pkgname,
-               }
+            'exact_match':True,
+            'source_name':self.pkgname,
+        }
         if not sorted_by_version:
             args['order_by_date'] = True
         if series:
@@ -437,18 +395,49 @@ class GitUbuntuSourceInformation(object):
 
         spph = self.archive.getPublishedSources(**args)
         if len(spph) == 0:
-            raise NoPublicationHistoryException("Is %s published in %s?" %
-                                                (self.pkgname, self.dist_name))
+            raise NoPublicationHistoryException(
+                "Is %s published in %s?" % (
+                    self.pkgname,
+                    self.dist_name,
+                )
+            )
 
         for srcpkg in spph:
             yield self.get_corrected_spi(srcpkg, workdir)
 
-    def launchpad_versions_published_after(self, head_versions, namespace, workdir=None, active_series_only=False):
+    def launchpad_versions_published_after(
+        self,
+        head_versions,
+        head_ref_namespace,
+        workdir,
+        active_series_only=False,
+    ):
+        """Provide Launchpad publishing information after a certain publish
+
+        Arguments:
+        head_versions - a dictionary of dictionaries, keyed by Git
+        branch names, where each element is a dictionary with elements
+        'version' and 'head'.
+        head_ref_namespace - a string Git ref namespace corresponding to
+        the branches in @head_versions
+        workdir - a string filesystem path to store the downloaded
+        source package in
+        active_series_only - a boolean indicating only publishing
+        records from active series should be considered
+
+        @head_versions[<branch>]['head'] is a pygit2.Branch object.
+        @head_versions[<branch>]['version'] is the string version of the
+        debian/changelog in the corresponnding Git branch.
+
+        Returns:
+        an iterable over GitUbuntuSourcePackageInformation objects, one
+        per Launchpad publishing record after @head_version
+        """
         args = {
-                'exact_match':True,
-                'source_name':self.pkgname,
-                'order_by_date':True,
-               }
+            'exact_match':True,
+            'source_name':self.pkgname,
+            'order_by_date':True,
+        }
 
         # we have the date of the commit too, so we can double-check
         # that it matches
@@ -469,14 +458,33 @@ class GitUbuntuSourceInformation(object):
         # Sanity check that the passed in srcpkg name has a publication
         # history
         if len(spph) == 0:
-            raise NoPublicationHistoryException("Is %s published in %s?" %
-                                                (self.pkgname, self.dist_name))
-        if len(head_versions) > 0:
+            raise NoPublicationHistoryException(
+                "Is %s published in %s?" % (
+                    self.pkgname,
+                    self.dist_name,
+                )
+            )
+        if head_versions:
             _spph = list()
             for spphr in spph:
-                spi = GitUbuntuSourcePackageInformation(spphr, self.dist_name,
-                                                        workdir=workdir)
-                if self._head_version_is_equal(head_versions, namespace, spi):
+                spi = GitUbuntuSourcePackageInformation(
+                    spphr,
+                    self.dist_name,
+                    workdir=workdir,
+                )
+                head_version = head_versions[
+                    gitubuntu.importerutils.head_ref_name(
+                        spi,
+                        head_ref_namespace,
+                    )
+                ]
+                if (
+                    head_version['version'] == spi.version and
+                    spi.date_published and
+                    int(spi.date_published.timestamp()) == head_version[
+                        'head'
+                    ].peel().commit_time
+                ):
                     break
                 _spph.append(spphr)
             spph = _spph