Merge ~nacc/git-ubuntu:bug-fixes-3 into git-ubuntu:master

[Nish Aravamudan]

Make jenkins happy.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:5d4441a4f987fe7242513d1430601224717d2b46
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/239/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Style Check
    SUCCESS: Unit Tests
    SUCCESS: Integration Tests
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/239/rebuild

review: Approve (continuous-integration)

[Nish Aravamudan (nacc)]

I plan on rebasing this and cleaning up the second commit so it's clearer. I'll also merge in the other approved fixes from a few of the other MPs.

[Andreas Hasenack (ahasenack)]

5d4441a4f987fe7242513d1430601224717d2b46 apt change: +1

[Andreas Hasenack (ahasenack)]

Comments inline.

I see some duplication of code, which was already there, but is unfortunate. Something for a follow-up branch?

review: Needs Information

[Andreas Hasenack (ahasenack)]

Could you also please assign the bugs to yourself and mark them as "in progress"? Just the lxd one is so.

[Andreas Hasenack (ahasenack)]

One more question inline.

review: Needs Information

[Nish Aravamudan (nacc)]

@Robie: the last two commits I need some help on. We talked in HO about HEAD^, but I forget what we decided as to the cleanup. For HEAD, I'm not sure how to make pristine-tar and parent-dir cooperate, when the former needs to know that we are in a Git repository and what content we need from it.

[Nish Aravamudan (nacc)]

On 10.01.2018 [12:26:35 -0000], Andreas Hasenack wrote:
> Review: Needs Information
>
> Comments inline.
>
> I see some duplication of code, which was already there, but is
> unfortunate. Something for a follow-up branch?

Absolutely. Patches welcome, as well :)

> Diff comments:
>
> > diff --git a/gitubuntu/build.py b/gitubuntu/build.py
> > index 072d3e7..06b491f 100644
> > --- a/gitubuntu/build.py
> > +++ b/gitubuntu/build.py
> > @@ -878,16 +873,32 @@ def do_build_host_exitstack(
> > old_pardir_contents = set(os.listdir(os.pardir))
> > run(['dpkg-buildpackage'] + list(rem_args))
> > new_pardir_contents = set(os.listdir(os.pardir))
>
> Do we still need to take this snapshot of the pardir before and after
> the build? See below my other comment when we parse the changes file.

Hrm, you're right, probably not. We can just look that all the contents
of the changes file are present. I believe here I was trying to avoid
copying the orig tarball twice.

> > - temp_pardir_contents = new_pardir_contents - old_pardir_contents
> > - built_pardir_contents = []
> > - for f in temp_pardir_contents:
> > - shutil.copy(
> > - os.path.join(os.pardir, f),
> > - os.path.join(oldcwd, os.pardir)
> > - )
> > - built_pardir_contents.append(os.path.join(os.pardir, f))
> >
> > - return built_pardir_contents
> > + changes_file = None
> > + built_pardir_contents = []
> > + changes_files = [f for f in new_pardir_contents if 'changes' in f]
>
> Suggestion to be more specific about the file we want:
> changes_files = [f for f in new_pardir_contents if f.endswith('.changes')]

Yes, probably worth doing.

> > + if len(changes_files) == 0:
> > + logging.error("No changes file")
> > + elif len(changes_files) > 1:
> > + logging.error("Multiple changes files?")
> > + else:
> > + changes_file = changes_files.pop()
> > + # Intersect changes file contents with the new files in /tmp
> > + # to determine what was newly built
> > + with open(os.path.join(os.pardir, changes_file), 'rb') as changes_fobj:
>
> Do we still need this intersection? Won't the changes file have an
> exact list of what we need?

Good point.

>
> > + changes = Changes(changes_fobj)
> > + changes_named_files = [f['name'] for f in changes['files']]
> > + for new_file in new_pardir_contents:
> > + if new_file in changes_named_files:
> > + shutil.copy(
> > + os.path.join(os.pardir, new_file),
> > + os.path.join(oldcwd, os.pardir),
> > + )
> > + built_pardir_contents.append(
> > + os.path.join(os.pardir, new_file)
> > + )
> > +
> > + return [changes_file,] + built_pardir_contents
> >
> > def get_cmd_in_origpath(cmd):
> > return shutil.which(
> > @@ -1141,7 +1164,32 @@ def do_build_lxd_exitstack(
> > if filename_stripped
> > )
> >
> > - built_temp_contents = new_temp_contents - orig_temp_contents
> > + changes_file = None
> > + built_temp_contents = []
> ...

[Nish Aravamudan (nacc)]

On 10.01.2018 [12:41:41 -0000], Andreas Hasenack wrote:
> Could you also please assign the bugs to yourself and mark them as "in
> progress"? Just the lxd one is so.

I believe they all are now from the new branch.

[Nish Aravamudan (nacc)]

On 10.01.2018 [12:52:57 -0000], Andreas Hasenack wrote:
> Review: Needs Information
>
> One more question inline.
>
> Diff comments:
>
> > diff --git a/gitubuntu/build.py b/gitubuntu/build.py
> > index 072d3e7..06b491f 100644
> > --- a/gitubuntu/build.py
> > +++ b/gitubuntu/build.py
> > @@ -1026,17 +1037,29 @@ def do_build_lxd_exitstack(
> > time.sleep(retry_backoffs[i])
> > else:
> > raise RuntimeError(
> > - "Failed to run apt in ephemeral build container"
> > + "Failed to run apt-get in ephemeral build container"
> > )
> >
> > + # lxd assumes that if SNAP is set in the environment, that it is
> > + # running as a snap. That is not necessarily true when git-ubuntu is
> > + # the snapped application. So unset SNAP before we exec.
> > + # LP: #1741949
>
> Does this happen only with lxc push? There are other lxc commands we
> run elsewhere, like lxc pull, and lxc exec.

Only lxc push/pull care about the hostfs. Both have been updated to use
this env in this patch.

>
> > + env_unset_SNAP = os.environ.copy()
> > try:
> > - run([
> > - lxc,
> > - 'file',
> > - 'push',
> > - archive_tarball_name,
> > - '%s/tmp/' % container_name,
> > - ])
> > + del env_unset_SNAP['SNAP']
> > + except KeyError:
> > + pass
> > + try:
> > + run(
> > + [
> > + lxc,
> > + 'file',
> > + 'push',
> > + archive_tarball_name,
> > + '%s/tmp/' % container_name,
> > + ],
> > + env=env_unset_SNAP,
> > + )
> > except Exception as e:
> > raise RuntimeError(
> > "Failed to push archive tarball to ephemeral build container"

--
Nishanth Aravamudan
Ubuntu Server
Canonical Ltd

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:66abeeca9e8fee25e4988c3232c433eabf0bb6e1
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/242/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Style Check
    SUCCESS: Unit Tests
    FAILED: Integration Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/242/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:cd23a85ab442fff9cb667dc0a5aaa915eb478f29
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/243/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Style Check
    SUCCESS: Unit Tests
    FAILED: Integration Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/243/rebuild

review: Needs Fixing (continuous-integration)

[Nish Aravamudan (nacc)]

Clearly still working on this. Feel free to not review until it passes CI.

On Jan 11, 2018 18:23, "Server Team CI bot" <
<email address hidden>> wrote:

> Review: Needs Fixing continuous-integration
>
> FAILED: Continuous integration, rev:cd23a85ab442fff9cb667dc0a5aaa9
> 15eb478f29
> https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/243/
> Executed test runs:
> SUCCESS: Checkout
> SUCCESS: Style Check
> SUCCESS: Unit Tests
> FAILED: Integration Tests
>
> Click here to trigger a rebuild:
> https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/243/rebuild
>
> --
> https://code.launchpad.net/~nacc/usd-importer/+git/usd-
> importer/+merge/336017
> You are the owner of ~nacc/usd-importer:bug-fixes-3.
>

[Andreas Hasenack (ahasenack)]

I haven't started a re-review of this yet, but I wanted to use it to avoid that bug where lxc pull fails because it's pulling in an unrelated directory, but then I got this new backtrace

I'm not sure yet what happened:

01/12/2018 11:15:17 - DEBUG:Executing: /usr/bin/lxc exec right-lab -- sudo -s -H -u ubuntu ls /tmp
01/12/2018 11:15:17 - DEBUG:Executing: /usr/bin/lxc file pull right-lab/tmp/collectd_5.7.2-2ubuntu1~ppa1_source.changes ..
01/12/2018 11:15:17 - DEBUG:Executing: /usr/bin/lxc file pull right-lab/tmp/collectd_5.7.2-2ubuntu1~ppa1.debian.tar.xz right-lab/tmp/collectd_5.7.2-2ubuntu1~ppa1.dsc right-lab/tmp/collectd_5.7.2-2ubuntu1~ppa1_source.buildinfo ..
01/12/2018 11:15:17 - DEBUG:Executing: git commit-tree --no-gpg-sign 5615196cde7013454b7a99ed35d8bcb630f858bf -p HEAD -F /tmp/tmpsyorh6sx
01/12/2018 11:15:17 - INFO:We automatically generated fixup changes relative to
HEAD as commit 4aead8c54f2875c666e7817ce1a15d207e7ddd0a.
If you would like to create a branch at that commit, run:
 git checkout -b <branch name> 4aead8c54f2875c666e7817ce1a15d207e7ddd0a
01/12/2018 11:15:17 - DEBUG:Executing: /usr/bin/lxc stop --force right-lab
01/12/2018 11:15:19 - DEBUG:Executing: git worktree prune
Traceback (most recent call last):
  File "/home/andreas/git/projects/usd-importer/bin/git-ubuntu", line 27, in <module>
    main()
  File "/home/andreas/git/projects/usd-importer/gitubuntu/__main__.py", line 243, in main
    sys.exit(args.func(args))
  File "/home/andreas/git/projects/usd-importer/gitubuntu/buildsource.py", line 103, in cli_main
    default_rem_args=default_rem_args,
  File "/home/andreas/git/projects/usd-importer/gitubuntu/build.py", line 698, in cli_main
    retry_backoffs=args.retry_backoffs,
  File "/home/andreas/git/projects/usd-importer/gitubuntu/build.py", line 490, in main
    retry_backoffs,
  File "/home/andreas/git/projects/usd-importer/gitubuntu/build.py", line 1330, in fetch_orig_and_build
    shutil.copy(f, os.path.join(oldcwd, os.pardir))
  File "/usr/lib/python3.6/shutil.py", line 241, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.6/shutil.py", line 120, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: 'collectd_5.7.2-2ubuntu1~ppa1_source.changes'

The files referenced in the lxc pull command are not in ".."

[Andreas Hasenack (ahasenack)]

Ah, I know, I committed changes to configure.ac instead of making a quilt patch.

[Nish Aravamudan (nacc)]

The backtrack is what is failing CI right now. Will debug today.

On Jan 12, 2018 05:22, "Andreas Hasenack" <email address hidden> wrote:

> Ah, I know, I committed changes to configure.ac instead of making a quilt
> patch.
> --
> https://code.launchpad.net/~nacc/usd-importer/+git/usd-
> importer/+merge/336017
> You are the owner of ~nacc/usd-importer:bug-fixes-3.
>

[Nish Aravamudan (nacc)]

On Fri, Jan 12, 2018 at 7:48 AM, Nish Aravamudan
<email address hidden> wrote:
> The backtrack is what is failing CI right now. Will debug today.

*backtrace and I believe the latest branch, force updated is good to
review/working.

-Nish

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:70a249980cdcc43ae8787bba9c29c8ef32cf249f
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/244/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Style Check
    SUCCESS: Unit Tests
    SUCCESS: Integration Tests
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/244/rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:db0ddf8ed22fcf0f0a736776111fd9dba5af12ec
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/245/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Style Check
    SUCCESS: Unit Tests
    SUCCESS: Integration Tests
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/245/rebuild

review: Approve (continuous-integration)

[Andreas Hasenack (ahasenack)]

First pass: some comments inline copied over from the previous MP

[Andreas Hasenack (ahasenack)]

Missed one

[Nish Aravamudan (nacc)]

Thanks Andreas. I will update the comments for the lxc calls (I responded
on the other MP) and just forgot to take your changes suggestion. Will
update the MP tomorrow.

On Jan 15, 2018 05:42, "Andreas Hasenack" <email address hidden> wrote:

> Missed one
>
> Diff comments:
>
> > diff --git a/gitubuntu/build.py b/gitubuntu/build.py
> > index 072d3e7..adece8e 100644
> > --- a/gitubuntu/build.py
> > +++ b/gitubuntu/build.py
> > @@ -1132,16 +1150,40 @@ def do_build_lxd_exitstack(
> > ['ls', '/tmp',],
> > user='ubuntu',
> > )
> > - new_temp_contents = set(
> > + changes_files = set(
> > filename_stripped
> > for filename_stripped in (
> > filename.strip() for filename in
> > stdout.splitlines()
> > )
> > - if filename_stripped
> > + if filename_stripped and '.changes' in filename_stripped
>
> Suggestion to be more specific:
> if filename_stripped and filename_stripped.endswith('.changes')
>
> > )
> >
> > - built_temp_contents = new_temp_contents - orig_temp_contents
> > + changes_file = None
> > + built_temp_contents = []
> > +
> > + if len(changes_files) == 0:
> > + logging.error("No changes file")
> > + elif len(changes_files) > 1:
> > + logging.error("Multiple changes files?")
> > + else:
> > + changes_file = changes_files.pop()
> > + cmd = [
> > + lxc,
> > + 'file',
> > + 'pull',
> > + '%s%s' % (container_name, os.path.join('/tmp',
> changes_file)),
> > + os.pardir,
> > + ]
> > + run(cmd, env=env_unset_SNAP)
> > + changes_file = os.path.abspath(
> > + os.path.join(os.pardir, changes_file)
> > + )
> > + with open(changes_file, 'rb') as changes_fobj:
> > + changes = Changes(changes_fobj)
> > + for f in changes['files']:
> > + built_temp_contents.append(f['name'])
> > +
> > cmd = [lxc, 'file', 'pull']
> > cmd.extend([
> > '%s%s' % (container_name, os.path.join('/tmp', f))
>
>
> --
> https://code.launchpad.net/~nacc/usd-importer/+git/usd-
> importer/+merge/336017
> You are the owner of ~nacc/usd-importer:bug-fixes-3.
>

[Robie Basak (racb)]

I tried to break 337e8bd into its own MP but it conflicts against master. This branch seems to be based on snap/beta instead of master.

review: Needs Fixing

[Nish Aravamudan (nacc)]

Hrm? Ah yes, i guess some packages were added to master but not to snap/beta.

However, I don't see how that commit could conflict, the only diff
between my local master (which was at upstream/snap/beta) and
upstream/master are in gitubuntu/source-package-whitelist.txt

On Thu, Jan 18, 2018 at 6:09 AM, Robie Basak <email address hidden> wrote:
> Review: Needs Fixing
>
> I tried to break 337e8bd into its own MP but it conflicts against master. This branch seems to be based on snap/beta instead of master.
> --
> https://code.launchpad.net/~nacc/usd-importer/+git/usd-importer/+merge/336017
> You are the owner of ~nacc/usd-importer:bug-fixes-3.

Unmerged commits

db0ddf8... by Nish Aravamudan on 2018-01-12

git_repository: modify use of TemporaryDirectory to avoid crosstalk

Currently, no matter where the cwd is, all temporary directories are
placed in /tmp. This means when we use the pardir of the temporary
directory, we end up using /tmp and that leads to crosstalk between code
paths and binaries.

We really want each temporary directory to be an isolated env, so use
nested TemporaryDirectories.

LP: #1734137.

46280ff... by Nish Aravamudan on 2018-01-06

build.fetch_orig: modify API to take repo and commit_hash

There is a false assumption in the fetch_orig() API that assumed
gbp-buildpackage would somehow leverage the changelog object we have as
a parameter. It does not, and additionally, it cannot -- the
gbp-buildpackage subcommand only uses data from HEAD currently. So we
need to:
  a) pass the repo and commit_hash to fetch_orig; and
  b) use a temporary worktree in fetch_orig*() to switch to the
commit_hash in repo so that the remaining commands dtrt.

b) does mean we are using a different GIT_DIR and WORK_TREE, so there
will be uncommitted changes relative to the GIT_DIR. Pass
--git-ignore-new to gbp to avoid this issue (we aren't actually using
gbp-buildpackage to build, and should switch to gbp-export-orig ASAP).

However, using b) breaks fetch_orig_from_parent_dir which does not know
about the Git repository at all. I am not sure how to resolve this yet,
and it's technically a corner case where an experienced developer wants
to provide us orig tarballs.

LP: #1738957

9092a15... by Nish Aravamudan on 2018-01-12

build: do not intersect changes file and built files

Just use the changes file contents to determine what was built.

d4507f9... by Nish Aravamudan on 2018-01-03

scripts/scriptutils: refactor srcpkgs_to_import_list to reduce code duplication

We iterate all of Ubuntu main, then all of Ubuntu universe, then Debian,
treating Debian source packages not present in Ubuntu as if they were in
universe.

I would prefer if the Ubuntu iterations were using one
components=['main', 'universe'], but then the caller would not know in
which component the given source package lives. In theory, we can use
src['directory'], but that seems sort of fragile to parse.

07d8b26... by Nish Aravamudan on 2018-01-11

git_repository.temporary_worktree: always prune worktrees

If an Exception was raised (e.g., ^C'ing git-ubuntu), the worktree would
not be pruned without this change.

53e7bd1... by Nish Aravamudan on 2018-01-11

git_repository.temporary_worktree: adjust GIT_WORK_TREE variable

gbp is throwing rather odd errors claiming the path for the temporary
work tree is not a Git repository. This appears to be due to having a
GIT_WORK_TREE environment variable and using a git-worktree that is
somewhere else. Properly set the environment variable in our context.

399ca4b... by Nish Aravamudan on 2018-01-11

scriptutils.pool_map_import_srcpkg: properly handle num_workers=0

Do not create a pool, just run map().

0a657cd... by Nish Aravamudan on 2018-01-11

git-ubuntu: properly handle symlinked binaries

7581ac8... by Nish Aravamudan on 2018-01-05

build: orig tarballs do not exist for native source packages

Do not try to push orig tarballs if they do not exist. Found while
investigating LP #1741472.

e7fa818... by Nish Aravamudan on 2018-01-05

dsc: return a list from orig_tarball_paths

The current Dsc class incorrectly handles native package orig tarballs
and returns None for the orig_tarball_path. It really needs to return
'nothing', as in an empty list (but the current API is a single return
value, not a list). When there is an orig tarball, there will be one
member in the list.

LP: #1741472

---

Note that I would like to use the dsc_builder code to test this, but
it's not yet merged.

diff --git a/bin/git-ubuntu b/bin/git-ubuntu
index 2b15471..48549fd 100755
--- a/bin/git-ubuntu
+++ b/bin/git-ubuntu
@@ -6,7 +6,12 @@ import os
 import sys
 # we know that our relative path doesn't include the module
 try:
-    realpath = os.readlink(__file__)
+    # If __file__ is a symlink, resolve to where it points to. If the symlink
+    # target is relative, then we need to join that to the directory __file__
+    # is inside in order to resolve it correctly. If the symlink target is
+    # absolute, then os.path.join is defined to ignore the first parameter
+    # here.
+    realpath = os.path.join(os.path.dirname(__file__), os.readlink(__file__))
 except OSError:
     realpath = __file__
 sys.path.insert(
diff --git a/gitubuntu/build.py b/gitubuntu/build.py
index 072d3e7..adece8e 100644
--- a/gitubuntu/build.py
+++ b/gitubuntu/build.py
@@ -45,6 +45,7 @@ from gitubuntu.source_information import (
 try:
     pkg = 'python3-debian'
     from debian.debfile import PART_EXTS
+    from debian.deb822 import Changes
     pkg = 'python3-petname'
     import petname
     pkg = 'python3-pygit2'
@@ -359,25 +360,27 @@ def fetch_orig_from_launchpad(changelog, source, pullfile, retries,
             spi.pull()
             dsc = GitUbuntuDsc(spi.dsc_pathname)
             orig_paths = dsc.all_tarball_paths
-            if dl_cache is None:
-                _symlink_paths_into_parent_dir(orig_paths)
-            else:
-                srcpkg_cache_dir = os.path.join(
-                    dl_cache,
-                    derive_source_from_changelog(changelog),
-                    changelog.srcpkg,
-                    changelog.upstream_version,
-                )
-                if not os.path.isdir(srcpkg_cache_dir):
-                    os.makedirs(srcpkg_cache_dir, exist_ok=True)
-                # populate cache
-                logging.debug("Caching dsc file")
-                _populate_cache(srcpkg_cache_dir, dsc.dsc_path, 'DSC')
 
-                cached_paths  = []
-                for path in orig_paths:
-                    cached_paths.append(_populate_cache(srcpkg_cache_dir, path))
-                _symlink_paths_into_parent_dir(cached_paths)
+            if orig_paths:
+                if dl_cache is None:
+                    _symlink_paths_into_parent_dir(orig_paths)
+                else:
+                    srcpkg_cache_dir = os.path.join(
+                        dl_cache,
+                        derive_source_from_changelog(changelog),
+                        changelog.srcpkg,
+                        changelog.upstream_version,
+                    )
+                    if not os.path.isdir(srcpkg_cache_dir):
+                        os.makedirs(srcpkg_cache_dir, exist_ok=True)
+                    # populate cache
+                    logging.debug("Caching dsc file")
+                    _populate_cache(srcpkg_cache_dir, dsc.dsc_path, 'DSC')
+
+                    cached_paths  = []
+                    for path in orig_paths:
+                        cached_paths.append(_populate_cache(srcpkg_cache_dir, path))
+                    _symlink_paths_into_parent_dir(cached_paths)
 
             return orig_paths
 
@@ -474,7 +477,7 @@ def main(
                     oldubuntu_version,
                 )
 
-        files = fetch_orig_and_build(
+        built_files = fetch_orig_and_build(
             repo,
             commitish,
             search_list,
@@ -486,14 +489,8 @@ def main(
             retries,
             retry_backoffs,
         )
-        if len(files) and sign:
-            changes_files = [f for f in files if 'changes' in f]
-            if len(changes_files) == 0:
-                logging.error("No changes file")
-            if len(changes_files) > 1:
-                logging.error("Multiple changes files?")
-
-            changes_file = changes_files.pop()
+        if built_files and sign:
+            changes_file = built_files[0]
             try:
                 logging.info("Signing changes file %s", changes_file)
                 run(['debsign', '-p%s' % _gpg_abspath(), changes_file,])
@@ -501,7 +498,7 @@ def main(
                 raise RuntimeError(
                     "Failed to sign changes file %s" % changes_file
                 ) from e
-        return files
+        return built_files
     except RuntimeError as e:
         traceback.print_exc()
         return []
@@ -875,19 +872,35 @@ def do_build_host_exitstack(
     stack.callback(os.chdir, oldcwd)
     os.chdir(os.path.join(tempdir, archive_prefix))
 
-    old_pardir_contents = set(os.listdir(os.pardir))
     run(['dpkg-buildpackage'] + list(rem_args))
-    new_pardir_contents = set(os.listdir(os.pardir))
-    temp_pardir_contents = new_pardir_contents - old_pardir_contents
+
+    changes_file = None
     built_pardir_contents = []
-    for f in temp_pardir_contents:
-        shutil.copy(
-            os.path.join(os.pardir, f),
-            os.path.join(oldcwd, os.pardir)
+    changes_files = [f for f in set(os.listdir(os.pardir)) if '.changes' in f]
+    if len(changes_files) == 0:
+        logging.error("No changes file")
+    elif len(changes_files) > 1:
+        logging.error("Multiple changes files?")
+    else:
+        changes_file = os.path.abspath(
+            shutil.copy(
+                os.path.join(os.pardir, changes_files.pop()),
+                os.path.join(oldcwd, os.pardir),
+            )
         )
-        built_pardir_contents.append(os.path.join(os.pardir, f))
+        with open(changes_file, 'rb') as changes_fobj:
+            changes = Changes(changes_fobj)
+            for f in changes['files']:
+                built_pardir_contents.append(
+                    os.path.abspath(
+                        shutil.copy(
+                            os.path.join(os.pardir, f['name']),
+                            os.path.join(oldcwd, os.pardir),
+                        )
+                    )
+                )
 
-    return built_pardir_contents
+    return [changes_file,] + built_pardir_contents
 
 def get_cmd_in_origpath(cmd):
     return shutil.which(
@@ -1006,9 +1019,9 @@ def do_build_lxd_exitstack(
 
     for i in range(retries+1):
         try:
-            _run_in_lxd(container_name, ['apt', 'update',])
+            _run_in_lxd(container_name, ['apt-get', 'update',])
             _run_in_lxd(container_name, [
-                'apt',
+                'apt-get',
                 'install',
                 '-y',
                 'devscripts',
@@ -1018,7 +1031,7 @@ def do_build_lxd_exitstack(
             break
         except Exception as e:
             logging.error(
-                "Failed to run apt in ephemeral build container "
+                "Failed to run apt-get in ephemeral build container "
                 "(attempt %d/%d)",
                 i+1, retries+1,
             )
@@ -1026,29 +1039,42 @@ def do_build_lxd_exitstack(
                 time.sleep(retry_backoffs[i])
     else:
         raise RuntimeError(
-            "Failed to run apt in ephemeral build container"
+            "Failed to run apt-get in ephemeral build container"
         )
 
+    # lxd assumes that if SNAP is set in the environment, that it is
+    # running as a snap. That is not necessarily true when git-ubuntu is
+    # the snapped application. So unset SNAP before we exec.
+    # LP: #1741949
+    env_unset_SNAP = os.environ.copy()
     try:
-        run([
-            lxc,
-            'file',
-            'push',
-            archive_tarball_name,
-            '%s/tmp/' % container_name,
-        ])
+        del env_unset_SNAP['SNAP']
+    except KeyError:
+        pass
+    try:
+        run(
+            [
+                lxc,
+                'file',
+                'push',
+                archive_tarball_name,
+                '%s/tmp/' % container_name,
+            ],
+            env=env_unset_SNAP,
+        )
     except Exception as e:
         raise RuntimeError(
             "Failed to push archive tarball to ephemeral build container"
         ) from e
 
-    cmd = [lxc, 'file', 'push']
-    cmd.extend(tarballs)
-    cmd.extend(['%s/tmp/' % container_name,])
-    try:
-        run(cmd)
-    except Exception as e:
-        raise RuntimeError("Failed to push orig tarballs to container") from e
+    if tarballs:
+        cmd = [lxc, 'file', 'push']
+        cmd.extend(tarballs)
+        cmd.extend(['%s/tmp/' % container_name,])
+        try:
+            run(cmd, env=env_unset_SNAP)
+        except Exception as e:
+            raise RuntimeError("Failed to push orig tarballs to container") from e
 
     logging.info("Copied build files to %s", container_name)
 
@@ -1073,14 +1099,6 @@ def do_build_lxd_exitstack(
         ['ls', '/tmp',],
         user='ubuntu',
     )
-    orig_temp_contents = set(
-        filename_stripped
-        for filename_stripped in (
-            filename.strip() for filename in
-            stdout.splitlines()
-        )
-        if filename_stripped
-    )
 
     container_path = '/tmp/%s-%s' % (
         changelog.srcpkg,
@@ -1132,16 +1150,40 @@ def do_build_lxd_exitstack(
         ['ls', '/tmp',],
         user='ubuntu',
     )
-    new_temp_contents = set(
+    changes_files = set(
         filename_stripped
         for filename_stripped in (
             filename.strip() for filename in
             stdout.splitlines()
         )
-        if filename_stripped
+        if filename_stripped and '.changes' in filename_stripped
     )
 
-    built_temp_contents = new_temp_contents - orig_temp_contents
+    changes_file = None
+    built_temp_contents = []
+
+    if len(changes_files) == 0:
+        logging.error("No changes file")
+    elif len(changes_files) > 1:
+        logging.error("Multiple changes files?")
+    else:
+        changes_file = changes_files.pop()
+        cmd = [
+            lxc,
+            'file',
+            'pull',
+            '%s%s' % (container_name, os.path.join('/tmp', changes_file)),
+            os.pardir,
+        ]
+        run(cmd, env=env_unset_SNAP)
+        changes_file = os.path.abspath(
+            os.path.join(os.pardir, changes_file)
+        )
+        with open(changes_file, 'rb') as changes_fobj:
+            changes = Changes(changes_fobj)
+            for f in changes['files']:
+                built_temp_contents.append(f['name'])
+
     cmd = [lxc, 'file', 'pull']
     cmd.extend([
             '%s%s' % (container_name, os.path.join('/tmp', f))
@@ -1149,39 +1191,57 @@ def do_build_lxd_exitstack(
         ]
     )
     cmd.extend([os.pardir,])
-    run(cmd)
+    run(cmd, env=env_unset_SNAP)
 
-    return [os.path.join(os.pardir, f) for f in built_temp_contents]
+    built_temp_contents = [
+        os.path.abspath(os.path.join(os.pardir, f))
+        for f in built_temp_contents
+    ]
 
+    return [changes_file,] + built_temp_contents
 
 def fetch_orig(
     orig_search_list,
-    changelog,
+    repo,
+    commit_hash,
 ):
+    changelog = gitubuntu.git_repository.Changelog.from_treeish(
+        repo.raw_repo,
+        repo.raw_repo.get(commit_hash),
+    )
+
     unaliased_orig_search_list = expand_changelog_source_aliases(
         orig_search_list,
         changelog,
     )
     # Follow searches already attempted as a 'changelog' source alias
     # may expand to a duplicate.
-    for entry in unique_everseen(unaliased_orig_search_list):
-        try:
-            mechanism_name = entry.mechanism.__name__
-        except AttributeError:
-            mechanism_name = entry.mechanism.func.__name__
-        tarballs = entry.mechanism(changelog, entry.source)
-        if tarballs is None:
-            logging.debug('%s(source=%s) failed',
+    oldcwd = os.getcwd()
+    with repo.temporary_worktree(commit_hash):
+        for entry in unique_everseen(unaliased_orig_search_list):
+            try:
+                mechanism_name = entry.mechanism.__name__
+            except AttributeError:
+                mechanism_name = entry.mechanism.func.__name__
+            tarballs = entry.mechanism(changelog, entry.source)
+            if tarballs is None:
+                logging.debug('%s(source=%s) failed',
+                    mechanism_name,
+                    entry.source,
+                )
+                continue  # search returned negative; try next search entry
+            copied_tarballs = []
+            for tarball in tarballs:
+                copied_tarballs.append(
+                    shutil.copy(tarball, os.path.join(oldcwd, os.pardir))
+                )
+
+            logging.info('Successfully fetched%susing %s(source=%s)',
+                ':\n' + '\n'.join(copied_tarballs) + '\n' if copied_tarballs else ' ',
                 mechanism_name,
                 entry.source,
             )
-            continue  # search returned negative; try next search entry
-        logging.info('Successfully fetched%susing %s(source=%s)',
-            ':\n' + '\n'.join(tarballs) + '\n' if tarballs else ' ',
-            mechanism_name,
-            entry.source,
-        )
-        return tarballs
+            return copied_tarballs
 
 def fetch_orig_and_build(
     repo,
@@ -1205,64 +1265,79 @@ def fetch_orig_and_build(
     )
     # Follow searches already attempted as a 'changelog' source alias
     # may expand to a duplicate.
-    for entry in unique_everseen(unaliased_orig_search_list):
-        try:
-            mechanism_name = entry.mechanism.__name__
-        except AttributeError:
-            mechanism_name = entry.mechanism.func.__name__
-        tarballs = entry.mechanism(changelog, entry.source)
-        if tarballs is None:
-            logging.debug('%s(source=%s) failed',
-                mechanism_name,
-                entry.source,
-            )
-            continue  # search returned negative; try next search entry
-        try:
-            built_files = do_build(
-                repo,
-                commitish,
-                tarballs,
-                rem_args,
-                use_lxd,
-                keep_build_env,
-                lxd_profile,
-                lxd_image,
-                retries,
-                retry_backoffs,
-            )
-        except RuntimeError as e:
-            if entry.must_build:
-                # Build failed and the search list declares this fatal
-                logging.error("Failed to build")
-                raise RuntimeError("Unable to build using %s(source=%s)" % (
+    commit_hash = str(
+        repo.get_commitish(commitish).peel(pygit2.Commit).id
+    )
+    oldcwd = os.getcwd()
+    with repo.temporary_worktree(commit_hash):
+        for entry in unique_everseen(unaliased_orig_search_list):
+            try:
+                mechanism_name = entry.mechanism.__name__
+            except AttributeError:
+                mechanism_name = entry.mechanism.func.__name__
+            tarballs = entry.mechanism(changelog, entry.source)
+            if tarballs is None:
+                logging.debug('%s(source=%s) failed',
                     mechanism_name,
                     entry.source,
-                )) from e
+                )
+                continue  # search returned negative; try next search entry
+            try:
+                built_files = do_build(
+                    repo,
+                    commitish,
+                    tarballs,
+                    rem_args,
+                    use_lxd,
+                    keep_build_env,
+                    lxd_profile,
+                    lxd_image,
+                    retries,
+                    retry_backoffs,
+                )
+            except RuntimeError as e:
+                if entry.must_build:
+                    # Build failed and the search list declares this fatal
+                    logging.error("Failed to build")
+                    raise RuntimeError("Unable to build using %s(source=%s)" % (
+                        mechanism_name,
+                        entry.source,
+                    )) from e
+                else:
+                    # Build failed, but the search list entry declares that
+                    # this should not be fatal and so we try the next search
+                    # entry
+                    # First, remove any tarballs fetched with this mechanism
+                    for tarball in tarballs:
+                        os.remove(tarball)
+                    logging.debug(
+                        "Ignoring failure to build using %s(source=%s)",
+                        mechanism_name,
+                        entry.source,
+                    )
+                    continue
             else:
-                # Build failed, but the search list entry declares that
-                # this should not be fatal and so we try the next search
-                # entry
-                # First, remove any tarballs fetched with this mechanism
-                for tarball in tarballs:
-                    os.remove(tarball)
-                logging.debug(
-                    "Ignoring failure to build using %s(source=%s)",
+                copied_built_files = []
+                for f in built_files:
+                    copied_built_files.append(
+                        os.path.abspath(
+                            shutil.copy(f, os.path.join(oldcwd, os.pardir))
+                        )
+                    )
+
+                logging.info(
+                    "Successfully built%susing %s(source=%s)",
+                    ":\n" + "\n".join(copied_built_files) + "\n"
+                    if copied_built_files else " ",
                     mechanism_name,
                     entry.source,
                 )
-                continue
+                return copied_built_files
         else:
-            logging.info('Successfully built%susing %s(source=%s)',
-                ':\n' + '\n'.join(built_files) + '\n' if built_files else ' ',
-                mechanism_name,
-                entry.source,
+            raise RuntimeError(
+                "All potential search mechanisms were tried, "
+                "but none produced a successful build."
             )
-            return built_files
-    else:
-        raise RuntimeError(
-            "All potential search mechanisms were tried, "
-            "but none produced a successful build."
-        )
 
 
 class NativenessMismatchError(Exception): pass
diff --git a/gitubuntu/dsc.py b/gitubuntu/dsc.py
index 1fc3129..f630b56 100644
--- a/gitubuntu/dsc.py
+++ b/gitubuntu/dsc.py
@@ -47,24 +47,48 @@ class GitUbuntuDsc(Dsc):
 
     @property
     def orig_tarball_path(self):
-        if self._orig:
+        """Get string filesystem path of this DSC's orig tarball
+
+        Returns:
+        list of string paths to the orig tarball
+
+        The list will either have zero or one elements. Zero elements
+        implies a native package.
+
+        Raises:
+        GitUbuntuDscError if tarball fails to verify
+        """
+        if self._orig is not None:
             return self._orig
-        orig = None
+        orig = []
         for entry in self['Files']:
             if orig_re.match(entry['name']):
-                if orig is not None:
-                    raise GitUbuntuDscError('Multiple base '
-                        'orig tarballs in DSC')
-                orig = os.path.join(self.dsc_dir, entry['name'])
-                logging.debug('Verifying orig tarball %s', orig)
-                if not self.verify_file(orig):
-                    raise GitUbuntuDscError('Unable to verify orig '
-                        'tarball %s' % orig)
+                if orig:
+                    raise GitUbuntuDscError(
+                        "Multiple base orig tarballs in DSC"
+                    )
+                _orig = os.path.join(self.dsc_dir, entry['name'])
+                orig.append(_orig)
+                logging.debug('Verifying orig tarball %s', _orig)
+                if not self.verify_file(_orig):
+                    raise GitUbuntuDscError(
+                        "Unable to verify orig tarball %s" % _orig
+                    )
+        assert len(orig) < 2
         self._orig = orig
         return self._orig
 
     @property
     def component_tarball_paths(self):
+        """Get string filesystem paths of this DSC's orig component tarball(s)
+
+        Returns:
+        dictionary of string path to component tarballs, keyed by
+        component names
+
+        Raises:
+        GitUbuntuDscError if a tarball fails to verify
+        """
         if self._components is not None:
             return self._components
         components = {}
@@ -83,4 +107,6 @@ class GitUbuntuDsc(Dsc):
 
     @property
     def all_tarball_paths(self):
-        return [self.orig_tarball_path] + list(self.component_tarball_paths.values())
+        return self.orig_tarball_path + list(
+            self.component_tarball_paths.values()
+        )
diff --git a/gitubuntu/exportorig.py b/gitubuntu/exportorig.py
index a04362c..f259ad0 100644
--- a/gitubuntu/exportorig.py
+++ b/gitubuntu/exportorig.py
@@ -88,10 +88,8 @@ def main(
     try:
         return gitubuntu.build.fetch_orig(
             orig_search_list=search_list,
-            changelog=gitubuntu.git_repository.Changelog.from_treeish(
-                repo.raw_repo,
-                repo.raw_repo.get(commit_hash),
-            ),
+            repo=repo,
+            commit_hash=commit_hash,
         )
     except gitubuntu.build.NativenessMismatchError as e:
         logging.error("%s" % e)
diff --git a/gitubuntu/git_repository.py b/gitubuntu/git_repository.py
index b04cd16..d5842b2 100644
--- a/gitubuntu/git_repository.py
+++ b/gitubuntu/git_repository.py
@@ -21,7 +21,14 @@ import gitubuntu.lint
 from gitubuntu.__main__ import top_level_defaults
 import gitubuntu.build
 from gitubuntu.dsc import component_tarball_matches
-from gitubuntu.run import run, runq, decode_binary
+from gitubuntu.run import (
+   decode_binary,
+   run,
+   runq,
+   run_gbp,
+   run_quilt,
+)
+import gitubuntu.versioning
 try:
     pkg = 'python3-debian'
     import debian.changelog
@@ -918,8 +925,9 @@ class GitUbuntuRepository:
             tarballs.append(
                 os.path.join(os.path.pardir, potential_main_tarballs[0])
             )
-            cmd = ['gbp', 'buildpackage', '--git-builder=/bin/true',
-                   '--git-pristine-tar', '--git-ignore-branch',
+            args = ['buildpackage', '--git-builder=/bin/true',
+                   '--git-ignore-new', '--git-pristine-tar',
+                   '--git-ignore-branch',
                    '--git-upstream-tag=%s/upstream/%s/%%(version)s%s' %
                    (namespace, dist, ext)]
             # This will probably break if the component tarballs get
@@ -935,11 +943,11 @@ class GitUbuntuRepository:
             tarballs.extend(map(lambda x : os.path.join(os.path.pardir, x),
                 list(potential_component_tarballs.values()))
             )
-            cmd.extend(map(lambda x : '--git-component=%s' % x,
+            args.extend(map(lambda x : '--git-component=%s' % x,
                 list(potential_component_tarballs.keys()))
             )
             with self.pristine_tar_branches(dist, namespace):
-                run(cmd)
+                run_gbp(args, env=self.env)
             return tarballs
 
         raise PristineTarNotFoundError(
@@ -1447,6 +1455,9 @@ class GitUbuntuRepository:
     def get_upstream_tag(self, version, namespace):
         return self.get_tag_reference(upstream_tag(version, namespace))
 
+    def get_orphan_tag(self, version, namespace):
+        return self.get_tag_reference(orphan_tag(version, namespace))
+
     def nearest_remote_branches(self, commit_hash, prefix=None,
         max_commits=100
     ):
@@ -1853,29 +1864,39 @@ class GitUbuntuRepository:
 
     @contextmanager
     def temporary_worktree(self, commitish, prefix=None):
-        with tempfile.TemporaryDirectory(prefix=prefix) as tempdir:
-            self.git_run(
-                [
-                    'worktree',
-                    'add',
-                    '--detach',
-                    '--force',
-                    tempdir,
-                    commitish,
-                ]
-            )
-
-            oldcwd = os.getcwd()
-            os.chdir(tempdir)
+        try:
+            with tempfile.TemporaryDirectory() as tempdir:
+                with tempfile.TemporaryDirectory(
+                    prefix=prefix,
+                    dir=tempdir,
+                ) as ttempdir:
+                    self.git_run(
+                        [
+                            'worktree',
+                            'add',
+                            '--detach',
+                            '--force',
+                            ttempdir,
+                            commitish,
+                        ]
+                    )
 
-            try:
-                yield
-            except:
-                raise
-            finally:
-                os.chdir(oldcwd)
+                    oldcwd = os.getcwd()
+                    os.chdir(ttempdir)
 
-        self.git_run(['worktree', 'prune'])
+                    orig_work_tree = self._env['GIT_WORK_TREE']
+                    try:
+                        self._env['GIT_WORK_TREE'] = ttempdir
+                        yield
+                    except:
+                        raise
+                    finally:
+                        self._env['GIT_WORK_TREE'] = orig_work_tree
+                        os.chdir(oldcwd)
+        except:
+            raise
+        finally:
+            self.git_run(['worktree', 'prune'])
 
     def tree_hash_after_command(self, commitish, cmd):
         with self.temporary_worktree(commitish):
@@ -1972,8 +1993,8 @@ class GitUbuntuRepository:
         # differentiate between applied an unapplied
         with self.temporary_worktree(commit_hash):
             try:
-                run(
-                    ['quilt', 'push', '-a'],
+                run_quilt(
+                    ['push', '-a'],
                     env=self.quilt_env_from_treeish_str(commit_hash),
                 )
                 # False if in an unapplied state, which is signified by
@@ -1998,8 +2019,8 @@ class GitUbuntuRepository:
                         raise e
 
                     try:
-                        run(
-                            ['quilt', 'push', '-a'],
+                        run_quilt(
+                            ['push', '-a'],
                             env=self.quilt_env_from_treeish_str(commit_hash),
                         )
                         # False if in an unapplied state
@@ -2064,10 +2085,8 @@ class GitUbuntuRepository:
                 for_merge=False,
                 no_pristine_tar=False,
             ),
-            changelog=Changelog.from_treeish(
-                self.raw_repo,
-                self.raw_repo.get(commit_hash)
-            ),
+            repo=self,
+            commit_hash=commit_hash,
         )
         logger.setLevel(oldLevel)
         # run dpkg-source
@@ -2080,22 +2099,32 @@ class GitUbuntuRepository:
 
             # create a nested temporary directory where we will recreate
             # the .pc directory
-            with tempfile.TemporaryDirectory(prefix=tempdir+'/') as ttempdir:
+            with tempfile.TemporaryDirectory(dir=tempdir) as ttempdir:
                 oldcwd = os.getcwd()
                 os.chdir(ttempdir)
 
                 for tarball in tarballs:
                     run(['tar', '-x', '--strip-components=1', '-f', tarball,])
 
-                # need the debia/patches
-                shutil.copytree(
-                    os.path.join(self.local_dir, 'debian',),
-                    'debian',
-                )
+                # in case the orig tarballs contain a debian/ directory
+                try:
+                    os.unlink('debian')
+                except IsADirectoryError:
+                    shutil.rmtree('debian')
+                except FileNotFoundError:
+                    # it is possible no debian/ dir exists yet
+                    pass
+
+                # need the debian dir from @commit_hash to recreate .pc
+                with self.temporary_worktree(commit_hash):
+                    shutil.copytree(
+                        'debian',
+                        os.path.join(ttempdir, 'debian'),
+                    )
 
                 # generate the equivalent .pc directory
-                run(
-                    ['quilt', 'push', '-a'],
+                run_quilt(
+                    ['push', '-a'],
                     env=self.quilt_env_from_treeish_str(commit_hash),
                     rcs=[2],
                 )
@@ -2184,9 +2213,10 @@ with other relevant fields (see below for details).
                         # if any patches add files that are untracked,
                         # remove them
                         run(['git', 'clean', '-f', '-d',])
-                        # reset all the other files to their status in
-                        # HEAD
-                        run(['git', 'checkout', commit_hash, '--', '*',])
+                        # reset all the other files to their status
+                        # before our modifications, preserving our d/p
+                        # changes
+                        run(['git', 'checkout', '--', '*',])
                         with open(fixup_patch_path, 'rb') as f:
                             run(['patch', '-Rp1',], input=f.read())
 
@@ -2278,14 +2308,14 @@ with other relevant fields (see below for details).
             'debian/changelog',
         ):
             with self.temporary_worktree(commit_hash):
-                run(
+                run_gbp(
                     [
-                        'gbp',
                         'dch',
                         '--snapshot',
                         '--ignore-branch',
                         '--since=%s' % str(parent_ref),
-                    ]
+                    ],
+                    env=self.env,
                 )
                 return self.dir_to_tree('.')
 
@@ -2339,3 +2369,80 @@ with other relevant fields (see below for details).
             pygit2.GIT_FILEMODE_TREE, # attr
         )
         return str(tb.write())
+
+    def find_ubuntu_merge_base(
+        self,
+        new_debian_id,
+        old_ubuntu_id,
+    ):
+        # For now, we have to account for both old and new versions of the
+        # importer. This can be removed/simplified once we reimport the
+        # world.
+        # Under the old algorithm, use git-merge-base, but also check that
+        # the merge base's version is what we expect
+        # If it is not, assume we are in the new algorithm and look for an
+        # import tag for the expected Debian version.
+        # If not found, return None
+        # LP: #1734364
+
+        # obtain version from appropriate changelog
+        expected_version, _ = self.get_changelog_versions_from_treeish(
+            str(old_ubuntu_id)
+        )
+
+        # extract corresponding Debian version
+        expected_version, _ = gitubuntu.versioning.split_version_string(
+            expected_version
+        )
+
+        merge_base_id = self.raw_repo.merge_base(
+            str(new_debian_id),
+            str(old_ubuntu_id),
+        )
+
+        version, _ = self.get_changelog_versions_from_treeish(
+            str(merge_base_id)
+        )
+
+        if version != expected_version:
+            logging.debug(
+                "Found a git-merge-base %s, but its version (%s) "
+                "is not as expected (%s). Assuming the new importer "
+                "algorithm and looking for an import tag instead.",
+                str(merge_base_id),
+                version,
+                expected_version,
+            )
+            merge_base_id = self.get_import_tag(
+                expected_version,
+                'pkg',
+            ).peel(pygit2.Commit).id
+
+            if not merge_base_id:
+                logging.error(
+                    "Unable to find an import tag for %s.",
+                    expected_version,
+                )
+                return None
+
+            try:
+                self.git_run(
+                    [
+                        'merge-base',
+                        '--is-ancestor',
+                        str(merge_base_id),
+                        str(old_ubuntu_id),
+                    ],
+                    verbose_on_failure=False,
+                )
+            except CalledProcessError as e:
+                logging.error(
+                    "Found an import tag for %s (commit: %s), but it is "
+                    "not an ancestor of %s.",
+                    expected_version,
+                    str(merge_base_id),
+                    str(old_ubuntu_id),
+                )
+                return None
+
+        return merge_base_id
diff --git a/gitubuntu/importer.py b/gitubuntu/importer.py
index e8fb4f8..ec157e7 100644
--- a/gitubuntu/importer.py
+++ b/gitubuntu/importer.py
@@ -53,7 +53,13 @@ from gitubuntu.git_repository import (
     PristineTarError,
     is_dir_3_0_quilt,
 )
-from gitubuntu.run import decode_binary, run, runq
+from gitubuntu.run import (
+    decode_binary,
+    run,
+    runq,
+    run_gbp,
+    run_quilt,
+)
 from gitubuntu.source_information import (
     GitUbuntuSourceInformation,
     NoPublicationHistoryException,
@@ -645,31 +651,29 @@ def import_orig(repo, dsc, namespace, version, dist):
         orig_tarball_path = dsc.orig_tarball_path
     except GitUbuntuDscError as e:
         raise GitUbuntuImportOrigError(
-            'Unable to import orig tarball in DSC for version %s' % version
+            "Unable to import orig tarball in DSC for version %s" % version
         ) from e
 
-    if orig_tarball_path is None:
-        # native packages only have a .tar.*
-        native_re = re.compile(r'.*\.tar\.[^.]+$')
-        for entry in dsc['Files']:
-            if native_re.match(entry['name']):
-                return
-        raise GitUbuntuImportOrigError('No orig tarball in '
-            'DSC for version %s.' % version
+    if orig_tarball_path:
+        orig_tarball_path = orig_tarball_path[0]
+    else:
+        # native packages do not have an orig tarball
+        if '-' not in version:
+            return
+        raise GitUbuntuImportOrigError(
+            "No orig tarball in DSC for version %s." % version
         )
 
     try:
         orig_component_paths = dsc.component_tarball_paths
     except GitUbuntuDscError as e:
         raise GitUbuntuImportOrigError(
-            'Unable to import component tarballs in DSC for version '
-            '%s' % version
+            "Unable to import component tarballs in DSC for version %s" %
+            version
         ) from e
 
-    orig_paths = [orig_tarball_path] + list(orig_component_paths.values())
-    if (not orig_tarball_path or
-        orig_imported(repo, orig_paths, namespace, dist)
-    ):
+    orig_paths = dsc.all_tarball_paths
+    if orig_imported(repo, orig_paths, namespace, dist):
         return
     if not all(map(os.path.exists, orig_paths)):
         raise GitUbuntuImportOrigError('Unable to find tarball: '
@@ -683,20 +687,20 @@ def import_orig(repo, dsc, namespace, version, dist):
             # https://github.com/agx/git-buildpackage/pull/16
             os.chdir(repo.local_dir)
 
-            ext = os.path.splitext(dsc.orig_tarball_path)[1]
+            ext = os.path.splitext(orig_tarball_path)[1]
 
             # make this non-fatal if upstream already exist as tagged?
-            cmd = ['gbp', 'import-orig', '--no-merge',
+            args = ['import-orig', '--no-merge',
                  '--upstream-branch', 'do-not-push',
                  '--pristine-tar', '--no-interactive',
                  '--no-symlink-orig',
                  '--upstream-tag=%s/upstream/%s/%%(version)s%s' %
                      (namespace, dist, ext)]
-            cmd.extend(map(lambda x: '--component=%s' % x,
+            args.extend(map(lambda x: '--component=%s' % x,
                 list(orig_component_paths.keys()))
             )
-            cmd.extend([orig_tarball_path,])
-            run(cmd, env=repo.env)
+            args.extend([orig_tarball_path,])
+            run_gbp(args, env=repo.env)
     except CalledProcessError as e:
         raise GitUbuntuImportOrigError(
             'Unable to import tarballs: %s' % orig_paths
@@ -783,18 +787,18 @@ def import_patches_applied_tree(repo, dsc_pathname):
         while True:
             try:
                 os.chdir(extracted_dir)
-                run(
-                    ['quilt', 'push'],
+                run_quilt(
+                    ['push'],
                     env=repo.quilt_env_from_treeish_str(import_tree_hash),
                     verbose_on_failure=False,
                 )
-                patch_name, _ = run(
-                    ['quilt', 'top'],
+                patch_name, _ = run_quilt(
+                    ['top'],
                     env=repo.quilt_env_from_treeish_str(import_tree_hash),
                 )
                 patch_name = patch_name.strip()
-                header, _  = run(
-                    ['quilt', 'header'],
+                header, _  = run_quilt(
+                    ['header'],
                     env=repo.quilt_env_from_treeish_str(import_tree_hash),
                 )
                 header = header.strip()
@@ -1167,9 +1171,16 @@ def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo):
                  unapplied_tip_version,
             )
 
-        # Walk changelog backwards until we find an imported version
+        # Walk changelog backwards until we find an imported or orphaned
+        # version, preferring imported versions
         for version in import_tree_versions:
-            unapplied_changelog_parent_tag = repo.get_import_tag(version, namespace)
+            unapplied_changelog_parent_tag = repo.get_import_tag(
+                version,
+                namespace,
+            ) or repo.get_orphan_tag(
+                version,
+                namespace,
+            )
             if unapplied_changelog_parent_tag is not None:
                 # sanity check that version from d/changelog of the
                 # tagged parent matches ours
@@ -1202,6 +1213,7 @@ def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo):
     # of the import tree, to obtain the changelog parent
     upload_tag = repo.get_upload_tag(spi.version, namespace)
     import_tag = repo.get_import_tag(spi.version, namespace)
+    orphan_tag = repo.get_orphan_tag(spi.version, namespace)
     if import_tag:
         if str(import_tag.peel().tree.id) != unapplied_import_tree_hash:
             logging.error(
@@ -1216,6 +1228,21 @@ def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo):
                 str(import_tag.peel().id),
             )
             return
+    elif orphan_tag:
+        if str(orphan_tag.peel().tree.id) != unapplied_import_tree_hash:
+            raise Exception(
+                "Found orphan tag %s, but the corresponding tree "
+                "does not match the published version. We cannot "
+                "currently support multiple orphan tags for the same "
+                "published version and will now exit.",
+                repo.tag_to_pretty_name(orphan_tag),
+            )
+        else:
+            repo.update_head_to_commit(
+                spi.head_name(namespace),
+                str(orphan_tag.peel().id),
+            )
+            return
     elif upload_tag:
         if str(upload_tag.peel().tree.id) != unapplied_import_tree_hash:
             logging.warn(
diff --git a/gitubuntu/lint.py b/gitubuntu/lint.py
index a7428e5..de130d0 100644
--- a/gitubuntu/lint.py
+++ b/gitubuntu/lint.py
@@ -443,8 +443,8 @@ def tree_after_merge_changelogs(repo, old, new):
     changelog_files = []
     try:
         merge_base_id = str(
-            repo.raw_repo.merge_base(
-               repo.get_commitish(old).id,
+            repo.find_ubuntu_merge_base(
+                repo.get_commitish(old).id,
                 repo.get_commitish(new).id
             )
         )
@@ -513,9 +513,11 @@ def do_merge_lint(repo, namespace, commitish_string, old, new_base):
     # for now, assume pkg/ubuntu/devel and pkg/debian/sid are the merge
     # points and that we can do this command. Ideally, we abstract the merge
     # API into a better place
-    merge_base_id = str(repo.raw_repo.merge_base(
-        repo.get_commitish(old).id,
-        repo.get_commitish(new_base).id)
+    merge_base_id = str(
+        repo.find_ubuntu_merge_base(
+            repo.get_commitish(old).id,
+            repo.get_commitish(new_base).id
+        )
     )
     # 1) check if old/debian, old/ubuntu and new/debian point to what we
     # expect - compare treeishs and commits
diff --git a/gitubuntu/merge.py b/gitubuntu/merge.py
index c16fe4b..c49d9ed 100644
--- a/gitubuntu/merge.py
+++ b/gitubuntu/merge.py
@@ -437,7 +437,7 @@ def main(
         logging.error(stdout)
         return 1
 
-    merge_base_id = repo.raw_repo.merge_base(
+    merge_base_id = repo.find_ubuntu_merge_base(
         onto_obj.id,
         commitish_obj.id,
     )
diff --git a/gitubuntu/run.py b/gitubuntu/run.py
index 4b1a86c..1fdcd6d 100644
--- a/gitubuntu/run.py
+++ b/gitubuntu/run.py
@@ -1,4 +1,5 @@
 import logging
+import os
 import subprocess
 
 
@@ -102,3 +103,17 @@ def decode_binary(binary, verbose=True):
             logging.warning("Failed to decode blob: %s", e)
             logging.warning("blob=%s", binary.decode(errors='replace'))
         return binary.decode('utf-8', errors='replace')
+
+def run_quilt(args, env=None, **kwargs):
+    cmd = ['quilt', '--quiltrc', '-'] + args
+    return run(cmd, env=env, **kwargs)
+
+def run_gbp(args, env=None, **kwargs):
+    cmd = ['gbp'] + args
+    if env:
+        env_no_gbp_conf = env.copy()
+    else:
+        env_no_gbp_conf = os.environ.copy()
+    env_no_gbp_conf['GBP_CONF_FILES'] = '/dev/null'
+
+    return run(cmd, env=env_no_gbp_conf, **kwargs)
diff --git a/gitubuntu/versioning.py b/gitubuntu/versioning.py
index 7b1eb7b..62e28ae 100644
--- a/gitubuntu/versioning.py
+++ b/gitubuntu/versioning.py
@@ -25,6 +25,7 @@ __all__ = [
    "next_development_version_string",
    "next_sru_version_string",
    "version_compare",
+   "split_version_string",
 ]
 
 # see _decompose_version_string for explanation of members
@@ -117,6 +118,27 @@ def _decompose_version_string(version_string):
         return _VersionComps._make([parts, None, None, None, None])
 
 
+def split_version_string(version_string):
+    """Return the Debian version and Ubuntu suffix, if present
+
+    Returns a 2-tuple of strings
+    """
+
+    if 'build' in version_string and 'ubuntu' in version_string:
+        raise ValueError(
+            "Not sure how to split %s: both build and " "ubuntu found" %
+            version_string
+        )
+
+    if 'build' in version_string:
+        deb_version, ubuntu_suffix = version_string.split('build')
+        return deb_version, 'build' + ubuntu_suffix
+    if 'ubuntu' in version_string:
+        deb_version, ubuntu_suffix = version_string.split('ubuntu')
+        return deb_version, 'ubuntu' + ubuntu_suffix
+    return version_string, ""
+
+
 def _bump_version_object(old_version_object, bump_function):
     """Return a new Version object with the correct attribute bumped
 
@@ -280,6 +302,19 @@ def test_decompose_version_string(test_input, expected):
 
 
 @pytest.mark.parametrize('test_input, expected', [
+    ('1.0-2', ('1.0-2', '')),
+    ('1.0-2ubuntu1', ('1.0-2', 'ubuntu1')),
+    ('1.0-2ubuntu1.3', ('1.0-2', 'ubuntu1.3')),
+    ('1.0-2build1', ('1.0-2', 'build1')),
+    ('1', ('1', '')),
+    ('1ubuntu2', ('1', 'ubuntu2')),
+    ('1.0-3ubuntu', ('1.0-3', 'ubuntu')),
+    ('1:1.0-1.1ubuntu1', ('1:1.0-1.1', 'ubuntu1')),
+])
+def test_split_version_string(test_input, expected):
+    assert split_version_string(test_input) == expected
+
[e-mail]('test_input, expected', [
     ('1.0-2', '1.0-2ubuntu1'),
     ('1.0-2ubuntu1', '1.0-2ubuntu2'),
     ('1.0-2ubuntu1.3', '1.0-2ubuntu2'),
diff --git a/scripts/import-source-packages.py b/scripts/import-source-packages.py
index 33ccbf2..0bc5dca 100755
--- a/scripts/import-source-packages.py
+++ b/scripts/import-source-packages.py
@@ -45,20 +45,15 @@ LOG_PATH = os.path.join(tempfile.gettempdir(), 'import-source-packages-log')
 
 def import_new_published_sources(
     num_workers,
-    whitelist,
-    blacklist,
-    phasing_main,
-    phasing_universe,
+    pkgnames_to_import,
     dry_run,
 ):
-    """import_new_published_source - Import all new publishes since a prior execution
+    """Import all new publishes since a prior execution
 
     Arguments:
     num_workers - integer number of worker processes to use
-    whitelist - a list of of packages to always import
-    blacklist - a list of packages to never import
-    phasing_main - a integer percentage of all packages in main to import
-    phasing_universe - a integer percentage of all packages in universe to import
+    pkgnames_to_import - list of source package names that should be
+    considered for import, if publish records are found
     dry_run - a boolean to indicate a dry-run operation
 
     Returns:
@@ -118,14 +113,7 @@ def import_new_published_sources(
                 # a publish timestamp before the last run
                 if spphr.date_published.timestamp() < timestamp.time:
                     break
-                if scriptutils.should_import_srcpkg(
-                    spphr.source_package_name,
-                    spphr.component_name,
-                    whitelist,
-                    blacklist,
-                    phasing_main,
-                    phasing_universe,
-                ):
+                if spphr.source_package_name in pkgnames_to_import:
                     dist_filtered_pkgnames.add(spphr.source_package_name)
 
         if not dist_filtered_pkgnames:
@@ -146,7 +134,6 @@ def import_new_published_sources(
                 link=str(dist_newest_spphr),
             )
 
-
     ret = scriptutils.pool_map_import_srcpkg(
         num_workers=num_workers,
         dry_run=dry_run,
@@ -158,7 +145,7 @@ def import_new_published_sources(
     return ret
 
 def read_timestamps():
-    """read_timestamps - Read saved timestamp values from LOG_PATH
+    """Read saved timestamp values from LOG_PATH
 
     If the log file is not readable (e.g., does not exist), the
     timestamps will be set to 24 hours before now.
@@ -194,7 +181,7 @@ def read_timestamps():
         )
 
 def write_timestamps(timestamps):
-    """write_timestamps - Write timestamp values to LOG_PATH
+    """Write timestamp values to LOG_PATH
 
     Arguments:
     timestamps - a dictionary with 'debian' and 'ubuntu' keys, each of which is
@@ -230,7 +217,7 @@ def main(
     phasing_universe=scriptutils.DEFAULTS.phasing_universe,
     dry_run=scriptutils.DEFAULTS.dry_run,
 ):
-    """main - Main entry point to the script
+    """Main entry point to the script
 
     Arguments:
     num_workers - integer number of worker threads to use
@@ -272,13 +259,21 @@ def main(
     mail_imported_srcpkgs = set()
     mail_failed_srcpkgs = set()
 
-    while True:
-        imported_srcpkgs, failed_srcpkgs = import_new_published_sources(
-            num_workers,
+    try:
+        pkgnames_to_import = scriptutils.srcpkgs_to_import_list(
             whitelist,
             blacklist,
             phasing_main,
             phasing_universe,
+        )
+    except RuntimeError as e:
+        print("%s" % e)
+        sys.exit(1)
+
+    while True:
+        imported_srcpkgs, failed_srcpkgs = import_new_published_sources(
+            num_workers,
+            pkgnames_to_import,
             dry_run,
         )
         print("Imported %d source packages" % len(imported_srcpkgs))
@@ -320,7 +315,7 @@ def main(
             time.sleep(sleep_interval_minutes * 60)
 
 def cli_main():
-    """cli_main - CLI entry point to script
+    """CLI entry point to script
     """
     parser = argparse.ArgumentParser(
         description='Script to import all source packages with phasing',
diff --git a/scripts/scriptutils.py b/scripts/scriptutils.py
index 912efd1..3608d58 100644
--- a/scripts/scriptutils.py
+++ b/scripts/scriptutils.py
@@ -1,13 +1,21 @@
+import bz2
 from collections import namedtuple
 import functools
+import gzip
 import hashlib
+import itertools
+import lzma
 import multiprocessing
 import os
 import sys
 import subprocess
 import time
+import urllib.request
 
 import pkg_resources
+import pytest
+
+from debian.deb822 import Sources
 
 # We expect to be running from a git repository in master for this
 # script, because the snap's python code is not exposed except within
@@ -24,6 +32,7 @@ sys.path.insert(
 )
 
 from gitubuntu.run import run
+from gitubuntu.source_information import GitUbuntuSourceInformation
 
 Defaults = namedtuple(
     'Defaults',
@@ -54,6 +63,8 @@ DEFAULTS = Defaults(
     use_whitelist=True,
 )
 
+UBUNTU_SOURCES_ROOT = 'http://archive.ubuntu.com/ubuntu/dists/'
+DEBIAN_SOURCES_ROOT = 'http://cdn-fastly.deb.debian.org/debian/dists/'
 
 def should_import_srcpkg(
     pkgname,
@@ -63,7 +74,7 @@ def should_import_srcpkg(
     phasing_main,
     phasing_universe,
 ):
-    """should_import_srcpkg - indicate if a given source package should be imported
+    """indicate if a given source package should be imported
 
     The phasing is implemented similarly to update-manager. If the
     md5sum of the source package name is less than the (appropriate
@@ -100,9 +111,183 @@ def should_import_srcpkg(
     # skip partner and multiverse for now
     return False
 
[e-mail]('pkgname, component, whitelist, blacklist, phasing_main, phasing_universe, expected', [
+    ('jimtcl', 'universe', [], [], 1, 0, False),
+    ('jimtcl', 'universe', [], [], 1, 100, True),
+    ('jimtcl', 'universe', ['jimtcl',], [], 1, 0, True),
+    ('linux', 'main', [], ['linux',], 100, 0, False),
+])
+def test_should_import_srcpkg(
+    pkgname,
+    component,
+    whitelist,
+    blacklist,
+    phasing_main,
+    phasing_universe,
+    expected,
+):
+    assert should_import_srcpkg(
+        pkgname=pkgname,
+        component=component,
+        whitelist=whitelist,
+        blacklist=blacklist,
+        phasing_main=phasing_main,
+        phasing_universe=phasing_universe,
+    ) == expected
+
+def _get_sources(
+    sources_url_root,
+    serieses,
+    components,
+    pocket_suffixes,
+):
+    """Helper generator of Sources file contents
+
+    Arguments:
+    sources_url_root: a string prefix up to "dists" for Sources URLs
+    serieses: a list of series strings
+    components: a list of component strings
+    pocket_suffixes: a list of pocket suffix strings
+
+    Yields:
+    The contents of all Sources files satisfying the requested
+    serieses X components X pocket_suffixes product.
+    """
+    compressions = {
+        '.xz': lzma.open,
+        '.bz2': bz2.open,
+        '.gz': gzip.open,
+    }
+
+    for component in components:
+        for series, pocket_suffix in itertools.product(
+            serieses,
+            pocket_suffixes,
+        ):
+            url = sources_url_root + '%s%s/%s/source/Sources' % (
+                series,
+                pocket_suffix,
+                component,
+            )
+            for compression, opener in compressions.items():
+                try:
+                    with urllib.request.urlopen(
+                        url + compression
+                    ) as source_url_file:
+                        yield opener(source_url_file, mode='r')
+                        break
+                except urllib.error.HTTPError:
+                    pass
+            else:
+                raise RuntimeError(
+                    "Unable to find any Sources file for component=%s, "
+                    "series=%s, pocket=%s" % (
+                        component,
+                        series,
+                        pocket_suffix,
+                    )
+                )
+    raise StopIteration()
+
+def srcpkgs_to_import_list(
+    whitelist,
+    blacklist,
+    phasing_main,
+    phasing_universe,
+):
+    """Obtain a list of source package names satisfying some criteria
+
+    Arguments:
+    whitelist - a list of of packages to always import
+    blacklist - a list of packages to never import
+    phasing_main - a integer percentage of all packages in main to import
+    phasing_universe - a integer percentage of all packages in universe to import
+
+    Packages only published in Debian are treated like universe
+    packages.
+
+    Returns:
+    A list of source package names that should be imported, which can be
+    empty.
+
+    Raises:
+    RuntimeError if any active component/series/pocket URL is not found
+    """
+
+    all_ubuntu_pkgnames = set()
+    pkgnames_to_import = set()
+    for sources in _get_sources(
+        sources_url_root=UBUNTU_SOURCES_ROOT,
+        serieses=GitUbuntuSourceInformation('ubuntu').active_series_name_list,
+        components=['main',],
+        pocket_suffixes=['-proposed', '-updates', '-security', ''],
+    ):
+        for src in Sources.iter_paragraphs(
+            sources,
+            use_apt_pkg=False,
+        ):
+            pkgname = src['Package']
+            all_ubuntu_pkgnames.add(pkgname)
+            if should_import_srcpkg(
+                pkgname=pkgname,
+                component='main',
+                whitelist=whitelist,
+                blacklist=blacklist,
+                phasing_main=phasing_main,
+                phasing_universe=phasing_universe,
+            ):
+                pkgnames_to_import.add(pkgname)
+
+    for sources in _get_sources(
+        sources_url_root=UBUNTU_SOURCES_ROOT,
+        serieses=GitUbuntuSourceInformation('ubuntu').active_series_name_list,
+        components=['universe',],
+        pocket_suffixes=['-proposed', '-updates', '-security', ''],
+    ):
+        for src in Sources.iter_paragraphs(
+            sources,
+            use_apt_pkg=False,
+        ):
+            pkgname = src['Package']
+            all_ubuntu_pkgnames.add(pkgname)
+            if should_import_srcpkg(
+                pkgname=pkgname,
+                component='universe',
+                whitelist=whitelist,
+                blacklist=blacklist,
+                phasing_main=phasing_main,
+                phasing_universe=phasing_universe,
+            ):
+                pkgnames_to_import.add(pkgname)
+
+    # find source packages only published in Debian and treat them like
+    # they are in Universe
+    for sources in _get_sources(
+        sources_url_root=DEBIAN_SOURCES_ROOT,
+        serieses=GitUbuntuSourceInformation('debian').all_series_name_list,
+        components=['main',],
+        pocket_suffixes = ['',],
+    ):
+        for src in Sources.iter_paragraphs(
+            sources,
+            use_apt_pkg=False,
+        ):
+            pkgname = src['Package']
+            if pkgname not in all_ubuntu_pkgnames:
+                if should_import_srcpkg(
+                    pkgname=pkgname,
+                    component='universe',
+                    whitelist=whitelist,
+                    blacklist=blacklist,
+                    phasing_main=phasing_main,
+                    phasing_universe=phasing_universe,
+                ):
+                    pkgnames_to_import.add(pkgname)
+
+    return pkgnames_to_import
 
 def import_srcpkg(pkgname, dry_run):
-    """import_srcpkg - Invoke git ubuntu import on @pkgname
+    """Invoke git ubuntu import on @pkgname
 
     Arguments:
     pkgname - string name of a source package
@@ -145,7 +330,7 @@ def setup_git_config(
     name='Ubuntu Git Importer',
     email='[e-mail]',
 ):
-    """setup_git_config - Ensure global required Git configuration values are set
+    """Ensure global required Git configuration values are set
 
     Arguments:
     name - string name to set as user.name in Git config
@@ -165,22 +350,26 @@ def pool_map_import_srcpkg(
     dry_run,
     pkgnames,
 ):
-    """pool_map_import_srcpkg - Use a multiprocessing.Pool to parallel
-    import source packages
+    """Use a multiprocessing.Pool to parallel import source packages
 
     Arguments:
     num_workers - integer number of worker processes to use
     dry_run - a boolean to indicate a dry-run operation
     pkgnames - a list of string names of source packages
     """
-    with multiprocessing.Pool(processes=num_workers) as pool:
-        results = pool.map(
-            functools.partial(
-                import_srcpkg,
-                dry_run=dry_run,
-            ),
-            pkgnames,
-        )
+    map_args = [
+        functools.partial(
+            import_srcpkg,
+            dry_run=dry_run,
+        ),
+        pkgnames,
+    ]
+
+    if num_workers:
+        with multiprocessing.Pool(processes=num_workers) as pool:
+            results = pool.map(*map_args)
+    else:
+        results = map(*map_args)
 
     return (
         [pkg for pkg, success in results if success],
diff --git a/scripts/source-package-walker.py b/scripts/source-package-walker.py
index 58c3d5a..0dcc2a8 100755
--- a/scripts/source-package-walker.py
+++ b/scripts/source-package-walker.py
@@ -9,18 +9,11 @@
 # Report on successful and failed imports
 
 import argparse
-import bz2
-import itertools
-import gzip
-import lzma
 import os
 import sys
-import urllib.request
 
 import scriptutils
 
-from debian.deb822 import Sources
-
 # We expect to be running from a git repository in master for this
 # script, because the snap's python code is not exposed except within
 # the snap
@@ -35,25 +28,17 @@ sys.path.insert(
     )
 )
 
-from gitubuntu.source_information import GitUbuntuSourceInformation
-
 def import_all_published_sources(
     num_workers,
-    whitelist,
-    blacklist,
-    phasing_main,
-    phasing_universe,
+    pkgnames_to_import,
     dry_run,
 ):
-    """import_all_published_sources - Import all publishes satisfying a
-    {white,black}list and phasing
+    """Import all publishes for a list of source package names
 
     Arguments:
     num_workers - integer number of worker processes to use
-    whitelist - a list of of packages to always import
-    blacklist - a list of packages to never import
-    phasing_main - a integer percentage of all packages in main to import
-    phasing_universe - a integer percentage of all packages in universe to import
+    pkgnames_to_import - list of source package names that should be
+    considered for import
     dry_run - a boolean to indicate a dry-run operation
 
     Returns:
@@ -61,72 +46,14 @@ def import_all_published_sources(
     successfully imported source packages, the second containing the
     names of all source packages that failed to import.
     """
-    serieses = GitUbuntuSourceInformation('ubuntu').active_series_name_list
-    components = ['main', 'universe']
-    pocket_suffixes = ['-proposed', '-updates', '-security', '']
-    compressions = {
-        '.xz': lzma.open,
-        '.bz2': bz2.open,
-        '.gz': gzip.open,
-    }
-
-    base_sources_url = (
-        'http://archive.ubuntu.com/ubuntu/dists/%s%s/%s/source/Sources'
-    )
-    pkgnames = set()
-    for component in components:
-        for series, pocket_suffix in itertools.product(
-            serieses,
-            pocket_suffixes,
-        ):
-            url = base_sources_url % (
-                series,
-                pocket_suffix,
-                component,
-            )
-            for compression, opener in compressions.items():
-                try:
-                    with urllib.request.urlopen(
-                        url + compression
-                    ) as source_url_file:
-                        with opener(source_url_file, mode='r') as sources:
-                            print(url + compression)
-                            for src in Sources.iter_paragraphs(
-                                sources,
-                                use_apt_pkg=False,
-                            ):
-                                pkgname = src['Package']
-                                if scriptutils.should_import_srcpkg(
-                                    pkgname,
-                                    component,
-                                    whitelist,
-                                    blacklist,
-                                    phasing_main,
-                                    phasing_universe,
-                                ):
-                                    pkgnames.add(pkgname)
-                        break
-                except urllib.error.HTTPError:
-                    pass
-            else:
-                print(
-                    "Unable to find any Sources file for component=%s, "
-                    "series=%s, pocket=%s" % (
-                        component,
-                        series,
-                        pocket_suffix,
-                    )
-                )
-                sys.exit(1)
-
-    if not pkgnames:
+    if not pkgnames_to_import:
         print("No relevant publishes found")
         return [], []
 
     return scriptutils.pool_map_import_srcpkg(
         num_workers=num_workers,
         dry_run=dry_run,
-        pkgnames=pkgnames,
+        pkgnames=pkgnames_to_import,
     )
 
 def main(
@@ -138,7 +65,7 @@ def main(
     dry_run=scriptutils.DEFAULTS.dry_run,
     use_whitelist=scriptutils.DEFAULTS.use_whitelist,
 ):
-    """main - Main entry point to the script
+    """Main entry point to the script
 
     Arguments:
     num_workers - integer number of worker threads to use
@@ -184,12 +111,20 @@ def main(
     except (FileNotFoundError, IOError):
         blacklist = list()
 
+    try:
+        pkgnames_to_import = scriptutils.srcpkgs_to_import_list(
+            whitelist,
+            blacklist,
+            phasing_main,
+            phasing_universe,
+        )
+    except RuntimeError as e:
+        print("%s" % e)
+        sys.exit(1)
+
     imported_srcpkgs, failed_srcpkgs = import_all_published_sources(
         num_workers,
-        whitelist,
-        blacklist,
-        phasing_main,
-        phasing_universe,
+        pkgnames_to_import,
         dry_run,
     )
     print(
@@ -206,7 +141,7 @@ def main(
     )
 
 def cli_main():
-    """cli_main - CLI entry point to script
+    """CLI entry point to script
     """
     parser = argparse.ArgumentParser(
         description='Script to import all source packages with phasing',
diff --git a/scripts/update-repository-alias.py b/scripts/update-repository-alias.py
index 5f2c8ae..a2d9fd6 100755
--- a/scripts/update-repository-alias.py
+++ b/scripts/update-repository-alias.py
@@ -27,7 +27,7 @@ sys.path.insert(
 from gitubuntu.source_information import launchpad_login_auth
 
 def update_git_repository(package, dry_run, unset):
-    """update_git_repository - set the default Git Repository on Launchpad for a source package
+    """Set the default Git Repository on Launchpad for a source package
 
     Arguments:
     package - string name of a source package
@@ -93,7 +93,7 @@ def main(
     unset=False,
     use_whitelist=scriptutils.DEFAULTS.use_whitelist,
 ):
-    """main - Main entry point to the script
+    """Main entry point to the script
 
     Set the default Git Repository target on Launchpad for all source
     packages imported to usd-import-team to the usd-import-team
@@ -162,22 +162,22 @@ def main(
     filtered_pkgnames = set()
     for pkgname in main_packages:
         if scriptutils.should_import_srcpkg(
-            pkgname,
-            'main',
-            whitelist,
-            blacklist,
-            phasing_main,
-            phasing_universe,
+            pkgname=pkgname,
+            component='main',
+            whitelist=whitelist,
+            blacklist=blacklist,
+            phasing_main=phasing_main,
+            phasing_universe=phasing_universe,
         ):
             filtered_pkgnames.add(pkgname)
     for pkgname in universe_packages:
         if scriptutils.should_import_srcpkg(
-            pkgname,
-            'universe',
-            whitelist,
-            blacklist,
-            phasing_main,
-            phasing_universe,
+            pkgname=pkgname,
+            component='universe',
+            whitelist=whitelist,
+            blacklist=blacklist,
+            phasing_main=phasing_main,
+            phasing_universe=phasing_universe,
         ):
             filtered_pkgnames.add(pkgname)
 
@@ -196,7 +196,7 @@ def main(
         )
 
 def cli_main():
-    """cli_main - main entry point for CLI
+    """main entry point for CLI
     """
     parser = argparse.ArgumentParser(
         description='Update the default Git repository for imported source packages',
diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml
index 0fbad2b..f8e6e5d 100644
--- a/snap/snapcraft.yaml
+++ b/snap/snapcraft.yaml
@@ -38,9 +38,8 @@ parts:
    gbp:
       plugin: python
       python-version: python3
-      source: https://github.com/nacc/git-buildpackage.git
-      source-type: git
-      source-branch: master-fix-tilde # want to use a tagged release once available
+      source: https://github.com/agx/git-buildpackage/archive/debian/0.9.6.tar.gz
+      source-type: tar
       requirements: requirements.txt
       stage:
          - -usr/bin/py*
@@ -51,8 +50,7 @@ parts:
       after: [git, pristine-tar, python3]
    git:
       plugin: autotools
-      # want to use an upstream release of 2.14.2 or later, once available
-      source: https://github.com/nacc/git/archive/v2.14.1_nacc_fixes.tar.gz
+      source: https://github.com/git/git/archive/v2.14.3.tar.gz
       source-type: tar
       build-packages:
         - libz-dev