lp:~nacc/git-ubuntu

Author: Nish Aravamudan
Author Date: 2018-06-13 03:29:05 UTC

importer: rework algorithm to minimize looping

Basically, our current main importer loop looks like:

for applied in unapplied, applied:
    for dist in debian, ubuntu:
        for new publishes in $dist relative to $applied $dist branches
            import $applied publish
                this updates the branch pointer for the pocket + series
                if dist == 'ubuntu': this also updates the devel pointers

That is a fair amount of branch manipulation that will be discarded (the
series might have multiple publishes). Esp. on the first/reimport import
(worst case).

So instead:

for dist in debian, ubuntu:
    for applied in unapplied, applied:
        for new publishes in $dist relative to $applied $dist branches
            if publish has already been imported: continue
            import $applied publish

update all affected branch pointers,
    where affected is defined by running track of refs we *would* have
    updated in the original algorithm, and we also store the 'last'
    commit that it would have pointed to. This consolidates the branch
    and devel updates into one place as well.
    This also makes the importer algorithm match the spec on some level
    -- update the commit graph by importing only new publishes (which
    would create new import tags and new applied tags, while verifying
    any repeated publish data matches exactly) and then forcibly moving
    branch pointers to where they are "now" in Launchpad. The branch
    pointers are not part of the commit graph, so they change in a
    distinct step.

In my testing, this drops the reimport time for ipsec-tools from ~65
minutes to ~42 minutes consistently. We could probably speed it up more
starting from this base, if need be -- e.g., I'm not sure it makes sense
to to do the import action itself in the loop. Perhaps it's better to
accumulate the set of unique publishes in a set and then iterate that
set in a second loop.

Robie: this does not address your idempotency concern, yet, but I think
the solution is to make changes to importer::import_publishes so that
the order of publication events follows some specific rule. Right now,
it would be if it appears in Debian first, I think.

LP: #1731554

Author: Nish Aravamudan
Author Date: 2018-06-05 16:11:54 UTC

lint: separate check for reconstruct and deconstruct tag

I'm not sure why I wrote the code like this originally, but the intent
was to run every check (even if it doesn't necessarily make any sense to
have a deconstruct tag if there is no a reconstruct tag, in this case).
We will emit all failures as we go through the checks. Currently, we
only check deconstruct if reconstruct exists -- but then we use the
deconstruct tag variable to check against the logical tag. This will
obviously error if the reconstruct tag didn't exist originally.

Also fix a bit of mis-indentation in the code.

LP: #1775041

Author: Nish Aravamudan
Author Date: 2018-06-01 02:37:46 UTC

importer: fix patches-applied error

Used the wrong variable, should be squashed up.

Author: Nish Aravamudan
Author Date: 2018-05-21 21:39:35 UTC

snap: order gnupg2 to be before devscripts

We want to make sure that our gnupg binary and libraries are built
before git-ubuntu. In 48249b21607fdfbb80af9d53e8d0b1375d8778c1, the
local build reversion was reverted in turn, but the ordering was missed.
This leads to inconsistent builds.

Author: Nish Aravamudan
Author Date: 2018-05-18 23:28:33 UTC

setup.py: lazr.restfulclient bump

Author: Nish Aravamudan
Author Date: 2018-04-27 16:10:00 UTC

git_repository: add close method and invoke it in a try/finally from importer

Push the ability to remove a repository's directory into the repository
itself and make caller's responsible for the removal.

Wrap importer.main in try/finally, where the finally clause calls
repo.close() directly. This removes our use of atexit, and ensures that
whether import succeeds or fails, the repository object we may have
created is cleaned up per the arguments.

LP: #1765219

Author: Nish Aravamudan
Author Date: 2018-04-17 22:01:31 UTC

find_ubuntu_merge_base: fix the code for the new imported Git layout

We no longer have a publishing parent to use as a Git merge-base, so we
have to rely on what we know about Ubuntu merges. We look for the
nearest (searching the changelog backwards) Debian version that has been
imported. That should provide us a delta point.

Note this also changes a necessary step in the merge workflow, which
needs to be done in lockstep on the wiki with landing this change.

LP: #1734364

Author: Nish Aravamudan
Author Date: 2018-04-20 22:50:59 UTC

importer: integrate Approved MPs as rich history sources

Author: Nish Aravamudan
Author Date: 2018-04-16 21:13:44 UTC

test_importer: reorder cases numerically

Author: Nish Aravamudan
Author Date: 2018-04-12 18:01:21 UTC

importer: do not shadow imported methods

Our local variables were shadowing the methods imported from
git_repository and this led to (future) test failures.

No functional changes.

Author: Nish Aravamudan
Author Date: 2018-04-10 23:03:25 UTC

importer: abstract out import_{un,}applied_dsc

No functional changes.

Author: Nish Aravamudan
Author Date: 2018-04-10 22:50:12 UTC

importer: refactor _commit_import to return the commit hash

Have the caller do the tagging and branch manipulation.

This should not be any functional change and allows for future fixes to
the algorithm.

Author: Nish Aravamudan
Author Date: 2018-04-10 22:47:34 UTC

importer._commit_import: use get_import_tag_msg()

No functional change.

Author: Nish Aravamudan
Author Date: 2018-04-10 20:37:16 UTC

importer: abstract out get_changelog_parent_commit

This was duplicated code between the import_{un,}applied_spi and needs
to be abstracted out anyways for future changes.

Add some unit tests. While not complete in and of themselves, the code
will be further tested with follow-on unit tests for the importer
methods.

There should be no functional change.

Author: Nish Aravamudan
Author Date: 2018-04-06 17:40:13 UTC

importer: do not pass spi to _commit_import

No functional changes, purely refactoring to allow for code reuse.

Author: Nish Aravamudan
Author Date: 2018-03-30 21:28:52 UTC

importer: remove import_patches_unapplied_tree

This is a trival wrapper around dsc_to_tree_hash that does
not buy us much.

Author: Nish Aravamudan
Author Date: 2018-03-28 23:00:05 UTC

importer: add create_{import,applied}_tag API

Currently unused, but will be. Add tests.

LP: #1755247

Author: Nish Aravamudan
Author Date: 2018-03-30 21:39:18 UTC

importer: stub out and test tag and commit message functions

Author: Nish Aravamudan
Author Date: 2018-03-28 19:23:34 UTC

source_builder: fix changelog_versions handling

Add unit tests to catch the mistaken code.

Author: Nish Aravamudan
Author Date: 2018-03-09 17:01:23 UTC

test_fixtures: for simplicity chdir to the repo local directory

This ensures the users of the fixtures do not need to do it
for themselves. This is only strictly required for importer testing,
which assumes it is in the correct directory.

Author: Nish Aravamudan
Author Date: 2018-03-08 22:44:38 UTC

source_builder: add file_contents attribute to SourceSpec

This allows for arbitrary file creation in the source package, to
manipulate the Git tree contents.

Also add a unit test.

Author: Nish Aravamudan
Author Date: 2018-03-06 20:50:23 UTC

git_repository: drop ensure_importer_branches_exist

Much like 1d658869b56b ("Create dsc branch on first use"), create the
pristine-tar necessary branches on first use, which all occurs within
the context manager in git_repository.py.

This should only result in one functional change, but should ensure
there are no side-effects of simply instantiating a GitUbuntuRepository
object and the helper function is no longer necessary.

The functional change is that when there is only publishing history in
one of Debian or Ubuntu, there will only be pristine-tar related
branches for the specific distribution. This requires, in turn, ensuring
that pristine_tar_list does not create any branches, so that `git ubuntu
build`'s usage continues to work as expected.

It is a separate issue as to whether these branches should exist in the
first place or not.

LP 1733895

---

Author: Nish Aravamudan
Author Date: 2018-03-30 21:12:37 UTC

repo_builder.SourceTree: allow for patches to be applied

Author: Nish Aravamudan
Author Date: 2018-03-26 22:29:05 UTC

repo_builder: add Repo.copy

This will allow for importer tests to start with a given
Repo object, specify the modifications to it, and then
generate the expected result structurally. The original
and modified-copy can then be compared with the to-be-written
Repo comparator.

Author: Nish Aravamudan
Author Date: 2018-03-28 22:28:06 UTC

importer: fix algorithm to match spec

This also fixes a logic issue with the changelog parent.

LP: #1761331
LP: #1754194
LP: #1754507
LP: #1754706
LP: #1734883
LP: #1761332

Author: Nish Aravamudan
Author Date: 2018-03-26 17:45:48 UTC

repo_builder: add Branch and Tag classes

These classes are used to extend the Repo class to include their
corresponding refs.

This also requires changing when replace_placeholders is called for
Repo() objects. We want to be able to run tests against Repo() objects
which might be otherwise invalid, so we need to defer the
replace_placeholders() call until write().

Author: Nish Aravamudan
Author Date: 2018-03-28 19:23:34 UTC

source_builder: fix changelog_versions handling

Add unit tests to catch the mistaken code.

Author: Nish Aravamudan
Author Date: 2018-03-27 17:00:54 UTC

repo_comparator: add basic equals() method

This method currently supports a simple check of specified refs in the
two repositories.

Author: Nish Aravamudan
Author Date: 2018-03-28 22:41:14 UTC

snap: also do not stage lib/*/gpg-error*

This is related to 4e07a28386140ba2d403a173684530af06b8017f ("snap:
remove duplicate libgpg-error and libgcrypt so files") and adds some
missed paths.

Author: Nish Aravamudan
Author Date: 2018-03-15 22:27:06 UTC

source-package-walker: add --old-phasing-{main,universe} arguments

These allow us to control the linear package set in a more
fine-tuned way when we are bumping the phasing.

Author: Nish Aravamudan
Author Date: 2018-03-06 18:13:25 UTC

import: make upload tags first-class

It is possible for us to modify our algorithm such that upload tags simply
get used directly, rather than as a parent. Effectively, when we
find a matching upload tag, we should see if it has the changelog
parent commit as ancestor (already done as a safety check),
and if so, then just repoint the import tag there.

Fully untested, but for discussion for Robie.

LP: #1734883

Author: Nish Aravamudan
Author Date: 2018-03-06 21:36:00 UTC

add test for case 1 in MP

Author: Nish Aravamudan
Author Date: 2018-03-06 18:10:49 UTC

import: check if an orphan tag matches the imported tree in all cases

If we attempt to use an import tag, but the tree does not match, we will
orphan, but it's possible this is the *second* time seeing the same
publish event, in which case we should possibly use the orphan tag's
tree this time.

LP: #1753800

Author: Nish Aravamudan
Author Date: 2017-12-04 09:17:09 UTC

import: handle orphan tags as changelog parents

It is possible that any given version might be an orphan import (due not
finding any changelog parents). When this happens, we want to ensure
that when importing the same (to-be-orphaned) publication record for a
different series, we simply use the same logic we have now for import
tags. That is, when determining what to do with a given published
version:

1) If there is a corresponding import tag for version
        1a) If the tag's treeish matches, just move the branch
        1b) If the tag's treeish does not match, orphan this publish
2) If there is a corresponding orphan tag for version
        1a) If the tag's treeish matches, just move the branch
        1b) If the tag's treeish does not match, fail (unhandled still)
3) If there is a corresponding upload tag for version
        1a) If the tag's treeish matches, incorporate the rich history
        1b) If the tag's treeish does not match, emit a warning but
                ignore this tag

2) is inserted by these changes.

A similar change should be done in the lookup of the unapplied changelog
parent itself, which previously only searched for import tags, but should
search for import or orphan tags. It is not incorrect to use an orphan
tag as a changelog parent, it just implies there is a break in the
publishing record relative to the changelog.

LP: #1733856

Author: Nish Aravamudan
Author Date: 2018-01-27 01:05:51 UTC

scripts/scriputils.py: treat Debian-only publishes as universe

For the purposes of phasing, we only want to consider Ubuntu publishing
data directly. But if something is only present in Debian, we do not
have any component data to use; presumably it is lower priority for
Ubuntu, though, so phase it like universe.

Author: Nish Aravamudan
Author Date: 2018-02-28 18:54:02 UTC

self-test: fix coverage

When running the self-test app from a directory that happens to contain
a gitubuntu Python package (e.g., the Git repository), we get a false
empty coverage report because the local pacakge is used to check
coverage.

Author: Nish Aravamudan
Author Date: 2018-02-28 16:59:27 UTC

apt_repo_test: gpg should call the snapped gpg-agent

This ensures self-compatibility and ideally not affecting the running
user's gpg-agent.

The order of arguments to gpg matters, hence where this option is added.

Author: Nish Aravamudan
Author Date: 2018-02-01 04:48:10 UTC

snap: add scripts to the snap

Author: Nish Aravamudan
Author Date: 2018-01-11 05:29:24 UTC

scriptutils.pool_map_import_srcpkg: properly handle num_workers=0

Do not create a pool, just run map().

Author: Nish Aravamudan
Author Date: 2018-02-23 18:04:21 UTC

snap: use people.canonical.com cache of tarballs

Periodically we see CI or snap build failures due to Git
hangups or timeouts. We do not need rich history to build,
so just create a `git archive` (instructions in the yaml)
which we put in a well-defined public location and use
those tarballs.

This should speed up building without any changes.

Author: Nish Aravamudan
Author Date: 2018-01-31 01:05:10 UTC

Will be squashed up to 515e59a3

Author: Nish Aravamudan
Author Date: 2018-01-30 19:21:11 UTC

gitubuntu/: debian.debian_support's Version.upstream version validation is wrong

For native Ubuntu packages with delta (e.g., php-defaults 57ubuntu1),
the underlying Version object parses the upstream_version as '57ubuntu1'
instead of '57'. This is because the regex used by that object is
correct for Debian packages only.

To fix this, we need to intercept any calls to that layer and check if
the resulting upstream_version contains 'ubuntu' and instead cut off the
suffix (ubuntu*).

Author: Nish Aravamudan
Author Date: 2018-01-10 21:44:49 UTC

gitubuntu: add API to call quilt and gbp

We want our quilt and gbp invocations to be reproducible when the
importer is run, or they might affect the resulting hashes. So use
application-specific logic via parameters (quilt) or environment (gbp)
to not load any configuration files.

LP: #1734137

Author: Nish Aravamudan
Author Date: 2018-01-30 00:42:08 UTC

snap: move to released version of Git

We were leveraging a bugfix for 2.14.1, which is now available upstream.

Author: Nish Aravamudan
Author Date: 2018-01-09 19:21:16 UTC

build: unset SNAP in lxc's env

Ever since lxc gained snap support, it assumed that if SNAP is set in
it's environment that lxc itself was running as a SNAP. This is not
necessarily the case when git-ubuntu itself is a classic snap (and thus
has SNAP set in its environment) but is calling lxc in the host. Unset
SNAP manually on lxc run() calls.

LP: #1741949

Author: Nish Aravamudan
Author Date: 2018-01-12 01:29:31 UTC

git_repository: modify use of TemporaryDirectory to avoid crosstalk

Currently, no matter where the cwd is, all temporary directories are
placed in /tmp. This means when we use the pardir of the temporary
directory, we end up using /tmp and that leads to crosstalk between code
paths and binaries.

We really want each temporary directory to be an isolated env, so use
nested TemporaryDirectories.

LP: #1734137.

Author: Nish Aravamudan
Author Date: 2018-01-11 00:47:07 UTC

properly append the correct path

Will be squashed up.

Author: Nish Aravamudan
Author Date: 2018-01-09 20:20:18 UTC

git_repository: handle debian removal exceptions in one block

Will be squashed up.

Author: Nish Aravamudan
Author Date: 2018-01-09 20:17:18 UTC

add launchpad bug reference in comment

Author: Nish Aravamudan
Author Date: 2018-01-06 00:05:12 UTC

gitubuntu: add basic DSC tests

I started with tests for the bug I just fixed. I wanted to test
before/after conditions, but since the actual logic of the 'before' was
wrong (it returned None which was the API, but then the API was
incorrectly used), it felt more natural to just test the new API
specification.

Author: Nish Aravamudan
Author Date: 2018-01-05 23:16:53 UTC

build: orig tarballs do not exist for native source packages

Do not try to push orig tarballs if they do not exist. Found while
investigating LP #1741472.

Author: Nish Aravamudan
Author Date: 2018-01-03 22:46:39 UTC

scripts/scriptutils: refactor srcpkgs_to_import_list to reduce code duplication

We iterate all of Ubuntu main, then all of Ubuntu universe, then Debian,
treating Debian source packages not present in Ubuntu as if they were in
universe.

I would prefer if the Ubuntu iterations were using one
components=['main', 'universe'], but then the caller would not know in
which component the given source package lives. In theory, we can use
src['directory'], but that seems sort of fragile to parse?

Will possibly be squashed up to c94f8a649dfe1e9ab5e4e90d8e722e4663aca22b
("scripts/scriputils.py: treat Debian-only publishes as universe"), or
left on its own for clarity.

Author: Nish Aravamudan
Author Date: 2017-12-04 15:20:58 UTC

GitUbuntuRepository._maybe_quiltify_tree_hash: correctly restore state

When we undo our potential quiltification, so that all we are left with
is a quilt patch, a series change, and a revert of the previously
un-quilted upstream changes, we were checking out to the original commit
for all files. This has the unintended consequence of discarding any
modifications staged for commit (new files, like
d/p/git-ubuntu-fixup.patch are preserved), specifically our changes to
d/p/series. Instead, just follow the `git status` advice and discard the
working tree changes that are not staged for commit, as we immediately
preceding this, stage all of d/p for commit.

LP: #1734371

Author: Nish Aravamudan
Author Date: 2017-12-04 09:07:41 UTC

lint: properly pass arguments to do_lint by keyword

They were passed in the wrong order before.

LP: #1735744.

Author: Nish Aravamudan
Author Date: 2017-11-17 01:36:35 UTC

importer: more consolidation

Author: Nish Aravamudan
Author Date: 2017-11-08 22:44:20 UTC

git ubuntu import: use a dbm cache to store importer progress

With the recent changes to the importer algorithm, we no longer can
identify a Launchpad publishing event by the commit information in the
repository -- a given SourcePackageRelease (source package name and
version) is only imported once (presuming all future publishes of the
same name and version match exactly). We rely on this in
launchpad_versions_published_after, which iterates the Launchpad data
backwards until we either match the commit data for a branch head, or
see Launchpad data from before one of the branch heads.

Change the importer code to take a --db-cache argument as a directory
containing a DBM cache for debian and ubuntu (this is needed, because
DBM are single-leve string-indexed string storage databases). Lookup the
source package name in the relevant cache before we lookup Launchpad
data, to obtain the last SPPHR used. Store the latest processed SPPHR
after iterating the Launchpad data.

Also update the scripts to support passing a persistent/consistent value
to the import for the cache.

LP: #1730734

Author: Nish Aravamudan
Author Date: 2017-11-07 16:01:53 UTC

importer: rework and move devel pointer moving

After discussion with Robie on IRC, we decided that 0f3c943054ab
("import: drop publishing parent functionality") and 5aa33fa08078 ("Also
reset devel heads") introduced a regression in the semantics of the
devel pointers.

Before those changes, the devel pointers were merged up so as to be
fast-forwarding, as we imported publication entries, if the publication
entry was newer than the current devel pointer. In other words, the
devel pointers were part of the commit graph itself.

After those changes, the devel pointers are more like symbolic
references, describing meta-state about the commit graph, rather than
integral to the graph itself:

A given series devel branch, after a successful import, points to the
latest publication record in a given series.

The ubuntu/devel branch, after a successful import, points to the latest
series devel branch.

Given these 'rules', we can stop updating the devel pointers in the main
import loop and just do so after we are done importing. All series
branch pointers are updated, which is unnecessary but will generally be
a no-op. This allows the method to not be aware of reimporting or not.

LP: #1730655

Fixes: 0f3c943054ab ("import: drop publishing parent functionality")
Fixes: 5aa33fa08078 ("Also reset devel heads")

Author: Nish Aravamudan
Author Date: 2017-11-02 20:49:34 UTC

source-package-walker.py: add script to linearly walk all source packages

It uses the same blacklist, whitelist and phasing that the other scripts
do.

Author: Nish Aravamudan
Author Date: 2017-11-10 17:43:13 UTC

build: symlink cache hits into parent directory

The other fetch* operations will result in either the relevant files or
symlinks to them being present in the parent directory. However, the
cache path does not do that, so if a deletes relevant files from .., but
not the cache, the build will succeed, but there will be no orig in the
parent directory. This is not in and of itself necessarily an error, as
the orig tarball is only needed for further use on merges and upstream
updates, but it is counter-intuitive and inconsistent.

LP: #1731513

Author: Nish Aravamudan
Author Date: 2017-11-09 17:12:05 UTC

git_repository._add_remote_by_fetch_url: force update remote-tracking refs

With the recent changes to the importer algorithm (e.g., dropping of
publishing parents, force moving of branch refs), it is no longer
guaranteed that the series branches are fast-forwarding. However, our
default refspecs installed by `git ubuntu clone` will error out when
they are not. Expect non-fastfowarding updates to the branches and tags,
as we implicitly want to treat the remote as authoritative for those
objects.

LP: #1731299

Author: Nish Aravamudan
Author Date: 2017-11-08 17:34:44 UTC

git_repository.Changelog: dpkg-parsechangelog returns 'unknown' in some cases

When a changelog version is unparseable, under RFC822, the return value
is 'unknown'. We might be able to obtain a valid value with the
debian.Changelog class, but we will fail to match the old output.

Note, we would obtain a valid and matching value with the manual shell
parsing, but we won't fallback to the latter because 'unknown' is not an
error condition.

Special-case 'unknown' from the dpkg-parsechangelog output as not being
failure when the old and new output does not match.

Add a unit-test for this case, as well.

LP: #1730780

Author: Nish Aravamudan
Author Date: 2017-11-07 23:42:33 UTC

scripts: update based upon review commennts

This will get squashed up.

Author: Nish Aravamudan
Author Date: 2017-11-03 16:09:44 UTC

build/buildsource: make rem_args a set

This allows for two broad changes:
1) We do not need to check for duplicates from user input.
2) We can consolidate the two cli_main methods almost completely.

This also fixes a bug with --for-merge not setting -sa, if extra
arguments are passed.

Author: Robie Basak
Author Date: 2017-10-26 16:44:55 UTC

Also reset devel heads

We have decided to reset the devel heads instead of generating merge
commits for them.

Author: Nish Aravamudan
Author Date: 2017-10-04 22:48:56 UTC

import: add hidden flag to allow patches-applied failures

It is possible that modern tooling cannot reproduce a historic source
package. This happens, for instance, when patches apply with fuzz and
that was allowed in the past but is no longer allowed. Other cases may
exist. Rather than attempting to resolve all such historical cases with
source patching, add a flag that a developer can use to skip such
failures, when encountered, and continue on. The history of the source
package should be comprehensible, albeit with holes for the failed
patches-applied imports.

Author: Nish Aravamudan
Author Date: 2017-10-04 18:50:11 UTC

import: drop publishing parent functionality

Author: Nish Aravamudan
Author Date: 2017-10-17 23:02:37 UTC

remote: add support for adding Debian remote

Parse pkg/ubuntu/devel:debian/control for X-Debian-Vcs-Git and, if
X-Debian-Vcs-Git is not found, Vcs-Git, when the --debian flag is given
to add. This also requires us to make --user a flag option, in order to
properly parse the arguments. One of --debian or --user is required.

LP: #1719707

Author: Nish Aravamudan
Author Date: 2017-10-05 00:40:27 UTC

import: make dpkg-source not so verbose as it can often fail

Author: Nish Aravamudan
Author Date: 2017-09-15 17:07:14 UTC

build: use changelog to determine source by default

Unless --for-merge is specified, in which case Debian is used
preferentially.

Author: Nish Aravamudan
Author Date: 2017-10-19 22:44:13 UTC

build: push derive_* calls into else (if not image)

This will be squashed up.

Author: Nish Aravamudan
Author Date: 2017-10-16 21:57:47 UTC

snap: do not snap pytho self-tests

This trims almost 10M (close to 10%) of the snap image.

Author: Nish Aravamudan
Author Date: 2017-10-04 14:45:48 UTC

source_info: fix typo s/_ddi/_udi/

Author: Nish Aravamudan
Author Date: 2017-09-20 19:10:51 UTC

doc/gitubuntu-completion.sh: update to be current

Author: Nish Aravamudan
Author Date: 2017-09-29 19:13:54 UTC

snap: build xdelta from source

Author: Nish Aravamudan
Author Date: 2017-09-20 23:23:14 UTC

git-ubuntu: respect GIT_DIR environment variable

We currently incorrectly assume the git directory is .git/ when
it is specifiably with either GIT_DIR or git --git-dir=. Luckily
the latter sets the former, so we can just use the environment
variable and default to .git if it is not set.

LP: #1718469

Author: Nish Aravamudan
Author Date: 2017-09-18 23:14:44 UTC

queue: ubuntu/<series>-devel branches are in one of two places

They either are in a local importer/ubuntu/<series>-devel branch or in a
remote-tracking pkg/ubuntu/<series>-devel branch.

Look in both places via a helper function.

LP: #1699541

Author: Nish Aravamudan
Author Date: 2017-09-12 23:41:07 UTC

source_information: add wrapper classes for {Debian,Ubuntu}SourcePackage

By default, ubuntutools.archive.{Debian,Ubuntu}SourcePackage use
internal masters and mirrors lists to download (see
ubuntutools.archive.SourcePackage::_source_urls). These lists are not
hosted on Launchpad, and so add additional external dependencies for our
tooling. Create trivial wrapper classes that instead empty the masters
and mirrors lists after the super-constructor runs.

LP: #1718019

Author: Nish Aravamudan
Author Date: 2017-09-16 02:48:01 UTC

lint: check if a bugfix is targetted at the correct branch

LP: #1711174

Author: Nish Aravamudan
Author Date: 2017-09-14 23:17:52 UTC

build: use -v for building merges

LP: #1713530

Author: Nish Aravamudan
Author Date: 2017-09-14 18:23:21 UTC

git_repository::commit_tree_hash: do not gpg sign import commits

We are importing other user publishes, which implies we do not have
signing rights for their keys. Do not attempt to sign import commits,
which only is an issue if a local config globally specifies it should
be done.

LP: #1705542

Author: Nish Aravamudan
Author Date: 2017-09-28 16:40:46 UTC

source_information: refactor pull() to be more pythonic

Author: Nish Aravamudan
Author Date: 2017-09-25 19:15:47 UTC

snap: rename part from main to git-ubuntu

With this change, the snap now builds successfully on Xenial, and runs
on Xenial and Artful.

LP: #1718499

Author: Nish Aravamudan
Author Date: 2017-09-18 20:36:46 UTC

switch to using shutil.which

Author: Nish Aravamudan
Author Date: 2017-09-18 20:05:51 UTC

git_repository: use a contextmanager for worktrees

This will be squasehd up

Author: Nish Aravamudan
Author Date: 2017-09-15 20:53:13 UTC

buildsource: try and detect if for-merge

Author: Nish Aravamudan
Author Date: 2017-09-13 20:24:33 UTC

lint: drop unnecessary local variable

Author: Nish Aravamudan
Author Date: 2017-09-13 17:43:16 UTC

review: drop unused sys

Author: Nish Aravamudan
Author Date: 2017-09-12 23:41:07 UTC

source_information: add wrapper classes for {Debian,Ubuntu}SourcePackage

By default, ubuntutools.archive.{Debian,Ubuntu}SourcePackage use
internal masters and mirrors lists to download (see
ubuntutools.archive.SourcePackage::_source_urls). These lists are not
hosted on Launchpad, and so add additional external dependencies for our
tooling. Create trivial wrapper classes that instead empty the masters
and mirrors lists after the super-constructor runs.

LP: #1659915

Author: Nish Aravamudan
Author Date: 2017-09-12 17:21:40 UTC

versioning: add support for non-series information in the version string

libvirt uploads (SRUs) have versions like 1.2.2-0ubuntu13.1.23 which break
our parsing, which assumes anything between ubuntu_maj and ubuntu_min
is either a series or invalid. Extend our parsing to allow for an "other"
field, which is set when the substring between maj and min cannot be
parsed as a version.

LP: #1716737

Author: Nish Aravamudan
Author Date: 2017-09-08 23:22:44 UTC

remote: rework so that main is a more useful function

This will be folded up with the other changes before merging.

Author: Nish Aravamudan
Author Date: 2017-09-11 21:06:29 UTC

snap: main part can be obtained from .

The jenkins job now clones a repo and builds a snap from
potential branches. This does not quite work if our yaml
always builds master. Have it use the local dir instead.

Author: Nish Aravamudan
Author Date: 2017-08-25 23:09:16 UTC

source_information: create working directory if needed

For the purposes of the cache from `git ubuntu build`, the cache may not
yet exist, so it needs to be created before pull() is invoked, or we get
a FileNotFoundError.

Author: Nish Aravamudan
Author Date: 2017-08-23 23:56:51 UTC

build: switch our implementation to the new algorithm

Using fetch_orig, iterate over a well-defined list of OrigSearchEntry
based upon our arguments.

LP: #1708691

Author: Nish Aravamudan
Author Date: 2017-07-19 17:20:56 UTC

git ubuntu import: update pristine-tar imports to use common code

1) Modify orig_imported() to take a set of paths to tarballs and a
distribution name. It will see if the given tarballs have been imported
identically to the current repository for the given distribution.

2) Use the gitubuntu/dsc.py accessors for the orig tarball and component
tarballs.

Author: Nish Aravamudan
Author Date: 2017-08-23 23:44:05 UTC

build: add derive_orig_search_list

This will take our args namespace and convert into an appropriate list
of OrigSearchEntrys for use by fetch_orig.

Author: Nish Aravamudan
Author Date: 2017-08-23 23:18:32 UTC

build: add unique_everseen helper function with test

Author: Nish Aravamudan
Author Date: 2017-08-23 23:56:51 UTC

build: switch our implementation to the new algorithm

Using fetch_orig, iterate over a well-defined list of OrigSearchEntry
based upon our arguments.

LP: #1708691

Author: Nish Aravamudan
Author Date: 2017-07-19 17:20:56 UTC

git ubuntu import: update pristine-tar imports to use common code

1) Modify orig_imported() to take a set of paths to tarballs and a
distribution name. It will see if the given tarballs have been imported
identically to the current repository for the given distribution.

2) Use the gitubuntu/dsc.py accessors for the orig tarball and component
tarballs.

Author: Nish Aravamudan
Author Date: 2017-08-23 23:55:02 UTC

build: add derive_orig_search_list

This will take our args namespace and convert into an appropriate list
of OrigSearchEntrys for use by fetch_orig.

Author: Nish Aravamudan
Author Date: 2017-08-23 23:18:32 UTC

build: add unique_everseen helper function with test