lp:~racb/usd-importer

Owned by Robie Basak
Get this repository:
git clone https://git.launchpad.net/~racb/usd-importer
Only Robie Basak can upload to this repository. If you are Robie Basak please log in for upload directions.

Branches

Name Last Modified Last Commit
fix_derive_codename_from_series.3 2019-08-05 13:07:30 UTC 2019-08-05
fixup! source_information: Fix ubuntu series

Author: Robie Basak
Author Date: 2019-08-05 12:51:19 UTC

fixup! source_information: Fix ubuntu series

Drop xfail now that the bug is fixed

importer-add-tests 2019-05-24 11:20:14 UTC 2019-05-24
wip: review comments and notes

Author: Robie Basak
Author Date: 2019-05-17 10:35:14 UTC

wip: review comments and notes

blacklist-wheezy 2019-04-25 16:42:23 UTC 2019-04-25
Blacklist wheezy as an active series

Author: Robie Basak
Author Date: 2019-04-25 12:25:28 UTC

Blacklist wheezy as an active series

optimise-api-calls 2019-04-25 09:16:36 UTC 2019-04-25
Cache some Launchpad object fetches

Author: Robie Basak
Author Date: 2019-04-23 11:45:33 UTC

Cache some Launchpad object fetches

As we iterate through Launchpad's source_package_publication_history
records, we keep looking up their distro_series objects via
GitUbuntuSourcePackageInformation.distribution.name and suchlike. This
results in a very large number of API calls to fetch the same
distro_series object from Launchpad every time, causing a severe
degradation in performance.

Since distribution_link is the same every time, we can instead key a
cache on this, eliminating the excessive number of API calls. The same
applies to some other similar calls.

As far as I can tell we've always made this excessive number of calls
here, but in practice something recently changed in the amount of time
these calls are taking, causing import performance to drop virtually to
a standstill. This optimisation restores reasonable performance again.

This introduces a new dependency on cachetools, which seems to be the
cleanest caching solution without having to reimplement it inside
git-ubuntu. python3-cachetools is available in Xenial and the module
seems to be stable upstream, so I hope this addition will not cause us
too much trouble.

unpin-pylint 2018-11-29 15:37:27 UTC 2018-11-29
Friendlier self-test output

Author: Robie Basak
Author Date: 2018-11-29 15:31:37 UTC

Friendlier self-test output

If we change directory to the location of the gitubuntu Python module
first, the commands to pylint and pytest are simpler and additionally
has the advantage that the output from pytest is nicer.

master 2018-11-08 11:51:47 UTC 2018-11-08
Merge remote-tracking branch 'smoser/fix/1801168-build-when-no-patches'

Author: Robie Basak
Author Date: 2018-11-08 11:51:47 UTC

Merge remote-tracking branch 'smoser/fix/1801168-build-when-no-patches'

httplib2-re-slash 2018-11-08 11:51:47 UTC 2018-11-08
Merge remote-tracking branch 'smoser/fix/1801168-build-when-no-patches'

Author: Robie Basak
Author Date: 2018-11-08 11:51:47 UTC

Merge remote-tracking branch 'smoser/fix/1801168-build-when-no-patches'

scriptutils-dry-run 2018-11-06 11:34:48 UTC 2018-11-06
Drop use of dry_run that is no longer defined

Author: Robie Basak
Author Date: 2018-11-06 11:34:48 UTC

Drop use of dry_run that is no longer defined

This fixes a regression caused by
d17507e57d60f744540088d6a1d4f56a80b92970.

cosmic-keyring 2018-11-05 16:29:28 UTC 2018-11-05
Bump ubuntu-keyring keyrings used in the snap

Author: Robie Basak
Author Date: 2018-11-05 14:41:49 UTC

Bump ubuntu-keyring keyrings used in the snap

LP: #1801725

importer-service-db 2018-11-01 16:55:43 UTC 2018-11-01
Mark "running" imports as errored on restart

Author: Robie Basak
Author Date: 2018-10-30 16:56:18 UTC

Mark "running" imports as errored on restart

fix-keyring-pypi-breakage 2018-11-01 14:17:07 UTC 2018-11-01
Pin keyring version to work around regression

Author: Robie Basak
Author Date: 2018-11-01 13:02:28 UTC

Pin keyring version to work around regression

importer-service-datadir 2018-10-23 12:28:41 UTC 2018-10-23
Use persistent state directory

Author: Robie Basak
Author Date: 2018-10-19 12:58:44 UTC

Use persistent state directory

Move the directory in which we keep persistent state into the one
specified by --data-directory, defaulting to /var/local/git-ubuntu.

This directory must exist.

This moves the previously used directory /tmp into somewhere that can
be expected to persist. This impacts git-ubuntu importer operators,
since they now need to make sure that a persistent area is ready before
use.

This applies to the importer service only. Users of the "git-ubuntu" CLI
tool, as opposed to the operators of the importer, are not affected.

gawk-readline-fix 2018-10-12 09:44:19 UTC 2018-10-12
Fix call into core snap awk binary

Author: Robie Basak
Author Date: 2018-10-12 09:44:19 UTC

Fix call into core snap awk binary

The core snap's loader is tied to its libc. usr/bin/awk inside the core
snap refers to /lib64/ld-linux-x86-64.so.2, which is therefore incorrect
when called from a classic snap.

Add a wrapper for awk which calls awk in the core snap correctly,
through its own loader.

Separately, we also need to set LD_LIBRARY_PATH so that the loader can
find the shared libraries supplied by the core snap that satisfy its own
dependencies, rather than look at the host system or into our own snap
which may not have them. This was the original bug. Fixing this revealed
the problem above. Both issues are resolved by wrapping this call.

This does raise some questions such as what it means for a classic snap
to be making use of a core snap's binaries, and whether this is an
appropriate thing to do given that the core snap's binaries all specify
the "wrong" loader path in the classic case since the core snap won't be
mounted at the root of the filesystem. This might need some refactoring
of how the git-ubuntu snap works in the future, but for now we can fix
the awk case to fix the current regression in master.

With thanks to Michael Vogt.

LP: #1796017

gawk-readline-fix-wip 2018-10-03 12:19:32 UTC 2018-10-03
Add awk integration test

Author: Robie Basak
Author Date: 2018-10-02 16:58:36 UTC

Add awk integration test

keyring-backend-fix 2018-09-28 15:47:00 UTC 2018-09-28
Drop keyrings.alt version restriction

Author: Robie Basak
Author Date: 2018-09-27 15:06:44 UTC

Drop keyrings.alt version restriction

The launchpadlib part requires "keyring" with no version restriction. By
requiring a particular version here, we're relying that some newest
version of keyring will work with this particular keyrings.alt, which
fails with keyring 15.0.0 vs. keyrings.alt 2.3. Since launchpadlib
already has no version restriction, do the same for keyrings.alt. Then
at least if "pip3 install" works, so should the snap.

Alternatively we could have added a version restriction to the
launchpadlib part, but at the moment that pattern isn't used anywhere,
and the requirements are taken from launchpadlib's setup.py with no
patching. If we want to restrict versions some other way, then we'll
need a major reworking of how the snap is put together.

Now the test checking that keyring can find a functional backend should
always pass.

LP: #1794041

improve-tag-dirty-tree-detection 2018-09-20 16:19:56 UTC 2018-09-20
tag: improve dirty tree detection

Author: Robie Basak
Author Date: 2018-09-20 16:18:48 UTC

tag: improve dirty tree detection

If raw_repo.status() returns entries that are GIT_STATUS_IGNORED then
they don't represent the tree being dirty, so ignore this case. This
fixes a case where "git ubuntu tag" refuses to work even when a working
tree is clean.

Related bugs:

https://bugs.launchpad.net/usd-importer/+bug/1687940
https://bugs.launchpad.net/usd-importer/+bug/1739000

proposed 2018-07-10 12:08:46 UTC 2018-07-10
Merge branch 'changelog-assertions'

Author: Robie Basak
Author Date: 2018-07-10 12:01:49 UTC

Merge branch 'changelog-assertions'

Some fixups were needed for the merge. The tests move from
git_repository.py to git_repository_test.py, one import gets removed
from git_repository.py (no longer needed from git_repository_test.py),
and a couple of references in new test code get prepended by "target.".

test-structure-consistency 2018-06-14 12:52:26 UTC 2018-06-14
Rename test files

Author: Robie Basak
Author Date: 2018-06-14 12:52:26 UTC

Rename test files

This is a style change: we're moving to keeping tests in foo_test.py.

changelog-assertions 2018-06-04 14:06:07 UTC 2018-06-04
Fix edge case changelog assertion check

Author: Robie Basak
Author Date: 2018-06-01 15:57:01 UTC

Fix edge case changelog assertion check

We should also check the length since zip()'s output is the shortest of
its two inputs. This fixes the associated test.

I've also simplified the associated error path code, since I think it's
sufficient to just raise an exception and not swallow it higher up the
stack. Problems with changelog parsing can be reproduced later easily
enough.

importer-cleanup-tmpdir-always 2018-05-21 10:08:17 UTC 2018-05-21
Drop inferred log message

Author: Robie Basak
Author Date: 2018-05-15 16:48:07 UTC

Drop inferred log message

This message was left behind to avoid a functional change in a previous
refactoring.

Unfortunately it has now become quite disconnected from the
implementation of logic that it is logging. It's at risk of becoming
wrong in the future if the implementation elsewhere is intentionally
changed.

Since it doesn't seem that important, drop it to avoid this happening.

snap-lazr-path 2018-05-16 13:11:56 UTC 2018-05-16
snap: fix lazr.restfulclient stage path

Author: Robie Basak
Author Date: 2018-05-16 13:11:56 UTC

snap: fix lazr.restfulclient stage path

The old path was incorrect, but snapcraft somehow previously worked with
it, perhaps due to a regexp match or similar. The current snapcraft does
not. This change fixes the current snapcraft build failure.

Thanks to Nish for identifying the root cause and fix.

snap-vendoring-updates 2018-04-27 11:01:34 UTC 2018-04-27
Document external dependency management policy

Author: Robie Basak
Author Date: 2018-04-27 11:01:34 UTC

Document external dependency management policy

External dependencies make the build more painful for developers and CI
in different environments, so we've agreed to prefer avoiding external
dependencies where possible.

apt-repo 2018-02-27 20:26:22 UTC 2018-02-27
Add gitubuntu/apt_repo_test/ to the snap

Author: Robie Basak
Author Date: 2018-02-27 20:26:22 UTC

Add gitubuntu/apt_repo_test/ to the snap

This is needed for the in-snap self test.

test-fixes 2018-02-23 13:05:22 UTC 2018-02-23
tests: changelog test files should move to pkg_resources

Author: Nish Aravamudan
Author Date: 2018-02-22 03:08:33 UTC

tests: changelog test files should move to pkg_resources

In order to access the changelog files regardless of running from the
Git repository or the snap, we need to have the test files in the
package and available via pkg_resources.

This should be done for all future testing infrastructure as we add it,
as well.

source-builder-changelog-versions 2018-02-21 10:21:09 UTC 2018-02-21
SourceSpec/SourceFiles: add changelog_versions

Author: Robie Basak
Author Date: 2018-02-12 15:58:02 UTC

SourceSpec/SourceFiles: add changelog_versions

This allows a Source to be created with a predefined list of versions in
debian/changelog. This is useful, for example, for control of changelog
parents during testing.

decouple-gitubunturepository 2018-02-19 14:02:05 UTC 2018-02-19
Prevent modification of GitUbuntuRepository env

Author: Robie Basak
Author Date: 2018-02-19 14:02:05 UTC

Prevent modification of GitUbuntuRepository env

Since it is common to take a GitUbuntuRepository instance env property
and then adjust it, make sure that modifying the dictionary returned
from the property getter does not modify the dictionary owned by the
instance itself. This adds some robustness against this common use
pattern.

source-tree 2018-02-12 15:00:57 UTC 2018-02-12
Add SourceTree

Author: Robie Basak
Author Date: 2018-01-25 15:20:13 UTC

Add SourceTree

commit-graph-builder 2018-02-07 15:27:36 UTC 2018-02-07
Add CommitGraph

Author: Robie Basak
Author Date: 2018-01-25 14:55:37 UTC

Add CommitGraph

derive-env 2018-02-07 13:49:52 UTC 2018-02-07
Replace self._env with a derived property

Author: Robie Basak
Author Date: 2018-02-06 13:29:07 UTC

Replace self._env with a derived property

As part of the drive to reduce stored state to improve testability,
derive self.env when we need it instead of working out in advance in the
constructor.

This is intended to have no functional changes.

GIT_WORK_TREE will end up differently normalised now, such as in
trailing slashes, since in testing pygit2.Repository.workdir was not
normalised like this and so a normalised assertion was needed. This
should not result in any change to git's behaviour.

import-tests 2018-01-24 15:03:18 UTC 2018-01-24
Initial test for import_unapplied_spi

Author: Robie Basak
Author Date: 2017-12-22 15:48:04 UTC

Initial test for import_unapplied_spi

Not got the verification at the end though.

source-builder 2018-01-24 14:42:22 UTC 2018-01-24
Add new source_builder module

Author: Robie Basak
Author Date: 2017-12-08 11:09:47 UTC

Add new source_builder module

unglobal-parent-overrides 2018-01-23 15:20:10 UTC 2018-01-23
Remove global _PARENT_OVERRIDES

Author: Robie Basak
Author Date: 2018-01-23 11:15:39 UTC

Remove global _PARENT_OVERRIDES

Make parent_overrides a regular parameter instead of a magic global
variable. This allows for easier testing along the "less initialisation"
theme.

create-dsc-on-first-use 2018-01-23 14:07:39 UTC 2018-01-23
Clean up catch-all exception

Author: Robie Basak
Author Date: 2018-01-19 17:50:57 UTC

Clean up catch-all exception

The only thing we want to catch here is the command returning a non-zero
exit status, which is subprocess.CalledProcessError. We shouldn't
swallow any other exceptions.

startup-pythonpath 2018-01-23 14:03:50 UTC 2018-01-23
wrapper: fix module search path

Author: Robie Basak
Author Date: 2018-01-23 14:02:34 UTC

wrapper: fix module search path

Fix the case when running git-ubuntu from the working tree via a
relative symlink.

nacc-breakdown/apt-get-not-apt 2018-01-22 13:34:04 UTC 2018-01-22
build: use apt-get over apt

Author: Nish Aravamudan
Author Date: 2018-01-09 16:12:30 UTC

build: use apt-get over apt

apt does not have a stable CLI API, so use the older, but very fine,
apt-get.

LP: #1734366

drop-unused-parameter 2018-01-22 11:35:24 UTC 2018-01-22
Drop unused parameter ubuntu_sinfo

Author: Robie Basak
Author Date: 2017-12-21 15:01:32 UTC

Drop unused parameter ubuntu_sinfo

This is used neither by import_applied_spi nor by import_unapplied_spi.

nacc-breakdown/run-quilt-gbp-correctly 2018-01-18 15:43:17 UTC 2018-01-18
gitubuntu: add API to call quilt and gbp

Author: Nish Aravamudan
Author Date: 2018-01-10 23:10:16 UTC

gitubuntu: add API to call quilt and gbp

LP: #1734137

nacc-breakdown/gbp-pbuilder-qemubuilder 2018-01-18 14:31:01 UTC 2018-01-18
gbp: do not use pbuilder or qemubuilder

Author: Nish Aravamudan
Author Date: 2018-01-10 21:44:49 UTC

gbp: do not use pbuilder or qemubuilder

nacc-breakdown/ubuntu-package-merge-base 2018-01-18 14:16:35 UTC 2018-01-18
gitubuntu: handle finding Ubuntu merge base with new importer layout

Author: Nish Aravamudan
Author Date: 2018-01-10 21:44:30 UTC

gitubuntu: handle finding Ubuntu merge base with new importer layout

`git ubuntu merge` relies on the old importer algorithm setting a
publishing parent around, so we can find the common ancestor between
new/debian and old/ubuntu, via `git merge-base`.

This works even for cases where the branch pointers are forcibly moved,
as long as old/debian is an ancestor of new/debian. But when old/ubuntu
was based on an NMU or security update, there is no guarantee that
new/debian will have that base in its history.

Add a helper method to find the Ubuntu merge point, which will try and
verify that the merge-base found by Git has the correct version. If it
does not, it will fall back to just using the import tag for that
version. The first case corresponds to imports with the old algorithm
and the second to imports with the new algorithm.

LP: #1734364

nacc-breakdown/lxc-snap-env 2018-01-18 14:05:30 UTC 2018-01-18
build: unset SNAP in lxc's env

Author: Nish Aravamudan
Author Date: 2018-01-09 19:21:16 UTC

build: unset SNAP in lxc's env

Ever since lxc gained snap support, it assumed that if SNAP is set in
it's environment that lxc itself was running as a SNAP. This is not
necessarily the case when git-ubuntu itself is a classic snap (and thus
has SNAP set in its environment) but is calling lxc in the host. Unset
SNAP manually on lxc run() calls.

LP: #1741949

dsc-builder 2017-12-22 15:48:04 UTC 2017-12-22
Initial test for import_unapplied_spi

Author: Robie Basak
Author Date: 2017-12-22 15:48:04 UTC

Initial test for import_unapplied_spi

Not got the verification at the end though.

fix-applied-branch-updates 2017-11-24 17:34:59 UTC 2017-11-24
Add quilt env handling tests

Author: Robie Basak
Author Date: 2017-11-24 17:34:59 UTC

Add quilt env handling tests

fix-applied-branch-updates-wip 2017-11-24 17:34:59 UTC 2017-11-24
Add quilt env handling tests

Author: Robie Basak
Author Date: 2017-11-24 17:34:59 UTC

Add quilt env handling tests

escape-git 2017-11-21 14:30:14 UTC 2017-11-21
Add .git escaping

Author: Robie Basak
Author Date: 2017-11-17 15:52:31 UTC

Add .git escaping

If a source package unpacks to contain any files or directories named
'.git', it cannot be represented in a git repository as-is because the
name '.git' is reserved by git and git will not permit it to be an entry
in a git tree object.

Instead we losslessly escape these names by prepending a '.' recursively.

See discussion thread at
http://lists.alioth.debian.org/pipermail/vcs-pkg-discuss/2017-May/000976.html
(continuing into June).

LP: #1730778

test-collection-hang 2017-11-21 12:56:57 UTC 2017-11-21
Do not import local Launchpad object

Author: Robie Basak
Author Date: 2017-11-21 11:47:10 UTC

Do not import local Launchpad object

This is a workaround for LP 1733388. It stops py.test-3 poking the
object's __getattr__ during test discovery, which currently results in a
hang as it prompts the user to log in to Launchpad but the prompt gets
hidden. And we don't want network communication during test discovery
anyway.

fix-importer-devel-branches 2017-11-15 14:34:53 UTC 2017-11-15
Initial tests for _devel_branch_updates

Author: Robie Basak
Author Date: 2017-11-15 10:45:05 UTC

Initial tests for _devel_branch_updates

repo-builder 2017-11-13 09:44:27 UTC 2017-11-13
Add test_git_repository.py

Author: Robie Basak
Author Date: 2017-10-31 17:26:27 UTC

Add test_git_repository.py

As a starting point, this adds tests for follow_symlinks_to_blob using
repo_builder.py.

modernize-scripts-review 2017-11-08 11:28:21 UTC 2017-11-08
Drop spphr_timestamp

Author: Robie Basak
Author Date: 2017-11-08 11:28:21 UTC

Drop spphr_timestamp

This function is no longer used.

dependency-loop 2017-10-30 16:10:07 UTC 2017-10-30
Fix dependency loop: move derive_target_branch

Author: Robie Basak
Author Date: 2017-10-30 16:06:01 UTC

Fix dependency loop: move derive_target_branch

Move derive_target_branch to git_repository.py. It is self contained and
needed from multiple subcommand implemnetations, such as build.py and
lint.py, and git_repository.py also needs it. To keep it in lint.py
creates a dependency loop making various modules unloadable (this
regressed in ae196ab).

This resolves the dependency loops so individual tests are runnable
again. Even if there is some technical debt in this not being exactly in
the ideal place now, it is materially better than it was before since
the tests are runnable.

Next I'd like to get the tests into CI. Then we can have some safety in
refactoring to move it to a final destination if we think it should
belong somewhere else.

commit-graph-v2 2017-10-27 15:40:17 UTC 2017-10-27
Also reset devel heads

Author: Robie Basak
Author Date: 2017-10-26 16:44:55 UTC

Also reset devel heads

We have decided to reset the devel heads instead of generating merge
commits for them.

lp1699541-queue-local-import 2017-10-23 14:03:12 UTC 2017-10-23
Use the branch tip found previously

Author: Robie Basak
Author Date: 2017-10-23 14:03:12 UTC

Use the branch tip found previously

changelog-parsing-fix 2017-08-14 13:31:36 UTC 2017-08-14
Test Changelog.distribution

Author: Robie Basak
Author Date: 2017-08-14 13:31:36 UTC

Test Changelog.distribution

This fails before the previous commit and now passes.

merge-fixes 2017-08-14 11:42:49 UTC 2017-08-14
Refactor merge subcommand commitish handling

Author: Robie Basak
Author Date: 2017-08-14 11:40:21 UTC

Refactor merge subcommand commitish handling

As we're now fetching a commitish object, use that to get the
treeish_string directly instead of looking it up again. This is cleaner
as it avoids the dual lookup, but my main motivation is that it isolates
the "treeish string"-ness more for eventual removal.

lp1698402 2017-08-11 16:01:03 UTC 2017-08-11
wip

Author: Robie Basak
Author Date: 2017-08-11 16:01:03 UTC

wip

versioning-revert 2017-07-26 14:42:14 UTC 2017-07-26
versioning: revert series related changes

Author: Robie Basak
Author Date: 2017-07-26 14:37:32 UTC

versioning: revert series related changes

This regressed tests.

Now we're back where we started. Tests pass for versions calculations
that we had working. The test for the SRU "same version in multiple
series" versioning exception fails as expected, and we're in a position
to be able to implement it.

lint-fixes-2 2017-07-26 12:16:03 UTC 2017-07-26
GitUbuntuRepository: silence noise on __init__

Author: Robie Basak
Author Date: 2017-07-26 12:07:30 UTC

GitUbuntuRepository: silence noise on __init__

logging.info() writes to the terminal by default, so do not use it in
such a common place. Library code shouln't write to the terminal except
by request of the caller.

lint-fixes 2017-07-25 17:46:25 UTC 2017-07-25
lint: correctly report lint failures on failures

Author: Robie Basak
Author Date: 2017-07-25 17:41:13 UTC

lint: correctly report lint failures on failures

We haven't agreed the best pattern to use for this yet. But currently
the return value of GitUbuntuLint.do_change_lint() is outright wrong,
returning only the status of the final check instead of all the checks
processed.

Fix this, at least for now, by correctly combining the results.

atomic-push 2017-07-13 14:47:30 UTC 2017-07-13
importer: push all refs atomically

Author: Robie Basak
Author Date: 2017-07-13 14:43:34 UTC

importer: push all refs atomically

Push any newly imported branches simultaneously with any new import tags
and atomically. This prevents any partial update and restores importer
idempotency.

This requires support for --atomic. It does exist on Xenial and seems to
work with Launchpad. I think this should be sufficient for our purposes.

LP: #1704139

queue-parent-fix 2017-07-12 09:55:06 UTC 2017-07-12
queue: do not clobber parent in series iteration

Author: Robie Basak
Author Date: 2017-07-12 09:55:06 UTC

queue: do not clobber parent in series iteration

If parent gets set for one series, it ends up being set for all of them,
which is wrong. Deconflate the parent requested by the caller from the
parent selected for use for a particular series by calling the latter
series_parent instead.

queue-typos 2017-07-12 09:29:31 UTC 2017-07-12
queue: fix typos for tag deletion

Author: Robie Basak
Author Date: 2017-07-12 09:29:31 UTC

queue: fix typos for tag deletion

tag_uri, not tag_url. Fixes NameErrors.

tag-print-name-only 2017-07-11 14:42:52 UTC 2017-07-11
tag: add --print-name-only

Author: Robie Basak
Author Date: 2017-07-11 14:42:52 UTC

tag: add --print-name-only

This is useful to get the dep14 substitution for scripting.

dsc-decoding 2017-06-30 11:28:10 UTC 2017-06-30
Open dsc in binary mode for better decoding

Author: Robie Basak
Author Date: 2017-06-30 11:28:10 UTC

Open dsc in binary mode for better decoding

Prior to Debian policy 3.8.1.0 (around March 2009), dsc files could be
encoded in anything as no encoding was mandated.

ubuntutools.archive.Dsc already has some support for auto-detection. It
appears to work, but the dsc must be opened in binary mode for this to
work.

This may mutate hashes if any other dscs are decoded differently as a
consequence of this change.

LP: #1700846

fix-empty-dir-symlink-traversal 2017-06-21 16:32:18 UTC 2017-06-21
Fix empty directory handling symlink traversal

Author: Robie Basak
Author Date: 2017-06-21 16:28:55 UTC

Fix empty directory handling symlink traversal

Fix the recursion to recurse only down real directories, not symlinks.

A symlink to a directory that has under the target directory an empty
directory fails the assertion in _create_replacement_tree_builder
because the provided entry is not a tree (a symlink entry is a blob).
This is non-sensical; we don't want to follow symlinks while traversing
for empty directories anyway.

Nish discovered this when importing websockify 0.5.1+dfsg1-1.

I had assumed that os.path.isdir() would not match on symlinks to
directories, but it does. Instead, use os.lstat() to check st_mode
directly.

queue-fixes-3b 2017-06-15 13:15:08 UTC 2017-06-15
Refactor queue sync

Author: Robie Basak
Author Date: 2017-06-15 13:12:18 UTC

Refactor queue sync

This was done in a spike. I don't think it's worth breaking it down for
this early stage script with only one known user (me). The changes
overlay each other significantly and I'm not confident the intermediate
commits are easy to get working anyway.

This adds --no-fetch, --source, --series, --parent, --orphan, --no-trim,
--new and --unapproved to modify the sync subcommand's behaviour. One
can now use this to pull queue tags into any git repository, which is
useful when working with package renames and similar.

I'm not sure it's appropriate for all of this functionality makes sense
to be in a command called "sync" any more; suggestions welcome.

queue-fixes-3 2017-05-31 13:25:48 UTC 2017-05-31
queue: stop tracking srcpkg_name

Author: Robie Basak
Author Date: 2017-05-31 13:25:48 UTC

queue: stop tracking srcpkg_name

We don't need it. Every series import is independent of the other. This
is intentional as the branches may change over a package rename; yet the
queue sync operation is still well-defined in this case.

queue-fixes-2 2017-05-24 15:01:03 UTC 2017-05-24
Fix queue devel ref lookup

Author: Robie Basak
Author Date: 2017-05-24 15:01:03 UTC

Fix queue devel ref lookup

Since pkg is now in refs/remotes/pkg/, look it up by the full reference
as it is not a local branch and so is no longer found by
GitUbuntuRepository.get_head_by_name().

changelog-parsing 2017-05-23 13:44:40 UTC 2017-05-23
Refactor changelog parsing

Author: Robie Basak
Author Date: 2017-05-23 11:25:16 UTC

Refactor changelog parsing

Replace the shell-based symlink workaround with a proper implementation
that uses pygit2 directly. This results in a significant speedup.

get_changelog_versions_from_treeish() previously returned (None, None)
if 'debian/changelog' was not in the supplied tree-ish. This was not
documented and happened implicitly due to the shell fallback behaviour
in parsing debian/changelog. get_heads_and_versions() appears to rely on
this behaviour, for example when asking for versions for the
'debian/pristine-tar' branch. This behaviour is made explicit in the
replacement function that uses the new changelog parsing.

emptydir-workaround 2017-05-17 15:50:19 UTC 2017-05-17
Add hooks to catch empty directory issue

Author: Robie Basak
Author Date: 2017-05-17 14:24:08 UTC

Add hooks to catch empty directory issue

Warn when finding trees affected by LP 1687057.

queue-pocket-copy 2017-04-20 13:49:23 UTC 2017-04-20
queue: support import of pocket copies

Author: Robie Basak
Author Date: 2017-04-20 13:48:24 UTC

queue: support import of pocket copies

In the case of a pocket copy, we must follow through to find a suitable
source_package_publishing_history object and use that to download the
source.

queue-fixes 2017-04-19 15:59:37 UTC 2017-04-19
usd/queue: fix queue handling

Author: Robie Basak
Author Date: 2017-04-19 15:55:57 UTC

usd/queue: fix queue handling

This broke in the refactoring in bcb7e8de.

queue 2017-04-05 18:42:14 UTC 2017-04-05
Add the "queue" command

Author: Robie Basak
Author Date: 2017-01-11 17:34:08 UTC

Add the "queue" command

refactorings-for-queue 2017-02-24 13:52:52 UTC 2017-02-24
Add msg option to USDGitRepository:annotated_tag

Author: Robie Basak
Author Date: 2017-01-11 17:24:29 UTC

Add msg option to USDGitRepository:annotated_tag

This allows the caller to specify the annotation, instead of assuming a
blank one.

lp/1661092 2017-02-02 11:02:34 UTC 2017-02-02
Follow symlinks reading debian/changelog

Author: Robie Basak
Author Date: 2017-02-02 11:01:10 UTC

Follow symlinks reading debian/changelog

LP: #1661092

nofetch.v2 2017-02-01 18:28:30 UTC 2017-02-01
usd import: add --no-fetch option

Author: Robie Basak
Author Date: 2017-01-19 10:36:44 UTC

usd import: add --no-fetch option

exception-handling 2017-01-19 12:50:15 UTC 2017-01-19
Correctly instantiate exceptions

Author: Robie Basak
Author Date: 2017-01-19 12:50:15 UTC

Correctly instantiate exceptions

We should be raising instances of exception classes, not the classes
themselves.

unapproved 2017-01-12 00:41:04 UTC 2017-01-12
fixup commit_tree_hash

Author: Robie Basak
Author Date: 2017-01-12 00:41:04 UTC

fixup commit_tree_hash

disable-applied 2016-12-14 12:37:59 UTC 2016-12-14
Disable applied patches imports

Author: Robie Basak
Author Date: 2016-12-14 12:37:59 UTC

Disable applied patches imports

This is a crude attempt to work around LP: #1649832 by not converting
the unapplied tips to applied tips.

namespaces-v1 2016-11-18 12:03:38 UTC 2016-11-18
Update "usd clone" to meet current namespace specs

Author: Robie Basak
Author Date: 2016-11-18 12:03:38 UTC

Update "usd clone" to meet current namespace specs

178 of 78 results
This repository contains Public information 
Everyone can see this information.

Subscribers