It is common for tests to use source_builder with a
repo_builder.SourceTree. Before this change, this was generated by
round-tripping through a packed source package (dsc, debian.tar.gz, etc)
which takes considerable time. This isn't really necessary but was the
easiest way to implement it at the time because the components already
existed.
Instead, we can arrange to construct the git objects directly, bypassing
generating the source package. On my system this speeds up the tests
significantly, going from 98 seconds to 32 seconds.
In the case that SourceTree is being called in patches-applied mode,
this isn't sufficient as we don't currently have a way to apply patches
purely in git. So we fall back to the previous method in this case.
Skipping the tests that use SourceTree in patches-applied mode saves
another ten seconds, so this suggests that there is yet more significant
performance improvement possible. But I'll leave this for another time.
repo_builder: relax restriction on object mutation
In practice we can mutate repo_builder objects before write() has been
called. I'd like to make use of this to build up Tree() entries
incrementally, so update the docstring to relax the restriction.
Since the implementation already allows this, no actual code change is
needed.
If the user is using GNOME, then it is expected that GNOME Keyring will
be used for Launchpad API token storage, which requires this module (via
keyring) to work.
python3-secretstorage depends on python3-cryptography, which must
explicitly be specified in snapcraft.yaml due to LP: #1918999.
I've been unable to write a test for this. Creating a collection in
secretservice (as implemented by gnome-keyring) doesn't work because a
prompt is required, and gnome-keyring apparently won't do that headless.
I found one reference to this at: https://github.com/99designs/aws-vault/issues/304#issuecomment-497387162
Otherwise, a simple test that does "import keyring;
keyring.backends.SecretService.Keyring().get_preferred_collection()"
might have worked.
We don't want imported repositories to languish; if imported once into
Launchpad, they should continue to be maintained.
Rather than relying on manual import into the whitelist, we can instead
just dynamically check to see if we have already imported a package, and
if so, treat it as if it is whitelisted.
This implements that, calling it a "team whitelist" and adding a
--team-whitelist option to the poller to specify which Launchpad team
should be consulted in order to determine if a package is already
imported.
As the logic gets more complex, we refactor it all into
a filter function created by
importer_service_poller.create_filter_func(). This neatly simplifies
importer_service.request_imports(). For this to work, we pull out the
Launchpad login into the caller, renaming the object to "lp" everywhere
to match convention elsewhere.
I chose to add has_import_repository() to importer_service.py as it's a
general function that feels like it isn't necessary poller specific. But
I placed create_filter_func() to importer_service_poller.py as the logic
of what to include and exclude is closely tied to the poller itself.
This nicely separates the decision logic and list management in the
poller from the search implementation in the main importer_service.py
service handling library.
As the blacklist and whitelist functionality is now pulled into
create_filter_func(), that functionality is removed from
srcpkgs_to_import_list() and should_import_srcpkg(). This invalidates
the admin tools import-source-packages.py, source-package-walker.py and
update-repository-alias.py, so I've removed them. These tools haven't
been used in a long time anyway. import-source-packages.py and
source-package-walker.py are redundant now that we have the queue driven
importer service implementation. update-repository-alias.py is currently
done by hand, but I intend to replace it with an
imported-repository-driven equivalent anyway.
Finally, this means that scriptutils.py becomes mostly redundant and it
might be worth removing the redundant parts and pulling what's left into
importer_service_poller.py, but this change seems to have gotten big
enough now that I'll leave that for another time.
