Allow uploaders to specify where rich history associated with an upload
can be found directly from the changes file. This allows rich history to
be provided by any uploader without concern for things like access
control for pushing upload tags.
This partly deprecates upload tags. The generally recommended approach
to supply rich history will be through the changes file. Upload tags
remain to preserve the rich history preservation mechanism on reimport.
After a transition to changes file based rich history is complete, we
might refactor the rich history preservation mechanism and then drop
support for upload tags altogether. However it might be useful to retain
upload tag support for testing purposes, as they are easier to arrange
than this new changes file mechanism.
For the retry logic, we add a dependency on tenacity. This module seems
to be the most widely used for this purpose, and is already in main in
Ubuntu Focal.
The underlying Launchpad source_package_publishing_history object may
contain a reference to a changes file associated with an upload. Add a
method to access this through the SourcePackageInformation object if it
exists.
In a following change we will need find_or_create_unapplied_commit() to
opportunitistically have access to the underlying
SourcePackageInformation object if there is one available, so that it
can find any provided rich history through that channel. So pass it
through when it exists.
Arrange for the caller to provide the warning messages instead of the
callee.
For this to work, we return the nature of the validation failure in an
exception instead of returning False to indicate a validation failure.
The exception itself provides the nature of the problem, so we can
reduce ourselves to printing a single warning.
The version parameter is no longer needed since it was only used to
generate the warning, which is now done by the callee.
This refactoring will allow the validate_upload_tag() function to become
generic to rich history validation in general, and no longer be specific
to upload tags.
If the user is using GNOME, then it is expected that GNOME Keyring will
be used for Launchpad API token storage, which requires this module (via
keyring) to work.
python3-secretstorage depends on python3-cryptography, which must
explicitly be specified in snapcraft.yaml due to LP: #1918999.
I've been unable to write a test for this. Creating a collection in
secretservice (as implemented by gnome-keyring) doesn't work because a
prompt is required, and gnome-keyring apparently won't do that headless.
I found one reference to this at: https://github.com/99designs/aws-vault/issues/304#issuecomment-497387162
Otherwise, a simple test that does "import keyring;
keyring.backends.SecretService.Keyring().get_preferred_collection()"
might have worked.
We don't want imported repositories to languish; if imported once into
Launchpad, they should continue to be maintained.
Rather than relying on manual import into the whitelist, we can instead
just dynamically check to see if we have already imported a package, and
if so, treat it as if it is whitelisted.
This implements that, calling it a "team whitelist" and adding a
--team-whitelist option to the poller to specify which Launchpad team
should be consulted in order to determine if a package is already
imported.
As the logic gets more complex, we refactor it all into
a filter function created by
importer_service_poller.create_filter_func(). This neatly simplifies
importer_service.request_imports(). For this to work, we pull out the
Launchpad login into the caller, renaming the object to "lp" everywhere
to match convention elsewhere.
I chose to add has_import_repository() to importer_service.py as it's a
general function that feels like it isn't necessary poller specific. But
I placed create_filter_func() to importer_service_poller.py as the logic
of what to include and exclude is closely tied to the poller itself.
This nicely separates the decision logic and list management in the
poller from the search implementation in the main importer_service.py
service handling library.
As the blacklist and whitelist functionality is now pulled into
create_filter_func(), that functionality is removed from
srcpkgs_to_import_list() and should_import_srcpkg(). This invalidates
the admin tools import-source-packages.py, source-package-walker.py and
update-repository-alias.py, so I've removed them. These tools haven't
been used in a long time anyway. import-source-packages.py and
source-package-walker.py are redundant now that we have the queue driven
importer service implementation. update-repository-alias.py is currently
done by hand, but I intend to replace it with an
imported-repository-driven equivalent anyway.
Finally, this means that scriptutils.py becomes mostly redundant and it
might be worth removing the redundant parts and pulling what's left into
importer_service_poller.py, but this change seems to have gotten big
enough now that I'll leave that for another time.
