Now that we are importing all* packages, allowlists (including
"allowlist_team") and phasing are no longer required, so we can remove
this functionality and remove significant sections of code.
Previously we were augmenting the "user-specified" allowlist with an
allowlist generated by reading the apt repository for source packages,
placing them in main and universe buckets, and then applying "phasing"
to them to allow us to include proportional subsets of main and
universe. This made sense at the time, but is no longer necessary.
The downside of the previous approach is that it locked the set of
source packages imported to those present at the time the poller was
last restarted. This would miss new source packages until after the
poller was restarted and a further upload was made to them after the
restart.
Now that we've imported the majority of packages currently published in
any pocket of any release that is not EOL, it's simpler to import any
package that isn't otherwise denylisted. We can maintain the denylist
with the set of edge case imports that still need attention to avoid
unnecessary importer resources.
This requires just a denylist and nothing else.
Since the apt repository parsing was being done solely to determine the
list of source packages in the archive for the purposes of the poller,
the apt repository reading and validation code can also be removed,
together with corresponding tests.
According to the import specification, it's fine to parse Thurs as
Thu, Tues as Tue, and Sept as Sep, as these are all unambiguous. This
fixes imports of the following:
Applications of multiple regular expression replacements are factored
out into _apply_re_substitutions(). This function isn't explicitly
tested; the comprehensive test coverage of its only caller
Changelog._parse_changelog_date() is sufficient.
In importing source packages that exist in non-EOL releases but have
since been deleted, these further twelve packages fail to import and
have never been imported, so add these to the denylist for now.
These package are not yet imported. They will need individual
investigation to fix the edge case issues that cause them to fail. In
the meantime, we won't attempt to import them automatically to avoid
blocking up worker slots.
These are installed and managed manually; these are the current ones in
use in production, with the exception that in
git-ubuntu-importer-service-worker@.service we're currently manually
specifying the IP to connect to, and so this is substituted out.
