Merge lp:~vorlon/ubuntu-archive-tools/update-i386-whitelist into lp:ubuntu-archive-tools

[Steve Langasek]

[lp:~vorlon/ubuntu-archive-tools/update-i386-whitelist]

1253. By Steve Langasek on 2019-12-05

Add a manual whitelist of ppa-only packages

[Adam Conrad (adconrad)]

The PPA hack is icky, but given I filed the relevant LP bug, I understand why it's there. Might do with a comment referencing the bug and encouraging removal.

My only other comment (not a blocker, cause I'm not convinced we'll ever have another partial arch, but it seems more in the spirit of the feature?) is that it should probably be named update-arch-whitelist and take a '-a' argument that fills in all the bits in the source that currently hardcode i386.

Approved with or without those changes, though I think I'd strongly prefer the latter happen (it can default to i386 for now, given that's our only partial arch, if that makes it more palatable to you).

review: Approve

[Adam Conrad (adconrad)]

Hah, and when I said "all the places that hardcode i386", I guess the number of places is exactly one (the packageset), but my brain had somehow filled in that the URL argument had a default too, while it appears not to.

Other tooling in ubuntu-archive-tools is vaguely inconsistent on that front, to be fair, but either defaulting to a local file URL (assuming it'll run on snakefruit most often) or an https one (assuming it's more likely to be run remotely) might not be terribad. I dunno. The URL being in --help makes it a no-brained to copy-pasta, so whatevs.

[lp:~vorlon/ubuntu-archive-tools/update-i386-whitelist]

1254. By Steve Langasek on 2019-12-05

fuller comment

=== added file 'update-i386-whitelist'
--- update-i386-whitelist	1970-01-01 00:00:00 +0000
+++ update-i386-whitelist	2019-12-05 18:29:44 +0000
@@ -0,0 +1,107 @@
+#!/usr/bin/python3
+
+# Copyright (C) 2020  Canonical Ltd.
+# Author: Steve Langasek <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+'''Synchronize the i386 source package whitelist in Launchpad with the output
+of germinate.
+
+USAGE:
+    update-i386-whitelist [--dry-run] https://people.canonical.com/~ubuntu-archive/germinate-output/i386.focal/i386+build-depends.sources
+'''
+
+from launchpadlib.launchpad import Launchpad
+import optparse
+from urllib.request import urlopen
+import sys
+
+def get_sources_from_url(url):
+    '''Download the germinate output and parse out the list of sources.
+
+    Returns list of source package names.
+    '''
+    sources = []
+
+    file = urlopen(url)
+    for i in file:
+        if i.startswith(b'Source') or i.startswith(b'---'):
+            continue
+        sources.append(i.decode('utf-8').split(' ',maxsplit=1)[0])
+    return sources
+
+def parse_options():
+    '''Parse command line arguments.
+
+    Return (options, source_package) tuple.
+    '''
+    parser = optparse.OptionParser(
+        usage='Usage: %prog [--dry-run] https://people.canonical.com/~ubuntu-archive/germinate-output/i386.focal/i386+build-depends.sources')
+    parser.add_option(
+        "--dry-run", help="don't change launchpad, just report the delta",
+        action="store_true")
+    parser.add_option(
+        "-s", dest="release", default=default_release, metavar="RELEASE",
+        help="release (default: %s)" % default_release)
+
+    (opts, args) = parser.parse_args()
+
+    if len(args) != 1:
+        parser.error('Need to specify a URL to sync from')
+
+    return (opts, args[0])
+
+
+if __name__ == '__main__':
+
+    default_release = 'focal'
+
+    (opts, url) = parse_options()
+
+    launchpad = Launchpad.login_with('update-i386-whitelist',
+                                     'production',
+                                     version="devel")
+    ubuntu = launchpad.distributions['ubuntu']
+    series = ubuntu.getSeries(name_or_version=opts.release)
+    archive = ubuntu.main_archive
+
+    sources = get_sources_from_url(url)
+
+    packageset = launchpad.packagesets.getByName(name='i386-whitelist',
+                                                 distroseries=series)
+    currentSet = set(packageset.getSourcesIncluded())
+    newSet = set(sources)
+    # hard-coded list of ppa-only additions; can maybe go away when
+    # https://bugs.launchpad.net/launchpad/+bug/1855069 is fixed, but this is
+    # also potentially useful for bootstrapping any additional packages into
+    # the archive if needed.
+    newSet.add(('gcc-10',))
+    print("Additions:" )
+    additions = list(newSet-currentSet)
+    additions.sort()
+    for i in additions:
+        print(" * %s" % i)
+    print("Removals:" )
+    removals = list(currentSet-newSet)
+    removals.sort()
+    for i in removals:
+        print(" * %s" % i)
+    if opts.dry_run:
+        print("--dry-run is set, doing nothing.")
+        sys.exit(0)
+
+    if additions:
+        packageset.addSources(names=additions)
+    if removals:
+        packageset.removeSources(names=removals)