Merge ~hyask/autopkgtest-cloud:skia/private_ppa_noble into autopkgtest-cloud:master

[Skia]

Main high level change is to enable the use of private PPAs on Noble.

Lower-level changes are:
  * move away from `apt-key`.
  * get the `release` in a way that is deb822 compatible.
  * remove the `noble`-specific quirk that was added when it became deb822-only.

[Skia (hyask)]

WARNING: this has **not** been tested yet, we are still waiting for a private PPA to be setup for our team.

[Skia (hyask)]

First round of test showed that we needed to add keyserver.ubuntu.com to `no_proxy`. With that fix, things are working. We just need to validate that it also works with a private PPA.

[Tim Andersson (andersson123)]

If you haven't seen yet, our private ppa has been set up :)

[Skia (hyask)]

Tested with our newly created private PPA on both Noble and Trusty.
Command-lines used, for memory:
* ./run-autopkgtest -s noble -a amd64 --trigger hello/2.10-3build1~hellothere1 --ppa hyask:thisismytoken@canonical-ubuntu-qa/private-ppa-testing:074721E255F8AC93F97019410847687141517A9C --swiftuser skia hello
* ./run-autopkgtest -s trusty -a amd64 --trigger hello/2.8-4ubuntu1~hellothere14.04 --ppa hyask:thisismytoken@canonical-ubuntu-qa/private-ppa-testing:074721E255F8AC93F97019410847687141517A9C --swiftuser skia hello

[Paride Legovini (paride)]

Nice simplification!

I wonder how apt-key was accessing keyserver.ubuntu.com before, without it being in no_proxy. Maybe is uses https by default, and that's allowed in the squid config, while you went for plain http?

Not worth overthinking: skipping the proxy is even better.

review: Approve

[Skia (hyask)]

I wondered the same, and my wild guess, is that `apt-key` uses `gpg` directly, and `gpg` may simply not support the `http_proxy` variable, thus bypassing it naturally without requiring `no_proxy`.

Thanks for the review!

diff --git a/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker b/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker
index 5689302..bfa35e7 100755
--- a/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker
+++ b/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker
@@ -912,33 +912,23 @@ def request(msg):
                     )
                 else:
                     ppaprefix = "http://"
-                if release == "noble":
-                    # add GPG key
-                    argv += [
-                        "--setup-commands",
-                        "apt-get install -y software-properties-common",
-                    ]
-                    # add apt source
-                    argv += [
-                        "--setup-commands",
-                        "add-apt-repository --enable-source --yes  ppa:%(u)s/%(p)s"
-                        % {"u": ppauser, "p": ppaname},
-                    ]
-                else:
-                    # add GPG key
-                    argv += [
-                        "--setup-commands",
-                        "apt-key adv --keyserver keyserver.ubuntu.com --recv-key "
-                        + fingerprint,
-                    ]
-                    # add apt source
-                    argv += [
-                        "--setup-commands",
-                        'REL=$(sed -rn "/^(deb|deb-src) .*(ubuntu.com|ftpmaster)/ { s/^[^ ]+ +(\[.*\] *)?[^ ]* +([^ -]+) +.*$/\\2/p; q }" /etc/apt/sources.list); '
-                        'echo "deb %(prefix)sppa.launchpad.net/%(u)s/%(p)s/ubuntu $REL main" > /etc/apt/sources.list.d/autopkgtest-%(u)s-%(p)s.list; '
-                        'echo "deb-src %(prefix)sppa.launchpad.net/%(u)s/%(p)s/ubuntu $REL main" >> /etc/apt/sources.list.d/autopkgtest-%(u)s-%(p)s.list;'
-                        % {"prefix": ppaprefix, "u": ppauser, "p": ppaname},
-                    ]
+                # add GPG key
+                # we need to `gpg --dearmor` because at least trusty doesn't seem to support asc keys and
+                # OpenPGP HKS protocol doesn't specify a way to directly download unarmored keys
+                # both wget and gpg are thus required on the testbed, and both are present on the cloud images by default
+                argv += [
+                    "--setup-commands",
+                    f"wget -O- 'http://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&v=1&search=0x{fingerprint}' | "
+                    f"gpg --dearmor > '/etc/apt/trusted.gpg.d/autopkgtest-{ppauser}-{ppaname}.gpg'",
+                ]
+                # add apt source
+                argv += [
+                    "--setup-commands",
+                    "REL=$(lsb_release -cs); "
+                    'echo "deb %(prefix)sppa.launchpad.net/%(u)s/%(p)s/ubuntu $REL main" > /etc/apt/sources.list.d/autopkgtest-%(u)s-%(p)s.list; '
+                    'echo "deb-src %(prefix)sppa.launchpad.net/%(u)s/%(p)s/ubuntu $REL main" >> /etc/apt/sources.list.d/autopkgtest-%(u)s-%(p)s.list;'
+                    % {"prefix": ppaprefix, "u": ppauser, "p": ppaname},
+                ]
 
             # put results into separate container, named by the last PPA
             container += "-%s-%s" % (ppauser, ppaname)
diff --git a/mojo/service-bundle b/mojo/service-bundle
index f2da41c..24d2d14 100644
--- a/mojo/service-bundle
+++ b/mojo/service-bundle
@@ -15,7 +15,7 @@
 {%- if stage_name == "production" or stage_name == "staging" %}
     {%- set http_proxy = "http://squid.internal:3128" %}
     {%- set https_proxy = "http://squid.internal:3128" %}
-    {%- set no_proxy = "127.0.0.1,127.0.1.1,login.ubuntu.com,localhost,localdomain,novalocal,internal,archive.ubuntu.com,ports.ubuntu.com,security.ubuntu.com,ddebs.ubuntu.com,changelogs.ubuntu.com,launchpadlibrarian.net,launchpadcontent.net,launchpad.net,10.24.0.0/24,keystone.ps5.canonical.com,objectstorage.prodstack5.canonical.com" %}
+    {%- set no_proxy = "127.0.0.1,127.0.1.1,login.ubuntu.com,localhost,localdomain,novalocal,internal,archive.ubuntu.com,ports.ubuntu.com,security.ubuntu.com,ddebs.ubuntu.com,changelogs.ubuntu.com,keyserver.ubuntu.com,launchpadlibrarian.net,launchpadcontent.net,launchpad.net,10.24.0.0/24,keystone.ps5.canonical.com,objectstorage.prodstack5.canonical.com" %}
 {%- endif %}
 
 description: "autopkgtest-cloud"