Merge ~hyask/autopkgtest-cloud:skia/always_dump_additional_params into autopkgtest-cloud:master

[Skia]

[Skia (hyask)]

Tested on staging: https://autopkgtest.staging.ubuntu.com/packages/mawk/mantic/amd64

Without all-proposed:
```
❯ curl https://autopkgtest.staging.ubuntu.com/results/autopkgtest-mantic/mantic/amd64/m/mawk/20240208_114537_49781@/artifacts.tar.gz 2>/dev/null | tar xvzOf - testinfo.json
testinfo.json
{
 "virt_server": "autopkgtest-virt-ssh -s /home/ubuntu/autopkgtest/ssh-setup/nova -- --flavor stag-cpu2-ram4-disk20 --security-groups <email address hidden> --name 'adt-mantic-amd64-mawk-20240208-114009-juju-97cf5d-stg-proposed-migration-environment-23-$UUID' --image adt/ubuntu-mantic-amd64-server --keyname testbed-juju-97cf5d-stg-proposed-migration-environment-23 --net-id=net_stg-proposed-migration-environment -e TERM=linux -e ''\"'\"'http_proxy=http://squid.internal:3128'\"'\"'' -e ''\"'\"'https_proxy=http://squid.internal:3128'\"'\"'' -e ''\"'\"'no_proxy=127.0.0.1,127.0.1.1,login.ubuntu.com,localhost,localdomain,novalocal,internal,archive.ubuntu.com,ports.ubuntu.com,security.ubuntu.com,ddebs.ubuntu.com,changelogs.ubuntu.com,launchpadlibrarian.net,launchpadcontent.net,launchpad.net,10.24.0.0/24,keystone.ps5.canonical.com,objectstorage.prodstack5.canonical.com'\"'\"'' --mirror=http://ftpmaster.internal/ubuntu/",
 "kernel_version": "Linux 6.5.0-17-generic #17-Ubuntu SMP PREEMPT_DYNAMIC Thu Jan 11 14:01:59 UTC 2024",
 "custom_environment": [
  "ADT_TEST_TRIGGERS=mawk/1.3.4.20230730-1"
 ],
 "nproc": "2",
 "cpu_model": "AMD EPYC-Rome Processor",
 "cpu_flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid",
 "all-proposed": null
}
```

With all-proposed:
```
❯ curl https://autopkgtest.staging.ubuntu.com/results/autopkgtest-mantic/mantic/amd64/m/mawk/20240208_114848_668c6@/artifacts.tar.gz 2>/dev/null | tar xvzOf - testinfo.json
testinfo.json
{
 "virt_server": "autopkgtest-virt-ssh -s /home/ubuntu/autopkgtest/ssh-setup/nova -- --flavor stag-cpu2-ram4-disk20 --security-groups <email address hidden> --name 'adt-mantic-amd64-mawk-20240208-114538-juju-97cf5d-stg-proposed-migration-environment-23-$UUID' --image adt/ubuntu-mantic-amd64-server --keyname testbed-juju-97cf5d-stg-proposed-migration-environment-23 --net-id=net_stg-proposed-migration-environment -e TERM=linux -e ''\"'\"'http_proxy=http://squid.internal:3128'\"'\"'' -e ''\"'\"'https_proxy=http://squid.internal:3128'\"'\"'' -e ''\"'\"'no_proxy=127.0.0.1,127.0.1.1,login.ubuntu.com,localhost,localdomain,novalocal,internal,archive.ubuntu.com,ports.ubuntu.com,security.ubuntu.com,ddebs.ubuntu.com,changelogs.ubuntu.com,launchpadlibrarian.net,launchpadcontent.net,launchpad.net,10.24.0.0/24,keystone.ps5.canoni...

[Tim Andersson (andersson123)]

I think it's better to have all-proposed=0 rather than all-proposed=null. Best they're the same type since multiple snippets of code interact with it

[Skia (hyask)]

Nothing is relying on `all-proposed` in `testinfo.json` yet, as it was not present until like last week. Moreover, setting this explicitly to 0 means taking a conscious decision to do so even if it was not specified, while `null` is more likely to imply that the option just wasn't given. It's a bit like the modern `Option` and `Result` types in Rust: the type is `int`, but it can be `null` so that you know explicitly there is no value to look for and react accordingly.

[Tim Andersson (andersson123)]

let's discuss in standup tomorrow maybe

[Tim Andersson (andersson123)]

Approved once the commit messages have been amended as we just discussed in a call

[Tim Andersson (andersson123)]

Yeeeeeee boy! LGTM. See if CI passes or tempfails then feel free to merge.

review: Approve

[Skia (hyask)]

CI again had some random transient failures, I'm merging this.

diff --git a/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker b/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker
index 3744f06..ed85962 100755
--- a/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker
+++ b/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker
@@ -321,9 +321,7 @@ def request_matches_per_package(package, arch, release, s):
     )
 
 
-def process_output_dir(
-    dir, pkgname, code, triggers, additional_params: dict = None
-):
+def process_output_dir(dir, pkgname, code, triggers, additional_params: dict):
     """Post-process output directory"""
 
     files = set(os.listdir(dir))
@@ -353,14 +351,13 @@ def process_output_dir(
                 json.dump(d, testinfo, indent=True)
             files.add("testinfo.json")
 
-    if additional_params not in [None, {}]:
-        d = {}
-        with open(os.path.join(dir, "testinfo.json"), "r") as testinfo:
-            d = json.load(testinfo)
-        for key, val in additional_params.items():
-            d[key] = val
-        with open(os.path.join(dir, "testinfo.json"), "w") as testinfo:
-            json.dump(d, testinfo, indent=True)
+    d = {}
+    with open(os.path.join(dir, "testinfo.json"), "r") as testinfo:
+        d = json.load(testinfo)
+    for key in KEYS_FOR_ADDITIONAL_PARAMS:
+        d[key] = additional_params.get(key)
+    with open(os.path.join(dir, "testinfo.json"), "w") as testinfo:
+        json.dump(d, testinfo, indent=True)
 
     with open(os.path.join(dir, "testpkg-version"), "r") as tpv:
         testpkg_version = tpv.read().split()[1]
@@ -1250,20 +1247,17 @@ def request(msg):
         # List of key=value strings to be sent as a rabbitmq msg
         # These variables are optional additional parameters users
         # can employ when requesting a test which alters the behaviour
-        additional_parameters = []
-        # We also create it as a dict to easily insert into a json file
-        additional_parameters_for_testinfo = {}
+        additional_parameters = {}
         for key in KEYS_FOR_ADDITIONAL_PARAMS:
             if params.get(key, None) is not None:
-                additional_parameters.append("%s=%s" % (key, params[key]))
-                additional_parameters_for_testinfo[key] = params[key]
+                additional_parameters[key] = params[key]
 
         (testpkg_version, duration, requester) = process_output_dir(
             out_dir,
             pkgname,
             code,
             params.get("triggers", []),
-            additional_parameters_for_testinfo,  # embed additional parameters in testinfo.json
+            additional_parameters,  # embed additional parameters in testinfo.json
         )
 
         # If two tests for the same package with different triggers finish at the
@@ -1386,7 +1380,12 @@ def request(msg):
             "requester": requester,
             "swift_dir": swift_dir,
             "triggers": triggers,
-            "env": ",".join(additional_parameters),
+            "env": ",".join(
+                [
+                    f"{key}={value}"
+                    for key, value in additional_parameters.items()
+                ]
+            ),
         }
     )
     complete_amqp.basic_publish(