Merge ~cpete/curtin:cdrom-info into curtin:master

[Chris Peterson]

curthooks: copy installation media metadata

Ubuntu ISOs contain metadata in the .disk/ directory, like the build date or sometimes an OEM Tracking ID, and we always want to copy this information over when we can. Subiquity currently always tries to write this information, and it's potentially useful for non-Subiquity installs, so let's move it here instead.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:528ebd94d94cfc0192ed57aebb50653a87bbf28d

No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want jenkins to rebuild you need to trigger it yourself):
https://code.launchpad.net/~cpete/curtin/+git/curtin/+merge/462092/+edit-commit-message

https://jenkins.canonical.com/server-team/job/curtin-ci/250/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-amd64/250/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-arm64/250/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-ppc64el/250/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-s390x/250/

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/curtin-ci/250//rebuild

review: Needs Fixing (continuous-integration)

[Dan Bungert (dbungert)]

The commit message will need fixing. I often just copy a useful commit message from the commit stream.

If you have several commits and we don't want them squashed then I suggest adding "do not squash" like was done in https://code.launchpad.net/~curtin-dev/curtin/+git/curtin/+merge/461452

With a fix to the destination (see comments) and making the CI robot happy, this should be fine. Please also pull into a VM test to confirm.

review: Needs Fixing

[Dan Bungert (dbungert)]

LGTM

review: Approve

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:9a8123e3e2f9493e59f052834a6b3eebff567695
https://jenkins.canonical.com/server-team/job/curtin-ci/254/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-amd64/254/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-arm64/254/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-ppc64el/254/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-s390x/254/

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/curtin-ci/254//rebuild

review: Approve (continuous-integration)

[Chris Peterson (cpete)]

Confirmed works in a VM. How do I tell the CI to retry? Edit: Nevermind, just slow I think.

[Olivier Gayot (ogayot)]

LGTM. Consider using pathlib next time :)

[Chris Peterson (cpete)]

@ogayot Thanks for the feedback!

> Consider using pathlib next time :)

Will do :)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:5b35ba5326110607b7f8f0578e67722296435856
https://jenkins.canonical.com/server-team/job/curtin-ci/257/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-amd64/257/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-arm64/257/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-ppc64el/257/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-s390x/257/

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/curtin-ci/257//rebuild

review: Approve (continuous-integration)

diff --git a/curtin/commands/curthooks.py b/curtin/commands/curthooks.py
index 63a24fe..9eb8831 100644
--- a/curtin/commands/curthooks.py
+++ b/curtin/commands/curthooks.py
@@ -920,6 +920,32 @@ def copy_crypttab(crypttab, target):
     shutil.copy(crypttab, os.path.sep.join([target, 'etc/crypttab']))
 
 
+def copy_cdrom(cdrom: str, target: str) -> None:
+    """Copy installation media metadata"""
+
+    base: str = f"{cdrom}/.disk"
+    ubuntu_dist_channel: str = f"{base}/ubuntu_dist_channel"
+    media_info: str = f"{base}/info"
+
+    logdir: str = os.path.join(target, "var/log/installer")
+    util.ensure_dir(logdir)
+
+    libdir: str = os.path.join(target, "var/lib")
+    util.ensure_dir(libdir)
+
+    if os.path.exists(media_info):
+        LOG.info("copying media-info into target")
+        shutil.copy(media_info, f"{logdir}/media-info")
+    else:
+        LOG.warning(f"{media_info} not found, skipping")
+
+    if os.path.exists(ubuntu_dist_channel):
+        LOG.info("copying ubuntu_dist_channel into target")
+        shutil.copy(ubuntu_dist_channel, f"{libdir}/ubuntu_dist_channel")
+    else:
+        LOG.warning(f"{ubuntu_dist_channel} not found, skipping")
+
+
 def copy_iscsi_conf(nodes_dir, target, target_nodes_dir='etc/iscsi/nodes'):
     if not nodes_dir:
         LOG.warn("nodes directory must be specified, not copying")
@@ -2168,6 +2194,14 @@ def builtin_curthooks(cfg, target, state):
             setup_grub(cfg, target, osfamily=osfamily,
                        variant=distro_info.variant)
 
+    # Copy information from installation media
+    with events.ReportEventStack(
+            name=f"{stack_prefix}/copy-cdrom-metadata",
+            reporting_enabled=True, level="INFO",
+            description="copying metadata from /cdrom"):
+        cdrom_loc: str = "/cdrom"
+        copy_cdrom(cdrom_loc, target)
+
 
 def curthooks(args):
     state = util.load_command_environment()
diff --git a/tests/unittests/test_curthooks.py b/tests/unittests/test_curthooks.py
index 61cad9f..4f65313 100644
--- a/tests/unittests/test_curthooks.py
+++ b/tests/unittests/test_curthooks.py
@@ -1933,6 +1933,42 @@ class TestCurthooksChzdev(CiTestCase):
         self.assertEqual(self.chzdev_import, output)
 
 
+class TestCurthooksCopyCdrom(CiTestCase):
+    with_logs = True
+
+    def setUp(self):
+        super().setUp()
+        self.host_dir = self.tmp_dir()
+        self.target = f"{self.host_dir}/target"
+        self.cdrom = f"{self.host_dir}/cdrom"
+
+    def test_pass_on_empty(self):
+        curthooks.copy_cdrom(self.cdrom, self.target)
+        logs = self.logs.getvalue()
+        self.assertIn("/cdrom/.disk/info not found", logs)
+        self.assertIn("/cdrom/.disk/ubuntu_dist_channel not found", logs)
+
+    def test_copy_on_exist(self):
+        util.write_file(f"{self.cdrom}/.disk/info", "")
+        util.write_file(f"{self.cdrom}/.disk/ubuntu_dist_channel", "")
+
+        curthooks.copy_cdrom(self.cdrom, self.target)
+
+        logs = self.logs.getvalue()
+        self.assertNotIn("/cdrom/.disk/info not found", logs)
+        self.assertNotIn("/cdrom/.disk/ubuntu_dist_channel not found", logs)
+        self.assertTrue(
+                os.path.exists(
+                    f"{self.target}/var/log/installer/media-info"
+                )
+        )
+        self.assertTrue(
+                os.path.exists(
+                    f"{self.target}/var/lib/ubuntu_dist_channel"
+                )
+        )
+
+
 class TestCurthooksCopyZkey(CiTestCase):
     def setUp(self):
         super(TestCurthooksCopyZkey, self).setUp()