Merge ubuntu-cdimage:make-canary-default-add-legacy into ubuntu-cdimage:main

[Łukasz Zemczak]

Prepare for switching canary to become the default Ubuntu (needs releasing the related livecd-rootfs changes). Add support for daily-legacy to still build the old Ubuntu ubiquity based images.

Prepare for switching canary to become the default Ubuntu (needs releasing the related livecd-rootfs changes). Add support for daily-legacy to still build the old Ubuntu ubiquity based images.

This is basically https://code.launchpad.net/~ubuntu-cdimage/ubuntu-cdimage/canary-as-default but modified.

This requires https://code.launchpad.net/~seb128/livecd-rootfs/+git/livecd-rootfs/+merge/433534 to be released after this lands. There is still some work to be done, like, legacy should really get its own seed once the main ones are switched to canary. But that's separate.

Please note that I did not rip out daily-canary support and just added -legacy next to it. The reason is that I heard from Ken the there are some plans of using 'canary' again for some other experimental things, so I don't want us to just keep ripping out and re-adding for no reason. We can clean up separately if we need so.

[Łukasz Zemczak (sil2100)]

Tests are passing - well, at least there's no new failures. I'll clean up the main branch with tests later.

[Brian Murray (brian-murray)]

review: Approve

diff --git a/README b/README
index 45bc00c..926a1f2 100644
--- a/README
+++ b/README
@@ -29,10 +29,15 @@ the following scripts, which may be called by hand:
     Build live CDs for all relevant Ubuntu architectures.
 
   * cron.daily-canary
-
+ 
     Build live CDs of the experimental canary layout for all relevant
     Ubuntu architectures.
 
+  * cron.daily-legacy
+
+    Build live CDs of the legacy (ubiquity-based) images for all relevant
+    Ubuntu architectures.
+
   * cron.dvd
 
     Build DVDs for all relevant Ubuntu architectures.  These are normally
diff --git a/bin/cron.daily-legacy b/bin/cron.daily-legacy
new file mode 100755
index 0000000..eff5f40
--- /dev/null
+++ b/bin/cron.daily-legacy
@@ -0,0 +1,43 @@
+#! /usr/bin/python3
+
+# Copyright (C) 2021 Canonical Ltd.
+# Author: Łukasz 'sil2100' Zemczak <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Build a set of desktop legacy (ubiquity) images."""
+
+from optparse import OptionParser
+import os
+import sys
+
+sys.path.insert(0, os.path.join(sys.path[0], os.pardir, "lib"))
+
+
+def main():
+    from cdimage.build import build_image_set
+    from cdimage.config import Config
+
+    parser = OptionParser("%prog")
+    parser.add_option(
+        "--live", default=False, action="store_true",
+        help="build live filesystems first")
+    options, _ = parser.parse_args()
+    config = Config(IMAGE_TYPE="daily-legacy")
+    config["CDIMAGE_LIVE"] = "1"
+    if not build_image_set(config, options):
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/etc/crontab b/etc/crontab
index 10844c9..58cff5a 100644
--- a/etc/crontab
+++ b/etc/crontab
@@ -52,7 +52,7 @@ PATH=/srv/cdimage.ubuntu.com/bin:/usr/sbin:/usr/bin:/sbin:/bin
 52 4 * * *	PROPOSED=0 DIST=jammy for-project ubuntu-base cron.daily-live --live
 # regular lunar daily builds
 41 7 * * *	for-project ubuntu cron.daily-live --live; SUBPROJECT=desktop-preinstalled for-project ubuntu cron.daily-preinstalled --live
-41 8 * * *	SUBPROJECT=canary for-project ubuntu cron.daily-canary --live
+41 8 * * *	SUBPROJECT=legacy for-project ubuntu cron.daily-legacy --live
 24 5 * * *	for-project kubuntu cron.daily-live --live
 40 1 * * *	for-project xubuntu cron.daily-live --live
 10 3 * * *	for-project ubuntu-server cron.daily-preinstalled --live; SUBPROJECT=live for-project ubuntu-server cron.daily-live --live
diff --git a/etc/livefs-launchpad b/etc/livefs-launchpad
index 025f725..979dec4 100644
--- a/etc/livefs-launchpad
+++ b/etc/livefs-launchpad
@@ -3,8 +3,9 @@
 # Like etc/default-arches, "PROJECT" is in fact PROJECT[-SUBPROJECT][-LOCALE].
 
 ubuntu			daily-live		*		*		ubuntu-cdimage/ubuntu
+ubuntu			daily-legacy		*		*		ubuntu-cdimage/ubuntu-legacy
+ubuntu-canary			daily-live		*		disco-		ubuntu-cdimage/ubuntu-canary
 ubuntu-desktop-preinstalled daily-preinstalled	focal-		*		ubuntu-cdimage/ubuntu-preinstalled
-ubuntu-canary			daily-canary		*		*		ubuntu-cdimage/ubuntu-canary
 kubuntu			daily-live		*		*		ubuntu-cdimage/kubuntu
 xubuntu			daily-live		*		*		ubuntu-cdimage/xubuntu
 ubuntu-server		daily			trusty-		*		ubuntu-cdimage/ubuntu-server
diff --git a/etc/purge-days b/etc/purge-days
index b778029..acdd921 100644
--- a/etc/purge-days
+++ b/etc/purge-days
@@ -2,6 +2,7 @@ daily 1
 daily-live 1
 daily-preinstalled 1
 daily-canary 1
+daily-legacy 1
 # Purge images older than 2 days => keep the most recent build.
 dvd 2
 # Ubuntu Core images are not purged per age
diff --git a/etc/qa-products b/etc/qa-products
index aa7a307..ea32841 100644
--- a/etc/qa-products
+++ b/etc/qa-products
@@ -41,6 +41,7 @@ Ubuntu Base s390x							ubuntu-base		daily				base					s390x			iso
 
 # Ubuntu Desktop 
 Ubuntu Desktop amd64						ubuntu			daily-live			desktop					amd64			iso
+Ubuntu Desktop (Legacy) amd64				ubuntu/legacy	daily-legacy		desktop-legacy			amd64			iso
 Ubuntu Desktop (Canary) amd64				ubuntu/canary	daily-canary		desktop-canary			amd64			iso
 Ubuntu Desktop arm64						ubuntu			daily-live			desktop					arm64			iso
 Ubuntu Desktop arm64+raspi					ubuntu/desktop-preinstalled			daily-preinstalled	preinstalled-desktop	arm64+raspi		iso
diff --git a/lib/cdimage/germinate.py b/lib/cdimage/germinate.py
index 4bd3ee2..af88a5f 100644
--- a/lib/cdimage/germinate.py
+++ b/lib/cdimage/germinate.py
@@ -315,6 +315,7 @@ class GerminateOutput:
             elif project == "ubuntu" and self.config["SUBPROJECT"] == "canary":
                 # ubuntu-desktop-installer
                 yield "canary-ship-live"
+                # TODO: we will probably need a legacy-ship-live seed
             else:
                 yield "ship-live"
         elif mode == "addon":
diff --git a/lib/cdimage/tests/test_tree.py b/lib/cdimage/tests/test_tree.py
index abf6369..c293665 100644
--- a/lib/cdimage/tests/test_tree.py
+++ b/lib/cdimage/tests/test_tree.py
@@ -269,6 +269,7 @@ class TestPublisher(TestCase):
             ("daily", "ubuntu-server", "focal", "legacy-server"),
             ("daily", "ubuntu", "precise", "alternate"),
             ("daily-canary", "ubuntu", "jammy", "desktop-canary"),
+            ("daily-legacy", "ubuntu", "lunar", "desktop-legacy"),
         ):
             self.config["PROJECT"] = project
             self.config["DIST"] = dist
@@ -1032,6 +1033,44 @@ class TestDailyTreePublisher(TestCase):
                 self.config.series),
             "%s-desktop-canary-amd64.iso" % self.config.series)
 
+    @mock.patch("cdimage.osextras.find_on_path", return_value=True)
+    @mock.patch("cdimage.tree.zsyncmake")
+    def test_publish_legacy_binary(self, mock_zsyncmake, *args):
+        publisher = self.make_publisher("ubuntu", "daily-legacy")
+        source_dir = publisher.image_output("amd64")
+        touch(os.path.join(
+            source_dir, "%s-desktop-legacy-amd64.raw" %
+            self.config.series))
+        touch(os.path.join(
+            source_dir, "%s-desktop-legacy-amd64.list" %
+            self.config.series))
+        touch(os.path.join(
+            source_dir, "%s-desktop-legacy-amd64.manifest" %
+            self.config.series))
+        self.capture_logging()
+        list(publisher.publish_binary("desktop-legacy", "amd64", "20201215"))
+        self.assertLogEqual([
+            "Publishing amd64 ...",
+            "Unknown file type 'empty'; assuming .iso",
+            "Publishing amd64 live manifest ...",
+            "Making amd64 zsync metafile ...",
+        ])
+        target_dir = os.path.join(publisher.publish_base, "20201215")
+        self.assertEqual([], os.listdir(source_dir))
+        self.assertCountEqual([
+            "%s-desktop-legacy-amd64.iso" % self.config.series,
+            "%s-desktop-legacy-amd64.list" % self.config.series,
+            "%s-desktop-legacy-amd64.manifest" % self.config.series,
+        ], os.listdir(target_dir))
+        mock_zsyncmake.assert_called_once_with(
+            os.path.join(
+                target_dir, "%s-desktop-legacy-amd64.iso" %
+                self.config.series),
+            os.path.join(
+                target_dir, "%s-desktop-legacy-amd64.iso.zsync" %
+                self.config.series),
+            "%s-desktop-legacy-amd64.iso" % self.config.series)
+
     def test_publish_livecd_base(self):
         publisher = self.make_publisher("livecd-base", "livecd-base")
         source_dir = os.path.join(
@@ -2138,6 +2177,9 @@ class TestChinaDailyTreePublisher(TestDailyTreePublisher):
     def test_publish_canary_binary(self):
         pass
 
+    def test_publish_legacy_binary(self):
+        pass
+
     def test_publish_livecd_base(self):
         pass
 
diff --git a/lib/cdimage/tree.py b/lib/cdimage/tree.py
index 0c5c592..3be7f9a 100644
--- a/lib/cdimage/tree.py
+++ b/lib/cdimage/tree.py
@@ -433,6 +433,8 @@ class Publisher:
             return "dvd"
         elif self.image_type == "daily-canary":
             return "desktop-canary"
+        elif self.image_type == "daily-legacy":
+            return "desktop-legacy"
         else:
             if self.project == "edubuntu":
                 return "addon"
@@ -463,6 +465,8 @@ class Publisher:
             return "daily"
         elif publish_type == "desktop-canary":
             return "daily-canary"
+        elif publish_type == "desktop-legacy":
+            return "daily-legacy"
         else:
             return None
 
@@ -517,6 +521,8 @@ class Publisher:
             return "desktop %s" % cd
         elif publish_type == "desktop-canary":
             return "canary desktop %s" % cd
+        elif publish_type == "desktop-legacy":
+            return "legacy desktop %s" % cd
         elif publish_type == "install":
             return "install %s" % cd
         elif publish_type == "alternate":
@@ -597,7 +603,7 @@ class Publisher:
                 "The live %s allows you to try %s without changing your "
                 "computer at all, and at your option to install it "
                 "permanently later.</p>" % (cd, capproject))
-        elif publish_type in ("desktop", "desktop-canary"):
+        elif publish_type in ("desktop", "desktop-canary", "desktop-legacy"):
             sentences.append(
                 "The desktop %s allows you to try %s without changing your "
                 "computer at all, and at your option to install it "
@@ -611,6 +617,10 @@ class Publisher:
                 sentences.append(
                     "This type of %s is experimental." %
                     cd)
+            if publish_type == "desktop-legacy":
+                sentences.append(
+                    "This type of %s uses the legacy installer." %
+                    cd)
             sentences.append(desktop_req)
             if self.project == "edubuntu":
                 sentences.append(
@@ -1197,6 +1207,7 @@ class Publisher:
             "preinstalled-touch", "preinstalled-core", "wubi",
             "live-core",
             "desktop-canary",
+            "desktop-legacy",
         )
 
         all_arches = (
@@ -3314,7 +3325,7 @@ class ReleasePublisher(Publisher):
 
     def want_manifest(self, publish_type, path):
         if publish_type in (
-            "live", "desktop", "desktop-canary", "netbook",
+            "live", "desktop", "desktop-canary", "desktop-legacy", "netbook",
             "uec", "server-uec", "core", "wubi", "server", "live-server",
             "legacy-server",
         ):