Merge lp:~ltrager/maas-images/v3_squashfs_only into lp:maas-images

[Lee Trager]

If the squashfs config option is set produce streams only with SquashFS images.

Any image which is only available as a root-image will be automatically
converted.

Sample stream is up at http://162.213.35.187/squashfs/ it does not contain Precise as maas-images currently ignores unsupported releases and we do not plan to support in in MAAS 2.3+.

[Andres Rodriguez (andreserl)]

Why squashfs *only*? We should continue to produce the root tarballs for trusty. There are still a lot of users on older versions of MAAS that would potentially be broken if we make the switch.

review: Needs Information

[Scott Moser (smoser)]

I agree with Andres. You'll need to make both image types available.

[Lee Trager (ltrager)]

Any version of MAAS which can use the V3 stream can deploy SquashFS images. In the current V3 stream if a release has a SquashFS image we only publish the SquashFS image. The only reason Trusty and Precise have yet to goto a SquashFS image is because they aren't available on http://cloud-images.ubuntu.com. If MAAS detects a release in the stream contains both a root-image and a SquashFS image the SquashFS image will be downloaded and the root-image will be ignored[1]. If a SquashFS image is added to an existing release version the root-image will be deleted and the SquashFS image will be downloaded[2]. So once the Trusty SquashFS image is released to the V3 stream no version of MAAS will use the root-image, even if its available.

[1] https://git.launchpad.net/maas/tree/src/maasserver/bootresources.py#n1084
[2] https://git.launchpad.net/maas/tree/src/maasserver/bootresources.py#n652

[lp:~ltrager/maas-images/v3_squashfs_only]

375. By Lee Trager on 2017-06-23

Merge trunk

376. By Lee Trager on 2017-06-23

If upstream contains a root-image publish it along side the SquashFS image.

[Lee Trager (ltrager)]

I've updated the MP to publish both the root-image.gz and SquashFS image for Trusty. As previously mentioned any version of MAAS which can use the V3 stream will ignore the root-image.gz. The only effect this should have is increasing the stream size.

[Blake Rouse (blake-rouse)]

Looks good. Thanks for the fixes.

review: Approve

=== modified file 'meph2/stream.py'
--- meph2/stream.py	2016-12-16 23:52:01 +0000
+++ meph2/stream.py	2017-06-23 02:07:18 +0000
@@ -155,10 +155,17 @@
     krd_packs = []
     squashfs = cfgdata.get('squashfs', False)
     base_boot_keys = ['boot-kernel', 'boot-initrd']
-    if squashfs and img_url.endswith('.squashfs'):
+    if squashfs:
+        # Only publish SquashFS root images when the squashfs option is set. If
+        # the image is a tar.gz it will be converted below.
         base_ikeys = base_boot_keys + ['squashfs', 'squashfs.manifest']
         manifest_path = PATH_FORMATS['squashfs.manifest'] % subs
         newpaths = set((PATH_FORMATS['squashfs'] % subs, manifest_path))
+        # If upstream is only providing a root-image include it in addition to
+        # the SquashFS image
+        if not img_url.endswith('.squashfs'):
+            base_ikeys += ['root-image.gz']
+            newpaths.update([rootimg_path])
     else:
         base_ikeys = base_boot_keys + ['root-image.gz', 'root-image.manifest']
         manifest_path = PATH_FORMATS['root-image.manifest'] % subs
@@ -300,8 +307,8 @@
         subprocess.check_call(gencmd)
         LOG.info("finished: %s" % gencmd)
 
+        base_dir = os.path.join(out_d, release, arch, version_name)
         if img_url.endswith('squashfs'):
-            base_dir = os.path.join(out_d, release, arch, version_name)
             src_squash = os.path.join(base_dir, os.path.basename(img_url))
             if squashfs:
                 # If we're publishing a SquashFS file rename it to its
@@ -318,6 +325,15 @@
                 # If we're not publishing the SquashFS image but used it to
                 # generate the root-img clean it up.
                 os.remove(src_squash)
+        elif squashfs and img_url.endswith('tar.gz'):
+            # If the stream is publishing SquashFS images convert any
+            # non-SquashFS image into a SquashFS image.
+            subprocess.check_call([
+                'sudo', 'env', 'PATH=%s' % os.environ.get('PATH'),
+                os.environ.get('IMG2SQUASHFS', 'img2squashfs'),
+                os.path.join(base_dir, 'root-image.gz'),
+                os.path.join(base_dir, 'squashfs'),
+                ])
 
     # get checksum and size of new files created
     file_info = {}