Merge ~stgraber/simplestreams:master into simplestreams:master

[Stéphane Graber]

Add combined_disk1-img_sha256 to download.json

This introduces a new combined_disk1-img_sha256 field for the
"lxd.tar.xz" file type in download entries.

This should be the only change that we need to get LXD virtual machine support to be happy with the existing cloud-images.

This will not caused any issues with any existing LXD servers out there as they specifically look for the squashfs or rootxz right now, all other combined sha256 hashes are just ignored.

[Stéphane Graber (stgraber)]

For now we'll go with the same "lxd.tar.xz" for both VM and container images.

If we need to start distinguishing between the two down the line, we can simply list two different files, both of filetype "lxd.tar.xz" and simply have the combined hashes for containers on one and combined hashes for vms on the other and have them each point to a different tarball.

That wouldn't need any client change to work and wouldn't even need defining any new filetype or anything on the server side.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:6b47642fecac1ee5873f0c1dda6addcb79e233e0
https://jenkins.ubuntu.com/server/job/simplestreams-ci/85/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-amd64/85/
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-ppc64el/85/
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-s390x/85/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/simplestreams-ci/85//rebuild

review: Approve (continuous-integration)

[Robert C Jennings (rcj)]

I'd like to request a minor fix to keep the code comment up-to-date regarding combined SHAs. Thanks.

review: Needs Fixing

c57243c... by Stéphane Graber on 2019-08-19

Update comment to cover combined_disk1-img_sha256

Signed-off-by: Stéphane Graber <email address hidden>

[Stéphane Graber (stgraber)]

Oh, good catch, updated the branch.

[Robert C Jennings (rcj)]

Thanks for the update... +1

review: Approve

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:c57243c7ed46927c92c18b1fddebf2220c0a0f28
https://jenkins.ubuntu.com/server/job/simplestreams-ci/86/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-amd64/86/
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-ppc64el/86/
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-s390x/86/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/simplestreams-ci/86//rebuild

review: Approve (continuous-integration)

diff --git a/tools/make-test-data b/tools/make-test-data
index 28b5e5b..a1e183d 100755
--- a/tools/make-test-data
+++ b/tools/make-test-data
@@ -297,8 +297,10 @@ def create_fake_file(prefix, item):
     # create a combined sha256 for lxd.tar.xz (metadata) and the root fs
     # - combined_sha256 and combined_rootxz_sha256 for the -root.tar.gz
     # - combined_squashfs_sha256 for the squashfs
+    # - combined_disk1-img_sha256 for the img
     ftype = item['ftype']
-    for name, extension in (('rootxz', '-root.tar.xz'),
+    for name, extension in (('disk1-img', '.img'),
+                            ('rootxz', '-root.tar.xz'),
                             ('squashfs', '.squashfs')):
         if (ftype == "lxd.tar.xz" and
                 'combined_{}_sha256'.format(name) not in FAKE_DATA[ftype]):
@@ -567,7 +569,8 @@ def create_image_data(query_tree, out_d, streamdir):
             lxd = os.path.basename(path)
 
             # find calculated combined checksums
-            for name, extension in (('rootxz', '-root.tar.xz'),
+            for name, extension in (('disk1-img', '.img'),
+                                    ('rootxz', '-root.tar.xz'),
                                     ('squashfs', '.squashfs')):
                 root = lxd.replace('-lxd.tar.xz', extension)
                 combined = os.path.join(dirname, ','.join([lxd, root]))