Merge ~raharper/curtin:fix/vmtest-image-sync-after-maas-url-rename into curtin:master

[Ryan Harper]

vmtest: fix image-sync after maas URL stream rename

Image syncing for vmtest has been failing since MAAS renamed their
daily image stream to stable. In an effort to keep the 'daily' URL
valid the website redirects this to 'stable' path. However the content
was also modified and includes references to files and URLs which have
the 'stable' name included. This broke vmtest image sync which expected
the 'daily' string to be present in filenames and content.

The observable result was that curtin would attempt to find content it
had just *synced* to /srv/images but when searching, it would load up
the cached content files which only referenced the 'daily' stream data
and would indicate that the target file was missing.

This branch moves vmtest to using the 'stable' maas image stream
directly. This value is controllable via an environment variable,
MAAS_IMAGE_STREAM, if set will be used to compose the IMAGE_SRC_URL
and the STREAM_BASE variable which is used to find cached content in
/srv/images/.vmtest_data/ directory.

LP: #1908543

[Michael Hudson-Doyle (mwhudson)]

This all makes sense to me. I guess we'll check it passes in jenkins first...

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:48e8c3c14c5fea63f0441370c8abe3bca11d4289
https://jenkins.ubuntu.com/server/job/curtin-ci/63/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/63/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/63/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/63/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/63/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/63//rebuild

review: Approve (continuous-integration)

diff --git a/tests/vmtests/image_sync.py b/tests/vmtests/image_sync.py
index e460e02..19b2d22 100644
--- a/tests/vmtests/image_sync.py
+++ b/tests/vmtests/image_sync.py
@@ -32,9 +32,11 @@ def environ_get(key, default):
     return os.environ.get(long_key, os.environ.get(key, default))
 
 
+MAAS_IMAGE_STREAM = environ_get("MAAS_IMAGE_STREAM", "stable")
 IMAGE_SRC_URL = environ_get(
     'IMAGE_SRC_URL',
-    "http://images.maas.io/ephemeral-v3/daily/streams/v1/index.sjson")
+    "http://images.maas.io/ephemeral-v3/%s/streams/v1/index.sjson" %
+    MAAS_IMAGE_STREAM)
 IMAGE_DIR = environ_get("IMAGE_DIR", "/srv/images")
 
 KEYRING = environ_get(
@@ -43,7 +45,7 @@ KEYRING = environ_get(
 ITEM_NAME_FILTERS = \
     ['ftype~(boot-initrd|boot-kernel|root-tgz|squashfs)']
 FORMAT_JSON = 'JSON'
-STREAM_BASE = 'com.ubuntu.maas:daily'
+STREAM_BASE = 'com.ubuntu.maas:' + MAAS_IMAGE_STREAM
 VMTEST_CONTENT_ID_PATH_MAP = {
     STREAM_BASE + ":v3:download": "streams/v1/vmtest.json",
     STREAM_BASE + ":centos-bases-download": "streams/v1/vmtest-centos.json",