Merge lp:~thomir-deactivatedaccount/core-image-publisher/build-cloud-image into lp:core-image-publisher

[Thomi Richards]

Add support for building and converting the cloud image.

Work in progress to add support for building and converting the cloud image

[Thomi Richards (thomir-deactivatedaccount)]

Happy to land once you fix the one note I made in the diff.

[Celso Providelo (cprov)]

As we discussed on IRC, this code will not work in production until the charm gets extended to grant sudo NOPASSWD powers to the worker user ('core-worker', IRC).

I don't necessarily want to block code changes due to production missing features, but I will only feel comfortable approving this changes if someone is already assigned to do the charm work. Otherwise it will get lost ...

review: Needs Information

[Para Siva (psivaa)]

One question inline. But that could be postponed for another MP.

[Celso Providelo (cprov)]

review: Approve

[Celso Providelo (cprov)]

Actually, u-d-f call needs a tweak (shell=True) and better check if qemu-img call is in the same situation.

review: Needs Fixing

[Celso Providelo (cprov)]

Thomi,

Ignore my comments about u-d-f calls requiring shell, but double check kombu docs about reject vs. requeue behaviour. I think psivaa is right, according to docs, we should call requeue() not reject().

review: Needs Fixing

[Thomi Richards (thomir-deactivatedaccount)]

Yeah, I agree - changed them all to requeue() calls instead.

[Kit Randel (blr)]

Looks fine, although I may be lacking some context. :)

review: Approve (code)

[Ubuntu CI Bot (uci-bot)]

No proposals found for merge of https://code.launchpad.net/~pwlars/core-image-publisher/fix-logger into https://code.launchpad.net/~canonical-ci-engineering/core-image-publisher/trunk.

[lp:~thomir-deactivatedaccount/core-image-publisher/build-cloud-image]

14. By Thomi Richards on 2015-03-29

Fix typo.

Unmerged revisions

=== modified file 'core_image_publisher/worker.py'
--- core_image_publisher/worker.py	2015-03-27 04:47:12 +0000
+++ core_image_publisher/worker.py	2015-03-29 20:04:52 +0000
@@ -18,6 +18,9 @@
 """Business logic or the service lives here."""
 
 import logging
+import os
+import subprocess
+import tempfile
 from core_image_publisher.queue import enqueue_message
 
 logger = logging.getLogger(__name__)
@@ -36,25 +39,63 @@
             e,
             extra=payload
         )
-        message.reject()
+        message.requeue()
         return
 
-    image_path = download_image(image_name, channel, device)
-    nova_image_path = convert_nova_image(image_path)
-    glance_image_name = upload_image_to_glance(nova_image_path)
+    with tempfile.TemporaryDirectory as tmpdir:
+        try:
+            image_path = download_image(image_name, channel, device, tmpdir)
+        except subprocess.CalledProcessError as e:
+            logger.error("Unable to download core image: %s", e, extra=payload)
+            message.requeue()
+            return
+        try:
+            nova_image_path = convert_nova_image(image_path)
+        except subprocess.CalledProcessError as e:
+            logger.error("Unable to convert core image to qcow2 image: %s", e,
+                extra=payload)
+            message.requeue()
+            return
+        glance_image_name = upload_image_to_glance(nova_image_path)
 
     payload['nova_image'] = glance_image_name
     enqueue_message(payload)
     message.ack()
 
 
-def download_image(name, channel, device):
+def download_image(name, channel, device, tmpdir):
     """Download the ubuntu code image, return a path to it on disk."""
+    image_path = os.path.join(tmpdir, 'core-{}.img'.format(name))
+    cmd = ['sudo',
+           'ubuntu-device-flash',
+           '--revision', name,
+           'core',
+           '--device' generic_amd64,
+           '--channel', channel,
+           '--size', '3',
+           '-o', image_path,
+           '--developer-mode',
+           '--cloud']
+    subprocess.check_call(cmd)
+    return image_path
 
 
 def convert_nova_image(image_path):
-    """Convert a core image to a nova image, return path to the converted image."""
+    """
+    Convert a core image to a nova image, return path to the converted image.
+    """
+    converted_image_path = '{}-cloud.img'.format(image_path.strip('.img'))
+    cmd = ['qemu-img',
+           'convert',
+           '-f', 'raw',
+           '-O', 'qcow2',
+           image_path,
+           converted_image_path]
+    subprocess.check_call(cmd)
+    return converted_image_path
 
 
 def upload_image_to_glance(nova_image_path):
-    """Upload the nova image to glance, returning the name of the image in glance."""
+    """
+    Upload the nova image to glance, returning the name of the image in glance.
+    """