Merge ~andreserl/maas:deploy_precise_squashfs into maas:master

[Andres Rodriguez]

LP: #1739761 - Addd the ability to deploy a precise squashfs image.

[MAAS Lander (maas-lander)]

UNIT TESTS
-b deploy_precise_squashfs lp:~andreserl/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/1410/console
COMMIT: 10051e55f7927e6548bc0b8141d8b6e3c9b9e2b3

review: Needs Fixing

[Lee Trager (ltrager)]

This tries to use the latest commissioning OS with the kernel being deployed. hwe-p isn't available with Trusty+ so this fails to boot.

review: Needs Fixing

[Lee Trager (ltrager)]

This branch fixes the kernel issue, it also tries to use the dd-gz image for deployment if available.

https://code.launchpad.net/~ltrager/maas/+git/maas/+merge/337110

[Scott Moser (smoser)]

Onto this branch, please add
diff --git a/src/maasserver/preseed.py b/src/maasserver/preseed.py
index 5d224f893..7b9e1663b 100644
--- a/src/maasserver/preseed.py
+++ b/src/maasserver/preseed.py
@@ -460,6 +460,8 @@ def get_curtin_installer_url(node):
         return 'cp:///media/root-ro'
     elif image['xinstall_type'] == 'tgz':
         url_prepend = ''
+ elif image['xinstall_type'] == 'squashfs':
+ url_prepend = 'fsimage:'
     else:
         url_prepend = '%s:' % image['xinstall_type']
     dyn_uri = '/'.join([

This way, you'll tell curtin that this url is a 'fsimage'.
rather than a 'squashfs' type.

[Lee Trager (ltrager)]

LGTM!

review: Approve

[MAAS Lander (maas-lander)]

UNIT TESTS
-b deploy_precise_squashfs lp:~andreserl/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/1478/console
COMMIT: d89465d7ac1b1efefcc9edc393539057d4f9792b

review: Needs Fixing

[MAAS Lander (maas-lander)]

LANDING
-b deploy_precise_squashfs lp:~andreserl/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/1489/consoleText

[MAAS Lander (maas-lander)]

LANDING
-b deploy_precise_squashfs lp:~andreserl/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/1494/consoleText

[MAAS Lander (maas-lander)]

LANDING
-b deploy_precise_squashfs lp:~andreserl/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/1499/consoleText

[MAAS Lander (maas-lander)]

UNIT TESTS
-b deploy_precise_squashfs lp:~andreserl/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/1503/console
COMMIT: 6c1660065c96fc5cfea971aa13c64ea538512256

review: Needs Fixing

[MAAS Lander (maas-lander)]

LANDING
-b deploy_precise_squashfs lp:~andreserl/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/1506/consoleText

[MAAS Lander (maas-lander)]

UNIT TESTS
-b deploy_precise_squashfs lp:~andreserl/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/1507/console
COMMIT: 4b81730a80ffe50990655ec26d11f59936826003

review: Needs Fixing

d384623... by Andres Rodriguez on 2018-02-09

Fix lint

diff --git a/src/maasserver/preseed.py b/src/maasserver/preseed.py
index f2ed29c..6eadf99 100644
--- a/src/maasserver/preseed.py
+++ b/src/maasserver/preseed.py
@@ -457,7 +457,14 @@ def get_curtin_installer_url(node):
     # Per etc/services cluster is opening port 5248 to serve images via HTTP
     image = get_curtin_image(node)
     if image['xinstall_type'] == 'squashfs':
-        return 'cp:///media/root-ro'
+        # XXX: roaksoax LP: #1739761 - Since the switch to squashfs (and drop
+        # of iscsi), precise is no longer deployable. To address a squashfs
+        # image is made available allowing it to be deployed in the
+        # commissioning ephemeral environment.
+        if series == 'precise':
+            url_prepend = "fsimage:"
+        else:
+            return 'cp:///media/root-ro'
     elif image['xinstall_type'] == 'tgz':
         url_prepend = ''
     else:
diff --git a/src/maasserver/rpc/boot.py b/src/maasserver/rpc/boot.py
index 3dbad53..85b63f8 100644
--- a/src/maasserver/rpc/boot.py
+++ b/src/maasserver/rpc/boot.py
@@ -251,13 +251,19 @@ def get_config(
 
         # Get the correct operating system and series based on the purpose
         # of the booting machine.
+        precise = False
         if purpose == "commissioning":
             osystem = Config.objects.get_config('commissioning_osystem')
             series = Config.objects.get_config('commissioning_distro_series')
         else:
             osystem = machine.get_osystem()
             series = machine.get_distro_series()
-            if purpose == "xinstall" and osystem != "ubuntu":
+            # XXX: roaksoax LP: #1739761 - Since the switch to squashfs (and
+            # drop of iscsi), precise is no longer deployable. To address a
+            # squashfs image is made available allowing it to be deployed in
+            # the commissioning ephemeral environment.
+            precise = True if series == "precise" else False
+            if purpose == "xinstall" and (osystem != "ubuntu" or precise):
                 # Use only the commissioning osystem and series, for operating
                 # systems other than Ubuntu. As Ubuntu supports HWE kernels,
                 # and needs to use that kernel to perform the installation.
@@ -273,7 +279,12 @@ def get_config(
         # subarchitecture. Since Ubuntu does not support architecture specific
         # hardware enablement kernels(i.e a highbank hwe-t kernel on precise)
         # we give precedence to any kernel defined in the subarchitecture field
-        if subarch == "generic" and machine.hwe_kernel:
+
+        # XXX: roaksoax LP: #1739761 - Do not override the subarch (used for
+        # the deployment ephemeral env) when deploying precise, provided that
+        # it uses the commissioning distro_series and hwe kernels are not
+        # needed.
+        if subarch == "generic" and machine.hwe_kernel and not precise:
             subarch = machine.hwe_kernel
         elif(subarch == "generic" and
              purpose == "commissioning" and
diff --git a/src/maasserver/rpc/tests/test_boot.py b/src/maasserver/rpc/tests/test_boot.py
index 84dfcf8..17898db 100644
--- a/src/maasserver/rpc/tests/test_boot.py
+++ b/src/maasserver/rpc/tests/test_boot.py
@@ -695,6 +695,27 @@ class TestGetConfig(MAASServerTestCase):
             rack_controller.system_id, local_ip, remote_ip, mac=mac)
         self.assertEqual(distro_series, observed_config["release"])
 
+    # XXX: roaksoax LP: #1739761 - Deploying precise is now done using
+    # the commissioning ephemeral environment.
+    def test__returns_commissioning_os_series_for_precise_xinstall(self):
+        self.patch(boot_module, 'get_boot_filenames').return_value = (
+            None, None, None)
+        commissioning_series = "xenial"
+        Config.objects.set_config(
+            "commissioning_distro_series", commissioning_series)
+        distro_series = "precise"
+        rack_controller = factory.make_RackController()
+        local_ip = factory.make_ip_address()
+        remote_ip = factory.make_ip_address()
+        node = self.make_node(
+            status=NODE_STATUS.DEPLOYING, osystem='ubuntu',
+            distro_series=distro_series, primary_rack=rack_controller)
+        mac = node.get_boot_interface().mac_address
+        observed_config = get_config(
+            rack_controller.system_id, local_ip, remote_ip, mac=mac)
+        self.assertEqual(observed_config["release"], commissioning_series)
+        self.assertEqual(node.distro_series, distro_series)
+
     def test__returns_commissioning_os_when_erasing_disks(self):
         self.patch(boot_module, 'get_boot_filenames').return_value = (
             None, None, None)
diff --git a/src/maasserver/tests/test_preseed.py b/src/maasserver/tests/test_preseed.py
index 6f5e7d6..76b5cc1 100644
--- a/src/maasserver/tests/test_preseed.py
+++ b/src/maasserver/tests/test_preseed.py
@@ -1577,6 +1577,36 @@ XJzKwRUEuJlIkVEZ72OtuoUMoBrjuADRlJQUW0ZbcmpOxjK1c6w08nhSvA==
                 series, boot_image['label'], xinstall_path),
             installer_url)
 
+    # XXX: roaksoax LP: #1739761 - Deploying precise is now done using
+    # the commissioning ephemeral environment.
+    def test_get_curtin_installer_url_returns_fsimage_precise_squashfs(self):
+        osystem = make_usable_osystem(self)
+        series = 'precise'
+        architecture = make_usable_architecture(self)
+        xinstall_path = factory.make_name('xi_path')
+        xinstall_type = factory.make_name('xi_type')
+        cluster_ip = factory.make_ipv4_address()
+        node = factory.make_Node_with_Interface_on_Subnet(
+            primary_rack=self.rpc_rack_controller, osystem=osystem['name'],
+            architecture=architecture, distro_series=series,
+            boot_cluster_ip=cluster_ip)
+        arch, subarch = architecture.split('/')
+        boot_image = make_rpc_boot_image(
+            osystem=osystem['name'], release=series,
+            architecture=arch, subarchitecture=subarch,
+            purpose='xinstall', xinstall_path=xinstall_path,
+            xinstall_type=xinstall_type)
+        self.patch(
+            preseed_module,
+            'get_boot_images_for').return_value = [boot_image]
+
+        installer_url = get_curtin_installer_url(node)
+        self.assertEqual(
+            '%s:http://%s:5248/images/%s/%s/%s/%s/%s/%s' % (
+                xinstall_type, cluster_ip, osystem['name'], arch, subarch,
+                series, boot_image['label'], xinstall_path),
+            installer_url)
+
     def test_get_curtin_installer_url_returns_cp_for_squashfs(self):
         osystem = make_usable_osystem(self)
         series = osystem['default_release']