Merge lp:~ltrager/maas/lp1629491 into lp:~maas-committers/maas/trunk

[Lee Trager]

Use validate_hwe_kernel when setting the commissiong kernel.

The kernel format changed from hwe-<letter> to hwe-<version> in 2.1. Users who upgraded from a previous version may still be using the old format in the min_hwe_kernel field. validate_hwe_kernel converts from the old format to the new transparently so commissioning continues to work.

[Blake Rouse (blake-rouse)]

Code looks good. As long as it works, ;-)

review: Approve

[MAAS Lander (maas-lander)]

The attempt to merge lp:~ltrager/maas/lp1629491 into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Hit:2 http://security.ubuntu.com/ubuntu xenial-security InRelease
Get:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [95.7 kB]
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Fetched 95.7 kB in 0s (216 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind avahi-utils bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common isc-dhcp-server libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-attr python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
avahi-utils is already the newest version (0.6.32~rc+dfsg-1ubuntu2).
build-essential is already the newest version (12.1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-formencode is already the newest version (1.3.0-0ubuntu5).
python-lxml is already the newest version (3.5.0-1build1).
python-netaddr is already the newest version (0.7....

=== modified file 'src/maasserver/models/tests/test_bootresource.py'
--- src/maasserver/models/tests/test_bootresource.py	2016-09-12 21:33:05 +0000
+++ src/maasserver/models/tests/test_bootresource.py	2016-10-03 17:54:21 +0000
@@ -649,9 +649,9 @@
                 name=self.name, rtype=BOOT_RESOURCE_TYPE.SYNCED,
                 architecture="%s/%s" % (self.arch, self.subarch))
         self.assertEqual(
-            self.kernels,
-            BootResource.objects.get_usable_hwe_kernels(
-                self.name, self.arch),
+            sorted(self.kernels),
+            sorted(BootResource.objects.get_usable_hwe_kernels(
+                self.name, self.arch)),
             "%s should return %s as its usable kernel" % (
                 self.name, self.kernels))
         self.assertEqual(

=== modified file 'src/maasserver/rpc/boot.py'
--- src/maasserver/rpc/boot.py	2016-09-30 22:08:43 +0000
+++ src/maasserver/rpc/boot.py	2016-10-03 17:54:21 +0000
@@ -7,6 +7,7 @@
     "get_config",
 ]
 
+from django.core.exceptions import ValidationError
 from maasserver.enum import INTERFACE_TYPE
 from maasserver.models import (
     BootResource,
@@ -29,6 +30,7 @@
     get_one,
     transactional,
 )
+from maasserver.utils.osystems import validate_hwe_kernel
 from provisioningserver.events import EVENT_TYPES
 from provisioningserver.rpc.exceptions import BootConfigNoResponse
 from provisioningserver.utils.twisted import synchronous
@@ -157,7 +159,12 @@
         elif(subarch == "generic" and
              purpose == "commissioning" and
              machine.min_hwe_kernel):
-            subarch = machine.min_hwe_kernel
+            try:
+                subarch = validate_hwe_kernel(
+                    None, machine.min_hwe_kernel, machine.architecture,
+                    osystem, series)
+            except ValidationError:
+                subarch = "no-such-kernel"
 
         # We don't care if the kernel opts is from the global setting or a tag,
         # just get the options

=== modified file 'src/maasserver/rpc/tests/test_boot.py'
--- src/maasserver/rpc/tests/test_boot.py	2016-08-18 17:31:05 +0000
+++ src/maasserver/rpc/tests/test_boot.py	2016-10-03 17:54:21 +0000
@@ -500,13 +500,42 @@
         remote_ip = factory.make_ip_address()
         node = factory.make_Node_with_Interface_on_Subnet(
             status=NODE_STATUS.COMMISSIONING,
-            min_hwe_kernel="hwe-v")
-        mac = node.get_boot_interface().mac_address
-        observed_config = get_config(
-            rack_controller.system_id, local_ip, remote_ip, mac=mac)
-        self.assertEqual(
-            "hwe-v",
-            observed_config["subarch"])
+            min_hwe_kernel="hwe-16.10")
+        arch = node.split_arch()[0]
+        ubuntu = factory.make_default_ubuntu_release_bootable(arch)
+        factory.make_usable_boot_resource(
+            name=ubuntu.name, architecture="%s/hwe-16.10" % arch,
+            kflavor='generic', rtype=BOOT_RESOURCE_TYPE.SYNCED)
+        mac = node.get_boot_interface().mac_address
+        observed_config = get_config(
+            rack_controller.system_id, local_ip, remote_ip, mac=mac)
+        self.assertEqual("hwe-16.10", observed_config["subarch"])
+
+    def test__commissioning_node_uses_min_hwe_kernel_converted(self):
+        rack_controller = factory.make_RackController()
+        local_ip = factory.make_ip_address()
+        remote_ip = factory.make_ip_address()
+        node = factory.make_Node_with_Interface_on_Subnet(
+            status=NODE_STATUS.COMMISSIONING,
+            min_hwe_kernel="hwe-x")
+        arch = node.split_arch()[0]
+        factory.make_default_ubuntu_release_bootable(arch)
+        mac = node.get_boot_interface().mac_address
+        observed_config = get_config(
+            rack_controller.system_id, local_ip, remote_ip, mac=mac)
+        self.assertEqual("hwe-16.04", observed_config["subarch"])
+
+    def test__commissioning_node_uses_min_hwe_kernel_reports_missing(self):
+        rack_controller = factory.make_RackController()
+        local_ip = factory.make_ip_address()
+        remote_ip = factory.make_ip_address()
+        node = factory.make_Node_with_Interface_on_Subnet(
+            status=NODE_STATUS.COMMISSIONING,
+            min_hwe_kernel="hwe-x")
+        mac = node.get_boot_interface().mac_address
+        observed_config = get_config(
+            rack_controller.system_id, local_ip, remote_ip, mac=mac)
+        self.assertEqual("no-such-kernel", observed_config["subarch"])
 
     def test__returns_ubuntu_os_series_for_ubuntu_xinstall(self):
         distro_series = random.choice(["trusty", "vivid", "wily", "xenial"])

=== modified file 'src/maasserver/utils/osystems.py'
--- src/maasserver/utils/osystems.py	2016-09-29 19:23:13 +0000
+++ src/maasserver/utils/osystems.py	2016-10-03 17:54:21 +0000
@@ -357,7 +357,10 @@
     else:
         raise ValueError("Unknown release or kernel %s!" % string)
 
-    if parts[0] == 'hwe':
+    # hwe kernels should only have a higher weight when using the new format
+    # which is hwe-<version>. This ensures the old format maps to the ga
+    # kernel.
+    if parts[0] == 'hwe' and len(parts[1]) > 1:
         weight += 1
 
     if release == 'rolling':

=== modified file 'src/maasserver/utils/tests/test_osystems.py'
--- src/maasserver/utils/tests/test_osystems.py	2016-09-30 13:56:04 +0000
+++ src/maasserver/utils/tests/test_osystems.py	2016-10-03 17:54:21 +0000
@@ -376,7 +376,7 @@
             }),
             ("Old style kernel", {
                 "string": "hwe-%s" % release['series'][0],
-                "expected": version_tuple + tuple([1]),
+                "expected": version_tuple + tuple([0]),
             }),
             ("GA kernel", {
                 "string": "ga-%s" % version_str,