Merge lp:~ltrager/maas/lp1654450 into lp:~maas-committers/maas/trunk

[Lee Trager]

Only include real kernels when listing all usable kernels.

Each kernel lists all the other kernels supported in that release. For example hwe-16.04 lists hwe-{p,q,r,s,t,u,v,w} as it contains the support proved by all of those kernels. MAAS was filtering out the older versions however edge kernels also display the next kernel as they are a preview release.

[Gavin Panella (allenap)]

Looks good, but read my comments before landing.

review: Approve

[MAAS Lander (maas-lander)]

The attempt to merge lp:~ltrager/maas/lp1654450 into lp:maas failed. Below is the output from the failed tests.

Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Hit:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Fetched 306 kB in 0s (612 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind avahi-utils bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common isc-dhcp-server libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libnss-wrapper libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-attr python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
authbind is already the newest version (2.1.1+nmu1).
avahi-utils is already the newest version (0.6.32~rc+dfsg-1ubuntu2).
build-essential is already the newest version (12.1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
git is already the newest version (1:2.7.4-0ubuntu1).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-formencode is already the newest version (1.3.0-0ubuntu5).
python-lxml is already the newest version (3.5.0-1build1).
python-netaddr is already the newest version (0.7.18-1).
python-netifaces is already the newest version (0.10.4-0.1build2).
python-psycopg2 is a...

=== modified file 'src/maasserver/models/bootresource.py'
--- src/maasserver/models/bootresource.py	2016-10-12 15:26:17 +0000
+++ src/maasserver/models/bootresource.py	2017-01-10 01:10:50 +0000
@@ -1,4 +1,4 @@
-# Copyright 2014-2016 Canonical Ltd.  This software is licensed under the
+# Copyright 2014-2017 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """Boot Resource."""
@@ -230,9 +230,10 @@
             return False
         return True
 
-    def get_usable_hwe_kernels(
-            self, name=None, architecture=None, kflavor=None):
-        """Return the set of usable kernels for architecture and release."""
+    def get_hwe_kernels(
+            self, name=None, architecture=None, kflavor=None,
+            include_subarches=False):
+        """Return the set of kernels."""
         from maasserver.utils.osystems import get_release_version_from_string
 
         if not name:
@@ -257,7 +258,7 @@
                     subarch_parts[1] = 'rolling'
                     kernels.add('-'.join(subarch_parts))
 
-            if "subarches" in resource.extra:
+            if include_subarches and "subarches" in resource.extra:
                 for subarch in resource.extra["subarches"].split(","):
                     if subarch.startswith("hwe-") or subarch.startswith("ga-"):
                         if kflavor is None:
@@ -277,6 +278,28 @@
         return sorted(
             kernels, key=lambda k: get_release_version_from_string(k))
 
+    def get_usable_hwe_kernels(
+            self, name=None, architecture=None, kflavor=None):
+        """Return the set of usable kernels for the given name, arch, kflavor.
+
+        Returns only the list of kernels which MAAS has downloaded. For example
+        if Trusty and Xenial have been downloaded this will return hwe-t,
+        ga-16.04, hwe-16.04, hwe-16.04-edge, hwe-16.04-lowlatency, and
+        hwe-16.04-lowlatency-edge."""
+        return self.get_hwe_kernels(name, architecture, kflavor, False)
+
+    def get_supported_hwe_kernels(
+            self, name=None, architecture=None, kflavor=None):
+        """Return the set of supported kernels for the given name, arch,
+        kflavor.
+
+        Returns the list of kernels downloaded by MAAS and the subarches each
+        of those kernels support. For example if Trusty and Xenial have been
+        downloaded this will return hwe-p, hwe-q, hwe-r, hwe-s, hwe-t, hwe-u,
+        hwe-v, hwe-w, ga-16.04, hwe-16.04, hwe-16.04-edge,
+        hwe-16.04-lowlatency, and hwe-16.04-lowlatency-edge."""
+        return self.get_hwe_kernels(name, architecture, kflavor, True)
+
     def get_kpackage_for_node(self, node):
         """Return the kernel package name for the kernel specified."""
         if not node.hwe_kernel:

=== modified file 'src/maasserver/models/tests/test_bootresource.py'
--- src/maasserver/models/tests/test_bootresource.py	2016-10-12 15:26:17 +0000
+++ src/maasserver/models/tests/test_bootresource.py	2017-01-10 01:10:50 +0000
@@ -1,4 +1,4 @@
-# Copyright 2014-2016 Canonical Ltd.  This software is licensed under the
+# Copyright 2014-2017 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """Tests for `BootResource`."""
@@ -174,6 +174,21 @@
         self.assertItemsEqual(
             arches, usable_arches)
 
+    def test_get_usable_hwe_kernel_doesnt_include_all_subarches(self):
+        factory.make_usable_boot_resource(
+            architecture='amd64/hwe-16.04',
+            extra={'subarches': 'hwe-p,hwe-t,hwe-16.04,hwe-16.10'})
+        self.assertEquals(
+            ['hwe-16.04'], BootResource.objects.get_usable_hwe_kernels())
+
+    def test_get_supported_hwe_kernel_includes_all_subarches(self):
+        factory.make_usable_boot_resource(
+            architecture='amd64/hwe-16.04',
+            extra={'subarches': 'hwe-p,hwe-t,hwe-16.04,hwe-16.10'})
+        self.assertEquals(
+            ['hwe-p', 'hwe-t', 'hwe-16.04', 'hwe-16.10'],
+            BootResource.objects.get_supported_hwe_kernels())
+
     def test_get_commissionable_resource_returns_iterable(self):
         os = factory.make_name('os')
         series = factory.make_name('series')

=== modified file 'src/maasserver/tests/test_forms_commissioning.py'
--- src/maasserver/tests/test_forms_commissioning.py	2016-07-30 01:17:54 +0000
+++ src/maasserver/tests/test_forms_commissioning.py	2017-01-10 01:10:50 +0000
@@ -40,18 +40,18 @@
 
     def test_commissioningform_contains_real_and_ui_choice(self):
         release = factory.pick_ubuntu_release()
-        name = "ubuntu/" + release
+        name = 'ubuntu/%s' % release
+        arch = factory.make_name('arch')
         kernel = 'hwe-' + release[0]
         # Disable boot sources signals otherwise the test fails due to unrun
         # post-commit tasks at the end of the test.
-        self.useFixture(SignalsDisabled("bootsources"))
+        self.useFixture(SignalsDisabled('bootsources'))
         factory.make_BootSourceCache(
             os=name,
             subarch=kernel,
             release=release)
         factory.make_usable_boot_resource(
-            name=name,
-            extra={'subarches': kernel},
+            name=name, architecture='%s/%s' % (arch, kernel),
             rtype=BOOT_RESOURCE_TYPE.SYNCED)
         Config.objects.set_config(
             'commissioning_distro_series',

=== modified file 'src/maasserver/utils/osystems.py'
--- src/maasserver/utils/osystems.py	2016-10-03 16:27:26 +0000
+++ src/maasserver/utils/osystems.py	2017-01-10 01:10:50 +0000
@@ -1,4 +1,4 @@
-# Copyright 2014-2016 Canonical Ltd.  This software is licensed under the
+# Copyright 2014-2017 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 """Utilities for working with operating systems."""
 
@@ -78,10 +78,8 @@
         for release in osystems:
             os_release = osystem + '/' + release['name']
             kernels[osystem][release['name']] = list_hwe_kernel_choices(
-                sorted([
-                    i for i in BootResource.objects.get_usable_hwe_kernels(
-                        os_release)
-                    if release_a_newer_than_b(i, release['name'])]))
+                sorted(BootResource.objects.get_usable_hwe_kernels(os_release))
+            )
     return kernels
 
 
@@ -473,7 +471,7 @@
     """Check that the min_hwe_kernel is avalible."""
     if not min_hwe_kernel or min_hwe_kernel == "":
         return ""
-    usable_kernels = BootResource.objects.get_usable_hwe_kernels()
+    usable_kernels = BootResource.objects.get_supported_hwe_kernels()
     if min_hwe_kernel not in usable_kernels:
         raise ValidationError('%s is not a usable kernel.' % min_hwe_kernel)
     else:

=== modified file 'src/maasserver/utils/tests/test_osystems.py'
--- src/maasserver/utils/tests/test_osystems.py	2016-10-03 16:27:26 +0000
+++ src/maasserver/utils/tests/test_osystems.py	2017-01-10 01:10:50 +0000
@@ -588,7 +588,7 @@
         kernel = factory.make_kernel_string(generic_only=True)
         self.patch(
             BootResource.objects,
-            'get_usable_hwe_kernels').return_value = (kernel,)
+            'get_supported_hwe_kernels').return_value = (kernel,)
         self.assertEquals(kernel, validate_min_hwe_kernel(kernel))
 
     def test_returns_empty_string_when_none(self):

=== modified file 'src/maasserver/websockets/handlers/general.py'
--- src/maasserver/websockets/handlers/general.py	2016-12-12 22:07:14 +0000
+++ src/maasserver/websockets/handlers/general.py	2017-01-10 01:10:50 +0000
@@ -1,4 +1,4 @@
-# Copyright 2015-2016 Canonical Ltd.  This software is licensed under the
+# Copyright 2015-2017 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """The general handler for the WebSocket connection."""
@@ -86,7 +86,7 @@
         the flavor during deployment.
         """
         return list_hwe_kernel_choices(
-            BootResource.objects.get_usable_hwe_kernels()
+            BootResource.objects.get_supported_hwe_kernels()
         )
 
     def default_min_hwe_kernel(self, params):