Merge lp:~blake-rouse/maas/fix-1612203-1611949 into lp:~maas-committers/maas/trunk

[Blake Rouse]

When selecting the best architecture to perform enlistment only select between fully imported images. Log a message if "no-such-image" occurs when enlistment is being performed.

[Andres Rodriguez (andreserl)]

Overall looks good to me. I'd like to add some more information to the message that is shown.

review: Needs Fixing

[Andres Rodriguez (andreserl)]

+1 just please improve the message.

review: Approve

[MAAS Lander (maas-lander)]

The attempt to merge lp:~blake-rouse/maas/fix-1612203-1611949 into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [95.7 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Fetched 190 kB in 0s (434 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind avahi-utils bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common isc-dhcp-server libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-attr python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
avahi-utils is already the newest version (0.6.32~rc+dfsg-1ubuntu2).
build-essential is already the newest version (12.1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-formencode is already the newest version (1.3.0-0ubuntu5).
python-lxml is already the newest version (3.5.0-1build1).
python-netaddr is already t...

=== modified file 'src/maasserver/models/bootresource.py'
--- src/maasserver/models/bootresource.py	2016-09-29 19:23:13 +0000
+++ src/maasserver/models/bootresource.py	2016-10-05 12:48:58 +0000
@@ -111,7 +111,7 @@
         """
         arches = set()
         for resource in self.all():
-            resource_set = resource.get_latest_set()
+            resource_set = resource.get_latest_complete_set()
             if (resource_set is not None and
                     resource_set.commissionable and
                     resource_set.xinstallable):
@@ -132,7 +132,7 @@
         name = '%s/%s' % (osystem, series)
         resources = self.filter(name=name).order_by('architecture')
         for resource in resources:
-            resource_set = resource.get_latest_set()
+            resource_set = resource.get_latest_complete_set()
             if resource_set is not None and resource_set.commissionable:
                 yield resource
 
@@ -243,7 +243,7 @@
                 architecture__startswith=architecture, name__startswith=name):
             if kflavor is not None and resource.kflavor != kflavor:
                 continue
-            resource_set = resource.get_latest_set()
+            resource_set = resource.get_latest_complete_set()
             if(resource_set is None or
                not resource_set.commissionable or
                not resource_set.xinstallable):
@@ -283,7 +283,7 @@
 
         resource = self.filter(name=os_release, architecture=hwe_arch).first()
         if resource:
-            latest_set = resource.get_latest_set()
+            latest_set = resource.get_latest_complete_set()
             if latest_set:
                 kernel = latest_set.files.filter(
                     filetype=BOOT_RESOURCE_FILE_TYPE.BOOT_KERNEL).first()
@@ -314,7 +314,7 @@
         resources = []
         for resource in self.filter(
                 rtype=BOOT_RESOURCE_TYPE.SYNCED, name__in=lts_releases):
-            resource_set = resource.get_latest_set()
+            resource_set = resource.get_latest_complete_set()
             if resource_set is not None and resource_set.commissionable:
                 resources.append(resource)
 

=== modified file 'src/maasserver/models/tests/test_bootresource.py'
--- src/maasserver/models/tests/test_bootresource.py	2016-10-03 17:53:44 +0000
+++ src/maasserver/models/tests/test_bootresource.py	2016-10-05 12:48:58 +0000
@@ -147,8 +147,10 @@
     def test_get_usable_architectures(self):
         arches = [
             '%s/%s' % (factory.make_name('arch'), factory.make_name('subarch'))
-            for _ in range(3)
+            for _ in range(4)
             ]
+        incomplete_arch = arches.pop()
+        factory.make_incomplete_boot_resource(architecture=incomplete_arch)
         for arch in arches:
             factory.make_usable_boot_resource(architecture=arch)
         usable_arches = BootResource.objects.get_usable_architectures()
@@ -188,6 +190,8 @@
         name = '%s/%s' % (os, series)
         resource = factory.make_usable_boot_resource(
             rtype=BOOT_RESOURCE_TYPE.SYNCED, name=name)
+        factory.make_incomplete_boot_resource(
+            rtype=BOOT_RESOURCE_TYPE.SYNCED, name=name)
         not_commissionable = factory.make_BootResource(
             rtype=BOOT_RESOURCE_TYPE.SYNCED, name=name)
         factory.make_BootResourceSet(not_commissionable)
@@ -643,6 +647,11 @@
                 factory.make_usable_boot_resource(
                     name=self.name, rtype=BOOT_RESOURCE_TYPE.SYNCED,
                     architecture="%s/%s" % (self.arch, i), kflavor=kflavor)
+                factory.make_incomplete_boot_resource(
+                    name=self.name, rtype=BOOT_RESOURCE_TYPE.SYNCED,
+                    architecture="%s/%s" % (
+                        self.arch, factory.make_name("incomplete")),
+                    kflavor=kflavor)
         else:
             generic_kernels = self.kernels
             factory.make_usable_boot_resource(

=== modified file 'src/maasserver/testing/factory.py'
--- src/maasserver/testing/factory.py	2016-09-30 21:08:56 +0000
+++ src/maasserver/testing/factory.py	2016-10-05 12:48:58 +0000
@@ -1541,6 +1541,33 @@
                 resource_set, filename=filetype, filetype=filetype, size=None)
         return resource
 
+    def make_incomplete_boot_resource(
+            self, rtype=None, name=None, architecture=None,
+            extra=None, version=None, label=None, kflavor=None, size=None,
+            bootloader_type=None):
+        resource = self.make_BootResource(
+            rtype=rtype, name=name, architecture=architecture, extra=extra,
+            kflavor=kflavor, bootloader_type=bootloader_type)
+        resource_set = self.make_BootResourceSet(
+            resource, version=version, label=label)
+        filetypes = {
+            BOOT_RESOURCE_FILE_TYPE.BOOT_KERNEL,
+            BOOT_RESOURCE_FILE_TYPE.BOOT_INITRD,
+        }
+        filetypes.add(random.choice([
+            BOOT_RESOURCE_FILE_TYPE.SQUASHFS_IMAGE,
+            BOOT_RESOURCE_FILE_TYPE.ROOT_IMAGE,
+        ]))
+        filetypes.add(random.choice(XINSTALL_TYPES))
+        for filetype in filetypes:
+            # Create a half completed file.
+            size = 512
+            content = factory.make_bytes(256)
+            self.make_boot_resource_file_with_content(
+                resource_set, filename=filetype, filetype=filetype,
+                size=size, content=content)
+        return resource
+
     def make_default_ubuntu_release_bootable(self, arch=None):
         if arch is None:
             arch = self.make_name('arch')

=== modified file 'src/provisioningserver/rackdservices/tftp.py'
--- src/provisioningserver/rackdservices/tftp.py	2016-06-30 18:27:39 +0000
+++ src/provisioningserver/rackdservices/tftp.py	2016-10-05 12:48:58 +0000
@@ -25,6 +25,7 @@
     send_event_node_mac_address,
 )
 from provisioningserver.kernel_opts import KernelParameters
+from provisioningserver.logger.log import get_maas_logger
 from provisioningserver.rpc.boot_images import list_boot_images
 from provisioningserver.rpc.exceptions import BootConfigNoResponse
 from provisioningserver.rpc.region import (
@@ -68,6 +69,9 @@
 from twisted.python.filepath import FilePath
 
 
+maaslog = get_maas_logger("tftp")
+
+
 def get_boot_image(params):
     """Get the boot image for the params on this rack controller."""
     # Match on purpose; enlist uses the commissioning purpose.
@@ -191,6 +195,13 @@
                     d.addErrback(
                         log.err,
                         "Failed to mark machine failed: %s" % description)
+                else:
+                    maaslog.error(
+                        "Enlistment failed to boot %s; missing required boot "
+                        "image %s/%s/%s/%s." % (
+                            params["remote_ip"],
+                            params['osystem'], params["arch"],
+                            params["subarch"], params["release"]))
                 params["label"] = "no-such-image"
             else:
                 params["label"] = boot_image["label"]