Merge lp:~blake-rouse/maas/fix-1516722 into lp:~maas-committers/maas/trunk

[Blake Rouse]

Modify the initial networking migration to use the orm only from the migration, instead of using the current node model from source.

I have tested this on upgrade from 1.8 to 1.9. This fixes the issue as previous the upgrade broke.

This does not have unit tests because they would have to be skipped anyway because the model is now different.

[Andres Rodriguez (andreserl)]

lgtm!

So when this lands we need to test in real deployments

upgrade from 1.7 to 1.9
upgrade from 1.8 to 1.9
upgrade from 1.9 to 1.9

I'll prepare rc2 and test some of this later today.

Thanks.

review: Approve

[MAAS Lander (maas-lander)]

The attempt to merge lp:~blake-rouse/maas/fix-1516722 into lp:maas failed. Below is the output from the failed tests.

Hit http://security.ubuntu.com xenial-security InRelease
Get:1 http://nova.clouds.archive.ubuntu.com xenial InRelease [218 kB]
Hit http://nova.clouds.archive.ubuntu.com xenial-updates InRelease
Hit http://security.ubuntu.com xenial-security/main Sources
Hit http://security.ubuntu.com xenial-security/universe Sources
Hit http://security.ubuntu.com xenial-security/main amd64 Packages
Hit http://security.ubuntu.com xenial-security/universe amd64 Packages
Hit http://security.ubuntu.com xenial-security/main Translation-en
Hit http://security.ubuntu.com xenial-security/universe Translation-en
Get:2 http://nova.clouds.archive.ubuntu.com xenial/main Sources [1,117 kB]
Get:3 http://nova.clouds.archive.ubuntu.com xenial/universe Sources [7,464 kB]
Get:4 http://nova.clouds.archive.ubuntu.com xenial/main amd64 Packages [1,441 kB]
Get:5 http://nova.clouds.archive.ubuntu.com xenial/universe amd64 Packages [6,955 kB]
Get:6 http://nova.clouds.archive.ubuntu.com xenial/main Translation-en [841 kB]
Get:7 http://nova.clouds.archive.ubuntu.com xenial/universe Translation-en [4,707 kB]
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/main Sources
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/universe Sources
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/universe Translation-en
Fetched 22.7 MB in 14s (1,620 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 authbind bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm pep8 phantomjs postgresql pxelinux pyflakes python-apt python-bson python-convoy python-coverage python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django16 python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-extras python-fixtures python-flake8 python-formencode python-hivex python-httplib2 python-jinja2 python-jsonschema python-lxml python-mock python-netaddr python-netifaces python-nose python-oauth python-openssl python-paramiko python-pexpect python-pip python-pocket-lint python-psycopg2 python-pyinotify python-pyparsing python-seamicroclient python-simplejson python-simplestreams python-sphinx python-subunit python-tempita python-testresources python-testscenarios python-testtools python-twisted python-txtftp python-tz python-unittest2 python-virtualenv python-yaml python-zope.interface socat syslinux-common tgt ubuntu-cloudimage-keyring virtualenv wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is ...

[MAAS Lander (maas-lander)]

The attempt to merge lp:~blake-rouse/maas/fix-1516722 into lp:maas failed. Below is the output from the failed tests.

Hit http://security.ubuntu.com xenial-security InRelease
Get:1 http://nova.clouds.archive.ubuntu.com xenial InRelease [218 kB]
Hit http://security.ubuntu.com xenial-security/main Sources
Hit http://nova.clouds.archive.ubuntu.com xenial-updates InRelease
Hit http://security.ubuntu.com xenial-security/universe Sources
Hit http://security.ubuntu.com xenial-security/main amd64 Packages
Hit http://security.ubuntu.com xenial-security/universe amd64 Packages
Hit http://security.ubuntu.com xenial-security/main Translation-en
Hit http://security.ubuntu.com xenial-security/universe Translation-en
Get:2 http://nova.clouds.archive.ubuntu.com xenial/main Sources [1,117 kB]
Get:3 http://nova.clouds.archive.ubuntu.com xenial/universe Sources [7,466 kB]
Get:4 http://nova.clouds.archive.ubuntu.com xenial/main amd64 Packages [1,441 kB]
Get:5 http://nova.clouds.archive.ubuntu.com xenial/universe amd64 Packages [6,956 kB]
Get:6 http://nova.clouds.archive.ubuntu.com xenial/main Translation-en [841 kB]
Get:7 http://nova.clouds.archive.ubuntu.com xenial/universe Translation-en [4,709 kB]
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/main Sources
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/universe Sources
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com xenial-updates/universe Translation-en
Fetched 22.7 MB in 14s (1,532 kB/s)

W: Failed to fetch http://nova.clouds.archive.ubuntu.com/ubuntu/dists/xenial/universe/i18n/Translation-en Hash Sum mismatch

E: Some index files failed to download. They have been ignored, or old ones used instead.

[Mike Pontillo (mpontillo)]

I was hoping to review this before it landed; looks good, but the one change I would have considered is to (rather than choose all READY nodes) choose all nodes *except* deployed and new.

=== modified file 'src/maasserver/migrations/0182_initial_networking_layout.py'
--- src/maasserver/migrations/0182_initial_networking_layout.py	2015-09-30 07:29:27 +0000
+++ src/maasserver/migrations/0182_initial_networking_layout.py	2015-11-18 16:51:19 +0000
@@ -1,27 +1,81 @@
 from django.db import models
-from maasserver.enum import NODE_STATUS
+from maasserver.enum import (
+    IPADDRESS_TYPE,
+    NODE_STATUS,
+    NODEGROUP_STATUS,
+    NODEGROUPINTERFACE_MANAGEMENT,
+)
 from maasserver.models.node import Node
 from south.db import db
 from south.utils import datetime_utils as datetime
 from south.v2 import DataMigration
 
 
+def get_managed_ngi_for_subnet(subnet):
+    """Return the cluster interface that manages this subnet."""
+    interfaces = subnet.nodegroupinterface_set.filter(
+        nodegroup__status=NODEGROUP_STATUS.ENABLED)
+    interfaces = interfaces.exclude(
+        management=NODEGROUPINTERFACE_MANAGEMENT.UNMANAGED)
+    return interfaces.first()
+
+
+def get_managed_subnet_for_interface(interface):
+    """Return the `Subnet` that manages the `VLAN` for this `interface`."""
+    for subnet in interface.vlan.subnet_set.all():
+        ngi = get_managed_ngi_for_subnet(subnet)
+        if ngi is not None:
+            return subnet
+    return None
+
+
 class Migration(DataMigration):
 
     def forwards(self, orm):
-        # We don't use orm here because we want the current Node model to
-        # include all of the networking methods we require. This will break
-        # if the node these methods get removed from the Node, so we assert
-        # that they at least exists.
-        assert hasattr(Node, "_clear_networking_configuration")
-        assert hasattr(Node, "set_initial_networking_configuration")
-        for node in Node.objects.filter(
+        now = datetime.datetime.now()
+
+        # Loop through all ready nodes and fix their interface configuration.
+        # Deploying and deployed nodes will already have an AUTO or STICKY IP
+        # address and on release MAAS will do the correct thing.
+        for node in orm['maasserver.Node'].objects.filter(
                 installable=True, status=NODE_STATUS.READY):
-            # The node has to be ready and we set the default networking
-            # configuration. Deploying or deployed nodes will already have a
-            # configuration with an AUTO IP or STATIC IP.
-            node._clear_networking_configuration()
-            node.set_initial_networking_configuration()
+
+            # Check if any IP addresses already exists for this node.
+            has_ip = False
+            for interface in node.interface_set.all():
+                has_ip = interface.ip_addresses.exclude(
+                    alloc_type=IPADDRESS_TYPE.DISCOVERED).exists()
+                if has_ip:
+                    break
+
+            # If already has an IP address then nothing needs to be done. If
+            # not then an AUTO IP address needs to be created on the boot
+            # interface or fallback to DHCP if the boot interface is not on
+            # a managed subnet.
+            if not has_ip:
+                # Get the boot interface for the node.
+                boot_interface = node.boot_interface
+                if boot_interface is None:
+                    boot_interface = node.interface_set.order_by('id').first()
+                if boot_interface is None:
+                    continue
+
+                # Get the subnet that manages the VLAN this interface is
+                # connected to.
+                subnet = get_managed_subnet_for_interface(boot_interface)
+                if subnet is not None:
+                    # Create an AUTO IP address for this subnet.
+                    sip = orm['maasserver.StaticIPAddress'].objects.create(
+                        alloc_type=IPADDRESS_TYPE.AUTO, user=None,
+                        ip=None, subnet=subnet, created=now, updated=now)
+                else:
+                    # No subnet managing the VLAN this interface is connected
+                    # to, so set the boot interface to just DHCP.
+                    sip = orm['maasserver.StaticIPAddress'].objects.create(
+                        alloc_type=IPADDRESS_TYPE.DHCP, user=None,
+                        ip=None, subnet=None, created=now, updated=now)
+                boot_interface.ip_addresses.add(sip)
+                boot_interface.save()
 
     def backwards(self, orm):
         # No need to go backward.