Merge lp:~mpontillo/maas/storage-recommission-fix-1575567 into lp:~maas-committers/maas/trunk

[Mike Pontillo]

Change commissioning script to handle the case where existing disks are renamed by the kernel (such as in some cases when a storage device is added).

[Mike Pontillo (mpontillo)]

Note: this needs to be backported to 1.9.

[Andres Rodriguez (andreserl)]

review: Needs Information

[Blake Rouse (blake-rouse)]

Looks good to me see comment about the rename. That needs to be looked into before landing.

review: Approve

[Mike Pontillo (mpontillo)]

Reply below.

[Mike Pontillo (mpontillo)]

Changed it to use the device.id. Thanks for the review.

[MAAS Lander (maas-lander)]

The attempt to merge lp:~mpontillo/maas/storage-recommission-fix-1575567 into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [94.5 kB]
Hit:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu xenial-security InRelease
Fetched 94.5 kB in 0s (220 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-coverage python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu3).
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
build-essential is already the newest version (12.1ubuntu2).
bzr is already the newest version (2.7.0-2ubuntu1).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
isc-dhcp-common is already the newest version (4.3.3-5ubuntu12).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
libpq-dev is already the newest version (9.5.2-1).
make is already the newest version (4.1-6).
postgresql is already the ne...

[Mike Pontillo (mpontillo)]

Unrelated random error.

======================================================================
ERROR: maasserver.tests.test_forms_interface.VLANInterfaceFormTest.test__edits_interface
----------------------------------------------------------------------
testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/tmp/tarmac/branch.a0EU75/src/maastesting/runtest.py", line 121, in _run_user
    result = function(*args, **kwargs)
  File "/home/ubuntu/.buildout/eggs/testtools-2.1.0-py3.5.egg/testtools/testcase.py", line 714, in _run_test_method
    return self._get_test_method()()
  File "/tmp/tarmac/branch.a0EU75/src/maasserver/tests/test_forms_interface.py", line 525, in test__edits_interface
    new_vlan = factory.make_VLAN(fabric=interface.vlan.fabric, vid=33)
  File "/tmp/tarmac/branch.a0EU75/src/maasserver/testing/factory.py", line 877, in make_VLAN
    vlan.save()
  File "/tmp/tarmac/branch.a0EU75/src/maasserver/models/cleansave.py", line 28, in save
    self.full_clean()
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 1171, in full_clean
    raise ValidationError(errors)
django.core.exceptions.ValidationError: {'__all__': ['VLAN with this Vid and Fabric already exists.']}

=== modified file 'src/metadataserver/models/commissioningscript.py'
--- src/metadataserver/models/commissioningscript.py	2016-04-11 16:23:26 +0000
+++ src/metadataserver/models/commissioningscript.py	2016-05-10 16:58:34 +0000
@@ -312,6 +312,15 @@
             block_device.tags = tags
             block_device.save()
         else:
+            # First check if there is an existing device with the same name.
+            # If so, we need to rename it. Its name will be changed back later,
+            # when we loop around to it.
+            existing = PhysicalBlockDevice.objects.filter(
+                node=node, name=name).all()
+            for device in existing:
+                # Use the device ID to ensure a unique temporary name.
+                device.name = "%s.%d" % (device.name, device.id)
+                device.save()
             # New block device. Create it on the node.
             PhysicalBlockDevice.objects.create(
                 node=node,

=== modified file 'src/metadataserver/models/tests/test_commissioningscript.py'
--- src/metadataserver/models/tests/test_commissioningscript.py	2016-05-06 11:41:05 +0000
+++ src/metadataserver/models/tests/test_commissioningscript.py	2016-05-10 16:58:34 +0000
@@ -482,6 +482,24 @@
             ]
         self.assertItemsEqual(device_names, created_names)
 
+    def test__handles_renamed_block_device(self):
+        devices = [self.make_block_device(name='sda', serial='first')]
+        node = factory.make_Node()
+        json_output = json.dumps(devices).encode('utf-8')
+        update_node_physical_block_devices(node, json_output, 0)
+        devices = [
+            self.make_block_device(name='sda', serial='second'),
+            self.make_block_device(name='sdb', serial='first'),
+        ]
+        json_output = json.dumps(devices).encode('utf-8')
+        update_node_physical_block_devices(node, json_output, 0)
+        device_names = [device['NAME'] for device in devices]
+        created_names = [
+            device.name
+            for device in PhysicalBlockDevice.objects.filter(node=node)
+            ]
+        self.assertItemsEqual(device_names, created_names)
+
     def test__only_updates_physical_block_devices(self):
         devices = [self.make_block_device() for _ in range(3)]
         node = factory.make_Node()