Merge lp:~returntoreptar/curtin/removable-drive-fixes into lp:~curtin-dev/curtin/trunk

[Robert Clark]

The past few months I have been having an issue with 5 or so of my machines which popped up due to an update to curtin. I have finally traced down the issue to a specific commit to the code (http://bazaar.launchpad.net/~curtin-dev/curtin/trunk/revision/202). In this commit "removable devices" are filtered out since lsblk returns the removable (RM) flag. Unfortunately this caused half of my servers to stop working with maas, since for some reason their main drives (/dev/sda, /dev/sdb, etc) were all labeled as removable by lsblk. I understand the reason for the change being so that curtin does not try to install to removable media, however it is too aggressive and causes the code to ignore legitimate installable media as well due to a bad report from lsblk.

The full error text encountered that is fixed by this commit is as follows:

list index out of range
Installation failed with exception: Unexpected error while running command.
Command: ['curtin', 'block-meta', 'simple']
Exit code: 3
Reason: -
Stdout: 'list index out of range\n'
Stderr: ''

This commit will also most likely help with https://bugs.launchpad.net/curtin/+bug/1451490 as I believe they are encountering a very similar problem as the one I was.

For reference, the specific hardware that this error is happening on:
HP proliant DL100 g2
IBM system x3650 M1
An old server from PSSC labs

So this problem does not seem to be constrained to just 1 vendor.

[Scott Moser (smoser)]

Hey, this looks good.

2 questions, though
a.) can we file a bug on lsblk stating that your devices are not removable?
b.) did this fail to install precise? trusty? wily?
  Just wondering to see if lsblk is fixed in newer ubuntu possibly.

[Robert Clark (returntoreptar)]

Thanks for the reply!

a.) Done, bug report filed here: https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/1532062 I will follow up with any other information needed there.

b.) I tried this on trusty and wily, both failed with the exact same error, so it seems this is still an issue in the latest ubuntu releases.

[Robert Clark (returntoreptar)]

review: Approve

[Robert Clark (returntoreptar)]

Is there anything else I need to do to this merge request at this point or is it just waiting on additional review?

[Dave Chiluk (chiluk)]

IIUC, I think Scott's point was that if it fails on the newer Ubuntu's then a bug needs to be opened against the upstream for lsblk as well, so that it correctly reports your drives as not removable.

=== modified file 'curtin/commands/block_meta.py'
--- curtin/commands/block_meta.py	2016-01-04 18:44:23 +0000
+++ curtin/commands/block_meta.py	2016-01-06 00:02:23 +0000
@@ -1150,6 +1150,18 @@
         devices = block.get_installable_blockdevs()
         LOG.warn("'%s' mode, no devices given. unused list: %s",
                  args.mode, devices)
+        # Check if the list of installable block devices is still empty after
+        # checking for block devices and filtering out the removable ones.
+        # In this case we may have a system which has its harddrives reported
+        # by lsblk incorrectly. In this case we search for installable blockdevices
+        # that are removable as a last resort before raising an exception.
+        if len(devices) == 0:
+            devices = block.get_installable_blockdevs(True)
+            LOG.warn("No non-removable, installable devices found. "
+                     "List populated with removable devices allowed: %s", devices)
+            # Fail gracefully if no devices are found, still.
+            if len(devices) == 0:
+                raise Exception("No valid target devices found that curtin can install on.")
 
     if len(devices) > 1:
         if args.devices is not None: