Merge lp:~wesley-wiedenmeier/curtin/curtin-1543263 into lp:~curtin-dev/curtin/trunk

[Wesley Wiedenmeier]

partitioning: limited support for odd ordering of partition numbers

This fixes a bug in the partitioning code where it assumes that the
previous partition is numbered N-1. That is not always the case.
The change here is very focused on:
 a.) fixing bug 1543263 and allowing maas to send configuration
     for power that "hides" the PReP partition as number 15
     so they don't have to really show it to the user.
 b.) Being non-invasive.

Ultimately, we want to re-write the partitioning code to walk
through the storage config and correctly fill in number, start
and end for each entry. That is a more invasive and time-consuming
change though.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:350
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~wesley-wiedenmeier/curtin/curtin-1543263/+merge/285825/+edit-commit-message

https://server-team-jenkins.canonical.com/job/curtin-ci/159/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/156/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci/159/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:350
https://server-team-jenkins.canonical.com/job/curtin-ci/160/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/157/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci/160/rebuild

review: Needs Fixing (continuous-integration)

[Scott Moser (smoser)]

nitpicks in line, but need fixing.

review: Needs Fixing

[lp:~wesley-wiedenmeier/curtin/curtin-1543263]

351. By Wesley Wiedenmeier on 2016-02-12

Dropped print statements and added a debug message for partition number

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:351
https://server-team-jenkins.canonical.com/job/curtin-ci/161/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/158/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci/161/rebuild

review: Needs Fixing (continuous-integration)

[lp:~wesley-wiedenmeier/curtin/curtin-1543263]

352. By Wesley Wiedenmeier on 2016-02-12

Use better style string formatting for syslog

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:352
https://server-team-jenkins.canonical.com/job/curtin-ci/162/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/159/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci/162/rebuild

review: Needs Fixing (continuous-integration)

[Scott Moser (smoser)]

I'm running a test with your suggested vmtest change at http://paste.ubuntu.com/15021746/
I approve of this for nwo.

review: Approve

[lp:~wesley-wiedenmeier/curtin/curtin-1543263]

353. By Wesley Wiedenmeier on 2016-02-12

Added test case for partition numbering

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:353
https://server-team-jenkins.canonical.com/job/curtin-ci/163/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/160/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci/163/rebuild

review: Needs Fixing (continuous-integration)

=== modified file 'curtin/commands/block_meta.py'
--- curtin/commands/block_meta.py	2016-02-11 18:52:29 +0000
+++ curtin/commands/block_meta.py	2016-02-12 04:14:05 +0000
@@ -503,6 +503,19 @@
     return logicaldisks
 
 
+def find_previous_partition(disk_id, part_id, storage_config):
+    last_partnum = 0
+    for item_id, command in storage_config.items():
+        if item_id == part_id:
+            return last_partnum
+        if command['type'] == 'partition' and command['device'] == disk_id:
+            if 'number' in item_id:
+                last_partnum = item_id['number']
+            else:
+                last_partnum += 1
+    return None
+
+
 def partition_handler(info, storage_config):
     device = info.get('device')
     size = info.get('size')
@@ -540,8 +553,11 @@
             previous_partition = "/sys/block/%s/%s%s/" % \
                 (disk_kname, disk_kname, extended_part_no)
         else:
+            pnum = find_previous_partition(device, info['id'], storage_config)
+            LOG.debug("previous partition number for '%s' found to be '%s'",
+                      info.get('id'), pnum)
             previous_partition = "/sys/block/%s/%s%s/" % \
-                (disk_kname, disk_kname, partnumber - 1)
+                (disk_kname, disk_kname, pnum)
         with open(os.path.join(previous_partition, "size"), "r") as fp:
             previous_size = int(fp.read())
         with open(os.path.join(previous_partition, "start"), "r") as fp:

=== modified file 'examples/tests/basic.yaml'
--- examples/tests/basic.yaml	2015-12-07 15:14:55 +0000
+++ examples/tests/basic.yaml	2016-02-12 04:14:05 +0000
@@ -54,3 +54,19 @@
         type: mount
         path: /btrfs
         device: btrfs_disk_fmt_id
+      - id: pnum_disk
+        type: disk
+        path: /dev/vde
+        name: sparedisk
+        wipe: superblock
+        ptable: gpt
+      - id: pnum_disk_p1
+        type: partition
+        number: 1
+        size: 1GB
+        device: pnum_disk
+      - id: pnum_disk_p2
+        type: partition
+        number: 10
+        size: 1GB
+        device: pnum_disk

=== modified file 'tests/vmtests/test_basic.py'
--- tests/vmtests/test_basic.py	2015-12-17 22:07:00 +0000
+++ tests/vmtests/test_basic.py	2016-02-12 04:14:05 +0000
@@ -13,7 +13,7 @@
     conf_file = "examples/tests/basic.yaml"
     install_timeout = 600
     boot_timeout = 120
-    extra_disks = ['128G', '128G']
+    extra_disks = ['128G', '128G', '4G']
     disk_to_check = {'main_disk': 1, 'main_disk': 2}
     collect_scripts = [textwrap.dedent("""
         cd OUTPUT_COLLECT_D
@@ -43,6 +43,21 @@
         blkid_info = self.get_blkid_data("blkid_output_vda")
         self.assertEquals(blkid_info["PTTYPE"], "dos")
 
+    def test_partition_numbers(self):
+        # vde should have partitions 1 and 10
+        disk = "vde"
+        proc_partitions_path = os.path.join(self.td.collect,
+                                            'proc_partitions')
+        self.assertTrue(os.path.exists(proc_partitions_path))
+        found = []
+        with open(proc_partitions_path, 'r') as fp:
+            for line in fp.readlines():
+                if disk in line:
+                    found.append(line.split()[3])
+        # /proc/partitions should have 3 lines with 'vde' in them.
+        expected = [disk + s for s in ["", "1", "10"]]
+        self.assertEqual(found, expected)
+
     def test_partitions(self):
         with open(os.path.join(self.td.collect, "fstab")) as fp:
             fstab_lines = fp.readlines()