Merge lp:~wesley-wiedenmeier/curtin/curtin-fix-sysfs-partition-data into lp:~curtin-dev/curtin/trunk

[Wesley Wiedenmeier]

The current implementation of sysfs_partition_data only works when a blockdev to a whole disk is specified. It does not work when the blockdev for a partition is specified or when only a sysfs path is specified. This function is used quite a decent bit by the storage tests in
lp:~wesley-wiedenmeier/curtin/partial-testing so the fix is useful even though it doesn't look like it will fix any current bugs.

See: http://paste.ubuntu.com/17082198/
Output with fix: http://paste.ubuntu.com/17082212/

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:379
https://server-team-jenkins.canonical.com/job/curtin-ci/240/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/237/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci/240/rebuild

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

This is good improvement. Comments on simplifying below.

review: Needs Fixing

[lp:~wesley-wiedenmeier/curtin/curtin-fix-sysfs-partition-data]

380. By Wesley Wiedenmeier on 2016-06-07

Improve code for sysfs_partition_data by ensuring that both blockdev and
sysfs_path will be set.

[Wesley Wiedenmeier (wesley-wiedenmeier)]

Makes sense, I just switched it over and it looks much nicer this way

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:380
https://server-team-jenkins.canonical.com/job/curtin-ci/242/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/239/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci/242/rebuild

review: Approve (continuous-integration)

[lp:~wesley-wiedenmeier/curtin/curtin-fix-sysfs-partition-data]

381. By Wesley Wiedenmeier on 2016-07-13

Cleanup to partnum assignment in sysfs_partition_data() based on diff comments
from:
https://code.launchpad.net/~wesley-wiedenmeier/curtin/partial-testing/+merge/297958

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:381
https://server-team-jenkins.canonical.com/job/curtin-ci-ppc64/11/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/307/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci-ppc64/11/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:381
https://server-team-jenkins.canonical.com/job/curtin-ci-ppc64/12/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/309/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci-ppc64/12/rebuild

review: Approve (continuous-integration)

[lp:~wesley-wiedenmeier/curtin/curtin-fix-sysfs-partition-data]

382. By Wesley Wiedenmeier on 2016-08-08

Merge from trunk

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:382
https://jenkins.ubuntu.com/server/job/curtin-ci/228/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=amd64/228
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=ppc64el/228

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/228/rebuild

review: Approve (continuous-integration)

[lp:~wesley-wiedenmeier/curtin/curtin-fix-sysfs-partition-data]

383. By Wesley Wiedenmeier on 2016-08-23

Merge from trunk

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:383
https://jenkins.ubuntu.com/server/job/curtin-ci/249/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=amd64/249
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=ppc64el/249

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/249/rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:383
https://jenkins.ubuntu.com/server/job/curtin-ci/400/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/400
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/400
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/400
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/400
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=vm-i386/400

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/400/rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:383
https://jenkins.ubuntu.com/server/job/curtin-ci/417/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/417
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/417
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/417
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/417
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=vm-i386/417

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/417/rebuild

review: Approve (continuous-integration)

[Joshua Powers (powersj)]

Moving to WIP

Unmerged revisions

383. By Wesley Wiedenmeier on 2016-08-23

Merge from trunk

=== modified file 'curtin/block/__init__.py'
--- curtin/block/__init__.py	2016-07-31 01:17:53 +0000
+++ curtin/block/__init__.py	2016-08-23 00:02:14 +0000
@@ -558,31 +558,33 @@
 def sysfs_partition_data(blockdev=None, sysfs_path=None):
     # given block device or sysfs_path, return a list of tuples
     # of (kernel_name, number, offset, size)
-    if blockdev is None and sysfs_path is None:
-        raise ValueError("Blockdev and sysfs_path cannot both be None")
-
     if blockdev:
+        blockdev = os.path.normpath(blockdev)
         sysfs_path = sys_block_path(blockdev)
-
-    ptdata = []
-    # /sys/class/block/dev has entries of 'kname' for each partition
+    elif sysfs_path:
+        # use normpath to ensure that paths with trailing slash work
+        sysfs_path = os.path.normpath(sysfs_path)
+        blockdev = os.path.join('/dev', os.path.basename(sysfs_path))
+    else:
+        raise ValueError("Blockdev and sysfs_path cannot both be None")
 
     # queue property is only on parent devices, ie, we can't read
     # /sys/class/block/vda/vda1/queue/* as queue is only on the
     # parent device
+    sysfs_prefix = sysfs_path
     (parent, partnum) = get_blockdev_for_partition(blockdev)
-    sysfs_prefix = sysfs_path
     if partnum:
         sysfs_prefix = sys_block_path(parent)
-
-    block_size = int(util.load_file(os.path.join(sysfs_prefix,
-                                    'queue/logical_block_size')))
-
-    block_size = int(
-        util.load_file(os.path.join(sysfs_path, 'queue/logical_block_size')))
+        partnum = int(partnum)
+
+    block_size = int(util.load_file(os.path.join(
+        sysfs_prefix, 'queue/logical_block_size')))
     unit = block_size
-    for d in os.listdir(sysfs_path):
-        partd = os.path.join(sysfs_path, d)
+
+    # /sys/class/block/dev has entries of 'kname' for each partition
+    ptdata = []
+    for d in os.listdir(sysfs_prefix):
+        partd = os.path.join(sysfs_prefix, d)
         data = {}
         for sfile in ('partition', 'start', 'size'):
             dfile = os.path.join(partd, sfile)
@@ -591,8 +593,9 @@
             data[sfile] = int(util.load_file(dfile))
         if 'partition' not in data:
             continue
-        ptdata.append((d, data['partition'], data['start'] * unit,
-                       data['size'] * unit,))
+        if partnum is None or data['partition'] == partnum:
+            ptdata.append((d, data['partition'], data['start'] * unit,
+                           data['size'] * unit,))
 
     return ptdata