Merge lp:~raharper/curtin/set_bcache_cache_mode into lp:~curtin-dev/curtin/trunk

[Ryan Harper]

Update curtin to write out cache mode for bcache devices if 'cache_mode' is present in storage config for bcache devices.
Update vmtest for bcache to confirm that we can set cache_mode; note it also confirms that the setting is persistent since we reboot between curtin install and vmtest validation.

Fixes: lp:1510334

[Scott Moser (smoser)]

one suggestion in line. other than that, looks good.
and if i'm wrong on that, then go ahead and commit anyway.

review: Approve

[lp:~raharper/curtin/set_bcache_cache_mode]

296. By Ryan Harper on 2015-10-30

Find bcache device name via backing device holders

Add function get_holders which reads /sys/class/block/<dev>/holders
Use get_holders to list the bcache backing device holders to find the
assigned bcache device name

297. By Ryan Harper on 2015-10-30

vmtest: switch bcache test to use check_file_stripped_line

Use common check function in bcache testcase

298. By Ryan Harper on 2015-10-30

make check fixes for bcache cache_mode changes

Apply cleanups as found by make check

299. By Ryan Harper on 2015-10-30

vmtest: switch bcache test to use regex

After looking at how stripped line output works, using a
regex is more flexible in the face of bcache adding/removing
cache modes.

[Ryan Harper (raharper)]

I've addressed two things:

1) Blake highlighted that relying the the bcache storage config id isn't robust; it could be any string though in practice it tends to the the correct value. Instead, use the backing device to look up the bcache device name via sysfs holders dir.

2) Switch testcase to use check_file_regex

=== modified file 'curtin/commands/block_meta.py'
--- curtin/commands/block_meta.py	2015-10-26 10:02:13 +0000
+++ curtin/commands/block_meta.py	2015-10-30 13:40:20 +0000
@@ -155,6 +155,22 @@
         util.subp(cmd, rcs=[0, 1, 2, 5], capture=True)
 
 
+def get_holders(devname):
+    if not devname:
+        return []
+
+    devname_sysfs = \
+        '/sys/class/block/{}/holders'.format(os.path.basename(devname))
+    if not os.path.exists(devname_sysfs):
+        err = ('No sysfs path to device holders:'
+               ' {}'.format(devname_sysfs))
+        LOG.error(err)
+        raise ValueError(err)
+
+    LOG.debug('Getting blockdev holders: {}'.format(devname_sysfs))
+    return os.listdir(devname_sysfs)
+
+
 def clear_holders(sys_block_path):
     holders = os.listdir(os.path.join(sys_block_path, "holders"))
     LOG.info("clear_holders running on '%s', with holders '%s'" %
@@ -882,6 +898,8 @@
                                                 storage_config)
     cache_device = get_path_to_storage_volume(info.get('cache_device'),
                                               storage_config)
+    cache_mode = info.get('cache_mode', None)
+
     if not backing_device or not cache_device:
         raise ValueError("backing device and cache device for bcache must be \
                 specified")
@@ -909,6 +927,25 @@
             fp.write(path)
             fp.close()
 
+    if cache_mode:
+        # find the actual bcache device name via sysfs using the
+        # backing device's holders directory.
+        holders = get_holders(backing_device)
+
+        if len(holders) != 1:
+            err = ('Invalid number of holding devices:'
+                   ' {}'.format(holders))
+            LOG.error(err)
+            raise ValueError(err)
+
+        [bcache_dev] = holders
+        LOG.info("Setting cache_mode on {} to {}".format(bcache_dev,
+                                                         cache_mode))
+        cache_mode_file = \
+            '/sys/block/{}/bcache/cache_mode'.format(info.get('id'))
+        with open(cache_mode_file, "w") as fp:
+            fp.write(cache_mode)
+
     if info.get('name'):
         # Make dname rule for this dev
         make_dname(info.get('id'), storage_config)

=== modified file 'examples/tests/mdadm_bcache.yaml'
--- examples/tests/mdadm_bcache.yaml	2015-08-12 14:26:23 +0000
+++ examples/tests/mdadm_bcache.yaml	2015-10-30 13:40:20 +0000
@@ -46,6 +46,7 @@
        name: cached_array
        backing_device: mddevice
        cache_device: sda5
+       cache_mode: writeback
      - id: sda1_root
        type: format
        fstype: ext4

=== modified file 'tests/vmtests/test_mdadm_bcache.py'
--- tests/vmtests/test_mdadm_bcache.py	2015-10-26 10:39:59 +0000
+++ tests/vmtests/test_mdadm_bcache.py	2015-10-30 13:40:20 +0000
@@ -50,13 +50,15 @@
         cd OUTPUT_COLLECT_D
         bcache-super-show /dev/vda6 > bcache_super_vda6
         ls /sys/fs/bcache > bcache_ls
+        cat /sys/block/bcache0/bcache/cache_mode > bcache_cache_mode
         """)]
     fstab_expected = {
         '/dev/bcache0': '/media/data'
     }
 
     def test_bcache_output_files_exist(self):
-        self.output_files_exist(["bcache_super_vda6", "bcache_ls"])
+        self.output_files_exist(["bcache_super_vda6", "bcache_ls",
+                                 "bcache_cache_mode"])
 
     def test_bcache_status(self):
         bcache_cset_uuid = None
@@ -68,6 +70,9 @@
         with open(os.path.join(self.td.mnt, "bcache_ls"), "r") as fp:
             self.assertTrue(bcache_cset_uuid in fp.read().splitlines())
 
+    def test_bcache_cachemode(self):
+        self.check_file_regex("bcache_cache_mode", r"\[writeback\]")
+
 
 class WilyTestMdadmBcache(TestMdadmBcacheAbs):
     __test__ = True