Merge ~raharper/curtin:fix/lvm-over-bcache into curtin:master

[Ryan Harper]

clear-holders: increase the level for devices with holders by one

In the case where clear-holders looks at a dependent device, if the
parent is already in the registry, we want to take the max level
and then increment this by one to ensure the dependent device is
shutdown first. This resolves the case where we have an LVM over
top a bcache device and we need to remove the LVM device before
any of the bcache devices can be removed as they may share a
cacheset in which case all bcache devices must be stopped before
the cacheset can be removed.

LP: #1844543

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:366110456364767cf8048a6637095d8863734ca6
https://jenkins.ubuntu.com/server/job/curtin-ci/3725/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/3725/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/3725/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/3725/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=torkoal/3725/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/3725//rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:c5897529e377c9afca83aea0256a1901832e8536
https://jenkins.ubuntu.com/server/job/curtin-ci/3726/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/3726/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/3726/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/3726/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=torkoal/3726/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/3726//rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:c5897529e377c9afca83aea0256a1901832e8536
https://jenkins.ubuntu.com/server/job/curtin-ci/3727/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/3727/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/3727/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/3727/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=torkoal/3727/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/3727//rebuild

review: Approve (continuous-integration)

[Chad Smith (chad.smith)]

Thanks for this test coverage. LGTM

review: Approve

diff --git a/curtin/block/clear_holders.py b/curtin/block/clear_holders.py
index bcc16e6..fb778f0 100644
--- a/curtin/block/clear_holders.py
+++ b/curtin/block/clear_holders.py
@@ -487,7 +487,7 @@ def plan_shutdown_holder_trees(holders_trees):
         # anything else regardless of whether it was added to the tree via
         # the cache device or backing device first
         if device in reg:
-            level = max(reg[device]['level'], level)
+            level = max(reg[device]['level'], level) + 1
 
         reg[device] = {'level': level, 'device': device,
                        'dev_type': tree['dev_type']}
diff --git a/examples/tests/bcache-ceph-nvme-simple.yaml b/examples/tests/bcache-ceph-nvme-simple.yaml
new file mode 100644
index 0000000..83cb04c
--- /dev/null
+++ b/examples/tests/bcache-ceph-nvme-simple.yaml
@@ -0,0 +1,107 @@
+storage:
+    config:
+    -   grub_device: true
+        id: sda
+        model: MM1000GBKAL
+        name: sda
+        ptable: gpt
+        serial: disk-a
+        type: disk
+        wipe: superblock
+    -   id: sdb
+        model: MM1000GBKAL
+        name: sdb
+        serial: disk-b
+        type: disk
+        wipe: superblock
+    -   id: nvme0n1
+        model: INTEL SSDPEDME400G4
+        name: nvme0n1
+        serial: nvme-CVMD552400
+        type: disk
+        wipe: superblock
+    -   backing_device: sdb
+        cache_device: nvme0n1
+        cache_mode: writeback
+        id: bcache1
+        name: bcache1
+        type: bcache
+    -   device: sda
+        id: sda-part1
+        name: sda-part1
+        number: 1
+        offset: 4194304B
+        size: 5G
+        type: partition
+        uuid: 1e27e7af-26dc-4af4-9ef5-aea928204997
+        wipe: superblock
+    -   device: sda
+        id: sda-part2
+        name: sda-part2
+        number: 2
+        size: 2G
+        type: partition
+        uuid: 0040d622-41f1-4596-842f-82d731ba9054
+        wipe: superblock
+    -   device: sda
+        id: sda-part3
+        name: sda-part3
+        number: 3
+        size: 2G
+        type: partition
+        uuid: cb59d827-662c-4da6-b1ef-7967218bd0db
+        wipe: superblock
+    -   backing_device: sda-part3
+        cache_device: nvme0n1
+        cache_mode: writeback
+        id: bcache0
+        name: bcache0
+        type: bcache
+    -   fstype: fat32
+        id: sda-part1_format
+        label: efi
+        type: format
+        uuid: 27638478-d881-43e5-a93c-1cac7aa60daa
+        volume: sda-part1
+    -   fstype: ext4
+        id: sda-part2_format
+        label: boot
+        type: format
+        uuid: cfd11d4f-d77f-4307-b372-b52e81c873f7
+        volume: sda-part2
+    -   fstype: ext4
+        id: bcache0_format
+        label: root
+        type: format
+        uuid: 63247841-195c-4939-83e4-cb834d61f95f
+        volume: bcache0
+    -   devices:
+        - bcache1
+        id: ceph-bcache-vg
+        name: ceph-bcache-vg
+        type: lvm_volgroup
+    -   id: ceph-bcache-lv-0
+        name: ceph-bcache-lv-0
+        size: 3G
+        type: lvm_partition
+        volgroup: ceph-bcache-vg
+    -   fstype: xfs
+        id: ceph-bcache-lv-0_format
+        volume: ceph-bcache-lv-0
+        type: format
+    -   device: bcache0_format
+        id: bcache0_mount
+        options: ''
+        path: /
+        type: mount
+    -   device: sda-part2_format
+        id: sda-part2_mount
+        options: ''
+        path: /boot
+        type: mount
+    -   device: sda-part1_format
+        id: sda-part1_mount
+        options: ''
+        path: /boot/efi
+        type: mount
+    version: 1
diff --git a/tests/vmtests/test_bcache_ceph.py b/tests/vmtests/test_bcache_ceph.py
index c35ee85..07d9bca 100644
--- a/tests/vmtests/test_bcache_ceph.py
+++ b/tests/vmtests/test_bcache_ceph.py
@@ -82,4 +82,27 @@ class DiscoTestBcacheCeph(relbase.disco, TestBcacheCeph):
 class EoanTestBcacheCeph(relbase.eoan, TestBcacheCeph):
     __test__ = True
 
+
+class TestBcacheCephLvm(TestBcacheCeph):
+    test_type = 'storage'
+    nr_cpus = 2
+    uefi = True
+    dirty_disks = True
+    extra_disks = ['20G', '20G']
+    nvme_disks = ['20G']
+    conf_file = "examples/tests/bcache-ceph-nvme-simple.yaml"
+
+    @skip_if_flag('expected_failure')
+    def test_bcache_output_files_exist(self):
+        self.output_files_exist([
+            "bcache-super-show.vda3",
+            "bcache-super-show.vdc",
+            "bcache-super-show.nvme0n1",
+        ])
+
+
+class BionicTestBcacheCephLvm(relbase.bionic, TestBcacheCephLvm):
+    __test__ = True
+
+
 # vi: ts=4 expandtab syntax=python