Merge ~raharper/curtin:fix/reuse-vg-partition-simple into curtin:master

[Ryan Harper]

block-meta: don't filter preserve=true devices, select by wipe

Now that wiping and preserving are independent values we consider
any disk or partition with a wipe setting for clearing holders.
An existing scenario where this matters is reusing a partition
which was previously part of a volume-group.

- Add a simplified vmtest scenario for reusing partition that
  was previously part of a volume-group

LP: #1837214

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:c5a55d313e6d731ed34bfc5bcb4612348ae49bd8
https://jenkins.ubuntu.com/server/job/curtin-ci/39/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/39/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/39/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/39/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/39/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/39//rebuild

review: Approve (continuous-integration)

[Michael Hudson-Doyle (mwhudson)]

Ah ha!

review: Approve

[Paride Legovini (paride)]

review: Approve

074f345... by Ryan Harper on 2020-04-20

Adjust docstring, remove filter comment.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:074f345594ed753a70100d856cb8617affc69e55
https://jenkins.ubuntu.com/server/job/curtin-ci/41/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/41/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/41/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/41/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/41/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/41//rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:074f345594ed753a70100d856cb8617affc69e55
https://jenkins.ubuntu.com/server/job/curtin-ci/45/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/45/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/45/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/45/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/45/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/45//rebuild

review: Approve (continuous-integration)

diff --git a/curtin/commands/block_meta.py b/curtin/commands/block_meta.py
index e3c4ce1..65e779f 100644
--- a/curtin/commands/block_meta.py
+++ b/curtin/commands/block_meta.py
@@ -1692,8 +1692,8 @@ def zfs_handler(info, storage_config):
 
 
 def get_device_paths_from_storage_config(storage_config):
-    """Returns a list of device paths in a storage config filtering out
-       preserved or devices which do not have wipe configuration.
+    """Returns a list of device paths in a storage config which have wipe
+       config enabled.
 
     :param: storage_config: Ordered dict of storage configation
     """
@@ -1701,8 +1701,6 @@ def get_device_paths_from_storage_config(storage_config):
     for (k, v) in storage_config.items():
         if v.get('type') in ['disk', 'partition']:
             if config.value_as_boolean(v.get('wipe')):
-                if config.value_as_boolean(v.get('preserve')):
-                    continue
                 try:
                     dpaths.append(
                         get_path_to_storage_volume(k, storage_config))
diff --git a/examples/tests/preserve-partition-wipe-vg-simple.yaml b/examples/tests/preserve-partition-wipe-vg-simple.yaml
new file mode 100644
index 0000000..662f34f
--- /dev/null
+++ b/examples/tests/preserve-partition-wipe-vg-simple.yaml
@@ -0,0 +1,62 @@
+showtrace: true
+
+bucket:
+  - &setup |
+    parted /dev/disk/by-id/virtio-disk-a --script -- \
+        mklabel gpt                   \
+        mkpart primary ext4 2MiB 4MiB \
+        set 1 bios_grub on            \
+        mkpart primary ext4 4GiB 7GiB
+    udevadm settle
+    ls -al /dev/disk/by-id
+    vgcreate --force --zero=y --yes root_vg /dev/disk/by-id/virtio-disk-a-part2
+    pvscan --cache
+    vgscan --mknodes --cache
+    vgs
+    lvcreate root_vg --name lv1_root --zero=y --wipesignatures=y \
+        --size 2G
+    udevadm settle
+    lvs
+    ls -al /dev/disk/by-id
+
+# Partition the disk now to test curtin's reuse of partitions.
+early_commands:
+  00-setup-disk: [sh, -exuc, *setup]
+
+storage:
+  config:
+  - ptable: gpt
+    serial: disk-a
+    preserve: true
+    name: disk-a
+    grub_device: true
+    type: disk
+    id: disk-sda
+  - device: disk-sda
+    size: 2097152
+    flag: bios_grub
+    preserve: true
+    number: 1
+    type: partition
+    id: disk-sda-part-1
+  - device: disk-sda
+    size: 3G
+    flag: linux
+    preserve: true
+    number: 2
+    wipe: zero
+    type: partition
+    id: disk-sda-part-2
+  - fstype: ext4
+    volume: disk-sda-part-2
+    preserve: false
+    type: format
+    id: format-0
+  - device: format-0
+    path: /
+    type: mount
+    id: mount-0
+  version: 1
+
+verbosity: 3
+
diff --git a/tests/vmtests/test_preserve_partition_wipe_vg.py b/tests/vmtests/test_preserve_partition_wipe_vg.py
index b779ad1..96346ff 100644
--- a/tests/vmtests/test_preserve_partition_wipe_vg.py
+++ b/tests/vmtests/test_preserve_partition_wipe_vg.py
@@ -33,4 +33,27 @@ class FocalTestPreserveWipeLvm(relbase.focal, TestPreserveWipeLvm):
     __test__ = True
 
 
+class TestPreserveWipeLvmSimple(VMBaseClass):
+    conf_file = "examples/tests/preserve-partition-wipe-vg-simple.yaml"
+    uefi = False
+    extra_collect_scripts = [textwrap.dedent("""
+        cd OUTPUT_COLLECT_D
+        ls /opt > ls-opt
+        exit 0
+        """)]
+
+
+class BionicTestPreserveWipeLvmSimple(relbase.bionic,
+                                      TestPreserveWipeLvmSimple):
+    __test__ = True
+
+
+class EoanTestPreserveWipeLvmSimple(relbase.eoan, TestPreserveWipeLvmSimple):
+    __test__ = True
+
+
+class FocalTestPreserveWipeLvmSimple(relbase.focal, TestPreserveWipeLvmSimple):
+    __test__ = True
+
+
 # vi: ts=4 expandtab syntax=python