~raharper/curtin:fix/vmtest-basic-scsi-pnum-disk

Last commit made on 2020-04-20
Get this branch:
git clone -b fix/vmtest-basic-scsi-pnum-disk https://git.launchpad.net/~raharper/curtin
Only Ryan Harper can upload to this branch. If you are Ryan Harper please log in for upload directions.

Branch merges

Branch information

Name:
fix/vmtest-basic-scsi-pnum-disk
Repository:
lp:~raharper/curtin

Recent commits

7341689... by Ryan Harper on 2020-04-20

vmtest: basic use dname to lookup disk with multiple partitions

e967eeb... by Ryan Harper on 2020-04-16

block-meta: Don't check the ptable type of a disk with no ptable

When re-using a disk without a partition table we do not need to check
the current partition table.

0fed7d6... by Ryan Harper on 2020-04-16

curthooks: always use ChrootableTarget.subp when calling efibootmgr

There was a bug recently introduced when adding the feature to remove
duplicate UEFI boot entries which removed use of ChrootableTarget
subp method. The assumption was that util.subp calls would also get
the target parameter set but this was not the case. The ephemeral
environment does not usually have efibootmgr package installed but
we ensure that it is present in the target system. This branch
replaces calls to util.subp with the ChrootableTarget subp method.

dc215b1... by Ryan Harper on 2020-04-16

storage: enable and use multipath during storage configuration

Curtin has attempted to avoid using multipath enabled during installation
as that requires reworking a number of areas with in curtin that use Linux
tooling which is not multipath friendly. In particular, partitions on
multipath devices appear as disk not as partitions and this makes programs
confused. However, some scenarios just cannot be handled sanely without
multipath devices enabled. One such scenario is creating an LVM root on
top of multipath devices. This branch adds a vmtest scenario to recreate
this deployment configuration. To resolve the issue curtin will now keep
multipath enabled if available and adjust partition handling to make use
of 'kpartx' and 'dmsetup' to create, manipulate and calculate partition
related information. There was quite a bit of fallout with this change
so this branch includes a large amount of changes to handle this all the
way back to Xenial. Xenial and Bionic do not enable multipath by default,
and curtin does not enable multipath in the ephemeral environment, rather
if multipath is presnet in the ephemeral environment then curtin will
make use of it.

Additional changes needed/used to resolve this issue:

- Add --shutdown-plan to curtin clear-holders CLI tool for dumping the
  clear-holders plan of action for a given storage configuration.
- Add --probe-data to curtin block-discover CLI to dump the probert probe
  data out to a file and skip converting to a storage config.
- clear-holders will reload multipath maps to ensure mp is active.
- Add use of --filter to lvm commands to only use mpath devices if mp is
  enabled.
- curthooks now modifies target /etc/lvm.conf with a devices filter if mp
  is enabled.
- Fix dname for mpath devices, using dm-uuid and handling the oddity that
  multipath partitions prefix their dm-uuid with part-X-.
- If multipath enabled, don't hard code root devices to mpath0

LP: #1869075

0832e4e... by Ryan Harper on 2020-04-10

block-discover: detect nvme multipath devices

NVME "multipath" devices are a virtual subsystem inside the kernel
that has nothing to do with traditional multipath. The kernel nvme
driver jumps through hoops to hide a controller node and pretend that
it is a block device when it is emphatically _not_. It's not suprising
that this breaks lots of things.

These phony block devices do present a virtual NVME block device and
curtin block-discover ignores composed virtual devices when
constructing storage-config 'type: disk' devices. We must include such
devices as 'type: disk'.

This patch narrows our filter to exclude any DEVPATH which is
/devices/virtual/block and NVME mp devices use
/devices/virtual/nvme-subsystem.

LP: #1868109

3240d84... by Ryan Harper on 2020-04-09

clear-holders: Tolerate vgchange errors during discovery

Curtin may be on a system where it has an incomplete lvm and it
should not exit with error if the volume group cannot be enabled.

LP: #1870037

2ae80dd... by Ryan Harper on 2020-04-08

block-meta: handle preserve with vtoc ptable

DASD devices have a 'vtoc' partition table type. When we're handling
a disk config with 'preserve': True we raised a ValueError expecting
either 'gpt' or 'msdos'. Fix this by refactoring the code to check
the expected ptable value and adding a check_vtoc_signature method.

LP: #1871158

b7cbe9d... by Ryan Harper on 2020-04-07

vmtest: use -partition file for TestReuseRAIDMemberPartition class

638c9a7... by Ryan Harper on 2020-04-03

format: extra_options should be a list type

Switch to a list for extra_options to allow for specifying whitespace
characters in values passed to mkfs command.

- tox: add block-schema validate env

4264649... by Paride Legovini on 2020-04-01

tox: add pyflakes to the default tox run

The new py3-pyflakes environment tests the code against a specific
version of pyflakes, and it is run by default. This in contrast
with the existing tip-pyflakes environment, which tests against the
latest pyflakes version and it is not part of the default tox run.