block-schema: update raid schema for preserve and metadata
Curtin can now accept type: raid storage config which sets the
preserve key booloean to keep a raid configuration and separately
set the 'wipe' key to clear the data on the raid device.
We also accept a 'metadata' version hint in the schema as curtin
passes this value along as info to users of block-discover.
clear-holders: increase the level for devices with holders by one
In the case where clear-holders looks at a dependent device, if the
parent is already in the registry, we want to take the max level
and then increment this by one to ensure the dependent device is
shutdown first. This resolves the case where we have an LVM over
top a bcache device and we need to remove the LVM device before
any of the bcache devices can be removed as they may share a
cacheset in which case all bcache devices must be stopped before
the cacheset can be removed.
tests: mock timestamp used in collect-log file creation
The unittest for curtin collect-logs uses a timestamp to
create the filename for the tarball. The selected in the
unittest versus when the call occurs may race if the delay
between the calls is greater than 1 second. Resolve this
unittest issue by providing the same timestamp value.
ChrootableTarget: mount /run to resolve lvm/mdadm issues which require it.
Newer lvm/mdadm/grub tools are expecting to find udev and other
information /run, so remove the Centos only change to add /run
and instead have ChrootableTarget always mount /run.
Update block-discover to annotate disks and partitions that are
part of a multipath device. Using probed data, curtin now
includes a 'multipath' key in type:disk and type:partition entries.
This allows readers of extracted config to understand the
duplicated information present.
Other changes:
- Update block schema to accept 'multipath' string
- Update documentation for storage 'multipath' string
- Update block-discover-to-config tool to handle probe data
without 'storage' top level key.
- Add unittests for multipath detection in probe data.
Consider disks and partitions when checking for holders that
need removal before they can be used. Additionally when shutting
down a raid, if it is inactive or broken, writes to the /dev/mdX
device will fail which is expected, don't exit instead, log a
non-fatal error and continue with shutting down the raid.
With these fixes TestReuseRAIDMemberPartition vmtests now pass.
install: export zpools if present in the storage-config
Curtin needs to run zpool export on the any pool it created
before booting as zfs-initramfs 0.8 now requires a pool to
be exported before being imported.
the second jenkins-runner will have to wait for the first one to
finish, as it's locked by vmtest-sync-images. A full vmtest run
can take several hours. I easily spotted successful runs that
took more than 16 hours on our Jenkins log. Setting the default
timeout to something like 20 hours should be safe, but I still
don't think it's a good default, as it is too high for all the
cases where only a subset of the tests is being run. The timeout
is too dependent on the hardware, on how busy the server is,
and on the set of tests which is run. No size fits all.
Other minor changes:
- Force CURTIN_VMTEST_IMAGE_LOCK_TIMEOUT to be an integer
- Remove unnecessary quotes around fixed strings
