vmtest: fix pollinate re.search to use groupdict()
In python3.6 re.search() returns an SRE_Match which has
support for dictionary indexing, however on older python3
we do not. Fallback to the supported SRE_Match.groupdict()
before indexing named matches.
The pollinate client in Ubuntu images supports custom data to be
appended to the user-agent value sent when contacting the pollen
server. This patch allows users to add any key/value pairs which
will be added to the pollinate client configuration.
By default, curtin will configure pollinate with the curtin version
and if being deployed by MAAS, the MAAS version. Users can also
disable configuration of the pollinate client.
Additionally fix the vmtest webhook logger to encode writes when
handling GETs.
vmtest: Remove xenial and trusty specific workaround for rooturl.
xenial and trusty required specifying rc-initrd-dns on the kernel
command line in order to enable dns transition from initramfs to
real root (rooturl). Those changes are now fixed in both releases
so we can remove this code.
The changes made the archive today. We would expect they
are in any daily image with a serial newer than that 20180212.
This test exercises filesystem creation, mount and unmount of all
supported filesystem types.
Also here is a fix for jfs filesystem creation. mkfs_jfs requires a '-q'
argument to force/quiet. Otherwise it will prompt to ask if you
really want to do that.
Two issues with dirty_disk mode. First, after the early
stage runs, we need to tear down the raid array so we can
disconnect the iscsi devices, second even when that is done
there is a bug in mdadm version in Artful and Bionic which
prevents successful testing (LP: #1753786)
tools/curtainer: Try harder to get source at binary version.
When we test curtin from the daily PPA, sometimes the source version
that is available is newer than the binary version available. That
causes mismatch of vmtest:code.
If 'apt-get source curtin/<version>' fails, then ask apt
what the url to the deb for 'curtin' at 'version' is.
Then replace _all.deb with .dsc and try dget on that url. This
should work because:
a.) curtin is a binary and a source package.
b.) source are generally published along side binaries.
c.) in the ppa case, the files are likely still available but only
the most current is published in the Sources for the archive.
If this proves to not work, then the next solution is to actually
provide a binary package with the files needed for vmtest.
clear_holders: wipe complex devices before disassembly
The curtin clear-holders code did not wipe the contents of the
assembled devices prior to shutting them down. In cases where
curtin re-assembles the same devices, in a RAID for example, then
when the RAID device comes on-line, it may prevent some tools from
creating new metadata on the target device. In particular, the bcache tools
do not want to overwrite an existing bcache superblock resulting in
a failed deployment when layering a bcache over a RAID volume.
The fix is in two parts. First, clear-holders will wipe the superblock of
the assembled device before it breaks it apart. Second, when creating a
bcache backing device, check first if the target device is already claimed
by the bcache layer; if so then raise a RuntimeError as clear holders should
have removed this device.
curtin.block
- expose 'exclusive' boolean and pass it through to context manager
curtin.block.clear_holders
- Add logic to wipe assembled devices before calling shutdown function
- introduce internal _wipe_superblock which only contains the retry
logic
- switch to using dmsetup remove for lvm destruction
- improve identify_raid, it incorrectly identified partitions on RAID0 and
RAID1 devices as raid devices, but they are actually partitions.
