vmtests: switch to MAAS v3 streams for images and kernels
Switching to v3 streams enables testing of Xenial HWE and HWE-edge
kernels and the squashfs root which MAAS 2.3 uses.
- Add v3 streams URL
- Introduce two attributes, ephemeral_ftype, target_ftype to determine
which files are needed for the ephemeral environment and which for the
target
- Add kflavor and subarch filters to streams query and release classes to
support kernel variants: ga, hwe, hwe-edge. We're currently only testing
subarch 'generic' and ignoring 'low-latency'
- When testing with hwe and hwe-edge, emit a kernel configuration to
curtin to ensure we install the matching kernel in the target image
- Make sure vmtests append collect scripts to the base class parameter
so all tests run the BaseVM collect_scripts (dpkg-query)
- Introduce new release classes, XenialGA, XenialHWE and XenialEdge
Note, XenialGA replaces the general Xenial release class, and that
not all test-cases run Xenial HWE/Edge kernels; most of the storage
ones do.
- Booting from a squashfs image requires some changes
- publish the squashfs image and boot VMs with root=URL (except trusty)
- utilize overlayfs to keep the ephemeral environment pristine and
use the squashfs image as a the curtin install source.
- Add some boot parameters to the ephemeral environment to enable
networking, including DNS for package installation.
- Use BOOTIF= parameter for multi-nic vmtests to ensure we only bring
up one nic for ephemeral networking. Sometimes multiple nics would
come online and then cause networking hiccups which prevented
installs from completing.
- Modify vmtest storage configurations for disks to all use a 'serial'
attribute. Currently we use 'disk-X' as the serial value where X
increments from 'a' -> 'z' as the test needs.
- jenkins-runner now prints additional information about what images
are found and available.
- tools/launch
- modify --append to allow multiple parameters to grow the array.
- fix bug in --publish which prevented more than a single file being
published
- allow PUBURL to be expanded in root-arg
- add a 'boot-image' parameter which is exported to the VM with a serial
number to provide a consistent path to the device,
/dev/disk/by-id/virtio-boot-disk
- drop '2G' boot size parameter to boot disks, instead allow qemu-img to
default to the size of backing image.
tests: update pack tests to clear out pyc files more completely.
When the test test_curtin_help_has_hacked_version was first written,
the file .pyc file that was known to be created for
my_path/my.py was my_path/my.pyc
Newer versions of python (at least in cpython 3.6), create:
version/__pycache__/version.cpython-36.pyc
This updates the test to remote those pyc files also.
The failure to remove the .pyc file results in a race condition
where sometimes the .pyc file is seen as valid even though the
.py file was written after it.
pack: fix packing when curtin is installed inside a snap.
When curtin is installed as part of a MAAS snap, then it would fail
to find its helpers directory. That is because instead of helpers
living in usr/lib/curtin/helpers they will be in
/snap/maas/<version>/usr/lib/curtin/helpers
But helpfully, SNAP will be set in the environment to point to
/snap/maas/<version>/
The fix is to adjust pack to trust the environment variable
and pack up the installed files based on that prefix.