~smoser/curtin:cleanup/curtainer-patch-version

Last commit made on 2018-05-23
Get this branch:
git clone -b cleanup/curtainer-patch-version https://git.launchpad.net/~smoser/curtin
Only Scott Moser can upload to this branch. If you are Scott Moser please log in for upload directions.

Branch merges

Branch information

Name:
cleanup/curtainer-patch-version
Repository:
lp:~smoser/curtin

Recent commits

9ca81ae... by Scott Moser

flip the logic.

patch the source by default.

b91e519... by Scott Moser

better identify where that .dist file came from

33836a6... by Scott Moser

curtainer: add --patch-version option

After adding 'CURTIN_EXE_VERSION' and 'CURTIN_VERSION' in output
we now see nice output for runs from trunk. But when running from
a package via curtainer, we see:
  CURTIN_VMTEST_CURTIN_EXE=.. curtin-vmtest-proposed-a-29 curtin
  CURTIN_VMTEST_CURTIN_EXE_VERSION=18.1-17-gae48e86f-0ubuntu1~17.10.1
  CURTIN_VMTEST_CURTIN_VERSION=18.1

That is confusing as it appears our vmtest version is different
from our exe version.

That is because the patching of the version that we do in debian/rules
only applies to the built binaries, the source still has the un-modified
@@PACKAGED_VERSION@@ in tact.

The change here is to add '--patch-version' to curtainer. It will
then know how to patch the version into the source it produces
so that running 'curtin' from that source will provide the correct
output. Note that curtin has verified that it downloaded the version
of the source that it is patching in, so this is known-to-be-correct.

48b24ca... by Scott Moser

pyflakes: fix unused variable references identified by pyflakes 2.0.0.

A newer version of pyflakes (2.0.0) was released over the last weekend.
It identifed some unused variables that version 1.6.0 did not identify.
The change here merely fixes those unused variables.

The fact that c-i hit this is not intended. A future fix is needed to
properly pin the version of pyflakes so that it does not happen again.

ae48e86... by Scott Moser

tests: replace usage of mock.assert_called

assert_called is only in mock 2 or newer, meaning this was failing
to run tests in a xenial build environment.

So just replace assert_called usage with assertTrue(mymock.called).

1883584... by Scott Moser

tools: jenkins-runner show curtin version in output.

I was looking at output of a jenkins run, and it was very non-clear
what actual version of curtin was running. The change here will
get 'curtin version' output into the CURTIN_VMTEST_VERSION variable
which then gets printed out with all the other CURTIN_VMTEST_* variables.

So, we see something like:
  CURTIN_VMTEST_CURTIN_EXE=./tools/curtin-from-container xc1 curtin
  CURTIN_VMTEST_CURTIN_EXE_VERSION=18.1-619-g4bf7750-0ubuntu1
  CURTIN_VMTEST_CURTIN_VERSION=18.1-17-g0fc297bd

5313b9e... by Ryan Harper

zfs: implement a supported check to handle i386

The zfs kernel module is not available for i386 so curtin cannot
expect that missing that module is fatal. Concentrate the various
checks for when we load the zfs kernel module into the zfs python
module. Add vmtest to exercise the i386 path.

LP: #1768709

9e6e66a... by Scott Moser

Support mount entries not tied to a device, including bind and tmpfs.

Previously an entry in the storage configuration of type 'mount'
was required to have a 'device' entry that was a reference to
an id of type 'format'. That meant that there was no way to add
a mount for anything that wasn't a traditional filesystem.

The change here allows a type 'mount' entry to not have a 'device'
as long as it has a 'spec' and a 'fstype' entry. The word 'spec'
comes from fstab(5).

It also adds support for the mount type to have a 'passno' and 'freq'
which are entries in a fstab file.

LP: #1767979

4bf7750... by Ryan Harper

block/clear_holders/mdadm: refactor handling of layered device wiping

The block layer handling of checking efi/dos signatures did not ensure
that it always used a devpath (/dev/<kname>) which resulted in
failures when attempting to identify dos partitions, including
extended partitions due to attempting to open /sys/class/block/<kname>
to look for metadata. In addition there is a window where
wiping/removing lvm devices in dos logical partitions renders the
/dev/<kname> path to the extended partition missing, no /dev/<kname>
We now re-read a block device's partition table after completing a
shutdown of each device. Doing this re-read triggered mdadm devices
which had been stopped to resume preventing wiping of the underlying
devices. A refactor of how raid devices are shutdown now includes
collecting the list of raid members and spares, failing each element
out of the array, stopping the rescan/rebuild of the array and finally
removing the mdadm metadata on the underlying devices.

With this in place, we can now successful run the LVM test-cases with
dirty_disk mode enabled and simultaneously continue to pass the
complex bcache/raid examples.

LP: #1768893
LP: #1769742

736808c... by Ryan Harper

clear_holders: only export zpools that have been imported

Curtin may encounter disks from a previous install which are
zfs_members but are part of a pool which isn't currently
imported. Attempting to export a pool that is not imported
results in a failure. Instead, only export a pool if it's
found in the list of currently imported pools.

Also enable Bionic ZfsRoot vmtest (BionicTestZfsRootFsType)
LP: #1770280