~raharper/curtin:fix/drop-disco-tests

Branch merges

Propose for merging

Merged into curtin:master

Server Team CI bot: Approve (continuous-integration) on 2020-02-04

Paride Legovini: Approve on 2020-02-04

Chad Smith: Approve on 2020-02-03

Dan Watkins (community): Needs Fixing on 2020-01-31

Diff: 769 lines (+56/-209)

38 files modified

tests/vmtests/test_basic.py (+0/-8)
tests/vmtests/test_basic_dasd.py (+0/-4)
tests/vmtests/test_bcache_basic.py (+0/-4)
tests/vmtests/test_bcache_bug1718699.py (+0/-4)
tests/vmtests/test_bcache_ceph.py (+0/-4)
tests/vmtests/test_bcache_partitions.py (+0/-4)
tests/vmtests/test_fs_battery.py (+0/-4)
tests/vmtests/test_iscsi.py (+0/-4)
tests/vmtests/test_journald_reporter.py (+0/-4)
tests/vmtests/test_lvm.py (+0/-4)
tests/vmtests/test_lvm_iscsi.py (+0/-4)
tests/vmtests/test_lvm_raid.py (+0/-4)
tests/vmtests/test_mdadm_bcache.py (+0/-34)
tests/vmtests/test_mdadm_iscsi.py (+0/-4)
tests/vmtests/test_multipath.py (+0/-4)
tests/vmtests/test_network.py (+0/-4)
tests/vmtests/test_network_alias.py (+0/-4)
tests/vmtests/test_network_bonding.py (+0/-4)
tests/vmtests/test_network_bridging.py (+0/-4)
tests/vmtests/test_network_ipv6.py (+0/-4)
tests/vmtests/test_network_ipv6_static.py (+0/-4)
tests/vmtests/test_network_ipv6_vlan.py (+0/-4)
tests/vmtests/test_network_mtu.py (+0/-8)
tests/vmtests/test_network_static.py (+0/-4)
tests/vmtests/test_network_static_routes.py (+0/-5)
tests/vmtests/test_network_vlan.py (+0/-4)
tests/vmtests/test_nvme.py (+0/-8)
tests/vmtests/test_pollinate_useragent.py (+0/-4)
tests/vmtests/test_preserve.py (+0/-4)
tests/vmtests/test_preserve_raid.py (+0/-4)
tests/vmtests/test_raid5_bcache.py (+0/-4)
tests/vmtests/test_reuse_raid_member.py (+0/-9)
tests/vmtests/test_simple.py (+0/-14)
tests/vmtests/test_uefi_basic.py (+0/-9)
tests/vmtests/test_zfsroot.py (+0/-10)
tools/remove-vmtest-release (+54/-0)
tools/run-pep8 (+1/-0)
tools/run-pyflakes (+1/-0)

api

Related source package recipes

No recipes using this branch. (?)

Create packaging recipe

Related snap packages

Related charm recipes

82b0be2... by Ryan Harper on 2020-02-04

remove-vmtest-release: remove 'class' match case, fix calling clean-file

00ba9f8... by Ryan Harper on 2020-02-03

tools/remove-vmtest-release: refactor and rename tool

Refactor remove-vmtest-release with the following changes:

- renamed to drop the trailing .py
- skip re-writing file if not modified
- ensure remaining content is not dropped after last class definition
  removed
- Add argparsing with parameters --distro-release, --path
- Handle single file or directory (--path tests/vmtests)
- Use distro.title() to capitalize the first letter of release
- Add comments in the parsing logic

43d6d28... by Ryan Harper on 2020-01-30

vmtests: remove Disco tests via tools/remove-vmtest-release.py

a9eba32... by Ryan Harper on 2020-01-30

Add tool to remove all vmtests for a specific release.

50ffca4... by Ryan Harper on 2020-01-30

clear-holders: ensure we wipe device even if multipath enabled not not mp

On Ubuntu releases where multipath is enabled but not in-use some devices
would fail to be wiped due to logic bug. This was found in the
BcacheBasic test. Refactor this logic to ensure that we wipe either
the in-use multipath dm device or the block device value originally
supplied to the wipe function.

941c43f... by Ryan Harper on 2020-01-29

block_meta: use reliable fs_spec entries for block devices

Entries in /etc/fstab currently use UUID= in almost all cases. This
is not reliable in the face of LVM snapshots, multipath, and other
scenarios where multiple block devices may have the same UUID values.

Resolve this by following the recommendations in the Ubuntu Wiki[1]
where we will use specific udev persistent block paths which embed
a *device* uuid in the path (dm-uuid, md-uuid, etc). In addition
to modifying the fs_spec value we write, also include a comment
above each entry which includes a terse device path which may help
users recogize a disk where a UUID will not. This brings curtin
to parity with other Ubuntu installers which include:

 # /mnt was mounted on /dev/sdb1 during installation

LP: #1851899

b475db2... by Ryan Harper on 2020-01-27

multipath: handle removal of multipath partitions correctly

Curtin could handle multipath-partition removal if it was given
the device-mapper device (dm-x) however, if it was fed the
underying scsi device (e.g. sda2) this would fail when curtin
would try to clear the partition requesting exclusive access
which it could not obtain due to the device-mapper device.

This branch implements a kernel scsi device lookup to map
sdXN -> dm-N mapping which resolves the issue by allowing
curtin to wipe the dm-X device even if it has been
provided the sdXN device.

Additional Changes
 - Enable dirty_disk mode on multipath test to reproduce
   failure

LP: #1857042

a45a54c... by Ryan Harper on 2020-01-16

vmtests: skip Focal deploying Centos70 ScsiBasic

In Focal, there is a bug (#1859858) in udev which returns a
truncated value for ID_SERIAL on some scsi devices; this does
not break dname rules when the ephemeral environment matches
the target system, however if Focal deploys Bionic or in this
case Centos70, the udev in the target system produces a
different ID_SERIAL value which prevents the dname rules from
matching. Skip this test until we get a udev fix.

c0c986e... by Ryan Harper on 2020-01-15

vmtests: fix network mtu tests, separating ifupdown vs networkd

On networkd based releases, some of the existing ipv6 MTU tests
will not work, by design. The curtin ifupdown helper scripts
installed ensure that the base network interface mtu is always
set correctly; networkd requires an explicit device MTU setting
in addition to an IPV6 MTU value if they are to differ.

- Add a networkd specific mtu test config
- Add a _skip_if_not_ifupdown to avoid tests when on networkd
- Update scripts to capture additional debugging info when on
  networkd

Additional changes needed:
    - Add '---' before adding the service masking to ensure
      we disable snapd during vmtest
    - Refactor journalctl commands to separately sync, flush to
      ensure we capture the compelte journal when we collect
    - allow test class to specify add_repos, system_upgrade and
      upgrade_packages, env still wins if set
    - use systemd from ppa:ddstreet/systemd until ipv6 mtu
      bug is fixed

4fa2d66... by Mike Pontillo <email address hidden> on 2020-01-10

doc: Fix kexec documentation bug.