~smoser/curtin:cleanup/cleanup-apt-test

Last commit made on 2018-01-18
Get this branch:
git clone -b cleanup/cleanup-apt-test https://git.launchpad.net/~smoser/curtin
Only Scott Moser can upload to this branch. If you are Scott Moser please log in for upload directions.

Branch merges

Branch information

Name:
cleanup/cleanup-apt-test
Repository:
lp:~smoser/curtin

Recent commits

a4c9636... by Scott Moser

tests: cleanup the apt custom sources test.

This test must have gone through some iteration in development.
It was testing in a very white-box way, and not really even validating
that the config was as expected (as made evident by the previous
lack of use of EXPECTED_CONVERTED_CONTENT).

The test now asserts that the two expected changes are made.

8c4af4e... by Scott Moser

apt: Be more lenient when replacing mirrors in /etc/apt/sources.list

Offical stock trusty and precise images contain /etc/apt/sources.list
that have a mirror like:
  http://archive.ubuntu.com/ubuntu
Where xenial and later have that as:
  http://archive.ubuntu.com/ubuntu/

When curtin "parsed" the existing /etc/apt/sources.list into a template
file to use for updating mirrors, it would not identify the trusty
version.

This fixes that parsing to be just as strict as before except for
to accomodate a missing trailing /.

LP: #1744038

6abd085... by Ryan Harper

vmtest: Drop Zesty release

Zesty is no longer supported as of 2018-01-13[1]

1. https://lists.ubuntu.com/archives/ubuntu-announce/2018-January/000227.html

89698cd... by Ryan Harper

vmtest: initialize logger with class names for easy parsing

When running vmtest in parallel, having each message include the
TestClass name helps developers map the output to specific tests when
reading the combined log.

8a7308e... by Scott Moser

packaging: Do not mention primary contributors in debian/changelog.

new-upstream-snapshot gives credit to committers by putting their
name in square brackets on the debian/changelog entries.

For some time in cloud-init we've stripped myself from that list.
I think it makes sense to strip out primary contributors to avoid
noise, but still give credit to new or less frequent contributors.

8b14506... by Scott Moser

packaging: Update new-upstream-snapshot to work with git.

After the move to git, we need to update new-upstream-snapshot.
This version comes from cloud-init, with an extra hunk in it to
handle the transition from bzr.

Commit new-upstream-snapshot to master, where it previously
only lived on ubuntu branches. Then, commit it also to the branches
so that it does not cause conflict.

fae8ffb... by Scott Moser

Bump pylint to 1.8.1.

Increase the version of pylint from 1.7.4 to 1.8.1

92e45a8... by Ryan Harper

vmtests: switch to MAAS v3 streams for images and kernels

Switching to v3 streams enables testing of Xenial HWE and HWE-edge
kernels and the squashfs root which MAAS 2.3 uses.

- Add v3 streams URL
- Introduce two attributes, ephemeral_ftype, target_ftype to determine
  which files are needed for the ephemeral environment and which for the
  target
- Add kflavor and subarch filters to streams query and release classes to
  support kernel variants: ga, hwe, hwe-edge. We're currently only testing
  subarch 'generic' and ignoring 'low-latency'
  - When testing with hwe and hwe-edge, emit a kernel configuration to
    curtin to ensure we install the matching kernel in the target image
  - Make sure vmtests append collect scripts to the base class parameter
    so all tests run the BaseVM collect_scripts (dpkg-query)
- Introduce new release classes, XenialGA, XenialHWE and XenialEdge
  Note, XenialGA replaces the general Xenial release class, and that
  not all test-cases run Xenial HWE/Edge kernels; most of the storage
  ones do.
- Booting from a squashfs image requires some changes
  - publish the squashfs image and boot VMs with root=URL (except trusty)
  - utilize overlayfs to keep the ephemeral environment pristine and
    use the squashfs image as a the curtin install source.
  - Add some boot parameters to the ephemeral environment to enable
    networking, including DNS for package installation.
  - Use BOOTIF= parameter for multi-nic vmtests to ensure we only bring
    up one nic for ephemeral networking. Sometimes multiple nics would
    come online and then cause networking hiccups which prevented
    installs from completing.
- Modify vmtest storage configurations for disks to all use a 'serial'
  attribute. Currently we use 'disk-X' as the serial value where X
  increments from 'a' -> 'z' as the test needs.
- jenkins-runner now prints additional information about what images
  are found and available.
- tools/launch
  - modify --append to allow multiple parameters to grow the array.
  - fix bug in --publish which prevented more than a single file being
    published
  - allow PUBURL to be expanded in root-arg
  - add a 'boot-image' parameter which is exported to the VM with a serial
    number to provide a consistent path to the device,
    /dev/disk/by-id/virtio-boot-disk
  - drop '2G' boot size parameter to boot disks, instead allow qemu-img to
    default to the size of backing image.

be9ab3c... by Scott Moser

tests: update pack tests to clear out pyc files more completely.

When the test test_curtin_help_has_hacked_version was first written,
the file .pyc file that was known to be created for
my_path/my.py was my_path/my.pyc

Newer versions of python (at least in cpython 3.6), create:
   version/__pycache__/version.cpython-36.pyc

This updates the test to remote those pyc files also.

The failure to remove the .pyc file results in a race condition
where sometimes the .pyc file is seen as valid even though the
.py file was written after it.

c5cb03e... by Scott Moser

debian/control: drop conflicts that had bzr version

this is fine as the bzr version (~bzr54) is not in any supported
release.. trusty is at 126-0ubuntu1 in released version even.