lp:~smoser/curtin/trunk.lp1662346

Branch merges

Propose for merging

No branches dependent on this one.

Merged into lp:~curtin-dev/curtin/trunk at revision 460

Joshua Powers (community): Approve on 2017-02-07

Server Team CI bot: Approve (continuous-integration) on 2017-02-07

curtin developers: Pending requested 2017-02-07

Diff: 19 lines (+9/-0)

1 file modified

curtin/commands/curthooks.py (+9/-0)

api

Related source package recipes

No recipes using this branch. (?)

Create packaging recipe

Related snap packages

Related bugs

Bug #1662346: vmtest: s390x requires zipl command

Medium

Fix Released

Link a bug report

Related blueprints

452. By Scott Moser on 2017-02-07

zipl is in s390-tools

451. By Scott Moser on 2017-02-07

fix variable name

450. By Scott Moser on 2017-02-07

merge from trunk

449. By Scott Moser on 2017-02-07

intended to be a list, not a set.

448. By Scott Moser on 2017-02-06

on s390x, install zipl in target.

447. By Christian Ehrhardt  on 2017-02-06

vmtest: Add the ability to add extra config files to test execution.

This is what I used to test multipath-tools via curtin, but it can be much
more helpful by generally allowing to add any external content

446. By Ryan Harper on 2017-02-06

content decoding in load_file, apply_net raise exception on errors

This patch series fixes two issues. First, subcommands of the apply_net
command were exiting non-zero but we failed to raise and exception
which let curtin hide an error. We've modified apply_net to re-raise
the exception when it occurs. Additionally update vmtest to look for
stack-traces in the installation log and mark a test failed if it
detects one; this should prevent future cases from re-occurring.

The second error is when loading a file with and encoding, load_file
did not handle this case. Merge in a version of load_file from
cloud-init which already handles this case. Introduce new
unittests to validate the function as it touches some block related
code where we read partition data directly.

Finally, the mock_open feature of unittest.mock only supports
binary data in version 2.0.0 or newer, so skip this unittest on
systems without a new-enough mock; note this does not affect
the function of the code on the same release, only the unittest.

445. By Ryan Harper on 2017-02-03

gpg: retry when recv'ing gpg keys fail

Retry gpg recv operations in case error is transient.
Add unittests to validate curtin.gpg including retry behavior

444. By Ryan Harper on 2017-02-01

Add clear_holders checks to disk and partition handlers

In some cases disks may contain block device metadata outside of the
current partition table. Due to Linux udev rules and behaviors;
creating or modifying a disks partition table may trigger udev rules
which may start up mdadm, bcache or lvm services which will obtain
exclusive access to the disk preventing installation from completing.

This branch addresses this issue with the following

- block.wipe_volume now uses os.O_EXCL flag to obtain exclusive access
  to a device; this fails when something else (like mdadm) obtains
  a file handle on the device with O_EXCL flag

- Upon failure to obtain exclusive access, we dump information about
  a devices block holders

- During disk handling, after creating a partition table we check for
  holders and then clear them if present.

- During partition handling, after creating a new partition we check
  for holders and then clear them if present.

443. By Ryan Harper on 2017-01-27

vmtest: overhaul image sync

Changes to support HWE and Centos images have exposed issues in the image
sync process. Our 'make sync-images' was using a simplestreams filter
that did not pickup any of the required HWE kernels which meant that
during vmtest runs we would trigger new image syncs. Compounding this
issue was the fact that the filter used was too-wide which picked up
things like di-initrd,di-kernel files.

Additionally for an empty IMAGE_DIR repository, there were two bugs; one
it assumed that it could load the vmtest.json/vmtest-centos.json files.
Second when the repo was empty, we did not return an iterable type.

The following changes have been made to achieve the goals:
  - make sync-images should download everything we need to run a complete
    vmtest run without acquiring any new files from our image server.
  - Handle empty repositories properly
  - Do not attempt to sync with the streams on each test, but only if
    specific needed files are missing
  - Do not sync streams during jenkings-runner; the previous
    make sync-images should have pulled everything