lp:~raharper/curtin/trunk.add-gpg-retry

Branch merges

Propose for merging

No branches dependent on this one.

Merged into lp:~curtin-dev/curtin/trunk at revision 445

Server Team CI bot: Approve (continuous-integration) on 2017-02-03

Scott Moser (community): Approve on 2017-02-03

Diff: 249 lines (+172/-7)

5 files modified

curtin/commands/apt_config.py (+2/-1)
curtin/gpg.py (+4/-4)
curtin/util.py (+3/-0)
tests/unittests/test_apt_source.py (+4/-2)
tests/unittests/test_gpg.py (+159/-0)

api

Related source package recipes

No recipes using this branch. (?)

Create packaging recipe

Related snap packages

Related bugs

Bug #1661337: curtin does not retry when gpg fails to recv key

Low

Fix Released

Link a bug report

Related blueprints

447. By Ryan Harper on 2017-02-03

Unittest cleanup after review

- Use list expansion for repeated calls
- Remove extra comma when calling gpg.recv_key

446. By Ryan Harper on 2017-02-02

Don't hardcode retries value in gpg module.

445. By Ryan Harper on 2017-02-02

gpg: retry when recv'ing gpg keys fail

Retry gpg recv operations in case error is transient.
Add unittests to validate curtin.gpg including retry behavior

444. By Ryan Harper on 2017-02-01

Add clear_holders checks to disk and partition handlers

In some cases disks may contain block device metadata outside of the
current partition table. Due to Linux udev rules and behaviors;
creating or modifying a disks partition table may trigger udev rules
which may start up mdadm, bcache or lvm services which will obtain
exclusive access to the disk preventing installation from completing.

This branch addresses this issue with the following

- block.wipe_volume now uses os.O_EXCL flag to obtain exclusive access
  to a device; this fails when something else (like mdadm) obtains
  a file handle on the device with O_EXCL flag

- Upon failure to obtain exclusive access, we dump information about
  a devices block holders

- During disk handling, after creating a partition table we check for
  holders and then clear them if present.

- During partition handling, after creating a new partition we check
  for holders and then clear them if present.

443. By Ryan Harper on 2017-01-27

vmtest: overhaul image sync

Changes to support HWE and Centos images have exposed issues in the image
sync process. Our 'make sync-images' was using a simplestreams filter
that did not pickup any of the required HWE kernels which meant that
during vmtest runs we would trigger new image syncs. Compounding this
issue was the fact that the filter used was too-wide which picked up
things like di-initrd,di-kernel files.

Additionally for an empty IMAGE_DIR repository, there were two bugs; one
it assumed that it could load the vmtest.json/vmtest-centos.json files.
Second when the repo was empty, we did not return an iterable type.

The following changes have been made to achieve the goals:
  - make sync-images should download everything we need to run a complete
    vmtest run without acquiring any new files from our image server.
  - Handle empty repositories properly
  - Do not attempt to sync with the streams on each test, but only if
    specific needed files are missing
  - Do not sync streams during jenkings-runner; the previous
    make sync-images should have pulled everything

442. By Ryan Harper on 2017-01-26

vmtest: skip apt-proxy test if not set

Testing of apt-proxy requires one to be set in the host. If a proxy
is not set then the test-case will fail. This was discovered on diglett
host which usally has an apt-proxy but recently it was disabled which
tripped this failure.

441. By Scott Moser on 2017-01-25

net: add new lines after rendered static routes.

Static routes were being rendered in /etc/network/interfaces
without a trailing newline after the 'pre-down' line.
Add unittest and vmtest to validate static route configurations.

440. By Scott Moser on 2017-01-20

vmtest: add 'webserv' helper

Rather than running python -m SimpleHTTPServer for the web server
add a program 'webserv' in tools/ dir and use that.

The benefits of this approach are:
 a.) no longer need python2 (we were invoking with 'python')
 b.) webserv listens on both ipv4 and ipv6 addresses.

439. By Scott Moser on 2017-01-20

vmtest: add CURTIN_VMTEST_CURTIN_EXE variable.

This allows us to control which 'curtin' program is used. The program
must support curtin's 'pack' interface, and needs to handle finding itself
appropriately.

Also here, we drop the PYTHONPATH setting in launch, as it is no longer
needed as our bin/curtin wrapper handles that.

The target use case is really for easily running vmtest with an installed
version of curtin, allowing us to easily verify stable release updates.

438. By Ryan Harper on 2017-01-19

multipath: don't run update-grub; setup_grub will handle this better.

During an update-grub in the target system when using multipath
the 'os-prober' tool used to discover other operating systems
to add to the grub menu does not handle duplicate paths and
mounts the same disk up twice causing unknown behavior which
sometimes results in timeouts during installation.

Dropping the direct call to update-grub and deferring to curtin's
setup_grub method (which disables os-prober) works around the
issue. Separate bug filed against os-prober.

LP: #1656369