Fix error in url_helper and vmtest from previous commit.
Previous commit had 2 issues:
a.) it pushed the majority of tests down "install from url" path,
which was not intended. The intent was to use 'cp://' whenever
the booted environment was the same as the target. The problem
occurred when target_release was the default None.
The fix here is just to cover target_release == None case.
b.) url_helper.download was broken for python2.7.
The isue was just that we expected the bytes written to be returned
from file().write. Python 2 file.write has no return value.
A unit test is added to test that code.
tools: fix curtainer to specify source version to apt-get source.
We were seeing version skew from the binary package to the source
package, and had recently added code to exit failure if we saw
that case.
This change will make curtainer explicitly request the version
of source that it has the binary for. It is possible that the
ppa would not have that version (if it had been supersceeded),
but hopefully the archive holds onto source for current binary
packages.
Add support for installing sources that are a filesystem image.
This adds support for sources that are a 'filesystem image' via local
file path or via http or https. It will work for any filesystem
image type that is mountable by the running kernel.
To accomplish this:
a.) if the source is a http/https url, then download it to temporary file.
b.) mount the file loopback (mount -o loop,ro).
c.) copy the contents to the target
d.) unmount
e.) if downloaded remove the file.
In order to do this we needed some mechanism for downloading a url
in pieces rather than all into memory as 'geturl' uses. So I've
added UrlReader and 'download'.
Also here, a fix for the default headers to include the actual curtin
version per version.version_string() rather than '.1'.
MAAS 2.3 and 2.4 plan to use this functionality to install Ubuntu 12.04
while booted in 16.04. There are two tests added that excercise that
path that will be enabled later. To support testing hwe here,
also have added explicit setting of target_kernel_package in the class.
clear-holders: detect and remove devices from kernel swap as needed.
During testing of subiquity the ephemeral environment may activate
existing swap partitions which then prevent curtin from using a device
during storage configuration. This branch introduces a change to TestBasic
to add a swap partition and a command to the dirty-disk mode to forcibly
enable the swap device to reproduce the error. The fix involves having
clear_holders detect and remove the device (disk or partition) from the
kernel swap via use of swapoff.
tests: remove xfs filesystem from basic uefi test.
The test_uefi_basic.py file reports to be 'basic' test of uefi.
However, it also tests xfs filesystem support.
In order to make it more basic, just drop the xfs.
The real motivation here is to later enable a precise (12.04)
installation test done inside a 14.04 or 16.04 environment.
Each of those scenarios cause mounting problems of the xfs partition
when the 12.04 kernel boots. That is because the mkfs.xfs command
enabled options that were not supported in the older kernel.
Older version of dpkg-parsechangelog in trusty does not seem to
support '--file=<file>' correctly. It does seem to support
--file <file> though. So we'll just use that format.
Writing to standard error from apply_net method would leak those
messages through to stderr when running tox. There was no value
in writing the messages as 'curtin' would still print them.
Also make apply_net consistent with other sub commands by changing
'apply_net_main' to let exceptions bubble up.
'curtin apply-net' will still write a message with the error
and will exit non-zero. If --showtrace is provided, it will
print the stack trace.
vmtest: Drop use of root-image.gz and maas2roottar.
The root-image.gz and its conversion into a root-tgz file are no
longer necessary. Dropping the legacy code has numerous benefits:
a.) no longer download root-image.gz files to local mirror
b.) do not uncompressing these.
c.) do not use maas2roottar which required sudo to create the root-tgz.
tools/curtainer: check that binary and source version are the same.
We saw a puzzling failure of vmtest against the daily archive.
The reason for failure was that the source package that was retrieved
via 'apt-get source' differed in version from the version installed.
This adds a check in curtainer that will exit failure if that is the case.
