cii: Use git hash pip dependency format for pylxd.
We want this git hash from master as it pylxd pip packages are not
as frequently published by this project. Master for pylxd is under
active development, so we do want to 'pin' which hash we pull.
This adds the specific requirements for integration testing to
a single file that can be referenced in other areas. It also enables
the read-dependencies script to install those packages.
MAAS: add check_instance_id based off oauth tokens.
This stores a hash of the OAuth tokens as an 'id' for the maas
datasource. Since new instances get new tokens created and those tokens
are written by curtin into datasource system config this will provide
a way to identify a new "instance" (install).
Due to additional platforms getting added this test was not taking into
account platform specific mirrors nor was it checking that no additional
entries were added.
This fixes the incorrectly named 'family' value for images as 'os'.
Families are already defined in util.py:OS_FAMILY_MAPPING and a family
is a collection of OSes.
This makes the properties function part of the super class of image as
it is only overrided by the lxd backend.
tests: rename test ssh keys to avoid appearance of leaking private keys.
While the generated ssh keys are throw away keys, generating SSH keys
with the default name (e.g. id_rsa) can trigger security scanners or
draw unnecessary attention.
The change here simply renames 'id_rsa' to 'cloud_init_rsa' to avoid
a false positive reported by a scanning tool.
This enables integration tests to utilize AWS EC2 as a testing platform by
utilizing the boto3 Python library.
Usage will create and delete a custom VPC for every run. All resources
will be tagged with the ec2 tag, 'cii', and the date (e.g.
cii-20171220-102452). The VPC is setup with both IPv4 and IPv6
capabilities, but will only hand out IPv4 addresses by default. Instances
will have complete Internet access and have full ingress and egress access
(i.e. no firewall).
SSH keys are generated with each run of the integration tests with the key
getting uploaded to AWS at the start of tests and deleted on exit. To
enable creation when the platform is setup the SSH generation code is
moved to be completed by the platform setup and not during image setup.
The nocloud-kvm platform was updated with this change.
Creating a custom image will utilize the same clean script,
boot_clean_script, that the LXD platform uses as well. The custom AMI is
generated, used, and de-registered after a test run.
The default instance type is set to t2.micro. This is one of the smallest
instance types and is free tier eligible.
The default timeout for ec2 was increased to 300 from 120 as many tests
hit up against the 2 minute timeout and depending on region load can
go over.
Documentation for the AWS platform was added with the expected
configuration files for the platform to be used. There are some
additional whitespace changes included as well.
pylint exception was added for paramiko and simplestreams. In the past
these were not already flagged due to no __init__.py in the subdirectories
of files that used these. boto3 was added to the list of dependencies in
the tox ci-test runner.
In order to grab console logs on EC2 the harness will now shut down an
instance before terminating and before collecting the console log. This
is to address a behavior of EC2 where the console log is refreshed very
infrequently, but one point when it is refreshed is after shutdown.
This fixes a traceback when attempting to bounce the network after
hostname resets.
In artful and bionic ifupdown package is no longer installed in default
cloud images. As such, Azure can't use those tools to bounce the network
informing DDNS about hostname changes. This doesn't affect DDNS updates
though because systemd-networkd is now watching hostname deltas and with
default behavior to SendHostname=True over dhcp for all hostname updates
which publishes DDNS for us.
