pkg build ci: Add make ci-deps-<distro> target to install pkgs
This change adds a couple of makefile targets for ci environments to
install all necessary dependencies for package builds and test runs.
It adds a number of arguments to ./tools/read-dependencies to facilitate
reading pip dependencies, translating pip deps to system package names and
optionally installing needed system-package dependencies on the local
system. This relocates all package dependency and translation logic into
./tools/read-dependencies instead of duplication found in packages/brpm
and packages/bddeb.
In this branch, we also define buildrequires as including all runtime
requires when rendering cloud-init.spec.in and debian/control files
because our package build infrastructure will also be running all unit
test during the package build process so we need runtime deps at build
time.
Additionally, this branch converts
packages/(redhat|suse)/cloud-init.spec.in from cheetah templates to jinja
to allow building python3 envs.
systemd: make cloud-final.service run before apt daily services.
This changes all cloud-init systemd units to run 'Before' the apt processes
that run daily and may cause a lock on the apt database.
apt-daily-upgrade.service contains 'After=apt-daily.service'.
Thus following order is enforced, so we can just be 'Before' the first.
apt-daily.service
apt-daily-upgrade.service
Note that this means only that apt-daily* will not run until
cloud-init has entirely finished. Any other processes running apt-get
operations are still affected by the global lock.
On some systems with python-libselinux a bug[1] related to recursive
restorecon fails but the distro release does not yet include
an update. This change will accept the error and log a warning.
On systems with selinux enabled, some of the networking commands executed
successfully do not return 0. Allow these commands to return 1 since the
output is valid.
Ultimately we need to get this information in some way so that we can
display it correctly. For now, work around the stack trace when selinux
does not allow us to collect it.
In cases where the config json specifies nameserver entries,
if there are interfaces configured to use dhcp, NetworkManager,
if enabled, will clobber the /etc/resolv.conf that cloud-init
has produced, which can break dns. If there are no interfaces
configured to use dhcp, NetworkManager could clobber
/etc/resolv.conf with an empty file.
This patch adds a mechanism for dropping additional configuration
into /etc/NetworkManager/conf.d/ and disables management of
/etc/resolv.conf by NetworkManager when nameserver information is
provided in the config.
Use distro release version to determine if we use systemd in redhat spec
The typical rpm build process will examine the spec file to determine
which packages should be installed in the boot root. This requires
the specfile to declare that it needs system. Provide this information
by checking which version in which the rpm is being built and exporting
requirements for systemd.
The network_state object's network and route keys would have different
information depending upon how the network_state object was populated.
This change cleans that up. Now:
* address will always contain an IP address.
* prefix will always include an integer value that is the
network_prefix for the address.
* netmask will be present only if the address is ipv4, and its
value will always correlate to the 'prefix'.
Integration Testing: tox env, pyxld 2.2.3, and revamp framework
Massive update to clean up and greatly enhance the integration testing
framework developed by Wesley Wiedenmeier.
- Updated tox environment to run integration test 'citest' to utilize
pylxd 2.2.3
- Add support for distro feature flags
- add framework for feature flags to release config with feature groups
and overrides allowed in any release conf override level
- add support for feature flags in platform and config handling
- during collect, skip testcases that require features not supported by
the image with a warning message
- Enable additional distros (i.e. centos, debian)
- Add 'bddeb' command to build a deb from the current working tree
cleanly in a container, so deps do not have to be installed on host
- Adds a command line option '--preserve-data' that ensures that
collected data will be left after tests run. This also allows the
directory to store collected data in during the run command to be
specified using '--data-dir'.
- Updated Read the Docs testing page and doc strings for pep 257
compliance
The added 'run-centos' does:
- Creates centos 6 or 7 lxd container
* Sets http_proxy variable for yum if set locally
* Creates centos user
- Push local tree
* Tar's up working directory
* Pushes to container and untars
- Installs pip and yum dependencies
- As user centos it can then based on flags:
* runs unittests
* run ./packages/brpm
* run ./packages/brpm --srpm
* artifact the built *.rpm