Merge ~chad.smith/ubuntu/+source/ubuntu-advantage-tools:upload-release-26 into ubuntu/+source/ubuntu-advantage-tools:ubuntu/devel

Proposed by Chad Smith
Status: Merged
Merged at revision: 83975b05565e50fa8fc8efc315c7d09babbd5f0a
Proposed branch: ~chad.smith/ubuntu/+source/ubuntu-advantage-tools:upload-release-26
Merge into: ubuntu/+source/ubuntu-advantage-tools:ubuntu/devel
Diff against target: 18644 lines (+13502/-1203)
107 files modified
.github/PULL_REQUEST_TEMPLATE.md (+33/-0)
.gitignore (+24/-0)
Jenkinsfile (+263/-0)
Makefile (+7/-1)
README.md (+277/-23)
RELEASES.md (+63/-0)
apt-hook/hook.cc (+61/-22)
debian/changelog (+225/-0)
debian/control (+4/-3)
debian/po/POTFILES.in (+1/-0)
debian/po/templates.pot (+36/-0)
debian/postinst (+112/-10)
debian/postrm (+4/-5)
debian/prerm (+2/-2)
debian/rules (+5/-7)
debian/ubuntu-advantage-tools.templates (+1/-1)
dev/null (+0/-7)
features/attach_invalidtoken.feature (+12/-5)
features/attach_validtoken.feature (+354/-9)
features/attached_commands.feature (+382/-53)
features/attached_enable.feature (+373/-67)
features/aws-ids.yaml (+4/-0)
features/azure-ids.yaml (+4/-0)
features/cloud.py (+744/-0)
features/environment.py (+649/-80)
features/gcp-ids.yaml (+3/-0)
features/staging_commands.feature (+395/-0)
features/steps/steps.py (+332/-22)
features/ubuntu_pro.feature (+354/-0)
features/ubuntu_upgrade.feature (+71/-0)
features/unattached_commands.feature (+91/-55)
features/unattached_status.feature (+68/-14)
features/util.py (+251/-85)
help_data.yaml (+68/-0)
integration-requirements.txt (+10/-0)
lib/__init__.py (+0/-0)
lib/reboot_cmds.py (+159/-0)
lib/upgrade_lts_contract.py (+99/-0)
release-upgrades.d/ubuntu-advantage-upgrades.cfg (+4/-0)
setup.py (+25/-2)
sru/release-25/trusty-hwe-livepatch.txt (+1787/-0)
sru/release-26/xenial-enable-fips (+188/-0)
systemd/ua-auto-attach.path (+9/-0)
systemd/ua-auto-attach.service (+2/-2)
systemd/ua-reboot-cmds.service (+13/-0)
test-requirements.txt (+0/-4)
tools/base_travis_integration_tests.sh (+23/-0)
tools/constraints-bionic.txt (+1/-0)
tools/constraints-mypy.txt (+3/-0)
tools/constraints-xenial.txt (+1/-0)
tools/make-release (+120/-0)
tools/refresh-aws-pro-ids (+63/-0)
tools/refresh-keyrings.sh (+15/-22)
tools/run_aws_travis_integration_tests.sh (+22/-0)
tools/run_azure_travis_integration_tests.sh (+28/-0)
tools/run_lxd_travis_integration_tests.sh (+29/-0)
tools/tox-lxd-runner (+89/-0)
tools/ua-dev-cloud-config.yaml (+12/-0)
tox.ini (+75/-8)
uaclient/apt.py (+83/-17)
uaclient/cli.py (+391/-58)
uaclient/clouds/gcp.py (+49/-0)
uaclient/clouds/identity.py (+5/-3)
uaclient/clouds/tests/test_gcp.py (+104/-0)
uaclient/config.py (+330/-8)
uaclient/conftest.py (+5/-1)
uaclient/contract.py (+122/-47)
uaclient/defaults.py (+4/-1)
uaclient/entitlements/__init__.py (+10/-0)
uaclient/entitlements/base.py (+153/-4)
uaclient/entitlements/cc.py (+23/-11)
uaclient/entitlements/cis.py (+5/-3)
uaclient/entitlements/esm.py (+4/-3)
uaclient/entitlements/fips.py (+307/-29)
uaclient/entitlements/livepatch.py (+59/-16)
uaclient/entitlements/repo.py (+183/-65)
uaclient/entitlements/tests/conftest.py (+8/-2)
uaclient/entitlements/tests/test_base.py (+52/-0)
uaclient/entitlements/tests/test_cc.py (+8/-3)
uaclient/entitlements/tests/test_cis.py (+16/-8)
uaclient/entitlements/tests/test_esm.py (+13/-12)
uaclient/entitlements/tests/test_fips.py (+616/-131)
uaclient/entitlements/tests/test_livepatch.py (+96/-6)
uaclient/entitlements/tests/test_repo.py (+281/-41)
uaclient/exceptions.py (+32/-0)
uaclient/pip.py (+29/-0)
uaclient/status.py (+108/-11)
uaclient/tests/test_apt.py (+115/-24)
uaclient/tests/test_cli.py (+306/-52)
uaclient/tests/test_cli_attach.py (+15/-0)
uaclient/tests/test_cli_auto_attach.py (+59/-1)
uaclient/tests/test_cli_detach.py (+48/-16)
uaclient/tests/test_cli_disable.py (+146/-14)
uaclient/tests/test_cli_enable.py (+384/-9)
uaclient/tests/test_cli_refresh.py (+14/-0)
uaclient/tests/test_cli_status.py (+204/-12)
uaclient/tests/test_config.py (+462/-19)
uaclient/tests/test_contract.py (+78/-45)
uaclient/tests/test_pip.py (+91/-0)
uaclient/tests/test_reboot_cmds.py (+124/-0)
uaclient/tests/test_status.py (+3/-0)
uaclient/tests/test_upgrade_lts_contract.py (+118/-0)
uaclient/tests/test_util.py (+83/-5)
uaclient/tests/test_version.py (+11/-3)
uaclient/util.py (+76/-7)
uaclient/version.py (+16/-4)
ubuntu-advantage.1 (+3/-3)
Reviewer Review Type Date Requested Status
Bryce Harrington (community) Approve
Canonical Server Pending
Review via email: mp+398470@code.launchpad.net

Description of the change

New upstream release of 26.1 into Hirsute.

We would like to preserve rich git commit history by adding an pkg/upload/26.1 tag
pkg/ubuntu/devel(hirsute) if possible. But, not certain how we do that as part of this PR review process.

Known lintian warnings in this version we plan on addressing for our 27.0 release before trying to upload to Hirsute during feature freeze[1]:

 - lintian issues:
     - https://github.com/canonical/ubuntu-advantage-client/issues/1384

PPA package pushed and building at https://launchpad.net/~chad.smith/+archive/ubuntu/uaclient-uploads

local tests can be run w/ tox -e py3/mypy/flake8 and are also run on every PR against ubuntu-advantage-client

Daily integration tests are also run on our Jenkins[2] (VPN needs to be active)

references:
[1] FFe bug for releasing 27.0 into hirsute in a couple weeks https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/1915354.

[2] Jenkins UA integration tests https://jenkins.ubuntu.com/server/view/ua/

To post a comment you must log in.
Revision history for this message
Chad Smith (chad.smith) wrote :
Download full text (87.0 KiB)

----- Autopkgtest results: ---------

csmith@downtown:/tmp/ubuntu-advantage-client (ubuntu/devel)$ autopkgtest -o autopkgtest-ua-tools -U ../out/ubuntu-advantage-tools_26.1~21.04.1.dsc -- lxd ubuntu-daily:xenial
autopkgtest [14:10:51]: starting date: 2021-02-22
autopkgtest [14:10:51]: version 5.13.1
autopkgtest [14:10:51]: host downtown; command line: /usr/bin/autopkgtest -o autopkgtest-ua-tools -U '../out/ubuntu-advantage-tools_26.1~21.04.1.dsc' -- lxd ubuntu-daily:xenial
autopkgtest [14:11:01]: @@@@@@@@@@@@@@@@@@@@ test bed setup
tee: /proc/self/fd/2: Permission denied
Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease
Hit:2 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Calculating upgrade...
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
The following packages will be upgraded:
  libldap-2.4-2
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 159 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libldap-2.4-2 amd64 2.4.42+dfsg-2ubuntu3.13 [159 kB]
Fetched 159 kB in 0s (218 kB/s)
(Reading database ... 25810 files and directories currently installed.)
Preparing to unpack .../libldap-2.4-2_2.4.42+dfsg-2ubuntu3.13_amd64.deb ...
Unpacking libldap-2.4-2:amd64 (2.4.42+dfsg-2ubuntu3.13) over (2.4.42+dfsg-2ubuntu3.12) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up libldap-2.4-2:amd64 (2.4.42+dfsg-2ubuntu3.13) ...
Processing triggers for libc-bin (2.23-0ubuntu11.2) ...
autopkgtest [14:11:28]: testbed dpkg architecture: amd64
autopkgtest [14:11:29]: testbed running kernel: Linux 5.8.0-43-generic #49-Ubuntu SMP Fri Feb 5 03:01:28 UTC 2021
autopkgtest [14:11:29]: @@@@@@@@@@@@@@@@@@@@ source ../out/ubuntu-advantage-tools_26.1~21.04.1.dsc
Reading package lists...
Building dependency tree...
Reading state information...
Correcting dependencies...Starting pkgProblemResolver with broken count: 0
Starting 2 pkgProblemResolver with broken count: 0
Done
 Done
Starting pkgProblemResolver with broken count: 0
Starting 2 pkgProblemResolver with broken count: 0
Done
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
The following additional packages will be installed:
  binutils dpkg-dev libdpkg-perl make
Suggested packages:
  binutils-doc debian-keyring gcc | c-compiler make-doc
Recommended packages:
  gcc | c-compiler build-essential fakeroot libalgorithm-merge-perl
  libfile-fcntllock-perl
The following NEW packages will be installed:
  binutils dpkg-dev libdpkg-perl make
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
Need to get 3,242 kB of archives.
After this operation, 17.6 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu xenial-updates/ma...

Revision history for this message
Bryce Harrington (bryce) wrote :
Download full text (5.8 KiB)

### Packaging Review ###

Changelog entries look great (lots and lots of changes!)

Per the RELEASES.md documentation, the version number for hirsute should rather be just '26.1'. The ~NN.04.M suffix is only used on stable series. I've updated this manually, hopefully this is ok.

Beyond that, I just spot-checked commits to ./debian/.

In running debuild -S and debuild, I notice the packaging introduces some git delta. It's all just tmp build chaff, nothing that's actually an issue, and this is not at all unusual for packages, but if you're looking to polish the packaging might be worth tidying up:

$ debuild -S
...
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
 deleted: .pytest_cache/README.md
 deleted: .pytest_cache/v/cache/nodeids
 deleted: .pytest_cache/v/cache/stepwise

$ debuild
...
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
 modified: .pytest_cache/README.md
 modified: .pytest_cache/v/cache/nodeids

Untracked files:
  (use "git add <file>..." to include in what will be committed)
 apt-hook/hook
 apt-hook/ubuntu-advantage.pot
 debian/files
 debian/ubuntu-advantage-pro.substvars
 debian/ubuntu-advantage-pro/

Most likely just stuff to add to .gitignore. The .pytest_cache directory is in the .git repository, but maybe it shouldn't be?

### Autopkgtest ###

Passed without issue.

### Lintian ###

I'm ignoring the lintian errors since sounds like you plan to investigate and address in the future, but just for feedback these are the errors/warnings that showed for me:

E: ubuntu-advantage-tools: no-debconf-config
W: ubuntu-advantage-tools source: build-depends-on-obsolete-package Build-Depends: dh-systemd => use debhelper (>= 9.20160709)
W: ubuntu-advantage-tools: groff-message usr/share/man/man1/ubuntu-advantage.1.gz command exited with status 2: /usr/lib/man-db/zsoelim | /usr/lib/man-db/manconv -f UTF-8:ISO-8859-1 -t UTF-8//IGNORE | preconv -e UTF-8 | groff -mandoc -Z -rLL=117n -rLT=117n -wmac -Tutf8
W: ubuntu-advantage-tools: groff-message usr/share/man/man1/ubuntu-advantage.1.gz troff: Signal 11 (core dumped)
W: ubuntu-advantage-tools source: package-uses-deprecated-debhelper-compat-version 9
W: ubuntu-advantage-tools: postinst-uses-db-input
W: ubuntu-advantage-pro: systemd-service-file-refers-to-unusual-wantedby-target lib/systemd/system/ua-auto-attach.service cloud-config.service

The man page core dumps happen to me for pretty much every package I do; I've never bothered to investigate and am guessing it's something quirky about my lxc environment.

### SRU text ###

I know this MP isn't an SRU, so am considering the SRU bug report just a draft, but I thought I'd give some feedback on it anyway.

"Version 27.0 will introduce the ability a new `ua fix <CVE_or_USN>` subcommand which will introspect the running system for a specific CVE or USN issue"

  - I think there's a small grammar issue there. Maybe drop "ability"?

"If we block the update to ubuntu-advantage-tools versio...

Read more...

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
0new file mode 1006440new file mode 100644
index 0000000..1fdfc76
--- /dev/null
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,33 @@
1## Proposed Commit Message
2<!-- Include a proposed commit message because all PRs can be merged in a variety of ways by the reviewer -->
3
4> summary: no more than 70 characters
5>
6> A description of what the change being made is and why it is being
7> made, if the summary line is insufficient. The blank line above is
8> required. This should be wrapped at 72 characters, but otherwise has
9> no particular length requirements.
10>
11> If you need to write multiple paragraphs, feel free.
12>
13> LP: #NNNNNNN (replace with the appropriate Launchpad bug reference if applicable)
14> Fixes: #NNNNNNN (replace with the appropriate github issue if applicable)
15
16## Test Steps
17<!-- Please include any steps necessary to verify (and reproduce if
18this is a bug fix) this change on a live deployed system,
19including any necessary configuration files, user-data,
20setup, and teardown. Scripts used may be attached directly to this PR. -->
21
22## Desired commit type::
23<!-- put an `x` in one merge type to allow the reviewer to merge for you -->
24 - [ ] Squash and merge: Using the "Proposed Commit Message"
25 - [ ] Create a merge commit and use "Proposed Commit Message"
26 - [ ] Rebase and merge, no merge commit, rely only on existing commit messages
27
28## Checklist:
29<!-- Go over all the following points, and put an `x` in all the boxes
30that apply. -->
31 - [ ] I have updated or added any unit tests accordingly
32 - [ ] I have updated or added any integration tests accordingly
33 - [ ] I have updated or added any documentation accordingly
diff --git a/.gitignore b/.gitignore
0new file mode 10064434new file mode 100644
index 0000000..2b2da67
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,24 @@
1artifacts/*
2*debhelper*
3debian/ubuntu-advantage-tools/
4debian/ubuntu-advantage-tools.substvars
5dev/entitlement-creds.json.*
6*.pyc
7*.pyo
8__pycache__
9.tox
10.pybuild
11.coverage
12.cache
13*.egg-info/
14.mypy_cache/
15
16# Ignore artifacts generated by bddeb
17*-1~bddeb*
18ubuntu-advantage-tools_*.orig.tar.gz
19ubuntu-advantage-tools_all.deb
20ubuntu_advantage_tools.dsc
21
22# test/jenkins artifacts
23pytest_results.xml
24reports/
diff --git a/.travis.yml b/.travis.yml
0deleted file mode 10064425deleted file mode 100644
index 9e571fa..0000000
--- a/.travis.yml
+++ /dev/null
@@ -1,83 +0,0 @@
1language: python
2dist: bionic
3
4install:
5 # Required so `git describe` will definitely find a tag; see
6 # https://github.com/travis-ci/travis-ci/issues/7422
7 - git fetch --unshallow
8 - make testdeps
9script:
10 - make test
11
12matrix:
13 fast_finish: true
14 include:
15 - python: 3.7
16 env: TOXENV=behave
17 script:
18 # bionic has lxd from deb installed, remove it first to avoid
19 # confusion over versions
20 - sudo apt-get remove --yes --purge lxd lxd-client
21 - sudo rm -Rf /var/lib/lxd
22 - sudo snap install lxd
23 - sudo lxd init --auto
24 - sudo usermod -a -G lxd $USER
25 - sg lxd -c 'make test'
26 - env:
27 PACKAGE_BUILD_SERIES=trusty
28 install:
29 - make travis-deb-install
30 script:
31 - make travis-deb-script
32 - env:
33 PACKAGE_BUILD_SERIES=xenial
34 install:
35 - make travis-deb-install
36 script:
37 - make travis-deb-script
38 - env:
39 PACKAGE_BUILD_SERIES=bionic
40 install:
41 - make travis-deb-install
42 script:
43 - make travis-deb-script
44 - env:
45 PACKAGE_BUILD_SERIES=eoan
46 install:
47 - make travis-deb-install
48 script:
49 - make travis-deb-script
50 - env:
51 PACKAGE_BUILD_SERIES=focal
52 install:
53 - make travis-deb-install
54 script:
55 - make travis-deb-script
56 - python: 3.4
57 env: TOXENV=py3-trusty,flake8-trusty
58 dist: trusty
59 - python: 3.5
60 env: TOXENV=py3-xenial,flake8-xenial
61 dist: xenial
62 - python: 3.6
63 env: TOXENV=py3-bionic,flake8-bionic
64 - python: 3.7
65 env: TOXENV=py3-eoan,flake8-eoan
66 - python: 3.8
67 env: TOXENV=py3,flake8
68 - python: 3.7
69 env: TOXENV=mypy
70 - python: 3.7
71 env: TOXENV=black
72 allow_failures:
73 - python: 3.7
74 env: TOXENV=behave
75 script:
76 # bionic has lxd from deb installed, remove it first to avoid
77 # confusion over versions
78 - sudo apt-get remove --yes --purge lxd lxd-client
79 - sudo rm -Rf /var/lib/lxd
80 - sudo snap install lxd
81 - sudo lxd init --auto
82 - sudo usermod -a -G lxd $USER
83 - sg lxd -c 'make test'
diff --git a/Jenkinsfile b/Jenkinsfile
84new file mode 1006440new file mode 100644
index 0000000..ebee6f9
--- /dev/null
+++ b/Jenkinsfile
@@ -0,0 +1,263 @@
1pipeline {
2 agent any
3
4 environment {
5 TMPDIR = "/tmp/$BUILD_TAG/"
6 UACLIENT_BEHAVE_JENKINS_BUILD_TAG = "${BUILD_TAG}"
7 UACLIENT_BEHAVE_JENKINS_CHANGE_ID = "${CHANGE_ID}"
8 UACLIENT_BEHAVE_BUILD_PR=1
9 UACLIENT_BEHAVE_CONTRACT_TOKEN = credentials('ua-contract-token')
10 UACLIENT_BEHAVE_AWS_ACCESS_KEY_ID = credentials('ua-aws-access-key-id')
11 UACLIENT_BEHAVE_AWS_SECRET_ACCESS_KEY = credentials(
12 'ua-aws-secret-access-key'
13 )
14 UACLIENT_BEHAVE_AZ_CLIENT_ID = credentials('ua-azure-client-id')
15 UACLIENT_BEHAVE_AZ_CLIENT_SECRET = credentials(
16 'ua-azure-client-secret'
17 )
18 UACLIENT_BEHAVE_AZ_TENANT_ID = credentials('ua-azure-tenant')
19 UACLIENT_BEHAVE_AZ_SUBSCRIPTION_ID = credentials(
20 'ua-azure-subscription-id'
21 )
22 UACLIENT_BEHAVE_CONTRACT_TOKEN_STAGING = credentials(
23 'ua-contract-token-staging'
24 )
25 JOB_SUFFIX = sh(returnStdout: true, script: "basename ${JOB_NAME}| cut -d'-' -f2").trim()
26 }
27
28 stages {
29 stage ('Setup Dependencies') {
30 steps {
31 deleteDir()
32 checkout scm
33 sh '''
34 python3 -m venv $TMPDIR
35 . $TMPDIR/bin/activate
36 pip install tox # for tox supporting --parallel--safe-build
37 '''
38 }
39 }
40 stage ('Lint and Style') {
41 parallel {
42 stage("flake8") {
43 steps {
44 sh '''
45 set +x
46 . $TMPDIR/bin/activate
47 tox --parallel--safe-build -e flake8
48 '''
49 }
50 }
51 stage("style") {
52 steps {
53 sh '''
54 set +x
55 . $TMPDIR/bin/activate
56 tox --parallel--safe-build -e black
57 '''
58 }
59 }
60 stage("mypy") {
61 steps {
62 sh '''
63 set +x
64 . $TMPDIR/bin/activate
65 tox --parallel--safe-build -e mypy
66 '''
67 }
68 }
69 }
70 }
71 stage ('Unit Tests') {
72 steps {
73 sh '''
74 set +x
75 . $TMPDIR/bin/activate
76 tox --parallel--safe-build -e py3
77 '''
78 }
79 }
80 stage ('Package builds') {
81 parallel {
82 stage ('Package build: 14.04') {
83 environment {
84 BUILD_SERIES = "trusty"
85 SERIES_VERSION = "14.04"
86 PKG_VERSION = sh(returnStdout: true, script: "dpkg-parsechangelog --show-field Version").trim()
87 NEW_PKG_VERSION = "${PKG_VERSION}~${SERIES_VERSION}~${JOB_SUFFIX}"
88 ARTIFACT_DIR = "${TMPDIR}${BUILD_SERIES}"
89 }
90 steps {
91 sh '''
92 set -x
93 mkdir ${ARTIFACT_DIR}
94 cp debian/changelog ${WORKSPACE}/debian/changelog-${SERIES_VERSION}
95 sed -i "s/${PKG_VERSION}/${NEW_PKG_VERSION}/" ${WORKSPACE}/debian/changelog-${SERIES_VERSION}
96 dpkg-source -l${WORKSPACE}/debian/changelog-${SERIES_VERSION} -b .
97 sbuild --nolog --verbose --dist=${BUILD_SERIES} --no-run-lintian --append-to-version=~${SERIES_VERSION} ../ubuntu-advantage-tools*${NEW_PKG_VERSION}*dsc
98 cp ./ubuntu-advantage-tools*${SERIES_VERSION}*.deb ${ARTIFACT_DIR}/ubuntu-advantage-tools-${BUILD_SERIES}.deb
99 cp ./ubuntu-advantage-pro*${SERIES_VERSION}*.deb ${ARTIFACT_DIR}/ubuntu-advantage-pro-${BUILD_SERIES}.deb
100 '''
101 }
102 }
103 stage ('Package build: 16.04') {
104 environment {
105 BUILD_SERIES = "xenial"
106 SERIES_VERSION = "16.04"
107 PKG_VERSION = sh(returnStdout: true, script: "dpkg-parsechangelog --show-field Version").trim()
108 NEW_PKG_VERSION = "${PKG_VERSION}~${SERIES_VERSION}~${JOB_SUFFIX}"
109 ARTIFACT_DIR = "${TMPDIR}${BUILD_SERIES}"
110 }
111 steps {
112 sh '''
113 set -x
114 mkdir ${ARTIFACT_DIR}
115 cp debian/changelog ${WORKSPACE}/debian/changelog-${SERIES_VERSION}
116 sed -i "s/${PKG_VERSION}/${NEW_PKG_VERSION}/" ${WORKSPACE}/debian/changelog-${SERIES_VERSION}
117 dpkg-source -l${WORKSPACE}/debian/changelog-${SERIES_VERSION} -b .
118 sbuild --nolog --verbose --dist=${BUILD_SERIES} --no-run-lintian --append-to-version=~${SERIES_VERSION} ../ubuntu-advantage-tools*${NEW_PKG_VERSION}*dsc
119 cp ./ubuntu-advantage-tools*${SERIES_VERSION}*.deb ${ARTIFACT_DIR}/ubuntu-advantage-tools-${BUILD_SERIES}.deb
120 cp ./ubuntu-advantage-pro*${SERIES_VERSION}*.deb ${ARTIFACT_DIR}/ubuntu-advantage-pro-${BUILD_SERIES}.deb
121 '''
122 }
123 }
124 stage ('Package build: 18.04') {
125 environment {
126 BUILD_SERIES = "bionic"
127 SERIES_VERSION = "18.04"
128 PKG_VERSION = sh(returnStdout: true, script: "dpkg-parsechangelog --show-field Version").trim()
129 NEW_PKG_VERSION = "${PKG_VERSION}~${SERIES_VERSION}~${JOB_SUFFIX}"
130 ARTIFACT_DIR = "${TMPDIR}${BUILD_SERIES}"
131 }
132 steps {
133 sh '''
134 set -x
135 mkdir ${ARTIFACT_DIR}
136 cp debian/changelog ${WORKSPACE}/debian/changelog-${SERIES_VERSION}
137 sed -i "s/${PKG_VERSION}/${NEW_PKG_VERSION}/" ${WORKSPACE}/debian/changelog-${SERIES_VERSION}
138 dpkg-source -l${WORKSPACE}/debian/changelog-${SERIES_VERSION} -b .
139 sbuild --nolog --verbose --dist=${BUILD_SERIES} --no-run-lintian --append-to-version=~${SERIES_VERSION} ../ubuntu-advantage-tools*${NEW_PKG_VERSION}*dsc
140 cp ./ubuntu-advantage-tools*${SERIES_VERSION}*.deb ${ARTIFACT_DIR}/ubuntu-advantage-tools-${BUILD_SERIES}.deb
141 cp ./ubuntu-advantage-pro*${SERIES_VERSION}*.deb ${ARTIFACT_DIR}/ubuntu-advantage-pro-${BUILD_SERIES}.deb
142 '''
143 }
144 }
145 stage ('Package build: 20.04') {
146 environment {
147 BUILD_SERIES = "focal"
148 SERIES_VERSION = "20.04"
149 PKG_VERSION = sh(returnStdout: true, script: "dpkg-parsechangelog --show-field Version").trim()
150 NEW_PKG_VERSION = "${PKG_VERSION}~${SERIES_VERSION}~${JOB_SUFFIX}"
151 ARTIFACT_DIR = "${TMPDIR}${BUILD_SERIES}"
152 }
153 steps {
154 sh '''
155 set -x
156 mkdir ${ARTIFACT_DIR}
157 cp debian/changelog ${WORKSPACE}/debian/changelog-${SERIES_VERSION}
158 sed -i "s/${PKG_VERSION}/${NEW_PKG_VERSION}/" ${WORKSPACE}/debian/changelog-${SERIES_VERSION}
159 dpkg-source -l${WORKSPACE}/debian/changelog-${SERIES_VERSION} -b .
160 sbuild --nolog --verbose --dist=${BUILD_SERIES} --no-run-lintian --append-to-version=~${SERIES_VERSION} ../ubuntu-advantage-tools*${NEW_PKG_VERSION}*dsc
161 cp ./ubuntu-advantage-tools*${SERIES_VERSION}*.deb ${ARTIFACT_DIR}/ubuntu-advantage-tools-${BUILD_SERIES}.deb
162 cp ./ubuntu-advantage-pro*${SERIES_VERSION}*.deb ${ARTIFACT_DIR}/ubuntu-advantage-pro-${BUILD_SERIES}.deb
163 '''
164 }
165 }
166 }
167 }
168 stage ('Integration Tests') {
169 parallel {
170 stage("lxc 14.04") {
171 environment {
172 UACLIENT_BEHAVE_DEBS_PATH = "${TMPDIR}trusty/"
173 UACLIENT_BEHAVE_ARTIFACT_DIR = "${TMPDIR}artifacts/behave-lxd-14.04"
174 }
175 steps {
176 sh '''
177 set +x
178 . $TMPDIR/bin/activate
179 tox --parallel--safe-build -e behave-lxd-14.04
180 '''
181 }
182 }
183 stage("lxc 16.04") {
184 environment {
185 UACLIENT_BEHAVE_DEBS_PATH = "${TMPDIR}xenial/"
186 UACLIENT_BEHAVE_ARTIFACT_DIR = "${TMPDIR}artifacts/behave-lxd-16.04"
187 }
188 steps {
189 sh '''
190 set +x
191 . $TMPDIR/bin/activate
192 tox --parallel--safe-build -e behave-lxd-16.04
193 '''
194 }
195 }
196 stage("lxc 18.04") {
197 environment {
198 UACLIENT_BEHAVE_DEBS_PATH = "${TMPDIR}bionic/"
199 UACLIENT_BEHAVE_ARTIFACT_DIR = "${TMPDIR}artifacts/behave-lxd-18.04"
200 }
201 steps {
202 sh '''
203 set +x
204 . $TMPDIR/bin/activate
205 tox --parallel--safe-build -e behave-lxd-18.04
206 '''
207 }
208 }
209 stage("lxc vm 20.04") {
210 environment {
211 UACLIENT_BEHAVE_DEBS_PATH = "${TMPDIR}focal/"
212 UACLIENT_BEHAVE_ARTIFACT_DIR = "${TMPDIR}artifacts/behave-vm-20.04"
213 }
214 steps {
215 sh '''
216 set +x
217 . $TMPDIR/bin/activate
218 tox --parallel--safe-build -e behave-vm-20.04
219 '''
220 }
221 }
222 stage("awspro 18.04") {
223 environment {
224 UACLIENT_BEHAVE_DEBS_PATH = "${TMPDIR}bionic/"
225 UACLIENT_BEHAVE_ARTIFACT_DIR = "${TMPDIR}artifacts/behave-awspro-18.04"
226 }
227 steps {
228 sh '''
229 set +x
230 . $TMPDIR/bin/activate
231 tox --parallel--safe-build -e behave-awspro-18.04
232 '''
233 }
234 }
235 }
236 }
237 }
238 post {
239 always {
240 script {
241 try {
242 sh '''
243 set +x
244 DATE=`date -d 'now+1day' +%m/%d/%Y`
245 git clone https://github.com/canonical/server-test-scripts.git
246 python3 server-test-scripts/ubuntu-advantage-client/lxd_cleanup.py --prefix ubuntu-behave-test-$CHANGE_ID --before-date $DATE || true
247 '''
248
249 junit "pytest_results.xml"
250 junit "reports/*.xml"
251 } catch (Exception e) {
252 echo e.toString()
253 currentBuild.result = 'UNSTABLE'
254 }
255 try {
256 archiveArtifacts "/tmp/${BUILD_TAG}/artifacts/**/*"
257 } catch (Exception e) {
258 echo "No integration test artifacts found. Presume success."
259 }
260 }
261 }
262 }
263}
diff --git a/Makefile b/Makefile
index d52d1e0..e8ce957 100644
--- a/Makefile
+++ b/Makefile
@@ -27,8 +27,12 @@ test:
27 @tox27 @tox
2828
29testdeps:29testdeps:
30 pip install -U six30ifneq (,$(findstring trusty,$(TOXENV)))
31 @echo Pinning virtualenv to 20.0.31 on trusty because 32 breaks py3.4
32 pip install virtualenv==20.0.31
33endif
31 pip install tox34 pip install tox
35 pip install tox-pip-version
3236
33travis-deb-install:37travis-deb-install:
34 git fetch --unshallow38 git fetch --unshallow
@@ -48,6 +52,8 @@ travis-deb-script:
48 # Use this to get a new shell where we're in the sbuild group52 # Use this to get a new shell where we're in the sbuild group
49 sudo -E su ${USER} -c 'mk-sbuild ${PACKAGE_BUILD_SERIES}'53 sudo -E su ${USER} -c 'mk-sbuild ${PACKAGE_BUILD_SERIES}'
50 sudo -E su ${USER} -c 'sbuild --nolog --verbose --dist=${PACKAGE_BUILD_SERIES} ../ubuntu-advantage-tools*.dsc'54 sudo -E su ${USER} -c 'sbuild --nolog --verbose --dist=${PACKAGE_BUILD_SERIES} ../ubuntu-advantage-tools*.dsc'
55 cp ./ubuntu-advantage-tools*.deb ubuntu-advantage-tools-${PACKAGE_BUILD_SERIES}.deb
56 cp ./ubuntu-advantage-pro*.deb ubuntu-advantage-tools-pro-${PACKAGE_BUILD_SERIES}.deb
5157
5258
53.PHONY: build clean test testdeps demo59.PHONY: build clean test testdeps demo
diff --git a/README.md b/README.md
index 10e88d2..c45c58b 100644
--- a/README.md
+++ b/README.md
@@ -1,22 +1,122 @@
1# Ubuntu Advantage Client1# Ubuntu Advantage Client
22
3[![Build Status](https://travis-ci.org/CanonicalLtd/ubuntu-advantage-client.svg?branch=master)](https://travis-ci.org/CanonicalLtd/ubuntu-advantage-client)3[![Build Status](https://travis-ci.com/canonical/ubuntu-advantage-client.svg?branch=master)](https://travis-ci.com/github/canonical/ubuntu-advantage-client)
44
5The Ubuntu Advantage client provides users with a simple mechanism to5The Ubuntu Advantage client provides users with a simple mechanism to
6view, enable, and disable offerings from Canonical on their system. The6view, enable, and disable offerings from Canonical on their system. The
7following entitlements are supported:7following entitlements are supported:
88
9- [Common Criteria EAL2 certification artifacts provisioning](https://ubuntu.com/cc-eal)9- [Common Criteria EAL2 certification artifacts provisioning](https://ubuntu.com/cc-eal)
10- [Canonical CIS Benchmark Audit Tool](https://ubuntu.com/cis)10- [Canonical CIS Benchmark Audit Tool](https://ubuntu.com/cis-audit)
11- [Ubuntu Extended Security Maintenance](https://ubuntu.com/esm)11- [Ubuntu Extended Security Maintenance](https://ubuntu.com/esm)
12- [FIPS 140-2 Certified Modules](https://ubuntu.com/fips)12- [FIPS 140-2 Certified Modules](https://ubuntu.com/fips)
13- [FIPS 140-2 Non-Certified Module Updates](https://ubuntu.com/fips-updates)13- [FIPS 140-2 Non-Certified Module Updates](https://ubuntu.com/fips)
14- [Livepatch Service](https://www.ubuntu.com/livepatch)14- [Livepatch Service](https://www.ubuntu.com/livepatch)
1515
16## Obtaining the Client16## Obtaining the Client
1717
18The client comes pre-installed on all Ubuntu systems. Users can run the18The client comes pre-installed on all Ubuntu systems in the debian packages
19`ua` command to learn more or view the manpage.19`ubuntu-advantage-tools` package. Ubuntu Pro images on AWS and Azure Ubuntu Pro
20images will also contain `ubuntu-advantage-pro` which automates machine attach
21on custom AWS and Azure images.
22
23Users can manually run the `ua` command to learn more or view the manpage.
24
25## Terminology
26 The following vocabulary is used to describe different aspects of the work
27Ubuntu Advantage Client performs:
28
29| Term | Meaning |
30| -------- | -------- |
31| UA Client | The python command line client represented in this ubuntu-advantage-client repository. It is installed on each Ubuntu machine and is the entry-point to enable any Ubuntu Advantage commercial service on an Ubuntu machine. |
32| Contract Server | The backend service exposing a REST API to which UA Client authenticates in order to obtain contract and commercial service information and manage which support services are active on a machine.|
33| Entitlement/Service | An Ubuntu Advantage commercial support service such as FIPS, ESM, Livepatch, CIS-Audit to which a contract may be entitled |
34| Affordance | Service-specific list of applicable architectures and Ubuntu series on which a service can run |
35| Directives | Service-specific configuration values which are applied to a service when enabling that service |
36| Obligations | Service-specific policies that must be instrumented for support of a service. Example: `enableByDefault: true` means that any attached machine **MUST** enable a service on attach |
37
38
39## Architecture
40Ubuntu Advantage client, hereafter "UA client", is python3-based command line
41utility. It provides a CLI to attach, detach, enable,
42disable and check status of support related services.
43
44The package `ubuntu-advantage-tools` also provides a C++ APT hook which helps
45advertise ESM service and available packages in MOTD and during various apt
46commands.
47
48The `ubuntu-advantage-pro` package delivers auto-attach auto-enable
49functionality via init scripts and systemd services for various cloud
50platforms.
51
52By default, Ubuntu machines are deployed in an unattached state. A machine can
53get manually or automatically attached to a specific contract by interacting
54with the Contract Server REST API. Any change in state of services or machine
55attach results in additional interactions with the Contract Server API to
56validate such operations. The contract server API is described by the
57[ua-contracts openapi spec](https://github.com/CanonicalLtd/ua-contracts/blob/develop/docs/contracts.yaml).
58
59### Attaching a machine
60Each Ubuntu SSO account holder has access to one or more contracts. To attach
61a machine to an Ubuntu Advantage contract:
62
63* An Ubuntu SSO account holder must obtain a contract token from
64https://ubuntu.com/advantage.
65* Run `sudo ua attach <contractToken>` on the machine
66 - Ubuntu Pro images for AWS and Azure perform an auto-attach without tokens
67* UA Client reads config from /etc/ubuntu-advantage/uaclient.conf to obtain
68 the contract_url (default: https://contracts.canonical.com)
69* UA Client POSTs to the Contract Server API @
70 <contract_url>/api/v1/context/machines/token providing the \<contractToken\>
71* The Contract Server responds with a JSON blob containing an unique machine
72 token, service credentials, affordances, directives and obligations to allow
73 enabling and disabling Ubuntu Advantage services
74* UA client writes the machine token API response to the root-readonly
75 /var/lib/ubuntu-advantage/machine-token.json
76* UA client auto-enables any services defined with
77 `obligations:{enableByDefault: true}`
78
79### Enabling a service
80Each service controlled by UA client will have a python module in
81uaclient/entitlements/\*.py which handles setup and teardown of services when
82enabled or disabled.
83
84If a contract entitles a machine to a service, `root` user can enable the
85service with `ua enable <service>`. If a service can be disabled
86`ua disabled <service>` will be permitted.
87
88The goal of the UA client is to remain simple and flexible and let the
89contracts backend drive dynamic changes in contract offerings and constraints.
90In pursuit of that goal, the UA client obtains most of it's service constraints
91from a machine token that it obtains from the Contract Server API.
92
93The UA Client is simple in that it relies on the machine token on the attached
94machine to describe whether a service is applicable for an environment and what
95configuration is required to properly enable that service.
96
97Any interactions with the Contract server API are defined as UAContractClient
98class methods in [uaclient/contract.py](uaclient/contract.py).
99
100## Directory layout
101The following describes the intent of UA client related directories:
102
103
104| File/Directory | Intent |
105| -------- | -------- |
106| ./tools | Helpful scripts used to publish, release or test various aspects of UA client |
107| ./features/ | Behave BDD integration tests for UA Client
108| ./uaclient/ | collection of python modules which will be packaged into ubuntu-advantage-tools package to deliver the UA Client CLI |
109| uaclient.entitlements | Service-specific \*Entitlement class definitions which perform enable, disable, status, and entitlement operations etc. All classes derive from base.py:UAEntitlement and many derive from repo.py:RepoEntitlement |
110| ./uaclient/cli.py | The entry-point for the command-line client
111| ./uaclient/clouds/ | Cloud-platform detection logic used in Ubuntu Pro to determine if a given should be auto-attached to a contract |
112| uaclient.contract | Module for interacting with the Contract Server API |
113| ./demo | Various stale developer scripts for setting up one-off demo environments. (Not needed often)
114| ./apt-hook/ | the C++ apt-hook delivering MOTD and apt command notifications about UA support services |
115| ./apt-conf.d/ | apt config files delivered to /etc/apt/apt-conf.d to automatically allow unattended upgrades of ESM security-related components |
116| /etc/ubuntu-advantage/uaclient.conf | Configuration file for the UA client.|
117| /var/lib/ubuntu-advantage/private | `root` read-only directory containing Contract API responses, machine-tokens and service credentials |
118| /var/log/ubuntu-advantage.log | `root` read-only log of ubuntu-advantage operations |
119
20120
21## Testing121## Testing
22122
@@ -42,19 +142,49 @@ directory and consist of two parts: `.feature` files that define the
42tests we want to run, and `.py` files which implement the underlying142tests we want to run, and `.py` files which implement the underlying
43logic for those tests.143logic for those tests.
44144
145By default, integration tests will do the folowing on a given cloud platform:
146 * Launch an instance running latest daily image of the target Ubuntu release
147 * Add the Ubuntu advantage client daily build PPA: [ppa:canonical-server/ua-client-daily](https://code.launchpad.net/~canonical-server/+archive/ubuntu/ua-client-daily)
148 * Install the appropriate ubuntu-advantage-tools and ubuntu-advantage-pro deb
149 * Stop the instance and snapshot it creating an updated bootable image for
150 test runs
151 * Launch a fresh instance based on the boot-image created and exercise tests
152
153The testing can be overridden to run using a local copy of the ubuntu-advantage-client source code instead of the daily PPA by providing the following environment variable to the behave test runner.
154```UACLIENT_BEHAVE_BUILD_PR=1```
155
45To run the tests, you can use `tox`:156To run the tests, you can use `tox`:
46157
47```shell158```shell
48tox -e behave159tox -e behave-20.04
49```160```
50161
51or, if you just want to run a specific file, or a test within a file:162or, if you just want to run a specific file, or a test within a file:
52163
53```shell164```shell
54tox -e behave features/unattached_commands.feature165tox -e behave-20.04 features/unattached_commands.feature
55tox -e behave features/unattached_commands.feature:55166tox -e behave-20.04 features/unattached_commands.feature:55
56```167```
57168
169As can be seen, this will run behave tests only for release 20.04 (Focal Fossa). We are currently
170supporting 4 distinct releases:
171
172* 20.04 (Focal Fossa)
173* 18.04 (Bionic Beaver)
174* 16.04 (Xenial Xerus)
175* 14.04 (Trusty Tahr)
176
177Therefore, to change which release to run the behave tests against, just change the release version
178on the behave command.
179
180Furthermore, when developing/debugging a new scenario:
181
182 1. Add a `@wip` tag decorator on the scenario
183 2. To only run @wip scenarios run: `tox -e behave-20.04 -- -w`
184 4. If you want to use a debugger:
185 a. Add ipdb to integration-requirements.txt
186 b. Add ipdb.set_trace() in the code block you wish to debug
187
58(If you're getting started with behave, we recommend at least reading188(If you're getting started with behave, we recommend at least reading
59through [the behave189through [the behave
60tutorial](https://behave.readthedocs.io/en/latest/tutorial.html) to get190tutorial](https://behave.readthedocs.io/en/latest/tutorial.html) to get
@@ -104,25 +234,138 @@ you need to add `-D reuse_container=container_name`:
104tox -e behave -D reuse_container=container_name234tox -e behave -D reuse_container=container_name
105```235```
106236
237#### Integration testing on EC2
238The following tox environments allow for testing focal on EC2:
239
240```
241 # To test ubuntu-pro-images on EC2
242 tox -e behave-awspro-20.04
243 # To test Canonical cloud images (non-ubuntu-pro) on EC2
244 tox -e behave-awsgeneric-20.04
245```
246
247To run the test for a different release, just update the release version string. For example,
248to run AWS pro xenial tests, you can run:
249
250```
251tox -e behave-awspro-16.04
252```
253
254In order to run EC2 tests the following environment variables are required:
255 - UACLIENT_BEHAVE_AWS_ACCESS_KEY_ID
256 - UACLIENT_BEHAVE_AWS_SECRET_ACCESS_KEY
257
258
259To specifically run non-ubuntu pro tests using canonical cloud-images an
260additional token obtained from https://ubuntu.com/advantage needs to be set:
261 - UACLIENT_BEHAVE_CONTRACT_TOKEN=<your_token>
262
263By default, the public AMIs for Ubuntu Pro testing used for each Ubuntu
264release are defined in features/aws-ids.yaml. These ami-ids are determined by
265running `./tools/refresh-aws-pro-ids`.
266
267Integration tests will read features/aws-ids.yaml to determine which default
268AMI id to use for each supported Ubuntu release.
269
270To update `features/aws-ids.yaml`, run `./tools/refresh-aws-pro-ids` and put up
271a pull request against this repo to updated that content from the ua-contracts
272marketplace definitions.
273
274* To manually run EC2 integration tests using packages from `ppa:canonical-server/ua-client-daily` provide the following environment vars:
275
276```sh
277UACLIENT_BEHAVE_AWS_ACCESS_KEY_ID=<blah> UACLIENT_BEHAVE_AWS_SECRET_KEY=<blah2> tox -e behave-awspro-20.04
278```
279
280* To manually run EC2 integration tests with a specific AMI Id provide the
281following environment variable to launch your specfic AMI instead of building
282a daily ubuntu-advantage-tools image.
283```sh
284UACLIENT_BEHAVE_REUSE_IMAGE=your-custom-ami tox -e behave-awspro-20.04
285```
286
287#### Integration testing on Azure
288The following tox environments allow for testing focal on Azure:
289
290```
291 # To test ubuntu-pro-images on EC2
292 tox -e behave-azurepro-20.04
293 # To test Canonical cloud images (non-ubuntu-pro) on EC2
294 tox -e behave-azuregeneric-20.04
295```
296
297To run the test for a different release, just update the release version string. For example,
298to run AWS pro xenial tests, you can run:
299
300```
301tox -e behave-azurepro-16.04
302```
303
304In order to run EC2 tests the following environment variables are required:
305 - UACLIENT_BEHAVE_AZ_CLIENT_ID
306 - UACLIENT_BEHAVE_AZ_CLIENT_SECRET
307 - UACLIENT_BEHAVE_AZ_SUBSCRIPTION_ID
308 - UACLIENT_BEHAVE_AZ_TENANT_ID
309
310
311To specifically run non-ubuntu pro tests using canonical cloud-images an
312additional token obtained from https://ubuntu.com/advantage needs to be set:
313 - UACLIENT_BEHAVE_CONTRACT_TOKEN=<your_token>
314
315* To manually run Azure integration tests using packages from `ppa:canonical-server/ua-client-daily` provide the following environment vars:
316
317```sh
318UACLIENT_BEHAVE_AZ_CLIENT_ID=<blah> UACLIENT_BEHAVE_AZ_CLIENT_SECRET=<blah2> UACLIENT_BEHAVE_AZ_SUBSCRIPTION_ID=<blah3> UACLIENT_BEHAVE_AZ_TENANT_ID=<blah4> tox -e behave-azurepro-20.04
319```
320
321* To manually run Azure integration tests with a specific Image Id provide the
322following environment variable to launch your specfic Image Id instead of building
323a daily ubuntu-advantage-tools image.
324```sh
325UACLIENT_BEHAVE_REUSE_IMAGE=your-custom-image-id tox -e behave-awspro-20.04
326```
327
107## Building328## Building
108329
109The packaging for the UA client package (ubuntu-advantage-tools) is330Creating ubuntu-advantage-tools and ubuntu-advantage-pro is created from the
110in-tree, so you can build the package the way you would normally build331debian/control file in this repository. You can build the
111a Debian package:332package the way you would normally build a Debian package:
333
112334
113```shell335```shell
114dpkg-buildpackage336dpkg-buildpackage -us -uc
115```337```
116338
117or, if you want to build for a target release other than the release339**Note** It will build the package with dependencies for the Ubuntu release on
118you're on, [configure sbuild](https://wiki.ubuntu.com/SimpleSbuild) and340which you are building, so it's best to build in a container of kvm for the
341release you are targeting.
342
343OR, if you want to build for a target release other than the release
344you're on:
345
346### using sbuild
347[configure sbuild](https://wiki.ubuntu.com/SimpleSbuild) and
119use that for the build:348use that for the build:
120349
350
121```shell351```shell
122debuild -S352debuild -S
123sbuild --dist=<target> ../ubuntu-advantage-tools_*.dsc353sbuild --dist=<target> ../ubuntu-advantage-tools_*.dsc
124```354```
125355
356### Setting up an lxc development container
357```shell
358lxc launch ubuntu-daily:trusty dev-t -c user.user-data="$(cat tools/ua-dev-cloud-config.yaml)"
359lxc exec dev-t bash
360```
361
362### Setting up a kvm development environment with multipass
363**Note:** There is a sample procedure documented in tools/multipass.md as well.
364```shell
365multipass launch daily:focal -n dev-f --cloud-init tools/ua-dev-cloud-config.yaml
366multipass connect dev-f
367```
368
126## Code Formatting369## Code Formatting
127370
128The `ubuntu-advantage-client` code base is formatted using371The `ubuntu-advantage-client` code base is formatted using
@@ -151,15 +394,26 @@ the project, you should install them via `dev-requirements.txt`.)
151On Launchpad, there is a [daily build recipe](https://code.launchpad.net/~canonical-server/+recipe/ua-client-daily),394On Launchpad, there is a [daily build recipe](https://code.launchpad.net/~canonical-server/+recipe/ua-client-daily),
152which will build the client and place it in the [ua-client-daily PPA](https://code.launchpad.net/~canonical-server/+archive/ubuntu/ua-client-daily).395which will build the client and place it in the [ua-client-daily PPA](https://code.launchpad.net/~canonical-server/+archive/ubuntu/ua-client-daily).
153396
154## Demo397## Remastering custom golden images based on Ubuntu PRO
155398
156Users can demo the client with a local backend. This can be done with399Vendors who wish to provide custom images based on Ubuntu PRO images can
157the following:400follow the procedure below:
158401
159```shell402* Launch the Ubuntu PRO golden image
160# Set up ua-contracts in a docker container in a bionic lxc on port 3000403* Customize your golden image as you see fit
161make demo404* If `ua status` shows attached, remove the UA artifacts to allow clean
162# Set up two clients pointing at the local contract server405 auto-attach on subsequent cloned VM launches
163./demo/run-uaclient --series disco406```bash
164./demo/run-uaclient --series xenial -b multipass407sudo ua detach
408sudo rm -rf /var/log/ubuntu-advantage.log # to remove credentials and tokens from logs
165```409```
410* Remove `cloud-init` first boot artifacts so the cloned VM boot is seen as a first boot
411```bash
412sudo cloud-init clean --logs
413sudo shutdown -h now
414```
415* Use your cloud platform to clone or snapshot this VM as a golden image
416
417
418## Releasing ubuntu-adantage-tools
419see [RELEASES.md](RELEASES.md)
diff --git a/RELEASES.md b/RELEASES.md
166new file mode 100644420new file mode 100644
index 0000000..e31dc37
--- /dev/null
+++ b/RELEASES.md
@@ -0,0 +1,63 @@
1# Ubuntu Advantage Client Releases
2
3## Release versioning schemes:
4
5Below are the versioning schemes used for publishing debs:
6
7| Build target | Version Format |
8| -------- | -------- |
9| Devel series upstream release | XX.YY |
10| Devel series bugfix release | XX.YY.Z~ubuntu1|
11| Stable series release | XX.YY~ubuntu1~18.04.1|
12| [Daily Build Recipe](https://code.launchpad.net/~canonical-server/+recipe/ua-client-daily) | XX.YY+<revtime>-g<commitish>~ubuntu1~18.04.1 |
13| Ubuntu PRO series release | binary copies of Daily PPA |
14
15## Supported upgrade use-cases based on version formats
16
17| Upgrade path | Version diffs |
18| LTS to LTS | 20.3~ubuntu1~14.04.1 -> 20.3~ubuntu1~16.04.1 |
19| LTS to Daily PPA | 20.3~ubuntu1~14.04.1 -> 20.3+202004011202~ubuntu1~14.04.1 |
20| Ubuntu PRO to latest <series>-updates | 20.3+202004011202~ubuntu1~14.04.1 -> 20.4~ubuntu1~14.04.1 |
21| Ubuntu PRO to Daily PPA | 20.3+202004011202~ubuntu1~14.04.1 -> 20.4+202004021202~ubuntu1~14.04.1 |
22
23
24## Devel Release Process
25
26Below is the procedure used to release ubuntu-advantage-client to an Ubuntu series:
27
28 1. git-ubuntu clone ubuntu-advantage-tools; cd ubuntu-advantage-tools
29 2. git remote add upstream git@github.com:canonical/ubuntu-advantage-client.git
30 3. git fetch upstream
31 4. git checkout upstream/release-24
32 5. git tag -a 24.3
33 6. git rebase --onto pkg/ubuntu/devel 24.2 24.3
34 7. git checkout -b pkg-upload-24.3
35 8. debuild -S
36 9. dput ppa:chad.smith/ua-client-uploads ./ubuntu-advantage-tools_24.3_source.changes
37 10. create a release tarfile/changes file using uss-tableflip's build-package script
38```
39 ./tools/make-release --series <SERIES> --ppa <PPA_URL_FOR_TEST_UPLOADS>
40 # follow printed procedure for dput to test PPA, and tag the released commitish
41```
42 11. Create a merge proposal such as [24.2 PR](https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/385073), [24.3 PR](https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/389745)
43 11. For SRUs: Create an "upload a new version bug" in https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bugs
44 12. Describe the need, provide testing PPA and describe test instructions
45 13. Ping appropriate maintainer about upload and verification
46 14. Validate tests, accept upload and ship it
47
48Previous release bugs:
49| ------- |
50| [20.3 Focal](https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/1869980) |
51
52
53## Ubuntu PRO Release Process
54
55Manually perform a binary package copy from Daily PPA to Premium PPA and notify image creators
56
57 1. [Open Daily PPA copy-package operation](https://code.launchpad.net/~canonical-server/+archive/ubuntu/ua-client-daily/+copy-packages)
58 2. Check Trusty, Xenial, Bionic package
59 3. Select Destination PPA: UA Client Premium [~canonical-server/ubuntu/ua-client-premium]
60 4. Select Destination series: The same series
61 5. Copy options: "Copy existing binaries
62 6. Click Copy packages
63 7. Notify Pro Image creatros about expected Premium PPA version (patviafore/rcj)
diff --git a/apt-hook/hook.cc b/apt-hook/hook.cc
index 56032f4..ed009de 100644
--- a/apt-hook/hook.cc
+++ b/apt-hook/hook.cc
@@ -33,8 +33,10 @@
33#include <locale.h>33#include <locale.h>
3434
35struct result {35struct result {
36 int enabled_esms;36 int enabled_esms_i;
37 int disabled_esms;37 int disabled_esms_i;
38 int enabled_esms_a;
39 int disabled_esms_a;
38};40};
3941
40// Return parent pid of specified pid, using /proc (pid might be self)42// Return parent pid of specified pid, using /proc (pid might be self)
@@ -122,10 +124,22 @@ static void check_esm_upgrade(pkgCache::PkgIterator pkg, pkgPolicy *policy, resu
122 // TODO: Just look at the origin, not pinning.124 // TODO: Just look at the origin, not pinning.
123 if (pf.File().Archive() != 0 && pf.File().Origin() == std::string("UbuntuESM"))125 if (pf.File().Archive() != 0 && pf.File().Origin() == std::string("UbuntuESM"))
124 {126 {
127 // Xenial and later should not be advertising unauthenticated ESM Infra apt repos
125 if (policy->GetPriority(pf.File()) == -32768)128 if (policy->GetPriority(pf.File()) == -32768)
126 res.disabled_esms++;129 res.disabled_esms_i++;
127 else130 else
128 res.enabled_esms++;131 res.enabled_esms_i++;
132
133 return;
134 }
135
136 if (pf.File().Archive() != 0 && pf.File().Origin() == std::string("UbuntuESMApps"))
137 {
138 // Xenial and later should not be advertising unauthenticated ESM Apps apt repos
139 if (policy->GetPriority(pf.File()) == -32768)
140 res.disabled_esms_a++;
141 else
142 res.enabled_esms_a++;
129143
130 return;144 return;
131 }145 }
@@ -187,15 +201,16 @@ int main(int argc, char *argv[])
187 // assert(ppid == getppid_of("self"));201 // assert(ppid == getppid_of("self"));
188 // }202 // }
189 assert(cmdline_eligible(make_cmdline("apt\0update\0")));203 assert(cmdline_eligible(make_cmdline("apt\0update\0")));
190 assert(cmdline_eligible(make_cmdline("apt-get\0update\0")));
191 assert(!cmdline_eligible(make_cmdline("apt-get\0install\0")));
192 assert(!cmdline_eligible(make_cmdline("apt\0install\0")));
193 assert(!cmdline_eligible(make_cmdline("apt\0install\0")));204 assert(!cmdline_eligible(make_cmdline("apt\0install\0")));
194 assert(cmdline_eligible(make_cmdline("aptitude\0upgrade\0")));205 assert(cmdline_eligible(make_cmdline("aptitude\0upgrade\0")));
195 assert(cmdline_eligible(make_cmdline("aptitude\0update\0")));206 assert(cmdline_eligible(make_cmdline("aptitude\0update\0")));
196 command_used = "";207 command_used = "";
197208
198 result res = {0, 0};209 result res = {0, 0, 0, 0};
210
211 // useful for testing
212 if (has_arg(argv, "test"))
213 command_used = "update";
199214
200 if (has_arg(argv, "test") || cmdline_eligible(getcmdline(getppid_of(getppid_of("self")))))215 if (has_arg(argv, "test") || cmdline_eligible(getcmdline(getppid_of(getppid_of("self")))))
201 get_update_count(res);216 get_update_count(res);
@@ -205,27 +220,51 @@ int main(int argc, char *argv[])
205 return 1;220 return 1;
206 }221 }
207222
208 if (res.enabled_esms > 0 && (command_used == "update"))223 if (command_used == "update")
209 {224 {
210 ioprintf(std::cout,225 if (res.enabled_esms_i > 0)
211 ngettext("%d of the updates is from UA Infrastructure ESM.",226 {
212 "%d of the updates are from UA Infrastructure ESM.",227 ioprintf(std::cout,
213 res.enabled_esms),228 ngettext("%d of the updates is from UA Infra: ESM.",
214 res.enabled_esms);229 "%d of the updates are from UA Infra: ESM.",
215 ioprintf(std::cout, "\n");230 res.enabled_esms_i),
231 res.enabled_esms_i);
232 ioprintf(std::cout, "\n");
233 }
234 if (res.enabled_esms_a > 0)
235 {
236 ioprintf(std::cout,
237 ngettext("%d of the updates is from UA Apps: ESM.",
238 "%d of the updates are from UA Apps: ESM.",
239 res.enabled_esms_a),
240 res.enabled_esms_a);
241 ioprintf(std::cout, "\n");
242 }
216 }243 }
217244
218 if (res.disabled_esms > 0)245 if (res.disabled_esms_i > 0 || res.disabled_esms_a > 0)
219 {246 {
220 if (command_used != "update")247 if (command_used != "update")
221 std::cout << std::endl;248 std::cout << std::endl;
222 ioprintf(std::cout,249 if (res.disabled_esms_i > 0)
223 ngettext("%d additional update is available with UA Infrastructure ESM.",250 {
224 "%d additional updates are available with UA Infrastructure ESM.",251 ioprintf(std::cout,
225 res.disabled_esms),252 ngettext("%d additional update is available with UA Infra: ESM.",
226 res.disabled_esms);253 "%d additional updates are available with UA Infra: ESM.",
254 res.disabled_esms_i),
255 res.disabled_esms_i);
256 ioprintf(std::cout, "\n");
257 }
258 if (res.disabled_esms_a > 0)
259 {
260 ioprintf(std::cout,
261 ngettext("%d additional update is available with UA Apps: ESM.",
262 "%d additional updates are available with UA Apps ESM.",
263 res.disabled_esms_a),
264 res.disabled_esms_a);
265 ioprintf(std::cout, "\n");
266 }
227267
228 ioprintf(std::cout, "\n");
229 ioprintf(std::cout, gettext("To see these additional updates run: apt list --upgradable"));268 ioprintf(std::cout, gettext("To see these additional updates run: apt list --upgradable"));
230 ioprintf(std::cout, "\n");269 ioprintf(std::cout, "\n");
231 ioprintf(std::cout, gettext("See https://ubuntu.com/advantage or run: sudo ua status"));270 ioprintf(std::cout, gettext("See https://ubuntu.com/advantage or run: sudo ua status"));
diff --git a/debian/changelog b/debian/changelog
index 001d980..1180c0f 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,228 @@
1ubuntu-advantage-tools (26.1~21.04.1) hirsute; urgency=medium
2
3 * New upstream release 26.1
4 - contract: block detach call to contract if machine-id change
5 - docs: add readme docs about mastering clean golden images
6 - fips: add reboot notices for fips operations (GH: #1368)
7 - livepatch: add retry when running canonical-livepatch status
8 (GH: #1360)
9 - util: use lru_cache to avoid re-reading os-release and machine-id
10 (GH: #1329)
11 - tests:
12 + add disable_auto_attach config to all test PRO vms
13 + add more log artifacts during failed integration test
14 + check cloudinit status after launching image
15 + mock leaking livepatch.application_status for fips test
16 + retry package installs on apt exit 100
17 - jenkins: parameterize build stages to avoid parallel job collision
18
19 -- Lucas Moura <lucas.moura@canonical.com> Fri, 19 Feb 2021 10:30:22 -0300
20
21ubuntu-advantage-tools (26.0.1~21.04.1) hirsute; urgency=medium
22
23 * auto-attach: fix comparing numeric iid
24
25 -- Lucas Moura <lucas.moura@canonical.com> Fri, 05 Feb 2021 14:10:09 -0300
26
27ubuntu-advantage-tools (26.0~21.04.1) hirsute; urgency=medium
28
29 * New upstream release 26.0:
30 - auto-attach: systemd unit to run before ua-reboot-cmds.service
31 - config: remove_notice should remove notices.json when empty
32 - fips:
33 + add notice if running a deactivated FIPS kernel (GH: #1348)
34 + block enabling FIPS on clouds using Xenial
35 + block enabling fips on GCP instances
36 + check /proc/sys/crypto/fips_enable to see if fips is enabled
37 + override fips metapackage when on bionic cloud
38 + update metapackage override logic on fips
39 - notices: clear lock file and notice when encountering any exception
40 (GH: #1326)
41 - reboot_cmds: retry on lock held errors due to pro auto-attach
42 - services: allow uaclient to disable services during enable
43 - status: include beta services in json formatted output with --all
44 (GH: #1341)
45 - tests:
46 + add FIPS tests to AWS and Azure bionic images
47 + add GCP pro test for focal machine
48 + add after_step collection of artifacts on failure
49 + remove proc file check after disabling fips
50 + pro: block auto-attach with cloud-config bootcmd
51 + add validation of systemd unit ua-reboot-cmds.service
52 + test enabling fips-updates when fips is enabled
53 - jenkins:
54 - add deb build stage to assert package builds
55 - use series-specific sbuild --build-dir avoid races
56 - use --append-to-version for each sbuild run to avoid races
57 - presume success when no integration artifacts created
58
59 -- Lucas Moura <lucas.moura@canonical.com> Thu, 04 Feb 2021 16:34:56 -0300
60
61ubuntu-advantage-tools (26.0~21.04.1~beta) hirsute; urgency=medium
62
63 * d/rules:
64 - add --with systemd to allow reboot init script
65 - do not remove lib/systemd/system folder
66 * d/postinst:
67 - create marker file when reboot script need to run:
68 - enable livepatch across trusty to xenial upgrade
69 - update fips on existing fips pro machines
70 * New upstream release 26.0~beta:
71 - gcp: add Google Cloud Platform support (GH #1269)
72 - fips:
73 + remove is_beta from fips sevices
74 + fips pro: add upgrade support to require reboot to unmark held fips pkgs
75 + update origin UbuntuFIPSUpdates
76 - status:
77 + add notice to tabular output
78 + held locks emit notice about Operation in progress
79 - cli: help sort output so trusty ordering matches xenial++
80 - cis: rename service from cis-audit
81 - config: provide config notices and add_notice and remove_notice methods
82 - contract: add resource-machine-access route and datapath
83 - init: add init script to run commands on reboot
84 - keys: add ubuntu-advantage-cis keyring
85 - livepatch: make livepatch react to enableByDefault delta
86 - log: log when we install pkgs because of contract delta
87 - make: drop six testdeps target
88 - pro: do not install pro debs on non-pro instances
89 - services: Update beta info for services (GH #1220)
90 - tools: add tox-lxd-runner, that execute the test command in a shell
91 - tools: refresh-keyrings handles cis keys. drop series-specific keys
92 - tests:
93 + add GCE support for integration tests
94 + add cis integration tests for unattached and pro
95 + add pytest constraint for mypy tests
96 + add unittests for reboot_cmds script
97 + fix esm package messages for new update notifier version
98 + pin importlib-metadata for mypy tests
99 + repo tests for request_resource_machine_access
100 + unit tests for config cache clearing and machine-access data
101 - jenkins:
102 + add basic Jenkinsfile for CI runs per PR
103 + add jenkins parseable test results
104 + add lxc cleanup stage on Jenkinsfile
105
106 -- Lucas Moura <lucas.moura@canonical.com> Thu, 14 Jan 2021 10:08:20 -0300
107
108ubuntu-advantage-tools (25.0~20.10.1) groovy; urgency=medium
109
110 * Release version 25.0
111
112 -- Chad Smith <chad.smith@canonical.com> Fri, 04 Dec 2020 13:32:16 -0700
113
114ubuntu-advantage-tools (25.0~20.10.1beta3) groovy; urgency=medium
115
116 * New upstream release 25.0~beta3:
117 - upgrade-lts-conract: noop during do-release-upgrade on unattached
118 (GH: #1255)
119 - ua-auto-attach: order systemd unit before cloud-config.service
120 - Update FIPSUpdates pin origin
121 - fips: unmark held fips packages for ubuntu pro fips image support
122 (GH: #1109)
123 - repo: handle changes to additionalPackages contract deltas
124 - repo: move package installation to install_packages method
125 - pro: trigger auto-attach as soon as instance-data.json is available
126 (GH: #1234)
127 - Conditionally install packages when enabling FIPS
128 - fips: allow disable (GH: #1168)
129 - cli: add trailing newline to argparse errors (GH: #1236)
130 - Install fips metapacking when enabling service
131 - integration test improvements:
132 + upgrade-test: fix upgrade path restart failures on trusty (GH: #1257)
133 + Fix integration test setup scripts (GH: #1253)
134 + strict checking for command success on behave
135 + Update tests to use new pycloudlib LXD abstraction
136 + Add upgrade scenario tests when FIPS is enabled
137 + Improve FIPS tests for checking packages
138 + Update esm-infra xenial lxd test
139 + Fix vm tests as esm-apps is beta service
140 + Fix azure generic integration testing
141 + Update esm-apps check on staging_commands tests
142 + Install pycloudlib for azure jobs only
143 + Fix shell condition in run_azure_travis_integration_tests.sh
144 + Update azure jobs on travis
145 + Update travis url in README
146 + Update travis scripts to use ppa only on master
147 + Fix cron event type check on travis yaml
148
149 -- Chad Smith <chad.smith@canonical.com> Wed, 02 Dec 2020 13:43:16 -0700
150
151ubuntu-advantage-tools (25.0~20.10.1~beta2) groovy; urgency=medium
152
153 * New upstream release 25.0~beta2:
154 - help: update esm-infra help text (GH: #1212)
155 - apt-hook: update apt cli messaging for UA Infra: ESM and UA Apps: ESM
156 product names
157 - help: update fips help docs (GH: #1213)
158 - help: revert CIS help doc URL (GH: #1211)
159 - help: add new fips help URLs to CLI help docs (GH: #1210)
160 - Show error when enabling service with invalid repo [Lucas Moura]
161 (GH: #954)
162 - Update beta info for services (#1220) [Lucas Moura] (GH: #1216)
163 - Do not enable fips when fips-updates is active [Lucas Moura] (GH: #1209)
164 - Add vm test commands in tox.ini (#1204) [Lucas Moura]
165
166 -- Chad Smith <chad.smith@canonical.com> Mon, 26 Oct 2020 20:01:21 -0600
167
168ubuntu-advantage-tools (25.0~20.10.1~beta1) groovy; urgency=medium
169
170 * Beta bug fix release
171 - status: fix missing description_override key after upgrade from
172 trusty (GH: #1201)
173 - During contract delta processing use _check_application_status_on_cache
174 instead of live service status
175
176 -- Chad Smith <chad.smith@canonical.com> Sat, 10 Oct 2020 21:47:21 -0600
177
178ubuntu-advantage-tools (25.0~20.10.1~beta) groovy; urgency=medium
179
180 * d/control:
181 - add po-debconf dependency and fix lintian not-using-po-debconf and
182 untranslatable-debconf-templates
183 - add ${misc:Depends} dep to ubuntu-advantage-pro to fix lintian
184 debhelper-but-no-misc-depends (GH: #1024)
185 * d/rules:
186 - drop --with systemd fix build-depends-on-obsolete-package
187 - set fix lintian warning extra:Depends even if empty
188 * d/postrm
189 - Add more gpg keys to be deleted in postrm for Xenial+ support
190 * d/postinst:
191 - do not unconfigure non-trusty esm. no series in apt filenames (GH: #1170)
192 - check if esm is already enabled (GH: #1095)
193 * New upstream release 25.0:
194 - Do not uninstall additionalPackages or livepatch when disabling services
195 - check for issubclass on clean_apt_files
196 - Add do-release-upgrade support for esm-infra and apps suites (GH: #1169)
197 - Apply contract deltas during do-release-upgrade operations
198 - cli: add ua help command
199 - cli: status add blocking --wait param and lock files for config change
200 - Fix livepatch behaviour on aws pro focal machine
201 - travis: drop inapplicable workspaces from specific awsgeneric release
202 jobs
203 - Add possible reboot text after enabling/disabling services
204 - apt-hook: package apt-hook and apt configuration files on all releases
205 (GH: #1150)
206 - Fix enable fail bug
207 - Add uaclient.conf override mechanism for auto-attach, beta services and
208 machine-token
209 - Support ESM Apps [Brian Murray] (GH: #930)
210 - Do not enable services if blocking services is active (GH: #1029)
211 - contract: handle 401 on invalid token, 403 on expired (GH: #1335)
212 - Hide beta services from default status output and enable/disable
213 operations (GH: #1079) (GH: #1091)
214 - fips: force apt noninteractive prompts during package installs
215 (GH: #1084)
216 - tests: add unit tests for aws-gov/aws-china cloud detection
217 - Add AWS China and GovCloud partitions [Robert Jennings]
218 - Disable beta services to be show/enabled without flag
219 - Add missing build_pr command to environment
220 - Use additionalPackages from service payload
221 - Add integration testing for Travis runs [patriciadomin] (GH: #856)
222 (GH: #857) (GH: #853)
223
224 -- Chad Smith <chad.smith@canonical.com> Mon, 28 Sep 2020 21:11:54 -0600
225
1ubuntu-advantage-tools (24.4) groovy; urgency=medium226ubuntu-advantage-tools (24.4) groovy; urgency=medium
2227
3 * New bug-fix-only release 24.4:228 * New bug-fix-only release 24.4:
diff --git a/debian/control b/debian/control
index 0fedb9c..9c3095e 100644
--- a/debian/control
+++ b/debian/control
@@ -5,16 +5,17 @@ Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
5Build-Depends: bash-completion,5Build-Depends: bash-completion,
6 debhelper (>=9),6 debhelper (>=9),
7 dh-python,7 dh-python,
8 dh-systemd,
8 gettext,9 gettext,
9 git,10 git,
10 libapt-pkg-dev,11 libapt-pkg-dev,
12 po-debconf,
11 python3 (>= 3.4),13 python3 (>= 3.4),
12 python3-flake8,14 python3-flake8,
13 python3-mock,15 python3-mock,
14 python3-pytest,16 python3-pytest,
15 python3-setuptools,17 python3-setuptools,
16 python3-yaml,18 python3-yaml
17 dh-systemd
18Standards-Version: 4.3.019Standards-Version: 4.3.0
19Homepage: https://buy.ubuntu.com20Homepage: https://buy.ubuntu.com
20Vcs-Git: https://github.com/CanonicalLtd/ubuntu-advantage-script.git21Vcs-Git: https://github.com/CanonicalLtd/ubuntu-advantage-script.git
@@ -37,7 +38,7 @@ Description: management tools for Ubuntu Advantage
3738
38Package: ubuntu-advantage-pro39Package: ubuntu-advantage-pro
39Architecture: any40Architecture: any
40Depends: ubuntu-advantage-tools (>=20.2)41Depends: ${misc:Depends}, ubuntu-advantage-tools (>=20.2)
41Replaces: ubuntu-advantage-tools (<<20.2)42Replaces: ubuntu-advantage-tools (<<20.2)
42Breaks: ubuntu-advantage-tools (<<20.2)43Breaks: ubuntu-advantage-tools (<<20.2)
43Description: utilities and services for Ubuntu Pro images44Description: utilities and services for Ubuntu Pro images
diff --git a/debian/po/POTFILES.in b/debian/po/POTFILES.in
44new file mode 10064445new file mode 100644
index 0000000..986f7cc
--- /dev/null
+++ b/debian/po/POTFILES.in
@@ -0,0 +1 @@
1[type: gettext/rfc822deb] ubuntu-advantage-tools.templates
diff --git a/debian/po/templates.pot b/debian/po/templates.pot
0new file mode 1006442new file mode 100644
index 0000000..9dd5145
--- /dev/null
+++ b/debian/po/templates.pot
@@ -0,0 +1,36 @@
1# SOME DESCRIPTIVE TITLE.
2# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
3# This file is distributed under the same license as the ubuntu-advantage-tools package.
4# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
5#
6#, fuzzy
7msgid ""
8msgstr ""
9"Project-Id-Version: ubuntu-advantage-tools\n"
10"Report-Msgid-Bugs-To: ubuntu-advantage-tools@packages.debian.org\n"
11"POT-Creation-Date: 2020-10-02 15:07-0600\n"
12"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
13"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
14"Language-Team: LANGUAGE <LL@li.org>\n"
15"Language: \n"
16"MIME-Version: 1.0\n"
17"Content-Type: text/plain; charset=CHARSET\n"
18"Content-Transfer-Encoding: 8bit\n"
19
20#. Type: note
21#. Description
22#: ../ubuntu-advantage-tools.templates:1001
23msgid "Ubuntu Pro support now requires ubuntu-advantage-pro"
24msgstr ""
25
26#. Type: note
27#. Description
28#: ../ubuntu-advantage-tools.templates:1001
29msgid "To install run the following:"
30msgstr ""
31
32#. Type: note
33#. Description
34#: ../ubuntu-advantage-tools.templates:1001
35msgid "sudo apt-get install ubuntu-advantage-pro"
36msgstr ""
diff --git a/debian/postinst b/debian/postinst
index 3b39c9d..5d942f9 100644
--- a/debian/postinst
+++ b/debian/postinst
@@ -11,12 +11,15 @@ UA_KEYRING_DIR="/usr/share/keyrings/"
11ESM_INFRA_KEY_TRUSTY="ubuntu-advantage-esm-infra-trusty.gpg"11ESM_INFRA_KEY_TRUSTY="ubuntu-advantage-esm-infra-trusty.gpg"
1212
13APT_SRC_DIR="/etc/apt/sources.list.d"13APT_SRC_DIR="/etc/apt/sources.list.d"
14APT_PREFERENCES_DIR="/etc/apt/preferences.d"
14ESM_APT_SOURCE_FILE_PRECISE="$APT_SRC_DIR/ubuntu-esm-precise.list"15ESM_APT_SOURCE_FILE_PRECISE="$APT_SRC_DIR/ubuntu-esm-precise.list"
15ESM_APT_SOURCE_FILE_TRUSTY="$APT_SRC_DIR/ubuntu-esm-trusty.list"16ESM_APT_SOURCE_FILE_TRUSTY="$APT_SRC_DIR/ubuntu-esm-trusty.list"
16ESM_INFRA_APT_SOURCE_FILE_TRUSTY="$APT_SRC_DIR/ubuntu-esm-infra-trusty.list"17ESM_INFRA_OLD_APT_SOURCE_FILE_TRUSTY="$APT_SRC_DIR/ubuntu-esm-infra-trusty.list"
18ESM_INFRA_APT_SOURCE_FILE_TRUSTY="$APT_SRC_DIR/ubuntu-esm-infra.list"
1719
18ESM_APT_PREF_FILE_TRUSTY="/etc/apt/preferences.d/ubuntu-esm-trusty"20ESM_APT_PREF_FILE_TRUSTY="/etc/apt/preferences.d/ubuntu-esm-trusty"
19ESM_INFRA_APT_PREF_FILE_TRUSTY="/etc/apt/preferences.d/ubuntu-esm-infra-trusty"21ESM_INFRA_OLD_APT_PREF_FILE_TRUSTY="/etc/apt/preferences.d/ubuntu-esm-infra-trusty"
22ESM_INFRA_APT_PREF_FILE_TRUSTY="/etc/apt/preferences.d/ubuntu-esm-infra"
2023
21MYARCH="$(dpkg --print-architecture)"24MYARCH="$(dpkg --print-architecture)"
22ESM_SUPPORTED_ARCHS="i386 amd64"25ESM_SUPPORTED_ARCHS="i386 amd64"
@@ -25,14 +28,80 @@ SYSTEMD_WANTS_AUTO_ATTACH_LINK="/etc/systemd/system/multi-user.target.wants/ua-a
25SYSTEMD_HELPER_ENABLED_AUTO_ATTACH_DSH="/var/lib/systemd/deb-systemd-helper-enabled/ua-auto-attach.service.dsh-also"28SYSTEMD_HELPER_ENABLED_AUTO_ATTACH_DSH="/var/lib/systemd/deb-systemd-helper-enabled/ua-auto-attach.service.dsh-also"
26SYSTEMD_HELPER_ENABLED_WANTS_LINK="/var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/ua-auto-attach.service"29SYSTEMD_HELPER_ENABLED_WANTS_LINK="/var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/ua-auto-attach.service"
2730
31REBOOT_CMD_MARKER_FILE="/var/lib/ubuntu-advantage/marker-reboot-cmds-required"
32
33# Rename apt config files for ua services removing ubuntu release names
34redact_ubuntu_release_from_ua_apt_filenames() {
35 DIR=$1
36 UA_SERVICES=$(python3 -c "
37from uaclient.entitlements import ENTITLEMENT_CLASS_BY_NAME
38print(*ENTITLEMENT_CLASS_BY_NAME.keys(), sep=' ')
39")
40
41 for file in "$DIR"/*; do
42 release_name=""
43 case "$file" in
44 *-trusty*)
45 release_name=trusty;;
46 *-xenial*)
47 release_name=xenial;;
48 *-bionic*)
49 release_name=bionic;;
50 *-focal*)
51 release_name=focal;;
52 *-groovy*)
53 release_name=groovy;;
54 *) release_name="";;
55 esac
56 if [ "$release_name" ]; then
57 # We have a ubuntu release name in the apt config.
58 # Remove $release_name from original $file.
59 new_file=${file%-${release_name}*}${file#*${release_name}}
60 for service in ${UA_SERVICES}; do
61 if [ "${file#*$service}" != "$file" ]; then
62 # Valid apt cfg file for an ubuntu-advantage service
63 mv "$file" "$new_file"
64 fi
65 done
66 fi
67 done
68}
69
70
71# Check cached service status from status.json and return 0 if enabled else 1
72check_service_is_enabled() {
73 service_name=$1
74 _RET=$(python3 -c "
75import os
76import json
77from uaclient.config import UAConfig
78cfg = UAConfig()
79status = cfg.read_cache('status-cache')
80if status:
81 for service in status['services']:
82 if service['name'] == '${service_name}':
83 print(service['status'])
84")
85 if [ "${_RET}" = "enabled" ]; then
86 return 0
87 else
88 return 1
89 fi
90}
91
2892
29unconfigure_esm() {93unconfigure_esm() {
30 rm -f $APT_TRUSTED_KEY_DIR/ubuntu-esm*gpg # Remove previous esm keys94 if ! check_service_is_enabled esm-infra; then
31 rm -f $APT_TRUSTED_KEY_DIR/$ESM_INFRA_KEY_TRUSTY95 rm -f $APT_TRUSTED_KEY_DIR/ubuntu-esm*gpg # Remove previous esm keys
32 rm -f $ESM_INFRA_APT_SOURCE_FILE_TRUSTY96 rm -f $APT_TRUSTED_KEY_DIR/$ESM_INFRA_KEY_TRUSTY
33 rm -f $ESM_APT_PREF_FILE_TRUSTY $ESM_INFRA_APT_PREF_FILE_TRUSTY97 rm -f $ESM_INFRA_APT_SOURCE_FILE_TRUSTY
98 rm -f $ESM_INFRA_OLD_APT_SOURCE_FILE_TRUSTY
99 rm -f $ESM_APT_PREF_FILE_TRUSTY $ESM_INFRA_OLD_APT_PREF_FILE_TRUSTY
100 rm -f $ESM_INFRA_APT_PREF_FILE_TRUSTY
101 fi
34}102}
35103
104
36configure_esm() {105configure_esm() {
37 rm -f $APT_TRUSTED_KEY_DIR/ubuntu-esm*gpg # Remove previous esm keys106 rm -f $APT_TRUSTED_KEY_DIR/ubuntu-esm*gpg # Remove previous esm keys
38 if [ ! -f "$APT_TRUSTED_KEY_DIR/$ESM_INFRA_KEY_TRUSTY" ]; then107 if [ ! -f "$APT_TRUSTED_KEY_DIR/$ESM_INFRA_KEY_TRUSTY" ]; then
@@ -64,6 +133,29 @@ EOF
64 fi133 fi
65}134}
66135
136
137# If held fips packages exist, we are on a FIPS PRO machine with FIPS enabled
138mark_reboot_for_fips_pro() {
139 FIPS_HOLDS=$(apt-mark showholds | grep -E 'fips|libssl1|openssh-client|openssh-server|linux-fips|openssl|strongswan' || exit 0)
140 if [ "$FIPS_HOLDS" ]; then
141 mark_reboot_cmds_as_needed MESSAGE_FIPS_REBOOT_REQUIRED
142 fi
143}
144
145
146mark_reboot_cmds_as_needed() {
147 msg_name=$1
148 if [ ! -f "$REBOOT_CMD_MARKER_FILE" ]; then
149 touch $REBOOT_CMD_MARKER_FILE
150 fi
151 python3 -c "
152from uaclient.config import UAConfig
153from uaclient.status import ${msg_name}
154cfg = UAConfig()
155cfg.add_notice(label='', description=${msg_name})
156"
157}
158
67case "$1" in159case "$1" in
68 configure)160 configure)
69 PREVIOUS_PKG_VER=$2161 PREVIOUS_PKG_VER=$2
@@ -94,10 +186,15 @@ case "$1" in
94 # ubuntu-advantage-pro package that should be installed186 # ubuntu-advantage-pro package that should be installed
95 . /usr/share/debconf/confmodule187 . /usr/share/debconf/confmodule
96 db_input high ubuntu-advantage-tools/suggest_pro_pkg || true188 db_input high ubuntu-advantage-tools/suggest_pro_pkg || true
97 db_go189 db_go || true
98 fi190 fi
99 fi191 fi
100192
193 # UA service PPAs support all ubuntu releases, no need to
194 # specialize apt config filenames per ubuntu release.
195 redact_ubuntu_release_from_ua_apt_filenames $APT_SRC_DIR
196 redact_ubuntu_release_from_ua_apt_filenames $APT_PREFERENCES_DIR
197
101 # CACHE_DIR is no longer present or used since 19.1198 # CACHE_DIR is no longer present or used since 19.1
102 rm -rf /var/cache/ubuntu-advantage-tools199 rm -rf /var/cache/ubuntu-advantage-tools
103 # machine-access cache files no longer present or used since 20.1200 # machine-access cache files no longer present or used since 20.1
@@ -111,10 +208,8 @@ case "$1" in
111 # 14.04 and unsupported arch208 # 14.04 and unsupported arch
112 unconfigure_esm209 unconfigure_esm
113 fi210 fi
114 else
115 # not 14.04, regardless of arch
116 unconfigure_esm
117 fi211 fi
212
118 if [ ! -f /var/log/ubuntu-advantage.log ]; then213 if [ ! -f /var/log/ubuntu-advantage.log ]; then
119 touch /var/log/ubuntu-advantage.log214 touch /var/log/ubuntu-advantage.log
120 fi215 fi
@@ -124,6 +219,13 @@ case "$1" in
124 if [ -d "$private_dir" ]; then219 if [ -d "$private_dir" ]; then
125 chmod 0700 "$private_dir"220 chmod 0700 "$private_dir"
126 fi221 fi
222
223 if [ "$VERSION_ID" = "16.04" ]; then
224 if echo "$PREVIOUS_PKG_VER" | grep -q "14.04"; then
225 mark_reboot_cmds_as_needed MESSAGE_LIVEPATCH_LTS_REBOOT_REQUIRED
226 fi
227 fi
228 mark_reboot_for_fips_pro
127 ;;229 ;;
128esac230esac
129231
diff --git a/debian/postrm b/debian/postrm
index 2112cf5..fb84664 100644
--- a/debian/postrm
+++ b/debian/postrm
@@ -16,11 +16,10 @@ remove_logs(){
16 rm -f /var/log/ubuntu-advantage.log*16 rm -f /var/log/ubuntu-advantage.log*
17}17}
1818
19remove_esm(){19remove_gpg_files(){
20 # config files created in postinst, need explicit handling on purge
21 rm -f /etc/apt/trusted.gpg.d/ubuntu-advantage-esm-infra-trusty.gpg20 rm -f /etc/apt/trusted.gpg.d/ubuntu-advantage-esm-infra-trusty.gpg
22 rm -f /etc/apt/sources.list.d/ubuntu-esm-infra-trusty.list21 rm -f /etc/apt/trusted.gpg.d/ubuntu-advantage-esm-apps.gpg
23 rm -f /etc/apt/preferences.d/ubuntu-esm-infra-trusty22 rm -f /etc/apt/trusted.gpg.d/ubuntu-advantage-fips.gpg
24}23}
2524
26case "$1" in25case "$1" in
@@ -28,7 +27,7 @@ case "$1" in
28 remove_apt_auth27 remove_apt_auth
29 remove_cache_dir28 remove_cache_dir
30 remove_logs29 remove_logs
31 remove_esm30 remove_gpg_files
32 ;;31 ;;
33esac32esac
3433
diff --git a/debian/prerm b/debian/prerm
index 4644c05..59b2b1b 100644
--- a/debian/prerm
+++ b/debian/prerm
@@ -5,9 +5,9 @@ set -e
55
6remove_apt_files() {6remove_apt_files() {
7 python3 -c '7 python3 -c '
8from uaclient.apt import clean_apt_sources8from uaclient.apt import clean_apt_files
99
10clean_apt_sources()10clean_apt_files()
11'11'
1212
13}13}
diff --git a/debian/rules b/debian/rules
index a535bbb..888dcde 100755
--- a/debian/rules
+++ b/debian/rules
@@ -39,26 +39,24 @@ endif
39endif39endif
4040
41override_dh_gencontrol:41override_dh_gencontrol:
42 [ -z '$(APT_PKG_DEPS)' ] || echo extra:Depends=$(APT_PKG_DEPS) >> debian/ubuntu-advantage-tools.substvars42 echo extra:Depends=$(APT_PKG_DEPS) >> debian/ubuntu-advantage-tools.substvars
43 dh_gencontrol43 dh_gencontrol
4444
45override_dh_auto_install:45override_dh_auto_install:
46 dh_auto_install --destdir=debian/ubuntu-advantage-tools46 dh_auto_install --destdir=debian/ubuntu-advantage-tools
47 flist=$$(find $(CURDIR)/debian/ -type f -name version.py) && sed -i 's,@@PACKAGED_VERSION@@,$(DEB_VERSION),' $${flist:-did-not-find-version-py-for-replacement}47 flist=$$(find $(CURDIR)/debian/ -type f -name version.py) && sed -i 's,@@PACKAGED_VERSION@@,$(DEB_VERSION),' $${flist:-did-not-find-version-py-for-replacement}
48ifeq (${VERSION_ID},"14.04")
49 make -C apt-hook DESTDIR=$(CURDIR)/debian/ubuntu-advantage-tools install48 make -C apt-hook DESTDIR=$(CURDIR)/debian/ubuntu-advantage-tools install
50endif
5149
52ifeq (${VERSION_ID},"14.04")50ifeq (${VERSION_ID},"14.04")
53 # Move ua-auto-attach.conf to ubuntu-advantage-pro51 # Move ua-auto-attach.conf out to ubuntu-advantage-pro
54 mkdir -p debian/ubuntu-advantage-pro/etc/init52 mkdir -p debian/ubuntu-advantage-pro/etc/init
55 mv debian/ubuntu-advantage-tools/etc/init/ua-auto-attach.conf debian/ubuntu-advantage-pro/etc/init/53 mv debian/ubuntu-advantage-tools/etc/init/ua-auto-attach.conf debian/ubuntu-advantage-pro/etc/init/
56 rmdir debian/ubuntu-advantage-tools/etc/init54 rmdir debian/ubuntu-advantage-tools/etc/init
57else55else
58 # Move ua-auto-attach.service to ubuntu-advantage-pro56 # Move ua-auto-attach.service out to ubuntu-advantage-pro
59 mkdir -p debian/ubuntu-advantage-pro/lib/systemd/system57 mkdir -p debian/ubuntu-advantage-pro/lib/systemd/system
60 mv debian/ubuntu-advantage-tools/lib/systemd/system/ua-auto-attach.service debian/ubuntu-advantage-pro/lib/systemd/system58 mv debian/ubuntu-advantage-tools/lib/systemd/system/ua-auto-attach.* debian/ubuntu-advantage-pro/lib/systemd/system
61 cd debian/ubuntu-advantage-tools && rmdir -p lib/systemd/system59 cd debian/ubuntu-advantage-tools
62endif60endif
6361
64override_dh_auto_clean:62override_dh_auto_clean:
diff --git a/debian/ubuntu-advantage-tools.templates b/debian/ubuntu-advantage-tools.templates
index 255d330..1b5c6f8 100644
--- a/debian/ubuntu-advantage-tools.templates
+++ b/debian/ubuntu-advantage-tools.templates
@@ -1,6 +1,6 @@
1Template: ubuntu-advantage-tools/suggest_pro_pkg1Template: ubuntu-advantage-tools/suggest_pro_pkg
2Type: note2Type: note
3Description: Ubuntu Pro support now requires ubuntu-advantage-pro3_Description: Ubuntu Pro support now requires ubuntu-advantage-pro
4 To install run the following:4 To install run the following:
5 .5 .
6 sudo apt-get install ubuntu-advantage-pro6 sudo apt-get install ubuntu-advantage-pro
diff --git a/features/attach_invalidtoken.feature b/features/attach_invalidtoken.feature
index 11a8ba8..4f3fc9a 100644
--- a/features/attach_invalidtoken.feature
+++ b/features/attach_invalidtoken.feature
@@ -1,15 +1,22 @@
1Feature: Command behaviour when trying to attach a machine to an Ubuntu1Feature: Command behaviour when trying to attach a machine to an Ubuntu
2 Advantage subscription using an invalid token2 Advantage subscription using an invalid token
33
4 Scenario: Attach command in a trusty lxd container4 @series.all
5 Given a trusty lxd container with ubuntu-advantage-tools installed5 Scenario Outline: Attach command in a machine
6 When I run `ua attach INVALID_TOKEN` with sudo6 Given a `<release>` machine with ubuntu-advantage-tools installed
7 Then I will see the following on stderr:7 When I verify that running `ua attach INVALID_TOKEN` `with sudo` exits `1`
8 Then stderr matches regexp:
8 """9 """
9 Invalid token. See https://ubuntu.com/advantage10 Invalid token. See https://ubuntu.com/advantage
10 """11 """
11 When I run `ua attach INVALID_TOKEN` as non-root12 When I verify that running `ua attach INVALID_TOKEN` `as non-root` exits `1`
12 Then I will see the following on stderr:13 Then I will see the following on stderr:
13 """14 """
14 This command must be run as root (try using sudo)15 This command must be run as root (try using sudo)
15 """16 """
17 Examples: ubuntu release
18 | release |
19 | trusty |
20 | xenial |
21 | bionic |
22 | focal |
diff --git a/features/attach_validtoken.feature b/features/attach_validtoken.feature
index 2a7bfc9..a25234f 100644
--- a/features/attach_validtoken.feature
+++ b/features/attach_validtoken.feature
@@ -1,10 +1,38 @@
1@uses.config.contract_token
1Feature: Command behaviour when attaching a machine to an Ubuntu Advantage2Feature: Command behaviour when attaching a machine to an Ubuntu Advantage
2 subscription using a valid token3 subscription using a valid token
34
4 @uses.config.contract_token5 @series.all
5 Scenario: Attach command in a trusty lxd container6 @uses.config.machine_type.lxd.container
6 Given a trusty lxd container with ubuntu-advantage-tools installed7 Scenario Outline: Attach command in a ubuntu lxd container
7 When I attach contract_token with sudo8 Given a `<release>` machine with ubuntu-advantage-tools installed
9 When I run `apt-get update` with sudo, retrying exit [100]
10 And I run `apt-get install -y <downrev_pkg>` with sudo, retrying exit [100]
11 When I verify that running ` --assume-yes --beta` `with sudo` exits `1`
12 And I run `/usr/lib/update-notifier/apt-check --human-readable` as non-root
13 Then if `<release>` in `trusty` and stdout matches regexp:
14 """
15 UA Infrastructure Extended Security Maintenance \(ESM\) is not enabled.
16
17 \d+ update(s)? can be installed immediately.
18 \d+ of these updates (is a|are) security update(s)?.
19 """
20 Then if `<release>` in `trusty` and stdout matches regexp:
21 """
22 Enable UA Infrastructure ESM to receive \d+ additional security update(s)?.
23 See https://ubuntu.com/advantage or run: sudo ua status
24 """
25 Then if `<release>` in `xenial or bionic` and stdout matches regexp:
26 """
27 \d+ package(s)? can be updated.
28 \d+ of these updates (is a|are) security update(s)?.
29 """
30 Then if `<release>` in `focal` and stdout matches regexp:
31 """
32 \d+ update(s)? can be installed immediately.
33 \d+ of these updates (is a|are) security update(s)?.
34 """
35 When I attach `contract_token` with sudo
8 Then stdout matches regexp:36 Then stdout matches regexp:
9 """37 """
10 ESM Infra enabled38 ESM Infra enabled
@@ -16,15 +44,332 @@ Feature: Command behaviour when attaching a machine to an Ubuntu Advantage
16 And stdout matches regexp:44 And stdout matches regexp:
17 """45 """
18 SERVICE ENTITLED STATUS DESCRIPTION46 SERVICE ENTITLED STATUS DESCRIPTION
19 cc-eal +yes +n/a +Common Criteria EAL2 Provisioning Packages47 esm-infra +yes +enabled +UA Infra: Extended Security Maintenance \(ESM\)
20 cis-audit +no +— +Center for Internet Security Audit Tools
21 esm-apps +no +— +UA Apps: Extended Security Maintenance
22 esm-infra +yes +enabled +UA Infra: Extended Security Maintenance
23 fips +yes +n/a +NIST-certified FIPS modules48 fips +yes +n/a +NIST-certified FIPS modules
24 fips-updates +yes +n/a +Uncertified security updates to FIPS modules49 fips-updates +yes +n/a +Uncertified security updates to FIPS modules
25 livepatch +yes +n/a +Canonical Livepatch service50 livepatch +yes +n/a +Canonical Livepatch service
26 """51 """
27 And I will see the following on stderr:52 And stderr matches regexp:
53 """
54 Enabling default service esm-infra
55 """
56 When I run `/usr/lib/update-notifier/apt-check --human-readable` as non-root
57 Then if `<release>` in `trusty` and stdout matches regexp:
58 """
59 UA (Infra:|Infrastructure) Extended Security Maintenance \(ESM\) is enabled.
60
61 \d+ update(s)? can be installed immediately.
62 \d+ of these updates (is|are) (fixed|provided) through UA (Infra:|Infrastructure) ESM.
63 \d+ of these updates (is a|are) security update(s)?.
64 """
65 Then if `<release>` in `focal` and stdout matches regexp:
66 """
67 UA (Infra:|Infrastructure) Extended Security Maintenance \(ESM\) is enabled.
68
69 \d+ update(s)? can be installed immediately.
70 \d+ of these updates (is|are) (fixed|provided) through UA (Infra:|Infrastructure) ESM.
71 \d+ of these updates (is a|are) security update(s)?.
72 To see these additional updates run: apt list --upgradable
73 """
74 Then if `<release>` in `xenial or bionic` and stdout matches regexp:
75 """
76 \d+ package(s)? can be updated.
77 \d+ of these updates (is a|are) security update(s)?.
78 """
79 Examples: ubuntu release packages
80 | release | downrev_pkg |
81 | trusty | libgit2-0=0.19.0-2 |
82 | xenial | libkrad0=1.13.2+dfsg-5 |
83 | bionic | libkrad0=1.16-2build1 |
84 | focal | hello=2.10-2ubuntu2 |
85
86 @series.all
87 @uses.config.machine_type.aws.generic
88 Scenario Outline: Attach command in a ubuntu lxd container
89 Given a `<release>` machine with ubuntu-advantage-tools installed
90 When I attach `contract_token` with sudo
91 Then stdout matches regexp:
92 """
93 ESM Infra enabled
94 """
95 And stdout matches regexp:
96 """
97 This machine is now attached to
98 """
99 And stdout matches regexp:
100 """
101 SERVICE ENTITLED STATUS DESCRIPTION
102 esm-infra +yes +enabled +UA Infra: Extended Security Maintenance \(ESM\)
103 fips +yes +<fips_status> +NIST-certified FIPS modules
104 fips-updates +yes +<fips_status> +Uncertified security updates to FIPS modules
105 livepatch +yes +<lp_status> +<lp_desc>
106 """
107 And stderr matches regexp:
108 """
109 Enabling default service esm-infra
110 """
111
112 Examples: ubuntu release livepatch status
113 | release | fips_status |lp_status | lp_desc |
114 | trusty | n/a |n/a | Available with the HWE kernel |
115 | xenial | disabled |enabled | Canonical Livepatch service |
116 | bionic | disabled |enabled | Canonical Livepatch service |
117 | focal | n/a |enabled | Canonical Livepatch service |
118
119 @series.all
120 @uses.config.machine_type.azure.generic
121 Scenario Outline: Attach command in a ubuntu lxd container
122 Given a `<release>` machine with ubuntu-advantage-tools installed
123 When I attach `contract_token` with sudo
124 Then stdout matches regexp:
125 """
126 ESM Infra enabled
127 """
128 And stdout matches regexp:
129 """
130 This machine is now attached to
131 """
132 And stdout matches regexp:
133 """
134 SERVICE ENTITLED STATUS DESCRIPTION
135 esm-infra +yes +enabled +UA Infra: Extended Security Maintenance \(ESM\)
136 fips +yes +<fips_status> +NIST-certified FIPS modules
137 fips-updates +yes +<fips_status> +Uncertified security updates to FIPS modules
138 livepatch +yes +<lp_status> +Canonical Livepatch service
139 """
140 And stderr matches regexp:
141 """
142 Enabling default service esm-infra
143 """
144
145 Examples: ubuntu release livepatch status
146 | release | lp_status | fips_status |
147 | trusty | disabled | n/a |
148 | xenial | n/a | n/a |
149 | bionic | n/a | disabled |
150 | focal | n/a | n/a |
151
152 @series.all
153 @uses.config.machine_type.gcp.generic
154 Scenario Outline: Attach command in a ubuntu lxd container
155 Given a `<release>` machine with ubuntu-advantage-tools installed
156 When I attach `contract_token` with sudo
157 Then stdout matches regexp:
158 """
159 ESM Infra enabled
160 """
161 And stdout matches regexp:
162 """
163 This machine is now attached to
164 """
165 And stdout matches regexp:
166 """
167 SERVICE ENTITLED STATUS DESCRIPTION
168 esm-infra +yes +enabled +UA Infra: Extended Security Maintenance \(ESM\)
169 fips +yes +<fips_status> +NIST-certified FIPS modules
170 fips-updates +yes +<fips_status> +Uncertified security updates to FIPS modules
171 livepatch +yes +<lp_status> +Canonical Livepatch service
172 """
173 And stderr matches regexp:
28 """174 """
29 Enabling default service esm-infra175 Enabling default service esm-infra
30 """176 """
177
178 Examples: ubuntu release livepatch status
179 | release | lp_status | fips_status |
180 | trusty | disabled | n/a |
181 | xenial | n/a | n/a |
182 | bionic | n/a | n/a |
183 | focal | n/a | n/a |
184
185 @series.bionic
186 @uses.config.machine_type.azure.generic
187 Scenario Outline: Attached enable of vm-based services in an ubuntu lxd vm
188 Given a `<release>` machine with ubuntu-advantage-tools installed
189 When I attach `contract_token_staging` with sudo
190 And I run `apt-get install openssh-client openssh-server strongswan -y` with sudo
191 And I run `apt-mark hold openssh-client openssh-server strongswan` with sudo
192 And I run `ua enable <fips-service> --assume-yes` with sudo
193 Then stdout matches regexp:
194 """
195 Updating package lists
196 Installing <fips-name> packages
197 <fips-name> enabled
198 A reboot is required to complete install
199 """
200 When I run `ua status --all` with sudo
201 Then stdout matches regexp:
202 """
203 <fips-service> +yes enabled
204 """
205 And I verify that running `apt update` `with sudo` exits `0`
206 And I verify that running `grep Traceback /var/log/ubuntu-advantage.log` `with sudo` exits `1`
207 And I verify that `openssh-server` is installed from apt source `<fips-apt-source>`
208 And I verify that `openssh-client` is installed from apt source `<fips-apt-source>`
209 And I verify that `strongswan` is installed from apt source `<fips-apt-source>`
210 And I verify that `openssh-server-hmac` is installed from apt source `<fips-apt-source>`
211 And I verify that `openssh-client-hmac` is installed from apt source `<fips-apt-source>`
212 And I verify that `strongswan-hmac` is installed from apt source `<fips-apt-source>`
213 When I run `apt-cache policy ubuntu-azure-fips` as non-root
214 Then stdout does not match regexp:
215 """
216 .*Installed: \(none\)
217 """
218 When I reboot the `<release>` machine
219 And I run `uname -r` as non-root
220 Then stdout matches regexp:
221 """
222 azure-fips
223 """
224 When I run `cat /proc/sys/crypto/fips_enabled` with sudo
225 Then I will see the following on stdout:
226 """
227 1
228 """
229 When I run `ua disable <fips-service> --assume-yes` with sudo
230 Then stdout matches regexp:
231 """
232 Updating package lists
233 """
234 When I run `apt-cache policy ubuntu-azure-fips` as non-root
235 Then stdout matches regexp:
236 """
237 .*Installed: \(none\)
238 """
239 When I reboot the `<release>` machine
240 Then I verify that `openssh-server` installed version matches regexp `fips`
241 And I verify that `openssh-client` installed version matches regexp `fips`
242 And I verify that `strongswan` installed version matches regexp `fips`
243 And I verify that `openssh-server-hmac` installed version matches regexp `fips`
244 And I verify that `openssh-client-hmac` installed version matches regexp `fips`
245 And I verify that `strongswan-hmac` installed version matches regexp `fips`
246 When I run `apt-mark unhold openssh-client openssh-server strongswan` with sudo
247 Then I will see the following on stdout:
248 """
249 openssh-client was already not hold.
250 openssh-server was already not hold.
251 strongswan was already not hold.
252 """
253 When I run `ua status --all` with sudo
254 Then stdout matches regexp:
255 """
256 <fips-service> +yes disabled
257 """
258
259 Examples: ubuntu release
260 | release | fips-name | fips-service |fips-apt-source |
261 | bionic | FIPS | fips |https://esm.staging.ubuntu.com/fips/ubuntu bionic/main |
262
263 @series.bionic
264 @uses.config.machine_type.aws.generic
265 Scenario Outline: Attached enable of vm-based services in an ubuntu lxd vm
266 Given a `<release>` machine with ubuntu-advantage-tools installed
267 When I attach `contract_token_staging` with sudo
268 And I run `ua disable livepatch` with sudo
269 And I run `apt-get install openssh-client openssh-server strongswan -y` with sudo
270 And I run `apt-mark hold openssh-client openssh-server strongswan` with sudo
271 And I run `ua enable <fips-service> --assume-yes` with sudo
272 Then stdout matches regexp:
273 """
274 Updating package lists
275 Installing <fips-name> packages
276 <fips-name> enabled
277 A reboot is required to complete install
278 """
279 When I run `ua status --all` with sudo
280 Then stdout matches regexp:
281 """
282 <fips-service> +yes enabled
283 """
284 And I verify that running `apt update` `with sudo` exits `0`
285 And I verify that running `grep Traceback /var/log/ubuntu-advantage.log` `with sudo` exits `1`
286 And I verify that `openssh-server` is installed from apt source `<fips-apt-source>`
287 And I verify that `openssh-client` is installed from apt source `<fips-apt-source>`
288 And I verify that `strongswan` is installed from apt source `<fips-apt-source>`
289 And I verify that `openssh-server-hmac` is installed from apt source `<fips-apt-source>`
290 And I verify that `openssh-client-hmac` is installed from apt source `<fips-apt-source>`
291 And I verify that `strongswan-hmac` is installed from apt source `<fips-apt-source>`
292 When I run `apt-cache policy ubuntu-aws-fips` as non-root
293 Then stdout does not match regexp:
294 """
295 .*Installed: \(none\)
296 """
297 When I reboot the `<release>` machine
298 And I run `uname -r` as non-root
299 Then stdout matches regexp:
300 """
301 aws-fips
302 """
303 When I run `cat /proc/sys/crypto/fips_enabled` with sudo
304 Then I will see the following on stdout:
305 """
306 1
307 """
308 When I run `ua disable <fips-service> --assume-yes` with sudo
309 Then stdout matches regexp:
310 """
311 Updating package lists
312 """
313 When I run `apt-cache policy ubuntu-aws-fips` as non-root
314 Then stdout matches regexp:
315 """
316 .*Installed: \(none\)
317 """
318 When I reboot the `<release>` machine
319 Then I verify that `openssh-server` installed version matches regexp `fips`
320 And I verify that `openssh-client` installed version matches regexp `fips`
321 And I verify that `strongswan` installed version matches regexp `fips`
322 And I verify that `openssh-server-hmac` installed version matches regexp `fips`
323 And I verify that `openssh-client-hmac` installed version matches regexp `fips`
324 And I verify that `strongswan-hmac` installed version matches regexp `fips`
325 When I run `apt-mark unhold openssh-client openssh-server strongswan` with sudo
326 Then I will see the following on stdout:
327 """
328 openssh-client was already not hold.
329 openssh-server was already not hold.
330 strongswan was already not hold.
331 """
332 When I run `ua status --all` with sudo
333 Then stdout matches regexp:
334 """
335 <fips-service> +yes disabled
336 """
337
338 Examples: ubuntu release
339 | release | fips-name | fips-service |fips-apt-source |
340 | bionic | FIPS | fips |https://esm.staging.ubuntu.com/fips/ubuntu bionic/main |
341
342 @series.xenial
343 @uses.config.machine_type.azure.generic
344 Scenario Outline: Attached enable of vm-based services in an ubuntu lxd vm
345 Given a `xenial` machine with ubuntu-advantage-tools installed
346 When I attach `contract_token_staging` with sudo
347 Then I verify that running `ua enable <fips_service> --assume-yes` `with sudo` exits `1`
348 And stdout matches regexp:
349 """
350 Ubuntu Xenial does not provide an Azure optimized FIPS kernel
351 """
352
353 Examples: fips
354 | fips_service |
355 | fips |
356 | fips-updates |
357
358 @series.bionic
359 @series.xenial
360 @uses.config.machine_type.gcp.generic
361 Scenario Outline: Attached enable of fips services in an ubuntu gcp vm
362 Given a `<release>` machine with ubuntu-advantage-tools installed
363 When I attach `contract_token_staging` with sudo
364 Then I verify that running `ua enable <fips_service> --assume-yes` `with sudo` exits `1`
365 And stdout matches regexp:
366 """
367 Ubuntu <release_title> does not provide a GCP optimized FIPS kernel
368 """
369
370 Examples: fips
371 | release | release_title | fips_service |
372 | xenial | Xenial | fips |
373 | xenial | Xenial | fips-updates |
374 | bionic | Bionic | fips |
375 | bionic | Bionic | fips-updates |
diff --git a/features/attached_commands.feature b/features/attached_commands.feature
index 24dfbfe..4b48f76 100644
--- a/features/attached_commands.feature
+++ b/features/attached_commands.feature
@@ -1,13 +1,14 @@
1@uses.config.contract_token1@uses.config.contract_token
2Feature: Command behaviour when attached to an UA subscription2Feature: Command behaviour when attached to an UA subscription
33
4 Scenario: Attached refresh in a trusty lxd container4 @series.all
5 Given a trusty lxd container with ubuntu-advantage-tools installed5 Scenario Outline: Attached refresh in a ubuntu machine
6 When I attach contract_token with sudo6 Given a `<release>` machine with ubuntu-advantage-tools installed
7 And I run `ua refresh` as non-root7 When I attach `contract_token` with sudo
8 Then I will see the following on stderr:8 Then I verify that running `ua refresh` `as non-root` exits `1`
9 And stderr matches regexp:
9 """10 """
10 This command must be run as root (try using sudo)11 This command must be run as root \(try using sudo\)
11 """12 """
12 When I run `ua refresh` with sudo13 When I run `ua refresh` with sudo
13 Then I will see the following on stdout:14 Then I will see the following on stdout:
@@ -15,43 +16,55 @@ Feature: Command behaviour when attached to an UA subscription
15 Successfully refreshed your subscription16 Successfully refreshed your subscription
16 """17 """
1718
18 Scenario: Attached disable of an already disabled service in a trusty lxd container19 Examples: ubuntu release
19 Given a trusty lxd container with ubuntu-advantage-tools installed20 | release |
20 When I attach contract_token with sudo21 | bionic |
21 And I run `ua disable livepatch` as non-root22 | focal |
22 Then I will see the following on stderr:23 | trusty |
24 | xenial |
25
26 @series.all
27 Scenario Outline: Attached disable of an already disabled service in a ubuntu machine
28 Given a `<release>` machine with ubuntu-advantage-tools installed
29 When I attach `contract_token` with sudo
30 Then I verify that running `ua disable livepatch` `as non-root` exits `1`
31 And stderr matches regexp:
23 """32 """
24 This command must be run as root (try using sudo)33 This command must be run as root \(try using sudo\)
25 """34 """
26 When I run `ua disable livepatch` with sudo35 And I verify that running `ua disable livepatch` `with sudo` exits `1`
27 Then I will see the following on stdout:36 And I will see the following on stdout:
28 """37 """
29 Livepatch is not currently enabled38 Livepatch is not currently enabled
30 See: sudo ua status39 See: sudo ua status
31 """40 """
3241
33 Scenario: Attached disable of an unknown service in a trusty lxd container42 Examples: ubuntu release
34 Given a trusty lxd container with ubuntu-advantage-tools installed43 | release |
35 When I attach contract_token with sudo44 | bionic |
36 And I run `ua disable foobar` as non-root45 | focal |
37 Then I will see the following on stderr:46 | trusty |
47 | xenial |
48
49 @series.all
50 Scenario Outline: Attached disable of a service in a ubuntu machine
51 Given a `<release>` machine with ubuntu-advantage-tools installed
52 When I attach `contract_token` with sudo
53 Then I verify that running `ua disable foobar` `as non-root` exits `1`
54 And stderr matches regexp:
38 """55 """
39 This command must be run as root (try using sudo)56 This command must be run as root \(try using sudo\)
40 """57 """
41 When I run `ua disable foobar` with sudo58 And I verify that running `ua disable foobar` `with sudo` exits `1`
42 Then I will see the following on stderr:59 And stderr matches regexp:
43 """60 """
44 Cannot disable 'foobar'61 Cannot disable unknown service 'foobar'.
45 For a list of services see: sudo ua status62 Try cc-eal, cis, esm-apps, esm-infra, fips, fips-updates, livepatch
46 """63 """
4764 And I verify that running `ua disable esm-infra` `as non-root` exits `1`
48 Scenario: Attached disable of an already enabled service in a trusty lxd container65 And stderr matches regexp:
49 Given a trusty lxd container with ubuntu-advantage-tools installed
50 When I attach contract_token with sudo
51 And I run `ua disable esm-infra` as non-root
52 Then I will see the following on stderr:
53 """66 """
54 This command must be run as root (try using sudo)67 This command must be run as root \(try using sudo\)
55 """68 """
56 When I run `ua disable esm-infra` with sudo69 When I run `ua disable esm-infra` with sudo
57 Then I will see the following on stdout:70 Then I will see the following on stdout:
@@ -61,26 +74,24 @@ Feature: Command behaviour when attached to an UA subscription
61 When I run `ua status` with sudo74 When I run `ua status` with sudo
62 Then stdout matches regexp:75 Then stdout matches regexp:
63 """76 """
64 esm-infra +yes +disabled +UA Infra: Extended Security Maintenance77 esm-infra +yes +disabled +UA Infra: Extended Security Maintenance \(ESM\)
65 """
66 When I run `apt-cache policy` with sudo
67 Then stdout matches regexp:
68 """
69 -32768 https://esm.ubuntu.com/ubuntu/ trusty-infra-updates/main amd64 Packages
70 """
71 And stdout matches regexp:
72 """
73 -32768 https://esm.ubuntu.com/ubuntu/ trusty-infra-security/main amd64 Packages
74 """78 """
79 And I verify that running `apt update` `with sudo` exits `0`
7580
81 Examples: ubuntu release
82 | release |
83 | bionic |
84 | focal |
85 | xenial |
7686
77 Scenario: Attached detach in a trusty lxd container87 @series.all
78 Given a trusty lxd container with ubuntu-advantage-tools installed88 Scenario Outline: Attached detach in a trusty machine
79 When I attach contract_token with sudo89 Given a `<release>` machine with ubuntu-advantage-tools installed
80 And I run `ua detach` as non-root90 When I attach `contract_token` with sudo
81 Then I will see the following on stderr:91 Then I verify that running `ua detach` `as non-root` exits `1`
92 And stderr matches regexp:
82 """93 """
83 This command must be run as root (try using sudo)94 This command must be run as root \(try using sudo\)
84 """95 """
85 When I run `ua detach --assume-yes` with sudo96 When I run `ua detach --assume-yes` with sudo
86 Then I will see the following on stdout:97 Then I will see the following on stdout:
@@ -90,18 +101,336 @@ Feature: Command behaviour when attached to an UA subscription
90 Updating package lists101 Updating package lists
91 This machine is now detached102 This machine is now detached
92 """103 """
93 When I run `ua status` as non-root104 When I run `ua status --all` as non-root
94 Then stdout matches regexp:105 Then stdout matches regexp:
95 """106 """
96 SERVICE AVAILABLE DESCRIPTION107 SERVICE AVAILABLE DESCRIPTION
97 cc-eal +no +Common Criteria EAL2 Provisioning Packages108 cc-eal +<cc-eal> +Common Criteria EAL2 Provisioning Packages
98 esm-apps +no +UA Apps: Extended Security Maintenance109 cis +<cis> +Center for Internet Security Audit Tools
99 esm-infra +yes +UA Infra: Extended Security Maintenance110 esm-apps +<esm-apps> +UA Apps: Extended Security Maintenance \(ESM\)
100 fips +no +NIST-certified FIPS modules111 esm-infra +yes +UA Infra: Extended Security Maintenance \(ESM\)
101 fips-updates +no +Uncertified security updates to FIPS modules112 fips +<fips> +NIST-certified FIPS modules
113 fips-updates +<fips> +Uncertified security updates to FIPS modules
102 livepatch +yes +Canonical Livepatch service114 livepatch +yes +Canonical Livepatch service
103 """115 """
104 And stdout matches regexp:116 And stdout matches regexp:
105 """117 """
106 This machine is not attached to a UA subscription.118 This machine is not attached to a UA subscription.
107 """119 """
120 And I verify that running `apt update` `with sudo` exits `0`
121
122 Examples: ubuntu release
123 | release | esm-apps | cc-eal | cis | fips | fips-update |
124 | bionic | yes | no | yes | yes | yes |
125 | focal | yes | no | no | no | no |
126 | trusty | no | no | no | no | no |
127 | xenial | yes | yes | yes | yes | yes |
128
129 @series.all
130 Scenario Outline: Attached auto-attach in a ubuntu machine
131 Given a `<release>` machine with ubuntu-advantage-tools installed
132 When I attach `contract_token` with sudo
133 Then I verify that running `ua auto-attach` `as non-root` exits `1`
134 And stderr matches regexp:
135 """
136 This command must be run as root \(try using sudo\)
137 """
138 When I run `ua auto-attach` with sudo
139 Then stderr matches regexp:
140 """
141 This machine is already attached
142 """
143
144 Examples: ubuntu release
145 | release |
146 | bionic |
147 | focal |
148 | trusty |
149 | xenial |
150
151 @series.all
152 Scenario Outline: Attached show version in a ubuntu machine
153 Given a `<release>` machine with ubuntu-advantage-tools installed
154 When I attach `contract_token` with sudo
155 And I run `ua version` as non-root
156 Then I will see the uaclient version on stdout
157 When I run `ua version` with sudo
158 Then I will see the uaclient version on stdout
159 When I run `ua --version` as non-root
160 Then I will see the uaclient version on stdout
161 When I run `ua --version` with sudo
162 Then I will see the uaclient version on stdout
163
164 Examples: ubuntu release
165 | release |
166 | bionic |
167 | focal |
168 | trusty |
169 | xenial |
170
171 @series.all
172 Scenario Outline: Unattached status in a ubuntu machine with feature overrides
173 Given a `<release>` machine with ubuntu-advantage-tools installed
174 When I create the file `/tmp/machine-token-overlay.json` with the following:
175 """
176 {
177 "machineTokenInfo": {
178 "contractInfo": {
179 "resourceEntitlements": [
180 {
181 "type": "cc-eal",
182 "entitled": false
183 }
184 ]
185 }
186 }
187 }
188 """
189 And I append the following on uaclient config:
190 """
191 features:
192 machine_token_overlay: "/tmp/machine-token-overlay.json"
193 disable_auto_attach: true
194 other: false
195 """
196 And I attach `contract_token` with sudo
197 And I run `ua status --all` with sudo
198 Then stdout matches regexp:
199 """
200 SERVICE ENTITLED STATUS DESCRIPTION
201 cc-eal no
202 """
203 When I run `ua --version` as non-root
204 Then I will see the uaclient version on stdout with features ` +disable_auto_attach +machine_token_overlay -other`
205 When I run `ua version` as non-root
206 Then I will see the uaclient version on stdout with features ` +disable_auto_attach +machine_token_overlay -other`
207 When I run `ua auto-attach` with sudo
208 Then stdout matches regexp:
209 """
210 Skipping auto-attach. Config disable_auto_attach is set.
211 """
212
213 Examples: ubuntu release
214 | release |
215 | bionic |
216 | focal |
217 | trusty |
218 | xenial |
219
220 @series.all
221 Scenario Outline: Attached disable of different services in a ubuntu machine
222 Given a `<release>` machine with ubuntu-advantage-tools installed
223 When I attach `contract_token` with sudo
224 Then I verify that running `ua disable esm-infra livepatch foobar` `as non-root` exits `1`
225 And stderr matches regexp:
226 """
227 This command must be run as root \(try using sudo\)
228 """
229 And I verify that running `ua disable esm-infra livepatch foobar` `with sudo` exits `1`
230 And I will see the following on stdout:
231 """
232 Updating package lists
233 Livepatch is not currently enabled
234 See: sudo ua status
235 """
236 And stderr matches regexp:
237 """
238 Cannot disable unknown service 'foobar'.
239 Try cc-eal, cis, esm-apps, esm-infra, fips, fips-updates, livepatch
240 """
241 When I run `ua status` with sudo
242 Then stdout matches regexp:
243 """
244 esm-infra +yes +disabled +UA Infra: Extended Security Maintenance \(ESM\)
245 """
246
247 Examples: ubuntu release
248 | release |
249 | bionic |
250 | focal |
251 | trusty |
252 | xenial |
253
254 @series.trusty
255 Scenario: Attached disable of an already enabled service in a trusty machine
256 Given a `trusty` machine with ubuntu-advantage-tools installed
257 When I attach `contract_token` with sudo
258 Then I verify that running `ua disable foobar` `as non-root` exits `1`
259 And stderr matches regexp:
260 """
261 This command must be run as root \(try using sudo\)
262 """
263 And I verify that running `ua disable foobar` `with sudo` exits `1`
264 And stderr matches regexp:
265 """
266 Cannot disable unknown service 'foobar'.
267 Try cc-eal, cis, esm-apps, esm-infra, fips, fips-updates, livepatch
268 """
269 And I verify that running `ua disable esm-infra` `as non-root` exits `1`
270 And stderr matches regexp:
271 """
272 This command must be run as root \(try using sudo\)
273 """
274 When I run `ua disable esm-infra` with sudo
275 Then I will see the following on stdout:
276 """
277 Updating package lists
278 """
279 When I run `ua status` with sudo
280 Then stdout matches regexp:
281 """
282 esm-infra +yes +disabled +UA Infra: Extended Security Maintenance \(ESM\)
283 """
284 And I verify that running `apt update` `with sudo` exits `0`
285 When I run `apt-cache policy` with sudo
286 Then apt-cache policy for the following url has permission `-32768`
287 """
288 https://esm.ubuntu.com/ubuntu/ trusty-infra-security/main amd64 Packages
289 """
290 And apt-cache policy for the following url has permission `-32768`
291 """
292 https://esm.ubuntu.com/ubuntu/ trusty-infra-updates/main amd64 Packages
293 """
294
295 @series.all
296 Scenario Outline: Help command on an attached machine
297 Given a `<release>` machine with ubuntu-advantage-tools installed
298 When I attach `contract_token` with sudo
299 And I run `ua help esm-infra` with sudo
300 Then I will see the following on stdout:
301 """
302 Name:
303 esm-infra
304
305 Entitled:
306 yes
307
308 Status:
309 enabled
310
311 Help:
312 esm-infra provides access to a private ppa which includes available high
313 and critical CVE fixes for Ubuntu LTS packages in the Ubuntu Main
314 repository between the end of the standard Ubuntu LTS security
315 maintenance and its end of life. It is enabled by default with
316 Extended Security Maintenance (ESM) for UA Apps and UA Infra.
317 You can find our more about the esm service at
318 https://ubuntu.com/security/esm
319 """
320 When I run `ua help esm-infra --format json` with sudo
321 Then I will see the following on stdout:
322 """
323 {"name": "esm-infra", "entitled": "yes", "status": "enabled", "help": "esm-infra provides access to a private ppa which includes available high\nand critical CVE fixes for Ubuntu LTS packages in the Ubuntu Main\nrepository between the end of the standard Ubuntu LTS security\nmaintenance and its end of life. It is enabled by default with\nExtended Security Maintenance (ESM) for UA Apps and UA Infra.\nYou can find our more about the esm service at\nhttps://ubuntu.com/security/esm\n"}
324 """
325 And I verify that running `ua help invalid-service` `with sudo` exits `1`
326 And I will see the following on stderr:
327 """
328 No help available for 'invalid-service'
329 """
330 When I run `ua --help` as non-root
331 Then stdout matches regexp:
332 """
333 Client to manage Ubuntu Advantage services on a machine.
334 - esm-infra: UA Infra: Extended Security Maintenance \(ESM\)
335 \(https://ubuntu.com/security/esm\)
336 - fips-updates: Uncertified security updates to FIPS modules
337 \(https://ubuntu.com/security/certifications#fips\)
338 - fips: NIST-certified FIPS modules
339 \(https://ubuntu.com/security/certifications#fips\)
340 - livepatch: Canonical Livepatch service
341 \(https://ubuntu.com/security/livepatch\)
342 """
343 When I run `ua help` as non-root
344 Then stdout matches regexp:
345 """
346 Client to manage Ubuntu Advantage services on a machine.
347 - esm-infra: UA Infra: Extended Security Maintenance \(ESM\)
348 \(https://ubuntu.com/security/esm\)
349 - fips-updates: Uncertified security updates to FIPS modules
350 \(https://ubuntu.com/security/certifications#fips\)
351 - fips: NIST-certified FIPS modules
352 \(https://ubuntu.com/security/certifications#fips\)
353 - livepatch: Canonical Livepatch service
354 \(https://ubuntu.com/security/livepatch\)
355 """
356 When I run `ua help --all` as non-root
357 Then stdout matches regexp:
358 """
359 Client to manage Ubuntu Advantage services on a machine.
360 - cc-eal: Common Criteria EAL2 Provisioning Packages
361 \(https://ubuntu.com/cc-eal\)
362 - cis: Center for Internet Security Audit Tools
363 \(https://ubuntu.com/security/certifications#cis\)
364 - esm-apps: UA Apps: Extended Security Maintenance \(ESM\)
365 \(https://ubuntu.com/security/esm\)
366 - esm-infra: UA Infra: Extended Security Maintenance \(ESM\)
367 \(https://ubuntu.com/security/esm\)
368 - fips-updates: Uncertified security updates to FIPS modules
369 \(https://ubuntu.com/security/certifications#fips\)
370 - fips: NIST-certified FIPS modules
371 \(https://ubuntu.com/security/certifications#fips\)
372 - livepatch: Canonical Livepatch service
373 \(https://ubuntu.com/security/livepatch\)
374 """
375
376 Examples: ubuntu release
377 | release |
378 | bionic |
379 | focal |
380 | trusty |
381 | xenial |
382
383 @series.all
384 Scenario Outline: Purge package after attaching it to a machine
385 Given a `<release>` machine with ubuntu-advantage-tools installed
386 When I attach `contract_token` with sudo
387 And I run `touch /etc/apt/preferences.d/ubuntu-esm-infra` with sudo
388 Then I verify that files exist matching `/var/log/ubuntu-advantage.log`
389 And I verify that running `test -d /var/lib/ubuntu-advantage` `with sudo` exits `0`
390 And I verify that files exist matching `/etc/apt/auth.conf.d/90ubuntu-advantage`
391 And I verify that files exist matching `/etc/apt/trusted.gpg.d/ubuntu-advantage-esm-infra-trusty.gpg`
392 And I verify that files exist matching `/etc/apt/sources.list.d/ubuntu-esm-infra.list`
393 And I verify that files exist matching `/etc/apt/preferences.d/ubuntu-esm-infra`
394 When I run `apt-get purge ubuntu-advantage-tools -y` with sudo, retrying exit [100]
395 Then stdout matches regexp:
396 """
397 Purging configuration files for ubuntu-advantage-tools
398 """
399 And I verify that no files exist matching `/var/log/ubuntu-advantage.log`
400 And I verify that no files exist matching `/var/lib/ubuntu-advantage`
401 And I verify that no files exist matching `/etc/apt/auth.conf.d/90ubuntu-advantage`
402 And I verify that no files exist matching `/etc/apt/sources.list.d/ubuntu-*`
403 And I verify that no files exist matching `/etc/apt/trusted.gpg.d/ubuntu-advantage-*`
404 And I verify that no files exist matching `/etc/apt/preferences.d/ubuntu-*`
405
406 Examples: ubuntu release
407 | release |
408 | bionic |
409 | focal |
410 | trusty |
411 | xenial |
412
413 @series.all
414 Scenario Outline: Enable command with invalid repositories in user machine
415 Given a `<release>` machine with ubuntu-advantage-tools installed
416 When I attach `contract_token` with sudo
417 And I run `ua disable esm-infra` with sudo
418 And I run `add-apt-repository ppa:ua-client/staging -y` with sudo, retrying exit [1]
419 And I run `apt update` with sudo
420 And I run `sed -i 's/ubuntu/ubun/' /etc/apt/sources.list.d/<ppa_file>.list` with sudo
421 And I run `ua enable esm-infra` with sudo
422 Then stdout matches regexp:
423 """
424 One moment, checking your subscription first
425 Updating package lists
426 APT update failed.
427 APT update failed to read APT config for the following URL:
428 - http://ppa.launchpad.net/ua-client/staging/ubun
429 """
430
431 Examples: ubuntu release
432 | release | ppa_file |
433 | trusty | ua-client-staging-trusty |
434 | xenial | ua-client-ubuntu-staging-xenial |
435 | bionic | ua-client-ubuntu-staging-bionic |
436 | focal | ua-client-ubuntu-staging-focal |
diff --git a/features/attached_enable.feature b/features/attached_enable.feature
index 86f2a56..8a36fce 100644
--- a/features/attached_enable.feature
+++ b/features/attached_enable.feature
@@ -1,131 +1,437 @@
1@uses.config.contract_token1@uses.config.contract_token
2Feature: Enable command behaviour when attached to an UA subscription2Feature: Enable command behaviour when attached to an UA subscription
33
4 @series.all
5 Scenario Outline: Attached enable Common Criteria service in a ubuntu machine
6 Given a `<release>` machine with ubuntu-advantage-tools installed
7 When I attach `contract_token` with sudo
8 Then I verify that running `ua enable cc-eal` `as non-root` exits `1`
9 And I will see the following on stderr:
10 """
11 This command must be run as root (try using sudo)
12 """
13 And I verify that running `ua enable cc-eal --beta` `with sudo` exits `1`
14 And I will see the following on stdout
15 """
16 One moment, checking your subscription first
17 <msg>
18 """
19 And I verify that running `ua enable cc-eal` `with sudo` exits `1`
20 And I will see the following on stdout:
21 """
22 One moment, checking your subscription first
23 """
24 And stderr matches regexp:
25 """
26 Cannot enable unknown service 'cc-eal'.
27 Try esm-infra, fips, fips-updates, livepatch
28 """
429
5 Scenario: Attached enable Livepatch service in a trusty lxd container30 Examples: ubuntu release
6 Given a trusty lxd container with ubuntu-advantage-tools installed31 | release | msg |
7 When I attach contract_token with sudo32 | bionic | CC EAL2 is not available for Ubuntu 18.04 LTS (Bionic Beaver). |
8 And I run `ua enable livepatch` as non-root33 | focal | CC EAL2 is not available for Ubuntu 20.04 LTS (Focal Fossa). |
9 Then I will see the following on stderr:34 | trusty | CC EAL2 is not available for Ubuntu 14.04 LTS (Trusty Tahr). |
35
36 @series.all
37 Scenario Outline: Attached enable a disabled beta service and unknown service in a ubuntu machine
38 Given a `<release>` machine with ubuntu-advantage-tools installed
39 When I attach `contract_token` with sudo
40 Then I verify that running `ua enable cc-eal foobar` `as non-root` exits `1`
41 And I will see the following on stderr:
10 """42 """
11 This command must be run as root (try using sudo)43 This command must be run as root (try using sudo)
12 """44 """
13 When I run `ua enable livepatch` with sudo45 And I verify that running `ua enable cc-eal foobar` `with sudo` exits `1`
14 Then I will see the following on stdout:46 And I will see the following on stdout:
15 """47 """
16 One moment, checking your subscription first48 One moment, checking your subscription first
17 Cannot install Livepatch on a container49 """
50 And stderr matches regexp:
51 """
52 Cannot enable unknown service 'foobar, cc-eal'.
53 Try esm-infra, fips, fips-updates, livepatch
18 """54 """
1955
56 Examples: ubuntu release
57 | release |
58 | bionic |
59 | focal |
60 | trusty |
61 | xenial |
2062
21 Scenario: Attached enable Common Criteria service in a trusty lxd container63 @series.all
22 Given a trusty lxd container with ubuntu-advantage-tools installed64 Scenario Outline: Attached enable of an unknown service in a ubuntu machine
23 When I attach contract_token with sudo65 Given a `<release>` machine with ubuntu-advantage-tools installed
24 And I run `ua enable cc-eal` as non-root66 When I attach `contract_token` with sudo
25 Then I will see the following on stderr:67 Then I verify that running `ua enable foobar` `as non-root` exits `1`
68 And I will see the following on stderr:
26 """69 """
27 This command must be run as root (try using sudo)70 This command must be run as root (try using sudo)
28 """71 """
29 When I run `ua enable cc-eal` with sudo72 And I verify that running `ua enable foobar` `with sudo` exits `1`
30 Then I will see the following on stdout73 And I will see the following on stdout:
31 """74 """
32 One moment, checking your subscription first75 One moment, checking your subscription first
33 CC EAL2 is not available for Ubuntu 14.04 LTS (Trusty Tahr).76 """
77 And stderr matches regexp:
78 """
79 Cannot enable unknown service 'foobar'.
80 Try esm-infra, fips, fips-updates, livepatch
34 """81 """
3582
83 Examples: ubuntu release
84 | release |
85 | bionic |
86 | focal |
87 | trusty |
88 | xenial |
3689
37 Scenario: Attached enable CIS Audit service in a trusty lxd container90 @series.all
38 Given a trusty lxd container with ubuntu-advantage-tools installed91 Scenario Outline: Attached enable of a known service already enabled (UA Infra) in a ubuntu machine
39 When I attach contract_token with sudo92 Given a `<release>` machine with ubuntu-advantage-tools installed
40 And I run `ua enable cis-audit` as non-root93 When I attach `contract_token` with sudo
41 Then I will see the following on stderr:94 Then I verify that running `ua enable esm-infra` `as non-root` exits `1`
95 And I will see the following on stderr:
42 """96 """
43 This command must be run as root (try using sudo)97 This command must be run as root (try using sudo)
44 """98 """
45 When I run `ua enable cis-audit` with sudo99 And I verify that running `ua enable esm-infra` `with sudo` exits `1`
46 Then I will see the following on stdout:100 Then I will see the following on stdout:
47 """101 """
48 One moment, checking your subscription first102 One moment, checking your subscription first
49 This subscription is not entitled to CIS Audit.103 ESM Infra is already enabled.
50 For more information see: https://ubuntu.com/advantage104 See: sudo ua status
51 """105 """
106 When I run `apt-cache policy` with sudo
107 Then apt-cache policy for the following url has permission `500`
108 """
109 <esm-infra-url> <release>-infra-updates/main amd64 Packages
110 """
111 And I verify that running `apt update` `with sudo` exits `0`
112 When I run `apt install -y <infra-pkg>` with sudo, retrying exit [100]
113 And I run `apt-cache policy <infra-pkg>` as non-root
114 Then stdout matches regexp:
115 """
116 \s*500 <esm-infra-url> <release>-infra-security/main amd64 Packages
117 \s*500 <esm-infra-url> <release>-infra-updates/main amd64 Packages
118 """
52119
53 Scenario: Attached enable UA Apps service in a trusty lxd container120 Examples: ubuntu release
54 Given a trusty lxd container with ubuntu-advantage-tools installed121 | release | infra-pkg | esm-infra-url |
55 When I attach contract_token with sudo122 | bionic | libkrad0 | https://esm.ubuntu.com/infra/ubuntu |
56 And I run `ua enable esm-apps` as non-root123 | focal | hello | https://esm.ubuntu.com/infra/ubuntu |
57 Then I will see the following on stderr:124 | trusty | libgit2-0 | https://esm.ubuntu.com/ubuntu/ |
125 | xenial | libkrad0 | https://esm.ubuntu.com/infra/ubuntu |
126
127 @series.xenial
128 @series.bionic
129 @series.focal
130 Scenario Outline: Attached enable of a know service shows update in a ubuntu machine
131 Given a `<release>` machine with ubuntu-advantage-tools installed
132 When I attach `contract_token` with sudo
133 Then I verify that running `ua enable esm-infra` `with sudo` exits `1`
134 And I will see the following on stdout:
135 """
136 One moment, checking your subscription first
137 ESM Infra is already enabled.
138 See: sudo ua status
139 """
140 When I run `apt install -y <pkg-version>` with sudo, retrying exit [100]
141 And I run `apt update` with sudo
142 Then stdout matches regexp
143 """
144 \d+ of the updates (is|are) from UA Infra: ESM
145 """
146 When I run `ua disable esm-infra` with sudo
147 And I run `apt update` with sudo
148 Then stdout does not match regexp
149 """
150 \d+ of the updates (is|are) from UA Infra: ESM
151 """
152
153 Examples: ubuntu release
154 | release | pkg-version |
155 | bionic | libkrad0=1.16-2build1 |
156 | focal | hello=2.10-2ubuntu2 |
157 | xenial | libkrad0=1.13.2+dfsg-5 |
158
159 @series.all
160 @uses.config.machine_type.lxd.container
161 Scenario Outline: Attached enable of non-container services in a ubuntu lxd container
162 Given a `<release>` machine with ubuntu-advantage-tools installed
163 When I attach `contract_token` with sudo
164 Then I verify that running `ua enable <service> <flag>` `as non-root` exits `1`
165 And I will see the following on stderr:
58 """166 """
59 This command must be run as root (try using sudo)167 This command must be run as root (try using sudo)
60 """168 """
61 When I run `ua enable esm-apps` with sudo169 And I verify that running `ua enable <service> <flag>` `with sudo` exits `1`
62 Then I will see the following on stdout:170 And I will see the following on stdout:
63 """171 """
64 One moment, checking your subscription first172 One moment, checking your subscription first
65 This subscription is not entitled to ESM Apps.173 Cannot install <title> on a container
66 For more information see: https://ubuntu.com/advantage
67 """174 """
68175
176 Examples: Un-supported services in containers
177 | release | service | title | flag |
178 | bionic | livepatch | Livepatch | |
179 | bionic | fips | FIPS | --assume-yes |
180 | bionic | fips-updates | FIPS Updates | --assume-yes |
181 | focal | livepatch | Livepatch | |
182 | focal | fips | FIPS | --assume-yes |
183 | focal | fips-updates | FIPS Updates | --assume-yes |
184 | trusty | livepatch | Livepatch | |
185 | trusty | fips | FIPS | --assume-yes |
186 | trusty | fips-updates | FIPS Updates | --assume-yes |
187 | xenial | livepatch | Livepatch | |
188 | xenial | fips | FIPS | --assume-yes |
189 | xenial | fips-updates | FIPS Updates | --assume-yes |
69190
70 Scenario: Attached enable NIST-certified FIPS service in a trusty lxd container191 @series.all
71 Given a trusty lxd container with ubuntu-advantage-tools installed192 Scenario Outline: Attached enable not entitled service in a ubuntu machine
72 When I attach contract_token with sudo193 Given a `<release>` machine with ubuntu-advantage-tools installed
73 And I run `ua enable fips` as non-root194 When I attach `contract_token` with sudo
74 Then I will see the following on stderr:195 Then I verify that running `ua enable <service>` `as non-root` exits `1`
196 And I will see the following on stderr:
75 """197 """
76 This command must be run as root (try using sudo)198 This command must be run as root (try using sudo)
77 """199 """
78 When I run `ua enable fips` with sudo200 And I verify that running `ua enable <service> --beta` `with sudo` exits `1`
79 Then I will see the following on stdout:201 And I will see the following on stdout:
80 """202 """
81 One moment, checking your subscription first203 One moment, checking your subscription first
82 Cannot install FIPS on a container204 This subscription is not entitled to <title>.
205 For more information see: https://ubuntu.com/advantage
83 """206 """
84207
208 Examples: not entitled services
209 | release | service | title |
210 | bionic | cis | CIS Audit |
211 | bionic | esm-apps | ESM Apps |
212 | focal | cis | CIS Audit |
213 | focal | esm-apps | ESM Apps |
214 | trusty | cis | CIS Audit |
215 | trusty | esm-apps | ESM Apps |
216 | xenial | cis | CIS Audit |
217 | xenial | esm-apps | ESM Apps |
85218
86 Scenario: Attached enable Uncertified FIPS service in a trusty lxd container219 @series.focal
87 Given a trusty lxd container with ubuntu-advantage-tools installed220 @uses.config.machine_type.lxd.vm
88 When I attach contract_token with sudo221 Scenario: Attached enable of vm-based services in a focal lxd vm
89 And I run `ua enable fips-updates` as non-root222 Given a `focal` machine with ubuntu-advantage-tools installed
223 When I attach `contract_token` with sudo
224 Then I verify that running `ua enable fips --assume-yes` `with sudo` exits `1`
225 And I will see the following on stdout:
226 """
227 One moment, checking your subscription first
228 FIPS is not available for Ubuntu 20.04 LTS (Focal Fossa).
229 """
230 And I verify that running `ua enable fips-updates --assume-yes` `with sudo` exits `1`
231 And I will see the following on stdout:
232 """
233 One moment, checking your subscription first
234 FIPS Updates is not available for Ubuntu 20.04 LTS (Focal Fossa).
235 """
236
237 @series.bionic
238 @series.xenial
239 @uses.config.machine_type.lxd.vm
240 Scenario Outline: Attached enable of vm-based services in a bionic lxd vm
241 Given a `<release>` machine with ubuntu-advantage-tools installed
242 When I attach `contract_token` with sudo
243 And I run `ua status` with sudo
244 Then stdout matches regexp:
245 """
246 esm-infra +yes +enabled +UA Infra: Extended Security Maintenance \(ESM\)
247 fips +yes +disabled +NIST-certified FIPS modules
248 fips-updates +yes +disabled +Uncertified security updates to FIPS modules
249 livepatch +yes +enabled +Canonical Livepatch service
250 """
251 When I run `ua disable livepatch` with sudo
252 Then I verify that running `canonical-livepatch status` `with sudo` exits `1`
253 And stdout matches regexp:
254 """
255 Machine is not enabled. Please run 'sudo canonical-livepatch enable' with the
256 token obtained from https://ubuntu.com/livepatch.
257 """
258 When I run `ua status` with sudo
259 Then stdout matches regexp:
260 """
261 esm-infra +yes +enabled +UA Infra: Extended Security Maintenance \(ESM\)
262 fips +yes +disabled +NIST-certified FIPS modules
263 fips-updates +yes +disabled +Uncertified security updates to FIPS modules
264 livepatch +yes +disabled +Canonical Livepatch service
265 """
266
267 Examples: ubuntu release
268 | release |
269 | xenial |
270 | bionic |
271
272 @series.bionic
273 @series.xenial
274 @uses.config.machine_type.lxd.vm
275 Scenario Outline: Attached enable livepatch on a machine with fips active
276 Given a `<release>` machine with ubuntu-advantage-tools installed
277 When I verify that running `canonical-livepatch status` `with sudo` exits `1`
90 Then I will see the following on stderr:278 Then I will see the following on stderr:
91 """279 """
92 This command must be run as root (try using sudo)280 sudo: canonical-livepatch: command not found
281 """
282 When I attach `contract_token` with sudo
283 Then stdout matches regexp:
284 """
285 Installing canonical-livepatch snap
286 Canonical livepatch enabled
93 """287 """
94 When I run `ua enable fips-updates` with sudo288 When I run `ua status` with sudo
289 Then stdout matches regexp:
290 """
291 livepatch yes enabled
292 """
293 When I run `canonical-livepatch status` with sudo
294 Then stdout matches regexp:
295 """
296 running: true
297 """
298
299 Examples: ubuntu release
300 | release |
301 | xenial |
302 | bionic |
303
304 @series.bionic
305 @uses.config.machine_type.lxd.vm
306 Scenario: Attached enable fips on a machine with livepatch active
307 Given a `bionic` machine with ubuntu-advantage-tools installed
308 When I attach `contract_token` with sudo
309 Then stdout matches regexp:
310 """
311 Updating package lists
312 ESM Infra enabled
313 Installing canonical-livepatch snap
314 Canonical livepatch enabled
315 """
316 When I run `ua disable livepatch` with sudo
317 And I run `ua enable fips --assume-yes` with sudo
95 Then I will see the following on stdout:318 Then I will see the following on stdout:
96 """319 """
97 One moment, checking your subscription first320 One moment, checking your subscription first
98 Cannot install FIPS Updates on a container321 Updating package lists
322 Installing FIPS packages
323 FIPS enabled
324 A reboot is required to complete install
325 """
326 When I append the following on uaclient config:
327 """
328 features:
329 block_disable_on_enable: true
330 """
331 Then I verify that running `ua enable livepatch` `with sudo` exits `1`
332 And I will see the following on stdout
333 """
334 One moment, checking your subscription first
335 Cannot enable Livepatch when FIPS is enabled
99 """336 """
100337
338 @series.bionic
339 @uses.config.machine_type.lxd.vm
340 Scenario: Attached enable livepatch on a machine with fips active
341 Given a `bionic` machine with ubuntu-advantage-tools installed
342 When I attach `contract_token` with sudo
343 Then stdout matches regexp:
344 """
345 Updating package lists
346 ESM Infra enabled
347 Installing canonical-livepatch snap
348 Canonical livepatch enabled
349 """
350 When I append the following on uaclient config:
351 """
352 features:
353 block_disable_on_enable: true
354 """
355 Then I verify that running `ua enable fips --assume-yes` `with sudo` exits `1`
356 And I will see the following on stdout
357 """
358 One moment, checking your subscription first
359 Cannot enable FIPS when Livepatch is enabled
360 """
101361
102 Scenario: Attached enable of an unknown service in a trusty lxd container362 @series.xenial
103 Given a trusty lxd container with ubuntu-advantage-tools installed363 @series.bionic
104 When I attach contract_token with sudo364 @uses.config.machine_type.lxd.vm
105 And I run `ua enable foobar` as non-root365 Scenario Outline: Attached enable fips on a machine with livepatch active
106 Then I will see the following on stderr:366 Given a `<release>` machine with ubuntu-advantage-tools installed
367 When I attach `contract_token` with sudo
368 Then stdout matches regexp:
107 """369 """
108 This command must be run as root (try using sudo)370 Updating package lists
371 ESM Infra enabled
109 """372 """
110 When I run `ua enable foobar` with sudo373 And stdout matches regexp:
111 Then I will see the following on stderr:374 """
375 Installing canonical-livepatch snap
376 Canonical livepatch enabled
377 """
378 When I run `ua enable fips --assume-yes` with sudo
379 Then I will see the following on stdout
380 """
381 One moment, checking your subscription first
382 Updating package lists
383 Installing FIPS packages
384 FIPS enabled
385 A reboot is required to complete install
112 """386 """
113 Cannot enable 'foobar'387 When I run `ua status` with sudo
114 For a list of services see: sudo ua status388 Then stdout matches regexp:
389 """
390 fips +yes +enabled
391 """
392 And stdout matches regexp:
393 """
394 livepatch +yes +n/a
115 """395 """
116396
117 Scenario: Attached enable of a known service already enabled (UA Infra) in a trusty lxd container397 Examples: ubuntu release
118 Given a trusty lxd container with ubuntu-advantage-tools installed398 | release |
119 When I attach contract_token with sudo399 | bionic |
120 And I run `ua enable esm-infra` as non-root400 | xenial |
121 Then I will see the following on stderr:401
402 @series.xenial
403 @series.bionic
404 @uses.config.machine_type.lxd.vm
405 Scenario Outline: Attached enable fips on a machine with fips-updates active
406 Given a `<release>` machine with ubuntu-advantage-tools installed
407 When I attach `contract_token` with sudo
408 Then stdout matches regexp:
122 """409 """
123 This command must be run as root (try using sudo)410 ESM Infra enabled
411 """
412 And stdout matches regexp:
413 """
414 Installing canonical-livepatch snap
415 Canonical livepatch enabled
124 """416 """
125 When I run `ua enable esm-infra` with sudo417 When I run `ua disable livepatch` with sudo
418 And I run `ua enable fips-updates --assume-yes` with sudo
126 Then I will see the following on stdout:419 Then I will see the following on stdout:
127 """420 """
128 One moment, checking your subscription first421 One moment, checking your subscription first
129 ESM Infra is already enabled.422 Updating package lists
130 See: sudo ua status423 Installing FIPS Updates packages
424 FIPS Updates enabled
425 A reboot is required to complete install
131 """426 """
427 When I verify that running `ua enable fips --assume-yes` `with sudo` exits `1`
428 Then I will see the following on stdout
429 """
430 One moment, checking your subscription first
431 Cannot enable FIPS when FIPS Updates is enabled
432 """
433
434 Examples: ubuntu release
435 | release |
436 | bionic |
437 | xenial |
diff --git a/features/aws-ids.yaml b/features/aws-ids.yaml
132new file mode 100644438new file mode 100644
index 0000000..e065192
--- /dev/null
+++ b/features/aws-ids.yaml
@@ -0,0 +1,4 @@
1bionic: ami-0e11aa8c6a6d58146
2focal: ami-0383ece2c0f6de239
3trusty: ami-0a64b4493bc9dc321
4xenial: ami-09e110a448d322f4a
diff --git a/features/azure-ids.yaml b/features/azure-ids.yaml
0new file mode 1006445new file mode 100644
index 0000000..0072983
--- /dev/null
+++ b/features/azure-ids.yaml
@@ -0,0 +1,4 @@
1bionic: "Canonical:0001-com-ubuntu-pro-bionic:pro-18_04-lts"
2focal: "Canonical:0001-com-ubuntu-pro-focal:pro-20_04-lts"
3trusty: "Canonical:0001-com-ubuntu-pro-trusty:pro-14_04-lts"
4xenial: "Canonical:0001-com-ubuntu-pro-xenial:pro-16_04-lts"
diff --git a/features/cloud.py b/features/cloud.py
0new file mode 1006445new file mode 100644
index 0000000..4443345
--- /dev/null
+++ b/features/cloud.py
@@ -0,0 +1,744 @@
1import json
2import os
3import logging
4import pycloudlib # type: ignore
5import time
6import yaml
7
8try:
9 from typing import Tuple, List, Optional # noqa
10except ImportError:
11 # typing isn't available on trusty, so ignore its absence
12 pass
13
14
15class Cloud:
16 """Base class for cloud providers that should be tested through behave.
17
18 :machine_type:
19 A string representing the type of machine to launch (pro or generic)
20 :region:
21 The region to create the cloud resources on
22 :param tag:
23 A tag to be used when creating the resources on the cloud provider
24 :timestamp_suffix:
25 Boolean set true to direct pycloudlib to append a timestamp to the end
26 of the provided tag.
27 """
28
29 name = ""
30 pro_ids_path = ""
31 env_vars: "Tuple[str, ...]" = ()
32
33 def __init__(
34 self,
35 machine_type: str,
36 region: "Optional[str]" = None,
37 tag: "Optional[str]" = None,
38 timestamp_suffix: bool = True,
39 ) -> None:
40 if tag:
41 self.tag = tag
42 else:
43 self.tag = "uaclient-ci"
44 self.machine_type = machine_type
45 self.region = region
46 self._api = None
47 self.key_name = pycloudlib.util.get_timestamped_tag(self.tag)
48 self.timestamp_suffix = timestamp_suffix
49
50 missing_env_vars = self.missing_env_vars()
51 if missing_env_vars:
52 logging.warning(
53 "".join(
54 [
55 "UACLIENT_BEHAVE_MACHINE_TYPE={} requires".format(
56 self.machine_type
57 ),
58 " the following env vars:\n",
59 *self.format_missing_env_vars(missing_env_vars),
60 ]
61 )
62 )
63
64 @property
65 def api(self) -> pycloudlib.cloud.BaseCloud:
66 """Return the api used to interact with the cloud provider."""
67 raise NotImplementedError
68
69 def _create_instance(
70 self,
71 series: str,
72 instance_name: "Optional[str]" = None,
73 image_name: "Optional[str]" = None,
74 user_data: "Optional[str]" = None,
75 ) -> pycloudlib.instance:
76 """Create an instance for on the cloud provider.
77
78 :param series:
79 The ubuntu release to be used when creating an instance. We will
80 create an image based on this value if the used does not provide
81 a image_name value
82 :param instance_name:
83 The name of the instance to be created
84 :param image_name:
85 The name of the image to be used when creating the instance
86 :param user_data:
87 The user data to be passed when creating the instance
88
89 :returns:
90 A cloud provider instance
91 """
92 raise NotImplementedError
93
94 def _check_cloudinit_status(
95 self, instance: pycloudlib.instance.BaseInstance
96 ) -> None:
97 """
98 Check if cloudinit was able to finish without errors.
99
100 :param instance:
101 An instance created on the cloud provider
102 """
103 result = instance.execute(["cloud-init", "status", "--wait", "--long"])
104
105 if result.failed:
106 raise OSError(
107 "cloud-init failed to start\n: out: %s\n error: %s"
108 % (result.stdout, result.stderr)
109 )
110
111 print("--- cloud-init succeeded")
112
113 def launch(
114 self,
115 series: str,
116 instance_name: "Optional[str]" = None,
117 image_name: "Optional[str]" = None,
118 user_data: "Optional[str]" = None,
119 ) -> pycloudlib.instance.BaseInstance:
120 """Create and wait for cloud provider instance to be ready.
121
122 :param series:
123 The ubuntu release to be used when creating an instance. We will
124 create an image based on this value if the used does not provide
125 a image_name value
126 :param instance_name:
127 The name of the instance to be created
128 :param image_name:
129 The name of the image to be used when creating the instance
130 :param user_data:
131 The user data to be passed when creating the instance
132
133 :returns:
134 An cloud provider instance
135 """
136 inst = self._create_instance(
137 series=series,
138 instance_name=instance_name,
139 image_name=image_name,
140 user_data=user_data,
141 )
142 print(
143 "--- {} instance launched: {}. Waiting for ssh access".format(
144 self.name, inst.name
145 )
146 )
147 time.sleep(15)
148 for sleep in (5, 10, 15):
149 try:
150 inst.wait()
151 break
152 except Exception as e:
153 print("--- Retrying instance.wait on {}".format(str(e)))
154
155 if series != "trusty":
156 self._check_cloudinit_status(inst)
157
158 return inst
159
160 def get_instance_id(
161 self, instance: pycloudlib.instance.BaseInstance
162 ) -> str:
163 """Return the instance identifier.
164
165 :param instance:
166 An instance created on the cloud provider
167
168 :returns:
169 The string of the unique instance id
170 """
171 return instance.id
172
173 def format_missing_env_vars(self, missing_env_vars: "List") -> "List[str]":
174 """Format missing env vars to be displayed in log.
175
176 :returns:
177 A list of env string formatted to be used when logging
178 """
179 return [" - {}\n".format(env_var) for env_var in missing_env_vars]
180
181 def missing_env_vars(self) -> "List[str]":
182 """Return a list of env variables necessary for this cloud provider.
183
184 :returns:
185 A list of string representing the missing variables
186 """
187 return [
188 env_name
189 for env_name in self.env_vars
190 if not getattr(
191 self, env_name.lower().replace("uaclient_behave_", "")
192 )
193 ]
194
195 def locate_image_name(self, series: str) -> str:
196 """Locate and return the image name to use for vm provision.
197
198 :param series:
199 The ubuntu release to be used when locating the image name
200
201 :returns:
202 A image name to use when provisioning a virtual machine
203 based on the series value
204 """
205 if not series:
206 raise ValueError(
207 "Must provide either series or image_name to launch azure"
208 )
209
210 if "pro" in self.machine_type:
211 with open(self.pro_ids_path, "r") as stream:
212 pro_ids = yaml.safe_load(stream.read())
213 image_name = pro_ids[series]
214 else:
215 image_name = self.api.daily_image(release=series)
216
217 return image_name
218
219 def manage_ssh_key(self, private_key_path: "Optional[str]" = None) -> None:
220 """Create and manage ssh key pairs to be used in the cloud provider.
221
222 :param private_key_path:
223 Location of the private key path to use. If None, the location
224 will be a default location.
225 """
226 cloud_name = self.name.lower().replace("_", "-")
227 pub_key_path = "{}-pubkey".format(cloud_name)
228 priv_key_path = "{}-privkey".format(cloud_name)
229 pub_key, priv_key = self.api.create_key_pair()
230
231 with open(pub_key_path, "w") as f:
232 f.write(pub_key)
233
234 with open(priv_key_path, "w") as f:
235 f.write(priv_key)
236
237 os.chmod(pub_key_path, 0o600)
238 os.chmod(priv_key_path, 0o600)
239
240 self.api.use_key(
241 public_key_path=pub_key_path, private_key_path=priv_key_path
242 )
243
244
245class EC2(Cloud):
246 """Class that represents the EC2 cloud provider.
247
248 :param aws_access_key_id:
249 The aws access key id
250 :param aws_secret_access_key:
251 The aws secret access key
252 :region:
253 The region to be used to create the aws instances
254 :machine_type:
255 A string representing the type of machine to launch (pro or generic)
256 :tag:
257 A tag to be used when creating the resources on the cloud provider
258 :timestamp_suffix:
259 Boolean set true to direct pycloudlib to append a timestamp to the end
260 of the provided tag.
261 """
262
263 name = "aws"
264 env_vars: "Tuple[str, ...]" = (
265 "aws_access_key_id",
266 "aws_secret_access_key",
267 )
268 pro_ids_path = "features/aws-ids.yaml"
269
270 def __init__(
271 self,
272 aws_access_key_id: "Optional[str]",
273 aws_secret_access_key: "Optional[str]",
274 machine_type: str,
275 region: "Optional[str]" = "us-east-2",
276 tag: "Optional[str]" = None,
277 timestamp_suffix: bool = True,
278 ) -> None:
279 self.aws_access_key_id = aws_access_key_id
280 self.aws_secret_access_key = aws_secret_access_key
281 logging.basicConfig(
282 filename="pycloudlib-behave.log", level=logging.DEBUG
283 )
284 super().__init__(
285 region=region,
286 machine_type=machine_type,
287 tag=tag,
288 timestamp_suffix=timestamp_suffix,
289 )
290
291 @property
292 def api(self) -> pycloudlib.cloud.BaseCloud:
293 """Return the api used to interact with the cloud provider."""
294 if self._api is None:
295 self._api = pycloudlib.EC2(
296 tag=self.tag,
297 access_key_id=self.aws_access_key_id,
298 secret_access_key=self.aws_secret_access_key,
299 region=self.region,
300 timestamp_suffix=self.timestamp_suffix,
301 )
302
303 return self._api
304
305 def manage_ssh_key(
306 self,
307 private_key_path: "Optional[str]" = None,
308 key_name: "Optional[str]" = None,
309 ) -> None:
310 """Create and manage ssh key pairs to be used in the cloud provider.
311
312 :param private_key_path:
313 Location of the private key path to use. If None, the location
314 will be a default location.
315 :param key_name:
316 Optional key_name to use when uploading to the cloud. Default is
317 uaclient-ci-<timestamp>
318 """
319 if key_name:
320 self.key_name = key_name
321 if not private_key_path:
322 if self.key_name in self.api.list_keys():
323 self.api.delete_key(self.key_name)
324
325 private_key_path = "ec2-{}.pem".format(self.key_name)
326 print(
327 "--- Creating local keyfile {} for EC2".format(
328 private_key_path
329 )
330 )
331 keypair = self.api.client.create_key_pair(KeyName=self.key_name)
332
333 with open(private_key_path, "w") as stream:
334 stream.write(keypair["KeyMaterial"])
335 os.chmod(private_key_path, 0o600)
336
337 self.api.use_key(private_key_path, private_key_path, self.key_name)
338
339 def _create_instance(
340 self,
341 series: str,
342 instance_name: "Optional[str]" = None,
343 image_name: "Optional[str]" = None,
344 user_data: "Optional[str]" = None,
345 ) -> pycloudlib.instance:
346 """Launch an instance on the cloud provider.
347
348 :param series:
349 The ubuntu release to be used when creating an instance. We will
350 create an image based on this value if the used does not provide
351 a image_name value
352 :param instance_name:
353 The name of the instance to be created
354 :param image_name:
355 The name of the image to be used when creating the instance
356 :param user_data:
357 The user data to be passed when creating the instance
358
359 :returns:
360 An AWS cloud provider instance
361 """
362 if not image_name:
363 image_name = self.locate_image_name(series)
364
365 print("--- Launching AWS image {}({})".format(image_name, series))
366 vpc = self.api.get_or_create_vpc(name="uaclient-integration")
367
368 try:
369 inst = self.api.launch(image_name, user_data=user_data, vpc=vpc)
370 except Exception as e:
371 print(str(e))
372 raise
373
374 return inst
375
376
377class Azure(Cloud):
378 """Class that represents the Azure cloud provider.
379
380 :param az_client_id:
381 The Azure client id
382 :param az_client_secret
383 The Azure client secret
384 :param az_tenant_id:
385 The Azure tenant id
386 :param az_subscription_id:
387 The Azure subscription id
388 :machine_type:
389 A string representing the type of machine to launch (pro or generic)
390 :region:
391 The region to create the resources on
392 :tag:
393 A tag to be used when creating the resources on the cloud provider
394 :timestamp_suffix:
395 Boolean set true to direct pycloudlib to append a timestamp to the end
396 of the provided tag.
397 """
398
399 name = "Azure"
400 env_vars: "Tuple[str, ...]" = (
401 "az_client_id",
402 "az_client_secret",
403 "az_tenant_id",
404 "az_subscription_id",
405 )
406 pro_ids_path = "features/azure-ids.yaml"
407
408 def __init__(
409 self,
410 machine_type: str,
411 region: "Optional[str]" = "centralus",
412 tag: "Optional[str]" = None,
413 timestamp_suffix: bool = True,
414 az_client_id: "Optional[str]" = None,
415 az_client_secret: "Optional[str]" = None,
416 az_tenant_id: "Optional[str]" = None,
417 az_subscription_id: "Optional[str]" = None,
418 ) -> None:
419 self.az_client_id = az_client_id
420 self.az_client_secret = az_client_secret
421 self.az_tenant_id = az_tenant_id
422 self.az_subscription_id = az_subscription_id
423
424 super().__init__(
425 machine_type=machine_type,
426 region=region,
427 tag=tag,
428 timestamp_suffix=timestamp_suffix,
429 )
430
431 @property
432 def api(self) -> pycloudlib.cloud.BaseCloud:
433 """Return the api used to interact with the cloud provider."""
434 if self._api is None:
435 self._api = pycloudlib.Azure(
436 tag=self.tag,
437 client_id=self.az_client_id,
438 client_secret=self.az_client_secret,
439 tenant_id=self.az_tenant_id,
440 subscription_id=self.az_subscription_id,
441 timestamp_suffix=self.timestamp_suffix,
442 )
443
444 return self._api
445
446 def get_instance_id(
447 self, instance: pycloudlib.instance.BaseInstance
448 ) -> str:
449 """Return the instance identifier.
450
451 :param instance:
452 An instance created on the cloud provider
453
454 :returns:
455 The string of the unique instance id
456 """
457 # For Azure, the API identifier uses the instance name
458 # instead of the instance id
459 return instance.name
460
461 def manage_ssh_key(self, private_key_path: "Optional[str]" = None) -> None:
462 """Create and manage ssh key pairs to be used in the cloud provider.
463
464 :param private_key_path:
465 Location of the private key path to use. If None, the location
466 will be a default location.
467 """
468 if not private_key_path:
469 private_key_path = "azure-priv-{}.pem".format(self.key_name)
470 pub_key_path = "azure-pub-{}.txt".format(self.key_name)
471 print(
472 "--- Creating local keyfile {} for Azure".format(
473 private_key_path
474 )
475 )
476 if not os.path.exists(private_key_path):
477 pub_key, priv_key = self.api.create_key_pair(
478 key_name=self.key_name
479 )
480
481 with open(pub_key_path, "w") as stream:
482 stream.write(pub_key)
483
484 with open(private_key_path, "w") as stream:
485 stream.write(priv_key)
486
487 os.chmod(pub_key_path, 0o600)
488 os.chmod(private_key_path, 0o600)
489
490 self.api.use_key(pub_key_path, private_key_path, self.key_name)
491
492 def _create_instance(
493 self,
494 series: str,
495 instance_name: "Optional[str]" = None,
496 image_name: "Optional[str]" = None,
497 user_data: "Optional[str]" = None,
498 ) -> pycloudlib.instance:
499 """Launch an instance on the cloud provider.
500
501 :param series:
502 The ubuntu release to be used when creating an instance. We will
503 create an image based on this value if the used does not provide
504 a image_name value
505 :param instance_name:
506 The name of the instance to be created
507 :param image_name:
508 The name of the image to be used when creating the instance
509 :param user_data:
510 The user data to be passed when creating the instance
511
512 :returns:
513 An AWS cloud provider instance
514 """
515 if not image_name:
516 image_name = self.locate_image_name(series)
517
518 print("--- Launching Azure image {}({})".format(image_name, series))
519 inst = self.api.launch(image_id=image_name, user_data=user_data)
520 return inst
521
522
523class GCP(Cloud):
524 name = "gcp"
525 pro_ids_path = "features/gcp-ids.yaml"
526
527 """Class that represents the Google Cloud Platform cloud provider.
528
529 :param gcp_credentials_path
530 The GCP credentials path to use when authentiacting to GCP
531 :param gcp_project
532 The name of the GCP project to be used
533 :machine_type:
534 A string representing the type of machine to launch (pro or generic)
535 :region:
536 The region to create the resources on
537 :tag:
538 A tag to be used when creating the resources on the cloud provider
539 :timestamp_suffix:
540 Boolean set true to direct pycloudlib to append a timestamp to the end
541 of the provided tag.
542 """
543
544 env_vars: "Tuple[str, ...]" = ("gcp_credentials_path", "gcp_project")
545
546 def __init__(
547 self,
548 machine_type: str,
549 region: "Optional[str]" = "us-west2",
550 tag: "Optional[str]" = None,
551 timestamp_suffix: bool = True,
552 zone: "Optional[str]" = "a",
553 gcp_credentials_path: "Optional[str]" = None,
554 gcp_project: "Optional[str]" = None,
555 ) -> None:
556 self.gcp_credentials_path = gcp_credentials_path
557 self.gcp_project = gcp_project
558 self.zone = zone
559
560 super().__init__(
561 machine_type=machine_type,
562 region=region,
563 tag=tag,
564 timestamp_suffix=timestamp_suffix,
565 )
566
567 self._set_service_account_email()
568
569 def _set_service_account_email(self):
570 """Set service account email if credentials provided."""
571 json_credentials = {}
572
573 if self.gcp_credentials_path:
574 with open(self.gcp_credentials_path, "r") as f:
575 json_credentials = json.load(f)
576
577 self.service_account_email = json_credentials.get("client_email")
578
579 @property
580 def api(self) -> pycloudlib.cloud.BaseCloud:
581 """Return the api used to interact with the cloud provider."""
582 if self._api is None:
583 self._api = pycloudlib.GCE(
584 tag=self.tag,
585 timestamp_suffix=self.timestamp_suffix,
586 credentials_path=self.gcp_credentials_path,
587 project=self.gcp_project,
588 zone=self.zone,
589 region=self.region,
590 service_account_email=self.service_account_email,
591 )
592
593 return self._api
594
595 def _create_instance(
596 self,
597 series: str,
598 instance_name: "Optional[str]" = None,
599 image_name: "Optional[str]" = None,
600 user_data: "Optional[str]" = None,
601 ) -> pycloudlib.instance:
602 """Launch an instance on the cloud provider.
603
604 :param series:
605 The ubuntu release to be used when creating an instance. We will
606 create an image based on this value if the used does not provide
607 a image_name value
608 :param instance_name:
609 The name of the instance to be created
610 :param image_name:
611 The name of the image to be used when creating the instance
612 :param user_data:
613 The user data to be passed when creating the instance
614
615 :returns:
616 An AWS cloud provider instance
617 """
618 if not image_name:
619 image_name = self.locate_image_name(series)
620
621 print("--- Launching GCP image {}({})".format(image_name, series))
622 inst = self.api.launch(image_id=image_name, user_data=user_data)
623 return inst
624
625
626class _LXD(Cloud):
627 name = "_lxd"
628
629 @property
630 def pycloudlib_cls(self):
631 """Return the pycloudlib cls to be used as an api."""
632 raise NotImplementedError
633
634 def _create_instance(
635 self,
636 series: str,
637 instance_name: "Optional[str]" = None,
638 image_name: "Optional[str]" = None,
639 user_data: "Optional[str]" = None,
640 ) -> pycloudlib.instance:
641 """Launch an instance on the cloud provider.
642
643 :param series:
644 The ubuntu release to be used when creating an instance. We will
645 create an image based on this value if the used does not provide
646 a image_name value
647 :param instance_name:
648 The name of the instance to be created
649 :param image_name:
650 The name of the image to be used when creating the instance
651 :param user_data:
652 The user data to be passed when creating the instance
653
654 :returns:
655 An AWS cloud provider instance
656 """
657 if not image_name:
658 image_name = self.locate_image_name(series)
659
660 image_type = self.name.title().replace("-", " ")
661
662 print(
663 "--- Launching {} image {}({})".format(
664 image_type, image_name, series
665 )
666 )
667
668 inst = self.api.launch(
669 name=instance_name, image_id=image_name, user_data=user_data
670 )
671 return inst
672
673 def get_instance_id(
674 self, instance: pycloudlib.instance.BaseInstance
675 ) -> str:
676 """Return the instance identifier.
677
678 :param instance:
679 An instance created on the cloud provider
680
681 :returns:
682 The string of the unique instance id
683 """
684 # For LXD, the API identifier uses the instance name
685 # instead of the instance id
686 return instance.name
687
688 def locate_image_name(self, series: str) -> str:
689 """Locate and return the image name to use for vm provision.
690
691 :param series:
692 The ubuntu release to be used when locating the image name
693
694 :returns:
695 A image name to use when provisioning a virtual machine
696 based on the series value
697 """
698 if not series:
699 raise ValueError(
700 "Must provide either series or image_name to launch azure"
701 )
702
703 image_name = self.api.daily_image(release=series)
704 return image_name
705
706 @property
707 def api(self) -> pycloudlib.cloud.BaseCloud:
708 """Return the api used to interact with the cloud provider."""
709 if self._api is None:
710 self._api = self.pycloudlib_cls(
711 tag=self.tag, timestamp_suffix=self.timestamp_suffix
712 )
713
714 return self._api
715
716
717class LXDVirtualMachine(_LXD):
718 name = "lxd-virtual-machine"
719
720 @property
721 def pycloudlib_cls(self):
722 """Return the pycloudlib cls to be used as an api."""
723 return pycloudlib.LXDVirtualMachine
724
725
726class LXDContainer(_LXD):
727 name = "lxd-container"
728
729 @property
730 def pycloudlib_cls(self):
731 """Return the pycloudlib cls to be used as an api."""
732 return pycloudlib.LXDContainer
733
734 def manage_ssh_key(self, private_key_path: "Optional[str]" = None) -> None:
735 """Create and manage ssh key pairs to be used in the cloud provider.
736
737 On a LXD container, we do not use ssh keys to communicate with the
738 instance. Therefore, this method should not be used.
739
740 :param private_key_path:
741 Location of the private key path to use. If None, the location
742 will be a default location.
743 """
744 pass
diff --git a/features/environment.py b/features/environment.py
index 54d00eb..9a53933 100644
--- a/features/environment.py
+++ b/features/environment.py
@@ -1,13 +1,70 @@
1import datetime1import datetime
2import errno
2import os3import os
3import subprocess4import itertools
4from typing import Dict, Union5import tempfile
6import textwrap
7import logging
8import pycloudlib # type: ignore
59
6from behave.model import Scenario10try:
11 from typing import Dict, Optional, Union, List, Tuple, Any # noqa: F401
12except ImportError:
13 # typing isn't available on trusty, so ignore its absence
14 pass
15
16from behave.model import Feature, Scenario
717
8from behave.runner import Context18from behave.runner import Context
919
10from features.util import launch_lxd_container, lxc_exec20import features.cloud as cloud
21
22from features.util import emit_spinner_on_travis, lxc_get_property, build_debs
23
24ALL_SUPPORTED_SERIES = ["bionic", "focal", "trusty", "xenial"]
25
26DAILY_PPA = "http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu"
27DAILY_PPA_KEYID = "8A295C4FB8B190B7"
28
29USERDATA_BLOCK_AUTO_ATTACH_IMG = """\
30#cloud-config
31bootcmd:
32 - cp /usr/bin/ua /usr/bin/ua.orig
33 - 'echo "#!/bin/sh\ndate >> /root/ua-calls\n" > /usr/bin/ua'
34 - chmod 755 /usr/bin/ua
35"""
36
37USERDATA_APT_SOURCE_PPA_TRUSTY = """\
38apt_sources: # for trusty
39 - source: deb {ppa_url} $RELEASE main
40 keyid: {ppa_keyid}
41packages: [{packages}]
42"""
43
44USERDATA_APT_SOURCE_PPA = """\
45apt:
46 sources:
47 ua-tools-ppa:
48 source: deb {ppa_url} $RELEASE main
49 keyid: {ppa_keyid}
50packages: [{packages}]
51"""
52
53PROCESS_LOG_TMPL = """\
54returncode: {}
55stdout:
56{stdout}
57stderr:
58{stderr}
59"""
60
61PROCESS_LOG_TMPL = """\
62returncode: {returncode}
63stdout:
64{stdout}
65stderr:
66{stderr}
67"""
1168
1269
13class UAClientBehaveConfig:70class UAClientBehaveConfig:
@@ -19,16 +76,32 @@ class UAClientBehaveConfig:
1976
20 :param contract_token:77 :param contract_token:
21 A valid contract token to use during attach scenarios78 A valid contract token to use during attach scenarios
79 :param contract_token_staging:
80 A valid staging contract token to use during attach scenarios
22 :param image_clean:81 :param image_clean:
23 This indicates whether the image created for this test run should be82 This indicates whether the image created for this test run should be
24 cleaned up when all tests are complete.83 cleaned up when all tests are complete.
84 :param machine_type:
85 The default machine_type to test: lxd.container, lxd.vm, azure.pro,
86 azure.generic, aws.pro or aws.generic
87 :param private_key_file:
88 Optional path to pre-existing private key file to use when connecting
89 launched VMs via ssh.
90 :param private_key_name:
91 Optional name of the cloud's named private key object to use when
92 connecting to launched VMs via ssh. Default: uaclient-integration.
25 :param reuse_image:93 :param reuse_image:
26 A string with an image name that should be used instead of building a94 A string with an image name that should be used instead of building a
27 fresh image for this test run. If specified, image_clean will be set95 fresh image for this test run. If specified, this image will not be
28 to False.96 deleted.
29 :param destroy_instances:97 :param destroy_instances:
30 This boolean indicates that test containers should be destroyed after98 This boolean indicates that test containers should be destroyed after
31 the completion. Set to False to leave instances running.99 the completion. Set to False to leave instances running.
100 :param debs_path:
101 Location of the debs to be used when lauching a focal integration
102 test. If that path is None, we will build those debs locally.
103 :param artifact_dir:
104 Location where test artifacts are emitted.
32 """105 """
33106
34 prefix = "UACLIENT_BEHAVE_"107 prefix = "UACLIENT_BEHAVE_"
@@ -36,48 +109,200 @@ class UAClientBehaveConfig:
36 # These variables are used in .from_environ() to convert the string109 # These variables are used in .from_environ() to convert the string
37 # environment variable input to the appropriate Python types for use within110 # environment variable input to the appropriate Python types for use within
38 # the test framework111 # the test framework
39 boolean_options = ["image_clean", "destroy_instances"]112 boolean_options = ["build_pr", "image_clean", "destroy_instances"]
40 str_options = ["contract_token", "reuse_image"]113 str_options = [
41 redact_options = ["contract_token"]114 "aws_access_key_id",
115 "aws_secret_access_key",
116 "az_client_id",
117 "az_client_secret",
118 "az_tenant_id",
119 "az_subscription_id",
120 "gcp_credentials_path",
121 "gcp_project",
122 "contract_token",
123 "contract_token_staging",
124 "machine_type",
125 "private_key_file",
126 "private_key_name",
127 "reuse_image",
128 "debs_path",
129 "artifact_dir",
130 "ppa",
131 "ppa_keyid",
132 ]
133 redact_options = [
134 "aws_access_key_id",
135 "aws_secret_access_key",
136 "az_client_id",
137 "az_client_secret",
138 "az_tenant_id",
139 "az_subscription_id",
140 "contract_token",
141 "contract_token_staging",
142 ]
42143
43 # This variable is used in .from_environ() but also to emit the "Config144 # This variable is used in .from_environ() but also to emit the "Config
44 # options" stanza in __init__145 # options" stanza in __init__
45 all_options = boolean_options + str_options146 all_options = boolean_options + str_options
147 cloud_api = None # type: pycloudlib.cloud.BaseCloud
148 cloud_manager = None # type: cloud.Cloud
46149
47 def __init__(150 def __init__(
48 self,151 self,
49 *,152 *,
153 aws_access_key_id: str = None,
154 aws_secret_access_key: str = None,
155 az_client_id: str = None,
156 az_client_secret: str = None,
157 az_tenant_id: str = None,
158 az_subscription_id: str = None,
159 gcp_credentials_path: str = None,
160 gcp_project: str = None,
161 build_pr: bool = False,
50 image_clean: bool = True,162 image_clean: bool = True,
51 destroy_instances: bool = True,163 destroy_instances: bool = True,
164 machine_type: str = "lxd.container",
165 private_key_file: str = None,
166 private_key_name: str = "uaclient-integration",
52 reuse_image: str = None,167 reuse_image: str = None,
53 contract_token: str = None168 contract_token: str = None,
169 contract_token_staging: str = None,
170 debs_path: str = None,
171 artifact_dir: str = None,
172 ppa: str = DAILY_PPA,
173 ppa_keyid: str = DAILY_PPA_KEYID,
174 cmdline_tags: "List" = []
54 ) -> None:175 ) -> None:
55 # First, store the values we've detected176 # First, store the values we've detected
177 self.aws_access_key_id = aws_access_key_id
178 self.aws_secret_access_key = aws_secret_access_key
179 self.az_client_id = az_client_id
180 self.az_client_secret = az_client_secret
181 self.az_tenant_id = az_tenant_id
182 self.az_subscription_id = az_subscription_id
183 self.gcp_credentials_path = gcp_credentials_path
184 self.gcp_project = gcp_project
185 self.build_pr = build_pr
56 self.contract_token = contract_token186 self.contract_token = contract_token
187 self.contract_token_staging = contract_token_staging
57 self.image_clean = image_clean188 self.image_clean = image_clean
58 self.destroy_instances = destroy_instances189 self.destroy_instances = destroy_instances
190 self.machine_type = machine_type
191 self.private_key_file = private_key_file
192 self.private_key_name = private_key_name
59 self.reuse_image = reuse_image193 self.reuse_image = reuse_image
60194 self.cmdline_tags = cmdline_tags
195 self.debs_path = debs_path
196 self.artifact_dir = artifact_dir
197 self.ppa = ppa
198 self.ppa_keyid = ppa_keyid
199 self.filter_series = set(
200 [
201 tag.split(".")[1]
202 for tag in cmdline_tags
203 if tag.startswith("series.") and "series.all" not in tag
204 ]
205 )
61 # Next, perform any required validation206 # Next, perform any required validation
62 if self.reuse_image is not None:207 if self.reuse_image is not None:
63 if self.image_clean:208 if self.image_clean:
64 print("reuse_image specified, setting image_clean = False")209 print(" Reuse_image specified, it will not be deleted.")
65 self.image_clean = False210
211 ignore_vars = () # type: Tuple[str, ...]
212 if "aws" not in self.machine_type:
213 ignore_vars += cloud.EC2.env_vars
214 if "azure" not in self.machine_type:
215 ignore_vars += cloud.Azure.env_vars
216 if "gcp" not in self.machine_type:
217 ignore_vars += cloud.GCP.env_vars
218 if "pro" in self.machine_type:
219 ignore_vars += (
220 "UACLIENT_BEHAVE_CONTRACT_TOKEN",
221 "UACLIENT_BEHAVE_CONTRACT_TOKEN_STAGING",
222 )
223 for env_name in ignore_vars:
224 attr_name = env_name.replace("UACLIENT_BEHAVE_", "").lower()
225 if getattr(self, attr_name):
226 print(
227 " --- Ignoring {} because machine_type is {}".format(
228 env_name, self.machine_type
229 )
230 )
231 setattr(self, attr_name, None)
232 timed_job_tag = datetime.datetime.utcnow().strftime(
233 "uaclient-ci-%m%d-%H%M-"
234 )
235 # Jenkinsfile provides us with UACLIENT_BEHAVE_JENKINS_BUILD_TAG
236 job_suffix = os.environ.get("UACLIENT_BEHAVE_JENKINS_BUILD_TAG")
237 print("--- job suffix: {}".format(job_suffix), flush=True)
238 if not job_suffix:
239 job_suffix = os.environ.get("CHANGE_ID", "dev")
240 else:
241 job_suffix = job_suffix.split("PR-")[-1]
242 timed_job_tag += str(job_suffix)
243 timed_job_tag = timed_job_tag.replace(".", "-")
244 if "aws" in self.machine_type:
245 self.cloud_manager = cloud.EC2(
246 aws_access_key_id,
247 aws_secret_access_key,
248 region=os.environ.get("AWS_DEFAULT_REGION", "us-east-2"),
249 machine_type=self.machine_type,
250 tag=timed_job_tag,
251 timestamp_suffix=False,
252 )
253 elif "azure" in self.machine_type:
254 self.cloud_manager = cloud.Azure(
255 az_client_id=az_client_id,
256 az_client_secret=az_client_secret,
257 az_tenant_id=az_tenant_id,
258 az_subscription_id=az_subscription_id,
259 machine_type=self.machine_type,
260 tag=timed_job_tag,
261 timestamp_suffix=False,
262 )
263 elif "gcp" in self.machine_type:
264 self.cloud_manager = cloud.GCP(
265 machine_type=self.machine_type,
266 tag=timed_job_tag,
267 timestamp_suffix=False,
268 gcp_credentials_path=self.gcp_credentials_path,
269 gcp_project=gcp_project,
270 )
271 elif "lxd.vm" in self.machine_type:
272 self.cloud_manager = cloud.LXDVirtualMachine(
273 machine_type=self.machine_type
274 )
275 else:
276 self.cloud_manager = cloud.LXDContainer(
277 machine_type=self.machine_type
278 )
279
280 self.cloud_api = self.cloud_manager.api
66281
67 # Finally, print the config options. This helps users debug the use of282 # Finally, print the config options. This helps users debug the use of
68 # config options, and means they'll be included in test logs in CI.283 # config options, and means they'll be included in test logs in CI.
69 print("Config options:")284 print("Config options:")
70 for option in self.all_options:285 for option in self.all_options:
71 value = getattr(self, option, "ERROR")286 value = getattr(self, option, "<UNSET>")
72 if option in self.redact_options and value not in (None, "ERROR"):287 if option in self.redact_options and value not in (
288 None,
289 "<UNSET>",
290 ):
73 value = "<REDACTED>"291 value = "<REDACTED>"
74 print(" {}".format(option), "=", value)292 print(" {}".format(option), "=", value)
75293
76 @classmethod294 @classmethod
77 def from_environ(cls) -> "UAClientBehaveConfig":295 def from_environ(cls, config) -> "UAClientBehaveConfig":
78 """Gather config options from os.environ and return a config object"""296 """Gather config options from os.environ and return a config object"""
79 # First, gather all known options297 # First, gather all known options
80 kwargs: Dict[str, Union[str, bool]] = {}298 kwargs: Dict[str, Union[str, bool, "List"]] = {}
299 # Preserve cmdline_tags for reference
300 if not config.tags.ands:
301 kwargs["cmdline_tags"] = []
302 else:
303 kwargs["cmdline_tags"] = list(
304 itertools.chain.from_iterable(config.tags.ands)
305 )
81 for key, value in os.environ.items():306 for key, value in os.environ.items():
82 if not key.startswith(cls.prefix):307 if not key.startswith(cls.prefix):
83 continue308 continue
@@ -98,96 +323,440 @@ class UAClientBehaveConfig:
98323
99324
100def before_all(context: Context) -> None:325def before_all(context: Context) -> None:
101 """behave will invoke this before anything else happens.326 """behave will invoke this before anything else happens."""
102327 context.config.setup_logging()
103 In this function, we launch a container, install ubuntu-advantage-tools and
104 then capture an image. This image is then reused by each feature, reducing
105 test execution time.
106 """
107 userdata = context.config.userdata328 userdata = context.config.userdata
108 context.reuse_container = userdata.get("reuse_container")329 if userdata:
109 context.config = UAClientBehaveConfig.from_environ()330 logging.debug("Userdata key / value pairs:")
110 if context.config.reuse_image is None:331 print("Userdata key / value pairs:")
111 create_trusty_uat_lxd_image(context)332 for key, value in userdata.items():
112 else:333 logging.debug(" - {} = {}".format(key, value))
113 context.image_name = context.config.reuse_image334 print(" - {} = {}".format(key, value))
114335 context.series_image_name = {}
115336 context.series_reuse_image = ""
116def before_scenario(context: Context, scenario: Scenario):337 context.reuse_container = {}
117 for tag in scenario.effective_tags:338 context.config = UAClientBehaveConfig.from_environ(context.config)
339 context.config.cloud_manager.manage_ssh_key()
340
341 if context.config.reuse_image:
342 series = lxc_get_property(
343 context.config.reuse_image, property_name="series", image=True
344 )
345 machine_type = lxc_get_property(
346 context.config.reuse_image,
347 property_name="machine_type",
348 image=True,
349 )
350 if machine_type:
351 print("Found machine_type: {vm_type}".format(vm_type=machine_type))
352 if series is not None:
353 context.series_reuse_image = series
354 context.series_image_name[series] = context.config.reuse_image
355 else:
356 print(" Could not check image series. It will not be used. ")
357 context.config.reuse_image = None
358
359 if userdata.get("reuse_container"):
360 inst = context.config.cloud_api.get_instance(
361 userdata.get("reuse_container")
362 )
363 codename = inst.execute(
364 ["grep", "UBUNTU_CODENAME", "/etc/os-release"]
365 ).strip()
366 [_, series] = codename.split("=")
367
368 context.reuse_container = {series: userdata.get("reuse_container")}
369 print(
370 textwrap.dedent(
371 """
372 You are providing a {series} container. Make sure you are running
373 this series tests. For instance: --tags=series.{series}""".format(
374 series=series
375 )
376 )
377 )
378
379
380def _should_skip_tags(context: Context, tags: "List") -> str:
381 """Return a reason if a feature or scenario should be skipped"""
382 machine_type = getattr(context.config, "machine_type", "")
383 machine_types = []
384
385 for tag in tags:
118 parts = tag.split(".")386 parts = tag.split(".")
119 if parts[0] == "uses":387 if parts[0] != "uses":
120 val = context388 continue # Only process @uses.* tags for skipping:
121 for attr in parts[1:]:389 val = context
122 val = getattr(val, attr, None)390 for idx, attr in enumerate(parts[1:], 1):
123 if val is None:391 val = getattr(val, attr, None)
124 scenario.skip(392 if attr == "machine_type":
125 reason="Skipped because tag value was None: {}".format(393 curr_machine_type = ".".join(parts[idx + 1 :])
126 tag394 machine_types.append(curr_machine_type)
395 if curr_machine_type == machine_type:
396 if machine_type.startswith("lxd"):
397 return ""
398
399 cloud_manager = context.config.cloud_manager
400 if cloud_manager and cloud_manager.missing_env_vars():
401 return "".join(
402 (
403 "Skipped: {} machine_type requires:\n".format(
404 machine_type
405 ),
406 *cloud_manager.format_missing_env_vars(
407 cloud_manager.missing_env_vars()
408 ),
409 )
127 )410 )
128 )411 return ""
412 break
413 if val is None:
414 return "Skipped: tag value was None: {}".format(tag)
415
416 if machine_types:
417 return "Skipped: machine type {} was not found in tags:\n {}".format(
418 machine_type, ", ".join(machine_types)
419 )
420
421 return ""
422
129423
424def before_feature(context: Context, feature: Feature):
425 reason = _should_skip_tags(context, feature.tags)
426 if reason:
427 feature.skip(reason=reason)
130428
131def _capture_container_as_image(container_name: str, image_name: str) -> None:429
132 """Capture a lxd container as an image.430def before_scenario(context: Context, scenario: Scenario):
431 """
432 In this function, we launch a container, install ubuntu-advantage-tools and
433 then capture an image. This image is then reused by each scenario, reducing
434 test execution time.
435 """
436 reason = _should_skip_tags(context, scenario.effective_tags)
437 if reason:
438 scenario.skip(reason=reason)
439
440
441FAILURE_FILES = (
442 "/etc/ubuntu-advantage/uaclient.log",
443 "/var/log/cloud-init.log",
444 "/var/log/ubuntu-advantage.log",
445 "/var/lib/cloud/instance/user-data.txt",
446 "/var/lib/cloud/instance/vendor-data.txt",
447)
448FAILURE_CMDS = {
449 "ua-version": ["ua", "version"],
450 "cloud-init-analyze": ["cloud-init", "analyze", "show"],
451 "cloud-init.status": ["cloud-init", "status", "--long"],
452 "status.json": ["ua", "status", "--all", "--format=json"],
453 "journal.log": ["journalctl", "-b", "0"],
454 "systemd-analyze-blame": ["systemd-analyze", "blame"],
455 "systemctl-status": ["systemctl", "status"],
456 "systemctl-status-ua-auto-attach": [
457 "systemctl",
458 "status",
459 "ua-auto-attach.service",
460 ],
461 "systemctl-status-ua-reboot-cmds": [
462 "systemctl",
463 "status",
464 "ua-reboot-cmds.service",
465 ],
466}
467
468
469def after_step(context, step):
470 """Collect test artifacts in the event of failure."""
471 if step.status == "failed":
472 if context.config.artifact_dir:
473 artifacts_dir = context.config.artifact_dir
474 else:
475 artifacts_dir = "artifacts"
476 artifacts_dir = os.path.join(
477 artifacts_dir,
478 "{}_{}".format(os.path.basename(step.filename), step.line),
479 )
480 if hasattr(context, "process"):
481 if not os.path.exists(artifacts_dir):
482 os.makedirs(artifacts_dir)
483 artifact_file = os.path.join(artifacts_dir, "process.log")
484 process = context.process
485 with open(artifact_file, "w") as stream:
486 stream.write(
487 PROCESS_LOG_TMPL.format(
488 returncode=process.returncode,
489 stdout=process.stdout,
490 stderr=process.stderr,
491 )
492 )
493
494 if hasattr(context, "instance"):
495 if not os.path.exists(artifacts_dir):
496 os.makedirs(artifacts_dir)
497 for log_file in FAILURE_FILES:
498 artifact_file = os.path.join(
499 artifacts_dir, os.path.basename(log_file)
500 )
501 print("-- pull instance:{} {}".format(log_file, artifact_file))
502 try:
503 context.instance.pull_file(log_file, artifact_file)
504 except IOError as e:
505 if e.errno == errno.EACCES:
506 result = context.instance.execute(
507 ["cat", log_file], use_sudo=True
508 )
509 with open(artifact_file, "w") as stream:
510 stream.write(result.stdout)
511 except RuntimeError:
512 # File did not exist
513 with open(artifact_file, "w") as stream:
514 stream.write("")
515 for artifact_file, cmd in FAILURE_CMDS.items():
516 result = context.instance.execute(cmd, use_sudo=True)
517 artifact_file = os.path.join(artifacts_dir, artifact_file)
518 with open(artifact_file, "w") as stream:
519 stream.write(result.stdout)
520
521
522def after_all(context):
523 if context.config.image_clean:
524 for key, image in context.series_image_name.items():
525 if key == context.series_reuse_image:
526 print(
527 " Not deleting this image: ",
528 context.series_image_name[key],
529 )
530 else:
531 context.config.cloud_api.delete_image(image)
532
533
534def _capture_container_as_image(
535 container_name: str,
536 image_name: str,
537 cloud_api: "pycloudlib.cloud.BaseCloud",
538) -> str:
539 """Capture a container as an image.
133540
134 :param container_name:541 :param container_name:
135 The name of the container to be captured. Note that this container542 The name of the container to be captured. Note that this container
136 will be stopped.543 will be stopped.
137 :param image_name:544 :param image_name:
138 The name under which the image should be published.545 The name under which the image should be published.
546 :param cloud_api: Optional pycloud BaseCloud api for applicable
547 machine_types.
548 """
549 print(
550 "--- Creating base image snapshot from vm {}".format(container_name)
551 )
552 inst = cloud_api.get_instance(container_name)
553 return cloud_api.snapshot(instance=inst)
554
555
556def build_debs_from_dev_instance(context: Context, series: str) -> "List[str]":
557 """Create a development instance, instal build dependencies and build debs
558
559
560 Will stop the development instance after deb build succeeds.
561
562 :return: A list of paths to applicable deb files published.
139 """563 """
140 subprocess.run(["lxc", "stop", container_name])564 time_suffix = datetime.datetime.now().strftime("%s%f")
141 subprocess.run(["lxc", "publish", container_name, "--alias", image_name])565 deb_paths = []
566
567 if context.config.debs_path:
568 print(
569 "--- Checking if debs can be reused in {}".format(
570 context.config.debs_path
571 )
572 )
573 debs_path = context.config.debs_path
574 if os.path.isdir(debs_path):
575 deb_paths = [
576 os.path.join(debs_path, deb_file)
577 for deb_file in os.listdir(debs_path)
578 if series in deb_file
579 ]
580
581 if len(deb_paths):
582 print("--- Reusing debs: {}".format(", ".join(deb_paths)))
583 else:
584 print("--- Could not find any debs to reuse. Building it locally")
585 print(
586 "--- Launching vm to build ubuntu-advantage*debs from local source"
587 )
588 build_container_name = (
589 "ubuntu-behave-image-pre-build-%s-" % series + time_suffix
590 )
591
592 cloud_manager = context.config.cloud_manager
593 if "pro" in context.config.machine_type:
594 user_data = USERDATA_BLOCK_AUTO_ATTACH_IMG
595 else:
596 user_data = ""
597 inst = cloud_manager.launch(
598 instance_name=build_container_name,
599 series=series,
600 user_data=user_data,
601 )
602
603 build_container_name = cloud_manager.get_instance_id(inst)
142604
605 with emit_spinner_on_travis("Building debs from local source... "):
606 deb_paths = build_debs(
607 build_container_name,
608 output_deb_dir=os.path.join(tempfile.gettempdir(), series),
609 cloud_api=context.config.cloud_api,
610 )
143611
144def create_trusty_uat_lxd_image(context: Context) -> None:612 if "pro" in context.config.machine_type:
145 """Create a trusty lxd image with ubuntu-advantage-tools installed613 return deb_paths
614 # Redact ubuntu-advantage-pro deb as inapplicable
615 return [deb_path for deb_path in deb_paths if "pro" not in deb_path]
616
617
618def create_uat_image(context: Context, series: str) -> None:
619 """Create a given series lxd image with ubuntu-advantage-tools installed
146620
147 This will launch a container, install ubuntu-advantage-tools, and publish621 This will launch a container, install ubuntu-advantage-tools, and publish
148 the image. The image's name is stored in context.image_name for use within622 the image. The image's name is stored in context.series_image_name for
149 step code.623 use within step code.
150624
151 :param context:625 :param context:
152 A `behave.runner.Context`; this will have `image_name` set on it.626 A `behave.runner.Context`; this will have `series.image_name` set on
627 it.
628 :param series:
629 A string representing the series name to create
153 """630 """
154631
155 def image_cleanup() -> None:632 if series in context.reuse_container:
156 if context.config.image_clean:633 print(
157 subprocess.run(["lxc", "image", "delete", context.image_name])634 "\n Reusing the existing container: ",
635 context.reuse_container[series],
636 )
637 return
638 time_suffix = datetime.datetime.now().strftime("%s%f")
639 deb_paths = []
640 if context.config.build_pr:
641 deb_paths = build_debs_from_dev_instance(context, series)
642
643 print(
644 "--- Launching VM to create a base image with updated ubuntu-advantage"
645 )
646
647 is_vm = bool(context.config.machine_type == "lxd.vm")
648 build_container_name = "ubuntu-behave-image-build-%s-%s" % (
649 "-vm" if is_vm else "",
650 series + time_suffix,
651 )
652
653 user_data = ""
654 if "pro" in context.config.machine_type:
655 user_data = USERDATA_BLOCK_AUTO_ATTACH_IMG
656 if not deb_paths:
657 if not user_data:
658 user_data = "#cloud-config\n"
659 ppa = context.config.ppa
660 ppa_keyid = context.config.ppa_keyid
661 if context.config.ppa.startswith("ppa:"):
662 ppa = ppa.replace("ppa:", "http://ppa.launchpad.net/") + "/ubuntu"
663 if series == "trusty":
664 packages = ["ubuntu-advantage-tools"]
665
666 if "pro" in context.config.machine_type:
667 packages.append("ubuntu-advantage-pro")
668
669 user_data += USERDATA_APT_SOURCE_PPA_TRUSTY.format(
670 ppa_url=ppa, ppa_keyid=ppa_keyid, packages=", ".join(packages)
671 )
158 else:672 else:
159 print("Image cleanup disabled, not deleting:", context.image_name)673 packages = ["openssh-server", "ubuntu-advantage-tools"]
160674
161 if context.reuse_container:675 if "pro" in context.config.machine_type:
162 print(" Reusing the existent container: ", context.reuse_container)676 packages.append("ubuntu-advantage-pro")
163 else:
164 now = datetime.datetime.now()
165 context.image_name = "behave-image-" + now.strftime("%s%f")
166 build_container_name = "behave-image-build-" + now.strftime("%s%f")
167 launch_lxd_container(context, "ubuntu:trusty", build_container_name)
168 _install_uat_in_container(build_container_name)
169 _capture_container_as_image(build_container_name, context.image_name)
170 context.add_cleanup(image_cleanup)
171677
678 user_data += USERDATA_APT_SOURCE_PPA.format(
679 ppa_url=ppa, ppa_keyid=ppa_keyid, packages=", ".join(packages)
680 )
681 inst = context.config.cloud_manager.launch(
682 instance_name=build_container_name, series=series, user_data=user_data
683 )
684 build_container_name = context.config.cloud_manager.get_instance_id(inst)
172685
173def _install_uat_in_container(container_name: str) -> None:686 _install_uat_in_container(
687 build_container_name,
688 series=series,
689 config=context.config,
690 deb_paths=deb_paths,
691 )
692
693 image_name = _capture_container_as_image(
694 build_container_name,
695 image_name="ubuntu-behave-image-%s-" % series + time_suffix,
696 cloud_api=context.config.cloud_api,
697 )
698 context.series_image_name[series] = image_name
699 inst.delete(wait=False)
700
701
702def _install_uat_in_container(
703 container_name: str,
704 series: str,
705 config: UAClientBehaveConfig,
706 deb_paths: "Optional[List[str]]" = None,
707) -> None:
174 """Install ubuntu-advantage-tools into the specified container708 """Install ubuntu-advantage-tools into the specified container
175709
176 :param container_name:710 :param container_name:
177 The name of the container into which ubuntu-advantage-tools should be711 The name of the container into which ubuntu-advantage-tools should be
178 installed.712 installed.
713 :param series: The name of the series that is being used
714 :param config: UAClientBehaveConfig
715 :param deb_paths: Optional paths to local deb files we need to install
179 """716 """
180 lxc_exec(717 cmds: "List[Any]" = [["systemctl", "is-system-running", "--wait"]]
181 container_name,718
182 [719 if deb_paths is None:
183 "sudo",720 deb_paths = []
184 "add-apt-repository",721
185 "--yes",722 if deb_paths:
186 "ppa:canonical-server/ua-client-daily",723 cmds.append(["sudo", "apt-get", "update", "-qqy"])
187 ],724
188 )725 deb_files = []
189 lxc_exec(container_name, ["sudo", "apt-get", "update", "-qq"])726 inst = config.cloud_api.get_instance(container_name)
190 lxc_exec(727
191 container_name,728 for deb_file in deb_paths:
192 ["sudo", "apt-get", "install", "-qq", "-y", "ubuntu-advantage-tools"],729 if "pro" in deb_file and "pro" not in config.machine_type:
193 )730 continue
731
732 deb_name = os.path.basename(deb_file)
733 deb_files.append("/tmp/" + deb_name)
734 inst.push_file(deb_file, "/tmp/" + deb_name)
735
736 if series == "trusty":
737 cmds.append(["sudo", "dpkg", "-i"] + deb_files)
738 else:
739 cmds.append(["sudo", "apt-get", "install", "-y"] + deb_files)
740
741 if "pro" in config.machine_type:
742 features = "features:\n disable_auto_attach: true\n"
743 conf_path = "/etc/ubuntu-advantage/uaclient.conf"
744 cmd = "printf '{}' > /tmp/uaclient.conf".format(features)
745 cmds.append('sh -c "{}"'.format(cmd))
746 cmds.append(
747 'sudo -- sh -c "cat /tmp/uaclient.conf >> {}"'.format(conf_path)
748 )
749 cmds.append("sudo ua detach --assume-yes")
750
751 cmds.append(["ua", "version"])
752 instance = config.cloud_api.get_instance(container_name)
753 for cmd in cmds: # type: ignore
754 result = instance.execute(cmd)
755 if result.failed:
756 print(
757 "--- Failed {}: out {} err {}".format(
758 cmd, result.stdout, result.stderr
759 )
760 )
761 elif "version" in cmd:
762 print("--- " + result)
diff --git a/features/gcp-ids.yaml b/features/gcp-ids.yaml
194new file mode 100644763new file mode 100644
index 0000000..a3ca5bd
--- /dev/null
+++ b/features/gcp-ids.yaml
@@ -0,0 +1,3 @@
1xenial: "projects/ubuntu-os-cloud-image-proposed/global/images/testing-ubuntu-pro-1604-xenial-v20210205"
2bionic: "projects/ubuntu-os-cloud-image-proposed/global/images/testing-ubuntu-pro-1804-bionic-v20210205"
3focal: "projects/ubuntu-os-cloud-image-proposed/global/images/testing-ubuntu-pro-2004-focal-v20210205"
diff --git a/features/staging_commands.feature b/features/staging_commands.feature
0new file mode 1006444new file mode 100644
index 0000000..90c212f
--- /dev/null
+++ b/features/staging_commands.feature
@@ -0,0 +1,395 @@
1@uses.config.contract_token_staging
2Feature: Enable command behaviour when attached to an UA staging subscription
3
4 @series.xenial
5 Scenario: Attached enable CC EAL service in a xenial lxd container
6 Given a `xenial` machine with ubuntu-advantage-tools installed
7 When I attach `contract_token_staging` with sudo
8 Then I verify that running `ua enable cc-eal` `as non-root` exits `1`
9 And I will see the following on stderr:
10 """
11 This command must be run as root (try using sudo)
12 """
13 When I run `ua enable cc-eal --beta` with sudo
14 Then I will see the following on stdout:
15 """
16 One moment, checking your subscription first
17 GPG key '/usr/share/keyrings/ubuntu-cc-keyring.gpg' not found
18 """
19 @series.xenial
20 @series.bionic
21 @series.focal
22 Scenario Outline: Attached enable esm-apps on a machine
23 Given a `<release>` machine with ubuntu-advantage-tools installed
24 When I attach `contract_token_staging` with sudo
25 And I run `ua status --all` as non-root
26 Then stdout matches regexp
27 """
28 esm-apps yes enabled UA Apps: Extended Security Maintenance \(ESM\)
29 """
30 And I verify that running `apt update` `with sudo` exits `0`
31 When I run `apt-cache policy` as non-root
32 Then apt-cache policy for the following url has permission `500`
33 """
34 https://esm.staging.ubuntu.com/apps/ubuntu <release>-apps-updates/main amd64 Packages
35 """
36 And apt-cache policy for the following url has permission `500`
37 """
38 https://esm.staging.ubuntu.com/apps/ubuntu <release>-apps-security/main amd64 Packages
39 """
40 And I verify that running `apt update` `with sudo` exits `0`
41 When I run `apt install -y <apps-pkg>` with sudo, retrying exit [100]
42 And I run `apt-cache policy <apps-pkg>` as non-root
43 Then stdout matches regexp:
44 """
45 Version table:
46 \s*\*\*\* .* 500
47 \s*500 https://esm.staging.ubuntu.com/apps/ubuntu <release>-apps-security/main amd64 Packages
48 """
49
50 Examples: ubuntu release
51 | release | apps-pkg |
52 | bionic | bundler |
53 | focal | ant |
54 | trusty | ant |
55 | xenial | jq |
56
57 @series.xenial
58 @series.bionic
59 @uses.config.machine_type.lxd.vm
60 Scenario Outline: Attached enable of vm-based services in an ubuntu lxd vm
61 Given a `<release>` machine with ubuntu-advantage-tools installed
62 When I attach `contract_token_staging` with sudo
63 And I run `ua disable livepatch` with sudo
64 And I run `apt-get install openssh-client openssh-server strongswan -y` with sudo, retrying exit [100]
65 And I run `apt-mark hold openssh-client openssh-server strongswan` with sudo
66 And I run `ua enable <fips-service> --assume-yes` with sudo
67 Then stdout matches regexp:
68 """
69 Updating package lists
70 Installing <fips-name> packages
71 <fips-name> enabled
72 A reboot is required to complete install
73 """
74 When I run `ua status --all` with sudo
75 Then stdout matches regexp:
76 """
77 <fips-service> +yes enabled
78 """
79 And stdout matches regexp:
80 """
81 FIPS support requires system reboot to complete configuration
82 """
83 And I verify that running `apt update` `with sudo` exits `0`
84 And I verify that running `grep Traceback /var/log/ubuntu-advantage.log` `with sudo` exits `1`
85 And I verify that `openssh-server` is installed from apt source `<fips-apt-source>`
86 And I verify that `openssh-client` is installed from apt source `<fips-apt-source>`
87 And I verify that `strongswan` is installed from apt source `<fips-apt-source>`
88 And I verify that `openssh-server-hmac` is installed from apt source `<fips-apt-source>`
89 And I verify that `openssh-client-hmac` is installed from apt source `<fips-apt-source>`
90 And I verify that `strongswan-hmac` is installed from apt source `<fips-apt-source>`
91 When I reboot the `<release>` machine
92 And I run `uname -r` as non-root
93 Then stdout matches regexp:
94 """
95 fips
96 """
97 When I run `cat /proc/sys/crypto/fips_enabled` with sudo
98 Then I will see the following on stdout:
99 """
100 1
101 """
102 When I run `ua status --all` with sudo
103 Then stdout does not match regexp:
104 """
105 FIPS support requires system reboot to complete configuration
106 """
107 When I run `ua disable <fips-service> --assume-yes` with sudo
108 Then stdout matches regexp:
109 """
110 Updating package lists
111 A reboot is required to complete disable operation
112 """
113 When I run `ua status --all` with sudo
114 Then stdout matches regexp:
115 """
116 Disabling FIPS requires system reboot to complete operation
117 """
118 When I run `apt-cache policy ubuntu-fips` as non-root
119 Then stdout matches regexp:
120 """
121 .*Installed: \(none\)
122 """
123 When I reboot the `<release>` machine
124 Then I verify that `openssh-server` installed version matches regexp `fips`
125 And I verify that `openssh-client` installed version matches regexp `fips`
126 And I verify that `strongswan` installed version matches regexp `fips`
127 And I verify that `openssh-server-hmac` installed version matches regexp `fips`
128 And I verify that `openssh-client-hmac` installed version matches regexp `fips`
129 And I verify that `strongswan-hmac` installed version matches regexp `fips`
130 When I run `apt-mark unhold openssh-client openssh-server strongswan` with sudo
131 Then I will see the following on stdout:
132 """
133 openssh-client was already not hold.
134 openssh-server was already not hold.
135 strongswan was already not hold.
136 """
137 When I run `ua status --all` with sudo
138 Then stdout matches regexp:
139 """
140 <fips-service> +yes disabled
141 """
142 Then stdout does not match regexp:
143 """
144 Disabling FIPS requires system reboot to complete operation
145 """
146
147 Examples: ubuntu release
148 | release | fips-name | fips-service |fips-apt-source |
149 | xenial | FIPS | fips |https://esm.staging.ubuntu.com/fips/ubuntu xenial/main |
150 | bionic | FIPS | fips |https://esm.staging.ubuntu.com/fips/ubuntu bionic/main |
151
152 @series.xenial
153 @series.bionic
154 @uses.config.machine_type.lxd.vm
155 Scenario Outline: Attached enable of vm-based services in an ubuntu lxd vm
156 Given a `<release>` machine with ubuntu-advantage-tools installed
157 When I attach `contract_token_staging` with sudo
158 And I run `ua disable livepatch` with sudo
159 And I run `apt-get install openssh-client openssh-server strongswan -y` with sudo, retrying exit [100]
160 When I run `ua enable <fips-service> --assume-yes` with sudo
161 Then stdout matches regexp:
162 """
163 Updating package lists
164 Installing <fips-name> packages
165 <fips-name> enabled
166 A reboot is required to complete install
167 """
168 When I run `ua status --all` with sudo
169 Then stdout matches regexp:
170 """
171 <fips-service> +yes enabled
172 """
173 And I verify that running `apt update` `with sudo` exits `0`
174 And I verify that running `grep Traceback /var/log/ubuntu-advantage.log` `with sudo` exits `1`
175 And I verify that `openssh-server` is installed from apt source `<fips-apt-source>`
176 And I verify that `openssh-client` is installed from apt source `<fips-apt-source>`
177 And I verify that `strongswan` is installed from apt source `<fips-apt-source>`
178 And I verify that `openssh-server-hmac` is installed from apt source `<fips-apt-source>`
179 And I verify that `openssh-client-hmac` is installed from apt source `<fips-apt-source>`
180 And I verify that `strongswan-hmac` is installed from apt source `<fips-apt-source>`
181 When I reboot the `<release>` machine
182 And I run `uname -r` as non-root
183 Then stdout matches regexp:
184 """
185 fips
186 """
187 When I run `cat /proc/sys/crypto/fips_enabled` with sudo
188 Then I will see the following on stdout:
189 """
190 1
191 """
192 When I run `ua disable <fips-service> --assume-yes` with sudo
193 Then stdout matches regexp:
194 """
195 Updating package lists
196 A reboot is required to complete disable operation
197 """
198 When I reboot the `<release>` machine
199 Then I verify that `openssh-server` installed version matches regexp `fips`
200 And I verify that `openssh-client` installed version matches regexp `fips`
201 And I verify that `strongswan` installed version matches regexp `fips`
202 And I verify that `openssh-server-hmac` installed version matches regexp `fips`
203 And I verify that `openssh-client-hmac` installed version matches regexp `fips`
204 And I verify that `strongswan-hmac` installed version matches regexp `fips`
205 When I run `apt-mark unhold openssh-client openssh-server strongswan` with sudo
206 Then I will see the following on stdout:
207 """
208 openssh-client was already not hold.
209 openssh-server was already not hold.
210 strongswan was already not hold.
211 """
212 When I run `ua status --all` with sudo
213 Then stdout matches regexp:
214 """
215 <fips-service> +yes disabled
216 """
217
218 Examples: ubuntu release
219 | release | fips-name | fips-service |fips-apt-source |
220 | xenial | FIPS Updates | fips-updates |https://esm.staging.ubuntu.com/fips-updates/ubuntu xenial-updates/main |
221 | bionic | FIPS Updates | fips-updates |https://esm.staging.ubuntu.com/fips-updates/ubuntu bionic-updates/main |
222
223 @series.xenial
224 @uses.config.machine_type.lxd.vm
225 Scenario Outline: Attached FIPS upgrade across LTS releases
226 Given a `<release>` machine with ubuntu-advantage-tools installed
227 When I attach `contract_token_staging` with sudo
228 And I run `apt-get install lsof` with sudo, retrying exit [100]
229 And I run `ua disable livepatch` with sudo
230 And I run `ua enable <fips-service> --assume-yes` with sudo
231 Then stdout matches regexp:
232 """
233 Updating package lists
234 Installing <fips-name> packages
235 <fips-name> enabled
236 A reboot is required to complete install
237 """
238 When I run `ua status --all` with sudo
239 Then stdout matches regexp:
240 """
241 <fips-service> +yes enabled
242 """
243 And I verify that running `apt update` `with sudo` exits `0`
244 When I reboot the `<release>` machine
245 And I run `uname -r` as non-root
246 Then stdout matches regexp:
247 """
248 fips
249 """
250 When I run `cat /proc/sys/crypto/fips_enabled` with sudo
251 Then I will see the following on stdout:
252 """
253 1
254 """
255 When I run `apt-get dist-upgrade -y --allow-downgrades` with sudo
256 # A package may need a reboot after running dist-upgrade
257 And I reboot the `<release>` machine
258 And I create the file `/etc/update-manager/release-upgrades.d/ua-test.cfg` with the following
259 """
260 [Sources]
261 AllowThirdParty=yes
262 """
263 Then I verify that running `do-release-upgrade --frontend DistUpgradeViewNonInteractive` `with sudo` exits `0`
264 When I reboot the `<release>` machine
265 And I run `lsb_release -cs` as non-root
266 Then I will see the following on stdout:
267 """
268 <next_release>
269 """
270 When I verify that running `egrep "disabled" /etc/apt/sources.list.d/<source-file>.list` `as non-root` exits `1`
271 Then I will see the following on stdout:
272 """
273 """
274 When I run `ua status --all` with sudo
275 Then stdout matches regexp:
276 """
277 <fips-service> +yes enabled
278 """
279 When I run `uname -r` as non-root
280 Then stdout matches regexp:
281 """
282 fips
283 """
284 When I run `cat /proc/sys/crypto/fips_enabled` with sudo
285 Then I will see the following on stdout:
286 """
287 1
288 """
289
290 Examples: ubuntu release
291 | release | next_release | fips-service | fips-name | source-file |
292 | xenial | bionic | fips | FIPS | ubuntu-fips |
293 | xenial | bionic | fips-updates | FIPS Updates | ubuntu-fips-updates |
294
295 @series.xenial
296 @series.bionic
297 Scenario Outline: Attached enable of cis service in a ubuntu machine
298 Given a `<release>` machine with ubuntu-advantage-tools installed
299 When I attach `contract_token_staging` with sudo
300 And I verify that running `ua enable cis --beta` `with sudo` exits `0`
301 Then I will see the following on stdout:
302 """
303 One moment, checking your subscription first
304 Updating package lists
305 Installing CIS Audit packages
306 CIS Audit enabled
307 """
308 When I run `apt-cache policy usg-cisbenchmark` as non-root
309 Then stdout does not match regexp:
310 """
311 .*Installed: \(none\)
312 """
313 And stdout matches regexp:
314 """
315 \s* 500 https://esm.staging.ubuntu.com/cis/ubuntu <release>/main amd64 Packages
316 """
317 When I run `apt-cache policy usg-common` as non-root
318 Then stdout does not match regexp:
319 """
320 .*Installed: \(none\)
321 """
322 And stdout matches regexp:
323 """
324 \s* 500 https://esm.staging.ubuntu.com/cis/ubuntu <release>/main amd64 Packages
325 """
326
327 Examples: not entitled services
328 | release |
329 | bionic |
330 | xenial |
331
332 @series.xenial
333 @series.bionic
334 @uses.config.machine_type.lxd.vm
335 Scenario Outline: Attached enable fips-updates on fips enabled vm
336 Given a `<release>` machine with ubuntu-advantage-tools installed
337 When I attach `contract_token_staging` with sudo
338 And I run `ua disable livepatch` with sudo
339 And I run `apt-get install openssh-client openssh-server strongswan -y` with sudo, retrying exit [100]
340 And I run `ua enable fips --assume-yes` with sudo
341 Then stdout matches regexp:
342 """
343 Updating package lists
344 Installing FIPS packages
345 FIPS enabled
346 A reboot is required to complete install
347 """
348 When I run `ua status --all` with sudo
349 Then stdout matches regexp:
350 """
351 fips +yes enabled
352 """
353 When I reboot the `<release>` machine
354 And I run `ua enable fips-updates --assume-yes` with sudo
355 Then stdout matches regexp:
356 """
357 Updating package lists
358 Installing FIPS Updates packages
359 FIPS Updates enabled
360 A reboot is required to complete install
361 """
362 When I run `ua status --all` with sudo
363 Then stdout matches regexp:
364 """
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches

to status/vote changes: