Merge ~lamoura/ubuntu/+source/ubuntu-advantage-tools:hirsute-devel-release-27 into ubuntu/+source/ubuntu-advantage-tools:ubuntu/hirsute-devel

Proposed by Lucas Albuquerque Medeiros de Moura
Status: Needs review
Proposed branch: ~lamoura/ubuntu/+source/ubuntu-advantage-tools:hirsute-devel-release-27
Merge into: ubuntu/+source/ubuntu-advantage-tools:ubuntu/hirsute-devel
Diff against target: 12025 lines (+8488/-712)
87 files modified
Jenkinsfile (+28/-35)
README.md (+29/-12)
RELEASES.md (+63/-4)
apt-hook/20apt-esm-hook.conf (+10/-2)
apt-hook/Makefile (+11/-4)
apt-hook/hook.cc (+264/-74)
apt-hook/json-hook-src/go.mod (+3/-0)
apt-hook/json-hook-src/json-hook.go (+253/-0)
apt-hook/json-hook-src/json-hook_test.go (+410/-0)
debian/changelog (+158/-1)
debian/control (+8/-2)
debian/lintian-overrides (+5/-0)
debian/po/templates.pot (+2/-2)
debian/rules (+10/-2)
debian/ubuntu-advantage-tools.config (+14/-0)
debian/ubuntu-advantage-tools.postinst (+150/-40)
debian/ubuntu-advantage-tools.templates (+1/-1)
dev/null (+0/-0)
features/attach_invalidtoken.feature (+1/-1)
features/attach_validtoken.feature (+63/-28)
features/attached_commands.feature (+12/-12)
features/attached_enable.feature (+62/-118)
features/cloud.py (+0/-12)
features/environment.py (+27/-17)
features/gcp-ids.yaml (+3/-3)
features/staging_commands.feature (+50/-5)
features/steps/steps.py (+75/-3)
features/ubuntu_pro.feature (+39/-9)
features/unattached_commands.feature (+388/-3)
features/util.py (+0/-81)
integration-requirements.txt (+1/-1)
lib/reboot_cmds.py (+12/-8)
lib/ua_update_messaging.py (+302/-0)
setup.py (+1/-0)
systemd/ua-messaging.service (+8/-0)
systemd/ua-messaging.timer (+11/-0)
tools/test_xenial_upgrade.sh (+224/-0)
tools/tox-lxd-runner (+2/-2)
tox.ini (+2/-2)
uaclient-devel.conf (+1/-0)
uaclient.conf (+1/-0)
uaclient/apt.py (+1/-0)
uaclient/cli.py (+87/-18)
uaclient/clouds/identity.py (+12/-0)
uaclient/clouds/tests/test_identity.py (+31/-0)
uaclient/config.py (+136/-8)
uaclient/contract.py (+1/-2)
uaclient/defaults.py (+10/-1)
uaclient/entitlements/esm.py (+53/-7)
uaclient/entitlements/fips.py (+9/-3)
uaclient/entitlements/livepatch.py (+3/-3)
uaclient/entitlements/repo.py (+3/-3)
uaclient/entitlements/tests/test_base.py (+2/-2)
uaclient/entitlements/tests/test_cc.py (+1/-1)
uaclient/entitlements/tests/test_esm.py (+145/-34)
uaclient/entitlements/tests/test_fips.py (+38/-1)
uaclient/entitlements/tests/test_livepatch.py (+8/-6)
uaclient/entitlements/tests/test_repo.py (+1/-1)
uaclient/exceptions.py (+12/-0)
uaclient/gpg.py (+1/-1)
uaclient/security.py (+1179/-0)
uaclient/serviceclient.py (+19/-2)
uaclient/status.py (+237/-37)
uaclient/testing/fakes.py (+4/-1)
uaclient/tests/test_apt.py (+5/-1)
uaclient/tests/test_cli.py (+2/-0)
uaclient/tests/test_cli_attach.py (+33/-3)
uaclient/tests/test_cli_auto_attach.py (+28/-0)
uaclient/tests/test_cli_detach.py (+31/-2)
uaclient/tests/test_cli_disable.py (+27/-3)
uaclient/tests/test_cli_enable.py (+31/-7)
uaclient/tests/test_cli_fix.py (+78/-0)
uaclient/tests/test_cli_refresh.py (+20/-1)
uaclient/tests/test_cli_status.py (+81/-7)
uaclient/tests/test_config.py (+162/-31)
uaclient/tests/test_contract.py (+1/-8)
uaclient/tests/test_reboot_cmds.py (+48/-1)
uaclient/tests/test_security.py (+2344/-0)
uaclient/tests/test_serviceclient.py (+24/-0)
uaclient/tests/test_status.py (+49/-1)
uaclient/tests/test_ua_update_messaging.py (+469/-0)
uaclient/tests/test_util.py (+237/-9)
uaclient/tests/test_version.py (+10/-3)
uaclient/util.py (+101/-18)
uaclient/version.py (+2/-2)
update-motd.d/88-esm-announce (+4/-0)
update-motd.d/91-contract-ua-esm-status (+4/-0)
Reviewer Review Type Date Requested Status
Bryce Harrington (community) Approve
Review via email: mp+401650@code.launchpad.net

Description of the change

This is a branch used only as a base for reviewing release 27 of uaclient. It brings all the functionality that we are going to release across ubuntu releases (xenial to hirsute).

To reproduce it:

1. Add the main project as upstream:
git remote add upstream <email address hidden>:canonical/ubuntu-advantage-client.git

2. git fetch upstream
3. Manually cherry-pick those commits:

140d0ee657d70e22109f17de02f9cc3a3e8df529
80b4172fa4dcac4355ab72842d8a9fe082d488fb
e134dd489aed8ec3cb5f8a5510aaf8eeff1091de
985f8d1eb74512c535bbe0f03cd30c3570d6ba2b
bad262200eb38d23a2885168ad1676503ec538ef
c76dcf249adb6f300d750408275d9129c5d00a7b
c6ceeedc536c7144fb5d10463dab5692f2970ef0
bb3d8928acd4ef7e6091ba2ecf69091b733c2774

4. git cherry-pick b2de092ebeb9480c837048c7551bc50fb3a25377..bb623147c73c2984f75b35e3947595c6027350ce

Skip all of merge commits and solve the conflicts on debian/changelog and debian/control

5. Add a commit bumping the project version to 27.0

To post a comment you must log in.
7a474e8... by Chad Smith

config: avoid tracebacks on invalid features value in uaclient.conf

If /etc/ubuntu-advantage/uaclient.conf contains unexpected values
in features: settings log a warning and return an empty dict.

Avoid tracing on every ua subcommand call.

Fixes: #1564

2d4d78c... by Lucas Albuquerque Medeiros de Moura

Remove redundant messaging from uaclient

Currently, uaclient is advertising esm services by allowing
motd to show messages saying that additional packages could
be installed if esm service was enabled. We don't need those
type of messages because update-notifier will handle them.

Also, we are updating the code to always show expired message
for esm-infra services, even if the distro is not ESM

851036e... by Grant Orndorff

test: uncomment additional xenial upgrade tests

3b3e646... by Chad Smith

apt-hook: avoid reporting and counting duplicate package names

Only track unique named package upgrades in ESM.
Both esm -updates and esm-security pockets can contain same
packages. Avoid counting duplicates.

Fixes: #1578

a0dc53a... by Chad Smith

messages: add optional (s) to apt messaging to include singular/plural pkgs

5aa8c38... by Grant Orndorff

fix: dont say reboot required when unnecessary (#1577)

Before, we were assuming that if the system reboot
flag was set, then it was because of our fix operation.
Now, we are carving out a case where we can safely know
that the reboot is not required because of our fix.
Specifically, if our fix operation didn't actually install
anything, then it couldn't have caused the reboot to be
required.

LP: #1926183

bba3fd7... by Lucas Albuquerque Medeiros de Moura

changelog for release 27.0

Revision history for this message
Chad Smith (chad.smith) wrote :

Note that functional equivalent bits have been uploaded to https://launchpad.net/~orndorffgrant/+archive/ubuntu/uaclient-staging-27 for hirsute and impish for testing purposes

Revision history for this message
Chad Smith (chad.smith) wrote :

Since we are now actualy SRUing into Hirsute we have to follow the SRU exception process for ua-tools @ https://wiki.ubuntu.com/UbuntuAdvantageToolsUpdates

So the debian/changelog needs to reflect our SRU process bug (and we won't link any other LP bugs in that SRU)

diff --git a/debian/changelog b/debian/changelog
index 9db7b0de..2bae3d2b 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,11 +1,11 @@
 ubuntu-advantage-tools (27.0~21.04.1) hirsute; urgency=medium

- * New upstream release 27.0:
+ * New upstream release 27.0: (LP: #1926361)
     - messages: add optional (s) to apt messaging to include
       singular/plural pkgs
     - apt-hook: avoid reporting and counting duplicate package
       names (GH: #1578)
- - fix: dont say reboot required when unnecessary (LP: #1926183)
+ - fix: dont say reboot required when unnecessary
     - test: uncomment additional xenial upgrade tests

  -- Lucas Moura <email address hidden> Tue, 27 Apr 2021 15:31:06 -0300

21e6c1e... by Lucas Albuquerque Medeiros de Moura

fix changelog

Revision history for this message
Bryce Harrington (bryce) wrote :

Given that this is essentially a new upstream release, for review purposes I'm going to treat this as an upstream package merge. In other words, rather than a detailed code review I'm mainly going to focus on the packaging changes and integration testing, and assume the main codebase changes have been adequately reviewed and tested already. (The short timeframe we have really doesn't permit a full review anyway.)

However, I did do a cursory review through the 10 commits just to familiarize myself and make sure there's nothing scary. Didn't find anything notable, (other than that this impressively hits four different programming languages!)

I ran into a test failure on initial build which went away after subsequent builds. I'm ignoring that since I couldn't reproduce it, but am noting it in case it comes up again. The issue seemed to involve not recognizing one of the files in lib/ as a python module since there is not a lib/__init__.py. If that crops up anywhere else, it may need adjustment for how it imports the module.

Other than that, the build and test cases all worked fine, and the package installed and upgraded/downgraded without issue.

I did run into a problem trying to use the test case from the SRU bug to reproduce the issue. See
https://pastebin.ubuntu.com/p/WYmK6467NF/. I also tested the libcaca0 package from the original bug report, and that gave similar results. Chad suggested testing against ruby2.7=2.7.2-4ubuntu1 with `ua fix USN-4922-2`, and that worked ok. I gather the metadata provided for CVEs isn't as complete as the metadata provided with USN's. However, it seems like it may be hard to justify this SRU if the original issue is not seen to be fixed by the change... I would suggest changing the test case to one like ruby2.7 that properly demonstrates the bug + fix, and adding a discussion in the SRU text at how this will also work for CVEs once the metadata issue is resolved.

Since the SRU process isn't required for devel versions, I'll go ahead and mark this approved for impish and proceed with uploading.

review: Approve
Revision history for this message
Bryce Harrington (bryce) wrote :

I also uploaded this hirsute package since the package itself is good to go. The SRU bug will need some attention though.

Unmerged commits

21e6c1e... by Lucas Albuquerque Medeiros de Moura

fix changelog

bba3fd7... by Lucas Albuquerque Medeiros de Moura

changelog for release 27.0

a0dc53a... by Chad Smith

messages: add optional (s) to apt messaging to include singular/plural pkgs

3b3e646... by Chad Smith

apt-hook: avoid reporting and counting duplicate package names

Only track unique named package upgrades in ESM.
Both esm -updates and esm-security pockets can contain same
packages. Avoid counting duplicates.

Fixes: #1578

5aa8c38... by Grant Orndorff

fix: dont say reboot required when unnecessary (#1577)

Before, we were assuming that if the system reboot
flag was set, then it was because of our fix operation.
Now, we are carving out a case where we can safely know
that the reboot is not required because of our fix.
Specifically, if our fix operation didn't actually install
anything, then it couldn't have caused the reboot to be
required.

LP: #1926183

851036e... by Grant Orndorff

test: uncomment additional xenial upgrade tests

2d4d78c... by Lucas Albuquerque Medeiros de Moura

Remove redundant messaging from uaclient

Currently, uaclient is advertising esm services by allowing
motd to show messages saying that additional packages could
be installed if esm service was enabled. We don't need those
type of messages because update-notifier will handle them.

Also, we are updating the code to always show expired message
for esm-infra services, even if the distro is not ESM

5ea5a1a... by Grant Orndorff

apt-hook: new json hook for security update counts

This new hook takes advantage of the newly created
"stats" json hooj in apt to print a message about the
number of security updates to be installed on
apt upgrade.

7a474e8... by Chad Smith

config: avoid tracebacks on invalid features value in uaclient.conf

If /etc/ubuntu-advantage/uaclient.conf contains unexpected values
in features: settings log a warning and return an empty dict.

Avoid tracing on every ua subcommand call.

Fixes: #1564

ede5523... by Lucas Albuquerque Medeiros de Moura

Bump version to 27

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/Jenkinsfile b/Jenkinsfile
2index ebee6f9..2082e19 100644
3--- a/Jenkinsfile
4+++ b/Jenkinsfile
5@@ -37,35 +37,28 @@ pipeline {
6 '''
7 }
8 }
9- stage ('Lint and Style') {
10- parallel {
11- stage("flake8") {
12- steps {
13- sh '''
14- set +x
15- . $TMPDIR/bin/activate
16- tox --parallel--safe-build -e flake8
17- '''
18- }
19- }
20- stage("style") {
21- steps {
22- sh '''
23- set +x
24- . $TMPDIR/bin/activate
25- tox --parallel--safe-build -e black
26- '''
27- }
28- }
29- stage("mypy") {
30- steps {
31- sh '''
32- set +x
33- . $TMPDIR/bin/activate
34- tox --parallel--safe-build -e mypy
35- '''
36- }
37- }
38+ stage("flake8") {
39+ steps {
40+ sh '''
41+ set +x
42+ tox -e flake8
43+ '''
44+ }
45+ }
46+ stage("style") {
47+ steps {
48+ sh '''
49+ set +x
50+ tox -e black
51+ '''
52+ }
53+ }
54+ stage("mypy") {
55+ steps {
56+ sh '''
57+ set +x
58+ tox -e mypy
59+ '''
60 }
61 }
62 stage ('Unit Tests') {
63@@ -170,7 +163,7 @@ pipeline {
64 stage("lxc 14.04") {
65 environment {
66 UACLIENT_BEHAVE_DEBS_PATH = "${TMPDIR}trusty/"
67- UACLIENT_BEHAVE_ARTIFACT_DIR = "${TMPDIR}artifacts/behave-lxd-14.04"
68+ UACLIENT_BEHAVE_ARTIFACT_DIR = "artifacts/behave-lxd-14.04"
69 }
70 steps {
71 sh '''
72@@ -183,7 +176,7 @@ pipeline {
73 stage("lxc 16.04") {
74 environment {
75 UACLIENT_BEHAVE_DEBS_PATH = "${TMPDIR}xenial/"
76- UACLIENT_BEHAVE_ARTIFACT_DIR = "${TMPDIR}artifacts/behave-lxd-16.04"
77+ UACLIENT_BEHAVE_ARTIFACT_DIR = "artifacts/behave-lxd-16.04"
78 }
79 steps {
80 sh '''
81@@ -196,7 +189,7 @@ pipeline {
82 stage("lxc 18.04") {
83 environment {
84 UACLIENT_BEHAVE_DEBS_PATH = "${TMPDIR}bionic/"
85- UACLIENT_BEHAVE_ARTIFACT_DIR = "${TMPDIR}artifacts/behave-lxd-18.04"
86+ UACLIENT_BEHAVE_ARTIFACT_DIR = "artifacts/behave-lxd-18.04"
87 }
88 steps {
89 sh '''
90@@ -209,7 +202,7 @@ pipeline {
91 stage("lxc vm 20.04") {
92 environment {
93 UACLIENT_BEHAVE_DEBS_PATH = "${TMPDIR}focal/"
94- UACLIENT_BEHAVE_ARTIFACT_DIR = "${TMPDIR}artifacts/behave-vm-20.04"
95+ UACLIENT_BEHAVE_ARTIFACT_DIR = "artifacts/behave-vm-20.04"
96 }
97 steps {
98 sh '''
99@@ -222,7 +215,7 @@ pipeline {
100 stage("awspro 18.04") {
101 environment {
102 UACLIENT_BEHAVE_DEBS_PATH = "${TMPDIR}bionic/"
103- UACLIENT_BEHAVE_ARTIFACT_DIR = "${TMPDIR}artifacts/behave-awspro-18.04"
104+ UACLIENT_BEHAVE_ARTIFACT_DIR = "artifacts/behave-awspro-18.04"
105 }
106 steps {
107 sh '''
108@@ -253,7 +246,7 @@ pipeline {
109 currentBuild.result = 'UNSTABLE'
110 }
111 try {
112- archiveArtifacts "/tmp/${BUILD_TAG}/artifacts/**/*"
113+ archiveArtifacts "artifacts/**/**/*"
114 } catch (Exception e) {
115 echo "No integration test artifacts found. Presume success."
116 }
117diff --git a/README.md b/README.md
118index c45c58b..7e140c0 100644
119--- a/README.md
120+++ b/README.md
121@@ -20,6 +20,15 @@ The client comes pre-installed on all Ubuntu systems in the debian packages
122 images will also contain `ubuntu-advantage-pro` which automates machine attach
123 on custom AWS and Azure images.
124
125+Additionally, there are 3 PPAs with different release channels of the Ubuntu Advantage Client:
126+
127+1. Stable: This contains stable builds only.
128+ - add with `sudo add-apt-repository ppa:ua-client/stable`
129+2. Staging: This contains builds that are being prepared for release to stable.
130+ - add with `sudo add-apt-repository ppa:ua-client/staging`
131+3. Daily: This PPA is updated every day with the latest changes.
132+ - add with `sudo add-apt-repository ppa:ua-client/daily`
133+
134 Users can manually run the `ua` command to learn more or view the manpage.
135
136 ## Terminology
137@@ -37,7 +46,7 @@ Ubuntu Advantage Client performs:
138
139
140 ## Architecture
141-Ubuntu Advantage client, hereafter "UA client", is python3-based command line
142+Ubuntu Advantage client, hereafter "UA client", is a python3-based command line
143 utility. It provides a CLI to attach, detach, enable,
144 disable and check status of support related services.
145
146@@ -83,7 +92,7 @@ enabled or disabled.
147
148 If a contract entitles a machine to a service, `root` user can enable the
149 service with `ua enable <service>`. If a service can be disabled
150-`ua disabled <service>` will be permitted.
151+`ua disable <service>` will be permitted.
152
153 The goal of the UA client is to remain simple and flexible and let the
154 contracts backend drive dynamic changes in contract offerings and constraints.
155@@ -120,7 +129,15 @@ The following describes the intent of UA client related directories:
156
157 ## Testing
158
159-All unit and lint tests are run using tox:
160+All unit and lint tests are run using `tox`. We also use `tox-pip-version` to specify an older pip version as a workaround: we have some required dependencies that can't meet the strict compatibility checks of current pip versions.
161+
162+First, install `tox` and `tox-pip-version` - you'll only have to do this once.
163+
164+```shell
165+make testdeps
166+```
167+
168+Then you can run the unit and lint tests:
169
170 ```shell
171 tox
172@@ -144,7 +161,7 @@ logic for those tests.
173
174 By default, integration tests will do the folowing on a given cloud platform:
175 * Launch an instance running latest daily image of the target Ubuntu release
176- * Add the Ubuntu advantage client daily build PPA: [ppa:canonical-server/ua-client-daily](https://code.launchpad.net/~canonical-server/+archive/ubuntu/ua-client-daily)
177+ * Add the Ubuntu advantage client daily build PPA: [ppa:ua-client/daily](https://code.launchpad.net/~ua-client/+archive/ubuntu/daily)
178 * Install the appropriate ubuntu-advantage-tools and ubuntu-advantage-pro deb
179 * Stop the instance and snapshot it creating an updated bootable image for
180 test runs
181@@ -181,9 +198,9 @@ Furthermore, when developing/debugging a new scenario:
182
183 1. Add a `@wip` tag decorator on the scenario
184 2. To only run @wip scenarios run: `tox -e behave-20.04 -- -w`
185- 4. If you want to use a debugger:
186- a. Add ipdb to integration-requirements.txt
187- b. Add ipdb.set_trace() in the code block you wish to debug
188+ 3. If you want to use a debugger:
189+ 1. Add ipdb to integration-requirements.txt
190+ 2. Add ipdb.set_trace() in the code block you wish to debug
191
192 (If you're getting started with behave, we recommend at least reading
193 through [the behave
194@@ -271,7 +288,7 @@ To update `features/aws-ids.yaml`, run `./tools/refresh-aws-pro-ids` and put up
195 a pull request against this repo to updated that content from the ua-contracts
196 marketplace definitions.
197
198-* To manually run EC2 integration tests using packages from `ppa:canonical-server/ua-client-daily` provide the following environment vars:
199+* To manually run EC2 integration tests using packages from `ppa:ua-client/daily` provide the following environment vars:
200
201 ```sh
202 UACLIENT_BEHAVE_AWS_ACCESS_KEY_ID=<blah> UACLIENT_BEHAVE_AWS_SECRET_KEY=<blah2> tox -e behave-awspro-20.04
203@@ -295,13 +312,13 @@ The following tox environments allow for testing focal on Azure:
204 ```
205
206 To run the test for a different release, just update the release version string. For example,
207-to run AWS pro xenial tests, you can run:
208+to run Azure pro xenial tests, you can run:
209
210 ```
211 tox -e behave-azurepro-16.04
212 ```
213
214-In order to run EC2 tests the following environment variables are required:
215+In order to run Azure tests the following environment variables are required:
216 - UACLIENT_BEHAVE_AZ_CLIENT_ID
217 - UACLIENT_BEHAVE_AZ_CLIENT_SECRET
218 - UACLIENT_BEHAVE_AZ_SUBSCRIPTION_ID
219@@ -312,7 +329,7 @@ To specifically run non-ubuntu pro tests using canonical cloud-images an
220 additional token obtained from https://ubuntu.com/advantage needs to be set:
221 - UACLIENT_BEHAVE_CONTRACT_TOKEN=<your_token>
222
223-* To manually run Azure integration tests using packages from `ppa:canonical-server/ua-client-daily` provide the following environment vars:
224+* To manually run Azure integration tests using packages from `ppa:ua-client/daily` provide the following environment vars:
225
226 ```sh
227 UACLIENT_BEHAVE_AZ_CLIENT_ID=<blah> UACLIENT_BEHAVE_AZ_CLIENT_SECRET=<blah2> UACLIENT_BEHAVE_AZ_SUBSCRIPTION_ID=<blah3> UACLIENT_BEHAVE_AZ_TENANT_ID=<blah4> tox -e behave-azurepro-20.04
228@@ -337,7 +354,7 @@ dpkg-buildpackage -us -uc
229 ```
230
231 **Note** It will build the package with dependencies for the Ubuntu release on
232-which you are building, so it's best to build in a container of kvm for the
233+which you are building, so it's best to build in a container or kvm for the
234 release you are targeting.
235
236 OR, if you want to build for a target release other than the release
237diff --git a/RELEASES.md b/RELEASES.md
238index e31dc37..98e8aaf 100644
239--- a/RELEASES.md
240+++ b/RELEASES.md
241@@ -54,10 +54,69 @@ Previous release bugs:
242
243 Manually perform a binary package copy from Daily PPA to Premium PPA and notify image creators
244
245- 1. [Open Daily PPA copy-package operation](https://code.launchpad.net/~canonical-server/+archive/ubuntu/ua-client-daily/+copy-packages)
246+ 1. [Open Daily PPA copy-package operation](https://code.launchpad.net/~ua-client/+archive/ubuntu/daily/+copy-packages)
247 2. Check Trusty, Xenial, Bionic package
248- 3. Select Destination PPA: UA Client Premium [~canonical-server/ubuntu/ua-client-premium]
249+ 3. Select Destination PPA: UA Client Premium [~ua-client/ubuntu/staging]
250 4. Select Destination series: The same series
251- 5. Copy options: "Copy existing binaries
252+ 5. Copy options: "Copy existing binaries"
253 6. Click Copy packages
254- 7. Notify Pro Image creatros about expected Premium PPA version (patviafore/rcj)
255+ 7. Notify Pro Image creators about expected Premium PPA version (patviafore/rcj/powersj)
256+
257+
258+## Release to PPA
259+
260+We manually upload the packages to our staging/stable PPAs. If you want to cut a new release and
261+upload to one of these PPAs, follow these steps:
262+
263+ 1. Do a `git cherry-pick` on the commits that should be included in the release
264+ 2. Update the debian/changelog file:
265+ * Create a new entry in the `debian/changelog` file:
266+ * You can do that by running ` dch --newversion <version-name>`
267+ * Remember to update the release from `UNRELEASED` to the most recently supported
268+ ubuntu release
269+ * Populate `debian/changelog` with the commits you have cherry-picked
270+ * You can do that by running `git log <first-cherry-pick-commit>..<last-cherry-pick-commit> | log2dch`
271+ * This will generate a list of commits that could be included in the changelog. If you don't
272+ have `log2dch`, you can get it from the [uss-tableflip](https://github.com/canonical/uss-tableflip)
273+ * You don't need to include all of the commits generated. Remember that the changelog should
274+ be read by the user to understand the new features/modifications in the package. If you
275+ think a commit will not add that much to the user experience, you can drop it from the
276+ changelog
277+ * To structure the changelog you can use the other entries as example. But we basically try
278+ keep this order: debian changes, new features/modifications, testing
279+ 3. Start building the package:
280+ * *WARNING* Build the package in a clean environment. The reason for that is because the package
281+ will contain everything that it is present in the folder. If you are storing credentials or
282+ other sensible development information in your folder, they will be uploaded too when we send
283+ the package to the ppa. A clean environment is the safest way to perform this.
284+ * Build the necessary artifacts that allow building the package
285+ * Guarantee that you have a gpg key in the system. You will use that gpg key to sign the
286+ package.
287+ * If you don't yet have a gpg key set up, follow the instructions
288+ [here](https://help.launchpad.net/YourAccount/ImportingYourPGPKey) to create a key,
289+ publish it to `hkp://keyserver.ubuntu.com`, and import it into Launchpad.
290+ * We can achieve that by running the `build-package` command when your current folder is the
291+ top of source tree. This script is also found on the [uss-tableflip](https://github.com/canonical/uss-tableflip) repo.
292+ * This script will generate all the package artifacts in the parent directory as `../out`.
293+ * Verify if we can build the package:
294+ * We can achieve that by running the `sbuild-it` command. This script is also found on the
295+ [uss-tableflip](https://github.com/canonical/uss-tableflip) repo.
296+ * Before you run `sbuild-it` for the first time, you'll need to set up a chroot for each Ubuntu release.
297+ Follow [these instructions](https://gist.github.com/smoser/14df5f0cd621e10d2282d7c90345e322#new-sbuild-creation)
298+ to create a chroot for each supported release.
299+ * To use it, you can just run `sbuild-it ../out/<package_name>.dsc
300+ * If the package was built sucessfully, you can move to the next step.
301+ 4. Repeat that for older ubuntu releases:
302+ * Currently, we test this build process for `Trusty(14.04)`, `Xenial(16.04)`, `Bionic(18.04)`,
303+ `Focal(20.10)`, `Groovy(20.10)` and `Hirsute(21.04)`
304+ * To test this other releases, just change the changelog to target those releases.
305+ PS: remember to also change the version number on the changelog. For example, suppose
306+ the new version is `1.1~20.04.1`. If you want to test Bionic now, change it to
307+ `1.1~18.04.1`.
308+ * Commit those changes and perform the `build-package` and `sbuild-it` steps for the release.
309+ * These commits are just local commits for this build process - do not push them to remote repository.
310+ 5. After all of the releases are tested, we can start uploading to the ppa. For each release, run
311+ the command `dput ppa:ua-client/stable ../out/<package_name>_source.changes`
312+ * Run this command for each release you are going to upload
313+ * Remember to have launchpad already properly configured in your system to allow you uploading
314+ packages to the ppa.
315diff --git a/apt-hook/20apt-esm-hook.conf b/apt-hook/20apt-esm-hook.conf
316index d6d1ad1..c4e4ebb 100644
317--- a/apt-hook/20apt-esm-hook.conf
318+++ b/apt-hook/20apt-esm-hook.conf
319@@ -1,7 +1,15 @@
320 APT::Update::Post-Invoke-Stats {
321- "[ ! -f /usr/lib/ubuntu-advantage/apt-esm-hook ] || /usr/lib/ubuntu-advantage/apt-esm-hook";
322+ "[ ! -f /usr/lib/ubuntu-advantage/apt-esm-hook ] || /usr/lib/ubuntu-advantage/apt-esm-hook post-invoke-stats";
323 };
324
325 APT::Install::Post-Invoke-Success {
326- "[ ! -f /usr/lib/ubuntu-advantage/apt-esm-hook ] || /usr/lib/ubuntu-advantage/apt-esm-hook";
327+ "[ ! -f /usr/lib/ubuntu-advantage/apt-esm-hook ] || /usr/lib/ubuntu-advantage/apt-esm-hook post-invoke-success";
328 };
329+
330+APT::Install::Pre-Invoke {
331+ "[ ! -f /usr/lib/ubuntu-advantage/apt-esm-hook ] || /usr/lib/ubuntu-advantage/apt-esm-hook pre-invoke";
332+}
333+
334+AptCli::Hooks::Upgrade {
335+ "[ ! -f /usr/lib/ubuntu-advantage/apt-esm-json-hook ] || /usr/lib/ubuntu-advantage/apt-esm-json-hook";
336+}
337diff --git a/apt-hook/Makefile b/apt-hook/Makefile
338index 8eeae10..15c4856 100644
339--- a/apt-hook/Makefile
340+++ b/apt-hook/Makefile
341@@ -1,16 +1,23 @@
342 all: build
343
344-build: hook ubuntu-advantage.pot
345+build: hook ubuntu-advantage.pot json-hook
346
347 ubuntu-advantage.pot: hook.cc
348 xgettext hook.cc -o ubuntu-advantage.pot
349
350 hook: hook.cc
351- $(CXX) -Wall -Wextra -pedantic $(CXXFLAGS) $(CPPFLAGS) $(LDFLAGS) -g -o hook hook.cc -lapt-pkg $(LDLIBS)
352+ $(CXX) -Wall -Wextra -pedantic -std=c++11 $(CXXFLAGS) $(CPPFLAGS) $(LDFLAGS) -g -o hook hook.cc -lapt-pkg $(LDLIBS)
353
354-install: hook
355+json-hook:
356+ cd json-hook-src && GOCACHE=/tmp/ go build json-hook.go
357+
358+install: hook json-hook
359 install -D -m 644 20apt-esm-hook.conf $(DESTDIR)/etc/apt/apt.conf.d/20apt-esm-hook.conf
360 install -D -m 755 hook $(DESTDIR)/usr/lib/ubuntu-advantage/apt-esm-hook
361+ install -D -m 755 json-hook-src/json-hook $(DESTDIR)/usr/lib/ubuntu-advantage/apt-esm-json-hook
362
363 clean:
364- rm -f hook ubuntu-advantage.pot
365+ rm -f hook ubuntu-advantage.pot json-hook-src/json-hook
366+
367+test:
368+ cd json-hook-src && go test
369diff --git a/apt-hook/hook.cc b/apt-hook/hook.cc
370index ed009de..5aed078 100644
371--- a/apt-hook/hook.cc
372+++ b/apt-hook/hook.cc
373@@ -23,20 +23,50 @@
374 #include <apt-pkg/policy.h>
375 #include <apt-pkg/strutl.h>
376
377+#include <algorithm>
378 #include <fstream>
379+#include <iostream>
380 #include <sstream>
381 #include <string>
382+#include <vector>
383
384 #include <assert.h>
385 #include <sys/stat.h>
386 #include <libintl.h>
387 #include <locale.h>
388
389+#define MOTD_ESM_SERVICE_STATUS_MESSAGE_STATIC_PATH "/var/lib/ubuntu-advantage/messages/motd-esm-service-status"
390+#define MOTD_APPS_NO_PKGS_TEMPLATE_PATH "/var/lib/ubuntu-advantage/messages/motd-no-packages-apps.tmpl"
391+#define MOTD_INFRA_NO_PKGS_TEMPLATE_PATH "/var/lib/ubuntu-advantage/messages/motd-no-packages-infra.tmpl"
392+#define MOTD_APPS_PKGS_TEMPLATE_PATH "/var/lib/ubuntu-advantage/messages/motd-packages-apps.tmpl"
393+#define MOTD_INFRA_PKGS_TEMPLATE_PATH "/var/lib/ubuntu-advantage/messages/motd-packages-infra.tmpl"
394+#define MOTD_APPS_PKGS_STATIC_PATH "/var/lib/ubuntu-advantage/messages/motd-packages-apps"
395+#define MOTD_INFRA_PKGS_STATIC_PATH "/var/lib/ubuntu-advantage/messages/motd-packages-infra"
396+#define APT_PRE_INVOKE_APPS_NO_PKGS_TEMPLATE_PATH "/var/lib/ubuntu-advantage/messages/apt-pre-invoke-no-packages-apps.tmpl"
397+#define APT_PRE_INVOKE_INFRA_NO_PKGS_TEMPLATE_PATH "/var/lib/ubuntu-advantage/messages/apt-pre-invoke-no-packages-infra.tmpl"
398+#define APT_PRE_INVOKE_APPS_PKGS_TEMPLATE_PATH "/var/lib/ubuntu-advantage/messages/apt-pre-invoke-packages-apps.tmpl"
399+#define APT_PRE_INVOKE_APPS_PKGS_STATIC_PATH "/var/lib/ubuntu-advantage/messages/apt-pre-invoke-packages-apps"
400+#define APT_PRE_INVOKE_INFRA_PKGS_TEMPLATE_PATH "/var/lib/ubuntu-advantage/messages/apt-pre-invoke-packages-infra.tmpl"
401+#define APT_PRE_INVOKE_INFRA_PKGS_STATIC_PATH "/var/lib/ubuntu-advantage/messages/apt-pre-invoke-packages-infra"
402+#define APT_PRE_INVOKE_MESSAGE_STATIC_PATH "/var/lib/ubuntu-advantage/messages/apt-pre-invoke-esm-service-status"
403+#define UBUNTU_NO_WARRANTY_STATIC_PATH "/var/lib/ubuntu-advantage/messages/ubuntu-no-warranty"
404+
405+
406+#define ESM_APPS_PKGS_COUNT_TEMPLATE_VAR "{ESM_APPS_PKG_COUNT}"
407+#define ESM_APPS_PACKAGES_TEMPLATE_VAR "{ESM_APPS_PACKAGES}"
408+#define ESM_INFRA_PKGS_COUNT_TEMPLATE_VAR "{ESM_INFRA_PKG_COUNT}"
409+#define ESM_INFRA_PACKAGES_TEMPLATE_VAR "{ESM_INFRA_PACKAGES}"
410+
411+enum Subcommand { PreInvoke, PostInvokeStats, PostInvokeSuccess, ProcessTemplates };
412+
413 struct result {
414 int enabled_esms_i;
415 int disabled_esms_i;
416+ std::vector<std::string> esm_i_packages;
417+
418 int enabled_esms_a;
419 int disabled_esms_a;
420+ std::vector<std::string> esm_a_packages;
421 };
422
423 // Return parent pid of specified pid, using /proc (pid might be self)
424@@ -56,7 +86,7 @@ static std::string getppid_of(std::string pid)
425 getline(stream, line);
426
427 if (line.find("PPid:") != 0)
428- continue;
429+ continue;
430
431 // Erase everything before a number
432 line.erase(0, line.find_first_of("0123456789"));
433@@ -121,28 +151,38 @@ static void check_esm_upgrade(pkgCache::PkgIterator pkg, pkgPolicy *policy, resu
434 {
435 for (pkgCache::VerFileIterator pf = ver.FileList(); !pf.end(); pf++)
436 {
437- // TODO: Just look at the origin, not pinning.
438- if (pf.File().Archive() != 0 && pf.File().Origin() == std::string("UbuntuESM"))
439- {
440- // Xenial and later should not be advertising unauthenticated ESM Infra apt repos
441- if (policy->GetPriority(pf.File()) == -32768)
442- res.disabled_esms_i++;
443- else
444- res.enabled_esms_i++;
445-
446- return;
447- }
448-
449- if (pf.File().Archive() != 0 && pf.File().Origin() == std::string("UbuntuESMApps"))
450- {
451- // Xenial and later should not be advertising unauthenticated ESM Apps apt repos
452- if (policy->GetPriority(pf.File()) == -32768)
453- res.disabled_esms_a++;
454- else
455- res.enabled_esms_a++;
456-
457- return;
458- }
459+ if (pf.File().Archive() != 0 && pf.File().Origin() == std::string("UbuntuESM"))
460+ {
461+ if (std::find(res.esm_i_packages.begin(), res.esm_i_packages.end(), pkg.Name()) == res.esm_i_packages.end()) {
462+ res.esm_i_packages.push_back(pkg.Name());
463+
464+ // Pin-Priority: never unauthenticated APT repos == -32768
465+ if (policy->GetPriority(pf.File()) == -32768)
466+ {
467+ res.disabled_esms_i++;
468+ }
469+ else
470+ {
471+ res.enabled_esms_i++;
472+ }
473+ }
474+ }
475+ if (pf.File().Archive() != 0 && pf.File().Origin() == std::string("UbuntuESMApps"))
476+ {
477+ if (std::find(res.esm_a_packages.begin(), res.esm_a_packages.end(), pkg.Name()) == res.esm_a_packages.end()) {
478+ res.esm_a_packages.push_back(pkg.Name());
479+
480+ // Pin-Priority: never unauthenticated APT repos == -32768
481+ if (policy->GetPriority(pf.File()) == -32768)
482+ {
483+ res.disabled_esms_a++;
484+ }
485+ else
486+ {
487+ res.enabled_esms_a++;
488+ }
489+ }
490+ }
491 }
492 }
493 }
494@@ -172,6 +212,155 @@ static int get_update_count(result &res)
495 return count;
496 }
497
498+
499+static void process_template_file(
500+ std::string template_file_name,
501+ std::string static_file_name,
502+ std::string esm_a_pkgs_count,
503+ std::string esm_a_pkgs,
504+ std::string esm_i_pkgs_count,
505+ std::string esm_i_pkgs
506+) {
507+ std::ifstream message_tmpl_file(template_file_name.c_str());
508+ if (message_tmpl_file.is_open()) {
509+ // This line loads the whole file contents into a string
510+ std::string message_tmpl((std::istreambuf_iterator<char>(message_tmpl_file)), (std::istreambuf_iterator<char>()));
511+
512+ message_tmpl_file.close();
513+
514+ // Process all template variables
515+ std::array<std::string, 4> tmpl_var_names = {
516+ ESM_APPS_PKGS_COUNT_TEMPLATE_VAR,
517+ ESM_APPS_PACKAGES_TEMPLATE_VAR,
518+ ESM_INFRA_PKGS_COUNT_TEMPLATE_VAR,
519+ ESM_INFRA_PACKAGES_TEMPLATE_VAR
520+ };
521+ std::array<std::string, 4> tmpl_var_vals = {
522+ esm_a_pkgs_count,
523+ esm_a_pkgs,
524+ esm_i_pkgs_count,
525+ esm_i_pkgs
526+ };
527+ for (uint i = 0; i < tmpl_var_names.size(); i++) {
528+ size_t pos = message_tmpl.find(tmpl_var_names[i]);
529+ if (pos != std::string::npos) {
530+ message_tmpl.replace(pos, tmpl_var_names[i].size(), tmpl_var_vals[i]);
531+ }
532+ }
533+
534+ std::ofstream message_static_file(static_file_name.c_str());
535+ if (message_static_file.is_open()) {
536+ message_static_file << message_tmpl;
537+ message_static_file.close();
538+ }
539+ } else {
540+ remove(static_file_name.c_str());
541+ }
542+}
543+
544+static void process_all_templates(
545+ int esm_a_pkgs_count,
546+ std::string esm_a_pkgs,
547+ int esm_i_pkgs_count,
548+ std::string esm_i_pkgs
549+) {
550+ int bytes_written;
551+ std::array<std::string, 4> static_file_names = {
552+ APT_PRE_INVOKE_APPS_PKGS_STATIC_PATH,
553+ MOTD_APPS_PKGS_STATIC_PATH,
554+ APT_PRE_INVOKE_INFRA_PKGS_STATIC_PATH,
555+ MOTD_INFRA_PKGS_STATIC_PATH,
556+ };
557+ std::array<std::string, 3> apt_static_files = {
558+ APT_PRE_INVOKE_APPS_PKGS_STATIC_PATH,
559+ APT_PRE_INVOKE_INFRA_PKGS_STATIC_PATH,
560+ UBUNTU_NO_WARRANTY_STATIC_PATH
561+ };
562+ std::array<std::string, 3> motd_static_files = {
563+ MOTD_APPS_PKGS_STATIC_PATH,
564+ MOTD_INFRA_PKGS_STATIC_PATH,
565+ UBUNTU_NO_WARRANTY_STATIC_PATH
566+ };
567+
568+ std::vector<std::string> template_file_names;
569+ if (esm_a_pkgs_count > 0) {
570+ template_file_names.push_back(APT_PRE_INVOKE_APPS_PKGS_TEMPLATE_PATH);
571+ template_file_names.push_back(MOTD_APPS_PKGS_TEMPLATE_PATH);
572+ } else {
573+ template_file_names.push_back(APT_PRE_INVOKE_APPS_NO_PKGS_TEMPLATE_PATH);
574+ template_file_names.push_back(MOTD_APPS_NO_PKGS_TEMPLATE_PATH);
575+ }
576+ if (esm_i_pkgs_count > 0) {
577+ template_file_names.push_back(APT_PRE_INVOKE_INFRA_PKGS_TEMPLATE_PATH);
578+ template_file_names.push_back(MOTD_INFRA_PKGS_TEMPLATE_PATH);
579+ } else {
580+ template_file_names.push_back(APT_PRE_INVOKE_INFRA_NO_PKGS_TEMPLATE_PATH);
581+ template_file_names.push_back(MOTD_INFRA_NO_PKGS_TEMPLATE_PATH);
582+ }
583+ for (uint i = 0; i < template_file_names.size(); i++) {
584+ process_template_file(
585+ template_file_names[i],
586+ static_file_names[i],
587+ std::to_string(esm_a_pkgs_count),
588+ esm_a_pkgs,
589+ std::to_string(esm_i_pkgs_count),
590+ esm_i_pkgs
591+ );
592+ }
593+
594+ std::ofstream apt_pre_invoke_msg;
595+ apt_pre_invoke_msg.open(APT_PRE_INVOKE_MESSAGE_STATIC_PATH);
596+ for (uint i = 0; i < apt_static_files.size(); i++) {
597+ std::ifstream message_file(apt_static_files[i]);
598+ if (message_file.is_open()) {
599+ apt_pre_invoke_msg << std::endl;
600+ apt_pre_invoke_msg << message_file.rdbuf();
601+ message_file.close();
602+ };
603+ }
604+ bytes_written = apt_pre_invoke_msg.tellp();
605+ if (bytes_written > 0) {
606+ // Then we wrote some content add trailing newline
607+ apt_pre_invoke_msg << std::endl;
608+ }
609+ apt_pre_invoke_msg.close();
610+ if (bytes_written == 0) {
611+ // We added nothing. Remove the file
612+ remove(APT_PRE_INVOKE_MESSAGE_STATIC_PATH);
613+ }
614+
615+ std::ofstream motd_msg;
616+ motd_msg.open(MOTD_ESM_SERVICE_STATUS_MESSAGE_STATIC_PATH);
617+ for (uint i = 0; i < motd_static_files.size(); i++) {
618+ std::ifstream message_file(motd_static_files[i]);
619+ if (message_file.is_open()) {
620+ if ( i > 0 ) {
621+ motd_msg << std::endl;
622+ }
623+ motd_msg << message_file.rdbuf();
624+ message_file.close();
625+ };
626+ }
627+ bytes_written = motd_msg.tellp();
628+ if (bytes_written > 0) {
629+ // Then we wrote some content add trailing newline
630+ motd_msg << std::endl;
631+ }
632+ motd_msg.close();
633+ if (bytes_written == 0) {
634+ // We added nothing. Remove the file
635+ remove(MOTD_ESM_SERVICE_STATUS_MESSAGE_STATIC_PATH);
636+ }
637+}
638+
639+static void output_file_if_present(std::string file_name) {
640+ std::ifstream message_file(file_name);
641+ if (message_file.is_open()) {
642+ std::cout << message_file.rdbuf();
643+ message_file.close();
644+ }
645+}
646+
647 // Preserves \0 bytes in a string literal
648 template<std::size_t n>
649 std::string make_cmdline(const char (&s)[n])
650@@ -191,6 +380,9 @@ bool has_arg(char **argv, const char *arg)
651 int main(int argc, char *argv[])
652 {
653 (void) argc; // unused
654+ Subcommand subcommand = ProcessTemplates;
655+ bool test_run = false;
656+
657 setlocale(LC_ALL, "");
658 textdomain("ubuntu-advantage");
659 // Self testing
660@@ -206,69 +398,67 @@ int main(int argc, char *argv[])
661 assert(cmdline_eligible(make_cmdline("aptitude\0update\0")));
662 command_used = "";
663
664- result res = {0, 0, 0, 0};
665+ if (has_arg(argv, "test")) {
666+ // useful for testing
667+ test_run = true;
668+ command_used = "upgrade";
669+ }
670+ if (has_arg(argv, "pre-invoke")) {
671+ subcommand = PreInvoke;
672+ } else if (has_arg(argv, "post-invoke-stats")) {
673+ subcommand = PostInvokeStats;
674+ } else if (has_arg(argv, "post-invoke-success")) {
675+ subcommand = PostInvokeSuccess;
676+ } else if (has_arg(argv, "process-templates")) {
677+ subcommand = ProcessTemplates;
678+ }
679
680- // useful for testing
681- if (has_arg(argv, "test"))
682- command_used = "update";
683+ if (!test_run && subcommand != ProcessTemplates && !cmdline_eligible(getcmdline(getppid_of(getppid_of("self"))))) {
684+ // Only run on valid apt commands, or when being used to process templates
685+ return 0;
686+ }
687
688- if (has_arg(argv, "test") || cmdline_eligible(getcmdline(getppid_of(getppid_of("self")))))
689- get_update_count(res);
690+ // Iterate over apt cache looking for esm packages
691+ result res = {0, 0, std::vector<std::string>(), 0, 0, std::vector<std::string>()};
692+ get_update_count(res);
693 if (_error->PendingError())
694 {
695 _error->DumpErrors();
696 return 1;
697 }
698
699- if (command_used == "update")
700- {
701- if (res.enabled_esms_i > 0)
702- {
703- ioprintf(std::cout,
704- ngettext("%d of the updates is from UA Infra: ESM.",
705- "%d of the updates are from UA Infra: ESM.",
706- res.enabled_esms_i),
707- res.enabled_esms_i);
708- ioprintf(std::cout, "\n");
709- }
710- if (res.enabled_esms_a > 0)
711- {
712- ioprintf(std::cout,
713- ngettext("%d of the updates is from UA Apps: ESM.",
714- "%d of the updates are from UA Apps: ESM.",
715- res.enabled_esms_a),
716- res.enabled_esms_a);
717- ioprintf(std::cout, "\n");
718+ // Compute all strings necessary to fill in templates
719+ std::string space_separated_esm_i_packages = "";
720+ if (res.esm_i_packages.size() > 0) {
721+ for (uint i = 0; i < res.esm_i_packages.size() - 1; i++) {
722+ space_separated_esm_i_packages.append(res.esm_i_packages[i]);
723+ space_separated_esm_i_packages.append(" ");
724 }
725+ space_separated_esm_i_packages.append(res.esm_i_packages[res.esm_i_packages.size() - 1]);
726 }
727-
728- if (res.disabled_esms_i > 0 || res.disabled_esms_a > 0)
729- {
730- if (command_used != "update")
731- std::cout << std::endl;
732- if (res.disabled_esms_i > 0)
733- {
734- ioprintf(std::cout,
735- ngettext("%d additional update is available with UA Infra: ESM.",
736- "%d additional updates are available with UA Infra: ESM.",
737- res.disabled_esms_i),
738- res.disabled_esms_i);
739- ioprintf(std::cout, "\n");
740+ std::string space_separated_esm_a_packages = "";
741+ if (res.esm_a_packages.size() > 0) {
742+ for (uint i = 0; i < res.esm_a_packages.size() - 1; i++) {
743+ space_separated_esm_a_packages.append(res.esm_a_packages[i]);
744+ space_separated_esm_a_packages.append(" ");
745 }
746- if (res.disabled_esms_a > 0)
747- {
748- ioprintf(std::cout,
749- ngettext("%d additional update is available with UA Apps: ESM.",
750- "%d additional updates are available with UA Apps ESM.",
751- res.disabled_esms_a),
752- res.disabled_esms_a);
753- ioprintf(std::cout, "\n");
754+ space_separated_esm_a_packages.append(res.esm_a_packages[res.esm_a_packages.size() - 1]);
755+ }
756+ std::string esm_i_packages_count = std::to_string(res.esm_i_packages.size());
757+ std::string esm_a_packages_count = std::to_string(res.esm_a_packages.size());
758+
759+ process_all_templates(
760+ res.esm_a_packages.size(),
761+ space_separated_esm_a_packages,
762+ res.esm_i_packages.size(),
763+ space_separated_esm_i_packages
764+ );
765+
766+ // Execute specified subcommand
767+ if (subcommand == PreInvoke) {
768+ if (command_used == "upgrade" || command_used == "dist-upgrade") {
769+ output_file_if_present(APT_PRE_INVOKE_MESSAGE_STATIC_PATH);
770 }
771-
772- ioprintf(std::cout, gettext("To see these additional updates run: apt list --upgradable"));
773- ioprintf(std::cout, "\n");
774- ioprintf(std::cout, gettext("See https://ubuntu.com/advantage or run: sudo ua status"));
775- ioprintf(std::cout, "\n");
776 }
777
778 return 0;
779diff --git a/apt-hook/json-hook-src/go.mod b/apt-hook/json-hook-src/go.mod
780new file mode 100644
781index 0000000..66726af
782--- /dev/null
783+++ b/apt-hook/json-hook-src/go.mod
784@@ -0,0 +1,3 @@
785+module json-hook
786+
787+go 1.2
788diff --git a/apt-hook/json-hook-src/json-hook.go b/apt-hook/json-hook-src/json-hook.go
789new file mode 100644
790index 0000000..0f2e3eb
791--- /dev/null
792+++ b/apt-hook/json-hook-src/json-hook.go
793@@ -0,0 +1,253 @@
794+package main
795+
796+import (
797+ "bufio"
798+ "encoding/json"
799+ "fmt"
800+ "io"
801+ "net"
802+ "os"
803+ "strconv"
804+)
805+
806+type jsonRPCPackageVersion struct {
807+ Id int `json:"id"`
808+ Version string `json:"version"`
809+ Architecture string `json:"architecture"`
810+ Pin int `json:"pin"`
811+ Origins []struct {
812+ Archive string `json:"archive"`
813+ Codename string `json:"codename"`
814+ Version string `json:"version"`
815+ Origin string `json:"origin"`
816+ Label string `json:"label"`
817+ Site string `json:"site"`
818+ } `json:"origins"`
819+}
820+type jsonRPC struct {
821+ JsonRPC string `json:"jsonrpc"`
822+ Method string `json:"method"`
823+ Params struct {
824+ Command string `json:"command"`
825+ UnknownPackages []string `json:"unknown-packages"`
826+ Packages []struct {
827+ Id int `json:"id"`
828+ Name string `json:"name"`
829+ Architecture string `json:"architecture"`
830+ Mode string `json:"mode"`
831+ Automatic bool `json:"automatic"`
832+ Versions struct {
833+ Candidate jsonRPCPackageVersion `json:"candidate"`
834+ Install jsonRPCPackageVersion `json:"install"`
835+ Current jsonRPCPackageVersion `json:"current"`
836+ } `json:"versions"`
837+ } `json:"packages"`
838+ } `json:"params"`
839+}
840+
841+func createUpdateMessage(standardSecurityCount int, esmInfraCount int, esmAppsCount int) string {
842+ displayStandard := true
843+ displayEsmInfra := true
844+ displayEsmApps := true
845+ if standardSecurityCount == 0 {
846+ displayStandard = false
847+ }
848+ if esmInfraCount == 0 {
849+ displayEsmInfra = false
850+ }
851+ if esmAppsCount == 0 {
852+ displayEsmApps = false
853+ }
854+
855+ if !displayStandard && !displayEsmInfra && !displayEsmApps {
856+ return ""
857+ }
858+
859+ esmInfraFirst := false
860+ esmAppsFirst := false
861+ if !displayStandard && displayEsmInfra {
862+ esmInfraFirst = true
863+ } else if !displayStandard && !displayEsmInfra && displayEsmApps {
864+ esmAppsFirst = true
865+ }
866+
867+ standardUpdates := ""
868+ esmInfraUpdates := ""
869+ esmAppsUpdates := ""
870+ if displayStandard {
871+ standardUpdates = fmt.Sprintf("%d standard security ", standardSecurityCount)
872+ if standardSecurityCount > 1 {
873+ standardUpdates += "updates"
874+ } else {
875+ standardUpdates += "update"
876+ }
877+ if displayEsmInfra && displayEsmApps {
878+ standardUpdates += ","
879+ }
880+ if displayEsmInfra || displayEsmApps {
881+ standardUpdates += " "
882+ }
883+ if (displayEsmInfra && !displayEsmApps) || (!displayEsmInfra && displayEsmApps) {
884+ standardUpdates += "and "
885+ }
886+ }
887+ if displayEsmInfra {
888+ esmInfraUpdates = fmt.Sprintf("%d esm-infra ", esmInfraCount)
889+ if esmInfraFirst {
890+ esmInfraUpdates += "security "
891+ }
892+ if esmInfraCount > 1 {
893+ esmInfraUpdates += "updates"
894+ } else {
895+ esmInfraUpdates += "update"
896+ }
897+ if displayEsmApps {
898+ esmInfraUpdates += " and "
899+ }
900+ }
901+ if displayEsmApps {
902+ esmAppsUpdates = fmt.Sprintf("%d esm-apps ", esmAppsCount)
903+ if esmAppsFirst {
904+ esmAppsUpdates += "security "
905+ }
906+ if esmAppsCount > 1 {
907+ esmAppsUpdates += "updates"
908+ } else {
909+ esmAppsUpdates += "update"
910+ }
911+ }
912+
913+ return standardUpdates + esmInfraUpdates + esmAppsUpdates
914+}
915+
916+func fromOriginAndArchive(pkgVersion jsonRPCPackageVersion, origin string, archive string) bool {
917+ for _, pkgOrigin := range pkgVersion.Origins {
918+ if pkgOrigin.Origin == origin && pkgOrigin.Archive == archive {
919+ return true
920+ }
921+ }
922+ return false
923+}
924+
925+func distroFromPackageOrigin(rpc *jsonRPC) string {
926+ for _, pkg := range rpc.Params.Packages {
927+ for _, origin := range pkg.Versions.Candidate.Origins {
928+ if origin.Codename != "" {
929+ return origin.Codename
930+ }
931+ }
932+ }
933+ return ""
934+}
935+
936+func countSecurityUpdates(rpc *jsonRPC) (int, int, int) {
937+ esmAppsCount := 0
938+ esmInfraCount := 0
939+ standardSecurityCount := 0
940+ distro := distroFromPackageOrigin(rpc)
941+ for _, pkg := range rpc.Params.Packages {
942+ if pkg.Mode == "upgrade" {
943+ if fromOriginAndArchive(pkg.Versions.Install, "UbuntuESMApps", fmt.Sprintf("%s-apps-security", distro)) {
944+ esmAppsCount++
945+ } else if fromOriginAndArchive(pkg.Versions.Install, "UbuntuESM", fmt.Sprintf("%s-infra-security", distro)) {
946+ esmInfraCount++
947+ } else if fromOriginAndArchive(pkg.Versions.Install, "Ubuntu", fmt.Sprintf("%s-security", distro)) {
948+ standardSecurityCount++
949+ }
950+ }
951+ }
952+ return standardSecurityCount, esmInfraCount, esmAppsCount
953+}
954+
955+// readRpc reads a apt json rpc protocol 0.2 message as described in
956+// https://salsa.debian.org/apt-team/apt/blob/main/doc/json-hooks-protocol.md#wire-protocol
957+func readRpc(r *bufio.Reader) (*jsonRPC, error) {
958+ line, err := r.ReadBytes('\n')
959+ if err != nil && err != io.EOF {
960+ return nil, fmt.Errorf("cannot read json-rpc: %v", err)
961+ }
962+
963+ var rpc jsonRPC
964+ if err := json.Unmarshal(line, &rpc); err != nil {
965+ return nil, err
966+ }
967+ // empty \n
968+ emptyNL, _, err := r.ReadLine()
969+ if err != nil {
970+ return nil, err
971+ }
972+ if string(emptyNL) != "" {
973+ return nil, fmt.Errorf("unexpected line: %q (empty)", emptyNL)
974+ }
975+
976+ return &rpc, nil
977+}
978+
979+func printEsmUpgrades() error {
980+ sockFd := os.Getenv("APT_HOOK_SOCKET")
981+ if sockFd == "" {
982+ return fmt.Errorf("cannot find APT_HOOK_SOCKET env")
983+ }
984+
985+ fd, err := strconv.Atoi(sockFd)
986+ if err != nil {
987+ return fmt.Errorf("expected APT_HOOK_SOCKET to be a decimal integer, found %q", sockFd)
988+ }
989+
990+ f := os.NewFile(uintptr(fd), "apt-hook-socket")
991+ if f == nil {
992+ return fmt.Errorf("cannot open file descriptor %v", fd)
993+ }
994+ defer f.Close()
995+
996+ conn, err := net.FileConn(f)
997+ if err != nil {
998+ return fmt.Errorf("cannot connect to %v: %v", fd, err)
999+ }
1000+ defer conn.Close()
1001+
1002+ r := bufio.NewReader(conn)
1003+
1004+ // handshake
1005+ rpc, err := readRpc(r)
1006+ if err != nil {
1007+ return err
1008+ }
1009+ if rpc.Method != "org.debian.apt.hooks.hello" {
1010+ return fmt.Errorf("expected 'hello' method, got: %v", rpc.Method)
1011+ }
1012+ if _, err := conn.Write([]byte(`{"jsonrpc":"2.0","id":0,"result":{"version":"0.2"}}` + "\n\n")); err != nil {
1013+ return err
1014+ }
1015+
1016+ // payload
1017+ rpc, err = readRpc(r)
1018+ if err != nil {
1019+ return err
1020+ }
1021+ if rpc.Method == "org.debian.apt.hooks.install.statistics" {
1022+ standardSecurityCount, esmInfraCount, esmAppsCount := countSecurityUpdates(rpc)
1023+ msg := createUpdateMessage(standardSecurityCount, esmInfraCount, esmAppsCount)
1024+ if msg != "" {
1025+ fmt.Println(msg)
1026+ }
1027+ }
1028+
1029+ // bye
1030+ rpc, err = readRpc(r)
1031+ if err != nil {
1032+ return err
1033+ }
1034+ if rpc.Method != "org.debian.apt.hooks.bye" {
1035+ return fmt.Errorf("expected 'bye' method, got: %v", rpc.Method)
1036+ }
1037+
1038+ return nil
1039+}
1040+
1041+func main() {
1042+ err := printEsmUpgrades()
1043+ if err != nil {
1044+ println(err.Error())
1045+ }
1046+}
1047diff --git a/apt-hook/json-hook-src/json-hook_test.go b/apt-hook/json-hook-src/json-hook_test.go
1048new file mode 100644
1049index 0000000..cfc1788
1050--- /dev/null
1051+++ b/apt-hook/json-hook-src/json-hook_test.go
1052@@ -0,0 +1,410 @@
1053+package main
1054+
1055+import (
1056+ "encoding/json"
1057+ "fmt"
1058+ "testing"
1059+)
1060+
1061+func TestCreateUpdateMessages(t *testing.T) {
1062+ type params struct {
1063+ standardSecurityCount int
1064+ esmInfraCount int
1065+ esmAppsCount int
1066+ expectedMessage string
1067+ }
1068+ testParamsList := []params{
1069+ params{0, 0, 0, ""},
1070+ params{0, 0, 1, "1 esm-apps security update"},
1071+ params{0, 0, 2, "2 esm-apps security updates"},
1072+ params{0, 1, 0, "1 esm-infra security update"},
1073+ params{0, 1, 1, "1 esm-infra security update and 1 esm-apps update"},
1074+ params{0, 1, 2, "1 esm-infra security update and 2 esm-apps updates"},
1075+ params{0, 2, 0, "2 esm-infra security updates"},
1076+ params{0, 2, 1, "2 esm-infra security updates and 1 esm-apps update"},
1077+ params{0, 2, 2, "2 esm-infra security updates and 2 esm-apps updates"},
1078+ params{1, 0, 0, "1 standard security update"},
1079+ params{1, 0, 1, "1 standard security update and 1 esm-apps update"},
1080+ params{1, 0, 2, "1 standard security update and 2 esm-apps updates"},
1081+ params{1, 1, 0, "1 standard security update and 1 esm-infra update"},
1082+ params{1, 1, 1, "1 standard security update, 1 esm-infra update and 1 esm-apps update"},
1083+ params{1, 1, 2, "1 standard security update, 1 esm-infra update and 2 esm-apps updates"},
1084+ params{1, 2, 0, "1 standard security update and 2 esm-infra updates"},
1085+ params{1, 2, 1, "1 standard security update, 2 esm-infra updates and 1 esm-apps update"},
1086+ params{1, 2, 2, "1 standard security update, 2 esm-infra updates and 2 esm-apps updates"},
1087+ params{2, 0, 0, "2 standard security updates"},
1088+ params{2, 0, 1, "2 standard security updates and 1 esm-apps update"},
1089+ params{2, 0, 2, "2 standard security updates and 2 esm-apps updates"},
1090+ params{2, 1, 0, "2 standard security updates and 1 esm-infra update"},
1091+ params{2, 1, 1, "2 standard security updates, 1 esm-infra update and 1 esm-apps update"},
1092+ params{2, 1, 2, "2 standard security updates, 1 esm-infra update and 2 esm-apps updates"},
1093+ params{2, 2, 0, "2 standard security updates and 2 esm-infra updates"},
1094+ params{2, 2, 1, "2 standard security updates, 2 esm-infra updates and 1 esm-apps update"},
1095+ params{2, 2, 2, "2 standard security updates, 2 esm-infra updates and 2 esm-apps updates"},
1096+ }
1097+
1098+ for i, testParams := range testParamsList {
1099+ t.Run(fmt.Sprintf("Case %d", i), func(t *testing.T) {
1100+ actual := createUpdateMessage(testParams.standardSecurityCount, testParams.esmInfraCount, testParams.esmAppsCount)
1101+ if actual != testParams.expectedMessage {
1102+ t.Logf("expected: \"%s\", got: \"%s\"", testParams.expectedMessage, actual)
1103+ t.Fail()
1104+ }
1105+ })
1106+ }
1107+}
1108+
1109+func TestCountSecurityUpdates(t *testing.T) {
1110+ type params struct {
1111+ rpc string
1112+ expectedStandardSecurityCount int
1113+ expectedEsmInfraCount int
1114+ expectedEsmAppsCount int
1115+ }
1116+ testParamsList := []params{
1117+ params{mockJson, 1, 2, 3},
1118+ }
1119+
1120+ for i, testParams := range testParamsList {
1121+ t.Run(fmt.Sprintf("Case %d", i), func(t *testing.T) {
1122+ rpc := &jsonRPC{}
1123+ if err := json.Unmarshal([]byte(mockJson), rpc); err != nil {
1124+ t.Error(err)
1125+ }
1126+
1127+ actualStandard, actualInfra, actualApps := countSecurityUpdates(rpc)
1128+ if actualStandard != testParams.expectedStandardSecurityCount {
1129+ t.Logf("expected: %d, got: %d", testParams.expectedStandardSecurityCount, actualStandard)
1130+ t.Fail()
1131+ }
1132+ if actualInfra != testParams.expectedEsmInfraCount {
1133+ t.Logf("expected: %d, got: %d", testParams.expectedEsmInfraCount, actualInfra)
1134+ t.Fail()
1135+ }
1136+ if actualApps != testParams.expectedEsmAppsCount {
1137+ t.Logf("expected: %d, got: %d", testParams.expectedEsmAppsCount, actualApps)
1138+ t.Fail()
1139+ }
1140+ })
1141+ }
1142+}
1143+
1144+const mockJson = `
1145+{
1146+ "jsonrpc": "2.0",
1147+ "method": "org.debian.apt.hooks.install.statistics",
1148+ "params": {
1149+ "command": "install",
1150+ "search-terms": [
1151+ "~U"
1152+ ],
1153+ "unknown-packages": [],
1154+ "packages": [
1155+ {
1156+ "id": 418,
1157+ "name": "base-files",
1158+ "architecture": "amd64",
1159+ "mode": "upgrade",
1160+ "automatic": true,
1161+ "versions": {
1162+ "candidate": {
1163+ "id": 86,
1164+ "version": "11ubuntu19",
1165+ "architecture": "amd64",
1166+ "pin": 500,
1167+ "origins": [
1168+ {
1169+ "archive": "hirsute-apps-security",
1170+ "codename": "hirsute",
1171+ "version": "21.04",
1172+ "origin": "UbuntuESMApps",
1173+ "label": "Ubuntu",
1174+ "site": ""
1175+ }
1176+ ]
1177+ },
1178+ "install": {
1179+ "id": 86,
1180+ "version": "11ubuntu19",
1181+ "architecture": "amd64",
1182+ "pin": 500,
1183+ "origins": [
1184+ {
1185+ "archive": "hirsute-apps-security",
1186+ "codename": "hirsute",
1187+ "version": "21.04",
1188+ "origin": "UbuntuESMApps",
1189+ "label": "Ubuntu",
1190+ "site": ""
1191+ }
1192+ ]
1193+ },
1194+ "current": {
1195+ "id": 95463,
1196+ "version": "11ubuntu18",
1197+ "architecture": "amd64",
1198+ "pin": 100,
1199+ "origins": []
1200+ }
1201+ }
1202+ },
1203+ {
1204+ "id": 1085,
1205+ "name": "elfutils",
1206+ "architecture": "amd64",
1207+ "mode": "upgrade",
1208+ "automatic": true,
1209+ "versions": {
1210+ "candidate": {
1211+ "id": 371,
1212+ "version": "0.183-8",
1213+ "architecture": "amd64",
1214+ "pin": 500,
1215+ "origins": [
1216+ {
1217+ "archive": "hirsute-apps-security",
1218+ "codename": "hirsute",
1219+ "version": "21.04",
1220+ "origin": "UbuntuESMApps",
1221+ "label": "Ubuntu",
1222+ "site": ""
1223+ }
1224+ ]
1225+ },
1226+ "install": {
1227+ "id": 371,
1228+ "version": "0.183-8",
1229+ "architecture": "amd64",
1230+ "pin": 500,
1231+ "origins": [
1232+ {
1233+ "archive": "hirsute-apps-security",
1234+ "codename": "hirsute",
1235+ "version": "21.04",
1236+ "origin": "UbuntuESMApps",
1237+ "label": "Ubuntu",
1238+ "site": ""
1239+ }
1240+ ]
1241+ },
1242+ "current": {
1243+ "id": 95472,
1244+ "version": "0.183-6",
1245+ "architecture": "amd64",
1246+ "pin": 100,
1247+ "origins": []
1248+ }
1249+ }
1250+ },
1251+ {
1252+ "id": 24709,
1253+ "name": "fdroidserver",
1254+ "architecture": "amd64",
1255+ "mode": "upgrade",
1256+ "automatic": false,
1257+ "versions": {
1258+ "candidate": {
1259+ "id": 14186,
1260+ "version": "2.0-1",
1261+ "architecture": "all",
1262+ "pin": 500,
1263+ "origins": [
1264+ {
1265+ "archive": "hirsute-infra-security",
1266+ "codename": "hirsute",
1267+ "version": "21.04",
1268+ "origin": "UbuntuESM",
1269+ "label": "Ubuntu",
1270+ "site": ""
1271+ },
1272+ {
1273+ "archive": "hirsute",
1274+ "codename": "hirsute",
1275+ "version": "21.04",
1276+ "origin": "Ubuntu",
1277+ "label": "Ubuntu",
1278+ "site": ""
1279+ }
1280+ ]
1281+ },
1282+ "install": {
1283+ "id": 14186,
1284+ "version": "2.0-1",
1285+ "architecture": "all",
1286+ "pin": 500,
1287+ "origins": [
1288+ {
1289+ "archive": "hirsute-infra-security",
1290+ "codename": "hirsute",
1291+ "version": "21.04",
1292+ "origin": "UbuntuESM",
1293+ "label": "Ubuntu",
1294+ "site": ""
1295+ },
1296+ {
1297+ "archive": "hirsute",
1298+ "codename": "hirsute",
1299+ "version": "21.04",
1300+ "origin": "Ubuntu",
1301+ "label": "Ubuntu",
1302+ "site": ""
1303+ }
1304+ ]
1305+ },
1306+ "current": {
1307+ "id": 95474,
1308+ "version": "1.1.9-1",
1309+ "architecture": "all",
1310+ "pin": 100,
1311+ "origins": []
1312+ }
1313+ }
1314+ },
1315+ {
1316+ "id": 238,
1317+ "name": "gdb",
1318+ "architecture": "amd64",
1319+ "mode": "upgrade",
1320+ "automatic": true,
1321+ "versions": {
1322+ "candidate": {
1323+ "id": 705,
1324+ "version": "10.1-2ubuntu2",
1325+ "architecture": "amd64",
1326+ "pin": 500,
1327+ "origins": [
1328+ {
1329+ "archive": "hirsute-infra-security",
1330+ "codename": "hirsute",
1331+ "version": "21.04",
1332+ "origin": "UbuntuESM",
1333+ "label": "Ubuntu",
1334+ "site": ""
1335+ }
1336+ ]
1337+ },
1338+ "install": {
1339+ "id": 705,
1340+ "version": "10.1-2ubuntu2",
1341+ "architecture": "amd64",
1342+ "pin": 500,
1343+ "origins": [
1344+ {
1345+ "archive": "hirsute-infra-security",
1346+ "codename": "hirsute",
1347+ "version": "21.04",
1348+ "origin": "UbuntuESM",
1349+ "label": "Ubuntu",
1350+ "site": ""
1351+ }
1352+ ]
1353+ },
1354+ "current": {
1355+ "id": 95475,
1356+ "version": "10.1-2ubuntu1",
1357+ "architecture": "amd64",
1358+ "pin": 100,
1359+ "origins": []
1360+ }
1361+ }
1362+ },
1363+ {
1364+ "id": 126271,
1365+ "name": "google-chrome-stable",
1366+ "architecture": "amd64",
1367+ "mode": "upgrade",
1368+ "automatic": true,
1369+ "versions": {
1370+ "candidate": {
1371+ "id": 95416,
1372+ "version": "90.0.4430.85-1",
1373+ "architecture": "amd64",
1374+ "pin": 500,
1375+ "origins": [
1376+ {
1377+ "archive": "hirsute-apps-security",
1378+ "codename": "hirsute",
1379+ "version": "1.0",
1380+ "origin": "UbuntuESMApps",
1381+ "label": "Google",
1382+ "site": "dl.google.com"
1383+ }
1384+ ]
1385+ },
1386+ "install": {
1387+ "id": 95416,
1388+ "version": "90.0.4430.85-1",
1389+ "architecture": "amd64",
1390+ "pin": 500,
1391+ "origins": [
1392+ {
1393+ "archive": "hirsute-apps-security",
1394+ "codename": "hirsute",
1395+ "version": "1.0",
1396+ "origin": "UbuntuESMApps",
1397+ "label": "Google",
1398+ "site": "dl.google.com"
1399+ }
1400+ ]
1401+ },
1402+ "current": {
1403+ "id": 95477,
1404+ "version": "90.0.4430.72-1",
1405+ "architecture": "amd64",
1406+ "pin": 100,
1407+ "origins": []
1408+ }
1409+ }
1410+ },
1411+ {
1412+ "id": 1499,
1413+ "name": "libasm1",
1414+ "architecture": "amd64",
1415+ "mode": "upgrade",
1416+ "automatic": true,
1417+ "versions": {
1418+ "candidate": {
1419+ "id": 1763,
1420+ "version": "0.183-8",
1421+ "architecture": "amd64",
1422+ "pin": 500,
1423+ "origins": [
1424+ {
1425+ "archive": "hirsute-security",
1426+ "codename": "hirsute",
1427+ "version": "21.04",
1428+ "origin": "Ubuntu",
1429+ "label": "Ubuntu",
1430+ "site": ""
1431+ }
1432+ ]
1433+ },
1434+ "install": {
1435+ "id": 1763,
1436+ "version": "0.183-8",
1437+ "architecture": "amd64",
1438+ "pin": 500,
1439+ "origins": [
1440+ {
1441+ "archive": "hirsute-security",
1442+ "codename": "hirsute",
1443+ "version": "21.04",
1444+ "origin": "Ubuntu",
1445+ "label": "Ubuntu",
1446+ "site": ""
1447+ }
1448+ ]
1449+ },
1450+ "current": {
1451+ "id": 95482,
1452+ "version": "0.183-6",
1453+ "architecture": "amd64",
1454+ "pin": 100,
1455+ "origins": []
1456+ }
1457+ }
1458+ }
1459+ ]
1460+ }
1461+}
1462+`
1463diff --git a/debian/changelog b/debian/changelog
1464index ee90158..3f0715e 100644
1465--- a/debian/changelog
1466+++ b/debian/changelog
1467@@ -1,10 +1,167 @@
1468+ubuntu-advantage-tools (27.0~21.04.1) hirsute; urgency=medium
1469+
1470+ * New upstream release 27.0: (LP: #1926361)
1471+ - messages: add optional (s) to apt messaging to include
1472+ singular/plural pkgs
1473+ - apt-hook: avoid reporting and counting duplicate package
1474+ names (GH: #1578)
1475+ - fix: dont say reboot required when unnecessary
1476+ - test: uncomment additional xenial upgrade tests
1477+
1478+ -- Lucas Moura <lucas.moura@canonical.com> Tue, 27 Apr 2021 15:31:06 -0300
1479+
1480+ubuntu-advantage-tools (27.0~21.04.1~beta3) hirsute; urgency=medium
1481+
1482+ * New upstream beta3 release:
1483+ - config: avoid tracebacks on invalid features value in uaclient.conf
1484+ (GH: #1564)
1485+ - apt-hook: new json hook for security update counts
1486+ - Remove redundant messaging from uaclient
1487+
1488+ -- Chad Smith <chad.smith@canonical.com> Fri, 23 Apr 2021 15:28:44 -0600
1489+
1490+ubuntu-advantage-tools (27.0~21.04.1~beta2) hirsute; urgency=medium
1491+
1492+ * d/control:
1493+ - add distro-info dependency
1494+ - add new debianutils dependency
1495+ - add optional dh-systemd | debhelper (>= 13.3) to fallback on hirsute
1496+ and later when dh-systemd is not present
1497+ * d/rules: enable and start ua-messaging.timer on package install
1498+ * d/postinst:
1499+ - configure esm on any LTS release avoid beta services
1500+ - configure esm-infra when is_active_esm and apps on LTS
1501+ - xenial enable unauthenticated apt source for apps/infra
1502+ * New upstream release 27.0~beta:
1503+ - apt-hook:
1504+ + adapt hook to process separate message templates
1505+ + esm-apps and esm-infra pkg counts not mutually-exclusive
1506+ + print static messages on apt upgrade/dist-upgrade (GH: #1546)
1507+ - config: create settings_overrides on config (GH: #1507)
1508+ - docs: add entry for uploading new version to ppa
1509+ - esm:
1510+ + add pin never when disabling esm-infra/apps on xenial
1511+ + enable infra when EOL LTS and apps on all LTS (GH: #1558)
1512+ - fips: add notice when installing over old fips
1513+ - fix:
1514+ + add links to ubuntu.com/gcp/aws in messaging when on non-PRO
1515+ + add notice to reboot operation on ua fix
1516+ + do not prompt user for beta services (GH: #1544)
1517+ + notify users if reboot is required (GH: #1476)
1518+ + update how the expired token logic works
1519+ + wrap output greater than 80 chars (GH: #1487)
1520+ - lib: fix notice handling on reboot script
1521+ - messages
1522+ + provide static message files for use in APT and MOTD
1523+ + update_ua_messages on attach/detach/disable
1524+ - mypy: add lib/ dir for coverage
1525+ - status: do not remove notices on non-root call (GH: #1518)
1526+ - subp: separate % format strings when logging (GH: #1520)
1527+ - systemd: add ua-messaging.timer to update ua MOTD and APT msgs
1528+ - update-motd.d: add conditional hooks for motd to source ua messages
1529+ - util: add is_lts and is_active_esm funtions to support ESM
1530+ - test
1531+ + add integration tests asserting esm-apps setup due to postinst
1532+ + manual test script for xenial upgrade
1533+ + trusty and xenial infra and apps disabled in pkg install
1534+ - behave: use unaltered cloud images unsetting UACLIENT_BEHAVE_PPA
1535+ - jenkins: make lint and style stage run sequentially
1536+
1537+ -- Lucas Moura <lucas.moura@canonical.com> Thu, 22 Apr 2021 14:16:26 -0300
1538+
1539+ubuntu-advantage-tools (27.0~21.04.1~beta) hirsute; urgency=medium
1540+
1541+ * d/*: prefix all the debhelper conf files with the package name
1542+ * d/control:
1543+ - add Rules-Requires-Root: no
1544+ - bump Standards-Version to 4.5.1
1545+ - make ubuntu-advantage-pro Architecture: all
1546+ * d/lintian-overrides:
1547+ - override maintainer-script-calls-service
1548+ - package-supports-alternative-init-but-no-init.d-script
1549+ * d/postinst: move the u-a-pro note to a config script
1550+ * d/ubuntu-advantage-tools.templates: suggest the use of apt
1551+ * New upstream release 27.0~beta:
1552+ - apt: add retry for apt-helper command (GH: #1431)
1553+ - cli: drop subcommand repeated help output, fix enable & refresh
1554+ (GH: #1440)
1555+ - config:
1556+ + allow parsing yaml delivered from env values
1557+ + environment variable support for feature overrides (GH: #1395)
1558+ + create config to add extra params to security url
1559+ - docs:
1560+ + add ppas and fix typos
1561+ + use Ubuntu Pro not Ubuntu PRO
1562+ + add stop "." punctuation to messages (GH: #1320)
1563+ - fips: fix FIPS message when disable operation fails
1564+ - fix:
1565+ + add basic UASecurityClient to which queries CVE and USNs
1566+ + add security_url to config
1567+ + check if service is enabled during ua fix (GH: #1462)
1568+ + closer representation of cve and usn responses
1569+ + filter usns by cve details (GH: #1470)
1570+ + fix regex to be more permissive and strict
1571+ + get_cve_affected_source_packages_status won't list not-affected
1572+ (GH: #1467)
1573+ + handle other package status when running ua fix (GH: #1435)
1574+ + improve error message for ua fix (GH: #1420)
1575+ + install pkg fixes when they are on standard pocket (GH: #1401)
1576+ + move timeout and retries to security client only
1577+ + only prompt for subscription attach for UA-related pkg updates
1578+ + parse all related USNS to a given CVE when fixing
1579+ + parse full API responses for related CVEs and USNs
1580+ + prefer USN.release_packages binary pkg versions to CVE src ver
1581+ (GH: #1436)
1582+ + prompt for new ua token when expired one is used (GH: #1475)
1583+ + prompt to emit pro suggestion on pro_clouds if unattached (GH: #1386)
1584+ + prompt to enable service during ua fix (GH: #1455)
1585+ + provide related CVE URLs instead of USNs (GH: #1456)
1586+ + raise errors when source_link is null or unexpected format
1587+ + show packages that were not fixed in the output
1588+ + update output for released packages in ua fix (GH: #1438)
1589+ + update message for invalid issue in ua fix (GH: #1433)
1590+ + use pocket values from USNs (GH: #1439)
1591+ - logs: emit error response on API errors and redact sensitive logs
1592+ (GH: #1424)
1593+ - serviceclient: add 10 second timeout and two retries to API calls
1594+ (GH: #1374)
1595+ - util:
1596+ + add error prompts on invalid selection
1597+ + add timeout to readurl
1598+ - tests:
1599+ + Add disable_auto_attach config to all test PRO vms
1600+ + add merge_usn_released_binary_package_versions tests
1601+ + add unittest coverage for override_usn_release_package_status
1602+ + drop traceback checks on fips integration tests
1603+ + refactor integration tests for ua fix cmd
1604+ + run status wait before detach in PRO tests
1605+ + use ssh to run commands on lxd containers
1606+ - jenkins: archiveArtifacts can only reference paths within workspace
1607+
1608+ -- Lucas Moura <lucas.moura@canonical.com> Tue, 30 Mar 2021 14:16:03 -0300
1609+
1610+ubuntu-advantage-tools (26.3~21.04.1) hirsute; urgency=medium
1611+
1612+ * d/control: add new debianutils dependency
1613+ * New upstream release 26.3
1614+ - util: improve is_container check for chroot
1615+ - cli: pass assume_yes param to services on detach (GH: #1530)
1616+
1617+ -- Grant Orndorff <grant.orndorff@canonical.com> Tue, 06 Apr 2021 14:26:20 -0300
1618+
1619 ubuntu-advantage-tools (26.2) hirsute; urgency=medium
1620
1621 * Drop dh-systemd build dependency.
1622
1623 -- Matthias Klose <doko@ubuntu.com> Wed, 10 Mar 2021 16:54:12 +0100
1624
1625-ubuntu-advantage-tools (26.1) hirsute; urgency=medium
1626+ubuntu-advantage-tools (26.2~21.04.1) hirsute; urgency=medium
1627+
1628+ * status: show beta services in status if enabled (GH: #1410)
1629+
1630+ -- Lucas Moura <lucas.moura@canonical.com> Tue, 02 Mar 2021 10:11:53 -0300
1631+
1632+ubuntu-advantage-tools (26.1~21.04.1) hirsute; urgency=medium
1633
1634 * New upstream release 26.1
1635 - contract: block detach call to contract if machine-id change
1636diff --git a/debian/control b/debian/control
1637index 87595b8..cd28a1f 100644
1638--- a/debian/control
1639+++ b/debian/control
1640@@ -4,27 +4,33 @@ Priority: important
1641 Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
1642 Build-Depends: bash-completion,
1643 debhelper (>=9),
1644+ debianutils,
1645 dh-python,
1646+ dh-systemd | debhelper (>= 13.3),
1647 gettext,
1648 git,
1649+ golang,
1650 libapt-pkg-dev,
1651 po-debconf,
1652 python3 (>= 3.4),
1653+ distro-info,
1654 python3-flake8,
1655 python3-mock,
1656 python3-pytest,
1657 python3-setuptools,
1658 python3-yaml
1659-Standards-Version: 4.3.0
1660+Standards-Version: 4.5.1
1661 Homepage: https://buy.ubuntu.com
1662 Vcs-Git: https://github.com/CanonicalLtd/ubuntu-advantage-script.git
1663 Vcs-Browser: https://github.com/CanonicalLtd/ubuntu-advantage-script
1664+Rules-Requires-Root: no
1665
1666 Package: ubuntu-advantage-tools
1667 Architecture: any
1668 Depends: ${misc:Depends},
1669 ${python3:Depends},
1670 ${shlibs:Depends},
1671+ distro-info,
1672 python3-pkg-resources,
1673 ${extra:Depends},
1674 Description: management tools for Ubuntu Advantage
1675@@ -36,7 +42,7 @@ Description: management tools for Ubuntu Advantage
1676 services in this package.
1677
1678 Package: ubuntu-advantage-pro
1679-Architecture: any
1680+Architecture: all
1681 Depends: ${misc:Depends}, ubuntu-advantage-tools (>=20.2)
1682 Replaces: ubuntu-advantage-tools (<<20.2)
1683 Breaks: ubuntu-advantage-tools (<<20.2)
1684diff --git a/debian/lintian-overrides b/debian/lintian-overrides
1685new file mode 100644
1686index 0000000..0d8ba20
1687--- /dev/null
1688+++ b/debian/lintian-overrides
1689@@ -0,0 +1,5 @@
1690+# False positive
1691+ubuntu-advantage-tools: maintainer-script-calls-service postinst:*
1692+
1693+# Ubuntu doesn't require init.d scripts
1694+ubuntu-advantage-tools: package-supports-alternative-init-but-no-init.d-script
1695diff --git a/debian/po/templates.pot b/debian/po/templates.pot
1696index 9dd5145..ffd2763 100644
1697--- a/debian/po/templates.pot
1698+++ b/debian/po/templates.pot
1699@@ -8,7 +8,7 @@ msgid ""
1700 msgstr ""
1701 "Project-Id-Version: ubuntu-advantage-tools\n"
1702 "Report-Msgid-Bugs-To: ubuntu-advantage-tools@packages.debian.org\n"
1703-"POT-Creation-Date: 2020-10-02 15:07-0600\n"
1704+"POT-Creation-Date: 2021-02-26 16:29+0100\n"
1705 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
1706 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
1707 "Language-Team: LANGUAGE <LL@li.org>\n"
1708@@ -32,5 +32,5 @@ msgstr ""
1709 #. Type: note
1710 #. Description
1711 #: ../ubuntu-advantage-tools.templates:1001
1712-msgid "sudo apt-get install ubuntu-advantage-pro"
1713+msgid "sudo apt install ubuntu-advantage-pro"
1714 msgstr ""
1715diff --git a/debian/rules b/debian/rules
1716index 888dcde..9bea522 100755
1717--- a/debian/rules
1718+++ b/debian/rules
1719@@ -19,8 +19,7 @@ APT_PKG_DEPS="apt (>= 1.8.1), apt-utils (>= 1.8.1), libapt-pkg5.90 (>= 1.8.1)"
1720 endif
1721
1722 %:
1723- dh $@ --with python3,bash-completion,systemd --buildsystem=pybuild \
1724- --no-start
1725+ dh $@ --with python3,bash-completion,systemd --buildsystem=pybuild
1726
1727 override_dh_auto_build:
1728 dh_auto_build
1729@@ -42,6 +41,15 @@ override_dh_gencontrol:
1730 echo extra:Depends=$(APT_PKG_DEPS) >> debian/ubuntu-advantage-tools.substvars
1731 dh_gencontrol
1732
1733+override_dh_systemd_enable:
1734+ dh_systemd_enable -pubuntu-advantage-pro ua-auto-attach.service
1735+ dh_systemd_enable -pubuntu-advantage-tools ua-reboot-cmds.service
1736+ dh_systemd_enable -pubuntu-advantage-tools ua-messaging.timer
1737+ dh_systemd_enable -pubuntu-advantage-tools ua-messaging.service
1738+
1739+override_dh_systemd_start:
1740+ dh_systemd_start -pubuntu-advantage-tools ua-messaging.timer
1741+
1742 override_dh_auto_install:
1743 dh_auto_install --destdir=debian/ubuntu-advantage-tools
1744 flist=$$(find $(CURDIR)/debian/ -type f -name version.py) && sed -i 's,@@PACKAGED_VERSION@@,$(DEB_VERSION),' $${flist:-did-not-find-version-py-for-replacement}
1745diff --git a/debian/ubuntu-advantage-tools.config b/debian/ubuntu-advantage-tools.config
1746new file mode 100644
1747index 0000000..92f4ee2
1748--- /dev/null
1749+++ b/debian/ubuntu-advantage-tools.config
1750@@ -0,0 +1,14 @@
1751+#!/bin/sh
1752+
1753+set -e
1754+
1755+. /usr/share/debconf/confmodule
1756+
1757+if dpkg --compare-versions "$PREVIOUS_PKG_VER" lt-nl "20.2~"; then
1758+ if dpkg --compare-versions "$PREVIOUS_PKG_VER" ge-nl "19.7~"; then
1759+ # Use debconf to alert the user to the additional
1760+ # ubuntu-advantage-pro package that should be installed
1761+ db_input high ubuntu-advantage-tools/suggest_pro_pkg || true
1762+ db_go || true
1763+ fi
1764+fi
1765diff --git a/debian/links b/debian/ubuntu-advantage-tools.links
1766similarity index 100%
1767rename from debian/links
1768rename to debian/ubuntu-advantage-tools.links
1769diff --git a/debian/manpages b/debian/ubuntu-advantage-tools.manpages
1770similarity index 100%
1771rename from debian/manpages
1772rename to debian/ubuntu-advantage-tools.manpages
1773diff --git a/debian/postinst b/debian/ubuntu-advantage-tools.postinst
1774similarity index 50%
1775rename from debian/postinst
1776rename to debian/ubuntu-advantage-tools.postinst
1777index 5d942f9..75581e3 100644
1778--- a/debian/postinst
1779+++ b/debian/ubuntu-advantage-tools.postinst
1780@@ -4,22 +4,39 @@ set -e
1781
1782 . /etc/os-release # For VERSION_ID
1783
1784+# Since UBUNTU_CODENAME isn't on trusty set it set a default if unknown
1785+if [ "" = "${UBUNTU_CODENAME}" ]; then
1786+ case "$VERSION_ID" in
1787+ 14.04) UBUNTU_CODENAME="trusty";;
1788+ *) UBUNTU_CODENAME="NO-UBUNTU_CODENAME-$VERSION_ID";;
1789+ esac
1790+fi
1791+
1792+# Needed even if this script doesn't call debconf, see:
1793+# https://lintian.debian.org/tags/postinst-does-not-load-confmodule.html
1794+. /usr/share/debconf/confmodule
1795
1796 APT_TRUSTED_KEY_DIR="/etc/apt/trusted.gpg.d"
1797 UA_KEYRING_DIR="/usr/share/keyrings/"
1798
1799 ESM_INFRA_KEY_TRUSTY="ubuntu-advantage-esm-infra-trusty.gpg"
1800+ESM_APPS_KEY="ubuntu-advantage-esm-apps.gpg"
1801
1802 APT_SRC_DIR="/etc/apt/sources.list.d"
1803 APT_PREFERENCES_DIR="/etc/apt/preferences.d"
1804 ESM_APT_SOURCE_FILE_PRECISE="$APT_SRC_DIR/ubuntu-esm-precise.list"
1805 ESM_APT_SOURCE_FILE_TRUSTY="$APT_SRC_DIR/ubuntu-esm-trusty.list"
1806 ESM_INFRA_OLD_APT_SOURCE_FILE_TRUSTY="$APT_SRC_DIR/ubuntu-esm-infra-trusty.list"
1807-ESM_INFRA_APT_SOURCE_FILE_TRUSTY="$APT_SRC_DIR/ubuntu-esm-infra.list"
1808+ESM_INFRA_APT_SOURCE_FILE="$APT_SRC_DIR/ubuntu-esm-infra.list"
1809+ESM_APPS_APT_SOURCE_FILE="$APT_SRC_DIR/ubuntu-esm-apps.list"
1810+FIPS_APT_SOURCE_FILE="$APT_SRC_DIR/ubuntu-fips.list"
1811
1812-ESM_APT_PREF_FILE_TRUSTY="/etc/apt/preferences.d/ubuntu-esm-trusty"
1813-ESM_INFRA_OLD_APT_PREF_FILE_TRUSTY="/etc/apt/preferences.d/ubuntu-esm-infra-trusty"
1814-ESM_INFRA_APT_PREF_FILE_TRUSTY="/etc/apt/preferences.d/ubuntu-esm-infra"
1815+OLD_CLIENT_FIPS_PPA="private-ppa.launchpad.net/ubuntu-advantage/fips/ubuntu"
1816+
1817+ESM_APT_PREF_FILE_TRUSTY="$APT_PREFERENCES_DIR/ubuntu-esm-trusty"
1818+ESM_INFRA_OLD_APT_PREF_FILE_TRUSTY="$APT_PREFERENCES_DIR/ubuntu-esm-infra-trusty"
1819+ESM_INFRA_APT_PREF_FILE="$APT_PREFERENCES_DIR/ubuntu-esm-infra"
1820+ESM_APPS_APT_PREF_FILE="$APT_PREFERENCES_DIR/ubuntu-esm-apps"
1821
1822 MYARCH="$(dpkg --print-architecture)"
1823 ESM_SUPPORTED_ARCHS="i386 amd64"
1824@@ -68,6 +85,50 @@ print(*ENTITLEMENT_CLASS_BY_NAME.keys(), sep=' ')
1825 }
1826
1827
1828+# Ubuntu LTS release all support-esm
1829+check_is_lts() {
1830+ release_name=$1
1831+ ubuntu-distro-info --supported-esm | grep -q "${release_name}"
1832+}
1833+
1834+
1835+# Check whether this series is under active ESM
1836+check_is_active_esm() {
1837+ release_name=$1
1838+ # Trusty doesn't support --series param
1839+ if [ "${release_name}" = "trusty" ]; then
1840+ return 0
1841+ else
1842+ _DAYS_UNTIL_ESM=$(ubuntu-distro-info --series "${release_name}" -yeol)
1843+ if [ "${_DAYS_UNTIL_ESM}" -lt "1" ]; then
1844+ return 0
1845+ fi
1846+ fi
1847+ return 1
1848+}
1849+
1850+# Check whether a given service is beta
1851+check_service_is_beta() {
1852+ service_name=$1
1853+ _IS_BETA_SVC=$(python3 -c "
1854+from uaclient.config import UAConfig
1855+from uaclient.entitlements import ENTITLEMENT_CLASS_BY_NAME
1856+ent_cls = ENTITLEMENT_CLASS_BY_NAME.get('${service_name}')
1857+if ent_cls:
1858+ cfg = UAConfig()
1859+ allow_beta = cfg.features.get('allow_beta', False)
1860+ print(all([ent_cls.is_beta, not allow_beta]))
1861+else:
1862+ print(True)
1863+")
1864+if [ "${_IS_BETA_SVC}" = "True" ]; then
1865+ return 0
1866+else
1867+ return 1
1868+fi
1869+}
1870+
1871+
1872 # Check cached service status from status.json and return 0 if enabled else 1
1873 check_service_is_enabled() {
1874 service_name=$1
1875@@ -92,47 +153,90 @@ if status:
1876
1877 unconfigure_esm() {
1878 if ! check_service_is_enabled esm-infra; then
1879- rm -f $APT_TRUSTED_KEY_DIR/ubuntu-esm*gpg # Remove previous esm keys
1880- rm -f $APT_TRUSTED_KEY_DIR/$ESM_INFRA_KEY_TRUSTY
1881- rm -f $ESM_INFRA_APT_SOURCE_FILE_TRUSTY
1882- rm -f $ESM_INFRA_OLD_APT_SOURCE_FILE_TRUSTY
1883- rm -f $ESM_APT_PREF_FILE_TRUSTY $ESM_INFRA_OLD_APT_PREF_FILE_TRUSTY
1884- rm -f $ESM_INFRA_APT_PREF_FILE_TRUSTY
1885+ rm -f "$APT_TRUSTED_KEY_DIR/ubuntu-esm*gpg" # Remove previous esm keys
1886+ rm -f "$APT_TRUSTED_KEY_DIR/$ESM_INFRA_KEY_TRUSTY"
1887+ rm -f "$ESM_INFRA_APT_SOURCE_FILE"
1888+ rm -f "$ESM_INFRA_OLD_APT_SOURCE_FILE_TRUSTY"
1889+ rm -f "$ESM_APT_PREF_FILE_TRUSTY" "$ESM_INFRA_OLD_APT_PREF_FILE_TRUSTY"
1890+ rm -f "$ESM_INFRA_APT_PREF_FILE"
1891+ fi
1892+ if ! check_service_is_enabled esm-apps; then
1893+ rm -f "$APT_TRUSTED_KEY_DIR/$ESM_APPS_KEY"
1894+ rm -f "$ESM_APPS_APT_SOURCE_FILE"
1895+ rm -f "$ESM_APPS_APT_PREF_FILE"
1896 fi
1897 }
1898
1899
1900-configure_esm() {
1901- rm -f $APT_TRUSTED_KEY_DIR/ubuntu-esm*gpg # Remove previous esm keys
1902- if [ ! -f "$APT_TRUSTED_KEY_DIR/$ESM_INFRA_KEY_TRUSTY" ]; then
1903- cp $UA_KEYRING_DIR/$ESM_INFRA_KEY_TRUSTY $APT_TRUSTED_KEY_DIR
1904+# Add visibility to a disabled ESM APT source by installing a GPG key and
1905+# preferences file to Pin never so packages won't get installed by apt update.
1906+install_esm_apt_key_and_source() {
1907+ service=$1 release=$2
1908+ apt_suite="${release}-${service}";
1909+ case "${service}" in
1910+ apps)
1911+ apt_origin="UbuntuESMApps"
1912+ apt_pref_file=${ESM_APPS_APT_PREF_FILE};
1913+ apt_source_file=${ESM_APPS_APT_SOURCE_FILE};
1914+ apt_key=${ESM_APPS_KEY};
1915+ ;;
1916+ infra)
1917+ apt_origin="UbuntuESM"
1918+ apt_pref_file=${ESM_INFRA_APT_PREF_FILE};
1919+ apt_source_file=${ESM_INFRA_APT_SOURCE_FILE};
1920+ apt_key=${ESM_INFRA_KEY_TRUSTY};
1921+ ;;
1922+ esac
1923+
1924+ # GPG key setup to avoid apt gpg key warnings
1925+ if [ ! -f "$APT_TRUSTED_KEY_DIR/$apt_key" ]; then
1926+ cp $UA_KEYRING_DIR/$apt_key $APT_TRUSTED_KEY_DIR
1927 fi
1928
1929- if [ -e "$ESM_APT_SOURCE_FILE_TRUSTY" ]; then
1930- mv $ESM_APT_SOURCE_FILE_TRUSTY $ESM_INFRA_APT_SOURCE_FILE_TRUSTY
1931- fi
1932- if [ -e "$ESM_APT_PREF_FILE_TRUSTY" ]; then
1933- mv $ESM_APT_PREF_FILE_TRUSTY $ESM_INFRA_APT_PREF_FILE_TRUSTY
1934+ # Migrate trusty legacy source list and preference file names
1935+ if [ "14.04" = "$VERSION_ID" ]; then
1936+ if [ -e "$ESM_APT_SOURCE_FILE_TRUSTY" ]; then
1937+ mv $ESM_APT_SOURCE_FILE_TRUSTY $ESM_INFRA_APT_SOURCE_FILE
1938+ fi
1939+ if [ -e "$ESM_APT_PREF_FILE_TRUSTY" ]; then
1940+ mv "$ESM_APT_PREF_FILE_TRUSTY" "$ESM_INFRA_APT_PREF_FILE"
1941+ fi
1942 fi
1943- if [ ! -e "$ESM_INFRA_APT_SOURCE_FILE_TRUSTY" ]; then
1944- cat > $ESM_INFRA_APT_SOURCE_FILE_TRUSTY <<EOF
1945+ # If preference file doesn't already exist, we aren't attached.
1946+ # Setup unauthenticated apt source list file and never-pin preference
1947+ if [ ! -e "${apt_source_file}" ]; then
1948+ # Unconfigured repo, so set it up as never-pinned
1949+ cat > ${apt_source_file} <<EOF
1950 # Written by ubuntu-advantage-tools
1951-deb https://esm.ubuntu.com/ubuntu trusty-infra-security main
1952-# deb-src https://esm.ubuntu.com/ubuntu trusty-infra-security main
1953+deb https://esm.ubuntu.com/${service}/ubuntu ${apt_suite}-security main
1954+# deb-src https://esm.ubuntu.com/${service}/ubuntu ${apt_suite}-security main
1955
1956-deb https://esm.ubuntu.com/ubuntu trusty-infra-updates main
1957-# deb-src https://esm.ubuntu.com/ubuntu trusty-infra-updates main
1958+deb https://esm.ubuntu.com/${service}/ubuntu ${apt_suite}-updates main
1959+# deb-src https://esm.ubuntu.com/${service}/ubuntu ${apt_suite}-updates main
1960 EOF
1961+
1962 # Automatically disable esm sources via apt preferences until enabled
1963- cat > $ESM_INFRA_APT_PREF_FILE_TRUSTY <<EOF
1964+ cat > "${apt_pref_file}" <<EOF
1965 # Written by ubuntu-advantage-tools
1966 Package: *
1967-Pin: release o=UbuntuESM, n=trusty
1968+Pin: release o=${apt_origin}, n=${release}
1969 Pin-Priority: never
1970 EOF
1971 fi
1972 }
1973
1974+configure_esm() {
1975+ rm -f $APT_TRUSTED_KEY_DIR/ubuntu-esm*gpg # Remove legacy esm keys
1976+ if check_is_active_esm "${UBUNTU_CODENAME}"; then
1977+ install_esm_apt_key_and_source "infra" "$UBUNTU_CODENAME"
1978+ fi
1979+ if ! check_service_is_beta esm-apps; then
1980+ if [ "${UBUNTU_CODENAME}" != "trusty" ]; then
1981+ install_esm_apt_key_and_source "apps" "$UBUNTU_CODENAME"
1982+ fi
1983+ fi
1984+}
1985+
1986
1987 # If held fips packages exist, we are on a FIPS PRO machine with FIPS enabled
1988 mark_reboot_for_fips_pro() {
1989@@ -143,11 +247,8 @@ mark_reboot_for_fips_pro() {
1990 }
1991
1992
1993-mark_reboot_cmds_as_needed() {
1994+add_notice() {
1995 msg_name=$1
1996- if [ ! -f "$REBOOT_CMD_MARKER_FILE" ]; then
1997- touch $REBOOT_CMD_MARKER_FILE
1998- fi
1999 python3 -c "
2000 from uaclient.config import UAConfig
2001 from uaclient.status import ${msg_name}
2002@@ -156,6 +257,14 @@ cfg.add_notice(label='', description=${msg_name})
2003 "
2004 }
2005
2006+mark_reboot_cmds_as_needed() {
2007+ msg_name=$1
2008+ if [ ! -f "$REBOOT_CMD_MARKER_FILE" ]; then
2009+ touch $REBOOT_CMD_MARKER_FILE
2010+ fi
2011+ add_notice "$msg_name"
2012+}
2013+
2014 case "$1" in
2015 configure)
2016 PREVIOUS_PKG_VER=$2
2017@@ -165,7 +274,7 @@ case "$1" in
2018 # https://github.com/CanonicalLtd/ubuntu-advantage-client/issues/693
2019 if [ -e "$ESM_APT_SOURCE_FILE_PRECISE" ]; then
2020 mv $ESM_APT_SOURCE_FILE_PRECISE \
2021- $ESM_INFRA_APT_SOURCE_FILE_TRUSTY
2022+ $ESM_INFRA_APT_SOURCE_FILE
2023 fi
2024
2025 # We changed the way we store public files in 19.5
2026@@ -182,11 +291,6 @@ case "$1" in
2027 rm -f $SYSTEMD_WANTS_AUTO_ATTACH_LINK
2028 rm -f $SYSTEMD_HELPER_ENABLED_AUTO_ATTACH_DSH
2029 rm -f $SYSTEMD_HELPER_ENABLED_WANTS_LINK
2030- # Use debconf to alert the user to the additional
2031- # ubuntu-advantage-pro package that should be installed
2032- . /usr/share/debconf/confmodule
2033- db_input high ubuntu-advantage-tools/suggest_pro_pkg || true
2034- db_go || true
2035 fi
2036 fi
2037
2038@@ -195,17 +299,23 @@ case "$1" in
2039 redact_ubuntu_release_from_ua_apt_filenames $APT_SRC_DIR
2040 redact_ubuntu_release_from_ua_apt_filenames $APT_PREFERENCES_DIR
2041
2042+ # Repo for FIPS packages changed from old client
2043+ if [ -f $FIPS_APT_SOURCE_FILE ]; then
2044+ if grep -q $OLD_CLIENT_FIPS_PPA $FIPS_APT_SOURCE_FILE; then
2045+ add_notice MESSAGE_FIPS_INSTALL_OUT_OF_DATE
2046+ fi
2047+ fi
2048+
2049 # CACHE_DIR is no longer present or used since 19.1
2050 rm -rf /var/cache/ubuntu-advantage-tools
2051 # machine-access cache files no longer present or used since 20.1
2052 rm -f /var/lib/ubuntu-advantage/private/machine-access-*.json
2053
2054- if [ "14.04" = "$VERSION_ID" ]; then
2055+ if check_is_lts "${UBUNTU_CODENAME}"; then
2056 if echo "$ESM_SUPPORTED_ARCHS" | grep -qw "$MYARCH"; then
2057- # 14.04 and supported arch
2058 configure_esm
2059 else
2060- # 14.04 and unsupported arch
2061+ # ESM supported release but unsupported arch
2062 unconfigure_esm
2063 fi
2064 fi
2065diff --git a/debian/postrm b/debian/ubuntu-advantage-tools.postrm
2066similarity index 100%
2067rename from debian/postrm
2068rename to debian/ubuntu-advantage-tools.postrm
2069diff --git a/debian/prerm b/debian/ubuntu-advantage-tools.prerm
2070similarity index 100%
2071rename from debian/prerm
2072rename to debian/ubuntu-advantage-tools.prerm
2073diff --git a/debian/ubuntu-advantage-tools.templates b/debian/ubuntu-advantage-tools.templates
2074index 1b5c6f8..ce1a66a 100644
2075--- a/debian/ubuntu-advantage-tools.templates
2076+++ b/debian/ubuntu-advantage-tools.templates
2077@@ -3,4 +3,4 @@ Type: note
2078 _Description: Ubuntu Pro support now requires ubuntu-advantage-pro
2079 To install run the following:
2080 .
2081- sudo apt-get install ubuntu-advantage-pro
2082+ sudo apt install ubuntu-advantage-pro
2083diff --git a/features/attach_invalidtoken.feature b/features/attach_invalidtoken.feature
2084index 4f3fc9a..6b8db52 100644
2085--- a/features/attach_invalidtoken.feature
2086+++ b/features/attach_invalidtoken.feature
2087@@ -12,7 +12,7 @@ Feature: Command behaviour when trying to attach a machine to an Ubuntu
2088 When I verify that running `ua attach INVALID_TOKEN` `as non-root` exits `1`
2089 Then I will see the following on stderr:
2090 """
2091- This command must be run as root (try using sudo)
2092+ This command must be run as root (try using sudo).
2093 """
2094 Examples: ubuntu release
2095 | release |
2096diff --git a/features/attach_validtoken.feature b/features/attach_validtoken.feature
2097index a25234f..d0e9612 100644
2098--- a/features/attach_validtoken.feature
2099+++ b/features/attach_validtoken.feature
2100@@ -2,26 +2,15 @@
2101 Feature: Command behaviour when attaching a machine to an Ubuntu Advantage
2102 subscription using a valid token
2103
2104- @series.all
2105+ @series.xenial
2106+ @series.bionic
2107+ @series.focal
2108 @uses.config.machine_type.lxd.container
2109 Scenario Outline: Attach command in a ubuntu lxd container
2110 Given a `<release>` machine with ubuntu-advantage-tools installed
2111 When I run `apt-get update` with sudo, retrying exit [100]
2112 And I run `apt-get install -y <downrev_pkg>` with sudo, retrying exit [100]
2113- When I verify that running ` --assume-yes --beta` `with sudo` exits `1`
2114- And I run `/usr/lib/update-notifier/apt-check --human-readable` as non-root
2115- Then if `<release>` in `trusty` and stdout matches regexp:
2116- """
2117- UA Infrastructure Extended Security Maintenance \(ESM\) is not enabled.
2118-
2119- \d+ update(s)? can be installed immediately.
2120- \d+ of these updates (is a|are) security update(s)?.
2121- """
2122- Then if `<release>` in `trusty` and stdout matches regexp:
2123- """
2124- Enable UA Infrastructure ESM to receive \d+ additional security update(s)?.
2125- See https://ubuntu.com/advantage or run: sudo ua status
2126- """
2127+ And I run `run-parts /etc/update-motd.d/` with sudo
2128 Then if `<release>` in `xenial or bionic` and stdout matches regexp:
2129 """
2130 \d+ package(s)? can be updated.
2131@@ -35,7 +24,7 @@ Feature: Command behaviour when attaching a machine to an Ubuntu Advantage
2132 When I attach `contract_token` with sudo
2133 Then stdout matches regexp:
2134 """
2135- ESM Infra enabled
2136+ UA Infra: ESM enabled
2137 """
2138 And stdout matches regexp:
2139 """
2140@@ -53,28 +42,74 @@ Feature: Command behaviour when attaching a machine to an Ubuntu Advantage
2141 """
2142 Enabling default service esm-infra
2143 """
2144- When I run `/usr/lib/update-notifier/apt-check --human-readable` as non-root
2145- Then if `<release>` in `trusty` and stdout matches regexp:
2146+ When I append the following on uaclient config:
2147+ """
2148+ features:
2149+ allow_beta: true
2150+ """
2151+ And I run `apt update` with sudo
2152+ And I run `python3 /usr/lib/ubuntu-advantage/ua_update_messaging.py` with sudo
2153+ And I run `apt install update-motd` with sudo, retrying exit [100]
2154+ And I run `update-motd` with sudo
2155+ Then if `<release>` in `focal` and stdout matches regexp:
2156 """
2157+ \* Introducing Extended Security Maintenance for Applications.
2158+ +Receive updates to over 30,000 software packages with your
2159+ +Ubuntu Advantage subscription. Free for personal use.
2160+
2161+ +https:\/\/ubuntu.com\/esm
2162+
2163 UA (Infra:|Infrastructure) Extended Security Maintenance \(ESM\) is enabled.
2164
2165 \d+ update(s)? can be installed immediately.
2166 \d+ of these updates (is|are) (fixed|provided) through UA (Infra:|Infrastructure) ESM.
2167 \d+ of these updates (is a|are) security update(s)?.
2168+ To see these additional updates run: apt list --upgradable
2169 """
2170- Then if `<release>` in `focal` and stdout matches regexp:
2171+ Then if `<release>` in `xenial or bionic` and stdout matches regexp:
2172 """
2173- UA (Infra:|Infrastructure) Extended Security Maintenance \(ESM\) is enabled.
2174+ \* Introducing Extended Security Maintenance for Applications.
2175+ +Receive updates to over 30,000 software packages with your
2176+ +Ubuntu Advantage subscription. Free for personal use.
2177
2178- \d+ update(s)? can be installed immediately.
2179- \d+ of these updates (is|are) (fixed|provided) through UA (Infra:|Infrastructure) ESM.
2180+ +https:\/\/ubuntu.com\/esm
2181+
2182+ UA Infra: Extended Security Maintenance \(ESM\) is enabled.
2183+
2184+ \d+ package(s)? can be updated.
2185 \d+ of these updates (is a|are) security update(s)?.
2186 To see these additional updates run: apt list --upgradable
2187 """
2188+ When I update contract to use `effectiveTo` as `days=-20`
2189+ And I run `python3 /usr/lib/ubuntu-advantage/ua_update_messaging.py` with sudo
2190+ And I run `update-motd` with sudo
2191 Then if `<release>` in `xenial or bionic` and stdout matches regexp:
2192 """
2193- \d+ package(s)? can be updated.
2194- \d+ of these updates (is a|are) security update(s)?.
2195+
2196+ \*Your UA Infra: ESM subscription has EXPIRED\*
2197+
2198+ \d+ additional security update\(s\) could have been applied via UA Infra: ESM.
2199+
2200+ Renew your UA services at https:\/\/ubuntu.com\/esm
2201+
2202+ """
2203+ Then if `<release>` in `xenial` and stdout matches regexp:
2204+ """
2205+
2206+ Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
2207+ applicable law.
2208+
2209+ """
2210+ When I run `apt upgrade --dry-run` with sudo
2211+ Then if `<release>` in `xenial` and stdout matches regexp:
2212+ """
2213+ \*Your UA Infra: ESM subscription has EXPIRED\*
2214+ Enabling UA Infra: ESM service would provide security updates for following
2215+ packages:
2216+ libkrad0
2217+ 1 esm-infra security update\(s\) NOT APPLIED. Renew your UA services at
2218+ https:\/\/ubuntu.com\/advantage
2219+
2220 """
2221 Examples: ubuntu release packages
2222 | release | downrev_pkg |
2223@@ -85,12 +120,12 @@ Feature: Command behaviour when attaching a machine to an Ubuntu Advantage
2224
2225 @series.all
2226 @uses.config.machine_type.aws.generic
2227- Scenario Outline: Attach command in a ubuntu lxd container
2228+ Scenario Outline: Attach command in an generic AWS Ubuntu VM
2229 Given a `<release>` machine with ubuntu-advantage-tools installed
2230 When I attach `contract_token` with sudo
2231 Then stdout matches regexp:
2232 """
2233- ESM Infra enabled
2234+ UA Infra: ESM enabled
2235 """
2236 And stdout matches regexp:
2237 """
2238@@ -123,7 +158,7 @@ Feature: Command behaviour when attaching a machine to an Ubuntu Advantage
2239 When I attach `contract_token` with sudo
2240 Then stdout matches regexp:
2241 """
2242- ESM Infra enabled
2243+ UA Infra: ESM enabled
2244 """
2245 And stdout matches regexp:
2246 """
2247@@ -156,7 +191,7 @@ Feature: Command behaviour when attaching a machine to an Ubuntu Advantage
2248 When I attach `contract_token` with sudo
2249 Then stdout matches regexp:
2250 """
2251- ESM Infra enabled
2252+ UA Infra: ESM enabled
2253 """
2254 And stdout matches regexp:
2255 """
2256diff --git a/features/attached_commands.feature b/features/attached_commands.feature
2257index 4b48f76..48b69eb 100644
2258--- a/features/attached_commands.feature
2259+++ b/features/attached_commands.feature
2260@@ -8,12 +8,12 @@ Feature: Command behaviour when attached to an UA subscription
2261 Then I verify that running `ua refresh` `as non-root` exits `1`
2262 And stderr matches regexp:
2263 """
2264- This command must be run as root \(try using sudo\)
2265+ This command must be run as root \(try using sudo\).
2266 """
2267 When I run `ua refresh` with sudo
2268 Then I will see the following on stdout:
2269 """
2270- Successfully refreshed your subscription
2271+ Successfully refreshed your subscription.
2272 """
2273
2274 Examples: ubuntu release
2275@@ -30,7 +30,7 @@ Feature: Command behaviour when attached to an UA subscription
2276 Then I verify that running `ua disable livepatch` `as non-root` exits `1`
2277 And stderr matches regexp:
2278 """
2279- This command must be run as root \(try using sudo\)
2280+ This command must be run as root \(try using sudo\).
2281 """
2282 And I verify that running `ua disable livepatch` `with sudo` exits `1`
2283 And I will see the following on stdout:
2284@@ -53,18 +53,18 @@ Feature: Command behaviour when attached to an UA subscription
2285 Then I verify that running `ua disable foobar` `as non-root` exits `1`
2286 And stderr matches regexp:
2287 """
2288- This command must be run as root \(try using sudo\)
2289+ This command must be run as root \(try using sudo\).
2290 """
2291 And I verify that running `ua disable foobar` `with sudo` exits `1`
2292 And stderr matches regexp:
2293 """
2294 Cannot disable unknown service 'foobar'.
2295- Try cc-eal, cis, esm-apps, esm-infra, fips, fips-updates, livepatch
2296+ Try cc-eal, cis, esm-apps, esm-infra, fips, fips-updates, livepatch.
2297 """
2298 And I verify that running `ua disable esm-infra` `as non-root` exits `1`
2299 And stderr matches regexp:
2300 """
2301- This command must be run as root \(try using sudo\)
2302+ This command must be run as root \(try using sudo\).
2303 """
2304 When I run `ua disable esm-infra` with sudo
2305 Then I will see the following on stdout:
2306@@ -91,7 +91,7 @@ Feature: Command behaviour when attached to an UA subscription
2307 Then I verify that running `ua detach` `as non-root` exits `1`
2308 And stderr matches regexp:
2309 """
2310- This command must be run as root \(try using sudo\)
2311+ This command must be run as root \(try using sudo\).
2312 """
2313 When I run `ua detach --assume-yes` with sudo
2314 Then I will see the following on stdout:
2315@@ -99,7 +99,7 @@ Feature: Command behaviour when attached to an UA subscription
2316 Detach will disable the following service:
2317 esm-infra
2318 Updating package lists
2319- This machine is now detached
2320+ This machine is now detached.
2321 """
2322 When I run `ua status --all` as non-root
2323 Then stdout matches regexp:
2324@@ -133,7 +133,7 @@ Feature: Command behaviour when attached to an UA subscription
2325 Then I verify that running `ua auto-attach` `as non-root` exits `1`
2326 And stderr matches regexp:
2327 """
2328- This command must be run as root \(try using sudo\)
2329+ This command must be run as root \(try using sudo\).
2330 """
2331 When I run `ua auto-attach` with sudo
2332 Then stderr matches regexp:
2333@@ -236,7 +236,7 @@ Feature: Command behaviour when attached to an UA subscription
2334 And stderr matches regexp:
2335 """
2336 Cannot disable unknown service 'foobar'.
2337- Try cc-eal, cis, esm-apps, esm-infra, fips, fips-updates, livepatch
2338+ Try cc-eal, cis, esm-apps, esm-infra, fips, fips-updates, livepatch.
2339 """
2340 When I run `ua status` with sudo
2341 Then stdout matches regexp:
2342@@ -258,13 +258,13 @@ Feature: Command behaviour when attached to an UA subscription
2343 Then I verify that running `ua disable foobar` `as non-root` exits `1`
2344 And stderr matches regexp:
2345 """
2346- This command must be run as root \(try using sudo\)
2347+ This command must be run as root \(try using sudo\).
2348 """
2349 And I verify that running `ua disable foobar` `with sudo` exits `1`
2350 And stderr matches regexp:
2351 """
2352 Cannot disable unknown service 'foobar'.
2353- Try cc-eal, cis, esm-apps, esm-infra, fips, fips-updates, livepatch
2354+ Try cc-eal, cis, esm-apps, esm-infra, fips, fips-updates, livepatch.
2355 """
2356 And I verify that running `ua disable esm-infra` `as non-root` exits `1`
2357 And stderr matches regexp:
2358diff --git a/features/attached_enable.feature b/features/attached_enable.feature
2359index 8a36fce..f6f0ec9 100644
2360--- a/features/attached_enable.feature
2361+++ b/features/attached_enable.feature
2362@@ -8,7 +8,7 @@ Feature: Enable command behaviour when attached to an UA subscription
2363 Then I verify that running `ua enable cc-eal` `as non-root` exits `1`
2364 And I will see the following on stderr:
2365 """
2366- This command must be run as root (try using sudo)
2367+ This command must be run as root (try using sudo).
2368 """
2369 And I verify that running `ua enable cc-eal --beta` `with sudo` exits `1`
2370 And I will see the following on stdout
2371@@ -24,7 +24,7 @@ Feature: Enable command behaviour when attached to an UA subscription
2372 And stderr matches regexp:
2373 """
2374 Cannot enable unknown service 'cc-eal'.
2375- Try esm-infra, fips, fips-updates, livepatch
2376+ Try esm-infra, fips, fips-updates, livepatch.
2377 """
2378
2379 Examples: ubuntu release
2380@@ -34,73 +34,39 @@ Feature: Enable command behaviour when attached to an UA subscription
2381 | trusty | CC EAL2 is not available for Ubuntu 14.04 LTS (Trusty Tahr). |
2382
2383 @series.all
2384- Scenario Outline: Attached enable a disabled beta service and unknown service in a ubuntu machine
2385+ Scenario Outline: Attached enable of a service in a ubuntu machine
2386 Given a `<release>` machine with ubuntu-advantage-tools installed
2387 When I attach `contract_token` with sudo
2388- Then I verify that running `ua enable cc-eal foobar` `as non-root` exits `1`
2389+ Then I verify that running `ua enable foobar` `as non-root` exits `1`
2390 And I will see the following on stderr:
2391 """
2392- This command must be run as root (try using sudo)
2393+ This command must be run as root (try using sudo).
2394 """
2395- And I verify that running `ua enable cc-eal foobar` `with sudo` exits `1`
2396+ And I verify that running `ua enable foobar` `with sudo` exits `1`
2397 And I will see the following on stdout:
2398 """
2399 One moment, checking your subscription first
2400 """
2401 And stderr matches regexp:
2402 """
2403- Cannot enable unknown service 'foobar, cc-eal'.
2404- Try esm-infra, fips, fips-updates, livepatch
2405- """
2406-
2407- Examples: ubuntu release
2408- | release |
2409- | bionic |
2410- | focal |
2411- | trusty |
2412- | xenial |
2413-
2414- @series.all
2415- Scenario Outline: Attached enable of an unknown service in a ubuntu machine
2416- Given a `<release>` machine with ubuntu-advantage-tools installed
2417- When I attach `contract_token` with sudo
2418- Then I verify that running `ua enable foobar` `as non-root` exits `1`
2419- And I will see the following on stderr:
2420- """
2421- This command must be run as root (try using sudo)
2422+ Cannot enable unknown service 'foobar'.
2423+ Try esm-infra, fips, fips-updates, livepatch.
2424 """
2425- And I verify that running `ua enable foobar` `with sudo` exits `1`
2426+ And I verify that running `ua enable cc-eal foobar` `with sudo` exits `1`
2427 And I will see the following on stdout:
2428 """
2429 One moment, checking your subscription first
2430 """
2431 And stderr matches regexp:
2432 """
2433- Cannot enable unknown service 'foobar'.
2434- Try esm-infra, fips, fips-updates, livepatch
2435- """
2436-
2437- Examples: ubuntu release
2438- | release |
2439- | bionic |
2440- | focal |
2441- | trusty |
2442- | xenial |
2443-
2444- @series.all
2445- Scenario Outline: Attached enable of a known service already enabled (UA Infra) in a ubuntu machine
2446- Given a `<release>` machine with ubuntu-advantage-tools installed
2447- When I attach `contract_token` with sudo
2448- Then I verify that running `ua enable esm-infra` `as non-root` exits `1`
2449- And I will see the following on stderr:
2450- """
2451- This command must be run as root (try using sudo)
2452+ Cannot enable unknown service 'foobar, cc-eal'.
2453+ Try esm-infra, fips, fips-updates, livepatch.
2454 """
2455 And I verify that running `ua enable esm-infra` `with sudo` exits `1`
2456 Then I will see the following on stdout:
2457 """
2458 One moment, checking your subscription first
2459- ESM Infra is already enabled.
2460+ UA Infra: ESM is already enabled.
2461 See: sudo ua status
2462 """
2463 When I run `apt-cache policy` with sudo
2464@@ -124,69 +90,41 @@ Feature: Enable command behaviour when attached to an UA subscription
2465 | trusty | libgit2-0 | https://esm.ubuntu.com/ubuntu/ |
2466 | xenial | libkrad0 | https://esm.ubuntu.com/infra/ubuntu |
2467
2468- @series.xenial
2469- @series.bionic
2470- @series.focal
2471- Scenario Outline: Attached enable of a know service shows update in a ubuntu machine
2472- Given a `<release>` machine with ubuntu-advantage-tools installed
2473- When I attach `contract_token` with sudo
2474- Then I verify that running `ua enable esm-infra` `with sudo` exits `1`
2475- And I will see the following on stdout:
2476- """
2477- One moment, checking your subscription first
2478- ESM Infra is already enabled.
2479- See: sudo ua status
2480- """
2481- When I run `apt install -y <pkg-version>` with sudo, retrying exit [100]
2482- And I run `apt update` with sudo
2483- Then stdout matches regexp
2484- """
2485- \d+ of the updates (is|are) from UA Infra: ESM
2486- """
2487- When I run `ua disable esm-infra` with sudo
2488- And I run `apt update` with sudo
2489- Then stdout does not match regexp
2490- """
2491- \d+ of the updates (is|are) from UA Infra: ESM
2492- """
2493-
2494- Examples: ubuntu release
2495- | release | pkg-version |
2496- | bionic | libkrad0=1.16-2build1 |
2497- | focal | hello=2.10-2ubuntu2 |
2498- | xenial | libkrad0=1.13.2+dfsg-5 |
2499-
2500 @series.all
2501 @uses.config.machine_type.lxd.container
2502 Scenario Outline: Attached enable of non-container services in a ubuntu lxd container
2503 Given a `<release>` machine with ubuntu-advantage-tools installed
2504 When I attach `contract_token` with sudo
2505- Then I verify that running `ua enable <service> <flag>` `as non-root` exits `1`
2506+ Then I verify that running `ua enable livepatch` `as non-root` exits `1`
2507 And I will see the following on stderr:
2508 """
2509- This command must be run as root (try using sudo)
2510+ This command must be run as root (try using sudo).
2511+ """
2512+ And I verify that running `ua enable livepatch` `with sudo` exits `1`
2513+ And I will see the following on stdout:
2514+ """
2515+ One moment, checking your subscription first
2516+ Cannot install Livepatch on a container.
2517+ """
2518+ And I verify that running `ua enable fips --assume-yes` `with sudo` exits `1`
2519+ And I will see the following on stdout:
2520+ """
2521+ One moment, checking your subscription first
2522+ Cannot install FIPS on a container.
2523 """
2524- And I verify that running `ua enable <service> <flag>` `with sudo` exits `1`
2525+ And I verify that running `ua enable fips-updates --assume-yes` `with sudo` exits `1`
2526 And I will see the following on stdout:
2527 """
2528 One moment, checking your subscription first
2529- Cannot install <title> on a container
2530+ Cannot install FIPS Updates on a container.
2531 """
2532
2533 Examples: Un-supported services in containers
2534- | release | service | title | flag |
2535- | bionic | livepatch | Livepatch | |
2536- | bionic | fips | FIPS | --assume-yes |
2537- | bionic | fips-updates | FIPS Updates | --assume-yes |
2538- | focal | livepatch | Livepatch | |
2539- | focal | fips | FIPS | --assume-yes |
2540- | focal | fips-updates | FIPS Updates | --assume-yes |
2541- | trusty | livepatch | Livepatch | |
2542- | trusty | fips | FIPS | --assume-yes |
2543- | trusty | fips-updates | FIPS Updates | --assume-yes |
2544- | xenial | livepatch | Livepatch | |
2545- | xenial | fips | FIPS | --assume-yes |
2546- | xenial | fips-updates | FIPS Updates | --assume-yes |
2547+ | release |
2548+ | bionic |
2549+ | focal |
2550+ | trusty |
2551+ | xenial |
2552
2553 @series.all
2554 Scenario Outline: Attached enable not entitled service in a ubuntu machine
2555@@ -195,26 +133,29 @@ Feature: Enable command behaviour when attached to an UA subscription
2556 Then I verify that running `ua enable <service>` `as non-root` exits `1`
2557 And I will see the following on stderr:
2558 """
2559- This command must be run as root (try using sudo)
2560+ This command must be run as root (try using sudo).
2561+ """
2562+ And I verify that running `ua enable cis --beta` `with sudo` exits `1`
2563+ And I will see the following on stdout:
2564+ """
2565+ One moment, checking your subscription first
2566+ This subscription is not entitled to CIS Audit
2567+ For more information see: https://ubuntu.com/advantage.
2568 """
2569- And I verify that running `ua enable <service> --beta` `with sudo` exits `1`
2570+ And I verify that running `ua enable esm-apps --beta` `with sudo` exits `1`
2571 And I will see the following on stdout:
2572 """
2573 One moment, checking your subscription first
2574- This subscription is not entitled to <title>.
2575- For more information see: https://ubuntu.com/advantage
2576+ This subscription is not entitled to UA Apps: ESM
2577+ For more information see: https://ubuntu.com/advantage.
2578 """
2579
2580 Examples: not entitled services
2581- | release | service | title |
2582- | bionic | cis | CIS Audit |
2583- | bionic | esm-apps | ESM Apps |
2584- | focal | cis | CIS Audit |
2585- | focal | esm-apps | ESM Apps |
2586- | trusty | cis | CIS Audit |
2587- | trusty | esm-apps | ESM Apps |
2588- | xenial | cis | CIS Audit |
2589- | xenial | esm-apps | ESM Apps |
2590+ | release |
2591+ | bionic |
2592+ | focal |
2593+ | trusty |
2594+ | xenial |
2595
2596 @series.focal
2597 @uses.config.machine_type.lxd.vm
2598@@ -309,7 +250,7 @@ Feature: Enable command behaviour when attached to an UA subscription
2599 Then stdout matches regexp:
2600 """
2601 Updating package lists
2602- ESM Infra enabled
2603+ UA Infra: ESM enabled
2604 Installing canonical-livepatch snap
2605 Canonical livepatch enabled
2606 """
2607@@ -321,7 +262,7 @@ Feature: Enable command behaviour when attached to an UA subscription
2608 Updating package lists
2609 Installing FIPS packages
2610 FIPS enabled
2611- A reboot is required to complete install
2612+ A reboot is required to complete install.
2613 """
2614 When I append the following on uaclient config:
2615 """
2616@@ -332,7 +273,7 @@ Feature: Enable command behaviour when attached to an UA subscription
2617 And I will see the following on stdout
2618 """
2619 One moment, checking your subscription first
2620- Cannot enable Livepatch when FIPS is enabled
2621+ Cannot enable Livepatch when FIPS is enabled.
2622 """
2623
2624 @series.bionic
2625@@ -343,7 +284,7 @@ Feature: Enable command behaviour when attached to an UA subscription
2626 Then stdout matches regexp:
2627 """
2628 Updating package lists
2629- ESM Infra enabled
2630+ UA Infra: ESM enabled
2631 Installing canonical-livepatch snap
2632 Canonical livepatch enabled
2633 """
2634@@ -356,7 +297,10 @@ Feature: Enable command behaviour when attached to an UA subscription
2635 And I will see the following on stdout
2636 """
2637 One moment, checking your subscription first
2638- Cannot enable FIPS when Livepatch is enabled
2639+ """
2640+ And I will see the following on stderr
2641+ """
2642+ Cannot enable FIPS when Livepatch is enabled.
2643 """
2644
2645 @series.xenial
2646@@ -368,7 +312,7 @@ Feature: Enable command behaviour when attached to an UA subscription
2647 Then stdout matches regexp:
2648 """
2649 Updating package lists
2650- ESM Infra enabled
2651+ UA Infra: ESM enabled
2652 """
2653 And stdout matches regexp:
2654 """
2655@@ -382,7 +326,7 @@ Feature: Enable command behaviour when attached to an UA subscription
2656 Updating package lists
2657 Installing FIPS packages
2658 FIPS enabled
2659- A reboot is required to complete install
2660+ A reboot is required to complete install.
2661 """
2662 When I run `ua status` with sudo
2663 Then stdout matches regexp:
2664@@ -407,7 +351,7 @@ Feature: Enable command behaviour when attached to an UA subscription
2665 When I attach `contract_token` with sudo
2666 Then stdout matches regexp:
2667 """
2668- ESM Infra enabled
2669+ UA Infra: ESM enabled
2670 """
2671 And stdout matches regexp:
2672 """
2673@@ -422,13 +366,13 @@ Feature: Enable command behaviour when attached to an UA subscription
2674 Updating package lists
2675 Installing FIPS Updates packages
2676 FIPS Updates enabled
2677- A reboot is required to complete install
2678+ A reboot is required to complete install.
2679 """
2680 When I verify that running `ua enable fips --assume-yes` `with sudo` exits `1`
2681 Then I will see the following on stdout
2682 """
2683 One moment, checking your subscription first
2684- Cannot enable FIPS when FIPS Updates is enabled
2685+ Cannot enable FIPS when FIPS Updates is enabled.
2686 """
2687
2688 Examples: ubuntu release
2689diff --git a/features/cloud.py b/features/cloud.py
2690index 4443345..13d3ea4 100644
2691--- a/features/cloud.py
2692+++ b/features/cloud.py
2693@@ -730,15 +730,3 @@ class LXDContainer(_LXD):
2694 def pycloudlib_cls(self):
2695 """Return the pycloudlib cls to be used as an api."""
2696 return pycloudlib.LXDContainer
2697-
2698- def manage_ssh_key(self, private_key_path: "Optional[str]" = None) -> None:
2699- """Create and manage ssh key pairs to be used in the cloud provider.
2700-
2701- On a LXD container, we do not use ssh keys to communicate with the
2702- instance. Therefore, this method should not be used.
2703-
2704- :param private_key_path:
2705- Location of the private key path to use. If None, the location
2706- will be a default location.
2707- """
2708- pass
2709diff --git a/features/environment.py b/features/environment.py
2710index 9a53933..0df3632 100644
2711--- a/features/environment.py
2712+++ b/features/environment.py
2713@@ -1,5 +1,4 @@
2714 import datetime
2715-import errno
2716 import os
2717 import itertools
2718 import tempfile
2719@@ -23,8 +22,8 @@ from features.util import emit_spinner_on_travis, lxc_get_property, build_debs
2720
2721 ALL_SUPPORTED_SERIES = ["bionic", "focal", "trusty", "xenial"]
2722
2723-DAILY_PPA = "http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu"
2724-DAILY_PPA_KEYID = "8A295C4FB8B190B7"
2725+DAILY_PPA = "http://ppa.launchpad.net/ua-client/daily/ubuntu"
2726+DAILY_PPA_KEYID = "6E34E7116C0BC933"
2727
2728 USERDATA_BLOCK_AUTO_ATTACH_IMG = """\
2729 #cloud-config
2730@@ -500,18 +499,14 @@ def after_step(context, step):
2731 )
2732 print("-- pull instance:{} {}".format(log_file, artifact_file))
2733 try:
2734- context.instance.pull_file(log_file, artifact_file)
2735- except IOError as e:
2736- if e.errno == errno.EACCES:
2737- result = context.instance.execute(
2738- ["cat", log_file], use_sudo=True
2739- )
2740- with open(artifact_file, "w") as stream:
2741- stream.write(result.stdout)
2742+ result = context.instance.execute(
2743+ ["cat", log_file], use_sudo=True
2744+ )
2745+ content = result.stdout if result.ok else ""
2746 except RuntimeError:
2747- # File did not exist
2748- with open(artifact_file, "w") as stream:
2749- stream.write("")
2750+ content = ""
2751+ with open(artifact_file, "w") as stream:
2752+ stream.write(content)
2753 for artifact_file, cmd in FAILURE_CMDS.items():
2754 result = context.instance.execute(cmd, use_sudo=True)
2755 artifact_file = os.path.join(artifacts_dir, artifact_file)
2756@@ -520,7 +515,9 @@ def after_step(context, step):
2757
2758
2759 def after_all(context):
2760- if context.config.image_clean:
2761+ if context.config.ppa == "":
2762+ print(" No custom images to delete. UACLIENT_BEHAVE_PPA is unset.")
2763+ elif context.config.image_clean:
2764 for key, image in context.series_image_name.items():
2765 if key == context.series_reuse_image:
2766 print(
2767@@ -635,6 +632,16 @@ def create_uat_image(context: Context, series: str) -> None:
2768 context.reuse_container[series],
2769 )
2770 return
2771+ ppa = context.config.ppa
2772+ if ppa == "":
2773+ image_name = context.config.cloud_manager.locate_image_name(series)
2774+ print(
2775+ "--- Unset UACLIENT_BEHAVE_PPA. Using ubuntu-advantage-tools "
2776+ + "from image: {}".format(image_name)
2777+ )
2778+ context.series_image_name[series] = image_name
2779+ return
2780+
2781 time_suffix = datetime.datetime.now().strftime("%s%f")
2782 deb_paths = []
2783 if context.config.build_pr:
2784@@ -741,11 +748,14 @@ def _install_uat_in_container(
2785 if "pro" in config.machine_type:
2786 features = "features:\n disable_auto_attach: true\n"
2787 conf_path = "/etc/ubuntu-advantage/uaclient.conf"
2788- cmd = "printf '{}' > /tmp/uaclient.conf".format(features)
2789+ cmd = "printf '{}' > /var/tmp/uaclient.conf".format(features)
2790 cmds.append('sh -c "{}"'.format(cmd))
2791 cmds.append(
2792- 'sudo -- sh -c "cat /tmp/uaclient.conf >> {}"'.format(conf_path)
2793+ 'sudo -- sh -c "cat /var/tmp/uaclient.conf >> {}"'.format(
2794+ conf_path
2795+ )
2796 )
2797+ cmds.append("sudo ua status --wait")
2798 cmds.append("sudo ua detach --assume-yes")
2799
2800 cmds.append(["ua", "version"])
2801diff --git a/features/gcp-ids.yaml b/features/gcp-ids.yaml
2802index a3ca5bd..fe37cfb 100644
2803--- a/features/gcp-ids.yaml
2804+++ b/features/gcp-ids.yaml
2805@@ -1,3 +1,3 @@
2806-xenial: "projects/ubuntu-os-cloud-image-proposed/global/images/testing-ubuntu-pro-1604-xenial-v20210205"
2807-bionic: "projects/ubuntu-os-cloud-image-proposed/global/images/testing-ubuntu-pro-1804-bionic-v20210205"
2808-focal: "projects/ubuntu-os-cloud-image-proposed/global/images/testing-ubuntu-pro-2004-focal-v20210205"
2809+xenial: "projects/ubuntu-os-pro-cloud/global/images/ubuntu-pro-1604-xenial-v20210329"
2810+bionic: "projects/ubuntu-os-pro-cloud/global/images/ubuntu-pro-1804-bionic-v20210329"
2811+focal: "projects/ubuntu-os-pro-cloud/global/images/ubuntu-pro-2004-focal-v20210329"
2812diff --git a/features/staging_commands.feature b/features/staging_commands.feature
2813index 90c212f..8946977 100644
2814--- a/features/staging_commands.feature
2815+++ b/features/staging_commands.feature
2816@@ -8,14 +8,15 @@ Feature: Enable command behaviour when attached to an UA staging subscription
2817 Then I verify that running `ua enable cc-eal` `as non-root` exits `1`
2818 And I will see the following on stderr:
2819 """
2820- This command must be run as root (try using sudo)
2821+ This command must be run as root (try using sudo).
2822 """
2823 When I run `ua enable cc-eal --beta` with sudo
2824 Then I will see the following on stdout:
2825 """
2826 One moment, checking your subscription first
2827- GPG key '/usr/share/keyrings/ubuntu-cc-keyring.gpg' not found
2828+ GPG key '/usr/share/keyrings/ubuntu-cc-keyring.gpg' not found.
2829 """
2830+
2831 @series.xenial
2832 @series.bionic
2833 @series.focal
2834@@ -46,6 +47,53 @@ Feature: Enable command behaviour when attached to an UA staging subscription
2835 \s*\*\*\* .* 500
2836 \s*500 https://esm.staging.ubuntu.com/apps/ubuntu <release>-apps-security/main amd64 Packages
2837 """
2838+ When I run `mkdir -p /var/lib/ubuntu-advantage/messages` with sudo
2839+ When I create the file `/var/lib/ubuntu-advantage/messages/apt-pre-invoke-no-packages-infra.tmpl` with the following
2840+ """
2841+ esm-infra-no {ESM_INFRA_PKG_COUNT}:{ESM_INFRA_PACKAGES}
2842+ """
2843+ When I create the file `/var/lib/ubuntu-advantage/messages/apt-pre-invoke-packages-infra.tmpl` with the following
2844+ """
2845+ esm-infra {ESM_INFRA_PKG_COUNT}:{ESM_INFRA_PACKAGES}
2846+ """
2847+ When I create the file `/var/lib/ubuntu-advantage/messages/apt-pre-invoke-packages-apps.tmpl` with the following
2848+ """
2849+ esm-apps {ESM_APPS_PKG_COUNT}:{ESM_APPS_PACKAGES}
2850+ """
2851+ When I create the file `/var/lib/ubuntu-advantage/messages/apt-pre-invoke-no-packages-apps.tmpl` with the following
2852+ """
2853+ esm-apps-no {ESM_APPS_PKG_COUNT}:{ESM_APPS_PACKAGES}
2854+ """
2855+ When I run `/usr/lib/ubuntu-advantage/apt-esm-hook process-templates` with sudo
2856+ When I run `cat /var/lib/ubuntu-advantage/messages/apt-pre-invoke-packages-apps` with sudo
2857+ Then stdout matches regexp:
2858+ """
2859+ esm-apps(-no)? \d+:(.*)?
2860+ """
2861+ When I run `cat /var/lib/ubuntu-advantage/messages/apt-pre-invoke-packages-infra` with sudo
2862+ Then stdout matches regexp:
2863+ """
2864+ esm-infra(-no)? \d+:(.*)?
2865+ """
2866+ When I create the file `/var/lib/ubuntu-advantage/messages/apt-pre-invoke-packages-infra.tmpl` with the following
2867+ """
2868+ esm-infra {ESM_INFRA_PKG_COUNT} {ESM_INFRA_PACKAGES}
2869+ """
2870+ When I create the file `/var/lib/ubuntu-advantage/messages/apt-pre-invoke-no-packages-infra.tmpl` with the following
2871+ """
2872+ esm-infra-no {ESM_INFRA_PKG_COUNT} {ESM_INFRA_PACKAGES}
2873+ """
2874+ When I create the file `/var/lib/ubuntu-advantage/messages/apt-pre-invoke-packages-apps.tmpl` with the following
2875+ """
2876+ esm-apps {ESM_APPS_PKG_COUNT} {ESM_APPS_PACKAGES}
2877+ """
2878+ When I run `apt upgrade --dry-run` with sudo
2879+ Then stdout matches regexp:
2880+ """
2881+ esm-apps(-no)? \d+.*
2882+
2883+ esm-infra(-no)? \d+.*
2884+ """
2885
2886 Examples: ubuntu release
2887 | release | apps-pkg |
2888@@ -81,7 +129,6 @@ Feature: Enable command behaviour when attached to an UA staging subscription
2889 FIPS support requires system reboot to complete configuration
2890 """
2891 And I verify that running `apt update` `with sudo` exits `0`
2892- And I verify that running `grep Traceback /var/log/ubuntu-advantage.log` `with sudo` exits `1`
2893 And I verify that `openssh-server` is installed from apt source `<fips-apt-source>`
2894 And I verify that `openssh-client` is installed from apt source `<fips-apt-source>`
2895 And I verify that `strongswan` is installed from apt source `<fips-apt-source>`
2896@@ -171,7 +218,6 @@ Feature: Enable command behaviour when attached to an UA staging subscription
2897 <fips-service> +yes enabled
2898 """
2899 And I verify that running `apt update` `with sudo` exits `0`
2900- And I verify that running `grep Traceback /var/log/ubuntu-advantage.log` `with sudo` exits `1`
2901 And I verify that `openssh-server` is installed from apt source `<fips-apt-source>`
2902 And I verify that `openssh-client` is installed from apt source `<fips-apt-source>`
2903 And I verify that `strongswan` is installed from apt source `<fips-apt-source>`
2904@@ -370,7 +416,6 @@ Feature: Enable command behaviour when attached to an UA staging subscription
2905 """
2906 When I reboot the `<release>` machine
2907 Then I verify that running `apt update` `with sudo` exits `0`
2908- And I verify that running `grep Traceback /var/log/ubuntu-advantage.log` `with sudo` exits `1`
2909 And I verify that `ubuntu-fips` is installed from apt source `<fips-apt-source>`
2910 And I verify that `openssh-server` is installed from apt source `<fips-apt-source>`
2911 And I verify that `openssh-client` is installed from apt source `<fips-apt-source>`
2912diff --git a/features/steps/steps.py b/features/steps/steps.py
2913index 928cfd7..8555ad6 100644
2914--- a/features/steps/steps.py
2915+++ b/features/steps/steps.py
2916@@ -18,7 +18,7 @@ from hamcrest import (
2917 from features.environment import create_uat_image
2918 from features.util import SLOW_CMDS, emit_spinner_on_travis, nullcontext
2919
2920-from uaclient.defaults import DEFAULT_CONFIG_FILE
2921+from uaclient.defaults import DEFAULT_CONFIG_FILE, DEFAULT_MACHINE_TOKEN_PATH
2922
2923
2924 CONTAINER_PREFIX = "ubuntu-behave-test-"
2925@@ -74,6 +74,16 @@ def given_a_machine(context, series):
2926 context.instance
2927 )
2928
2929+ if series == "trusty":
2930+ # On trusty, the distro-info package is not directly
2931+ # installed when we install the ubuntu-advantage-client
2932+ # deb. This is fixing that problem on trusty
2933+ when_i_run_command(
2934+ context=context,
2935+ command="apt-get install -f -y",
2936+ user_spec="with sudo",
2937+ )
2938+
2939 def cleanup_instance() -> None:
2940 if not context.config.destroy_instances:
2941 print(
2942@@ -118,7 +128,9 @@ def when_i_retry_run_command(context, command, user_spec, exit_codes):
2943
2944
2945 @when("I run `{command}` {user_spec}")
2946-def when_i_run_command(context, command, user_spec, verify_return=True):
2947+def when_i_run_command(
2948+ context, command, user_spec, verify_return=True, stdin=None
2949+):
2950 prefix = get_command_prefix_for_user_spec(user_spec)
2951 slow_cmd_spinner = nullcontext
2952 for slow_cmd in SLOW_CMDS:
2953@@ -128,7 +140,7 @@ def when_i_run_command(context, command, user_spec, verify_return=True):
2954
2955 full_cmd = prefix + shlex.split(command)
2956 with slow_cmd_spinner():
2957- result = context.instance.execute(full_cmd)
2958+ result = context.instance.execute(full_cmd, stdin=stdin)
2959
2960 process = subprocess.CompletedProcess(
2961 args=full_cmd,
2962@@ -148,6 +160,59 @@ def when_i_run_command(context, command, user_spec, verify_return=True):
2963 context.process = process
2964
2965
2966+@when("I fix `{issue}` by attaching to a subscription with `{token_type}`")
2967+def when_i_fix_a_issue_by_attaching(context, issue, token_type):
2968+ token = getattr(context.config, token_type)
2969+ when_i_run_command(
2970+ context=context,
2971+ command="ua fix {}".format(issue),
2972+ user_spec="with sudo",
2973+ stdin="a\n{}\n".format(token),
2974+ )
2975+
2976+
2977+@when("I fix `{issue}` by enabling required service")
2978+def when_i_fix_a_issue_by_enabling_service(context, issue):
2979+ when_i_run_command(
2980+ context=context,
2981+ command="ua fix {}".format(issue),
2982+ user_spec="with sudo",
2983+ stdin="e\n",
2984+ )
2985+
2986+
2987+@when("I fix `{issue}` by updating expired token")
2988+def when_i_fix_a_issue_by_updating_expired_token(context, issue):
2989+ token = getattr(context.config, "contract_token")
2990+ when_i_run_command(
2991+ context=context,
2992+ command="ua fix {}".format(issue),
2993+ user_spec="with sudo",
2994+ stdin="r\n{}\n".format(token),
2995+ )
2996+
2997+
2998+@when("I update contract to use `{contract_field}` as `{new_value}`")
2999+def when_i_update_contract_field_to_new_value(
3000+ context, contract_field, new_value
3001+):
3002+ if contract_field == "effectiveTo":
3003+ if "days=" in new_value: # Set timedelta offset from current day
3004+ now = datetime.datetime.utcnow()
3005+ contract_expiry = now + datetime.timedelta(days=int(new_value[5:]))
3006+ new_value = contract_expiry.strftime("%Y-%m-%dT00:00:00Z")
3007+ when_i_run_command(
3008+ context,
3009+ 'sed -i \'s/"{}": "[^"]*"/"{}": "{}"/g\' {}'.format(
3010+ contract_field,
3011+ contract_field,
3012+ new_value,
3013+ DEFAULT_MACHINE_TOKEN_PATH,
3014+ ),
3015+ user_spec="with sudo",
3016+ )
3017+
3018+
3019 @when("I attach `{token_type}` {user_spec}")
3020 def when_i_attach_staging_token(context, token_type, user_spec):
3021 token = getattr(context.config, token_type)
3022@@ -225,6 +290,13 @@ def then_conditional_stdout_matches_regexp(context, value1, value2):
3023 then_stdout_matches_regexp(context)
3024
3025
3026+@then("if `{value1}` in `{value2}` and stdout does not match regexp")
3027+def then_conditional_stdout_does_not_match_regexp(context, value1, value2):
3028+ """Only apply regex assertion if value1 in value2."""
3029+ if value1 in value2.split(" or "):
3030+ then_stdout_does_not_match_regexp(context)
3031+
3032+
3033 @then("stdout matches regexp")
3034 def then_stdout_matches_regexp(context):
3035 assert_that(context.process.stdout.strip(), matches_regexp(context.text))
3036diff --git a/features/ubuntu_pro.feature b/features/ubuntu_pro.feature
3037index 5dd0dbb..522781e 100644
3038--- a/features/ubuntu_pro.feature
3039+++ b/features/ubuntu_pro.feature
3040@@ -15,7 +15,17 @@ Feature: Command behaviour when attached to an UA subscription
3041 """
3042 And I run `ua auto-attach` with sudo
3043 And I run `ua status --wait` as non-root
3044- And I run `ua status --all` as non-root
3045+ And I run `ua status` as non-root
3046+ Then stdout matches regexp:
3047+ """
3048+ SERVICE ENTITLED STATUS DESCRIPTION
3049+ esm-apps +yes +enabled +UA Apps: Extended Security Maintenance \(ESM\)
3050+ esm-infra +yes +enabled +UA Infra: Extended Security Maintenance \(ESM\)
3051+ fips +yes +<fips-s> +NIST-certified FIPS modules
3052+ fips-updates +yes +<fips-s> +Uncertified security updates to FIPS modules
3053+ livepatch +yes +enabled +Canonical Livepatch service
3054+ """
3055+ When I run `ua status --all` as non-root
3056 Then stdout matches regexp:
3057 """
3058 SERVICE ENTITLED STATUS DESCRIPTION
3059@@ -96,7 +106,17 @@ Feature: Command behaviour when attached to an UA subscription
3060 """
3061 And I run `ua auto-attach` with sudo
3062 And I run `ua status --wait` as non-root
3063- And I run `ua status --all` as non-root
3064+ And I run `ua status` as non-root
3065+ Then stdout matches regexp:
3066+ """
3067+ SERVICE ENTITLED STATUS DESCRIPTION
3068+ esm-apps +yes +enabled +UA Apps: Extended Security Maintenance \(ESM\)
3069+ esm-infra +yes +enabled +UA Infra: Extended Security Maintenance \(ESM\)
3070+ fips +yes +<fips-s> +NIST-certified FIPS modules
3071+ fips-updates +yes +<fips-s> +Uncertified security updates to FIPS modules
3072+ livepatch +yes +enabled +Canonical Livepatch service
3073+ """
3074+ When I run `ua status --all` as non-root
3075 Then stdout matches regexp:
3076 """
3077 SERVICE ENTITLED STATUS DESCRIPTION
3078@@ -177,7 +197,17 @@ Feature: Command behaviour when attached to an UA subscription
3079 """
3080 And I run `ua auto-attach` with sudo
3081 And I run `ua status --wait` as non-root
3082- And I run `ua status --all` as non-root
3083+ And I run `ua status` as non-root
3084+ Then stdout matches regexp:
3085+ """
3086+ SERVICE ENTITLED STATUS DESCRIPTION
3087+ esm-apps +yes +enabled +UA Apps: Extended Security Maintenance \(ESM\)
3088+ esm-infra +yes +enabled +UA Infra: Extended Security Maintenance \(ESM\)
3089+ fips +yes +<fips-s> +NIST-certified FIPS modules
3090+ fips-updates +yes +<fips-s> +Uncertified security updates to FIPS modules
3091+ livepatch +yes +enabled +Canonical Livepatch service
3092+ """
3093+ When I run `ua status --all` as non-root
3094 Then stdout matches regexp:
3095 """
3096 SERVICE ENTITLED STATUS DESCRIPTION
3097@@ -207,7 +237,7 @@ Feature: Command behaviour when attached to an UA subscription
3098 https://esm.ubuntu.com/apps/ubuntu <release>-apps-security/main amd64 Packages
3099 """
3100 And I verify that running `apt update` `with sudo` exits `0`
3101- When I run `apt install -y <infra-pkg>/<release>-infra-security>` with sudo, retrying exit [100]
3102+ When I run `apt install -y <infra-pkg>/<release>-infra-security` with sudo, retrying exit [100]
3103 And I run `apt-cache policy <infra-pkg>` as non-root
3104 Then stdout matches regexp:
3105 """
3106@@ -218,7 +248,7 @@ Feature: Command behaviour when attached to an UA subscription
3107 """
3108 Installed: .*[~+]esm
3109 """
3110- When I run `apt install -y <apps-pkg>/<release>-apps-security>` with sudo, retrying exit [100]
3111+ When I run `apt install -y <apps-pkg>/<release>-apps-security` with sudo, retrying exit [100]
3112 And I run `apt-cache policy <apps-pkg>` as non-root
3113 Then stdout matches regexp:
3114 """
3115@@ -238,10 +268,10 @@ Feature: Command behaviour when attached to an UA subscription
3116 """
3117
3118 Examples: ubuntu release
3119- | release | cc-eal-s | cis-s | infra-pkg | apps-pkg |
3120- | xenial | disabled | disabled | libkrad0 | jq |
3121- | bionic | n/a | disabled | libkrad0 | bundler |
3122- | focal | n/a | n/a | hello | ant |
3123+ | release | fips-s | cc-eal-s | cis-s | infra-pkg | apps-pkg |
3124+ | xenial | n/a | disabled | disabled | libkrad0 | jq |
3125+ | bionic | n/a | n/a | disabled | libkrad0 | bundler |
3126+ | focal | n/a | n/a | n/a | hello | ant |
3127
3128 @series.trusty
3129 @uses.config.machine_type.aws.pro
3130diff --git a/features/unattached_commands.feature b/features/unattached_commands.feature
3131index 0f73393..e2877ba 100644
3132--- a/features/unattached_commands.feature
3133+++ b/features/unattached_commands.feature
3134@@ -6,7 +6,7 @@ Feature: Command behaviour when unattached
3135 When I verify that running `ua auto-attach` `as non-root` exits `1`
3136 Then stderr matches regexp:
3137 """
3138- This command must be run as root \(try using sudo\)
3139+ This command must be run as root \(try using sudo\).
3140 """
3141 When I run `ua auto-attach` with sudo
3142 Then stderr matches regexp:
3143@@ -22,13 +22,93 @@ Feature: Command behaviour when unattached
3144 | trusty | nocloudnet |
3145 | xenial | lxd |
3146
3147+ @series.trusty
3148+ @series.xenial
3149+ Scenario Outline: Disabled unattached APT policy apt-hook for infra and apps
3150+ Given a `<release>` machine with ubuntu-advantage-tools installed
3151+ When I run `apt update` with sudo
3152+ When I run `apt-cache policy` with sudo
3153+ Then if `<release>` in `trusty` and stdout matches regexp:
3154+ """
3155+ -32768 <esm-infra-url> <release>-infra-security/main amd64 Packages
3156+ """
3157+ Then if `<release>` in `xenial` and stdout matches regexp:
3158+ """
3159+ -32768 <esm-infra-url> <release>-infra-updates/main amd64 Packages
3160+ """
3161+ Then if `<release>` in `trusty or xenial` and stdout does not match regexp:
3162+ """
3163+ -32768 <esm-apps-url> <release>-apps-updates/main amd64 Packages
3164+ """
3165+ Then if `<release>` in `trusty or xenial` and stdout does not match regexp:
3166+ """
3167+ -32768 <esm-apps-url> <release>-apps-security/main amd64 Packages
3168+ """
3169+ When I append the following on uaclient config:
3170+ """
3171+ features:
3172+ allow_beta: true
3173+ """
3174+ And I run `dpkg-reconfigure ubuntu-advantage-tools` with sudo
3175+ And I run `apt-get update` with sudo
3176+ When I run `apt-cache policy` with sudo
3177+ Then if `<release>` in `trusty` and stdout does not match regexp:
3178+ """
3179+ -32768 <esm-apps-url> <release>-apps-updates/main amd64 Packages
3180+ """
3181+ Then if `<release>` in `trusty` and stdout does not match regexp:
3182+ """
3183+ -32768 <esm-apps-url> <release>-apps-security/main amd64 Packages
3184+ """
3185+ Then if `<release>` in `xenial` and stdout matches regexp:
3186+ """
3187+ -32768 <esm-apps-url> <release>-apps-updates/main amd64 Packages
3188+ """
3189+ Then if `<release>` in `xenial` and stdout matches regexp:
3190+ """
3191+ -32768 <esm-apps-url> <release>-apps-security/main amd64 Packages
3192+ """
3193+ When I append the following on uaclient config:
3194+ """
3195+ features:
3196+ allow_beta: true
3197+ """
3198+ And I run `python3 /usr/lib/ubuntu-advantage/ua_update_messaging.py` with sudo
3199+ And I run `run-parts /etc/update-motd.d/` with sudo
3200+ Then if `<release>` in `xenial` and stdout matches regexp:
3201+ """
3202+ \* Introducing Extended Security Maintenance for Applications.
3203+ +Receive updates to over 30,000 software packages with your
3204+ +Ubuntu Advantage subscription. Free for personal use.
3205+
3206+ +https:\/\/ubuntu.com\/esm
3207+
3208+ UA Infra: Extended Security Maintenance \(ESM\) is not enabled.
3209+ """
3210+ # Check that json hook is installed properly
3211+ When I run `ls /usr/lib/ubuntu-advantage` with sudo
3212+ Then stdout matches regexp:
3213+ """
3214+ apt-esm-json-hook
3215+ """
3216+ When I run `cat /etc/apt/apt.conf.d/20apt-esm-hook.conf` with sudo
3217+ Then stdout matches regexp:
3218+ """
3219+ apt-esm-json-hook
3220+ """
3221+
3222+ Examples: ubuntu release
3223+ | release | esm-infra-url | esm-apps-url |
3224+ | trusty | https://esm.ubuntu.com/ubuntu/ | NOTTESTED |
3225+ | xenial | https://esm.ubuntu.com/infra/ubuntu | https://esm.ubuntu.com/apps/ubuntu |
3226+
3227 @series.all
3228 Scenario Outline: Unattached commands that requires enabled user in a ubuntu machine
3229 Given a `<release>` machine with ubuntu-advantage-tools installed
3230 When I verify that running `ua <command>` `as non-root` exits `1`
3231 Then I will see the following on stderr:
3232 """
3233- This command must be run as root (try using sudo)
3234+ This command must be run as root (try using sudo).
3235 """
3236 When I verify that running `ua <command>` `with sudo` exits `1`
3237 Then stderr matches regexp:
3238@@ -54,7 +134,7 @@ Feature: Command behaviour when unattached
3239 When I verify that running `ua <command> <service>` `as non-root` exits `1`
3240 Then I will see the following on stderr:
3241 """
3242- This command must be run as root (try using sudo)
3243+ This command must be run as root (try using sudo).
3244 """
3245 When I verify that running `ua <command> <service>` `with sudo` exits `1`
3246 Then stderr matches regexp:
3247@@ -121,3 +201,308 @@ Feature: Command behaviour when unattached
3248 | focal |
3249 | trusty |
3250 | xenial |
3251+
3252+ @series.focal
3253+ Scenario Outline: Fix command on an unattached machine
3254+ Given a `<release>` machine with ubuntu-advantage-tools installed
3255+ When I verify that running `ua fix CVE-1800-123456` `as non-root` exits `1`
3256+ Then I will see the following on stderr:
3257+ """
3258+ Error: CVE-1800-123456 not found.
3259+ """
3260+ When I verify that running `ua fix USN-12345-12` `as non-root` exits `1`
3261+ Then I will see the following on stderr:
3262+ """
3263+ Error: USN-12345-12 not found.
3264+ """
3265+ When I verify that running `ua fix CVE-12345678-12` `as non-root` exits `1`
3266+ Then I will see the following on stderr:
3267+ """
3268+ Error: issue "CVE-12345678-12" is not recognized.
3269+ Usage: "ua fix CVE-yyyy-nnnn" or "ua fix USN-nnnn"
3270+ """
3271+ When I verify that running `ua fix USN-12345678-12` `as non-root` exits `1`
3272+ Then I will see the following on stderr:
3273+ """
3274+ Error: issue "USN-12345678-12" is not recognized.
3275+ Usage: "ua fix CVE-yyyy-nnnn" or "ua fix USN-nnnn"
3276+ """
3277+ When I run `apt install -y libawl-php=0.60-1 --allow-downgrades` with sudo
3278+ And I run `ua fix USN-4539-1` with sudo
3279+ Then stdout matches regexp:
3280+ """
3281+ USN-4539-1: AWL vulnerability
3282+ Found CVEs:
3283+ https://ubuntu.com/security/CVE-2020-11728
3284+ 1 affected package is installed: awl
3285+ \(1/1\) awl:
3286+ A fix is available in Ubuntu standard updates.
3287+ .*\{ apt update && apt install --only-upgrade -y libawl-php \}.*
3288+ .*✔.* USN-4539-1 is resolved.
3289+ """
3290+ When I run `ua fix CVE-2020-28196` as non-root
3291+ Then stdout matches regexp:
3292+ """
3293+ CVE-2020-28196: Kerberos vulnerability
3294+ https://ubuntu.com/security/CVE-2020-28196
3295+ 1 affected package is installed: krb5
3296+ \(1/1\) krb5:
3297+ A fix is available in Ubuntu standard updates.
3298+ The update is already installed.
3299+ .*✔.* CVE-2020-28196 is resolved.
3300+ """
3301+
3302+ Examples: ubuntu release details
3303+ | release |
3304+ | focal |
3305+
3306+ @series.xenial
3307+ Scenario Outline: Fix command on an unattached machine
3308+ Given a `<release>` machine with ubuntu-advantage-tools installed
3309+ When I run `apt install -y libawl-php` with sudo
3310+ And I run `ua fix USN-4539-1` as non-root
3311+ Then stdout matches regexp:
3312+ """
3313+ USN-4539-1: AWL vulnerability
3314+ Found CVEs:
3315+ https://ubuntu.com/security/CVE-2020-11728
3316+ 1 affected package is installed: awl
3317+ \(1/1\) awl:
3318+ Ubuntu security engineers are investigating this issue.
3319+ 1 package is still affected: awl
3320+ .*✘.* USN-4539-1 is not resolved.
3321+ """
3322+ When I run `ua fix CVE-2020-28196` as non-root
3323+ Then stdout matches regexp:
3324+ """
3325+ CVE-2020-28196: Kerberos vulnerability
3326+ https://ubuntu.com/security/CVE-2020-28196
3327+ 1 affected package is installed: krb5
3328+ \(1/1\) krb5:
3329+ A fix is available in Ubuntu standard updates.
3330+ The update is already installed.
3331+ .*✔.* CVE-2020-28196 is resolved.
3332+ """
3333+ When I run `DEBIAN_FRONTEND=noninteractive apt-get install -y expat=2.1.0-7 swish-e matanza ghostscript` with sudo
3334+ And I verify that running `ua fix CVE-2017-9233` `with sudo` exits `1`
3335+ Then stdout matches regexp:
3336+ """
3337+ CVE-2017-9233: Expat vulnerability
3338+ https://ubuntu.com/security/CVE-2017-9233
3339+ 3 affected packages are installed: expat, matanza, swish-e
3340+ \(1/3, 2/3\) matanza, swish-e:
3341+ Ubuntu security engineers are investigating this issue.
3342+ """
3343+ And stderr matches regexp:
3344+ """
3345+ Error: CVE-2017-9233 metadata defines no fixed version for expat.
3346+ 3 packages are still affected: expat, matanza, swish-e
3347+ .*✘.* CVE-2017-9233 is not resolved.
3348+ """
3349+
3350+ Examples: ubuntu release details
3351+ | release |
3352+ | xenial |
3353+
3354+ @uses.config.contract_token
3355+ @series.trusty
3356+ Scenario Outline: Fix command on an unattached machine
3357+ Given a `<release>` machine with ubuntu-advantage-tools installed
3358+ When I run `ua fix USN-4539-1` as non-root
3359+ Then stdout matches regexp:
3360+ """
3361+ USN-4539-1: AWL vulnerability
3362+ Found CVEs:
3363+ https://ubuntu.com/security/CVE-2020-11728
3364+ No affected packages are installed.
3365+ .*✔.* USN-4539-1 does not affect your system.
3366+ """
3367+ When I run `ua fix CVE-2020-15180` as non-root
3368+ Then stdout matches regexp:
3369+ """
3370+ CVE-2020-15180: MariaDB vulnerabilities
3371+ https://ubuntu.com/security/CVE-2020-15180
3372+ No affected packages are installed.
3373+ .*✔.* CVE-2020-15180 does not affect your system.
3374+ """
3375+ When I run `ua fix CVE-2020-28196` as non-root
3376+ Then stdout matches regexp:
3377+ """
3378+ CVE-2020-28196: Kerberos vulnerability
3379+ https://ubuntu.com/security/CVE-2020-28196
3380+ 1 affected package is installed: krb5
3381+ \(1/1\) krb5:
3382+ A fix is available in UA Infra.
3383+ Package fixes cannot be installed.
3384+ To install them, run this command as root \(try using sudo\)
3385+ 1 package is still affected: krb5
3386+ .*✘.* CVE-2020-28196 is not resolved.
3387+ """
3388+ When I fix `USN-4747-2` by attaching to a subscription with `contract_token`
3389+ Then stdout matches regexp:
3390+ """
3391+ USN-4747-2: GNU Screen vulnerability
3392+ Found CVEs:
3393+ https://ubuntu.com/security/CVE-2021-26937
3394+ 1 affected package is installed: screen
3395+ \(1/1\) screen:
3396+ A fix is available in UA Infra.
3397+ The update is not installed because this system is not attached to a
3398+ subscription.
3399+
3400+ Choose: \[S\]ubscribe at ubuntu.com \[A\]ttach existing token \[C\]ancel
3401+ > Enter your token \(from https://ubuntu.com/advantage\) to attach this system:
3402+ > .*\{ ua attach .*\}.*
3403+ Updating package lists
3404+ UA Infra: ESM enabled
3405+ """
3406+ And stdout matches regexp:
3407+ """
3408+ .*\{ apt update && apt install --only-upgrade -y screen \}.*
3409+ .*✔.* USN-4747-2 is resolved.
3410+ """
3411+ When I run `apt-get install -y screen=4.1.0~20120320gitdb59704-9 --force-yes` with sudo
3412+ And I run `ua disable esm-infra` with sudo
3413+ And I fix `USN-4747-2` by enabling required service
3414+ Then stdout matches regexp:
3415+ """
3416+ USN-4747-2: GNU Screen vulnerability
3417+ Found CVEs:
3418+ https://ubuntu.com/security/CVE-2021-26937
3419+ 1 affected package is installed: screen
3420+ \(1/1\) screen:
3421+ A fix is available in UA Infra.
3422+ The update is not installed because this system does not have
3423+ esm-infra enabled.
3424+
3425+ Choose: \[E\]nable esm-infra \[C\]ancel
3426+ > .*\{ ua enable esm-infra \}.*
3427+ One moment, checking your subscription first
3428+ Updating package lists
3429+ UA Infra: ESM enabled
3430+ """
3431+ And stdout matches regexp:
3432+ """
3433+ .*\{ apt update && apt install --only-upgrade -y screen \}.*
3434+ .*✔.* USN-4747-2 is resolved.
3435+ """
3436+ When I run `apt-get install -y screen=4.1.0~20120320gitdb59704-9 --force-yes` with sudo
3437+ And I update contract to use `effectiveTo` as `1999-12-01T00:00:00Z`
3438+ And I fix `USN-4747-2` by updating expired token
3439+ Then stdout matches regexp:
3440+ """
3441+ USN-4747-2: GNU Screen vulnerability
3442+ Found CVEs:
3443+ https://ubuntu.com/security/CVE-2021-26937
3444+ 1 affected package is installed: screen
3445+ \(1/1\) screen:
3446+ A fix is available in UA Infra.
3447+ The update is not installed because this system is attached to an
3448+ expired subscription.
3449+
3450+ Choose: \[R\]enew your subscription \(at https://ubuntu.com/advantage\) \[C\]ancel
3451+ > Enter your new token to renew UA subscription on this system:
3452+ > .*\{ ua detach \}.*
3453+ Detach will disable the following service:
3454+ esm-infra
3455+ Updating package lists
3456+ This machine is now detached.
3457+ .*\{ ua attach .* \}.*
3458+ Updating package lists
3459+ UA Infra: ESM enabled
3460+ """
3461+ And stdout matches regexp:
3462+ """
3463+ .*\{ apt update && apt install --only-upgrade -y screen \}.*
3464+ .*✔.* USN-4747-2 is resolved.
3465+ """
3466+
3467+ Examples: ubuntu release
3468+ | release |
3469+ | trusty |
3470+
3471+ @series.bionic
3472+ Scenario: Fix command on an unattached machine
3473+ Given a `bionic` machine with ubuntu-advantage-tools installed
3474+ When I verify that running `ua fix CVE-1800-123456` `as non-root` exits `1`
3475+ Then I will see the following on stderr:
3476+ """
3477+ Error: CVE-1800-123456 not found.
3478+ """
3479+ When I verify that running `ua fix USN-12345-12` `as non-root` exits `1`
3480+ Then I will see the following on stderr:
3481+ """
3482+ Error: USN-12345-12 not found.
3483+ """
3484+ When I verify that running `ua fix CVE-12345678-12` `as non-root` exits `1`
3485+ Then I will see the following on stderr:
3486+ """
3487+ Error: issue "CVE-12345678-12" is not recognized.
3488+ Usage: "ua fix CVE-yyyy-nnnn" or "ua fix USN-nnnn"
3489+ """
3490+ When I verify that running `ua fix USN-12345678-12` `as non-root` exits `1`
3491+ Then I will see the following on stderr:
3492+ """
3493+ Error: issue "USN-12345678-12" is not recognized.
3494+ Usage: "ua fix CVE-yyyy-nnnn" or "ua fix USN-nnnn"
3495+ """
3496+ When I run `apt install -y libawl-php` with sudo
3497+ And I run `ua fix USN-4539-1` as non-root
3498+ Then stdout matches regexp:
3499+ """
3500+ USN-4539-1: AWL vulnerability
3501+ Found CVEs:
3502+ https://ubuntu.com/security/CVE-2020-11728
3503+ 1 affected package is installed: awl
3504+ \(1/1\) awl:
3505+ Ubuntu security engineers are investigating this issue.
3506+ 1 package is still affected: awl
3507+ .*✘.* USN-4539-1 is not resolved.
3508+ """
3509+ When I run `ua fix CVE-2020-28196` as non-root
3510+ Then stdout matches regexp:
3511+ """
3512+ CVE-2020-28196: Kerberos vulnerability
3513+ https://ubuntu.com/security/CVE-2020-28196
3514+ 1 affected package is installed: krb5
3515+ \(1/1\) krb5:
3516+ A fix is available in Ubuntu standard updates.
3517+ The update is already installed.
3518+ .*✔.* CVE-2020-28196 is resolved.
3519+ """
3520+ When I run `apt-get install xterm=330-1ubuntu2 -y` with sudo
3521+ And I run `ua fix CVE-2021-27135` as non-root
3522+ Then stdout matches regexp:
3523+ """
3524+ CVE-2021-27135: xterm vulnerability
3525+ https://ubuntu.com/security/CVE-2021-27135
3526+ 1 affected package is installed: xterm
3527+ \(1/1\) xterm:
3528+ A fix is available in Ubuntu standard updates.
3529+ Package fixes cannot be installed.
3530+ To install them, run this command as root \(try using sudo\)
3531+ 1 package is still affected: xterm
3532+ .*✘.* CVE-2021-27135 is not resolved.
3533+ """
3534+ When I run `ua fix CVE-2021-27135` with sudo
3535+ Then stdout matches regexp:
3536+ """
3537+ CVE-2021-27135: xterm vulnerability
3538+ https://ubuntu.com/security/CVE-2021-27135
3539+ 1 affected package is installed: xterm
3540+ \(1/1\) xterm:
3541+ A fix is available in Ubuntu standard updates.
3542+ .*\{ apt update && apt install --only-upgrade -y xterm \}.*
3543+ .*✔.* CVE-2021-27135 is resolved.
3544+ """
3545+ When I run `ua fix CVE-2021-27135` with sudo
3546+ Then stdout matches regexp:
3547+ """
3548+ CVE-2021-27135: xterm vulnerability
3549+ https://ubuntu.com/security/CVE-2021-27135
3550+ 1 affected package is installed: xterm
3551+ \(1/1\) xterm:
3552+ A fix is available in Ubuntu standard updates.
3553+ The update is already installed.
3554+ .*✔.* CVE-2021-27135 is resolved.
3555+ """
3556diff --git a/features/util.py b/features/util.py
3557index 8af945c..cacd992 100644
3558--- a/features/util.py
3559+++ b/features/util.py
3560@@ -18,33 +18,6 @@ LXC_PROPERTY_MAP = {
3561 SLOW_CMDS = ["do-release-upgrade"] # Commands which will emit dots on travis
3562 SOURCE_PR_TGZ = os.path.join(tempfile.gettempdir(), "pr_source.tar.gz")
3563 UA_DEBS = frozenset({"ubuntu-advantage-tools.deb", "ubuntu-advantage-pro.deb"})
3564-VM_PROFILE_TMPL = "behave-{}"
3565-
3566-
3567-# For Xenial and Bionic vendor-data required to setup lxd-agent
3568-# Additionally xenial needs to launch images:ubuntu/16.04/cloud
3569-# because it contains the HWE kernel which has vhost-vsock support
3570-LXC_SETUP_VENDORDATA = textwrap.dedent(
3571- """\
3572- config:
3573- user.vendor-data: |
3574- #cloud-config
3575- {custom_cfg}
3576- write_files:
3577- - path: /var/lib/cloud/scripts/per-once/setup-lxc.sh
3578- encoding: b64
3579- permissions: '0755'
3580- owner: root:root
3581- content: |
3582- IyEvYmluL3NoCmlmICEgZ3JlcCBseGRfY29uZmlnIC9wcm9jL21vdW50czsgdGhlbgogICAgbWtk
3583- aXIgLXAgL3J1bi9seGRhZ2VudAogICAgbW91bnQgLXQgOXAgY29uZmlnIC9ydW4vbHhkYWdlbnQK
3584- ICAgIFZJUlQ9JChzeXN0ZW1kLWRldGVjdC12aXJ0KQogICAgY2FzZSAkVklSVCBpbgogICAgICAg
3585- IHFlbXV8a3ZtKQogICAgICAgICAgICAoY2QgL3J1bi9seGRhZ2VudC8gJiYgLi9pbnN0YWxsLnNo
3586- KQogICAgICAgICAgICB1bW91bnQgL3J1bi9seGRhZ2VudAogICAgICAgICAgICBzeXN0ZW1jdGwg
3587- c3RhcnQgbHhkLWFnZW50LTlwIGx4ZC1hZ2VudAogICAgICAgICAgICA7OwogICAgICAgICopCiAg
3588- ICBlc2FjCmZpCg==
3589- """
3590-)
3591
3592
3593 BUILD_FROM_TGZ = textwrap.dedent(
3594@@ -65,60 +38,6 @@ BUILD_FROM_TGZ = textwrap.dedent(
3595 )
3596
3597
3598-def lxc_create_vm_profile(series: str):
3599- """Create a vm profile to enable launching kvm instances"""
3600-
3601- content_tmpl = textwrap.dedent(
3602- """\
3603- {vendordata}
3604- description: Default LXD profile for {series} VMs
3605- devices:
3606- config:
3607- source: cloud-init:config
3608- type: disk
3609- eth0:
3610- name: eth0
3611- network: lxdbr0
3612- type: nic
3613- root:
3614- path: /
3615- pool: default
3616- type: disk
3617- name: vm
3618- """
3619- )
3620- if series == "xenial":
3621- # FIXME: Xenial images from images:ubuntu/16.04/cloud have HWE kernel
3622- # but no openssh-server (which fips testing would expect)
3623- # Work with CPC to get vhost-vsock support if possible to use
3624- # ubuntu-daily:xenial images
3625- content = content_tmpl.format(
3626- vendordata=LXC_SETUP_VENDORDATA.format(
3627- custom_cfg="packages: [openssh-server]"
3628- ),
3629- series=series,
3630- )
3631- elif series == "bionic":
3632- content = content_tmpl.format(
3633- vendordata=LXC_SETUP_VENDORDATA.format(custom_cfg=""),
3634- series=series,
3635- )
3636- elif series == "focal":
3637- content = content_tmpl.format(vendordata="config: {}", series=series)
3638- else:
3639- raise RuntimeError(
3640- "===No lxc mv support for series {}====".format(series)
3641- )
3642- output = subprocess.check_output(["lxc", "profile", "list"])
3643- profile_name = VM_PROFILE_TMPL.format(series)
3644- if " {} ".format(profile_name) not in output.decode("utf-8"):
3645- subprocess.run(["lxc", "profile", "create", profile_name])
3646- proc = subprocess.Popen(
3647- ["lxc", "profile", "edit", profile_name], stdin=subprocess.PIPE
3648- )
3649- proc.communicate(content.encode())
3650-
3651-
3652 def lxc_get_property(name: str, property_name: str, image: bool = False):
3653 """Check series name of either an image or a container.
3654
3655diff --git a/integration-requirements.txt b/integration-requirements.txt
3656index e29790e..b4e36d7 100644
3657--- a/integration-requirements.txt
3658+++ b/integration-requirements.txt
3659@@ -1,7 +1,7 @@
3660 # Integration testing
3661 behave
3662 PyHamcrest
3663-pycloudlib @ git+https://github.com/canonical/pycloudlib.git@1ac9d4c82fdfd5cb1407f70b8a2b17e02953569d
3664+pycloudlib @ git+https://github.com/canonical/pycloudlib.git@cab5db70bdd5588aabcf7217e655ff90d2dee487
3665
3666
3667 # Simplestreams is not found on PyPi so pull from repo directly
3668diff --git a/lib/__init__.py b/lib/__init__.py
3669deleted file mode 100644
3670index e69de29..0000000
3671--- a/lib/__init__.py
3672+++ /dev/null
3673diff --git a/lib/reboot_cmds.py b/lib/reboot_cmds.py
3674index 6a753d5..7cc5dfd 100644
3675--- a/lib/reboot_cmds.py
3676+++ b/lib/reboot_cmds.py
3677@@ -120,15 +120,19 @@ def process_reboot_operations(args, cfg):
3678 if os.path.exists(reboot_cmd_marker_file):
3679 logging.debug("Running process contract deltas on reboot ...")
3680
3681- fix_pro_pkg_holds(cfg)
3682- refresh_contract(cfg)
3683- process_remaining_deltas(cfg)
3684-
3685- cfg.delete_cache_key("marker-reboot-cmds")
3686+ try:
3687+ fix_pro_pkg_holds(cfg)
3688+ refresh_contract(cfg)
3689+ process_remaining_deltas(cfg)
3690
3691- logging.debug(
3692- "Completed running process contract deltas on reboot ..."
3693- )
3694+ cfg.delete_cache_key("marker-reboot-cmds")
3695+ cfg.remove_notice("", status.MESSAGE_REBOOT_SCRIPT_FAILED)
3696+ logging.debug("Successfully ran all commands on reboot.")
3697+ except Exception as e:
3698+ msg = "Failed running commands on reboot."
3699+ msg += str(e)
3700+ logging.error(msg)
3701+ cfg.add_notice("", status.MESSAGE_REBOOT_SCRIPT_FAILED)
3702
3703
3704 def main(cfg):
3705diff --git a/lib/ua_update_messaging.py b/lib/ua_update_messaging.py
3706new file mode 100644
3707index 0000000..d8e645e
3708--- /dev/null
3709+++ b/lib/ua_update_messaging.py
3710@@ -0,0 +1,302 @@
3711+#!/usr/bin/env python3
3712+
3713+"""
3714+Update messaging text for use in MOTD and APT custom Ubuntu Advantage messages.
3715+
3716+Messaging files will be emitted to /var/lib/ubuntu-advantage/message-* which
3717+will be sourced by apt-hook/hook.cc and various /etc/update-motd.d/ hooks to
3718+present updated text about Ubuntu Advantage service and token state.
3719+"""
3720+
3721+import enum
3722+import logging
3723+import os
3724+
3725+try:
3726+ from typing import Dict, List, Optional, Tuple # noqa
3727+except ImportError:
3728+ # typing isn't available on trusty, so ignore its absence
3729+ pass
3730+
3731+from uaclient.cli import setup_logging
3732+from uaclient import config
3733+from uaclient import entitlements
3734+from uaclient import defaults
3735+from uaclient.status import (
3736+ MESSAGE_ANNOUNCE_ESM,
3737+ MESSAGE_CONTRACT_EXPIRED_APT_NO_PKGS_TMPL,
3738+ MESSAGE_CONTRACT_EXPIRED_APT_PKGS_TMPL,
3739+ MESSAGE_CONTRACT_EXPIRED_GRACE_PERIOD_TMPL,
3740+ MESSAGE_CONTRACT_EXPIRED_MOTD_PKGS_TMPL,
3741+ MESSAGE_CONTRACT_EXPIRED_SOON_TMPL,
3742+ MESSAGE_DISABLED_MOTD_NO_PKGS_TMPL,
3743+ MESSAGE_DISABLED_APT_PKGS_TMPL,
3744+ MESSAGE_UBUNTU_NO_WARRANTY,
3745+ ApplicationStatus,
3746+)
3747+from uaclient import util
3748+
3749+
3750+@enum.unique
3751+class ContractExpiryStatus(enum.Enum):
3752+ NONE = 0
3753+ ACTIVE = 1
3754+ ACTIVE_EXPIRED_SOON = 2
3755+ EXPIRED_GRACE_PERIOD = 3
3756+ EXPIRED = 4
3757+
3758+
3759+# Type of message file used for external messaging (APT and MOTD)
3760+@enum.unique
3761+class ExternalMessage(enum.Enum):
3762+ MOTD_APPS_NO_PKGS = "motd-no-packages-apps.tmpl"
3763+ MOTD_INFRA_NO_PKGS = "motd-no-packages-infra.tmpl"
3764+ MOTD_APPS_PKGS = "motd-packages-apps.tmpl"
3765+ MOTD_INFRA_PKGS = "motd-packages-infra.tmpl"
3766+ APT_PRE_INVOKE_APPS_NO_PKGS = "apt-pre-invoke-no-packages-apps.tmpl"
3767+ APT_PRE_INVOKE_INFRA_NO_PKGS = "apt-pre-invoke-no-packages-infra.tmpl"
3768+ APT_PRE_INVOKE_APPS_PKGS = "apt-pre-invoke-packages-apps.tmpl"
3769+ APT_PRE_INVOKE_INFRA_PKGS = "apt-pre-invoke-packages-infra.tmpl"
3770+ APT_PRE_INVOKE_SERVICE_STATUS = "apt-pre-invoke-esm-service-status"
3771+ MOTD_ESM_SERVICE_STATUS = "motd-esm-service-status"
3772+ ESM_ANNOUNCE = "motd-esm-announce"
3773+ UBUNTU_NO_WARRANTY = "ubuntu-no-warranty"
3774+
3775+
3776+def get_contract_expiry_status(
3777+ cfg: config.UAConfig
3778+) -> "Tuple[ContractExpiryStatus, int]":
3779+ """Return a tuple [ContractExpiryStatus, num_days]"""
3780+ if not cfg.is_attached:
3781+ return ContractExpiryStatus.NONE, 0
3782+
3783+ grace_period = defaults.CONTRACT_EXPIRY_GRACE_PERIOD_DAYS
3784+ pending_expiry = defaults.CONTRACT_EXPIRY_PENDING_DAYS
3785+ remaining_days = cfg.contract_remaining_days
3786+ if 0 <= remaining_days <= pending_expiry:
3787+ return ContractExpiryStatus.ACTIVE_EXPIRED_SOON, remaining_days
3788+ elif -grace_period <= remaining_days < 0:
3789+ return ContractExpiryStatus.EXPIRED_GRACE_PERIOD, remaining_days
3790+ elif remaining_days < -grace_period:
3791+ return ContractExpiryStatus.EXPIRED, remaining_days
3792+ return ContractExpiryStatus.ACTIVE, remaining_days
3793+
3794+
3795+def _write_template_or_remove(msg: str, tmpl_file: str):
3796+ """Write a template to tmpl_file.
3797+
3798+ When msg is empty, remove both tmpl_file and the generated msg.
3799+ """
3800+ if msg:
3801+ util.write_file(tmpl_file, msg)
3802+ else:
3803+ util.remove_file(tmpl_file)
3804+ if tmpl_file.endswith(".tmpl"):
3805+ util.remove_file(tmpl_file.replace(".tmpl", ""))
3806+
3807+
3808+def _write_esm_service_msg_templates(
3809+ cfg: config.UAConfig,
3810+ ent: entitlements.base.UAEntitlement,
3811+ expiry_status: ContractExpiryStatus,
3812+ remaining_days: int,
3813+ pkgs_file: str,
3814+ no_pkgs_file: str,
3815+ motd_pkgs_file: str,
3816+ motd_no_pkgs_file: str,
3817+ no_warranty_file: str,
3818+):
3819+
3820+ pkgs_msg = no_pkgs_msg = motd_pkgs_msg = motd_no_pkgs_msg = ""
3821+ no_warranty_msg = ""
3822+ tmpl_prefix = ent.name.upper().replace("-", "_")
3823+ tmpl_pkg_count_var = "{{{}_PKG_COUNT}}".format(tmpl_prefix)
3824+ tmpl_pkg_names_var = "{{{}_PACKAGES}}".format(tmpl_prefix)
3825+ if ent.application_status()[0] == ApplicationStatus.ENABLED:
3826+ if expiry_status == ContractExpiryStatus.ACTIVE_EXPIRED_SOON:
3827+ pkgs_msg = MESSAGE_CONTRACT_EXPIRED_SOON_TMPL.format(
3828+ title=ent.title,
3829+ remaining_days=remaining_days,
3830+ url=defaults.BASE_UA_URL,
3831+ )
3832+ # Same cautionary message when contract is about to expire
3833+ motd_pkgs_msg = motd_no_pkgs_msg = no_pkgs_msg = pkgs_msg
3834+ elif expiry_status == ContractExpiryStatus.EXPIRED_GRACE_PERIOD:
3835+ grace_period_remaining = (
3836+ defaults.CONTRACT_EXPIRY_GRACE_PERIOD_DAYS + remaining_days
3837+ )
3838+ pkgs_msg = MESSAGE_CONTRACT_EXPIRED_GRACE_PERIOD_TMPL.format(
3839+ title=ent.title,
3840+ expired_date=cfg.contract_expiry_datetime.strftime("%d %b %Y"),
3841+ remaining_days=grace_period_remaining,
3842+ url=defaults.BASE_UA_URL,
3843+ )
3844+ # Same cautionary message when in grace period
3845+ motd_pkgs_msg = motd_no_pkgs_msg = no_pkgs_msg = pkgs_msg
3846+ elif expiry_status == ContractExpiryStatus.EXPIRED:
3847+ if util.is_active_esm(util.get_platform_info()["series"]):
3848+ no_warranty_msg = MESSAGE_UBUNTU_NO_WARRANTY
3849+ pkgs_msg = MESSAGE_CONTRACT_EXPIRED_APT_PKGS_TMPL.format(
3850+ pkg_num=tmpl_pkg_count_var,
3851+ pkg_names=tmpl_pkg_names_var,
3852+ title=ent.title,
3853+ name=ent.name,
3854+ url=defaults.BASE_UA_URL,
3855+ )
3856+ no_pkgs_msg = MESSAGE_CONTRACT_EXPIRED_APT_NO_PKGS_TMPL.format(
3857+ title=ent.title, url=defaults.BASE_ESM_URL
3858+ )
3859+ motd_no_pkgs_msg = no_pkgs_msg
3860+ motd_pkgs_msg = MESSAGE_CONTRACT_EXPIRED_MOTD_PKGS_TMPL.format(
3861+ title=ent.title,
3862+ pkg_num=tmpl_pkg_count_var,
3863+ url=defaults.BASE_ESM_URL,
3864+ )
3865+ elif expiry_status != ContractExpiryStatus.EXPIRED: # Service not enabled
3866+ pkgs_msg = MESSAGE_DISABLED_APT_PKGS_TMPL.format(
3867+ title=ent.title,
3868+ pkg_num=tmpl_pkg_count_var,
3869+ pkg_names=tmpl_pkg_names_var,
3870+ url=defaults.BASE_ESM_URL,
3871+ )
3872+ no_pkgs_msg = MESSAGE_DISABLED_MOTD_NO_PKGS_TMPL.format(
3873+ title=ent.title, url=defaults.BASE_ESM_URL
3874+ )
3875+
3876+ msg_dir = os.path.join(cfg.data_dir, "messages")
3877+ _write_template_or_remove(
3878+ no_warranty_msg, os.path.join(msg_dir, no_warranty_file)
3879+ )
3880+ _write_template_or_remove(no_pkgs_msg, os.path.join(msg_dir, no_pkgs_file))
3881+ _write_template_or_remove(pkgs_msg, os.path.join(msg_dir, pkgs_file))
3882+ _write_template_or_remove(
3883+ motd_no_pkgs_msg, os.path.join(msg_dir, motd_no_pkgs_file)
3884+ )
3885+ _write_template_or_remove(
3886+ motd_pkgs_msg, os.path.join(msg_dir, motd_pkgs_file)
3887+ )
3888+
3889+
3890+def write_apt_and_motd_templates(cfg: config.UAConfig, series: str) -> None:
3891+ """Write messaging templates about available esm packages.
3892+
3893+ :param cfg: UAConfig instance for this environment.
3894+ :param series: string of Ubuntu release series: 'xenial'.
3895+ """
3896+ apps_no_pkg_file = ExternalMessage.APT_PRE_INVOKE_APPS_NO_PKGS.value
3897+ apps_pkg_file = ExternalMessage.APT_PRE_INVOKE_APPS_PKGS.value
3898+ infra_no_pkg_file = ExternalMessage.APT_PRE_INVOKE_INFRA_NO_PKGS.value
3899+ infra_pkg_file = ExternalMessage.APT_PRE_INVOKE_INFRA_PKGS.value
3900+ motd_apps_no_pkg_file = ExternalMessage.MOTD_APPS_NO_PKGS.value
3901+ motd_apps_pkg_file = ExternalMessage.MOTD_APPS_PKGS.value
3902+ motd_infra_no_pkg_file = ExternalMessage.MOTD_INFRA_NO_PKGS.value
3903+ motd_infra_pkg_file = ExternalMessage.MOTD_INFRA_PKGS.value
3904+ no_warranty_file = ExternalMessage.UBUNTU_NO_WARRANTY.value
3905+
3906+ apps_cls = entitlements.ENTITLEMENT_CLASS_BY_NAME["esm-apps"]
3907+ apps_inst = apps_cls(cfg)
3908+ config_allow_beta = util.is_config_value_true(
3909+ config=cfg.cfg, path_to_value="features.allow_beta"
3910+ )
3911+ apps_not_beta = bool(config_allow_beta or not apps_cls.is_beta)
3912+ infra_cls = entitlements.ENTITLEMENT_CLASS_BY_NAME["esm-infra"]
3913+ infra_inst = infra_cls(cfg)
3914+
3915+ expiry_status, remaining_days = get_contract_expiry_status(cfg)
3916+
3917+ if series != "trusty" and apps_not_beta:
3918+ _write_esm_service_msg_templates(
3919+ cfg,
3920+ apps_inst,
3921+ expiry_status,
3922+ remaining_days,
3923+ apps_pkg_file,
3924+ apps_no_pkg_file,
3925+ motd_apps_pkg_file,
3926+ motd_apps_no_pkg_file,
3927+ no_warranty_file,
3928+ )
3929+
3930+ # We only have esm-infra apt alerts for esm distros.
3931+ # However, if we have expired credentials, we will
3932+ # produce esm-infra message showing that the contract is
3933+ # expiring/expired.
3934+ infra_status, _ = infra_inst.application_status()
3935+ is_infra_enabled = infra_status == ApplicationStatus.ENABLED
3936+ if is_infra_enabled or util.is_active_esm(series):
3937+ _write_esm_service_msg_templates(
3938+ cfg,
3939+ infra_inst,
3940+ expiry_status,
3941+ remaining_days,
3942+ infra_pkg_file,
3943+ infra_no_pkg_file,
3944+ motd_infra_pkg_file,
3945+ motd_infra_no_pkg_file,
3946+ no_warranty_file,
3947+ )
3948+
3949+
3950+def write_esm_announcement_message(cfg: config.UAConfig, series: str) -> None:
3951+ """Write human-readable messages if ESM is offered on this LTS release.
3952+
3953+ Do not write ESM announcements on trusty, esm-apps is enable or beta.
3954+
3955+ :param cfg: UAConfig instance for this environment.
3956+ :param series: string of Ubuntu release series: 'xenial'.
3957+ """
3958+ apps_cls = entitlements.ENTITLEMENT_CLASS_BY_NAME["esm-apps"]
3959+ apps_inst = apps_cls(cfg)
3960+ enabled_status = ApplicationStatus.ENABLED
3961+ apps_not_enabled = apps_inst.application_status()[0] != enabled_status
3962+ config_allow_beta = util.is_config_value_true(
3963+ config=cfg.cfg, path_to_value="features.allow_beta"
3964+ )
3965+ apps_not_beta = bool(config_allow_beta or not apps_cls.is_beta)
3966+
3967+ msg_dir = os.path.join(cfg.data_dir, "messages")
3968+ esm_news_file = os.path.join(msg_dir, ExternalMessage.ESM_ANNOUNCE.value)
3969+ if all([series != "trusty", apps_not_beta, apps_not_enabled]):
3970+ util.write_file(esm_news_file, "\n" + MESSAGE_ANNOUNCE_ESM)
3971+ else:
3972+ util.remove_file(esm_news_file)
3973+
3974+
3975+def update_apt_and_motd_messages(cfg: config.UAConfig) -> None:
3976+ """Emit templates and human-readable status messages in msg_dir.
3977+
3978+ These structured messages will be sourced by both /etc/update.motd.d
3979+ and APT UA-configured hooks. APT hook content will orginate from
3980+ apt-hook/hook.cc
3981+
3982+ Call esm-apt-hook process-templates to render final human-readable
3983+ messages.
3984+
3985+ :param cfg: UAConfig instance for this environment.
3986+ """
3987+ setup_logging(logging.INFO, logging.DEBUG)
3988+ logging.debug("Updating UA messages for APT and MOTD.")
3989+ msg_dir = os.path.join(cfg.data_dir, "messages")
3990+ if not os.path.exists(msg_dir):
3991+ os.makedirs(msg_dir)
3992+
3993+ series = util.get_platform_info()["series"]
3994+ if not util.is_lts(series):
3995+ # ESM is only on LTS releases. Remove all messages and templates.
3996+ for msg_enum in ExternalMessage:
3997+ msg_path = os.path.join(msg_dir, msg_enum.value)
3998+ util.remove_file(msg_path)
3999+ if msg_path.endswith(".tmpl"):
4000+ util.remove_file(msg_path.replace(".tmpl", ""))
4001+ return
4002+
4003+ # Announce ESM availabilty on active ESM LTS releases
4004+ write_esm_announcement_message(cfg, series)
4005+ write_apt_and_motd_templates(cfg, series)
4006+ # Now that we've setup/cleanedup templates render them with apt-hook
4007+ util.subp(["/usr/lib/ubuntu-advantage/apt-esm-hook", "process-templates"])
4008+
4009+
4010+if __name__ == "__main__":
4011+ cfg = config.UAConfig()
4012+ update_apt_and_motd_messages(cfg=cfg)
4013diff --git a/setup.py b/setup.py
4014index a8a597c..0702d03 100644
4015--- a/setup.py
4016+++ b/setup.py
4017@@ -40,6 +40,7 @@ def _get_version():
4018 def _get_data_files():
4019 data_files = [
4020 ("/etc/ubuntu-advantage", ["uaclient.conf", "help_data.yaml"]),
4021+ ("/etc/update-motd.d", glob.glob("update-motd.d/*")),
4022 ("/usr/lib/ubuntu-advantage", glob.glob("lib/[!_]*")),
4023 ("/usr/share/keyrings", glob.glob("keyrings/*")),
4024 (
4025diff --git a/systemd/ua-messaging.service b/systemd/ua-messaging.service
4026new file mode 100644
4027index 0000000..7bc57d2
4028--- /dev/null
4029+++ b/systemd/ua-messaging.service
4030@@ -0,0 +1,8 @@
4031+[Unit]
4032+Description=Ubuntu Advantage APT and MOTD Messages
4033+After=network.target network-online.target systemd-networkd.service ua-auto-attach.service
4034+Wants=ua-auto-attach.service
4035+
4036+[Service]
4037+Type=oneshot
4038+ExecStart=/usr/bin/python3 /usr/lib/ubuntu-advantage/ua_update_messaging.py
4039diff --git a/systemd/ua-messaging.timer b/systemd/ua-messaging.timer
4040new file mode 100644
4041index 0000000..fee91b7
4042--- /dev/null
4043+++ b/systemd/ua-messaging.timer
4044@@ -0,0 +1,11 @@
4045+[Unit]
4046+Description=Ubuntu Advantage update messaging
4047+
4048+[Timer]
4049+OnCalendar=*-*-* 3,15:00
4050+RandomizedDelaySec=1h
4051+Persistent=true
4052+OnStartupSec=1min
4053+
4054+[Install]
4055+WantedBy=timers.target
4056diff --git a/tools/test_xenial_upgrade.sh b/tools/test_xenial_upgrade.sh
4057new file mode 100644
4058index 0000000..691581b
4059--- /dev/null
4060+++ b/tools/test_xenial_upgrade.sh
4061@@ -0,0 +1,224 @@
4062+#!/usr/bin/bash
4063+
4064+set -x
4065+set -e
4066+name=test-xenial-ua-upgrade
4067+
4068+function h1() {
4069+ set +x
4070+ echo ""
4071+ echo ""
4072+ echo ""
4073+ echo "############################################################################"
4074+ echo "## $1"
4075+ echo "############################################################################"
4076+ echo ""
4077+ set -x
4078+}
4079+function h2() {
4080+ set +x
4081+ echo ""
4082+ echo ""
4083+ echo "-> $1"
4084+ echo "----------------------------------------------------------------------------"
4085+ echo ""
4086+ set -x
4087+}
4088+
4089+
4090+function setup() {
4091+ tool=$1
4092+
4093+ h2 "Make sure we're up to date"
4094+ $tool exec $name -- sudo apt update
4095+ $tool exec $name -- sudo apt upgrade -y
4096+ $tool exec $name -- sudo apt install ubuntu-advantage-tools -y
4097+
4098+ h2 "Initial State"
4099+ $tool exec $name -- apt-cache policy ubuntu-advantage-tools
4100+ $tool exec $name -- sudo ubuntu-advantage status
4101+ $tool exec $name -- dpkg-query -L ubuntu-advantage-tools
4102+}
4103+function setup_container() {
4104+ h1 "Setting up fresh container"
4105+ lxc delete --force $name || true
4106+ lxc launch ubuntu-daily:xenial $name
4107+ sleep 10
4108+
4109+ setup lxc
4110+}
4111+function setup_vm() {
4112+ h1 "Setting up fresh vm"
4113+ multipass delete -p $name || true
4114+ multipass launch -n $name xenial
4115+ sleep 10
4116+
4117+ setup multipass
4118+}
4119+function teardown_container() {
4120+ lxc delete --force $name || true
4121+}
4122+function teardown_vm() {
4123+ multipass delete -p $name || true
4124+}
4125+
4126+function install_new_ua() {
4127+ tool=$1
4128+ h2 "Set up to use daily ppa"
4129+ $tool exec $name -- sudo add-apt-repository ppa:ua-client/daily -y
4130+ $tool exec $name -- sudo apt update
4131+
4132+ h2 "Actually install - verify there are no errors"
4133+ $tool exec $name -- sudo apt install ubuntu-advantage-tools -y
4134+}
4135+
4136+function attach_ua() {
4137+ tool=$1
4138+ set +x
4139+ echo "+ $tool exec $name -- sudo ua attach \$UACLIENT_BEHAVE_CONTRACT_TOKEN"
4140+ $tool exec $name -- sudo ua attach $UACLIENT_BEHAVE_CONTRACT_TOKEN
4141+ set -x
4142+}
4143+
4144+
4145+
4146+
4147+function test_upgrade_in_container() {
4148+ setup_container
4149+
4150+ h1 "Upgrade to UA 27 while unattached"
4151+
4152+ install_new_ua lxc
4153+
4154+ h2 "Check for leftover files from old version - verify nothing unexpected is left behind"
4155+ lxc exec $name -- ls -l /etc/update-motd.d/99-esm /usr/share/keyrings/ubuntu-esm-keyring.gpg /usr/share/keyrings/ubuntu-fips-keyring.gpg /usr/share/man/man1/ubuntu-advantage.1.gz /usr/share/doc/ubuntu-advantage-tools/copyright /usr/share/doc/ubuntu-advantage-tools/changelog.gz /usr/bin/ubuntu-advantage || true
4156+
4157+ h2 "New Status - verify esm-infra available but not enabled; esm-apps not visible"
4158+ lxc exec $name -- ua status
4159+
4160+ h2 "Attach - verify esm-infra automatically enabled; esm-apps not visible"
4161+ attach_ua lxc
4162+
4163+ h2 "Detaching before destruction"
4164+ lxc exec $name -- ua detach --assume-yes
4165+
4166+ teardown_container
4167+}
4168+
4169+
4170+function test_upgrade_with_livepatch_in_vm() {
4171+
4172+ setup_vm
4173+
4174+ h1 "Upgrade to UA 27 while old version has livepatch enabled"
4175+
4176+ h2 "Enable livepatch on old version"
4177+ set +x
4178+ echo "+ multipass exec $name -- ubuntu-advantage enable-livepatch \$LIVEPATCH_TOKEN"
4179+ multipass exec $name -- sudo ubuntu-advantage enable-livepatch $LIVEPATCH_TOKEN
4180+ set -x
4181+
4182+ h2 "Status before - old UA and livepatch say enabled"
4183+ multipass exec $name -- sudo canonical-livepatch status
4184+ multipass exec $name -- sudo ubuntu-advantage status
4185+
4186+ install_new_ua multipass
4187+
4188+ h2 "Status after upgrade - livepatch still enabled but new UA doesn't report it"
4189+ multipass exec $name -- sudo canonical-livepatch status
4190+ multipass exec $name -- sudo ua status
4191+
4192+ h2 "Attach - verify that livepatch is disabled and re-enabled"
4193+ attach_ua multipass
4194+
4195+ h2 "Status after attach - both livepatch and UA should say enabled"
4196+ multipass exec $name -- sudo canonical-livepatch status
4197+ multipass exec $name -- sudo ua status
4198+
4199+ h2 "Detaching before destruction"
4200+ multipass exec $name -- sudo ua detach --assume-yes
4201+
4202+ teardown_vm
4203+}
4204+
4205+function test_upgrade_with_fips_in_vm() {
4206+
4207+ setup_vm
4208+
4209+ h1 "Upgrade to UA 27 while old version has fips enabled"
4210+
4211+ h2 "Manual fips check says disabled (file doesn't exist)"
4212+ multipass exec $name -- sudo cat /proc/sys/crypto/fips_enabled || true
4213+
4214+ h2 "Enable fips on old version"
4215+ set +x
4216+ echo "+ multipass exec $name -- ubuntu-advantage enable-fips \$FIPS_CREDS"
4217+ multipass exec $name -- sudo ubuntu-advantage enable-fips $FIPS_CREDS
4218+ set -x
4219+
4220+ h2 "Reboot to finish fips activation"
4221+ multipass exec $name -- sudo reboot || true
4222+ sleep 20
4223+
4224+ h2 "Status before upgrade - old UA says fips is enabled, manual check agrees"
4225+ multipass exec $name -- sudo ubuntu-advantage status
4226+ multipass exec $name -- sudo cat /proc/sys/crypto/fips_enabled
4227+
4228+ h2 "Source added by old client is present"
4229+ multipass exec $name -- sudo ls /etc/apt/sources.list.d
4230+ multipass exec $name -- sudo grep -o private-ppa.launchpad.net/ubuntu-advantage/fips/ubuntu /etc/apt/sources.list.d/ubuntu-fips-xenial.list
4231+
4232+ install_new_ua multipass
4233+
4234+ h2 "Status after upgrade - new UA won't say anything is enabled, but a manual check still says fips is enabled"
4235+ multipass exec $name -- sudo ua status
4236+ multipass exec $name -- sudo cat /proc/sys/crypto/fips_enabled
4237+
4238+ h2 "Source file added by old client is renamed but contents left unchanged"
4239+ multipass exec $name -- sudo ls /etc/apt/sources.list.d
4240+ multipass exec $name -- sudo grep -o private-ppa.launchpad.net/ubuntu-advantage/fips/ubuntu /etc/apt/sources.list.d/ubuntu-fips.list
4241+
4242+ h2 "Attach - only esm-infra will be auto-enabled"
4243+ attach_ua multipass
4244+
4245+ h2 "Status after attach - new UA will say fips is disabled, livepatch is n/a, and there is a notice to enable fips"
4246+ multipass exec $name -- sudo ua status
4247+ multipass exec $name -- sudo cat /proc/sys/crypto/fips_enabled
4248+
4249+ h2 "Enable fips on new UA - This will re-install fips packages and ask to reboot again"
4250+ multipass exec $name -- sudo ua enable fips --assume-yes
4251+
4252+
4253+ h2 "Status after enabled but before reboot - UA says fips enabled, notice to enable fips is gone"
4254+ multipass exec $name -- sudo ua status
4255+ multipass exec $name -- sudo cat /proc/sys/crypto/fips_enabled
4256+
4257+ h2 "Source added by old client is replaced with new source"
4258+ set +x
4259+ echo "multipass exec $name -- sudo grep \$FIPS_CREDS /etc/apt/sources.list.d/ubuntu-fips.list && echo \"FAIL: found oldclient FIPS creds\" || echo \"SUCCESS: Migrated to new client creds\""
4260+ multipass exec $name -- sudo grep $FIPS_CREDS /etc/apt/sources.list.d/ubuntu-fips.list && echo "FAIL: found oldclient FIPS creds" || echo "SUCCESS: Migrated to new client creds"
4261+ set -x
4262+ multipass exec $name -- sudo ls /etc/apt/sources.list.d
4263+ multipass exec $name -- sudo cat /etc/apt/sources.list.d/ubuntu-fips.list
4264+ multipass exec $name -- sudo apt update
4265+
4266+ h2 "Check to make sure we have a valid ubuntu-*-fips metapackage installed"
4267+ multipass exec $name -- sudo grep "install" /var/log/ubuntu-advantage.log
4268+
4269+ h2 "Reboot to finish second fips activation"
4270+ multipass exec $name -- sudo reboot || true
4271+ sleep 20
4272+
4273+ h2 "Status after reboot - new UA will say fips is enabled, manual check agrees"
4274+ multipass exec $name -- sudo ua status
4275+ multipass exec $name -- sudo cat /proc/sys/crypto/fips_enabled
4276+
4277+ h2 "Detaching before destruction"
4278+ multipass exec $name -- sudo ua detach --assume-yes
4279+
4280+ teardown_vm
4281+}
4282+
4283+test_upgrade_in_container
4284+test_upgrade_with_livepatch_in_vm
4285+test_upgrade_with_fips_in_vm
4286diff --git a/tools/tox-lxd-runner b/tools/tox-lxd-runner
4287index 40c5e48..cd80af0 100755
4288--- a/tools/tox-lxd-runner
4289+++ b/tools/tox-lxd-runner
4290@@ -57,9 +57,9 @@ python_minor=$(lxc exec "$container" -- python3 -c 'import sys; print(sys.versio
4291 if ((python_minor > 5)); then
4292 get_pip_url="https://bootstrap.pypa.io/get-pip.py"
4293 elif ((python_minor == 5)); then
4294- get_pip_url="https://bootstrap.pypa.io/3.5/get-pip.py"
4295+ get_pip_url="https://bootstrap.pypa.io/pip/3.5/get-pip.py"
4296 elif ((python_minor == 4)); then
4297- get_pip_url="https://bootstrap.pypa.io/3.4/get-pip.py"
4298+ get_pip_url="https://bootstrap.pypa.io/pip/3.4/get-pip.py"
4299 else
4300 echo "Unsupported Python version (3.$python_minor)"
4301 exit 1
4302diff --git a/tox.ini b/tox.ini
4303index fa53bb5..19a5b56 100644
4304--- a/tox.ini
4305+++ b/tox.ini
4306@@ -46,8 +46,8 @@ commands =
4307 flake8-bionic: flake8 features
4308 mypy: mypy --python-version 3.4 uaclient/
4309 mypy: mypy --python-version 3.5 uaclient/
4310- mypy: mypy --python-version 3.6 uaclient/ features/
4311- mypy-focal: mypy --python-version 3.7 uaclient/ features/
4312+ mypy: mypy --python-version 3.6 uaclient/ features/ lib/
4313+ mypy-focal: mypy --python-version 3.7 uaclient/ features/ lib/
4314 black: black --check --diff uaclient/ features/ lib/ setup.py
4315 behave-lxd-14.04: behave -v {posargs} --tags="series.trusty,series.all" --tags="~upgrade"
4316 behave-lxd-16.04: behave -v {posargs} --tags="series.xenial,series.all" --tags="~upgrade"
4317diff --git a/uaclient-devel.conf b/uaclient-devel.conf
4318index e21d64d..c8386c8 100644
4319--- a/uaclient-devel.conf
4320+++ b/uaclient-devel.conf
4321@@ -1,5 +1,6 @@
4322 # Development UA Client config file. YAML
4323 contract_url: 'https://contracts.staging.canonical.com'
4324+security_url: 'https://ubuntu.com/security'
4325 data_dir: /var/tmp/uaclient
4326 log_level: debug
4327 log_file: ubuntu-advantage-devel.log
4328diff --git a/uaclient.conf b/uaclient.conf
4329index 8dc2f1a..9e5def8 100644
4330--- a/uaclient.conf
4331+++ b/uaclient.conf
4332@@ -1,5 +1,6 @@
4333 # Ubuntu-Advantage client config file.
4334 contract_url: 'https://contracts.canonical.com'
4335+security_url: 'https://ubuntu.com/security'
4336 data_dir: /var/lib/ubuntu-advantage
4337 log_level: debug
4338 log_file: /var/log/ubuntu-advantage.log
4339diff --git a/uaclient/apt.py b/uaclient/apt.py
4340index 93630b4..7b80d87 100644
4341--- a/uaclient/apt.py
4342+++ b/uaclient/apt.py
4343@@ -58,6 +58,7 @@ def assert_valid_apt_credentials(repo_url, username, password):
4344 os.path.join(tmpd, "apt-helper-output"),
4345 ],
4346 timeout=APT_HELPER_TIMEOUT,
4347+ retry_sleeps=APT_RETRIES,
4348 )
4349 except util.ProcessExecutionError as e:
4350 if e.exit_code == 100:
4351diff --git a/uaclient/cli.py b/uaclient/cli.py
4352index 87dc767..b6a4ed5 100644
4353--- a/uaclient/cli.py
4354+++ b/uaclient/cli.py
4355@@ -8,6 +8,7 @@ import json
4356 import logging
4357 import os
4358 import pathlib
4359+import re
4360 import sys
4361 import textwrap
4362 import time
4363@@ -23,6 +24,7 @@ from uaclient import config
4364 from uaclient import contract
4365 from uaclient import entitlements
4366 from uaclient import exceptions
4367+from uaclient import security
4368 from uaclient import status as ua_status
4369 from uaclient import util
4370 from uaclient import version
4371@@ -88,19 +90,11 @@ class UAArgumentParser(argparse.ArgumentParser):
4372 self.non_beta_services_desc,
4373 self.beta_services_desc,
4374 ]
4375-
4376- if all([desc_var is None for desc_var in desc_vars]):
4377- super().print_help(file=file)
4378- elif show_all:
4379- self.description = "\n".join(
4380- [self.base_desc]
4381- + sorted(self.non_beta_services_desc + self.beta_services_desc)
4382- )
4383- else:
4384- self.description = "\n".join(
4385- [self.base_desc] + sorted(self.non_beta_services_desc)
4386- )
4387-
4388+ if any(desc_vars):
4389+ services = sorted(self.non_beta_services_desc)
4390+ if show_all:
4391+ services = sorted(services + self.beta_services_desc)
4392+ self.description = "\n".join([self.base_desc] + services)
4393 super().print_help(file=file)
4394
4395
4396@@ -195,6 +189,10 @@ def assert_not_attached(f):
4397 def auto_attach_parser(parser):
4398 """Build or extend an arg parser for auto-attach subcommand."""
4399 parser.prog = "auto-attach"
4400+ parser.description = (
4401+ "Automatically attach an Ubuntu Advantage token on Ubuntu Pro"
4402+ " images."
4403+ )
4404 parser.usage = USAGE_TMPL.format(name=NAME, command=parser.prog)
4405 parser._optionals.title = "Flags"
4406 return parser
4407@@ -204,6 +202,10 @@ def attach_parser(parser):
4408 """Build or extend an arg parser for attach subcommand."""
4409 parser.usage = USAGE_TMPL.format(name=NAME, command="attach <token>")
4410 parser.prog = "attach"
4411+ parser.description = (
4412+ "Attach this machine to Ubuntu Advantage with a token obtained"
4413+ " from https://ubuntu.com/advantage"
4414+ )
4415 parser._optionals.title = "Flags"
4416 parser.add_argument(
4417 "token",
4418@@ -221,11 +223,56 @@ def attach_parser(parser):
4419 return parser
4420
4421
4422+def fix_parser(parser):
4423+ """Build or extend an arg parser for fix subcommand."""
4424+ parser.usage = USAGE_TMPL.format(
4425+ name=NAME, command="fix <CVE-yyyy-nnnn+>|<USN-nnnn-d+>"
4426+ )
4427+ parser.prog = "fix"
4428+ parser.description = (
4429+ "Inspect and resolve CVEs and USNs (Ubuntu Security Notices) on this"
4430+ " machine."
4431+ )
4432+ parser._optionals.title = "Flags"
4433+ parser.add_argument(
4434+ "security_issue",
4435+ help=(
4436+ "Security vulnerability ID to inspect and resolve on this system."
4437+ " Format: CVE-yyyy-nnnn, CVE-yyyy-nnnnnnn or USN-nnnn-dd"
4438+ ),
4439+ )
4440+ return parser
4441+
4442+
4443+def refresh_parser(parser):
4444+ """Build or extend an arg parser for refresh subcommand."""
4445+ parser.prog = "refresh"
4446+ parser.description = (
4447+ "Refresh existing Ubuntu Advantage contract and update services."
4448+ )
4449+ parser.usage = USAGE_TMPL.format(name=NAME, command=parser.prog)
4450+ parser._optionals.title = "Flags"
4451+ return parser
4452+
4453+
4454+def action_fix(args, cfg, **kwargs):
4455+ if not re.match(security.CVE_OR_USN_REGEX, args.security_issue):
4456+ msg = (
4457+ 'Error: issue "{}" is not recognized.\n'
4458+ 'Usage: "ua fix CVE-yyyy-nnnn" or "ua fix USN-nnnn"'
4459+ ).format(args.security_issue)
4460+ raise exceptions.UserFacingError(msg)
4461+
4462+ security.fix_security_issue_id(cfg, args.security_issue)
4463+ return 0
4464+
4465+
4466 def detach_parser(parser):
4467 """Build or extend an arg parser for detach subcommand."""
4468 usage = USAGE_TMPL.format(name=NAME, command="detach")
4469 parser.usage = usage
4470 parser.prog = "detach"
4471+ parser.description = "Detach this machine from Ubuntu Advantage services."
4472 parser._optionals.title = "Flags"
4473 parser.add_argument(
4474 "--assume-yes",
4475@@ -240,6 +287,9 @@ def help_parser(parser):
4476 usage = USAGE_TMPL.format(name=NAME, command="help [service]")
4477 parser.usage = usage
4478 parser.prog = "help"
4479+ parser.description = (
4480+ "Provide detailed information about Ubuntu Advantage services."
4481+ )
4482 parser._positionals.title = "Arguments"
4483 parser.add_argument(
4484 "service",
4485@@ -276,6 +326,7 @@ def enable_parser(parser):
4486 usage = USAGE_TMPL.format(
4487 name=NAME, command="enable <service> [<service>]"
4488 )
4489+ parser.description = "Enable an Ubuntu Advantage service."
4490 parser.usage = usage
4491 parser.prog = "enable"
4492 parser._positionals.title = "Arguments"
4493@@ -286,7 +337,7 @@ def enable_parser(parser):
4494 nargs="+",
4495 help=(
4496 "the name(s) of the Ubuntu Advantage services to enable."
4497- " One of: {}".format(entitlements.RELEASED_ENTITLEMENTS_STR),
4498+ " One of: {}".format(entitlements.RELEASED_ENTITLEMENTS_STR)
4499 ),
4500 )
4501 parser.add_argument(
4502@@ -305,6 +356,7 @@ def disable_parser(parser):
4503 usage = USAGE_TMPL.format(
4504 name=NAME, command="disable <service> [<service>]"
4505 )
4506+ parser.description = "Disable an Ubuntu Advantage service."
4507 parser.usage = usage
4508 parser.prog = "disable"
4509 parser._positionals.title = "Arguments"
4510@@ -330,6 +382,9 @@ def status_parser(parser):
4511 """Build or extend an arg parser for status subcommand."""
4512 usage = USAGE_TMPL.format(name=NAME, command="status")
4513 parser.usage = usage
4514+ parser.description = (
4515+ "Output the status information for Ubuntu Advantage services."
4516+ )
4517 parser.prog = "status"
4518 # This formatter_class ensures that our formatting below isn't lost
4519 parser.formatter_class = argparse.RawDescriptionHelpFormatter
4520@@ -441,7 +496,7 @@ def action_disable(args, cfg, **kwargs):
4521 ret &= _perform_disable(entitlement, cfg, assume_yes=args.assume_yes)
4522
4523 if entitlements_not_found:
4524- valid_names = "Try " + entitlements.ALL_ENTITLEMENTS_STR
4525+ valid_names = "Try " + entitlements.ALL_ENTITLEMENTS_STR + "."
4526 service_msg = "\n".join(
4527 textwrap.wrap(valid_names, width=80, break_long_words=False)
4528 )
4529@@ -542,7 +597,7 @@ def action_enable(args, cfg, **kwargs):
4530 valid_names = entitlements.RELEASED_ENTITLEMENTS_STR
4531 service_msg = "\n".join(
4532 textwrap.wrap(
4533- "Try " + valid_names, width=80, break_long_words=False
4534+ "Try " + valid_names + ".", width=80, break_long_words=False
4535 )
4536 )
4537 tmpl = ua_status.MESSAGE_INVALID_SERVICE_OP_FAILURE_TMPL
4538@@ -580,7 +635,7 @@ def _detach(cfg: config.UAConfig, assume_yes: bool) -> int:
4539 """
4540 to_disable = []
4541 for ent_cls in entitlements.ENTITLEMENT_CLASSES:
4542- ent = ent_cls(cfg)
4543+ ent = ent_cls(cfg=cfg, assume_yes=assume_yes)
4544 if ent.can_disable(silent=True):
4545 to_disable.append(ent)
4546 if to_disable:
4547@@ -597,6 +652,7 @@ def _detach(cfg: config.UAConfig, assume_yes: bool) -> int:
4548 contract_id = cfg.machine_token["machineTokenInfo"]["contractInfo"]["id"]
4549 contract_client.detach_machine_from_contract(machine_token, contract_id)
4550 cfg.delete_cache()
4551+ config.update_ua_messages(cfg)
4552 print(ua_status.MESSAGE_DETACH_SUCCESS)
4553 return 0
4554
4555@@ -614,10 +670,12 @@ def _attach_with_token(
4556 logging.exception(exc)
4557 print(ua_status.MESSAGE_ATTACH_FAILURE)
4558 cfg.status() # Persist updated status in the event of partial attach
4559+ config.update_ua_messages(cfg)
4560 return 1
4561 except exceptions.UserFacingError as exc:
4562 logging.warning(exc.msg)
4563 cfg.status() # Persist updated status in the event of partial attach
4564+ config.update_ua_messages(cfg)
4565 return 1
4566 contract_name = cfg.machine_token["machineTokenInfo"]["contractInfo"][
4567 "name"
4568@@ -628,6 +686,7 @@ def _attach_with_token(
4569 )
4570 )
4571
4572+ config.update_ua_messages(cfg)
4573 action_status(args=None, cfg=cfg)
4574 return 0
4575
4576@@ -804,6 +863,13 @@ def get_parser():
4577 help="refresh Ubuntu Advantage services from contracts server",
4578 )
4579 parser_refresh.set_defaults(action=action_refresh)
4580+ refresh_parser(parser_refresh)
4581+ parser_fix = subparsers.add_parser(
4582+ "fix",
4583+ help="check for and mitigate the impact of a CVE/USN on this system",
4584+ )
4585+ parser_fix.set_defaults(action=action_fix)
4586+ fix_parser(parser_fix)
4587 parser_version = subparsers.add_parser(
4588 "version", help="show version of {}".format(NAME)
4589 )
4590@@ -869,6 +935,7 @@ def action_refresh(args, cfg):
4591 with util.disable_log_to_console():
4592 logging.exception(exc)
4593 raise exceptions.UserFacingError(ua_status.MESSAGE_REFRESH_FAILURE)
4594+
4595 print(ua_status.MESSAGE_REFRESH_SUCCESS)
4596 return 0
4597
4598@@ -989,7 +1056,9 @@ def main(sys_argv=None):
4599 log_level = cfg.log_level
4600 console_level = logging.DEBUG if args.debug else logging.INFO
4601 setup_logging(console_level, log_level, cfg.log_file)
4602- logging.debug("Executed with sys.argv: %r", sys_argv)
4603+ logging.debug(
4604+ util.redact_sensitive_logs("Executed with sys.argv: %r" % sys_argv)
4605+ )
4606 return args.action(args, cfg)
4607
4608
4609diff --git a/uaclient/clouds/identity.py b/uaclient/clouds/identity.py
4610index fa24c36..95f9a19 100644
4611--- a/uaclient/clouds/identity.py
4612+++ b/uaclient/clouds/identity.py
4613@@ -6,6 +6,7 @@ from uaclient import exceptions
4614 from uaclient import clouds
4615 from uaclient import status
4616 from uaclient import util
4617+from uaclient.config import apply_config_settings_override
4618
4619 try:
4620 from typing import Dict, Optional, Type # noqa: F401
4621@@ -21,6 +22,16 @@ CLOUDINIT_INSTANCE_ID_FILE = "/var/lib/cloud/data/instance-id"
4622 # Mapping of datasource names to cloud-id responses. Trusty compat with Xenial+
4623 DATASOURCE_TO_CLOUD_ID = {"azurenet": "azure", "ec2": "aws", "gce": "gcp"}
4624
4625+CLOUD_TYPE_TO_TITLE = {
4626+ "aws": "AWS",
4627+ "aws-china": "AWS China",
4628+ "aws-gov": "AWS Gov",
4629+ "azure": "Azure",
4630+ "gcp": "GCP",
4631+}
4632+
4633+PRO_CLOUDS = ["aws", "azure", "gcp"]
4634+
4635
4636 def get_instance_id(
4637 _iid_file: str = CLOUDINIT_INSTANCE_ID_FILE
4638@@ -47,6 +58,7 @@ def get_cloud_type_from_result_file(
4639 return DATASOURCE_TO_CLOUD_ID.get(dsname, dsname)
4640
4641
4642+@apply_config_settings_override("cloud_type")
4643 def get_cloud_type() -> "Optional[str]":
4644 if util.which("cloud-id"):
4645 # Present in cloud-init on >= Xenial
4646diff --git a/uaclient/clouds/tests/test_identity.py b/uaclient/clouds/tests/test_identity.py
4647index af320e1..3b8e494 100644
4648--- a/uaclient/clouds/tests/test_identity.py
4649+++ b/uaclient/clouds/tests/test_identity.py
4650@@ -97,6 +97,37 @@ class TestGetCloudType:
4651 ):
4652 assert get_cloud_type() is None
4653
4654+ @pytest.mark.parametrize(
4655+ "settings_overrides",
4656+ (
4657+ (
4658+ """
4659+ settings_overrides:
4660+ cloud_type: "azure"
4661+ """
4662+ ),
4663+ (
4664+ """
4665+ settings_overrides:
4666+ other_setting: "blah"
4667+ """
4668+ ),
4669+ ),
4670+ )
4671+ @mock.patch("uaclient.util.load_file")
4672+ @mock.patch(M_PATH + "util.which", return_value="/usr/bin/cloud-id")
4673+ @mock.patch(M_PATH + "util.subp", return_value=("test", ""))
4674+ def test_cloud_type_when_using_settings_override(
4675+ self, m_subp, m_which, m_load_file, settings_overrides
4676+ ):
4677+ if "azure" in settings_overrides:
4678+ expected_value = "azure"
4679+ else:
4680+ expected_value = "test"
4681+
4682+ m_load_file.return_value = settings_overrides
4683+ assert get_cloud_type() == expected_value
4684+
4685
4686 @mock.patch(M_PATH + "get_cloud_type")
4687 class TestCloudInstanceFactory:
4688diff --git a/uaclient/config.py b/uaclient/config.py
4689index 5b2de5e..e4b2f0c 100644
4690--- a/uaclient/config.py
4691+++ b/uaclient/config.py
4692@@ -1,9 +1,11 @@
4693 import copy
4694 from datetime import datetime
4695+from functools import wraps
4696 import json
4697 import logging
4698 import os
4699 import re
4700+import sys
4701 import yaml
4702 from collections import namedtuple, OrderedDict
4703
4704@@ -47,6 +49,7 @@ MERGE_ID_KEY_MAP = {
4705 "availableResources": "name",
4706 "resourceEntitlements": "type",
4707 }
4708+UNSET_SETTINGS_OVERRIDE_KEY = "_unset"
4709
4710
4711 # A data path is a filename, and an attribute ("private") indicating whether it
4712@@ -69,6 +72,7 @@ class UAConfig:
4713
4714 _entitlements = None # caching to avoid repetitive file reads
4715 _machine_token = None # caching to avoid repetitive file reading
4716+ _contract_expiry_datetime = None
4717
4718 def __init__(
4719 self, cfg: "Dict[str, Any]" = None, series: str = None
4720@@ -93,6 +97,10 @@ class UAConfig:
4721 def contract_url(self):
4722 return self.cfg.get("contract_url", "https://contracts.canonical.com")
4723
4724+ @property
4725+ def security_url(self):
4726+ return self.cfg.get("security_url", "https://ubuntu.com/security")
4727+
4728 def check_lock_info(self) -> "Tuple[int, str]":
4729 """Return lock info if config lock file is present the lock is active.
4730
4731@@ -209,14 +217,48 @@ class UAConfig:
4732 return self._entitlements
4733
4734 @property
4735+ def contract_expiry_datetime(self) -> "datetime":
4736+ """Return a datetime of the attached contract expiration."""
4737+ if not self._contract_expiry_datetime:
4738+ contractInfo = self.machine_token["machineTokenInfo"][
4739+ "contractInfo"
4740+ ]
4741+ self._contract_expiry_datetime = datetime.strptime(
4742+ contractInfo["effectiveTo"], "%Y-%m-%dT%H:%M:%SZ"
4743+ )
4744+
4745+ return self._contract_expiry_datetime
4746+
4747+ @property
4748 def is_attached(self):
4749 """Report whether this machine configuration is attached to UA."""
4750 return bool(self.machine_token) # machine_token is removed on detach
4751
4752 @property
4753+ def contract_remaining_days(self) -> int:
4754+ """Report num days until contract expiration based on effectiveTo
4755+
4756+ :return: A positive int representing the number of days the attached
4757+ contract remains in effect. Return a negative int for the number
4758+ of days beyond contract's effectiveTo date.
4759+ """
4760+ delta = self.contract_expiry_datetime.date() - datetime.utcnow().date()
4761+ return delta.days
4762+
4763+ @property
4764 def features(self):
4765 """Return a dictionary of any features provided in uaclient.conf."""
4766- return self.cfg.get("features", {})
4767+ features = self.cfg.get("features")
4768+ if features:
4769+ if isinstance(features, dict):
4770+ return features
4771+ else:
4772+ logging.warning(
4773+ "Unexpected uaclient.conf features value."
4774+ " Expected dict, but found %s",
4775+ features,
4776+ )
4777+ return {}
4778
4779 @property
4780 def machine_token(self):
4781@@ -364,7 +406,11 @@ class UAConfig:
4782 released_resources.append(resource)
4783 continue
4784
4785- if not ent_cls.is_beta:
4786+ enabled_status = status.UserFacingStatus.ACTIVE.value
4787+ if (
4788+ not ent_cls.is_beta
4789+ or resource.get("status", "") == enabled_status
4790+ ):
4791 released_resources.append(resource)
4792
4793 if released_resources:
4794@@ -518,6 +564,7 @@ class UAConfig:
4795
4796 Write the status-cache when called by root.
4797 """
4798+
4799 if os.getuid() != 0:
4800 response = cast("Dict[str, Any]", self.read_cache("status-cache"))
4801 if not response:
4802@@ -530,6 +577,15 @@ class UAConfig:
4803 if os.getuid() == 0:
4804 self.write_cache("status-cache", response)
4805
4806+ # Try to remove fix reboot notices if not applicable
4807+ if not util.should_reboot():
4808+ self.remove_notice(
4809+ "",
4810+ status.MESSAGE_ENABLE_REBOOT_REQUIRED_TMPL.format(
4811+ operation="fix operation"
4812+ ),
4813+ )
4814+
4815 config_allow_beta = util.is_config_value_true(
4816 config=self.cfg, path_to_value="features.allow_beta"
4817 )
4818@@ -620,19 +676,69 @@ def parse_config(config_path=None):
4819 for key, value in os.environ.items():
4820 key = key.lower()
4821 if key.startswith("ua_"):
4822- env_keys[key[3:]] = value # Strip leading UA_
4823+ if "ua_features_" in key:
4824+ key = key[12:] # String leading UA_FEATURES_
4825+
4826+ # Users can provide a yaml file to override
4827+ # config behavor. If they do, we are going
4828+ # to load that yaml and update the config
4829+ # with it
4830+ if value.endswith("yaml"):
4831+ if os.path.exists(value):
4832+ value = yaml.safe_load(util.load_file(value))
4833+ else:
4834+ raise exceptions.UserFacingError(
4835+ "Could not find yaml file: {}".format(value)
4836+ )
4837+
4838+ if "features" not in cfg:
4839+ cfg["features"] = {key: value}
4840+ else:
4841+ cfg["features"][key] = value
4842+ else:
4843+ env_keys[key[3:]] = value # Strip leading UA_
4844 cfg.update(env_keys)
4845 cfg["log_level"] = cfg["log_level"].upper()
4846 cfg["data_dir"] = os.path.expanduser(cfg["data_dir"])
4847- if not util.is_service_url(cfg["contract_url"]):
4848- raise exceptions.UserFacingError(
4849- "Invalid url in config. contract_url: {}".format(
4850- cfg["contract_url"]
4851+ for key in ("contract_url", "security_url"):
4852+ if not util.is_service_url(cfg[key]):
4853+ raise exceptions.UserFacingError(
4854+ "Invalid url in config. {}: {}".format(key, cfg[key])
4855 )
4856- )
4857 return cfg
4858
4859
4860+def apply_config_settings_override(override_key: str):
4861+ """Decorator used to override function return by config settings.
4862+
4863+ To identify if we should override the function return, we check
4864+ if the config object has the expected override key, we use it
4865+ has, we will use the key value as the function return. Otherwise
4866+ we will call the function normally.
4867+
4868+ @param override_key: key to be looked for in the settings_override
4869+ entry in the config dict. If that key is present, we will return
4870+ its value as the function return.
4871+ """
4872+
4873+ def wrapper(f):
4874+ @wraps(f)
4875+ def new_f():
4876+ cfg = parse_config()
4877+ value_override = cfg.get("settings_overrides", {}).get(
4878+ override_key, UNSET_SETTINGS_OVERRIDE_KEY
4879+ )
4880+
4881+ if value_override != UNSET_SETTINGS_OVERRIDE_KEY:
4882+ return value_override
4883+
4884+ return f()
4885+
4886+ return new_f
4887+
4888+ return wrapper
4889+
4890+
4891 def depth_first_merge_overlay_dict(base_dict, overlay_dict):
4892 """Merge the contents of overlay dict into base_dict not only on top-level
4893 keys, but on all on the depths of the overlay_dict object. For example,
4894@@ -678,3 +784,25 @@ def depth_first_merge_overlay_dict(base_dict, overlay_dict):
4895 base_dict[key] = value
4896 else:
4897 base_dict[key] = value
4898+
4899+
4900+def update_ua_messages(cfg: UAConfig):
4901+ """Helper to load and run ua_update_messaging.
4902+
4903+ This is needed because we don't have /usr/lib/ubuntu-advantage
4904+ python scripts in our path and we don't want to shell out with
4905+ subp to call python3 /path/to/ua_update_messaging.py.
4906+ """
4907+ sys.path.append("/usr/lib/ubuntu-advantage")
4908+ try:
4909+ __import__("ua_update_messaging")
4910+ update_msgs = getattr(
4911+ sys.modules["ua_update_messaging"], "update_apt_and_motd_messages"
4912+ )
4913+ update_msgs(cfg)
4914+ except ImportError:
4915+ logging.debug(
4916+ "Unable to update UA messages. Cannot import ua_update_messaging."
4917+ )
4918+ finally:
4919+ sys.path.pop()
4920diff --git a/uaclient/contract.py b/uaclient/contract.py
4921index 32c4347..a616b7a 100644
4922--- a/uaclient/contract.py
4923+++ b/uaclient/contract.py
4924@@ -1,5 +1,4 @@
4925 import logging
4926-import urllib
4927
4928 from uaclient import clouds
4929 from uaclient import exceptions
4930@@ -97,7 +96,7 @@ class UAContractClient(serviceclient.UAServiceClient):
4931 "kernel": platform["kernel"],
4932 }
4933 resource_response, headers = self.request_url(
4934- API_V1_RESOURCES + "?" + urllib.parse.urlencode(query_params)
4935+ API_V1_RESOURCES, query_params=query_params
4936 )
4937 return resource_response
4938
4939diff --git a/uaclient/defaults.py b/uaclient/defaults.py
4940index 129bc14..1eef1fd 100644
4941--- a/uaclient/defaults.py
4942+++ b/uaclient/defaults.py
4943@@ -6,14 +6,23 @@ any of our dependencies installed.
4944 """
4945
4946 UAC_ETC_PATH = "/etc/ubuntu-advantage/"
4947+DEFAULT_DATA_DIR = "/var/lib/ubuntu-advantage"
4948+DEFAULT_MACHINE_TOKEN_PATH = DEFAULT_DATA_DIR + "/private/machine-token.json"
4949 DEFAULT_CONFIG_FILE = UAC_ETC_PATH + "uaclient.conf"
4950 DEFAULT_HELP_FILE = UAC_ETC_PATH + "help_data.yaml"
4951 DEFAULT_UPGRADE_CONTRACT_FLAG_FILE = UAC_ETC_PATH + "request-update-contract"
4952 BASE_CONTRACT_URL = "https://contracts.canonical.com"
4953+BASE_SECURITY_URL = "https://ubuntu.com/security"
4954+BASE_UA_URL = "https://ubuntu.com/advantage"
4955+BASE_ESM_URL = "https://ubuntu.com/esm"
4956+PRINT_WRAP_WIDTH = 80
4957+CONTRACT_EXPIRY_GRACE_PERIOD_DAYS = 14
4958+CONTRACT_EXPIRY_PENDING_DAYS = 20
4959
4960 CONFIG_DEFAULTS = {
4961 "contract_url": BASE_CONTRACT_URL,
4962- "data_dir": "/var/lib/ubuntu-advantage",
4963+ "security_url": BASE_SECURITY_URL,
4964+ "data_dir": DEFAULT_DATA_DIR,
4965 "log_level": "INFO",
4966 "log_file": "/var/log/ubuntu-advantage.log",
4967 }
4968diff --git a/uaclient/entitlements/esm.py b/uaclient/entitlements/esm.py
4969index 866967f..2decf4b 100644
4970--- a/uaclient/entitlements/esm.py
4971+++ b/uaclient/entitlements/esm.py
4972@@ -1,5 +1,6 @@
4973 from uaclient.entitlements import repo
4974 from uaclient import util
4975+from uaclient.config import update_ua_messages
4976
4977 try:
4978 from typing import Optional # noqa: F401
4979@@ -11,31 +12,76 @@ except ImportError:
4980 class ESMBaseEntitlement(repo.RepoEntitlement):
4981 help_doc_url = "https://ubuntu.com/security/esm"
4982
4983+ def enable(self, *, silent_if_inapplicable: bool = False) -> bool:
4984+ enable_performed = super().enable(
4985+ silent_if_inapplicable=silent_if_inapplicable
4986+ )
4987+ if enable_performed:
4988+ update_ua_messages(self.cfg)
4989+ return enable_performed
4990+
4991+ def disable(self, silent=False) -> bool:
4992+ disable_performed = super().disable(silent=silent)
4993+ if disable_performed:
4994+ update_ua_messages(self.cfg)
4995+ return disable_performed
4996+
4997
4998 class ESMAppsEntitlement(ESMBaseEntitlement):
4999 origin = "UbuntuESMApps"
5000 name = "esm-apps"
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches

to status/vote changes: