Enable support for trusty ESM

Thanks, questions about this upload:

How do I know that keyrings/ubuntu-esm-v2-keyring.gpg is authentic? Not that I don't trust you, but when dealing with the installation of gpg keys that will be trusted by apt, it is useful to have a trust path that can be independently verified by someone other than the uploader (even if the set of people that can verify it is still limited, e.g. archive admins etc).

--- ubuntu-advantage-tools-10ubuntu0.14.04.2/apt.conf.d/51ubuntu-advantage-esm
1970-01-01 00:00:00.000000000 +0000
+++ ubuntu-advantage-tools-10ubuntu0.14.04.3/apt.conf.d/51ubuntu-advantage-esm
2019-04-18 17:24:38.000000000 +0000
@@ -0,0 +1,3 @@
+Unattended-Upgrade::Allowed-Origins {
+ "${distro_id}ESM:${distro_codename}-security";
+};

I think this is worth calling out in the changelog, since it changes the configuration of the system for all users even if the user does not enable esm.

+_apt_add_auth() {
+ local repo_host="$1"
+ local credentials="$2"
+
+ local login password
+ login=$(echo "$credentials" | cut -d: -f1)
+ password=$(echo "$credentials" | cut -d: -f2)
+ [ -d "$APT_AUTH_DIR" ] || mkdir -p "$APT_AUTH_DIR"
+ [ -f "$APT_AUTH_FILE" ] || touch "$APT_AUTH_FILE"
+ chmod 600 "$APT_AUTH_FILE"
+ echo "machine ${repo_host}/ login ${login} password ${password}" \
+ >>"$APT_AUTH_FILE"
+}

Does this mean that if I run 'ua enable-esm' twice, the file gets two entries? (Should this instead be > instead of >> so that it's idempotent?)

+_apt_remove_auth() {
+ local repo_host="$1"
+
+ sed -i "/^machine ${repo_host}\/ login/d" "$APT_AUTH_FILE"
+}

Given that this file is /etc/apt/auth.conf.d/90ubuntu-advantage which is exclusive to ESM, why sedding this out instead of deleting the file?

+deb https://${ESM_REPO_HOST}/ubuntu ${SERIES}-updates main
+# deb-src https://${ESM_REPO_HOST}/ubuntu ${SERIES}-updates main
+EOF

I would suggest that we don't enable -updates at this stage, and defer that until the new client lands.

To verify the key, you can add this to sources.list and run apt-get update, it will complain about a missing key (if you don't have it), and you can then use it and verify the complaint is gone.

deb https://esm.ubuntu.com/ubuntu trusty-security main

> Does this mean that if I run 'ua enable-esm' twice, the file gets two entries? (Should this instead be > instead of >> so that it's idempotent?)

Bummer, I thought it was gated on an is_esm_enabled check. Will fix. >> was used because in other releases other services (like fips) would also use an auth.conf file, and it was the same file (no .d existed).

> Given that this file is /etc/apt/auth.conf.d/90ubuntu-advantage which is exclusive to ESM, why sedding this out instead of deleting the file?

It used to be just /etc/apt/auth.conf, but an apt SRU allowed us to use auth.conf.d and I opted to switch to that format, because it's what the new client is using. I also opted to not change that code since it would still work and I wouldn't have to change anything else, not even tests, and the consequence is a zero-sized file if you disable esm. But on purge it gets removed.

> +deb https://${ESM_REPO_HOST}/ubuntu ${SERIES}-updates main
> +# deb-src https://${ESM_REPO_HOST}/ubuntu ${SERIES}-updates main
> +EOF
> I would suggest that we don't enable -updates at this stage, and defer that until the new client lands.

I'll check

> I would suggest that we don't enable -updates at this stage,
> and defer that until the new client lands.

Withdrawing that comment; I was reminded that since at this stage this will only be enabled in sources.list for machines that have opted in to esm, there is no significant penalty for having -updates also enabled by default.

I have verified the authenticity of ubuntu-esm-v2-keyring.gpg by this method:

$ gpg --no-default-keyring --keyring ./keyrings/ubuntu-esm-v2-keyring.gpg --list-keys
gpg: please do a --check-trustdb
./keyrings/ubuntu-esm-v2-keyring.gpg
------------------------------------
pub 4096R/4067E40313CB4B13 2019-04-17
uid Ubuntu Extended Security Maintenance Automatic Signing Key v2 <email address hidden>
sub 4096R/349F0F98EF1B9BA3 2019-04-17

 gpg --no-default-keyring --keyring ./keyrings/ubuntu-esm-v2-keyring.gpg --verify /var/lib/apt/lists/esm.ubuntu.com_ubuntu_dists_trusty-security_Release{.gpg,}
gpg: Signature made Thu Apr 18 18:15:02 2019 UTC
gpg: using RSA key 4067E40313CB4B13
gpg: please do a --check-trustdb
gpg: Good signature from "Ubuntu Extended Security Maintenance Automatic Signing Key v2 <email address hidden>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 56F7 650A 24C9 E9EC F87C 4D8D 4067 E403 13CB 4B13
$

/var/lib/apt/lists/esm.ubuntu.com_ubuntu_dists_trusty-security_Release{.gpg,} were downloaded by apt via https. The esm.ubuntu.com https endpoint is secured with a certificate issued by cn=Let's Encrypt Authority X3, a CA we have a high degree of confidence in (and is not issued by a random other CA that might have been compromised elsewhere).

This is enough for now.

Thanks for the review, re-uploaded with the fix to prevent re-enabling esm with it's already enabled.

Hello Andreas, or anyone else affected,

Accepted ubuntu-advantage-tools into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/10ubuntu0.14.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Tested: 10ubuntu0.14.04.3 from direct deb download on ubuntu-daily:14.04 LXD
Verified it doesn't show as not available anymore.
Enabled and was able to install ansible from ESM.

All working here. Direct download of: ubuntu-advantage-tools_10ubuntu0.14.04.3_all.deb

Package installs, upgrades ubuntu-advantage-tools, enables ESM with credentials from the canonical support dashboard, then I ran 'ubuntu-advantage disable-esm`, and the apt source was removed from the listing. I installed ansible from the trusty-security pocket, and it upgraded correctly.

Verified Success on new LXD ubuntu-daily:trusty.

We found a case there installing esm packages will fail with a 401.

ubuntu-advantage-tools needs apt (and libapt-pkg4.12) at version 1.0.1ubuntu2.22 or higher, otherwise it won't recognize the /etc/apt/auth.conf.d/90ubuntu-advantage file which is where the credentials are stored. apt older than 1.0.1ubuntu2.22 only knows about /etc/apt/auth.conf

Adding a depends on libapt-pkg4.12 (>= 1.0.1ubuntu2.22) to the ubuntu-advantage-tools package, perahps also on apt, should address this, but I worry about adding apt related dependencies to a package that will likely be upgraded in the same apt transaction. It's also something that other ua offerings doesn't need, like livepatch.

Another possibility is to add code to ubuntu-advantage-tools to install a newer apt if needed, when esm is enabled.

Re:#9, We have updated the instructions in the Ubuntu Advantage KB to specify upgrading apt versions first.

This bug was fixed in the package ubuntu-advantage-tools - 10ubuntu0.14.04.3

---------------
ubuntu-advantage-tools (10ubuntu0.14.04.3) trusty; urgency=medium

  * Enable support for Trusty ESM (LP: #1825239)
  * Install an unattended-upgrades configuration that allows for the
    UbuntuESM trusty-security origin.
  * Re-enable tests at package build time, just not flake8 as python3-flake8
    is in universe:
    - d/rules: run tests
    - d/control: add test dependencies

 -- Andreas Hasenack <email address hidden> Thu, 18 Apr 2019 15:20:23 +0000

The verification of the Stable Release Update for ubuntu-advantage-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.