Merge ~renanrodrigo/ubuntu/+source/ubuntu-advantage-tools:upload-27.11.3-kinetic into ubuntu/+source/ubuntu-advantage-tools:ubuntu/devel

[Renan Rodrigo]

This is the 27.11.3 release of ubuntu-advantage-tools.
This brings a couple hotfixes to bugs opened right after 27.11.2 was released.
Please refer to the changelog to see what has changed

[Robie Basak (racb)]

A couple of comments. This isn't a full review.

1) Let's be honest. Moving the message to the end, and/or providing a command to change configuration to opt-out, is certainly not fixing the spirit of bug 1992026 as it was filed, and it's distasteful to pretend that it is. I doubt that those who marked themselves as affected would consider that bug resolved or even mitigated by these changes. By all means go ahead and make these changes if you think they provide mitigation, but the correct status of bug 1992026 should then be "Won't Fix" or "Opinion", perhaps with a comment explaining what mitigating steps you are taking instead. "Fix Released" however would be wrong. Feel free to use different bugs to track what you _are_ changing, for SRU purposes, etc.

2) That "-q/-qq" doesn't suppress the message certainly seems like a bug to me. If you can't fix it directly in your hook because apt doesn't expose that information to you, then maybe it needs fixing by first changing the hook interface in apt. I suggest you file a separate bug to track that with both apt and ubuntu-advantage-tools tasks. It isn't reasonable to say that it can't be fixed in ubuntu-advantage-tools unless there's at least an open bug against apt.

[Renan Rodrigo (renanrodrigo)]

Thanks for the comments @racb.

1) We decided that product people will be responsible for updating that bug, either commenting or changing status will be up to them. We removed the (#) from the changelog entry referencing the bug, so we keep the pointer but automation does not consider it fixed.

2) https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1992472 filed against APT. I doubt they will fix and release this to all LTS before our message goes away anyway, so I am not linking ourselves there. We can leverage the functionality in the future if any other message comes up.

[Robie Basak (racb)]

> We removed the (#) from the changelog entry referencing the bug, so we keep the pointer but automation does not consider it fixed.

OK, but you do need a bug to track the regular SRU verification of the new release, please, and it won't be able to be this 1992026 bug then. Since we allow major version updates for this package, and you have test coverage for the changes you're making, this doesn't need anything more than your usual test plan, but we need a bug to track the status of that QA process.

Alternatively, you can do this SRU without relying on your exception and have a separate bug to track every change and verify each one, but I assume that's more tedious and so you'll want to do it the former way.

[Grant Orndorff (orndorffgrant)]

Thanks racb!

renanrodrigo will be out for a while so I'll take this over in the morning. I'll create a bug to track the QA like we usually do and reference that in the changelog.

There also may be some minor tweaks pending a discussion early tomorrow.

Not sure if I can update this MP since it's owned by renanrodrigo or if I'll need to create a new one.

Regardless, I'll ping you again as soon as this is ready.

[Robie Basak (racb)]

SRU +1 on commit 9417e483c9eb15e012bf73c1ae1beffb4903886f

Your implementation of "pro config set apt_news=False" seems fine. However, I suggest that users who want to disable that also want to disable everything else this package does in relation to the apt hook, so removing /etc/apt/apt.conf.d/20apt-esm-hook.conf would be my recommendation to those users instead. This is a configuration change in /etc so must be preserved by packaging policy.

review: Approve (ubuntu-sru)

[Robie Basak (racb)]

> SRU +1 on commit 9417e483c9eb15e012bf73c1ae1beffb4903886f

(but with bug reference changes as discussed)

[Paride Legovini (paride)]

I am also +1 on the technical aspects of this MP. Robie's comments on the non strictly technical aspects are way more insightful than what I could have come up with, I agree with them and I'm happy with the middle ground you found, given that there is no perfect fix to the concerns that were raised. I can proceed sponsoring once the bug references are updated, just ping me.

As a general comment on this MP: I think this could have been uploaded as initially proposed. The discussion here did improve the upload and allow for better bug tracking and management, but bug statuses can be adjusted after uploading, and same for filing extra bugs. This is all basically invisible to the end user. In case it isn't clear: I'm writing this with your PPU application in mind :-)

review: Approve

[Grant Orndorff (orndorffgrant)]

Thank you racb and paride for the reviews!

As mentioned in my last comment there are some tweaks incoming. You can see a preview of what they might be here: https://github.com/canonical/ubuntu-advantage-client/pull/2293

They will at least include:
* modifying this output to only appear when using `apt` (and not `apt-get`)
* fixing https://github.com/canonical/ubuntu-advantage-client/issues/2294
* a change to the message itself, still not finalized

I've been told the message will be finalized by tomorrow, so hopefully by tomorrow I will have everything in place (SRU bug, updated MP) for another round of review and we can get this out by early next week.

I'll be sure to ping you both when it is ready.

Unmerged commits

9417e48... by Renan Rodrigo on 2022-10-10

chore: fix the changelog entry for 27.11.3

Signed-off-by: Renan Rodrigo <email address hidden>

520d708... by Renan Rodrigo on 2022-10-10

new release 27.11.3

Signed-off-by: Renan Rodrigo <email address hidden>

e15a247... by Lucas Albuquerque Medeiros de Moura on 2022-10-06

collect-logs: Do not fail if we can't read a file

Currently, if we fail to read a file, we will error out
during the execution of the command. We are updating that
so that we will just warn the user that a file could not be read
and proceed with the command.

LP: #1991858

6e4c406... by Renan Rodrigo on 2022-10-10

tests: add check for APT messages disabling apt_news

Signed-off-by: Renan Rodrigo <email address hidden>

41f5a7a... by Grant Orndorff on 2022-10-07

messaging: move pro beta message and add config option to disable it

The placement of the "Try Ubuntu Pro beta..." message in apt output is
considered distasteful. To make it somewhat more sensible, we're moving
the message to the bottom of apt output, so that it is more clearly a
separate banner from the normal output.

We're also adding a config option to disable this post-apt-output
banner: pro config set apt_news=false

LP: #1992026

74f2f55... by Renan Rodrigo on 2022-10-03

tests: fix 'or' condition in docker feature

Signed-off-by: Renan Rodrigo <email address hidden>

e663f76... by Grant Orndorff on 2022-09-30

tests: various fixups to bring tests inline with expectations

diff --git a/apt-hook/json-hook.cc b/apt-hook/json-hook.cc
index 0efaa81..bacf033 100644
--- a/apt-hook/json-hook.cc
+++ b/apt-hook/json-hook.cc
@@ -257,23 +257,31 @@ int run()
 
     json_object_put(hello_req.root_msg);
 
-    jsonrpc_request stats_req;
-    success = read_jsonrpc_request(socket_in, stats_req);
+    jsonrpc_request hook_req;
+    success = read_jsonrpc_request(socket_in, hook_req);
     if (!success) {
         std::cerr << "ua-hook: failed to read hook msg" << std::endl;
         return 0;
     }
-    if (stats_req.method == "org.debian.apt.hooks.install.statistics") {
+    if (hook_req.method == "org.debian.apt.hooks.install.statistics") {
         security_package_counts counts;
-        success = count_security_packages_from_apt_stats_json(stats_req.params, counts);
+        success = count_security_packages_from_apt_stats_json(hook_req.params, counts);
         if (success) {
             std::string message = create_count_message(counts);
             if (message != "") {
                 std::cout << message << std::endl;
             }
         }
+    } else if (hook_req.method == "org.debian.apt.hooks.install.post") {
+        std::ifstream apt_news_flag_file("/var/lib/ubuntu-advantage/flags/show-apt-news");
+        if (apt_news_flag_file.is_open()) {
+            std::cout << std::endl;
+            std::cout << "Try Ubuntu Pro beta with a free personal subscription on up to 5 machines." << std::endl;
+            std::cout << "Learn more at https://ubuntu.com/pro" << std::endl;
+            apt_news_flag_file.close();
+        }
     }
-    json_object_put(stats_req.root_msg);
+    json_object_put(hook_req.root_msg);
 
     jsonrpc_request bye_req;
     success = read_jsonrpc_request(socket_in, bye_req);
diff --git a/debian/changelog b/debian/changelog
index a5acfc4..f6dbb14 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,15 @@
+ubuntu-advantage-tools (27.11.3~22.10.1) kinetic; urgency=medium
+
+  * New upstream release 27.11.3:
+    - d/postinst: remove the old Ubuntu Pro message and set up the
+      configurable flag instead
+    - apport-hook: do not fail if a file cannot be read (LP: #1991858)
+    - config: add a flag to remove the APT related news (LP: 1992026)
+      (GH: #2288)
+    - messaging: move the Ubuntu Pro beta message to the end of the APT output
+
+ -- Renan Rodrigo <[e-mail]>  Mon, 10 Oct 2022 15:08:51 -0300
+
 ubuntu-advantage-tools (27.11.2~22.10.1) kinetic; urgency=medium
 
   * New upstream release 27.11.2: (LP: #1991173)
diff --git a/debian/ubuntu-advantage-tools.postinst b/debian/ubuntu-advantage-tools.postinst
index 4a677fc..aa2a1a3 100644
--- a/debian/ubuntu-advantage-tools.postinst
+++ b/debian/ubuntu-advantage-tools.postinst
@@ -57,6 +57,11 @@ OLD_LICENSE_CHECK_MARKER_FILE="/var/lib/ubuntu-advantage/marker-license-check"
 MACHINE_TOKEN_FILE="/var/lib/ubuntu-advantage/private/machine-token.json"
 PUBLIC_MACHINE_TOKEN_FILE="/var/lib/ubuntu-advantage/machine-token.json"
 
+UA_MESSAGES_DIR="/var/lib/ubuntu-advantage/messages"
+APT_ESM_MESSAGE_FILE="$UA_MESSAGES_DIR/apt-pre-invoke-esm-service-status"
+ESM_APPS_APT_MSG_FILES="$UA_MESSAGES_DIR/apt-pre-invoke-no-packages-apps.tmpl $UA_MESSAGES_DIR/apt-pre-invoke-no-packages-apps $UA_MESSAGES_DIR/apt-pre-invoke-packages-apps.tmpl $UA_MESSAGES_DIR/apt-pre-invoke-packages-apps"
+UA_FLAGS_DIR="/var/lib/ubuntu-advantage/flags"
+APT_NEWS_FLAG_FILE="$UA_FLAGS_DIR/show-apt-news"
 
 # Rename apt config files for ua services removing ubuntu release names
 redact_ubuntu_release_from_ua_apt_filenames() {
@@ -384,6 +389,23 @@ machine_token_file.write(content=content)
 "
 }
 
+migrate_ubuntu_pro_beta_banner() {
+    PREVIOUS_PKG_VER=$1
+    # This only shipped in 27.11.2~
+    if dpkg --compare-versions "$PREVIOUS_PKG_VER" ge "27.11.2~" \
+       && dpkg --compare-versions "$PREVIOUS_PKG_VER" lt "27.11.3~"; then
+        # if the banner was present, remove it and add the new flag file which is configurable
+        if [ -f $APT_ESM_MESSAGE_FILE ]; then
+            if cat $APT_ESM_MESSAGE_FILE | grep -q "Try Ubuntu Pro beta"; then
+                rm -f $APT_ESM_MESSAGE_FILE
+                rm -f $ESM_APPS_APT_MSG_FILES
+                mkdir -p $UA_FLAGS_DIR
+                touch $APT_NEWS_FLAG_FILE
+            fi
+        fi
+    fi
+}
+
 case "$1" in
     configure)
       PREVIOUS_PKG_VER=$2
@@ -462,6 +484,7 @@ case "$1" in
       if [ -f $MACHINE_TOKEN_FILE ] && [ ! -f $PUBLIC_MACHINE_TOKEN_FILE ]; then
           create_public_machine_token_file
       fi
+      migrate_ubuntu_pro_beta_banner $PREVIOUS_PKG_VER
       ;;
 esac
 
diff --git a/features/api_full_auto_attach.feature b/features/api_full_auto_attach.feature
index 240e06f..c65fca5 100644
--- a/features/api_full_auto_attach.feature
+++ b/features/api_full_auto_attach.feature
@@ -27,7 +27,7 @@ Feature: Full Auto-Attach Endpoint
         """
         Then stdout matches regexp:
         """
-        livepatch     +yes +disabled  +Canonical Livepatch service
+        livepatch     +yes +(disabled|n/a)  +Canonical Livepatch service
         """
         Examples:
            | release |
diff --git a/features/apt_messages.feature b/features/apt_messages.feature
index 27d67cb..eee585d 100644
--- a/features/apt_messages.feature
+++ b/features/apt_messages.feature
@@ -225,9 +225,10 @@ Feature: APT Messages
         Building dependency tree...
         Reading state information...
         Calculating upgrade...
+        0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
+
         Try Ubuntu Pro beta with a free personal subscription on up to 5 machines.
         Learn more at https://ubuntu.com/pro
-        0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
         """
         When I attach `contract_token` with sudo
         When I run `apt-get upgrade --dry-run` with sudo
@@ -267,9 +268,17 @@ Feature: APT Messages
         Building dependency tree...
         Reading state information...
         Calculating upgrade...
+        0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded\.
+
+        Try Ubuntu Pro beta with a free personal subscription on up to 5 machines.
+        Learn more at https:\/\/ubuntu.com\/pro
+        """
+        When I run `pro config set apt_news=False` with sudo
+        And I run `apt-get upgrade` with sudo
+        Then stdout does not match regexp:
+        """
         Try Ubuntu Pro beta with a free personal subscription on up to 5 machines.
         Learn more at https:\/\/ubuntu.com\/pro
-        0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded\.
         """
         Examples: ubuntu release
           | release |
diff --git a/features/cloud_pro_clone.feature b/features/cloud_pro_clone.feature
index f08bff0..48aa17a 100644
--- a/features/cloud_pro_clone.feature
+++ b/features/cloud_pro_clone.feature
@@ -2,7 +2,6 @@ Feature: Creating golden images based on Cloud Ubuntu Pro instances
 
     @series.lts
     @uses.config.machine_type.aws.pro
-    @uses.config.machine_type.azure.pro
     @uses.config.machine_type.gcp.pro
     Scenario Outline: Create a Pro fips-updates image and launch
         Given a `<release>` machine with ubuntu-advantage-tools installed
diff --git a/features/daemon.feature b/features/daemon.feature
index 593c86d..0b33c37 100644
--- a/features/daemon.feature
+++ b/features/daemon.feature
@@ -104,9 +104,10 @@ Feature: Pro Upgrade Daemon only runs in environments where necessary
         """
         Active: active \(running\)
         """
-        Then on `xenial`, systemd status output says memory usage is less than `14` MB
+        # TODO find out what caused memory to go up, try to lower it again
+        Then on `xenial`, systemd status output says memory usage is less than `15` MB
         Then on `bionic`, systemd status output says memory usage is less than `13` MB
-        Then on `focal`, systemd status output says memory usage is less than `11` MB
+        Then on `focal`, systemd status output says memory usage is less than `12` MB
         Then on `jammy`, systemd status output says memory usage is less than `12` MB
 
         When I run `cat /var/log/ubuntu-advantage-daemon.log` with sudo
diff --git a/features/docker.feature b/features/docker.feature
index e4ac590..2e82266 100644
--- a/features/docker.feature
+++ b/features/docker.feature
@@ -19,7 +19,9 @@ Feature: Build docker images with pro services
             apt-get update \
             && apt-get install --no-install-recommends -y ubuntu-advantage-tools ca-certificates \
 
-            && dpkg -i /ua.deb \
+            && ((dpkg -i /ua.deb || true)) \
+
+            && apt-get install -f \
 
             && pro attach --attach-config /run/secrets/ua-attach-config \
 
diff --git a/features/environment.py b/features/environment.py
index 39013d0..bfe0042 100644
--- a/features/environment.py
+++ b/features/environment.py
@@ -23,10 +23,6 @@ DEFAULT_UA_PPA_KEYID = "6E34E7116C0BC933"
 
 USERDATA_BLOCK_AUTO_ATTACH_IMG = """\
 #cloud-config
-[bootcmd, once]:
- - cp /usr/bin/ua /usr/bin/ua.orig
- - 'echo "#!/bin/sh\ndate >> /root/ua-calls\n" > /usr/bin/ua'
- - chmod 755 /usr/bin/ua
 """
 
 # we can't use write_files because it will clash with the
@@ -40,8 +36,10 @@ runcmd:
 USERDATA_RUNCMD_ENABLE_PROPOSED = """
 runcmd:
   - printf \"deb http://archive.ubuntu.com/ubuntu/ {series}-proposed main\" > /etc/apt/sources.list.d/uaclient-proposed.list
+  - printf \"deb http://archive.ubuntu.com/ubuntu/ {series}-proposed universe\" > /etc/apt/sources.list.d/uaclient-proposed-universe.list
   - "printf \\"Package: *\\nPin: release a={series}-proposed\\nPin-Priority: 400\\n\\" > /etc/apt/preferences.d/lower-proposed"
   - "printf \\"Package: ubuntu-advantage-tools\\nPin: release a={series}-proposed\\nPin-Priority: 1001\\n\\" > /etc/apt/preferences.d/uaclient-proposed"
+  - "printf \\"Package: ubuntu-advantage-pro\\nPin: release a={series}-proposed\\nPin-Priority: 1001\\n\\" > /etc/apt/preferences.d/uaclientpro-proposed"
 """  # noqa: E501
 
 USERDATA_APT_SOURCE_PPA = """\
@@ -333,6 +331,8 @@ class UAClientBehaveConfig:
 def before_all(context: Context) -> None:
     """behave will invoke this before anything else happens."""
     context.config.setup_logging()
+    logging.getLogger("botocore").setLevel(logging.INFO)
+    logging.getLogger("boto3").setLevel(logging.INFO)
     userdata = context.config.userdata
     if userdata:
         logging.debug("Userdata key / value pairs:")
diff --git a/features/ubuntu_pro.feature b/features/ubuntu_pro.feature
index f23dcde..c373a95 100644
--- a/features/ubuntu_pro.feature
+++ b/features/ubuntu_pro.feature
@@ -23,6 +23,7 @@ Feature: Command behaviour when auto-attached in an ubuntu PRO image
         """
         And I verify `/var/log/squid/access.log` is empty on `proxy` machine
         When I run `pro auto-attach` with sudo
+        When I run `pro status --all` with sudo
         Then stdout matches regexp:
             """
             SERVICE       +ENTITLED  STATUS    DESCRIPTION
@@ -94,6 +95,7 @@ Feature: Command behaviour when auto-attached in an ubuntu PRO image
         """
         And I verify `/var/log/squid/access.log` is empty on `proxy` machine
         When I run `pro auto-attach` with sudo
+        When I run `pro status --all` with sudo
         Then stdout matches regexp:
             """
             SERVICE       +ENTITLED  STATUS    DESCRIPTION
@@ -165,6 +167,7 @@ Feature: Command behaviour when auto-attached in an ubuntu PRO image
         """
         And I verify `/var/log/squid/access.log` is empty on `proxy` machine
         When I run `pro auto-attach` with sudo
+        When I run `pro status --all` with sudo
         Then stdout matches regexp:
             """
             SERVICE       +ENTITLED  STATUS    DESCRIPTION
diff --git a/integration-requirements.txt b/integration-requirements.txt
index 9eaf14f..de2698d 100644
--- a/integration-requirements.txt
+++ b/integration-requirements.txt
@@ -2,7 +2,7 @@
 behave
 jsonschema
 PyHamcrest
-pycloudlib @ git+https://github.com/canonical/pycloudlib.git@25cca01b24d111e00e91a86eb1021b88592e069e
+pycloudlib @ git+https://github.com/canonical/pycloudlib.git@e7c4a42eb98a914b084b253c3f96c960de42e8fa
 toml==0.10
 
 
diff --git a/uaclient/actions.py b/uaclient/actions.py
index 9b1af00..3f602e8 100644
--- a/uaclient/actions.py
+++ b/uaclient/actions.py
@@ -160,6 +160,23 @@ def _write_command_output_to_file(
         system.write_file(filename, out)
 
 
+def _get_state_files(cfg: config.UAConfig):
+    # include cfg log files here because they could be set to non default
+    return [
+        cfg.cfg_path or DEFAULT_CONFIG_FILE,
+        cfg.log_file,
+        cfg.timer_log_file,
+        cfg.daemon_log_file,
+        cfg.data_path("jobs-status"),
+        CLOUD_BUILD_INFO,
+        *(
+            entitlement.repo_list_file_tmpl.format(name=entitlement.name)
+            for entitlement in entitlements.ENTITLEMENT_CLASSES
+            if issubclass(entitlement, entitlements.repo.RepoEntitlement)
+        ),
+    ]
+
+
 def collect_logs(cfg: config.UAConfig, output_dir: str):
     """
     Write all relevant Ubuntu Pro logs to the specified directory
@@ -199,28 +216,19 @@ def collect_logs(cfg: config.UAConfig, output_dir: str):
             return_codes=[0, 3],
         )
 
-    # include cfg log files here because they could be set to non default
-    state_files = [
-        cfg.cfg_path or DEFAULT_CONFIG_FILE,
-        cfg.log_file,
-        cfg.timer_log_file,
-        cfg.daemon_log_file,
-        cfg.data_path("jobs-status"),
-        CLOUD_BUILD_INFO,
-        *(
-            entitlement.repo_list_file_tmpl.format(name=entitlement.name)
-            for entitlement in entitlements.ENTITLEMENT_CLASSES
-            if issubclass(entitlement, entitlements.repo.RepoEntitlement)
-        ),
-    ]
+    state_files = _get_state_files(cfg)
 
     # also get default logrotated log files
     for f in state_files + glob.glob(DEFAULT_LOG_PREFIX + "*"):
         if os.path.isfile(f):
             try:
                 content = system.load_file(f)
-            except PermissionError as e:
-                logging.warning(e)
+            except Exception as e:
+                # If we fail to load that file for any reason we will
+                # not break the command, we will instead warn the user
+                # about the issue and try to process the other files
+                logging.warning("Failed to load file: %s\n%s", f, str(e))
+                continue
             content = util.redact_sensitive_logs(content)
             if os.getuid() == 0:
                 # if root, overwrite the original with redacted content
diff --git a/uaclient/cli.py b/uaclient/cli.py
index 3473414..55f8bec 100644
--- a/uaclient/cli.py
+++ b/uaclient/cli.py
@@ -49,7 +49,9 @@ from uaclient.entitlements.entitlement_status import (
     CanEnableFailureReason,
 )
 from uaclient.jobs.update_messaging import (
+    clear_apt_news_flag,
     refresh_motd,
+    set_apt_news_flag,
     update_apt_and_motd_messages,
 )
 
@@ -1022,6 +1024,13 @@ def action_config_set(args, *, cfg, **kwargs):
                     "<value> for interval must be a positive integer."
                 ).format(set_key, set_value)
             )
+    elif set_key == "apt_news":
+        set_value = set_value.lower() == "true"
+        if set_value:
+            set_apt_news_flag(cfg)
+        else:
+            clear_apt_news_flag(cfg)
+
     setattr(cfg, set_key, set_value)
 
 
diff --git a/uaclient/config.py b/uaclient/config.py
index 2b43d55..bb7bb06 100644
--- a/uaclient/config.py
+++ b/uaclient/config.py
@@ -49,6 +49,7 @@ UA_CONFIGURABLE_KEYS = (
     "update_messaging_timer",
     "update_status_timer",
     "metering_timer",
+    "apt_news",
 )
 
 # Basic schema validation top-level keys for parse_config handling
@@ -304,6 +305,17 @@ class UAConfig:
         self.cfg["ua_config"]["polling_error_retry_delay"] = value
         self.write_cfg()
 
+    @property
+    def apt_news(self) -> bool:
+        return self.cfg.get("ua_config", {}).get("apt_news", True)
+
+    @apt_news.setter
+    def apt_news(self, value: bool):
+        if "ua_config" not in self.cfg:
+            self.cfg["ua_config"] = {}
+        self.cfg["ua_config"]["apt_news"] = value
+        self.write_cfg()
+
     def check_lock_info(self) -> Tuple[int, str]:
         """Return lock info if config lock file is present the lock is active.
 
diff --git a/uaclient/jobs/update_messaging.py b/uaclient/jobs/update_messaging.py
index 4a3893f..8d24cf0 100644
--- a/uaclient/jobs/update_messaging.py
+++ b/uaclient/jobs/update_messaging.py
@@ -28,7 +28,6 @@ from uaclient.messages import (
     CONTRACT_EXPIRED_MOTD_SOON_TMPL,
     DISABLED_APT_NO_PKGS_TMPL,
     DISABLED_APT_PKGS_TMPL,
-    TRY_UBUNTU_PRO_BETA,
 )
 
 XENIAL_ESM_URL = "https://ubuntu.com/16-04"
@@ -68,6 +67,21 @@ UPDATE_NOTIFIER_MOTD_SCRIPT = (
 )
 
 
+def set_apt_news_flag(cfg: config.UAConfig):
+    if (
+        system.is_current_series_lts()
+        and not system.is_current_series_active_esm()
+        and not cfg.is_attached
+    ):
+        system.create_file(
+            os.path.join(cfg.data_dir, "flags", "show-apt-news")
+        )
+
+
+def clear_apt_news_flag(cfg: config.UAConfig):
+    system.remove_file(os.path.join(cfg.data_dir, "flags", "show-apt-news"))
+
+
 def get_contract_expiry_status(
     cfg: config.UAConfig,
 ) -> Tuple[ContractExpiryStatus, int]:
@@ -311,32 +325,6 @@ def write_apt_and_motd_templates(cfg: config.UAConfig, series: str) -> None:
             ],
         )
 
-    if (
-        system.is_current_series_lts()
-        and not system.is_active_esm(series)
-        and not cfg.is_attached
-    ):
-        _write_template_or_remove(
-            TRY_UBUNTU_PRO_BETA, os.path.join(msg_dir, apps_no_pkg_file)
-        )
-        _write_template_or_remove(
-            TRY_UBUNTU_PRO_BETA, os.path.join(msg_dir, apps_pkg_file)
-        )
-        _write_template_or_remove("", os.path.join(msg_dir, infra_no_pkg_file))
-        _write_template_or_remove("", os.path.join(msg_dir, infra_pkg_file))
-        _write_template_or_remove(
-            "", os.path.join(msg_dir, motd_apps_no_pkg_file)
-        )
-        _write_template_or_remove(
-            "", os.path.join(msg_dir, motd_apps_pkg_file)
-        )
-        _write_template_or_remove(
-            "", os.path.join(msg_dir, motd_infra_no_pkg_file)
-        )
-        _write_template_or_remove(
-            "", os.path.join(msg_dir, motd_infra_pkg_file)
-        )
-
 
 def write_esm_announcement_message(cfg: config.UAConfig, series: str) -> None:
     """Write human-readable messages if ESM is offered on this LTS release.
@@ -396,6 +384,12 @@ def update_apt_and_motd_messages(cfg: config.UAConfig) -> bool:
 
     # Announce ESM availabilty on active ESM LTS releases
     # write_esm_announcement_message(cfg, series)
+
+    if cfg.apt_news:
+        set_apt_news_flag(cfg)
+    else:
+        clear_apt_news_flag(cfg)
+
     write_apt_and_motd_templates(cfg, series)
     # Now that we've setup/cleanedup templates render them with apt-hook
     system.subp(
diff --git a/uaclient/messages.py b/uaclient/messages.py
index 95f1605..e3061be 100644
--- a/uaclient/messages.py
+++ b/uaclient/messages.py
@@ -1034,7 +1034,3 @@ Package names in {bold}bold{end_bold} currently have an available update
 with '{{service}}' enabled""".format(
     bold=TxtColor.BOLD, end_bold=TxtColor.ENDC
 )
-
-TRY_UBUNTU_PRO_BETA = """\
-Try Ubuntu Pro beta with a free personal subscription on up to 5 machines.
-Learn more at https://ubuntu.com/pro"""
diff --git a/uaclient/system.py b/uaclient/system.py
index ef48d0c..3983e84 100644
--- a/uaclient/system.py
+++ b/uaclient/system.py
@@ -1,6 +1,7 @@
 import datetime
 import logging
 import os
+import pathlib
 import re
 import subprocess
 import time
@@ -205,6 +206,11 @@ def is_active_esm(series: str) -> bool:
 
 
 @lru_cache(maxsize=None)
+def is_current_series_active_esm() -> bool:
+    return is_active_esm(get_platform_info()["series"])
+
+
+@lru_cache(maxsize=None)
 def is_container(run_path: str = "/run") -> bool:
     """Checks to see if this code running in a container of some sort"""
 
@@ -350,6 +356,13 @@ def load_file(filename: str, decode: bool = True) -> str:
     return content.decode("utf-8")
 
 
+def create_file(filename: str, mode: int = 0o644) -> None:
+    logging.debug("Creating file: %s", filename)
+    os.makedirs(os.path.dirname(filename), exist_ok=True)
+    pathlib.Path(filename).touch()
+    os.chmod(filename, mode)
+
+
 def write_file(filename: str, content: str, mode: int = 0o644) -> None:
     """Write content to the provided filename encoding it if necessary.
 
diff --git a/uaclient/tests/test_actions.py b/uaclient/tests/test_actions.py
index 6f77d7f..12c05f1 100644
--- a/uaclient/tests/test_actions.py
+++ b/uaclient/tests/test_actions.py
@@ -1,8 +1,10 @@
+import logging
+
 import mock
 import pytest
 
 from uaclient import exceptions, messages
-from uaclient.actions import attach_with_token, auto_attach
+from uaclient.actions import attach_with_token, auto_attach, collect_logs
 from uaclient.exceptions import ContractAPIError, NonAutoAttachImageError
 from uaclient.tests.test_cli_auto_attach import fake_instance_factory
 
@@ -141,3 +143,36 @@ class TestAutoAttach:
             auto_attach(cfg, fake_instance_factory())
 
         assert unexpected_error == excinfo.value
+
+
[e-mail]("uaclient.actions._write_command_output_to_file")
+class TestCollectLogs:
+    @pytest.mark.parametrize("caplog_text", [logging.WARNING], indirect=True)
+    @mock.patch("os.getuid")
+    @mock.patch("uaclient.system.write_file")
+    @mock.patch("uaclient.system.load_file")
+    @mock.patch("uaclient.actions._get_state_files")
+    @mock.patch("glob.glob")
+    def test_collect_logs_invalid_file(
+        self,
+        m_glob,
+        m_get_state_files,
+        m_load_file,
+        m_write_file,
+        m_getuid,
+        m_write_cmd,
+        caplog_text,
+    ):
+        m_get_state_files.return_value = ["a", "b"]
+        m_load_file.side_effect = [UnicodeError("test"), "test"]
+        m_getuid.return_value = 1
+        m_glob.return_value = []
+
+        with mock.patch("os.path.isfile", return_value=True):
+            collect_logs(cfg=mock.MagicMock(), output_dir="test")
+
+        assert 2 == m_load_file.call_count
+        assert [mock.call("a"), mock.call("b")] == m_load_file.call_args_list
+        assert 1 == m_write_file.call_count
+        assert [mock.call("test/b", "test")] == m_write_file.call_args_list
+        assert "Failed to load file: a\n" in caplog_text()
diff --git a/uaclient/tests/test_cli_config_set.py b/uaclient/tests/test_cli_config_set.py
index 66aaf82..d8a7c49 100644
--- a/uaclient/tests/test_cli_config_set.py
+++ b/uaclient/tests/test_cli_config_set.py
@@ -16,7 +16,8 @@ positional arguments:
                   must be one of: http_proxy, https_proxy, apt_http_proxy,
                   apt_https_proxy, ua_apt_http_proxy, ua_apt_https_proxy,
                   global_apt_http_proxy, global_apt_https_proxy,
-                  update_messaging_timer, update_status_timer, metering_timer
+                  update_messaging_timer, update_status_timer, metering_timer,
+                  apt_news
 
 Flags:
   -h, --help      show this help message and exit
diff --git a/uaclient/tests/test_cli_config_show.py b/uaclient/tests/test_cli_config_show.py
index d4582fd..ebf760c 100644
--- a/uaclient/tests/test_cli_config_show.py
+++ b/uaclient/tests/test_cli_config_show.py
@@ -117,6 +117,7 @@ global_apt_https_proxy  http://global_apt_https_proxy
 update_messaging_timer  None
 update_status_timer     None
 metering_timer          None
+apt_news                True
 """
                 == out
             )
diff --git a/uaclient/tests/test_cli_config_unset.py b/uaclient/tests/test_cli_config_unset.py
index 115bc86..42c3625 100644
--- a/uaclient/tests/test_cli_config_unset.py
+++ b/uaclient/tests/test_cli_config_unset.py
@@ -15,7 +15,7 @@ positional arguments:
               http_proxy, https_proxy, apt_http_proxy, apt_https_proxy,
               ua_apt_http_proxy, ua_apt_https_proxy, global_apt_http_proxy,
               global_apt_https_proxy, update_messaging_timer,
-              update_status_timer, metering_timer
+              update_status_timer, metering_timer, apt_news
 
 Flags:
   -h, --help  show this help message and exit
diff --git a/uaclient/tests/test_config.py b/uaclient/tests/test_config.py
index bd8a49e..d3b493c 100644
--- a/uaclient/tests/test_config.py
+++ b/uaclient/tests/test_config.py
@@ -295,6 +295,7 @@ UA_CFG_DICT = {
     "ua_config": {
         "apt_http_proxy": None,
         "apt_https_proxy": None,
+        "apt_news": True,
         "global_apt_http_proxy": None,
         "global_apt_https_proxy": None,
         "ua_apt_http_proxy": None,
@@ -316,7 +317,9 @@ class TestUAConfigKeys:
     ):
         """Getters and settings are available fo UA_CONFIGURABLE_KEYS."""
         cfg = FakeConfig()
-        assert None is getattr(cfg, attr_name, None)
+        assert UA_CFG_DICT["ua_config"][attr_name] == getattr(
+            cfg, attr_name, None
+        )
         cfg_non_members = ("apt_http_proxy", "apt_https_proxy")
         if attr_name not in cfg_non_members:
             setattr(cfg, attr_name, attr_name + "value")
diff --git a/uaclient/version.py b/uaclient/version.py
index 54776f6..c9b72ab 100644
--- a/uaclient/version.py
+++ b/uaclient/version.py
@@ -18,7 +18,7 @@ from uaclient.defaults import CANDIDATE_CACHE_PATH, UAC_TMP_PATH
 from uaclient.exceptions import ProcessExecutionError
 from uaclient.system import subp
 
-__VERSION__ = "27.11.2"
+__VERSION__ = "27.11.3"
 PACKAGED_VERSION = "@@PACKAGED_VERSION@@"
 
 CANDIDATE_REGEX = r"Candidate: (?P<candidate>.*?)\n"