Merge ~lamoura/ubuntu/+source/update-notifier:update-notifier-esm-support into ubuntu/+source/update-notifier:ubuntu/xenial-devel

[Lucas Albuquerque Medeiros de Moura]

Currently, the apt-check script is configured to only handle package count for ESM Infra. We are now updating the logic to also handle ESM Apps packages as well.

Furthermore, we are also updating the messaging that is created in apt-check. We are advertising ESM Apps if the service is disabled and only messaging about ESM Infra if the distro is already on ESM mode.

Finally, this PR also fixes LP #1883315, since we now check to see if the system has esm-infra or esm-apps before performing a package count for package with esm origins

[Lucas Albuquerque Medeiros de Moura (lamoura)]

This is the result of running autopkgtest on the xenial package uploaded to:
https://launchpad.net/~lamoura/+archive/ubuntu/update-notifier-test-ppa/

----------------------
autopkgtest [10:59:37]: test nose-tests: [-----------------------
Get:1 http://localhost:46825/canary-file.txt [4 B]
Fetched 4 B in 0s (0 B/s)
..........Get:1 http://localhost:23274/canary-file.txt [4 B]
Fetched 4 B in 0s (0 B/s)
.Get:1 http://localhost:8629/canary-file.txt [4 B]
Fetched 4 B in 0s (0 B/s)
....Get:1 http://localhost:10077/canary-file.txt [4 B]
Fetched 4 B in 0s (0 B/s)
..
----------------------------------------------------------------------
Ran 17 tests in 2.312s

OK
autopkgtest [10:59:40]: test nose-tests: -----------------------]
autopkgtest [10:59:41]: test nose-tests: - - - - - - - - - - results - - - - - - - - - -
nose-tests PASS
autopkgtest [10:59:41]: @@@@@@@@@@@@@@@@@@@@ summary
nose-tests PASS
-------------------------------

I have redacted the output here because of the number of lines. But you can reproduce the command running:

autopkgtest -U ../out/update-notifier_3.168.14.dsc -- lxd ubuntu-daily:xenial

[Lucas Albuquerque Medeiros de Moura (lamoura)]

Also, I have uploaded a test script that can be used here in the SRU bug for this release;
https://bugs.launchpad.net/ubuntu/+source/update-notifier/+bug/1924766

[Bryce Harrington (bryce)]

Hi Lucas,

I'm still in the middle of reviewing but wanted to give some early feedback on the SRU bug report, you can consider working on in the interim.

I see this is aiming to be an SRU to xenial, that fixes two bugs. In general, the SRU team will want to see SRU templates on both referenced bugs; it looks to me like in this case that would be appropriate, so I'd suggest filling out LP: #1883315 with impact/test case/regression etc.

In both bug reports, for the [Where problems could occur], I like to think from the context of if after some user upgrades to this release and encounter a bug, how could they roughly determine if their issue might or might not relate to this change? IOW, what kinds of problems would you want to look at as perhaps having caused? In this case, in addition to what you wrote about unexpected oddnesses in the MOTD text, I would also consider if a crash (python stack trace) showed up in the log files for whatever triggers apt-check.py. Obviously also erroneous displays of ESM update availability or counts would also be worth consideration.

I notice that LP: #1924766 targeted xenial and -devel, but 1883315 targets only -devel. I've updated both bug reports to have tags for all the other currently supported releases. If the bug does not exist in some or all of those releases, has already been fixed, or is not going to be fixed for some reason, then those bug tasks can be marked Invalid, Fix Released, or WontFix, accordingly; this will communicate status to the SRU team and other bug reviewers. I did not add a bug task for trusty, as I'm not sure what the support status for ESM update-notifier is there, but we can also add that if it's at all relevant. You should also add yourself as assignee for the Xenial bug tasks for each bug report, and optionally add yourself for any other releases you plan on also SRUing.

[Bryce Harrington (bryce)]

Here's part 2 of my review, this one is focusing on the code and test changes.

I'm assuming this has already gone through a peer code review process, so am assuming the overall logic has been thoroughly doublechecked. Adding the two logical paths makes a bit of a state machine here, so the logic is starting to get a bit complex. However, the new test cases look like they're well thought out and thorough. I read through and spot checked code but found no obvious errors.

If we ever get to a point where a third path needs fitted in (like a third service type, or splitting stable/edge apps or some such), it may be worth moving to a class/subclass abstraction. Some of this logic feels like it might be easier to write tests for if it was encapsulated in a class object.

One minor thing I would recommend as 'needs fixing' is adding code docs to the new helper functions being added. The new is_* and have_* routines are short and simple so the docs should be easy to add as well.

review: Needs Fixing

[Bryce Harrington (bryce)]

For part 3, looking at the packaging changes; in this case jus adding python3-distro-info as a dependency. I doublechecked this package is in all supported releases and LGTM.

The only other import change is unittest.mock in the test case. This is already listed in Build-Depends.

For part 4 I'll run the tests and verify the functionality, but as it's already getting late in my day I'll leave that to tackle tomorrow.

[Lucas Albuquerque Medeiros de Moura (lamoura)]

Hi Bryce, thank you for the review. I agree that if add more services in this script, create a class structure for them will be the best way going forward.

Additionally, I have added the code docs for the new methods.

[Bryce Harrington (bryce)]

Yesterday I ran through the tests and found the functionality seems to be in order. The updates look good.

$ debuild -S -uc -us -i
 dpkg-buildpackage -rfakeroot -d -us -uc -i -I.bzr -I.svn -I.git -S -i
dpkg-buildpackage: source package update-notifier
dpkg-buildpackage: source version 3.168.14
dpkg-buildpackage: source distribution xenial
dpkg-buildpackage: source changed by Lucas Moura <email address hidden>
 dpkg-source -i -I.bzr -I.svn -I.git -i --before-build update-notifier-gu
 fakeroot debian/rules clean
dh clean --with python3
   dh_testdir
   dh_auto_clean
   dh_clean
 dpkg-source -i -I.bzr -I.svn -I.git -i -b update-notifier-gu
dpkg-source: info: using source format '3.0 (native)'
dpkg-source: info: building update-notifier in update-notifier_3.168.14.tar.xz
dpkg-source: info: building update-notifier in update-notifier_3.168.14.dsc
 dpkg-genchanges -S >../update-notifier_3.168.14_source.changes
dpkg-genchanges: including full source code in upload
 dpkg-source -i -I.bzr -I.svn -I.git -i --after-build update-notifier-gu
dpkg-buildpackage: source-only upload: Debian-native package
Now running lintian...
W: update-notifier source: ancient-standards-version 3.9.4 (current is 3.9.7)
Finished running lintian.

$ debsign ../update-notifier_3.168.14_source.changes
 signfile dsc update-notifier_3.168.14.dsc A661100B3DAC1D4F2CAD8A54E603B2578FB8F0FB

 fixup_changes dsc update-notifier_3.168.14.dsc update-notifier_3.168.14_source.changes
 signfile changes update-notifier_3.168.14_source.changes A661100B3DAC1D4F2CAD8A54E603B2578FB8F0FB

Successfully signed dsc, changes files

$ dput ubuntu update-notifier_3.168.14_source.changes
Checking signature on .changes
gpg: /home/bryce/pkg/UpdateNotifier/review-mp401473/update-notifier_3.168.14_source.changes: Valid signature from E603B2578FB8F0FB
Checking signature on .dsc
gpg: /home/bryce/pkg/UpdateNotifier/review-mp401473/update-notifier_3.168.14.dsc: Valid signature from E603B2578FB8F0FB
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading update-notifier_3.168.14.dsc: done.
  Uploading update-notifier_3.168.14.tar.xz: done.
  Uploading update-notifier_3.168.14_source.changes: done.
Successfully uploaded packages.

$ git ubuntu tag --upload
$ git push pkg upload/3.168.14
Counting objects: 13, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (13/13), done.
Writing objects: 100% (13/13), 4.35 KiB | 0 bytes/s, done.
Total 13 (delta 9), reused 0 (delta 0)
To ssh://<email address hidden>/ubuntu/+source/update-notifier
 * [new tag] upload/3.168.14 -> upload/3.168.14

review: Approve

[Bryce Harrington (bryce)]

There are some errors for a few of the dngettext() calls. This routine chooses between two strings based on singular vs. plural, but in a few places the plural form is being used for both. I suggest some edits to correct these inline. I think I caught them all but you should doublecheck.

It would probably be wise to also add test cases for the singular cases of each string.

review: Needs Fixing

[Bryce Harrington (bryce)]

Fixes look good, thanks, and I've verified the tests pass and did a local smoke test.

Tagged and uploaded for xenial:

triage-xenial+16.04:~/pkg/UpdateNotifier/review-mp401473$ dput -f ubuntu update-notifier_3.168.14_source.changes
Checking signature on .changes
gpg: Signature made Tue 27 Apr 2021 12:50:46 AM UTC using RSA key ID 8FB8F0FB
gpg: Good signature from "Bryce Harrington <email address hidden>"
gpg: aka "Bryce Harrington <email address hidden>"
gpg: aka "Bryce Harrington <email address hidden>"
Good signature on /home/bryce/pkg/UpdateNotifier/review-mp401473/update-notifier_3.168.14_source.changes.
Checking signature on .dsc
gpg: Signature made Tue 27 Apr 2021 12:50:46 AM UTC using RSA key ID 8FB8F0FB
gpg: Good signature from "Bryce Harrington <email address hidden>"
gpg: aka "Bryce Harrington <email address hidden>"
gpg: aka "Bryce Harrington <email address hidden>"
Good signature on /home/bryce/pkg/UpdateNotifier/review-mp401473/update-notifier_3.168.14.dsc.
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading update-notifier_3.168.14.dsc: done.
  Uploading update-notifier_3.168.14.tar.xz: done.
  Uploading update-notifier_3.168.14_source.changes: done.
Successfully uploaded packages.

triage-xenial+16.04:~/pkg/UpdateNotifier/review-mp401473/update-notifier-gu$ git ubuntu tag -f --upload
/snap/git-ubuntu/current/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Updated tag 'upload/3.168.14' (was 1f8fcad)
triage-xenial+16.04:~/pkg/UpdateNotifier/review-mp401473/update-notifier-gu$ git push -f pkg upload/3.168.14
Counting objects: 17, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (17/17), done.
Writing objects: 100% (17/17), 2.97 KiB | 0 bytes/s, done.
Total 17 (delta 13), reused 0 (delta 0)
To ssh://<email address hidden>/ubuntu/+source/update-notifier
 + 1f8fcad...7474f3b upload/3.168.14 -> upload/3.168.14 (forced update)

I had to force push since the previous MP upload was already in place. I'm assuming the previous upload was pulled early enough we don't have to re-rev the version numbers; if not we'll know soon enough...

review: Approve

diff --git a/data/apt_check.py b/data/apt_check.py
index 28026a4..2f2bb3d 100755
--- a/data/apt_check.py
+++ b/data/apt_check.py
@@ -10,13 +10,16 @@ import sys
 from optparse import OptionParser
 import gettext
 import subprocess
+import distro_info
 
 SYNAPTIC_PINFILE = "/var/lib/synaptic/preferences"
 DISTRO = subprocess.check_output(
     ["lsb_release", "-c", "-s"],
     universal_newlines=True).strip()
 
-ESM_ORIGINS = ("UbuntuESM", "UbuntuESMApps")
+ESM_INFRA_ORIGIN = "UbuntuESM"
+ESM_APPS_ORIGIN = "UbuntuESMApps"
+ESM_ORIGINS = (ESM_INFRA_ORIGIN, ESM_APPS_ORIGIN)
 
 
 def _(msg):
@@ -47,8 +50,8 @@ def saveDistUpgrade(cache, depcache):
 def isSecurityUpgrade(ver):
     " check if the given version is a security update (or masks one) "
     security_pockets = [("Ubuntu", "%s-security" % DISTRO),
-                        ("UbuntuESM", "%s-infra-security" % DISTRO),
-                        ("UbuntuESMApps", "%s-apps-security" % DISTRO),
+                        (ESM_INFRA_ORIGIN, "%s-infra-security" % DISTRO),
+                        (ESM_APPS_ORIGIN, "%s-apps-security" % DISTRO),
                         ("gNewSense", "%s-security" % DISTRO),
                         ("Debian", "%s-updates" % DISTRO)]
     for (file, index) in ver.file_list:
@@ -58,14 +61,24 @@ def isSecurityUpgrade(ver):
     return False
 
 
-def isESMUpgrade(ver):
+def _isESMUpgrade(ver, esm_origin):
     " check if the given version is a security update (or masks one) "
     for (file, index) in ver.file_list:
-        if file.origin in ESM_ORIGINS and file.archive.startswith(DISTRO):
+        if file.origin == esm_origin and file.archive.startswith(DISTRO):
             return True
     return False
 
 
+def isESMAppsUpgrade(ver):
+    " check if the given version is a ESM Apps upgrade "
+    return _isESMUpgrade(ver, esm_origin=ESM_APPS_ORIGIN)
+
+
+def isESMInfraUpgrade(ver):
+    " check if the given version is a ESM Infra upgrade "
+    return _isESMUpgrade(ver, esm_origin=ESM_INFRA_ORIGIN)
+
+
 def write_package_names(outstream, cache, depcache):
     " write out package names that change to outstream "
     pkgs = [pkg for pkg in cache.packages if depcache.marked_install(pkg)
@@ -73,11 +86,90 @@ def write_package_names(outstream, cache, depcache):
     outstream.write("\n".join([p.name for p in pkgs]))
 
 
+def is_esm_distro():
+    " check if the current distro is ESM or not "
+    ubuntu_distro = distro_info.UbuntuDistroInfo()
+
+    is_esm_supported = bool(
+        DISTRO in ubuntu_distro.supported_esm()
+    )
+
+    is_not_currently_supported = bool(
+        DISTRO in ubuntu_distro.unsupported()
+    )
+
+    return is_esm_supported and is_not_currently_supported
+
+
+def is_lts_distro():
+    " check if the current distro is LTS or not"
+    return distro_info.UbuntuDistroInfo().is_lts(DISTRO)
+
+
+def _output_esm_package_count(outstream, service_type, esm_pkg_count):
+    " output the number of packages upgrades related to esm service "
+    if esm_pkg_count > 0:
+        outstream.write("\n")
+        outstream.write(gettext.dngettext("update-notifier",
+                                          "%d of these updates "
+                                          "is an UA %s: ESM "
+                                          "security update.",
+                                          "%d of these updates "
+                                          "are UA %s: ESM "
+                                          "security updates.",
+                                          esm_pkg_count) %
+                        (esm_pkg_count, service_type))
+
+
+def _output_esm_package_alert(
+    outstream, service_type, disabled_pkg_count
+):
+    " output the number of upgradable packages if esm service was enabled "
+    outstream.write("\n")
+    if disabled_pkg_count > 0:
+        outstream.write("\n")
+        outstream.write(gettext.dngettext("update-notifier",
+                                          "%i additional security "
+                                          "update can be applied "
+                                          "with UA %s: ESM\nLearn "
+                                          "more about enabling UA "
+                                          "%s: ESM service at "
+                                          "https://ubuntu.com/esm",
+                                          "%i additional security "
+                                          "updates can be applied "
+                                          "with UA %s: ESM\nLearn "
+                                          "more about enabling UA "
+                                          "%s: ESM service at "
+                                          "https://ubuntu.com/esm",
+                                          disabled_pkg_count) %
+                        (disabled_pkg_count, service_type, service_type))
+    else:
+        outstream.write("\n")
+        outstream.write(gettext.dgettext("update-notifier",
+                                         "Enable UA %s: ESM to "
+                                         "receive additional future "
+                                         "security updates.") %
+                        service_type)
+
+        outstream.write("\n")
+        outstream.write(
+            gettext.dgettext("update-notifier",
+                             "See https://ubuntu.com/esm "
+                             "or run: sudo ua status")
+        )
+
+
 def write_human_readable_summary(outstream, upgrades, security_updates,
-                                 esm_updates, have_esm, disabled_esm_updates):
+                                 esm_infra_updates, esm_apps_updates,
+                                 have_esm_infra, have_esm_apps,
+                                 disabled_esm_infra_updates,
+                                 disabled_esm_apps_updates):
+
     " write out human summary summary to outstream "
-    if have_esm is not None:
-        if have_esm:
+    esm_distro = is_esm_distro()
+    lts_distro = is_lts_distro()
+    if have_esm_infra is not None and esm_distro:
+        if have_esm_infra:
             outstream.write(gettext.dgettext("update-notifier",
                                              "UA Infra: Extended "
                                              "Security Maintenance (ESM) is "
@@ -87,63 +179,51 @@ def write_human_readable_summary(outstream, upgrades, security_updates,
                                              "UA Infra: Extended "
                                              "Security Maintenance (ESM) is "
                                              "not enabled."))
-        outstream.write("\n\n")
-
-    outstream.write(gettext.dngettext("update-notifier",
-                                      "%i package can be updated.",
-                                      "%i packages can be updated.",
-                                      upgrades) %
-                    upgrades)
-    outstream.write("\n")
-    if esm_updates > 0:
+        if upgrades > 0:
+            outstream.write("\n\n")
+    if upgrades > 0:
+        outstream.write(
+            gettext.dngettext("update-notifier",
+                              "%i update can be applied immediately.",
+                              "%i updates can be applied immediately.",
+                              upgrades) % upgrades
+        )
+
+    _output_esm_package_count(
+        outstream, service_type="Infra", esm_pkg_count=esm_infra_updates)
+    _output_esm_package_count(
+        outstream, service_type="Apps", esm_pkg_count=esm_apps_updates)
+
+    if security_updates > 0:
+        outstream.write("\n")
         outstream.write(gettext.dngettext("update-notifier",
-                                          "%i of these updates is fixed "
-                                          "through UA Infra: ESM.",
+                                          "%i of these updates is a "
+                                          "standard security update.",
                                           "%i of these updates are "
-                                          "fixed through UA "
-                                          "Infra: ESM.",
-                                          esm_updates) %
-                        esm_updates)
-        outstream.write("\n")
-    outstream.write(gettext.dngettext("update-notifier",
-                                      "%i of these updates is a "
-                                      "security update.",
-                                      "%i of these updates are "
-                                      "security updates.",
-                                      security_updates) %
-                    security_updates)
-    if upgrades > 0 or security_updates > 0 or esm_updates > 0:
+                                          "standard security updates.",
+                                          security_updates) %
+                        security_updates)
+
+    if any([upgrades, security_updates, esm_infra_updates, esm_apps_updates]):
         outstream.write("\n")
         outstream.write(gettext.dgettext("update-notifier",
                                          "To see these additional updates "
                                          "run: apt list --upgradable"))
-    if have_esm is not None and not have_esm:
-        outstream.write("\n")
-        if disabled_esm_updates > 0:
-            outstream.write("\n")
-            outstream.write(gettext.dngettext("update-notifier",
-                                              "Enable UA Infra: ESM "
-                                              "to receive %i additional "
-                                              "security update.",
-                                              "Enable UA Infra: ESM "
-                                              "to receive %i additional "
-                                              "security updates.",
-                                              disabled_esm_updates) %
-                            disabled_esm_updates)
-        else:
-            outstream.write("\n")
-            outstream.write(gettext.dgettext("update-notifier",
-                                             "Enable UA Infra: ESM to "
-                                             "receive additional future "
-                                             "security updates."))
-        outstream.write("\n")
-        outstream.write(gettext.dgettext("update-notifier",
-                                         "See https://ubuntu.com/security/esm "
-                                         "or run: sudo ua status"))
+
+    if have_esm_apps is not None and not have_esm_apps and lts_distro:
+        _output_esm_package_alert(
+            outstream, service_type="Apps",
+            disabled_pkg_count=disabled_esm_apps_updates)
+
+    if have_esm_infra is not None and not have_esm_infra and esm_distro:
+        _output_esm_package_alert(
+            outstream, service_type="Infra",
+            disabled_pkg_count=disabled_esm_infra_updates)
+
     outstream.write("\n")
 
 
-def has_disabled_esm_security_update(depcache, pkg):
+def has_disabled_esm_security_update(depcache, pkg, esm_origin):
     " check if we have a disabled ESM security update "
     inst_ver = pkg.current_ver
     if not inst_ver:
@@ -154,12 +234,42 @@ def has_disabled_esm_security_update(depcache, pkg):
             break
 
         for (file, index) in ver.file_list:
-            if (file.origin in ESM_ORIGINS and file.archive.startswith(DISTRO)
+            if (file.origin == esm_origin and file.archive.startswith(DISTRO)
                     and depcache.policy.get_priority(file) == -32768):
                 return True
     return False
 
 
+def has_disabled_esm_apps_security_update(depcache, pkg):
+    " check if we have a disabled ESM Apps security update "
+    return has_disabled_esm_security_update(depcache, pkg, ESM_APPS_ORIGIN)
+
+
+def has_disabled_esm_infra_security_update(depcache, pkg):
+    " check if we have a disabled ESM Infra security update "
+    return has_disabled_esm_security_update(depcache, pkg, ESM_INFRA_ORIGIN)
+
+
+def has_esm_service(cache, depcache, esm_origin):
+    " check if we have an enabled ESM service in the machine "
+    have_esm = None
+    for file in cache.file_list:
+        origin = file.origin
+        if origin == esm_origin and file.archive.startswith(DISTRO):
+            # In case of multiple ESM repos, one enabled is sufficient.
+            if depcache.policy.get_priority(file) == -32768:
+                # We found a disabled ESM repository, but we'll only count
+                # ESM as disabled here if we have not found any other ESM
+                # repo, so one ESM repo being enabled means ESM is enabled.
+                if have_esm is None:
+                    have_esm = False
+            else:
+                have_esm = True
+                break
+
+    return have_esm
+
+
 def init():
     " init the system, be nice "
     # FIXME: do a ionice here too?
@@ -202,31 +312,26 @@ def run(options=None):
 
     # Check if we have ESM enabled or disabled; and if it exists in the
     # first place.
-    have_esm = None        # None == does not exist
-    for file in cache.file_list:
-        if file.origin in ESM_ORIGINS and file.archive.startswith(DISTRO):
-            # In case of multiple ESM repos, one enabled is sufficient.
-            if depcache.policy.get_priority(file) == -32768:
-                # We found a disabled ESM repository, but we'll only count
-                # ESM as disabled here if we have not found any other ESM
-                # repo, so one ESM repo being enabled means ESM is enabled.
-                if have_esm is None:
-                    have_esm = False
-            else:
-                have_esm = True
-                break
+    have_esm_infra = has_esm_service(
+        cache, depcache, esm_origin=ESM_INFRA_ORIGIN)
+    have_esm_apps = has_esm_service(
+        cache, depcache, esm_origin=ESM_APPS_ORIGIN)
 
     # analyze the ugprade
     upgrades = 0
     security_updates = 0
-    esm_updates = 0
-    disabled_esm_updates = 0
+    esm_apps_updates = 0
+    esm_infra_updates = 0
+    disabled_esm_apps_updates = 0
+    disabled_esm_infra_updates = 0
 
     # we need another cache that has more pkg details
     with apt.Cache() as aptcache:
         for pkg in cache.packages:
-            if has_disabled_esm_security_update(depcache, pkg):
-                disabled_esm_updates += 1
+            if has_disabled_esm_apps_security_update(depcache, pkg):
+                disabled_esm_apps_updates += 1
+            if has_disabled_esm_infra_security_update(depcache, pkg):
+                disabled_esm_infra_updates += 1
 
             # skip packages that are not marked upgraded/installed
             if not (depcache.marked_install(pkg)
@@ -240,10 +345,14 @@ def run(options=None):
                 continue
             # check for security upgrades
             if isSecurityUpgrade(cand_ver):
-                if isESMUpgrade(cand_ver):
-                    esm_updates += 1
+                if have_esm_apps and isESMAppsUpgrade(cand_ver):
+                    esm_apps_updates += 1
+                elif have_esm_infra and isESMInfraUpgrade(cand_ver):
+                    esm_infra_updates += 1
+                else:
+                    security_updates += 1
+
                 upgrades += 1
-                security_updates += 1
                 continue
 
             # check to see if the update is a phased one
@@ -267,9 +376,11 @@ def run(options=None):
                         and apt_pkg.version_compare(ver.ver_str,
                                                     inst_ver.ver_str) <= 0):
                     continue
-                if isESMUpgrade(ver):
-                    esm_updates += 1
-                if isSecurityUpgrade(ver):
+                if have_esm_apps and isESMAppsUpgrade(cand_ver):
+                    esm_apps_updates += 1
+                elif have_esm_infra and isESMInfraUpgrade(cand_ver):
+                    esm_infra_updates += 1
+                elif isSecurityUpgrade(ver):
                     security_updates += 1
                     break
 
@@ -278,8 +389,10 @@ def run(options=None):
         write_package_names(sys.stderr, cache, depcache)
     elif options and options.readable_output:
         write_human_readable_summary(sys.stdout, upgrades, security_updates,
-                                     esm_updates, have_esm,
-                                     disabled_esm_updates)
+                                     esm_infra_updates, esm_apps_updates,
+                                     have_esm_infra, have_esm_apps,
+                                     disabled_esm_infra_updates,
+                                     disabled_esm_apps_updates)
     else:
         # print the number of regular upgrades and the number of
         # security upgrades
diff --git a/debian/changelog b/debian/changelog
index 197b459..333a5b4 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,18 @@
+update-notifier (3.168.14) xenial; urgency=medium
+
+  * data/apt_check.py:
+    - Add support to handle packages from ESM Apps in addition to ESM Infra
+      and only display alerts if the distro is ESM. (LP: #1924766)
+    - Do not display a count of ESM packages if the system does not have ESM
+      enabled. (LP: #1883315)
+    - Make distinction betweem standard security updates and ESM updates
+      when performing package counts. (LP: #1926208)
+    - use 'applied' instead of 'installed', redact 0 of these updates are
+      security updates, and correct singular messages
+  * debian/control: Add a dependency on python3-distro-info.
+
+ -- Lucas Moura <[e-mail]>  Tue, 20 Apr 2021 10:20:21 -0300
+
 update-notifier (3.168.13) xenial; urgency=medium
 
   * Fix pep8 autopkgtest failures in the right control file (LP: #1906436)
diff --git a/debian/control b/debian/control
index a3b383a..1966167 100644
--- a/debian/control
+++ b/debian/control
@@ -15,6 +15,7 @@ Build-Depends: debhelper (>= 7.0.50~),
                python3,
                python3-apt,
                python3-debian,
+               python3-distro-info,
                python3-mock
 Standards-Version: 3.9.4
 Vcs-Bzr: http://bazaar.launchpad.net/~ubuntu-core-dev/update-notifier/ubuntu
@@ -48,6 +49,7 @@ Depends: ${shlibs:Depends},
          ${misc:Depends},
          python3:any,
          python3-apt, python3-dbus, python3-debian, debconf,
+         python3-distro-info,
          patch, update-manager-core (>= 1:16.04.6)
 Recommends: libpam-modules (>= 1.0.1-9ubuntu3)
 Suggests: policykit-1
diff --git a/tests/test_motd.py b/tests/test_motd.py
index a876937..335adce 100755
--- a/tests/test_motd.py
+++ b/tests/test_motd.py
@@ -3,11 +3,11 @@
 
 import apt_check
 import io
-import os
-import subprocess
 import unittest
 import textwrap
 
+from unittest import mock
+
 
 def get_message(*args, **kwds):
     with io.StringIO() as stream:
@@ -16,134 +16,388 @@ def get_message(*args, **kwds):
 
 
 class TestMotd(unittest.TestCase):
-    """ ensure that the tree is pep8 clean """
+    """ Validate /etc/motd text """
 
-    def test_esm_disabled_upto_date_esm_avail(self):
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_disabled_upto_date_esm_avail(
+        self, _m_esm_distro, _m_is_lts
+    ):
         self.assertEqual(
             get_message(upgrades=0, security_updates=0,
-                        esm_updates=0, have_esm=False,
-                        disabled_esm_updates=23),
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=False, have_esm_apps=False,
+                        disabled_esm_infra_updates=1,
+                        disabled_esm_apps_updates=0),
             textwrap.dedent(
-                """
+                """\
                 UA Infra: Extended Security Maintenance (ESM) is not enabled.
 
-                0 updates can be installed immediately.
-                0 of these updates are security updates.
+                Enable UA Apps: ESM to receive additional future security updates.
+                See https://ubuntu.com/esm or run: sudo ua status
 
-                Enable UA Infra: ESM to receive 23 additional security updates.
-                See https://ubuntu.com/security/esm or run: sudo ua status
-                """).lstrip())
+                1 additional security update can be applied with UA Infra: ESM
+                Learn more about enabling UA Infra: ESM service at https://ubuntu.com/esm
+                """))
 
-    def test_esm_disabled_security_esm_avail(self):
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_disabled_security_esm_avail(
+        self, _m_esm_distro, _m_is_lts
+    ):
         self.assertEqual(
-            get_message(upgrades=15, security_updates=7,
-                        esm_updates=0, have_esm=False,
-                        disabled_esm_updates=23),
+            get_message(upgrades=15, security_updates=1,
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=False, have_esm_apps=False,
+                        disabled_esm_infra_updates=23,
+                        disabled_esm_apps_updates=0),
             textwrap.dedent(
-                """
+                """\
                 UA Infra: Extended Security Maintenance (ESM) is not enabled.
 
-                15 updates can be installed immediately.
-                7 of these updates are security updates.
+                15 updates can be applied immediately.
+                1 of these updates is a standard security update.
                 To see these additional updates run: apt list --upgradable
 
-                Enable UA Infra: ESM to receive 23 additional security updates.
-                See https://ubuntu.com/security/esm or run: sudo ua status
-                """).lstrip())
+                Enable UA Apps: ESM to receive additional future security updates.
+                See https://ubuntu.com/esm or run: sudo ua status
+
+                23 additional security updates can be applied with UA Infra: ESM
+                Learn more about enabling UA Infra: ESM service at https://ubuntu.com/esm
+                """))
 
-    def test_esm_disabled_security_no_esm_avail(self):
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_disabled_security_no_esm_avail(
+        self, _m_esm_distro, _m_is_lts
+    ):
         self.assertEqual(
             get_message(upgrades=15, security_updates=7,
-                        esm_updates=0, have_esm=False,
-                        disabled_esm_updates=0),
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=False, have_esm_apps=False,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=0),
             textwrap.dedent(
-                """
+                """\
                 UA Infra: Extended Security Maintenance (ESM) is not enabled.
 
-                15 updates can be installed immediately.
-                7 of these updates are security updates.
+                15 updates can be applied immediately.
+                7 of these updates are standard security updates.
                 To see these additional updates run: apt list --upgradable
 
+                Enable UA Apps: ESM to receive additional future security updates.
+                See https://ubuntu.com/esm or run: sudo ua status
+
                 Enable UA Infra: ESM to receive additional future security updates.
-                See https://ubuntu.com/security/esm or run: sudo ua status
-                """).lstrip())
+                See https://ubuntu.com/esm or run: sudo ua status
+                """))
 
-    def test_esm_disabled_nosecurity(self):
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_disabled_nosecurity(
+        self, _m_esm_distro, _m_is_lts
+    ):
         self.assertEqual(
             get_message(upgrades=15, security_updates=0,
-                        esm_updates=0, have_esm=False,
-                        disabled_esm_updates=0),
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=False, have_esm_apps=False,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=0),
             textwrap.dedent(
-                """
+                """\
                 UA Infra: Extended Security Maintenance (ESM) is not enabled.
 
-                15 updates can be installed immediately.
-                0 of these updates are security updates.
+                15 updates can be applied immediately.
                 To see these additional updates run: apt list --upgradable
 
+                Enable UA Apps: ESM to receive additional future security updates.
+                See https://ubuntu.com/esm or run: sudo ua status
+
                 Enable UA Infra: ESM to receive additional future security updates.
-                See https://ubuntu.com/security/esm or run: sudo ua status
-                """).lstrip())
+                See https://ubuntu.com/esm or run: sudo ua status
+                """))
 
-    def test_esm_disabled_noupdates(self):
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_disabled_noupdates(
+        self, _m_esm_distro, _m_is_lts
+    ):
         self.assertEqual(
             get_message(upgrades=0, security_updates=0,
-                        esm_updates=0, have_esm=False,
-                        disabled_esm_updates=0),
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=False, have_esm_apps=False,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=0),
             textwrap.dedent(
-                """
+                """\
                 UA Infra: Extended Security Maintenance (ESM) is not enabled.
 
-                0 updates can be installed immediately.
-                0 of these updates are security updates.
+                Enable UA Apps: ESM to receive additional future security updates.
+                See https://ubuntu.com/esm or run: sudo ua status
 
                 Enable UA Infra: ESM to receive additional future security updates.
-                See https://ubuntu.com/security/esm or run: sudo ua status
-                """).lstrip())
+                See https://ubuntu.com/esm or run: sudo ua status
+                """))
 
-    def test_esm_enabled_nosecurity(self):
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_enabled_nosecurity(
+        self, _m_esm_distro, _m_is_lts
+    ):
         self.assertEqual(
             get_message(upgrades=35, security_updates=0,
-                        esm_updates=13, have_esm=True,
-                        disabled_esm_updates=0),
+                        esm_infra_updates=13, esm_apps_updates=0,
+                        have_esm_infra=True, have_esm_apps=False,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=0),
             textwrap.dedent(
-                """
+                """\
                 UA Infra: Extended Security Maintenance (ESM) is enabled.
 
-                35 updates can be installed immediately.
-                13 of these updates are fixed through UA Infra: ESM.
-                0 of these updates are security updates.
+                35 updates can be applied immediately.
+                13 of these updates are UA Infra: ESM security updates.
                 To see these additional updates run: apt list --upgradable
-                """).lstrip())
 
-    def test_esm_enabled_somesecurity(self):
+                Enable UA Apps: ESM to receive additional future security updates.
+                See https://ubuntu.com/esm or run: sudo ua status
+                """))
+
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_enabled_somesecurity(
+        self, _m_esm_distro, _m_is_lts
+    ):
         self.assertEqual(
             get_message(upgrades=47, security_updates=7,
-                        esm_updates=13, have_esm=True,
-                        disabled_esm_updates=0),
+                        esm_infra_updates=13, esm_apps_updates=0,
+                        have_esm_infra=True, have_esm_apps=False,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=0),
             textwrap.dedent(
-                """
+                """\
                 UA Infra: Extended Security Maintenance (ESM) is enabled.
 
-                47 updates can be installed immediately.
-                13 of these updates are fixed through UA Infra: ESM.
-                7 of these updates are security updates.
+                47 updates can be applied immediately.
+                13 of these updates are UA Infra: ESM security updates.
+                7 of these updates are standard security updates.
                 To see these additional updates run: apt list --upgradable
-                """).lstrip())
 
-    def test_esm_enabled_noupdates(self):
+                Enable UA Apps: ESM to receive additional future security updates.
+                See https://ubuntu.com/esm or run: sudo ua status
+                """))
+
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_enabled_noupdates(
+        self, _m_esm_distro, _m_is_lts
+    ):
         self.assertEqual(
             get_message(upgrades=0, security_updates=0,
-                        esm_updates=0, have_esm=True,
-                        disabled_esm_updates=0),
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=True, have_esm_apps=False,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=10),
+            textwrap.dedent(
+                """\
+                UA Infra: Extended Security Maintenance (ESM) is enabled.
+
+                10 additional security updates can be applied with UA Apps: ESM
+                Learn more about enabling UA Apps: ESM service at https://ubuntu.com/esm
+                """))
+
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_and_esm_apps_enabled(
+        self, _m_esm_distro, _m_is_lts
+    ):
+        self.assertEqual(
+            get_message(upgrades=30, security_updates=0,
+                        esm_infra_updates=15, esm_apps_updates=15,
+                        have_esm_infra=True, have_esm_apps=True,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=0),
             textwrap.dedent(
                 """
                 UA Infra: Extended Security Maintenance (ESM) is enabled.
 
-                0 updates can be installed immediately.
-                0 of these updates are security updates.
+                30 updates can be applied immediately.
+                15 of these updates are UA Infra: ESM security updates.
+                15 of these updates are UA Apps: ESM security updates.
+                To see these additional updates run: apt list --upgradable
                 """).lstrip())
 
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_disable_and_esm_apps_enabled(
+        self, _m_esm_distro, _m_is_lts
+    ):
+        self.assertEqual(
+            get_message(upgrades=30, security_updates=12,
+                        esm_infra_updates=0, esm_apps_updates=15,
+                        have_esm_infra=False, have_esm_apps=True,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=0),
+            textwrap.dedent(
+                """
+                UA Infra: Extended Security Maintenance (ESM) is not enabled.
+
+                30 updates can be applied immediately.
+                15 of these updates are UA Apps: ESM security updates.
+                12 of these updates are standard security updates.
+                To see these additional updates run: apt list --upgradable
+
+                Enable UA Infra: ESM to receive additional future security updates.
+                See https://ubuntu.com/esm or run: sudo ua status
+                """).lstrip())
+
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_disabled_wih_pkgs_and_esm_apps_enabled(
+        self, _m_esm_distro, _m_is_lts
+    ):
+        self.assertEqual(
+            get_message(upgrades=30, security_updates=15,
+                        esm_infra_updates=0, esm_apps_updates=1,
+                        have_esm_infra=False, have_esm_apps=True,
+                        disabled_esm_infra_updates=40,
+                        disabled_esm_apps_updates=0),
+            textwrap.dedent(
+                """
+                UA Infra: Extended Security Maintenance (ESM) is not enabled.
+
+                30 updates can be applied immediately.
+                1 of these updates is an UA Apps: ESM security update.
+                15 of these updates are standard security updates.
+                To see these additional updates run: apt list --upgradable
+
+                40 additional security updates can be applied with UA Infra: ESM
+                Learn more about enabling UA Infra: ESM service at https://ubuntu.com/esm
+                """).lstrip())
+
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_esm_infra_disabled_and_esm_apps_disabled_with_pkgs(
+        self, _m_esm_distro, _m_is_lts
+    ):
+        self.assertEqual(
+            get_message(upgrades=30, security_updates=18,
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=False, have_esm_apps=False,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=40),
+            textwrap.dedent(
+                """\
+                UA Infra: Extended Security Maintenance (ESM) is not enabled.
+
+                30 updates can be applied immediately.
+                18 of these updates are standard security updates.
+                To see these additional updates run: apt list --upgradable
+
+                40 additional security updates can be applied with UA Apps: ESM
+                Learn more about enabling UA Apps: ESM service at https://ubuntu.com/esm
+
+                Enable UA Infra: ESM to receive additional future security updates.
+                See https://ubuntu.com/esm or run: sudo ua status
+                """))
+
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=True)
+    def test_no_esm_infra_and_apps_index_in_esm_distro(
+        self, _m_esm_distro, _m_is_lts
+    ):
+        self.assertEqual(
+            get_message(upgrades=30, security_updates=18,
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=None, have_esm_apps=None,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=0),
+            textwrap.dedent(
+                """\
+                30 updates can be applied immediately.
+                18 of these updates are standard security updates.
+                To see these additional updates run: apt list --upgradable
+                """))
+
+    @mock.patch("apt_check.is_lts_distro", return_value=False)
+    @mock.patch("apt_check.is_esm_distro", return_value=False)
+    def test_no_esm_infra_and_apps_index_in_non_lts_distro(
+        self, _m_esm_distro, _m_is_lts
+    ):
+        self.assertEqual(
+            get_message(upgrades=30, security_updates=18,
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=None, have_esm_apps=None,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=0),
+            textwrap.dedent(
+                """\
+                30 updates can be applied immediately.
+                18 of these updates are standard security updates.
+                To see these additional updates run: apt list --upgradable
+                """))
+
+    @mock.patch("apt_check.is_lts_distro", return_value=False)
+    @mock.patch("apt_check.is_esm_distro", return_value=False)
+    def test_esm_infra_disabled_and_esm_apps_disabled_with_no_esm_distro(
+        self, _m_esm_distro, _m_is_lts,
+    ):
+        self.assertEqual(
+            get_message(upgrades=30, security_updates=18,
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=False, have_esm_apps=False,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=40),
+            textwrap.dedent(
+                """\
+                30 updates can be applied immediately.
+                18 of these updates are standard security updates.
+                To see these additional updates run: apt list --upgradable
+                """))
+
+    @mock.patch("apt_check.is_lts_distro", return_value=False)
+    @mock.patch("subprocess.Popen")
+    def test_message_for_distro_that_will_not_go_into_esm_mode(
+        self, m_popen, _m_is_lts
+    ):
+        comm_mock = mock.MagicMock()
+        comm_mock.communicate.return_value = (
+            "(unknown)".encode("utf-8"), "")
+        m_popen.return_value = comm_mock
+
+        self.assertEqual(
+            get_message(upgrades=30, security_updates=18,
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=False, have_esm_apps=False,
+                        disabled_esm_infra_updates=0,
+                        disabled_esm_apps_updates=40),
+            textwrap.dedent(
+                """\
+                30 updates can be applied immediately.
+                18 of these updates are standard security updates.
+                To see these additional updates run: apt list --upgradable
+                """))
+
+    @mock.patch("apt_check.is_lts_distro", return_value=True)
+    @mock.patch("apt_check.is_esm_distro", return_value=False)
+    def test_message_for_lts_distro_not_in_esm_mode_yet(
+        self, _m_is_esm, _m_is_lts
+    ):
+        self.assertEqual(
+            get_message(upgrades=30, security_updates=18,
+                        esm_infra_updates=0, esm_apps_updates=0,
+                        have_esm_infra=False, have_esm_apps=False,
+                        disabled_esm_infra_updates=10,
+                        disabled_esm_apps_updates=40),
+            textwrap.dedent(
+                """\
+                30 updates can be applied immediately.
+                18 of these updates are standard security updates.
+                To see these additional updates run: apt list --upgradable
+
+                40 additional security updates can be applied with UA Apps: ESM
+                Learn more about enabling UA Apps: ESM service at https://ubuntu.com/esm
+                """))
+
 
 if __name__ == "__main__":
     import logging