Merge ~chad.smith/software-properties:ua-status-from-json into software-properties:ubuntu/master

[Chad Smith]

utils: prefer ua status from status.json. Support schema 0.1 format

Adjust gtk.utils.get_ua_status to prefer reading
/var/lib/ubuntu-advantage/status.json instead of invoking
ua status on the commandline due to a network roundtrip that
is performed while running the command.

This status.json file will exist on all machines attached to an
Ubuntu Advantage subscription.

Unattached machines will persist status.json due to a systemd timer
that will sync current unattached or attached status to
/var/lib/ubuntu-advantage/status.json.

Allow get_ua_status will now also check a _schema_version key from
ua status which will log if the schema version has changed from the
expected version "0.1".

Changes in schema version may imply incompatibility with reading
UA status.

bumped changelog version here. not sure if it's needed or not

[Chad Smith (chad.smith)]

Testing performed
   1. launched desktop install of Jammy daily desktop images
   2. confirmed failure path from LP: #1939732

jammy-desktop:~$ dpkg-query --show software-properties-gtk
software-properties-gtk 0.99.13
jammy-desktop:~$ software-properties-gtk
No ua status file written:
[Errno 2] No such file or directory: '/var/lib/ubuntu-advantage/status.json'
No ua status file written:
[Errno 2] No such file or directory: '/var/lib/ubuntu-advantage/status.json'
No ua status file written:
[Errno 2] No such file or directory: '/var/lib/ubuntu-advantage/status.json'

jammy-desktop:~$ sudo cp utils.py /usr/lib/python3/dist-packages/softwareproperties/gtk/utils.py

jammy-desktop:~$ dpkg-query --show software-properties-gtk

# no error on gtk dialog load on absent /var/lib/ubuntu-advantage/status.json
jammy-desktop:~$ [ -f /var/lib/ubuntu-advantage/status.json ] && echo "status.json exists"
jammy-desktop:~$ software-properties-gtk
# click "Updates" table in dialog and see "Basic Security Maintenance"

# Attach to a subscription
jammy-desktop:~$ ua attach <mytoken>

jammy-desktop:~$ software-properties-gtk
# no errors
# click "Updates" table in dialog and see "Extended Security Maintenance" with an expiry from my contract: 12/31/1999
# no errors printed to console

# editing _schema_version to print informational messages on CLI
jammy-desktop:~$ sudo sed -i 's/"_schema_version": "0.1"/"_schema_version": "0.3"/' /var/lib/ubuntu-advantage/status.json
jammy-desktop:~$ software-properties-gtk
UA status schema version change: 0.3

[Chad Smith (chad.smith)]

pushed upload to https://launchpad.net/~chad.smith/+archive/ubuntu/software-properties for testing

[Robert Ancell (robert-ancell)]

Change LGTM, thanks @chad.smith!

review: Approve

diff --git a/debian/changelog b/debian/changelog
index 847c543..ccadab3 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,12 @@
+software-properties (0.99.15) jammy; urgency=medium
+
+  * utils: prefer /var/lib/ubuntu-advantage/status.json over ua status
+    - Handle absent /var/lib/ubuntu-advantage/status.json for non-root
+      users (LP: #1939732)
+    - print unexcepted errors and if _schema_version not equal to 0.1
+
+ -- Chad Smith <[e-mail]>  Fri, 29 Oct 2021 15:46:58 -0600
+
 software-properties (0.99.14) jammy; urgency=medium
 
   * cloudarchive: Enable support for the Yoga Ubuntu Cloud Archive on
diff --git a/softwareproperties/gtk/utils.py b/softwareproperties/gtk/utils.py
index d478671..522faa0 100644
--- a/softwareproperties/gtk/utils.py
+++ b/softwareproperties/gtk/utils.py
@@ -26,6 +26,7 @@ import gi
 gi.require_version("Gtk", "3.0")
 from gi.repository import Gio, Gtk
 import json
+import os
 import subprocess
 
 import logging
@@ -73,37 +74,52 @@ def current_distro():
         if release.series == distro.codename:
             return release
 
+
 def get_ua_status():
     """Return a dict of all UA status information or empty dict on error."""
-    # status.json will exist on any attached system or any unattached system
-    # which has already run `ua status`. Calling ua status directly on
-    # network disconnected machines will raise a TimeoutException trying to
-    # access contracts.canonical.com/v1/resources.
-    try:
-        # Success writes UA_STATUS_JSON
-        result = subprocess.run(['ua', 'status', '--format=json'], capture_output=True)
-    except Exception as e:
-        print("Failed to call ubuntu advantage client:\n%s" % e)
-        return {}
-    if result.returncode != 0:
-        print("Ubuntu advantage client returned code %d" % result.returncode)
+    # status.json will exist on any attached system. It will also be created
+    # by the systemd timer ua-timer which will update UA_STATUS_JSON every 12
+    # hours to reflect current status of UA subscription services.
+    # Invoking `ua status` with subp will result in a network call to
+    # contracts.canonical.com which could raise Timeouts on network limited
+    # machines. So, prefer the status.json file when possible.
+    status_json = ""
+    if os.path.exists(UA_STATUS_JSON):
+        with open(UA_STATUS_JSON) as stream:
+            status_json = stream.read()
+    else:
+        try:
+            # Success writes UA_STATUS_JSON
+            result = subprocess.run(
+                ['ua', 'status', '--format=json'], stdout=subprocess.PIPE
+            )
+        except Exception as e:
+            print("Failed to run `ua status`:\n%s" % e)
+            return {}
+        if result.returncode != 0:
+            print(
+                "Ubuntu Advantage client returned code %d" % result.returncode
+            )
+            return {}
+        status_json = result.stdout
+    if not status_json:
+        print(
+            "Warning: no Ubuntu Advantage status found."
+            " Is ubuntu-advantage-tools installed?"
+        )
         return {}
-
-    try:
-        status_file = open(UA_STATUS_JSON, "r")
-    except Exception as e:
-        print("No ua status file written:\n%s" % e)
-        return {}
-
-    with status_file as stream:
-        status_json = stream.read()
     try:
         status = json.loads(status_json)
     except json.JSONDecodeError as e:
         print("Failed to parse ubuntu advantage client JSON:\n%s" % e)
         return {}
+    if status.get("_schema_version", "0.1") != "0.1":
+        print(
+            "UA status schema version change: %s" % status["_schema_version"]
+        )
     return status
 
+
 def get_ua_service_status(service_name='esm-infra', status=None):
     """Get service availability and status for a specific UA service.
 
@@ -112,26 +128,27 @@ def get_ua_service_status(service_name='esm-infra', status=None):
         - attached contract is entitled to the service
         - unattached machine reports service "availability" as "yes"
       :str service_status: will be one of the following:
-        - "disabled" when the service is available but not active
+        - "disabled" when the service is available and applicable but not
+          active
         - "enabled" when the service is available and active
-        - "n/a" when the service is not applicable on the environment or not
+        - "n/a" when the service is not applicable to the environment or not
           entitled for the attached contract
     """
     if not status:
         status = get_ua_status()
+    # Assume unattached on empty status dict
     available = False
     service_status = "n/a"
     for service in status.get("services", []):
-        if service.get("name") == service_name:
-            if service.get("available"):  # then we are not attached
-                if service["available"] == "yes":
-                     available = True
-                     service_status = "disabled"  # Disabled since unattached
-            else:  # attached
-                available = service.get("entitled") == "yes"
-                service_status = service.get("status")  # Will be enabled, disabled or n/a
+        if service.get("name") != service_name:
+            continue
+        if "available" in service:
+            available = bool("yes" == service["available"])
+        if "status" in service:
+            service_status = service["status"]  # enabled, disabled or n/a
     return (available, service_status)
 
+
 def retry(exceptions, tries=10, delay=0.1, backoff=2):
     """
     Retry calling the decorated function using an exponential backoff.