Merge ~pieq/bugit/+git/qabro:1904847-new-sku-and-cid-cli-arguments into bugit:master

[Pierre Equoy]

Description
===========

Allow QA Engineer to use the `--cid` and `--sku` (or -c and -k, respectively) arguments when invoking qabro to fill the CID and the SKU information.

If a CID is added this way, it is also added as a tag as per cwayne's suggestion [1].

This branch also fixes some stuff needed to be able to build qabro again (sosreport was updated to v4.1 in the repositories, and it broke patches and snapcraft recipe).

How it was tested
=================

Tested as an amd64 snap, running qabro with different arguments:

qabro
-> no CID, no SKU added in the bug description

qabro -c 202012-25387 -k MDN-DVT2-C1 -t "tag1 tag2"
-> CID added in the bug description as well as in the tags list (see below)
-> SKU added in the bug description
-> tags added: tag1, tag2, 202012-25387

[1] https://chat.canonical.com/canonical/pl/ee316bp4p7ndjcjbm6f7byapzc

[Sylvain Pineau (sylvain-pineau)]

LGTM (good idea to also add them to the bug desc)

review: Approve

[StanleyHuang (stanley31)]

Looks good to me.

review: Approve

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8ad30f7..c6d804b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,16 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ## [Unreleased]
+### Added
+- Provide argument to add SKU and CID from command line using `--sku` and `--cid`, respectively. If a CID is provided, it is also added as a tag ([LP #1904847](https://bugs.launchpad.net/qabro/+bug/1904847))
+
+### Changed
+- Get buildstamp (image information) from different locations to match IoT and PC projects for both Ubuntu classic and core
+- Updated snap recipe and patches to work with new version of sosreport (v4.1)
+
+### Fixed
+- Attach acpidump output only if process call OK (to avoid problem on ARM devices with no ACPI tables) ([LP #1903229](https://bugs.launchpad.net/qabro/+bug/1903229))
+- Ignore lspci-related errors when getting standard info ([LP #1910193](https://bugs.launchpad.net/qabro/+bug/1910193))
 
 ## [0.14] - 2020-09-28
 ### Added
@@ -118,4 +128,4 @@ Initial version.
 [0.4]: https://git.launchpad.net/qabro/log/?qt=range&q=v0.3...v0.4
 [0.3]: https://git.launchpad.net/qabro/log/?qt=range&q=v0.2...v0.3
 [0.2]: https://git.launchpad.net/qabro/log/?qt=range&q=v0.1...v0.2
-[0.1]: https://git.launchpad.net/qabro/tag/?h=v0.1
\ No newline at end of file
+[0.1]: https://git.launchpad.net/qabro/tag/?h=v0.1
diff --git a/patches/sosreport-apt-remove-user-pwd.patch b/patches/sosreport-apt-remove-user-pwd.patch
index b8fddec..524e4cb 100644
--- a/patches/sosreport-apt-remove-user-pwd.patch
+++ b/patches/sosreport-apt-remove-user-pwd.patch
@@ -3,7 +3,7 @@
 @@ -9,6 +9,7 @@
  # See the LICENSE file in the source distribution for further information.
  
- from sos.plugins import Plugin, UbuntuPlugin, DebianPlugin
+ from sos.report.plugins import Plugin, UbuntuPlugin, DebianPlugin
 +from glob import glob
  
  
diff --git a/qabro/__init__.py b/qabro/__init__.py
index 5604dcb..7169068 100644
--- a/qabro/__init__.py
+++ b/qabro/__init__.py
@@ -51,6 +51,8 @@ def main():
     parser.add_argument("-s", "--series", help="Series e.g. focal")
     parser.add_argument("-t", "--tags",
                         help="Tags (such as Platform tag) separated by spaces")
+    parser.add_argument("-k", "--sku", help="SKU of the device under test")
+    parser.add_argument("-c", "--cid", help="Canonical ID (CID) of the device under test")
     args = parser.parse_args()
     if args.fwts:
         if not args.project:
@@ -62,14 +64,20 @@ def main():
         start_ui(assignee=args.assignee,
                  project=args.project,
                  series=args.series,
-                 tags=args.tags)
+                 tags=args.tags,
+                 sku=args.sku,
+                 cid=args.cid)
 
 
-def start_ui(assignee='', project='', series='', tags=''):
+def start_ui(assignee='', project='', series='', tags='', sku='', cid=''):
+    tags += " {}".format(cid)
+    tags = " ".join(tags.split()) # Remove unwanted spaces
     ui = ReportScreen(assignee=assignee.lower(),
                       project=project.lower(),
                       series=series.lower(),
-                      tags=tags.lower())
+                      tags=tags.lower(),
+                      sku=sku.strip().upper(),
+                      cid=cid)
     ui.run()
     ba = BugAssistant(ui.report())
     while True:
diff --git a/qabro/bug_assistant.py b/qabro/bug_assistant.py
index 75ea219..bfa58b1 100644
--- a/qabro/bug_assistant.py
+++ b/qabro/bug_assistant.py
@@ -266,39 +266,9 @@ class AttachmentAssistant:
                               snap.props.confinement.value_nick)
         self.attachments["snap_list.log"] = out.encode()
 
-    def _get_last_checkbox_session_path(self, usr_home="/home/$USER", root_home="/root", root_dir="/"):
-        """Return path of the most recent Checkbox session directory."""
-        rootpath = ''
-        latest_session_dir = ''
-        generic_path = os.path.join(root_dir, "var/tmp/checkbox-ng/sessions")
-        classic_path = os.path.expandvars(os.path.join(usr_home, ".cache/plainbox/sessions"))
-        if os.path.exists(generic_path):  # for any checkbox installs based on checkbox-ng >= 1.10
-            rootpath = generic_path
-        elif os.path.exists(classic_path):  # for classic checkbox installs based on checkbox-ng <= 1.9
-            rootpath = classic_path
-        else:  # for snap checkbox installs based on checkbox-ng <= 1.9
-            snaps = qabro.utils.get_snaps()
-            snap_list = [e.props.name for e in snaps if 'checkbox-' in e.props.name]
-            for snap_name in snap_list:
-                sp = [os.path.join(usr_home, "snap/{}/current/.cache/plainbox/sessions"),
-                      os.path.join(root_home, "snap/{}/current/.cache/plainbox/sessions")]
-                for p in sp:
-                    snap_path = os.path.expandvars(p.format(snap_name))
-                    if os.path.exists(snap_path):
-                        rootpath = snap_path
-        # Get path of the most recent Checkbox session
-        if rootpath:
-            sessions_dirs = [os.path.join(rootpath, f) for f in os.listdir(rootpath)
-                             if os.path.isdir(os.path.join(rootpath, f))]
-            if sessions_dirs:
-                latest_session_dir = max(sessions_dirs, key=lambda x: os.stat(x).st_mtime)
-        else:
-            print('There does not seem to be any Checkbox session here...')
-        return latest_session_dir
-
     def attach_last_checkbox_session(self):
         """Archive and attach the most recent Checkbox session available."""
-        latest_session_dir = self._get_last_checkbox_session_path()
+        latest_session_dir = qabro.utils.get_last_checkbox_session_path()
         if latest_session_dir:
             session_tgz = os.path.expandvars('$HOME/checkbox-session.tgz')
             if os.path.exists(session_tgz):
@@ -315,7 +285,7 @@ class AttachmentAssistant:
         """
         p = os.getenv("PATH")
         lp = os.getenv("LD_LIBRARY_PATH")
-        latest_session_dir = self._get_last_checkbox_session_path()
+        latest_session_dir = qabro.utils.get_last_checkbox_session_path()
         if p and lp:
             if latest_session_dir:
                 logpath = os.path.join(latest_session_dir, "CHECKBOX_DATA",
diff --git a/qabro/ui.py b/qabro/ui.py
index f83c516..436eb68 100644
--- a/qabro/ui.py
+++ b/qabro/ui.py
@@ -44,11 +44,13 @@ class ReportScreen:
             elif d["category"] == "vendors":
                 tags["vendors"] = d["tags"]
 
-    def __init__(self, assignee='', project='', series='', tags=''):
+    def __init__(self, assignee='', project='', series='', tags='', sku='', cid=''):
         # Adding standard info (CPU, BIOS, GPU, etc.)  to the description
         std_info = AttachmentAssistant.get_standard_info()
-        std_info_str = '\n'.join(['{}: {}'.format(elt, std_info[elt]) for elt in std_info])
-        self.default_description += "[Additional information]\nSKU: \n" + std_info_str
+        std_info_str = "\n".join(["{}: {}".format(elt, std_info[elt]) for elt in std_info])
+        cid_str = "\nCID: {}".format(cid)
+        sku_str = "\nSKU: {}".format(sku)
+        self.default_description += "[Additional information]" + cid_str + sku_str + "\n" + std_info_str
 
         self._title = urwid.LineBox(urwid.Edit(), 'Bug Title')
         self._description = urwid.LineBox(urwid.Edit(edit_text=self.default_description,
diff --git a/qabro/utils.py b/qabro/utils.py
index 8c3e4e8..a89ed24 100644
--- a/qabro/utils.py
+++ b/qabro/utils.py
@@ -1,12 +1,41 @@
 #!/usr/bin/env python3
 import gi
+import gzip
 import json
-import pprint
-import socket
+import os
 
 gi.require_version('Snapd', '1')
 from gi.repository import Snapd
 
+def get_last_checkbox_session_path(usr_home="/home/$USER", root_home="/root", root_dir="/"):
+    """Return path of the most recent Checkbox session directory."""
+    rootpath = ''
+    latest_session_dir = ''
+    generic_path = os.path.join(root_dir, "var/tmp/checkbox-ng/sessions")
+    classic_path = os.path.expandvars(os.path.join(usr_home, ".cache/plainbox/sessions"))
+    if os.path.exists(generic_path):  # for any checkbox installs based on checkbox-ng >= 1.10
+        rootpath = generic_path
+    elif os.path.exists(classic_path):  # for classic checkbox installs based on checkbox-ng <= 1.9
+        rootpath = classic_path
+    else:  # for snap checkbox installs based on checkbox-ng <= 1.9
+        snaps = get_snaps()
+        snap_list = [e.props.name for e in snaps if 'checkbox-' in e.props.name]
+        for snap_name in snap_list:
+            sp = [os.path.join(usr_home, "snap/{}/current/.cache/plainbox/sessions"),
+                  os.path.join(root_home, "snap/{}/current/.cache/plainbox/sessions")]
+            for p in sp:
+                snap_path = os.path.expandvars(p.format(snap_name))
+                if os.path.exists(snap_path):
+                    rootpath = snap_path
+    # Get path of the most recent Checkbox session
+    if rootpath:
+        sessions_dirs = [os.path.join(rootpath, f) for f in os.listdir(rootpath)
+                         if os.path.isdir(os.path.join(rootpath, f))]
+        if sessions_dirs:
+            latest_session_dir = max(sessions_dirs, key=lambda x: os.stat(x).st_mtime)
+    else:
+        print('There does not seem to be any Checkbox session here...')
+    return latest_session_dir
 
 def get_snaps():
     client = Snapd.Client()
diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml
index 346125e..a7608a5 100644
--- a/snap/snapcraft.yaml
+++ b/snap/snapcraft.yaml
@@ -62,11 +62,11 @@ parts:
       snapcraftctl build
       # Patch to add a bunch of $SNAP-related paths in the $PATH environment
       # variable used by sosreport to call other commands.
-      patch $SNAPCRAFT_PART_INSTALL/usr/share/sosreport/sos/policies/ubuntu.py $SNAPCRAFT_STAGE/patches/sosreport-env-path-snap.patch
+      patch $SNAPCRAFT_PART_INSTALL/usr/lib/python3/dist-packages/sos/policies/distros/ubuntu.py $SNAPCRAFT_STAGE/patches/sosreport-env-path-snap.patch
       # patches from YC Chen
-      patch $SNAPCRAFT_PART_INSTALL/usr/share/sosreport/sos/plugins/apt.py $SNAPCRAFT_STAGE/patches/sosreport-apt-remove-user-pwd.patch
+      patch $SNAPCRAFT_PART_INSTALL/usr/lib/python3/dist-packages/sos/report/plugins/apt.py $SNAPCRAFT_STAGE/patches/sosreport-apt-remove-user-pwd.patch
       # patch to be able to have data from `dpkg -l` command
-      patch $SNAPCRAFT_PART_INSTALL/usr/share/sosreport/sos/plugins/dpkg.py $SNAPCRAFT_STAGE/patches/sosreport-dpkg-l.patch
+      patch $SNAPCRAFT_PART_INSTALL/usr/lib/python3/dist-packages/sos/report/plugins/dpkg.py $SNAPCRAFT_STAGE/patches/sosreport-dpkg-l.patch
 
   sosreport-plugins:
     after: [sosreport]