Merge ~fourdollars/pc-enablement/+git/oem-scripts:master into ~oem-solutions-engineers/pc-enablement/+git/oem-scripts:master

[Shih-Yuan Lee]

./mir-bug -q check 1891988 --json ~/projects/oem-hw-info/platform-info/somerville/fossa-beric-tgl.json
...
diff --git a/debian/control b/debian/control
index 0a41830..271e6f9 100644
--- a/debian/control
+++ b/debian/control
@@ -12,6 +12,6 @@ Architecture: all
 Depends: ${misc:Depends}, ubuntu-oem-keyring
 XB-Modaliases: ${modaliases}
 XB-Ubuntu-OEM-Kernel-Flavour: oem
-Description: hardware support for Somerville Beric-Tgl platform
- This is a metapackage for Somerville Beric-Tgl platform. It installs
+Description: hardware support for Dell Inspiron 3501, Vostro 3400/3500
+ This is a metapackage for Dell Inspiron 3501, Vostro 3400/3500. It installs
  packages needed to support this hardware fully.
...

$ ./mir-bug -q check 1925019 --json ~/projects/oem-hw-info/platform-info/stella.cmit/fossa-rattata.json
...
diff --git a/debian/control b/debian/control
index 07aa4b2..fd6d608 100644
--- a/debian/control
+++ b/debian/control
@@ -12,6 +12,6 @@ Architecture: all
 Depends: ${misc:Depends}, ubuntu-oem-keyring
 XB-Modaliases: ${modaliases}
 XB-Ubuntu-OEM-Kernel-Flavour: oem
-Description: hardware support for Stella Cmit Rattata platform
- This is a metapackage for Stella Cmit Rattata platform. It installs
+Description: hardware support for HP ProBook 440 G8 Notebook PC
+ This is a metapackage for HP ProBook 440 G8 Notebook PC. It installs
  packages needed to support this hardware fully.
...

$ ./mir-bug -q check 1924726 --json ~/projects/oem-hw-info/platform-info/sutton.newell/fossa-aelfwine.json
...
diff --git a/debian/control b/debian/control
index af0527e..8172068 100644
--- a/debian/control
+++ b/debian/control
@@ -12,14 +12,13 @@ Architecture: all
 Depends: ${misc:Depends}, ubuntu-oem-keyring, linux-oem-20.04b | linux-oem-20.04, oem-sutton.newell-meta
 XB-Modaliases: ${modaliases}
 XB-Ubuntu-OEM-Kernel-Flavour: oem
-Description: hardware support for Sutton Newell Aelfwine platform
- This is a metapackage for Sutton Newell Aelfwine platform. It installs
+Description: hardware support for Lenovo ThinkPad T14s Gen 2, X13 Gen 2
+ This is a metapackage for Lenovo ThinkPad T14s Gen 2, X13 Gen 2. It installs
  packages needed to support this hardware fully.

 Package: oem-sutton.newell-factory-aelfwine-meta
 Architecture: all
 Depends: ${misc:Depends}, oem-sutton.newell-factory-meta
-Suggests: lenovo-doc-aelfwine
-Description: hardware support for Sutton Newell Aelfwine platform (factory)
- This is a factory metapackage for Sutton Newell Aelfwine platform.
+Description: hardware support for Lenovo ThinkPad T14s Gen 2, X13 Gen 2 (factory)
+ This is a factory metapackage for Lenovo ThinkPad T14s Gen 2, X13 Gen 2.
  It installs packages needed to support this hardware fully. (factory)
...

[Shih-Yuan Lee (fourdollars)]

[BOT]
$ cat oem-scripts-1.5-181c9f8-in-docker-focal-summary.log
autopkgtest-collect-credentials PASS
autopkgtest-oem-scripts-auto PASS
pkg-somerville-meta PASS
pkg-stella-meta PASS
pkg-sutton-meta PASS
bug-bind PASS
get-private-ppa PASS
jq-lp PASS
launchpad-api PASS
lp-bug PASS
oem-meta-packages PASS
pkg-list PASS
review-merge-proposal PASS
run-autopkgtest PASS
setup-apt-dir PASS
mir-bug SKIP exit status 77 and marked as skippable
git-url-insteadof-setting PASS
recovery-from-iso.sh PASS
mir-bug-verification PASS
https://paste.ubuntu.com/p/MmhYx3PgBN/ oem-scripts-1.5-181c9f8-in-docker-focal-complete.log

[Yuan-Chen Cheng (ycheng-twn)]

I am not a fan of re-use variable like this

+ if args.json:
+ args.json = json.load(args.json)

other look good. Up to the author to change it or not.

review: Approve

diff --git a/debian/changelog b/debian/changelog
index 676969f..9180f60 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,11 @@
+oem-scripts (1.5) UNRELEASED; urgency=medium
+
+  * oem-meta-packages: Make the '--output' of the 'collect' subcommand
+    unnecessary.
+  * mir-bug: Make it able to read the product names from the platform json.
+
+ -- Shih-Yuan Lee (FourDollars) <[e-mail]>  Mon, 19 Jul 2021 18:09:17 +0800
+
 oem-scripts (1.4) focal; urgency=medium
 
   * debian/tests/mir-bug-verification: Workaround for update-grub in Linux
diff --git a/debian/tests/oem-meta-packages b/debian/tests/oem-meta-packages
index de367d5..65466a7 100644
--- a/debian/tests/oem-meta-packages
+++ b/debian/tests/oem-meta-packages
@@ -24,7 +24,7 @@ cat > platforms.json << ENDLINE
     "Customer": "Dell",
     "Group": "N/A",
     "Codename": "fossa-beric-icl",
-    "Platform": "Bullseye N3 15 ICL-U(SMB0) ",
+    "Platform": "Bullseye N3 15 ICL-U(SMB0)",
     "MarketName": "Inspiron 3501",
     "PlatformLPTag": "fossa-beric-icl"
   }
@@ -35,10 +35,17 @@ export LAUNCHPAD_API="https://api.launchpad.net/"
 
 oem-meta-packages --quiet list
 
-oem-meta-packages --quiet collect --json platforms.json -o meta-info.json
-jq -r < meta-info.json
-oem-meta-packages --quiet collect --meta oem-somerville-beric-icl-meta -o meta-info.json
-jq -r < meta-info.json
+oem-meta-packages --quiet collect --json platforms.json  # --output meta-info.json (by default)
+if [ "x$(jq -r '."oem-somerville-beric-icl-meta".new_desc' < meta-info.json)" != "xDell Inspiron 3501, Vostro 3401/3501" ]; then
+    jq -S -M < meta-info.json
+    exit 1
+fi
+
+oem-meta-packages --quiet collect --meta oem-somerville-beric-icl-meta  # --output oem-somerville-beric-icl-meta.json (by default)
+if [ "x$(jq -r '."oem-somerville-beric-icl-meta".oem.git.market_name' < oem-somerville-beric-icl-meta.json)" != "xInspiron 3501, Vostro 3401/3501" ]; then
+    jq -S -M < oem-somerville-beric-icl-meta.json
+    exit 1
+fi
 
 oem-meta-packages --dry-run update --json meta-info.json
 
diff --git a/mir-bug b/mir-bug
index 37f38fc..f954f62 100755
--- a/mir-bug
+++ b/mir-bug
@@ -19,6 +19,7 @@
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 
 import argparse
+import collections
 import difflib
 import json
 import lazr
@@ -100,6 +101,11 @@ update.add_argument(
     action="store_true",
     help="Update the bug to Fix Committed, also affects 'Ubuntu', and subscribe 'ubuntu-sponsors' and 'ubuntu-desktop'.",
 )
+update.add_argument(
+    "--json",
+    help="Specify the platform json file to read the information.",
+    type=argparse.FileType("r", encoding="UTF-8"),
+)
 
 check = subparsers.add_parser(
     "check", help="[-h] [--ready] [--skip] [--tz=UTC-8] bugNumber"
@@ -120,6 +126,11 @@ check.add_argument(
     action="store_true",
     help="Check if the bug is Fix Committed, also affects 'Ubuntu', and subscribe 'ubuntu-sponsors' and 'ubuntu-desktop'.",
 )
+check.add_argument(
+    "--json",
+    help="Specify the platform json file to read the information.",
+    type=argparse.FileType("r", encoding="UTF-8"),
+)
 
 collect = subparsers.add_parser("collect", help="[-h] [--ubuntu-certified] jsonFile")
 collect.add_argument(
@@ -197,6 +208,38 @@ Please use "oem-metapackage-mir-check" in lp:ubuntu-archive-tools to verify this
 pattern = re.compile(r".*\[MIR\]\W*oem-([^-]*)-(.*)-meta\W*")
 
 
+# Python 3.9 supports this.
+def remove_prefix(s, prefix):
+    return s[len(prefix) :] if s.startswith(prefix) else s
+
+
+def _grouping_market_names(market_names: list, maxsplit=1) -> str:
+    # Remove empty item
+    while "" in market_names:
+        market_names.remove("")
+    tmp = collections.defaultdict(list)
+    space_in_model = False
+    try:
+        for market_name in sorted(market_names):
+            if maxsplit == 1:
+                name, model = market_name.split(maxsplit=maxsplit)
+                tmp[name].append(model)
+                if " " in model:
+                    space_in_model = True
+            elif maxsplit == 2:
+                brand, name, model = market_name.split(maxsplit=maxsplit)
+                tmp[brand + " " + name].append(model)
+                if " " in model:
+                    space_in_model = True
+    except ValueError:
+        return ", ".join(sorted(market_names))
+
+    if space_in_model:
+        return ", ".join(f"{name} {', '.join(models)}" for name, models in tmp.items())
+    else:
+        return ", ".join(f"{name} {'/'.join(models)}" for name, models in tmp.items())
+
+
 def create_bug(lp, oemCodename, platformCodename, deviceName):
     info("Creating bug...")
     tempList = oemCodename.split(".")
@@ -963,6 +1006,28 @@ def check_and_update_git_repo(
     kernel_flavour, kernel_meta, market_name, ids = _get_items_from_git(
         project, branch, pkg_name
     )
+
+    if args.json:
+        if group == "cmit":
+            new_market_name = _grouping_market_names(
+                args.json["productnames"], maxsplit=2
+            )
+            new_market_name = remove_prefix(new_market_name, "HP ")
+        else:
+            new_market_name = _grouping_market_names(args.json["productnames"])
+        if market_name is None:
+            market_name = new_market_name
+        if market_name != new_market_name:
+            if args.skip or not update:
+                warning(
+                    f"You may need to use '{new_market_name}' instead of '{market_name}'?"
+                )
+            elif yes_or_ask(
+                yes,
+                f"Do you want to use '{new_market_name}' instead of '{market_name}'?",
+            ):
+                market_name = new_market_name
+
     if kernel_flavour == "default":
         if kernel_meta is None:
             kernel_meta = "linux-generic-hwe-20.04"
@@ -970,7 +1035,7 @@ def check_and_update_git_repo(
             pass
         else:
             critical(
-                f"{pkg_name}'s {branch} branch has unexpected kernel_flavour=${kernel_flavour}, kernel_meta={kernel_meta}."
+                f"{pkg_name}'s {branch} branch has unexpected kernel_flavour={kernel_flavour}, kernel_meta={kernel_meta}."
             )
             exit(1)
     elif kernel_flavour == "oem":
@@ -980,7 +1045,7 @@ def check_and_update_git_repo(
             pass
         else:
             critical(
-                f"{pkg_name}'s {branch} branch has unexpected kernel_flavour=${kernel_flavour}, kernel_meta={kernel_meta}."
+                f"{pkg_name}'s {branch} branch has unexpected kernel_flavour={kernel_flavour}, kernel_meta={kernel_meta}."
             )
             exit(1)
 
@@ -1142,8 +1207,12 @@ if args.subcommand:
 if args.subcommand == "create":
     create_bug(lp, args.oemCodename, args.platformCodename, args.deviceName)
 elif args.subcommand == "update":
+    if args.json:
+        args.json = json.load(args.json)
     update_bug(lp, args.bugNumber, args.yes)
 elif args.subcommand == "check":
+    if args.json:
+        args.json = json.load(args.json)
     check_bug(lp, args.bugNumber)
 elif args.subcommand == "collect":
     collect_bugs(lp, args.json)
diff --git a/oem-meta-packages b/oem-meta-packages
index ad37f43..24d7d4b 100755
--- a/oem-meta-packages
+++ b/oem-meta-packages
@@ -166,11 +166,11 @@ subparsers = parser.add_subparsers(dest="subcommand")
 
 collect = subparsers.add_parser(
     "collect",
-    help="[-h] --json platforms.json | --meta oem-qemu-meta -o|--output meta-info.json",
+    help="[-h] --json platforms.json | --meta oem-qemu-meta [-o|--output meta-info.json]",
     formatter_class=argparse.RawDescriptionHelpFormatter,
     epilog="""
 For example,
-    oem-meta-packages collect --json platforms.json --output meta-info.json
+    oem-meta-packages collect --json platforms.json [--output meta-info.json]
 
 platforms.json:
 
@@ -204,7 +204,7 @@ platforms.json:
   ...
 ]
 
-oem-meta-packages collect --meta oem-qemu-meta --output meta-info.json
+oem-meta-packages collect --meta oem-qemu-meta [--output oem-qemu-meta.json]
 
 Collect the information of OEM metapackages in PPAs and devel/staging archives.""",
 )
@@ -220,7 +220,6 @@ collect.add_argument(
 collect.add_argument(
     "-o",
     "--output",
-    required=True,
     help="Specify a filename to write the meta information.",
     type=argparse.FileType("w", encoding="UTF-8"),
 )
@@ -322,6 +321,7 @@ update.add_argument(
 )
 
 args = parser.parse_args()
+BASEDIR = os.getcwd()
 
 setup_logging(debug=args.debug, quiet=args.quiet)
 
@@ -2126,8 +2126,21 @@ elif args.subcommand == "collect":
     else:
         print("You need to use --json or --meta.")
         exit(1)
-    args.output.write(json.dumps(pkgInfo, indent=4, sort_keys=True, cls=CustomEncoder))
-    args.output.write("\n")
+    if args.output:
+        args.output.write(
+            json.dumps(pkgInfo, indent=4, sort_keys=True, cls=CustomEncoder) + "\n"
+        )
+        args.output.write("\n")
+    elif args.json:
+        with open(f"{BASEDIR}/meta-info.json", "w+") as f:
+            f.write(
+                json.dumps(pkgInfo, indent=4, sort_keys=True, cls=CustomEncoder) + "\n"
+            )
+    elif args.meta:
+        with open(f"{BASEDIR}/{args.meta}.json", "w+") as f:
+            f.write(
+                json.dumps(pkgInfo, indent=4, sort_keys=True, cls=CustomEncoder) + "\n"
+            )
 elif args.subcommand == "staging-copy":
     oem_scripts_config_ini = os.path.join(
         os.environ["HOME"], ".config/oem-scripts/config.ini"