Merge ~alextu/plainbox-provider-pc-sanity:fix-leak-wrong-pkgs-unit-test into ~os369510/plainbox-provider-pc-sanity:fix-leak-wrong-pkgs

[Alex Tu]

[Alex Tu (alextu)]

$ bats test/test-screen-pkg-not-public.bat ✓ allow pkgs came from ubuntu archive
 ✓ screen out pkgs we hacked
 ✓ screen out pkgs only in unexpected archive.
 ✓ screen out pkgs not in any archive.
 ✓ screen out pkgs only on oem archive.

5 tests, 0 failures

tested the changed screen-pkg-not-public.sh on ubuntu@dell-bto-focal-fossa-201810-26535:
https://paste.ubuntu.com/p/hPDZgsCNbj/

[Alex Tu (alextu)]

the log of build debian package after adopt shellcheck and bats in debian/rules: https://pastebin.canonical.com/p/kbMf9wDd68/

[Alex Tu (alextu)]

a test build passed in docker container : https://pastebin.canonical.com/p/HpvsfKQdnG/

[Alex Tu (alextu)]

rebased https://code.launchpad.net/~alextu/plainbox-provider-pc-sanity/+git/plainbox-provider-pc-sanity/+merge/392446

[Shih-Yuan Lee (fourdollars)]

There are some conflictions.

review: Needs Fixing

[Alex Tu (alextu)]

I rebased to target branch ~os369510/plainbox-provider-pc-sanity:fix-leak-wrong-pkgs , so that fixed the conflict there.

╰─>bats test/test-screen-pkg-not-public.bat
 ✓ allow pkgs came from ubuntu archive
 ✓ screen out pkgs we hacked
 ✓ screen out pkgs only in unexpected archive.
 ✓ screen out pkgs not in any archive.
 ✓ screen out pkgs only on oem archive.

5 tests, 0 failures

And it build passed in focal docker container:
https://paste.ubuntu.com/p/QYyWNCdKcr/

[Alex Tu (alextu)]

u@u-Latitude-5520:/var/tmp/checkbox-providers/plainbox-provider-pc-sanity$ bats test/test-screen-pkg-not-public.bat
 ✓ run_main() reflect the JOB_STATUS
 ✓ allow pkgs came from ubuntu archive
 ✓ screen out pkgs we hacked
 ✓ screen out pkgs only in unexpected archive.
 ✓ screen out pkgs not in any archive.
 ✓ screen out pkgs only on oem archive.

6 tests, 0 failures

[Rex Tsai (chihchun)]

Good work on the code changes and the unit test.

This MP includes adding new test case for bin/edid_continuous_frequency_check.py and new functions for bin/screen-pkg-not-public.sh. I would prefer to do it separately.

Why this MP is targeting ~os369510/plainbox-provider-pc-sanity:fix-leak-wrong-pkgs but not the project branch?

[jeremyszu (os369510)]

@Alex,

Please review following comments.
I think those are easy to fix.

review: Needs Fixing

[Alex Tu (alextu)]

> Good work on the code changes and the unit test.
>
> This MP includes adding new test case for
> bin/edid_continuous_frequency_check.py and new functions for bin/screen-pkg-
> not-public.sh. I would prefer to do it separately.
>
> Why this MP is targeting ~os369510/plainbox-provider-pc-sanity:fix-leak-wrong-
> pkgs but not the project branch?

Because this MP is triggered by the difficulty of reviewing ~os369510/plainbox-provider-pc-sanity:fix-leak-wrong-pkgs. So, I refactored it based on ~os369510/plainbox-provider-pc-sanity:fix-leak-wrong-pkgs (which is targeting to extend the coverage) to add unit testing.

[Alex Tu (alextu)]

done the chage:

ubuntu@dell-bto-focal-fossa-201810-26535$ checkbox-cli run com.canonical.certification::miscellanea/screen-pkg-not-public

https://paste.ubuntu.com/p/nfz4hVxnV9/

$ bats test/test-screen-pkg-not-public.bat
 ✓ run_main() reflect the JOB_STATUS
 ✓ allow pkgs came from ubuntu archive
 ✓ screen out pkgs we hacked
 ✓ screen out pkgs only in unexpected archive.
 ✓ screen out pkgs not in any archive.
 ✓ screen out pkgs only on oem archive.

6 tests, 0 failures

[Alex Tu (alextu)]

also the autopkgtest result after merging master : https://code.launchpad.net/~oem-solutions-engineers/plainbox-provider-pc-sanity/+git/plainbox-provider-pc-sanity/+ref/master

autopkgresult :
https://paste.ubuntu.com/p/NF2v5wydzS/

[Alex Tu (alextu)]

also the autopkgtest result after merging master : https://code.launchpad.net/~oem-solutions-engineers/plainbox-provider-pc-sanity/+git/plainbox-provider-pc-sanity/+ref/master

$ ./autopkgtest.sh 2>&1
https://paste.ubuntu.com/p/w9kWtQB2GR/

[jeremyszu (os369510)]

See my comment below.

review: Needs Fixing

[jeremyszu (os369510)]

LGTM, thanks.

review: Approve

diff --git a/README.rst b/README.rst
index 702788e..d8af49f 100644
--- a/README.rst
+++ b/README.rst
@@ -2,3 +2,6 @@ Plainbox Provider for Kernel SRU testing
 ========================================
 
 This provider contains test cases and a test plan for pc sanity testing.
+
+It depends on packages from ppa:checkbox-dev/ppa in build time.
+The launchpad recipe build it daily : https://code.launchpad.net/~oem-solutions-engineers/+recipe/plainbox-provider-pc-sanity-daily-1
diff --git a/bin/edid_continuous_frequency_check.py b/bin/edid_continuous_frequency_check.py
index 368b992..5db54a5 100755
--- a/bin/edid_continuous_frequency_check.py
+++ b/bin/edid_continuous_frequency_check.py
@@ -5,14 +5,16 @@ import os
 import glob
 import argparse
 
+
 def is_laptop():
     with open('/sys/class/dmi/id/chassis_type', 'r') as fobj:
-        chassis_type=fobj.read().replace('\n', '')
+        chassis_type = fobj.read().replace('\n', '')
         if chassis_type == '10' or chassis_type == '9' or chassis_type == '13':
             return True
     print("Not laptop, notebook or AIO platform, abort EDID check.")
     return False
 
+
 def find_internal_panel_edid():
     """ Find internal panel edid """
     drmdir = '/sys/class/drm/'
@@ -55,7 +57,7 @@ def get_horizontal_addressable_video_in_pixels(data_block_of_DTD):
     """
     havip = bytearray((data_block_of_DTD[4] >> 4).to_bytes(1, byteorder='big'))
     havip.append(data_block_of_DTD[2])
-    # print("DTD: Horizontal Addressable Video in Pixels: %d" % int.from_bytes(havip, byteorder='big') )
+    # print("DTD: Horizontal Addressable Video in Pixels: %d" % int.from_bytes(havip, byteorder='big') ) # noqa: E501
     return int.from_bytes(havip, byteorder='big')
 
 
@@ -63,22 +65,22 @@ def maximum_horizontal_active_pixels(byte12, byte13):
     return (8 * (byte13 + 256 * (0x03 & byte12)))
 
 
-def check_display_range_limits_descriptor(data_block_of_DRLD, horizontal_address_video_in_pixels):
+def check_display_range_limits_descriptor(data_block_of_DRLD, horizontal_address_video_in_pixels):  # noqa: E501
     """ Check Display Range Limits Descriptor with CVT support
         Address: 0x48h
         Size: 18 bytes
     """
     # Debug dump DRLD
-    # print("DRLD: Dump: %02X" %  int.from_bytes(data_block_of_DRLD, byteorder='big'))
+    # print("DRLD: Dump: %02X" %  int.from_bytes(data_block_of_DRLD, byteorder='big'))# noqa: E501
 
     # check tag 0xFD
     if (data_block_of_DRLD[3] == 0xfd):
         if (data_block_of_DRLD[10] == 0x04):
-            if (data_block_of_DRLD[5] < data_block_of_DRLD[6]) and (data_block_of_DRLD[7] < data_block_of_DRLD[8]):
-                if horizontal_address_video_in_pixels == maximum_horizontal_active_pixels(data_block_of_DRLD[12], data_block_of_DRLD[13]):
+            if (data_block_of_DRLD[5] < data_block_of_DRLD[6]) and (data_block_of_DRLD[7] < data_block_of_DRLD[8]):  # noqa: E501
+                if horizontal_address_video_in_pixels == maximum_horizontal_active_pixels(data_block_of_DRLD[12], data_block_of_DRLD[13]):  # noqa: E501
                     return True
                 else:
-                    print("horizontal_address_video_in_pixels != maximum_horizontal_active_pixels ")
+                    print("horizontal_address_video_in_pixels != maximum_horizontal_active_pixels ")  # noqa: E501
             else:
                 print("Minimum/Maximum Vertical/Horizontal Freq value error")
         else:
@@ -90,7 +92,7 @@ def check_display_range_limits_descriptor(data_block_of_DRLD, horizontal_address
 
 def main():
     parser = argparse.ArgumentParser()
-    parser.add_argument("--edid", help="path of a EDID file from /sys/class/drm/", type=str)
+    parser.add_argument("--edid", help="path of a EDID file from /sys/class/drm/", type=str)  # noqa: E501
     args = parser.parse_args()
     if not is_laptop():
         return 0
@@ -101,13 +103,13 @@ def main():
         edidfile = args.edid
 
     with open(edidfile, 'rb') as fobj:
-        version, revision = get_edid_version(read_edid(fobj, 0x12, 1), read_edid(fobj, 0x13, 1))
+        version, revision = get_edid_version(read_edid(fobj, 0x12, 1), read_edid(fobj, 0x13, 1))  # noqa: E501
         if version == 1 and revision >= 4:
             if is_continuous_frequency_display(read_edid(fobj, 0x18, 1)):
-                if check_display_range_limits_descriptor(read_edid(fobj, 0x48, 18), get_horizontal_addressable_video_in_pixels(read_edid(fobj, 0x36, 18))):
+                if check_display_range_limits_descriptor(read_edid(fobj, 0x48, 18), get_horizontal_addressable_video_in_pixels(read_edid(fobj, 0x36, 18))):  # noqa: E501
                     print("EDID Display Range Limits Descriptor checke: pass")
                 else:
-                    raise SystemExit("EDID Display Range Limits Descriptor check failed!")
+                    raise SystemExit("EDID Display Range Limits Descriptor check failed!")  # noqa: E501
             else:
                 print("Display is noncontinuous frequency")
         else:
diff --git a/bin/idle-delay-60.sh b/bin/idle-delay-60.sh
index 37e579d..db18e33 100755
--- a/bin/idle-delay-60.sh
+++ b/bin/idle-delay-60.sh
@@ -1 +1,3 @@
+#!/bin/bash
+
 gsettings set org.gnome.desktop.session idle-delay 60
diff --git a/bin/screen-pkg-not-public.sh b/bin/screen-pkg-not-public.sh
old mode 100644
new mode 100755
index 3eec17c..a3a3ae8
--- a/bin/screen-pkg-not-public.sh
+++ b/bin/screen-pkg-not-public.sh
@@ -1,22 +1,26 @@
 #!/bin/bash
 
 set -e
+readonly pkg_pass="0"
+readonly pkg_failed="1"
 allowlist_git="https://git.launchpad.net/~oem-solutions-engineers/pc-enablement/+git/oem-gap-allow-list"
+pf_meta_pkg=""
+pf_factory_meta_pkg=""
 oem=""
 platform=""
 allowlst_folder=""
-STATUS="pass"
+JOB_STATUS="pass"
 clean() {
    rm -rf "$allowlst_folder"
    [ -z "$1" ] || exit "$1"
-   [ "$STATUS" != "pass" ] && exit 1
+   [ "$JOB_STATUS" != "pass" ] && exit 1
    exit 0
 }
 prepare() {
     oem="$(grep -q sutton <(ubuntu-report show | grep DCD) && echo sutton)" ||\
     oem="$(grep -q stella <(ubuntu-report show | grep DCD) && echo stella)" ||\
     oem="$(grep -q somerville <(ubuntu-report show | grep DCD) && echo somerville)" ||\
-    (>&2 echo "[ERROR][CODE]got an empty OEM codename in ${FUNCNAME[1]}" && clean 1)
+    (>&2 echo "[ERROR][CODE]got an empty OEM codename in ${FUNCNAME[0]}" && clean 1)
     case "$oem" in
         "somerville")
             platform="$(ubuntu-report show | grep DCD | awk -F'+' '{print $2}')"
@@ -33,30 +37,33 @@ prepare() {
             done
             ;;
         *)
-            >&2 echo "[ERROR][CODE]we should not be here in ${FUNCNAME[1]} : ${LINENO}" && clean 1
+            >&2 echo "[ERROR][CODE]we should not be here in ${FUNCNAME[0]} : ${LINENO}" && clean 1
             ;;
     esac
-    [ -z "$platform" ] && (>&2 echo "[ERROR][CODE]got an empty platform name for $oem in ${FUNCNAME[1]}" && clean 1)
+    [ -n "$platform" ] || (>&2 echo "[ERROR][CODE]got an empty platform name for $oem in ${FUNCNAME[0]}" && clean 1)
     (sudo apt-get update > /dev/null || (>&2 echo "[ERROR]apt-get update failed, please check it." | exit 1)) && sudo apt-get install -y git > /dev/null
+    pf_meta_pkg="$(dpkg -S /etc/apt/sources.list.d/oem-"${oem}"-*-meta.list | awk '{print $1}' | sed 's/://Ig')"
+    pf_factory_meta_pkg="${pf_meta_pkg/oem-${oem}-/oem-${oem}-factory-}"
     echo "[INFO] getting allowlist from $allowlist_git."
     [ -n "$allowlist_git" ] &&\
-    (git clone --depth=1 "$allowlist_git" || (>&2 echo "[ERROR]git clone ""$allowlist_git"" filed, please check it." | exit 1)) &&\
-    allowlst_folder="$PWD"/"$(basename "$allowlist_git")"
+    allowlst_folder="$PWD"/"$(basename "$allowlist_git")" &&\
+    rm -rf "$allowlst_folder" &&\
+    (git clone --depth=1 "$allowlist_git" || (>&2 echo "[ERROR]git clone ""$allowlist_git"" failed, please check it." | exit 1))
     echo "[INFO] git hash of current allowlist: $(git -C "$allowlst_folder" rev-parse --short HEAD)"
 }
 pkg_need_allowing() {
-    [ -z "$1" ] && >&2 echo "[ERROR][CODE]got an empty pkg in ${FUNCNAME[1]}" && clean 1
-    >&2 echo "[ERROR] Please send a MP to $allowlist_git for manager review $1" && STATUS="failed"
+    [ -n "$1" ] || ( >&2 echo "[ERROR][CODE]got an empty pkg in ${FUNCNAME[0]}" && clean 1)
+    >&2 echo "[ERROR] Please send a MP to $allowlist_git for manager review $1" && JOB_STATUS="failed"
 }
 pkg_need_update() {
-    [ -z "$1" ] && >&2 echo "[ERROR][CODE]got an empty pkg in ${FUNCNAME[1]}" && clean 1
+    [ -n "$1" ] && (>&2 echo "[ERROR][CODE]got an empty pkg in ${FUNCNAME[0]}" && clean 1)
     >&2 echo "[ERROR] find a update-able pkg: $1 $2" && pkg_need_allowing "$1"
 }
 # return 0 for allowing
 # return 1 for not allowing
 if_allowing() {
     local allowed="NO"
-    [ -z "$1" ] && >&2 echo "[ERROR][CODE]got an empty pkg in ${FUNCNAME[1]}" && clean 1
+    [ -n "$1" ] || (>&2 echo "[ERROR][CODE]got an empty pkg in ${FUNCNAME[0]}" && clean 1)
 
     # check if the pkg on allow list.
     for F in "$allowlst_folder"/testtools "$allowlst_folder"/common "$allowlst_folder"/"$oem"/common "$allowlst_folder"/"$oem"/"$platform"; do
@@ -71,24 +78,16 @@ if_allowing() {
     fi
 }
 pkg_not_public() {
-    [ -z "$1" ] && >&2 echo "[ERROR][CODE]got an empty pkg in" "${FUNCNAME[1]}" && clean 1
+    [ -n "$1" ] || (>&2 echo "[ERROR][CODE]got an empty pkg in" "${FUNCNAME[0]}" && clean 1)
     # check if the pkg on allow list.
-    if_allowing "$1" || (>&2 echo "[ERROR] find a packge not on public archive:" "$1" "$2" && pkg_need_allowing "$1")
+    if_allowing "$1" || (>&2 echo "[ERROR] find a packge not on public archive:" "$1" "$2" && pkg_need_allowing "$1" && return $pkg_failed)
 }
-prepare
-echo "[INFO] staring screen all installed packages."
-
-pf_meta_pkg="$(dpkg -S /etc/apt/sources.list.d/oem-"${oem}"-*-meta.list | awk '{print $1}' | sed 's/://Ig')"
-pf_factory_meta_pkg="${pf_meta_pkg/oem-${oem}-/oem-${oem}-factory-}"
-
-while IFS= read -r line; do
-    progress=">""$progress"
-    [ "${#progress}" == "70" ] && echo "$progress" && progress=""
-
+screen_pkg() {
+    [ -n "$1" ] || (>&2 echo "[ERROR][CODE]got an empty input in" "${FUNCNAME[0]}" && clean 1)
+    line="$1"
     pkg_name="$(echo "${line}" | awk '{print $2}')"
     pkg_ver="$(echo "${line}" | awk '{print $3}')"
     pkg_curr_madison="$(apt-cache madison "${pkg_name}" | grep "$pkg_ver" || true)"
-
     # FIXME: I don't think the upgradable is need to in "id: miscellanea/screen-pkg-not-public"
     # Should have the other something like "id: miscellanea/check-oem-pkg-updatable"
     # Remove this section can speed up this test scope
@@ -96,7 +95,7 @@ while IFS= read -r line; do
     # TODO: detect somerville only?
     if [ -z "${pkg_ver##*oem*}" ] || [ -z "${pkg_ver##*somerville*}" ]; then
         if if_allowing "$pkg_name"; then
-            continue
+            return $pkg_pass
         else
             can_pkg_ver="$(apt-cache policy "$pkg_name" | grep Candidate | awk '{print $2}')"
             if dpkg --compare-versions "$can_pkg_ver" "gt" "$pkg_ver"; then
@@ -115,10 +114,13 @@ while IFS= read -r line; do
         if [ -n "${pkg_can_madison}" ] &&
             { [ -z "${pkg_can_madison##*security.ubuntu.com/ubuntu*}" ] ||
             [ -z "${pkg_can_madison##*archive.ubuntu.com/ubuntu*}" ]; }; then
-            continue
+            return $pkg_pass
+        fi
+        if pkg_not_public "$pkg_name" "$pkg_ver"; then
+            return $pkg_pass
+        else
+            return $pkg_failed
         fi
-        pkg_not_public "$pkg_name" "$pkg_ver"
-        continue
     fi
 
     # If the installed package is from ubuntu-archive then we're good
@@ -126,7 +128,7 @@ while IFS= read -r line; do
     #  all source list are under control at lease before GM)
     if [ -z "${pkg_curr_madison##*security.ubuntu.com/ubuntu*}" ] ||
         [ -z "${pkg_curr_madison##*archive.ubuntu.com/ubuntu*}" ]; then
-        continue
+        return $pkg_pass
     fi
 
     # If the installed package is from canonical-archive then we need to
@@ -134,15 +136,33 @@ while IFS= read -r line; do
     if [ -z "${pkg_curr_madison##*archive.canonical.com*}" ]; then
         # If the package is platform meta package then it should be control by meta generator
         if [ "$pkg_name" == "$pf_meta_pkg" ] || [ "$pkg_name" == "$pf_factory_meta_pkg" ]; then
-            continue
+            return $pkg_pass
         fi
         # Otherwise, need to review
-        pkg_not_public "$pkg_name" "$pkg_ver"
-        continue
+        if pkg_not_public "$pkg_name" "$pkg_ver"; then
+            return $pkg_pass
+        else
+            return $pkg_failed
+        fi
     fi
-
     # For unkown source of package (e.g. from a ppa), then review
-    pkg_not_public "$pkg_name" "$pkg_ver"
+    if pkg_not_public "$pkg_name" "$pkg_ver"; then
+        return $pkg_pass
+    else
+        return $pkg_failed
+    fi
+}
 
-done < <(dpkg -l | grep 'ii')
-clean
+run_main() {
+    prepare
+    >&2 echo "[INFO] staring screen all installed packages."
+    while IFS= read -r line; do
+        progress=">""$progress"
+        [ "${#progress}" == "70" ] && echo "$progress" && progress=""
+        screen_pkg "$line" || JOB_STATUS="failed"
+    done < <(dpkg -l | grep 'ii')
+    clean
+}
+if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
+  run_main
+fi
diff --git a/debian/control b/debian/control
index 5c17129..0912a6e 100644
--- a/debian/control
+++ b/debian/control
@@ -3,7 +3,7 @@ Section: utils
 Priority: optional
 Maintainer: Commercial Engineering <[e-mail]>
 Standards-Version: 3.9.7
-Build-Depends: debhelper (>= 9), dh-modaliases
+Build-Depends: debhelper (>= 9), dh-modaliases, intltool, python3-checkbox-ng, plainbox-provider-checkbox, flake8, bats, shellcheck
 Vcs-Browser: https://code.launchpad.net/~ville-team/ville/+git/plainbox-provider-pc-sanity
 
 Package: plainbox-provider-pc-sanity
diff --git a/debian/rules b/debian/rules
index 02c4840..1265262 100755
--- a/debian/rules
+++ b/debian/rules
@@ -11,7 +11,10 @@ override_dh_auto_build:
 	python3 manage.py i18n
 
 override_dh_auto_test:
-	python3 manage.py validate 
+	python3 manage.py validate
+	shellcheck bin/*.sh
+	bats test/*.bat
+	flake8 bin/*.py
 
 override_dh_auto_install:
 	python3 manage.py install --prefix=/usr --layout=unix \
diff --git a/po/POTFILES.in b/po/POTFILES.in
index f0e418b..a3aa462 100644
--- a/po/POTFILES.in
+++ b/po/POTFILES.in
@@ -1,3 +1,3 @@
 [encoding: UTF-8]
-[type: gettext/rfc822deb] units/pc-sanity.pxu
+[type: gettext/rfc822deb] units/pc-sanity/pc-sanity.pxu
 manage.py
diff --git a/test/test-screen-pkg-not-public.bats b/test/test-screen-pkg-not-public.bats
new file mode 100644
index 0000000..9e8bb5a
--- /dev/null
+++ b/test/test-screen-pkg-not-public.bats
@@ -0,0 +1,165 @@
+#!/usr/bin/env bats
+
+# execute this test file by `bats test/test-screen-pkg-not-public.bats`
+BIN_FOLDER="bin"
+
+function if_allowing() {
+    return 1
+}
+
+function setup() {
+    source "$BIN_FOLDER"/screen-pkg-not-public.sh
+    oem="somerville"
+    platform="fossa-melisa"
+}
+
+@test "run_main() reflect the JOB_STATUS" {
+    set -e
+    status_failed=1
+    status_passed=0
+    function screen_pkg(){
+        return "$status_simulate"
+    }
+    function prepare(){
+        echo "empty"
+    }
+    function clean(){
+        echo "empty"
+    }
+    function dpkg(){
+        echo "ii"
+    }
+
+    status_simulate=$status_passed
+    # we expect it pass.
+    run_main
+    [ "$JOB_STATUS" == "pass" ]
+
+    status_simulate=$status_failed
+    # we expect it failed.
+    run_main
+    [ "$JOB_STATUS" == "failed" ]
+
+}
+
+function apt-cache() {
+   case "$1" in
+    "madison")
+    echo "$aptcache_medison_string"
+    ;;
+    "policy")
+    echo "$apt_cache_policy_string"
+    ;;
+    *)
+    return 1;
+    ;;
+   esac
+}
+
+@test "allow pkgs came from ubuntu archive" {
+    set -e
+    aptcache_medison_string="
+    ubuntu-desktop |    1.450.2 | http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
+    ubuntu-desktop |      1.450 | http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
+    "
+    apt_cache_policy_string="\
+    ubuntu-desktop:\
+      Installed: 1.450.2\
+      Candidate: 1.450.2\
+      Version table:\
+     *** 1.450.2 500\
+            500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages\
+            100 /var/lib/dpkg/status\
+         1.450 500\
+            500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages\
+    "
+    export -f apt-cache
+    # we expect it passed.
+    run screen_pkg "ii  ubuntu-desktop                                1.450.2                                     amd64        The Ubuntu desktop system"
+    [ "$status" -eq 0 ]
+    unset -f apt-cache
+
+}
+
+@test "screen out pkgs we hacked" {
+    set -e
+    dpkg_list_string="ii  ubiquity                                      20.04.15.2somerville2                       amd64        Ubuntu live CD installer"
+    aptcache_medison_string="
+    ubiquity | 20.04.15.2 | http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
+    ubiquity |   20.04.15 | http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
+    "
+    apt_cache_policy_string="
+    ubiquity:
+      Installed: 20.04.15.2somerville2
+      Candidate: 20.04.15.2somerville2
+      Version table:
+     *** 20.04.15.2somerville2 100
+            100 /var/lib/dpkg/status
+         20.04.15.2 500
+            500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
+         20.04.15 500
+            500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
+    "
+    export -f apt-cache
+    # we expect it failed.
+    run screen_pkg "$dpkg_list_string"
+    [ "$status" -eq 1 ]
+    unset -f apt-cache
+}
+
+@test "screen out pkgs only in unexpected archive." {
+    set -e
+    dpkg_list_string="ii  fwts                                          20.09.00-0ubuntu1~f                         amd64        FirmWare Test Suite"
+    aptcache_medison_string="
+    fwts | 20.09.00-0ubuntu1~f | http://ppa.launchpad.net/checkbox-dev/ppa/ubuntu focal/main amd64 Packages
+    fwts | 20.03.00-0ubuntu1 | http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages
+    "
+    apt_cache_policy_string="
+    fwts:
+      Installed: 20.09.00-0ubuntu1~f
+      Candidate: 20.09.00-0ubuntu1~f
+      Version table:
+     *** 20.09.00-0ubuntu1~f 500
+            500 http://ppa.launchpad.net/checkbox-dev/ppa/ubuntu focal/main amd64 Packages
+            100 /var/lib/dpkg/status
+         20.03.00-0ubuntu1 500
+            500 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages
+    "
+
+    export -f apt-cache
+    # we expect it failed.
+    run screen_pkg "$dpkg_list_string"
+    [ "$status" -eq 1 ]
+    unset -f apt-cache
+}
+
+@test "screen out pkgs not in any archive." {
+    set -e
+    # we expect it failed.
+    run screen_pkg "ii  a-nonexisted-pkg                                1.187.3                                     all          Firmware for Linux kernel drivers"
+    [ "$status" -eq 1 ]
+    echo in bat: JOB_STATUS=$JOB_STATUS
+}
+
+@test "screen out pkgs only on oem archive." {
+    set -e
+    dpkg_list_string="ii  pkg-only-on-oemarchive                                20.04ubuntu7                                     all          Firmware for Linux kernel drivers"
+    aptcache_medison_string="
+    pkg-only-on-oem-archive | 20.04ubuntu7 | http://dell.archive.canonical.com focal/somerville-melisa amd64 Packages
+    "
+    apt_cache_policy_string="
+    pkg-only-on-oem-archive:
+     Installed: 20.04ubuntu7
+     Candidate: 20.04ubuntu7
+     Version table:
+    *** 20.04ubuntu7 500
+    500 http://dell.archive.canonical.com focal/somerville-melisa amd64 Packages
+    500 http://dell.archive.canonical.com focal/somerville-melisa i386 Packages
+    "
+
+    export -f apt-cache
+    # we expect it failed.
+    run screen_pkg "$dpkg_list_string"
+    [ "$status" -eq 1 ]
+    unset -f apt-cache
+}