Merge ~bladernr/plainbox-provider-checkbox:fwts-log-checks into plainbox-provider-checkbox:master

[Jeff Lane ]

Add new jobs to run fwts olog (power only) and klog checks and attach the logs from those.
Add testplan for server cert to miscellanea/test-plan.pxu

[Jonathan Cave (jocave)]

Suggestion of an improvement below

review: Needs Fixing

[Jeff Lane  (bladernr)]

Thanks. Fixed. It was a hanger-on from the first version of that that did have FWTS jobs, but I decided to pull those out separately for ordering reasons.

review: Needs Resubmitting

[Jonathan Cave (jocave)]

Sorry, noticed two other things to help improve quality of definitions, could you add a category_id and summary to the job defs where missing (I prefer summary to description)? These are displayed in the reports we generate and curses UI so keeps things pretty.

review: Needs Fixing

[Jeff Lane  (bladernr)]

Done

review: Needs Resubmitting

[Jonathan Cave (jocave)]

Thanks

review: Approve

[Devices Certification Bot (ce-certification-qa)]

The merge was fine but running tests failed.

[trusty] starting container
[trusty] (timing) 0.00user 0.00system 0:00.25elapsed 6%CPU (0avgtext+0avgdata 5004maxresident)k
[trusty] (timing) 0inputs+72outputs (0major+1089minor)pagefaults 0swaps
[trusty] provisioning container
[trusty] (timing) 74.46user 110.37system 7:20.66elapsed 41%CPU (0avgtext+0avgdata 68972maxresident)k
[trusty] (timing) 8032inputs+1609656outputs (15major+1201739minor)pagefaults 0swaps
[trusty-testing] Starting tests...
Found a test script: ./requirements/container-tests-provider-checkbox
[trusty-testing] container-tests-provider-checkbox: FAIL
[trusty-testing] stdout: https://paste.ubuntu.com/p/P84JBvr6xW/
[trusty-testing] stderr: https://paste.ubuntu.com/p/YtVbWgD78b/
[trusty-testing] (timing) Command exited with non-zero status 1
[trusty-testing] (timing) 17.79user 1.17system 0:35.54elapsed 53%CPU (0avgtext+0avgdata 61280maxresident)k
[trusty-testing] (timing) 0inputs+100800outputs (0major+472212minor)pagefaults 0swaps
[trusty-testing] Fixing file permissions in source directory
[trusty-testing] Destroying container
Name: trusty-testing
State: STOPPED
[xenial] starting container
[xenial] (timing) 0.00user 0.00system 0:00.33elapsed 2%CPU (0avgtext+0avgdata 5064maxresident)k
[xenial] (timing) 0inputs+72outputs (0major+1088minor)pagefaults 0swaps
[xenial] provisioning container
[xenial] (timing) 33.26user 14.54system 5:23.56elapsed 14%CPU (0avgtext+0avgdata 83492maxresident)k
[xenial] (timing) 14336inputs+1831184outputs (20major+1535524minor)pagefaults 0swaps
[xenial-testing] Starting tests...
Found a test script: ./requirements/container-tests-provider-checkbox
[xenial-testing] container-tests-provider-checkbox: FAIL
[xenial-testing] stdout: https://paste.ubuntu.com/p/gnqCrMZFWq/
[xenial-testing] stderr: https://paste.ubuntu.com/p/MsDj4XHdwv/
[xenial-testing] (timing) Command exited with non-zero status 1
[xenial-testing] (timing) 17.01user 0.87system 0:33.54elapsed 53%CPU (0avgtext+0avgdata 63172maxresident)k
[xenial-testing] (timing) 32inputs+101152outputs (0major+475176minor)pagefaults 0swaps
[xenial-testing] Fixing file permissions in source directory
[xenial-testing] Destroying container
Name: xenial-testing
State: STOPPED
[bionic] starting container
[bionic] (timing) 0.00user 0.00system 0:00.27elapsed 2%CPU (0avgtext+0avgdata 4984maxresident)k
[bionic] (timing) 0inputs+72outputs (0major+1086minor)pagefaults 0swaps
[bionic] provisioning container
[bionic] (timing) 38.36user 17.32system 6:41.49elapsed 13%CPU (0avgtext+0avgdata 92168maxresident)k
[bionic] (timing) 17232inputs+2260672outputs (5major+1954273minor)pagefaults 0swaps
[bionic-testing] Starting tests...
Found a test script: ./requirements/container-tests-provider-checkbox
[bionic-testing] container-tests-provider-checkbox: FAIL
[bionic-testing] stdout: https://paste.ubuntu.com/p/YtDV5sm2DZ/
[bionic-testing] stderr: https://paste.ubuntu.com/p/tvpyC4fCRq/
[bionic-testing] (timing) Command exited with non-zero status 1
[bionic-testing] (timing) 16.92user 1.15system 0:32.69elapsed 55%CPU (0avgtext+0avgdata 55416maxresident)k
[bionic-te...

review: Needs Fixing

[Jeff Lane  (bladernr)]

fixed the stupid unit test failure... and filed a bug because the unit test is wrong.

review: Needs Resubmitting

[Jeff Lane  (bladernr)]

self-approve

review: Approve

[Devices Certification Bot (ce-certification-qa)]

The merge was fine but running tests failed.

[trusty] starting container
[trusty] (timing) 0.00user 0.00system 0:00.23elapsed 3%CPU (0avgtext+0avgdata 5084maxresident)k
[trusty] (timing) 0inputs+72outputs (0major+1096minor)pagefaults 0swaps
[trusty] provisioning container
[trusty] (timing) 74.61user 110.14system 7:21.43elapsed 41%CPU (0avgtext+0avgdata 68968maxresident)k
[trusty] (timing) 6008inputs+1609688outputs (23major+1199019minor)pagefaults 0swaps
[trusty-testing] Starting tests...
Found a test script: ./requirements/container-tests-provider-checkbox
[trusty-testing] container-tests-provider-checkbox: FAIL
[trusty-testing] stdout: https://paste.ubuntu.com/p/cM2QGbRQSK/
[trusty-testing] stderr: https://paste.ubuntu.com/p/zkcpNqJvxs/
[trusty-testing] (timing) Command exited with non-zero status 1
[trusty-testing] (timing) 17.36user 1.15system 1:12.92elapsed 25%CPU (0avgtext+0avgdata 61224maxresident)k
[trusty-testing] (timing) 0inputs+100800outputs (0major+470772minor)pagefaults 0swaps
[trusty-testing] Fixing file permissions in source directory
[trusty-testing] Destroying container
Name: trusty-testing
State: STOPPED
[xenial] starting container
[xenial] (timing) 0.00user 0.00system 0:00.30elapsed 2%CPU (0avgtext+0avgdata 4988maxresident)k
[xenial] (timing) 0inputs+72outputs (0major+1088minor)pagefaults 0swaps
[xenial] provisioning container
[xenial] (timing) 33.50user 14.75system 6:04.89elapsed 13%CPU (0avgtext+0avgdata 83488maxresident)k
[xenial] (timing) 9360inputs+1831192outputs (6major+1536692minor)pagefaults 0swaps
[xenial-testing] Starting tests...
Found a test script: ./requirements/container-tests-provider-checkbox
[xenial-testing] container-tests-provider-checkbox: FAIL
[xenial-testing] stdout: https://paste.ubuntu.com/p/rtmhFRWrcd/
[xenial-testing] stderr: https://paste.ubuntu.com/p/jjQ48VhScz/
[xenial-testing] (timing) Command exited with non-zero status 1
[xenial-testing] (timing) 16.48user 0.94system 0:36.79elapsed 47%CPU (0avgtext+0avgdata 63216maxresident)k
[xenial-testing] (timing) 0inputs+101128outputs (0major+474841minor)pagefaults 0swaps
[xenial-testing] Fixing file permissions in source directory
[xenial-testing] Destroying container
Name: xenial-testing
State: STOPPED
[bionic] starting container
[bionic] (timing) 0.00user 0.00system 0:00.29elapsed 2%CPU (0avgtext+0avgdata 5088maxresident)k
[bionic] (timing) 0inputs+72outputs (0major+1093minor)pagefaults 0swaps
[bionic] provisioning container
[bionic] (timing) 38.22user 17.47system 6:38.82elapsed 13%CPU (0avgtext+0avgdata 92064maxresident)k
[bionic] (timing) 10400inputs+2260664outputs (8major+1957480minor)pagefaults 0swaps
[bionic-testing] Starting tests...
Found a test script: ./requirements/container-tests-provider-checkbox
[bionic-testing] container-tests-provider-checkbox: FAIL
[bionic-testing] stdout: https://paste.ubuntu.com/p/JXQbfTzVFw/
[bionic-testing] stderr: https://paste.ubuntu.com/p/z4KY2wHfcW/
[bionic-testing] (timing) Command exited with non-zero status 1
[bionic-testing] (timing) 16.63user 0.94system 0:31.16elapsed 56%CPU (0avgtext+0avgdata 55380maxresident)k
[bionic-testi...

review: Needs Fixing

diff --git a/units/miscellanea/jobs.pxu b/units/miscellanea/jobs.pxu
index 5d8ee64..760d301 100644
--- a/units/miscellanea/jobs.pxu
+++ b/units/miscellanea/jobs.pxu
@@ -314,11 +314,14 @@ _description:
 
 plugin:shell
 id: miscellanea/oops
+category_id: com.canonical.plainbox::miscellanea
 estimated_duration: 10.0
 requires: executable.name == 'fwts'
 user: root
 _description:
  Run Firmware Test Suite (fwts) oops tests.
+_summary:
+ Run FWTS OOPs check
 environ: PLAINBOX_SESSION_SHARE
 command:
  checkbox-support-fwts_test -l $PLAINBOX_SESSION_SHARE/fwts_oops_results.log -t oops
@@ -331,6 +334,52 @@ command:
  [ -e ${PLAINBOX_SESSION_SHARE}/fwts_oops_results.log ] && xz -c ${PLAINBOX_SESSION_SHARE}/fwts_oops_results.log
 _description: Attaches the FWTS oops results log to the submission
 
+plugin:shell
+id: miscellanea/olog
+category_id: com.canonical.plainbox::miscellanea
+estimated_duration: 10.0
+requires:
+ executable.name == 'fwts'
+ cpuinfo.platform in ("ppc64el")
+user: root
+_description:
+ Run Firmware Test Suite (fwts) olog tests (ppc64el only).
+_summary:
+ Run FWTS OLOG check on ppc64el
+environ: PLAINBOX_SESSION_SHARE
+command:
+ checkbox-support-fwts_test -l $PLAINBOX_SESSION_SHARE/fwts_olog_results.log -t olog
+
+plugin: attachment
+category_id: com.canonical.plainbox::miscellanea
+estimated_duration: 0.5
+id: miscellanea/olog_results.log
+command:
+ [ -e ${PLAINBOX_SESSION_SHARE}/fwts_olog_results.log ] && xz -c ${PLAINBOX_SESSION_SHARE}/fwts_olog_results.log
+_description: Attaches the FWTS olog results log to the submission
+
+plugin:shell
+id: miscellanea/klog
+category_id: com.canonical.plainbox::miscellanea
+estimated_duration: 10.0
+requires: executable.name == 'fwts'
+user: root
+_description:
+ Run Firmware Test Suite (fwts) klog tests.
+_summary:
+ Run FWTS Kernel Log check
+environ: PLAINBOX_SESSION_SHARE
+command:
+ checkbox-support-fwts_test -l $PLAINBOX_SESSION_SHARE/fwts_klog_results.log -t oops
+
+plugin: attachment
+category_id: com.canonical.plainbox::miscellanea
+estimated_duration: 0.5
+id: miscellanea/klog_results.log
+command:
+ [ -e ${PLAINBOX_SESSION_SHARE}/fwts_klog_results.log ] && xz -c ${PLAINBOX_SESSION_SHARE}/fwts_klog_results.log
+_description: Attaches the FWTS klog results log to the submission
+
 plugin: shell
 category_id: com.canonical.plainbox::miscellanea
 estimated_duration: 0.1
diff --git a/units/miscellanea/test-plan.pxu b/units/miscellanea/test-plan.pxu
index 758386e..9c17b3a 100644
--- a/units/miscellanea/test-plan.pxu
+++ b/units/miscellanea/test-plan.pxu
@@ -53,3 +53,25 @@ include:
     miscellanea/oops_results.log
 bootstrap_include:
     fwts
+
+id: server-miscellaneous
+unit: test plan
+_name: Miscellaneous server cert tests
+_description:
+ Miscellaneous server tests (log checks, dmi data, etc)
+mandatory_include:
+    miscellanea/submission-resources
+    miscellanea/get_make_and_model
+    miscellanea/get_maas_version               certification-status=blocker
+    miscellanea/efi_boot_mode                  certification-status=blocker
+    miscellanea/secure_boot_mode
+    miscellanea/efi_pxeboot
+    miscellanea/check_prerelease
+    miscellanea/cpus_are_not_samples
+    miscellanea/ipmi_test                      certification-status=blocker
+    miscellanea/bmc_info
+    miscellanea/fwupdate
+    miscellanea/dmitest_server
+    miscellanea/sosreport
+    miscellanea/sosreport_attachment
+include: