Merge lp:~cypressyew/checkbox/fwts-generator-fix into lp:checkbox

[Po-Hsu Lin]

* Move the uefirtvariable test from the QA test list to NON-CERT list, as we will have a dedicated one, so we won't test it again in the fwts_destop_diagnosis test.

* Move the out-of-list tests (wakealarm... etc) back to the NON-CERT list. This is essential for the generator, otherwise this job won't be generated as it can't be seem from fwts_test --list output
(BTW, we will still test the wakealarm test twice, in power-management/ and firmware/)

* Rename the fwts local job with generator_ prefix.

For the fix in test plan, I will paste the link to the bug later.

[Zygmunt Krynicki (zyga)]

+1a

review: Approve

[Sylvain Pineau (sylvain-pineau)]

The attempt to merge lp:~cypressyew/checkbox/fwts-generator-fix into lp:checkbox failed. Below is the output from the failed tests.

[precise] starting container
[precise] (timing) 0.09user 0.08system 0:05.33elapsed 3%CPU (0avgtext+0avgdata 10200maxresident)k
[precise] (timing) 0inputs+32outputs (0major+8707minor)pagefaults 0swaps
[precise] provisioning container
[precise] (timing) 40.44user 10.63system 1:05.45elapsed 78%CPU (0avgtext+0avgdata 94880maxresident)k
[precise] (timing) 0inputs+19080outputs (0major+1590382minor)pagefaults 0swaps
[precise-testing] Starting tests...
Found a test script: ./checkbox-ng/requirements/container-tests-checkbox-ng-unit
[precise-testing] container-tests-checkbox-ng-unit: PASS
[precise-testing] (timing) 0.90user 0.19system 0:01.18elapsed 92%CPU (0avgtext+0avgdata 50420maxresident)k
[precise-testing] (timing) 0inputs+1976outputs (0major+27764minor)pagefaults 0swaps
Found a test script: ./checkbox-support/requirements/container-tests-checkbox-support
[precise-testing] container-tests-checkbox-support: PASS
[precise-testing] (timing) 25.12user 0.20system 0:25.35elapsed 99%CPU (0avgtext+0avgdata 150464maxresident)k
[precise-testing] (timing) 0inputs+1320outputs (0major+40387minor)pagefaults 0swaps
Found a test script: ./checkbox-touch/requirements/container-tests-touch-unit-tests
[precise-testing] container-tests-touch-unit-tests: PASS
[precise-testing] (timing) 0.01user 0.01system 0:00.03elapsed 77%CPU (0avgtext+0avgdata 2176maxresident)k
[precise-testing] (timing) 0inputs+8outputs (0major+2485minor)pagefaults 0swaps
Found a test script: ./plainbox/plainbox/impl/providers/categories/requirements/container-tests-provider-categories
[precise-testing] container-tests-provider-categories: PASS
[precise-testing] (timing) 0.99user 0.08system 0:01.10elapsed 98%CPU (0avgtext+0avgdata 45820maxresident)k
[precise-testing] (timing) 0inputs+64outputs (0major+14259minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/001-container-tests-plainbox-egg-info
[precise-testing] 001-container-tests-plainbox-egg-info: PASS
[precise-testing] (timing) 0.22user 0.06system 0:00.29elapsed 96%CPU (0avgtext+0avgdata 14500maxresident)k
[precise-testing] (timing) 0inputs+96outputs (0major+6171minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox
[precise-testing] container-tests-plainbox: PASS
[precise-testing] (timing) 53.93user 1.40system 0:55.53elapsed 99%CPU (0avgtext+0avgdata 196548maxresident)k
[precise-testing] (timing) 0inputs+3256outputs (0major+199495minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox-documentation
[precise-testing] container-tests-plainbox-documentation: PASS
[precise-testing] (timing) 149.84user 0.64system 2:30.76elapsed 99%CPU (0avgtext+0avgdata 176888maxresident)k
[precise-testing] (timing) 0inputs+39856outputs (0major+53681minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox-integration
[precise-testing] container-tests-plainbox-integration: PASS
[precise-testing] ...

[Sylvain Pineau (sylvain-pineau)]

The attempt to merge lp:~cypressyew/checkbox/fwts-generator-fix into lp:checkbox failed. Below is the output from the failed tests.

[precise] starting container
[precise] (timing) 0.08user 0.07system 0:05.31elapsed 3%CPU (0avgtext+0avgdata 10196maxresident)k
[precise] (timing) 0inputs+32outputs (0major+8743minor)pagefaults 0swaps
[precise] provisioning container
[precise] (timing) 41.17user 10.47system 0:59.68elapsed 86%CPU (0avgtext+0avgdata 94892maxresident)k
[precise] (timing) 0inputs+19072outputs (0major+1591357minor)pagefaults 0swaps
[precise-testing] Starting tests...
Found a test script: ./checkbox-ng/requirements/container-tests-checkbox-ng-unit
[precise-testing] container-tests-checkbox-ng-unit: PASS
[precise-testing] (timing) 0.93user 0.16system 0:01.19elapsed 92%CPU (0avgtext+0avgdata 50420maxresident)k
[precise-testing] (timing) 0inputs+1976outputs (0major+27764minor)pagefaults 0swaps
Found a test script: ./checkbox-support/requirements/container-tests-checkbox-support
[precise-testing] container-tests-checkbox-support: PASS
[precise-testing] (timing) 25.83user 0.25system 0:26.19elapsed 99%CPU (0avgtext+0avgdata 150468maxresident)k
[precise-testing] (timing) 0inputs+1320outputs (0major+40417minor)pagefaults 0swaps
Found a test script: ./checkbox-touch/requirements/container-tests-touch-unit-tests
[precise-testing] container-tests-touch-unit-tests: PASS
[precise-testing] (timing) 0.01user 0.01system 0:00.03elapsed 77%CPU (0avgtext+0avgdata 2176maxresident)k
[precise-testing] (timing) 0inputs+8outputs (0major+2488minor)pagefaults 0swaps
Found a test script: ./plainbox/plainbox/impl/providers/categories/requirements/container-tests-provider-categories
[precise-testing] container-tests-provider-categories: PASS
[precise-testing] (timing) 1.01user 0.11system 0:01.13elapsed 98%CPU (0avgtext+0avgdata 45832maxresident)k
[precise-testing] (timing) 0inputs+64outputs (0major+14264minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/001-container-tests-plainbox-egg-info
[precise-testing] 001-container-tests-plainbox-egg-info: PASS
[precise-testing] (timing) 0.24user 0.04system 0:00.29elapsed 96%CPU (0avgtext+0avgdata 14500maxresident)k
[precise-testing] (timing) 0inputs+96outputs (0major+6170minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox
[precise-testing] container-tests-plainbox: PASS
[precise-testing] (timing) 53.11user 1.35system 0:54.63elapsed 99%CPU (0avgtext+0avgdata 196568maxresident)k
[precise-testing] (timing) 0inputs+3256outputs (0major+195897minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox-documentation
[precise-testing] container-tests-plainbox-documentation: PASS
[precise-testing] (timing) 148.88user 0.58system 2:29.75elapsed 99%CPU (0avgtext+0avgdata 176860maxresident)k
[precise-testing] (timing) 0inputs+39856outputs (0major+53344minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox-integration
[precise-testing] container-tests-plainbox-integration: PASS
[precise-testing] ...

[Sylvain Pineau (sylvain-pineau)]

The attempt to merge lp:~cypressyew/checkbox/fwts-generator-fix into lp:checkbox failed. Below is the output from the failed tests.

[precise] starting container
[precise] (timing) 0.08user 0.07system 0:04.31elapsed 3%CPU (0avgtext+0avgdata 10200maxresident)k
[precise] (timing) 0inputs+32outputs (0major+8260minor)pagefaults 0swaps
[precise] provisioning container
[precise] (timing) 40.25user 10.52system 0:58.81elapsed 86%CPU (0avgtext+0avgdata 94848maxresident)k
[precise] (timing) 0inputs+19088outputs (0major+1591072minor)pagefaults 0swaps
[precise-testing] Starting tests...
Found a test script: ./checkbox-ng/requirements/container-tests-checkbox-ng-unit
[precise-testing] container-tests-checkbox-ng-unit: PASS
[precise-testing] (timing) 0.89user 0.20system 0:01.18elapsed 92%CPU (0avgtext+0avgdata 50420maxresident)k
[precise-testing] (timing) 0inputs+1976outputs (0major+27764minor)pagefaults 0swaps
Found a test script: ./checkbox-support/requirements/container-tests-checkbox-support
[precise-testing] container-tests-checkbox-support: PASS
[precise-testing] (timing) 25.23user 0.20system 0:25.47elapsed 99%CPU (0avgtext+0avgdata 150408maxresident)k
[precise-testing] (timing) 0inputs+1320outputs (0major+40418minor)pagefaults 0swaps
Found a test script: ./checkbox-touch/requirements/container-tests-touch-unit-tests
[precise-testing] container-tests-touch-unit-tests: PASS
[precise-testing] (timing) 0.01user 0.01system 0:00.03elapsed 78%CPU (0avgtext+0avgdata 2180maxresident)k
[precise-testing] (timing) 0inputs+8outputs (0major+2487minor)pagefaults 0swaps
Found a test script: ./plainbox/plainbox/impl/providers/categories/requirements/container-tests-provider-categories
[precise-testing] container-tests-provider-categories: PASS
[precise-testing] (timing) 0.98user 0.12system 0:01.12elapsed 98%CPU (0avgtext+0avgdata 45828maxresident)k
[precise-testing] (timing) 0inputs+64outputs (0major+14262minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/001-container-tests-plainbox-egg-info
[precise-testing] 001-container-tests-plainbox-egg-info: PASS
[precise-testing] (timing) 0.23user 0.06system 0:00.30elapsed 96%CPU (0avgtext+0avgdata 14508maxresident)k
[precise-testing] (timing) 0inputs+96outputs (0major+6173minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox
[precise-testing] container-tests-plainbox: PASS
[precise-testing] (timing) 54.26user 1.41system 0:56.13elapsed 99%CPU (0avgtext+0avgdata 196580maxresident)k
[precise-testing] (timing) 0inputs+3256outputs (0major+199498minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox-documentation
[precise-testing] container-tests-plainbox-documentation: PASS
[precise-testing] (timing) 150.89user 0.67system 2:32.11elapsed 99%CPU (0avgtext+0avgdata 176832maxresident)k
[precise-testing] (timing) 0inputs+39856outputs (0major+52827minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox-integration
[precise-testing] container-tests-plainbox-integration: PASS
[precise-testing] ...

[Sylvain Pineau (sylvain-pineau)]

The attempt to merge lp:~cypressyew/checkbox/fwts-generator-fix into lp:checkbox failed. Below is the output from the failed tests.

[precise] starting container
[precise] (timing) 0.08user 0.08system 0:04.31elapsed 3%CPU (0avgtext+0avgdata 10196maxresident)k
[precise] (timing) 0inputs+32outputs (0major+8231minor)pagefaults 0swaps
[precise] provisioning container
[precise] (timing) 40.46user 10.38system 1:05.05elapsed 78%CPU (0avgtext+0avgdata 94848maxresident)k
[precise] (timing) 0inputs+19088outputs (0major+1591268minor)pagefaults 0swaps
[precise-testing] Starting tests...
Found a test script: ./checkbox-ng/requirements/container-tests-checkbox-ng-unit
[precise-testing] container-tests-checkbox-ng-unit: PASS
[precise-testing] (timing) 0.94user 0.16system 0:01.21elapsed 91%CPU (0avgtext+0avgdata 50416maxresident)k
[precise-testing] (timing) 0inputs+1976outputs (0major+27783minor)pagefaults 0swaps
Found a test script: ./checkbox-support/requirements/container-tests-checkbox-support
[precise-testing] container-tests-checkbox-support: PASS
[precise-testing] (timing) 25.64user 0.19system 0:25.89elapsed 99%CPU (0avgtext+0avgdata 150400maxresident)k
[precise-testing] (timing) 0inputs+1320outputs (0major+39902minor)pagefaults 0swaps
Found a test script: ./checkbox-touch/requirements/container-tests-touch-unit-tests
[precise-testing] container-tests-touch-unit-tests: PASS
[precise-testing] (timing) 0.00user 0.02system 0:00.03elapsed 78%CPU (0avgtext+0avgdata 2184maxresident)k
[precise-testing] (timing) 0inputs+8outputs (0major+2488minor)pagefaults 0swaps
Found a test script: ./plainbox/plainbox/impl/providers/categories/requirements/container-tests-provider-categories
[precise-testing] container-tests-provider-categories: PASS
[precise-testing] (timing) 1.02user 0.07system 0:01.11elapsed 98%CPU (0avgtext+0avgdata 45824maxresident)k
[precise-testing] (timing) 0inputs+64outputs (0major+14259minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/001-container-tests-plainbox-egg-info
[precise-testing] 001-container-tests-plainbox-egg-info: PASS
[precise-testing] (timing) 0.25user 0.03system 0:00.29elapsed 96%CPU (0avgtext+0avgdata 14504maxresident)k
[precise-testing] (timing) 0inputs+96outputs (0major+6170minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox
[precise-testing] container-tests-plainbox: PASS
[precise-testing] (timing) 54.76user 1.34system 0:56.28elapsed 99%CPU (0avgtext+0avgdata 196584maxresident)k
[precise-testing] (timing) 0inputs+3256outputs (0major+199526minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox-documentation
[precise-testing] container-tests-plainbox-documentation: PASS
[precise-testing] (timing) 147.40user 0.78system 2:28.49elapsed 99%CPU (0avgtext+0avgdata 176900maxresident)k
[precise-testing] (timing) 0inputs+39856outputs (0major+53173minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox-integration
[precise-testing] container-tests-plainbox-integration: PASS
[precise-testing] ...

=== modified file 'providers/plainbox-provider-checkbox/bin/fwts_test'
--- providers/plainbox-provider-checkbox/bin/fwts_test	2015-09-01 09:45:56 +0000
+++ providers/plainbox-provider-checkbox/bin/fwts_test	2015-10-07 08:59:47 +0000
@@ -30,11 +30,11 @@
             'mpcheck',
             'msr',
             'nx',
-            'uefirtvariable',
             'version']
 # These are advanced tests that shouldn't affect certification status
 NON_CERT_TESTS = ['bios_info',
                   'cmosdump',
+                  'cpufreq',
                   'crs',
                   'crsdump',
                   'csm',
@@ -59,7 +59,9 @@
                   'uefidump',
                   'uefirtmisc',
                   'uefirttime',
-                  'uefivarinfo'
+                  'uefirtvariable',
+                  'uefivarinfo',
+                  'wakealarm'
                   ]
 # The following tests will record logs in a separate file for the HWE team
 HWE_TESTS = ['mtrr',
@@ -295,7 +297,7 @@
     elif args.test:
         tests.extend(args.test)
     elif args.all:
-        tests.extend(['wakealarm', 'cpufreq', 'maxfreq'] + TESTS)
+        tests.extend(TESTS)
     elif args.hwe:
         tests.extend(HWE_TESTS)
     elif args.qa:

=== modified file 'providers/plainbox-provider-checkbox/jobs/firmware.txt.in'
--- providers/plainbox-provider-checkbox/jobs/firmware.txt.in	2015-09-08 22:04:46 +0000
+++ providers/plainbox-provider-checkbox/jobs/firmware.txt.in	2015-10-07 08:59:47 +0000
@@ -1,6 +1,6 @@
 plugin: local
 _summary: Automated tests for firmware using Firmware Test Suite.
-id: firmware/fwts
+id: firmware/generator_fwts
 estimated_duration: 0.5
 requires: package.name == 'fwts'
 _description: Automated tests for firmware using Firmware Test Suite.
@@ -19,7 +19,7 @@
 
 plugin: local
 _summary: Automated tests for firmware using Firmware Test Suite.
-id: firmware/fwts_logs
+id: firmware/generator_fwts_logs
 estimated_duration: 0.5
 requires: package.name == 'fwts'
 _description: Automated tests for firmware using Firmware Test Suite.