Merge lp:~cypressyew/checkbox/add-before-s3-graphic-tests into lp:checkbox

[Po-Hsu Lin]

Add driver_version_check, compiz_check, video and glxgears tests to be tested after suspend as well.

They will be included in both the client-cert dev whitelist and 14.04 test plan.

I haven't modify the driver_version_after_suspend job to make it do the driver consistency check, for now we could use the output for reference, and improve the test case later, I will open a bug for it later.

[Po-Hsu Lin (cypressyew)]

Verified with 201211-12071, I+N systems, driver_version_check, compiz_check, video, glxgears tests after suspend works as expected

[Sylvain Pineau (sylvain-pineau)]

Thanks for working on this, +1

review: Approve

=== modified file 'providers/plainbox-provider-certification-client/units/client-cert-14-04.pxu'
--- providers/plainbox-provider-certification-client/units/client-cert-14-04.pxu	2015-05-18 06:25:15 +0000
+++ providers/plainbox-provider-certification-client/units/client-cert-14-04.pxu	2015-05-18 10:56:52 +0000
@@ -274,8 +274,12 @@
     suspend/2_suspend-single-log-attach_.*
     suspend/2_hybrid_sleep_.*                              certification-status=non-blocker
     suspend/2_hybrid-sleep-single-log-attach_.*
+    suspend/2_compiz_check_after_suspend_.*                certification-status=blocker
+    suspend/2_driver_version_after_suspend_.*              certification-status=blocker
     suspend/2_resolution_after_suspend_.*                  certification-status=blocker
     suspend/2_display_after_suspend_.*                     certification-status=blocker
+    suspend/2_glxgears_after_suspend_.*                    certification-status=blocker
+    suspend/2_video_after_suspend_.*                       certification-status=blocker
     suspend/2_cycle_resolutions_after_suspend_.*
     suspend/2_xrandr_screens_after_suspend_.*.tar.gz
     power-management/2_hibernate_advanced_.*               certification-status=non-blocker
@@ -289,8 +293,12 @@
     suspend/suspend-single-log-attach
     suspend/1_hybrid_sleep_.*                              certification-status=non-blocker
     suspend/1_hybrid-sleep-single-log-attach_.*
+    suspend/compiz_check_after_suspend                     certification-status=blocker
+    suspend/driver_version_after_suspend                   certification-status=blocker
     suspend/resolution_after_suspend                       certification-status=blocker
     suspend/display_after_suspend                          certification-status=blocker
+    suspend/glxgears_after_suspend                         certification-status=blocker
+    suspend/video_after_suspend                            certification-status=blocker
     suspend/cycle_resolutions_after_suspend
     suspend/xrandr_screens_after_suspend.tar.gz
     keys/sleep                                             certification-status=blocker

=== modified file 'providers/plainbox-provider-certification-client/whitelists/client-cert.whitelist'
--- providers/plainbox-provider-certification-client/whitelists/client-cert.whitelist	2015-05-18 06:25:15 +0000
+++ providers/plainbox-provider-certification-client/whitelists/client-cert.whitelist	2015-05-18 10:56:52 +0000
@@ -289,8 +289,12 @@
 suspend/2_suspend-single-log-attach_.*
 suspend/2_hybrid_sleep_.*
 suspend/2_hybrid-sleep-single-log-attach_.*
+suspend/2_compiz_check_after_suspend_.*
+suspend/2_driver_version_after_suspend_.*
 suspend/2_resolution_after_suspend_.*
 suspend/2_display_after_suspend_.*
+suspend/2_glxgears_after_suspend_.*
+suspend/2_video_after_suspend_.*
 suspend/2_cycle_resolutions_after_suspend_.*
 suspend/2_xrandr_screens_after_suspend.tar.gz
 power-management/2_hibernate_advanced_.*
@@ -304,8 +308,12 @@
 suspend/suspend-single-log-attach
 suspend/1_hybrid_sleep_.*
 suspend/1_hybrid-sleep-single-log-attach_.*
+suspend/compiz_check_after_suspend
+suspend/driver_version_after_suspend
 suspend/resolution_after_suspend
 suspend/display_after_suspend
+suspend/glxgears_after_suspend
+suspend/video_after_suspend
 suspend/cycle_resolutions_after_suspend
 suspend/xrandr_screens_after_suspend.tar.gz
 keys/sleep

=== modified file 'providers/plainbox-provider-checkbox/jobs/suspend.txt.in'
--- providers/plainbox-provider-checkbox/jobs/suspend.txt.in	2015-04-29 07:54:05 +0000
+++ providers/plainbox-provider-checkbox/jobs/suspend.txt.in	2015-05-18 10:56:52 +0000
@@ -2481,3 +2481,137 @@
       Did the cursor move?
  EOF
 
+id: suspend/generator_compiz_check_after_suspend
+requires: device.category == 'VIDEO'
+_description: Compiz support tests after suspend
+_summary: Compiz support tests after suspend
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: shell
+ id: suspend/`echo ${index}`_compiz_check_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: package.name == 'nux-tools'
+ command: ! /usr/lib/nux/unity_support_test -c -p 2>&1 | ansi_parser | grep -e ":\(\s\+\)no$" -ie "error"
+ estimated_duration: 0.130
+ _description: Check that $product hardware is able to run compiz after suspend
+ _summary: Test Compiz support for $product after suspend
+ EOF
+
+plugin: shell
+id: suspend/compiz_check_after_suspend
+depends: suspend/suspend_advanced
+requires: package.name == 'nux-tools'
+command: ! /usr/lib/nux/unity_support_test -c -p 2>&1 | ansi_parser | grep -e ":\(\s\+\)no$" -ie "error"
+estimated_duration: 0.130
+_description: Check that the hardware is able to run compiz after suspend
+_summary: Test Compiz support after suspend
+
+id: suspend/generator_glxgears_after_suspend
+requires: device.category == 'VIDEO'
+_description: glxgears tests after suspend
+_summary: glxgears tests after suspend
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: user-interact-verify
+ id: suspend/`echo ${index}`_glxgears_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: package.name == 'mesa-utils'
+ command: glxgears; true
+ _summary: Test that glxgears works for $product after suspend
+ _description:
+  PURPOSE:
+      This test tests the basic 3D capabilities of your $product video card after suspend
+  STEPS:
+      1. Click "Test" to execute an OpenGL demo. Press ESC at any time to close.
+      2. Verify that the animation is not jerky or slow.
+  VERIFICATION:
+      1. Did the 3d animation appear?
+      2. Was the animation free from slowness/jerkiness?
+ EOF
+
+plugin: user-interact-verify
+id: suspend/glxgears_after_suspend
+depends: suspend/suspend_advanced
+requires:
+ package.name == 'mesa-utils'
+ device.category == 'VIDEO'
+command: glxgears; true
+_summary: Test that glxgears works after suspend
+_description:
+ PURPOSE:
+     This test tests the basic 3D capabilities of your $product video card after suspend
+ STEPS:
+     1. Click "Test" to execute an OpenGL demo. Press ESC at any time to close.
+     2. Verify that the animation is not jerky or slow.
+ VERIFICATION:
+     1. Did the 3d animation appear?
+     2. Was the animation free from slowness/jerkiness?
+
+id: suspend/generator_video_after_suspend
+requires: device.category == 'VIDEO'
+_description: Display video test after suspend
+_summary: Display video test after suspend
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: suspend/`echo ${index}`_video_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ plugin: user-interact-verify
+ requires:
+  package.name == 'xorg'
+  package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0'
+ command: gst_pipeline_test -t 2 'videotestsrc ! videoconvert ! autovideosink' || gst_pipeline_test -t 2 'videotestsrc ! ffmpegcolorspace ! autovideosink'
+ _summary: Test that video can be displayed after suspend with ${product}
+ _description:
+  PURPOSE:
+      This test will test the default display after suspend with a sample video
+  STEPS:
+      1. Click "Test" to display a video test.
+  VERIFICATION:
+      Do you see color bars and static?
+ EOF
+
+id: suspend/video_after_suspend
+plugin: user-interact-verify
+depends: suspend/suspend_advanced
+requires:
+ device.category == 'VIDEO'
+ package.name == 'xorg'
+ package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0'
+command: gst_pipeline_test -t 2 'videotestsrc ! videoconvert ! autovideosink' || gst_pipeline_test -t 2 'videotestsrc ! ffmpegcolorspace ! autovideosink'
+_summary: Test that video can be displayed after suspend
+_description:
+ PURPOSE:
+     This test will test the default display after suspend with a sample video
+ STEPS:
+     1. Click "Test" to display a video test.
+ VERIFICATION:
+     Do you see color bars and static?
+
+id: suspend/generator_driver_version_after_suspend
+requires: device.category == 'VIDEO'
+_description: Driver version tests after suspend
+_summary: Driver version tests after suspend
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: shell
+ id: suspend/`echo ${index}`_driver_version_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ command: graphics_driver
+ estimated_duration: 0.500
+ _description: Parses Xorg.0.Log and discovers the running X driver and version after suspend for the $product graphics card
+ _summary: Test X driver/version after suspend for $product
+ EOF
+
+id: suspend/driver_version_after_suspend
+requires: device.category == 'VIDEO'
+depends: suspend/suspend_advanced
+plugin: shell
+command: graphics_driver
+estimated_duration: 0.500
+_description: Parses Xorg.0.Log and discovers the running X driver and version after suspend
+_summary: Test X driver/version after suspend
+