Merge lp:~cypressyew/checkbox/solutionD into lp:checkbox

[Po-Hsu Lin]

* Fix missing jobs for hybrid graphic system by using local jobs

* Don't use local jobs for suspend-related test on 1st card, to solve the dependency issue on suspend_advanced job.

You can find the dependency chart here:
https://launchpadlibrarian.net/204221329/SolutionD.html
(Chart updated on Apr.24, removing dependency between 1_graphic_switch and suspend_advanced job, see comment for more information)

Possible issue:
* Those non-graphic related tests after suspend (Bluetooth, wireless, media card... etc) will be tested after hibernate on a hybrid graphic system, not sure if it's acceptable.
* For selecting suspend related job on hybrid graphic system will be a bit more complicated, because they are not grouped together anymore (e.g. you will need to select the 2_resolution_before_suspend for 2nd card, and resolution_before_suspend for 1st card), screenshot could be found here: http://i.imgur.com/5NcUgXV.png

[Sylvain Pineau (sylvain-pineau)]

From your D diagram, suspend/suspend_advanced should depend on graphics/1_switch_card. Otherwise this suspend could come well before what's expected.

The suspend.txt file is easier to review here (direct link to the line with suspend_advanced): http://bazaar.launchpad.net/~cypressyew/checkbox/solutionD/view/head:/providers/plainbox-provider-checkbox/jobs/suspend.txt.in#L251

Could you confirm that it's a job dependency missing?

review: Needs Information

[Po-Hsu Lin (cypressyew)]

Hello,
Yes it's a missing dependency, ideally it should depend on that job to make sure it's running on the correct card.
However, for using that dependency in suspend/suspend_advanced, it will be written as:
depneds: graphics/1_switch_card_*
But this kind of expression is only valid in local jobs.
I wonder if there is any other way to do this.

I updated the dependency chart, basically just removed the dependency between these two jobs.
https://launchpadlibrarian.net/204221329/SolutionD.html

[Sylvain Pineau (sylvain-pineau)]

ok, thanks for the confirmation.

This is an incredible work, thanks a lot for having initiated the fix and for the detailed diagrams.
+1

review: Approve

=== modified file 'providers/plainbox-provider-certification-client/units/client-cert-14-04.pxu'
--- providers/plainbox-provider-certification-client/units/client-cert-14-04.pxu	2015-03-10 11:08:12 +0000
+++ providers/plainbox-provider-certification-client/units/client-cert-14-04.pxu	2015-04-23 06:37:30 +0000
@@ -31,6 +31,7 @@
     xinput
     # Tests grouped by categories. Categories start with '__'
     __audio__
+    audio/genereator_.*
     audio/alsa_info_collect
     audio/alsa_info_attachment
     audio/speaker-headphone-plug-detection                 certification-status=blocker
@@ -40,9 +41,8 @@
     audio/list_devices
     audio/playback_auto                                    certification-status=blocker
     audio/playback_headphones                              certification-status=blocker
-    audio/playback_hdmi                                    certification-status=blocker
-    audio/playback_displayport                             certification-status=blocker
     audio/alsa_record_playback_automated                   certification-status=blocker
+    # See Monitor / Graphic test section for hybrid-graphic monitor audio tests
     __benchmarks__
     # See Automated Tests section at the end of this file
     __bluetooth__
@@ -77,20 +77,34 @@
     firewire/remove                                        certification-status=blocker
     __firmware__
     # See Automated Tests section at the end of this file
+    __monitor__
+    monitor/generator_.*
+    monitor/1_powersaving_.*                               certification-status=blocker
+    monitor/1_dim_brightness_.*                            certification-status=blocker
+    monitor/1_displayport_.*                               certification-status=blocker
+    audio/1_playback_displayport_.*                        certification-status=blocker
+    monitor/1_dvi_.*                                       certification-status=blocker
+    monitor/1_hdmi_.*                                      certification-status=blocker
+    audio/1_playback_hdmi_.*                               certification-status=blocker
+    monitor/1_rca_.*                                       certification-status=blocker
+    monitor/1_svideo_.*                                    certification-status=blocker
+    monitor/1_vga_.*                                       certification-status=blocker
+    monitor/1_multi-head_.*                                certification-status=blocker
+    # See Graphic Tests section for monitor test on second graphic card
     __graphics__
     graphics/generator_.*
+    graphics/xorg-version                                  certification-status=blocker
+    graphics/xorg-failsafe                                 certification-status=blocker
+    graphics/xorg-process                                  certification-status=blocker
+    graphics/VESA_drivers_not_in_use                       certification-status=blocker
     graphics/1_maximum_resolution_.*                       certification-status=blocker
     graphics/1_glxgears_.*                                 certification-status=blocker
-    graphics/xorg-version                                  certification-status=blocker
     graphics/1_driver_version_.*                           certification-status=blocker
     graphics/1_compiz_check_.*                             certification-status=blocker
     graphics/1_rotation_.*                                 certification-status=blocker
     graphics/1_display_.*                                  certification-status=blocker
     graphics/1_minimum_resolution_.*
-    graphics/xorg-failsafe                                 certification-status=blocker
-    graphics/xorg-process                                  certification-status=blocker
     graphics/1_cycle_resolution_.*
-    graphics/VESA_drivers_not_in_use                       certification-status=blocker
     graphics/2_switch_card_.*                              certification-status=blocker
     graphics/2_maximum_resolution_.*                       certification-status=blocker
     graphics/2_glxgears_.*                                 certification-status=blocker
@@ -100,6 +114,17 @@
     graphics/2_display_.*                                  certification-status=blocker
     graphics/2_minimum_resolution_.*
     graphics/2_cycle_resolution_.*
+    monitor/2_powersaving_.*                               certification-status=blocker
+    monitor/2_dim_brightness_.*                            certification-status=blocker
+    monitor/2_displayport_.*                               certification-status=blocker
+    audio/2_playback_displayport_.*                        certification-status=blocker
+    monitor/2_dvi_.*                                       certification-status=blocker
+    monitor/2_hdmi_.*                                      certification-status=blocker
+    audio/2_playback_hdmi_.*                               certification-status=blocker
+    monitor/2_rca_.*                                       certification-status=blocker
+    monitor/2_svideo_.*                                    certification-status=blocker
+    monitor/2_vga_.*                                       certification-status=blocker
+    monitor/2_multi-head_.*                                certification-status=blocker
     __hibernate__
     #See below, power-management/hibernate_advanced
     __input__
@@ -157,16 +182,6 @@
     miscellanea/oops                                       certification-status=blocker
     miscellanea/oops_results.log
     # See Automated Tests section at the end of this file
-    __monitor__
-    monitor/powersaving                                    certification-status=blocker
-    monitor/dim_brightness                                 certification-status=blocker
-    monitor/displayport                                    certification-status=blocker
-    monitor/dvi                                            certification-status=blocker
-    monitor/hdmi                                           certification-status=blocker
-    monitor/rca                                            certification-status=blocker
-    monitor/svideo                                         certification-status=blocker
-    monitor/vga                                            certification-status=blocker
-    monitor/multi-head                                     certification-status=blocker
     __mobilebroadband__
     mobilebroadband/gsm_connection                         certification-status=non-blocker
     mobilebroadband/cdma_connection                        certification-status=non-blocker
@@ -192,6 +207,7 @@
     optical/bluray-write_.*                                certification-status=blocker
     __power-management__
     # See Automated Tests section at the end of this file
+    power-management/generator_.*
     power-management/lid                                   certification-status=blocker
     power-management/lid_close                             certification-status=blocker
     power-management/lid_open                              certification-status=blocker
@@ -242,38 +258,52 @@
     wireless/wireless_connection_wpa_ac                    certification-status=blocker
     wireless/wireless_connection_open_ac                   certification-status=blocker
     __suspend__
+    suspend/generator_.*
     suspend/network_before_suspend                         certification-status=blocker
-    suspend/resolution_before_suspend                      certification-status=blocker
     suspend/audio_before_suspend                           certification-status=blocker
     suspend/cpu_before_suspend                             certification-status=blocker
     suspend/memory_before_suspend                          certification-status=blocker
     suspend/bluetooth_obex_send_before_suspend             certification-status=blocker
     suspend/bluetooth_obex_browse_before_suspend           certification-status=blocker
     suspend/bluetooth_obex_get_before_suspend              certification-status=blocker
+    # Test card 2 first, if present, since it's the one we will be using
+    # after coming back from suspend if the system has hybrid graphics.
+    suspend/2_resolution_before_suspend_.*                 certification-status=blocker
+    suspend/2_suspend_after_switch_to_card_.*              certification-status=blocker
+    suspend/2_suspend-time-check_.*                        certification-status=non-blocker
+    suspend/2_suspend-single-log-attach_.*
+    suspend/2_hybrid_sleep_.*                              certification-status=non-blocker
+    suspend/2_hybrid-sleep-single-log-attach_.*
+    suspend/2_resolution_after_suspend_.*                  certification-status=blocker
+    suspend/2_display_after_suspend_.*                     certification-status=blocker
+    suspend/2_cycle_resolutions_after_suspend_.*           certification-status=blocker
+    suspend/2_xrandr_screens_after_suspend_.*.tar.gz
+    power-management/2_hibernate_advanced_.*               certification-status=non-blocker
+    power-management/2_hibernate-single-log-attach_.*
+    # Now we ask to switch to the first graphics card. This requires a restart
+    # of checkbox.
+    graphics/1_switch_card_.*                              certification-status=blocker
+    suspend/resolution_before_suspend                      certification-status=blocker
     suspend/suspend_advanced                               certification-status=blocker
     suspend/suspend-time-check                             certification-status=non-blocker
     suspend/suspend-single-log-attach
-    suspend/hybrid_sleep                                   certification-status=non-blocker
-    suspend/hybrid-sleep-single-log-attach
+    suspend/1_hybrid_sleep_.*                              certification-status=non-blocker
+    suspend/1_hybrid-sleep-single-log-attach_.*
+    suspend/resolution_after_suspend                       certification-status=blocker
+    suspend/display_after_suspend                          certification-status=blocker
+    suspend/cycle_resolutions_after_suspend                certification-status=blocker
+    suspend/xrandr_screens_after_suspend.tar.gz
     keys/sleep                                             certification-status=blocker
     led/power-blink-suspend                                certification-status=blocker
     led/suspend                                            certification-status=blocker
     suspend/network_after_suspend                          certification-status=blocker
-    suspend/resolution_after_suspend                       certification-status=blocker
     suspend/audio_after_suspend                            certification-status=blocker
     suspend/cpu_after_suspend                              certification-status=blocker
     suspend/memory_after_suspend                           certification-status=blocker
-    suspend/generator_display_after_suspend
-    suspend/generator_suspend_after_switch_to_card
-    # Test card 2 first, if present, since it's the one we will be using
-    # after coming back from suspend if the system has hybrid graphics.
-    suspend/2_display_after_suspend.*                      certification-status=blocker
     suspend/bluetooth_detect_after_suspend                 certification-status=blocker
     suspend/bluetooth_obex_send_after_suspend              certification-status=blocker
     suspend/bluetooth_obex_browse_after_suspend            certification-status=blocker
     suspend/bluetooth_obex_get_after_suspend               certification-status=blocker
-    suspend/cycle_resolutions_after_suspend                certification-status=blocker
-    suspend/xrandr_screens_after_suspend.tar.gz
     suspend/record_playback_after_suspend                  certification-status=blocker
     suspend/pointing-after-suspend
     suspend/pointing-after-suspend_.*                      certification-status=blocker
@@ -298,14 +328,8 @@
     suspend/wireless_connection_after_suspend_open_n       certification-status=blocker
     suspend/wireless_connection_after_suspend_wpa_ac       certification-status=blocker
     suspend/wireless_connection_after_suspend_open_ac      certification-status=blocker
-    # Now we ask to switch to the first graphics card. This requires a restart
-    # of checkbox. If the system only has one card, the test instructions
-    # should tell the user to just do nothing.
-    graphics/1_switch_card_.*                              certification-status=blocker
-    suspend/1_suspend_after_switch_to_card_.*              certification-status=blocker
-    suspend/1_display_after_suspend.*                      certification-status=blocker
-    power-management/hibernate_advanced                    certification-status=non-blocker
-    power-management/hibernate-single-log-attach
+    power-management/1_hibernate_advanced_.*               certification-status=non-blocker
+    power-management/1_hibernate-single-log-attach._*
     keys/hibernate                                         certification-status=blocker
     # The following tests should run BEFORE the automated tests. The reboot and
     # power off tests will also give us a clean system to start the stress run

=== modified file 'providers/plainbox-provider-certification-client/whitelists/client-cert.whitelist'
--- providers/plainbox-provider-certification-client/whitelists/client-cert.whitelist	2015-04-16 07:29:03 +0000
+++ providers/plainbox-provider-certification-client/whitelists/client-cert.whitelist	2015-04-23 06:37:30 +0000
@@ -36,6 +36,7 @@
 xinput
 # Tests grouped by categories. Categories start with '__'
 __audio__
+audio/genereator_.*
 audio/alsa_info_collect
 audio/alsa_info_attachment
 audio/speaker-headphone-plug-detection
@@ -45,10 +46,8 @@
 audio/list_devices
 audio/playback_auto
 audio/playback_headphones
-audio/playback_hdmi
-audio/playback_displayport
-audio/playback_thunderbolt
 audio/alsa_record_playback_automated
+# See Monitor / Graphic test section for hybrid-graphic monitor audio tests
 __benchmarks__
 # See Automated Tests section at the end of this file
 __bluetooth__
@@ -88,20 +87,37 @@
 # See the Thunderbolt monitor test for the daisy-chain test
 __firmware__
 # See Automated Tests section at the end of this file
+__monitor__
+monitor/generator_.*
+monitor/1_powersaving_.*
+monitor/1_dim_brightness_.*
+monitor/1_displayport_.*
+audio/1_playback_displayport_.*
+monitor/1_dvi_.*
+monitor/1_hdmi_.*
+audio/1_playback_hdmi_.*
+monitor/1_thunderbolt_.*
+audio/1_playback_thunderbolt_.*
+thunderbolt/daisy-chain
+monitor/1_rca_.*
+monitor/1_svideo_.*
+monitor/1_vga_.*
+monitor/1_multi-head_.*
+# See Graphic Tests section for monitor test on second graphic card
 __graphics__
 graphics/generator_.*
+graphics/xorg-version
+graphics/xorg-failsafe
+graphics/xorg-process
+graphics/VESA_drivers_not_in_use
 graphics/1_maximum_resolution_.*
 graphics/1_glxgears_.*
-graphics/xorg-version
 graphics/1_driver_version_.*
 graphics/1_compiz_check_.*
 graphics/1_rotation_.*
 graphics/1_display_.*
 graphics/1_minimum_resolution_.*
-graphics/xorg-failsafe
-graphics/xorg-process
 graphics/1_cycle_resolution_.*
-graphics/VESA_drivers_not_in_use
 graphics/2_switch_card_.*
 graphics/2_maximum_resolution_.*
 graphics/2_glxgears_.*
@@ -111,6 +127,19 @@
 graphics/2_display_.*
 graphics/2_minimum_resolution_.*
 graphics/2_cycle_resolution_.*
+monitor/2_powersaving_.*
+monitor/2_dim_brightness_.*
+monitor/2_displayport_.*
+audio/2_playback_displayport_.*
+monitor/2_dvi_.*
+monitor/2_hdmi_.*
+audio/2_playback_hdmi_.*
+monitor/2_thunderbolt_.*
+audio/2_playback_thunderbolt_.*
+monitor/2_rca_.*
+monitor/2_svideo_.*
+monitor/2_vga_.*
+monitor/2_multi-head_.*
 __hibernate__
 #See below, power-management/hibernate_advanced
 __input__
@@ -168,18 +197,6 @@
 miscellanea/oops
 miscellanea/oops_results.log
 # See Automated Tests section at the end of this file
-__monitor__
-monitor/powersaving
-monitor/dim_brightness
-monitor/displayport
-monitor/dvi
-monitor/hdmi
-monitor/rca
-monitor/svideo
-monitor/thunderbolt
-thunderbolt/daisy-chain
-monitor/vga
-monitor/multi-head
 __mobilebroadband__
 mobilebroadband/gsm_connection
 mobilebroadband/cdma_connection
@@ -205,6 +222,7 @@
 optical/bluray-write_.*
 __power-management__
 # See Automated Tests section at the end of this file
+power-management/generator_.*
 power-management/lid
 power-management/lid_close
 power-management/lid_open
@@ -255,38 +273,52 @@
 wireless/wireless_connection_wpa_ac
 wireless/wireless_connection_open_ac
 __suspend__
+suspend/generator_.*
 suspend/network_before_suspend
-suspend/resolution_before_suspend
 suspend/audio_before_suspend
 suspend/cpu_before_suspend
 suspend/memory_before_suspend
 suspend/bluetooth_obex_send_before_suspend
 suspend/bluetooth_obex_browse_before_suspend
 suspend/bluetooth_obex_get_before_suspend
+# Test card 2 first, if present, since it's the one we will be using
+# after coming back from suspend if the system has hybrid graphics.
+suspend/2_resolution_before_suspend_.*
+suspend/2_suspend_after_switch_to_card_.*
+suspend/2_suspend-time-check_.*
+suspend/2_suspend-single-log-attach_.*
+suspend/2_hybrid_sleep_.*
+suspend/2_hybrid-sleep-single-log-attach_.*
+suspend/2_resolution_after_suspend_.*
+suspend/2_display_after_suspend_.*
+suspend/2_cycle_resolutions_after_suspend_.*
+suspend/2_xrandr_screens_after_suspend.tar.gz
+power-management/2_hibernate_advanced_.*
+power-management/2_hibernate-single-log-attach_.*
+# Now we ask to switch to the first graphics card. This requires a restart
+# of checkbox.
+graphics/1_switch_card_.*
+suspend/resolution_before_suspend
 suspend/suspend_advanced
 suspend/suspend-time-check
 suspend/suspend-single-log-attach
-suspend/hybrid_sleep
-suspend/hybrid-sleep-single-log-attach
+suspend/1_hybrid_sleep_.*
+suspend/1_hybrid-sleep-single-log-attach_.*
+suspend/resolution_after_suspend
+suspend/display_after_suspend
+suspend/cycle_resolutions_after_suspend
+suspend/xrandr_screens_after_suspend.tar.gz
 keys/sleep
 led/power-blink-suspend
 led/suspend
 suspend/network_after_suspend
-suspend/resolution_after_suspend
 suspend/audio_after_suspend
 suspend/cpu_after_suspend
 suspend/memory_after_suspend
-suspend/generator_display_after_suspend
-suspend/generator_suspend_after_switch_to_card
-# Test card 2 first, if present, since it's the one we will be using
-# after coming back from suspend if the system has hybrid graphics.
-suspend/2_display_after_suspend.*
 suspend/bluetooth_detect_after_suspend
 suspend/bluetooth_obex_send_after_suspend
 suspend/bluetooth_obex_browse_after_suspend
 suspend/bluetooth_obex_get_after_suspend
-suspend/cycle_resolutions_after_suspend
-suspend/xrandr_screens_after_suspend.tar.gz
 suspend/record_playback_after_suspend
 suspend/pointing-after-suspend
 suspend/pointing-after-suspend_.*
@@ -311,12 +343,6 @@
 suspend/wireless_connection_after_suspend_open_n
 suspend/wireless_connection_after_suspend_wpa_ac
 suspend/wireless_connection_after_suspend_open_ac
-# Now we ask to switch to the first graphics card. This requires a restart
-# of checkbox. If the system only has one card, the test instructions should
-# tell the user to just do nothing.
-graphics/1_switch_card_.*
-suspend/1_suspend_after_switch_to_card_.*
-suspend/1_display_after_suspend.*
 power-management/hibernate_advanced
 power-management/hibernate-single-log-attach
 keys/hibernate

=== modified file 'providers/plainbox-provider-checkbox/jobs/audio.txt.in'
--- providers/plainbox-provider-checkbox/jobs/audio.txt.in	2015-04-15 03:29:00 +0000
+++ providers/plainbox-provider-checkbox/jobs/audio.txt.in	2015-04-23 06:37:30 +0000
@@ -34,57 +34,71 @@
  VERIFICATION:
      Did you hear a tone?
 
-plugin: user-interact-verify
-id: audio/playback_hdmi
-estimated_duration: 30.0
-requires:
- device.category == 'AUDIO'
- package.name == 'alsa-base'
- package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0'
- package.name == 'pulseaudio-utils'
-command:
- audio_settings store --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings
- audio_settings set --verbose --device=hdmi --volume=50
- gst_pipeline_test -t 2 --device hdmi 'audiotestsrc wave=sine freq=512 ! audioconvert ! audioresample ! autoaudiosink'
- EXIT_CODE=$?
- audio_settings restore --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings
- exit $EXIT_CODE
-_description:
- PURPOSE:
-     HDMI audio interface verification
- STEPS:
-     1. Plug an external HDMI device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test)
-     2. Click the Test button
- VERIFICATION:
-     Did you hear the sound from the HDMI device?
-
-plugin: user-interact-verify
-id: audio/playback_displayport
-estimated_duration: 30.0
-requires:
- device.category == 'AUDIO'
- package.name == 'alsa-base'
- package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0'
- package.name == 'pulseaudio-utils'
-command:
-  audio_settings store --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings
-  audio_settings set --verbose --device=hdmi --volume=50
-  gst_pipeline_test -t 2 --device hdmi 'audiotestsrc wave=sine freq=512 ! audioconvert ! audioresample ! autoaudiosink'
-  EXIT_CODE=$?
-  audio_settings restore --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings
-  exit $EXIT_CODE
-_description:
- PURPOSE:
-     DisplayPort audio interface verification
- STEPS:
-     1. Plug an external DisplayPort device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test)
-     2. Click the Test button
- VERIFICATION:
-     Did you hear the sound from the DisplayPort device?
-
-plugin: user-interact-verify
-id: audio/playback_thunderbolt
-estimated_duration: 30.0
+id: audio/generator_playback_hdmi
+requires:
+ device.category == 'AUDIO'
+ package.name == 'alsa-base'
+ package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0'
+ package.name == 'pulseaudio-utils'
+_description: HDMI audio test
+_summary: HDMI audio test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: audio/`echo ${index}`_playback_hdmi_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: display.hdmi == 'supported'
+ plugin: user-interact-verify
+ estimated_duration: 30.0
+ command:
+  audio_settings store --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings
+  audio_settings set --verbose --device=hdmi --volume=50
+  gst_pipeline_test -t 2 --device hdmi 'audiotestsrc wave=sine freq=512 ! audioconvert ! audioresample ! autoaudiosink'
+  EXIT_CODE=$?
+  audio_settings restore --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings
+  exit $EXIT_CODE
+ _description:
+  PURPOSE:
+      HDMI audio interface verification
+  STEPS:
+      1. Plug an external HDMI device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test)
+      2. Click the Test button
+  VERIFICATION:
+      Did you hear the sound from the HDMI device?
+ EOF
+
+id: audio/generator_playback_displayport
+requires:
+ device.category == 'AUDIO'
+ package.name == 'alsa-base'
+ package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0'
+ package.name == 'pulseaudio-utils'
+_description: DisplayPort audio test
+_summary: DisplayPort audio test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: audio/`echo ${index}`_playback_displayport_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: display.dp == 'supported'
+ plugin: user-interact-verify
+ estimated_duration: 30.0
+ command:
+  audio_settings store --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings
+  audio_settings set --verbose --device=hdmi --volume=50
+  gst_pipeline_test -t 2 --device hdmi 'audiotestsrc wave=sine freq=512 ! audioconvert ! audioresample ! autoaudiosink'
+  EXIT_CODE=$?
+  audio_settings restore --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings
+  exit $EXIT_CODE
+ _description:
+  PURPOSE:
+      DisplayPort audio interface verification
+  STEPS:
+      1. Plug an external DisplayPort device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test)
+      2. Click the Test button
+  VERIFICATION:
+      Did you hear the sound from the DisplayPort device?
+ EOF
+
+id: audio/generator_playback_thunderbolt
 imports: from 2013.com.canonical.plainbox import manifest
 requires:
  manifest.has_thunderbolt == 'True'
@@ -92,21 +106,30 @@
  package.name == 'alsa-base'
  package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0'
  package.name == 'pulseaudio-utils'
+_description: Thunderbolt audio test
+_summary: Thunderbolt audio test
+plugin: local
 command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: audio/`echo ${index}`_playback_thunderbolt_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ plugin: user-interact-verify
+ estimated_duration: 30.0
+ command:
   audio_settings store --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings
   audio_settings set --verbose --device=hdmi --volume=50
   gst_pipeline_test -t 2 --device hdmi 'audiotestsrc wave=sine freq=512 ! audioconvert ! audioresample ! autoaudiosink'
   EXIT_CODE=$?
   audio_settings restore --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings
   exit $EXIT_CODE
-_description:
- PURPOSE:
-     Thunderbolt audio interface verification
- STEPS:
-     1. Plug an external Thunderbolt device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test)
-     2. Click the Test button
- VERIFICATION:
-     Did you hear the sound from the Thunderbolt device?
+ _description:
+  PURPOSE:
+      Thunderbolt audio interface verification
+  STEPS:
+      1. Plug an external Thunderbolt device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test)
+      2. Click the Test button
+  VERIFICATION:
+      Did you hear the sound from the Thunderbolt device?
+ EOF
 
 plugin: user-interact-verify
 id: audio/playback_headphones

=== modified file 'providers/plainbox-provider-checkbox/jobs/graphics.txt.in'
--- providers/plainbox-provider-checkbox/jobs/graphics.txt.in	2015-04-16 14:23:08 +0000
+++ providers/plainbox-provider-checkbox/jobs/graphics.txt.in	2015-04-23 06:37:30 +0000
@@ -221,20 +221,28 @@
  _summary: Test graphic memory info for $product.
  EOF
 
-plugin: user-interact-verify
-id: graphics/display
-requires:
- package.name == 'xorg'
- package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0'
-command: gst_pipeline_test -t 2 'videotestsrc ! videoconvert ! autovideosink' || gst_pipeline_test -t 2 'videotestsrc ! ffmpegcolorspace ! autovideosink' 
-_summary: Test that video can be displayed
-_description:
- PURPOSE:
-     This test will test the default display
- STEPS:
-     1. Click "Test" to display a video test.
- VERIFICATION:
-     Do you see color bars and static?
+id: graphics/generator_display
+requires: device.category == 'VIDEO'
+_description: Display test
+_summary: Display test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: graphics/`echo ${index}`_display_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ plugin: user-interact-verify
+ requires:
+  package.name == 'xorg'
+  package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0'
+ command: gst_pipeline_test -t 2 'videotestsrc ! videoconvert ! autovideosink' || gst_pipeline_test -t 2 'videotestsrc ! ffmpegcolorspace ! autovideosink'
+ _summary: Test that video can be displayed with ${product}
+ _description:
+  PURPOSE:
+      This test will test the default display
+  STEPS:
+      1. Click "Test" to display a video test.
+  VERIFICATION:
+      Do you see color bars and static?
+ EOF
 
 plugin: shell
 id: graphics/VESA_drivers_not_in_use

=== modified file 'providers/plainbox-provider-checkbox/jobs/hibernate.txt.in'
--- providers/plainbox-provider-checkbox/jobs/hibernate.txt.in	2014-09-02 16:33:03 +0000
+++ providers/plainbox-provider-checkbox/jobs/hibernate.txt.in	2015-04-23 06:37:30 +0000
@@ -25,6 +25,40 @@
  VERIFICATION:
      Did the system successfully hibernate and did it work properly after waking up?
 
+id: power-management/generator_hibernate_advanced
+requires:
+  sleep.disk == 'supported'
+  rtc.state == 'supported'
+_description: Hibernate test
+_summary: Hibernate test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: user-interact-verify
+ id: power-management/`echo ${index}`_hibernate_advanced_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ user: root
+ environ: PLAINBOX_SESSION_SHARE
+ estimated_duration: 300.00
+ command:
+  if type -P fwts >/dev/null; then
+   echo "Calling fwts"
+   fwts_test -l $PLAINBOX_SESSION_SHARE/`echo ${index}`_hibernate-single -f none -s s4 --s4-device-check --s4-device-check-delay=45 --s4-sleep-delay=120
+  else
+   echo "Calling sleep_test"
+   sleep_test -s disk -w 120
+  fi
+ _description:
+  PURPOSE:
+      This test will check to make sure your system can successfully hibernate (if supported)
+  STEPS:
+      1. Click on Test
+      2. The system will hibernate and should wake itself within 5 minutes
+      3. If your system does not wake itself after 5 minutes, please press the power button to wake the system manually
+      4. If the system fails to resume from hibernate, please restart System Testing and mark this test as Failed
+  VERIFICATION:
+      Did the system successfully hibernate and did it work properly after waking up?
+ EOF
+
 plugin: shell
 id: power-management/hibernate-single-log-check
 command: [ -e $PLAINBOX_SESSION_SHARE/hibernate-single.log ] && sleep_test_log_check -v s4 $PLAINBOX_SESSION_SHARE/hibernate-single.log
@@ -33,7 +67,22 @@
 
 plugin: attachment
 id: power-management/hibernate-single-log-attach
+depends: power-management/hibernate_advanced
 command: [ -e $PLAINBOX_SESSION_SHARE/hibernate-single.log ] && cat $PLAINBOX_SESSION_SHARE/hibernate-single.log
 estimated_duration: 0.5
 _description: attaches log from single hibernate/resume test to results
 
+id: power-management/generator_hibernate-single-log-attach
+_description: Hibernate log attach
+_summary: Hibernate log attach
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: attachment
+ id: power-management/`echo ${index}`_hibernate-single-log-attach_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ depends: power-management/`echo ${index}`_hibernate_advanced_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ estimated_duration: 0.5
+ command: [ -e $PLAINBOX_SESSION_SHARE/`echo ${index}`_hibernate-single.log ] && cat $PLAINBOX_SESSION_SHARE/`echo ${index}`_hibernate-single.log
+ _description: attaches log from single hibernate/resume test to results
+ EOF
+

=== modified file 'providers/plainbox-provider-checkbox/jobs/led.txt.in'
--- providers/plainbox-provider-checkbox/jobs/led.txt.in	2014-07-28 15:17:23 +0000
+++ providers/plainbox-provider-checkbox/jobs/led.txt.in	2015-04-23 06:37:30 +0000
@@ -24,6 +24,7 @@
 plugin: manual
 id: led/suspend
 estimated_duration: 30.0
+depends: suspend/suspend_advanced
 _description:
  PURPOSE:
      Suspend LED verification.

=== modified file 'providers/plainbox-provider-checkbox/jobs/monitor.txt.in'
--- providers/plainbox-provider-checkbox/jobs/monitor.txt.in	2015-04-22 09:48:46 +0000
+++ providers/plainbox-provider-checkbox/jobs/monitor.txt.in	2015-04-23 06:37:30 +0000
@@ -1,50 +1,179 @@
-plugin: manual
-id: monitor/vga
-requires: display.vga == 'supported' or display.dp == 'supported'
-_description:
- PURPOSE:
-     This test will check your VGA port.
- STEPS:
-     Skip this test if your system does not have a VGA port.
-     1. Connect a display (if not already connected) to the VGA port on your system
- VERIFICATION:
-     Was the desktop displayed correctly on both screens?
-
-plugin: manual
-id: monitor/dvi
-requires: display.dvi == 'supported'
-_description:
- PURPOSE:
-     This test will check your DVI port.
- STEPS:
-     Skip this test if your system does not have a DVI port.
-     1. Connect a display (if not already connected) to the DVI port on your system
- VERIFICATION:
-     Was the desktop displayed correctly on both screens?
-
-plugin: manual
-id: monitor/displayport
-requires: display.dp == 'supported'
-_description:
- PURPOSE:
-     This test will check your DisplayPort port.
- STEPS:
-     Skip this test if your system does not have a DisplayPort port.
-     1. Connect a display (if not already connected) to the DisplayPort port on your system
- VERIFICATION:
-     Was the desktop displayed correctly on both screens?
-
-plugin: manual
-id: monitor/hdmi
-requires: display.hdmi == 'supported'
-_description:
- PURPOSE:
-     This test will check your HDMI port.
- STEPS:
-     Skip this test if your system does not have a HDMI port.
-     1. Connect a display (if not already connected) to the HDMI port on your system
- VERIFICATION:
-     Was the desktop displayed correctly on both screens?
+id: monitor/generator_vga
+_description: VGA port test
+_summary: VGA port test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: monitor/`echo ${index}`_vga_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: display.vga == 'supported' or display.dp == 'supported'
+ plugin: manual
+ _description:
+  PURPOSE:
+      This test will check your VGA port.
+  STEPS:
+      Skip this test if your system does not have a VGA port.
+      1. Connect a display (if not already connected) to the VGA port on your system
+  VERIFICATION:
+      Was the desktop displayed correctly on both screens?
+ EOF
+
+id: monitor/generator_dvi
+_description: DVI port test
+_summary: DVI port test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: monitor/`echo ${index}`_dvi_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: display.dvi == 'supported'
+ plugin: manual
+ _description:
+  PURPOSE:
+      This test will check your DVI port.
+  STEPS:
+      Skip this test if your system does not have a DVI port.
+      1. Connect a display (if not already connected) to the DVI port on your system
+  VERIFICATION:
+      Was the desktop displayed correctly on both screens?
+ EOF
+
+id: monitor/generator_displayport
+_description: DisplayPort test
+_summary: DisplayPort test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: monitor/`echo ${index}`_displayport_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: display.dp == 'supported'
+ plugin: manual
+ _description:
+  PURPOSE:
+      This test will check your DisplayPort port.
+  STEPS:
+      Skip this test if your system does not have a DisplayPort port.
+      1. Connect a display (if not already connected) to the DisplayPort port on your system
+  VERIFICATION:
+      Was the desktop displayed correctly on both screens?
+ EOF
+
+id: monitor/generator_hdmi
+_description: HDMI port test
+_summary: HDMI port test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: monitor/`echo ${index}`_hdmi_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: display.hdmi == 'supported'
+ plugin: manual
+ _description:
+  PURPOSE:
+      This test will check your HDMI port.
+  STEPS:
+      Skip this test if your system does not have a HDMI port.
+      1. Connect a display (if not already connected) to the HDMI port on your system
+  VERIFICATION:
+      Was the desktop displayed correctly on both screens?
+ EOF
+
+id: monitor/generator_svideo
+_description: S-VIDEO port test
+_summary: S-VIDEO port test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: monitor/`echo ${index}`_svideo_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: display.svideo == 'supported'
+ plugin: manual
+ _description:
+  PURPOSE:
+      This test will check your S-VIDEO port.
+  STEPS:
+      Skip this test if your system does not have a S-VIDEO port.
+      1. Connect a display (if not already connected) to the S-VIDEO port on your system
+  VERIFICATION:
+      Was the desktop displayed correctly on both screens?
+ EOF
+
+id: monitor/generator_rca
+_description: RCA port test
+_summary: RCA port test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: monitor/`echo ${index}`_rca_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: display.rca == 'supported'
+ plugin: manual
+ _description:
+  PURPOSE:
+      This test will check your RCA port.
+  STEPS:
+      Skip this test if your system does not have a RCA port.
+      1. Connect a display (if not already connected) to the RCA port on your system
+  VERIFICATION:
+      Was the desktop displayed correctly on both screens?
+ EOF
+
+id: monitor/generator_multi-head
+_description: Multi-monitor test
+_summary: Multi-monitor test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: monitor/`echo ${index}`_multi-head_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: dmi.product in ['Desktop','Low Profile Desktop','Tower','Mini Tower']
+ plugin: manual
+ _description:
+  PURPOSE:
+      This test verifies that multi-monitor output works on your desktop system. This is NOT the same test as the external monitor tests you would run on your laptop.  You will need two monitors to perform this test.
+  STEPS:
+      Skip this test if your video card does not support multiple monitors.
+      1. If your second monitor is not already connected, connect it now
+      2. Open the "Displays" tool (open the dash and search for "Displays")
+      3. Configure your output to provide one desktop across both monitors
+      4. Open any application and drag it from one monitor to the next.
+  VERIFICATION:
+      Was the stretched desktop displayed correctly across both screens?
+ EOF
+
+id: monitor/generator_powersaving
+_description: Monitor power-saving test
+_summary: Monitor power-saving test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: monitor/`echo ${index}`_powersaving_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ plugin: user-interact-verify
+ command: xset dpms force off
+ _description:
+  PURPOSE:
+      This test will check your monitor power saving capabilities
+  STEPS:
+      1. Click "Test" to try the power saving capabilities of your monitor
+      2. Press any key or move the mouse to recover
+  VERIFICATION:
+      Did the monitor go blank and turn on again?
+ EOF
+
+id: monitor/generator_dim_brightness
+_description: Screen dimming test
+_summary: Screen dimming test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ id: monitor/`echo ${index}`_dim_brightness_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: dmi.product in ['Notebook','Laptop','Portable','All In One','All-In-One','AIO']
+ plugin: user-interact-verify
+ user: root
+ command: brightness_test
+ _description:
+  PURPOSE:
+      This test will test changes to screen brightness
+  STEPS:
+      1. Click "Test" to try to dim the screen.
+      2. Check if the screen was dimmed approximately to half of the maximum brightness.
+      3. The screen will go back to the original brightness in 2 seconds.
+  VERIFICATION:
+      Was your screen dimmed approximately to half of the maximum brightness?
+ EOF
 
 plugin: manual
 id: monitor/thunderbolt
@@ -63,69 +192,3 @@
      Was the desktop displayed correctly on the Thunderbolt-connected screen in
      every mode?
 
-plugin: manual
-id: monitor/svideo
-requires: display.svideo == 'supported'
-_description:
- PURPOSE:
-     This test will check your S-VIDEO port.
- STEPS:
-     Skip this test if your system does not have a S-VIDEO port.
-     1. Connect a display (if not already connected) to the S-VIDEO port on your system
- VERIFICATION:
-     Was the desktop displayed correctly on both screens?
-
-plugin: manual
-id: monitor/rca
-requires: display.rca == 'supported'
-_description:
- PURPOSE:
-     This test will check your RCA port.
- STEPS:
-     Skip this test if your system does not have a RCA port.
-     1. Connect a display (if not already connected) to the RCA port on your system
- VERIFICATION:
-     Was the desktop displayed correctly on both screens?
-
-plugin: manual
-id: monitor/multi-head
-requires: dmi.product in ['Desktop','Low Profile Desktop','Tower','Mini Tower']
-_description:
- PURPOSE:
-     This test verifies that multi-monitor output works on your desktop system. This is NOT the same test as the external monitor tests you would run on your laptop.  You will need two monitors to perform this test.
- STEPS:
-     Skip this test if your video card does not support multiple monitors.
-     1. If your second monitor is not already connected, connect it now
-     2. Open the "Displays" tool (open the dash and search for "Displays")
-     3. Configure your output to provide one desktop across both monitors
-     4. Open any application and drag it from one monitor to the next.
- VERIFICATION:
-     Was the stretched desktop displayed correctly across both screens?
-
-plugin: user-interact-verify
-id: monitor/powersaving
-command: xset dpms force off
-_description:
- PURPOSE:
-     This test will check your monitor power saving capabilities
- STEPS:
-     1. Click "Test" to try the power saving capabilities of your monitor
-     2. Press any key or move the mouse to recover
- VERIFICATION:
-     Did the monitor go blank and turn on again?
-
-plugin: user-interact-verify
-id: monitor/dim_brightness
-requires: dmi.product in ['Notebook','Laptop','Portable','All In One','All-In-One','AIO']
-user: root
-command: brightness_test
-_description:
- PURPOSE:
-     This test will test changes to screen brightness
- STEPS:
-     1. Click "Test" to try to dim the screen.
-     2. Check if the screen was dimmed approximately to half of the maximum brightness.
-     3. The screen will go back to the original brightness in 2 seconds.
- VERIFICATION:
-     Was your screen dimmed approximately to half of the maximum brightness?
-

=== modified file 'providers/plainbox-provider-checkbox/jobs/suspend.txt.in'
--- providers/plainbox-provider-checkbox/jobs/suspend.txt.in	2015-01-26 15:38:31 +0000
+++ providers/plainbox-provider-checkbox/jobs/suspend.txt.in	2015-04-23 06:37:30 +0000
@@ -11,6 +11,20 @@
 _description: Record the current resolution before suspending.
 command: xrandr -q |grep '*'| awk '{print $1}' > $PLAINBOX_SESSION_SHARE/resolution_before_suspend.txt
 
+plugin: local
+id: suspend/generator_resolution_before_suspend
+_description: Record the current resolution before suspending.
+_summary: Record the current resolution before suspending.
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: shell
+ id: suspend/`echo ${index}`_resolution_before_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ depends: graphics/`echo ${index}`_switch_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ estimated_duration: 1.2
+ _description: Record the current resolution before suspending.
+ command: xrandr -q |grep '*'| awk '{print $1}' > $PLAINBOX_SESSION_SHARE/`echo ${index}`_resolution_before_suspend.txt
+ EOF
+
 plugin: shell
 id: suspend/audio_before_suspend
 estimated_duration: 1.0
@@ -247,7 +261,7 @@
   set -o pipefail; fwts_test -f none -l $PLAINBOX_SESSION_SHARE/suspend_single -s s3 --s3-sleep-delay=30 --s3-device-check --s3-device-check-delay=45 | tee $PLAINBOX_SESSION_SHARE/suspend_single_times.log
  else
   echo "Calling sleep_test"
-  set -o pipefail; sleep_test -p | tee $PLAINBOX_SESSION_SHARE/suspend_single_times.log
+  set -o pipefail; sleep_test -p | tee $PLAINBOX_SESSION_SHARE/2_suspend_single_times.log
  fi
 estimated_duration: 90.0
 _description:
@@ -271,16 +285,19 @@
  cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
  plugin: user-interact-verify
  id: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
- depends: suspend/suspend_advanced
+ requires:
+  sleep.mem == 'supported'
+  rtc.state == 'supported'
+ depends: graphics/`echo ${index}`_switch_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
  user: root
  environ: PLAINBOX_SESSION_SHARE
  command:
   if type -P fwts >/dev/null; then
    echo "Calling fwts"
-   set -o pipefail; fwts_test -f none -s s3 --s3-sleep-delay=30 --s3-device-check --s3-device-check-delay=45
+   set -o pipefail; fwts_test -f none -l $PLAINBOX_SESSION_SHARE/`echo ${index}`_suspend_single -s s3 --s3-sleep-delay=30 --s3-device-check --s3-device-check-delay=45 | tee $PLAINBOX_SESSION_SHARE/`echo ${index}`_suspend_single_times.log
   else
    echo "Calling sleep_test"
-   set -o pipefail; sleep_test -p
+   set -o pipefail; sleep_test -p | tee $PLAINBOX_SESSION_SHARE/`echo ${index}`_suspend_single_times.log
   fi
  estimated_duration: 90.0
  _summary: Test suspend/resume after switching to $product
@@ -310,22 +327,29 @@
 command: set -o pipefail; fwts_test -f none -l $PLAINBOX_SESSION_SHARE/suspend_single -s s3 --s3-sleep-delay=30 --s3-device-check --s3-device-check-delay=45 | tee $PLAINBOX_SESSION_SHARE/suspend_single_times.log
 estimated_duration: 90.000
 
-plugin: user-interact-verify
-id: suspend/hybrid_sleep
-user: root
+id: suspend/generator_hybrid_sleep
+_description: Hybrid sleep test
+_summary: Hybrid sleep test
+plugin: local
 command:
- set -o pipefail; fwts_test -f none -l $PLAINBOX_SESSION_SHARE/hybrid_sleep_single -s s3 --s3-hybrid --s3-sleep-delay=30 --s3-device-check --s3-device-check-delay=45 --pm-method=pm-utils | tee $PLAINBOX_SESSION_SHARE/hybrid_sleep_single_times.log
-estimated_duration: 90.0
-_description:
- PURPOSE:
-     This test will check hybrid sleep and resume
- STEPS:
-     1. Click "Test" and your system will go into hybrid sleep mode for about 30 - 60 seconds
-     2. Observe the Power LED to see if it blinks or changes color during suspend
-     3. If your system does not wake itself up after 60 seconds, please press the power button momentarily to wake the system manually
-     4. If your system fails to wake at all and must be rebooted, restart System Testing after reboot and mark this test as Failed
- VERIFICATION:
-     Did your system enter hybrid sleep and then resume correctly?
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: user-interact-verify
+ id: suspend/`echo ${index}`_hybrid_sleep_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ user: root
+ command:
+  set -o pipefail; fwts_test -f none -l $PLAINBOX_SESSION_SHARE/hybrid_sleep_single -s s3 --s3-hybrid --s3-sleep-delay=30 --s3-device-check --s3-device-check-delay=45 --pm-method=pm-utils | tee $PLAINBOX_SESSION_SHARE/hybrid_sleep_single_times.log
+ estimated_duration: 90.0
+ _description:
+  PURPOSE:
+      This test will check hybrid sleep and resume
+  STEPS:
+      1. Click "Test" and your system will go into hybrid sleep mode for about 30 - 60 seconds
+      2. Observe the Power LED to see if it blinks or changes color during suspend
+      3. If your system does not wake itself up after 60 seconds, please press the power button momentarily to wake the system manually
+      4. If your system fails to wake at all and must be rebooted, restart System Testing after reboot and mark this test as Failed
+  VERIFICATION:
+      Did your system enter hybrid sleep and then resume correctly?
+ EOF
 
 plugin: shell
 id: suspend/suspend-single-log-check
@@ -336,10 +360,25 @@
 
 plugin: attachment
 id: suspend/suspend-single-log-attach
+depends: suspend/suspend_advanced
 command: [ -e $PLAINBOX_SESSION_SHARE/suspend_single.log ] && cat $PLAINBOX_SESSION_SHARE/suspend_single.log
 _description:
  Attaches the log from the single suspend/resume test to the results
 
+id: suspend/generator_suspend-single-log-attach
+_description: Suspend log attach
+_summary: Suspend log attach
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: attachment
+ id: suspend/`echo ${index}`_suspend-single-log-attach_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ command: [ -e $PLAINBOX_SESSION_SHARE/`echo ${index}`_suspend_single.log ] && cat $PLAINBOX_SESSION_SHARE/`echo ${index}`_suspend_single.log
+ _description:
+  Attaches the log from the single suspend/resume test to the results
+ EOF
+
 plugin: shell
 id: suspend/hybrid-sleep-single-log-check
 estimated_duration: 1.2
@@ -347,11 +386,19 @@
 _description:
  Automated check of the hybrid sleep log to look for errors reported by fwts
 
-plugin: attachment
-id: suspend/hybrid-sleep-single-log-attach
-command: [ -e $PLAINBOX_SESSION_SHARE/hybrid_sleep_single.log ] && cat $PLAINBOX_SESSION_SHARE/hybrid_sleep_single.log
-_description:
- Attaches the log from the single hybrid sleep/resume test to the results
+id: suspend/generator_hybrid-sleep-single-log-attach
+_description: Hybrid sleep log attach
+_summary: Hybrid sleep log attach
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: attachment
+ id: suspend/`echo ${index}`_hybrid-sleep-single-log-attach_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ depends: suspend/`echo ${index}`_hybrid_sleep_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ command: [ -e $PLAINBOX_SESSION_SHARE/`echo ${index}`_hybrid_sleep_single.log ] && cat $PLAINBOX_SESSION_SHARE/`echo ${index}`_hybrid_sleep_single.log
+ _description:
+  Attaches the log from the single hybrid sleep/resume test to the results
+ EOF
 
 plugin: shell
 id: suspend/suspend-time-check
@@ -360,6 +407,20 @@
 _description:
  Checks the sleep times to ensure that a machine suspends and resumes within a given threshold
 
+id: suspend/generator_suspend-time-check
+_description: Suspend time check test
+_summary: Suspend time check test
+plugin: local
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: shell
+ id: suspend/`echo ${index}`_suspend-time-check_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ estimated_duration: 1.2
+ command: [ -e $PLAINBOX_SESSION_SHARE/`echo ${index}`_suspend_single_times.log ] && sleep_time_check $PLAINBOX_SESSION_SHARE/`echo ${index}`_suspend_single_times.log
+ _description: Checks the sleep times to ensure that a machine suspends and resumes within a given threshold
+ EOF
+
 plugin: user-interact-verify
 id: power-management/usb_wakeup_keyboard
 user: root
@@ -406,6 +467,20 @@
 _description: Test to see that we have the same resolution after resuming as before.
 command: xrandr -q |grep '*'| awk '{print $1}' | diff $PLAINBOX_SESSION_SHARE/resolution_before_suspend.txt -
 
+plugin: local
+id: suspend/generator_resolution_after_suspend
+_description: Resolution consistency check after suspend with the valuse we got before suspend.
+_summary: Resolution consistency check after suspend with the valuse we got before suspend.
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: shell
+ id: suspend/`echo ${index}`_resolution_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ estimated_duration: 1.2
+ depends: suspend/`echo ${index}`_resolution_before_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ _description: Test to see that we have the same resolution after resuming as before.
+ command: xrandr -q |grep '*'| awk '{print $1}' | diff $PLAINBOX_SESSION_SHARE/`echo ${index}`_resolution_before_suspend.txt -
+ EOF
+
 plugin: shell
 id: suspend/audio_after_suspend
 estimated_duration: 1.0
@@ -481,7 +556,7 @@
  cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
  plugin: manual
  id: suspend/`echo ${index}`_display_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
- depends: suspend/suspend_advanced
+ depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
  _summary: Test display function after suspend for $product
  _description:
   PURPOSE:
@@ -1303,6 +1378,28 @@
  VERIFICATION:
      Did your display look fine in the detected mode?
 
+plugin: local
+id: suspend/generator_cycle_resolutions_after_suspend
+requires: device.category == 'VIDEO'
+_description: Cycle resolution after suspend tests
+_summary: Cycle resolution after suspend tests
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: user-interact-verify
+ id: suspend/`echo ${index}`_cycle_resolutions_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ requires: package.name == 'xorg'
+ depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ estimated_duration: 120.0
+ command: xrandr_cycle --keyword=`echo ${index}`_after_suspend --screenshot-dir $PLAINBOX_SESSION_SHARE
+ _description:
+  PURPOSE:
+      This test will cycle through the detected display modes
+  STEPS:
+      1. Click "Test" and the display will cycle trough the display modes
+  VERIFICATION:
+      Did your display look fine in the detected mode?
+ EOF
+
 plugin: shell
 id: suspend/cycle_resolutions_after_suspend_auto
 estimated_duration: 1.2
@@ -1319,6 +1416,19 @@
 command: [ -e $PLAINBOX_SESSION_SHARE/xrandr_screens_after_suspend.tgz ] && cat $PLAINBOX_SESSION_SHARE/xrandr_screens_after_suspend.tgz
 _description: This attaches screenshots from the suspend/cycle_resolutions_after_suspend test to the results submission.
 
+plugin: local
+id: suspend/generator_xrandr_screens_after_suspend.tar.gz
+_description: Attach screenshots from the suspend/cycle_resolution_after_suspend test to the results submission.
+_summary: Attach screenshots from the suspend/cycle_resolution_after_suspend test to the results submission.
+command:
+ cat <<'EOF' | run_templates -t -s 'graphics_card_resource'
+ plugin: attachment
+ id: suspend/`echo ${index}`_xrandr_screens_after_suspend.tar.gz
+ depends: suspend/`echo ${index}`_cycle_resolutions_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`
+ command: [ -e $PLAINBOX_SESSION_SHARE/`echo ${index}`_xrandr_screens_after_suspend.tgz ] && cat $PLAINBOX_SESSION_SHARE/`echo ${index}`_xrandr_screens_after_suspend.tgz
+ _description: This attaches screenshots from the suspend/cycle_resolutions_after_suspend test to the results submission.
+ EOF
+
 plugin: shell
 id: suspend/record_playback_after_suspend
 estimated_duration: 10.0