Merge lp:~zyga/checkbox/fix-exporter-crash into lp:checkbox

[Zygmunt Krynicki]

work in progress to address the crash that happens at the end of 'plainbox sru' run.

The tests work on precise and raring but quantal, for some reason, produces different results. I'm checking that to see what's going on.

This branch also needs to be rebased on the very old and unused 'device profiles' branch so that plainbox sru tests are only started inside virtualbox.

UPDATE 1:

Running on quantal is now fixed, this was a missing dependency on python3-dbus on jobs that run network_device_info script. I've added that dependency and marked it for installation with vagrant. I've fixed a few other things in that script as I had a look at it. I _strongly_ believe that we need to do an audit of each script and job and start looking after them more than we currently do (this is a part of the jobbox proposal).

The comment about device profiles still stands so please don't land this yet

UPDATE 2:

Fixed a few things that were breaking tests (indeterministic iteration over a few sets).
I've also fixed the text exporter to add newlines (yay) and fixed all exporters to have
deterministic output.

Still needs device profiles work to only run in VMs

UPDATE 3:

Added explicit requirements on all environment variables so that stuff is not supported (with an explicit reason) rather than failing.

Still needs device profiles work to only run in VMs

UPDATE 4:

Finished checking jobs that fail. Out of the list only memory/info and memory/check still fail but for not reasons caused by plainbox. A number of jobs are not supported
those will require adding a few packages (most notably udisks and some gir packages)
so that we can run all tests reliably for SRU (SRU vagrant mode should install those packages).

Again before landing this branch needs to support device profiles to only run on VMs

(the branch name is somewhat inappropriate now, as a small version of this, with just the fix, is proposed separately).

[lp:~zyga/checkbox/fix-exporter-crash]

2030. By Zygmunt Krynicki on 2013-04-04

vagrant: install python3-dbus

This package is needed by scripts that talk over dbus and have been
ported to python3

Signed-off-by: Zygmunt Krynicki <email address hidden>

2031. By Zygmunt Krynicki on 2013-04-04

jobs: networking/detect: add dependency on python3-dbus

This package required python3-dbus to work. Let's make it explicit

Signed-off-by: Zygmunt Krynicki <email address hidden>

2032. By Zygmunt Krynicki on 2013-04-04

scripts: network_device_info: remove unused imports

Signed-off-by: Zygmunt Krynicki <email address hidden>

2033. By Zygmunt Krynicki on 2013-04-04

scripts: network_device_info: fix PEP8 issues

Signed-off-by: Zygmunt Krynicki <email address hidden>

2034. By Zygmunt Krynicki on 2013-04-04

scrips: network_device_info: rework style

Using filter() with lambda is deprecated in favor of list comprehensions

Signed-off-by: Zygmunt Krynicki <email address hidden>

2035. By Zygmunt Krynicki on 2013-04-04

integration_tests: refactor XDGSandBoxMixIn from IntegrationTests

Signed-off-by: Zygmunt Krynicki <email address hidden>

2036. By Zygmunt Krynicki on 2013-04-04

integration_tests: fix PEP8 issues

Signed-off-by: Zygmunt Krynicki <email address hidden>

2037. By Zygmunt Krynicki on 2013-04-04

integration_tests: rename IntegrationTests to CheckBoxJobResponse

Signed-off-by: Zygmunt Krynicki <email address hidden>

2038. By Zygmunt Krynicki on 2013-04-04

depmgr: iterate over dependencies in deterministic order

The DependencySolver._visit() job iterates over each dependency as
computed by the DependencySolver._get_dependency_set() method. This
returns a set so the ordering is unspecified.

It makes sense to make this part deterministic so that we always run
jobs in the same order. It also fixes random "problems" for integration
tests that check the verbatim copy of the output.

Signed-off-by: Zygmunt Krynicki <email address hidden>

2039. By Zygmunt Krynicki on 2013-04-04

session: iterate over dependencies in deterministic order

There is one aspect of _recompute_job_readiness() that is
not deterministic. The loop that checks if each direct dependency
ran correctly iterates over a set returned by job.get_direct_dependencies()

Being deterministic is a desirable property so let's fix this by
sorting the dependencies.

Signed-off-by: Zygmunt Krynicki <email address hidden>

2040. By Zygmunt Krynicki on 2013-04-04

integration_tests: add smoke test for 'plainbox sru' command

Signed-off-by: Zygmunt Krynicki <email address hidden>

2041. By Zygmunt Krynicki on 2013-04-04

MANIFEST.in: put .xml and .txt files into source tarball

Signed-off-by: Zygmunt Krynicki <email address hidden>

2042. By Zygmunt Krynicki on 2013-04-04

daily-package-testing: run unit and integration tests

Signed-off-by: Zygmunt Krynicki <email address hidden>

2043. By Zygmunt Krynicki on 2013-04-04

exporter: text: add missing newline to each output line

Signed-off-by: Zygmunt Krynicki <email address hidden>

2044. By Zygmunt Krynicki on 2013-04-04

exporter: text: provide deterministic output

Signed-off-by: Zygmunt Krynicki <email address hidden>

2045. By Zygmunt Krynicki on 2013-04-04

exporter: prettify some of the canned output

Signed-off-by: Zygmunt Krynicki <email address hidden>

2046. By Zygmunt Krynicki on 2013-04-04

exporter: rfc822: provide deterministic output

Signed-off-by: Zygmunt Krynicki <email address hidden>

2047. By Zygmunt Krynicki on 2013-04-04

jobs: add explicit requires: environment.XXX for each $XXX used

The exceptions are $CHECKBOX_SHARE and $CHECKBOX_DATA as those are
generated internally by the job runner so are always available.

Signed-off-by: Zygmunt Krynicki <email address hidden>

2048. By Zygmunt Krynicki on 2013-04-04

test-data: update test data after adding environment requirements

Signed-off-by: Zygmunt Krynicki <email address hidden>

2049. By Zygmunt Krynicki on 2013-04-04

jobs: usb: update requires to depend on correct packages

There are a number of changes but they can be classified into distinct
groups:

Stuff that runs removable_storage_{watcher,test}. All of those jobs
should depend on python3-dbus, python3-gi, gir1.2-gudev-1.0 and one
of udisks or udisks2

One shell job that uses udev and hdparm directly got those dependencies

One job uses keyboard_test, it depends on python3-gi and the gir file for
GTK whichc is in package gir1.2-gtk-3.0

Signed-off-by: Zygmunt Krynicki <email address hidden>

2050. By Zygmunt Krynicki on 2013-04-04

test-data: update after declaring dependencies for usb jobs

Signed-off-by: Zygmunt Krynicki <email address hidden>

2051. By Zygmunt Krynicki on 2013-04-04

jobs: dmi_attachment: do not read private files

This job fails in plainbox because grub exits with error code 2. This
is caused by grub failing to open any of

/sys/class/dmi/id/{board_serial,product_serial,product_uuid}

All of those files are only readable by root as they contain unique
identifiers that can be used to trace the user somehow.

The new script does away with reading files it cannot read, it also
uses a slightly different syntax by having a space after the leading
colon (the filename)

Signed-off-by: Zygmunt Krynicki <email address hidden>

2052. By Zygmunt Krynicki on 2013-04-04

test-data: update after fixing dmi_attachment

Signed-off-by: Zygmunt Krynicki <email address hidden>

2053. By Zygmunt Krynicki on 2013-04-04

jobs: attachment: fix acpi_sleep_attachment if /proc/acpi/sleep is gone

This job returns a non-zero exit status from [ if /proc/acpi/sleep
does not exist. This shows up in plainbox as a failed job.

Since there is no way to express file dependencies (no path resource)
I've changed it to always succeed

Signed-off-by: Zygmunt Krynicki <email address hidden>

2054. By Zygmunt Krynicki on 2013-04-04

test-data: update after fixing acpi_sleep_attachment

Signed-off-by: Zygmunt Krynicki <email address hidden>

[Daniel Manrique (roadmr)]

So, we discussed how to arrange/split this if necessary. One thing we mentioned was having one commit for fixing each script, and one commit for each job file (*.txt.in).

In practice, I found it very easy to branch this and read each commit in turn, as they are quite small and very well separated logically. I don't think any further splitting is necessary and the whole branch achieves one purpose so this fits in well with how it's done. It already contains commits for each logical change to each script, and putting the job file fixes in only a few commits that touch many files is not as unwieldy as I thought, since the changes are logically related (i.e. adding explicit checks for environment variables to all the tests that need them).

This is just a comment since it's marked as work in progress, but this looks good in general and I'd +1 without problems once it's complete.

Unmerged revisions

2054. By Zygmunt Krynicki on 2013-04-04

test-data: update after fixing acpi_sleep_attachment

Signed-off-by: Zygmunt Krynicki <email address hidden>

2053. By Zygmunt Krynicki on 2013-04-04

jobs: attachment: fix acpi_sleep_attachment if /proc/acpi/sleep is gone

This job returns a non-zero exit status from [ if /proc/acpi/sleep
does not exist. This shows up in plainbox as a failed job.

Since there is no way to express file dependencies (no path resource)
I've changed it to always succeed

Signed-off-by: Zygmunt Krynicki <email address hidden>

2052. By Zygmunt Krynicki on 2013-04-04

test-data: update after fixing dmi_attachment

Signed-off-by: Zygmunt Krynicki <email address hidden>

2051. By Zygmunt Krynicki on 2013-04-04

jobs: dmi_attachment: do not read private files

This job fails in plainbox because grub exits with error code 2. This
is caused by grub failing to open any of

/sys/class/dmi/id/{board_serial,product_serial,product_uuid}

All of those files are only readable by root as they contain unique
identifiers that can be used to trace the user somehow.

The new script does away with reading files it cannot read, it also
uses a slightly different syntax by having a space after the leading
colon (the filename)

Signed-off-by: Zygmunt Krynicki <email address hidden>

2050. By Zygmunt Krynicki on 2013-04-04

test-data: update after declaring dependencies for usb jobs

Signed-off-by: Zygmunt Krynicki <email address hidden>

2049. By Zygmunt Krynicki on 2013-04-04

jobs: usb: update requires to depend on correct packages

There are a number of changes but they can be classified into distinct
groups:

Stuff that runs removable_storage_{watcher,test}. All of those jobs
should depend on python3-dbus, python3-gi, gir1.2-gudev-1.0 and one
of udisks or udisks2

One shell job that uses udev and hdparm directly got those dependencies

One job uses keyboard_test, it depends on python3-gi and the gir file for
GTK whichc is in package gir1.2-gtk-3.0

Signed-off-by: Zygmunt Krynicki <email address hidden>

2048. By Zygmunt Krynicki on 2013-04-04

test-data: update test data after adding environment requirements

Signed-off-by: Zygmunt Krynicki <email address hidden>

2047. By Zygmunt Krynicki on 2013-04-04

jobs: add explicit requires: environment.XXX for each $XXX used

The exceptions are $CHECKBOX_SHARE and $CHECKBOX_DATA as those are
generated internally by the job runner so are always available.

Signed-off-by: Zygmunt Krynicki <email address hidden>

2046. By Zygmunt Krynicki on 2013-04-04

exporter: rfc822: provide deterministic output

Signed-off-by: Zygmunt Krynicki <email address hidden>

2045. By Zygmunt Krynicki on 2013-04-04

exporter: prettify some of the canned output

Signed-off-by: Zygmunt Krynicki <email address hidden>

=== modified file 'Vagrantfile'
--- Vagrantfile	2013-03-20 14:34:45 +0000
+++ Vagrantfile	2013-04-04 16:49:24 +0000
@@ -49,6 +49,7 @@
   # Later on those could be installed on demand to test how we behave without
   # them but for now that's good enough. Little by little...
   config.vm.provision :shell, :inline => "apt-get install --yes fwts"
+  config.vm.provision :shell, :inline => "apt-get install --yes python3-dbus"
   # Develop plainbox so that we have it in $PATH
   config.vm.provision :shell, :inline => "cd /vagrant/plainbox/ && python3 setup.py develop"
   # Create a cool symlink so that everyone knows where to go to

=== modified file 'jobs/info.txt.in'
--- jobs/info.txt.in	2012-12-17 21:41:58 +0000
+++ jobs/info.txt.in	2013-04-04 16:49:24 +0000
@@ -16,7 +16,7 @@
 
 name: dmi_attachment
 plugin: attachment
-command: grep -r . /sys/class/dmi/id/ 2>/dev/null
+command: for f in /sys/class/dmi/id/*; do [ -r $f -a -f $f ] && cat $f | sed -e "s/^/$(basename $f): /"; done
 _description: Attaches info on DMI
 
 name: dmidecode_attachment
@@ -101,7 +101,7 @@
 
 plugin: attachment
 name: acpi_sleep_attachment
-command: [ -e /proc/acpi/sleep ] && cat /proc/acpi/sleep
+command: [ -e /proc/acpi/sleep ] && cat /proc/acpi/sleep || :
 _description: Attaches the contents of /proc/acpi/sleep if it exists.
 
 plugin: local

=== modified file 'jobs/networking.txt.in'
--- jobs/networking.txt.in	2012-12-13 18:35:08 +0000
+++ jobs/networking.txt.in	2013-04-04 16:49:24 +0000
@@ -4,6 +4,7 @@
  device.category == 'NETWORK' or device.category == 'WIRELESS'
  package.name == 'module-init-tools'
  package.name == 'pciutils'
+ package.name == 'python3-dbus'
 command: network_device_info
 _description: Test to detect the available network controllers
 
@@ -75,6 +76,7 @@
 
 plugin: shell
 name: networking/ping
+requires: environment.CHECKBOX_SERVER != ""
 command: internet_test $CHECKBOX_SERVER
 _description:
  Automated test case to verify availability of some system on the network
@@ -82,6 +84,9 @@
 
 plugin: shell
 name: networking/http
+requires:
+    package.name == "wget"
+    environment.TRANSFER_SERVER != ""
 command: wget -SO /dev/null http://$TRANSFER_SERVER
 _description:
  Automated test case to make sure that it's possible to download files through HTTP
@@ -95,13 +100,17 @@
 
 plugin: shell
 name: networking/ssh
-requires: package.name == 'openssh-client'
+requires:
+    package.name == 'openssh-client'
+    environment.CHECKBOX_SERVER != ""
 command: if [ $CHECKBOX_SERVER ]; then ssh -q -o 'StrictHostKeyChecking=no' -o "UserKnownHostsFile=/tmp/ssh_test_$$" -l ubuntu $CHECKBOX_SERVER "uname -a" && rm /tmp/ssh_test_$$; fi
 _description: Verify that an installation of checkbox-server on the network can be reached over SSH.
 
 plugin: shell
 name: networking/printer
-requires: package.name == 'cups-client'
+requires:
+    package.name == 'cups-client'
+    environment.CHECKBOX_SERVER != ""
 command: network_printer_test -s $CHECKBOX_SERVER
 _description: Try to enable a remote printer on the network and print a test page.
 

=== modified file 'jobs/peripheral.txt.in'
--- jobs/peripheral.txt.in	2013-01-21 22:21:29 +0000
+++ jobs/peripheral.txt.in	2013-04-04 16:49:24 +0000
@@ -33,6 +33,9 @@
 plugin: shell
 name: peripheral/external-usb-modem-http
 depends: peripheral/external-usb-modem
+requires:
+    package.name == "wget"
+    environment.TRANSFER_SERVER != ""
 command: wget -SO /dev/null http://$TRANSFER_SERVER
 _description:
  Automated test case to make sure that it's possible to download files through HTTP

=== modified file 'jobs/power-management.txt.in'
--- jobs/power-management.txt.in	2012-12-03 14:30:41 +0000
+++ jobs/power-management.txt.in	2013-04-04 16:49:24 +0000
@@ -164,6 +164,7 @@
 name: power-management/battery_drain_movie
 requires:
  package.name == 'upower'
+ environment.MOVIE_VAR != ''
 depends: power-management/unplug_ac
 _description: Checks the battery drain while watching a movie.  Reports time
  until empty and capacity as well.  Requires MOVIE_VAR to be set.

=== modified file 'jobs/suspend.txt.in'
--- jobs/suspend.txt.in	2013-03-29 18:21:40 +0000
+++ jobs/suspend.txt.in	2013-04-04 16:49:24 +0000
@@ -43,6 +43,7 @@
  package.name == 'bluez'
  package.name == 'obexd-client'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then
@@ -69,6 +70,7 @@
  package.name == 'bluez'
  package.name == 'obexftp'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then
@@ -95,6 +97,7 @@
  package.name == 'bluez'
  package.name == 'obexftp'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then
@@ -121,6 +124,7 @@
  package.name == 'bluez'
  package.name == 'obexftp'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then
@@ -312,6 +316,8 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.WPA_BG_SSID != ''
+ environment.WPA_BG_PSK != ''
 user: root
 environ: WPA_BG_SSID WPA_BG_PSK
 command: trap "rm -f /etc/NetworkManager/system-connections/$WPA_BG_SSID" EXIT; create_connection $WPA_BG_SSID --security=wpa --key=$WPA_BG_PSK; internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -325,6 +331,7 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.OPEN_BG_SSID != ''
 user: root
 environ: OPEN_BG_SSID
 command: trap "rm -f /etc/NetworkManager/system-connections/$OPEN_BG_SSID" EXIT; create_connection $OPEN_BG_SSID; internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -338,6 +345,8 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.WPA_N_SSID != ''
+ environment.WPA_N_PSK != ''
 user: root
 environ: WPA_N_SSID WPA_N_PSK
 command: trap "rm -f /etc/NetworkManager/system-connections/$WPA_N_SSID" EXIT; create_connection $WPA_N_SSID --security=wpa --key=$WPA_N_PSK; internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -351,6 +360,7 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.OPEN_N_SSID != ''
 user: root
 environ: OPEN_N_SSID
 command: trap "rm -f /etc/NetworkManager/system-connections/$OPEN_N_SSID" EXIT; create_connection $OPEN_N_SSID; internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -364,6 +374,8 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.WPA_BG_SSID != ''
+ environment.WPA_BG_PSK != ''
 user: root
 environ: WPA_BG_SSID WPA_BG_PSK
 command: trap "rm -f /etc/NetworkManager/system-connections/$WPA_BG_SSID" EXIT; create_connection $WPA_BG_SSID --security=wpa --key=$WPA_BG_PSK; internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -377,6 +389,7 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.OPEN_BG_SSID != ''
 user: root
 environ: OPEN_BG_SSID
 command: trap "rm -f /etc/NetworkManager/system-connections/$OPEN_BG_SSID" EXIT; create_connection $OPEN_BG_SSID; internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -390,6 +403,8 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.WPA_N_SSID != ''
+ environment.WPA_N_PSK != ''
 user: root
 environ: WPA_N_SSID WPA_N_PSK
 command: trap "rm -f /etc/NetworkManager/system-connections/$WPA_N_SSID" EXIT; create_connection $WPA_N_SSID --security=wpa --key=$WPA_N_PSK; internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -403,6 +418,7 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.OPEN_N_SSID != ''
 user: root
 environ: OPEN_N_SSID
 command: trap "rm -f /etc/NetworkManager/system-connections/$OPEN_N_SSID" EXIT; create_connection $OPEN_N_SSID; internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -416,6 +432,8 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'single'
+ environment.ROUTER_SSID != ''
+ environment.ROUTER_PSK != ''
 user: root
 environ: ROUTER_SSID ROUTER_PSK
 command: trap "nmcli con delete id $ROUTER_SSID" EXIT; create_connection $ROUTER_SSID --security=wpa --key=$ROUTER_PSK && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -438,6 +456,7 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'single'
+ environment.ROUTER_SSID != ''
 user: root
 environ: ROUTER_SSID
 command: trap "nmcli con delete id $ROUTER_SSID" EXIT; create_connection $ROUTER_SSID && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -460,6 +479,8 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'single'
+ environment.ROUTER_SSID != ''
+ environment.ROUTER_PSK != ''
 user: root
 environ: ROUTER_SSID ROUTER_PSK
 command: trap "nmcli con delete id $ROUTER_SSID" EXIT; create_connection $ROUTER_SSID --security=wpa --key=$ROUTER_PSK && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -482,6 +503,7 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'single'
+ environment.ROUTER_SSID != ''
 user: root
 environ: ROUTER_SSID
 command: trap "nmcli con delete id $ROUTER_SSID" EXIT; create_connection $ROUTER_SSID && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -529,6 +551,7 @@
  package.name == 'bluez'
  package.name == 'obexd-client'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then
@@ -556,6 +579,7 @@
  package.name == 'bluez'
  package.name == 'obexd-client'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then
@@ -583,6 +607,7 @@
  package.name == 'bluez'
  package.name == 'obexftp'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then
@@ -610,6 +635,7 @@
  package.name == 'bluez'
  package.name == 'obexftp'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then
@@ -637,6 +663,7 @@
  package.name == 'bluez'
  package.name == 'obexftp'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then
@@ -664,6 +691,7 @@
  package.name == 'bluez'
  package.name == 'obexftp'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then
@@ -691,6 +719,7 @@
  package.name == 'bluez'
  package.name == 'obexftp'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then
@@ -718,6 +747,7 @@
  package.name == 'bluez'
  package.name == 'obexftp'
  device.category == 'BLUETOOTH'
+ environment.BTDEVADDR != ''
 command:
   if [ -z "$BTDEVADDR" ]
   then

=== modified file 'jobs/usb.txt.in'
--- jobs/usb.txt.in	2013-03-07 07:46:10 +0000
+++ jobs/usb.txt.in	2013-04-04 16:49:24 +0000
@@ -6,6 +6,11 @@
 plugin: manual
 name: usb/disk_detect
 depends: usb/detect
+requires:
+ package.name == 'python3-dbus'
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gudev-1.0'
+ package.name == 'udisks' or package.name == 'udisks2'
 command: removable_storage_test -l usb
 _description:
  PURPOSE:
@@ -21,6 +26,9 @@
 plugin: manual
 name: usb/HID
 depends: usb/detect
+requires:
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gtk-3.0'
 command: keyboard_test
 _description:
  PURPOSE:
@@ -35,6 +43,11 @@
 plugin: user-interact
 name: usb/insert
 depends: usb/detect
+requires:
+ package.name == 'python3-dbus'
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gudev-1.0'
+ package.name == 'udisks' or package.name == 'udisks2'
 command: removable_storage_watcher insert usb
 _description:
  PURPOSE:
@@ -52,6 +65,10 @@
 name: usb3/insert
 requires:
  usb.usb3 == 'supported'
+ package.name == 'python3-dbus'
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gudev-1.0'
+ package.name == 'udisks' or package.name == 'udisks2'
 command: removable_storage_watcher -m 500000000 insert usb
 _description:
  PURPOSE:
@@ -68,6 +85,11 @@
 plugin: user-interact
 name: usb/remove
 depends: usb/insert
+requires:
+ package.name == 'python3-dbus'
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gudev-1.0'
+ package.name == 'udisks' or package.name == 'udisks2'
 command: removable_storage_watcher remove usb
 _description:
  PURPOSE:
@@ -85,6 +107,10 @@
 depends: usb3/insert
 requires:
  usb.usb3 == 'supported'
+ package.name == 'python3-dbus'
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gudev-1.0'
+ package.name == 'udisks' or package.name == 'udisks2'
 command: removable_storage_watcher -m 500000000 remove usb
 _description:
  PURPOSE:
@@ -100,6 +126,11 @@
 plugin: manual
 name: usb/storage-transfer
 depends: usb/insert
+requires:
+ package.name == 'python3-dbus'
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gudev-1.0'
+ package.name == 'udisks' or package.name == 'udisks2'
 user: root
 command: removable_storage_test -s 268400000 usb
 _description:
@@ -117,6 +148,10 @@
 name: usb3/storage-transfer
 requires:
  usb.usb3 == 'supported'
+ package.name == 'python3-dbus'
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gudev-1.0'
+ package.name == 'udisks' or package.name == 'udisks2'
 depends: usb3/insert
 user: root
 command: removable_storage_test -s 268400000 -m 500000000 -p 7 usb
@@ -134,6 +169,11 @@
 plugin: shell
 name: usb/storage-automated
 depends: usb/insert
+requires:
+ package.name == 'python3-dbus'
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gudev-1.0'
+ package.name == 'udisks' or package.name == 'udisks2'
 user: root
 command: removable_storage_test -s 268400000 usb
 _description:
@@ -143,6 +183,10 @@
 name: usb3/storage-automated
 requires:
  usb.usb3 == 'supported'
+ package.name == 'python3-dbus'
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gudev-1.0'
+ package.name == 'udisks' or package.name == 'udisks2'
 depends: usb3/insert
 user: root
 command: removable_storage_test -s 268400000 -m 500000000 -p 7 usb
@@ -151,6 +195,11 @@
 
 plugin: shell
 name: usb/storage-preinserted
+requires:
+ package.name == 'python3-dbus'
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gudev-1.0'
+ package.name == 'udisks' or package.name == 'udisks2'
 user: root
 command: removable_storage_test -l usb && removable_storage_test -s 268400000 usb
 _description:
@@ -163,6 +212,10 @@
 user: root
 requires:
  usb.usb3 == 'supported'
+ package.name == 'python3-dbus'
+ package.name == 'python3-gi'
+ package.name == 'gir1.2-gudev-1.0'
+ package.name == 'udisks' or package.name == 'udisks2'
 command: removable_storage_test -l usb && removable_storage_test -s 268400000 -m 500000000 -p 7 usb
 _description:
  This is an automated version of usb3/storage-automated and assumes that the
@@ -194,6 +247,8 @@
  name: usb/usb3_read_performance_`ls /sys$path/block`
  requires:
   device.path == "$path"
+  package.name == "udev"
+  package.name == "hdparm"
   block_device.`ls /sys$path/block`_state == 'removable' and block_device.`ls /sys$path/block`_usb3 == 'supported'
  description: USB3 read performance test for $product
  user: root

=== modified file 'jobs/user_apps.txt.in'
--- jobs/user_apps.txt.in	2012-07-25 19:08:46 +0000
+++ jobs/user_apps.txt.in	2013-04-04 16:49:24 +0000
@@ -66,6 +66,7 @@
 plugin: manual
 name: software/nautilus_file_create
 depends: software/nautilus_folder_create
+requires: environment.HOME != ''
 command: nautilus $HOME/"Test Folder"
 _description:
  PURPOSE:
@@ -81,6 +82,7 @@
 plugin: manual
 name: software/nautilus_file_copy
 depends: software/nautilus_file_create
+requires: environment.HOME != ''
 command: nautilus $HOME/"Test Folder"
 _description:
  PURPOSE:
@@ -98,6 +100,7 @@
 plugin: manual
 name: software/nautilus_file_move
 depends: software/nautilus_file_copy
+requires: environment.HOME != ''
 command: nautilus $HOME/"Test Folder"
 _description:
  PURPOSE:
@@ -114,6 +117,7 @@
 plugin: manual
 name: software/nautilus_file_delete
 depends: software/nautilus_file_create
+requires: environment.HOME != ''
 command: nautilus $HOME/"Test Folder"
 _description:
  PURPOSE:

=== modified file 'jobs/wireless.txt.in'
--- jobs/wireless.txt.in	2012-10-16 20:45:22 +0000
+++ jobs/wireless.txt.in	2013-04-04 16:49:24 +0000
@@ -38,6 +38,8 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.WPA_BG_SSID != ''
+ environment.WPA_BG_PSK != ''
 user: root
 environ: WPA_BG_SSID WPA_BG_PSK
 command: trap "nmcli con delete id $WPA_BG_SSID" EXIT; create_connection $WPA_BG_SSID --security=wpa --key=$WPA_BG_PSK && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -50,6 +52,7 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.OPEN_BG_SSID != ''
 user: root
 environ: OPEN_BG_SSID
 command: trap "nmcli con delete id $OPEN_BG_SSID" EXIT; create_connection $OPEN_BG_SSID && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -62,6 +65,8 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.WPA_N_SSID != ''
+ environment.WPA_N_PSK != ''
 user: root
 environ: WPA_N_SSID WPA_N_PSK
 command: trap "nmcli con delete id $WPA_N_SSID" EXIT; create_connection $WPA_N_SSID --security=wpa --key=$WPA_N_PSK && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -74,6 +79,7 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'multiple'
+ environment.OPEN_N_SSID != ''
 user: root
 environ: OPEN_N_SSID
 command: trap "nmcli con delete id $OPEN_N_SSID" EXIT; create_connection $OPEN_N_SSID &&  internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -86,6 +92,8 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'single'
+ environment.ROUTER_SSID != ''
+ environment.ROUTER_PSK != ''
 user: root
 environ: ROUTER_SSID ROUTER_PSK
 command: trap "nmcli con delete id $ROUTER_SSID" EXIT; create_connection $ROUTER_SSID --security=wpa --key=$ROUTER_PSK && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -107,6 +115,7 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'single'
+ environment.ROUTER_SSID != ''
 user: root
 environ: ROUTER_SSID
 command: trap "nmcli con delete id $ROUTER_SSID" EXIT; create_connection $ROUTER_SSID && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -128,6 +137,8 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'single'
+ environment.ROUTER_SSID != ''
+ environment.ROUTER_PSK != ''
 user: root
 environ: ROUTER_SSID ROUTER_PSK
 command: trap "nmcli con delete id $ROUTER_SSID" EXIT; create_connection $ROUTER_SSID --security=wpa --key=$ROUTER_PSK && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -149,6 +160,7 @@
 requires:
  device.category == 'WIRELESS'
  environment.ROUTERS == 'single'
+ environment.ROUTER_SSID != ''
 user: root
 environ: ROUTER_SSID
 command: trap "nmcli con delete id $ROUTER_SSID" EXIT; create_connection $ROUTER_SSID && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -170,6 +182,9 @@
 requires:
  package.name == 'iperf'
  device.category == 'WIRELESS'
+ environment.WPA_BG_SSID != ''
+ environment.WPA_BG_PSK != ''
+ environment.SERVER_IPERF != ''
 user: root
 environ: WPA_BG_SSID WPA_BG_PSK SERVER_IPERF
 command: trap "nmcli con delete id $WPA_BG_SSID" EXIT; create_connection $WPA_BG_SSID --security=wpa --key=$WPA_BG_PSK && iperf -c $SERVER_IPERF -t 300 -i 30
@@ -181,6 +196,9 @@
 requires:
  package.name == 'iperf'
  device.category == 'WIRELESS'
+ environment.WPA_BG_SSID != ''
+ environment.WPA_BG_PSK != ''
+ environment.SERVER_IPERF != ''
 user: root
 environ: WPA_BG_SSID WPA_BG_PSK SERVER_IPERF
 command: trap "nmcli con delete id $WPA_BG_SSID" EXIT; create_connection $WPA_BG_SSID --security=wpa --key=$WPA_BG_PSK && iperf -c $SERVER_IPERF -t 300 -i 30 -u -b 100m -p 5050
@@ -189,7 +207,9 @@
 
 plugin: shell
 name: wireless/wireless_connection_open_a
-requires: device.category == 'WIRELESS'
+requires:
+ device.category == 'WIRELESS'
+ environment.OPEN_A_SSID != ''
 user: root
 environ: OPEN_A_SSID
 command: trap "nmcli con delete id $OPEN_A_SSID" EXIT; create_connection $OPEN_A_SSID && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -200,7 +220,9 @@
 
 plugin: shell
 name: wireless/wireless_connection_open_b
-requires: device.category == 'WIRELESS'
+requires:
+ device.category == 'WIRELESS'
+ environment.OPEN_B_SSID != ''
 user: root
 environ: OPEN_B_SSID
 command: trap "nmcli con delete id $OPEN_B_SSID" EXIT; create_connection $OPEN_B_SSID && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`
@@ -211,7 +233,9 @@
 
 plugin: shell
 name: wireless/wireless_connection_open_g
-requires: device.category == 'WIRELESS'
+requires:
+ device.category == 'WIRELESS'
+ environment.OPEN_G_SSID != ''
 user: root
 environ: OPEN_G_SSID
 command: trap "nmcli con delete id $OPEN_G_SSID" EXIT; create_connection $OPEN_G_SSID && internet_test --interface=`nmcli dev status | awk '/802-11-wireless/ {print $1}'`

=== modified file 'plainbox/MANIFEST.in'
--- plainbox/MANIFEST.in	2013-03-15 15:49:18 +0000
+++ plainbox/MANIFEST.in	2013-04-04 16:49:24 +0000
@@ -1,7 +1,7 @@
 include README.md
 include COPYING
 include mk-interesting-graphs.sh
-recursive-include plainbox/test-data/integration-tests *.json
+recursive-include plainbox/test-data/ *.json *.xml *.txt
 recursive-include docs *.rst
 include docs/conf.py
 include plainbox/data/report/hardware-1_0.rng

=== modified file 'plainbox/daily-package-testing/Vagrantfile'
--- plainbox/daily-package-testing/Vagrantfile	2013-03-20 14:34:45 +0000
+++ plainbox/daily-package-testing/Vagrantfile	2013-04-04 16:49:24 +0000
@@ -45,4 +45,6 @@
   config.vm.provision :shell, :inline => "apt-get update"
   # Install plainbox
   config.vm.provision :shell, :inline => "apt-get install --yes plainbox"
+  # Install python3-mock for testing
+  config.vm.provision :shell, :inline => "apt-get install --yes python3-mock"
 end

=== modified file 'plainbox/daily-package-testing/test-in-vagrant.sh'
--- plainbox/daily-package-testing/test-in-vagrant.sh	2013-03-15 17:27:19 +0000
+++ plainbox/daily-package-testing/test-in-vagrant.sh	2013-04-04 16:49:24 +0000
@@ -32,6 +32,25 @@
         echo "[$target] stdout: $(pastebinit vagrant-logs/$target.help.log)"
         echo "[$target] stderr: $(pastebinit vagrant-logs/$target.help.err)"
     fi
+    # Test that plainbox self-test -u works
+    if vagrant ssh $target -c 'plainbox self-test -u' >vagrant-logs/$target.unit-tests.log 2>vagrant-logs/$target.unit-tests.err; then
+        echo "[$target] packaged PlainBox plainbox self-test -u: pass"
+    else
+        outcome=1
+        echo "[$target] packaged PlainBox plainbox self-test -u: pass"
+        echo "[$target] stdout: $(pastebinit vagrant-logs/$target.unit-tests.log)"
+        echo "[$target] stderr: $(pastebinit vagrant-logs/$target.unit-tests.err)"
+    fi
+    # Test that plainbox self-test -i works
+    if vagrant ssh $target -c 'sudo plainbox self-test -i' >vagrant-logs/$target.integration-tests.log 2>vagrant-logs/$target.integration-tests.err; then
+        echo "[$target] packaged PlainBox plainbox self-test -i: pass"
+    else
+        outcome=1
+        echo "[$target] packaged PlainBox plainbox self-test -i: pass"
+        echo "[$target] stdout: $(pastebinit vagrant-logs/$target.integration-tests.log)"
+        echo "[$target] stderr: $(pastebinit vagrant-logs/$target.integration-tests.err)"
+    fi
+    # All done
     case $VAGRANT_DONE_ACTION in
         suspend)
             # Suspend the target to conserve resources

=== modified file 'plainbox/plainbox/impl/commands/run.py'
--- plainbox/plainbox/impl/commands/run.py	2013-04-02 23:30:49 +0000
+++ plainbox/plainbox/impl/commands/run.py	2013-04-04 16:49:24 +0000
@@ -27,7 +27,6 @@
 """
 
 from argparse import FileType
-from io import RawIOBase
 from logging import getLogger
 from os.path import join
 import sys
@@ -35,6 +34,7 @@
 from plainbox.impl.commands import PlainBoxCommand
 from plainbox.impl.commands.checkbox import CheckBoxCommandMixIn
 from plainbox.impl.depmgr import DependencyDuplicateError
+from plainbox.impl.exporter import ByteStringStreamTranslator
 from plainbox.impl.exporter import get_all_exporters
 from plainbox.impl.result import JobResult
 from plainbox.impl.runner import JobRunner
@@ -45,37 +45,6 @@
 logger = getLogger("plainbox.commands.run")
 
 
-class ByteStringStreamTranslator(RawIOBase):
-    """
-    This is a sort of "impedance matcher" that bridges the gap between
-    something that expects to write strings to a stream and a stream
-    that expects to receive bytes. Instead of using, for instance, an
-    intermediate in-memory IO object, this decodes on the fly and
-    has the same interface as a writable stream, so it can be initialized
-    with the destination string stream and then passed to something
-    (usually a dump-style function) that writes bytes.
-    """
-
-    def __init__(self, dest_stream, encoding):
-        """ Create a stream that will take bytes, decode them into strings
-            according to the specified encoding, and then write them
-            as bytes into the destination stream.
-            :param dest_stream: the destination string stream.
-            :param encoding: Encoding with which bytes data is encoded.
-                It will be decoded using the same encoding to obtain
-                the string to be written.
-        """
-        self.dest_stream = dest_stream
-        self.encoding = encoding
-
-    def write(self, data):
-        """ Writes to the stream, takes bytes and decodes them per the
-            object's specified encoding prior to writing.
-            :param data: the chunk of data to write.
-        """
-        return self.dest_stream.write(data.decode(self.encoding))
-
-
 class RunCommand(PlainBoxCommand, CheckBoxCommandMixIn):
 
     def invoked(self, ns):
@@ -198,8 +167,8 @@
             print("[ Results ]".center(80, '='))
             #This requires a bit more finesse, as exporters output bytes
             #and stdout needs a string.
-            translating_stream = ByteStringStreamTranslator(ns.output_file,
-                                                                "utf-8")
+            translating_stream = ByteStringStreamTranslator(
+                ns.output_file, "UTF-8")
             exporter.dump(data, translating_stream)
         else:
             print("Saving results to {}".format(ns.output_file.name))

=== modified file 'plainbox/plainbox/impl/commands/sru.py'
--- plainbox/plainbox/impl/commands/sru.py	2013-03-21 09:48:53 +0000
+++ plainbox/plainbox/impl/commands/sru.py	2013-04-04 16:49:24 +0000
@@ -29,14 +29,15 @@
 from logging import getLogger
 import os
 
+from plainbox.impl.applogic import get_matching_job_list
+from plainbox.impl.checkbox import CheckBox, WhiteList
 from plainbox.impl.commands import PlainBoxCommand
 from plainbox.impl.depmgr import DependencyDuplicateError
+from plainbox.impl.exporter import ByteStringStreamTranslator
+from plainbox.impl.exporter.xml import XMLSessionStateExporter
 from plainbox.impl.result import JobResult
 from plainbox.impl.runner import JobRunner
 from plainbox.impl.session import SessionState
-from plainbox.impl.exporter.xml import XMLSessionStateExporter
-from plainbox.impl.checkbox import CheckBox, WhiteList
-from plainbox.impl.applogic import get_matching_job_list
 
 
 logger = getLogger("plainbox.commands.sru")
@@ -101,7 +102,8 @@
         print("Saving results to {0}".format(self.ns.fallback_file))
         data = self.exporter.get_session_data_subset(self.session)
         with open(self.ns.fallback_file, "wt", encoding="UTF-8") as stream:
-            self.exporter.dump(data, stream)
+            translating_stream = ByteStringStreamTranslator(stream, "UTF-8")
+            self.exporter.dump(data, translating_stream)
 
     def _run_all_jobs(self):
         again = True

=== modified file 'plainbox/plainbox/impl/commands/test_run.py'
--- plainbox/plainbox/impl/commands/test_run.py	2013-04-02 23:30:49 +0000
+++ plainbox/plainbox/impl/commands/test_run.py	2013-04-04 16:49:24 +0000
@@ -25,20 +25,4 @@
 Test definitions for plainbox.impl.run module
 """
 
-from unittest import TestCase
-from io import StringIO, BytesIO
-
-from plainbox.impl.commands.run import ByteStringStreamTranslator
-
-
-class TestRun(TestCase):
-
-    def test_byte_string_translator(self):
-        dest_stream = StringIO()
-        source_stream = BytesIO(b'This is a bytes literal')
-        encoding = 'utf-8'
-
-        translator = ByteStringStreamTranslator(dest_stream, encoding)
-        translator.write(source_stream.getvalue())
-
-        self.assertEqual('This is a bytes literal', dest_stream.getvalue())
+# NOTE: this file is blank after some tests moved to other modules.

=== modified file 'plainbox/plainbox/impl/depmgr.py'
--- plainbox/plainbox/impl/depmgr.py	2013-03-19 22:36:23 +0000
+++ plainbox/plainbox/impl/depmgr.py	2013-04-04 16:49:24 +0000
@@ -268,7 +268,7 @@
             if trail is None:
                 trail = [job]
             # Visit each dependency
-            for dep_type, job_name in self._get_dependency_set(job):
+            for dep_type, job_name in sorted(self._get_dependency_set(job)):
                 logger.debug("Found dependency %s: %s", dep_type, job_name)
                 # Dependency is just a name, we need to resolve it
                 # to a job instance. This can fail (missing dependencies)

=== modified file 'plainbox/plainbox/impl/exporter/__init__.py'
--- plainbox/plainbox/impl/exporter/__init__.py	2013-03-18 13:46:10 +0000
+++ plainbox/plainbox/impl/exporter/__init__.py	2013-04-04 16:49:24 +0000
@@ -28,6 +28,7 @@
 
 from abc import ABCMeta, abstractmethod
 from collections import OrderedDict
+from io import RawIOBase
 from logging import getLogger
 import base64
 
@@ -226,6 +227,42 @@
         # TODO: Add a way for the stream to be binary as well.
 
 
+class ByteStringStreamTranslator(RawIOBase):
+    """
+    This is a sort of "impedance matcher" that bridges the gap between
+    something that expects to write strings to a stream and a stream
+    that expects to receive bytes. Instead of using, for instance, an
+    intermediate in-memory IO object, this decodes on the fly and
+    has the same interface as a writable stream, so it can be initialized
+    with the destination string stream and then passed to something
+    (usually a dump-style function) that writes bytes.
+    """
+
+    def __init__(self, dest_stream, encoding):
+        """
+        Create a stream that will take bytes, decode them into strings
+        according to the specified encoding, and then write them
+        as bytes into the destination stream.
+
+        :param dest_stream:
+            the destination string stream.
+
+        :param encoding:
+            Encoding with which bytes data is encoded. It will be decoded
+            using the same encoding to obtain the string to be written.
+
+        """
+        self.dest_stream = dest_stream
+        self.encoding = encoding
+
+    def write(self, data):
+        """ Writes to the stream, takes bytes and decodes them per the
+            object's specified encoding prior to writing.
+            :param data: the chunk of data to write.
+        """
+        return self.dest_stream.write(data.decode(self.encoding))
+
+
 def get_all_exporters():
     """
     Discover and load all exporter classes.

=== modified file 'plainbox/plainbox/impl/exporter/json.py'
--- plainbox/plainbox/impl/exporter/json.py	2013-04-02 23:30:48 +0000
+++ plainbox/plainbox/impl/exporter/json.py	2013-04-04 16:49:24 +0000
@@ -37,15 +37,19 @@
 
     OPTION_MACHINE_JSON = 'machine-json'
 
-    SUPPORTED_OPTION_LIST = (SessionStateExporterBase.SUPPORTED_OPTION_LIST +
-                             (OPTION_MACHINE_JSON,))
+    SUPPORTED_OPTION_LIST = (
+        SessionStateExporterBase.SUPPORTED_OPTION_LIST + (
+            OPTION_MACHINE_JSON,))
 
     def dump(self, data, stream):
         if self.OPTION_MACHINE_JSON in self._option_list:
-            encoder = json.JSONEncoder(ensure_ascii=False,
-                             indent=None, separators=(',', ':'))
+            encoder = json.JSONEncoder(
+                ensure_ascii=False,
+                indent=None,
+                separators=(',', ':'))
         else:
-            encoder = json.JSONEncoder(ensure_ascii=False, indent=4)
-
+            encoder = json.JSONEncoder(
+                ensure_ascii=False,
+                indent=4)
         for chunk in encoder.iterencode(data):
-            stream.write(chunk.encode('utf-8'))
+            stream.write(chunk.encode('UTF-8'))

=== modified file 'plainbox/plainbox/impl/exporter/rfc822.py'
--- plainbox/plainbox/impl/exporter/rfc822.py	2013-04-02 23:30:48 +0000
+++ plainbox/plainbox/impl/exporter/rfc822.py	2013-04-04 16:49:24 +0000
@@ -41,8 +41,8 @@
     def dump(self, data, stream):
         entry = OrderedDict()
         string_stream = StringIO()
-        for job_name, job_data in data['result_map'].items():
+        for job_name, job_data in sorted(data['result_map'].items()):
             entry['name'] = job_name
             entry.update(job_data)
             dump_rfc822_records(entry, string_stream)
-        stream.write(string_stream.getvalue().encode('utf-8'))
+        stream.write(string_stream.getvalue().encode('UTF-8'))

=== modified file 'plainbox/plainbox/impl/exporter/test_init.py'
--- plainbox/plainbox/impl/exporter/test_init.py	2013-03-03 16:40:22 +0000
+++ plainbox/plainbox/impl/exporter/test_init.py	2013-04-04 16:49:24 +0000
@@ -1,8 +1,9 @@
 # This file is part of Checkbox.
 #
-# Copyright 2012 Canonical Ltd.
+# Copyright 2012, 2013 Canonical Ltd.
 # Written by:
 #   Zygmunt Krynicki <[e-mail]>
+#   Daniel Manrique  <[e-mail]>
 #
 # Checkbox is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -24,14 +25,16 @@
 Test definitions for plainbox.impl.exporter module
 """
 
+from io import StringIO, BytesIO
 from tempfile import TemporaryDirectory
 from unittest import TestCase
 
+from plainbox.impl.exporter import ByteStringStreamTranslator
 from plainbox.impl.exporter import SessionStateExporterBase
 from plainbox.impl.exporter import classproperty
-from plainbox.impl.session import SessionState
 from plainbox.impl.job import JobDefinition
 from plainbox.impl.result import JobResult, IOLogRecord
+from plainbox.impl.session import SessionState
 from plainbox.impl.testing_utils import make_io_log, make_job, make_job_result
 
 
@@ -203,3 +206,16 @@
                 (0, 'stdout', 'Zm9vCg=='),
                 (1, 'stderr', 'YmFyCg=='),
                 (2, 'stdout', 'cXV4eAo=')])
+
+
+class ByteStringStreamTranslatorTests(TestCase):
+
+    def test_smoke(self):
+        dest_stream = StringIO()
+        source_stream = BytesIO(b'This is a bytes literal')
+        encoding = 'utf-8'
+
+        translator = ByteStringStreamTranslator(dest_stream, encoding)
+        translator.write(source_stream.getvalue())
+
+        self.assertEqual('This is a bytes literal', dest_stream.getvalue())

=== modified file 'plainbox/plainbox/impl/exporter/test_json.py'
--- plainbox/plainbox/impl/exporter/test_json.py	2013-04-02 23:30:48 +0000
+++ plainbox/plainbox/impl/exporter/test_json.py	2013-04-04 16:49:24 +0000
@@ -44,10 +44,12 @@
         data = {'foo': 'bar'}
         stream = BytesIO()
         exporter.dump(data, stream)
-        self.assertEqual(stream.getvalue(), (
+        expected_bytes = (
             '{\n'
             '    "foo": "bar"\n'
-            '}').encode('utf-8'))
+            '}'
+        ).encode('UTF-8')
+        self.assertEqual(stream.getvalue(), expected_bytes)
 
     def test_machine_dump(self):
         exporter = self.exporter_cls(option_list=[
@@ -55,4 +57,7 @@
         data = {'foo': 'bar'}
         stream = BytesIO()
         exporter.dump(data, stream)
-        self.assertEqual(stream.getvalue(), '{"foo":"bar"}'.encode('utf-8'))
+        expected_bytes = (
+            '{"foo":"bar"}'
+        ).encode('UTF-8')
+        self.assertEqual(stream.getvalue(), expected_bytes)

=== modified file 'plainbox/plainbox/impl/exporter/test_rfc822.py'
--- plainbox/plainbox/impl/exporter/test_rfc822.py	2013-04-02 23:30:48 +0000
+++ plainbox/plainbox/impl/exporter/test_rfc822.py	2013-04-04 16:49:24 +0000
@@ -39,5 +39,9 @@
         data = {'result_map': {'job_name': {'outcome': 'fail'}}}
         stream = BytesIO()
         exporter.dump(data, stream)
-        expected_bytes = "name: job_name\noutcome: fail\n\n".encode('utf-8')
+        expected_bytes = (
+            "name: job_name\n"
+            "outcome: fail\n"
+            "\n"
+        ).encode('UTF-8')
         self.assertEqual(stream.getvalue(), expected_bytes)

=== modified file 'plainbox/plainbox/impl/exporter/test_text.py'
--- plainbox/plainbox/impl/exporter/test_text.py	2013-04-02 23:30:48 +0000
+++ plainbox/plainbox/impl/exporter/test_text.py	2013-04-04 16:49:24 +0000
@@ -39,5 +39,5 @@
         data = {'result_map': {'job_name': {'outcome': 'fail'}}}
         stream = BytesIO()
         exporter.dump(data, stream)
-        expected_bytes = "job_name: fail".encode('utf-8')
+        expected_bytes = "job_name: fail\n".encode('utf-8')
         self.assertEqual(stream.getvalue(), expected_bytes)

=== modified file 'plainbox/plainbox/impl/exporter/text.py'
--- plainbox/plainbox/impl/exporter/text.py	2013-04-02 23:30:48 +0000
+++ plainbox/plainbox/impl/exporter/text.py	2013-04-04 16:49:24 +0000
@@ -36,6 +36,6 @@
     """
 
     def dump(self, data, stream):
-        for job_name, job_data in data['result_map'].items():
-            stream.write("{}: {}".format(
-                         job_name, job_data['outcome']).encode('utf-8'))
+        for job_name, job_data in sorted(data['result_map'].items()):
+            stream.write("{}: {}\n".format(
+                job_name, job_data['outcome']).encode('utf-8'))

=== modified file 'plainbox/plainbox/impl/integration_tests.py'
--- plainbox/plainbox/impl/integration_tests.py	2013-03-15 15:49:18 +0000
+++ plainbox/plainbox/impl/integration_tests.py	2013-04-04 16:49:24 +0000
@@ -27,6 +27,7 @@
 """
 
 from tempfile import TemporaryDirectory
+from unittest import TestCase
 import json
 import os
 import shutil
@@ -40,11 +41,15 @@
 from plainbox.testing_utils.testcases import TestCaseWithParameters
 
 
-class IntegrationTests(TestCaseWithParameters):
-
-    parameter_names = ('job_name',)
+class XDGSandBoxMixIn:
+    """
+    Mix-In for TestCase-like classes that defines setUp() and tearDown() so
+    that tests are executed with custom, isolated XDG_CACHE_HOME that is
+    removed once testing is complete.
+    """
 
     def setUp(self):
+        super(XDGSandBoxMixIn, self).setUp()
         # session data are kept in XDG_CACHE_HOME/plainbox/.session
         # To avoid resuming a real session, we have to select a temporary
         # location instead
@@ -52,8 +57,27 @@
         self._env = os.environ
         os.environ['XDG_CACHE_HOME'] = self._sandbox
 
+    def tearDown(self):
+        shutil.rmtree(self._sandbox)
+        os.environ = self._env
+        super(XDGSandBoxMixIn, self).tearDown()
+
+
+class CheckBoxJobResponse(XDGSandBoxMixIn, TestCaseWithParameters):
+    """
+    Set of tests that check actual responses of checkbox jobs against baseline
+    values. The jobs are started with 'plainbox run' so this tests a lot of
+    infrastructure that is normally skipped.
+
+    The tests are parametrized by job name, jobs are discovered by looking for
+    expected results in plainbox/test-data/integration-tests directory.
+    """
+
+    parameter_names = ('job_name',)
+
     @classmethod
-    def _gen_job_name_values(cls, package='plainbox', root='test-data/integration-tests/'):
+    def _gen_job_name_values(cls, package='plainbox',
+                             root='test-data/integration-tests/'):
         """
         Discover job names for jobs that we have reference data for
 
@@ -66,7 +90,8 @@
                 for item in cls._gen_job_name_values(package, resource_name):
                     yield item
             elif resource_name.endswith('.json'):
-                yield resource_name[len('test-data/integration-tests/'):-len('.json')]
+                yield resource_name[
+                    len('test-data/integration-tests/'):-len('.json')]
 
     @classmethod
     def get_parameter_values(cls):
@@ -101,12 +126,37 @@
         # [ At this time TestIO and TemporaryDirectory are gone ]
         # Load the expected results and keep them in memory
         reference_path = resource_filename(
-            "plainbox", "test-data/integration-tests/{}.json".format(self.parameters.job_name))
+            "plainbox", "test-data/integration-tests/{}.json".format(
+                self.parameters.job_name))
         with open(reference_path, encoding='UTF-8') as stream:
             expected_result = json.load(stream)
         # Check that results match expected values
         self.assertEqual(actual_result, expected_result)
 
-    def tearDown(self):
-        shutil.rmtree(self._sandbox)
-        os.environ = self._env
+
+class CheckBoxSRUTests(XDGSandBoxMixIn, TestCase):
+
+    def test_smoke(self):
+        # Create a scratch directory so that we can save results there. The
+        # shared directory is also used for running tests as some test jobs
+        # leave junk around the current directory.
+        with TemporaryDirectory() as scratch_dir:
+            # Redirect all standard IO so that the test is silent.
+            # Run the script, having relocated to the scratch directory
+            # Capture SystemExit that is always raised by main() so that
+            # we can observe the return code as well.
+            with TestIO(combined=True) as test_io, TestCwd(scratch_dir),\
+                    self.assertRaises(SystemExit) as call:
+                main(['sru', '0123456789ABCDE', '0123456789ABCDE.xml',
+                      # Use the discard port to send data to nowhere in
+                      # particular
+                      '--destination=http://127.0.0.1:9/'])
+            # Check the return code for correctness
+            self.assertEqual(call.exception.args, (0,))
+            # Check that the output is the same as reference baseline
+            reference_path = resource_filename(
+                "plainbox", "test-data/sru-virtualbox-precise.txt")
+            with open(reference_path, mode="rt", encoding='UTF-8') as stream:
+                expected_output = stream.read()
+            self.maxDiff = None
+            self.assertEqual(test_io.combined, expected_output)

=== modified file 'plainbox/plainbox/impl/session.py'
--- plainbox/plainbox/impl/session.py	2013-04-03 12:05:43 +0000
+++ plainbox/plainbox/impl/session.py	2013-04-04 16:49:24 +0000
@@ -815,7 +815,7 @@
                         related_expression=exc.expression)
                     job_state.readiness_inhibitor_list.append(inhibitor)
             # Check if all job dependencies ran successfully
-            for dep_name in job.get_direct_dependencies():
+            for dep_name in sorted(job.get_direct_dependencies()):
                 dep_job_state = self._job_state_map[dep_name]
                 # If the dependency did not have a chance to run yet add the
                 # PENDING_DEP inhibitor.

=== added file 'plainbox/plainbox/test-data/sru-virtualbox-precise.txt'
--- plainbox/plainbox/test-data/sru-virtualbox-precise.txt	1970-01-01 00:00:00 +0000
+++ plainbox/plainbox/test-data/sru-virtualbox-precise.txt	2013-04-04 16:49:24 +0000
@@ -0,0 +1,171 @@
+- package: pass
+- device: pass
+- codecs_attachment: not-supported
+  * resource expression "device.driver == 'HDA Intel'" evaluates to false
+- cpuinfo_attachment: pass
+- dmesg_attachment: pass
+- dmi_attachment: pass
+- dmidecode_attachment: pass
+- efi_attachment: pass
+- lspci_attachment: pass
+- meminfo_attachment: pass
+- modprobe_attachment: pass
+- modules_attachment: pass
+- sysctl_attachment: pass
+- sysfs_attachment: pass
+- udev_attachment: pass
+- lsmod_attachment: pass
+- acpi_sleep_attachment: pass
+- __audio__: pass
+- __bluetooth__: pass
+- __camera__: pass
+- __cpu__: pass
+- __disk__: pass
+- __graphics__: pass
+- __info__: pass
+- __install__: pass
+- __mediacard__: pass
+- __memory__: pass
+- __networking__: pass
+- __power-management__: pass
+- __suspend__: pass
+- __usb__: pass
+- __wireless__: pass
+- power-management/fwts_wakealarm: pass
+- power-management/fwts_wakealarm-log-attach: pass
+- power-management/rtc: pass
+- power-management/tickless_idle: pass
+- cpu/scaling_test: pass
+- cpu/scaling_test-log-attach: pass
+- cpu/offlining_test: pass
+- cpuinfo: pass
+- cpu/topology: not-supported
+  * resource expression "int(cpuinfo.count) > 1 and (cpuinfo.platform == 'i386' or cpuinfo.platform == 'x86_64')" evaluates to false
+- networking/detect: pass
+- suspend/network_before_suspend: pass
+- suspend/audio_before_suspend: not-supported
+  * resource expression "device.category == 'AUDIO'" evaluates to false
+- suspend/cpu_before_suspend: pass
+- suspend/memory_before_suspend: pass
+- environment: pass
+- suspend/bluetooth_obex_send_before_suspend: not-supported
+  * resource expression "package.name == 'bluez'" evaluates to false
+- suspend/bluetooth_obex_browse_before_suspend: not-supported
+  * resource expression "package.name == 'bluez'" evaluates to false
+- suspend/bluetooth_obex_get_before_suspend: not-supported
+  * resource expression "package.name == 'bluez'" evaluates to false
+- sleep: pass
+- suspend/suspend_advanced_auto: not-supported
+  * resource expression "sleep.mem == 'supported'" evaluates to false
+- suspend/suspend_advanced: not-supported
+  * resource expression "sleep.mem == 'supported'" evaluates to false
+- suspend/audio_after_suspend: not-supported
+  * resource expression "device.category == 'AUDIO'" evaluates to false
+  * required dependency 'suspend/audio_before_suspend' has failed
+  * required dependency 'suspend/suspend_advanced' has failed
+- suspend/cpu_after_suspend_auto: not-supported
+  * required dependency 'suspend/suspend_advanced_auto' has failed
+- suspend/memory_after_suspend_auto: not-supported
+  * required dependency 'suspend/suspend_advanced_auto' has failed
+- suspend/wireless_connection_after_suspend_wpa_bg_auto: not-supported
+  * resource expression "device.category == 'WIRELESS'" evaluates to false
+  * required dependency 'suspend/suspend_advanced_auto' has failed
+- suspend/wireless_connection_after_suspend_open_bg_auto: not-supported
+  * resource expression "device.category == 'WIRELESS'" evaluates to false
+  * required dependency 'suspend/suspend_advanced_auto' has failed
+- suspend/wireless_connection_after_suspend_wpa_n_auto: not-supported
+  * resource expression "device.category == 'WIRELESS'" evaluates to false
+  * required dependency 'suspend/suspend_advanced_auto' has failed
+- suspend/wireless_connection_after_suspend_open_n_auto: not-supported
+  * resource expression "device.category == 'WIRELESS'" evaluates to false
+  * required dependency 'suspend/suspend_advanced_auto' has failed
+- suspend/bluetooth_obex_send_after_suspend_auto: not-supported
+  * resource expression "package.name == 'bluez'" evaluates to false
+  * required dependency 'suspend/suspend_advanced_auto' has failed
+- suspend/bluetooth_obex_browse_after_suspend_auto: not-supported
+  * resource expression "package.name == 'bluez'" evaluates to false
+  * required dependency 'suspend/suspend_advanced_auto' has failed
+- suspend/bluetooth_obex_get_after_suspend_auto: not-supported
+  * resource expression "package.name == 'bluez'" evaluates to false
+  * required dependency 'suspend/suspend_advanced_auto' has failed
+- suspend/record_playback_after_suspend: not-supported
+  * resource expression "device.category == 'AUDIO'" evaluates to false
+  * required dependency 'suspend/suspend_advanced' has failed
+- suspend/screenshot_after_suspend: not-supported
+  * resource expression "package.name == 'fswebcam'" evaluates to false
+  * required dependency 'suspend/suspend_advanced_auto' has failed
+- screenshot_after_suspend.jpg: not-supported
+  * required dependency 'suspend/screenshot_after_suspend' has failed
+- suspend/gpu_lockup_after_suspend: not-supported
+  * resource expression "package.name == 'wmctrl'" evaluates to false
+  * required dependency 'suspend/suspend_advanced_auto' has failed
+- suspend/wifi_resume_time: not-supported
+  * resource expression "device.category == 'WIRELESS'" evaluates to false
+  * required dependency 'suspend/suspend_advanced' has failed
+- suspend/network_resume_time: not-supported
+  * required dependency 'suspend/suspend_advanced' has failed
+- bluetooth/detect-output: not-supported
+  * resource expression "package.name == 'bluez'" evaluates to false
+- memory/info: fail
+- memory/check: fail
+- disk/read_performance: pass
+- graphics/xorg-version: not-supported
+  * resource expression 'package.name == "x11-utils"' evaluates to false
+- graphics/xorg-process: not-supported
+  * resource expression "package.name == 'xorg'" evaluates to false
+- graphics/xorg-failsafe: not-supported
+  * resource expression "package.name == 'xorg'" evaluates to false
+- graphics/compiz_check: not-supported
+  * resource expression "package.name == 'nux-tools'" evaluates to false
+- graphics/screenshot: not-supported
+  * resource expression "package.name == 'fswebcam'" evaluates to false
+- screenshot.jpg: not-supported
+  * required dependency 'graphics/screenshot' has failed
+- graphics/screenshot_fullscreen_video: not-supported
+  * resource expression "package.name == 'fswebcam'" evaluates to false
+- screenshot_fullscreen_video.jpg: not-supported
+  * required dependency 'graphics/screenshot_fullscreen_video' has failed
+- camera/detect: not-supported
+  * resource expression "device.category == 'CAPTURE'" evaluates to false
+- camera/multiple-resolution-images: not-supported
+  * resource expression "package.name == 'fswebcam' or package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0'" evaluates to false
+  * required dependency 'camera/detect' has failed
+- mediacard/sd-preinserted: not-supported
+  * resource expression "device.category == 'CARDREADER'" evaluates to false
+- install/apt-get-gets-updates: pass
+- audio/alsa_record_playback_automated: not-supported
+  * resource expression "package.name == 'python-gst0.10'" evaluates to false
+- networking/internet: pass
+- networking/ping: not-supported
+  * resource expression 'environment.CHECKBOX_SERVER != ""' evaluates to false
+- networking/http: not-supported
+  * resource expression 'environment.TRANSFER_SERVER != ""' evaluates to false
+- usb/detect: pass
+- usb/storage-preinserted: not-supported
+  * resource expression "package.name == 'udisks' or package.name == 'udisks2'" evaluates to false
+- cdimage: pass
+- dpkg: pass
+- gconf: not-supported
+  * resource expression 'package.name == "gconf2"' evaluates to false
+- lsb: pass
+- meminfo: pass
+- module: pass
+- dmi: pass
+- efi: pass
+- uname: pass
+- block_device: pass
+- xinput: not-supported
+  * resource expression 'package.name == "xinput"' evaluates to false
+- wireless/wireless_scanning: not-supported
+  * resource expression "package.name == 'network-manager'" evaluates to false
+- wireless/wireless_connection_wpa_bg: not-supported
+  * resource expression "device.category == 'WIRELESS'" evaluates to false
+- wireless/wireless_connection_open_bg: not-supported
+  * resource expression "device.category == 'WIRELESS'" evaluates to false
+- wireless/wireless_connection_wpa_n: not-supported
+  * resource expression "device.category == 'WIRELESS'" evaluates to false
+- wireless/wireless_connection_open_n: not-supported
+  * resource expression "device.category == 'WIRELESS'" evaluates to false
+- wireless/monitor_wireless_connection_udp: not-supported
+  * resource expression "package.name == 'iperf'" evaluates to false
+Saving results to 0123456789ABCDE.xml

=== modified file 'scripts/network_device_info'
--- scripts/network_device_info	2012-12-21 10:39:07 +0000
+++ scripts/network_device_info	2013-04-04 16:49:24 +0000
@@ -20,14 +20,14 @@
 #
 # Copyright (C) 2012 Canonical, Ltd.
 
-import os
-import re
+from subprocess import check_output, CalledProcessError, STDOUT
 import sys
+
 import dbus
-import sys
+
 from checkbox.parsers.modinfo import ModinfoParser
 from checkbox.parsers.udevadm import UdevadmParser
-from subprocess import check_output, CalledProcessError, STDOUT
+
 
 # This example lists basic information about network interfaces known to NM
 devtypes = {1: "Ethernet",
@@ -132,7 +132,7 @@
             return None
 
         if not stream:
-            print ("Error: modinfo returned nothing", file=sys.stderr)
+            print("Error: modinfo returned nothing", file=sys.stderr)
             return None
         else:
             parser = ModinfoParser(stream)
@@ -166,7 +166,7 @@
 
     # Get a proxy for the base NetworkManager object
     proxy = bus.get_object("org.freedesktop.NetworkManager",
-        "/org/freedesktop/NetworkManager")
+                           "/org/freedesktop/NetworkManager")
     manager = dbus.Interface(proxy, "org.freedesktop.NetworkManager")
 
     # Get all devices known to NM and print their properties
@@ -193,11 +193,12 @@
     two passed in lists, devices from Network Manager and devices from lspci.
     """
     # now check that the count (by type) matches - WiFi
-    nm_type_devices = list(filter(lambda dev: dev.gettype() == devtype,
-        nm_devices))
+    nm_type_devices = [dev for dev in nm_devices if dev.gettype() == devtype]
     udevtype = 'WIRELESS' if devtype == 'WiFi' else 'NETWORK'
-    udev_type_devices = list(filter(lambda dev: dev.category == udevtype,
-        udev_devices))
+    udev_type_devices = [
+        udev
+        for udev in udev_devices
+        if udev.category == udevtype]
     if len(nm_type_devices) != len(udev_type_devices):
         print("ERROR: %s devices missing: udev showed %d devices, but "
               "NetworkManager saw %d devices" % (devtype,