Merge lp:~aacid/unity8/fixruntests into lp:unity8

[Albert Astals Cid]

Make runtests fake a test error if make fails

This means that we will have 2 errors if there is a test that is run and fails, which is sub-optimal, but at least we'll have also a failure if the test doesn't fail because it was not run for example (i.e. missing dbus-launch).

* Are there any related MPs required for this MP to build/function as expected?

In order to keep qmltests green on CI this should be landed together with:
https://code.launchpad.net/~aacid/unity8/use_dbus_test_runner/+merge/257784
https://code.launchpad.net/~aacid/unity8/more_stable_lvwpht/+merge/257749

 * Did you perform an exploratory manual test run of your code change and any related functionality?
Yes

 * Did you make sure that your branch does not contain spurious tags?
Yes

 * If you changed the packaging (debian), did you subscribe the ubuntu-unity team to this MP?
N/A

 * If you changed the UI, has there been a design review?
N/A

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:1753
http://jenkins.qa.ubuntu.com/job/unity8-ci/5638/
Executed test runs:
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-vivid-touch/2524
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-phablet-qmluitests-vivid/801/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity8-vivid-amd64-ci/803
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity8-vivid-i386-ci/803
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-runner-vivid-mako/2187
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/2522
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/2522/artifact/work/output/*zip*/output.zip
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/20020

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity8-ci/5638/rebuild

review: Needs Fixing (continuous-integration)

[lp:~aacid/unity8/fixruntests]

1754. By Albert Astals Cid on 2015-05-04

Instead of failing the .sh create a fake "fail" .xml

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:1754
http://jenkins.qa.ubuntu.com/job/unity8-ci/5671/
Executed test runs:
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-vivid-touch/2618
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/unity-phablet-qmluitests-vivid/836
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity8-vivid-amd64-ci/836/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity8-vivid-i386-ci/836
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-runner-vivid-mako/2267
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/2616
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/2616/artifact/work/output/*zip*/output.zip
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/20159

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity8-ci/5671/rebuild

review: Needs Fixing (continuous-integration)

[Michael Zanetti (mzanetti)]

 * Did you perform an exploratory manual test run of the code change and any related functionality?

This is just for jenkins, and jenkins behaves as expected now.

 * Did CI run pass? If not, please explain why.

no, which is the purpose of this branch.

 * Did you make sure that the branch does not contain spurious tags?

yes

review: Approve

=== modified file 'runtests.sh'
--- runtests.sh	2013-12-17 17:57:07 +0000
+++ runtests.sh	2015-05-04 08:46:50 +0000
@@ -42,6 +42,9 @@
 cd -
 
 make -k qmltests
+if [ $? -ne 0 ]; then
+    echo '<?xml version="1.0" encoding="UTF-8" ?><testsuite errors="0" failures="1" tests="1" name="makeExitStatusTest"><properties/><testcase result="fail" name="makeExitStatus"><failure message="Make test did not suceed" result="fail"/></testcase><system-err/></testsuite>' > testMakeExitStatus.xml
+fi
 
 cd -