Merge lp:~canonical-platform-qa/qa-jenkins-jobs/ust-style-improvements into lp:qa-jenkins-jobs

[Allan LeSage]

Style improvements for ubuntu-system-tests.

Improve names, use anchor-based inheritance, include scripty builders instead of listing in-job.

[Christopher Lee (veebers)]

LGTM I loaded it into my test jenkins instance and all seemed fine.
I like the use of the templating for the jobs.

review: Approve

[Max Brustkern (nuclearbob)]

Let's make sure the _PT jobs are removed if we're deprecating those in favor of _test.

I see on ust_{channel}_{device}_{test_level}, we now provide device, but not node. Do we no longer need to link jobs to nodes?

Also, it looks like you define &DEFAULT_UST_PUBLISHERS but don't use it yet. That makes sense to me, since it seems likely we may want to add arbitrary new publishers to jobs in the future, I just want to make sure there's not a use of that somewhere that I'm missing.

[lp:~canonical-platform-qa/qa-jenkins-jobs/ust-style-improvements]

59. By Allan LeSage on 2016-04-28

Remove default publishers.

[Allan LeSage (allanlesage)]

The device property is set in the "anchor" slash parent template, although this is not easy to see in the diff.

Removed &DEFAULT_UST_PUBLISHERS b/c I was getting ahead of myself.

One more round Max!

[Max Brustkern (nuclearbob)]

I see it now! Looks good!

review: Approve

=== added file 'ubuntu-system-tests/flash-device-builder.sh'
--- ubuntu-system-tests/flash-device-builder.sh	1970-01-01 00:00:00 +0000
+++ ubuntu-system-tests/flash-device-builder.sh	2016-04-28 20:18:56 +0000
@@ -0,0 +1,11 @@
+# use plars' lifeboat scripts to get our adb serial
+rm -rf lifeboat
+git clone $LIFEBOAT_BRANCH lifeboat
+export ANDROID_SERIAL=$(./lifeboat/get-device-info serial $NODE_NAME)
+
+rm -rf qa-jenkins-jobs
+bzr branch $QA_JENKINS_JOBS_BRANCH qa-jenkins-jobs
+
+./qa-jenkins-jobs/scripts/get-recovery-image.sh
+./qa-jenkins-jobs/scripts/recover-device.sh
+./qa-jenkins-jobs/scripts/flash-device.sh

=== modified file 'ubuntu-system-tests/jobs.yaml'
--- ubuntu-system-tests/jobs.yaml	2016-04-26 15:38:52 +0000
+++ ubuntu-system-tests/jobs.yaml	2016-04-28 20:18:56 +0000
@@ -31,8 +31,8 @@
             device_channel: 'meizu.en'
     jobs:
         - 'flash-device'
-        - 'ust_{channel}_{device}_{test_level}_PT'
         - 'ust_{channel}_{device}_{test_level}'
+        - 'ust_{channel}_{device}_{test_level}_test'
 
 - job-template:
     name: 'flash-device'
@@ -64,22 +64,11 @@
 - builder:
     name: flash-device-builder
     builders:
-        - shell: |
-            # use plars' lifeboat scripts to get our adb serial
-            rm -rf lifeboat
-            git clone $LIFEBOAT_BRANCH lifeboat
-            export ANDROID_SERIAL=$(./lifeboat/get-device-info serial $NODE_NAME)
-
-            rm -rf qa-jenkins-jobs
-            bzr branch $QA_JENKINS_JOBS_BRANCH qa-jenkins-jobs
-
-            ./qa-jenkins-jobs/scripts/get-recovery-image.sh
-            ./qa-jenkins-jobs/scripts/recover-device.sh
-            ./qa-jenkins-jobs/scripts/flash-device.sh
-
-- job-template:
-    name: 'ust_{channel}_{device}_{test_level}_PT'
-    description: "Triggered upon image update."
+        - shell:
+            !include-raw: 'flash-device-builder.sh'
+
+- job-template: &UST_DEFAULT_JOB_TEMPLATE
+    name: 'ust_default_job_template'
     parameters:
         - ubuntu-system-tests-parameters:
             test_level: '{test_level}'
@@ -103,24 +92,21 @@
             results: 'results/artifacts/test-results.xml'
 
 - job-template:
+    <<: *UST_DEFAULT_JOB_TEMPLATE
+    name: 'ust_{channel}_{device}_{test_level}_test'
+    description: |
+        Provided for experimentation--e.g. re-running of failed tests.
+
+- job-template:
+    <<: *UST_DEFAULT_JOB_TEMPLATE
     name: 'ust_{channel}_{device}_{test_level}'
-    description: "Provided for experimentation--e.g. re-running of failed tests--in way that won't contribute to the pass/fail record of the canonical job."
-    parameters:
-        - ubuntu-system-tests-parameters:
-            test_level: '{test_level}'
+    description: |
+        Triggered by image publication, reports results to PractiTest.
+    triggers:
+        - ubuntu-system-tests-triggers:
             channel: '{channel}'
+            device: '{device}'
             device_channel: '{device_channel}'
-    node: '{device}'
-    builders:
-        - clear-artifacts
-        - flash-device-builder
-        - ubuntu-system-tests-builder
-    publishers:
-        - archive:
-            artifacts: 'results/**/*'
-            allow-empty: 'true'
-        - junit:
-            results: 'results/artifacts/test-results.xml'
 
 - parameter:
     name: ubuntu-system-tests-parameters
@@ -173,65 +159,5 @@
 - builder:
     name: ubuntu-system-tests-builder
     builders:
-        - shell: |
-            # use plars' lifeboat scripts to get our adb serial
-            rm -rf lifeboat
-            # we occasionally see 500s from Launchpad git service
-            GIT_RETRY=0
-            until git clone $LIFEBOAT_BRANCH lifeboat || [ $GIT_RETRY -gt 10 ]; do
-                sleep 1
-                GIT_RETRY=$(($GIT_RETRY+1))
-                echo "git clone lifeboat attempt failed, trying again . . .'
-            done
-            if [ $GIT_RETRY -gt 10 ]; then
-                echo "error: git clone lifeboat failed!"
-                exit 1
-            fi
-
-            export ANDROID_SERIAL=$(./lifeboat/get-device-info serial $NODE_NAME)
-
-            rm -rf $WORKSPACE/results
-            mkdir $WORKSPACE/results
-            export OUTPUTDIR="$WORKSPACE/results"
-
-            rm -rf qa-jenkins-jobs
-            bzr branch $QA_JENKINS_JOBS_BRANCH qa-jenkins-jobs
-
-            export UBUNTU_SYSTEM_TESTS_CONFIG_BRANCH="lp:ubuntu-system-tests-config"
-            export CONFIG_PATH=${WORKSPACE}/ubuntu-system-tests.conf
-            ./qa-jenkins-jobs/scripts/ubuntu-system-tests/write-config.sh
-            cat >> $CONFIG_PATH <<EOF
-            device_serial = $ANDROID_SERIAL
-            output_dir = $OUTPUTDIR
-            EOF
-            cat $CONFIG_PATH
-
-            case "$TESTS_TO_RUN" in
-              "regression")
-                TESTS_TO_RUN=""
-                ;;
-              "sanity")
-                TESTS_TO_RUN=$(bzr cat "$UBUNTU_SYSTEM_TESTS_BRANCH/test_lists/sanity.tests" | tr '\n' ' ')
-                ;;
-              "")
-                echo "Please specify tests to run!" >&2
-                exit 1
-                ;;
-            esac
-
-            recover_device_flash_and_run_tests () {
-              ./qa-jenkins-jobs/scripts/ubuntu-system-tests/run-ubuntu-system-tests.sh "$1"
-            }
-            recover_device_flash_and_run_tests "$(echo "$TESTS_TO_RUN")"
-
-            # default to sanity for testing, TODO: consider exiting if not sanity or regression
-            case "${TESTS_TO_RUN}" in
-              "regression")
-                export TEST_LEVEL=regression
-                ;;
-              *)
-                export TEST_LEVEL=sanity
-                ;;
-            esac
-            # "1SS" is the name of the lab
-            ./qa-jenkins-jobs/scripts/ubuntu-system-tests/report-results-to-practitest.sh $WORKSPACE/results/artifacts/test-results.subunit $TEST_LEVEL 1SS
+        - shell:
+            !include-raw: 'ubuntu-system-tests-builder.sh'

=== added file 'ubuntu-system-tests/ubuntu-system-tests-builder.sh'
--- ubuntu-system-tests/ubuntu-system-tests-builder.sh	1970-01-01 00:00:00 +0000
+++ ubuntu-system-tests/ubuntu-system-tests-builder.sh	2016-04-28 20:18:56 +0000
@@ -0,0 +1,50 @@
+# use plars' lifeboat scripts to get our adb serial
+rm -rf lifeboat
+git clone $LIFEBOAT_BRANCH lifeboat
+export ANDROID_SERIAL=$(./lifeboat/get-device-info serial $NODE_NAME)
+
+rm -rf $WORKSPACE/results
+mkdir $WORKSPACE/results
+export OUTPUTDIR="$WORKSPACE/results"
+
+rm -rf qa-jenkins-jobs
+bzr branch $QA_JENKINS_JOBS_BRANCH qa-jenkins-jobs
+
+export UBUNTU_SYSTEM_TESTS_CONFIG_BRANCH="lp:ubuntu-system-tests-config"
+export CONFIG_PATH=${WORKSPACE}/ubuntu-system-tests.conf
+./qa-jenkins-jobs/scripts/ubuntu-system-tests/write-config.sh
+cat >> $CONFIG_PATH <<EOF
+device_serial = $ANDROID_SERIAL
+output_dir = $OUTPUTDIR
+EOF
+cat $CONFIG_PATH
+
+case "$TESTS_TO_RUN" in
+  "regression")
+    TESTS_TO_RUN=""
+    ;;
+  "sanity")
+    TESTS_TO_RUN=$(bzr cat "$UBUNTU_SYSTEM_TESTS_BRANCH/test_lists/sanity.tests" | tr '\n' ' ')
+    ;;
+  "")
+    echo "Please specify tests to run!" >&2
+    exit 1
+    ;;
+esac
+
+recover_device_flash_and_run_tests () {
+  ./qa-jenkins-jobs/scripts/ubuntu-system-tests/run-ubuntu-system-tests.sh "$1"
+}
+recover_device_flash_and_run_tests "$(echo "$TESTS_TO_RUN")"
+
+# default to sanity for testing, TODO: consider exiting if not sanity or regression
+case "${TESTS_TO_RUN}" in
+  "regression")
+    export TEST_LEVEL=regression
+    ;;
+  *)
+    export TEST_LEVEL=sanity
+    ;;
+esac
+# "1SS" is the name of the lab
+./qa-jenkins-jobs/scripts/ubuntu-system-tests/report-results-to-practitest.sh $WORKSPACE/results/artifacts/test-results.subunit $TEST_LEVEL 1SS