Merge lp:~canonical-platform-qa/qa-jenkins-jobs/aut-grafana into lp:qa-jenkins-jobs

[Max Brustkern]

Add grafana uploads to auto-upgrade-testing jobs

This branch adds grafana uploading to the auto-upgrade-testing jobs, and makes changes to the macro to facilitate passing the needed information there. There may be better ways to handle this, so I am open to suggestions. I am also still testing this, so it's not ready to land quite yet, but probably will be shortly.

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:205
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/75/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/241
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/241/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/239
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/239/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1550/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/75/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/qa-jenkins-jobs/aut-grafana]

206. By Max Brustkern on 2017-02-08

Merged trunk

207. By Max Brustkern on 2017-02-08

Additional macro usage

208. By Max Brustkern on 2017-02-08

Finished updates to integrate new trunk

[Max Brustkern (nuclearbob)]

I've tested one of these with the new trunk updates now, and they should be ready to go.

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:208
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/88/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/258
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/258/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/256
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/256/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1572/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/88/rebuild

review: Approve (continuous-integration)

[Sergio Cazzolato (sergio-j-cazzolato)]

A comment inline

[Max Brustkern (nuclearbob)]

I'll try to get to that builder later today.

[lp:~canonical-platform-qa/qa-jenkins-jobs/aut-grafana]

209. By Max Brustkern on 2017-02-09

Updated grafana macro

210. By Max Brustkern on 2017-02-09

Updated jobs for new macro

[Max Brustkern (nuclearbob)]

This now includes/requires this:
https://code.launchpad.net/~canonical-platform-qa/qa-jenkins-jobs/results-type-grafana-upload/+merge/316884

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:210
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/91/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/264
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/264/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/262
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/262/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1580/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/91/rebuild

review: Approve (continuous-integration)

[Sergio Cazzolato (sergio-j-cazzolato)]

comments inline

review: Needs Fixing

[Sergio Cazzolato (sergio-j-cazzolato)]

comment inline

[Max Brustkern (nuclearbob)]

Responses made. If you think YYYYMMDD won't be ambiguous with regard to ISO build numbers, I can add that.

[lp:~canonical-platform-qa/qa-jenkins-jobs/aut-grafana]

211. By Max Brustkern on 2017-02-10

Added date

212. By Max Brustkern on 2017-02-10

Merged trunk

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:211
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/92/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/268
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/268/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/266
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/266/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1582/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/92/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/qa-jenkins-jobs/aut-grafana]

213. By Max Brustkern on 2017-02-10

Added shebang

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:213
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/93/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/269
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-amd64-package/269/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/267
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-xenial-i386-package/267/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1583/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/93/rebuild

review: Approve (continuous-integration)

[Sergio Cazzolato (sergio-j-cazzolato)]

Code lgtm

review: Approve

=== modified file 'jobs/auto-upgrade-testing/jobs.yaml'
--- jobs/auto-upgrade-testing/jobs.yaml	2016-06-22 19:16:00 +0000
+++ jobs/auto-upgrade-testing/jobs.yaml	2017-02-10 15:02:21 +0000
@@ -16,6 +16,12 @@
 # with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 - project:
+    name: 'auto-upgrade-testing-setup'
+    node: 'venonat-upgrade'
+    jobs:
+        - 'auto-upgrade-testing-setup'
+
+- project:
     # Requires: Post Build Script plugin.
     name: auto-upgrade-testing
     # TODO: automatically generate this list, possibly with a blacklist
@@ -33,24 +39,46 @@
         # This runs daily and allows jenkins to stagger the jobs
         - timed: 'H H * * *'
     builders:
+        - clear-artifacts:
+        - branch-bzr-project:
+            project-url: "lp:auto-upgrade-testing-specifications"
+            project-name: "auto-upgrade-testing-specifications"
+        - branch-bzr-project:
+            project-url: "lp:auto-upgrade-testing"
+            project-name: "auto-upgrade-testing"
         - shell: |
-            #TODO: get rid of all sudo here
-            sudo rm -rf *
-            # Branch the profile details
-            export PROFILES_BRANCH=lp:auto-upgrade-testing-specifications
-            export PROFILE_BRANCH_DIR=$WORKSPACE/testing-profiles
-            bzr branch $PROFILES_BRANCH $PROFILE_BRANCH_DIR
-
             # Construct profile usage
             export PROFILE_NAME="{profile}.yaml"
-            export PROFILE=$PROFILE_BRANCH_DIR/profiles/$PROFILE_NAME
-
-            export BRANCH=lp:auto-upgrade-testing # Until this is packaged, we branch for each run
-            export BRANCH_DIR=$WORKSPACE/auto-upgrade-testing
-            bzr branch $BRANCH $BRANCH_DIR
-            cd $BRANCH_DIR
-
-            sudo python3 -m upgrade_testing.command_line -c $PROFILE --provision --results-dir "$WORKSPACE/{profile}/"
+            export PROFILE=$WORKSPACE/auto-upgrade-testing-specifications/profiles/$PROFILE_NAME
+
+            # Write parameters to files for grafana upload
+            OS=$(grep flavor $PROFILE | sed 's/^.*flavor: //')
+            ARCH=$(grep arch $PROFILE | sed 's/^.*arch: //')
+            RELEASE=$(grep -m1 -A1 "releases:" $PROFILE | tail -1 | grep -o "\"[^\"]*\"" | tr -d '"')
+            BUILD=$(date +%Y%m%d)
+            cat > global_info.json <<EOF
+            {{
+                "device": "vm",
+                "model": "",
+                "os": "$OS",
+                "release": "$RELEASE",
+                "arch": "$ARCH",
+                "class": "upgrade",
+                "build": "$BUILD"
+            }}
+            EOF
+
+            cd auto-upgrade-testing
+
+            set +e
+            sudo python3 -m upgrade_testing.command_line -c $PROFILE --provision --results-dir "$WORKSPACE/ubuntu-trusty-xenial-basic-amd64_qemu/"
+            EXITSTATUS=$?
+            cp $WORKSPACE/ubuntu-trusty-xenial-basic-amd64_qemu/*/artifacts/upgrade_run/junit.xml $WORKSPACE/junit.xml
+            exit $EXITSTATUS
+        - grafana-tests-upload:
+            results_type: 'xunit'
+            results_file: 'junit.xml'
+            info_file: 'global_info.json'
     publishers:
         - postbuildscript:
             builders:
@@ -60,3 +88,13 @@
         - archive:
             artifacts: '**/*'
             allow-empty: 'true'
+        - junit:
+            results: 'junit.xml'
+- job:
+    name: 'auto-upgrade-testing-setup'
+    description: |
+        This job ensures we have everything needed to run auto-upgrade-testing
+        It can be run again to update to the latest versions of dependencies
+        Defined in lp:qa-jenkins-jobs
+    builders:
+        shell: pip3 install --user junitparser

=== modified file 'jobs/macros.yaml'
--- jobs/macros.yaml	2017-02-09 17:24:54 +0000
+++ jobs/macros.yaml	2017-02-10 15:02:21 +0000
@@ -43,12 +43,9 @@
             EOF
 
 - builder:
-    name: grafana-tests-upload
+    name: grafana-info
     builders:
-        - grafana-setup:
         - shell: |
-            #!/bin/bash
-            source $WORKSPACE/grafana.conf
             if [ -z "$BUILDSTAMP" ] ; then
                 BUILDSTAMP=$(grep -o '([0-9.]\+)' media_info | tr -d '()')
             fi
@@ -63,7 +60,15 @@
                 "build": "$BUILDSTAMP"
             }}
             EOF
-            $GRAFANA_INTEGRATION_PATH/run_script sync-tests -rt {results_type} -rf {results_file} -if global_info.json -m tests_01 -t $(date +%s)
+
+- builder:
+    name: grafana-tests-upload
+    builders:
+        - grafana-setup:
+        - shell: |
+            #!/bin/bash -ex
+            source $WORKSPACE/grafana.conf
+            $GRAFANA_INTEGRATION_PATH/run_script sync-tests -rt {results_type} -rf {results_file} -if {info_file} -m tests_01 -t $(date +%s)
 
 - publisher: # This publisher archives all job artifacts
     name: 'archive-artifacts'

=== modified file 'jobs/ubiquity/jobs.yaml'
--- jobs/ubiquity/jobs.yaml	2017-02-09 17:24:54 +0000
+++ jobs/ubiquity/jobs.yaml	2017-02-10 15:02:21 +0000
@@ -120,15 +120,17 @@
             ubiquity_branch: '{ubiquity_branch}'
         - generate-junit-files:
             source_dir: 'results/var/local/autopilot/junit'
-        - grafana-tests-upload:
-            results_type: 'xunit'
-            results_file: 'results/var/local/autopilot/junit/*.xml'
+        - grafana-info:
             device: 'vm'
             model: ''
             os: '{flavor}'
             release: '{release}'
             arch: '{arch}'
             class: 'ubiquity'
+        - grafana-tests-upload:
+            results_type: 'xunit'
+            results_file: 'results/var/local/autopilot/junit/*.xml'
+            info_file: 'global_info.json'
 
     publishers:
         - archive-artifacts