QA Jenkins Jobs

Merge lp:~canonical-platform-qa/qa-jenkins-jobs/app-launch-tests-trigger into lp:qa-jenkins-jobs

  1. app-launch-tests-trigger
  2. Merge into trunk
Proposed by Richard Huddie
Status: Work in progress
Proposed branch: lp:~canonical-platform-qa/qa-jenkins-jobs/app-launch-tests-trigger
Merge into: lp:qa-jenkins-jobs
Diff against target: 351 lines (+240/-12)
4 files modified
jobs/ubuntu-system-tests/jobs.yaml (+84/-8)
jobs/ubuntu-system-tests/new_version_published_trigger.py (+154/-0)
jobs/ubuntu-system-tests/ubuntu-system-tests-image-builder.sh (+1/-2)
jobs/ubuntu-system-tests/ubuntu-system-tests-run-tests-builder.sh (+1/-2)
To merge this branch: bzr merge lp:~canonical-platform-qa/qa-jenkins-jobs/app-launch-tests-trigger
Reviewer Review Type Date Requested Status
Canonical Platform QA Jenkins Pending
Review via email: mp+317458@code.launchpad.net
To post a comment you must log in.
lp:~canonical-platform-qa/qa-jenkins-jobs/app-launch-tests-trigger updated
223. By Richard Huddie

Add suite to the build image parameters.

224. By Richard Huddie

Remove the trigger jobs and replace with image builder triggers.

225. By Richard Huddie

Add jobs for individual apps.

226. By Richard Huddie

Add transformer class to evaluate suites.

Unmerged revisions

226. By Richard Huddie

Add transformer class to evaluate suites.

225. By Richard Huddie

Add jobs for individual apps.

224. By Richard Huddie

Remove the trigger jobs and replace with image builder triggers.

223. By Richard Huddie

Add suite to the build image parameters.

222. By Richard Huddie

Use reverse trigger to avoid specifying suite.

221. By Richard Huddie

Add common params to the build image trigger.

220. By Richard Huddie

Add trigger job.

219. By Richard Huddie

Use a list of snap names as parameter.

218. By Richard Huddie

Use a single json file to save results.

217. By Richard Huddie

Python snap revision checker working.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'jobs/ubuntu-system-tests/jobs.yaml'
--- jobs/ubuntu-system-tests/jobs.yaml 2017-02-14 16:32:35 +0000
+++ jobs/ubuntu-system-tests/jobs.yaml 2017-02-22 13:47:51 +0000
@@ -18,8 +18,50 @@
18- project:18- project:
19 name: ubuntu-system-tests19 name: ubuntu-system-tests
20 suite:20 suite:
21 - app-launch:21 - launch-all-apps:
22 filters: 'filter:169737 filter:169738'22 filters: 'filter:169737 filter:169738'
23 snap-triggers: 'unity8-session ubuntu-app-platform'
24 deb-triggers: 'unity8'
25 - launch-addressbook-app:
26 filters: 'filter:171066'
27 snap-triggers: 'address-book-app'
28 deb-triggers: 'address-book-app'
29 - launch-calculator-app:
30 filters: 'filter:171072'
31 snap-triggers: 'ubuntu-calculator-app'
32 deb-triggers: ''
33 - launch-calendar-app:
34 filters: 'filter:171073'
35 snap-triggers: 'ubuntu-calendar-app'
36 deb-triggers: ''
37 - launch-camera-app:
38 filters: 'filter:171069'
39 snap-triggers: 'camera-app'
40 deb-triggers: ''
41 - launch-clock-app:
42 filters: 'filter:171074'
43 snap-triggers: 'ubuntu-clock-app'
44 deb-triggers: ''
45 - launch-filemanager-app:
46 filters: 'filter:171075'
47 snap-triggers: 'ubuntu-filemanager-app'
48 deb-triggers: ''
49 - launch-gallery-app:
50 filters: 'filter:171070'
51 snap-triggers: 'gallery-app'
52 deb-triggers: 'gallery-app'
53 - launch-systemsettings-app:
54 filters: 'filter:171077'
55 snap-triggers: 'unity8-session'
56 deb-triggers: 'ubuntu-system-settings'
57 - launch-terminal-app:
58 filters: 'filter:171076'
59 snap-triggers: 'ubuntu-terminal-app'
60 deb-triggers: 'ubuntu-terminal-app'
61 - launch-webbrowser-app:
62 filters: 'filter:171004'
63 snap-triggers: 'webbrowser-app'
64 deb-triggers: 'webbrowser-app'
23 flavor:65 flavor:
24 - ubuntu66 - ubuntu
25 release:67 release:
@@ -36,7 +78,7 @@
36 - snap78 - snap
37 - deb79 - deb
38 jobs:80 jobs:
39 - 'ust-build-image-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'81 - 'ust-build-image-{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'
40 - 'ust-test-{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'82 - 'ust-test-{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'
4183
42- job-template: &UST_APP_LAUNCH_JOB_TEMPLATE84- job-template: &UST_APP_LAUNCH_JOB_TEMPLATE
@@ -46,7 +88,7 @@
4688
47- job-template:89- job-template:
48 <<: *UST_APP_LAUNCH_JOB_TEMPLATE90 <<: *UST_APP_LAUNCH_JOB_TEMPLATE
49 name: 'ust-build-image-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'91 name: 'ust-build-image-{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'
50 description: 'Build image for device.'92 description: 'Build image for device.'
51 parameters:93 parameters:
52 - ubuntu-system-tests-common-parameters:94 - ubuntu-system-tests-common-parameters:
@@ -56,10 +98,20 @@
56 flavor: '{flavor}'98 flavor: '{flavor}'
57 model: '{model}'99 model: '{model}'
58 arch: '{arch}'100 arch: '{arch}'
101 suite: '{suite}'
102 - ubuntu-system-tests-trigger-parameters:
103 snap-triggers: '{snap-triggers}'
104 deb-triggers: '{deb-triggers}'
59 builders:105 builders:
60 - clear-artifacts106 - clear-artifacts
61 - ubuntu-system-tests-config-builder107 - ubuntu-system-tests-config-builder
62 - ubuntu-system-tests-image-builder108 - ubuntu-system-tests-image-builder
109 triggers:
110 - script:
111 # This is every minute!
112 cron: '* * * * *'
113 script:
114 !include-raw: 'new_version_published_trigger.py'
63 publishers:115 publishers:
64 - trigger:116 - trigger:
65 project: 'ust-test-{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'117 project: 'ust-test-{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'
@@ -80,9 +132,9 @@
80 flavor: '{flavor}'132 flavor: '{flavor}'
81 model: '{model}'133 model: '{model}'
82 arch: '{arch}'134 arch: '{arch}'
135 suite: '{suite}'
83 - ubuntu-system-tests-run-parameters:136 - ubuntu-system-tests-run-parameters:
84 filters: '{filters}'137 filters: '{filters}'
85 suite: '{suite}'
86 builders:138 builders:
87 - clear-artifacts139 - clear-artifacts
88 - ubuntu-system-tests-config-builder140 - ubuntu-system-tests-config-builder
@@ -135,6 +187,14 @@
135 name: DEVICE187 name: DEVICE
136 default: '{device}'188 default: '{device}'
137 description: Name of device, either vm or adb.189 description: Name of device, either vm or adb.
190 - string:
191 name: SUITE
192 default: '{suite}'
193 description: Name of the test suite being run.
194 - string:
195 name: SHARED_IMAGE_PATH
196 default: '/var/cache/ust/img/{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}.img'
197 description: Path of the image file to use for testing.
138198
139- parameter:199- parameter:
140 name: ubuntu-system-tests-run-parameters200 name: ubuntu-system-tests-run-parameters
@@ -143,10 +203,26 @@
143 name: FILTERS203 name: FILTERS
144 default: '{filters}'204 default: '{filters}'
145 description: List of Practitest filters IDs to look-up required test IDs.205 description: List of Practitest filters IDs to look-up required test IDs.
146 - string:206
147 name: SUITE207- parameter:
148 default: '{suite}'208 name: ubuntu-system-tests-trigger-parameters
149 description: Name of the test suite being run.209 parameters:
210 - string:
211 name: SNAP_TRIGGERS
212 default: '{snap-triggers}'
213 description: List of snap names to monitor for updates.
214 - string:
215 name: DEB_TRIGGERS
216 default: '{deb-triggers}'
217 description: List of deb names to monitor for updates.
218 - string:
219 name: SNAP_CHANNEL
220 default: 'edge'
221 description: Channel to install snaps from.
222 - string:
223 name: CORE_SERIES
224 default: '16'
225 description: Series of Ubuntu-core used.
150226
151- builder:227- builder:
152 name: ubuntu-system-tests-config-builder228 name: ubuntu-system-tests-config-builder
153229
=== added file 'jobs/ubuntu-system-tests/new_version_published_trigger.py'
--- jobs/ubuntu-system-tests/new_version_published_trigger.py 1970-01-01 00:00:00 +0000
+++ jobs/ubuntu-system-tests/new_version_published_trigger.py 2017-02-22 13:47:51 +0000
@@ -0,0 +1,154 @@
1#!/usr/bin/python3
2
3import ast
4import json
5import os
6import requests
7import sys
8
9from collections import OrderedDict
10
11STORE_QUERY_URL = (
12 'https://search.apps.ubuntu.com/api/v1/snaps/details/{{n}}?channel={{c}}'
13)
14BUILD_TRIGGER_FOLDER = 'build-triggers'
15SNAP_DATA_FILE_NAME = 'snap-data.json'
16KEY_SNAP = '{{n}}-{{a}}-{{s}}-{{c}}'
17KEY_REV = 'revision'
18STATUS_TRIGGER_NEW_BUILD = 0
19STATUS_NO_NEW_BUILD = 1
20
21
22class Transformer(ast.NodeTransformer):
23 """Class to safely evaluate strings containing specified Python values."""
24
25 ALLOWED_NAMES = set(['OrderedDict'])
26 ALLOWED_NODE_TYPES = set([
27 'Expression',
28 'Tuple',
29 'Call',
30 'Name',
31 'Load',
32 'Str',
33 'Num',
34 'List',
35 'Dict',
36 ])
37
38 def visit_Name(self, node):
39 """Check that name is allowed and traverse onto any child nodes."""
40 if node.id not in self.ALLOWED_NAMES:
41 raise RuntimeError(
42 'Name access to {{n}} is not allowed'.format(n=node.id))
43 return self.generic_visit(node)
44
45 def generic_visit(self, node):
46 """Check that node type is allowed visit all child nodes."""
47 node_type = type(node).__name__
48 if node_type not in self.ALLOWED_NODE_TYPES:
49 raise RuntimeError(
50 'Invalid expression: {{n}}'.format(n=node_type))
51 return ast.NodeTransformer.generic_visit(self, node)
52
53 def evaluate(self, source):
54 """Safely evaluate the source string and return evaluated object."""
55 tree = ast.parse(source, mode='eval')
56 self.visit(tree)
57 compiled = compile(tree, '<AST>', 'eval')
58 return eval(compiled, dict(OrderedDict=OrderedDict))
59
60
61def get_build_trigger_path(work_dir, *args):
62 """Return path of build trigger folder.
63 :param work_dir: Work directory being used.
64 :param args: Any other file names to append to path.
65 """
66 return os.path.join(work_dir, BUILD_TRIGGER_FOLDER, *args)
67
68
69def get_snap_data_filepath(work_dir):
70 """Return dictionary of all saved snap data."""
71 return os.path.join(work_dir, SNAP_DATA_FILE_NAME)
72
73
74def get_snap_data(work_dir):
75 """Return dictionary of all saved snap data."""
76 file_name = get_snap_data_filepath(work_dir)
77 if not os.path.isfile(file_name):
78 return {{}}
79 with open(file_name) as f:
80 data = f.read()
81 return json.loads(data) if data else {{}}
82
83
84def get_published_snap_revision(name, arch, series, channel):
85 """Return revision of specified published snap."""
86 url = STORE_QUERY_URL.format(n=name, c=channel)
87 headers = {{'X-Ubuntu-Series': series, 'X-Ubuntu-Architecture': arch}}
88 return int(requests.get(url, headers=headers).json()[KEY_REV])
89
90
91def get_current_snap_revision(snap_data, name, arch, series, channel):
92 """Return revision of currently tested snap."""
93 key_snap = KEY_SNAP.format(n=name, a=arch, s=series, c=channel)
94 try:
95 rev = int(snap_data[key_snap][KEY_REV])
96 except (KeyError, TypeError, ValueError):
97 rev = -1
98 return rev
99
100
101def save_published_snap_revision(work_dir, snap_data, name, arch, series,
102 channel, revision):
103 """Save the published snap revision for future comparison."""
104 key_snap = KEY_SNAP.format(n=name, a=arch, s=series, c=channel)
105 if key_snap not in snap_data:
106 snap_data[key_snap] = {{}}
107 snap_data[key_snap][KEY_REV] = revision
108 file_path = get_snap_data_filepath(work_dir)
109 with open(file_path, 'w') as f:
110 f.write(json.dumps(snap_data))
111
112
113def run_snap_trigger_check(work_dir, names, arch, series, channel):
114 """Return status indicating if a new revision of snap is published."""
115 status = STATUS_NO_NEW_BUILD
116 # Check through each app to see if new version available.
117 # Update revision in snap data to keep saved data up to date.
118 for name in names:
119 snap_data = get_snap_data(work_dir)
120 current = get_current_snap_revision(
121 snap_data, name, arch, series, channel)
122 published = get_published_snap_revision(name, arch, series, channel)
123 if published > current:
124 save_published_snap_revision(
125 work_dir, snap_data, name, arch, series, channel, published)
126 status = STATUS_TRIGGER_NEW_BUILD
127 return status
128
129
130def main():
131 # Currently this only supports checking for new snap version.
132 # TODO: Add checking for new deb version.
133 suites = Transformer().evaluate(os.getenv('SUITE'))
134 for suite in suites:
135 for key in suite.keys():
136 print('*************')
137 print(key)
138 print(suite[key])
139 print(suite[key]['filters'])
140 print(suite[key]['snap-triggers'])
141 print(suite[key]['deb-triggers'])
142 return STATUS_NO_NEW_BUILD
143 if os.getenv('UNITY8_MODE') == 'snap':
144 work_dir = os.getenv('WORKSPACE')
145 names = os.getenv('SNAP_NAMES').split()
146 arch = os.getenv('ARCH')
147 series = os.getenv('CORE_SERIES')
148 channel = os.getenv('SNAP_CHANNEL')
149 return run_snap_trigger_check(work_dir, names, arch, series, channel)
150 return STATUS_NO_NEW_BUILD
151
152
153if __name__ == "__main__":
154 sys.exit(main())
0155
=== modified file 'jobs/ubuntu-system-tests/ubuntu-system-tests-image-builder.sh'
--- jobs/ubuntu-system-tests/ubuntu-system-tests-image-builder.sh 2017-02-14 15:39:47 +0000
+++ jobs/ubuntu-system-tests/ubuntu-system-tests-image-builder.sh 2017-02-22 13:47:51 +0000
@@ -4,7 +4,6 @@
4export UBUNTU_SYSTEM_TESTS_BRANCH=lp:ubuntu-system-tests4export UBUNTU_SYSTEM_TESTS_BRANCH=lp:ubuntu-system-tests
5export QA_JENKINS_JOBS_BRANCH=lp:qa-jenkins-jobs5export QA_JENKINS_JOBS_BRANCH=lp:qa-jenkins-jobs
6export ISO_PATH=/var/cache/utah/iso/$RELEASE-$MODEL-$ARCH.iso6export ISO_PATH=/var/cache/utah/iso/$RELEASE-$MODEL-$ARCH.iso
7export SHARED_IMG_PATH=/var/cache/ust/img/$RELEASE-$MODEL-$ARCH-unity8-$UNITY8_MODE.img
8export OUTPUTDIR=$WORKSPACE/results7export OUTPUTDIR=$WORKSPACE/results
9export CACHE_PATH=$WORKSPACE/.ust-cache8export CACHE_PATH=$WORKSPACE/.ust-cache
10export IMG_PATH=$CACHE_PATH/disk.img9export IMG_PATH=$CACHE_PATH/disk.img
@@ -31,6 +30,6 @@
31python3 -m ubuntu_system_tests.run setup --headless --silent --config $CONFIG_PATH --img $IMG_PATH30python3 -m ubuntu_system_tests.run setup --headless --silent --config $CONFIG_PATH --img $IMG_PATH
3231
33# copy the resulting image file to shared location for use by tests32# copy the resulting image file to shared location for use by tests
34sudo cp $IMG_PATH $SHARED_IMG_PATH33sudo cp $IMG_PATH $SHARED_IMAGE_PATH
3534
36deactivate # our virtualenv35deactivate # our virtualenv
3736
=== modified file 'jobs/ubuntu-system-tests/ubuntu-system-tests-run-tests-builder.sh'
--- jobs/ubuntu-system-tests/ubuntu-system-tests-run-tests-builder.sh 2017-02-14 15:39:47 +0000
+++ jobs/ubuntu-system-tests/ubuntu-system-tests-run-tests-builder.sh 2017-02-22 13:47:51 +0000
@@ -2,7 +2,6 @@
2set -x2set -x
33
4export UBUNTU_SYSTEM_TESTS_BRANCH=lp:ubuntu-system-tests4export UBUNTU_SYSTEM_TESTS_BRANCH=lp:ubuntu-system-tests
5export SHARED_IMG_PATH=/var/cache/ust/img/$RELEASE-$MODEL-$ARCH-unity8-$UNITY8_MODE.img
6export OUTPUTDIR=$WORKSPACE/results5export OUTPUTDIR=$WORKSPACE/results
7export CONFIG_PATH=$WORKSPACE/ubuntu-system-tests.conf6export CONFIG_PATH=$WORKSPACE/ubuntu-system-tests.conf
8export https_proxy=https://squid.internal:3128/7export https_proxy=https://squid.internal:3128/
@@ -22,7 +21,7 @@
2221
23# Run tests from each test filter22# Run tests from each test filter
24for FILTER in $FILTERS; do23for FILTER in $FILTERS; do
25 python3 -m ubuntu_system_tests.run run --headless --silent --config $CONFIG_PATH --img $SHARED_IMG_PATH $APPEND_RESULTS --filters $FILTER24 python3 -m ubuntu_system_tests.run run --headless --silent --config $CONFIG_PATH --img $SHARED_IMAGE_PATH $APPEND_RESULTS --filters $FILTER
26 # Use --append-results arg for all test runs but the first25 # Use --append-results arg for all test runs but the first
27 if [ -z $APPEND_RESULTS ]; then26 if [ -z $APPEND_RESULTS ]; then
28 APPEND_RESULTS="--append-results"27 APPEND_RESULTS="--append-results"

Subscribers

People subscribed via source and target branches