Merge lp:~canonical-platform-qa/qa-jenkins-jobs/app-launch-tests-trigger into lp:qa-jenkins-jobs

Proposed by Richard Huddie
Status: Work in progress
Proposed branch: lp:~canonical-platform-qa/qa-jenkins-jobs/app-launch-tests-trigger
Merge into: lp:qa-jenkins-jobs
Diff against target: 351 lines (+240/-12)
4 files modified
jobs/ubuntu-system-tests/jobs.yaml (+84/-8)
jobs/ubuntu-system-tests/new_version_published_trigger.py (+154/-0)
jobs/ubuntu-system-tests/ubuntu-system-tests-image-builder.sh (+1/-2)
jobs/ubuntu-system-tests/ubuntu-system-tests-run-tests-builder.sh (+1/-2)
To merge this branch: bzr merge lp:~canonical-platform-qa/qa-jenkins-jobs/app-launch-tests-trigger
Reviewer Review Type Date Requested Status
Canonical Platform QA Jenkins Pending
Review via email: mp+317458@code.launchpad.net
To post a comment you must log in.
223. By Richard Huddie

Add suite to the build image parameters.

224. By Richard Huddie

Remove the trigger jobs and replace with image builder triggers.

225. By Richard Huddie

Add jobs for individual apps.

226. By Richard Huddie

Add transformer class to evaluate suites.

Unmerged revisions

226. By Richard Huddie

Add transformer class to evaluate suites.

225. By Richard Huddie

Add jobs for individual apps.

224. By Richard Huddie

Remove the trigger jobs and replace with image builder triggers.

223. By Richard Huddie

Add suite to the build image parameters.

222. By Richard Huddie

Use reverse trigger to avoid specifying suite.

221. By Richard Huddie

Add common params to the build image trigger.

220. By Richard Huddie

Add trigger job.

219. By Richard Huddie

Use a list of snap names as parameter.

218. By Richard Huddie

Use a single json file to save results.

217. By Richard Huddie

Python snap revision checker working.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'jobs/ubuntu-system-tests/jobs.yaml'
2--- jobs/ubuntu-system-tests/jobs.yaml 2017-02-14 16:32:35 +0000
3+++ jobs/ubuntu-system-tests/jobs.yaml 2017-02-22 13:47:51 +0000
4@@ -18,8 +18,50 @@
5 - project:
6 name: ubuntu-system-tests
7 suite:
8- - app-launch:
9+ - launch-all-apps:
10 filters: 'filter:169737 filter:169738'
11+ snap-triggers: 'unity8-session ubuntu-app-platform'
12+ deb-triggers: 'unity8'
13+ - launch-addressbook-app:
14+ filters: 'filter:171066'
15+ snap-triggers: 'address-book-app'
16+ deb-triggers: 'address-book-app'
17+ - launch-calculator-app:
18+ filters: 'filter:171072'
19+ snap-triggers: 'ubuntu-calculator-app'
20+ deb-triggers: ''
21+ - launch-calendar-app:
22+ filters: 'filter:171073'
23+ snap-triggers: 'ubuntu-calendar-app'
24+ deb-triggers: ''
25+ - launch-camera-app:
26+ filters: 'filter:171069'
27+ snap-triggers: 'camera-app'
28+ deb-triggers: ''
29+ - launch-clock-app:
30+ filters: 'filter:171074'
31+ snap-triggers: 'ubuntu-clock-app'
32+ deb-triggers: ''
33+ - launch-filemanager-app:
34+ filters: 'filter:171075'
35+ snap-triggers: 'ubuntu-filemanager-app'
36+ deb-triggers: ''
37+ - launch-gallery-app:
38+ filters: 'filter:171070'
39+ snap-triggers: 'gallery-app'
40+ deb-triggers: 'gallery-app'
41+ - launch-systemsettings-app:
42+ filters: 'filter:171077'
43+ snap-triggers: 'unity8-session'
44+ deb-triggers: 'ubuntu-system-settings'
45+ - launch-terminal-app:
46+ filters: 'filter:171076'
47+ snap-triggers: 'ubuntu-terminal-app'
48+ deb-triggers: 'ubuntu-terminal-app'
49+ - launch-webbrowser-app:
50+ filters: 'filter:171004'
51+ snap-triggers: 'webbrowser-app'
52+ deb-triggers: 'webbrowser-app'
53 flavor:
54 - ubuntu
55 release:
56@@ -36,7 +78,7 @@
57 - snap
58 - deb
59 jobs:
60- - 'ust-build-image-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'
61+ - 'ust-build-image-{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'
62 - 'ust-test-{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'
63
64 - job-template: &UST_APP_LAUNCH_JOB_TEMPLATE
65@@ -46,7 +88,7 @@
66
67 - job-template:
68 <<: *UST_APP_LAUNCH_JOB_TEMPLATE
69- name: 'ust-build-image-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'
70+ name: 'ust-build-image-{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'
71 description: 'Build image for device.'
72 parameters:
73 - ubuntu-system-tests-common-parameters:
74@@ -56,10 +98,20 @@
75 flavor: '{flavor}'
76 model: '{model}'
77 arch: '{arch}'
78+ suite: '{suite}'
79+ - ubuntu-system-tests-trigger-parameters:
80+ snap-triggers: '{snap-triggers}'
81+ deb-triggers: '{deb-triggers}'
82 builders:
83 - clear-artifacts
84 - ubuntu-system-tests-config-builder
85 - ubuntu-system-tests-image-builder
86+ triggers:
87+ - script:
88+ # This is every minute!
89+ cron: '* * * * *'
90+ script:
91+ !include-raw: 'new_version_published_trigger.py'
92 publishers:
93 - trigger:
94 project: 'ust-test-{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}'
95@@ -80,9 +132,9 @@
96 flavor: '{flavor}'
97 model: '{model}'
98 arch: '{arch}'
99+ suite: '{suite}'
100 - ubuntu-system-tests-run-parameters:
101 filters: '{filters}'
102- suite: '{suite}'
103 builders:
104 - clear-artifacts
105 - ubuntu-system-tests-config-builder
106@@ -135,6 +187,14 @@
107 name: DEVICE
108 default: '{device}'
109 description: Name of device, either vm or adb.
110+ - string:
111+ name: SUITE
112+ default: '{suite}'
113+ description: Name of the test suite being run.
114+ - string:
115+ name: SHARED_IMAGE_PATH
116+ default: '/var/cache/ust/img/{suite}-{flavor}-{release}-{model}-{arch}-{device}-unity8-{unity8-mode}.img'
117+ description: Path of the image file to use for testing.
118
119 - parameter:
120 name: ubuntu-system-tests-run-parameters
121@@ -143,10 +203,26 @@
122 name: FILTERS
123 default: '{filters}'
124 description: List of Practitest filters IDs to look-up required test IDs.
125- - string:
126- name: SUITE
127- default: '{suite}'
128- description: Name of the test suite being run.
129+
130+- parameter:
131+ name: ubuntu-system-tests-trigger-parameters
132+ parameters:
133+ - string:
134+ name: SNAP_TRIGGERS
135+ default: '{snap-triggers}'
136+ description: List of snap names to monitor for updates.
137+ - string:
138+ name: DEB_TRIGGERS
139+ default: '{deb-triggers}'
140+ description: List of deb names to monitor for updates.
141+ - string:
142+ name: SNAP_CHANNEL
143+ default: 'edge'
144+ description: Channel to install snaps from.
145+ - string:
146+ name: CORE_SERIES
147+ default: '16'
148+ description: Series of Ubuntu-core used.
149
150 - builder:
151 name: ubuntu-system-tests-config-builder
152
153=== added file 'jobs/ubuntu-system-tests/new_version_published_trigger.py'
154--- jobs/ubuntu-system-tests/new_version_published_trigger.py 1970-01-01 00:00:00 +0000
155+++ jobs/ubuntu-system-tests/new_version_published_trigger.py 2017-02-22 13:47:51 +0000
156@@ -0,0 +1,154 @@
157+#!/usr/bin/python3
158+
159+import ast
160+import json
161+import os
162+import requests
163+import sys
164+
165+from collections import OrderedDict
166+
167+STORE_QUERY_URL = (
168+ 'https://search.apps.ubuntu.com/api/v1/snaps/details/{{n}}?channel={{c}}'
169+)
170+BUILD_TRIGGER_FOLDER = 'build-triggers'
171+SNAP_DATA_FILE_NAME = 'snap-data.json'
172+KEY_SNAP = '{{n}}-{{a}}-{{s}}-{{c}}'
173+KEY_REV = 'revision'
174+STATUS_TRIGGER_NEW_BUILD = 0
175+STATUS_NO_NEW_BUILD = 1
176+
177+
178+class Transformer(ast.NodeTransformer):
179+ """Class to safely evaluate strings containing specified Python values."""
180+
181+ ALLOWED_NAMES = set(['OrderedDict'])
182+ ALLOWED_NODE_TYPES = set([
183+ 'Expression',
184+ 'Tuple',
185+ 'Call',
186+ 'Name',
187+ 'Load',
188+ 'Str',
189+ 'Num',
190+ 'List',
191+ 'Dict',
192+ ])
193+
194+ def visit_Name(self, node):
195+ """Check that name is allowed and traverse onto any child nodes."""
196+ if node.id not in self.ALLOWED_NAMES:
197+ raise RuntimeError(
198+ 'Name access to {{n}} is not allowed'.format(n=node.id))
199+ return self.generic_visit(node)
200+
201+ def generic_visit(self, node):
202+ """Check that node type is allowed visit all child nodes."""
203+ node_type = type(node).__name__
204+ if node_type not in self.ALLOWED_NODE_TYPES:
205+ raise RuntimeError(
206+ 'Invalid expression: {{n}}'.format(n=node_type))
207+ return ast.NodeTransformer.generic_visit(self, node)
208+
209+ def evaluate(self, source):
210+ """Safely evaluate the source string and return evaluated object."""
211+ tree = ast.parse(source, mode='eval')
212+ self.visit(tree)
213+ compiled = compile(tree, '<AST>', 'eval')
214+ return eval(compiled, dict(OrderedDict=OrderedDict))
215+
216+
217+def get_build_trigger_path(work_dir, *args):
218+ """Return path of build trigger folder.
219+ :param work_dir: Work directory being used.
220+ :param args: Any other file names to append to path.
221+ """
222+ return os.path.join(work_dir, BUILD_TRIGGER_FOLDER, *args)
223+
224+
225+def get_snap_data_filepath(work_dir):
226+ """Return dictionary of all saved snap data."""
227+ return os.path.join(work_dir, SNAP_DATA_FILE_NAME)
228+
229+
230+def get_snap_data(work_dir):
231+ """Return dictionary of all saved snap data."""
232+ file_name = get_snap_data_filepath(work_dir)
233+ if not os.path.isfile(file_name):
234+ return {{}}
235+ with open(file_name) as f:
236+ data = f.read()
237+ return json.loads(data) if data else {{}}
238+
239+
240+def get_published_snap_revision(name, arch, series, channel):
241+ """Return revision of specified published snap."""
242+ url = STORE_QUERY_URL.format(n=name, c=channel)
243+ headers = {{'X-Ubuntu-Series': series, 'X-Ubuntu-Architecture': arch}}
244+ return int(requests.get(url, headers=headers).json()[KEY_REV])
245+
246+
247+def get_current_snap_revision(snap_data, name, arch, series, channel):
248+ """Return revision of currently tested snap."""
249+ key_snap = KEY_SNAP.format(n=name, a=arch, s=series, c=channel)
250+ try:
251+ rev = int(snap_data[key_snap][KEY_REV])
252+ except (KeyError, TypeError, ValueError):
253+ rev = -1
254+ return rev
255+
256+
257+def save_published_snap_revision(work_dir, snap_data, name, arch, series,
258+ channel, revision):
259+ """Save the published snap revision for future comparison."""
260+ key_snap = KEY_SNAP.format(n=name, a=arch, s=series, c=channel)
261+ if key_snap not in snap_data:
262+ snap_data[key_snap] = {{}}
263+ snap_data[key_snap][KEY_REV] = revision
264+ file_path = get_snap_data_filepath(work_dir)
265+ with open(file_path, 'w') as f:
266+ f.write(json.dumps(snap_data))
267+
268+
269+def run_snap_trigger_check(work_dir, names, arch, series, channel):
270+ """Return status indicating if a new revision of snap is published."""
271+ status = STATUS_NO_NEW_BUILD
272+ # Check through each app to see if new version available.
273+ # Update revision in snap data to keep saved data up to date.
274+ for name in names:
275+ snap_data = get_snap_data(work_dir)
276+ current = get_current_snap_revision(
277+ snap_data, name, arch, series, channel)
278+ published = get_published_snap_revision(name, arch, series, channel)
279+ if published > current:
280+ save_published_snap_revision(
281+ work_dir, snap_data, name, arch, series, channel, published)
282+ status = STATUS_TRIGGER_NEW_BUILD
283+ return status
284+
285+
286+def main():
287+ # Currently this only supports checking for new snap version.
288+ # TODO: Add checking for new deb version.
289+ suites = Transformer().evaluate(os.getenv('SUITE'))
290+ for suite in suites:
291+ for key in suite.keys():
292+ print('*************')
293+ print(key)
294+ print(suite[key])
295+ print(suite[key]['filters'])
296+ print(suite[key]['snap-triggers'])
297+ print(suite[key]['deb-triggers'])
298+ return STATUS_NO_NEW_BUILD
299+ if os.getenv('UNITY8_MODE') == 'snap':
300+ work_dir = os.getenv('WORKSPACE')
301+ names = os.getenv('SNAP_NAMES').split()
302+ arch = os.getenv('ARCH')
303+ series = os.getenv('CORE_SERIES')
304+ channel = os.getenv('SNAP_CHANNEL')
305+ return run_snap_trigger_check(work_dir, names, arch, series, channel)
306+ return STATUS_NO_NEW_BUILD
307+
308+
309+if __name__ == "__main__":
310+ sys.exit(main())
311
312=== modified file 'jobs/ubuntu-system-tests/ubuntu-system-tests-image-builder.sh'
313--- jobs/ubuntu-system-tests/ubuntu-system-tests-image-builder.sh 2017-02-14 15:39:47 +0000
314+++ jobs/ubuntu-system-tests/ubuntu-system-tests-image-builder.sh 2017-02-22 13:47:51 +0000
315@@ -4,7 +4,6 @@
316 export UBUNTU_SYSTEM_TESTS_BRANCH=lp:ubuntu-system-tests
317 export QA_JENKINS_JOBS_BRANCH=lp:qa-jenkins-jobs
318 export ISO_PATH=/var/cache/utah/iso/$RELEASE-$MODEL-$ARCH.iso
319-export SHARED_IMG_PATH=/var/cache/ust/img/$RELEASE-$MODEL-$ARCH-unity8-$UNITY8_MODE.img
320 export OUTPUTDIR=$WORKSPACE/results
321 export CACHE_PATH=$WORKSPACE/.ust-cache
322 export IMG_PATH=$CACHE_PATH/disk.img
323@@ -31,6 +30,6 @@
324 python3 -m ubuntu_system_tests.run setup --headless --silent --config $CONFIG_PATH --img $IMG_PATH
325
326 # copy the resulting image file to shared location for use by tests
327-sudo cp $IMG_PATH $SHARED_IMG_PATH
328+sudo cp $IMG_PATH $SHARED_IMAGE_PATH
329
330 deactivate # our virtualenv
331
332=== modified file 'jobs/ubuntu-system-tests/ubuntu-system-tests-run-tests-builder.sh'
333--- jobs/ubuntu-system-tests/ubuntu-system-tests-run-tests-builder.sh 2017-02-14 15:39:47 +0000
334+++ jobs/ubuntu-system-tests/ubuntu-system-tests-run-tests-builder.sh 2017-02-22 13:47:51 +0000
335@@ -2,7 +2,6 @@
336 set -x
337
338 export UBUNTU_SYSTEM_TESTS_BRANCH=lp:ubuntu-system-tests
339-export SHARED_IMG_PATH=/var/cache/ust/img/$RELEASE-$MODEL-$ARCH-unity8-$UNITY8_MODE.img
340 export OUTPUTDIR=$WORKSPACE/results
341 export CONFIG_PATH=$WORKSPACE/ubuntu-system-tests.conf
342 export https_proxy=https://squid.internal:3128/
343@@ -22,7 +21,7 @@
344
345 # Run tests from each test filter
346 for FILTER in $FILTERS; do
347- python3 -m ubuntu_system_tests.run run --headless --silent --config $CONFIG_PATH --img $SHARED_IMG_PATH $APPEND_RESULTS --filters $FILTER
348+ python3 -m ubuntu_system_tests.run run --headless --silent --config $CONFIG_PATH --img $SHARED_IMAGE_PATH $APPEND_RESULTS --filters $FILTER
349 # Use --append-results arg for all test runs but the first
350 if [ -z $APPEND_RESULTS ]; then
351 APPEND_RESULTS="--append-results"

Subscribers

People subscribed via source and target branches