Merge lp:~robru/ubuntu-test-cases/mp-testing into lp:ubuntu-test-cases

Proposed by Robert Bruce Park
Status: Superseded
Proposed branch: lp:~robru/ubuntu-test-cases/mp-testing
Merge into: lp:ubuntu-test-cases
Diff against target: 6933 lines (+6392/-0) (has conflicts)
101 files modified
README-cli.rst (+80/-0)
jenkins/custom-demo.py (+32/-0)
jenkins/production.py (+90/-0)
jenkins/setup_jenkins.py (+213/-0)
jenkins/staging.py (+22/-0)
jenkins/templates/touch-base.xml.jinja2 (+93/-0)
jenkins/templates/touch-master.xml.jinja2 (+121/-0)
jenkins/templates/touch-smoke.xml.jinja2 (+156/-0)
jenkins/testconfig.py (+177/-0)
scripts/assert-image (+28/-0)
scripts/click-buddy-install.sh (+114/-0)
scripts/combine_results (+121/-0)
scripts/dashboard.py (+298/-0)
scripts/device (+191/-0)
scripts/get-adb-id (+44/-0)
scripts/jenkins.sh (+186/-0)
scripts/junit2utah.py (+70/-0)
scripts/mp_provision (+299/-0)
scripts/provision.sh (+144/-0)
scripts/reboot-and-wait (+41/-0)
scripts/run-autopilot-tests.sh (+243/-0)
scripts/run-smoke (+364/-0)
scripts/run-touch-upgrade.sh (+46/-0)
scripts/statsd.py (+35/-0)
scripts/test_um_provision.py (+101/-0)
selftests/test_junit2utah.py (+49/-0)
selftests/test_reboot_and_wait.py (+66/-0)
selftests/test_run_smoke.py (+143/-0)
selftests/test_statsd.py (+64/-0)
tests/click_image_tests/check_preinstalled_list/check_preinstalled_list.py (+61/-0)
tests/click_image_tests/check_preinstalled_list/tc_control (+10/-0)
tests/click_image_tests/master.run (+5/-0)
tests/click_image_tests/tslist.run (+1/-0)
tests/customizations/master.run (+5/-0)
tests/customizations/setup.sh (+10/-0)
tests/customizations/ts_control (+1/-0)
tests/customizations/tslist.auto (+4/-0)
tests/default/ifconfig/tc_control (+13/-0)
tests/default/install/tc_control (+12/-0)
tests/default/master.run (+5/-0)
tests/default/netstat/tc_control (+12/-0)
tests/default/ping/pingtest.sh (+12/-0)
tests/default/ping/tc_control (+12/-0)
tests/default/pwd/tc_control (+9/-0)
tests/default/pwd/test.sh (+18/-0)
tests/default/route/tc_control (+12/-0)
tests/default/systemsettle/systemsettle.sh (+127/-0)
tests/default/systemsettle/tc_control (+9/-0)
tests/default/ts_control (+3/-0)
tests/default/tslist.run (+10/-0)
tests/default/uname/tc_control (+12/-0)
tests/default/unity8/tc_control (+9/-0)
tests/default/vmstat/tc_control (+12/-0)
tests/eventstat/eventstat/eventstat.sh (+5/-0)
tests/eventstat/eventstat/setup.sh (+7/-0)
tests/eventstat/eventstat/tc_control (+12/-0)
tests/eventstat/master.run (+5/-0)
tests/eventstat/tslist.run (+1/-0)
tests/memevent/master.run (+5/-0)
tests/memevent/setup.sh (+8/-0)
tests/memevent/ts_control (+1/-0)
tests/memevent/tslist.auto (+7/-0)
tests/memevent/ubuntu_test_cases/__init__.py (+6/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/__init__.py (+1/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/__init__.py (+21/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/browser.py (+103/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/camera.py (+58/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/gallery.py (+41/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/media_player.py (+53/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/matchers.py (+37/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/probes.py (+170/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/smem-tabs (+687/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/tests/__init__.py (+1/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/tests/test_memory_usage.py (+79/-0)
tests/sdk/master.run (+5/-0)
tests/sdk/run_test.sh (+20/-0)
tests/sdk/setup.sh (+12/-0)
tests/sdk/ts_control (+1/-0)
tests/sdk/tslist.auto (+3/-0)
tests/security/master.run (+5/-0)
tests/security/run_test.sh (+25/-0)
tests/security/setup.sh (+12/-0)
tests/security/ts_control (+1/-0)
tests/security/tslist.auto (+6/-0)
tests/smem/master.run (+15/-0)
tests/smem/smem/smem-tabs (+687/-0)
tests/smem/smem/smem.sh (+12/-0)
tests/smem/smem/tc_control (+11/-0)
tests/smem/tslist.run (+1/-0)
tests/systemsettle/systemsettle-after/tc_control (+9/-0)
tests/systemsettle/systemsettle-before/tc_control (+9/-0)
tests/systemsettle/systemsettle.sh (+127/-0)
tests/systemsettle/tslist.run (+2/-0)
utils/host/adb-shell (+23/-0)
utils/host/autopilot-list (+18/-0)
utils/host/autopilot-run (+7/-0)
utils/host/prepare-autopilot-test.sh (+7/-0)
utils/host/reboot-and-unlock.sh (+15/-0)
utils/target/autopilot-list (+13/-0)
utils/target/autopilot-run (+6/-0)
utils/target/prepare-autopilot-test.sh (+17/-0)
Conflict adding file scripts.  Moved existing file to scripts.moved.
To merge this branch: bzr merge lp:~robru/ubuntu-test-cases/mp-testing
Reviewer Review Type Date Requested Status
Leo Arias Pending
Francis Ginther Pending
Ubuntu Test Case Developers Pending
Review via email: mp+225555@code.launchpad.net

This proposal has been superseded by a proposal from 2014-07-03.

Commit message

Restart unity8 for ubuntu_experience_tests

To post a comment you must log in.

Unmerged revisions

231. By Robert Bruce Park

Restart unity8 for ubuntu_experience_tests

230. By Francis Ginther

Use new phablet-config welcome-wizard support to disable wizard when provisioning.

229. By Francis Ginther

Use correct TEST_SOURCE_DIR.

228. By Francis Ginther

Fixed missing '['.

227. By Francis Ginther

Allow other sourch paths for click tests.

226. By Francis Ginther

Add reboot-and-wait call to unlock-device.

225. By Francis Ginther

Don't install the packages returned by testconfig.py, these will replace the packages that we provisioned from the MP.

224. By Francis Ginther

Change default IMAGE_OPT arguments to utopic-proposed.

223. By Francis Ginther

Use unlock-device script from unity8-autopilot

222. By Francis Ginther

Add collection of system-image-cli.log and remove some debug output.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'README-cli.rst'
2--- README-cli.rst 1970-01-01 00:00:00 +0000
3+++ README-cli.rst 2014-07-03 19:35:55 +0000
4@@ -0,0 +1,80 @@
5+Touch Testing From the CLI
6+==========================
7+
8+The touch testing execution framework was written so that its very easy to
9+run tests from home in the exact same way test are run in the lab. The only
10+thigs you need are:
11+
12+ * This bzr branch
13+ * The phablet-tools_ package
14+ * An Ubuntu Touch supported_ device
15+
16+.. _phablet-tools: http://launchpad.net/phablet-tools
17+.. _supported: http://wiki.ubuntu.com/Touch/Devices
18+
19+There are two pieces to touch testing, provisioning and test execution. These
20+functions are independent of one another. eg, If your device already
21+has the proper image/configuration, you can simply use the test-runner.
22+
23+Provisioning
24+------------
25+
26+The provisioning script is a simple wrapper to commands from phablet-tools
27+to get a device ready for testing. Provisioning is performed with the
28+scripts/provision.sh command. Running::
29+
30+ ./scripts/provision.sh -h
31+
32+will list supported options.
33+
34+NOTE: provision.sh requires a path to a network-manager wifi connection that
35+can be copied to the target device. By default this is set to
36+/home/ubuntu/magners-wifi. This can be overriden with the -n parameter.
37+
38+Executing Tests
39+---------------
40+
41+The touch testing repository supports both autopilot and UTAH test definitions.
42+
43+Executing Autopilot Tests
44+~~~~~~~~~~~~~~~~~~~~~~~~~
45+
46+One or more autopilot tests can be run on the target device using the command::
47+
48+ ./scripts/run-autopilot-tests.sh
49+
50+This is a small wrapper that uses phablet-tools to drive the tests. The
51+script can run one or more autopilot tests. By default it will reboot the
52+device between each test and ensure the device is settled using the
53+*system-settle* script. Both of those options can be disabled via command
54+line options. By default the script will create a directory named
55+*clientlogs* and then a subdirectory for each testsuite with result files.
56+These sub-directories include a xUnit XML formatted file, *test_results.xml*,
57+as well as several log files from the device to help with debugging failures.
58+
59+An example testing two applications::
60+
61+ ./scripts/run-autopilot-tests.sh -a dropping_letters_app -a music_app
62+
63+Executing UTAH Tests
64+~~~~~~~~~~~~~~~~~~~~
65+
66+Executing UTAH tests locally will require you to install the UTAH client
67+package from a PPA::
68+
69+ sudo add-apt-repository ppa:utah/stable
70+ sudo apt-get update
71+ sudo apt-get install utah-client
72+
73+With that package installed UTAH tests can be run with::
74+
75+ ./scripts/jenkins.sh
76+
77+This script runs one test at a time and will put its test artifacts under the
78+*clientlogs* directory similar to the autopilot runner. The UTAH result file
79+will be named clientlogs/utah.yaml.
80+
81+An example of running the sdk test suite::
82+
83+ ./scripts/jenkins.sh -a sdk
84+
85
86=== added directory 'jenkins'
87=== added file 'jenkins/custom-demo.py'
88--- jenkins/custom-demo.py 1970-01-01 00:00:00 +0000
89+++ jenkins/custom-demo.py 2014-07-03 19:35:55 +0000
90@@ -0,0 +1,32 @@
91+# The configuration matrix of our production device testing
92+
93+JENKINS = 'http://q-jenkins.ubuntu-ci:8080'
94+
95+
96+def _url(channel, device):
97+ return 'http://system-image.ubuntu.com/ubuntu-touch/%s/%s/index.json' \
98+ % (channel, device)
99+
100+
101+TRUSTY_MATRIX = [
102+ {
103+ 'image-type': 'touch_custom_demo',
104+ 'include-qa': True,
105+ 'dashboard-host': 'ci.ubuntu.com',
106+ 'dashboard-port': '80',
107+ 'dashboard-user': 'doanac',
108+ 'devices': [
109+ {
110+ 'name': 'mako',
111+ 'slave-label': 'daily-mako',
112+ 'trigger_url': _url('trusty-proposed-customized-demo', 'mako'),
113+ },
114+ ],
115+ 'IMAGE_OPT': 'export IMAGE_OPT="ubuntu-system -b '
116+ '--channel ubuntu-touch/trusty-proposed-customized-demo"'
117+ },
118+]
119+
120+MATRIX = {
121+ 'trusty': TRUSTY_MATRIX,
122+}
123
124=== added file 'jenkins/production.py'
125--- jenkins/production.py 1970-01-01 00:00:00 +0000
126+++ jenkins/production.py 2014-07-03 19:35:55 +0000
127@@ -0,0 +1,90 @@
128+# The configuration matrix of our production device testing
129+
130+JENKINS = 'http://q-jenkins.ubuntu-ci:8080'
131+
132+
133+def _url(channel, device):
134+ return 'http://system-image.ubuntu.com/ubuntu-touch/%s/%s/index.json' \
135+ % (channel, device)
136+
137+
138+TRUSTY_MATRIX = [
139+ {
140+ 'image-type': 'touch_sf4p',
141+ 'include-qa': False,
142+ 'dashboard-host': 'ci.ubuntu.com',
143+ 'dashboard-port': '80',
144+ 'dashboard-user': 'doanac',
145+ 'devices': [
146+ {
147+ 'name': 'mako',
148+ 'slave-label': 'daily-mako',
149+ 'trigger_url': _url('trusty-proposed', 'mako'),
150+ },
151+ ],
152+ },
153+ {
154+ 'image-type': 'touch',
155+ 'include-qa': True,
156+ 'dashboard-host': 'ci.ubuntu.com',
157+ 'dashboard-port': '80',
158+ 'dashboard-user': 'doanac',
159+ 'devices': [
160+ {
161+ 'name': 'mako',
162+ 'slave-label': 'daily-mako',
163+ 'trigger_url': _url('trusty-proposed', 'mako'),
164+ },
165+ {
166+ 'name': 'flo',
167+ 'slave-label': 'daily-flo',
168+ 'trigger_url': _url('trusty-proposed', 'flo'),
169+ },
170+ {
171+ 'name': 'manta',
172+ 'slave-label': 'daily-manta',
173+ 'trigger_url': _url('trusty-proposed', 'manta'),
174+ },
175+ ],
176+ },
177+ {
178+ 'image-type': 'touch_custom',
179+ 'include-qa': False,
180+ 'dashboard-host': 'ci.ubuntu.com',
181+ 'dashboard-port': '80',
182+ 'dashboard-user': 'doanac',
183+ 'devices': [
184+ {
185+ 'name': 'mako',
186+ 'slave-label': 'daily-mako',
187+ 'trigger_url': _url('trusty-proposed-customized', 'mako'),
188+ },
189+ ],
190+ 'IMAGE_OPT': 'export IMAGE_OPT="--bootstrap '
191+ '--channel trusty-proposed-customized"'
192+ },
193+]
194+
195+SAUCY_MATRIX = [
196+ {
197+ 'image-type': 'touch_mir',
198+ 'include-qa': True,
199+ 'dashboard-host': 'ci.ubuntu.com',
200+ 'dashboard-port': '80',
201+ 'dashboard-user': 'doanac',
202+ 'devices': [
203+ {
204+ 'name': 'mako',
205+ 'slave-label': 'daily-mako',
206+ 'trigger_url': _url('saucy-proposed', 'mako'),
207+ },
208+ ],
209+ 'IMAGE_OPT': 'export IMAGE_OPT="--bootstrap '
210+ '--channel saucy-proposed"'
211+ },
212+]
213+
214+MATRIX = {
215+ 'trusty': TRUSTY_MATRIX,
216+ 'saucy': SAUCY_MATRIX,
217+}
218
219=== added file 'jenkins/setup_jenkins.py'
220--- jenkins/setup_jenkins.py 1970-01-01 00:00:00 +0000
221+++ jenkins/setup_jenkins.py 2014-07-03 19:35:55 +0000
222@@ -0,0 +1,213 @@
223+#!/usr/bin/env python
224+
225+# Ubuntu Testing Automation Harness
226+# Copyright 2013 Canonical Ltd.
227+
228+# This program is free software: you can redistribute it and/or modify it
229+# under the terms of the GNU General Public License version 3, as published
230+# by the Free Software Foundation.
231+
232+# This program is distributed in the hope that it will be useful, but
233+# WITHOUT ANY WARRANTY; without even the implied warranties of
234+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
235+# PURPOSE. See the GNU General Public License for more details.
236+
237+# You should have received a copy of the GNU General Public License along
238+# with this program. If not, see <http://www.gnu.org/licenses/>.
239+
240+import argparse
241+import imp
242+import jenkins
243+import jinja2
244+import logging
245+import os
246+
247+import testconfig
248+
249+from distro_info import UbuntuDistroInfo
250+DEV_SERIES = UbuntuDistroInfo().devel()
251+
252+
253+def _get_parser():
254+ """Create and return command line parser.
255+
256+ :returns: command line parser
257+ :rtype: argparse.ArgumentParser
258+
259+ """
260+ parser = argparse.ArgumentParser(
261+ description='Create/Update upgrade testing jenkins jobs.')
262+ parser.add_argument("-d", "--dryrun", action="store_true",
263+ help="Dry run mode. Don't execute jenkins commands.")
264+ parser.add_argument("-u", "--username",
265+ help="username to use when logging into Jenkins.")
266+ parser.add_argument("-p", "--password",
267+ help="username to use when logging into Jenkins")
268+ parser.add_argument("--dashboard-key", default="",
269+ help="The api-key for dashboard updates")
270+ parser.add_argument("-b", "--branch", default="lp:ubuntu-test-cases/touch",
271+ help="The branch this is located. default=%(default)s")
272+ parser.add_argument("-c", "--config", required=True,
273+ type=argparse.FileType('r'),
274+ help="The job config to use.")
275+ parser.add_argument("-P", "--publish", action="store_true",
276+ help="Publish at the end of the job")
277+ parser.add_argument("--prefix",
278+ help="Prefix to add to the beginning of the job name")
279+ parser.add_argument("-s", "--series", default=DEV_SERIES,
280+ help=("series of Ubuntu to download "
281+ "(default=%(default)s)"))
282+ parser.add_argument("-w", "--wait", type=int, default=300,
283+ help=("How long to wait after jenkins triggers the"
284+ "install-and-boot job (default=%(default)d)"))
285+ return parser
286+
287+
288+def _get_jenkins(url, username, password):
289+ logging.info('Attempting to login to jenkins at %s', url)
290+ if username is not None:
291+ logging.info('...with authentication as user: %s', username)
292+ instance = jenkins.Jenkins(url, username=username, password=password)
293+ else:
294+ logging.info('...without authentication')
295+ instance = jenkins.Jenkins(url)
296+
297+ return instance
298+
299+
300+def _get_environment():
301+ base = os.path.join(os.path.dirname(__file__), 'templates')
302+ return jinja2.Environment(
303+ loader=jinja2.FileSystemLoader(base),
304+ undefined=jinja2.StrictUndefined,
305+ trim_blocks=True)
306+
307+
308+def _publish(instance, env, args, template, jobname, **params):
309+ tmpl = env.get_template(template)
310+ cfg = tmpl.render(**params)
311+ if args.dryrun:
312+ _dryrun_func(jobname, cfg)
313+ return
314+ if instance.job_exists(jobname):
315+ logging.info('reconfiguring job %s', jobname)
316+ instance.reconfig_job(jobname, cfg)
317+ else:
318+ logging.info('creating job %s', jobname)
319+ instance.create_job(jobname, cfg)
320+
321+
322+def _configure_smoke(instance, env, args, config_item, device):
323+ defserial = '$(${BZRDIR}/scripts/get-adb-id ${NODE_NAME})'
324+
325+ params = {
326+ 'name': device['slave-label'],
327+ 'serial': device.get('serial', defserial),
328+ 'publish': args.publish,
329+ 'branch': args.branch,
330+ 'imagetype': config_item['image-type'],
331+ 'image_opt': config_item.get('IMAGE_OPT', ''),
332+ 'statsd_key': config_item.get('statsd-key', ''),
333+ 'dashboard_host': config_item.get('dashboard-host', ''),
334+ 'dashboard_port': config_item.get('dashboard-port', ''),
335+ 'dashboard_prefix': config_item.get('dashboard-prefix', ''),
336+ 'dashboard_user': config_item.get('dashboard-user', ''),
337+ 'dashboard_key': args.dashboard_key,
338+ }
339+
340+ prefix = ""
341+ if(args.prefix):
342+ prefix = args.prefix + "-"
343+
344+ job = '{}{}-{}-{}-smoke-daily'.format(
345+ prefix, args.series, config_item['image-type'], device['name'])
346+ _publish(instance, env, args, 'touch-smoke.xml.jinja2', job, **params)
347+ return job
348+
349+
350+def _configure_qa_job(instance, env, args, config_item, device, test):
351+ defserial = '$(${BZRDIR}/scripts/get-adb-id ${NODE_NAME})'
352+ params = {
353+ 'name': device['slave-label'],
354+ 'serial': device.get('serial', defserial),
355+ 'publish': args.publish,
356+ 'branch': args.branch,
357+ 'imagetype': config_item['image-type'],
358+ 'image_opt': config_item.get('IMAGE_OPT', ''),
359+ 'test': test.name,
360+ }
361+ prefix = ""
362+ if(args.prefix):
363+ prefix = args.prefix + "-"
364+ job = test.fmt.format(prefix=prefix,
365+ series=args.series,
366+ testname=test.name,
367+ imagetype=config_item['image-type'],
368+ type=device['name'])
369+ _publish(instance, env, args, 'touch-base.xml.jinja2', job, **params)
370+ return job
371+
372+
373+def _configure_qa_jobs(instance, env, args, config_item, device):
374+ tests = testconfig.TESTSUITES
375+ tests = testconfig.filter_tests(tests, config_item['image-type'])
376+ tests = [t for t in tests if not t.smoke]
377+ jobs = []
378+ for t in tests:
379+ j = _configure_qa_job(instance, env, args, config_item, device, t)
380+ jobs.append(j)
381+ return jobs
382+
383+
384+def _configure_master(instance, env, args, config_item, device, smoke, jobs):
385+ params = {
386+ 'branch': args.branch,
387+ 'trigger_url': device.get('trigger_url', ''),
388+ 'wait': args.wait,
389+ 'projects': jobs,
390+ 'smoke_job': smoke,
391+ }
392+ prefix = ""
393+ if(args.prefix):
394+ prefix = args.prefix + "-"
395+ job = testconfig.DEF_FMT.format(prefix=prefix,
396+ series=args.series,
397+ testname='master',
398+ imagetype=config_item['image-type'],
399+ type=device['name'])
400+ _publish(instance, env, args, 'touch-master.xml.jinja2', job, **params)
401+ return job
402+
403+
404+def _dryrun_func(jobname, config):
405+ logging.debug(jobname)
406+ logging.debug(config)
407+
408+
409+def main():
410+ logging.basicConfig(level=logging.DEBUG)
411+ args = _get_parser().parse_args()
412+
413+ config = imp.load_source('', 'config.py', args.config)
414+ if args.series not in config.MATRIX:
415+ print('"%s" series is not supported by this config.' % args.series)
416+ exit(1)
417+
418+ jenkins_inst = _get_jenkins(config.JENKINS, args.username, args.password)
419+ if args.dryrun:
420+ jenkins_inst.create_job = _dryrun_func
421+ jenkins_inst.reconfig_job = _dryrun_func
422+
423+ env = _get_environment()
424+
425+ for item in config.MATRIX[args.series]:
426+ for device in item['devices']:
427+ job = _configure_smoke(jenkins_inst, env, args, item, device)
428+ jobs = []
429+ if item.get('include-qa'):
430+ jobs = _configure_qa_jobs(
431+ jenkins_inst, env, args, item, device)
432+ _configure_master(jenkins_inst, env, args, item, device, job, jobs)
433+
434+if __name__ == '__main__':
435+ main()
436
437=== added file 'jenkins/staging.py'
438--- jenkins/staging.py 1970-01-01 00:00:00 +0000
439+++ jenkins/staging.py 2014-07-03 19:35:55 +0000
440@@ -0,0 +1,22 @@
441+# The configuration matrix of our staging device testing
442+
443+JENKINS = 'http://dev-jenkins.ubuntu-ci:8080/'
444+
445+TRUSTY_MATRIX = [
446+ {
447+ 'image-type': 'touch',
448+ 'statsd-key': 'ubuntu-ci.daily-image.staging',
449+ 'include-qa': True,
450+ 'devices': [
451+ {
452+ 'name': 'mako',
453+ 'slave-label': 'mako',
454+ #'trigger_url': 'http://system-image.ubuntu.com/trusty-proposed/mako/index.json',
455+ }
456+ ],
457+ },
458+]
459+
460+MATRIX = {
461+ 'trusty': TRUSTY_MATRIX,
462+}
463
464=== added directory 'jenkins/templates'
465=== added file 'jenkins/templates/touch-base.xml.jinja2'
466--- jenkins/templates/touch-base.xml.jinja2 1970-01-01 00:00:00 +0000
467+++ jenkins/templates/touch-base.xml.jinja2 2014-07-03 19:35:55 +0000
468@@ -0,0 +1,93 @@
469+<?xml version='1.0' encoding='UTF-8'?>
470+<project>
471+ <actions/>
472+ <description>
473+&lt;pre&gt;
474+#NOTE: Automatically created from a script as part of daily smoke testing&#xd;
475+ {{branch}}&#xd;
476+&lt;/pre&gt;
477+ </description>
478+ <keepDependencies>false</keepDependencies>
479+ <properties>
480+ <hudson.model.ParametersDefinitionProperty>
481+ <parameterDefinitions>
482+ <hudson.model.StringParameterDefinition>
483+ <name>INSTALL_URL</name>
484+ <description>A URL to the previous job. If provided this job will use the same install options as it used. If the device executing the job happens to have the exact same image, then provisioning can be skipped.
485+ </description>
486+ <defaultValue></defaultValue>
487+ </hudson.model.StringParameterDefinition>
488+ </parameterDefinitions>
489+ </hudson.model.ParametersDefinitionProperty>
490+ <com.sonyericsson.rebuild.RebuildSettings>
491+ <autoRebuild>false</autoRebuild>
492+ </com.sonyericsson.rebuild.RebuildSettings>
493+ </properties>
494+ <scm class="hudson.scm.NullSCM"/>
495+ <assignedNode>{{ name }}</assignedNode>
496+ <canRoam>false</canRoam>
497+ <disabled>false</disabled>
498+ <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
499+ <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
500+ <concurrentBuild>true</concurrentBuild>
501+ <builders>
502+ <hudson.tasks.Shell>
503+ <command><![CDATA[
504+set -e
505+rm -rf *
506+
507+BRANCH="{{branch}}"
508+BZRDIR=`echo "$BRANCH" | awk -F/ '{ print $(NF) }'`
509+BZRDIR=$(readlink -f $BZRDIR)
510+bzr branch ${BRANCH} ${BZRDIR}
511+
512+export ANDROID_SERIAL={{serial}}
513+export IMAGE_TYPE={{imagetype}}
514+
515+{{image_opt}}
516+${BZRDIR}/scripts/run-smoke -t {{test}}
517+
518+# move results to base directory as expected by dashboard:
519+mv clientlogs/{{test}}/* ./clientlogs/
520+rm -rf clientlogs/{{test}}
521+
522+{% if test == 'memevent' %}
523+adb pull /tmp/memory_usage.json ./clientlogs/
524+{% endif %}
525+ ]]></command>
526+ </hudson.tasks.Shell>
527+ </builders>
528+ <publishers>
529+ <hudson.tasks.ArtifactArchiver>
530+ <artifacts>clientlogs/*, clientlogs/**/*</artifacts>
531+ <latestOnly>false</latestOnly>
532+ </hudson.tasks.ArtifactArchiver>
533+ <hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
534+ <regexp>^= TOUCH IMAGE VERSION:([0-9]+.*)</regexp>
535+ <regexpForFailed>^= TOUCH IMAGE VERSION:([0-9]+.*)</regexpForFailed>
536+ <setForMatrix>false</setForMatrix>
537+ </hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
538+ <hudson.tasks.Mailer>
539+ <recipients>paul.larson@canonical.com para.siva@canonical.com</recipients>
540+ <dontNotifyEveryUnstableBuild>false</dontNotifyEveryUnstableBuild>
541+ <sendToIndividuals>false</sendToIndividuals>
542+ </hudson.tasks.Mailer>
543+{% if publish %}
544+ <hudson.plugins.build__publisher.BuildPublisher>
545+ <publishUnstableBuilds>true</publishUnstableBuilds>
546+ <publishFailedBuilds>true</publishFailedBuilds>
547+ <postActions class="vector"/>
548+ </hudson.plugins.build__publisher.BuildPublisher>
549+{% endif %}
550+ </publishers>
551+ <buildWrappers>
552+ <hudson.plugins.build__timeout.BuildTimeoutWrapper>
553+ <timeoutMinutes>60</timeoutMinutes>
554+ <failBuild>true</failBuild>
555+ <writingDescription>false</writingDescription>
556+ <timeoutPercentage>0</timeoutPercentage>
557+ <timeoutType>absolute</timeoutType>
558+ <timeoutMinutesElasticDefault>3</timeoutMinutesElasticDefault>
559+ </hudson.plugins.build__timeout.BuildTimeoutWrapper>
560+ </buildWrappers>
561+</project>
562
563=== added file 'jenkins/templates/touch-master.xml.jinja2'
564--- jenkins/templates/touch-master.xml.jinja2 1970-01-01 00:00:00 +0000
565+++ jenkins/templates/touch-master.xml.jinja2 2014-07-03 19:35:55 +0000
566@@ -0,0 +1,121 @@
567+<?xml version='1.0' encoding='UTF-8'?>
568+<project>
569+ <actions/>
570+ <description>
571+&lt;pre&gt;
572+#NOTE: Automatically created from a script as part of daily smoke testing&#xd;
573+ {{branch}}&#xd;
574+&lt;/pre&gt;
575+ </description>
576+ <keepDependencies>false</keepDependencies>
577+ <properties>
578+ <hudson.plugins.throttleconcurrents.ThrottleJobProperty>
579+ <maxConcurrentPerNode>0</maxConcurrentPerNode>
580+ <maxConcurrentTotal>0</maxConcurrentTotal>
581+ <throttleEnabled>false</throttleEnabled>
582+ <throttleOption>project</throttleOption>
583+ </hudson.plugins.throttleconcurrents.ThrottleJobProperty>
584+ <hudson.plugins.build__publisher.ExternalProjectProperty/>
585+ </properties>
586+ <scm class="hudson.scm.NullSCM"/>
587+ <assignedNode>master</assignedNode>
588+ <canRoam>false</canRoam>
589+ <disabled>false</disabled>
590+ <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
591+ <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
592+{%if trigger_url %}
593+ <triggers class="vector">
594+ <com.redfin.hudson.UrlChangeTrigger>
595+ <spec></spec>
596+ <url>{{trigger_url}}</url>
597+ </com.redfin.hudson.UrlChangeTrigger>
598+ </triggers>
599+{% endif %}
600+ <concurrentBuild>false</concurrentBuild>
601+ <builders>
602+ <hudson.tasks.Shell>
603+ <command><![CDATA[
604+set -e
605+
606+# give the image time to show up before running tests
607+sleep {{wait}}
608+ ]]></command>
609+ </hudson.tasks.Shell>
610+ <!-- our smoke job. this must run successfully or we won't continue -->
611+ <hudson.plugins.parameterizedtrigger.TriggerBuilder>
612+ <configs>
613+ <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
614+ <configs class="java.util.Collections$EmptyList"/>
615+ <projects>{{smoke_job}}</projects>
616+ <condition>ALWAYS</condition>
617+ <triggerWithNoParameters>false</triggerWithNoParameters>
618+ <block>
619+ <buildStepFailureThreshold>
620+ <name>FAILURE</name>
621+ <ordinal>2</ordinal>
622+ <color>RED</color>
623+ </buildStepFailureThreshold>
624+ <unstableThreshold>
625+ <name>UNSTABLE</name>
626+ <ordinal>1</ordinal>
627+ <color>YELLOW</color>
628+ </unstableThreshold>
629+ <failureThreshold>
630+ <name>FAILURE</name>
631+ <ordinal>2</ordinal>
632+ <color>RED</color>
633+ </failureThreshold>
634+ </block>
635+ <buildAllNodesWithLabel>false</buildAllNodesWithLabel>
636+ </hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
637+ </configs>
638+ </hudson.plugins.parameterizedtrigger.TriggerBuilder>
639+
640+ <hudson.tasks.Shell>
641+ <command><![CDATA[
642+set -e
643+
644+# Grab the INSTALL_URL from the smoke-job to help avoid reflashing the
645+# downstream jobs
646+child_number=$(env | grep ^TRIGGERED_BUILD_NUMBER_ | cut -d= -f2)
647+cat >child.params <<EOF
648+INSTALL_URL=${JENKINS_URL}job/{{smoke_job}}/${child_number}
649+EOF
650+
651+echo params for downstream jobs:
652+cat child.params
653+ ]]></command>
654+ </hudson.tasks.Shell>
655+
656+{% for project in projects %}
657+ <hudson.plugins.parameterizedtrigger.TriggerBuilder>
658+ <configs>
659+ <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
660+ <configs>
661+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
662+ <propertiesFile>child.params</propertiesFile>
663+ <failTriggerOnMissing>true</failTriggerOnMissing>
664+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
665+ </configs>
666+ <projects>{{project}}</projects>
667+ <condition>ALWAYS</condition>
668+ <triggerWithNoParameters>false</triggerWithNoParameters>
669+ <buildAllNodesWithLabel>false</buildAllNodesWithLabel>
670+ </hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
671+ </configs>
672+ </hudson.plugins.parameterizedtrigger.TriggerBuilder>
673+{% endfor %}
674+ </builders>
675+ <publishers/>
676+ <buildWrappers>
677+ <hudson.plugins.build__timeout.BuildTimeoutWrapper>
678+ <timeoutMinutes>300</timeoutMinutes>
679+ <failBuild>true</failBuild>
680+ <writingDescription>false</writingDescription>
681+ <timeoutPercentage>0</timeoutPercentage>
682+ <timeoutType>absolute</timeoutType>
683+ <timeoutMinutesElasticDefault>3</timeoutMinutesElasticDefault>
684+ </hudson.plugins.build__timeout.BuildTimeoutWrapper>
685+ </buildWrappers>
686+</project>
687+
688
689=== added file 'jenkins/templates/touch-smoke.xml.jinja2'
690--- jenkins/templates/touch-smoke.xml.jinja2 1970-01-01 00:00:00 +0000
691+++ jenkins/templates/touch-smoke.xml.jinja2 2014-07-03 19:35:55 +0000
692@@ -0,0 +1,156 @@
693+<?xml version='1.0' encoding='UTF-8'?>
694+<project>
695+ <actions/>
696+ <description>
697+<![CDATA[
698+This job provides a flexible way to execute all the tests in our daily image
699+testing process that contribute to the
700+<a href="http://reports.qa.ubuntu.com/smokeng/">QA Dashboard</a>.
701+The job is parameterized to give flexibility in what gets tested. A couple of
702+common ways to run this job are:
703+<dl>
704+ <dt>Full Test Run</dt>
705+ <dd>TESTS=ALL</dd>
706+ <dd>APPS=ALL</dd>
707+ <dt>Re-run a Failed Autopilot Test</dt>
708+ <dd>INSTALL_URL=http://dev-jenkins:8080/job/saucy-touch_ro-mako/9</dd>
709+ <dd>APPS=friends_app</dd>
710+ <dt>Re-run a Failed UTAH Test</dt>
711+ <dd>INSTALL_URL=http://dev-jenkins:8080/job/saucy-touch_ro-mako/9</dd>
712+ <dd>TESTS=security</dd>
713+</dl>
714+<pre>
715+#NOTE: Automatically created from a script as part of daily smoke testing
716+ {{branch}}
717+</pre>
718+{% if statsd_key %}
719+<h3>Graphite Reports</h3>
720+<ul>
721+ <li><a href="https://graphite.engineering.canonical.com/graphlot/?width=841&height=770&&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.provision.mean%2C0.000016667%29%2C%22Provisioning%20Time(minutes)%22%29&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.test_autopilot.mean%2C0.000016667%29%2C%22Autopilot%20Tests%20Time%22%29&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.test_utah.mean%2C0.000016667%29%2C%22UTAH%20Tests%20Time%22%29&from=-1weeks">Timings (1 week)</a></li>
722+ <li><a href="https://graphite.engineering.canonical.com/graphlot/?width=841&height=770&&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.provision.mean%2C0.000016667%29%2C%22Provisioning%20Time(minutes)%22%29&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.test_autopilot.mean%2C0.000016667%29%2C%22Autopilot%20Tests%20Time%22%29&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.test_utah.mean%2C0.000016667%29%2C%22UTAH%20Tests%20Time%22%29&from=-4weeks">Timings (4 weeks)</a></li>
723+</ul>
724+{% endif %}
725+]]>
726+ </description>
727+ <keepDependencies>false</keepDependencies>
728+ <properties>
729+ <hudson.model.ParametersDefinitionProperty>
730+ <parameterDefinitions>
731+ <hudson.model.StringParameterDefinition>
732+ <name>INSTALL_URL</name>
733+ <description>A URL to the previous job. If provided this job will use the same install options as it used. If the device executing the job happens to have the exact same image, then provisioning can be skipped.
734+ </description>
735+ <defaultValue></defaultValue>
736+ </hudson.model.StringParameterDefinition>
737+ <hudson.model.StringParameterDefinition>
738+ <name>TESTS</name>
739+ <description>A space separated list of utah tests to run. "ALL" can be used to run all known utah tests.
740+ </description>
741+ <defaultValue>ALL</defaultValue>
742+ </hudson.model.StringParameterDefinition>
743+ <hudson.model.StringParameterDefinition>
744+ <name>APPS</name>
745+ <description>A space separated list of autopilot tests to run. "ALL" can be used to run all known tests.
746+ </description>
747+ <defaultValue>ALL</defaultValue>
748+ </hudson.model.StringParameterDefinition>
749+ </parameterDefinitions>
750+ </hudson.model.ParametersDefinitionProperty>
751+ <com.sonyericsson.rebuild.RebuildSettings>
752+ <autoRebuild>false</autoRebuild>
753+ </com.sonyericsson.rebuild.RebuildSettings>
754+ </properties>
755+ <scm class="hudson.scm.NullSCM"/>
756+ <assignedNode>{{ name }}</assignedNode>
757+ <canRoam>false</canRoam>
758+ <disabled>false</disabled>
759+ <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
760+ <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
761+ <triggers class="vector"/>
762+ <concurrentBuild>true</concurrentBuild>
763+ <builders>
764+ <hudson.tasks.Shell>
765+ <command>set -e
766+BRANCH=&quot;{{branch}}&quot;
767+BZRDIR=`echo &quot;$BRANCH&quot; | awk -F/ &apos;{ print $(NF) }&apos;`
768+BZRDIR=$(readlink -f $BZRDIR)
769+[ -d $BZRDIR ] &amp;&amp; rm -rf $BZRDIR
770+bzr branch ${BRANCH} ${BZRDIR}
771+
772+export ANDROID_SERIAL={{serial}}
773+export IMAGE_TYPE={{imagetype}}
774+
775+{% if statsd_key %}
776+# the txstatsd package is too old, use the one from LP:
777+[ -d txstatsd ] &amp;&amp; rm -rf txstatsd
778+bzr branch -r 109 lp:txstatsd
779+export PYTHONPATH=`pwd`/txstatsd
780+
781+export STATSD_KEY={{statsd_key}}.{{imagetype}}
782+{% endif %}
783+
784+{% if dashboard_host %}
785+export DASHBOARD_HOST="{{dashboard_host}}"
786+{% endif %}
787+{% if dashboard_user %}
788+export DASHBOARD_USER="{{dashboard_user}}"
789+{% endif %}
790+{% if dashboard_key %}
791+set +x # don't let this leak into the public
792+export DASHBOARD_KEY="{{dashboard_key}}"
793+set -x
794+{% endif %}
795+{% if dashboard_prefix %}
796+export DASHBOARD_PREFIX="{{dashboard_prefix}}"
797+{% endif %}
798+{% if dashboard_port %}
799+export DASHBOARD_PORT="{{dashboard_port}}"
800+{% endif %}
801+
802+{{image_opt}}
803+${BZRDIR}/scripts/run-smoke
804+ </command>
805+ </hudson.tasks.Shell>
806+ </builders>
807+ <publishers>
808+ <hudson.tasks.ArtifactArchiver>
809+ <artifacts>clientlogs/**</artifacts>
810+ <latestOnly>false</latestOnly>
811+ </hudson.tasks.ArtifactArchiver>
812+ <hudson.tasks.junit.JUnitResultArchiver>
813+ <testResults>clientlogs/**/*.xml</testResults>
814+ <keepLongStdio>true</keepLongStdio>
815+ <testDataPublishers>
816+ <hudson.plugins.junitattachments.AttachmentPublisher/>
817+ </testDataPublishers>
818+ </hudson.tasks.junit.JUnitResultArchiver>
819+ <hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
820+ <regexp>^= TOUCH IMAGE VERSION:([0-9]+.*)</regexp>
821+ <regexpForFailed>^= TOUCH IMAGE VERSION:([0-9]+.*)</regexpForFailed>
822+ <setForMatrix>false</setForMatrix>
823+ </hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
824+{% if publish %}
825+ <hudson.tasks.Mailer>
826+ <recipients>paul.larson@canonical.com</recipients>
827+ <dontNotifyEveryUnstableBuild>false</dontNotifyEveryUnstableBuild>
828+ <sendToIndividuals>false</sendToIndividuals>
829+ </hudson.tasks.Mailer>
830+ <hudson.plugins.build__publisher.BuildPublisher>
831+ <publishUnstableBuilds>true</publishUnstableBuilds>
832+ <publishFailedBuilds>true</publishFailedBuilds>
833+ <postActions class="vector"/>
834+ </hudson.plugins.build__publisher.BuildPublisher>
835+{% endif %}
836+ </publishers>
837+ <buildWrappers>
838+ <hudson.plugins.build__timeout.BuildTimeoutWrapper>
839+ <timeoutMinutes>300</timeoutMinutes>
840+ <failBuild>true</failBuild>
841+ <writingDescription>false</writingDescription>
842+ <timeoutPercentage>0</timeoutPercentage>
843+ <timeoutType>absolute</timeoutType>
844+ <timeoutMinutesElasticDefault>3</timeoutMinutesElasticDefault>
845+ </hudson.plugins.build__timeout.BuildTimeoutWrapper>
846+ </buildWrappers>
847+</project>
848+
849
850=== added file 'jenkins/testconfig.py'
851--- jenkins/testconfig.py 1970-01-01 00:00:00 +0000
852+++ jenkins/testconfig.py 2014-07-03 19:35:55 +0000
853@@ -0,0 +1,177 @@
854+#!/usr/bin/env python
855+
856+# Ubuntu Testing Automation Harness
857+# Copyright 2013 Canonical Ltd.
858+
859+# This program is free software: you can redistribute it and/or modify it
860+# under the terms of the GNU General Public License version 3, as published
861+# by the Free Software Foundation.
862+
863+# This program is distributed in the hope that it will be useful, but
864+# WITHOUT ANY WARRANTY; without even the implied warranties of
865+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
866+# PURPOSE. See the GNU General Public License for more details.
867+
868+# You should have received a copy of the GNU General Public License along
869+# with this program. If not, see <http://www.gnu.org/licenses/>.
870+
871+import argparse
872+
873+DEF_FMT = '{prefix}{series}-{imagetype}-{type}-smoke-{testname}'
874+IDLE_FMT = '{prefix}{testname}-{series}-{imagetype}-armhf-install-idle-{type}'
875+
876+
877+class Test(object):
878+ def __init__(self, name, fmt=DEF_FMT):
879+ self.name = name
880+ self.fmt = fmt
881+ self.smoke = fmt == DEF_FMT
882+
883+
884+class APTest(Test):
885+ def __init__(self, name, app=None, pkgs=None):
886+ Test.__init__(self, name)
887+ self.pkgs = pkgs
888+ if not app:
889+ # convert share-app-autopilot to share_app
890+ app = name.replace('-', '_').replace('_autopilot', '')
891+ self.app = app
892+
893+
894+TESTSUITES = [
895+ Test('default'),
896+]
897+TESTSUITES += [
898+ APTest('friends-app-autopilot', pkgs=['friends-app-autopilot']),
899+ APTest('mediaplayer-app-autopilot', pkgs=['mediaplayer-app-autopilot']),
900+ APTest('gallery-app-autopilot'),
901+ APTest('webbrowser-app-autopilot', pkgs=['webbrowser-app-autopilot']),
902+ APTest('unity8-autopilot', 'unity8', pkgs=['url-dispatcher-tools']),
903+ APTest('notes-app-autopilot'),
904+ APTest('camera-app-autopilot'),
905+ APTest('dialer-app-autopilot', pkgs=['dialer-app-autopilot']),
906+ APTest('messaging-app-autopilot', pkgs=['messaging-app-autopilot']),
907+ APTest('address-book-app-autopilot', pkgs=['address-book-app-autopilot']),
908+ APTest('calendar-app-autopilot'),
909+ APTest('music-app-autopilot'),
910+ APTest('dropping-letters-app-autopilot'),
911+ APTest('sudoku-app-autopilot'),
912+ APTest('ubuntu-calculator-app-autopilot'),
913+ APTest('ubuntu-clock-app-autopilot'),
914+ APTest('ubuntu-filemanager-app-autopilot'),
915+ APTest('shorts-app-autopilot'),
916+ APTest('ubuntu-terminal-app-autopilot'),
917+ APTest('ubuntu-weather-app-autopilot'),
918+ APTest('ubuntu-ui-toolkit-autopilot', 'ubuntuuitoolkit',
919+ ['ubuntu-ui-toolkit-autopilot']),
920+ APTest('ubuntu-system-settings-autopilot',
921+ pkgs=['ubuntu-system-settings-autopilot']),
922+ APTest('ubuntu-system-settings-online-accounts-autopilot',
923+ 'online_accounts_ui',
924+ ['ubuntu-system-settings-online-accounts-autopilot']),
925+]
926+TESTSUITES += [
927+ Test('click_image_tests'),
928+ Test('sdk'),
929+ Test('security'),
930+ Test('eventstat', IDLE_FMT),
931+ Test('smem', IDLE_FMT),
932+ Test('memevent',
933+ '{prefix}{testname}-{series}-{imagetype}-armhf-default-{type}'),
934+]
935+
936+
937+def filter_tests(tests, image_type):
938+ if image_type:
939+ func = globals().get('get_tests_%s' % image_type)
940+ if func:
941+ tests = func(tests)
942+ elif image_type not in ['touch', 'touch_mir', 'touch_custom_demo']:
943+ print('Unsupported image type: %s' % image_type)
944+ exit(1)
945+ return tests
946+
947+
948+def _get_tests(test_type, image_type):
949+ # take all our known tests, then call filter tests which should give
950+ # us a list of tests customized for things like sf4p or touch_custom.
951+ # then eliminate tests that aren't of the proper test_type
952+ tests = [t for t in filter_tests(TESTSUITES, image_type)
953+ if type(t) == test_type and t.fmt == DEF_FMT]
954+ return tests
955+
956+
957+def _handle_utah(args):
958+ tests = _get_tests(Test, args.image_type)
959+ if args.with_autopilot:
960+ tests = [t for t in TESTSUITES if t.fmt == DEF_FMT]
961+ print(' '.join([t.name for t in tests]))
962+
963+
964+def _handle_ap_apps(args):
965+ tests = _get_tests(APTest, args.image_type)
966+ print(' '.join([t.app for t in tests]))
967+
968+
969+def _handle_ap_packages(args):
970+ pkgs = []
971+ tests = _get_tests(APTest, args.image_type)
972+ for test in tests:
973+ if not args.app or test.app in args.app:
974+ if test.pkgs:
975+ pkgs.extend(test.pkgs)
976+ print(' '.join(pkgs))
977+
978+
979+def get_tests_touch_sf4p(common_tests):
980+ tests = []
981+ test_set = [
982+ 'install-and-boot',
983+ 'default',
984+ 'unity8-autopilot',
985+ ]
986+
987+ tests = [t for t in common_tests if t.name in test_set]
988+ return tests
989+
990+
991+def get_tests_touch_custom(common_tests):
992+ tests = common_tests
993+ tests.insert(1, Test('customizations'))
994+ return tests
995+
996+
997+def _get_parser():
998+ parser = argparse.ArgumentParser(
999+ description='List information on configured tests for touch')
1000+ sub = parser.add_subparsers(title='Commands', metavar='')
1001+
1002+ p = sub.add_parser('utah', help='List UTAH tests')
1003+ p.set_defaults(func=_handle_utah)
1004+ p.add_argument('-i', '--image-type',
1005+ help='Return list of test configured for an image type.')
1006+ p.add_argument('-a', '--with-autopilot', action='store_true',
1007+ help='Include autopilot tests that can be run under UTAH.')
1008+
1009+ p = sub.add_parser('apps', help='List autopilot application names')
1010+ p.set_defaults(func=_handle_ap_apps)
1011+ p.add_argument('-i', '--image-type',
1012+ help='Return list of test configured for an image type.')
1013+
1014+ p = sub.add_parser('packages', help='List packages required for autopilot')
1015+ p.set_defaults(func=_handle_ap_packages)
1016+ g = p.add_mutually_exclusive_group()
1017+ g.add_argument('-i', '--image-type',
1018+ help='''If no apps are listed, limit to tests for an
1019+ image type.''')
1020+ g.add_argument('-a', '--app', action='append',
1021+ help='Autopilot test application. eg share_app')
1022+ return parser
1023+
1024+
1025+def main():
1026+ args = _get_parser().parse_args()
1027+ return args.func(args)
1028+
1029+if __name__ == '__main__':
1030+ exit(main())
1031
1032=== added directory 'scripts'
1033=== renamed directory 'scripts' => 'scripts.moved'
1034=== added file 'scripts/assert-image'
1035--- scripts/assert-image 1970-01-01 00:00:00 +0000
1036+++ scripts/assert-image 2014-07-03 19:35:55 +0000
1037@@ -0,0 +1,28 @@
1038+#!/bin/sh
1039+
1040+set -e
1041+
1042+BASEDIR=$(readlink -f $(dirname $0)/..)
1043+
1044+# a simple script to ensure the proper image is installed on the target
1045+
1046+[ -z $INSTALL_URL ] && echo "= INSTALL_URL not set. Using target as-is." && exit 0
1047+
1048+echo "Ensuring target has proper image..."
1049+REQUIRED_UUID=$(curl ${INSTALL_URL}/artifact/clientlogs/.ci-uuid)
1050+ACTUAL_UUID=$(adb shell "cat /home/phablet/.ci-uuid | tr -d '\r\n'")
1051+if [ "$REQUIRED_UUID" != "$ACTUAL_UUID" ] ; then
1052+ echo "= UUIDs $REQUIRED_UUID != $ACTUAL_UUID"
1053+ ARGS=$(curl ${INSTALL_URL}/artifact/clientlogs/.ci-flash-args | tr -d '\r\n')
1054+ CUST=$(curl ${INSTALL_URL}/artifact/clientlogs/.ci-customizations | tr -d '\r\n')
1055+ echo "reprovisioning device with: $ARGS"
1056+ echo "customizing device with: $CUST"
1057+ UUID=$REQUIRED_UUID IMAGE_OPT=$ARGS CUSTOMIZE=$CUST \
1058+ ${BASEDIR}/scripts/provision.sh
1059+else
1060+ echo "= UUIDS match, reusing image on target"
1061+ echo $REQUIRED_UUID > clientlogs/.ci-uuid
1062+ curl ${INSTALL_URL}/artifact/clientlogs/.ci-flash-args > clientlogs/.ci-flash-args
1063+ curl ${INSTALL_URL}/artifact/clientlogs/.ci-customizations > clientlogs/.ci-customizations
1064+fi
1065+
1066
1067=== added file 'scripts/click-buddy-install.sh'
1068--- scripts/click-buddy-install.sh 1970-01-01 00:00:00 +0000
1069+++ scripts/click-buddy-install.sh 2014-07-03 19:35:55 +0000
1070@@ -0,0 +1,114 @@
1071+#!/bin/sh
1072+# This program is free software: you can redistribute it and/or modify it
1073+# under the terms of the the GNU General Public License version 3, as
1074+# published by the Free Software Foundation.
1075+#
1076+# This program is distributed in the hope that it will be useful, but
1077+# WITHOUT ANY WARRANTY; without even the implied warranties of
1078+# MERCHANTABILITY, SATISFACTORY QUALITY or FITNESS FOR A PARTICULAR
1079+# PURPOSE. See the applicable version of the GNU General Public
1080+# License for more details.
1081+#.
1082+# You should have received a copy of the GNU General Public License
1083+# along with this program. If not, see <http://www.gnu.org/licenses/>.
1084+#
1085+# Copyright (C) 2014 Canonical, Ltd.
1086+#
1087+# Author: Francis Ginther <francis.ginther@canonical.com>
1088+set -x
1089+export LC_ALL=C
1090+
1091+usage() {
1092+cat <<EOF
1093+usage: $0 [OPTIONS]
1094+
1095+Installs a custom built click app and autopilot tests
1096+
1097+OPTIONS:
1098+ -h Show this message
1099+ -s Specify the serial of the device to install (see adb $ADBOPTS devices)
1100+
1101+ --source-dir sources used to backreference the package
1102+ --click the click package to install
1103+EOF
1104+}
1105+
1106+ADBOPTS=""
1107+SOURCE_DIR=""
1108+CLICK=""
1109+
1110+TEST_DIR='tests/autopilot'
1111+DEVICE_USER='phablet'
1112+
1113+ARGS=$(getopt -o s:h -l "source-dir:,click:,help" -n "$0" -- "$@")
1114+
1115+if [ $? -ne 0 ] ; then
1116+ exit 1
1117+fi
1118+eval set -- "$ARGS"
1119+
1120+while true; do
1121+ case "$1" in
1122+ -h|--help)
1123+ usage
1124+ exit 0
1125+ ;;
1126+ -s)
1127+ shift;
1128+ ADBOPTS="-s $1"
1129+ shift;
1130+ ;;
1131+ --click)
1132+ shift
1133+ CLICK="$1"
1134+ shift
1135+ ;;
1136+ --source-dir)
1137+ shift
1138+ SOURCE_DIR="$(readlink -f $1)"
1139+ shift
1140+ ;;
1141+ --)
1142+ shift;
1143+ break;
1144+ ;;
1145+ esac
1146+done
1147+
1148+set -e
1149+adb $ADBOPTS push "$CLICK" /tmp
1150+click_name=`basename $CLICK`
1151+adb $ADBOPTS shell sudo -u $DEVICE_USER bash -ic "pkcon install-local /tmp/$click_name"
1152+
1153+# The PY2_SUBDIR logic is from lp:phablet-tools phablet-click-test-setup
1154+# It is replicated here because we are using a merged branch, not something
1155+# that is present in launchpad.
1156+PY2_SUBDIR='legacy-py2'
1157+TARGET_DIR=/home/$DEVICE_USER/autopilot/$PY2_SUBDIR
1158+
1159+#phablet-click-test-setup --click $CLICK
1160+click_info=`adb $ADBOPTS shell click info /tmp/$click_name`
1161+if echo $click_info | grep -q 'x-test'; then
1162+ if echo $click_info | grep -q 'autopilot'; then
1163+ echo "Looks like python3-autopilot tests"
1164+ TARGET_DIR=/home/$DEVICE_USER/autopilot
1165+ fi
1166+fi
1167+if [ -d $SOURCE_DIR/tests/autopilot ]; then
1168+ TEST_SOURCE_DIR=$SOURCE_DIR/tests/autopilot
1169+else
1170+ if [ -d $SOURCE_DIR/app/tests/autopilot ]; then
1171+ TEST_SOURCE_DIR=$SOURCE_DIR/app/tests/autopilot
1172+ else
1173+ echo "ERROR: Can't find test sources"
1174+ exit 1
1175+ fi
1176+fi
1177+adb $ADBOPTS shell mkdir -p $TARGET_DIR
1178+adb $ADBOPTS push $TEST_SOURCE_DIR $TARGET_DIR
1179+adb $ADBOPTS shell chown -R "$DEVICE_USER":"$DEVICE_USER" /home/$DEVICE_USER/autopilot
1180+set +e
1181+
1182+echo Allowing autopilot to play well with apparmor
1183+phablet-config autopilot --dbus-probe enable
1184+echo "Ready to run autopilot"
1185
1186=== added file 'scripts/combine_results'
1187--- scripts/combine_results 1970-01-01 00:00:00 +0000
1188+++ scripts/combine_results 2014-07-03 19:35:55 +0000
1189@@ -0,0 +1,121 @@
1190+#!/usr/bin/env python3
1191+
1192+"""
1193+We always run the system-settle test before and after an autopilot test. This
1194+script takes the results of the before/after results and combines them in with
1195+the junit xml results from the autopilot test so we have one unified report.
1196+"""
1197+
1198+
1199+import os
1200+import sys
1201+
1202+from xml.etree import ElementTree
1203+
1204+
1205+PRE_COMBINE = [
1206+ ('settle_before', 'settle_before'),
1207+ ('setup_setup', 'setup'),
1208+]
1209+
1210+POST_COMBINE = [
1211+ ('settle_after', 'settle_after'),
1212+ ('setup_teardown', 'teardown'),
1213+]
1214+
1215+
1216+def _build_node(classname, name, rcfile, stdout):
1217+ e = ElementTree.Element('testcase')
1218+ e.attrib['classname'] = classname
1219+ e.attrib['name'] = name
1220+
1221+ if not os.path.exists(rcfile):
1222+ return None, False
1223+
1224+ rc = int(open(rcfile).read())
1225+ if rc != 0:
1226+ f = ElementTree.Element('failure')
1227+ e.append(f)
1228+ f.attrib['type'] = 'testtools.testresult.real._StringException'
1229+ f.text = open(stdout).read()
1230+ return e, rc != 0
1231+
1232+
1233+def _get_results(apfile):
1234+ try:
1235+ tree = ElementTree.parse(apfile)
1236+ except Exception as ex:
1237+ e = ElementTree.Element('testsuite')
1238+ tree = ElementTree.ElementTree(e)
1239+ e.attrib['errors'] = '1'
1240+ e.attrib['failures'] = '0'
1241+ e.attrib['tests'] = '1'
1242+
1243+ # make a guess at the classname:
1244+ classname = os.path.basename(os.path.dirname(apfile))
1245+
1246+ t = ElementTree.Element('testcase')
1247+ e.append(t)
1248+ t.attrib['classname'] = classname
1249+ t.attrib['name'] = 'phablet-test-run'
1250+
1251+ f = ElementTree.Element('failure')
1252+ t.append(f)
1253+ f.attrib['type'] = 'testtools.testresult.real._StringException'
1254+ f.text = str(ex)
1255+
1256+ return tree
1257+
1258+
1259+def _get_classname(results):
1260+ cname = results[0].attrib.get('classname')
1261+ if cname:
1262+ cname = cname.split('.')[0]
1263+ else:
1264+ cname = '???'
1265+ return cname
1266+
1267+
1268+def combine(resdir):
1269+ ap_file = os.path.join(resdir, 'test_results.xml')
1270+ tree = _get_results(ap_file)
1271+ ap_results = tree.getroot()
1272+ added_results = 0
1273+
1274+ errors = int(ap_results.attrib['errors'])
1275+
1276+ classname = _get_classname(ap_results)
1277+
1278+ for basename, label in PRE_COMBINE:
1279+ rc = os.path.join(resdir, basename + '.rc')
1280+ log = os.path.join(resdir, basename + '.log')
1281+ node, failed = _build_node(classname, label, rc, log)
1282+ if node is not None:
1283+ ap_results.insert(0, node)
1284+ if failed:
1285+ errors += 1
1286+ added_results += 1
1287+
1288+ for basename, label in POST_COMBINE:
1289+ rc = os.path.join(resdir, basename + '.rc')
1290+ log = os.path.join(resdir, basename + '.log')
1291+ node, failed = _build_node(classname, label, rc, log)
1292+ if node is not None:
1293+ ap_results.append(node)
1294+ if failed:
1295+ errors += 1
1296+ added_results += 1
1297+
1298+ num = int(ap_results.attrib['tests']) + added_results
1299+ ap_results.attrib['tests'] = str(num)
1300+ ap_results.attrib['errors'] = str(errors)
1301+
1302+ tree.write(ap_file)
1303+
1304+
1305+if __name__ == '__main__':
1306+ if len(sys.argv) != 2:
1307+ print('usage: {} <results directory>'.format(sys.argv[0]))
1308+ sys.exit(1)
1309+
1310+ combine(sys.argv[1])
1311
1312=== added file 'scripts/dashboard.py'
1313--- scripts/dashboard.py 1970-01-01 00:00:00 +0000
1314+++ scripts/dashboard.py 2014-07-03 19:35:55 +0000
1315@@ -0,0 +1,298 @@
1316+#!/usr/bin/python
1317+
1318+import argparse
1319+import datetime
1320+import json
1321+import logging
1322+import os
1323+
1324+import yaml
1325+
1326+from httplib import ACCEPTED, HTTPConnection, HTTPException, OK, CREATED
1327+from urllib import urlencode
1328+from urlparse import urlparse
1329+
1330+log = logging.getLogger()
1331+
1332+
1333+class API(object):
1334+ def __init__(self, host=None, port=None, user=None, key=None, prefix=None):
1335+ if not host:
1336+ host = os.environ.get('DASHBOARD_HOST', None)
1337+ if not port:
1338+ port = int(os.environ.get('DASHBOARD_PORT', '80'))
1339+ if not user:
1340+ user = os.environ.get('DASHBOARD_USER', None)
1341+ if not key:
1342+ key = os.environ.get('DASHBOARD_KEY', None)
1343+ if not prefix:
1344+ prefix = os.environ.get('DASHBOARD_PREFIX', None)
1345+
1346+ self.host = host
1347+ self.port = port
1348+ self.resource_base = prefix
1349+
1350+ self._headers = None
1351+ if user and key:
1352+ self._headers = {
1353+ 'Content-Type': 'application/json',
1354+ 'Authorization': 'ApiKey %s:%s' % (user, key)
1355+ }
1356+ # mod_wsgi will strip the Authorization header, but tastypie
1357+ # allows it as GET param also. More details for fixing apache:
1358+ # http://django-tastypie.rtfd.org/en/latest/authentication.html
1359+ self._auth_param = '?username=%s&api_key=%s' % (user, key)
1360+
1361+ def _connect(self):
1362+ if self.host:
1363+ return HTTPConnection(self.host, self.port)
1364+ return None
1365+
1366+ def _http_get(self, resource):
1367+ con = self._connect()
1368+ if not con:
1369+ # we just mock this for the case where the caller wants to
1370+ # use our API transparently enabled/disabled
1371+ return {}
1372+
1373+ if self.resource_base:
1374+ resource = self.resource_base + resource
1375+
1376+ logging.debug('doing get on: %s', resource)
1377+ headers = {'Content-Type': 'application/json'}
1378+ con.request('GET', resource, headers=headers)
1379+ resp = con.getresponse()
1380+ if resp.status != OK:
1381+ msg = ''
1382+ try:
1383+ msg = resp.read().decode()
1384+ except:
1385+ pass
1386+ fmt = '%d error getting resource(%s): %s'
1387+ raise HTTPException(fmt % (resp.status, resource, msg))
1388+ data = json.loads(resp.read().decode())
1389+ if len(data['objects']) == 0:
1390+ raise HTTPException('resource not found: %s' % resource)
1391+ assert len(data['objects']) == 1
1392+ return data['objects'][0]['resource_uri']
1393+
1394+ def _http_post(self, resource, params):
1395+ con = self._connect()
1396+ if not con or not self._headers:
1397+ return None
1398+
1399+ if self.resource_base:
1400+ resource = self.resource_base + resource
1401+ resource += self._auth_param
1402+
1403+ params = json.dumps(params)
1404+ log.debug('posting (%s): %s', resource, params)
1405+ con.request('POST', resource, params, self._headers)
1406+ resp = con.getresponse()
1407+ if resp.status != CREATED:
1408+ msg = ''
1409+ try:
1410+ msg = str(resp.getheaders())
1411+ msg += resp.read().decode()
1412+ except:
1413+ pass
1414+ raise HTTPException(
1415+ '%d creating resource(%s): %s' % (resp.status, resource, msg))
1416+ uri = resp.getheader('Location')
1417+ return urlparse(uri).path
1418+
1419+ def _http_patch(self, resource, params):
1420+ con = self._connect()
1421+ if not con or not self._headers:
1422+ return None
1423+
1424+ resource += self._auth_param
1425+
1426+ con.request('PATCH', resource, json.dumps(params), self._headers)
1427+ resp = con.getresponse()
1428+ if resp.status != ACCEPTED:
1429+ msg = ''
1430+ try:
1431+ msg = resp.getheaders()
1432+ except:
1433+ pass
1434+ raise HTTPException(
1435+ '%d patching resource(%s): %s' % (resp.status, resource, msg))
1436+ return resource
1437+
1438+ @staticmethod
1439+ def _uri_to_pk(resource_uri):
1440+ if resource_uri:
1441+ return resource_uri.split('/')[-2]
1442+ return None # we are mocked
1443+
1444+ def job_get(self, name):
1445+ resource = '/smokeng/api/v1/job/?' + urlencode({'name': name})
1446+ return self._http_get(resource)
1447+
1448+ def job_add(self, name):
1449+ resource = '/smokeng/api/v1/job/'
1450+ params = {
1451+ 'name': name,
1452+ 'url': 'http://jenkins.qa.ubuntu.com/job/' + name + '/'
1453+ }
1454+ return self._http_post(resource, params)
1455+
1456+ def build_add(self, job_name, job_number):
1457+ try:
1458+ logging.debug('trying to find job: %s', job_name)
1459+ job = self.job_get(job_name)
1460+ except HTTPException:
1461+ job = self.job_add(job_name)
1462+ logging.info('job is: %s', job)
1463+
1464+ resource = '/smokeng/api/v1/build/'
1465+ params = {
1466+ 'build_number': job_number,
1467+ 'job': job,
1468+ 'ran_at': datetime.datetime.now().isoformat(),
1469+ 'build_description': 'inprogress',
1470+ }
1471+ return self._http_post(resource, params)
1472+
1473+ def _image_get(self, build_number, release, variant, arch, flavor):
1474+ resource = '/smokeng/api/v1/image/?'
1475+ resource += urlencode({
1476+ 'build_number': build_number,
1477+ 'release': release,
1478+ 'flavor': flavor,
1479+ 'variant': variant,
1480+ 'arch': arch,
1481+ })
1482+ return self._http_get(resource)
1483+
1484+ def image_add(self, build_number, release, variant, arch, flavor):
1485+ try:
1486+ img = self._image_get(build_number, release, variant, arch, flavor)
1487+ return img
1488+ except HTTPException:
1489+ # image doesn't exist so go continue and create
1490+ pass
1491+
1492+ resource = '/smokeng/api/v1/image/'
1493+ params = {
1494+ 'build_number': build_number,
1495+ 'release': release,
1496+ 'flavor': flavor,
1497+ 'variant': variant,
1498+ 'arch': arch,
1499+ }
1500+ return self._http_post(resource, params)
1501+
1502+ def result_get(self, image, test):
1503+ # deal with getting resource uri's as parameters instead of id's
1504+ image = API._uri_to_pk(image)
1505+
1506+ resource = '/smokeng/api/v1/result/?'
1507+ resource += urlencode({
1508+ 'image': image,
1509+ 'name': test,
1510+ })
1511+ return self._http_get(resource)
1512+
1513+ def _result_status(self, image, build, test, status, results=None):
1514+ create = False
1515+ params = {
1516+ 'ran_at': datetime.datetime.now().isoformat(),
1517+ 'status': status,
1518+ 'jenkins_build': build,
1519+ }
1520+ if results:
1521+ params['results'] = results
1522+
1523+ try:
1524+ resource = self.result_get(image, test)
1525+ except HTTPException:
1526+ create = True
1527+ resource = '/smokeng/api/v1/result/'
1528+ params['image'] = image
1529+ params['name'] = test
1530+
1531+ if create:
1532+ return self._http_post(resource, params)
1533+ else:
1534+ return self._http_patch(resource, params)
1535+
1536+ def result_queue(self, image, build, test):
1537+ return self._result_status(image, build, test, 0)
1538+
1539+ def result_running(self, image, build, test):
1540+ return self._result_status(image, build, test, 1)
1541+
1542+ def result_syncing(self, image, build, test, results):
1543+ return self._result_status(image, build, test, 2, results)
1544+
1545+
1546+def _result_running(api, args):
1547+ return api.result_running(args.image, args.build, args.test)
1548+
1549+
1550+def _result_syncing(api, args):
1551+ results = {}
1552+ with open(args.results) as f:
1553+ results = yaml.safe_load(f.read())
1554+ return api.result_syncing(args.image, args.build, args.test, results)
1555+
1556+
1557+def _set_args(parser, names, func):
1558+ for n in names:
1559+ parser.add_argument(n, required=True)
1560+ parser.set_defaults(func=func)
1561+
1562+
1563+def _get_parser():
1564+ parser = argparse.ArgumentParser(
1565+ description='Interact with the CI dashboard API')
1566+
1567+ sub = parser.add_subparsers(title='Commands', metavar='')
1568+
1569+ args = ['--image', '--build', '--test']
1570+ p = sub.add_parser('result-running', help='Set a SmokeResult "Running".')
1571+ _set_args(p, args, _result_running)
1572+
1573+ p = sub.add_parser('result-syncing', help='Set a SmokeResult "Syncing".')
1574+ _set_args(p, args, _result_syncing)
1575+ p.add_argument('--results', required=True, help='UTAH yaml file')
1576+
1577+ return parser
1578+
1579+
1580+def _assert_env():
1581+ required = [
1582+ 'DASHBOARD_HOST', 'DASHBOARD_PORT', 'DASHBOARD_USER', 'DASHBOARD_KEY']
1583+ missing = []
1584+ for r in required:
1585+ if r not in os.environ:
1586+ missing.append(r)
1587+ if len(missing):
1588+ print('Missing the following environment variables:')
1589+ for x in missing:
1590+ print(' %s' % x)
1591+ exit(1)
1592+
1593+
1594+def _main(args):
1595+ _assert_env()
1596+
1597+ api = API()
1598+ try:
1599+ val = args.func(api, args)
1600+ if val:
1601+ if '/?' in val:
1602+ log.debug('stripping api-key from response')
1603+ val = val.split('/?')[0] + '/'
1604+ print(val)
1605+ except HTTPException as e:
1606+ print('ERROR: %s' % e)
1607+ exit(1)
1608+
1609+ exit(0)
1610+
1611+if __name__ == '__main__':
1612+ args = _get_parser().parse_args()
1613+ exit(_main(args))
1614
1615=== added file 'scripts/device'
1616--- scripts/device 1970-01-01 00:00:00 +0000
1617+++ scripts/device 2014-07-03 19:35:55 +0000
1618@@ -0,0 +1,191 @@
1619+#!/usr/bin/python
1620+
1621+import argparse
1622+import contextlib
1623+import fcntl
1624+import imp
1625+import logging
1626+import os
1627+import sys
1628+import time
1629+
1630+
1631+class DeviceException(Exception):
1632+ """Just a handy way to pass errors to the CLI."""
1633+ pass
1634+
1635+
1636+@contextlib.contextmanager
1637+def _device_pool(args):
1638+ lockfile = os.path.join(args.device_pool, 'lockfile')
1639+ with open(lockfile, 'w') as f:
1640+ logging.debug('aquiring lock for device pool')
1641+ try:
1642+ fcntl.lockf(f, fcntl.LOCK_EX)
1643+ yield args.device_pool
1644+ finally:
1645+ fcntl.lockf(f, fcntl.LOCK_UN)
1646+
1647+
1648+def _get_tokenfile():
1649+ return '.device.%d' % os.getppid()
1650+
1651+
1652+def _lock_device(device_file):
1653+ logging.info('locking file %s', device_file)
1654+
1655+ name = os.path.basename(device_file)
1656+ lockfile = device_file + '.lck'
1657+
1658+ with open(device_file) as f:
1659+ contents = f.read()
1660+ with open(_get_tokenfile(), 'w') as w:
1661+ w.write(contents)
1662+ if contents[-1] != '\n':
1663+ w.write('\n')
1664+ w.write('DEVICE_NAME=%s\n' % name)
1665+ w.write('LOCK_FILE=%s\n' % lockfile)
1666+ os.rename(device_file, lockfile)
1667+
1668+
1669+def _get_by_name(args):
1670+ loops = 0
1671+ while True:
1672+ with _device_pool(args) as pool:
1673+ f = os.path.join(pool, args.name)
1674+ if os.path.exists(f):
1675+ _lock_device(f)
1676+ break
1677+ elif os.path.exists(f + '.lck'):
1678+ # throttle back logging as time progresses
1679+ if (loops < 10) or \
1680+ (loops < 100 and loops % 4 == 0) or \
1681+ (loops % 12 == 0):
1682+ logging.info(
1683+ 'device is locked, waiting... attempts(%d)', loops)
1684+ else:
1685+ raise DeviceException('No such device exists.')
1686+ time.sleep(5)
1687+ loops += 1
1688+
1689+
1690+def _get_by_type(args):
1691+ while True:
1692+ with _device_pool(args) as pool:
1693+ match = False
1694+ for d in os.listdir(pool):
1695+ if args.type in d:
1696+ match = True
1697+ if not d.endswith('.lck'):
1698+ _lock_device(os.path.join(pool, d))
1699+ return
1700+ if not match:
1701+ raise DeviceException(
1702+ 'No "%s" devices found in pool.' % args.type)
1703+ time.sleep(5)
1704+
1705+
1706+def _get_by_env(args, jobname):
1707+ # find the device portion of a string like:
1708+ # saucy-touch_ro-DEVICETYPE-smoke-ubuntu-terminal-app-autopilot
1709+ parts = jobname.split('-')
1710+ if len(parts) < 3:
1711+ raise DeviceException('Invalid jobname: %s.' % jobname)
1712+ args.type = parts[2]
1713+ _get_by_type(args)
1714+
1715+
1716+def _device_get(args):
1717+ job = os.environ.get('JOBNAME', False)
1718+
1719+ if args.name:
1720+ _get_by_name(args)
1721+ elif args.type:
1722+ _get_by_type(args)
1723+ elif job:
1724+ _get_by_env(args, job)
1725+ else:
1726+ raise DeviceException('no device, type or $JOBNAME specified.')
1727+
1728+
1729+def _device_free(args):
1730+ if not os.path.exists(args.tokenfile):
1731+ raise DeviceException('tokenfile(%s) does not exist.' % args.tokenfile)
1732+
1733+ lockfile = None
1734+ with open(args.tokenfile) as f:
1735+ for line in f.readlines():
1736+ if line.startswith('LOCK_FILE='):
1737+ lockfile = line.split('=', 2)[1].strip()
1738+ break
1739+
1740+ device_file = os.path.splitext(lockfile)[0]
1741+ with _device_pool(args):
1742+ logging.info('releasing lock for %s', device_file)
1743+ os.rename(lockfile, device_file)
1744+ os.unlink(args.tokenfile)
1745+
1746+
1747+def _populate(args):
1748+ if os.path.exists(args.device_pool):
1749+ raise DeviceException('pool(%s) already exists.' % args.device_pool)
1750+ os.makedirs(args.device_pool)
1751+
1752+ # jump through some hoops to import this module since it has dashes
1753+ # and doesn't have a .py extension
1754+ module = 'get-adb-id'
1755+ fname = os.path.join(os.path.dirname(__file__), module)
1756+ adb = imp.load_source(module, fname)
1757+
1758+ # import the config
1759+ config_module = imp.load_source('config', args.config)
1760+ for config in config_module.MATRIX.values():
1761+ for item in config:
1762+ for device in item['devices']:
1763+ name = device['name']
1764+ serial = device.get('serial', None)
1765+ if not serial:
1766+ serial = adb.DEVICES[name]
1767+ with open(os.path.join(args.device_pool, name), 'w') as f:
1768+ f.write('ANDROID_SERIAL=%s' % serial)
1769+
1770+
1771+def _get_args():
1772+ parser = argparse.ArgumentParser(
1773+ description='''Provides a mechanism to aquire a lock on a device in
1774+ a given resource pool. The lock info will be written in
1775+ the format .device.<calling pid>. This file can be sourced
1776+ by a shell script to get ANDROID_SERIAL etc.''')
1777+ parser.add_argument('--device-pool', default='/var/lib/jenkins/ci-devices',
1778+ help='location of the device pool. %(default)s')
1779+
1780+ sub = parser.add_subparsers(title='Commands', metavar='')
1781+ p = sub.add_parser('get', help='Aquire a lock on an idle device.')
1782+ grp = p.add_mutually_exclusive_group()
1783+ grp.add_argument('--name', help='Get device by name')
1784+ grp.add_argument('--type', help='Get device by type')
1785+ p.set_defaults(func=_device_get)
1786+
1787+ p = sub.add_parser('free', help='Release a lock on a device.')
1788+ p.add_argument('--tokenfile', default=_get_tokenfile(),
1789+ help='token file for device reservation. %(default)s')
1790+ p.set_defaults(func=_device_free)
1791+
1792+ p = sub.add_parser('populate', help='Initialize a device pool for usage')
1793+ p.add_argument('-c', '--config', required=True,
1794+ help='The jenkins config to find devices in.')
1795+ p.set_defaults(func=_populate)
1796+
1797+ return parser.parse_args()
1798+
1799+
1800+if __name__ == '__main__':
1801+ logging.basicConfig(level=logging.INFO)
1802+ args = _get_args()
1803+ rc = 0
1804+ try:
1805+ args.func(args)
1806+ except DeviceException as e:
1807+ rc = 1
1808+ print('ERROR: %s' % e)
1809+ sys.exit(rc)
1810
1811=== added file 'scripts/get-adb-id'
1812--- scripts/get-adb-id 1970-01-01 00:00:00 +0000
1813+++ scripts/get-adb-id 2014-07-03 19:35:55 +0000
1814@@ -0,0 +1,44 @@
1815+#!/usr/bin/python
1816+
1817+import sys
1818+
1819+DEVICES = {
1820+ "maguro-01": "0149BD7E0A019003",
1821+ "maguro-02": "0149BD7E0200E00F",
1822+ "maguro-03": "0149C7A50A00C011",
1823+ "maguro-04": "0149BDCC0500701B",
1824+ "maguro-05": "0149BD3418014013",
1825+ "maguro-06": "014E058C0F00C013",
1826+ "maguro-07": "01498FE717002014",
1827+ "ps-mako-01": "0090f741e3d141bc",
1828+ "ps-mako-04": "04cbcc545f5328a5",
1829+ "mako-01": "01aa3d7a5dcba4a2",
1830+ "mako-02": "01ade38b552014d4",
1831+ "mako-03": "04c6714ed7c863f2",
1832+ "mako-05": "01b22f82dc5cec63",
1833+ "mako-06": "04ed70928fdc13ba",
1834+ "mako-07": "01e2f64788556934",
1835+ "mako-08": "04ea16a163930769",
1836+ "mako-10": "01ce848e48dfa6a2",
1837+ "mako-11": "04ed727c929709ba",
1838+ "grouper-02": "015d1884951c020e",
1839+ "grouper-03": "015d18ad5c27f80d",
1840+ "grouper-04": "015d188445242611",
1841+ "grouper-05": "015d21d9151c0019",
1842+ "manta-01": "R32D102RPZL",
1843+ "manta-02": "R32D102RPPK",
1844+ "manta-04": "R32D203DDZR",
1845+ "manta-05": "R32D203DMBY",
1846+ "flo-01": "09f306dc",
1847+ "flo-02": "08dbee36",
1848+ "flo-03": "09d55fa8",
1849+}
1850+
1851+if __name__ == '__main__':
1852+ name = sys.argv[1]
1853+
1854+ try:
1855+ print(DEVICES[name])
1856+ except KeyError:
1857+ print("Unknown device name: '%s'" % name)
1858+ sys.exit(-1)
1859
1860=== added file 'scripts/jenkins.sh'
1861--- scripts/jenkins.sh 1970-01-01 00:00:00 +0000
1862+++ scripts/jenkins.sh 2014-07-03 19:35:55 +0000
1863@@ -0,0 +1,186 @@
1864+#!/bin/bash
1865+
1866+## This is the script jenkins should run to execute various touch applications
1867+
1868+set -e
1869+
1870+BASEDIR=$(dirname $(readlink -f $0))/..
1871+
1872+RESDIR="${RESDIR-`pwd`/clientlogs}"
1873+UTAHFILE=${RESDIR}/utah.yaml
1874+UTAH_PHABLET_CMD="${UTAH_PHABLET_CMD-/usr/share/utah/examples/run_utah_phablet.py}"
1875+
1876+
1877+usage() {
1878+ cat <<EOF
1879+usage: $0 -a APP [-s ANDROID_SERIAL] [-p FILE -p FILE ...] [-Q]
1880+
1881+Provisions the given device with the latest build
1882+
1883+OPTIONS:
1884+ -h Show this message
1885+ -s Specify the serial of the device to install
1886+ -a The application under the "tests" directory to test
1887+ -p Extra file to pull from target (absolute path or relative to /home/phablet)
1888+ -Q "Quick" don't do a reboot of the device before running the test
1889+
1890+EOF
1891+}
1892+
1893+PIDS=""
1894+
1895+cleanup() {
1896+ set +e
1897+ echo "killing child pids: $PIDS"
1898+ for p in $PIDS ; do
1899+ kill $p
1900+ done
1901+}
1902+
1903+test_from_host() {
1904+ export PATH=${BASEDIR}/utils/host:${PATH}
1905+
1906+ # allow for certain commands to run from host/target
1907+ # see unity8-autopilot/ts_control for example
1908+ export TARGET_PREFIX=adb-shell
1909+
1910+ [ -z $ANDROID_SERIAL ] || ADBOPTS="-s $ANDROID_SERIAL"
1911+
1912+ sudo TARGET_PREFIX=$TARGET_PREFIX PATH="${PATH}" ${UTAH_PHABLET_CMD} \
1913+ ${ADBOPTS} \
1914+ --from-host \
1915+ --whoopsie \
1916+ --results-dir ${RESDIR} \
1917+ --skip-install --skip-network --skip-utah \
1918+ --pull /var/crash \
1919+ --pull /home/phablet/.cache/upstart \
1920+ --pull /tmp/xmlresults \
1921+ $EXTRA_PULL \
1922+ -l ${TESTSUITE_HOST}/master.run
1923+
1924+ # make sure the user running this script can remove its artifacts.
1925+ sudo chown -R `whoami` ${RESDIR}
1926+}
1927+
1928+assert_image() {
1929+ [ -z $INSTALL_URL ] && return
1930+ echo "Ensuring target has proper image..."
1931+ REQUIRED_UUID=$(curl ${INSTALL_URL}/artifact/clientlogs/.ci-uuid)
1932+ ACTUAL_UUID=$(adb shell "cat /home/phablet/.ci-uuid | tr -d '\r\n'")
1933+ if [ "$REQUIRED_UUID" != "$ACTUAL_UUID" ] ; then
1934+ echo "UUIDs $REQUIRED_UUID != $ACTUAL_UUID, reprovisioning device..."
1935+ ARGS=$(curl ${INSTALL_URL}/artifact/clientlogs/.ci-utah-args | tr -d '\r\n')
1936+ UUID=$REQUIRED_UUID IMAGE_OPT=$ARGS ${BASEDIR}/scripts/provision.sh
1937+ else
1938+ echo "UUIDS match"
1939+ fi
1940+}
1941+
1942+main() {
1943+ rm -rf $RESDIR
1944+ mkdir $RESDIR
1945+
1946+ assert_image
1947+
1948+ # print the build date so the jenkins job can use it as the
1949+ # build description
1950+ adb pull /var/log/installer/media-info ${RESDIR}
1951+ BUILDID=$(adb shell cat /home/phablet/.ci-version)
1952+ echo "= TOUCH IMAGE VERSION:$BUILDID"
1953+
1954+ adb shell "top -n1 -b" > ${RESDIR}/top.log
1955+
1956+ set -x
1957+ adb shell 'rm -f /var/crash/*'
1958+ if [ -z $QUICK ] ; then
1959+ # get the phone in sane place
1960+ adb reboot
1961+ # sometimes reboot doesn't happen fast enough, so add a little
1962+ # delay to help ensure its actually rebooted and we didn't just
1963+ # connect back to the device before it rebooted
1964+ adb wait-for-device
1965+ sleep 5
1966+ adb wait-for-device
1967+ phablet-network --skip-setup -t 90s
1968+ adb shell powerd-cli active &
1969+ PIDS="$PIDS $!"
1970+ adb shell powerd-cli display on &
1971+ PIDS="$PIDS $!"
1972+ else
1973+ echo "SKIPPING phone reboot..."
1974+ fi
1975+
1976+ ${BASEDIR}/utils/host/adb-shell "aa-clickhook -f --include=/usr/share/autopilot-touch/apparmor/click.rules"
1977+
1978+ echo "launching test from the host...."
1979+ test_from_host
1980+ adb shell 'rm -f /var/crash/*'
1981+
1982+ if ! `grep "^errors: [!0]" < $UTAHFILE >/dev/null` ; then
1983+ echo "errors found"
1984+ EXITCODE=1
1985+ fi
1986+ if ! `grep "^failures: [!0]" < $UTAHFILE >/dev/null` ; then
1987+ echo "failures found"
1988+ EXITCODE=2
1989+ fi
1990+ echo "Results Summary"
1991+ echo "---------------"
1992+ egrep '^(errors|failures|passes|fetch_errors):' $UTAHFILE
1993+ exit $EXITCODE
1994+}
1995+
1996+while getopts p:s:a:Qh opt; do
1997+ case $opt in
1998+ h)
1999+ usage
2000+ exit 0
2001+ ;;
2002+ s)
2003+ export ANDROID_SERIAL=$OPTARG
2004+ ;;
2005+ a)
2006+ APP=$OPTARG
2007+ ;;
2008+ p)
2009+ EXTRA_PULL_FILE=$OPTARG
2010+
2011+ if [ ! -z $EXTRA_PULL_FILE ]; then
2012+ # relative paths are assumed to be relative to /home/phablet
2013+ E_P_START=`echo $EXTRA_PULL_FILE | cut -c1`
2014+
2015+ if [ $E_P_START = '/' ]; then
2016+ EXTRA_PULL="$EXTRA_PULL --pull $EXTRA_PULL_FILE"
2017+ else
2018+ EXTRA_PULL="$EXTRA_PULL --pull /home/phablet/$EXTRA_PULL_FILE"
2019+ fi
2020+ fi
2021+ ;;
2022+ Q)
2023+ QUICK=1
2024+ ;;
2025+ esac
2026+done
2027+
2028+if [ -z $ANDROID_SERIAL ] ; then
2029+ # ensure we only have one device attached
2030+ lines=$(adb devices | wc -l)
2031+ if [ $lines -gt 3 ] ; then
2032+ echo "ERROR: More than one device attached, please use -s option"
2033+ echo
2034+ usage
2035+ exit 1
2036+ fi
2037+fi
2038+if [ -z $APP ] ; then
2039+ echo "ERROR: No app specified"
2040+ usage
2041+ exit 1
2042+fi
2043+
2044+TESTSUITE_HOST=$(readlink -f ${BASEDIR}/tests/${APP})
2045+TESTSUITE_TARGET_BASE=/tmp/tests
2046+TESTSUITE_TARGET=${TESTSUITE_TARGET_BASE}/$(basename ${TESTSUITE_HOST})
2047+
2048+trap cleanup TERM INT EXIT
2049+main
2050
2051=== added file 'scripts/junit2utah.py'
2052--- scripts/junit2utah.py 1970-01-01 00:00:00 +0000
2053+++ scripts/junit2utah.py 2014-07-03 19:35:55 +0000
2054@@ -0,0 +1,70 @@
2055+#!/usr/bin/python
2056+
2057+import datetime
2058+import sys
2059+
2060+from xml.etree import ElementTree
2061+
2062+import yaml
2063+
2064+
2065+def _convert_testcase(tc):
2066+ x = {
2067+ 'testcase': tc.attrib['name'],
2068+ 'testsuite': tc.attrib['classname'],
2069+ 'command': 'autopilot',
2070+ 'cmd_type': 'testcase_test',
2071+ 'stdout': '',
2072+ 'stderr': '',
2073+ 'returncode': 0
2074+ }
2075+ t = tc.attrib.get('time', False)
2076+ if t:
2077+ x['time_delta'] = t
2078+
2079+ for e in tc.getchildren():
2080+ if e.tag in ('failure', 'error'):
2081+ x['stderr'] = e.text
2082+ x['returncode'] = 1
2083+ elif e.tag == 'skip':
2084+ # NOTE: this isn't a real thing in UTAH. However, the
2085+ # qa-dashboard code doesn't care and will display it as skipped
2086+ x['cmd_type'] = 'testcase_skipped'
2087+ x['stdout'] = e.text
2088+ else:
2089+ raise RuntimeError('Unknown element type: %s' % e.tag)
2090+ return x
2091+
2092+
2093+def _get_results(stream):
2094+ tree = ElementTree.fromstring(stream.read())
2095+ results = {
2096+ 'errors': int(tree.attrib.get('errors', '0')),
2097+ 'failures': int(tree.attrib.get('failures', '0')),
2098+ 'commands': [],
2099+ 'fetch_errors': 0,
2100+ 'uname': 'n/a',
2101+ 'media-info': 'n/a',
2102+ 'install_type': 'n/a',
2103+ 'arch': 'n/a',
2104+ 'release': 'n/a',
2105+ 'build_number': 'n/a',
2106+ 'name': 'unamed',
2107+ 'runlist': 'n/a',
2108+ 'ran_at': datetime.datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S'),
2109+ }
2110+ results['passes'] = \
2111+ int(tree.attrib['tests']) - results['errors'] - results['failures']
2112+
2113+ for tc in tree.getchildren():
2114+ results['commands'].append(_convert_testcase(tc))
2115+ return results
2116+
2117+
2118+def _main(stream):
2119+ results = _get_results(stream)
2120+ print(yaml.safe_dump(results, default_flow_style=False))
2121+
2122+
2123+if __name__ == '__main__':
2124+ exit(_main(sys.stdin))
2125
2126=== added file 'scripts/mp_provision'
2127--- scripts/mp_provision 1970-01-01 00:00:00 +0000
2128+++ scripts/mp_provision 2014-07-03 19:35:55 +0000
2129@@ -0,0 +1,299 @@
2130+#!/usr/bin/env python
2131+# This program is free software: you can redistribute it and/or modify it
2132+# under the terms of the the GNU General Public License version 3, as
2133+# published by the Free Software Foundation.
2134+#
2135+# This program is distributed in the hope that it will be useful, but
2136+# WITHOUT ANY WARRANTY; without even the implied warranties of
2137+# MERCHANTABILITY, SATISFACTORY QUALITY or FITNESS FOR A PARTICULAR
2138+# PURPOSE. See the applicable version of the GNU General Public
2139+# License for more details.
2140+#.
2141+# You should have received a copy of the GNU General Public License
2142+# along with this program. If not, see <http://www.gnu.org/licenses/>.
2143+#
2144+# Copyright (C) 2014 Canonical, Ltd.
2145+#
2146+# Author: Francis Ginther <francis.ginther@canonical.com>
2147+
2148+import argparse
2149+import atexit
2150+import logging
2151+import os
2152+import shutil
2153+import subprocess
2154+import sys
2155+import tempfile
2156+import textwrap
2157+import urllib2
2158+
2159+MAX_RETRIES = 3
2160+HERE = os.path.abspath(os.path.dirname(__file__))
2161+log = logging.getLogger()
2162+
2163+
2164+def parse_arguments():
2165+ parser = argparse.ArgumentParser(
2166+ description='''Provisions the given device to run tests against a
2167+ jenkins upstream merger job.''')
2168+ parser.add_argument('-s', '--serial',
2169+ help='Android device serial number.')
2170+ parser.add_argument('-f', '--flash',
2171+ default=False,
2172+ action='store_true',
2173+ help='Flash the device with phablet-flash.')
2174+ parser.add_argument('-n', '--network-config',
2175+ default=None,
2176+ required=True,
2177+ help='A phablet-config network configuration file.')
2178+ parser.add_argument('-u', '--package-url',
2179+ default=None,
2180+ help='''The url to the jenkins build package archive
2181+ (a zip file).''')
2182+ parser.add_argument('-H', '--hooks',
2183+ default=None,
2184+ help='''The hooks parameter from the jenkins job which
2185+ reference PPAs.''')
2186+ parser.add_argument('-P', '--ppa',
2187+ default=[],
2188+ action='append',
2189+ help='A ppa to install (can be repeated).')
2190+ parser.add_argument('-p', '--package',
2191+ default=[],
2192+ action='append',
2193+ help='A package to install (can be repeated).')
2194+ parser.add_argument('--source-branch',
2195+ default=None,
2196+ help='The bzr source branch (MP) backing the packages.')
2197+ parser.add_argument('--target-branch',
2198+ default=None,
2199+ help='The bzr target trunk branch.')
2200+ parser.add_argument('--packaging-branch',
2201+ default=None,
2202+ help='The bzr target packaging branch.')
2203+ return parser.parse_args()
2204+
2205+
2206+def get_url(url):
2207+ file_name = 'package_archive.zip'
2208+ try:
2209+ req = urllib2.Request(url)
2210+ f = urllib2.urlopen(req)
2211+ response = f.read()
2212+ f.close()
2213+ except IOError as e:
2214+ raise EnvironmentError('Failed to open url {}: '
2215+ '{}'.format(url, e))
2216+ try:
2217+ with open(file_name, 'w') as archive_file:
2218+ archive_file.write(response)
2219+ return file_name
2220+ except IOError as e:
2221+ raise EnvironmentError('Failed to create archive file: '
2222+ '{}'.format(file_name, e))
2223+
2224+def cleanup(directory):
2225+ if os.path.exists(directory) and os.path.isdir(directory):
2226+ log.info('Removing directory %s' % directory)
2227+ shutil.rmtree(directory)
2228+
2229+
2230+def create_temp_dir():
2231+ directory = tempfile.mkdtemp(dir=os.getcwd())
2232+ atexit.register(cleanup, directory)
2233+ return directory
2234+
2235+
2236+def find_click_package(archive_dir):
2237+ '''Returns a single click package filename if found.'''
2238+ click_package = None
2239+ for root, dirs, files in os.walk(archive_dir):
2240+ for filename in files:
2241+ if filename.endswith('click'):
2242+ if not click_package:
2243+ click_package = os.path.join(root, filename)
2244+ else:
2245+ raise EnvironmentError('Found too many click packages')
2246+ return click_package
2247+
2248+
2249+def find_deb_package(archive_dir):
2250+ '''Searches for the presence of a deb package file.'''
2251+ for root, dirs, files in os.walk(archive_dir):
2252+ for filename in files:
2253+ if filename.endswith('deb'):
2254+ return True
2255+ return False
2256+
2257+
2258+def get_jenkins_package_archive(url):
2259+ '''Downloads and unzips a jenkins archive.
2260+
2261+ Returns a tuple with a list of archive dirs and a click package.'''
2262+ if not url:
2263+ return ([], None)
2264+ archive_file = get_url(url)
2265+ archive_dir = create_temp_dir()
2266+ unzip_cmd = ['unzip', archive_file, '-d', archive_dir]
2267+ subprocess.check_call(unzip_cmd)
2268+ click_package = find_click_package(archive_dir)
2269+ if find_deb_package(archive_dir):
2270+ return ([archive_dir], click_package)
2271+ else:
2272+ return ([], click_package)
2273+
2274+
2275+def bzr_build(recipe, target):
2276+ '''Executes the bzr build command on the given recipe.'''
2277+ try:
2278+ bzr_cmd = ['/usr/bin/bzr', 'build', recipe, target]
2279+ subprocess.check_call(bzr_cmd)
2280+ except subprocess.CalledProcessError as e:
2281+ raise RuntimeError('Error while preparing from recipe: {}'.format(
2282+ e.message))
2283+
2284+
2285+RECIPE = textwrap.dedent('''\
2286+ # bzr-builder format 0.4
2287+ {source_branch}
2288+ {merge_cmd}
2289+ {nest_part_cmd}
2290+ ''')
2291+def get_click_source(source_branch, target_branch, packaging_branch):
2292+ '''Get the source branch(es) for populating the click tests.'''
2293+ if not (source_branch or target_branch):
2294+ return None
2295+ recipe_path = 'bzr.recipe'
2296+ trunk_dir = os.path.join(create_temp_dir(), 'trunk')
2297+ if target_branch:
2298+ merge_cmd = 'merge target {}'.format(target_branch)
2299+ else:
2300+ merge_cmd = '# No target branch'
2301+ if packaging_branch:
2302+ nest_part_cmd = 'nest-part ubuntu {} debian'.format(packaging_branch)
2303+ else:
2304+ nest_part_cmd = '# No packaging branch'
2305+
2306+ recipe = RECIPE.format(source_branch=source_branch,
2307+ merge_cmd=merge_cmd,
2308+ nest_part_cmd=nest_part_cmd)
2309+ log.info(recipe)
2310+ with open(recipe_path , 'w') as recipe_file:
2311+ recipe_file.write(recipe)
2312+ bzr_exception = None
2313+ retry_count = 0
2314+ while retry_count < MAX_RETRIES:
2315+ try:
2316+ bzr_build(recipe_path, trunk_dir)
2317+ return trunk_dir
2318+ except Exception as bzr_exception:
2319+ log.error('Exception: {}'.format(bzr_exception))
2320+ retry_count += 1
2321+ raise bzr_exception
2322+
2323+
2324+PPA_HOOK_MAGIC = 'D09add_ppa'
2325+def convert_hook_to_ppa(hooks):
2326+ '''Extracts any PPAs indicated by the pbuilderjenkins hooks.'''
2327+ hook_list = []
2328+ if not hooks:
2329+ return []
2330+ for hook in hooks.split(' '):
2331+ if hook.startswith(PPA_HOOK_MAGIC):
2332+ (magic, ppa_team, ppa_name) = hook.split('~')
2333+ hook_list.append('ppa:{}/{}'.format(ppa_team, ppa_name))
2334+ return hook_list
2335+
2336+
2337+def provision(serial=None, flash=False, network_config=None,
2338+ archive_directories=[], ppa_list=[], deb_packages=[],
2339+ click_package=None, click_source=None):
2340+ '''Constructs and executes the provision commands.'''
2341+ # TODO: Remove the '-w' option to set the image writable once the unlock
2342+ # logic no longer requires installation of unity8-autopilot
2343+ provision_cmd = [os.path.join(HERE, 'provision.sh'), '-w']
2344+ click_cmd = [os.path.join(HERE, 'click-buddy-install.sh')]
2345+
2346+ # Click provisioning is always based on the click package
2347+ os.putenv('SKIP_CLICK', 'skip_click')
2348+ if serial:
2349+ provision_cmd.extend(['-s', serial])
2350+ if not flash:
2351+ os.putenv('SKIP_FLASH', 'skip_flash')
2352+ if network_config:
2353+ provision_cmd.extend(['-n', network_config])
2354+ if click_package:
2355+ click_cmd.extend(['--click', click_package])
2356+ if click_source:
2357+ click_cmd.extend(['--source-dir', click_source])
2358+ for archive in archive_directories:
2359+ provision_cmd.extend(['-D', archive])
2360+ for ppa in ppa_list:
2361+ provision_cmd.extend(['-P', ppa])
2362+ for package in deb_packages:
2363+ provision_cmd.extend(['-p', package])
2364+ log.info(provision_cmd)
2365+ subprocess.check_call(provision_cmd)
2366+ if click_package:
2367+ log.info(click_cmd)
2368+ subprocess.check_call(click_cmd)
2369+
2370+
2371+def get_args_from_environment(args):
2372+ '''Fill in options from the environment.'''
2373+ if not args.package_url:
2374+ args.package_url = os.getenv('package_url')
2375+ if not args.hooks:
2376+ args.hooks = os.getenv('hooks')
2377+ if not args.ppa:
2378+ ppa = os.getenv('ppa')
2379+ if ppa:
2380+ args.ppa = ppa.split()
2381+ if not args.package:
2382+ package = os.getenv('package')
2383+ if package:
2384+ args.package = package.split()
2385+ if not args.source_branch:
2386+ args.source_branch = os.getenv('source_branch')
2387+ if not args.source_branch:
2388+ # The jenkins MP testing uses 'landing_candidate' as the bzr source
2389+ args.source_branch = os.getenv('landing_candidate')
2390+ if not args.target_branch:
2391+ args.target_branch = os.getenv('target_branch')
2392+ if not args.packaging_branch:
2393+ args.packaging_branch = os.getenv('packaging_branch')
2394+ return args
2395+
2396+
2397+def main():
2398+ args = parse_arguments()
2399+ args = get_args_from_environment(args)
2400+
2401+ if not (args.package or args.package_url):
2402+ print('Must provision with packages or a url.')
2403+ return -1
2404+ rc = 1
2405+ try:
2406+ (archive_dir, click_package) = get_jenkins_package_archive(
2407+ args.package_url)
2408+ click_source = get_click_source(args.source_branch,
2409+ args.target_branch,
2410+ args.packaging_branch)
2411+ if click_source and not click_package:
2412+ print('Bzr source was specified but no click package was found.')
2413+ return -1
2414+
2415+ ppa_list = args.ppa
2416+ ppa_list.extend(convert_hook_to_ppa(args.hooks))
2417+ package_list = args.package
2418+ provision(args.serial, args.flash, args.network_config, archive_dir,
2419+ ppa_list, package_list, click_package, click_source)
2420+ rc = 0
2421+ except EnvironmentError as e:
2422+ log.error(e)
2423+ return -1
2424+ return rc
2425+
2426+
2427+if __name__ == '__main__':
2428+ sys.exit(main())
2429
2430=== added file 'scripts/provision.sh'
2431--- scripts/provision.sh 1970-01-01 00:00:00 +0000
2432+++ scripts/provision.sh 2014-07-03 19:35:55 +0000
2433@@ -0,0 +1,144 @@
2434+#!/bin/bash
2435+
2436+## This is the script jenkins should run to provision a device in the lab
2437+
2438+set -e
2439+
2440+BASEDIR=$(dirname $(readlink -f $0))
2441+
2442+RESDIR=`pwd`/clientlogs
2443+
2444+NETWORK_FILE="${NETWORK_FILE-/home/ubuntu/magners-wifi}"
2445+
2446+IMAGE_OPT="${IMAGE_OPT---bootstrap --channel utopic-proposed}"
2447+UUID="${UUID-$(uuidgen -r)}"
2448+
2449+usage() {
2450+cat <<EOF
2451+usage: $0 [-s ANDROID_SERIAL] [-n NETWORK_FILE] [-P ppa] [-p package] [-w]
2452+
2453+Provisions the given device with the latest build
2454+
2455+OPTIONS:
2456+ -h Show this message
2457+ -s Specify the serial of the device to install
2458+ -n Select network file
2459+ -D add a debian package dir to the target (can be repeated)
2460+ -P add the ppa to the target (can be repeated)
2461+ -p add the package to the target (can be repeated)
2462+ -w make the system writeable (implied with -p and -P arguments)
2463+
2464+EOF
2465+}
2466+
2467+image_info() {
2468+ # mark the version we installed in /home/phablet/.ci-[uuid,flash-args]
2469+ # adb shell messes up \n's with \r\n's so do the whole of the regex on the target
2470+ IMAGEVER=$(adb shell "system-image-cli -i | sed -n -e 's/version version: \([0-9]*\)/\1/p' -e 's/version ubuntu: \([0-9]*\)/\1/p' -e 's/version device: \([0-9]*\)/\1/p' | paste -s -d:")
2471+ CHAN=$(adb shell "system-image-cli -i | sed -n -e 's/channel: \(.*\)/\1/p' | paste -s -d:")
2472+ REV=$(echo $IMAGEVER | cut -d: -f1)
2473+ echo "$IMAGE_OPT" | grep -q "\-\-revision" || IMAGE_OPT="${IMAGE_OPT} --revision $REV"
2474+ echo "$IMAGE_OPT" | grep -q "\-\-channel" || IMAGE_OPT="${IMAGE_OPT} --channel $CHAN"
2475+ adb shell "echo '${IMAGEVER}' > /home/phablet/.ci-version"
2476+ echo $UUID > $RESDIR/.ci-uuid
2477+ adb push $RESDIR/.ci-uuid /home/phablet/
2478+ cat > $RESDIR/.ci-flash-args <<EOF
2479+$IMAGE_OPT
2480+EOF
2481+ adb push $RESDIR/.ci-flash-args /home/phablet/.ci-flash-args
2482+ echo $CUSTOMIZE > $RESDIR/.ci-customizations
2483+ adb push $RESDIR/.ci-customizations /home/phablet/.ci-customizations
2484+}
2485+
2486+log() {
2487+ echo = $(date): $*
2488+}
2489+
2490+while getopts i:s:n:D:P:p:wh opt; do
2491+ case $opt in
2492+ h)
2493+ usage
2494+ exit 0
2495+ ;;
2496+ n)
2497+ NETWORK_FILE=$OPTARG
2498+ ;;
2499+ s)
2500+ export ANDROID_SERIAL=$OPTARG
2501+ ;;
2502+ i)
2503+ IMAGE_TYPE=$OPTARG
2504+ ;;
2505+ w)
2506+ # making this a non-zero length string enables the logic
2507+ CUSTOMIZE=" "
2508+ ;;
2509+ D)
2510+ CUSTOMIZE="$CUSTOMIZE --package-dir $OPTARG"
2511+ ;;
2512+ P)
2513+ CUSTOMIZE="$CUSTOMIZE --ppa $OPTARG"
2514+ ;;
2515+ p)
2516+ CUSTOMIZE="$CUSTOMIZE -p $OPTARG"
2517+ ;;
2518+ esac
2519+done
2520+
2521+if [ -z $ANDROID_SERIAL ] ; then
2522+ # ensure we only have one device attached
2523+ lines=$(adb devices | wc -l)
2524+ if [ $lines -gt 3 ] ; then
2525+ echo "ERROR: More than one device attached, please use -s option"
2526+ echo
2527+ usage
2528+ exit 1
2529+ fi
2530+fi
2531+
2532+if [ ! -f $NETWORK_FILE ] ; then
2533+ echo "ERROR: NETWORK_FILE, $NETWORK_FILE, not found"
2534+ exit 1
2535+fi
2536+
2537+set -x
2538+[ -d $RESDIR ] && rm -rf $RESDIR
2539+mkdir -p $RESDIR
2540+
2541+if [ -n "$SKIP_FLASH" ]; then
2542+ log "SKIPPING phablet-flash AS REQUESTED"
2543+else
2544+ log "FLASHING DEVICE"
2545+ ubuntu-device-flash $IMAGE_OPT
2546+ adb wait-for-device
2547+ sleep 20 #give the system a little time
2548+fi
2549+
2550+if [ -n "$SKIP_CLICK" ]; then
2551+ log "SKIPPING phablet-click-test-setup AS REQUESTED"
2552+else
2553+ log "SETTING UP CLICK PACKAGES"
2554+ phablet-click-test-setup
2555+fi
2556+
2557+log "SETTING UP WIFI"
2558+phablet-network -n $NETWORK_FILE
2559+
2560+if [ "$IMAGE_TYPE" = "touch_sf4p" ]; then
2561+ adb shell rm -f /home/phablet/.display-mir
2562+fi
2563+
2564+phablet-config welcome-wizard --disable
2565+phablet-config edges-intro --disable
2566+
2567+# get our target-based utilities into our PATH
2568+adb push ${BASEDIR}/../utils/target /home/phablet/bin
2569+
2570+image_info
2571+
2572+if [ -n "$CUSTOMIZE" ] ; then
2573+ log "CUSTOMIZING IMAGE"
2574+ phablet-config writable-image $CUSTOMIZE
2575+ # Make sure whoopsie-upload-all can work (bug #1245524)
2576+ adb shell "sed -i '/Waiting for whoopsie/ a\ subprocess.call([\"restart\", \"whoopsie\"])' /usr/share/apport/whoopsie-upload-all"
2577+fi
2578
2579=== added file 'scripts/reboot-and-wait'
2580--- scripts/reboot-and-wait 1970-01-01 00:00:00 +0000
2581+++ scripts/reboot-and-wait 2014-07-03 19:35:55 +0000
2582@@ -0,0 +1,41 @@
2583+#!/usr/bin/python
2584+
2585+import argparse
2586+import logging
2587+import time
2588+
2589+from phabletutils.device import AndroidBridge
2590+
2591+
2592+def _get_arg_parser():
2593+ parser = argparse.ArgumentParser(
2594+ description='Reboot device and waits for networking to become active.')
2595+ parser.add_argument('-s', '--serial', help='Device serial')
2596+ parser.add_argument('-n', '--num-tries', type=int, default=3,
2597+ help='''How many times to retry on failure.
2598+ default=%(default)d''')
2599+ return parser
2600+
2601+
2602+def main(args):
2603+ device = AndroidBridge(args.serial)
2604+ device.wait_for_device()
2605+ for i in range(args.num_tries):
2606+ device.reboot()
2607+ device.wait_for_device()
2608+ time.sleep(5)
2609+ device.wait_for_device()
2610+ try:
2611+ device.wait_for_network()
2612+ return 0
2613+ except:
2614+ pass # try the loop again
2615+ logging.error('device failed to start and activate networking')
2616+ return 1
2617+
2618+
2619+if __name__ == '__main__':
2620+ logging.basicConfig(level=logging.INFO)
2621+ logging.getLogger().name = 'reboot-and-wait'
2622+ args = _get_arg_parser().parse_args()
2623+ exit(main(args))
2624
2625=== added file 'scripts/run-autopilot-tests.sh'
2626--- scripts/run-autopilot-tests.sh 1970-01-01 00:00:00 +0000
2627+++ scripts/run-autopilot-tests.sh 2014-07-03 19:35:55 +0000
2628@@ -0,0 +1,243 @@
2629+#!/bin/sh
2630+
2631+set -e
2632+
2633+BASEDIR=$(dirname $(readlink -f $0))/..
2634+RESDIR=`pwd`/clientlogs
2635+
2636+export PATH=${BASEDIR}/utils/host:${PATH}
2637+export TARGET_PREFIX=adb-shell
2638+
2639+
2640+usage() {
2641+ cat <<EOF
2642+usage: $0 -a APP [-s ANDROID_SERIAL] [-Q] [-o results_dir] [-S]
2643+
2644+Runs a set of autopilot tests on the target
2645+
2646+OPTIONS:
2647+ -h Show this message
2648+ -s Specify the serial of the device to test
2649+ -a The application to test (can be repeated)
2650+ -o Specify the directory to place results in.
2651+ Default: $RESDIR
2652+ -Q "Quick" don't do a reboot of the device before/between testsuites.
2653+ -S Skip the system-settle tests that run before/after each testsuite.
2654+
2655+EOF
2656+}
2657+
2658+log_error() {
2659+ echo ERROR: $* >> ${RESDIR}/runner-errors.txt
2660+}
2661+
2662+setup_test() {
2663+ app=$1
2664+ label=$2
2665+ odir=$3
2666+ {
2667+ pkgs=$(${BASEDIR}/jenkins/testconfig.py packages -a $app)
2668+ if [ "$label" = "setup" ] ; then
2669+ # During MP testing, the device is already provisioned
2670+ # with the necessary -autopilot packages. Installing
2671+ # the package(s) from testconfig.py can actually
2672+ # install the wrong packages from the archive.
2673+ #adb-shell sudo apt-get install -yq --force-yes $pkgs
2674+ # Grab the image info
2675+ adb-shell "system-image-cli --info > /tmp/system-image-cli.log"
2676+ adb pull /tmp/system-image-cli.log $odir
2677+ adb-shell "rm /tmp/system-image-cli.log || true"
2678+ # Grab the full list of installed packages
2679+ adb-shell "dpkg -l > /tmp/dpkg-l.log"
2680+ adb pull /tmp/dpkg-l.log $odir
2681+ adb-shell "rm /tmp/dpkg-l.log || true"
2682+ else
2683+ adb-shell sudo apt-get autoremove --purge -y $pkgs
2684+ # Grab dmesg
2685+ adb-shell "dmesg > /tmp/dmesg.log"
2686+ adb pull /tmp/dpkg-l.log $odir
2687+ adb-shell "rm /tmp/dmesg.log || true"
2688+ fi
2689+ echo $? > ${odir}/setup_${label}.rc
2690+ } 2>&1 | tee ${odir}/setup_${label}.log
2691+}
2692+
2693+system_settle() {
2694+ [ -z $NOSETTLE ] || return 0
2695+
2696+ label=$1
2697+ odir=$2
2698+ rc=0
2699+ timeout=120s
2700+ if [ "$label" = "before" ] ; then
2701+ timeout=300s
2702+ fi
2703+
2704+ settle=${BASEDIR}/tests/systemsettle/systemsettle.sh
2705+ {
2706+ export UTAH_PROBE_DIR=${odir} # needed for log file location
2707+ timeout $timeout $settle -c5 -d6 -p 97.5 -l $label || rc=1
2708+ echo $rc > ${odir}/settle_${label}.rc
2709+ } 2>&1 | tee ${odir}/settle_${label}.log
2710+}
2711+
2712+test_app() {
2713+ app=$1
2714+
2715+ odir=${RESDIR}/${app}
2716+ [ -d $odir ] && rm -rf $odir
2717+ mkdir -p $odir || return 1
2718+
2719+ system_settle before $odir
2720+ phablet-config autopilot --dbus-probe enable || \
2721+ (log_error "'autopilot dbus-probe enable' failed"; return 1)
2722+
2723+ setup_test $app setup $odir
2724+
2725+ NOSHELL=""
2726+ [ "$app" = "unity8" ] && NOSHELL="-n"
2727+ [ "$app" = "ubuntu_experience_tests" ] && NOSHELL="-n"
2728+
2729+ phablet-test-run \
2730+ $NOSHELL \
2731+ -o ${odir} \
2732+ -a /var/crash -a /home/phablet/.cache/upstart \
2733+ -v $app || true
2734+ system_settle after $odir
2735+ setup_test $app teardown $odir
2736+ ${BASEDIR}/scripts/combine_results ${odir}
2737+}
2738+
2739+reboot_wait() {
2740+ if [ -z $QUICK ] ; then
2741+ reboot-and-unlock.sh
2742+ FILES="/var/crash/* /home/phablet/.cache/upstart/*.log*"
2743+ if ! adb shell "rm -rf $FILES" ; then
2744+ log_error "unable to remove crash and log files, retrying"
2745+ adb wait-for-device
2746+ adb shell "rm -rf $FILES"
2747+ fi
2748+ else
2749+ echo "SKIPPING phone reboot..."
2750+ fi
2751+}
2752+
2753+grab_powerd() {
2754+ echo "grabbing powerd cli locks..."
2755+ adb shell powerd-cli active &
2756+ PIDS="$!"
2757+ adb shell powerd-cli display on &
2758+ PIDS="$PIDS $!"
2759+}
2760+
2761+release_powerd() {
2762+ if [ -n "$PIDS" ] ; then
2763+ echo "killing child pids: $PIDS"
2764+ for p in $PIDS ; do
2765+ kill $p || true
2766+ done
2767+ PIDS=""
2768+ fi
2769+}
2770+
2771+dashboard_update() {
2772+ # only try and update the dashboard if we are configured to
2773+ [ -z $DASHBOARD_KEY ] && return 0
2774+ [ -z $DASHBOARD_BUILD ] && return 0
2775+ [ -z $DASHBOARD_IMAGE ] && return 0
2776+ ${BASEDIR}/scripts/dashboard.py $* \
2777+ --image $DASHBOARD_IMAGE \
2778+ --build $DASHBOARD_BUILD || true
2779+}
2780+
2781+dashboard_result_running() {
2782+ dashboard_update result-running --test $1
2783+}
2784+
2785+dashboard_result_syncing() {
2786+ xunit=${RESDIR}/${app}/test_results.xml
2787+ [ -f $xunit ] || return 0
2788+
2789+ # save a utah.yaml version of the results so the dashboard can process
2790+ cat $xunit | ${BASEDIR}/scripts/junit2utah.py > ${RESDIR}/${app}/utah.yaml
2791+ dashboard_update result-syncing --test $1 --results ${RESDIR}/${app}/utah.yaml
2792+}
2793+
2794+main() {
2795+ # print the build date so the jenkins job can use it as the
2796+ # build description
2797+ BUILDID=$(adb shell cat /home/phablet/.ci-version)
2798+ echo "= TOUCH IMAGE VERSION:$BUILDID"
2799+
2800+ [ -d $RESDIR ] || mkdir -p $RESDIR
2801+
2802+ set -x
2803+
2804+ for app in $APPS ; do
2805+ set +x
2806+ echo "========================================================"
2807+ echo "= testing $app"
2808+ echo "========================================================"
2809+ set -x
2810+ dashboard_result_running $app
2811+ reboot_wait
2812+
2813+ grab_powerd
2814+
2815+ if ! test_app $app ; then
2816+ log_error "testing $app, retrying"
2817+ # we sometimes see sporatic adb failures that seem to
2818+ # related to MTP. This adds a retry for the test.
2819+ # test_app only fails on a device error (not a test
2820+ # case error)
2821+ adb wait-for-device
2822+ test_app $app
2823+ fi
2824+ dashboard_result_syncing $app
2825+
2826+ release_powerd
2827+ done
2828+}
2829+
2830+while getopts s:a:o:QSh opt; do
2831+ case $opt in
2832+ h)
2833+ usage
2834+ exit 0
2835+ ;;
2836+ s)
2837+ export ANDROID_SERIAL=$OPTARG
2838+ ;;
2839+ o)
2840+ RESDIR=$OPTARG
2841+ ;;
2842+ a)
2843+ APPS="$APPS $OPTARG"
2844+ ;;
2845+ Q)
2846+ QUICK=1
2847+ ;;
2848+ S)
2849+ NOSETTLE=1
2850+ ;;
2851+ esac
2852+done
2853+
2854+if [ -z $ANDROID_SERIAL ] ; then
2855+ # ensure we only have one device attached
2856+ lines=$(adb devices | wc -l)
2857+ if [ $lines -gt 3 ] ; then
2858+ echo "ERROR: More than one device attached, please use -s option"
2859+ echo
2860+ usage
2861+ exit 1
2862+ fi
2863+fi
2864+if [ -z "$APPS" ] ; then
2865+ echo "ERROR: No app specified"
2866+ usage
2867+ exit 1
2868+fi
2869+
2870+trap release_powerd TERM INT EXIT
2871+main
2872
2873=== added file 'scripts/run-smoke'
2874--- scripts/run-smoke 1970-01-01 00:00:00 +0000
2875+++ scripts/run-smoke 2014-07-03 19:35:55 +0000
2876@@ -0,0 +1,364 @@
2877+#!/usr/bin/python
2878+
2879+import argparse
2880+import datetime
2881+import logging
2882+import os
2883+import shutil
2884+import subprocess
2885+
2886+import yaml
2887+
2888+from phabletutils.environment import detect_device
2889+
2890+import dashboard
2891+import statsd
2892+
2893+log = logging.getLogger()
2894+script_dir = os.path.dirname(__file__)
2895+res_dir = os.path.join(os.getcwd(), 'clientlogs')
2896+
2897+dashboard_api = dashboard.API()
2898+
2899+
2900+class SerialAction(argparse.Action):
2901+ def __call__(self, parser, namespace, values, option_string=None):
2902+ log.info('android serial: %s', values[0])
2903+ os.environ['ANDROID_SERIAL'] = values[0]
2904+
2905+
2906+class DebugAction(argparse.Action):
2907+ def __call__(self, parser, namespace, values, option_string=None):
2908+ log.setLevel(level=logging.DEBUG)
2909+ log.debug('debug logging enabled')
2910+
2911+
2912+def _serial_required():
2913+ required = 'ANDROID_SERIAL' not in os.environ
2914+ if required:
2915+ try:
2916+ out = subprocess.check_output(['adb', 'devices'])
2917+ required = (len(out.decode().split('\n')) != 4)
2918+ except subprocess.CalledProcessError as e:
2919+ logging.debug('error getting adb devices: %s', e)
2920+ return required
2921+
2922+
2923+def _get_parser():
2924+ parser = argparse.ArgumentParser(
2925+ description='Run the complete test-execution-service suite.')
2926+
2927+ parser.add_argument('-s', '--serial', action=SerialAction, nargs=1,
2928+ required=_serial_required(),
2929+ help='Android serial if more than one device present')
2930+ parser.add_argument('--debug', action=DebugAction, nargs=0,
2931+ help='''Enable debug logging.''')
2932+
2933+ parser.add_argument('--install-url',
2934+ help='''Flash with image from previous jenkins job.
2935+ This option will check if the device already has image
2936+ noted from this URL and will skip provisioning. The URL
2937+ should be the path the job like:
2938+ http://q-jenkins:8080/job/<your job>/<build number>''')
2939+ parser.add_argument('-p', '--package', action='append',
2940+ help='Additional packages to install on target.')
2941+ parser.add_argument('-P', '--ppa', action='append',
2942+ help='Additional PPA to configure on target.')
2943+ parser.add_argument('-a', '--app', action='append',
2944+ help='Autopilot tests tor run.')
2945+ parser.add_argument('-t', '--test', action='append',
2946+ help='UTAH tests tor run.')
2947+ parser.add_argument('-n', '--no-provision', action='store_true',
2948+ help='Skip provisioning of the target device')
2949+ parser.add_argument('--hooks-dir',
2950+ help='''A directory containing scripts to be run after
2951+ the target has been provisioned and before testing.''')
2952+ parser.add_argument('--image-opt',
2953+ help='Options to pass to phablet-flash')
2954+ parser.add_argument('--image-type', default='touch',
2955+ help='''Image type being tested. This can be changed
2956+ to 'touch_sf4p' so that SurfaceFlinger will be used
2957+ instead of Mir. default=%(default)s''')
2958+ return parser
2959+
2960+
2961+def _arg_from_env(args, attr, envkey, array):
2962+ val = os.environ.get(envkey, False)
2963+ if val:
2964+ if array:
2965+ setattr(args, attr, val.split())
2966+ else:
2967+ setattr(args, attr, val)
2968+ del os.environ[envkey]
2969+
2970+
2971+def _merge_env(args):
2972+ '''When run in Jenkins everything comes as environment variables.
2973+
2974+ Its makes a much simpler job this way. While command line args are
2975+ much easier for a user.
2976+ '''
2977+ _arg_from_env(args, 'app', 'APPS', True)
2978+ _arg_from_env(args, 'test', 'TESTS', True)
2979+ _arg_from_env(args, 'package', 'PACKAGES', True)
2980+ _arg_from_env(args, 'ppa', 'PPAS', True)
2981+ _arg_from_env(args, 'image_opt', 'IMAGE_OPT', False)
2982+ _arg_from_env(args, 'image_type', 'IMAGE_TYPE', False)
2983+ _arg_from_env(args, 'install_url', 'INSTALL_URL', False)
2984+
2985+
2986+def _assert_args(args):
2987+ if args.install_url:
2988+ # this means you shouldn't specify packages, ppas, or image options
2989+ if args.package or args.ppa or args.image_opt:
2990+ msg = 'ERROR: --install-url can\'t be used with ' \
2991+ '--package, -ppa, or --image_opt'
2992+ print(msg)
2993+ return False
2994+
2995+ # don't bother the install_url check, a user might be copy/pasting and
2996+ # doesn't hurt. Its just good to not encourage it.
2997+ _merge_env(args)
2998+
2999+ script = os.path.join(script_dir, '../jenkins/testconfig.py')
3000+ if args.package and args.package[0] == 'ALL':
3001+ logging.info('Discovering all required dependencies')
3002+ out = subprocess.check_output(
3003+ [script, 'packages', '-i', args.image_type])
3004+ args.package = [x for x in out.decode().split()]
3005+
3006+ if args.app and args.app[0] == 'ALL':
3007+ logging.info('Discovering all autopilot tests')
3008+ out = subprocess.check_output(
3009+ [script, 'apps', '-i', args.image_type])
3010+ args.app = [x for x in out.decode().split()]
3011+
3012+ if args.test and args.test[0].startswith('ALL'):
3013+ logging.info('Discovering all UTAH tests')
3014+ argv = [script, 'utah', '-i', args.image_type]
3015+ if args.test[0] == 'ALL_INCLUDING_AUTOPILOT':
3016+ argv.append('-a')
3017+ out = subprocess.check_output(argv)
3018+ args.test = [x for x in out.decode().split()]
3019+
3020+ logging.debug('ARGS: %r', args)
3021+
3022+ statsd.gauge_it('PACKAGES', args.package)
3023+ statsd.gauge_it('APPS', args.app)
3024+ statsd.gauge_it('TESTS', args.test)
3025+
3026+ return True
3027+
3028+
3029+def _run(args, ignore_error=False):
3030+ try:
3031+ logging.info('Running: %s', ' '.join(args))
3032+ subprocess.check_call(args)
3033+ except subprocess.CalledProcessError:
3034+ if ignore_error:
3035+ logging.error('failed to run %r, continuing', args)
3036+ else:
3037+ exit(1)
3038+
3039+
3040+def _image_info():
3041+ info = subprocess.check_output(['adb', 'shell', 'system-image-cli', '-i'])
3042+ v_ver = u_ver = d_ver = channel = None
3043+ for line in info.split('\n'):
3044+ if not line.strip():
3045+ continue
3046+ key, val = line.split(':', 1)
3047+ if key == 'version version':
3048+ v_ver = val.strip()
3049+ elif key == 'version ubuntu':
3050+ u_ver = val.strip()
3051+ elif key == 'version device':
3052+ d_ver = val.strip()
3053+ elif key == 'channel':
3054+ channel = val.strip()
3055+ ver = '%s:%s:%s' % (v_ver, u_ver, d_ver)
3056+ return ver, channel
3057+
3058+
3059+def _assert_image(args):
3060+ log.info('checking if device has proper image ...')
3061+ os.environ['INSTALL_URL'] = args.install_url
3062+ _run([os.path.join(script_dir, 'assert-image')])
3063+
3064+
3065+def _write_utah(start, end, passed):
3066+ passes = failures = rc = 0
3067+ if passed:
3068+ passes = 1
3069+ else:
3070+ rc = failures = 1
3071+
3072+ delta = '%s' % (end - start)
3073+ start = start.strftime('%Y-%m-%d %H:%M:%S')
3074+ data = {
3075+ 'name': 'install-and-boot',
3076+ 'errors': 0,
3077+ 'failures': failures,
3078+ 'passes': passes,
3079+ 'fetch_errors': 0,
3080+ 'uname': 'n/a',
3081+ 'media-info': 'n/a',
3082+ 'install_type': 'n/a',
3083+ 'arch': 'n/a',
3084+ 'release': 'n/a',
3085+ 'build_number': 'n/a',
3086+ 'runlist': 'n/a',
3087+ 'ran_at': start,
3088+ 'commands': [{
3089+ 'cmd_type': 'testcase_test',
3090+ 'command': 'provision',
3091+ 'returncode': rc,
3092+ 'start_time': start,
3093+ 'time_delta': delta,
3094+ 'stderr': '',
3095+ 'stdout': '',
3096+ 'testcase': 'boot',
3097+ 'testsuite': 'install-and-boot',
3098+ }]
3099+ }
3100+ path = os.path.join(res_dir, 'install-and-boot')
3101+ if not os.path.exists(path):
3102+ os.mkdir(path)
3103+ with open(os.path.join(path, 'utah.yaml'), 'w') as f:
3104+ f.write(yaml.safe_dump(data, default_flow_style=False))
3105+
3106+
3107+def _post_install_hooks(args):
3108+ if not args.hooks_dir:
3109+ return
3110+ log.info('running post install hooks ...')
3111+ if not os.path.isdir(args.hooks_dir):
3112+ log.warn('hooks directory (%s) does not exist ... skipping',
3113+ args.hooks_dir)
3114+ for hook in sorted(os.listdir(args.hooks_dir)):
3115+ s = os.stat(os.path.join(args.hooks_dir, hook))
3116+ if s.st_mode & os.path.stat.S_IXUSR == 0:
3117+ log.warn('skipping hook (%s) - not executable', hook)
3118+ continue
3119+ log.info('executing hook: %s', hook)
3120+ hook = os.path.join(args.hooks_dir, hook)
3121+ subprocess.check_call([hook])
3122+
3123+
3124+def _provision(args):
3125+ log.info('provisioning device ...')
3126+ if args.image_opt:
3127+ log.debug('overriding IMAGE_OPT with: %s', args.image_opt)
3128+ os.environ['IMAGE_OPT'] = args.image_opt
3129+
3130+ cargs = [os.path.join(script_dir, 'provision.sh'), '-i', args.image_type]
3131+
3132+ if args.package:
3133+ for p in args.package:
3134+ cargs.extend(['-p', p])
3135+ if args.ppa:
3136+ for p in args.ppa:
3137+ cargs.extend(['-P', p])
3138+ if not args.ppa and not args.package:
3139+ # All tests require a writeable system the -p and -P args
3140+ # implicitly create a writable system. so we have to ensure here:
3141+ cargs.append('-w')
3142+
3143+ with statsd.time_it('provision'):
3144+ start = datetime.datetime.utcnow()
3145+ passed = False
3146+ try:
3147+ _run(cargs)
3148+ _post_install_hooks(args)
3149+ passed = True
3150+ finally:
3151+ end = datetime.datetime.utcnow()
3152+ _write_utah(start, end, passed)
3153+
3154+
3155+def _test_autopilot(args, build, image):
3156+ if args.app:
3157+ if build:
3158+ os.environ['DASHBOARD_BUILD'] = build
3159+ if image:
3160+ os.environ['DASHBOARD_IMAGE'] = image
3161+ cargs = [os.path.join(script_dir, 'run-autopilot-tests.sh')]
3162+ for app in args.app:
3163+ cargs.extend(['-a', app])
3164+ with statsd.time_it('APPS'):
3165+ _run(cargs)
3166+
3167+
3168+def _sync_results(build, image, test, fname):
3169+ with open(fname) as f:
3170+ d = yaml.safe_load(f)
3171+ dashboard_api.result_syncing(image, build, test, d)
3172+
3173+
3174+def _test_utah(args, build, image):
3175+ if args.test:
3176+ cargs = [os.path.join(script_dir, 'jenkins.sh')]
3177+ with statsd.time_it('TESTS'):
3178+ for test in args.test:
3179+ os.environ['RESDIR'] = os.path.join(res_dir, test)
3180+ dashboard_api.result_running(image, build, test)
3181+ _run(cargs + ['-a', test, '-p', '/tmp/results'],
3182+ ignore_error=True)
3183+ fname = os.path.join(res_dir, test, 'utah.yaml')
3184+ _sync_results(build, image, test, fname)
3185+
3186+
3187+def _image_add(args):
3188+ build_number, channel = _image_info()
3189+ # channel is: ubuntu-touch/trusty-proposed
3190+ # we need "trusty"
3191+ release = channel.split('/')[1].split('-')[0]
3192+ return dashboard_api.image_add(
3193+ build_number, release, args.image_type, detect_device(None), 'ubuntu')
3194+
3195+
3196+def main(args):
3197+ with statsd.time_it('main'):
3198+ if os.path.exists(res_dir):
3199+ logging.info('deleting old result directory: %s', res_dir)
3200+ shutil.rmtree(res_dir)
3201+ os.mkdir(res_dir)
3202+
3203+ job_name = os.environ.get('JOB_NAME', '')
3204+ job_number = os.environ.get('BUILD_NUMBER', '')
3205+ build = dashboard_api.build_add(job_name, job_number)
3206+
3207+ if args.no_provision:
3208+ logging.info('Skipping the provisioning step as requested')
3209+ elif args.install_url:
3210+ _assert_image(args)
3211+ else:
3212+ _provision(args)
3213+
3214+ # TODO - this should be incororated into provision and assert_image
3215+ # so that the status is updated *before* flashing rather than after
3216+ image = _image_add(args)
3217+
3218+ if args.test:
3219+ for x in args.test:
3220+ dashboard_api.result_queue(image, build, x)
3221+ if args.app:
3222+ for x in args.app:
3223+ dashboard_api.result_queue(image, build, x)
3224+
3225+ _test_utah(args, build, image)
3226+ _test_autopilot(args, build, image)
3227+
3228+ return 0
3229+
3230+
3231+if __name__ == '__main__':
3232+ logging.basicConfig(level=logging.INFO)
3233+ log.name = 'run-smoke'
3234+ dashboard.log = logging.getLogger('dashboard')
3235+
3236+ args = _get_parser().parse_args()
3237+ if not _assert_args(args):
3238+ exit(1)
3239+
3240+ exit(main(args))
3241
3242=== added file 'scripts/run-touch-upgrade.sh'
3243--- scripts/run-touch-upgrade.sh 1970-01-01 00:00:00 +0000
3244+++ scripts/run-touch-upgrade.sh 2014-07-03 19:35:55 +0000
3245@@ -0,0 +1,46 @@
3246+#!/bin/bash
3247+
3248+## This is the script jenkins should run to test upgrading a system image
3249+## in the lab.
3250+## Intersting environment variables that must be set:
3251+## ANDROID_SERIAL - specify another android device
3252+## RUNLIST - the path the runlist
3253+## NETWORK_FILE - specify an alternative network file (passed to runlist)
3254+## UPGRADE_FROM - the revision to upgrade from, eg -1 (passed to runlist)
3255+
3256+set -eux
3257+
3258+BASEDIR=$(dirname $(readlink -f $0))
3259+
3260+RESDIR=`pwd`/clientlogs
3261+
3262+UTAH_PHABLET_CMD="${UTAH_PHABLET_CMD-/usr/share/utah/examples/run_utah_phablet.py}"
3263+RUNLIST=${RUNLIST-"`pwd`/smoke-touch-apps/upgrade/master.run"}
3264+ANDROID_SERIAL="${ANDROID_SERIAL-015d1884b20c1c0f}" #doanac's nexus7 at home
3265+NETWORK_FILE="${NETWORK_FILE-/home/ubuntu/magners-wifi}"
3266+
3267+rm -rf clientlogs
3268+mkdir clientlogs
3269+
3270+export ANDROID_SERIAL=$ANDROID_SERIAL
3271+
3272+set +e
3273+sudo NETWORK_FILE=$NETWORK_FILE \
3274+ $UTAH_PHABLET_CMD -s $ANDROID_SERIAL \
3275+ --from-host --skip-install --skip-utah --skip-network -l $RUNLIST \
3276+ --results-dir=$RESDIR
3277+EXITCODE=$?
3278+
3279+UTAHFILE=$RESDIR/utah.yaml
3280+if ! `grep "^errors: [!0]" < $UTAHFILE >/dev/null` ; then
3281+ echo "errors found"
3282+ EXITCODE=1
3283+fi
3284+if ! `grep "^failures: [!0]" < $UTAHFILE >/dev/null` ; then
3285+ echo "failures found"
3286+ EXITCODE=2
3287+fi
3288+echo "Results Summary"
3289+echo "---------------"
3290+egrep '^(errors|failures|passes|fetch_errors):' $UTAHFILE
3291+exit $EXITCODE
3292
3293=== added file 'scripts/statsd.py'
3294--- scripts/statsd.py 1970-01-01 00:00:00 +0000
3295+++ scripts/statsd.py 2014-07-03 19:35:55 +0000
3296@@ -0,0 +1,35 @@
3297+#!/usr/bin/python
3298+
3299+import os
3300+import contextlib
3301+import time
3302+
3303+
3304+_namespace = os.environ.get('STATSD_KEY')
3305+if _namespace:
3306+ from txstatsd.client import UdpStatsDClient
3307+ from txstatsd.metrics.timermetric import TimerMetric
3308+ from txstatsd.metrics.gaugemetric import GaugeMetric
3309+ _host = os.environ.get('SERVER', 'snakefruit.canonical.com')
3310+ _port = int(os.environ.get('PORT', '10041'))
3311+ _client = UdpStatsDClient(_host, _port)
3312+ _client.connect()
3313+
3314+
3315+@contextlib.contextmanager
3316+def time_it(key):
3317+ start = time.time()
3318+ try:
3319+ yield
3320+ finally:
3321+ if _namespace:
3322+ m = TimerMetric(_client, _namespace + '.' + key)
3323+ m.mark(time.time() - start)
3324+
3325+
3326+def gauge_it(key, array):
3327+ val = 0
3328+ if array:
3329+ val = len(array)
3330+ if _namespace:
3331+ GaugeMetric(_client, _namespace + '.' + key).mark(val)
3332
3333=== added file 'scripts/test_um_provision.py'
3334--- scripts/test_um_provision.py 1970-01-01 00:00:00 +0000
3335+++ scripts/test_um_provision.py 2014-07-03 19:35:55 +0000
3336@@ -0,0 +1,101 @@
3337+#!/usr/bin/env python
3338+
3339+from mock import (MagicMock, patch)
3340+import os
3341+from testtools import TestCase
3342+from testscenarios import TestWithScenarios
3343+
3344+from um_provision import convert_hook_to_ppa, provision
3345+
3346+class UpstreamMergerProvision(TestCase):
3347+ def setUp(self):
3348+ super(UpstreamMergerProvision, self).setUp()
3349+
3350+class ConvertHookToPpa(TestWithScenarios, UpstreamMergerProvision):
3351+ scenarios = [
3352+ ('single_ppa_hook',
3353+ {
3354+ 'hooks': 'D09add_ppa~team-name~ppa-name',
3355+ 'ppa': ['ppa:team-name/ppa-name']}
3356+ ),
3357+ ('single_hook_not_a_ppa',
3358+ {
3359+ 'hooks': 'D10not_a_ppa_hook',
3360+ 'ppa': []}
3361+ ),
3362+ ('multiple_ppa_hooks',
3363+ {
3364+ 'hooks': 'D09add_ppa~team-name~ppa-name D09add_ppa~team2~ppa2 '
3365+ 'D10not_a_ppa_hook',
3366+ 'ppa': ['ppa:team-name/ppa-name', 'ppa:team2/ppa2']}
3367+ ),
3368+ ('no_hooks',
3369+ {
3370+ 'hooks': '',
3371+ 'ppa': []}
3372+ )
3373+ ]
3374+
3375+ def setUp(self):
3376+ super(ConvertHookToPpa, self).setUp()
3377+
3378+ def test_convert_hook_to_ppa(self):
3379+ self.assertEqual(self.ppa, convert_hook_to_ppa(self.hooks))
3380+
3381+
3382+class Provision(TestWithScenarios, UpstreamMergerProvision):
3383+ scenarios = [
3384+ ('specify_serial',
3385+ {'args': {'serial': 'abcdef1234567890'},
3386+ 'cmd': ['provision.sh', '-s', 'abcdef1234567890']}
3387+ ),
3388+ ('specify_single_archive_directory',
3389+ {'args': {'archive_directories': ['archive-1']},
3390+ 'cmd': ['provision.sh', '-D', 'archive-1']}
3391+ ),
3392+ ('specify_multiple_archive_directories',
3393+ {'args': {'archive_directories': ['archive-1', 'archive-2']},
3394+ 'cmd': ['provision.sh', '-D', 'archive-1', '-D', 'archive-2']}
3395+ ),
3396+ ('specify_single_ppa',
3397+ {'args': {'ppa_list': ['ppa:team-1/name-1']},
3398+ 'cmd': ['provision.sh', '-P', 'ppa:team-1/name-1']}
3399+ ),
3400+ ('specify_multiple_ppas',
3401+ {'args': {'ppa_list': ['ppa:team-1/name-1', 'ppa:team-2/name-2']},
3402+ 'cmd': ['provision.sh', '-P', 'ppa:team-1/name-1',
3403+ '-P', 'ppa:team-2/name-2']}
3404+ ),
3405+ ('specify_single_deb_package',
3406+ {'args': {'deb_packages': ['package-1']},
3407+ 'cmd': ['provision.sh', '-p', 'package-1']}
3408+ ),
3409+ ('specify_multiple_deb_package',
3410+ {'args': {'deb_packages': ['package-1', 'package-2']},
3411+ 'cmd': ['provision.sh', '-p', 'package-1', '-p', 'package-2']}
3412+ ),
3413+ ('specify_ppa_and_package',
3414+ {'args': {'ppa_list': ['ppa:team-1/name-1'],
3415+ 'deb_packages': ['package-1']},
3416+ 'cmd': ['provision.sh', '-P', 'ppa:team-1/name-1',
3417+ '-p', 'package-1']}
3418+ ),
3419+ ('specify_archive_dir_and_package',
3420+ {'args': {'archive_directories': ['archive-1'],
3421+ 'deb_packages': ['package-1']},
3422+ 'cmd': ['provision.sh', '-D', 'archive-1', '-p', 'package-1']}
3423+ ),
3424+ ('specify_archive_dir_ppa_and_package',
3425+ {'args': {'archive_directories': ['archive_dir'],
3426+ 'ppa_list': ['ppa:team-1/name-1'],
3427+ 'deb_packages': ['package-1']},
3428+ 'cmd': ['provision.sh', '-D', 'archive_dir',
3429+ '-P', 'ppa:team-1/name-1', '-p', 'package-1']}
3430+ ),
3431+ ]
3432+
3433+ @patch('subprocess.check_call')
3434+ def test_provision(self, call_mock):
3435+ provision(**self.args)
3436+ call_mock.assert_called_with(self.cmd)
3437+
3438
3439=== added directory 'selftests'
3440=== added file 'selftests/test_junit2utah.py'
3441--- selftests/test_junit2utah.py 1970-01-01 00:00:00 +0000
3442+++ selftests/test_junit2utah.py 2014-07-03 19:35:55 +0000
3443@@ -0,0 +1,49 @@
3444+import StringIO
3445+import os
3446+import sys
3447+import unittest
3448+
3449+here = os.path.dirname(__file__)
3450+sys.path.append(os.path.join(here, '../scripts'))
3451+
3452+import junit2utah
3453+
3454+RESULT = '''
3455+<testsuite errors="0" failures="1" name="" tests="4" time="0.001">
3456+<testcase classname="classname" name="testFails" time="0.000">
3457+<failure type="testtools.testresult.real._StringException">
3458+testtools.testresult.real._StringException: Traceback (most recent call last):
3459+ File "junit2utah.py", line 14, in testFails
3460+ self.assertFalse(True)
3461+ File "/usr/lib/python2.7/unittest/case.py", line 418, in assertFalse
3462+ raise self.failureException(msg)
3463+AssertionError: True is not false
3464+</failure>
3465+</testcase>
3466+<testcase classname="classname" name="testPasses" time="0.100"/>
3467+<testcase classname="classname" name="testPasses2" time="0.200"/>
3468+<testcase classname="classname" name="testSkip" time="0.000">
3469+<skip>ensure skip works</skip>
3470+</testcase>
3471+</testsuite>
3472+'''
3473+
3474+
3475+class TestJunit2Utah(unittest.TestCase):
3476+ def testFull(self):
3477+ stream = StringIO.StringIO(RESULT)
3478+ results = junit2utah._get_results(stream)
3479+ self.assertEquals(3, results['passes'])
3480+ self.assertEquals(1, results['failures'])
3481+ self.assertEquals(0, results['errors'])
3482+
3483+ tcs = results['commands']
3484+ self.assertEqual('classname', tcs[0]['testsuite'])
3485+ self.assertEqual('testFails', tcs[0]['testcase'])
3486+ self.assertIn('AssertionError', tcs[0]['stderr'])
3487+
3488+ self.assertEqual('0.200', tcs[2]['time_delta'])
3489+
3490+ self.assertEqual('classname', tcs[3]['testsuite'])
3491+ self.assertEqual('testSkip', tcs[3]['testcase'])
3492+ self.assertEqual('ensure skip works', tcs[3]['stdout'])
3493
3494=== added file 'selftests/test_reboot_and_wait.py'
3495--- selftests/test_reboot_and_wait.py 1970-01-01 00:00:00 +0000
3496+++ selftests/test_reboot_and_wait.py 2014-07-03 19:35:55 +0000
3497@@ -0,0 +1,66 @@
3498+# Ubuntu Test Cases for Touch
3499+# Copyright 2013 Canonical Ltd.
3500+
3501+# This program is free software: you can redistribute it and/or modify it
3502+# under the terms of the GNU General Public License version 3, as published
3503+# by the Free Software Foundation.
3504+
3505+# This program is distributed in the hope that it will be useful, but
3506+# WITHOUT ANY WARRANTY; without even the implied warranties of
3507+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
3508+# PURPOSE. See the GNU General Public License for more details.
3509+
3510+# You should have received a copy of the GNU General Public License along
3511+# with this program. If not, see <http://www.gnu.org/licenses/>.
3512+
3513+from __future__ import print_function
3514+
3515+import imp
3516+import mock
3517+import os
3518+import unittest
3519+
3520+
3521+class TestRebootAndWait(unittest.TestCase):
3522+
3523+ """Simple set of tests to make sure the reboot-and-wait command works."""
3524+
3525+ def setUp(self):
3526+ # do some trickery to load this as module
3527+ module = 'reboot-and-wait'
3528+ fname = os.path.join(os.path.dirname(__file__), '../scripts', module)
3529+ m = imp.load_source(module, fname)
3530+ self._main = m.main
3531+ self._get_parser = m._get_arg_parser
3532+
3533+ @mock.patch('phabletutils.device.AndroidBridge.reboot')
3534+ def testRebootFail(self, reboot):
3535+ reboot.side_effect = RuntimeError('foo')
3536+ args = self._get_parser().parse_args([])
3537+ with self.assertRaisesRegexp(RuntimeError, 'foo'):
3538+ self._main(args)
3539+
3540+ @mock.patch('phabletutils.device.AndroidBridge.reboot')
3541+ @mock.patch('phabletutils.device.AndroidBridge.wait_for_device')
3542+ def testWaitForDeviceFail(self, wait_for, reboot):
3543+ wait_for.side_effect = RuntimeError('foo')
3544+ args = self._get_parser().parse_args([])
3545+ with self.assertRaisesRegexp(RuntimeError, 'foo'):
3546+ self._main(args)
3547+ reboot.assert_called_once_with()
3548+
3549+ @mock.patch('phabletutils.device.AndroidBridge.reboot')
3550+ @mock.patch('phabletutils.device.AndroidBridge.wait_for_device')
3551+ @mock.patch('phabletutils.device.AndroidBridge.wait_for_network')
3552+ def testRetries(self, wait_for_net, wait_for_dev, reboot):
3553+ args = self._get_parser().parse_args([])
3554+ wait_for_net.side_effect = RuntimeError('foo')
3555+ self.assertEquals(1, self._main(args))
3556+ self.assertEquals(args.num_tries, reboot.call_count)
3557+
3558+ # now make sure it can recover after a single network failure
3559+ reboot.reset_mock()
3560+ wait_for_net.reset_mock()
3561+ wait_for_net.side_effect = [RuntimeError('foo'), None]
3562+ self.assertEquals(0, self._main(args))
3563+ self.assertEquals(2, reboot.call_count)
3564
3565=== added file 'selftests/test_run_smoke.py'
3566--- selftests/test_run_smoke.py 1970-01-01 00:00:00 +0000
3567+++ selftests/test_run_smoke.py 2014-07-03 19:35:55 +0000
3568@@ -0,0 +1,143 @@
3569+# Ubuntu Test Cases for Touch
3570+# Copyright 2013 Canonical Ltd.
3571+
3572+# This program is free software: you can redistribute it and/or modify it
3573+# under the terms of the GNU General Public License version 3, as published
3574+# by the Free Software Foundation.
3575+
3576+# This program is distributed in the hope that it will be useful, but
3577+# WITHOUT ANY WARRANTY; without even the implied warranties of
3578+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
3579+# PURPOSE. See the GNU General Public License for more details.
3580+
3581+# You should have received a copy of the GNU General Public License along
3582+# with this program. If not, see <http://www.gnu.org/licenses/>.
3583+
3584+import imp
3585+import mock
3586+import os
3587+import subprocess
3588+import sys
3589+import unittest
3590+
3591+
3592+class TestRunSmoke(unittest.TestCase):
3593+
3594+ """Simple set of tests to make sure the smoke-run command works."""
3595+
3596+ def setUp(self):
3597+ # load the module
3598+ module = 'run-smoke'
3599+ path = os.path.join(os.path.dirname(__file__), '../scripts')
3600+ sys.path.append(path)
3601+ fname = os.path.join(path, module)
3602+ self.run_smoke = imp.load_source(module, fname)
3603+ if 'ANDROID_SERIAL' in os.environ:
3604+ del os.environ['ANDROID_SERIAL']
3605+
3606+ @mock.patch.dict('os.environ')
3607+ @mock.patch('subprocess.check_output')
3608+ def testSerialRequired(self, check_output):
3609+ '''Ensure android serial is required when appropriate'''
3610+ check_output.return_value = '1'.encode()
3611+ self.assertTrue(self.run_smoke._serial_required())
3612+
3613+ check_output.return_value = '1\n2\n3\n4'.encode()
3614+ self.assertFalse(self.run_smoke._serial_required())
3615+
3616+ check_output.return_value = '1\n2\n3\n4\n5'.encode()
3617+ self.assertTrue(self.run_smoke._serial_required())
3618+
3619+ # make sure serial isn't required if specified in env
3620+ os.environ['ANDROID_SERIAL'] = 'foo'
3621+ check_output.return_value = '1\n2\n3\n4\n5'.encode()
3622+ self.assertFalse(self.run_smoke._serial_required())
3623+
3624+ @mock.patch('statsd.gauge_it')
3625+ def testAssertArgs(self, gauge):
3626+ '''Ensure install-url is used properly'''
3627+ patterns = [
3628+ (['--install-url', 'x', '-p', 'x'], False),
3629+ (['--install-url', 'x', '-P', 'x'], False),
3630+ (['--install-url', 'x', '--image-opt', 'x'], False),
3631+ (['-p', 'x', '-P', 'x', '--image-opt', 'x'], True),
3632+ ]
3633+ for pat, val in patterns:
3634+ args = self.run_smoke._get_parser().parse_args(['-s', 'foo'] + pat)
3635+ self.assertEqual(val, self.run_smoke._assert_args(args))
3636+
3637+ # ensure the -p ALL pulls in all packages
3638+ gauge.reset_mock()
3639+ args = self.run_smoke._get_parser().parse_args(['-p', 'ALL'])
3640+ self.assertTrue(self.run_smoke._assert_args(args))
3641+ self.assertTrue(len(args.package) > 1)
3642+ args = gauge.call_args_list[0][0]
3643+ self.assertEqual('PACKAGES', args[0])
3644+ self.assertLess(1, len(args[1]))
3645+
3646+ args = gauge.call_args_list[1][0]
3647+ self.assertEqual('APPS', args[0])
3648+ self.assertIsNone(args[1])
3649+ args = gauge.call_args_list[2][0]
3650+ self.assertEqual('TESTS', args[0])
3651+ self.assertIsNone(args[1])
3652+
3653+ # don't bother checking gauge calls for the remaining "ALL" tests
3654+
3655+ # ensure the -a ALL pulls in all APPS
3656+ args = self.run_smoke._get_parser().parse_args(['-a', 'ALL'])
3657+ self.assertTrue(self.run_smoke._assert_args(args))
3658+ self.assertTrue(len(args.app) > 1)
3659+
3660+ # ensure the -t ALL pulls in all TESTS
3661+ args = self.run_smoke._get_parser().parse_args(['-t', 'ALL'])
3662+ self.assertTrue(self.run_smoke._assert_args(args))
3663+ self.assertTrue(len(args.test) > 1)
3664+
3665+ def testAssertArgsEnv(self):
3666+ '''Ensure we pull in environment variables that jenkins uses.'''
3667+ with mock.patch.dict('os.environ'):
3668+ os.environ['APPS'] = 'apps'
3669+ os.environ['TESTS'] = 'tests'
3670+ os.environ['PACKAGES'] = 'packages'
3671+ os.environ['PPAS'] = 'ppas'
3672+ os.environ['IMAGE_TYPE'] = 'type'
3673+ os.environ['INSTALL_URL'] = 'url'
3674+ os.environ['IMAGE_OPT'] = 'opts opts'
3675+ os.environ['ANDROID_SERIAL'] = 'foo'
3676+
3677+ args = self.run_smoke._get_parser().parse_args([])
3678+ self.assertTrue(self.run_smoke._assert_args(args))
3679+
3680+ self.assertEqual(args.app, ['apps'])
3681+ self.assertEqual(args.test, ['tests'])
3682+ self.assertEqual(args.package, ['packages'])
3683+ self.assertEqual(args.ppa, ['ppas'])
3684+ self.assertEqual(args.image_type, 'type')
3685+ self.assertEqual(args.install_url, 'url')
3686+ self.assertEqual(args.image_opt, 'opts opts')
3687+
3688+ def testProvision(self):
3689+ orig = os.environ.get('IMAGE_OPT', '')
3690+ with mock.patch.object(self.run_smoke, '_run') as run:
3691+ args = self.run_smoke._get_parser().parse_args(
3692+ ['-s', 'foo', '--image-opt', 'FOOBAR', '-p', '1', '-p', '2'])
3693+ self.run_smoke._provision(args)
3694+ self.assertTrue(run.called)
3695+ val = os.environ.get('IMAGE_OPT')
3696+ os.environ['IMAGE_OPT'] = orig
3697+ self.assertEqual('FOOBAR', val)
3698+
3699+ def testUtahTests(self):
3700+ args = self.run_smoke._get_parser().parse_args(
3701+ ['-s', 'foo', '-t', 'a', '-t', 'b'])
3702+ with mock.patch.object(self.run_smoke, '_run') as run:
3703+ with mock.patch.dict('os.environ'):
3704+ run.side_effects = [subprocess.CalledProcessError, None]
3705+ with mock.patch.object(self.run_smoke, '_sync_results') as sr:
3706+ self.run_smoke._test_utah(args, None, None)
3707+ p = os.path.join(self.run_smoke.res_dir, 'b')
3708+ # ensuring b ran means, that we handled the failure of test
3709+ # 'a' and that the environment is setup correctly
3710+ self.assertEqual(os.environ['RESDIR'], p)
3711+ self.assertTrue(sr.called)
3712
3713=== added file 'selftests/test_statsd.py'
3714--- selftests/test_statsd.py 1970-01-01 00:00:00 +0000
3715+++ selftests/test_statsd.py 2014-07-03 19:35:55 +0000
3716@@ -0,0 +1,64 @@
3717+# Ubuntu Test Cases for Touch
3718+# Copyright 2013 Canonical Ltd.
3719+
3720+# This program is free software: you can redistribute it and/or modify it
3721+# under the terms of the GNU General Public License version 3, as published
3722+# by the Free Software Foundation.
3723+
3724+# This program is distributed in the hope that it will be useful, but
3725+# WITHOUT ANY WARRANTY; without even the implied warranties of
3726+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
3727+# PURPOSE. See the GNU General Public License for more details.
3728+
3729+# You should have received a copy of the GNU General Public License along
3730+# with this program. If not, see <http://www.gnu.org/licenses/>.
3731+
3732+import os
3733+import mock
3734+import unittest
3735+import sys
3736+import time
3737+
3738+path = os.path.join(os.path.dirname(__file__), '../scripts')
3739+sys.path.append(path)
3740+
3741+import statsd
3742+
3743+
3744+class TestStatsd(unittest.TestCase):
3745+
3746+ def setUp(self):
3747+ self.origkey = os.environ.get('STATSD_KEY')
3748+ os.environ['STATSD_KEY'] = 'prefix'
3749+ reload(statsd)
3750+
3751+ def tearDown(self):
3752+ if not self.origkey:
3753+ del os.environ['STATSD_KEY']
3754+ else:
3755+ os.environ['STATSD_KEY'] = self.origkey
3756+ reload(statsd)
3757+
3758+ """Simple set of tests to make sure the statsd calls work."""
3759+
3760+ @mock.patch('txstatsd.metrics.metric.Metric.write')
3761+ def testGaugeIt(self, _statsd):
3762+ statsd.gauge_it('foo', None)
3763+ _statsd.assert_called_with('prefix.foo:0|g')
3764+ _statsd.reset_mock()
3765+
3766+ statsd.gauge_it('foo', [])
3767+ _statsd.assert_called_with('prefix.foo:0|g')
3768+ _statsd.reset_mock()
3769+
3770+ statsd.gauge_it('foo', [1, 2, 3])
3771+ _statsd.assert_called_with('prefix.foo:3|g')
3772+ _statsd.reset_mock()
3773+
3774+ @mock.patch('txstatsd.metrics.metric.Metric.write')
3775+ def testTimeIt(self, _statsd):
3776+ with statsd.time_it('foo'):
3777+ time.sleep(0.1)
3778+ # should have a timing of about 100ms
3779+ self.assertRegexpMatches(
3780+ _statsd.call_args[0][0], 'prefix.foo:100.[\d+]|ms')
3781
3782=== added directory 'tests'
3783=== added directory 'tests/click_image_tests'
3784=== added directory 'tests/click_image_tests/check_preinstalled_list'
3785=== added file 'tests/click_image_tests/check_preinstalled_list/check_preinstalled_list.py'
3786--- tests/click_image_tests/check_preinstalled_list/check_preinstalled_list.py 1970-01-01 00:00:00 +0000
3787+++ tests/click_image_tests/check_preinstalled_list/check_preinstalled_list.py 2014-07-03 19:35:55 +0000
3788@@ -0,0 +1,61 @@
3789+#!/usr/bin/python3
3790+# Copyright (C) 2013 Canonical Ltd.
3791+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
3792+
3793+# This program is free software: you can redistribute it and/or modify
3794+# it under the terms of the GNU General Public License as published by
3795+# the Free Software Foundation; version 3 of the License.
3796+#
3797+# This program is distributed in the hope that it will be useful,
3798+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3799+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3800+# GNU General Public License for more details.
3801+#
3802+# You should have received a copy of the GNU General Public License
3803+# along with this program. If not, see <http://www.gnu.org/licenses/>.
3804+
3805+import subprocess
3806+import unittest
3807+import urllib.request
3808+
3809+
3810+click_list_url = \
3811+ 'http://people.canonical.com/~ubuntu-archive/click_packages/click_list'
3812+
3813+
3814+def get_install_list():
3815+ request = urllib.request.urlopen(click_list_url).read().decode('utf-8')
3816+ click_files = [x for x in request.split('\n') if x]
3817+ click_apps = {}
3818+ for entry in click_files:
3819+ entry_parts = entry.split('_')
3820+ click_apps[entry_parts[0]] = entry_parts[1]
3821+ return click_apps
3822+
3823+
3824+def get_image_list():
3825+ click_list = subprocess.check_output(
3826+ ['adb', 'shell', 'sudo', '-u', 'phablet',
3827+ 'bash', '-ic', 'click list']).decode('utf-8').split('\n')
3828+ click_entries = [x for x in click_list if x]
3829+ click_apps = {}
3830+ for entry in click_entries:
3831+ entry_parts = entry.split('\t')
3832+ click_apps[entry_parts[0]] = entry_parts[1].strip()
3833+ return click_apps
3834+
3835+
3836+class ClickPreinstalled(unittest.TestCase):
3837+
3838+ def setUp(self):
3839+ self.image_list = get_image_list()
3840+ self.install_list = get_install_list()
3841+ print('Search for %s on image' % self.install_list.keys())
3842+
3843+ def testPreinstalled(self):
3844+ for entry in self.install_list:
3845+ self.assertIn(entry, self.image_list.keys())
3846+
3847+
3848+if __name__ == '__main__':
3849+ unittest.main()
3850
3851=== added file 'tests/click_image_tests/check_preinstalled_list/tc_control'
3852--- tests/click_image_tests/check_preinstalled_list/tc_control 1970-01-01 00:00:00 +0000
3853+++ tests/click_image_tests/check_preinstalled_list/tc_control 2014-07-03 19:35:55 +0000
3854@@ -0,0 +1,10 @@
3855+description: check that all the expected click packages are installed
3856+dependencies: a touch read-only image
3857+action: |
3858+ 1. download the list of click packages
3859+ 2. list the current click packages installed on the device
3860+expected_results: |
3861+ 1. the current list of click packages from the archive should match
3862+type: userland
3863+timeout: 120
3864+command: ./check_preinstalled_list.py
3865
3866=== added file 'tests/click_image_tests/master.run'
3867--- tests/click_image_tests/master.run 1970-01-01 00:00:00 +0000
3868+++ tests/click_image_tests/master.run 2014-07-03 19:35:55 +0000
3869@@ -0,0 +1,5 @@
3870+---
3871+testsuites:
3872+ - name: click_image_tests
3873+ fetch_method: dev
3874+ fetch_location: ./
3875
3876=== added file 'tests/click_image_tests/tslist.run'
3877--- tests/click_image_tests/tslist.run 1970-01-01 00:00:00 +0000
3878+++ tests/click_image_tests/tslist.run 2014-07-03 19:35:55 +0000
3879@@ -0,0 +1,1 @@
3880+- test: check_preinstalled_list
3881
3882=== added directory 'tests/customizations'
3883=== added file 'tests/customizations/master.run'
3884--- tests/customizations/master.run 1970-01-01 00:00:00 +0000
3885+++ tests/customizations/master.run 2014-07-03 19:35:55 +0000
3886@@ -0,0 +1,5 @@
3887+---
3888+testsuites:
3889+ - name: customizations
3890+ fetch_method: dev
3891+ fetch_location: ./
3892
3893=== added file 'tests/customizations/setup.sh'
3894--- tests/customizations/setup.sh 1970-01-01 00:00:00 +0000
3895+++ tests/customizations/setup.sh 2014-07-03 19:35:55 +0000
3896@@ -0,0 +1,10 @@
3897+#!/bin/sh
3898+
3899+set -ex
3900+
3901+BZR_HOME=/dev/null bzr export customization_tests lp:savilerow/tests
3902+
3903+# copy the autopilot scripts over if needed (ie we are testing from HOST)
3904+[ -z $ANDROID_SERIAL ] || adb push ./customization_tests /home/phablet/autopilot/customization_tests
3905+
3906+prepare-autopilot-test.sh
3907
3908=== added file 'tests/customizations/ts_control'
3909--- tests/customizations/ts_control 1970-01-01 00:00:00 +0000
3910+++ tests/customizations/ts_control 2014-07-03 19:35:55 +0000
3911@@ -0,0 +1,1 @@
3912+ts_setup: ./setup.sh
3913
3914=== added file 'tests/customizations/tslist.auto'
3915--- tests/customizations/tslist.auto 1970-01-01 00:00:00 +0000
3916+++ tests/customizations/tslist.auto 2014-07-03 19:35:55 +0000
3917@@ -0,0 +1,4 @@
3918+-
3919+ discovery_cmd: autopilot-list customization_tests
3920+ test_cmd: autopilot-run {}
3921+
3922
3923=== added directory 'tests/default'
3924=== added directory 'tests/default/ifconfig'
3925=== added file 'tests/default/ifconfig/tc_control'
3926--- tests/default/ifconfig/tc_control 1970-01-01 00:00:00 +0000
3927+++ tests/default/ifconfig/tc_control 2014-07-03 19:35:55 +0000
3928@@ -0,0 +1,13 @@
3929+description: make sure wlan0 exists
3930+dependencies: none
3931+action: |
3932+ 1. ifconfig
3933+ 2. Make sure there is a wlan0
3934+expected_results: |
3935+ 1. there is a wlan0 on the system
3936+type: userland
3937+timeout: 60
3938+command: $TARGET_PREFIX ifconfig | grep wlan0
3939+#build_cmd:
3940+#tc_setup:
3941+#tc_cleanup:
3942
3943=== added directory 'tests/default/install'
3944=== added file 'tests/default/install/tc_control'
3945--- tests/default/install/tc_control 1970-01-01 00:00:00 +0000
3946+++ tests/default/install/tc_control 2014-07-03 19:35:55 +0000
3947@@ -0,0 +1,12 @@
3948+description: apt-get install works
3949+dependencies: openssh-server
3950+action: |
3951+ 1. apt-get install -y curl
3952+expected_results: |
3953+ 1. apt-get install can install a package and ends with exit code 0
3954+type: userland
3955+timeout: 120
3956+command: $TARGET_PREFIX apt-get install -y curl
3957+#build_cmd:
3958+#tc_setup:
3959+tc_cleanup: $TARGET_PREFIX apt-get remove -y curl
3960
3961=== added file 'tests/default/master.run'
3962--- tests/default/master.run 1970-01-01 00:00:00 +0000
3963+++ tests/default/master.run 2014-07-03 19:35:55 +0000
3964@@ -0,0 +1,5 @@
3965+---
3966+testsuites:
3967+ - name: smoke_touch
3968+ fetch_method: dev
3969+ fetch_location: ./
3970
3971=== added directory 'tests/default/netstat'
3972=== added file 'tests/default/netstat/tc_control'
3973--- tests/default/netstat/tc_control 1970-01-01 00:00:00 +0000
3974+++ tests/default/netstat/tc_control 2014-07-03 19:35:55 +0000
3975@@ -0,0 +1,12 @@
3976+description: netstat -l lists all the services
3977+dependencies: none
3978+action: |
3979+ 1. netstat -l
3980+expected_results: |
3981+ 1. netstat shows output and exists cleanly
3982+type: userland
3983+timeout: 60
3984+command: $TARGET_PREFIX netstat -l
3985+#build_cmd:
3986+#tc_setup:
3987+#tc_cleanup:
3988
3989=== added directory 'tests/default/ping'
3990=== added file 'tests/default/ping/pingtest.sh'
3991--- tests/default/ping/pingtest.sh 1970-01-01 00:00:00 +0000
3992+++ tests/default/ping/pingtest.sh 2014-07-03 19:35:55 +0000
3993@@ -0,0 +1,12 @@
3994+#!/bin/sh
3995+
3996+set -e
3997+
3998+if [ -z $TARGET_PREFIX ] ; then
3999+ echo "RUNNING ON TARGET"
4000+ ping -c 5 $(ip route show | head -n1 |cut -d" " -f3)
4001+else
4002+ echo "RUNNING FROM HOST"
4003+ ${TARGET_PREFIX} ping -c 5 '$(ip route show | head -n1 |cut -d" " -f3)'
4004+fi
4005+
4006
4007=== added file 'tests/default/ping/tc_control'
4008--- tests/default/ping/tc_control 1970-01-01 00:00:00 +0000
4009+++ tests/default/ping/tc_control 2014-07-03 19:35:55 +0000
4010@@ -0,0 +1,12 @@
4011+description: ping -c 5 $gateway
4012+dependencies: None
4013+action: |
4014+ 1. ping -c 5 $gateway
4015+expected_results: |
4016+ 1. ping ends without an error
4017+type: userland
4018+timeout: 60
4019+command: ./pingtest.sh
4020+#build_cmd:
4021+#tc_setup:
4022+#tc_cleanup:
4023
4024=== added directory 'tests/default/pwd'
4025=== added file 'tests/default/pwd/tc_control'
4026--- tests/default/pwd/tc_control 1970-01-01 00:00:00 +0000
4027+++ tests/default/pwd/tc_control 2014-07-03 19:35:55 +0000
4028@@ -0,0 +1,9 @@
4029+description: pwd works
4030+dependencies: none
4031+action: |
4032+ 1. cd to /tmp and verify which is the current directory
4033+expected_results: |
4034+ 1. Current directory is /tmp
4035+type: userland
4036+timeout: 60
4037+command: ./test.sh
4038
4039=== added file 'tests/default/pwd/test.sh'
4040--- tests/default/pwd/test.sh 1970-01-01 00:00:00 +0000
4041+++ tests/default/pwd/test.sh 2014-07-03 19:35:55 +0000
4042@@ -0,0 +1,18 @@
4043+#!/bin/sh
4044+
4045+set -e
4046+
4047+if [ -n "$TARGET_PREFIX" ] ; then
4048+ echo "RUNNING FROM HOST"
4049+else
4050+ echo "RUNNING ON TARGET"
4051+ TARGET_PREFIX="/bin/sh -c"
4052+fi
4053+
4054+#when it comes from adb we get a \r\n that should be removed
4055+dir=$($TARGET_PREFIX "cd /tmp; pwd" | head -n1 | tr -d '\r\n')
4056+if [ $dir != "/tmp" ] ; then
4057+ echo "failed to change directory"
4058+ exit 1
4059+fi
4060+exit 0
4061
4062=== added directory 'tests/default/route'
4063=== added file 'tests/default/route/tc_control'
4064--- tests/default/route/tc_control 1970-01-01 00:00:00 +0000
4065+++ tests/default/route/tc_control 2014-07-03 19:35:55 +0000
4066@@ -0,0 +1,12 @@
4067+description: route shows wlan0 routes
4068+dependencies: none
4069+action: |
4070+ 1. route -n | grep wlan0
4071+expected_results: |
4072+ 1. route shows wlan0 routes
4073+type: userland
4074+timeout: 60
4075+command: $TARGET_PREFIX route -n | grep wlan0
4076+#build_cmd:
4077+#tc_setup:
4078+#tc_cleanup:
4079
4080=== added directory 'tests/default/systemsettle'
4081=== added file 'tests/default/systemsettle/systemsettle.sh'
4082--- tests/default/systemsettle/systemsettle.sh 1970-01-01 00:00:00 +0000
4083+++ tests/default/systemsettle/systemsettle.sh 2014-07-03 19:35:55 +0000
4084@@ -0,0 +1,127 @@
4085+#!/bin/bash
4086+
4087+# Configuration variables:
4088+# TARGET_PREFIX - Allows this to be run from the host, by providings something
4089+# like TARGET_PREFIX="adb shell"
4090+# UTAH_PROBE_DIR - optionally where to save log files so utah will grab them
4091+
4092+set -e
4093+
4094+[ -z $UTAH_PROBE_DIR ] && UTAH_PROBE_DIR="/tmp"
4095+
4096+# default exit code storage
4097+dump_error=1
4098+
4099+calc () { awk "BEGIN{ print $* }" ;}
4100+
4101+function show_usage() {
4102+ echo "Usage:"
4103+ echo " $0 [options]"
4104+ echo "Options:"
4105+ echo " -r run forever without exiting"
4106+ echo " -p minimum idle percent to wait for (Default: 99)"
4107+ echo " -c number of times to run top at each iteration (Default: 10)"
4108+ echo " -d seconds to delay between each top iteration (Default: 6)"
4109+ echo " -i top measurements to ignore from each loop (Default: 1)"
4110+ echo " -m maximum loops of top before giving up if minimum idle"
4111+ echo " percent is not reached (Default: 10)"
4112+ echo " -l label to include for the top_log file"
4113+ echo " -s sleep timeout for %cpu calculation (Default: 10)"
4114+ exit 129
4115+}
4116+
4117+while getopts "h?rp:c:d:i:m:l:s:" opt; do
4118+ case "$opt" in
4119+ h|\?) show_usage
4120+ ;;
4121+ r) settle_prefix='-'
4122+ ;;
4123+ p) idle_avg_min=$OPTARG
4124+ ;;
4125+ c) top_repeat=$OPTARG
4126+ ;;
4127+ d) top_wait=$OPTARG
4128+ ;;
4129+ i) top_ignore=$OPTARG
4130+ ;;
4131+ m) settle_max=$OPTARG
4132+ ;;
4133+ l) top_log_label=$OPTARG
4134+ ;;
4135+ s) sleep_len=$OPTARG
4136+ ;;
4137+ esac
4138+done
4139+
4140+sleep_len=${sleep_len:-10}
4141+HZ=`getconf CLK_TCK`
4142+# minimum average idle level required to succeed
4143+idle_avg_min=${idle_avg_min:-99}
4144+# measurement details: top $top_wait $top_repeat
4145+top_repeat=${top_repeat:-10}
4146+top_wait=${top_wait:-6}
4147+# how many samples to ignore
4148+top_ignore=${top_ignore:-1}
4149+# how many total attempts to settle the system
4150+settle_max=${settle_max:-10}
4151+
4152+top_log="$UTAH_PROBE_DIR/top$top_log_label.log"
4153+
4154+# set and calc more runtime values
4155+top_tail=`calc $top_repeat - $top_ignore`
4156+settle_count=0
4157+idle_avg=0
4158+
4159+echo "System Settle run - quiesce the system"
4160+echo "--------------------------------------"
4161+echo
4162+echo " idle_avg_min = '$idle_avg_min'"
4163+echo " top_repeat = '$top_repeat'"
4164+echo " top_wait = '$top_wait'"
4165+echo " top_ignore = '$top_ignore'"
4166+echo " settle_max = '$settle_max'"
4167+echo " run_forever = '$settle_prefix' (- = yes)"
4168+echo " log files = $top_log $top_log.reduced"
4169+echo
4170+
4171+while test `calc $idle_avg '<' $idle_avg_min` = 1 -a "$settle_prefix$settle_count" -lt "$settle_max"; do
4172+ echo -n "Starting system idle measurement (run: $settle_count) ... "
4173+
4174+ # get top
4175+ echo "TOP DUMP (after settle run: $settle_count)" >> $top_log
4176+ echo "========================" >> $top_log
4177+ ${TARGET_PREFIX} COLUMNS=900 top -c -b -d $top_wait -n $top_repeat >> $top_log
4178+ cat $top_log | grep '.Cpu.*' | tail -n $top_tail > $top_log.reduced
4179+ echo >> $top_log
4180+
4181+ # Instead of using top, we need to use /proc/stat and compensate for
4182+ # the number of cpus and any frequency scaling that could be in effect
4183+ cpu_avg=$({
4184+ cat /proc/stat
4185+ sleep "$sleep_len"
4186+ cat /proc/stat
4187+ } | awk '
4188+ BEGIN { iter = 0 }
4189+ /^cpu / { iter = iter + 1; count = 0; next }
4190+ /^cpu/ { S[iter] = S[iter] + ($2+$3+$4+$6); count = count + 1;
4191+next }
4192+ END { print (S[2] - S[1]) * 100 / ('"$HZ"' * count * '"$sleep_len"') }
4193+ ')
4194+ idle_avg=`calc 100 - $cpu_avg`
4195+ settle_count=`calc $settle_count + 1`
4196+
4197+ echo " DONE."
4198+ echo
4199+ echo "Measurement:"
4200+ echo " + idle level: $idle_avg"
4201+ echo
4202+done
4203+
4204+if test `calc $idle_avg '<' $idle_avg_min` = 1; then
4205+ echo "system not settled. FAIL"
4206+ exit 1
4207+else
4208+ echo "system settled. SUCCESS"
4209+ exit 0
4210+fi
4211+
4212
4213=== added file 'tests/default/systemsettle/tc_control'
4214--- tests/default/systemsettle/tc_control 1970-01-01 00:00:00 +0000
4215+++ tests/default/systemsettle/tc_control 2014-07-03 19:35:55 +0000
4216@@ -0,0 +1,9 @@
4217+description: check if system settles to idle average > 99.25%
4218+dependencies: none
4219+action: |
4220+ 1. Take CPU load samples for 10 minutes and fail if average idle never goes above 99.25% percent
4221+expected_results: |
4222+ 1. When doing nothing, system calms down to at least 99.25% idle level
4223+type: userland
4224+timeout: 720
4225+command: ./systemsettle.sh -c5 -d6 -p 97.5
4226
4227=== added file 'tests/default/ts_control'
4228--- tests/default/ts_control 1970-01-01 00:00:00 +0000
4229+++ tests/default/ts_control 2014-07-03 19:35:55 +0000
4230@@ -0,0 +1,3 @@
4231+#build_cmd: echo building
4232+#ts_setup: echo setting up
4233+#ts_cleanup: echo cleaning up
4234
4235=== added file 'tests/default/tslist.run'
4236--- tests/default/tslist.run 1970-01-01 00:00:00 +0000
4237+++ tests/default/tslist.run 2014-07-03 19:35:55 +0000
4238@@ -0,0 +1,10 @@
4239+- test: pwd
4240+- test: uname
4241+- test: vmstat
4242+- test: systemsettle
4243+- test: netstat
4244+- test: ifconfig
4245+- test: route
4246+- test: ping
4247+- test: install
4248+- test: unity8
4249
4250=== added directory 'tests/default/uname'
4251=== added file 'tests/default/uname/tc_control'
4252--- tests/default/uname/tc_control 1970-01-01 00:00:00 +0000
4253+++ tests/default/uname/tc_control 2014-07-03 19:35:55 +0000
4254@@ -0,0 +1,12 @@
4255+description: uname shows an ubuntu-phablet image installed
4256+dependencies: CHANGE_ME
4257+action: |
4258+ 1. uname -a|grep ubuntu-phablet
4259+expected_results: |
4260+ 1. ubuntu-phablet is installed as expected
4261+type: userland
4262+timeout: 60
4263+command: $TARGET_PREFIX uname -a | grep ubuntu-phablet
4264+#build_cmd:
4265+#tc_setup:
4266+#tc_cleanup:
4267
4268=== added directory 'tests/default/unity8'
4269=== added file 'tests/default/unity8/tc_control'
4270--- tests/default/unity8/tc_control 1970-01-01 00:00:00 +0000
4271+++ tests/default/unity8/tc_control 2014-07-03 19:35:55 +0000
4272@@ -0,0 +1,9 @@
4273+description: check for unity8 process
4274+dependencies: None
4275+action: |
4276+ 1. pgrep unity8
4277+expected_results: |
4278+ 1. pgrep exits with 0, indicating it found a matching process
4279+type: userland
4280+timeout: 60
4281+command: $TARGET_PREFIX /usr/bin/pgrep unity8
4282
4283=== added directory 'tests/default/vmstat'
4284=== added file 'tests/default/vmstat/tc_control'
4285--- tests/default/vmstat/tc_control 1970-01-01 00:00:00 +0000
4286+++ tests/default/vmstat/tc_control 2014-07-03 19:35:55 +0000
4287@@ -0,0 +1,12 @@
4288+description: Execute vmstat and capture output
4289+dependencies: none
4290+action: |
4291+ 1. vmstat
4292+expected_results: |
4293+ 1. vmstat finishes without an error
4294+type: userland
4295+timeout: 60
4296+command: $TARGET_PREFIX vmstat
4297+#build_cmd:
4298+#tc_setup:
4299+#tc_cleanup:
4300
4301=== added directory 'tests/eventstat'
4302=== added directory 'tests/eventstat/eventstat'
4303=== added file 'tests/eventstat/eventstat/eventstat.sh'
4304--- tests/eventstat/eventstat/eventstat.sh 1970-01-01 00:00:00 +0000
4305+++ tests/eventstat/eventstat/eventstat.sh 2014-07-03 19:35:55 +0000
4306@@ -0,0 +1,5 @@
4307+#!/bin/bash
4308+: ${DURATION:=60}
4309+: ${COUNT:=10}
4310+${TARGET_PREFIX} "echo \"{\\\"duration\\\": $DURATION, \\\"count\\\": $COUNT}\" > /tmp/results/options.json"
4311+${TARGET_PREFIX} "eventstat -CSl -r /tmp/results/eventstat.csv $DURATION $COUNT > /tmp/results/eventstat.log"
4312
4313=== added file 'tests/eventstat/eventstat/setup.sh'
4314--- tests/eventstat/eventstat/setup.sh 1970-01-01 00:00:00 +0000
4315+++ tests/eventstat/eventstat/setup.sh 2014-07-03 19:35:55 +0000
4316@@ -0,0 +1,7 @@
4317+#!/bin/bash
4318+
4319+${TARGET_PREFIX} mkdir -p /tmp/results
4320+NO_UNLOCK=1 PKGS="eventstat" prepare-autopilot-test.sh
4321+#Let the system quiet down for a little while
4322+sleep 300
4323+
4324
4325=== added file 'tests/eventstat/eventstat/tc_control'
4326--- tests/eventstat/eventstat/tc_control 1970-01-01 00:00:00 +0000
4327+++ tests/eventstat/eventstat/tc_control 2014-07-03 19:35:55 +0000
4328@@ -0,0 +1,12 @@
4329+description: Gather memory usage information
4330+dependencies: none
4331+action: |
4332+ 1. gather eventstat data
4333+expected_results: |
4334+ 1. eventstat.log should be saved to /tmp/results
4335+type: userland
4336+timeout: 800
4337+command: ./eventstat.sh
4338+#build_cmd:
4339+tc_setup: ./setup.sh
4340+#tc_cleanup:
4341
4342=== added file 'tests/eventstat/master.run'
4343--- tests/eventstat/master.run 1970-01-01 00:00:00 +0000
4344+++ tests/eventstat/master.run 2014-07-03 19:35:55 +0000
4345@@ -0,0 +1,5 @@
4346+---
4347+testsuites:
4348+ - name: eventstat
4349+ fetch_method: dev
4350+ fetch_location: ./
4351
4352=== added file 'tests/eventstat/tslist.run'
4353--- tests/eventstat/tslist.run 1970-01-01 00:00:00 +0000
4354+++ tests/eventstat/tslist.run 2014-07-03 19:35:55 +0000
4355@@ -0,0 +1,1 @@
4356+- test: eventstat
4357
4358=== added directory 'tests/memevent'
4359=== added file 'tests/memevent/master.run'
4360--- tests/memevent/master.run 1970-01-01 00:00:00 +0000
4361+++ tests/memevent/master.run 2014-07-03 19:35:55 +0000
4362@@ -0,0 +1,5 @@
4363+---
4364+testsuites:
4365+ - name: memevent
4366+ fetch_method: dev
4367+ fetch_location: ./
4368
4369=== added file 'tests/memevent/setup.sh'
4370--- tests/memevent/setup.sh 1970-01-01 00:00:00 +0000
4371+++ tests/memevent/setup.sh 2014-07-03 19:35:55 +0000
4372@@ -0,0 +1,8 @@
4373+#!/bin/sh
4374+
4375+set -e
4376+
4377+# copy the autopilot scripts over if needed
4378+[ -z $ANDROID_SERIAL ] || adb push ./ubuntu_test_cases /home/phablet/autopilot/ubuntu_test_cases
4379+
4380+PKGS="camera-app-autopilot gallery-app-autopilot mediaplayer-app-autopilot webbrowser-app-autopilot" prepare-autopilot-test.sh
4381
4382=== added file 'tests/memevent/ts_control'
4383--- tests/memevent/ts_control 1970-01-01 00:00:00 +0000
4384+++ tests/memevent/ts_control 2014-07-03 19:35:55 +0000
4385@@ -0,0 +1,1 @@
4386+ts_setup: ./setup.sh
4387
4388=== added file 'tests/memevent/tslist.auto'
4389--- tests/memevent/tslist.auto 1970-01-01 00:00:00 +0000
4390+++ tests/memevent/tslist.auto 2014-07-03 19:35:55 +0000
4391@@ -0,0 +1,7 @@
4392+---
4393+# The discovery command is run from a directory containing the whole suite
4394+# The test command is run from a sub directory in that
4395+# We need to run each command in the directory that contains ubuntu_test_cases
4396+# so that autopilot can find our python modules
4397+- discovery_cmd: cd memevent; autopilot-list ubuntu_test_cases.memory_usage_measurement
4398+ test_cmd: cd ..; autopilot-run ubuntu_test_cases.memory_usage_measurement.tests.{}
4399
4400=== added directory 'tests/memevent/ubuntu_test_cases'
4401=== added file 'tests/memevent/ubuntu_test_cases/__init__.py'
4402--- tests/memevent/ubuntu_test_cases/__init__.py 1970-01-01 00:00:00 +0000
4403+++ tests/memevent/ubuntu_test_cases/__init__.py 2014-07-03 19:35:55 +0000
4404@@ -0,0 +1,6 @@
4405+"""ubuntu_test_cases package
4406+
4407+Warning: This python package is installed by a different debian package in a
4408+different branch. It's been been created here only because it's convenient to
4409+run autopilot directly from the branch.
4410+"""
4411
4412=== added directory 'tests/memevent/ubuntu_test_cases/memory_usage_measurement'
4413=== added file 'tests/memevent/ubuntu_test_cases/memory_usage_measurement/__init__.py'
4414--- tests/memevent/ubuntu_test_cases/memory_usage_measurement/__init__.py 1970-01-01 00:00:00 +0000
4415+++ tests/memevent/ubuntu_test_cases/memory_usage_measurement/__init__.py 2014-07-03 19:35:55 +0000
4416@@ -0,0 +1,1 @@
4417+"""Event based memory usage measurements test cases."""
4418
4419=== added directory 'tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps'
4420=== added file 'tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/__init__.py'
4421--- tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/__init__.py 1970-01-01 00:00:00 +0000
4422+++ tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/__init__.py 2014-07-03 19:35:55 +0000
4423@@ -0,0 +1,21 @@
4424+"""Application classes used to drive them easily with autopilot."""
4425+
4426+from autopilot.input import Keyboard, Mouse, Pointer, Touch
4427+from autopilot.platform import model
4428+
4429+
4430+class App(object):
4431+
4432+ """Application class with common code."""
4433+
4434+ def __init__(self, tc):
4435+ self.tc = tc
4436+ self.app = None
4437+ self.window = None
4438+
4439+ self.keyboard = Keyboard.create()
4440+ if model() == 'Desktop':
4441+ input_device = Mouse.create()
4442+ else:
4443+ input_device = Touch.create()
4444+ self.pointer = Pointer(input_device)
4445
4446=== added file 'tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/browser.py'
4447--- tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/browser.py 1970-01-01 00:00:00 +0000
4448+++ tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/browser.py 2014-07-03 19:35:55 +0000
4449@@ -0,0 +1,103 @@
4450+"""Browser application to write autopilot test cases easily."""
4451+
4452+from testtools.matchers import Contains, Equals
4453+
4454+from ubuntuuitoolkit.emulators import UbuntuUIToolkitEmulatorBase
4455+from webbrowser_app.emulators.browser import Browser
4456+
4457+from ubuntu_test_cases.memory_usage_measurement.apps import App
4458+from ubuntu_test_cases.memory_usage_measurement.matchers import (
4459+ DoesNotChange,
4460+ Eventually,
4461+)
4462+
4463+
4464+class BrowserApp(App):
4465+
4466+ """Browser application."""
4467+
4468+ TYPING_DELAY = 0.01
4469+
4470+ def assert_chrome_eventually_hidden(self):
4471+ """Make sure chrome is eventually hidden."""
4472+
4473+ toolbar = self.window.get_toolbar()
4474+ self.tc.assertThat(toolbar.opened, Eventually(Equals(False)))
4475+ self.tc.assertThat(toolbar.animating, Eventually(Equals(False)))
4476+
4477+ def assert_page_eventually_loaded(self, url):
4478+ """Make sure page is eventually loaded."""
4479+ webview = self.window.get_current_webview()
4480+ self.tc.assertThat(webview.url, Eventually(Equals(url)))
4481+ # loadProgress == 100 ensures that a page has actually loaded
4482+ self.tc.assertThat(webview.loadProgress, Eventually(Equals(100)))
4483+ self.tc.assertThat(webview.loading, Eventually(Equals(False)))
4484+
4485+ def clear_address_bar(self):
4486+ """Clear address bar."""
4487+ self.focus_address_bar()
4488+ clear_button = self.window.get_address_bar_clear_button()
4489+ self.tc.assertThat(lambda: (clear_button.x,
4490+ clear_button.y,
4491+ clear_button.y),
4492+ Eventually(DoesNotChange()))
4493+ self.pointer.move_to_object(clear_button)
4494+ self.pointer.click()
4495+ text_field = self.window.get_address_bar_text_field()
4496+ self.tc.assertThat(text_field.text, Eventually(Equals("")))
4497+
4498+ def ensure_chrome_is_hidden(self):
4499+ """Make sure chrome is hidden."""
4500+ webview = self.window.get_current_webview()
4501+ self.pointer.move_to_object(webview)
4502+ self.pointer.click()
4503+ self.assert_chrome_eventually_hidden()
4504+
4505+ def focus_address_bar(self):
4506+ """Make sure address bar is focused."""
4507+ address_bar = self.window.get_address_bar()
4508+ self.pointer.move_to_object(address_bar)
4509+ self.pointer.click()
4510+ self.tc.assertThat(address_bar.activeFocus, Eventually(Equals(True)))
4511+
4512+ def go_to_url(self, url):
4513+ """Go to given url.
4514+
4515+ :param url: URL that should be loaded in the browser
4516+ :type url: str
4517+
4518+ """
4519+ self.ensure_chrome_is_hidden()
4520+ self.window.open_toolbar()
4521+ self.clear_address_bar()
4522+ self.type_in_address_bar(url)
4523+ self.keyboard.press_and_release("Enter")
4524+
4525+ def launch(self):
4526+ """Launch application."""
4527+ args = [
4528+ 'webbrowser-app',
4529+ '--fullscreen',
4530+ ('--desktop_file_hint='
4531+ '/usr/share/applications/webbrowser-app.desktop'),
4532+ ]
4533+ self.app = self.tc.launch_test_application(
4534+ *args,
4535+ app_type='qt',
4536+ emulator_base=UbuntuUIToolkitEmulatorBase
4537+ )
4538+ self.window = self.app.select_single(Browser)
4539+ self.window.visible.wait_for(True)
4540+
4541+ def type_in_address_bar(self, text):
4542+ """Type text in address bar.
4543+
4544+ :param text: Text to be typed in the address bar.
4545+ :type text: str
4546+
4547+ """
4548+ address_bar = self.window.get_address_bar()
4549+ self.tc.assertThat(address_bar.activeFocus, Eventually(Equals(True)))
4550+ self.keyboard.type(text, delay=self.TYPING_DELAY)
4551+ text_field = self.window.get_address_bar_text_field()
4552+ self.tc.assertThat(text_field.text, Eventually(Contains(text)))
4553
4554=== added file 'tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/camera.py'
4555--- tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/camera.py 1970-01-01 00:00:00 +0000
4556+++ tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/camera.py 2014-07-03 19:35:55 +0000
4557@@ -0,0 +1,58 @@
4558+"""Camera application to write autopilot test cases easily."""
4559+
4560+import logging
4561+import os
4562+
4563+from glob import glob
4564+
4565+from camera_app.emulators.main_window import MainWindow as CameraWindow
4566+from testtools.matchers import Equals, HasLength
4567+
4568+from ubuntu_test_cases.memory_usage_measurement.apps import App
4569+from ubuntu_test_cases.memory_usage_measurement.matchers import Eventually
4570+
4571+LOGGER = logging.getLogger(__file__)
4572+
4573+
4574+class CameraApp(App):
4575+
4576+ """Camera application."""
4577+
4578+ IMAGE_FILENAME_PATTERN = os.path.expanduser('~/Pictures/image*')
4579+
4580+ def _remove_image_files(self):
4581+ """Remove all image files.
4582+
4583+ This is useful not only to cleanup, but also to check that just one
4584+ image file has been written to disk.
4585+
4586+ """
4587+ for filename in glob(self.IMAGE_FILENAME_PATTERN):
4588+ LOGGER.debug('Removing image file: {}'.format(filename))
4589+ os.remove(filename)
4590+
4591+ def launch(self):
4592+ """Launch application."""
4593+ args = [
4594+ 'camera-app',
4595+ '--fullscreen',
4596+ ('--desktop_file_hint='
4597+ '/usr/share/applications/camera-app.desktop'),
4598+ ]
4599+ self.app = self.tc.launch_test_application(*args, app_type='qt')
4600+ self.window = CameraWindow(self.app)
4601+
4602+ def take_picture(self):
4603+ """Click on the exposure button to take picture."""
4604+ self._remove_image_files()
4605+
4606+ exposure_button = self.window.get_exposure_button()
4607+ self.tc.assertThat(exposure_button.enabled, Eventually(Equals(True)))
4608+
4609+ LOGGER.debug('Taking picture...')
4610+ self.pointer.move_to_object(exposure_button)
4611+ self.pointer.click()
4612+
4613+ # Wait for image file to be produced before memory usage measurement
4614+ self.tc.assertThat(lambda: glob(self.IMAGE_FILENAME_PATTERN),
4615+ Eventually(HasLength(1)))
4616
4617=== added file 'tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/gallery.py'
4618--- tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/gallery.py 1970-01-01 00:00:00 +0000
4619+++ tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/gallery.py 2014-07-03 19:35:55 +0000
4620@@ -0,0 +1,41 @@
4621+"""Gallery application to write autopilot test cases easily."""
4622+
4623+import os
4624+import shutil
4625+import tempfile
4626+
4627+from ubuntu_test_cases.memory_usage_measurement.apps import App
4628+
4629+
4630+class GalleryApp(App):
4631+
4632+ """Gallery application."""
4633+
4634+ SAMPLE_DIR = '/usr/lib/python2.7/dist-packages/gallery_app/data/default'
4635+
4636+ def __init__(self, tc):
4637+ super(GalleryApp, self).__init__(tc)
4638+ self.temp_sample_dir = None
4639+ self._copy_sample_dir()
4640+
4641+ def _copy_sample_dir(self):
4642+ """Copy sample directory to a temporary location.
4643+
4644+ This is useful to provide some default content.
4645+
4646+ """
4647+ self.temp_sample_dir = tempfile.mkdtemp(prefix='gallery-app-test-')
4648+ self.tc.addCleanup(shutil.rmtree, self.temp_sample_dir)
4649+ self.temp_sample_dir = os.path.join(self.temp_sample_dir, 'data')
4650+ shutil.copytree(self.SAMPLE_DIR, self.temp_sample_dir)
4651+
4652+ def launch(self):
4653+ """Launch the application."""
4654+ args = [
4655+ 'gallery-app',
4656+ ('--desktop_file_hint='
4657+ '/usr/share/applications/gallery-app.desktop'),
4658+ self.temp_sample_dir,
4659+ ]
4660+ self.app = self.tc.launch_test_application(*args,
4661+ app_type='qt')
4662
4663=== added file 'tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/media_player.py'
4664--- tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/media_player.py 1970-01-01 00:00:00 +0000
4665+++ tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/media_player.py 2014-07-03 19:35:55 +0000
4666@@ -0,0 +1,53 @@
4667+"""Media player application to write autopilot test cases easily."""
4668+
4669+import os
4670+
4671+from testtools.matchers import Equals, GreaterThan
4672+
4673+from mediaplayer_app.emulators.main_window import (
4674+ MainWindow as MediaPlayerWindow)
4675+
4676+from ubuntu_test_cases.memory_usage_measurement.apps import App
4677+from ubuntu_test_cases.memory_usage_measurement.matchers import Eventually
4678+
4679+
4680+class MediaPlayerApp(App):
4681+
4682+ """Media player application."""
4683+
4684+ # Default content
4685+ VIDEOS_DIR = 'file:///usr/share/mediaplayer-app/videos/'
4686+
4687+ def assert_playback_finished(self):
4688+ """Media player memory usage after playing a file."""
4689+ time_line = self.window.get_slider()
4690+
4691+ # Time line value isn't set to maximum value after playback is finished
4692+ # (LP: #1190555)
4693+ maximum_value = time_line.maximumValue - 2.0
4694+ self.tc.assertThat(time_line.value,
4695+ Eventually(GreaterThan(maximum_value)))
4696+
4697+ def launch(self, movie_file=None):
4698+ """Launch application.
4699+
4700+ :param movie_file:
4701+ Relative path to movie file (uses default content directory as
4702+ root).
4703+ :type movie_file: str
4704+
4705+ """
4706+ binary = 'mediaplayer-app'
4707+ args = [
4708+ binary,
4709+ '--fullscreen',
4710+ ('--desktop_file_hint='
4711+ '/usr/share/applications/mediaplayer-app.desktop'),
4712+ ]
4713+ if movie_file:
4714+ args.insert(1, os.path.join(self.VIDEOS_DIR, movie_file))
4715+
4716+ self.app = self.tc.launch_test_application(*args, app_type='qt')
4717+ self.window = MediaPlayerWindow(self.app)
4718+ self.tc.assertThat(self.window.get_qml_view().visible,
4719+ Eventually(Equals(True)))
4720
4721=== added file 'tests/memevent/ubuntu_test_cases/memory_usage_measurement/matchers.py'
4722--- tests/memevent/ubuntu_test_cases/memory_usage_measurement/matchers.py 1970-01-01 00:00:00 +0000
4723+++ tests/memevent/ubuntu_test_cases/memory_usage_measurement/matchers.py 2014-07-03 19:35:55 +0000
4724@@ -0,0 +1,37 @@
4725+"""Custom matchers used by the tests."""
4726+
4727+from autopilot.matchers import Eventually as EventuallyMatcher
4728+from testtools.matchers import Mismatch
4729+
4730+
4731+def Eventually(matcher):
4732+ """Wrapper around autopilot.matchers.Eventually.
4733+
4734+ The aim of the wrapper is just use a different timeout default
4735+
4736+ :param matcher: A testools-like matcher that tests the desired condition
4737+ :type matcher: object
4738+ :returns: A value depending on the matcher protocol
4739+ :rtype: None | a mismatch object with information about the mismatch
4740+
4741+ """
4742+ return EventuallyMatcher(matcher, timeout=40)
4743+
4744+
4745+class DoesNotChange(object):
4746+ """Match if two consecutive values are equal."""
4747+ def __init__(self):
4748+ self.old_value = None
4749+ self.value = None
4750+
4751+ def __str__(self):
4752+ return 'DoesNotChange()'
4753+
4754+ def match(self, value):
4755+ self.old_value = self.value
4756+ self.value = value
4757+ if self.value != self.old_value:
4758+ return Mismatch('Current value ({}) does not match old value ({})'
4759+ .format(self.value, self.old_value))
4760+
4761+ return None
4762
4763=== added file 'tests/memevent/ubuntu_test_cases/memory_usage_measurement/probes.py'
4764--- tests/memevent/ubuntu_test_cases/memory_usage_measurement/probes.py 1970-01-01 00:00:00 +0000
4765+++ tests/memevent/ubuntu_test_cases/memory_usage_measurement/probes.py 2014-07-03 19:35:55 +0000
4766@@ -0,0 +1,170 @@
4767+"""Probes used to take measurements4 while the tests are executed."""
4768+
4769+import itertools
4770+import logging
4771+import os
4772+import re
4773+import subprocess
4774+
4775+from contextlib import contextmanager
4776+from time import time
4777+
4778+LOGGER = logging.getLogger(__file__)
4779+
4780+
4781+class SmemProbe(object):
4782+
4783+ """Memory usage probe."""
4784+
4785+ BINARY = os.path.join(os.path.dirname(__file__), 'smem-tabs')
4786+ THRESHOLDS = {
4787+ 'foreground': 262144, # 256MB
4788+ 'background': 131072, # 128MB
4789+ }
4790+
4791+ def __init__(self):
4792+ self.pids = [] # List of pids that should be monitored
4793+ self.readings = [] # List of readings
4794+ self.current_reading = None
4795+ self.threshold_exceeded_summary = []
4796+
4797+ @contextmanager
4798+ def probe(self, event):
4799+ """Run start and stop methods in a contex manager."""
4800+ self.start(event)
4801+ yield
4802+ self.stop(event)
4803+
4804+ def start(self, event):
4805+ """Start measurement.
4806+
4807+ This method is not actually used, but defined to be consistent with the
4808+ probes API.
4809+
4810+ :param event: Event name
4811+ :type event: str
4812+ :returns: None
4813+
4814+ """
4815+ LOGGER.debug('smem start: {}'.format(event))
4816+ self.current_reading = {
4817+ 'event': event,
4818+ 'start_time': time(),
4819+ }
4820+
4821+ def stop(self, event):
4822+ """Stop measurement.
4823+
4824+ Run smem and get memory usage for a given set PIDs.
4825+
4826+ :param event: Event name
4827+ :type event: str
4828+ :returns: None
4829+
4830+ """
4831+ LOGGER.debug('smem stop: {}'.format(event))
4832+ LOGGER.debug('Running {!r}...'.format(self.BINARY))
4833+ output = subprocess.check_output(self.BINARY)
4834+ parser = SmemParser()
4835+ pids_info = parser.parse(output)
4836+ threshold_exceeded_pids = self._calculate_threshold_exceeded(pids_info)
4837+ print '{:-^72}'.format(event)
4838+ for pid in self.pids:
4839+ print('PID: {pid}, command: {command}, PSS: {pss}, USS: {uss}'
4840+ .format(**pids_info[pid]))
4841+
4842+ self.current_reading['stop_time'] = time()
4843+ self.current_reading['data'] = pids_info
4844+ self.readings.append(self.current_reading)
4845+ if threshold_exceeded_pids:
4846+ self.threshold_exceeded_summary.append(
4847+ (event, threshold_exceeded_pids))
4848+ self.current_reading = None
4849+
4850+ def _calculate_threshold_exceeded(self, pids_info):
4851+ """Calculate thresholds for the given set of pids.
4852+
4853+ :param pids_info:
4854+ Memory usage data for a give set of pids.
4855+
4856+ ..note::
4857+ This parameter is modified in place to add a delta of the PSS
4858+ and the threshold.
4859+ :type pids_info: list(dict)
4860+
4861+ """
4862+ # It's assumed that the pid for the foreground application is the one
4863+ # that was last added to the probe
4864+ foreground_pid = self.pids[-1]
4865+ foreground_threshold = self.THRESHOLDS['foreground']
4866+ background_pids = self.pids[:-1]
4867+ background_threshold = self.THRESHOLDS['background']
4868+
4869+ pids_and_thresholds = itertools.chain(
4870+ [(foreground_pid, foreground_threshold)],
4871+ itertools.product(background_pids, [background_threshold]))
4872+
4873+ threshold_exceeded_pids = []
4874+ for pid, threshold in pids_and_thresholds:
4875+ if pid in pids_info:
4876+ pid_info = pids_info[pid]
4877+ delta = pid_info['pss'] - threshold
4878+ pid_info['threshold_exceeded'] = delta
4879+ if delta > 0:
4880+ threshold_exceeded_pids.append(pid)
4881+ return threshold_exceeded_pids
4882+
4883+ @property
4884+ def report(self):
4885+ """Return report with all the readings that have been made."""
4886+ return {'pids': self.pids,
4887+ 'thresholds': self.THRESHOLDS,
4888+ 'readings': self.readings}
4889+
4890+
4891+class SmemParser(object):
4892+
4893+ """Parser object to map smem output to data structure."""
4894+
4895+ SMEM_LINE = re.compile(
4896+ r'\s*(?P<pid>\d+)'
4897+ r'\s+(?P<user>\w+)'
4898+ r'\s+(?P<command>.+?)'
4899+ r'\s+(?P<swap>\d+)'
4900+ r'\s+(?P<uss>\d+)'
4901+ r'\s+(?P<pss>\d+)'
4902+ r'\s+(?P<rss>\d+)')
4903+
4904+ def parse(self, output):
4905+ """Parse smem output.
4906+
4907+ :param output: The report printed to stdout by smem
4908+ :type output: str
4909+ :returns: Data structure that can be serialized
4910+ :rtype: dict(dict)
4911+
4912+ """
4913+ pids_info = filter(None,
4914+ [self._parse_line(line)
4915+ for line in output.splitlines()[1:]])
4916+ return {pid_info['pid']: pid_info for pid_info in pids_info}
4917+
4918+ def _parse_line(self, line):
4919+ """Parse a single smem output line.
4920+
4921+ :param line:
4922+ A single line containing all the fields to parse for a process.
4923+ :type line: str
4924+ :returns: Data structure with the parsed fields
4925+ :rtype: dict
4926+
4927+ """
4928+ match = self.SMEM_LINE.match(line)
4929+ if not match:
4930+ LOGGER.warning('Unable to parse smem output: {}'.format(line))
4931+ return None
4932+
4933+ pid_info = match.groupdict()
4934+ for key in ('pid', 'swap', 'uss', 'pss', 'rss'):
4935+ pid_info[key] = int(pid_info[key])
4936+ return pid_info
4937
4938=== added file 'tests/memevent/ubuntu_test_cases/memory_usage_measurement/smem-tabs'
4939--- tests/memevent/ubuntu_test_cases/memory_usage_measurement/smem-tabs 1970-01-01 00:00:00 +0000
4940+++ tests/memevent/ubuntu_test_cases/memory_usage_measurement/smem-tabs 2014-07-03 19:35:55 +0000
4941@@ -0,0 +1,687 @@
4942+#!/usr/bin/env python
4943+#
4944+# smem - a tool for meaningful memory reporting
4945+#
4946+# Copyright 2008-2009 Matt Mackall <mpm@selenic.com>
4947+#
4948+# This software may be used and distributed according to the terms of
4949+# the GNU General Public License version 2 or later, incorporated
4950+# herein by reference.
4951+
4952+import re, os, sys, pwd, optparse, errno, tarfile
4953+
4954+warned = False
4955+
4956+class procdata(object):
4957+ def __init__(self, source):
4958+ self._ucache = {}
4959+ self._gcache = {}
4960+ self.source = source and source or ""
4961+ self._memdata = None
4962+ def _list(self):
4963+ return os.listdir(self.source + "/proc")
4964+ def _read(self, f):
4965+ return file(self.source + '/proc/' + f).read()
4966+ def _readlines(self, f):
4967+ return self._read(f).splitlines(True)
4968+ def _stat(self, f):
4969+ return os.stat(self.source + "/proc/" + f)
4970+
4971+ def pids(self):
4972+ '''get a list of processes'''
4973+ return [int(e) for e in self._list()
4974+ if e.isdigit() and not iskernel(e)]
4975+ def mapdata(self, pid):
4976+ return self._readlines('%s/smaps' % pid)
4977+ def memdata(self):
4978+ if self._memdata is None:
4979+ self._memdata = self._readlines('meminfo')
4980+ return self._memdata
4981+ def version(self):
4982+ return self._readlines('version')[0]
4983+ def pidname(self, pid):
4984+ try:
4985+ l = self.pidcmd(pid).split(' ')[0]
4986+ return os.path.basename(l)
4987+ except:
4988+ return '?'
4989+ def pidcmd(self, pid):
4990+ try:
4991+ c = self._read('%s/cmdline' % pid)[:-1]
4992+ return c.replace('\0', ' ')
4993+ except:
4994+ return '?'
4995+ def piduser(self, pid):
4996+ try:
4997+ return self._stat('%d' % pid).st_uid
4998+ except:
4999+ return -1
5000+ def pidgroup(self, pid):
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches