Merge lp:~nskaggs/ubuntu-test-cases/fix-1367654 into lp:ubuntu-test-cases

[Nicholas Skaggs]

Add address-book-service-testability depends for calendar

Add address-book-service-testability depends for calendar

Unmerged revisions

290. By Nicholas Skaggs on 2014-09-10

add new calendar depends

289. By Paul Larson on 2014-09-09

also specify password when recovering, in case we soon have stable images with the new adb

288. By Paul Larson on 2014-09-08

run security tests that require root under sudo

287. By Paul Larson on 2014-09-05

Fix provisioning to work with the new rtm and adbd as phablet user

286. By Paul Larson on 2014-09-05

Check to make sure that autoreporting is enabled, if it isn't then the .upload file won't ever be created.

285. By Paul Larson on 2014-08-27

hard kill powerd-cli on the device after the test run

284. By Paul Larson on 2014-08-27

also capture whoopsie.log

283. By Paul Larson on 2014-08-27

add more support for phablet user running adbd

282. By Paul Larson on 2014-08-22

a few more tweaks on health check

281. By Paul Larson on 2014-08-22

update health check thresholds for minor tweaks

=== added file 'README-cli.rst'
--- README-cli.rst	1970-01-01 00:00:00 +0000
+++ README-cli.rst	2014-09-10 13:48:46 +0000
@@ -0,0 +1,80 @@
+Touch Testing From the CLI
+==========================
+
+The touch testing execution framework was written so that its very easy to
+run tests from home in the exact same way test are run in the lab. The only
+thigs you need are:
+
+ * This bzr branch
+ * The phablet-tools_ package
+ * An Ubuntu Touch supported_ device
+
+.. _phablet-tools: http://launchpad.net/phablet-tools
+.. _supported: http://wiki.ubuntu.com/Touch/Devices
+
+There are two pieces to touch testing, provisioning and test execution. These
+functions are independent of one another. eg, If your device already
+has the proper image/configuration, you can simply use the test-runner.
+
+Provisioning
+------------
+
+The provisioning script is a simple wrapper to commands from phablet-tools
+to get a device ready for testing. Provisioning is performed with the
+scripts/provision.sh command. Running::
+
+  ./scripts/provision.sh -h
+
+will list supported options.
+
+NOTE: provision.sh requires a path to a network-manager wifi connection that
+can be copied to the target device. By default this is set to
+/home/ubuntu/magners-wifi. This can be overriden with the -n parameter.
+
+Executing Tests
+---------------
+
+The touch testing repository supports both autopilot and UTAH test definitions.
+
+Executing Autopilot Tests
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+One or more autopilot tests can be run on the target device using the command::
+
+  ./scripts/run-autopilot-tests.sh
+
+This is a small wrapper that uses phablet-tools to drive the tests. The
+script can run one or more autopilot tests. By default it will reboot the
+device between each test and ensure the device is settled using the
+*system-settle* script. Both of those options can be disabled via command
+line options. By default the script will create a directory named
+*clientlogs* and then a subdirectory for each testsuite with result files.
+These sub-directories include a xUnit XML formatted file, *test_results.xml*,
+as well as several log files from the device to help with debugging failures.
+
+An example testing two applications::
+
+ ./scripts/run-autopilot-tests.sh -a dropping_letters_app -a music_app
+
+Executing UTAH Tests
+~~~~~~~~~~~~~~~~~~~~
+
+Executing UTAH tests locally will require you to install the UTAH client
+package from a PPA::
+
+  sudo add-apt-repository ppa:utah/stable
+  sudo apt-get update
+  sudo apt-get install utah-client
+
+With that package installed UTAH tests can be run with::
+
+  ./scripts/jenkins.sh
+
+This script runs one test at a time and will put its test artifacts under the
+*clientlogs* directory similar to the autopilot runner. The UTAH result file
+will be named clientlogs/utah.yaml.
+
+An example of running the sdk test suite::
+
+  ./scripts/jenkins.sh -a sdk
+
 
=== added directory 'jenkins'
=== added file 'jenkins/custom-demo.py'
--- jenkins/custom-demo.py	1970-01-01 00:00:00 +0000
+++ jenkins/custom-demo.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,32 @@
+# The configuration matrix of our production device testing
+
+JENKINS = 'http://q-jenkins.ubuntu-ci:8080'
+
+
+def _url(channel, device):
+    return 'http://system-image.ubuntu.com/ubuntu-touch/%s/%s/index.json' \
+           % (channel, device)
+
+
+TRUSTY_MATRIX = [
+    {
+        'image-type': 'touch_custom_demo',
+        'include-qa': True,
+        'dashboard-host': 'ci.ubuntu.com',
+        'dashboard-port': '80',
+        'dashboard-user': 'doanac',
+        'devices': [
+            {
+                'name': 'mako',
+                'slave-label': 'daily-mako',
+                'trigger_url': _url('trusty-proposed-customized-demo', 'mako'),
+            },
+        ],
+        'IMAGE_OPT': 'export IMAGE_OPT="ubuntu-system -b '
+                     '--channel ubuntu-touch/trusty-proposed-customized-demo"'
+    },
+]
+
+MATRIX = {
+    'trusty': TRUSTY_MATRIX,
+}
 
=== added file 'jenkins/production.py'
--- jenkins/production.py	1970-01-01 00:00:00 +0000
+++ jenkins/production.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,92 @@
+# The configuration matrix of our production device testing
+
+JENKINS = 'http://q-jenkins.ubuntu-ci:8080'
+
+
+def _url(channel, device):
+    return 'http://system-image.ubuntu.com/ubuntu-touch/%s/%s/index.json' \
+           % (channel, device)
+
+
+UTOPIC_MATRIX = [
+    {
+        'image-type': 'touch',
+        'statsd-key': 'ubuntu-ci.daily-image.production',
+        'include-qa': True,
+        'dashboard-host': 'ci.ubuntu.com',
+        'dashboard-port': '80',
+        'dashboard-user': 'uci-bot',
+        'devices': [
+            {
+                'name': 'mako',
+                'slave-label': 'daily-mako',
+                'trigger_url': _url('devel-proposed', 'mako'),
+                'num-workers': 3,
+            },
+            {
+                'name': 'flo',
+                'slave-label': 'daily-flo',
+                'trigger_url': _url('devel-proposed', 'flo'),
+                'num-workers': 2,
+            },
+            {
+                'name': 'manta',
+                'slave-label': 'daily-manta',
+                'trigger_url': _url('devel-proposed', 'manta'),
+                'num-workers': 2,
+            },
+        ],
+    },
+    {
+        'image-type': 'touch_stable',
+        'statsd-key': 'ubuntu-ci.daily-image.production',
+        'include-qa': True,
+        'dashboard-host': 'ci.ubuntu.com',
+        'dashboard-port': '80',
+        'dashboard-user': 'uci-bot',
+        'devices': [
+            {
+                'name': 'mako',
+                'slave-label': 'daily-mako',
+                'trigger_url': _url('ubuntu-rtm/14.09-proposed', 'mako'),
+                'num-workers': 3,
+            },
+            {
+                'name': 'flo',
+                'slave-label': 'daily-flo',
+                'trigger_url': _url('ubuntu-rtm/14.09-proposed', 'flo'),
+                'num-workers': 2,
+            },
+            {
+                'name': 'manta',
+                'slave-label': 'daily-manta',
+                'trigger_url': _url('ubuntu-rtm/14.09-proposed', 'manta'),
+                'num-workers': 2,
+            },
+        ],
+        'IMAGE_OPT': 'export IMAGE_OPT="--bootstrap --developer-mode '
+                     '--channel ubuntu-touch/ubuntu-rtm/14.09-proposed"'
+    },
+    {
+        'image-type': 'touch_custom',
+        'statsd-key': 'ubuntu-ci.daily-image.production',
+        'include-qa': False,
+        'dashboard-host': 'ci.ubuntu.com',
+        'dashboard-port': '80',
+        'dashboard-user': 'uci-bot',
+        'devices': [
+            {
+                'name': 'mako',
+                'slave-label': 'daily-mako',
+                'trigger_url': _url('utopic-proposed-customized', 'mako'),
+            },
+        ],
+        'IMAGE_OPT': 'export IMAGE_OPT="--bootstrap --developer-mode '
+                     '--channel ubuntu-touch/utopic-proposed-customized"'
+    },
+]
+
+
+MATRIX = {
+    'utopic': UTOPIC_MATRIX,
+}
 
=== added file 'jenkins/setup_jenkins.py'
--- jenkins/setup_jenkins.py	1970-01-01 00:00:00 +0000
+++ jenkins/setup_jenkins.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,222 @@
+#!/usr/bin/env python
+
+# Ubuntu Testing Automation Harness
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import argparse
+import imp
+import jenkins
+import jinja2
+import logging
+import os
+
+import testconfig
+
+from distro_info import UbuntuDistroInfo
+DEV_SERIES = UbuntuDistroInfo().devel()
+
+
+def _get_parser():
+    """Create and return command line parser.
+
+    :returns: command line parser
+    :rtype: argparse.ArgumentParser
+
+    """
+    parser = argparse.ArgumentParser(
+        description='Create/Update upgrade testing jenkins jobs.')
+    parser.add_argument("-d", "--dryrun", action="store_true",
+                        help="Dry run mode. Don't execute jenkins commands.")
+    parser.add_argument("-u", "--username",
+                        help="username to use when logging into Jenkins.")
+    parser.add_argument("-p", "--password",
+                        help="username to use when logging into Jenkins")
+    parser.add_argument("--dashboard-key", default="",
+                        help="The api-key for dashboard updates")
+    parser.add_argument("-b", "--branch", default="lp:ubuntu-test-cases/touch",
+                        help="The branch this is located. default=%(default)s")
+    parser.add_argument("-c", "--config", required=True,
+                        type=argparse.FileType('r'),
+                        help="The job config to use.")
+    parser.add_argument("-P", "--publish", action="store_true",
+                        help="Publish at the end of the job")
+    parser.add_argument("--prefix",
+                        help="Prefix to add to the beginning of the job name")
+    parser.add_argument("-s", "--series", default=DEV_SERIES,
+                        help=("series of Ubuntu to download "
+                              "(default=%(default)s)"))
+    parser.add_argument("-w", "--wait", type=int, default=300,
+                        help=("How long to wait after jenkins triggers the"
+                              "install-and-boot job (default=%(default)d)"))
+    return parser
+
+
+def _get_jenkins(url, username, password):
+    logging.info('Attempting to login to jenkins at %s', url)
+    if username is not None:
+        logging.info('...with authentication as user: %s', username)
+        instance = jenkins.Jenkins(url, username=username, password=password)
+    else:
+        logging.info('...without authentication')
+        instance = jenkins.Jenkins(url)
+
+    return instance
+
+
+def _get_environment():
+    base = os.path.join(os.path.dirname(__file__), 'templates')
+    return jinja2.Environment(
+        loader=jinja2.FileSystemLoader(base),
+        undefined=jinja2.StrictUndefined,
+        trim_blocks=True)
+
+
+def _publish(instance, env, args, template, jobname, **params):
+    tmpl = env.get_template(template)
+    cfg = tmpl.render(**params)
+    if args.dryrun:
+        _dryrun_func(jobname, cfg)
+        return
+    if instance.job_exists(jobname):
+        logging.info('reconfiguring job %s', jobname)
+        instance.reconfig_job(jobname, cfg)
+    else:
+        logging.info('creating job %s', jobname)
+        instance.create_job(jobname, cfg)
+
+
+def _configure_smoke(instance, env, args, config_item, device):
+    defserial = '$(${BZRDIR}/scripts/get-adb-id ${NODE_NAME})'
+
+    params = {
+        'name': device['slave-label'],
+        'device_type': device['name'],
+        'serial': device.get('serial', defserial),
+        'publish': args.publish,
+        'branch': args.branch,
+        'imagetype': config_item['image-type'],
+        'series': args.series,
+        'image_opt': config_item.get('IMAGE_OPT', ''),
+        'statsd_key': config_item.get('statsd-key', ''),
+        'dashboard_host': config_item.get('dashboard-host', ''),
+        'dashboard_port': config_item.get('dashboard-port', ''),
+        'dashboard_prefix': config_item.get('dashboard-prefix', ''),
+        'dashboard_user': config_item.get('dashboard-user', ''),
+        'dashboard_key': args.dashboard_key,
+    }
+
+    prefix = ""
+    if(args.prefix):
+        prefix = args.prefix + "-"
+
+    job = '{}{}-{}-{}-smoke-daily'.format(
+        prefix, args.series, config_item['image-type'], device['name'])
+    _publish(instance, env, args, 'touch-smoke.xml.jinja2', job, **params)
+    return job
+
+
+def _configure_qa_job(instance, env, args, config_item, device, test):
+    defserial = '$(${BZRDIR}/scripts/get-adb-id ${NODE_NAME})'
+    #If the slave is specified for this test, set it
+    slave = getattr(test, 'device', device['slave-label'])
+    params = {
+        'name': slave,
+        'device_type': device['name'],
+        'serial': device.get('serial', defserial),
+        'publish': args.publish,
+        'branch': args.branch,
+        'imagetype': config_item['image-type'],
+        'series': args.series,
+        'image_opt': config_item.get('IMAGE_OPT', ''),
+        'timeout': test.timeout,
+        'test': test.name,
+    }
+    prefix = ""
+    if(args.prefix):
+        prefix = args.prefix + "-"
+    job = test.fmt.format(prefix=prefix,
+                          series=args.series,
+                          testname=test.name,
+                          imagetype=config_item['image-type'],
+                          type=device['name'])
+    _publish(instance, env, args, 'touch-base.xml.jinja2', job, **params)
+    return job
+
+
+def _configure_qa_jobs(instance, env, args, config_item, device):
+    tests = list(testconfig.TESTSUITES)
+    tests = testconfig.filter_tests(tests, config_item['image-type'],
+                                    device['name'])
+    tests = [t for t in tests if not t.smoke]
+    jobs = []
+    for t in tests:
+        j = _configure_qa_job(instance, env, args, config_item, device, t)
+        jobs.append(j)
+    return jobs
+
+
+def _configure_master(instance, env, args, config_item, device, smoke, jobs):
+    params = {
+        'branch': args.branch,
+        'trigger_url': device.get('trigger_url', ''),
+        'wait': args.wait,
+        'projects': jobs,
+        'smoke_job': smoke,
+        'num_workers': device.get('num-workers', 1),
+    }
+    prefix = ""
+    if(args.prefix):
+        prefix = args.prefix + "-"
+    job = testconfig.DEF_FMT.format(prefix=prefix,
+                                    series=args.series,
+                                    testname='master',
+                                    imagetype=config_item['image-type'],
+                                    type=device['name'])
+    _publish(instance, env, args, 'touch-master.xml.jinja2', job, **params)
+    return job
+
+
+def _dryrun_func(jobname, config):
+    logging.debug(jobname)
+    logging.debug(config)
+
+
+def main():
+    logging.basicConfig(level=logging.DEBUG)
+    args = _get_parser().parse_args()
+
+    config = imp.load_source('', 'config.py', args.config)
+    if args.series not in config.MATRIX:
+        print('"%s" series is not supported by this config.' % args.series)
+        exit(1)
+
+    jenkins_inst = _get_jenkins(config.JENKINS, args.username, args.password)
+    if args.dryrun:
+        jenkins_inst.create_job = _dryrun_func
+        jenkins_inst.reconfig_job = _dryrun_func
+
+    env = _get_environment()
+
+    for item in config.MATRIX[args.series]:
+        for device in item['devices']:
+            job = _configure_smoke(jenkins_inst, env, args, item, device)
+            jobs = []
+            if item.get('include-qa'):
+                jobs = _configure_qa_jobs(
+                    jenkins_inst, env, args, item, device)
+            _configure_master(jenkins_inst, env, args, item, device, job, jobs)
+
+if __name__ == '__main__':
+    main()
 
=== added file 'jenkins/staging.py'
--- jenkins/staging.py	1970-01-01 00:00:00 +0000
+++ jenkins/staging.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,42 @@
+# The configuration matrix of our staging device testing
+
+JENKINS = 'http://dev-jenkins.ubuntu-ci:8080/'
+
+UTOPIC_MATRIX = [
+    {
+        'image-type': 'touch',
+        'statsd-key': 'ubuntu-ci.daily-image.staging',
+        'include-qa': True,
+        'dashboard-host': 'dashboard.ubuntu-ci',
+        'dashboard-port': '8080',
+        'dashboard-user': 'ci-bot',
+        'devices': [
+            {
+                'name': 'mako',
+                'slave-label': 'mako',
+                #'trigger_url': 'http://system-image.ubuntu.com/utopic-proposed/mako/index.json',
+            }
+        ],
+    },
+    {
+        'image-type': 'touch_stable',
+        'statsd-key': 'ubuntu-ci.daily-image.staging',
+        'include-qa': True,
+        'dashboard-host': 'dashboard.ubuntu-ci',
+        'dashboard-port': '8080',
+        'dashboard-user': 'ci-bot',
+        'devices': [
+            {
+                'name': 'mako',
+                'slave-label': 'mako',
+                #'trigger_url': 'http://system-image.ubuntu.com/utopic-proposed/mako/index.json',
+            }
+        ],
+        'IMAGE_OPT': 'export IMAGE_OPT="--bootstrap --developer-mode '
+                     '--channel ubuntu-touch/staging-stable-proposed"'
+    },
+]
+
+MATRIX = {
+    'utopic': UTOPIC_MATRIX,
+}
 
=== added directory 'jenkins/templates'
=== added file 'jenkins/templates/touch-base.xml.jinja2'
--- jenkins/templates/touch-base.xml.jinja2	1970-01-01 00:00:00 +0000
+++ jenkins/templates/touch-base.xml.jinja2	2014-09-10 13:48:46 +0000
@@ -0,0 +1,97 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<project>
+  <actions/>
+  <description>
+&lt;pre&gt;
+#NOTE: Automatically created from a script as part of daily smoke testing&#xd;
+  {{branch}}&#xd;
+&lt;/pre&gt;
+  </description>
+  <keepDependencies>false</keepDependencies>
+  <properties>
+    <hudson.model.ParametersDefinitionProperty>
+      <parameterDefinitions>
+        <hudson.model.StringParameterDefinition>
+          <name>INSTALL_URL</name>
+          <description>A URL to the previous job. If provided this job will use the same install options as it used. If the device executing the job happens to have the exact same image, then provisioning can be skipped.
+	  </description>
+          <defaultValue></defaultValue>
+        </hudson.model.StringParameterDefinition>
+      </parameterDefinitions>
+    </hudson.model.ParametersDefinitionProperty>
+    <com.sonyericsson.rebuild.RebuildSettings>
+      <autoRebuild>false</autoRebuild>
+    </com.sonyericsson.rebuild.RebuildSettings>
+  </properties>
+  <scm class="hudson.scm.NullSCM"/>
+  <assignedNode>{{ name }}</assignedNode>
+  <canRoam>false</canRoam>
+  <disabled>false</disabled>
+  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
+  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
+  <concurrentBuild>true</concurrentBuild>
+  <builders>
+    <hudson.tasks.Shell>
+      <command><![CDATA[
+set -e
+sudo rm -rf *
+
+BRANCH="{{branch}}"
+BZRDIR=`echo "$BRANCH" | awk -F/ '{ print $(NF) }'`
+BZRDIR=$(readlink -f $BZRDIR)
+bzr branch ${BRANCH} ${BZRDIR}
+
+export ANDROID_SERIAL={{serial}}
+export IMAGE_TYPE={{imagetype}}
+export IMAGE_SERIES={{series}}
+export DEVICE_TYPE={{device_type}}
+
+${BZRDIR}/scripts/recover.py ${NODE_NAME}
+
+{{image_opt}}
+${BZRDIR}/scripts/run-smoke -t {{test}}
+
+# move results to base directory as expected by dashboard:
+mv clientlogs/{{test}}/* ./clientlogs/
+rm -rf clientlogs/{{test}}
+
+{% if test == 'memevent' %}
+adb pull /tmp/memory_usage.json ./clientlogs/
+{% endif %}
+      ]]></command>
+    </hudson.tasks.Shell>
+  </builders>
+  <publishers>
+    <hudson.tasks.ArtifactArchiver>
+      <artifacts>clientlogs/*, clientlogs/**/*</artifacts>
+      <latestOnly>false</latestOnly>
+    </hudson.tasks.ArtifactArchiver>
+    <hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
+      <regexp>^= TOUCH IMAGE VERSION:([0-9]+.*)</regexp>
+      <regexpForFailed>^= TOUCH IMAGE VERSION:([0-9]+.*)</regexpForFailed>
+      <setForMatrix>false</setForMatrix>
+    </hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
+    <hudson.tasks.Mailer>
+      <recipients>[e-mail] [e-mail]</recipients>
+      <dontNotifyEveryUnstableBuild>false</dontNotifyEveryUnstableBuild>
+      <sendToIndividuals>false</sendToIndividuals>
+    </hudson.tasks.Mailer>
+{% if publish %}
+    <hudson.plugins.build__publisher.BuildPublisher>
+      <publishUnstableBuilds>true</publishUnstableBuilds>
+      <publishFailedBuilds>true</publishFailedBuilds>
+      <postActions class="vector"/>
+    </hudson.plugins.build__publisher.BuildPublisher>
+{% endif %}
+  </publishers>
+  <buildWrappers>
+      <hudson.plugins.build__timeout.BuildTimeoutWrapper>
+        <timeoutMinutes>{{timeout}}</timeoutMinutes>
+      <failBuild>true</failBuild>
+      <writingDescription>false</writingDescription>
+      <timeoutPercentage>0</timeoutPercentage>
+      <timeoutType>absolute</timeoutType>
+      <timeoutMinutesElasticDefault>3</timeoutMinutesElasticDefault>
+    </hudson.plugins.build__timeout.BuildTimeoutWrapper>
+  </buildWrappers>
+</project>
 
=== added file 'jenkins/templates/touch-master.xml.jinja2'
--- jenkins/templates/touch-master.xml.jinja2	1970-01-01 00:00:00 +0000
+++ jenkins/templates/touch-master.xml.jinja2	2014-09-10 13:48:46 +0000
@@ -0,0 +1,80 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<com.cloudbees.plugins.flow.BuildFlow>
+  <actions/>
+  <description>
+&lt;pre&gt;
+#NOTE: Automatically created from a script as part of daily smoke testing&#xd;
+  {{branch}}&#xd;
+&lt;/pre&gt;
+  </description>
+  <keepDependencies>false</keepDependencies>
+  <properties>
+    <hudson.model.ParametersDefinitionProperty>
+      <parameterDefinitions>
+        <hudson.model.StringParameterDefinition>
+          <name>sleep</name>
+          <description>Seconds to sleep before starting the job
+          </description>
+          <defaultValue>{{ wait }}</defaultValue>
+        </hudson.model.StringParameterDefinition>
+      </parameterDefinitions>
+    </hudson.model.ParametersDefinitionProperty>
+    <hudson.plugins.throttleconcurrents.ThrottleJobProperty>
+      <maxConcurrentPerNode>0</maxConcurrentPerNode>
+      <maxConcurrentTotal>0</maxConcurrentTotal>
+      <throttleEnabled>false</throttleEnabled>
+      <throttleOption>project</throttleOption>
+    </hudson.plugins.throttleconcurrents.ThrottleJobProperty>
+    <hudson.plugins.build__publisher.ExternalProjectProperty/>
+  </properties>
+  <scm class="hudson.scm.NullSCM"/>
+  <assignedNode>master</assignedNode>
+  <canRoam>false</canRoam>
+  <disabled>false</disabled>
+  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
+  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
+{%if trigger_url %}
+  <triggers class="vector">
+    <com.redfin.hudson.UrlChangeTrigger>
+      <spec></spec>
+      <url>{{trigger_url}}</url>
+    </com.redfin.hudson.UrlChangeTrigger>
+  </triggers>
+{% endif %}
+  <concurrentBuild>false</concurrentBuild>
+  <builders/>
+  <publishers/>
+  <buildWrappers>
+      <hudson.plugins.build__timeout.BuildTimeoutWrapper>
+        <timeoutMinutes>300</timeoutMinutes>
+      <failBuild>true</failBuild>
+      <writingDescription>false</writingDescription>
+      <timeoutPercentage>0</timeoutPercentage>
+      <timeoutType>absolute</timeoutType>
+      <timeoutMinutesElasticDefault>3</timeoutMinutesElasticDefault>
+    </hudson.plugins.build__timeout.BuildTimeoutWrapper>
+  </buildWrappers>
+  <dsl><![CDATA[
+// give the image time to show up before running tests
+out.println("sleeping for a bit")
+def sleep = build.environment.get("sleep").toLong()
+build.sleep(sleep * 1000)
+
+ignore(UNSTABLE) {
+    join = parallel ([
+{% for x in range(num_workers) %}
+        worker_{{x}}:  { build("{{smoke_job}}", worker_idx: {{x}}, workers: {{num_workers}}) },
+{% endfor %}
+    ])
+}
+{% if projects %}
+out.println("kicking off downstream projects in parellel")
+install_url = build.environment.get("HUDSON_URL") + join.worker_0.lastBuild.build.url
+parallel (
+{% for project in projects %}
+    {build("{{project}}", INSTALL_URL: install_url)},
+{% endfor %}
+)
+{% endif %}
+  ]]></dsl>
+</com.cloudbees.plugins.flow.BuildFlow>
 
=== added file 'jenkins/templates/touch-smoke.xml.jinja2'
--- jenkins/templates/touch-smoke.xml.jinja2	1970-01-01 00:00:00 +0000
+++ jenkins/templates/touch-smoke.xml.jinja2	2014-09-10 13:48:46 +0000
@@ -0,0 +1,178 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<project>
+  <actions/>
+  <description>
+<![CDATA[
+This job provides a flexible way to execute all the tests in our daily image
+testing process that contribute to the
+<a href="http://reports.qa.ubuntu.com/smokeng/">QA Dashboard</a>.
+The job is parameterized to give flexibility in what gets tested. A couple of
+common ways to run this job are:
+<dl>
+  <dt>Full Test Run</dt>
+  <dd>TESTS=ALL</dd>
+  <dd>APPS=ALL</dd>
+  <dt>Re-run a Failed Autopilot Test</dt>
+  <dd>INSTALL_URL=http://dev-jenkins:8080/job/utopic-touch_ro-mako/9</dd>
+  <dd>APPS=dropping_letters_app</dd>
+  <dt>Re-run a Failed UTAH Test</dt>
+  <dd>INSTALL_URL=http://dev-jenkins:8080/job/utopic-touch_ro-mako/9</dd>
+  <dd>TESTS=security</dd>
+</dl>
+<pre>
+#NOTE: Automatically created from a script as part of daily smoke testing
+  {{branch}}
+</pre>
+{% if statsd_key %}
+<h3>Graphite Reports</h3>
+<ul>
+  <li><a href="https://graphite.engineering.canonical.com/graphlot/?width=841&height=770&&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.provision.mean%2C0.000016667%29%2C%22Provisioning%20Time(minutes)%22%29&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.test_autopilot.mean%2C0.000016667%29%2C%22Autopilot%20Tests%20Time%22%29&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.test_utah.mean%2C0.000016667%29%2C%22UTAH%20Tests%20Time%22%29&from=-1weeks">Timings (1 week)</a></li>
+  <li><a href="https://graphite.engineering.canonical.com/graphlot/?width=841&height=770&&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.provision.mean%2C0.000016667%29%2C%22Provisioning%20Time(minutes)%22%29&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.test_autopilot.mean%2C0.000016667%29%2C%22Autopilot%20Tests%20Time%22%29&target=alias%28scale%28statsd.{{statsd_key}}.{{imagetype}}.test_utah.mean%2C0.000016667%29%2C%22UTAH%20Tests%20Time%22%29&from=-4weeks">Timings (4 weeks)</a></li>
+</ul>
+{% endif %}
+]]>
+  </description>
+  <keepDependencies>false</keepDependencies>
+  <properties>
+    <hudson.model.ParametersDefinitionProperty>
+      <parameterDefinitions>
+        <hudson.model.StringParameterDefinition>
+          <name>INSTALL_URL</name>
+          <description>A URL to the previous job. If provided this job will use the same install options as it used. If the device executing the job happens to have the exact same image, then provisioning can be skipped.
+	  </description>
+          <defaultValue></defaultValue>
+        </hudson.model.StringParameterDefinition>
+        <hudson.model.StringParameterDefinition>
+          <name>TESTS</name>
+          <description>A space separated list of utah tests to run. "ALL" can be used to run all known utah tests.
+	  </description>
+          <defaultValue>ALL</defaultValue>
+        </hudson.model.StringParameterDefinition>
+	<hudson.model.StringParameterDefinition>
+          <name>APPS</name>
+          <description>A space separated list of autopilot tests to run. "ALL" can be used to run all known tests.
+	  </description>
+          <defaultValue>ALL</defaultValue>
+        </hudson.model.StringParameterDefinition>
+	<hudson.model.StringParameterDefinition>
+          <name>REVISION</name>
+          <description>The image revision to test with.
+	  </description>
+          <defaultValue></defaultValue>
+        </hudson.model.StringParameterDefinition>
+	<hudson.model.StringParameterDefinition>
+          <name>workers</name>
+          <description>The number of workers
+	  </description>
+          <defaultValue></defaultValue>
+        </hudson.model.StringParameterDefinition>
+	<hudson.model.StringParameterDefinition>
+          <name>worker_idx</name>
+          <description>The index of this worker
+	  </description>
+          <defaultValue></defaultValue>
+        </hudson.model.StringParameterDefinition>
+      </parameterDefinitions>
+    </hudson.model.ParametersDefinitionProperty>
+    <com.sonyericsson.rebuild.RebuildSettings>
+      <autoRebuild>false</autoRebuild>
+    </com.sonyericsson.rebuild.RebuildSettings>
+  </properties>
+  <scm class="hudson.scm.NullSCM"/>
+  <assignedNode>{{ name }}</assignedNode>
+  <canRoam>false</canRoam>
+  <disabled>false</disabled>
+  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
+  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
+  <triggers class="vector"/>
+  <concurrentBuild>true</concurrentBuild>
+  <builders>
+    <hudson.tasks.Shell>
+      <command>set -e
+BRANCH=&quot;{{branch}}&quot;
+BZRDIR=`echo &quot;$BRANCH&quot; | awk -F/ &apos;{ print $(NF) }&apos;`
+BZRDIR=$(readlink -f $BZRDIR)
+[ -d $BZRDIR ] &amp;&amp; rm -rf $BZRDIR
+bzr branch ${BRANCH} ${BZRDIR}
+
+export ANDROID_SERIAL={{serial}}
+export IMAGE_TYPE={{imagetype}}
+export IMAGE_SERIES={{series}}
+export DEVICE_TYPE={{device_type}}
+
+${BZRDIR}/scripts/recover.py ${NODE_NAME}
+
+{% if statsd_key %}
+# the txstatsd package is too old, use the one from LP:
+[ -d txstatsd ] &amp;&amp; rm -rf txstatsd
+bzr branch -r 109 lp:txstatsd
+export PYTHONPATH=`pwd`/txstatsd
+
+export STATSD_KEY={{statsd_key}}.{{imagetype}}
+{% endif %}
+
+{% if dashboard_host %}
+export DASHBOARD_HOST="{{dashboard_host}}"
+{% endif %}
+{% if dashboard_user %}
+export DASHBOARD_USER="{{dashboard_user}}"
+{% endif %}
+{% if dashboard_key %}
+set +x   # don't let this leak into the public
+export DASHBOARD_KEY="{{dashboard_key}}"
+set -x
+{% endif %}
+{% if dashboard_prefix %}
+export DASHBOARD_PREFIX="{{dashboard_prefix}}"
+{% endif %}
+{% if dashboard_port %}
+export DASHBOARD_PORT="{{dashboard_port}}"
+{% endif %}
+
+{{image_opt}}
+${BZRDIR}/scripts/run-smoke
+      </command>
+    </hudson.tasks.Shell>
+  </builders>
+  <publishers>
+    <hudson.tasks.ArtifactArchiver>
+      <artifacts>clientlogs/**</artifacts>
+      <latestOnly>false</latestOnly>
+    </hudson.tasks.ArtifactArchiver>
+    <hudson.tasks.junit.JUnitResultArchiver>
+      <testResults>clientlogs/**/*.xml</testResults>
+      <keepLongStdio>true</keepLongStdio>
+      <testDataPublishers>
+        <hudson.plugins.junitattachments.AttachmentPublisher/>
+      </testDataPublishers>
+    </hudson.tasks.junit.JUnitResultArchiver>
+    <hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
+      <regexp>^= TOUCH IMAGE VERSION:([0-9]+.*)</regexp>
+      <regexpForFailed>^= TOUCH IMAGE VERSION:([0-9]+.*)</regexpForFailed>
+      <setForMatrix>false</setForMatrix>
+    </hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
+{% if publish %}
+    <hudson.tasks.Mailer>
+      <recipients>[e-mail]</recipients>
+      <dontNotifyEveryUnstableBuild>false</dontNotifyEveryUnstableBuild>
+      <sendToIndividuals>false</sendToIndividuals>
+    </hudson.tasks.Mailer>
+    <hudson.plugins.build__publisher.BuildPublisher>
+      <publishUnstableBuilds>true</publishUnstableBuilds>
+      <publishFailedBuilds>true</publishFailedBuilds>
+      <postActions class="vector"/>
+    </hudson.plugins.build__publisher.BuildPublisher>
+{% endif %}
+  </publishers>
+  <buildWrappers>
+      <hudson.plugins.build__timeout.BuildTimeoutWrapper>
+          <timeoutMinutes>300</timeoutMinutes>
+      <failBuild>true</failBuild>
+      <writingDescription>false</writingDescription>
+      <timeoutPercentage>0</timeoutPercentage>
+      <timeoutType>absolute</timeoutType>
+      <timeoutMinutesElasticDefault>3</timeoutMinutesElasticDefault>
+    </hudson.plugins.build__timeout.BuildTimeoutWrapper>
+  </buildWrappers>
+</project>
+
 
=== added file 'jenkins/testconfig.py'
--- jenkins/testconfig.py	1970-01-01 00:00:00 +0000
+++ jenkins/testconfig.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,207 @@
+#!/usr/bin/env python
+
+# Ubuntu Testing Automation Harness
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import argparse
+
+DEF_FMT = '{prefix}{series}-{imagetype}-{type}-smoke-{testname}'
+IDLE_FMT = '{prefix}{testname}-{series}-{imagetype}-armhf-install-idle-{type}'
+
+
+class Test(object):
+    def __init__(self, name, timeout=60, fmt=DEF_FMT, smoke=True):
+        self.name = name
+        self.timeout = timeout
+        self.fmt = fmt
+        self.smoke = smoke
+
+
+class DevTest(Test):
+    def __init__(self, name, device):
+        Test.__init__(self, name, fmt=IDLE_FMT, smoke=False)
+        self.device = device
+
+
+class APTest(Test):
+    def __init__(self, name, app=None, pkgs=None):
+        Test.__init__(self, name)
+        self.pkgs = pkgs
+        if not app:
+            # convert share-app-autopilot to share_app
+            app = name.replace('-', '_').replace('_autopilot', '')
+        self.app = app
+
+
+TESTSUITES = [
+    Test('default'),
+]
+TESTSUITES += [
+    APTest('mediaplayer-app-autopilot', pkgs=['mediaplayer-app-autopilot']),
+    APTest('gallery-app-autopilot'),
+    APTest('webbrowser-app-autopilot', pkgs=['webbrowser-app-autopilot']),
+    APTest('unity8-autopilot', 'unity8'),
+    APTest('camera-app-autopilot'),
+    APTest('dialer-app-autopilot', pkgs=['dialer-app-autopilot']),
+    APTest('messaging-app-autopilot', pkgs=['messaging-app-autopilot']),
+    APTest('address-book-app-autopilot', pkgs=['address-book-app-autopilot']),
+    APTest('calendar-app-autopilot', pkgs=['address-book-service-dummy',  'address-book-service-testability']),
+    APTest('reminders-autopilot', pkgs=['python3-dbusmock', 'python3-requests-oauthlib', 'account-plugin-evernote-sandbox']),
+    APTest('music-app-autopilot'),
+    APTest('dropping-letters-app-autopilot'),
+    APTest('sudoku-app-autopilot'),
+    APTest('ubuntu-calculator-app-autopilot'),
+    APTest('ubuntu-clock-app-autopilot'),
+    APTest('filemanager', pkgs=['python3-lxml']),
+    APTest('shorts-app-autopilot'),
+    APTest('ubuntu-terminal-app-autopilot'),
+    APTest('ubuntu-weather-app-autopilot'),
+    APTest('ubuntu-ui-toolkit-autopilot', 'ubuntuuitoolkit',
+           ['ubuntu-ui-toolkit-autopilot']),
+    APTest('ubuntu-system-settings-autopilot',
+           pkgs=['ubuntu-system-settings-autopilot']),
+    APTest('ubuntu-system-settings-online-accounts-autopilot',
+           'online_accounts_ui',
+           ['ubuntu-system-settings-online-accounts-autopilot']),
+]
+TESTSUITES += [
+    Test('click_image_tests'),
+    Test('sdk'),
+    Test('security'),
+    Test('eventstat', fmt=IDLE_FMT, smoke=False),
+    Test('smem', fmt=IDLE_FMT, smoke=False),
+    Test('health-check', timeout=300, smoke=False),
+    Test('memevent',
+         fmt='{prefix}{testname}-{series}-{imagetype}-armhf-default-{type}',
+         smoke=False),
+    Test('bootspeed',
+         fmt='{prefix}{testname}-{series}-{imagetype}-{type}-{type}',
+         smoke=False),
+]
+
+
+def filter_tests(tests, image_type, device_type=None):
+    if image_type:
+        func = globals().get('get_tests_%s' % image_type)
+        if func:
+            tests = func(tests)
+        elif image_type not in ['touch_stable', 'touch',
+                                'touch_custom_demo']:
+            print('Unsupported image type: %s' % image_type)
+            exit(1)
+    if device_type:
+        func = globals().get('get_tests_%s' % device_type)
+        if func:
+            tests = func(tests)
+    return tests
+
+
+def _get_tests(test_type, image_type):
+    # take all our known tests, then call filter tests which should give
+    # us a list of tests customized for things like touch_custom.
+    # then eliminate tests that aren't of the proper test_type
+    tests = [t for t in filter_tests(TESTSUITES, image_type)
+             if type(t) == test_type and t.fmt == DEF_FMT]
+    return tests
+
+
+def _split_work(tests, total_workers, worker_idx):
+    assigned = []
+    for x in range(len(tests)):
+        if x % total_workers == worker_idx:
+            assigned.append(tests[x])
+    return assigned
+
+
+def _handle_utah(args):
+    tests = _get_tests(Test, args.image_type)
+    if args.with_autopilot:
+        tests = [t for t in TESTSUITES if t.fmt == DEF_FMT]
+    tests = _split_work(tests, args.total_workers, args.worker)
+    print(' '.join([t.name for t in tests if t.smoke is True]))
+
+
+def _handle_ap_apps(args):
+    tests = _get_tests(APTest, args.image_type)
+    tests = _split_work(tests, args.total_workers, args.worker)
+    print(' '.join([t.app for t in tests]))
+
+
+def _handle_ap_packages(args):
+    pkgs = []
+    tests = _get_tests(APTest, args.image_type)
+    for test in tests:
+        if not args.app or test.app in args.app:
+            if test.pkgs:
+                pkgs.extend(test.pkgs)
+    print(' '.join(pkgs))
+
+
+def get_tests_mako(common_tests):
+    tests = common_tests
+    tests.extend([DevTest('suspend-blocker', 'mako-01')])
+    return tests
+
+
+def get_tests_touch_custom(common_tests):
+    tests = common_tests
+    tests.insert(1, Test('customizations'))
+    return tests
+
+
+def _get_parser():
+    parser = argparse.ArgumentParser(
+        description='List information on configured tests for touch')
+    sub = parser.add_subparsers(title='Commands', metavar='')
+
+    p = sub.add_parser('utah', help='List UTAH tests')
+    p.set_defaults(func=_handle_utah)
+    p.add_argument('-i', '--image-type',
+                   help='Return list of test configured for an image type.')
+    p.add_argument('-a', '--with-autopilot', action='store_true',
+                   help='Include autopilot tests that can be run under UTAH.')
+    p.add_argument('-t', '--total-workers', type=int, default=1,
+                   help='''The total number of workers available for running
+                        tests.''')
+    p.add_argument('-w', '--worker', type=int, default=0,
+                   help='The worker to allocate applications for testing to.')
+
+    p = sub.add_parser('apps', help='List autopilot application names')
+    p.set_defaults(func=_handle_ap_apps)
+    p.add_argument('-i', '--image-type',
+                   help='Return list of test configured for an image type.')
+    p.add_argument('-t', '--total-workers', type=int, default=1,
+                   help='''The total number of workers available for running
+                        tests.''')
+    p.add_argument('-w', '--worker', type=int, default=0,
+                   help='The worker to allocate applications for testing to.')
+
+    p = sub.add_parser('packages', help='List packages required for autopilot')
+    p.set_defaults(func=_handle_ap_packages)
+    g = p.add_mutually_exclusive_group()
+    g.add_argument('-i', '--image-type',
+                   help='''If no apps are listed, limit to tests for an
+                        image type.''')
+    g.add_argument('-a', '--app', action='append',
+                   help='Autopilot test application. eg share_app')
+    return parser
+
+
+def main():
+    args = _get_parser().parse_args()
+    return args.func(args)
+
+if __name__ == '__main__':
+    exit(main())
 
=== added directory 'scripts'
=== renamed directory 'scripts' => 'scripts.moved'
=== added file 'scripts/assert-image'
--- scripts/assert-image	1970-01-01 00:00:00 +0000
+++ scripts/assert-image	2014-09-10 13:48:46 +0000
@@ -0,0 +1,30 @@
+#!/bin/sh
+
+set -e
+
+BASEDIR=$(readlink -f $(dirname $0)/..)
+
+# a simple script to ensure the proper image is installed on the target
+
+[ -z $INSTALL_URL ] && echo "= INSTALL_URL not set. Using target as-is." && exit 0
+
+echo "Ensuring target has proper image..."
+REQUIRED_UUID=$(curl ${INSTALL_URL}/artifact/clientlogs/.ci-uuid)
+ACTUAL_UUID=$(adb shell "cat /home/phablet/.ci-uuid | tr -d '\r\n'")
+if [ "$REQUIRED_UUID" != "$ACTUAL_UUID" ] ; then
+	echo "= UUIDs $REQUIRED_UUID != $ACTUAL_UUID"
+	ARGS=$(curl ${INSTALL_URL}/artifact/clientlogs/.ci-flash-args | tr -d '\r\n')
+	CUST=$(curl ${INSTALL_URL}/artifact/clientlogs/.ci-customizations | tr -d '\r\n')
+	echo "reprovisioning device with: $ARGS"
+	echo "customizing device with: $CUST"
+        #Make the image writable if we reprovision by adding a space
+        #after to CUSTOMIZE so it is never empty
+	UUID=$REQUIRED_UUID IMAGE_OPT=$ARGS CUSTOMIZE="$CUST " \
+		${BASEDIR}/scripts/provision.sh
+else
+	echo "= UUIDS match, reusing image on target"
+	echo $REQUIRED_UUID > clientlogs/.ci-uuid
+	curl ${INSTALL_URL}/artifact/clientlogs/.ci-flash-args > clientlogs/.ci-flash-args
+	curl ${INSTALL_URL}/artifact/clientlogs/.ci-customizations > clientlogs/.ci-customizations
+fi
+
 
=== added file 'scripts/combine_results'
--- scripts/combine_results	1970-01-01 00:00:00 +0000
+++ scripts/combine_results	2014-09-10 13:48:46 +0000
@@ -0,0 +1,124 @@
+#!/usr/bin/env python3
+
+"""
+We always run the system-settle test before and after an autopilot test. This
+script takes the results of the before/after results and combines them in with
+the junit xml results from the autopilot test so we have one unified report.
+"""
+
+
+import os
+import sys
+
+from xml.etree import ElementTree
+
+
+PRE_COMBINE = [
+    ('settle_before', 'settle_before'),
+    ('setup_setup', 'setup'),
+]
+
+POST_COMBINE = [
+    ('settle_after', 'settle_after'),
+    ('setup_teardown', 'teardown'),
+]
+
+
+def _build_node(classname, name, rcfile, stdout):
+    e = ElementTree.Element('testcase')
+    e.attrib['classname'] = classname
+    e.attrib['name'] = name
+
+    if not os.path.exists(rcfile):
+        return None, False
+
+    rc = int(open(rcfile).read())
+    if rc != 0:
+        f = ElementTree.Element('failure')
+        e.append(f)
+        f.attrib['type'] = 'testtools.testresult.real._StringException'
+        f.text = open(stdout).read()
+    return e, rc != 0
+
+
+def _get_results(apfile):
+    try:
+        tree = ElementTree.parse(apfile)
+    except Exception as ex:
+        e = ElementTree.Element('testsuite')
+        tree = ElementTree.ElementTree(e)
+        e.attrib['errors'] = '1'
+        e.attrib['failures'] = '0'
+        e.attrib['tests'] = '1'
+
+        # make a guess at the classname:
+        classname = os.path.basename(os.path.dirname(apfile))
+
+        t = ElementTree.Element('testcase')
+        e.append(t)
+        t.attrib['classname'] = classname
+        t.attrib['name'] = 'phablet-test-run'
+
+        f = ElementTree.Element('failure')
+        t.append(f)
+        f.attrib['type'] = 'testtools.testresult.real._StringException'
+        f.text = str(ex)
+
+    return tree
+
+
+def _get_classname(results):
+    if len(results) < 1:
+        return '???'
+
+    cname = results[0].attrib.get('classname')
+    if cname:
+        cname = cname.split('.')[0]
+    else:
+        cname = '???'
+    return cname
+
+
+def combine(resdir):
+    ap_file = os.path.join(resdir, 'test_results.xml')
+    tree = _get_results(ap_file)
+    ap_results = tree.getroot()
+    added_results = 0
+
+    errors = int(ap_results.attrib['errors'])
+
+    classname = _get_classname(ap_results)
+
+    for basename, label in PRE_COMBINE:
+        rc = os.path.join(resdir, basename + '.rc')
+        log = os.path.join(resdir, basename + '.log')
+        node, failed = _build_node(classname, label, rc, log)
+        if node is not None:
+            ap_results.insert(0, node)
+            if failed:
+                errors += 1
+            added_results += 1
+
+    for basename, label in POST_COMBINE:
+        rc = os.path.join(resdir, basename + '.rc')
+        log = os.path.join(resdir, basename + '.log')
+        node, failed = _build_node(classname, label, rc, log)
+        if node is not None:
+            ap_results.append(node)
+            if failed:
+                errors += 1
+            added_results += 1
+
+    num = int(ap_results.attrib['tests']) + added_results
+    ap_results.attrib['tests'] = str(num)
+    ap_results.attrib['errors'] = str(errors)
+
+    tree.write(ap_file)
+
+
+if __name__ == '__main__':
+    if len(sys.argv) != 2:
+        print('usage: {} <results directory>'.format(sys.argv[0]))
+        sys.exit(1)
+
+    combine(sys.argv[1])
 
=== added file 'scripts/dashboard.py'
--- scripts/dashboard.py	1970-01-01 00:00:00 +0000
+++ scripts/dashboard.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,305 @@
+#!/usr/bin/python
+
+import argparse
+import datetime
+import json
+import logging
+import os
+
+import yaml
+
+from httplib import ACCEPTED, HTTPConnection, HTTPException, OK, CREATED
+from urllib import urlencode
+from urlparse import urlparse
+
+log = logging.getLogger()
+
+
+class API(object):
+    def __init__(self, host=None, port=None, user=None, key=None, prefix=None):
+        if not host:
+            host = os.environ.get('DASHBOARD_HOST', None)
+        if not port:
+            port = int(os.environ.get('DASHBOARD_PORT', '80'))
+        if not user:
+            user = os.environ.get('DASHBOARD_USER', None)
+        if not key:
+            key = os.environ.get('DASHBOARD_KEY', None)
+        if not prefix:
+            prefix = os.environ.get('DASHBOARD_PREFIX', None)
+
+        self.host = host
+        self.port = port
+        self.resource_base = prefix
+
+        self._headers = None
+        if user and key:
+            self._headers = {
+                'Content-Type': 'application/json',
+                'Authorization': 'ApiKey %s:%s' % (user, key)
+            }
+            # mod_wsgi will strip the Authorization header, but tastypie
+            # allows it as GET param also. More details for fixing apache:
+            #  http://django-tastypie.rtfd.org/en/latest/authentication.html
+            self._auth_param = '?username=%s&api_key=%s' % (user, key)
+
+    def _connect(self):
+        if self.host:
+            return HTTPConnection(self.host, self.port)
+        return None
+
+    def _http_get(self, resource):
+        con = self._connect()
+        if not con:
+            # we just mock this for the case where the caller wants to
+            # use our API transparently enabled/disabled
+            return {}
+
+        if self.resource_base:
+            resource = self.resource_base + resource
+
+        logging.debug('doing get on: %s', resource)
+        headers = {'Content-Type': 'application/json'}
+        con.request('GET', resource, headers=headers)
+        resp = con.getresponse()
+        if resp.status != OK:
+            msg = ''
+            try:
+                msg = resp.read().decode()
+            except:
+                pass
+            fmt = '%d error getting resource(%s): %s'
+            raise HTTPException(fmt % (resp.status, resource, msg))
+        data = json.loads(resp.read().decode())
+        if len(data['objects']) == 0:
+            raise HTTPException('resource not found: %s' % resource)
+        assert len(data['objects']) == 1
+        return data['objects'][0]['resource_uri']
+
+    def _http_post(self, resource, params):
+        con = self._connect()
+        if not con or not self._headers:
+            return None
+
+        if self.resource_base:
+            resource = self.resource_base + resource
+        resource += self._auth_param
+
+        params = json.dumps(params)
+        log.debug('posting (%s): %s', resource, params)
+        con.request('POST', resource, params, self._headers)
+        resp = con.getresponse()
+        if resp.status != CREATED:
+            msg = ''
+            try:
+                msg = str(resp.getheaders())
+                msg += resp.read().decode()
+            except:
+                pass
+            raise HTTPException(
+                '%d creating resource(%s): %s' % (resp.status, resource, msg))
+        uri = resp.getheader('Location')
+        return urlparse(uri).path
+
+    def _http_patch(self, resource, params):
+        con = self._connect()
+        if not con or not self._headers:
+            return None
+
+        resource += self._auth_param
+
+        con.request('PATCH', resource, json.dumps(params), self._headers)
+        resp = con.getresponse()
+        if resp.status != ACCEPTED:
+            msg = ''
+            try:
+                msg = resp.getheaders()
+            except:
+                pass
+            raise HTTPException(
+                '%d patching resource(%s): %s' % (resp.status, resource, msg))
+        return resource
+
+    @staticmethod
+    def _uri_to_pk(resource_uri):
+        if resource_uri:
+            return resource_uri.split('/')[-2]
+        return None  # we are mocked
+
+    def job_get(self, name):
+        resource = '/smokeng/api/v1/job/?' + urlencode({'name': name})
+        return self._http_get(resource)
+
+    def job_add(self, name):
+        resource = '/smokeng/api/v1/job/'
+        params = {
+            'name': name,
+            'url': 'http://jenkins.qa.ubuntu.com/job/' + name + '/'
+        }
+        return self._http_post(resource, params)
+
+    def build_add(self, job_name, job_number):
+        try:
+            logging.debug('trying to find job: %s', job_name)
+            job = self.job_get(job_name)
+        except HTTPException:
+            job = self.job_add(job_name)
+        logging.info('job is: %s', job)
+
+        resource = '/smokeng/api/v1/build/'
+        params = {
+            'build_number': job_number,
+            'job': job,
+            'ran_at': datetime.datetime.now().isoformat(),
+            'build_description': 'inprogress',
+        }
+        return self._http_post(resource, params)
+
+    def _image_get(self, build_number, release, variant, arch, flavor):
+        resource = '/smokeng/api/v1/image/?'
+        resource += urlencode({
+            'build_number': build_number,
+            'release': release,
+            'flavor': flavor,
+            'variant': variant,
+            'arch': arch,
+        })
+        return self._http_get(resource)
+
+    def image_add(self, build_number, release, variant, arch, flavor):
+        try:
+            img = self._image_get(build_number, release, variant, arch, flavor)
+            return img
+        except HTTPException:
+            # image doesn't exist so go continue and create
+            pass
+
+        resource = '/smokeng/api/v1/image/'
+        params = {
+            'build_number': build_number,
+            'release': release,
+            'flavor': flavor,
+            'variant': variant,
+            'arch': arch,
+        }
+        try:
+            return self._http_post(resource, params)
+        except HTTPException:
+            # race situation. Both callers saw _image_get fail and tried to
+            # create. Only one of them can succeed, so the failed call should
+            # now safely be able to get the image created by the other
+            img = self._image_get(build_number, release, variant, arch, flavor)
+            return img
+
+    def result_get(self, image, test):
+        # deal with getting resource uri's as parameters instead of id's
+        image = API._uri_to_pk(image)
+
+        resource = '/smokeng/api/v1/result/?'
+        resource += urlencode({
+            'image': image,
+            'name': test,
+        })
+        return self._http_get(resource)
+
+    def _result_status(self, image, build, test, status, results=None):
+        create = False
+        params = {
+            'ran_at': datetime.datetime.now().isoformat(),
+            'status': status,
+            'jenkins_build': build,
+        }
+        if results:
+            params['results'] = results
+
+        try:
+            resource = self.result_get(image, test)
+        except HTTPException:
+            create = True
+            resource = '/smokeng/api/v1/result/'
+            params['image'] = image
+            params['name'] = test
+
+        if create:
+            return self._http_post(resource, params)
+        else:
+            return self._http_patch(resource, params)
+
+    def result_queue(self, image, build, test):
+        return self._result_status(image, build, test, 0)
+
+    def result_running(self, image, build, test):
+        return self._result_status(image, build, test, 1)
+
+    def result_syncing(self, image, build, test, results):
+        return self._result_status(image, build, test, 2, results)
+
+
+def _result_running(api, args):
+    return api.result_running(args.image, args.build, args.test)
+
+
+def _result_syncing(api, args):
+    results = {}
+    with open(args.results) as f:
+        results = yaml.safe_load(f.read())
+    return api.result_syncing(args.image, args.build, args.test, results)
+
+
+def _set_args(parser, names, func):
+    for n in names:
+        parser.add_argument(n, required=True)
+    parser.set_defaults(func=func)
+
+
+def _get_parser():
+    parser = argparse.ArgumentParser(
+        description='Interact with the CI dashboard API')
+
+    sub = parser.add_subparsers(title='Commands', metavar='')
+
+    args = ['--image', '--build', '--test']
+    p = sub.add_parser('result-running', help='Set a SmokeResult "Running".')
+    _set_args(p, args, _result_running)
+
+    p = sub.add_parser('result-syncing', help='Set a SmokeResult "Syncing".')
+    _set_args(p, args, _result_syncing)
+    p.add_argument('--results', required=True, help='UTAH yaml file')
+
+    return parser
+
+
+def _assert_env():
+    required = [
+        'DASHBOARD_HOST', 'DASHBOARD_PORT', 'DASHBOARD_USER', 'DASHBOARD_KEY']
+    missing = []
+    for r in required:
+        if r not in os.environ:
+            missing.append(r)
+    if len(missing):
+        print('Missing the following environment variables:')
+        for x in missing:
+            print('  %s' % x)
+        exit(1)
+
+
+def _main(args):
+    _assert_env()
+
+    api = API()
+    try:
+        val = args.func(api, args)
+        if val:
+            if '/?' in val:
+                log.debug('stripping api-key from response')
+                val = val.split('/?')[0] + '/'
+            print(val)
+    except HTTPException as e:
+        print('ERROR: %s' % e)
+        exit(1)
+
+    exit(0)
+
+if __name__ == '__main__':
+    args = _get_parser().parse_args()
+    exit(_main(args))
 
=== added file 'scripts/device_info.py'
--- scripts/device_info.py	1970-01-01 00:00:00 +0000
+++ scripts/device_info.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,100 @@
+#!/usr/bin/env python
+
+import re
+import subprocess
+
+
+class TouchDevice(object):
+    def __init__(self, devtype, serial, relay_url=None, bank=None,
+                 power_pin=None, volume_pin=None):
+        self.devtype = devtype
+        self.serial = serial
+        self.relay_url = relay_url
+        self.bank = bank
+        self.power_pin = power_pin
+        self.volume_pin = volume_pin
+
+DEVICES = {
+    "krillin-01": TouchDevice("krillin", "JB011018"),
+    "krillin-02": TouchDevice("krillin", "JB010894"),
+    "krillin-03": TouchDevice("krillin", "JB015156"),
+    "krillin-04": TouchDevice("krillin", "JB006885"),
+    "krillin-05": TouchDevice("krillin", "JB015256"),
+    "ps-mako-01": TouchDevice("mako", "0090f741e3d141bc"),
+    "ps-mako-02": TouchDevice("mako", "04ccca120acd4dea"),
+    "ps-mako-03": TouchDevice("mako", "04cb53b598546534"),
+    "ps-mako-04": TouchDevice("mako", "04cbcc545f5328a5"),
+    "mako-01": TouchDevice("mako", "01aa3d7a5dcba4a2"),
+    "mako-02": TouchDevice("mako", "01ade38b552014d4"),
+    "mako-03": TouchDevice("mako", "04c6714ed7c863f2"),
+    "mako-04": TouchDevice("mako", "04df89cf0f9d0933"),
+    "mako-05": TouchDevice("mako", "01b22f82dc5cec63"),
+    "mako-06": TouchDevice("mako", "04ed70928fdc13ba"),
+    "mako-07": TouchDevice("mako", "01e2f64788556934"),
+    "mako-08": TouchDevice("mako", "04ea16a163930769"),
+    "mako-09": TouchDevice("mako", "04fda12ea08fe3c7"),
+    "mako-10": TouchDevice("mako", "01ce848e48dfa6a2"),
+    "mako-11": TouchDevice("mako", "04ed727c929709ba"),
+    #If looking at the LAB wiki page, subtract 1 from the bank and pin numbers
+    #from what it says on the wiki (our numbers start at 0)
+    "mako-12": TouchDevice("mako", "00693fd555c9186a",
+                           relay_url="http://qa-relay-control.ubuntu-ci",
+                           bank=0, power_pin=1, volume_pin=2),
+    "mako-13": TouchDevice("mako", "0084e99c5315731b",
+                           relay_url="http://qa-relay-control.ubuntu-ci",
+                           bank=0, power_pin=3, volume_pin=4),
+    "mako-14": TouchDevice("mako", "007c6d84d348838e",
+                           relay_url="http://qa-relay-control.ubuntu-ci",
+                           bank=0, power_pin=5, volume_pin=6),
+    "mako-15": TouchDevice("mako", "00763b4a61ce0f87",
+                           relay_url="http://qa-relay-control.ubuntu-ci",
+                           bank=1, power_pin=0, volume_pin=1),
+    "mako-16": TouchDevice("mako", "017121eacf5282c4",
+                           relay_url="http://qa-relay-control.ubuntu-ci",
+                           bank=1, power_pin=2, volume_pin=3),
+    #mako-17 has a broken screen but should work, on ashes
+    "mako-17": TouchDevice("mako", "04e0d2f6d3cab77d"),
+    "mako-18": TouchDevice("mako", "027b981a4c1110dd"),
+    "mako-19": TouchDevice("mako", "021c8cdfd5d38602"),
+    "mako-20": TouchDevice("mako", "05083705e0d29402"),
+    "ps-manta-01": TouchDevice("manta", "R32D203DDZR"),
+    "manta-01": TouchDevice("manta", "R32D102RPZL"),
+    "manta-02": TouchDevice("manta", "R32D102RPPK"),
+    "manta-03": TouchDevice("manta", "R32D200N4YH"),
+    "manta-05": TouchDevice("manta", "R32D203DMBY"),  # Out of lab for now
+    "flo-01": TouchDevice("flo", "09f306dc"),
+    "flo-02": TouchDevice("flo", "08dbee36"),
+    "flo-03": TouchDevice("flo", "09d55fa8"),
+    "flo-04": TouchDevice("flo", "09e68682"),
+    "flo-05": TouchDevice("flo", "0a22f7cf"),
+    "flo-06": TouchDevice("flo", "08f09bb0"),
+}
+
+
+def get_state(serial):
+    """
+    Check adb and fastboot to determine the state a device is in.
+    Possible return values are:
+        device, recovery, unknown, bootloader, disconnected
+    """
+    pattern = "{}\t(.+)\n".format(serial)
+    adb_devices = subprocess.check_output(['adb', 'devices'])
+    found = re.search(pattern, adb_devices)
+    if not found:
+        #Otherwise, check fastboot
+        fastboot_devices = subprocess.check_output(['fastboot', 'devices'])
+        found = re.search(pattern, fastboot_devices)
+    if found:
+        state = found.group(1)
+        return state
+    else:
+        return 'disconnected'
+
+
+def get_serial(name):
+    return DEVICES.get(name).serial
+
+
+def get_power(name):
+    device = DEVICES.get(name)
+    return (device.relay_url, device.bank, device.power_pin, device.volume_pin)
 
=== added file 'scripts/get-adb-id'
--- scripts/get-adb-id	1970-01-01 00:00:00 +0000
+++ scripts/get-adb-id	2014-09-10 13:48:46 +0000
@@ -0,0 +1,13 @@
+#!/usr/bin/python
+
+import sys
+from device_info import get_serial
+
+if __name__ == '__main__':
+    name = sys.argv[1]
+
+    try:
+        print(get_serial(name))
+    except:
+        print("Unknown device name: '%s'" % name)
+        sys.exit(-1)
 
=== added file 'scripts/get-device-info'
--- scripts/get-device-info	1970-01-01 00:00:00 +0000
+++ scripts/get-device-info	2014-09-10 13:48:46 +0000
@@ -0,0 +1,37 @@
+#!/usr/bin/env python
+
+import argparse
+import device_info
+
+
+def _get_state(args):
+    print(device_info.get_state(device_info.get_serial(args.name)))
+
+
+def _get_serial(args):
+    print(device_info.get_serial(args.name))
+
+
+def _get_power(args):
+    print(device_info.get_power(args.name))
+
+
+def _get_parser():
+    parser = argparse.ArgumentParser(
+        description='Get information about a device')
+    sub = parser.add_subparsers(title='Commands', metavar='')
+    serial = sub.add_parser('serial', help='Get serial for a device name')
+    serial.set_defaults(func=_get_serial)
+    serial.add_argument('name', help='Device name')
+    power = sub.add_parser('power', help='Get power control info for a device')
+    power.set_defaults(func=_get_power)
+    power.add_argument('name', help='Device name')
+    state = sub.add_parser('state', help='Get device state for a device')
+    state.set_defaults(func=_get_state)
+    state.add_argument('name', help='Device name')
+    return parser
+
+
+if __name__ == '__main__':
+    args = _get_parser().parse_args()
+    exit(args.func(args))
 
=== added file 'scripts/jenkins.sh'
--- scripts/jenkins.sh	1970-01-01 00:00:00 +0000
+++ scripts/jenkins.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,189 @@
+#!/bin/bash
+
+## This is the script jenkins should run to execute various touch applications
+
+set -e
+
+BASEDIR=$(dirname $(readlink -f $0))/..
+
+RESDIR="${RESDIR-`pwd`/clientlogs}"
+UTAHFILE=${RESDIR}/utah.yaml
+UTAH_PHABLET_CMD="${UTAH_PHABLET_CMD-/usr/share/utah/examples/run_utah_phablet.py}"
+
+
+usage() {
+	cat <<EOF
+usage: $0 -a APP [-s ANDROID_SERIAL] [-p FILE -p FILE ...] [-Q]
+
+Provisions the given device with the latest build
+
+OPTIONS:
+  -h	Show this message
+  -s    Specify the serial of the device to install
+  -a    The application under the "tests" directory to test
+  -p    Extra file to pull from target (absolute path or relative to /home/phablet)
+  -Q    "Quick" don't do a reboot of the device before running the test
+
+EOF
+}
+
+PIDS=""
+
+cleanup() {
+	set +e
+	echo "killing child pids: $PIDS"
+	for p in $PIDS ; do
+		kill $p
+	done
+}
+
+test_from_host() {
+	export PATH=${BASEDIR}/utils/host:${PATH}
+
+	# allow for certain commands to run from host/target
+	# see unity8-autopilot/ts_control for example
+	export TARGET_PREFIX=adb-shell
+
+	[ -z $ANDROID_SERIAL ] || ADBOPTS="-s $ANDROID_SERIAL"
+
+	sudo TARGET_PREFIX=$TARGET_PREFIX PATH="${PATH}" ${UTAH_PHABLET_CMD} \
+		${ADBOPTS} \
+		--from-host \
+		--whoopsie \
+		--results-dir ${RESDIR} \
+		--skip-install --skip-network --skip-utah \
+		--pull /var/crash \
+		--pull /home/phablet/.cache/upstart \
+		--pull /tmp/xmlresults \
+                --pull /var/log/syslog \
+                --pull /var/log/kern.log \
+                --pull /var/log/upstart/whoopsie.log \
+		$EXTRA_PULL \
+		-l ${TESTSUITE_HOST}/master.run
+
+	# make sure the user running this script can remove its artifacts.
+	sudo chown -R `whoami` ${RESDIR}
+}
+
+assert_image() {
+	[ -z $INSTALL_URL ] && return
+	echo "Ensuring target has proper image..."
+	REQUIRED_UUID=$(curl ${INSTALL_URL}/artifact/clientlogs/.ci-uuid)
+	ACTUAL_UUID=$(adb shell "cat /home/phablet/.ci-uuid | tr -d '\r\n'")
+	if [ "$REQUIRED_UUID" != "$ACTUAL_UUID" ] ; then
+		echo "UUIDs $REQUIRED_UUID != $ACTUAL_UUID, reprovisioning device..."
+		ARGS=$(curl ${INSTALL_URL}/artifact/clientlogs/.ci-utah-args | tr -d '\r\n')
+		UUID=$REQUIRED_UUID IMAGE_OPT=$ARGS ${BASEDIR}/scripts/provision.sh
+	else
+		echo "UUIDS match"
+	fi
+}
+
+main() {
+	rm -rf $RESDIR
+	mkdir $RESDIR
+
+	assert_image
+
+	# print the build date so the jenkins job can use it as the
+	# build description
+	adb pull /var/log/installer/media-info ${RESDIR}
+	BUILDID=$(adb shell cat /home/phablet/.ci-version)
+	echo "= TOUCH IMAGE VERSION:$BUILDID"
+
+	adb shell "top -n1 -b" > ${RESDIR}/top.log
+
+	set -x
+	adb shell 'sudo rm -f /var/crash/*'
+	if [ -z $QUICK ] ; then
+		# get the phone in sane place
+		adb reboot
+		# sometimes reboot doesn't happen fast enough, so add a little
+		# delay to help ensure its actually rebooted and we didn't just
+		# connect back to the device before it rebooted
+		adb wait-for-device
+		sleep 5
+		adb wait-for-device
+		phablet-network --skip-setup -t 90s
+		adb shell sudo powerd-cli active &
+		PIDS="$PIDS $!"
+		adb shell sudo powerd-cli display on &
+		PIDS="$PIDS $!"
+	else
+		echo "SKIPPING phone reboot..."
+	fi
+
+	${BASEDIR}/utils/host/adb-shell "sudo aa-clickhook -f --include=/usr/share/autopilot-touch/apparmor/click.rules"
+
+	echo "launching test from the host...."
+	test_from_host
+	adb shell 'sudo rm -f /var/crash/*'
+
+	if ! `grep "^errors: [!0]" < $UTAHFILE >/dev/null` ; then
+		echo "errors found"
+		EXITCODE=1
+	fi
+	if ! `grep "^failures: [!0]" < $UTAHFILE >/dev/null` ; then
+		echo "failures found"
+		EXITCODE=2
+	fi
+	echo "Results Summary"
+	echo "---------------"
+	egrep '^(errors|failures|passes|fetch_errors):' $UTAHFILE
+	exit $EXITCODE
+}
+
+while getopts p:s:a:Qh opt; do
+    case $opt in
+    h)
+        usage
+        exit 0
+        ;;
+    s)
+        export ANDROID_SERIAL=$OPTARG
+        ;;
+    a)
+        APP=$OPTARG
+        ;;
+	p)
+		EXTRA_PULL_FILE=$OPTARG
+
+		if [ ! -z $EXTRA_PULL_FILE ]; then
+			# relative paths are assumed to be relative to /home/phablet
+			E_P_START=`echo $EXTRA_PULL_FILE | cut -c1`
+
+			if [ $E_P_START = '/' ]; then
+				EXTRA_PULL="$EXTRA_PULL --pull $EXTRA_PULL_FILE"
+			else
+				EXTRA_PULL="$EXTRA_PULL --pull /home/phablet/$EXTRA_PULL_FILE"
+			fi
+		fi
+		;;
+    Q)
+        QUICK=1
+        ;;
+  esac
+done
+
+if [ -z $ANDROID_SERIAL ] ; then
+    # ensure we only have one device attached
+    lines=$(adb devices | wc -l)
+    if [ $lines -gt 3 ] ; then
+        echo "ERROR: More than one device attached, please use -s option"
+	echo
+        usage
+        exit 1
+    fi
+fi
+if [ -z $APP ] ; then
+    echo "ERROR: No app specified"
+    usage
+    exit 1
+fi
+
+TESTSUITE_HOST=$(readlink -f ${BASEDIR}/tests/${APP})
+TESTSUITE_TARGET_BASE=/tmp/tests
+TESTSUITE_TARGET=${TESTSUITE_TARGET_BASE}/$(basename ${TESTSUITE_HOST})
+
+trap cleanup TERM INT EXIT
+main
 
=== added file 'scripts/junit2utah.py'
--- scripts/junit2utah.py	1970-01-01 00:00:00 +0000
+++ scripts/junit2utah.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,70 @@
+#!/usr/bin/python
+
+import datetime
+import sys
+
+from xml.etree import ElementTree
+
+import yaml
+
+
+def _convert_testcase(tc):
+    x = {
+        'testcase': tc.attrib['name'],
+        'testsuite': tc.attrib['classname'],
+        'command': 'autopilot',
+        'cmd_type': 'testcase_test',
+        'stdout': '',
+        'stderr': '',
+        'returncode': 0
+    }
+    t = tc.attrib.get('time', False)
+    if t:
+        x['time_delta'] = t
+
+    for e in tc.getchildren():
+        if e.tag in ('failure', 'error'):
+            x['stderr'] = e.text
+            x['returncode'] = 1
+        elif e.tag == 'skip':
+            # NOTE: this isn't a real thing in UTAH. However, the
+            # qa-dashboard code doesn't care and will display it as skipped
+            x['cmd_type'] = 'testcase_skipped'
+            x['stdout'] = e.text
+        else:
+            raise RuntimeError('Unknown element type: %s' % e.tag)
+    return x
+
+
+def _get_results(stream):
+    tree = ElementTree.fromstring(stream.read())
+    results = {
+        'errors': int(tree.attrib.get('errors', '0')),
+        'failures': int(tree.attrib.get('failures', '0')),
+        'commands': [],
+        'fetch_errors': 0,
+        'uname': 'n/a',
+        'media-info': 'n/a',
+        'install_type': 'n/a',
+        'arch': 'n/a',
+        'release': 'n/a',
+        'build_number': 'n/a',
+        'name': 'unamed',
+        'runlist': 'n/a',
+        'ran_at': datetime.datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S'),
+    }
+    results['passes'] = \
+        int(tree.attrib['tests']) - results['errors'] - results['failures']
+
+    for tc in tree.getchildren():
+        results['commands'].append(_convert_testcase(tc))
+    return results
+
+
+def _main(stream):
+    results = _get_results(stream)
+    print(yaml.safe_dump(results, default_flow_style=False))
+
+
+if __name__ == '__main__':
+    exit(_main(sys.stdin))
 
=== added file 'scripts/ncd_usb.py'
--- scripts/ncd_usb.py	1970-01-01 00:00:00 +0000
+++ scripts/ncd_usb.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,48 @@
+#! /usr/bin/python
+
+"""A utility to control the USB relay's in the QA lab."""
+
+import argparse
+import sys
+import urllib2
+
+
+def set_relay(urlbase, bank, relay, on):
+    # the values 100/108 came from the JavaScript of our web-based management
+    # system for the relays. The meaning of the values isn't documented.
+    if on:
+        relay += 108
+    else:
+        relay += 100
+    cmd = '254,{},{}'.format(relay, bank + 1)
+    url = '{}/cgi-bin/runcommand.sh?1:cmd={}'.format(urlbase, cmd)
+    resp = urllib2.urlopen(url)
+    resp = resp.read()
+    if 'OK' not in resp:
+        print('ERROR: bad response: {}'.format(resp))
+        sys.exit(1)
+
+
+def _get_parser():
+    parser = argparse.ArgumentParser(
+        description='Toggles an NCD relay connected to a USB cable on/off')
+    parser.add_argument('-u', '--url',
+                        default='http://qa-relay-control.ubuntu-ci',
+                        help='NCD relay URL. default=%(default)s')
+    parser.add_argument('-b', '--bank', type=int, required=True,
+                        help='NCD relay 0-based bank ID.')
+    parser.add_argument('-r', '--relay', type=int, required=True,
+                        help='NCD relay 0-based relay ID.')
+    parser.add_argument('action', metavar='action',
+                        choices=('on', 'off'),
+                        help='action to perform on|off')
+    return parser
+
+
+if __name__ == '__main__':
+    args = _get_parser().parse_args()
+
+    # NOTE: when the relay is ON usb is actually OFF. ie the logic
+    # is backwards between them, thus action=off actually turns
+    # the relay on
+    set_relay(args.url, args.bank, args.relay, args.action == 'off')
 
=== added file 'scripts/provision.sh'
--- scripts/provision.sh	1970-01-01 00:00:00 +0000
+++ scripts/provision.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,175 @@
+#!/bin/bash
+
+## This is the script jenkins should run to provision a device in the lab
+
+set -e
+
+BASEDIR=$(dirname $(readlink -f $0))
+export PATH=${BASEDIR}/../utils/host:${PATH}
+
+RESDIR=`pwd`/clientlogs
+
+NETWORK_FILE="${NETWORK_FILE-/home/ubuntu/magners-wifi}"
+
+IMAGE_OPT="${IMAGE_OPT---bootstrap --developer-mode --channel ubuntu-touch/devel-proposed}"
+UUID="${UUID-$(uuidgen -r)}"
+
+usage() {
+cat <<EOF
+usage: $0 [-s ANDROID_SERIAL] [-n NETWORK_FILE] [-P ppa] [-p package] [-r revision] [-w]
+
+Provisions the given device with the latest build
+
+OPTIONS:
+  -h	Show this message
+  -s    Specify the serial of the device to install
+  -n    Select network file
+  -P    add the ppa to the target (can be repeated)
+  -p    add the package to the target (can be repeated)
+  -r    Specify the image revision to flash
+  -w    make the system writeable (implied with -p and -P arguments)
+
+EOF
+}
+
+image_info() {
+	# mark the version we installed in /home/phablet/.ci-[uuid,flash-args]
+	# adb shell messes up \n's with \r\n's so do the whole of the regex on the target
+	IMAGEVER=$(adb shell "sudo system-image-cli -i | sed -n -e 's/version version: \([0-9]*\)/\1/p' -e 's/version ubuntu: \([0-9]*\)/\1/p' -e 's/version device: \([0-9]*\)/\1/p' | paste -s -d:")
+	CHAN=$(adb shell "sudo system-image-cli -i | sed -n -e 's/channel: \(.*\)/\1/p' | paste -s -d:")
+	REV=$(echo $IMAGEVER | cut -d: -f1)
+	echo "$IMAGE_OPT" | grep -q "\-\-revision" || IMAGE_OPT="${IMAGE_OPT} --revision $REV"
+	echo "$IMAGE_OPT" | grep -q "\-\-channel" || IMAGE_OPT="${IMAGE_OPT} --channel $CHAN"
+	adb shell "echo '${IMAGEVER}' > /home/phablet/.ci-version"
+	echo $UUID > $RESDIR/.ci-uuid
+	adb push $RESDIR/.ci-uuid /home/phablet/
+	cat > $RESDIR/.ci-flash-args <<EOF
+$IMAGE_OPT
+EOF
+	adb push $RESDIR/.ci-flash-args /home/phablet/.ci-flash-args
+	echo $CUSTOMIZE > $RESDIR/.ci-customizations
+	adb push $RESDIR/.ci-customizations /home/phablet/.ci-customizations
+}
+
+log() {
+	echo = $(date): $*
+}
+
+set_hwclock() {
+	log "SETTING HWCLOCK TO CURRENT TIME"
+        # Use ip for ntp.ubuntu.com in case resolving doesn't work yet
+	adb-shell sudo ntpdate 91.189.94.4 || log "WARNING: could not set ntpdate"
+	# hwclock sync has to happen after we set writable image
+	adb-shell sudo hwclock -w || log "WARNING: could not sync hwclock"
+	log "Current date on device is:"
+	adb shell date
+	log "Current hwclock on device is:"
+	adb shell hwclock
+}
+
+while getopts i:s:n:P:p:r:wh opt; do
+	case $opt in
+	h)
+		usage
+		exit 0
+		;;
+	n)
+		NETWORK_FILE=$OPTARG
+		;;
+	s)
+		export ANDROID_SERIAL=$OPTARG
+		;;
+	i)
+		IMAGE_TYPE=$OPTARG
+		;;
+	w)
+		# making this a non-zero length string enables the logic
+		CUSTOMIZE=" "
+		;;
+	P)
+		CUSTOMIZE="$CUSTOMIZE --ppa $OPTARG"
+		;;
+	p)
+		CUSTOMIZE="$CUSTOMIZE -p $OPTARG"
+		;;
+	r)
+		IMAGE_OPT="$IMAGE_OPT --revision $OPTARG"
+		;;
+
+	esac
+done
+
+if [ -z $ANDROID_SERIAL ] ; then
+	# ensure we only have one device attached
+	lines=$(adb devices | wc -l)
+	if [ $lines -gt 3 ] ; then
+		echo "ERROR: More than one device attached, please use -s option"
+		echo
+		usage
+		exit 1
+	fi
+fi
+
+if [ ! -f $NETWORK_FILE ] && [ -z $USE_EMULATOR ] ; then
+	echo "ERROR: NETWORK_FILE, $NETWORK_FILE, not found"
+	exit 1
+fi
+
+set -x
+[ -d $RESDIR ] && rm -rf $RESDIR
+mkdir -p $RESDIR
+
+if [ -z $USE_EMULATOR ] ; then
+	log "FLASHING DEVICE"
+        if [ "${DEVICE_TYPE}" = "krillin" ]; then
+		# reboot to recovery for krillin
+		adb reboot recovery
+                # Wait for recovery to boot
+                sleep 30
+        else
+		adb reboot bootloader
+	fi
+	ubuntu-device-flash --password ubuntuci $IMAGE_OPT
+	adb wait-for-device
+	sleep 60  #give the system a little time
+else
+	log "CREATING EMULATOR"
+	ubuntu-emulator destroy --yes $ANDROID_SERIAL || true
+	sudo ubuntu-emulator create $ANDROID_SERIAL $IMAGE_OPT
+	${BASEDIR}/reboot-and-wait
+fi
+
+if [ -z $USE_EMULATOR ] ; then
+	log "SETTING UP WIFI"
+	phablet-network -n $NETWORK_FILE
+fi
+
+phablet-config welcome-wizard --disable
+
+if [ -n "$CUSTOMIZE" ] ; then
+	log "CUSTOMIZING IMAGE"
+	phablet-config writable-image $CUSTOMIZE
+fi
+
+log "SETTING UP SUDO"
+adb shell "echo ubuntuci |sudo -S bash -c 'echo phablet ALL=\(ALL\) NOPASSWD: ALL > /etc/sudoers.d/phablet && chmod 600 /etc/sudoers.d/phablet'"
+
+# FIXME: Can't do this through phablet-config for now because it needs auth
+# phablet-config edges-intro --disable
+adb shell "sudo dbus-send --system --print-reply --dest=org.freedesktop.Accounts /org/freedesktop/Accounts/User32011 org.freedesktop.DBus.Properties.Set string:com.canonical.unity.AccountsService string:demo-edges variant:boolean:false"
+
+log "SETTING UP CLICK PACKAGES"
+CLICK_TEST_OPTS=""
+channel_name=$(adb shell "sudo system-image-cli -i | sed -n -e 's/channel: \(.*\)/\1/p' | paste -s -d:")
+# FIXME: workaround for phablet-click-test-setup to pull the right sources
+if [[ $channel_name == *rtm* ]] ; then
+	CLICK_TEST_OPTS="--distribution ubuntu-rtm --series 14.09"
+fi
+phablet-click-test-setup $CLICK_TEST_OPTS
+
+# get our target-based utilities into our PATH
+adb push ${BASEDIR}/../utils/target /home/phablet/bin
+
+image_info
+
+set_hwclock
 
=== added file 'scripts/reboot-and-wait'
--- scripts/reboot-and-wait	1970-01-01 00:00:00 +0000
+++ scripts/reboot-and-wait	2014-09-10 13:48:46 +0000
@@ -0,0 +1,54 @@
+#!/usr/bin/python
+
+import argparse
+import logging
+import os
+import subprocess
+import time
+
+from phabletutils.device import AndroidBridge
+
+EMULATOR = os.environ.get('USE_EMULATOR', '')
+
+
+def _get_arg_parser():
+    parser = argparse.ArgumentParser(
+        description='Reboot device and waits for networking to become active.')
+    parser.add_argument('-s', '--serial', help='Device serial')
+    parser.add_argument('-n', '--num-tries', type=int, default=3,
+                        help='''How many times to retry on failure.
+                             default=%(default)d''')
+    return parser
+
+
+def main(args):
+    device = AndroidBridge(args.serial)
+    device.wait_for_device()
+    for i in range(args.num_tries):
+        device.reboot()
+        device.wait_for_device()
+        time.sleep(5)
+        device.wait_for_device()
+        try:
+            device.wait_for_network()
+            return 0
+        except:
+            pass  # try the loop again
+    logging.error('device failed to start and activate networking')
+    return 1
+
+
+def emulator_main(args):
+    emulator = os.path.join(os.path.dirname(__file__), 'run-emulator')
+    subprocess.check_call([emulator])
+
+
+if __name__ == '__main__':
+    logging.basicConfig(level=logging.INFO)
+    logging.getLogger().name = 'reboot-and-wait'
+    args = _get_arg_parser().parse_args()
+    if EMULATOR:
+        logging.info('using emulator logic for reboot')
+        exit(emulator_main(args))
+    else:
+        exit(main(args))
 
=== added file 'scripts/recover.py'
--- scripts/recover.py	1970-01-01 00:00:00 +0000
+++ scripts/recover.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,124 @@
+#!/usr/bin/env python
+
+import device_info
+import logging
+import subprocess
+import sys
+import time
+from ncd_usb import set_relay
+
+log = logging.getLogger()
+logging.basicConfig(level=logging.INFO)
+
+
+class DeviceError(Exception):
+    pass
+
+
+def _reimage_from_fastboot(serial):
+    #Starting from fastboot mode, put a known-good image on the device
+    log.info("Flashing the last stable image")
+    subprocess.check_output(['ubuntu-device-flash', '--serial', serial,
+                             '--channel', 'ubuntu-touch/stable',
+                             '--bootstrap', '--password', 'ubuntuci'])
+    return _wait_for_device(serial, 600)
+
+
+def _wait_for_device(serial, timeout=120):
+    # Wait for the device to come up to a good/booted state
+    log.info("Waiting for the device to become available")
+    try:
+        subprocess.check_call(['timeout', str(timeout), 'adb', '-s',
+                               serial, 'wait-for-device'])
+    except:
+        log.error("Timed out waiting for reboot. Recover device manually")
+        raise
+    dev_state = device_info.get_state(serial)
+    if dev_state != 'device':
+        raise DeviceError("Device in state: {0}, still not available after "
+                          "{1} seconds".format(dev_state, timeout))
+    else:
+        log.info("Device is now available")
+        return 0
+
+
+def _wait_for_fastboot(serial, timeout=120):
+    if timeout > 10:
+        wait = 10
+    else:
+        wait = timeout
+    waited = 0
+    while waited < timeout:
+        state = device_info.get_state(serial)
+        if state == 'fastboot':
+            return 0
+        time.sleep(wait)
+        waited += wait
+    else:
+        state = device_info.get_state(serial)
+        if state == 'fastboot':
+            return 0
+        log.error("Timed out waiting for fastboot. Recover device manually")
+        raise DeviceError("Device in state: {0}, still not available after "
+                          "{1} seconds".format(state, timeout))
+
+
+def _mako_to_bootloader(urlbase, bank, power=1, volume=2):
+    """
+    This just works on mako for certain, but that's all we have connected
+    right now. After this runs, the device should be in the bootloader
+    """
+    log.info("Forcing the device to enter the bootloader")
+    #Power off the device from any state
+    set_relay(urlbase, bank, power, 1)
+    time.sleep(10)
+    set_relay(urlbase, bank, power, 0)
+    time.sleep(10)
+    #Enter the bootloader
+    set_relay(urlbase, bank, volume, 1)
+    set_relay(urlbase, bank, power, 1)
+    time.sleep(5)
+    set_relay(urlbase, bank, volume, 0)
+    set_relay(urlbase, bank, power, 0)
+
+
+def _full_recovery(device_name):
+    #we only support mako at the moment
+    (url, bank, power, volume) = device_info.get_power(device_name)
+    if None in (url, bank, power, volume):
+        #This device does not have information about relays
+        raise DeviceError("Full recovery not possible with this device")
+    _mako_to_bootloader(url, bank, power, volume)
+    serial = device_info.get_serial(device_name)
+    _wait_for_fastboot(serial)
+    _reimage_from_fastboot(serial)
+
+
+def recover(device):
+    try:
+        serial = device_info.get_serial(device)
+    except AttributeError:
+        log.error("No device found for '{}'".format(device))
+        raise
+    state = device_info.get_state(serial)
+    if state in ('device', 'recovery'):
+        #The device can proceed with testing
+        return 0
+    if state == 'fastboot':
+        #The device is in fastboot right now, we need it booted first
+        return _reimage_from_fastboot(serial)
+    if state in ('unknown', 'disconnected'):
+        #The device is in an unknown state, we need full recovery
+        return _full_recovery(device)
+    #In theory, we should never get here, but....
+    raise DeviceError("Device '{}' is in an unknown state!".format(device))
+
+
+if __name__ == '__main__':
+    name = sys.argv[1]
+    try:
+        print(recover(name))
+    except AttributeError:
+        #This is what we'll get if it's an unknown device, raise for
+        #everything else so we get better debugging information
+        sys.exit(-1)
 
=== added file 'scripts/run-autopilot-tests.sh'
--- scripts/run-autopilot-tests.sh	1970-01-01 00:00:00 +0000
+++ scripts/run-autopilot-tests.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,254 @@
+#!/bin/sh
+
+set -e
+
+BASEDIR=$(dirname $(readlink -f $0))/..
+RESDIR=`pwd`/clientlogs
+
+export PATH=${BASEDIR}/utils/host:${PATH}
+export TARGET_PREFIX=adb-shell
+
+
+usage() {
+	cat <<EOF
+usage: $0 -a APP [-s ANDROID_SERIAL] [-Q] [-o results_dir] [-S]
+
+Runs a set of autopilot tests on the target
+
+OPTIONS:
+  -h	Show this message
+  -s    Specify the serial of the device to test
+  -a    The application to test (can be repeated)
+  -o    Specify the directory to place results in.
+        Default: $RESDIR
+  -Q    "Quick" don't do a reboot of the device before/between testsuites.
+  -S    Skip the system-settle tests that run before/after each testsuite.
+
+EOF
+}
+
+log_error() {
+	echo ERROR: $* >> ${RESDIR}/runner-errors.txt
+}
+
+setup_test() {
+	app=$1
+	label=$2
+	odir=$3
+	{
+		pkgs=$(${BASEDIR}/jenkins/testconfig.py packages -a $app)
+		if [ "$label" = "setup" ] ; then
+			adb-shell sudo apt-get install -yq --force-yes $pkgs
+		else
+			#Always remove dbus-x11 because it causes
+			#problems when we leave it around
+			pkgs="$pkgs dbus-x11"
+			adb-shell sudo apt-get autoremove --purge -y $pkgs \
+				|| /bin/true
+		fi
+		echo $? > ${odir}/setup_${label}.rc
+	} 2>&1 | tee ${odir}/setup_${label}.log
+}
+
+system_settle() {
+	[ -z $NOSETTLE ] || return 0
+
+	label=$1
+	odir=$2
+	rc=0
+	timeout=120s
+	if [ "$label" = "before" ] ; then
+		timeout=300s
+	fi
+
+	settle=${BASEDIR}/tests/systemsettle/systemsettle.sh
+	{
+		export UTAH_PROBE_DIR=${odir}  # needed for log file location
+		timeout $timeout $settle -c5 -d6 -p 97.5 -l $label || rc=1
+		echo $rc > ${odir}/settle_${label}.rc
+	} 2>&1 | tee ${odir}/settle_${label}.log
+}
+
+test_app() {
+	app=$1
+
+	odir=${RESDIR}/${app}
+	[ -d $odir ] && rm -rf $odir
+	mkdir -p $odir || return 1
+
+	system_settle before $odir
+	phablet-config autopilot --dbus-probe enable || \
+		(log_error "'autopilot dbus-probe enable' failed"; return 1)
+	adb-shell /home/phablet/bin/check-clickhook-rules || \
+		(log_error "some click profiles missing autopilot rules")
+
+	setup_test $app setup $odir
+
+	NOSHELL=""
+	[ "$app" = "unity8" ] && NOSHELL="-n"
+	EXTRA=""
+        # Use --timeout-profile=long only if we are using the emulator
+	[ -z $USE_EMULATOR ] || EXTRA="-A '--timeout-profile=long'"
+
+	phablet-test-run \
+		$NOSHELL $EXTRA \
+		-o ${odir} -f subunit \
+		-a /var/crash -a /home/phablet/.cache/upstart \
+                -a /var/log/syslog -a /var/log/kern.log \
+                -a /var/log/upstart/whoopsie.log \
+		-v $app || true
+
+	system_settle after $odir
+	setup_test $app teardown $odir
+	if [ -f ${odir}/test_results.subunit ] ; then
+		cat ${odir}/test_results.subunit | subunit2junitxml > ${odir}/test_results.xml
+	fi
+	${BASEDIR}/scripts/combine_results ${odir}
+}
+
+reboot_wait() {
+	if [ -z $QUICK ] ; then
+		reboot-and-unlock.sh
+		FILES="/var/crash/* /home/phablet/.cache/upstart/*.log*"
+		if ! adb shell "sudo rm -rf $FILES" ; then
+			log_error "unable to remove crash and log files, retrying"
+			adb wait-for-device
+			adb shell "sudo rm -rf $FILES"
+		fi
+	else
+		echo "SKIPPING phone reboot..."
+	fi
+}
+
+if [ -z $USE_EMULATOR ] ; then
+grab_powerd() {
+	echo "grabbing powerd cli locks..."
+	adb shell sudo powerd-cli active &
+	PIDS="$!"
+	adb shell sudo powerd-cli display on &
+	PIDS="$PIDS $!"
+}
+
+release_powerd() {
+	if [ -n "$PIDS" ] ; then
+		echo "killing child pids: $PIDS"
+		for p in $PIDS ; do
+			kill $p || true
+		done
+		PIDS=""
+	fi
+        adb shell sudo pkill powerd-cli
+}
+
+else
+grab_powerd() {
+	#emulator does not use powerd, so this is noop
+	return 0
+}
+
+release_powerd() {
+	#emulator does not use powerd, so this is noop
+	return 0
+}
+fi
+
+dashboard_update() {
+	# only try and update the dashboard if we are configured to
+	[ -z $DASHBOARD_KEY ] && return 0
+	[ -z $DASHBOARD_BUILD ] && return 0
+	[ -z $DASHBOARD_IMAGE ] && return 0
+	${BASEDIR}/scripts/dashboard.py $* \
+		--image $DASHBOARD_IMAGE \
+		--build $DASHBOARD_BUILD || true
+}
+
+dashboard_result_running() {
+	dashboard_update result-running --test $1
+}
+
+dashboard_result_syncing() {
+	xunit=${RESDIR}/${app}/test_results.xml
+	[ -f $xunit ] || return 0
+
+	# save a utah.yaml version of the results so the dashboard can process
+	cat $xunit | ${BASEDIR}/scripts/junit2utah.py > ${RESDIR}/${app}/utah.yaml
+	dashboard_update result-syncing --test $1 --results ${RESDIR}/${app}/utah.yaml
+}
+
+main() {
+	[ -d $RESDIR ] || mkdir -p $RESDIR
+
+	set -x
+
+	for app in $APPS ; do
+		set +x
+		echo "========================================================"
+		echo "= testing $app"
+		echo "========================================================"
+		set -x
+		dashboard_result_running $app
+		reboot_wait
+
+		grab_powerd
+
+		if ! test_app $app ; then
+			log_error "testing $app, retrying"
+			# we sometimes see sporatic adb failures that seem to
+			# related to MTP. This adds a retry for the test.
+			# test_app only fails on a device error (not a test
+			# case error)
+			adb wait-for-device
+			test_app $app
+		fi
+		dashboard_result_syncing $app
+
+		release_powerd
+	done
+}
+
+while getopts s:a:o:QSh opt; do
+	case $opt in
+	h)
+		usage
+		exit 0
+		;;
+	s)
+		export ANDROID_SERIAL=$OPTARG
+		;;
+	o)
+		RESDIR=$OPTARG
+		;;
+	a)
+		APPS="$APPS $OPTARG"
+		;;
+	Q)
+		QUICK=1
+		;;
+	S)
+		NOSETTLE=1
+		;;
+	esac
+done
+
+if [ -z $ANDROID_SERIAL ] ; then
+	# ensure we only have one device attached
+	lines=$(adb devices | wc -l)
+	if [ $lines -gt 3 ] ; then
+		echo "ERROR: More than one device attached, please use -s option"
+		echo
+		usage
+		exit 1
+	fi
+fi
+if [ -z "$APPS" ] ; then
+	echo "ERROR: No app specified"
+	usage
+	exit 1
+fi
+
+trap release_powerd TERM INT EXIT
+if [ -n "$USE_EMULATOR" ] ; then
+	echo "disabling system-settle testing for emulator"
+	NOSETTLE=1
+fi
+main
 
=== added file 'scripts/run-emulator'
--- scripts/run-emulator	1970-01-01 00:00:00 +0000
+++ scripts/run-emulator	2014-09-10 13:48:46 +0000
@@ -0,0 +1,97 @@
+#!/usr/bin/python
+
+import logging
+import os
+import subprocess
+import sys
+import time
+
+FULL_RETRIES = 3
+# unity8 takes a long time because it needs apparmor to start. apparmor is
+# slow because its parsing all the profiles on first boot. qemu seems to
+# be inconsisent, sometimes this takes 8 tries, sometimes >40.
+UNITY_RETRIES = 50
+UNITY_WAIT = 20
+ADB_RETRIES = 3
+ADB_WAIT = 120
+
+EMULATOR_ARCH = os.environ.get('EMULATOR_ARCH', 'x86')
+
+
+class RetryException(Exception):
+    pass
+
+
+def _kill():
+    logging.info('killing all emulator pids')
+    with open('/dev/null', 'w') as f:
+        subprocess.call(['killall', 'ubuntu-emulator'], stderr=f)
+        time.sleep(1)
+        subprocess.call(['killall', 'emulator-' + EMULATOR_ARCH], stderr=f)
+        time.sleep(1)
+
+
+def _launch():
+    logging.info('launching emulator...')
+    subprocess.Popen(['ubuntu-emulator', 'run', os.environ['ANDROID_SERIAL']])
+
+
+def _adb_wait(retries, timeout):
+    timeout = '%ds' % timeout
+    for i in range(retries):
+        logging.info('waiting for emulator via adb (%d of %d)...', i, retries)
+        rc = subprocess.call(['timeout', timeout, 'adb', 'wait-for-device'])
+        if rc == 0:
+            return
+        # the emulator isn't always being detected by the adb-server
+        # running kill-server works around this. NOTE: this is only
+        # safe when run on a slave hooked up to a single emulator
+        logging.info('emulator not found, restarting adbd')
+        subprocess.check_call(['adb', 'kill-server'])
+    return RetryException('emulator not found via adb')
+
+
+def _unity_wait(retries, timeout):
+    for i in range(retries):
+        logging.info('waiting for unity8 (%d of %d)...', i, retries)
+        time.sleep(timeout)
+        try:
+            out = subprocess.check_output(
+                ['adb', 'shell', 'sudo -i -u phablet status unity8'])
+            if 'start/running' in out:
+                return
+        except subprocess.CalledProcessError:
+            logging.info('adb shell failed, retrying')
+    raise RetryException('unity8 not running on device')
+
+
+def main():
+    for i in range(FULL_RETRIES):
+        try:
+            _kill()
+            _launch()
+            _adb_wait(ADB_RETRIES, ADB_WAIT)
+            logging.info('emulator is running, waiting on unity8')
+            if EMULATOR_ARCH == 'arm':
+                logging.info('sleeping for 160s to wait for ARM emulator')
+                time.sleep(160)
+            _unity_wait(UNITY_RETRIES, UNITY_WAIT)
+            logging.info('emulator is booted and ready')
+            return 0
+        except RetryException as e:
+            logging.warn('emulator failed to boot: %s', e.message)
+            logging.warn('kill and retry %d more times', FULL_RETRIES - i)
+            next
+    logging.error('emulator failed to boot')
+    _kill()
+    return 1
+
+if __name__ == '__main__':
+    handler = logging.StreamHandler(stream=sys.stderr)
+    formatter = logging.Formatter(
+        '%(asctime)s %(levelname)s: %(message)s', datefmt='%H:%M:%S')
+    handler.setFormatter(formatter)
+    l = logging.getLogger('')
+    l.addHandler(handler)
+    l.setLevel(logging.INFO)
+    exit(main())
 
=== added file 'scripts/run-smoke'
--- scripts/run-smoke	1970-01-01 00:00:00 +0000
+++ scripts/run-smoke	2014-09-10 13:48:46 +0000
@@ -0,0 +1,395 @@
+#!/usr/bin/python
+
+import argparse
+import datetime
+import logging
+import os
+import shutil
+import subprocess
+
+import yaml
+
+from phabletutils.environment import detect_device
+
+import dashboard
+import statsd
+
+EMULATOR = os.environ.get('USE_EMULATOR')
+if EMULATOR:
+    def fake_detect(serial, device=None):
+        log.info('faking detect device for emulator')
+        return 'emulator'
+
+    # detect_device doesn't support the emulator
+    globals()['detect_device'] = fake_detect
+    if 'ANDROID_SERIAL' not in os.environ:
+        # we need something here or "serial required" logic fails
+        os.environ['ANDROID_SERIAL'] = 'emulator-5554'
+
+log = logging.getLogger()
+script_dir = os.path.dirname(__file__)
+res_dir = os.path.join(os.getcwd(), 'clientlogs')
+
+dashboard_api = dashboard.API()
+
+
+class SerialAction(argparse.Action):
+    def __call__(self, parser, namespace, values, option_string=None):
+        log.info('android serial: %s', values[0])
+        os.environ['ANDROID_SERIAL'] = values[0]
+
+
+class DebugAction(argparse.Action):
+    def __call__(self, parser, namespace, values, option_string=None):
+        log.setLevel(level=logging.DEBUG)
+        log.debug('debug logging enabled')
+
+
+def _serial_required():
+    required = 'ANDROID_SERIAL' not in os.environ
+    if required:
+        try:
+            out = subprocess.check_output(['adb', 'devices'])
+            required = (len(out.decode().split('\n')) != 4)
+        except subprocess.CalledProcessError as e:
+            logging.debug('error getting adb devices: %s', e)
+    return required
+
+
+def _get_parser():
+    parser = argparse.ArgumentParser(
+        description='Run the complete test-execution-service suite.')
+
+    parser.add_argument('-s', '--serial', action=SerialAction, nargs=1,
+                        required=_serial_required(),
+                        help='Android serial if more than one device present')
+    parser.add_argument('--debug', action=DebugAction, nargs=0,
+                        help='''Enable debug logging.''')
+
+    parser.add_argument('--install-url',
+                        help='''Flash with image from previous jenkins job.
+                        This option will check if the device already has image
+                        noted from this URL and will skip provisioning. The URL
+                        should be the path the job like:
+                        http://q-jenkins:8080/job/<your job>/<build number>''')
+    parser.add_argument('-p', '--package', action='append',
+                        help='Additional packages to install on target.')
+    parser.add_argument('-P', '--ppa', action='append',
+                        help='Additional PPA to configure on target.')
+    parser.add_argument('-a', '--app', action='append',
+                        help='Autopilot tests tor run.')
+    parser.add_argument('-t', '--test', action='append',
+                        help='UTAH tests tor run.')
+    parser.add_argument('-r', '--revision', help='Image revision to install.')
+    parser.add_argument('-n', '--no-provision', action='store_true',
+                        help='Skip provisioning of the target device')
+    parser.add_argument('--hooks-dir',
+                        help='''A directory containing scripts to be run after
+                        the target has been provisioned and before testing.''')
+    parser.add_argument('--image-opt',
+                        help='Options to pass to phablet-flash')
+    parser.add_argument('--image-type', default='touch',
+                        help='''Image type being tested. This can be changed
+                        to 'touch_sf4p' so that SurfaceFlinger will be used
+                        instead of Mir. default=%(default)s''')
+    parser.add_argument('--num-workers', type=int, default=1,
+                        help='''The total number of workers available for
+                        running tests.''')
+    parser.add_argument('--worker-idx', type=int, default=0,
+                        help='The worker to allocate testing work to.')
+    return parser
+
+
+def _arg_from_env(args, attr, envkey, array):
+    val = os.environ.get(envkey, False)
+    if val:
+        if array:
+            setattr(args, attr, val.split())
+        else:
+            setattr(args, attr, val)
+        del os.environ[envkey]
+
+
+def _merge_env(args):
+    '''When run in Jenkins everything comes as environment variables.
+
+    Its makes a much simpler job this way. While command line args are
+    much easier for a user.
+    '''
+    _arg_from_env(args, 'app', 'APPS', True)
+    _arg_from_env(args, 'test', 'TESTS', True)
+    _arg_from_env(args, 'package', 'PACKAGES', True)
+    _arg_from_env(args, 'ppa', 'PPAS', True)
+    _arg_from_env(args, 'image_opt', 'IMAGE_OPT', False)
+    _arg_from_env(args, 'image_type', 'IMAGE_TYPE', False)
+    _arg_from_env(args, 'install_url', 'INSTALL_URL', False)
+    _arg_from_env(args, 'revision', 'REVISION', False)
+    _arg_from_env(args, 'num_workers', 'workers', False)
+    _arg_from_env(args, 'worker_idx', 'worker_idx', False)
+
+
+def _assert_args(args):
+    if args.install_url:
+        # this means you shouldn't specify packages, ppas, or image options
+        if args.package or args.ppa or args.image_opt:
+            msg = 'ERROR: --install-url can\'t be used with ' \
+                  '--package, -ppa, or --image_opt'
+            print(msg)
+            return False
+
+    # don't bother the install_url check, a user might be copy/pasting and
+    # doesn't hurt. Its just good to not encourage it.
+    _merge_env(args)
+
+    script = os.path.join(script_dir, '../jenkins/testconfig.py')
+    if args.package and args.package[0] == 'ALL':
+        logging.info('Discovering all required dependencies')
+        out = subprocess.check_output(
+            [script, 'packages', '-i', args.image_type])
+        args.package = [x for x in out.decode().split()]
+
+    if args.app and args.app[0] == 'ALL':
+        logging.info('Discovering all autopilot tests')
+        out = subprocess.check_output(
+            [script, 'apps', '-i', args.image_type,
+             '-t', str(args.num_workers), '-w', str(args.worker_idx)])
+        args.app = [x for x in out.decode().split()]
+        logging.info('Autopilot test list: {}'.format(' '.join(args.app)))
+
+    if args.test and args.test[0].startswith('ALL'):
+        logging.info('Discovering all UTAH tests')
+        argv = [script, 'utah', '-i', args.image_type,
+                '-t', str(args.num_workers), '-w', str(args.worker_idx)]
+        if args.test[0] == 'ALL_INCLUDING_AUTOPILOT':
+            argv.append('-a')
+        out = subprocess.check_output(argv)
+        args.test = [x for x in out.decode().split()]
+        logging.info('Utah test list: {}'.format(' '.join(args.test)))
+
+    logging.debug('ARGS: %r', args)
+
+    statsd.gauge_it('PACKAGES', args.package)
+    statsd.gauge_it('APPS', args.app)
+    statsd.gauge_it('TESTS', args.test)
+
+    return True
+
+
+def _run(args, ignore_error=False):
+    try:
+        logging.info('Running: %s', ' '.join(args))
+        subprocess.check_call(args)
+    except subprocess.CalledProcessError:
+        if ignore_error:
+            logging.error('failed to run %r, continuing', args)
+        else:
+            exit(1)
+
+
+def _image_info():
+    info = subprocess.check_output(['adb', 'shell', 'sudo',
+                                    'system-image-cli', '-i'])
+    v_ver = u_ver = d_ver = channel = None
+    for line in info.split('\n'):
+        if not line.strip():
+            continue
+        key, val = line.split(':', 1)
+        if key == 'version version':
+            v_ver = val.strip()
+        elif key == 'version ubuntu':
+            u_ver = val.strip()
+        elif key == 'version device':
+            d_ver = val.strip()
+        elif key == 'channel':
+            channel = val.strip()
+    ver = '%s:%s:%s' % (v_ver, u_ver, d_ver)
+    # required for the jenkins job's build description
+    print('= TOUCH IMAGE VERSION:' + ver)
+    return ver, channel
+
+
+def _assert_image(args):
+    log.info('checking if device has proper image ...')
+    os.environ['INSTALL_URL'] = args.install_url
+    _run([os.path.join(script_dir, 'assert-image')])
+
+
+def _write_utah(start, end, passed):
+    passes = failures = rc = 0
+    if passed:
+        passes = 1
+    else:
+        rc = failures = 1
+
+    delta = '%s' % (end - start)
+    start = start.strftime('%Y-%m-%d %H:%M:%S')
+    data = {
+        'name': 'install-and-boot',
+        'errors': 0,
+        'failures': failures,
+        'passes': passes,
+        'fetch_errors': 0,
+        'uname': 'n/a',
+        'media-info': 'n/a',
+        'install_type': 'n/a',
+        'arch': 'n/a',
+        'release': 'n/a',
+        'build_number': 'n/a',
+        'runlist': 'n/a',
+        'ran_at': start,
+        'commands': [{
+            'cmd_type': 'testcase_test',
+            'command': 'provision',
+            'returncode': rc,
+            'start_time': start,
+            'time_delta': delta,
+            'stderr': '',
+            'stdout': '',
+            'testcase': 'boot',
+            'testsuite': 'install-and-boot',
+        }]
+    }
+    path = os.path.join(res_dir, 'install-and-boot')
+    if not os.path.exists(path):
+        os.mkdir(path)
+    with open(os.path.join(path, 'utah.yaml'), 'w') as f:
+        f.write(yaml.safe_dump(data, default_flow_style=False))
+
+
+def _post_install_hooks(args):
+    if not args.hooks_dir:
+        return
+    log.info('running post install hooks ...')
+    if not os.path.isdir(args.hooks_dir):
+        log.warn('hooks directory (%s) does not exist ... skipping',
+                 args.hooks_dir)
+    for hook in sorted(os.listdir(args.hooks_dir)):
+        s = os.stat(os.path.join(args.hooks_dir, hook))
+        if s.st_mode & os.path.stat.S_IXUSR == 0:
+            log.warn('skipping hook (%s) - not executable', hook)
+            continue
+        log.info('executing hook: %s', hook)
+        hook = os.path.join(args.hooks_dir, hook)
+        subprocess.check_call([hook])
+
+
+def _provision(args):
+    log.info('provisioning device ...')
+    if args.image_opt:
+        log.debug('overriding IMAGE_OPT with: %s', args.image_opt)
+        os.environ['IMAGE_OPT'] = args.image_opt
+
+    cargs = [os.path.join(script_dir, 'provision.sh'), '-i', args.image_type]
+
+    if args.package:
+        for p in args.package:
+            cargs.extend(['-p', p])
+    if args.ppa:
+        for p in args.ppa:
+            cargs.extend(['-P', p])
+    if not args.ppa and not args.package:
+        # All tests require a writeable system the -p and -P args
+        # implicitly create a writable system. so we have to ensure here:
+        cargs.append('-w')
+    if args.revision:
+        cargs.extend(['-r', args.revision])
+
+    with statsd.time_it('provision'):
+        start = datetime.datetime.utcnow()
+        passed = False
+        try:
+            _run(cargs)
+            _post_install_hooks(args)
+            passed = True
+        finally:
+            end = datetime.datetime.utcnow()
+            _write_utah(start, end, passed)
+
+
+def _test_autopilot(args, build, image):
+    if args.app:
+        if build:
+            os.environ['DASHBOARD_BUILD'] = build
+        if image:
+            os.environ['DASHBOARD_IMAGE'] = image
+        cargs = [os.path.join(script_dir, 'run-autopilot-tests.sh')]
+        for app in args.app:
+            cargs.extend(['-a', app])
+        with statsd.time_it('APPS'):
+            _run(cargs)
+
+
+def _sync_results(build, image, test, fname):
+    with open(fname) as f:
+        d = yaml.safe_load(f)
+        dashboard_api.result_syncing(image, build, test, d)
+
+
+def _test_utah(args, build, image):
+    if args.test:
+        cargs = [os.path.join(script_dir, 'jenkins.sh')]
+        with statsd.time_it('TESTS'):
+            for test in args.test:
+                os.environ['RESDIR'] = os.path.join(res_dir, test)
+                dashboard_api.result_running(image, build, test)
+                _run(cargs + ['-a', test, '-p', '/tmp/results'],
+                     ignore_error=True)
+                fname = os.path.join(res_dir, test, 'utah.yaml')
+                _sync_results(build, image, test, fname)
+
+
+def _image_add(args):
+    build_number, channel = _image_info()
+    # get the release series (ex. trusty, utopic)
+    # this is set in the job environment variable IMAGE_SERIES
+    release = os.environ.get('IMAGE_SERIES')
+    if release:
+        return dashboard_api.image_add(build_number, release, args.image_type,
+                                       detect_device(None), 'ubuntu')
+
+
+def main(args):
+    with statsd.time_it('main'):
+        if os.path.exists(res_dir):
+            logging.info('deleting old result directory: %s', res_dir)
+            shutil.rmtree(res_dir)
+        os.mkdir(res_dir)
+
+        job_name = os.environ.get('JOB_NAME', '')
+        job_number = os.environ.get('BUILD_NUMBER', '')
+        build = dashboard_api.build_add(job_name, job_number)
+
+        if args.no_provision:
+            logging.info('Skipping the provisioning step as requested')
+        elif args.install_url:
+            _assert_image(args)
+        else:
+            _provision(args)
+
+        # TODO - this should be incororated into provision and assert_image
+        # so that the status is updated *before* flashing rather than after
+        image = _image_add(args)
+
+        if args.test:
+            for x in args.test:
+                dashboard_api.result_queue(image, build, x)
+        if args.app:
+            for x in args.app:
+                dashboard_api.result_queue(image, build, x)
+
+        _test_utah(args, build, image)
+        _test_autopilot(args, build, image)
+
+    return 0
+
+
+if __name__ == '__main__':
+    logging.basicConfig(level=logging.INFO)
+    log.name = 'run-smoke'
+    dashboard.log = logging.getLogger('dashboard')
+
+    args = _get_parser().parse_args()
+    if not _assert_args(args):
+        exit(1)
+
+    exit(main(args))
 
=== added file 'scripts/run-touch-upgrade.sh'
--- scripts/run-touch-upgrade.sh	1970-01-01 00:00:00 +0000
+++ scripts/run-touch-upgrade.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,46 @@
+#!/bin/bash
+
+## This is the script jenkins should run to test upgrading a system image
+## in the lab.
+## Intersting environment variables that must be set:
+##  ANDROID_SERIAL - specify another android device
+##  RUNLIST - the path the runlist
+##  NETWORK_FILE - specify an alternative network file (passed to runlist)
+##  UPGRADE_FROM - the revision to upgrade from, eg -1 (passed to runlist)
+
+set -eux
+
+BASEDIR=$(dirname $(readlink -f $0))
+
+RESDIR=`pwd`/clientlogs
+
+UTAH_PHABLET_CMD="${UTAH_PHABLET_CMD-/usr/share/utah/examples/run_utah_phablet.py}"
+RUNLIST=${RUNLIST-"`pwd`/smoke-touch-apps/upgrade/master.run"}
+ANDROID_SERIAL="${ANDROID_SERIAL-015d1884b20c1c0f}"  #doanac's nexus7 at home
+NETWORK_FILE="${NETWORK_FILE-/home/ubuntu/magners-wifi}"
+
+rm -rf clientlogs
+mkdir clientlogs
+
+export ANDROID_SERIAL=$ANDROID_SERIAL
+
+set +e
+sudo NETWORK_FILE=$NETWORK_FILE \
+	$UTAH_PHABLET_CMD -s $ANDROID_SERIAL \
+	--from-host --skip-install --skip-utah --skip-network  -l $RUNLIST \
+	--results-dir=$RESDIR
+EXITCODE=$?
+
+UTAHFILE=$RESDIR/utah.yaml
+if ! `grep "^errors: [!0]" < $UTAHFILE >/dev/null` ; then
+  echo "errors found"
+  EXITCODE=1
+fi
+if ! `grep "^failures: [!0]" < $UTAHFILE >/dev/null` ; then
+  echo "failures found"
+  EXITCODE=2
+fi
+echo "Results Summary"
+echo "---------------"
+egrep '^(errors|failures|passes|fetch_errors):' $UTAHFILE
+exit $EXITCODE
 
=== added file 'scripts/statsd.py'
--- scripts/statsd.py	1970-01-01 00:00:00 +0000
+++ scripts/statsd.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,35 @@
+#!/usr/bin/python
+
+import os
+import contextlib
+import time
+
+
+_namespace = os.environ.get('STATSD_KEY')
+if _namespace:
+    from txstatsd.client import UdpStatsDClient
+    from txstatsd.metrics.timermetric import TimerMetric
+    from txstatsd.metrics.gaugemetric import GaugeMetric
+    _host = os.environ.get('SERVER', 'snakefruit.canonical.com')
+    _port = int(os.environ.get('PORT', '10041'))
+    _client = UdpStatsDClient(_host, _port)
+    _client.connect()
+
+
[e-mail]
+def time_it(key):
+    start = time.time()
+    try:
+        yield
+    finally:
+        if _namespace:
+            m = TimerMetric(_client, _namespace + '.' + key)
+            m.mark(time.time() - start)
+
+
+def gauge_it(key, array):
+    val = 0
+    if array:
+        val = len(array)
+    if _namespace:
+        GaugeMetric(_client, _namespace + '.' + key).mark(val)
 
=== added directory 'selftests'
=== added file 'selftests/test_junit2utah.py'
--- selftests/test_junit2utah.py	1970-01-01 00:00:00 +0000
+++ selftests/test_junit2utah.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,49 @@
+import StringIO
+import os
+import sys
+import unittest
+
+here = os.path.dirname(__file__)
+sys.path.append(os.path.join(here, '../scripts'))
+
+import junit2utah
+
+RESULT = '''
+<testsuite errors="0" failures="1" name="" tests="4" time="0.001">
+<testcase classname="classname" name="testFails" time="0.000">
+<failure type="testtools.testresult.real._StringException">
+testtools.testresult.real._StringException: Traceback (most recent call last):
+  File "junit2utah.py", line 14, in testFails
+    self.assertFalse(True)
+  File "/usr/lib/python2.7/unittest/case.py", line 418, in assertFalse
+    raise self.failureException(msg)
+AssertionError: True is not false
+</failure>
+</testcase>
+<testcase classname="classname" name="testPasses" time="0.100"/>
+<testcase classname="classname" name="testPasses2" time="0.200"/>
+<testcase classname="classname" name="testSkip" time="0.000">
+<skip>ensure skip works</skip>
+</testcase>
+</testsuite>
+'''
+
+
+class TestJunit2Utah(unittest.TestCase):
+    def testFull(self):
+        stream = StringIO.StringIO(RESULT)
+        results = junit2utah._get_results(stream)
+        self.assertEquals(3, results['passes'])
+        self.assertEquals(1, results['failures'])
+        self.assertEquals(0, results['errors'])
+
+        tcs = results['commands']
+        self.assertEqual('classname', tcs[0]['testsuite'])
+        self.assertEqual('testFails', tcs[0]['testcase'])
+        self.assertIn('AssertionError', tcs[0]['stderr'])
+
+        self.assertEqual('0.200', tcs[2]['time_delta'])
+
+        self.assertEqual('classname', tcs[3]['testsuite'])
+        self.assertEqual('testSkip', tcs[3]['testcase'])
+        self.assertEqual('ensure skip works', tcs[3]['stdout'])
 
=== added file 'selftests/test_reboot_and_wait.py'
--- selftests/test_reboot_and_wait.py	1970-01-01 00:00:00 +0000
+++ selftests/test_reboot_and_wait.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,66 @@
+# Ubuntu Test Cases for Touch
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from __future__ import print_function
+
+import imp
+import mock
+import os
+import unittest
+
+
+class TestRebootAndWait(unittest.TestCase):
+
+    """Simple set of tests to make sure the reboot-and-wait command works."""
+
+    def setUp(self):
+        # do some trickery to load this as module
+        module = 'reboot-and-wait'
+        fname = os.path.join(os.path.dirname(__file__), '../scripts', module)
+        m = imp.load_source(module, fname)
+        self._main = m.main
+        self._get_parser = m._get_arg_parser
+
+    @mock.patch('phabletutils.device.AndroidBridge.reboot')
+    def testRebootFail(self, reboot):
+        reboot.side_effect = RuntimeError('foo')
+        args = self._get_parser().parse_args([])
+        with self.assertRaisesRegexp(RuntimeError, 'foo'):
+            self._main(args)
+
+    @mock.patch('phabletutils.device.AndroidBridge.reboot')
+    @mock.patch('phabletutils.device.AndroidBridge.wait_for_device')
+    def testWaitForDeviceFail(self, wait_for, reboot):
+        wait_for.side_effect = RuntimeError('foo')
+        args = self._get_parser().parse_args([])
+        with self.assertRaisesRegexp(RuntimeError, 'foo'):
+            self._main(args)
+        reboot.assert_called_once_with()
+
+    @mock.patch('phabletutils.device.AndroidBridge.reboot')
+    @mock.patch('phabletutils.device.AndroidBridge.wait_for_device')
+    @mock.patch('phabletutils.device.AndroidBridge.wait_for_network')
+    def testRetries(self, wait_for_net, wait_for_dev, reboot):
+        args = self._get_parser().parse_args([])
+        wait_for_net.side_effect = RuntimeError('foo')
+        self.assertEquals(1, self._main(args))
+        self.assertEquals(args.num_tries, reboot.call_count)
+
+        # now make sure it can recover after a single network failure
+        reboot.reset_mock()
+        wait_for_net.reset_mock()
+        wait_for_net.side_effect = [RuntimeError('foo'), None]
+        self.assertEquals(0, self._main(args))
+        self.assertEquals(2, reboot.call_count)
 
=== added file 'selftests/test_run_smoke.py'
--- selftests/test_run_smoke.py	1970-01-01 00:00:00 +0000
+++ selftests/test_run_smoke.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,143 @@
+# Ubuntu Test Cases for Touch
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import imp
+import mock
+import os
+import subprocess
+import sys
+import unittest
+
+
+class TestRunSmoke(unittest.TestCase):
+
+    """Simple set of tests to make sure the smoke-run command works."""
+
+    def setUp(self):
+        # load the module
+        module = 'run-smoke'
+        path = os.path.join(os.path.dirname(__file__), '../scripts')
+        sys.path.append(path)
+        fname = os.path.join(path, module)
+        self.run_smoke = imp.load_source(module, fname)
+        if 'ANDROID_SERIAL' in os.environ:
+            del os.environ['ANDROID_SERIAL']
+
+    @mock.patch.dict('os.environ')
+    @mock.patch('subprocess.check_output')
+    def testSerialRequired(self, check_output):
+        '''Ensure android serial is required when appropriate'''
+        check_output.return_value = '1'.encode()
+        self.assertTrue(self.run_smoke._serial_required())
+
+        check_output.return_value = '1\n2\n3\n4'.encode()
+        self.assertFalse(self.run_smoke._serial_required())
+
+        check_output.return_value = '1\n2\n3\n4\n5'.encode()
+        self.assertTrue(self.run_smoke._serial_required())
+
+        # make sure serial isn't required if specified in env
+        os.environ['ANDROID_SERIAL'] = 'foo'
+        check_output.return_value = '1\n2\n3\n4\n5'.encode()
+        self.assertFalse(self.run_smoke._serial_required())
+
+    @mock.patch('statsd.gauge_it')
+    def testAssertArgs(self, gauge):
+        '''Ensure install-url is used properly'''
+        patterns = [
+            (['--install-url', 'x', '-p', 'x'], False),
+            (['--install-url', 'x', '-P', 'x'], False),
+            (['--install-url', 'x', '--image-opt', 'x'], False),
+            (['-p', 'x', '-P', 'x', '--image-opt', 'x'], True),
+        ]
+        for pat, val in patterns:
+            args = self.run_smoke._get_parser().parse_args(['-s', 'foo'] + pat)
+            self.assertEqual(val, self.run_smoke._assert_args(args))
+
+        # ensure the -p ALL pulls in all packages
+        gauge.reset_mock()
+        args = self.run_smoke._get_parser().parse_args(['-p', 'ALL'])
+        self.assertTrue(self.run_smoke._assert_args(args))
+        self.assertTrue(len(args.package) > 1)
+        args = gauge.call_args_list[0][0]
+        self.assertEqual('PACKAGES', args[0])
+        self.assertLess(1, len(args[1]))
+
+        args = gauge.call_args_list[1][0]
+        self.assertEqual('APPS', args[0])
+        self.assertIsNone(args[1])
+        args = gauge.call_args_list[2][0]
+        self.assertEqual('TESTS', args[0])
+        self.assertIsNone(args[1])
+
+        # don't bother checking gauge calls for the remaining "ALL" tests
+
+        # ensure the -a ALL pulls in all APPS
+        args = self.run_smoke._get_parser().parse_args(['-a', 'ALL'])
+        self.assertTrue(self.run_smoke._assert_args(args))
+        self.assertTrue(len(args.app) > 1)
+
+        # ensure the -t ALL pulls in all TESTS
+        args = self.run_smoke._get_parser().parse_args(['-t', 'ALL'])
+        self.assertTrue(self.run_smoke._assert_args(args))
+        self.assertTrue(len(args.test) > 1)
+
+    def testAssertArgsEnv(self):
+        '''Ensure we pull in environment variables that jenkins uses.'''
+        with mock.patch.dict('os.environ'):
+            os.environ['APPS'] = 'apps'
+            os.environ['TESTS'] = 'tests'
+            os.environ['PACKAGES'] = 'packages'
+            os.environ['PPAS'] = 'ppas'
+            os.environ['IMAGE_TYPE'] = 'type'
+            os.environ['INSTALL_URL'] = 'url'
+            os.environ['IMAGE_OPT'] = 'opts opts'
+            os.environ['ANDROID_SERIAL'] = 'foo'
+
+            args = self.run_smoke._get_parser().parse_args([])
+            self.assertTrue(self.run_smoke._assert_args(args))
+
+            self.assertEqual(args.app, ['apps'])
+            self.assertEqual(args.test, ['tests'])
+            self.assertEqual(args.package, ['packages'])
+            self.assertEqual(args.ppa, ['ppas'])
+            self.assertEqual(args.image_type, 'type')
+            self.assertEqual(args.install_url, 'url')
+            self.assertEqual(args.image_opt, 'opts opts')
+
+    def testProvision(self):
+        orig = os.environ.get('IMAGE_OPT', '')
+        with mock.patch.object(self.run_smoke, '_run') as run:
+            args = self.run_smoke._get_parser().parse_args(
+                ['-s', 'foo', '--image-opt', 'FOOBAR', '-p', '1', '-p', '2'])
+            self.run_smoke._provision(args)
+            self.assertTrue(run.called)
+            val = os.environ.get('IMAGE_OPT')
+            os.environ['IMAGE_OPT'] = orig
+            self.assertEqual('FOOBAR', val)
+
+    def testUtahTests(self):
+        args = self.run_smoke._get_parser().parse_args(
+            ['-s', 'foo', '-t', 'a', '-t', 'b'])
+        with mock.patch.object(self.run_smoke, '_run') as run:
+            with mock.patch.dict('os.environ'):
+                run.side_effects = [subprocess.CalledProcessError, None]
+                with mock.patch.object(self.run_smoke, '_sync_results') as sr:
+                    self.run_smoke._test_utah(args, None, None)
+                    p = os.path.join(self.run_smoke.res_dir, 'b')
+                    # ensuring b ran means, that we handled the failure of test
+                    # 'a' and that the environment is setup correctly
+                    self.assertEqual(os.environ['RESDIR'], p)
+                    self.assertTrue(sr.called)
 
=== added file 'selftests/test_statsd.py'
--- selftests/test_statsd.py	1970-01-01 00:00:00 +0000
+++ selftests/test_statsd.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,64 @@
+# Ubuntu Test Cases for Touch
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+import mock
+import unittest
+import sys
+import time
+
+path = os.path.join(os.path.dirname(__file__), '../scripts')
+sys.path.append(path)
+
+import statsd
+
+
+class TestStatsd(unittest.TestCase):
+
+    def setUp(self):
+        self.origkey = os.environ.get('STATSD_KEY')
+        os.environ['STATSD_KEY'] = 'prefix'
+        reload(statsd)
+
+    def tearDown(self):
+        if not self.origkey:
+            del os.environ['STATSD_KEY']
+        else:
+            os.environ['STATSD_KEY'] = self.origkey
+        reload(statsd)
+
+    """Simple set of tests to make sure the statsd calls work."""
+
+    @mock.patch('txstatsd.metrics.metric.Metric.write')
+    def testGaugeIt(self, _statsd):
+        statsd.gauge_it('foo', None)
+        _statsd.assert_called_with('prefix.foo:0|g')
+        _statsd.reset_mock()
+
+        statsd.gauge_it('foo', [])
+        _statsd.assert_called_with('prefix.foo:0|g')
+        _statsd.reset_mock()
+
+        statsd.gauge_it('foo', [1, 2, 3])
+        _statsd.assert_called_with('prefix.foo:3|g')
+        _statsd.reset_mock()
+
+    @mock.patch('txstatsd.metrics.metric.Metric.write')
+    def testTimeIt(self, _statsd):
+        with statsd.time_it('foo'):
+            time.sleep(0.1)
+        # should have a timing of about 100ms
+        self.assertRegexpMatches(
+            _statsd.call_args[0][0], 'prefix.foo:100.[\d+]|ms')
 
=== added directory 'tests'
=== added directory 'tests/bootspeed'
=== added directory 'tests/bootspeed/bootchart'
=== added file 'tests/bootspeed/bootchart/run.py'
--- tests/bootspeed/bootchart/run.py	1970-01-01 00:00:00 +0000
+++ tests/bootspeed/bootchart/run.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,73 @@
+#!/usr/bin/python
+
+import datetime
+import json
+import os
+import shutil
+import subprocess
+
+
+def _get_dashboard_data(timing_file):
+    info = subprocess.check_output(['adb', 'shell', 'system-image-cli', '-i'])
+    data = {
+        'image_md5': 'n/a',
+        'machine_mac': 'ff:ff:ff:ff:ff:ff',
+        'ran_at': datetime.datetime.now().isoformat(),
+        'kernel_init': 0.0,
+    }
+    for line in info.split('\r\n'):
+        if not line:
+            continue
+        key, val = line.split(':', 1)
+        val = val.strip()
+        if key == 'device name':
+            data['image_arch'] = val
+        elif key == 'channel':
+            # get 'touch' and 'trusty' from something like:
+            # ubuntu-touch/trusty-proposed
+            variant, release = val.split('/')
+            data['image_variant'] = variant.split('-')[1]
+            data['image_release'] = release.split('-')[0]
+        elif key == 'version version':
+            data['number'] = val
+        elif key == 'version ubuntu':
+            data['version_ubuntu'] = val
+        elif key == 'version device':
+            data['version_device'] = val
+    data['build_number'] = '%s:%s:%s' % (
+        data['number'], data['version_ubuntu'], data['version_device'])
+
+    with open(timing_file) as f:
+        # the timings file is sequence of readings (in hundredths of a second):
+        #  line 0 - the total boot time
+        #  line X - the time from boot until the given annotation *started*
+        timings = [float(x) / 100.0 for x in f.read().split('\n') if x]
+        data['boot'] = timings[0]
+        data['kernel'] = timings[1]
+        data['plumbing'] = timings[2] - timings[1]
+        data['xorg'] = timings[3] - timings[2]
+        data['desktop'] = timings[0] - timings[3]
+    return data
+
+
+def chart(results_dir):
+    timings = os.path.join(results_dir, 'timings')
+    os.environ['CHARTOPTS'] = ' '.join([
+        '--crop-after=unity8',
+        '--annotate=mountall',
+        '--annotate=lightdm',
+        '--annotate=unity8',
+        '--annotate-file=%s' % timings,
+    ])
+    subprocess.check_call(['phablet-bootchart', '-n', '-k',
+                           '-w', '/home/ubuntu/magners-wifi',
+                           '-o', results_dir])
+    data = _get_dashboard_data(timings)
+    with open(os.path.join(results_dir, 'boot.json'), 'w') as f:
+        json.dump(data, f, indent=4)
+
+if __name__ == '__main__':
+    # run_utah_phablet will pass us a "UTAH_PROBE_DIR":
+    resdir = os.environ.get('UTAH_PROBE_DIR', '/tmp/results')
+    for x in range(3):
+        chart(os.path.join(resdir, str(x)))
 
=== added file 'tests/bootspeed/bootchart/setup.sh'
--- tests/bootspeed/bootchart/setup.sh	1970-01-01 00:00:00 +0000
+++ tests/bootspeed/bootchart/setup.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,4 @@
+#!/bin/bash
+
+${TARGET_PREFIX} apt-get install bootchart
+
 
=== added file 'tests/bootspeed/bootchart/tc_control'
--- tests/bootspeed/bootchart/tc_control	1970-01-01 00:00:00 +0000
+++ tests/bootspeed/bootchart/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,12 @@
+description: run phablet-bootchart on a phone
+dependencies: none
+action: |
+  1. setup bootchart
+  2. reboot 3 times
+  3. collect
+expected_results: |
+  1. boot chart data
+type: userland
+timeout: 2000
+command: ./run.py
+tc_setup: ./setup.sh
 
=== added file 'tests/bootspeed/master.run'
--- tests/bootspeed/master.run	1970-01-01 00:00:00 +0000
+++ tests/bootspeed/master.run	2014-09-10 13:48:46 +0000
@@ -0,0 +1,5 @@
+---
+testsuites:
+  - name: bootspeed
+    fetch_method: dev
+    fetch_location: ./
 
=== added file 'tests/bootspeed/tslist.run'
--- tests/bootspeed/tslist.run	1970-01-01 00:00:00 +0000
+++ tests/bootspeed/tslist.run	2014-09-10 13:48:46 +0000
@@ -0,0 +1,1 @@
+- test: bootchart
 
=== added directory 'tests/click_image_tests'
=== added directory 'tests/click_image_tests/check_preinstalled_list'
=== added file 'tests/click_image_tests/check_preinstalled_list/check_preinstalled_list.py'
--- tests/click_image_tests/check_preinstalled_list/check_preinstalled_list.py	1970-01-01 00:00:00 +0000
+++ tests/click_image_tests/check_preinstalled_list/check_preinstalled_list.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,61 @@
+#!/usr/bin/python3
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import subprocess
+import unittest
+import urllib.request
+
+
+click_list_url = \
+    'http://people.canonical.com/~ubuntu-archive/click_packages/click_list'
+
+
+def get_install_list():
+    request = urllib.request.urlopen(click_list_url).read().decode('utf-8')
+    click_files = [x for x in request.split('\n') if x]
+    click_apps = {}
+    for entry in click_files:
+        entry_parts = entry.split('_')
+        click_apps[entry_parts[0]] = entry_parts[1]
+    return click_apps
+
+
+def get_image_list():
+    click_list = subprocess.check_output(
+        ['adb', 'shell', 'sudo', '-iu', 'phablet',
+         'click', 'list']).decode('utf-8').split('\n')
+    click_entries = [x for x in click_list if x]
+    click_apps = {}
+    for entry in click_entries:
+        entry_parts = entry.split('\t')
+        click_apps[entry_parts[0]] = entry_parts[1].strip()
+    return click_apps
+
+
+class ClickPreinstalled(unittest.TestCase):
+
+    def setUp(self):
+        self.image_list = get_image_list()
+        self.install_list = get_install_list()
+        print('Search for {} on image'.format(list(self.install_list.keys())))
+
+    def testPreinstalled(self):
+        for entry in self.install_list:
+            self.assertIn(entry, list(self.image_list.keys()))
+
+
+if __name__ == '__main__':
+    unittest.main()
 
=== added file 'tests/click_image_tests/check_preinstalled_list/tc_control'
--- tests/click_image_tests/check_preinstalled_list/tc_control	1970-01-01 00:00:00 +0000
+++ tests/click_image_tests/check_preinstalled_list/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,10 @@
+description: check that all the expected click packages are installed
+dependencies: a touch read-only image
+action: |
+  1. download the list of click packages
+  2. list the current click packages installed on the device
+expected_results: |
+  1. the current list of click packages from the archive should match
+type: userland
+timeout: 120
+command: ./check_preinstalled_list.py
 
=== added file 'tests/click_image_tests/master.run'
--- tests/click_image_tests/master.run	1970-01-01 00:00:00 +0000
+++ tests/click_image_tests/master.run	2014-09-10 13:48:46 +0000
@@ -0,0 +1,5 @@
+---
+testsuites:
+  - name: click_image_tests
+    fetch_method: dev
+    fetch_location: ./
 
=== added file 'tests/click_image_tests/tslist.run'
--- tests/click_image_tests/tslist.run	1970-01-01 00:00:00 +0000
+++ tests/click_image_tests/tslist.run	2014-09-10 13:48:46 +0000
@@ -0,0 +1,1 @@
+- test: check_preinstalled_list
 
=== added directory 'tests/customizations'
=== added file 'tests/customizations/master.run'
--- tests/customizations/master.run	1970-01-01 00:00:00 +0000
+++ tests/customizations/master.run	2014-09-10 13:48:46 +0000
@@ -0,0 +1,5 @@
+---
+testsuites:
+  - name: customizations
+    fetch_method: dev
+    fetch_location: ./
 
=== added file 'tests/customizations/setup.sh'
--- tests/customizations/setup.sh	1970-01-01 00:00:00 +0000
+++ tests/customizations/setup.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,10 @@
+#!/bin/sh
+
+set -ex
+
+BZR_HOME=/dev/null bzr export customization_tests lp:savilerow/tests
+
+# copy the autopilot scripts over if needed (ie we are testing from HOST)
+[ -z $ANDROID_SERIAL ] || adb push ./customization_tests /home/phablet/autopilot/customization_tests
+
+prepare-autopilot-test.sh
 
=== added file 'tests/customizations/ts_control'
--- tests/customizations/ts_control	1970-01-01 00:00:00 +0000
+++ tests/customizations/ts_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,1 @@
+ts_setup: ./setup.sh
 
=== added file 'tests/customizations/tslist.auto'
--- tests/customizations/tslist.auto	1970-01-01 00:00:00 +0000
+++ tests/customizations/tslist.auto	2014-09-10 13:48:46 +0000
@@ -0,0 +1,4 @@
+-
+  discovery_cmd: autopilot-list -3 customization_tests
+  test_cmd: autopilot-run -3 {}
+
 
=== added directory 'tests/default'
=== added directory 'tests/default/apport'
=== added file 'tests/default/apport/tc_control'
--- tests/default/apport/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/apport/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,9 @@
+description: check that apport is enabled
+dependencies: none
+action: |
+  1. status apport | grep running
+expected_results: |
+  1. status apport says that it is running
+type: userland
+timeout: 60
+command: $TARGET_PREFIX status apport | grep running
 
=== added directory 'tests/default/ifconfig'
=== added file 'tests/default/ifconfig/tc_control'
--- tests/default/ifconfig/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/ifconfig/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,13 @@
+description: make sure wlan0 exists
+dependencies: none
+action: |
+  1. ifconfig
+  2. Make sure there is a wlan0
+expected_results: |
+  1. there is a wlan0 on the system
+type: userland
+timeout: 60
+command: $TARGET_PREFIX ifconfig | grep wlan0
+#build_cmd:
+#tc_setup:
+#tc_cleanup:
 
=== added directory 'tests/default/install'
=== added file 'tests/default/install/tc_control'
--- tests/default/install/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/install/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,12 @@
+description: apt-get install works
+dependencies: openssh-server
+action: |
+  1. apt-get install -y curl
+expected_results: |
+  1. apt-get install can install a package and ends with exit code 0
+type: userland
+timeout: 120
+command: $TARGET_PREFIX sudo apt-get install -y curl
+#build_cmd:
+#tc_setup:
+tc_cleanup: $TARGET_PREFIX sudo apt-get remove -y curl
 
=== added file 'tests/default/master.run'
--- tests/default/master.run	1970-01-01 00:00:00 +0000
+++ tests/default/master.run	2014-09-10 13:48:46 +0000
@@ -0,0 +1,5 @@
+---
+testsuites:
+  - name: smoke_touch
+    fetch_method: dev
+    fetch_location: ./
 
=== added directory 'tests/default/netstat'
=== added file 'tests/default/netstat/tc_control'
--- tests/default/netstat/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/netstat/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,12 @@
+description: netstat -l lists all the services
+dependencies: none
+action: |
+  1. netstat -l
+expected_results: |
+  1. netstat shows output and exists cleanly
+type: userland
+timeout: 60
+command: $TARGET_PREFIX netstat -l
+#build_cmd:
+#tc_setup:
+#tc_cleanup:
 
=== added directory 'tests/default/ping'
=== added file 'tests/default/ping/pingtest.sh'
--- tests/default/ping/pingtest.sh	1970-01-01 00:00:00 +0000
+++ tests/default/ping/pingtest.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,12 @@
+#!/bin/sh
+
+set -e
+
+if [ -z $TARGET_PREFIX ] ; then
+    echo "RUNNING ON TARGET"
+    ping -c 5 $(ip route show | head -n1 |cut -d" " -f3)
+else
+    echo "RUNNING FROM HOST"
+    ${TARGET_PREFIX} ping -c 5 '$(ip route show | head -n1 |cut -d" " -f3)'
+fi
+
 
=== added file 'tests/default/ping/tc_control'
--- tests/default/ping/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/ping/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,12 @@
+description: ping -c 5 $gateway
+dependencies: None
+action: |
+  1. ping -c 5 $gateway
+expected_results: |
+  1. ping ends without an error
+type: userland
+timeout: 60
+command: ./pingtest.sh
+#build_cmd:
+#tc_setup:
+#tc_cleanup:
 
=== added directory 'tests/default/pwd'
=== added file 'tests/default/pwd/tc_control'
--- tests/default/pwd/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/pwd/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,9 @@
+description: pwd works
+dependencies: none
+action: |
+  1. cd to /tmp and verify which is the current directory
+expected_results: |
+  1. Current directory is /tmp
+type: userland
+timeout: 60
+command: ./test.sh
 
=== added file 'tests/default/pwd/test.sh'
--- tests/default/pwd/test.sh	1970-01-01 00:00:00 +0000
+++ tests/default/pwd/test.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,18 @@
+#!/bin/sh
+
+set -e
+
+if [ -n "$TARGET_PREFIX" ] ; then
+	echo "RUNNING FROM HOST"
+else
+	echo "RUNNING ON TARGET"
+	TARGET_PREFIX="/bin/sh -c"
+fi
+
+#when it comes from adb we get a \r\n that should be removed
+dir=$($TARGET_PREFIX "cd /tmp; pwd" | head -n1 | tr -d '\r\n')
+if [ $dir != "/tmp" ] ; then
+	echo "failed to change directory"
+	exit 1
+fi
+exit 0
 
=== added directory 'tests/default/route'
=== added file 'tests/default/route/tc_control'
--- tests/default/route/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/route/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,12 @@
+description: route shows wlan0 routes
+dependencies: none
+action: |
+  1. route -n | grep wlan0
+expected_results: |
+  1. route shows wlan0 routes
+type: userland
+timeout: 60
+command: $TARGET_PREFIX route -n | grep wlan0
+#build_cmd:
+#tc_setup:
+#tc_cleanup:
 
=== added directory 'tests/default/systemsettle'
=== added file 'tests/default/systemsettle/systemsettle.sh'
--- tests/default/systemsettle/systemsettle.sh	1970-01-01 00:00:00 +0000
+++ tests/default/systemsettle/systemsettle.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,127 @@
+#!/bin/bash
+
+# Configuration variables:
+#  TARGET_PREFIX - Allows this to be run from the host, by providings something
+#                  like TARGET_PREFIX="adb shell"
+#  UTAH_PROBE_DIR - optionally where to save log files so utah will grab them
+
+set -e
+
+[ -z $UTAH_PROBE_DIR ] && UTAH_PROBE_DIR="/tmp"
+
+# default exit code storage
+dump_error=1
+
+calc () { awk "BEGIN{ print $* }" ;}
+
+function show_usage() {
+   echo "Usage:"
+   echo " $0 [options]"
+   echo "Options:"
+   echo " -r  run forever without exiting"
+   echo " -p  minimum idle percent to wait for (Default: 99)"
+   echo " -c  number of times to run top at each iteration (Default: 10)"
+   echo " -d  seconds to delay between each top iteration (Default: 6)"
+   echo " -i  top measurements to ignore from each loop (Default: 1)"
+   echo " -m  maximum loops of top before giving up if minimum idle"
+   echo "     percent is not reached (Default: 10)"
+   echo " -l  label to include for the top_log file"
+   echo " -s  sleep timeout for %cpu calculation (Default: 10)"
+   exit 129
+}
+
+while getopts "h?rp:c:d:i:m:l:s:" opt; do
+    case "$opt" in
+        h|\?) show_usage
+              ;;
+        r)    settle_prefix='-'
+              ;;
+        p)    idle_avg_min=$OPTARG
+              ;;
+        c)    top_repeat=$OPTARG
+              ;;
+        d)    top_wait=$OPTARG
+              ;;
+        i)    top_ignore=$OPTARG
+              ;;
+        m)    settle_max=$OPTARG
+              ;;
+        l)    top_log_label=$OPTARG
+              ;;
+        s)    sleep_len=$OPTARG
+              ;;
+    esac
+done
+
+sleep_len=${sleep_len:-10}
+HZ=`getconf CLK_TCK`
+# minimum average idle level required to succeed
+idle_avg_min=${idle_avg_min:-99}
+# measurement details: top $top_wait $top_repeat
+top_repeat=${top_repeat:-10}
+top_wait=${top_wait:-6}
+# how many samples to ignore
+top_ignore=${top_ignore:-1}
+# how many total attempts to settle the system
+settle_max=${settle_max:-10}
+
+top_log="$UTAH_PROBE_DIR/top$top_log_label.log"
+
+# set and calc more runtime values
+top_tail=`calc $top_repeat - $top_ignore`
+settle_count=0
+idle_avg=0
+
+echo "System Settle run - quiesce the system"
+echo "--------------------------------------"
+echo
+echo "  idle_avg_min   = '$idle_avg_min'"
+echo "  top_repeat  = '$top_repeat'"
+echo "  top_wait    = '$top_wait'"
+echo "  top_ignore  = '$top_ignore'"
+echo "  settle_max     = '$settle_max'"
+echo "  run_forever    = '$settle_prefix' (- = yes)"
+echo "  log files   = $top_log $top_log.reduced"
+echo
+
+while test `calc $idle_avg '<' $idle_avg_min` = 1 -a "$settle_prefix$settle_count" -lt "$settle_max"; do
+  echo -n "Starting system idle measurement (run: $settle_count) ... "
+
+  # get top
+  echo "TOP DUMP (after settle run: $settle_count)" >> $top_log
+  echo "========================" >> $top_log
+  ${TARGET_PREFIX} COLUMNS=900 top -c -b -d $top_wait -n $top_repeat >> $top_log
+  cat $top_log | grep '.Cpu.*' | tail -n $top_tail > $top_log.reduced
+  echo >> $top_log
+
+  # Instead of using top, we need to use /proc/stat and compensate for
+  # the number of cpus and any frequency scaling that could be in effect
+  cpu_avg=$({
+    cat /proc/stat
+    sleep "$sleep_len"
+    cat /proc/stat
+  } | awk '
+    BEGIN       { iter = 0 }
+    /^cpu /     { iter = iter + 1; count = 0; next }
+    /^cpu/      { S[iter] = S[iter] + ($2+$3+$4+$6); count = count + 1;
+next }
+    END     { print (S[2] - S[1]) * 100 / ('"$HZ"' * count * '"$sleep_len"') }
+  ')
+  idle_avg=`calc 100 - $cpu_avg`
+  settle_count=`calc $settle_count + 1`
+
+  echo " DONE."
+  echo
+  echo "Measurement:"
+  echo "  + idle level: $idle_avg"
+  echo
+done
+
+if test `calc $idle_avg '<' $idle_avg_min` = 1; then
+  echo "system not settled. FAIL"
+  exit 1
+else
+  echo "system settled. SUCCESS"
+  exit 0
+fi
+
 
=== added file 'tests/default/systemsettle/tc_control'
--- tests/default/systemsettle/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/systemsettle/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,9 @@
+description: check if system settles to idle average > 99.25%
+dependencies: none
+action: |
+  1. Take CPU load samples for 10 minutes and fail if average idle never goes above 99.25% percent
+expected_results: |
+  1. When doing nothing, system calms down to at least 99.25% idle level
+type: userland
+timeout: 720
+command: ./systemsettle.sh -c5 -d6 -p 97.5
 
=== added file 'tests/default/ts_control'
--- tests/default/ts_control	1970-01-01 00:00:00 +0000
+++ tests/default/ts_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,3 @@
+#build_cmd: echo building
+#ts_setup: echo setting up
+#ts_cleanup: echo cleaning up
 
=== added file 'tests/default/tslist.run'
--- tests/default/tslist.run	1970-01-01 00:00:00 +0000
+++ tests/default/tslist.run	2014-09-10 13:48:46 +0000
@@ -0,0 +1,12 @@
+- test: pwd
+- test: uname
+- test: vmstat
+- test: systemsettle
+- test: netstat
+- test: ifconfig
+- test: route
+- test: ping
+- test: install
+- test: unity8
+- test: apport
+- test: whoopsie
 
=== added directory 'tests/default/uname'
=== added file 'tests/default/uname/tc_control'
--- tests/default/uname/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/uname/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,12 @@
+description: uname shows an ubuntu-phablet image installed
+dependencies: CHANGE_ME
+action: |
+  1. uname -a|grep ubuntu-phablet
+expected_results: |
+  1. ubuntu-phablet is installed as expected
+type: userland
+timeout: 60
+command: $TARGET_PREFIX uname -a | grep ubuntu-phablet
+#build_cmd:
+#tc_setup:
+#tc_cleanup:
 
=== added directory 'tests/default/unity8'
=== added file 'tests/default/unity8/tc_control'
--- tests/default/unity8/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/unity8/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,9 @@
+description: check for unity8 process
+dependencies: None
+action: |
+  1. pgrep unity8
+expected_results: |
+  1. pgrep exits with 0, indicating it found a matching process
+type: userland
+timeout: 60
+command: $TARGET_PREFIX /usr/bin/pgrep unity8
 
=== added directory 'tests/default/vmstat'
=== added file 'tests/default/vmstat/tc_control'
--- tests/default/vmstat/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/vmstat/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,12 @@
+description: Execute vmstat and capture output
+dependencies: none
+action: |
+  1. vmstat
+expected_results: |
+  1. vmstat finishes without an error
+type: userland
+timeout: 60
+command: $TARGET_PREFIX vmstat
+#build_cmd:
+#tc_setup:
+#tc_cleanup:
 
=== added directory 'tests/default/whoopsie'
=== added file 'tests/default/whoopsie/tc_control'
--- tests/default/whoopsie/tc_control	1970-01-01 00:00:00 +0000
+++ tests/default/whoopsie/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,9 @@
+description: verify whoopsie functionality
+dependencies: None
+action: |
+  1. trigger a segfault and verify that whoopsie uploads it
+expected_results: |
+  1. /var/crash/_bin_sleep.0.uploaded is created, indicating a successful upload
+type: userland
+timeout: 720
+command: ./whoopsie-test.sh
 
=== added file 'tests/default/whoopsie/whoopsie-test.sh'
--- tests/default/whoopsie/whoopsie-test.sh	1970-01-01 00:00:00 +0000
+++ tests/default/whoopsie/whoopsie-test.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,86 @@
+#!/bin/sh
+
+set -e
+
+cleanup()
+{
+	stop whoopsie || true
+	rm -f /var/crash/_bin_sleep.0.*
+	start whoopsie
+}
+
+if [ ! -e /var/lib/apport/autoreport ]; then
+    echo "Automatic crash reporting not enabled."
+    exit 1
+fi
+
+if [ -e /var/crash/_bin_sleep.0.crash ] || [ -e /var/crash/_bin_sleep.0.upload ] \
+   || [ -e /var/crash/_bin_sleep.0.uploaded ]
+then
+	echo "Error: existing crash data found on disk.  Did a previous run fail?"
+	exit 1
+fi
+
+trap cleanup 0 2 3 5 10 13 15
+
+echo "Configuring whoopsie for staging"
+stop whoopsie
+if ! start whoopsie CRASH_DB_URL=https://daisy.staging.ubuntu.com
+then
+	echo "Failed to start whoopsie"
+	exit 1
+fi
+
+sleep 10 &
+echo "Background process id $!"
+ps $!
+echo "Sending SIGSEGV"
+kill -SEGV $!
+
+echo "Polling for .crash file"
+crash_found=false
+for i in $(seq 0 99); do
+	if [ -e /var/crash/_bin_sleep.0.crash ]; then
+		echo "Crash file created after $(($i*2)) seconds."
+		crash_found=:
+		break
+	fi
+	sleep 2
+done
+if ! $crash_found; then
+	echo "whoopsie failed to create crash file within 200 seconds."
+	exit 1
+fi
+
+echo "Polling for .upload file"
+upload_found=false
+for i in $(seq 0 19); do
+	if [ -e /var/crash/_bin_sleep.0.upload ]; then
+		echo "Upload file created after $i seconds."
+		upload_found=:
+		break
+	fi
+	sleep 1
+done
+if ! $upload_found; then
+	echo "whoopsie_upload_all failed to run within 20 seconds."
+	exit 1
+fi
+
+echo "Polling for .uploaded file"
+uploaded_found=false
+for i in $(seq 0 79); do
+	if [ -e /var/crash/_bin_sleep.0.uploaded ]; then
+		echo ".uploaded file created after $(($i*5)) seconds."
+		uploaded_found=:
+		break
+	fi
+	sleep 5
+done
+if ! $uploaded_found; then
+	echo "whoopsie failed to upload crash file within 400 seconds."
+	exit 1
+fi
+
+echo "crash file successfully uploaded."
+exit 0
 
=== added directory 'tests/eventstat'
=== added directory 'tests/eventstat/eventstat'
=== added file 'tests/eventstat/eventstat/eventstat.sh'
--- tests/eventstat/eventstat/eventstat.sh	1970-01-01 00:00:00 +0000
+++ tests/eventstat/eventstat/eventstat.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,5 @@
+#!/bin/bash
+: ${DURATION:=60}
+: ${COUNT:=10}
+${TARGET_PREFIX} "echo \"{\\\"duration\\\": $DURATION, \\\"count\\\": $COUNT}\" > /tmp/results/options.json"
+${TARGET_PREFIX} "eventstat -CSl -r /tmp/results/eventstat.csv $DURATION $COUNT > /tmp/results/eventstat.log"
 
=== added file 'tests/eventstat/eventstat/setup.sh'
--- tests/eventstat/eventstat/setup.sh	1970-01-01 00:00:00 +0000
+++ tests/eventstat/eventstat/setup.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+${TARGET_PREFIX} mkdir -p /tmp/results
+NO_UNLOCK=1 PKGS="eventstat" prepare-autopilot-test.sh
+#Let the system quiet down for a little while
+sleep 300
+
 
=== added file 'tests/eventstat/eventstat/tc_control'
--- tests/eventstat/eventstat/tc_control	1970-01-01 00:00:00 +0000
+++ tests/eventstat/eventstat/tc_control	2014-09-10 13:48:46 +0000
@@ -0,0 +1,12 @@
+description: Gather memory usage information
+dependencies: none
+action: |
+  1. gather eventstat data
+expected_results: |
+  1. eventstat.log should be saved to /tmp/results
+type: userland
+timeout: 800
+command: ./eventstat.sh
+#build_cmd:
+tc_setup: ./setup.sh
+#tc_cleanup:
 
=== added file 'tests/eventstat/master.run'
--- tests/eventstat/master.run	1970-01-01 00:00:00 +0000
+++ tests/eventstat/master.run	2014-09-10 13:48:46 +0000
@@ -0,0 +1,5 @@
+---
+testsuites:
+  - name: eventstat
+    fetch_method: dev
+    fetch_location: ./
 
=== added file 'tests/eventstat/tslist.run'
--- tests/eventstat/tslist.run	1970-01-01 00:00:00 +0000
+++ tests/eventstat/tslist.run	2014-09-10 13:48:46 +0000
@@ -0,0 +1,1 @@
+- test: eventstat
 
=== added directory 'tests/health-check'
=== added file 'tests/health-check/health-check-test-get-pids.py'
--- tests/health-check/health-check-test-get-pids.py	1970-01-01 00:00:00 +0000
+++ tests/health-check/health-check-test-get-pids.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,38 @@
+#!/usr/bin/python3
+
+import os
+import psutil
+import platform
+
+basedir = os.path.dirname(__file__)
+default_threshold_path = os.path.join(
+    os.path.join(basedir, 'thresholds'), platform.machine())
+
+ignore_procs = [
+    'health-check', 'init', 'cat', 'vi', 'emacs', 'getty', 'csh', 'bash',
+    'sh', 'powerd', 'thermald', 'mpdecision', 'polkitd', 'whoopsie',
+    'mediascanner-service', 'window-stack-bridge', 'dconf-service',
+    'pulseaudio', 'hud-service', 'indicator-bluetooth-service',
+    'indicator-location-service', 'indicator-sound-service',
+    'indicator-secret-agent', 'mtp-server', 'address-book-service',
+    'dnsmasq', 'systemd-logind', 'systemd-udevd']
+
+with open(os.path.join(basedir, 'procmapping.txt'), 'w') as mapping:
+    procnames = {}
+    for p in psutil.process_iter():
+        try:
+            proc = p.as_dict(attrs=['pid', 'name'])
+            if os.getpgid(proc['pid']) != 0:
+                procname = os.path.basename(proc['name'])
+                fname = os.path.join(
+                    default_threshold_path, procname + ".threshold")
+                count = procnames.get(procname, 0)
+                procnames[procname] = count + 1
+                if count > 0:
+                    procname = '%s_%d' % (procname, count)
+                if not proc['name'] in ignore_procs:
+                    if os.path.isfile(fname):
+                        mapping.write('%s:%d\n' % (procname, proc['pid']))
+                        print(procname)
+        except:
+            pass
 
=== added file 'tests/health-check/health-check-test-pid.py'
--- tests/health-check/health-check-test-pid.py	1970-01-01 00:00:00 +0000
+++ tests/health-check/health-check-test-pid.py	2014-09-10 13:48:46 +0000
@@ -0,0 +1,198 @@
+#!/usr/bin/python3
+#
+#
+# Copyright (C) 2013 Canonical
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA
+#
+#
+# Syntax:
+# 	health-check-test-pid.py pid [ path-to-threshold-files ]
+#
+# The process name is resolved and the tool will use a
+# `procname`.threshold file to compare against.  If this file does not exist,
+# default.threshold is used.
+#
+
+import json
+import os
+import subprocess
+import sys
+import psutil
+import platform
+
+#
+# Default test run durations in seconds
+#
+default_duration = 60
+
+default_threshold_path = os.path.join(
+    os.path.join(os.path.dirname(__file__), 'thresholds'), platform.machine())
+
+default_pass_unknown_process = True
+
+
+def read_threshold(procname):
+    """Parse thresholds file.
+
+    lines starting with '#' are comments
+    format is: key value, e.g.
+       health-check.cpu-load.cpu-load-total.total-cpu-percent  0.5
+       health-check.cpu-load.cpu-load-total.user-cpu-percent   0.5
+       health-check.cpu-load.cpu-load-total.system-cpu-percent 0.5
+    """
+    fname = default_threshold_path + "/" + procname + ".threshold"
+    thresholds = {}
+    n = 0
+
+    with open(fname) as file:
+        for line in file:
+            n = n + 1
+            if len(line) > 1 and not line.startswith("#"):
+                tmp = line.split()
+                if len(tmp) == 2:
+                    thresholds[tmp[0]] = tmp[1]
+                else:
+                    sys.stderr.write(
+                        "Threshold file %s line %d format error" % (fname, n))
+
+    return thresholds
+
+
+def check_threshold(data, key, fullkey, threshold):
+    """Locate a threshold in the JSON data, compare it to the threshold"""
+
+    try:
+        d = data[key[0]]
+    except KeyError:
+        sys.stderr.write(
+            "health-check JSON data does not have key " + fullkey + "\n")
+        return (True, "Attribute not found and ignored")
+
+    key = key[1:]
+    if len(key) > 0:
+        return check_threshold(d, key, fullkey, threshold)
+    else:
+        val = float(d)
+        if threshold >= val:
+            cmp = str(threshold) + " >= " + str(val)
+            return (True, cmp)
+        else:
+            cmp = str(threshold) + " < " + str(val)
+            return (False, cmp)
+
+
+def check_thresholds(procname, data, thresholds):
+    print("process: " + procname)
+    failed = False
+    for key in sorted(thresholds.keys()):
+        if key.startswith("health-check"):
+            threshold = float(thresholds[key])
+            (ret, str) = check_threshold(data, key.split('.'), key, threshold)
+            if ret:
+                msg = "PASSED"
+            else:
+                msg = "FAILED"
+                failed = True
+
+            sys.stderr.write(msg + ": " + str + ": " + key + "\n")
+
+    return failed
+
+
+def health_check(pid, procname):
+    """run health-check on a given process
+
+    :return: True if failed, False if passed
+    """
+    thresholds = read_threshold(procname)
+    #
+    #  Can't test without thresholds
+    #
+    if len(thresholds) == 0:
+        if default_pass_unknown_process:
+            sys.stderr.write("No thresholds for process " + procname + "\n")
+            sys.stderr.write("Defaulting to pass this test\n")
+            return False
+        else:
+            thresholds = read_threshold("default")
+            if len(thresholds) == 0:
+                sys.stderr.write(
+                    "No thresholds for process " + procname + "\n")
+            else:
+                sys.stderr.write(
+                    "Using default thresholds for process " + procname + "\n")
+
+    duration = default_duration
+
+    if 'duration' in thresholds:
+        duration = int(thresholds['duration'])
+
+    filename = "health-check-" + str(pid) + ".json"
+    args = [
+        'health-check', '-c', '-f', '-d', str(duration),
+        '-w', '-W', '-r', '-p', str(pid), '-o', filename
+    ]
+
+    try:
+        subprocess.check_output(args)
+        with open(filename) as f:
+            data = json.load(f)
+            return check_thresholds(procname, data, thresholds)
+    except subprocess.CalledProcessError as e:
+        print(e)
+        exit(1)
+
+
+def get_proc(pid):
+    proc = None
+    try:
+        p = psutil.Process(pid)
+        pgid = os.getpgid(pid)
+        if pgid == 0:
+            sys.stderr.write(
+                "Cannot run health-check on kernel task with PID(%d)\n" % pid)
+        else:
+            proc = p
+    except psutil.NoSuchProcess as e:
+        sys.stderr.write('%s\n' % e)
+    except OSError as e:
+        if e.errno == 3:
+            sys.stderr.write("Cannot find pgid on process with PID %d\n" % pid)
+    return proc.as_dict(attrs=['pid', 'name'])
+
+
+def main(pid):
+    if os.getuid() != 0:
+        sys.stderr.write("Need to run as root\n")
+        exit(1)
+
+    p = get_proc(pid)
+    if not p:
+        exit(1)
+
+    procname = os.path.basename(p['name'])
+    if (health_check(pid, procname)):
+        exit(1)
+    else:
+        exit(0)
+
+if __name__ == '__main__':
+    if len(sys.argv) != 2:
+        sys.stderr.write("Usage: %s PID\n" % sys.argv[0])
+        exit(1)
+
+    pid = int(sys.argv[1])
+    main(pid)
 
=== added file 'tests/health-check/master.run'
--- tests/health-check/master.run	1970-01-01 00:00:00 +0000
+++ tests/health-check/master.run	2014-09-10 13:48:46 +0000
@@ -0,0 +1,5 @@
+---
+testsuites:
+  - name: health-check
+    fetch_method: dev
+    fetch_location: ./
 
=== added file 'tests/health-check/setup.sh'
--- tests/health-check/setup.sh	1970-01-01 00:00:00 +0000
+++ tests/health-check/setup.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,10 @@
+#!/bin/sh -x
+echo "Setting up"
+
+${TARGET_PREFIX} apt-get install health-check
+
+${TARGET_PREFIX} mkdir -p /tmp/results
+
+adb push . /tmp/healthcheck
+
+exit 0
 
=== added file 'tests/health-check/test.sh'
--- tests/health-check/test.sh	1970-01-01 00:00:00 +0000
+++ tests/health-check/test.sh	2014-09-10 13:48:46 +0000
@@ -0,0 +1,15 @@
+#!/bin/sh
+
+procname=$1
+RC=0
+
+pidline=$(cat /tmp/healthcheck/procmapping.txt | grep $procname:)
+sed -i "/$pidline/d" /tmp/healthcheck/procmapping.txt
+pid=$(echo $pidline | awk -F: '{print $NF}')
+
+echo "Testing $procname - $pid"
+/tmp/healthcheck/health-check-test-pid.py $pid
+RC=$?
+
+cp *.json /tmp/results
+exit $RC
 
=== added directory 'tests/health-check/thresholds'
=== added directory 'tests/health-check/thresholds/armv7l'
=== added file 'tests/health-check/thresholds/armv7l/NetworkManager.threshold'
--- tests/health-check/thresholds/armv7l/NetworkManager.threshold	1970-01-01 00:00:00 +0000
+++ tests/health-check/thresholds/armv7l/NetworkManager.threshold	2014-09-10 13:48:46 +0000
@@ -0,0 +1,73 @@
+#
+# Default "IDLE" process thresholds.  Typically, an idle process
+# should be doing very little, so this profile checks some simple
+# process specific settings to see if we can find "busy" idle processes
+#
+# Length of test in seconds
+#
+duration 300
+#
+# CPU loading, should be really low for idle processes
+#
+health-check.cpu-load.cpu-load-total.total-cpu-percent	0.9
+health-check.cpu-load.cpu-load-total.user-cpu-percent	0.5
+health-check.cpu-load.cpu-load-total.system-cpu-percent	0.6
+#
+# Page faults, ignore these
+#
+#health-check.page-faults.page-faults-total.minor-page-faults-total-rate 10
+#health-check.page-faults.page-faults-total.major-page-faults-total-rate 10
+#health-check.page-faults.page-faults-total.total-page-faults-total-rate 10
+#
+# Wakeup Events, NetworkManager can be quite busy
+#
+health-check.wakeup-events.wakeup-events-total.wakeup-total-rate 1.0
+#
+# Context Switches, ignore these
+#
+#health-check.context-switches.context-switches-total.context-switch-total-rate 10
+#
+# Files, NetworkManager can do quite a lot of I/O at times
+#
+health-check.file-access.file-access-total.access-count-total-rate 0.5
+#health-check.file-io-operations.file-io-operations-total.open-call-total-rate 0.05
+#health-check.file-io-operations.file-io-operations-total.close-call-total-rate 0.05
+#health-check.file-io-operations.file-io-operations-total.read-call-total-rate 0.05
+#health-check.file-io-operations.file-io-operations-total.write-call-total-rate 0.05
+#
+# System calls poll rates
+#
+#health-check.system-calls.system-calls-total.system-call-count-total-rate 10
+#health-check.polling-system-calls.polling-system-calls-total.system-call-total-rate 10
+health-check.polling-system-calls.polling-system-calls-total.poll-count-infinite-total-rate 0
+health-check.polling-system-calls.polling-system-calls-total.poll-count-zero-total-rate 0
+#
+# File system syncs, an idle process should not do any
+#
+health-check.file-system-syncs.sync-system-calls-total.fdatasync-call-count-total-rate 0
+health-check.file-system-syncs.sync-system-calls-total.fsync-call-count-total-rate 0
+health-check.file-system-syncs.sync-system-calls-total.sync-call-count-total-rate 0
+health-check.file-system-syncs.sync-system-calls-total.syncfs-call-count-total-rate 0
+#
+# Memory usage, can't have a generic memory limit, so ignore
+#
+#health-check.memory-usage.memory-usage-total.stack-size-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.stack-rss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.stack-pss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.heap-size-total-kbytes 9000
+#health-check.memory-usage.memory-usage-total.heap-rss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.heap-pss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.mapped-size-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.mapped-rss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.mapped-pss-total-kbytes 1000000
+#
+# Memory change,  should not grow more than say 10K an hour
+#
+# health-check.memory-change.memory-change-total.
+health-check.heap-usage-via-brk.heap-usage-via-brk-total.brk-size-total-Kbytes-rate   0.002777778
+health-check.memory-usage-via-mmap.memory-usage-via-mmap-total.mmap-total-Kbytes-rate 0.002777778
+#
+# Network, should be fairly idle
+#
+health-check.network-connections.network-connections-total.send-total-rate 1024
+health-check.network-connections.network-connections-total.receive-total-rate 1024
 
=== added file 'tests/health-check/thresholds/armv7l/accounts-daemon.threshold'
--- tests/health-check/thresholds/armv7l/accounts-daemon.threshold	1970-01-01 00:00:00 +0000
+++ tests/health-check/thresholds/armv7l/accounts-daemon.threshold	2014-09-10 13:48:46 +0000
@@ -0,0 +1,73 @@
+#
+# Default "IDLE" process thresholds.  Typically, an idle process
+# should be doing very little, so this profile checks some simple
+# process specific settings to see if we can find "busy" idle processes
+#
+# Length of test in seconds
+#
+duration 300
+#
+# CPU loading, should be really low for idle processes
+#
+health-check.cpu-load.cpu-load-total.total-cpu-percent	0.05
+health-check.cpu-load.cpu-load-total.user-cpu-percent	0.02
+health-check.cpu-load.cpu-load-total.system-cpu-percent	0.03
+#
+# Page faults, ignore these
+#
+#health-check.page-faults.page-faults-total.minor-page-faults-total-rate 10
+#health-check.page-faults.page-faults-total.major-page-faults-total-rate 10
+#health-check.page-faults.page-faults-total.total-page-faults-total-rate 10
+#
+# Wakeup Events, anything more than 1 a second is busy
+#
+health-check.wakeup-events.wakeup-events-total.wakeup-total-rate 0.4
+#
+# Context Switches, ignore these
+#
+#health-check.context-switches.context-switches-total.context-switch-total-rate 10
+#
+# Files, really should not be doing much so allow for 1 a minute top
+#
+health-check.file-access.file-access-total.access-count-total-rate 0.016666667
+health-check.file-io-operations.file-io-operations-total.open-call-total-rate 0.016666667
+health-check.file-io-operations.file-io-operations-total.close-call-total-rate 0.016666667
+health-check.file-io-operations.file-io-operations-total.read-call-total-rate 0.016666667
+health-check.file-io-operations.file-io-operations-total.write-call-total-rate 0.016666667
+#
+# System calls, ignore these
+#
+#health-check.system-calls.system-calls-total.system-call-count-total-rate 10
+#health-check.polling-system-calls.polling-system-calls-total.system-call-total-rate 10
+#health-check.polling-system-calls.polling-system-calls-total.poll-count-infinite-total-rate 10
+#health-check.polling-system-calls.polling-system-calls-total.poll-count-zero-total-rate 0
+#
+# File system syncs, an idle process should not do any
+#
+health-check.file-system-syncs.sync-system-calls-total.fdatasync-call-count-total-rate 0
+health-check.file-system-syncs.sync-system-calls-total.fsync-call-count-total-rate 0
+health-check.file-system-syncs.sync-system-calls-total.sync-call-count-total-rate 0
+health-check.file-system-syncs.sync-system-calls-total.syncfs-call-count-total-rate 0
+#
+# Memory usage, can't have a generic memory limit, so ignore
+#
+#health-check.memory-usage.memory-usage-total.stack-size-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.stack-rss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.stack-pss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.heap-size-total-kbytes 25000
+#health-check.memory-usage.memory-usage-total.heap-rss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.heap-pss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.mapped-size-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.mapped-rss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.mapped-pss-total-kbytes 1000000
+#
+# Memory change,  should not grow more than say 10K an hour
+#
+# health-check.memory-change.memory-change-total.
+health-check.heap-usage-via-brk.heap-usage-via-brk-total.brk-size-total-Kbytes-rate   0.002777778
+health-check.memory-usage-via-mmap.memory-usage-via-mmap-total.mmap-total-Kbytes-rate 0.002777778
+#
+# Network, should be fairly idle
+#
+health-check.network-connections.network-connections-total.send-total-rate 16
+health-check.network-connections.network-connections-total.receive-total-rate 16
 
=== added file 'tests/health-check/thresholds/armv7l/address-book-service.threshold'
--- tests/health-check/thresholds/armv7l/address-book-service.threshold	1970-01-01 00:00:00 +0000
+++ tests/health-check/thresholds/armv7l/address-book-service.threshold	2014-09-10 13:48:46 +0000
@@ -0,0 +1,73 @@
+#
+# Default "IDLE" process thresholds.  Typically, an idle process
+# should be doing very little, so this profile checks some simple
+# process specific settings to see if we can find "busy" idle processes
+#
+# Length of test in seconds
+#
+duration 300
+#
+# CPU loading, should be really low for idle processes
+#
+health-check.cpu-load.cpu-load-total.total-cpu-percent	0.01
+health-check.cpu-load.cpu-load-total.user-cpu-percent	0.01
+health-check.cpu-load.cpu-load-total.system-cpu-percent	0.01
+#
+# Page faults, ignore these
+#
+#health-check.page-faults.page-faults-total.minor-page-faults-total-rate 10
+#health-check.page-faults.page-faults-total.major-page-faults-total-rate 10
+#health-check.page-faults.page-faults-total.total-page-faults-total-rate 10
+#
+# Wakeup Events, NetworkManager can be quite busy
+#
+health-check.wakeup-events.wakeup-events-total.wakeup-total-rate 0.1
+#
+# Context Switches, ignore these
+#
+#health-check.context-switches.context-switches-total.context-switch-total-rate 10
+#
+# Files, NetworkManager can do quite a lot of I/O at times
+#
+health-check.file-access.file-access-total.access-count-total-rate 0.5
+#health-check.file-io-operations.file-io-operations-total.open-call-total-rate 0.05
+#health-check.file-io-operations.file-io-operations-total.close-call-total-rate 0.05
+#health-check.file-io-operations.file-io-operations-total.read-call-total-rate 0.05
+#health-check.file-io-operations.file-io-operations-total.write-call-total-rate 0.05
+#
+# System calls poll rates
+#
+#health-check.system-calls.system-calls-total.system-call-count-total-rate 10
+#health-check.polling-system-calls.polling-system-calls-total.system-call-total-rate 10
+health-check.polling-system-calls.polling-system-calls-total.poll-count-infinite-total-rate 0
+health-check.polling-system-calls.polling-system-calls-total.poll-count-zero-total-rate 0
+#
+# File system syncs, an idle process should not do any
+#
+health-check.file-system-syncs.sync-system-calls-total.fdatasync-call-count-total-rate 0
+health-check.file-system-syncs.sync-system-calls-total.fsync-call-count-total-rate 0
+health-check.file-system-syncs.sync-system-calls-total.sync-call-count-total-rate 0
+health-check.file-system-syncs.sync-system-calls-total.syncfs-call-count-total-rate 0
+#
+# Memory usage, can't have a generic memory limit, so ignore
+#
+#health-check.memory-usage.memory-usage-total.stack-size-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.stack-rss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.stack-pss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.heap-size-total-kbytes 9000
+#health-check.memory-usage.memory-usage-total.heap-rss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.heap-pss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.mapped-size-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.mapped-rss-total-kbytes 1000000
+#health-check.memory-usage.memory-usage-total.mapped-pss-total-kbytes 1000000
+#
+# Memory change,  should not grow more than say 10K an hour
+#
+# health-check.memory-change.memory-change-total.
+health-check.heap-usage-via-brk.heap-usage-via-brk-total.brk-size-total-Kbytes-rate   0.002777778
+health-check.memory-usage-via-mmap.memory-usage-via-mmap-total.mmap-total-Kbytes-rate 0.002777778
+#
+# Network, should be fairly idle
+#
+health-check.network-connections.network-connections-total.send-total-rate 1024
+health-check.network-connections.network-connections-total.receive-total-rate 1024
 
=== added file 'tests/health-check/thresholds/armv7l/bluetoothd.threshold'
--- tests/health-check/thresholds/armv7l/bluetoothd.threshold	1970-01-01 00:00:00 +0000
+++ tests/health-check/thresholds/armv7l/bluetoothd.threshold	2014-09-10 13:48:46 +0000
@@ -0,0 +1,73 @@
+#
+# Default "IDLE" process thresholds.  Typically, an idle process
+# should be doing very little, so this profile checks some simple
+# process specific settings to see if we can find "busy" idle processes
+#
+# Length of test in seconds
+#
+duration 300
+#
+# CPU loading, should be really low for idle processes
+#
+health-check.cpu-load.cpu-load-total.total-cpu-percent	0.01
+health-check.cpu-load.cpu-load-total.user-cpu-percent	0.01
+health-check.cpu-load.cpu-load-total.system-cpu-percent	0.01
+#
+# Page faults, ignore these
+#
+#health-check.page-faults.page-faults-total.minor-page-faults-total-rate 10
+#health-check.page-faults.page-faults-total.major-page-faults-total-rate 10
+#health-check.page-faults.page-faults-total.total-page-faults-total-rate 10
+#
+# Wakeup Events, NetworkManager can be quite busy
+#
+health-check.wakeup-events.wakeup-events-total.wakeup-total-rate 0.5
+#
+# Context Switches, ignore these
+#
+#health-check.context-switches.context-switches-total.context-switch-total-rate 10
+#
+# Files, NetworkManager can do quite a lot of I/O at times
+#
+health-check.file-access.file-access-total.access-count-total-rate 0.5
+#health-check.file-io-operations.file-io-operations-total.open-call-total-rate 0.05
+#health-check.file-io-operations.file-io-operations-total.close-call-total-rate 0.05
+#health-check.file-io-operations.file-io-operations-total.read-call-total-rate 0.05
+#health-check.file-io-operations.file-io-operations-total.write-call-total-rate 0.05
+#
+# System calls poll rates
+#
+#health-check.system-calls.system-calls-total.system-call-count-total-rate 10
+#health-check.polling-system-calls.polling-system-calls-total.system-call-total-rate 10
+health-check.polling-system-calls.polling-system-calls-total.poll-count-infinite-total-rate 0
+health-check.polling-system-calls.polling-system-calls-total.poll-count-zero-total-rate 0
+#