Merge lp:~canonical-platform-qa/ubuntu-system-tests/refactor-command-line into lp:ubuntu-system-tests

[Richard Huddie]

Refactor command line interface to seperate setup and test execution.

Command line now uses 2 commands: setup or run.

setup is used to provision the device ready for testing and allows: apt-get update, dist-upgrade, cache dependencies or install tests options. The default setup if no arguments are provided with setup command are to apt-get update, dist-upgrade, install apt-cacher and then download dependencies. It will only mount the file system and reboot when it needs to.

run is used only to run suites of tests on the device and won't try and change the state of the device.

Use -h option to get full list of options for each command.

Example to setup a device where network is already done:
python3 -m ubuntu_system_tests.run setup --network-ready

Example to run a test on a device that has been setup already:

python3 -m ubuntu_system_tests.run run -n -t ubuntu_system_tests.tests.scopes.test_manage_scopes.ManageScopesTest.test_manage_scopes_displays_scopes

When this lands the following MP into qa-jenkins-jobs will also need to land to keep the jenkins jobs working: https://code.launchpad.net/~canonical-platform-qa/qa-jenkins-jobs/ust-cmdline-refactor/+merge/298529

Readme and setup tests updated too, but there is another task to give the readme a complete overhaul.

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:463
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/266/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/399
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/399/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/399
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/399/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/346/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/266/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/ubuntu-system-tests/refactor-command-line]

464. By Richard Huddie on 2016-06-28

Add sync command before reboot and report failure state of reboot.

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:464
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/267/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/401
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/401/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/401
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/401/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/348/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/267/rebuild

review: Approve (continuous-integration)

[Allan LeSage (allanlesage)]

Able to get a passing test with this in Jenkins, suggest we land and debug further as necessary, one teensy comment.

review: Approve

[lp:~canonical-platform-qa/ubuntu-system-tests/refactor-command-line]

465. By Richard Huddie on 2016-06-29

Fix typo.

[Richard Huddie (rhuddie)]

> Able to get a passing test with this in Jenkins, suggest we land and debug
> further as necessary, one teensy comment.

Good spot, fixed the typo.

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:465
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/268/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/411
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/411/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/411
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/411/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/358/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/268/rebuild

review: Approve (continuous-integration)

=== modified file 'README.rst'
--- README.rst	2016-06-10 13:25:06 +0000
+++ README.rst	2016-06-29 08:06:07 +0000
@@ -37,7 +37,7 @@
 
 ::
 
-  $ sudo add-apt-repository ppa:canonical-platform-qa/ppa
+  $ sudo add-apt-repository ppa:canonical-platform-qa/ubuntu-system-tests
   $ sudo apt-get update
   $ sudo apt-get install ubuntu-system-tests-host
 
@@ -238,7 +238,7 @@
 
 ::
 
-    $ run-system-tests
+    $ system-tests run
 
 Running specific sets of tests
 ------------------------------
@@ -249,12 +249,12 @@
 just output the list of tests as an argument to the tool, like so:
 
 ::
-    $ run-system-tests `cat test_lists/sanity.tests`
+    $ system-tests run `cat test_lists/sanity.tests`
 
 Command line arguments
 ----------------------
 
-You can pass arguments to the run-system-tests command to change the default
+You can pass arguments to the system-tests command to change the default
 behavior. In the following sections you will find some information about how
 to use the arguments for some common scenarios.
 
@@ -263,7 +263,7 @@
 
 ::
 
-  $ run-system-tests --help
+  $ system-tests --help
 
 Run with a different configuration file
 ---------------------------------------
@@ -273,7 +273,7 @@
 
 ::
 
-  $ run-system-tests --config myconfig.conf
+  $ system-tests run --config myconfig.conf
 
 By default, the config file should be in the *~/.config* directory, but you can
 use a config file from a different directory by setting the *XDG_CONFIG_HOME*
@@ -282,7 +282,7 @@
 
 ::
 
-  $ XDG_CONFIG_HOME=/tmp run-system-tests --config myconfig.conf
+  $ XDG_CONFIG_HOME=/tmp system-tests run --config myconfig.conf
 
 You can use the different configuration files to store values for different
 test profiles that you commonly use. For example, if you usually have two
@@ -327,7 +327,7 @@
 
 ::
 
-    $ run-system-tests --config-section arale
+    $ system-tests run --config-section arale
 
 Using this option also allows concurrent execution of tests on multiple devices
 from the same host machine. In order to do this *device_serial* and
@@ -371,7 +371,7 @@
 
 ::
 
- $ run-system-tests --touch-visualization ubuntu_system_tests.tests.test_tutorial.TutorialTestCase.test_complete_tutorial
+ $ system-tests run --touch-visualization ubuntu_system_tests.tests.test_tutorial.TutorialTestCase.test_complete_tutorial
 
 Install tests dependencies in the device
 ----------------------------------------
@@ -384,7 +384,7 @@
 
 ::
 
- $ run-system-tests --install-tests package1.module1
+ $ system-tests setup --install-tests
 
 Upgrade device using dist-upgrade
 ---------------------------------
@@ -396,7 +396,7 @@
 
 ::
 
- $ run-system-tests --dist-upgrade package1.module1
+ $ system-tests setup --dist-upgrade
 
 Cache downloaded dependencies on the device
 -------------------------------------------
@@ -419,7 +419,7 @@
 
 ::
 
- $ run-system-tests --cache-deps package1.module1
+ $ system-tests setup --cache-deps
 
 This will install the dependency cacher on the device and then reboot before
 continuing to execute the tests as normal. Dependencies will be downloaded
@@ -438,31 +438,31 @@
 Easy device setup
 -----------------
 
-To allow devices be updated and prepared for testing easily *--setup* or *-a*
-option can be used. This will use the --update-apt, --dist-upgrade and
+To allow devices be updated and prepared for testing easily use setup command.
+This will use the --update-apt, --dist-upgrade and
 --cache-deps options to configure the device ready for testing.
 
 ::
 
- $ run-system-tests --setup package1.module1
+ $ system-tests setup
 
 
 Specify the list of tests to run
 --------------------------------
 
-The run-system-tests command receives an optional list of tests to run. For
+The system-tests run command receives an optional list of tests to run. For
 example, to run only the tests with id package1.module1.test1 and
 package2.module2.test2:
 
 ::
 
- $ run-system-tests package1.module1.test1 package2.module2.test2
+ $ system-tests run package1.module1.test1 package2.module2.test2
 
 You can also pass a package or module id to run all the tests defined in it,
 like:
 
 ::
 
-  $ run-system-tests package1 package2.module2
+  $ system-tests run package1 package2.module2
 
 If you don't pass any test as argument it will run all the tests in the suite.

=== modified file 'debian/tests/control'
--- debian/tests/control	2016-05-26 14:25:26 +0000
+++ debian/tests/control	2016-06-29 08:06:07 +0000
@@ -1,4 +1,4 @@
-Tests: systemtests
+Tests: systemtests, setup
 # Many tests output the emulators deprecation message to stderr.
 Restrictions: allow-stderr
 Depends: address-book-app,
@@ -11,7 +11,7 @@
          gallery-app,
          gallery-app-autopilot,
          mediaplayer-app,
-         mediaplayer-app-autopilot,
+         mediaplayer-app-autopilot,  # TMPINSTALL
          messaging-app,
          messaging-app-autopilot,  # TMPINSTALL
          oxideqt-chromedriver,  # TMPINSTALL

=== added file 'debian/tests/setup'
--- debian/tests/setup	1970-01-01 00:00:00 +0000
+++ debian/tests/setup	2016-06-29 08:06:07 +0000
@@ -0,0 +1,29 @@
+#!/usr/bin/python3
+
+#
+# Ubuntu System Tests
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import sys
+
+
+def main():
+    print('Setup successfully completed.')
+
+
+if __name__ == '__main__':
+    sys.exit(main())

=== renamed file 'run-system-tests' => 'system-tests'
=== modified file 'ubuntu_system_tests/host/command_line.py'
--- ubuntu_system_tests/host/command_line.py	2016-06-10 13:25:06 +0000
+++ ubuntu_system_tests/host/command_line.py	2016-06-29 08:06:07 +0000
@@ -19,142 +19,186 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-import deb822
-import glob
-import json
 import logging
 import os
 import subprocess
 import sys
-import tempfile
 
 from argparse import ArgumentParser
+from collections import namedtuple
+
 from ubuntu_system_tests.host import (
+    commands as cmds,
+    debian,
     results,
-    testdesc
 )
 from ubuntu_system_tests.common import (
     clean_dir,
     config,
-    ssh,
 )
 
 logger = logging.getLogger(__name__)
-_ADT_RUN_SHORT_TIMEOUT = 600  # 10 minutes.
-_ADT_RUN_TEST_TIMEOUT = 86400  # 24 hours
-
-# apt-cacher-ng definitions
-APT_CACHER = 'apt-cacher-ng'
-APT_CACHER_CACHE_DIR = '/var/cache/apt-cacher-ng'
-APT_CACHER_LOG_DIR = '/var/log/apt-cacher-ng/'
-APT_CACHER_INIT_FILE = '/etc/init/apt-cacher-ng.conf'
-APT_PROXY = 'Acquire::http { Proxy "http://127.0.0.1:3142"; };'
-APT_PROXY_FILE = '/etc/apt/apt.conf.d/01proxy'
-EXT_CACHE_DIR = '/userdata/apt-cacher-ng/cache'
-EXT_LOG_DIR = '/userdata/apt-cacher-ng/logs'
-PERMISSION = '755'
-
-# Installation path for shared files
-INSTALLED_SHARE_DIR = '/usr/share/ubuntu-system-tests'
-# Installation path for installed python scripts
-INSTALLED_PYTHON_DIR = '/usr/lib/python'
+
+Option = namedtuple('Option', ['args', 'kwargs'])
+OPTION_CACHE_DEPS = Option(
+    ['--cache-deps', '-d'],
+    {'action': 'store_true',
+     'help': 'Cache dependencies on the device so they don\'t have to be '
+     'downloaded for every test run.',
+     'default': False})
+OPTION_CONFIG = Option(
+    ['--config', '-c'],
+    {'help': 'The config file to use for this run.',
+     'default': 'ubuntu-system-tests.conf'})
+OPTION_CONFIG_SECTION = Option(
+    ['--config-section', '-e'],
+    {'help': 'The section name to use in config file.',
+     'default': config.KEY_DEFAULT})
+OPTION_DIST_UPGRADE = Option(
+    ['--dist-upgrade', '-p'],
+    {'action': 'store_true',
+     'help': 'Perform a dist-upgrade on the device to install latest '
+     'available updates from archive.',
+     'default': False})
+OPTION_INSTALL_TESTS = Option(
+    ['--install-tests', '-i'],
+    {'action': 'store_true',
+     'help': 'Install all dependencies permanently in the device instead '
+     'of in the /tmp directory. This includes all production and test '
+     'dependencies.',
+     'default': False})
+OPTION_NETWORK_READY = Option(
+    ['--network-ready', '-n'],
+    {'action': 'store_true',
+     'help': 'Assume that the network connection is already setup.',
+     'default': False})
+OPTION_SILENT = Option(
+    ['--silent', '-s'],
+    {'action': 'store_true',
+     'help': 'Do not prompt for missing user config options, use default '
+     'values instead.',
+     'default': False})
+OPTION_SUITES = Option(
+    ['suites'],
+    {'nargs': '*',
+     'help': 'The list of test suites to run. If left blank then all tests '
+             'from ubuntu_system_tests.tests will be run.'})
+OPTION_TOUCH_VISUALIZATION = Option(
+    ['--touch-visualization', '-t'],
+    {'action': 'store_true',
+     'help': 'Enable visualization of touch points on screen.',
+     'default': False})
+OPTION_UPDATE_APT = Option(
+    ['--update-apt', '-u'],
+    {'action': 'store_true',
+     'help': 'Update apt lists before installing the dependencies.',
+     'default': False})
+OPTION_VERBOSE = Option(
+    ['--verbose', '-v'],
+    {'action': 'store_true',
+     'help': 'Provide verbose logging.',
+     'default': False})
+
+COMMON_ARGS = [
+    OPTION_CONFIG,
+    OPTION_CONFIG_SECTION,
+    OPTION_NETWORK_READY,
+    OPTION_VERBOSE]
+RUN_ARGS = [
+    OPTION_SILENT,
+    OPTION_SUITES,
+    OPTION_TOUCH_VISUALIZATION]
+SETUP_ARGS = [
+    OPTION_CACHE_DEPS,
+    OPTION_DIST_UPGRADE,
+    OPTION_INSTALL_TESTS,
+    OPTION_UPDATE_APT
+]
+
+COMMAND_RUN = 'run'
+COMMAND_SETUP = 'setup'
 
 
 def main(argv=None):
-    if argv is None:
-        argv = sys.argv[1:]
-    args = _parse_arguments(argv)
-    if args.setup:
-        args.update_apt = args.cache_deps = args.dist_upgrade = True
-    return run_system_tests(args.config, args.config_section, args.suites,
-                            args.update_apt, args.network_ready,
-                            args.touch_visualization, args.dist_upgrade,
-                            args.install_tests, args.silent, args.cache_deps,
-                            args.verbose)
+    args = _parse_arguments(argv or sys.argv[1:])
+    if args.command not in [COMMAND_RUN, COMMAND_SETUP]:
+        logger.warning(
+            'Command option must be either "{r}" or "{s}". Use -h option for '
+            'further help.'.format(r=COMMAND_RUN, s=COMMAND_SETUP))
+        return
+
+    if args.command == COMMAND_RUN:
+        return run_system_tests(args)
+    else:
+        return setup_system_tests(args)
+
+
+def _add_options(parser, options):
+    """Add the specified list of options to the option parser object."""
+    for option in options:
+        parser.add_argument(*option.args, **option.kwargs)
 
 
 def _parse_arguments(argv=None):
-    parser = ArgumentParser('Run system tests for Ubuntu Touch.')
-    parser.add_argument('--config', '-c',
-                        help='The config file to use for this run.',
-                        default='ubuntu-system-tests.conf')
-    parser.add_argument('--config-section', '-e',
-                        help='The section name to use in config file.',
-                        default=config.KEY_DEFAULT)
-    parser.add_argument(
-        '--update-apt', '-u', action='store_true',
-        help='Update apt lists before installing the dependencies',
-        default=False)
-    parser.add_argument(
-        '--touch-visualization', '-t', action='store_true',
-        help='Enable visualization of touch points on screen.')
-    parser.add_argument(
-        '--network-ready', '-n', action='store_true',
-        help='Assume that the network connection is already setup.')
-    parser.add_argument(
-        '--dist-upgrade', '-p', action='store_true',
-        help='Perform a dist-upgrade on the device to install latest '
-             'available updates from archive.',
-        default=False)
-    parser.add_argument(
-        '--install-tests', '-i', action='store_true',
-        help='Install all dependencies permanently in the device instead of '
-             'in the /tmp directory. This includes all production and test '
-             'dependencies.',
-        default=False)
-    parser.add_argument(
-        '--silent', '-s', action='store_true',
-        help='Do not prompt for missing user config options, use default '
-        'values instead.')
-    parser.add_argument(
-        '--cache-deps', '-d', action='store_true',
-        help='Cache dependencies on the device so they don\'t have to be '
-             'downloaded for every test run.')
-    parser.add_argument(
-        '--setup', '-a', action='store_true',
-        help='Setup the device ready for testing. This will use --update-apt, '
-             '--dist-upgrade and --cache-deps options to update the device '
-             'and cache dependencies so it is ready for testing.')
-    parser.add_argument(
-        'suites', nargs='*', help='The list of test suites to run.')
-    parser.add_argument(
-        '--verbose', '-v', action='store_true',
-        help='Provide verbose logging.', default=False)
+    """Parse the provided arguments and return in Namespace object."""
+    parser = ArgumentParser(
+        'ubuntu-system-tests',
+        description='Autopilot system test suite for Ubuntu Touch devices. '
+        'Allows devices to be setup for testing and then used to execute '
+        'suites of autopilot tests from ubuntu-system-tests or other '
+        'projects. For further help use -h option with required command.')
+    subparsers = parser.add_subparsers(
+        dest='command',
+        help='Command options')
+    run = subparsers.add_parser(
+        COMMAND_RUN,
+        help='Run tests on the device',
+        description='Executes a list of tests on a connected device once it '
+        'has been setup using the setup command. If no tests are specified '
+        'to be run then all tests from ubuntu_system_tests.tests will be '
+        'executed.')
+    _add_options(run, COMMON_ARGS + RUN_ARGS)
+    setup = subparsers.add_parser(
+        COMMAND_SETUP,
+        help='Setup device ready for testing',
+        description='Sets up the device ready for testing. This can include '
+        'performing apt-get update and dist-upgrade operations. Also it is '
+        'possible to install apt-cacher-ng to allow downloaded dependencies '
+        'to be cached on the device so they don\'t need to be downloaded for '
+        'every test run. If no arguments are specified then the default '
+        'operation will be to perform apt-get update, dist-upgrade and '
+        'install apt-cacher-ng. If any single option is specified then only '
+        'this action will be performed.')
+    _add_options(setup, COMMON_ARGS + SETUP_ARGS)
     return parser.parse_args(args=argv)
 
 
-def run_system_tests(conf_name, config_section, tests_to_run, update_apt=False,
-                     network_ready=False, touch_visualization=False,
-                     dist_upgrade=False, install_tests=False, silent=False,
-                     cache_deps=False, verbose=False):
+def setup_system_tests(args):
+    """Validate arguments and execute required setup commands on device."""
+    if ((args.cache_deps or args.dist_upgrade or args.install_tests) and
+            not args.update_apt):
+        # An install operation is requested without update_apt, so set
+        # update_apt to True.
+        args.update_apt = True
+    elif not (args.cache_deps or args.dist_upgrade or args.update_apt):
+        # There were no optional args specified.
+        # In this case apply the default actions, which are:
+        # update_apt, cache_deps and dist_upgrade
+        args.cache_deps = args.dist_upgrade = args.update_apt = True
+    return run_autopkg_setup(args)
+
+
+def run_system_tests(args):
     """Run system tests in a device connected via USB and report the results.
-    :param conf_name: The name of the configuration file that will be used
-        for the system tests.
-    :param config_section: The section name to use in the config file.
-    :param tests_to_run: A list of tests to run.
-    :param update_apt: Boolean representing whether apt-get update should be
-        run on the device before installing dependencies.
-    :param network_ready: Boolean representing whether phablet-network should
-    be run to provision the network on the device.
-    :param dist_upgrade: Upgrade the device using dist-upgrade.
-    :param install_tests: Install the test dependencies permanently in the
-                          device instead of in the /tmp directory.
-    :param touch_visualization: Enable visualization of touch points on screen
-    :param silent: If True then do not prompt for user config options, instead
-                   use the default value.
-    :param cache_deps: If True then cache the downloaded dependencies so that
-                       they don't have to be downloaded for every test run.
+    :param args: command line options.
     :returns: 0 when successful, other value when failed.
 
     """
-    config_stack = _get_config_stack(conf_name, config_section, silent)
     print_warnings()
-    clean_output_directory(config_stack)
+    config_stack = _prepare_config(args)
 
     enable_autopilot_introspection(config_stack)
-    return_code = run_autopkg_test(
-        config_stack, config_section, tests_to_run, update_apt, network_ready,
-        touch_visualization, dist_upgrade, install_tests, cache_deps, verbose)
+    return_code = run_autopkg_test(config_stack, args)
 
     if return_code <= 6:
         # Return codes higher than 6 mean that there was a problem with adt or
@@ -165,6 +209,14 @@
     return return_code
 
 
+def _prepare_config(args):
+    """Get config object for specified arguments and clear previous results."""
+    config_stack = _get_config_stack(
+        args.config, args.config_section, vars(args).get('silent', False))
+    clean_output_directory(config_stack)
+    return config_stack
+
+
 def get_camera_warning():
     return '- Please ensure device is raised from surface to allow rear ' \
            'camera to take a photograph during camera tests.'
@@ -214,523 +266,101 @@
     return config_stack
 
 
-def _get_serial_argument(config_stack):
-    device_serial = config_stack.get('device_serial')
-    if device_serial:
-        return ['-s', device_serial]
-    else:
-        # No serial in the config, run the commands as if only one device is
-        # connected.
-        return []
-
-
 def enable_autopilot_introspection(config_stack):
-    command = _build_enable_autopilot_introspection_command(config_stack)
+    """Enable autopilot introspection on the required device."""
+    command = cmds.build_enable_autopilot_introspection_command(config_stack)
     subprocess.check_output(command)
 
 
-def _build_enable_autopilot_introspection_command(config_stack):
-    command = (
-        ['phablet-config'] +
-        _get_serial_argument(config_stack) +
-        ['autopilot', '--dbus-probe', 'enable']
-    )
-    return command
-
-
-def run_autopkg_test(config_stack, config_section, tests_to_run, update_apt,
-                     network_ready, touch_visualization, dist_upgrade,
-                     install_tests, cache_deps, verbose):
-    """ Run the autopkgtest in the revice for the tests selected
+def run_autopkg_setup(args):
+    """Run setup commands on the required device, rebooting if required.
+    :param args: Command line arguments for setup command.
+    :return: Setup status code, 0 indicating setup was successful. Any error
+             status encountered during setup will be translated to error
+             status 16 from autopkgtest which indicates a testbed failure.
+    """
+    config_stack = _prepare_config(args)
+    adt_run_cmd = cmds.build_adt_run_setup_command(
+        config_stack,
+        args.config_section,
+        cache_deps=args.cache_deps,
+        dist_upgrade=args.dist_upgrade,
+        install_tests=args.install_tests,
+        network_ready=args.network_ready,
+        update_apt=args.update_apt,
+        verbose=args.verbose)
+
+    return run_setup_commands(adt_run_cmd, config_stack)
+
+
+def run_setup_commands(setup_cmd, config_stack):
+    """Run setup commands to prepare the device for test execution.
+
+    This will run the setup command on the device. If this causes a failure
+    on the device, or a reboot is required, the device will reboot. The setup
+    command will then be executed on the device again. If the second attempt
+    at setup fails then the failure status will be reported and setup will
+    stop.
+
+    :param setup_cmd: Command to execute to set the device up.
+    :param config_stack: Device configuration data.
+    :return: Setup status code, 0 indicating setup was successful. Any error
+             status encountered during setup will be translated to error
+             status 16 from autopkgtest which indicates a testbed failure.
+    """
+    count = 0
+    status = 1
+    while (status and count < 2):
+        clean_output_directory(config_stack)
+        status = subprocess.call(setup_cmd)
+        if status and count < 1:
+            # Error was reported after first setup attempt,
+            # so reboot device and try again.
+            logger.warning(
+                'Rebooting device after setup completed with '
+                'status {s}.'.format(s=status))
+            reboot_status = _reboot_device(config_stack)
+            if reboot_status:
+                logger.warning(
+                    'Device reboot completed with status '
+                    '{s}.'.format(s=reboot_status))
+                return reboot_status
+        count += 1
+    return status
+
+
+def _reboot_device(config_stack):
+    """Reboot the device specified in config.
+    :param config_stack: Device configuration.
+    :return: Status of reboot command.
+    """
+    for reboot_command in cmds._get_adb_reboot_commands(config_stack):
+        status = subprocess.call(reboot_command)
+        if status:
+            # Error status reported from reboot, so don't attempt any further
+            # commands and return status value.
+            break
+    return status
+
+
+def run_autopkg_test(config_stack, args):
+    """ Run the autopkgtest in the device for the tests selected
     :param config_stack: The config objects
-    :param config_section: The name of config section to use.
-    :param tests_to_run: List of tests to run.
-    :param update_apt: True if the packages hasve to be updated
-    :param network_ready: True if the network set in the config has to be
-    configured in the device
-    :param dist_upgrade: Upgrade the device using dist-upgrade.
-    :param install_tests: Install the test dependencies permanently in the
-                          device instead of in the /tmp directory.
-    :param cache_deps: If True then cache the downloaded dependencies so that
-                       they don't have to be downloaded for every test run.
-    :param verbose: True to provide verbose logging.
+    :param args: command line arguments
     :return: the autopkgtest execution exit code
     """
-    extra_depends = _get_extra_dependencies(
-        config_stack.get('tests_to_run', '').split()
-    )
-    if cache_deps:
-        adt_run_cmd = _build_adt_run_test_command(
-            config_stack,
-            config_section,
-            update_apt=update_apt,
-            network_ready=network_ready,
-            cache_deps=True,
-            verbose=verbose)
-        subprocess.call(adt_run_cmd)
-        clean_output_directory(config_stack)
-        logger.info('apt-cacher-ng installed on device')
-        update_apt = False
-
-    if dist_upgrade:
-        adt_run_cmd = _build_adt_run_test_command(
-            config_stack,
-            config_section,
-            update_apt=update_apt,
-            network_ready=network_ready,
-            dist_upgrade=True,
-            verbose=verbose)
-        subprocess.call(adt_run_cmd)
-        clean_output_directory(config_stack)
-        update_apt = False
-
-    if install_tests:
-        adt_run_cmd = _build_adt_run_test_command(
-            config_stack,
-            config_section,
-            extra_depends=extra_depends,
-            update_apt=update_apt,
-            network_ready=network_ready,
-            install_tests=True,
-            verbose=verbose)
-        subprocess.call(adt_run_cmd)
-        clean_output_directory(config_stack)
-        update_apt = False
-
-    adt_run_cmd = _build_adt_run_test_command(
+    extra_depends = debian.get_extra_dependencies(args.suites)
+    adt_run_cmd = cmds.build_adt_run_test_command(
         config_stack,
-        config_section,
-        tests_to_run=tests_to_run,
+        args.config_section,
+        tests_to_run=args.suites,
         extra_depends=extra_depends,
-        update_apt=update_apt,
-        network_ready=network_ready,
-        touch_visualization=touch_visualization,
-        add_ppa=True,
-        verbose=verbose)
+        network_ready=args.network_ready,
+        touch_visualization=args.touch_visualization,
+        verbose=args.verbose)
     return subprocess.call(adt_run_cmd)
 
 
-def _get_extra_dependencies(tests_to_run):
-    """
-    Get any extra dependencies required to run the list of
-    tests provided
-
-    :param tests_to_run: The list of tests to run
-    :returns: A list of the extra dependencies
-    """
-    depends_map = _read_dependency_files(
-        os.path.realpath('ubuntu_system_tests/external_tests')
-    )
-    extra_depends = []
-    for test in tests_to_run:
-        extra_depends.extend(_extra_dependencies_for_test(test, depends_map))
-    return list(set(extra_depends))
-
-
-def _read_dependency_files(deps_path):
-    """
-    Read the information about how dependencies map
-    to test names from the .json files in the path
-    provided.
-
-    :param deps_path: The path to the directory containing the files
-    :returns: A dictionary associating test prefixes with lists of dependencies
-    """
-    depends_map = {}
-    for file in glob.glob(deps_path + '/*.json'):
-        with open(file) as deps_file:
-            for depends in json.loads(deps_file.read())['test_dependencies']:
-                depends_map[depends['id']] = depends['depends']
-    return depends_map
-
-
-def _extra_dependencies_for_test(test, depends_map):
-    """
-    Calculates the extra dependencies required for a particular test
-
-    :param test: The test to get the dependencies for
-    :param depends_map: The dependency mapping
-    :returns: A list of the extra dependencies
-    """
-    extra_dependencies = []
-    for id in depends_map.keys():
-        if test.startswith(id):
-            extra_dependencies.append(depends_map[id])
-    return extra_dependencies
-
-
-def _create_temp_control_file(dependencies,
-                              control_file='debian/tests/control'):
-    """Create a copy of the test suite control file, appending the list of
-    dependencies
-
-    :param dependencies: List of dependencies to append to file
-    :returns: File path to the new control file
-    """
-    # create a temporary file
-    _, tmp_path = tempfile.mkstemp(text=True)
-    # open existing control file
-    with open(os.path.realpath(control_file), 'r') as tests_control:
-        new_content = _append_dependencies(
-            tests_control, dependencies)
-        # write the new content to the temp file
-        with open(tmp_path, 'w') as tmp_file:
-            tmp_file.write(new_content)
-    # save the path in config
-    return tmp_path
-
-
-def _append_dependencies(tests_control, dependencies):
-    """Append the list of dependencies to the file content and return"""
-    content = ''
-    for pkg in deb822.Packages.iter_paragraphs(tests_control):
-        if pkg['Tests'] == 'systemtests':
-            current_deps = pkg['Depends']
-            for dep in dependencies:
-                current_deps += '\n         {},'.format(dep)
-            pkg['Depends'] = current_deps
-        content += pkg.dump() + '\n'
-    return content.rstrip()
-
-
-def _is_installed():
-    """Return True if the suite is running from installed location."""
-    return os.path.realpath(__file__).startswith(INSTALLED_PYTHON_DIR)
-
-
-def _get_debian_path():
-    """Return the path of the debian folder.
-    This will depend on whether tests are running from installed location
-    or local source directory.
-    """
-    return INSTALLED_SHARE_DIR if _is_installed() else os.getcwd()
-
-
-def _build_basic_adt_run_test_command(config_stack, extra_depends, verbose):
-    """ Build and return the first part of the adt-run """
-    output_dir = config_stack.get('output_dir')
-
-    run_cmd = ['adt-run']
-    if verbose:
-        run_cmd.append('-ddd')
-    if extra_depends:
-        run_cmd.append('--override-control={}'.format(
-            _create_temp_control_file(extra_depends))
-        )
-    run_cmd.extend([
-        # Updating and installing can take a while.
-        '--timeout-short={}'.format(_ADT_RUN_SHORT_TIMEOUT),
-        # Regression tests can take up to 8 hours, no reason to cut short.
-        '--timeout-test={}'.format(_ADT_RUN_TEST_TIMEOUT),
-        '-B',
-        '--unbuilt-tree={}'.format(_get_debian_path()),
-        '--output-dir',
-        output_dir,
-        # Even if the config file on the host has a different name, on the
-        # testbed it's copied with the default name.
-        '--copy',
-        '{src}:{dest}'.format(
-            src=config_stack.file_path,
-            dest=config.get_device_config_file_path()),
-    ])
-    return run_cmd
-
-
-def _build_adt_run_test_command(config_stack, config_section,
-                                tests_to_run=None, extra_depends=None,
-                                update_apt=False, network_ready=False,
-                                touch_visualization=False, add_ppa=False,
-                                dist_upgrade=False, install_tests=False,
-                                cache_deps=False, verbose=False):
-
-    device_password = config_stack.get('device_password')
-    run_cmd = _build_basic_adt_run_test_command(
-        config_stack, extra_depends, verbose)
-
-    ssh_cmd = (
-        ['ssh', '-s', 'adb', '--', '-p', device_password] +
-        _get_serial_argument(config_stack)
-    )
-    run_cmd.extend(_get_set_tests_to_run_command(tests_to_run))
-    run_cmd.extend(_get_set_config_section_command(config_section))
-    run_cmd.extend(_get_set_touch_visualization(touch_visualization))
-    run_cmd.extend(_get_network_setup_command(config_stack, network_ready))
-    run_cmd.extend(_get_update_apt_command(update_apt))
-    run_cmd.extend(_get_apparmor_rules_update_command())
-    run_cmd.extend(ssh.get_ssh_config_commands())
-
-    if cache_deps:
-        run_cmd.extend(_get_apt_cacher_setup_command())
-        return run_cmd + ['---'] + ssh_cmd
-
-    if dist_upgrade:
-        run_cmd.extend(_get_dist_upgrade_setup_command())
-        return run_cmd + ['---'] + ssh_cmd
-
-    run_cmd = _add_ppa_commands(add_ppa, run_cmd)
-
-    if install_tests:
-        run_cmd.extend(_get_install_tests_setup_command())
-    else:
-        run_cmd.extend(_get_fs_ro_command())
-
-    return run_cmd + ['---'] + ssh_cmd
-
-
-def _add_ppa_commands(add_ppa, run_cmd):
-    """Add the ppa setup commands if required and return command list."""
-    if add_ppa:
-        run_cmd.extend(_get_fs_rw_command())
-        run_cmd.extend(_get_ppa_and_testability_setup_command())
-    return run_cmd
-
-
-def _get_set_tests_to_run_command(tests_to_run):
-    """Return command to set list of tests as environment variable."""
-    tests = ' '.join(tests_to_run or [])
-    return _get_set_env_var_command('TESTS_TO_RUN', tests)
-
-
-def _get_set_config_section_command(config_section):
-    """Return command to set config section name as environment variable."""
-    return _get_set_env_var_command('CONFIG_SECTION', config_section)
-
-
-def _get_set_touch_visualization(touch_visualization):
-    """Return command to set touch visualization as environment variable."""
-    return _get_set_env_var_command(
-        'TOUCH_VISUALIZATION', str(touch_visualization).lower())
-
-
-def _get_set_env_var_command(name, value):
-    """Return command to set environment variable on device."""
-    return ['--env={n}={v}'.format(n=name, v=value)]
-
-
-def _get_update_apt_command(update_apt):
-    """Return command to update apt."""
-    if update_apt:
-        return [
-            '--setup-commands',
-            # emulate pitti's convenience 'ro-apt-update' script, see
-            # http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/commit/?id=a51f26f61c376  # NOQA
-            'echo "Mounting rw to update"; RETRY=0; '
-            'until mount -o remount,rw / || [ $RETRY -gt 10 ]; do sleep 1; '
-            'RETRY=$(($RETRY+1)); echo "mount attempt failed, trying '
-            'again..."; done; if [ $RETRY -gt 10 ]; then exit 1; fi; '
-            'echo "apt-get updating"; apt-get update || '
-            '(sleep 15; apt-get update); '
-            'echo "Mounting ro after updating"; RETRY=0; '
-            'until mount -o remount,ro / || [ $RETRY -gt 60 ]; do '
-            'sleep 1; RETRY=$(($RETRY+1)); echo "mount attempt failed, '
-            'trying again..."; done; if [ $RETRY -gt 60 ]; then exit 1; fi'
-        ]
-    return []
-
-
-def _get_device_network_command(wifi_ssid, wifi_password):
-    """Return command to configure wifi network on device.
-
-    :param wifi_ssid: SSID of required wifi network
-    :param wifi_password: Password of required wifi network
-
-    """
-    cmd_fmt = ('nmcli device wifi connect '
-               '\'{ssid}\' password \'{password}\' 2>/dev/null || true')
-    return cmd_fmt.format(ssid=wifi_ssid, password=wifi_password)
-
-
-def _get_fs_rw_command():
-    return [
-        '--setup-commands',
-        'echo "Mounting rw to install packages"; '
-        'sync; RETRY=0; until mount -o remount,rw / || [ $RETRY -gt 10 ]; '
-        'do sleep 1; RETRY=$(($RETRY+1)); echo "mount attempt failed, trying '
-        'again..."; done; if [ $RETRY -gt 10 ]; then exit 1; fi'
-    ]
-
-
-def _get_fs_ro_command():
-    return [
-        '--setup-commands',
-        'echo "Mounting ro after installing packages"; '
-        'sync; RETRY=0; until mount -o remount,ro / || [ $RETRY -gt 10 ]; '
-        'do sleep 1; RETRY=$(($RETRY+1)); echo "mount attempt failed, trying '
-        'again..."; done; if [ $RETRY -gt 10 ]; then exit 1; fi'
-    ]
-
-
-def _get_network_setup_command(config_stack, network_ready):
-    """ The network needs to be configured on the device
-    This setup-command should be the first to run so that wi-fi is ready
-    """
-    cmd = []
-    if not network_ready:
-        network_cmd = _get_device_network_command(
-            config_stack.get('wifi_ssid'),
-            config_stack.get('wifi_password'))
-        cmd = ['--setup-commands', network_cmd]
-
-    return cmd + [
-        '--setup-commands', 'echo \'Checking device network connection...\'; '
-        'RETRY=0; until wget -q --spider launchpad.net || [ $RETRY -gt 30 ]; '
-        'do sleep 1; RETRY=$(($RETRY+1)); echo "wget attempt $RETRY failed, '
-        ' trying again..."; done; if [ $RETRY -gt 30 ]; then echo \'Device '
-        'has no network connection!\'; exit 1; fi'
-    ]
-
-
-def _get_ppa_and_testability_setup_command():
-    return [
-        '--setup-commands',
-        'if ! ls /etc/apt/sources.list.d/canonical-platform-qa* > /dev/null '
-        '2>&1; then '
-        'apt-add-repository -y ppa:canonical-platform-qa/selenium; '
-        'apt-add-repository -y ppa:canonical-platform-qa/ubuntu-system-tests; '
-        'apt-get --no-list-cleanup update -o Dir::Etc::SourceList=/dev/null; '
-        'fi',
-        '--setup-commands',
-        'if apt-cache policy qttestability-autopilot | '
-        'grep -q "Installed: (none)"; then '
-        'apt-get install -y --no-install-recommends qttestability-autopilot; '
-        'fi'
-    ]
-
-
-def _get_reboot_command():
-    """Return command to reboot device."""
-    return ['--setup-commands', 'reboot']
-
-
-def _get_apt_cacher_autostart_command():
-    """Write the apt-cacher-ng conf file into /etc/init/apt-cacher-ng.conf
-    to re-launch apt-cacher-ng on each subsequent reboot.
-
-    This method assumes that upstart is used as the init system. If this
-    changes in future then this method will need updating to accomodate that.
-    """
-    script = ('start on runlevel [2345]\n'
-              'script\n' +
-              _get_mount_dir_command(EXT_LOG_DIR, APT_CACHER_LOG_DIR) +
-              _get_mount_dir_command(EXT_CACHE_DIR, APT_CACHER_CACHE_DIR) +
-              _get_apt_cacher_start_command() +
-              '\nend script')
-    return ['--setup-commands',
-            _get_create_file_command(
-                content=script, path=APT_CACHER_INIT_FILE)]
-
-
-def _get_apt_cacher_install_command():
-    """Return command to install apt-cacher-ng on device and configure it to
-    use userdata partion for caching and logging."""
-    return [
-        '--setup-commands',
-        _get_install_package_command(APT_CACHER) +
-        _get_apt_cacher_stop_command() +
-        _get_create_folder_command(
-            path=EXT_LOG_DIR, owner=APT_CACHER, permissions=PERMISSION) +
-        _get_create_folder_command(
-            path=EXT_CACHE_DIR, owner=APT_CACHER, permissions=PERMISSION) +
-        _get_create_folder_command(
-            path=APT_CACHER_CACHE_DIR, owner=APT_CACHER,
-            permissions=PERMISSION) +
-        _get_mount_dir_command(EXT_LOG_DIR, APT_CACHER_LOG_DIR) +
-        _get_mount_dir_command(EXT_CACHE_DIR, APT_CACHER_CACHE_DIR) +
-        _get_create_file_command(content=APT_PROXY, path=APT_PROXY_FILE) +
-        _get_apt_cacher_start_command()]
-
-
-def _get_create_folder_command(path, owner, permissions):
-    """Return command to create a folder with specified path, owner and
-    permissions."""
-    return ('if ! [ -d {path} ]; then '
-            'mkdir -p {path}; fi; '
-            'chown {owner}:{owner} {path}; '
-            'chmod {permissions} {path}; ').format(
-        path=path, owner=owner, permissions=permissions)
-
-
-def _get_install_package_command(package):
-    """Return command to install package on device."""
-    return ('apt-get -y install {package}; ').format(package=package)
-
-
-def _get_mount_dir_command(target, source):
-    """Return command to create a symlink between source and target
-    directories."""
-    return 'mount -o bind {target} {source}; '.format(
-        target=target, source=source)
-
-
-def _get_create_file_command(content, path):
-    """Return command to create file with specified content and path."""
-    return 'echo \'{content}\' | tee {path}; '.format(
-        content=content, path=path)
-
-
-def _get_apt_cacher_start_command():
-    """Return command to start apt-cacher-ng process."""
-    return ('/usr/sbin/apt-cacher-ng -c /etc/apt-cacher-ng '
-            'pidfile=/var/run/apt-cacher-ng/pid '
-            'SocketPath=/var/run/apt-cacher-ng/socket '
-            'foreground=0; ')
-
-
-def _get_apt_cacher_stop_command():
-    """Return command to stop apt-cacher-ng process."""
-    return 'pkill apt-cacher-ng || true; '
-
-
-def _get_dist_upgrade_command():
-    """Return command to dist-upgrade device."""
-    return ['--setup-commands', 'apt-get -y dist-upgrade']
-
-
-def _get_apt_cacher_setup_command():
-    """Return command to install and configure apt-cacher-ng on device."""
-    return (_get_fs_rw_command() + _get_apt_cacher_install_command() +
-            _get_apt_cacher_autostart_command() + _get_reboot_command())
-
-
-def _get_dist_upgrade_setup_command():
-    """Return command to dist-upgrade the device."""
-    return (_get_fs_rw_command() + _get_dist_upgrade_command() +
-            _get_reboot_command())
-
-
-def _get_install_tests_setup_command():
-    """ Get a list with all the dependencies from the tests/control file and
-    when the apt-get install does not install any package, the device is set
-    as ro and exit, otherwise it is rebooted and will start as ro
-    """
-    dir_name = _get_debian_path()
-    dependencies = testdesc.get_all_dependencies(dir_name, 'TMPINSTALL')
-    install_command = 'apt-get -o dir::cache::archives="/tmp/" -y install'
-    clean_command = 'apt-get clean'
-    for dep in dependencies:
-        install_command += ' {}'.format(dep.split()[0])
-
-    return ['--setup-commands', clean_command,
-            '--setup-commands', install_command,
-            '--setup-commands', 'sleep 5',
-            '--setup-commands', clean_command,
-            '--setup-commands', 'rm -f /userdata/.writable_image',
-            '--setup-commands', 'reboot']
-
-
-def _get_apparmor_rules_update_command():
-    rules_file = '/var/cache/apparmor/click-ap.rules'
-    update_command = 'printf "dbus (receive, send) ' \
-                     'bus=session path=/com/canonical/Autopilot/**,\n' \
-                     '/tmp/adt-run.** r,\n" > {rules};' \
-                     'aa-clickhook --include={rules}'.format(rules=rules_file)
-    return [
-        '--setup-commands',
-        'if [ ! -e {rules} ];then {command};fi'.format(rules=rules_file,
-                                                       command=update_command)
-    ]
-
-
 def generate_user_report(config_stack):
     """Create a human readable report from the subunit results."""
     artifacts_directory = results.get_artifacts_directory(config_stack)

=== added file 'ubuntu_system_tests/host/commands.py'
--- ubuntu_system_tests/host/commands.py	1970-01-01 00:00:00 +0000
+++ ubuntu_system_tests/host/commands.py	2016-06-29 08:06:07 +0000
@@ -0,0 +1,535 @@
+#!/usr/bin/env python3
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu System Tests
+# Copyright (C) 2014-2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+from ubuntu_system_tests.common import (
+    config,
+    ssh,
+)
+from ubuntu_system_tests.host import (
+    debian,
+    testdesc,
+)
+
+# apt-cacher-ng definitions
+APT_CACHER = 'apt-cacher-ng'
+APT_CACHER_CACHE_DIR = '/var/cache/apt-cacher-ng'
+APT_CACHER_LOG_DIR = '/var/log/apt-cacher-ng/'
+APT_CACHER_INIT_FILE = '/etc/init/apt-cacher-ng.conf'
+APT_PROXY = 'Acquire::http { Proxy "http://127.0.0.1:3142"; };'
+APT_PROXY_FILE = '/etc/apt/apt.conf.d/01proxy'
+EXT_CACHE_DIR = '/userdata/apt-cacher-ng/cache'
+EXT_LOG_DIR = '/userdata/apt-cacher-ng/logs'
+PERMISSION = '755'
+
+QTTESTABILITY_AUTOPILOT = 'qttestability-autopilot'
+
+PPA_LIST = ['ppa:canonical-platform-qa/selenium',
+            'ppa:canonical-platform-qa/ubuntu-system-tests']
+INSTALL_LIST = [QTTESTABILITY_AUTOPILOT]
+
+
+_ADT_RUN_SHORT_TIMEOUT = 600  # 10 minutes.
+_ADT_RUN_TEST_TIMEOUT = 86400  # 24 hours
+
+REBOOT_FILE = '/tmp/ust-reboot-required'
+
+STATUS_NO_CONNECTION = 1
+STATUS_FS_RO_MOUNT_ERROR = 2
+STATUS_FS_RW_MOUNT_ERROR = 3
+STATUS_REBOOT_REQUIRED = 4
+
+
+def build_basic_adt_run_test_command(config_stack, extra_depends, verbose):
+    """Build and return first part of adt-run command for test execution."""
+    run_cmd = _get_adt_run_cmd_start(config_stack, verbose)
+    if extra_depends:
+        run_cmd.append('--override-control={}'.format(
+            debian.create_temp_control_file(extra_depends))
+        )
+    run_cmd.extend([
+        # Updating and installing can take a while.
+        '--timeout-short={}'.format(_ADT_RUN_SHORT_TIMEOUT),
+        # Regression tests can take up to 8 hours, no reason to cut short.
+        '--timeout-test={}'.format(_ADT_RUN_TEST_TIMEOUT),
+        '--testname=systemtests',
+        '-B',
+        '--unbuilt-tree={}'.format(debian.get_debian_path()),
+        # Even if the config file on the host has a different name, on the
+        # testbed it's copied with the default name.
+        '--copy',
+        '{src}:{dest}'.format(
+            src=config_stack.file_path,
+            dest=config.get_device_config_file_path()),
+    ])
+    return run_cmd
+
+
+def build_basic_adt_run_setup_command(config_stack, verbose, cache_deps):
+    """Build and return first part of adt-run command for device setup."""
+    run_cmd = _get_adt_run_cmd_start(config_stack, verbose)
+    if cache_deps:
+        # Run the setup script from the control file listing all required
+        # dependencies. This will ensure all required dependencies are
+        # downloaded and cached as part of the setup command.
+        run_cmd.append('--testname=setup')
+    else:
+        # Dependency caching is not required. In this case don't download
+        # any dependencies by using a control file with no dependencies.
+        tmp_control = debian.create_temp_file('Tests: setup\nDepends: \n')
+        run_cmd.append('--override-control={}'.format(tmp_control))
+    run_cmd.extend([
+        '-B',
+        '--unbuilt-tree={}'.format(debian.get_debian_path())])
+    return run_cmd
+
+
+def build_adt_run_setup_command(config_stack, config_section, update_apt=False,
+                                network_ready=False, dist_upgrade=False,
+                                install_tests=False, cache_deps=False,
+                                verbose=False):
+    """Build and return adt-run command for device setup."""
+    run_cmd = build_basic_adt_run_setup_command(
+        config_stack, verbose, cache_deps)
+
+    run_cmd.extend(_get_delete_reboot_required_setup_command())
+    run_cmd.extend(_get_set_config_section_command(config_section))
+    run_cmd.extend(_get_network_setup_command(config_stack, network_ready))
+    run_cmd.extend(_get_disable_auto_reboot_setup_command())
+    run_cmd.extend(_get_add_ppa_if_required_setup_command())
+    run_cmd.extend(_get_update_apt_setup_command(update_apt))
+    run_cmd.extend(_get_install_test_pkgs_if_required_setup_command())
+    run_cmd.extend(_get_apt_cacher_install_setup_command(cache_deps))
+    run_cmd.extend(_get_dist_upgrade_if_required_setup_command(dist_upgrade))
+    run_cmd.extend(_get_install_tests_setup_command(install_tests))
+    run_cmd.extend(_get_fs_ro_setup_command())
+    run_cmd.extend(_get_apparmor_rules_update_setup_command())
+    run_cmd.extend(_get_check_reboot_required_setup_command())
+
+    return run_cmd + ['---'] + _get_ssh_command(config_stack)
+
+
+def build_adt_run_test_command(config_stack, config_section,
+                               tests_to_run=None, extra_depends=None,
+                               network_ready=False, touch_visualization=False,
+                               verbose=False):
+    """Build and return adt-run command for executing tests on device."""
+    run_cmd = build_basic_adt_run_test_command(
+        config_stack, extra_depends, verbose)
+
+    run_cmd.extend(_get_set_tests_to_run_command(tests_to_run))
+    run_cmd.extend(_get_set_config_section_command(config_section))
+    run_cmd.extend(_get_set_touch_visualization(touch_visualization))
+    run_cmd.extend(_get_network_setup_command(config_stack, network_ready))
+    run_cmd.extend(_get_apparmor_rules_update_setup_command())
+    run_cmd.extend(ssh.get_ssh_config_commands())
+
+    return run_cmd + ['---'] + _get_ssh_command(config_stack)
+
+
+def _get_adt_run_cmd_start(config_stack, verbose):
+    """Return the start of adt_run command."""
+    output_dir = config_stack.get('output_dir')
+    run_cmd = ['adt-run', '--output-dir', output_dir]
+    if verbose:
+        run_cmd.append('-ddd')
+    return run_cmd
+
+
+def _get_ssh_command(config_stack):
+    """Return command parameters for ssh connection to requried device."""
+    device_password = config_stack.get('device_password')
+    return (['ssh', '-s', 'adb', '--', '-p', device_password] +
+            _get_serial_argument(config_stack))
+
+
+def build_enable_autopilot_introspection_command(config_stack):
+    """Return command to enable autopilot introspection on required device."""
+    command = (
+        ['phablet-config'] +
+        _get_serial_argument(config_stack) +
+        ['autopilot', '--dbus-probe', 'enable']
+    )
+    return command
+
+
+def _get_adb_reboot_commands(config_stack):
+    """Return list of adb commands to reboot device."""
+    return [
+        _get_adb_command(config_stack, 'shell', 'sync'),
+        _get_adb_command(config_stack, 'reboot')]
+
+
+def _get_adb_command(config_stack, *args):
+    """Return adb command to run specified command args on connected device."""
+    return ['adb'] + _get_serial_argument(config_stack) + list(args)
+
+
+def _get_serial_argument(config_stack):
+    """Return serial commands from config, or empty list if not specified."""
+    device_serial = config_stack.get('device_serial')
+    if device_serial:
+        return ['-s', device_serial]
+    else:
+        # No serial in the config, run the commands as if only one device is
+        # connected.
+        return []
+
+
+def _get_set_tests_to_run_command(tests_to_run):
+    """Return command to set list of tests as environment variable."""
+    tests = ' '.join(tests_to_run or [])
+    return _get_set_env_var_adt_command('TESTS_TO_RUN', tests)
+
+
+def _get_set_config_section_command(config_section):
+    """Return command to set config section name as environment variable."""
+    return _get_set_env_var_adt_command('CONFIG_SECTION', config_section)
+
+
+def _get_set_touch_visualization(touch_visualization):
+    """Return command to set touch visualization as environment variable."""
+    return _get_set_env_var_adt_command(
+        'TOUCH_VISUALIZATION', str(touch_visualization).lower())
+
+
+def _get_set_env_var_adt_command(name, value):
+    """Return command to set environment variable on device."""
+    return ['--env={n}={v}'.format(n=name, v=value)]
+
+
+def _get_update_apt_command():
+    """Return command to apt-get update."""
+    return ('echo "apt-get updating"; apt-get update || '
+            '(sleep 15; apt-get update); ')
+
+
+def _get_update_apt_setup_command(update_apt):
+    """Return setup command to apt-get update."""
+    if update_apt:
+        return [
+            '--setup-commands',
+            _get_fs_rw_command() +
+            _get_update_apt_command()]
+    return []
+
+
+def _get_device_network_command(wifi_ssid, wifi_password):
+    """Return command to configure wifi network on device.
+
+    :param wifi_ssid: SSID of required wifi network
+    :param wifi_password: Password of required wifi network
+
+    """
+    cmd_fmt = ('nmcli device wifi connect '
+               '\'{ssid}\' password \'{password}\' 2>/dev/null || true')
+    return cmd_fmt.format(ssid=wifi_ssid, password=wifi_password)
+
+
+def _get_is_fs_ro_command():
+    """Return command to indicate if file system is currently read-only."""
+    return 'grep \' /\ ext\' /proc/mounts | grep -q \'ro\''
+
+
+def _get_is_fs_rw_command():
+    """Return command to indicate if file system is currently writable."""
+    return 'grep \' /\ ext\' /proc/mounts | grep -q \'rw\''
+
+
+def _get_fs_rw_command():
+    """Return command to set file system read-write if not already."""
+    return _get_if_command(_get_is_fs_ro_command(), _get_set_fs_rw_command())
+
+
+def _get_fs_ro_setup_command():
+    """Return setup command to set file system read-only if not already."""
+    return [
+        '--setup-commands',
+        _get_if_command(_get_is_fs_rw_command(), _get_set_fs_ro_command())
+        ]
+
+
+def _get_set_fs_rw_command():
+    """Return command to set file system writable, retyring on failure."""
+    return ('echo "Mounting file system rw"; '
+            'sync; RETRY=0; until mount -o remount,rw / || [ $RETRY -gt 10 ]; '
+            'do sleep 1; RETRY=$(($RETRY+1)); echo "mount attempt failed, '
+            'trying again..."; done; if [ $RETRY -gt 10 ]; then exit '
+            '{status}; fi; '.format(status=STATUS_FS_RW_MOUNT_ERROR))
+
+
+def _get_set_fs_ro_command():
+    """Return command to set file system read-only, retyring on failure."""
+    return ('echo "Mounting file system ro"; '
+            'sync; RETRY=0; until mount -o remount,ro / || [ $RETRY -gt 10 ]; '
+            'do sleep 1; RETRY=$(($RETRY+1)); echo "mount attempt failed, '
+            'trying again..."; done; if [ $RETRY -gt 10 ]; then exit '
+            '{status}; fi; '.format(status=STATUS_FS_RO_MOUNT_ERROR))
+
+
+def _get_network_setup_command(config_stack, network_ready):
+    """ The network needs to be configured on the device
+    This setup-command should be the first to run so that wi-fi is ready
+    """
+    cmd = []
+    if not network_ready:
+        network_cmd = _get_device_network_command(
+            config_stack.get('wifi_ssid'),
+            config_stack.get('wifi_password'))
+        cmd = ['--setup-commands', network_cmd]
+
+    return cmd + [
+        '--setup-commands', 'echo \'Checking device network connection...\'; '
+        'RETRY=0; until wget -q --spider launchpad.net || [ $RETRY -gt 60 ]; '
+        'do sleep 1; RETRY=$(($RETRY+1)); echo "wget attempt $RETRY failed, '
+        ' trying again..."; done; if [ $RETRY -gt 60 ]; then echo \'Device '
+        'has no network connection!\'; exit {status}; fi'.format(
+            status=STATUS_NO_CONNECTION)
+    ]
+
+
+def _get_apt_cacher_autostart_command():
+    """Write the apt-cacher-ng conf file into /etc/init/apt-cacher-ng.conf
+    to re-launch apt-cacher-ng on each subsequent reboot.
+
+    This method assumes that upstart is used as the init system. If this
+    changes in future then this method will need updating to accomodate that.
+    """
+    script = ('start on runlevel [2345]\n'
+              'script\n' +
+              _get_mount_dir_command(EXT_LOG_DIR, APT_CACHER_LOG_DIR) +
+              _get_mount_dir_command(EXT_CACHE_DIR, APT_CACHER_CACHE_DIR) +
+              _get_apt_cacher_start_command() +
+              '\nend script')
+    return _get_create_file_command(content=script, path=APT_CACHER_INIT_FILE)
+
+
+def _get_apt_cacher_install_setup_command(apt_cacher_required):
+    """Return command to install apt-cacher-ng on device and configure it to
+    use userdata partion for caching and logging. If apt-cacher-ng is already
+    installed on the device then no action will be taken."""
+    if not apt_cacher_required:
+        return []
+    is_apt_cacher_installed = _get_packages_installed_command(APT_CACHER)
+    install_apt_cacher_cmds = (
+        _get_fs_rw_command() +
+        _get_install_packages_command(APT_CACHER) +
+        _get_apt_cacher_stop_command() +
+        _get_create_folder_command(
+            path=EXT_LOG_DIR, owner=APT_CACHER, permissions=PERMISSION) +
+        _get_create_folder_command(
+            path=EXT_CACHE_DIR, owner=APT_CACHER, permissions=PERMISSION) +
+        _get_create_folder_command(
+            path=APT_CACHER_CACHE_DIR, owner=APT_CACHER,
+            permissions=PERMISSION) +
+        _get_mount_dir_command(EXT_LOG_DIR, APT_CACHER_LOG_DIR) +
+        _get_mount_dir_command(EXT_CACHE_DIR, APT_CACHER_CACHE_DIR) +
+        _get_create_file_command(content=APT_PROXY, path=APT_PROXY_FILE) +
+        _get_apt_cacher_start_command() +
+        _get_apt_cacher_autostart_command())
+    return ['--setup-commands', _get_if_not_command(
+        is_apt_cacher_installed, install_apt_cacher_cmds)]
+
+
+def _get_if_not_command(condition, commands):
+    """Return if command for specified condition and commands."""
+    return 'if ! ({condition}); then {commands} fi; '.format(
+        condition=condition, commands=commands)
+
+
+def _get_if_command(condition, commands):
+    """Return if not command for specified condition and commands."""
+    return 'if ({condition}); then {commands} fi; '.format(
+        condition=condition, commands=commands)
+
+
+def _get_create_folder_command(path, owner, permissions):
+    """Return command to create a folder with specified path, owner and
+    permissions."""
+    return ('if ! [ -d {path} ]; then '
+            'mkdir -p {path}; fi; '
+            'chown {owner}:{owner} {path}; '
+            'chmod {permissions} {path}; ').format(
+        path=path, owner=owner, permissions=permissions)
+
+
+def _get_install_test_packages_command():
+    """Return command to install required test packages."""
+    return (_get_fs_rw_command() +
+            _get_install_packages_command(' '.join(INSTALL_LIST)))
+
+
+def _get_test_packages_installed_command():
+    """Return command to indicate whether all required packages are already
+    installed or not.
+    :param apt_cacher_required: Whether apt-cahcer-ng is required to be
+                                installed or not.
+    """
+    return _get_packages_installed_command(' '.join(INSTALL_LIST))
+
+
+def _get_packages_installed_command(packages):
+    """Return command to tell if all listed packages are installed or not."""
+    return ('! apt-cache policy {packages} | '
+            'grep -q \'Installed: (none)\'; '.format(packages=packages))
+
+
+def _get_add_ppa_required_command():
+    """Return command to indicate if any ppa is required to be added."""
+    return ('! ls /etc/apt/sources.list.d/canonical-platform-qa* > /dev/null '
+            '2>&1')
+
+
+def _get_add_ppas_command():
+    """Return command to add all required ppas."""
+    cmd = ''
+    for ppa in PPA_LIST:
+        cmd += 'apt-add-repository -y {ppa}; '.format(ppa=ppa)
+    return _get_fs_rw_command() + cmd
+
+
+def _get_add_ppa_if_required_setup_command():
+    """Return setup command to add required ppas to device if required."""
+    return ['--setup-commands', _get_if_command(
+        _get_add_ppa_required_command(), _get_add_ppas_command())]
+
+
+def _dist_upgrade_required_command():
+    """Return command to indicate if any packages need to be updated as part
+    of a dist-upgrade command."""
+    return ('su - $ADT_NORMAL_USER -c \'apt-get dist-upgrade -s\' | '
+            'grep -q \'Inst\\|Conf\\|Remv\'; ')
+
+
+def _get_dist_upgrade_if_required_setup_command(required):
+    """Return setup command to perform dist-upgrade if required."""
+    if not required:
+        return []
+    return ['--setup-commands', _get_if_command(
+        _dist_upgrade_required_command(), _get_dist_upgrade_command())]
+
+
+def _get_install_test_pkgs_if_required_setup_command():
+    """Return setup command to install test packages if required."""
+    return ['--setup-commands',
+            _get_if_not_command(
+                _get_test_packages_installed_command(),
+                _get_install_test_packages_command())]
+
+
+def _get_install_packages_command(packages):
+    """Return command to install packages on device."""
+    return ('apt-get -y --no-install-recommends install {packages}; ').format(
+        packages=packages)
+
+
+def _get_mount_dir_command(target, source):
+    """Return command to create a symlink between source and target
+    directories."""
+    return 'mount -o bind {target} {source}; '.format(
+        target=target, source=source)
+
+
+def _get_create_file_command(content, path):
+    """Return command to create file with specified content and path."""
+    return 'echo \'{content}\' | tee {path}; '.format(
+        content=content, path=path)
+
+
+def _get_apt_cacher_start_command():
+    """Return command to start apt-cacher-ng process."""
+    return ('/usr/sbin/apt-cacher-ng -c /etc/apt-cacher-ng '
+            'pidfile=/var/run/apt-cacher-ng/pid '
+            'SocketPath=/var/run/apt-cacher-ng/socket '
+            'foreground=0; ')
+
+
+def _get_apt_cacher_stop_command():
+    """Return command to stop apt-cacher-ng process."""
+    return 'pkill apt-cacher-ng || true; '
+
+
+def _get_dist_upgrade_command():
+    """Return command to dist-upgrade device."""
+    return (_get_set_reboot_required_command() + 'apt-get -y dist-upgrade; ')
+
+
+def _get_check_reboot_required_setup_command():
+    """If a reboot is required, then exit with status to indicate that."""
+    return [
+        '--setup-commands',
+        'if [ -f {p} ]; then exit {s}; fi; '.format(
+            p=REBOOT_FILE, s=STATUS_REBOOT_REQUIRED)]
+
+
+def _get_set_reboot_required_command():
+    """Create file to indicate that reboot is required."""
+    return 'if [ ! -f {p} ]; then touch {p}; fi; '.format(p=REBOOT_FILE)
+
+
+def _get_delete_reboot_required_setup_command():
+    """Delete file to indicate that reboot is required."""
+    return [
+        '--setup-commands',
+        'if [ -f {p} ]; then rm -f {p}; fi; '.format(p=REBOOT_FILE)]
+
+
+def _get_disable_auto_reboot_setup_command():
+    """Return setup command to disable automatic rebooting by autopkgtest."""
+    return [
+        '--setup-commands',
+        'if [ ! -f /run/autopkgtest_no_reboot.stamp ]; then '
+        'touch /run/autopkgtest_no_reboot.stamp; fi; ']
+
+
+def _get_install_tests_setup_command(required):
+    """ Get a list with all the dependencies from the tests/control file and
+    when the apt-get install does not install any package, the device is set
+    as ro and exit, otherwise it is rebooted and will start as ro
+    """
+    if not required:
+        return []
+    dir_name = debian.get_debian_path()
+    dependencies = testdesc.get_all_dependencies(dir_name, 'TMPINSTALL')
+    install_command = 'apt-get -o dir::cache::archives="/tmp/" -y install'
+    clean_command = 'apt-get clean; '
+    for dep in dependencies:
+        install_command += ' {}'.format(dep.split()[0])
+    install_command += '; sleep 5; '
+
+    return ['--setup-commands',
+            _get_fs_rw_command() +
+            clean_command +
+            install_command +
+            clean_command]
+
+
+def _get_apparmor_rules_update_setup_command():
+    """Return setup command to update apparmor for autopilot introspection."""
+    rules_file = '/var/cache/apparmor/click-ap.rules'
+    update_command = 'printf "dbus (receive, send) ' \
+                     'bus=session path=/com/canonical/Autopilot/**,\n' \
+                     '/tmp/adt-run.** r,\n" > {rules};' \
+                     'aa-clickhook --include={rules}'.format(rules=rules_file)
+    return [
+        '--setup-commands',
+        'if [ ! -e {rules} ];then {command};fi'.format(rules=rules_file,
+                                                       command=update_command)
+    ]

=== added file 'ubuntu_system_tests/host/debian.py'
--- ubuntu_system_tests/host/debian.py	1970-01-01 00:00:00 +0000
+++ ubuntu_system_tests/host/debian.py	2016-06-29 08:06:07 +0000
@@ -0,0 +1,130 @@
+#!/usr/bin/env python3
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu System Tests
+# Copyright (C) 2014-2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import deb822
+import glob
+import json
+import os
+import tempfile
+
+# Installation path for shared files
+INSTALLED_SHARE_DIR = '/usr/share/ubuntu-system-tests'
+# Installation path for installed python scripts
+INSTALLED_PYTHON_DIR = '/usr/lib/python'
+
+
+def get_extra_dependencies(tests_to_run):
+    """
+    Get any extra dependencies required to run the list of
+    tests provided
+
+    :param tests_to_run: The list of tests to run
+    :returns: A list of the extra dependencies
+    """
+    depends_map = _read_dependency_files(
+        os.path.realpath('ubuntu_system_tests/external_tests')
+    )
+    extra_depends = []
+    for test in tests_to_run:
+        extra_depends.extend(_extra_dependencies_for_test(test, depends_map))
+    return list(set(extra_depends))
+
+
+def _read_dependency_files(deps_path):
+    """
+    Read the information about how dependencies map
+    to test names from the .json files in the path
+    provided.
+
+    :param deps_path: The path to the directory containing the files
+    :returns: A dictionary associating test prefixes with lists of dependencies
+    """
+    depends_map = {}
+    for file in glob.glob(deps_path + '/*.json'):
+        with open(file) as deps_file:
+            for depends in json.loads(deps_file.read())['test_dependencies']:
+                depends_map[depends['id']] = depends['depends']
+    return depends_map
+
+
+def _extra_dependencies_for_test(test, depends_map):
+    """
+    Calculates the extra dependencies required for a particular test
+
+    :param test: The test to get the dependencies for
+    :param depends_map: The dependency mapping
+    :returns: A list of the extra dependencies
+    """
+    extra_dependencies = []
+    for id in depends_map.keys():
+        if test.startswith(id):
+            extra_dependencies.append(depends_map[id])
+    return extra_dependencies
+
+
+def create_temp_control_file(dependencies,
+                             control_file='debian/tests/control'):
+    """Create a copy of the test suite control file, appending the list of
+    dependencies
+
+    :param dependencies: List of dependencies to append to file
+    :returns: File path to the new control file
+    """
+    # open existing control file
+    with open(os.path.realpath(control_file), 'r') as tests_control:
+        new_content = _append_dependencies(
+            tests_control, dependencies)
+    return create_temp_file(new_content)
+
+
+def create_temp_file(content):
+    """Create a temporary file with specified content and return path to it."""
+    # create a temporary file and write content to it
+    _, tmp_path = tempfile.mkstemp(text=True)
+    with open(tmp_path, 'w') as tmp_file:
+        tmp_file.write(content)
+    return tmp_path
+
+
+def _append_dependencies(tests_control, dependencies):
+    """Append the list of dependencies to the file content and return"""
+    content = ''
+    for pkg in deb822.Packages.iter_paragraphs(tests_control):
+        if pkg['Tests'].startswith('systemtests'):
+            current_deps = pkg['Depends']
+            for dep in dependencies:
+                current_deps += '\n         {},'.format(dep)
+            pkg['Depends'] = current_deps
+        content += pkg.dump() + '\n'
+    return content.rstrip()
+
+
+def _is_installed():
+    """Return True if the suite is running from installed location."""
+    return os.path.realpath(__file__).startswith(INSTALLED_PYTHON_DIR)
+
+
+def get_debian_path():
+    """Return the path of the debian folder.
+    This will depend on whether tests are running from installed location
+    or local source directory.
+    """
+    return INSTALLED_SHARE_DIR if _is_installed() else os.getcwd()

=== modified file 'ubuntu_system_tests/selftests/test_command_line.py'
--- ubuntu_system_tests/selftests/test_command_line.py	2016-05-26 16:40:52 +0000
+++ ubuntu_system_tests/selftests/test_command_line.py	2016-06-29 08:06:07 +0000
@@ -18,10 +18,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-import fixtures
-import json
 import os
-from tempfile import NamedTemporaryFile
 from unittest import mock
 
 import testscenarios
@@ -29,248 +26,14 @@
 from ubuntu_system_tests import selftests
 from ubuntu_system_tests.host import command_line
 from ubuntu_system_tests.common import config
-
-
-def check_any_item_in_list_contains_value(list, value):
-    return any(item.find(value) for item in list)
-
-
-def check_any_item_in_list_startswith_value(list, value):
-    return any(item.startswith(value) for item in list)
-
-
-class CommandLineTestCase(selftests.ConfigBaseTestCase):
-
-    def setUp(self):
-        super().setUp()
-        self.config_stack = config.UbuntuSystemTestsConfig()
-
-    def test_setup_network_command_contains_credentials(self):
-        self.set_config_values()
-
-        command = command_line._get_device_network_command(
-            'test_ssid', 'test_password')
-
-        self.assertTrue(command.startswith(
-            'nmcli device wifi connect \'test_ssid\' '
-            'password \'test_password\''))
-
-    def test_enable_introspection_with_serial(self):
-        serial = 'test_serial'
-        self.set_config_values(device_serial=serial)
-
-        command = command_line._build_enable_autopilot_introspection_command(
-            self.config_stack)
-
-        self.assertEqual(['phablet-config', '-s', 'test_serial'], command[:3])
-
-    def test_enable_introspection_without_serial(self):
-        self.set_config_values(device_serial='')
-
-        command = command_line._build_enable_autopilot_introspection_command(
-            self.config_stack)
-
-        self.assertNotIn('-s', command)
-
-    def test_adt_run_must_be_called_with_short_timeout(self):
-        self.set_config_values()
-
-        command = command_line._build_adt_run_test_command(
-            self.config_stack, config.KEY_DEFAULT)
-        command = ' '.join(command)
-
-        run_arguments, _ = command.split('---')
-        self.assertIn('--timeout-short=600', run_arguments)
-
-    def test_ppa_setup_commands_set_image_readwrite(self):
-        self.set_config_values()
-        self.assertIn(
-            'mount -o remount,rw /',
-            ' '.join(command_line._build_adt_run_test_command(
-                self.config_stack, config.KEY_DEFAULT, add_ppa=True))
-        )
-
-    def test_adt_run_command_set_image_back_to_readonly(self):
-        self.set_config_values()
-        self.assertIn(
-            'mount -o remount,ro /',
-            ' '.join(command_line._build_adt_run_test_command(
-                self.config_stack, config.KEY_DEFAULT))
-        )
-
-    def test_ppa_setup_commands_only_update_ppa_details(self):
-        aptget_command = str('apt-get --no-list-cleanup update -o'
-                             ' Dir::Etc::SourceList=/dev/null')
-        self.assertTrue(
-            check_any_item_in_list_contains_value(
-                command_line._get_ppa_and_testability_setup_command(),
-                aptget_command)
-        )
-
-    def test_ppa_setup_commands_install_testability_libs(self):
-        self.assertTrue(
-            check_any_item_in_list_contains_value(
-                command_line._get_ppa_and_testability_setup_command(),
-                'apt-get install -y --no-install-recommends '
-                'qttestability-autopilot')
-        )
-
-    def test_ppa_setup_commands_used_in_adt_run(self):
-        self.set_config_values()
-        command = command_line._build_adt_run_test_command(
-            self.config_stack, config.KEY_DEFAULT, add_ppa=True)
-        command = ' '.join(command)
-
-        run_arguments, _ = command.split('---')
-        self.assertIn(
-            'apt-add-repository -y ppa:canonical-platform-qa/selenium',
-            run_arguments
-        )
-
-    def test_adt_run_with_serial(self):
-        serial = 'test_serial'
-        self.set_config_values(device_serial=serial)
-
-        command = command_line._build_adt_run_test_command(
-            self.config_stack, config.KEY_DEFAULT)
-        command = ' '.join(command)
-
-        _, adb_arguments = command.split('ssh -s adb --')
-        self.assertIn('-s test_serial', adb_arguments)
-
-    def test_adt_run_without_serial(self):
-        self.set_config_values(device_serial='')
-
-        command = command_line._build_adt_run_test_command(
-            self.config_stack, config.KEY_DEFAULT)
-        command = ' '.join(command)
-
-        _, adb_arguments = command.split('ssh -s adb --')
-        self.assertNotIn('-s test_serial', adb_arguments)
-
-    def test_adt_run_without_apt_update_must_be_called_without_setup_command(
-            self):
-        self.set_config_values()
-
-        command = command_line._build_adt_run_test_command(
-            self.config_stack, config.KEY_DEFAULT, update_apt=False)
-        command = ' '.join(command)
-
-        run_arguments, _ = command.split('---')
-        self.assertNotIn('apt-get update', run_arguments)
-
-    def test_adt_run_with_apt_update_must_be_called_with_setup_command(self):
-        self.set_config_values()
-
-        command = command_line._build_adt_run_test_command(
-            self.config_stack, config.KEY_DEFAULT, update_apt=True)
-        command = ' '.join(command)
-
-        run_arguments, _ = command.split('---')
-        self.assertIn('apt-get update', run_arguments)
-
-    def test_adt_run_command_with_network_ready_creates_no_network(self):
-        self.set_config_values()
-
-        command = command_line._build_adt_run_test_command(
-            self.config_stack, config.KEY_DEFAULT, network_ready=True)
-
-        self.assertFalse(check_any_item_in_list_startswith_value(
-            command, 'nmcli device wifi connect'))
-
-    def test_adt_run_command_with_default_parameters_creates_network(self):
-        self.set_config_values()
-
-        command = command_line._build_adt_run_test_command(
-            self.config_stack, config.KEY_DEFAULT)
-
-        self.assertTrue(check_any_item_in_list_startswith_value(
-            command, 'nmcli device wifi connect'))
-
-    def test_get_extra_dependencies(self):
-        tests = ('app1.tests.test_module.TestClass.test_x '
-                 'app2.tests.test_module.TestClass.test_y '
-                 'app3.tests.test_module.TestClass.test_z')
-        self.set_config_values(tests_to_run=tests)
-        expected_depends = ['dep1',
-                            'dep2']
-        # Mock _read_dependency_files
-        with mock.patch(
-            'ubuntu_system_tests.host.command_line._read_dependency_files'
-        ) as mock_read_depends:
-            mock_read_depends.return_value = {
-                'app2.tests': 'dep1',
-                'app3.tests': 'dep2'
-            }
-            extra_depends = command_line._get_extra_dependencies(
-                self.config_stack.get('tests_to_run').split()
-            )
-        self.assertCountEqual(extra_depends, expected_depends)
-
-    def test_read_dependency_file(self):
-        with fixtures.TempDir() as deps_dir:
-            deps_file1 = open(os.path.join(deps_dir.path, 'deps1.json'), 'w')
-            deps_file2 = open(os.path.join(deps_dir.path, 'deps2.json'), 'w')
-            deps_json1 = {
-                "test_dependencies": [
-                    {"id": "app1.tests", "depends": "dep1"}
-                ]
-            }
-            deps_json2 = {
-                "test_dependencies": [
-                    {"id": "app2.tests", "depends": "dep2"}
-                ]
-            }
-            deps_file1.write(json.dumps(deps_json1))
-            deps_file2.write(json.dumps(deps_json2))
-            deps_file1.close()
-            deps_file2.close()
-
-            expected = {'app1.tests': 'dep1', 'app2.tests': 'dep2'}
-            actual = command_line._read_dependency_files(deps_dir.path)
-            self.assertEqual(expected, actual)
-
-    def test_extra_dependencies_for_test(self):
-        test = 'app1.tests.test_module.TestClass.test_y'
-        dependencies_map = {'app1.tests': 'dep1', 'app2.tests': 'dep2'}
-        extra_depends = command_line._extra_dependencies_for_test(
-            test, dependencies_map
-        )
-        self.assertEqual(extra_depends, ['dep1'])
-
-    def test_create_temp_control(self):
-        with NamedTemporaryFile() as control:
-            control.write("""Tests: systemtests
-Depends: dep1,
-         dep2,
-         dep3,
-
-Tests: someothertests
-Depends: other1,
-         other2,
-         other3,""".encode('utf-8'))
-            control.flush()
-            temp_control = command_line._create_temp_control_file(
-                ['dep4', 'dep5'], control.name
-            )
-            with open(temp_control) as temp:
-                tmp_control_content = temp.read()
-                expected_content = """Tests: systemtests
-Depends: dep1,
-         dep2,
-         dep3,
-         dep4,
-         dep5,
-
-Tests: someothertests
-Depends: other1,
-         other2,
-         other3,"""
-                self.assertEqual(tmp_control_content, expected_content)
-
-
-class RunSystemTestsCommandsTestCase(selftests.ConfigBaseTestCase,
-                                     testscenarios.TestWithScenarios):
+from ubuntu_system_tests.selftests.utils import (
+    get_cmdline_run_params,
+    get_cmdline_setup_params,
+)
+
+
+class SetupSystemTestsCommandsTestCase(selftests.ConfigBaseTestCase,
+                                       testscenarios.TestWithScenarios):
 
     scenarios = [
         # The nmcli network setup command is passed as --setup-command option
@@ -278,16 +41,12 @@
         ('without network setup', {
             'keyword_arguments': {'update_apt': False,
                                   'network_ready': True},
-            'expected_commands': ['phablet-config',
-                                  'grep',
-                                  'adt-run'],
+            'expected_commands': ['adt-run'],
         }),
         ('with network setup', {
             'keyword_arguments': {'update_apt': False,
                                   'network_ready': False},
-            'expected_commands': ['phablet-config',
-                                  'grep',
-                                  'adt-run'],
+            'expected_commands': ['adt-run'],
         }),
     ]
 
@@ -296,12 +55,18 @@
         self.config_stack = config.UbuntuSystemTestsConfig()
 
     @mock.patch('ubuntu_system_tests.host.command_line.generate_user_report')
-    def test_run_system_tests_commands(self, mock_generate_user_report):
+    def test_setup_system_tests_commands(self, mock_generate_user_report):
         def get_command_from_call(call):
             """Return the name of the command, without the arguments."""
-            return call[0][0][0]
+            cmd_list = call[0][0]
+            if type(cmd_list) == str:
+                cmd_list = cmd_list.split(' ')
+            return cmd_list[0]
 
         self.set_config_values()
+        conf_name = os.path.basename(self.config_stack.file_path)
+        params = get_cmdline_setup_params(
+            config=conf_name, **self.keyword_arguments)
 
         # Do not execute the subprocesses, just record their calls.
         with mock.patch('subprocess.Popen') as mock_popen:
@@ -310,10 +75,7 @@
             enter.communicate = mock.Mock(return_value=(0, 'dummy'))
             enter.poll = mock.Mock(return_value=0)
             enter.wait = mock.Mock(return_value=0)
-
-            conf_name = os.path.basename(self.config_stack.file_path)
-            command_line.run_system_tests(conf_name, config.KEY_DEFAULT, [],
-                                          **self.keyword_arguments)
+            command_line.setup_system_tests(params)
 
         # The first call doesn't run any command.
         calls = mock_popen.call_args_list[1:]
@@ -321,11 +83,11 @@
         self.assertEqual(self.expected_commands, commands)
 
 
-class MainPassingCommandLineArgumentsTestCase(testscenarios.TestWithScenarios):
+class SetupCommandLineArgumentsTestCase(testscenarios.TestWithScenarios):
 
+    SETUP_CMD = 'setup'
     DEFAULT_CONFIG_FILE = 'ubuntu-system-tests.conf'
     DEFAULT_CONFIG_SECT = config.KEY_DEFAULT
-    RUN_ALL_TESTS = []
     DO_APT_UPDATE = True
     DONT_APT_UPDATE = False
     NETWORK_ALREADY_SETUP = True
@@ -334,201 +96,254 @@
     DONT_DIST_UPGRADE = False
     DO_INSTALL_TESTS = True
     DONT_INSTALL_TESTS = False
-    VISUALIZATION_ON = True
-    VISUALIZATION_OFF = False
-    DO_USE_SILENT = True
-    DONT_USE_SILENT = False
     DO_USE_CACHE_DEPS = True
     DONT_USE_CACHE_DEPS = False
     DO_USE_VERBOSE_LOGGING = True
     DONT_USE_VERBOSE_LOGGING = False
 
     scenarios = [
-        ('no arguments', {
+        ('default setup arguments', {
             'command_line_arguments': [],
             'parsed_arguments': (
+                SETUP_CMD,
                 DEFAULT_CONFIG_FILE,
                 DEFAULT_CONFIG_SECT,
-                RUN_ALL_TESTS,
-                DONT_APT_UPDATE,
                 NETWORK_NOT_ALREADY_SETUP,
-                VISUALIZATION_OFF,
-                DONT_DIST_UPGRADE,
+                DONT_USE_VERBOSE_LOGGING,
+                DO_USE_CACHE_DEPS,
+                DO_DIST_UPGRADE,
                 DONT_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
+                DO_APT_UPDATE,
             )
         }),
         ('alternative config', {
             'command_line_arguments': ['--config=alternative.conf'],
             'parsed_arguments': (
+                SETUP_CMD,
                 'alternative.conf',
                 DEFAULT_CONFIG_SECT,
-                RUN_ALL_TESTS,
-                DONT_APT_UPDATE,
                 NETWORK_NOT_ALREADY_SETUP,
-                VISUALIZATION_OFF,
-                DONT_DIST_UPGRADE,
+                DONT_USE_VERBOSE_LOGGING,
+                DO_USE_CACHE_DEPS,
+                DO_DIST_UPGRADE,
                 DONT_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
+                DO_APT_UPDATE,
             )
         }),
         ('alternative config section', {
             'command_line_arguments': ['--config-section=custom-section'],
             'parsed_arguments': (
+                SETUP_CMD,
                 DEFAULT_CONFIG_FILE,
                 'custom-section',
-                RUN_ALL_TESTS,
-                DONT_APT_UPDATE,
                 NETWORK_NOT_ALREADY_SETUP,
-                VISUALIZATION_OFF,
-                DONT_DIST_UPGRADE,
+                DONT_USE_VERBOSE_LOGGING,
+                DO_USE_CACHE_DEPS,
+                DO_DIST_UPGRADE,
                 DONT_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
+                DO_APT_UPDATE,
             )
         }),
-        ('with apt-get update', {
+        ('only apt-get update', {
             'command_line_arguments': ['--update-apt'],
             'parsed_arguments': (
+                SETUP_CMD,
                 DEFAULT_CONFIG_FILE,
                 DEFAULT_CONFIG_SECT,
-                RUN_ALL_TESTS,
+                NETWORK_NOT_ALREADY_SETUP,
+                DONT_USE_VERBOSE_LOGGING,
+                DONT_USE_CACHE_DEPS,
+                DONT_DIST_UPGRADE,
+                DONT_INSTALL_TESTS,
                 DO_APT_UPDATE,
-                NETWORK_NOT_ALREADY_SETUP,
-                VISUALIZATION_OFF,
-                DONT_DIST_UPGRADE,
-                DONT_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
-            )
-        }),
-        ('one test suite', {
-            'command_line_arguments': ['test_suite'],
-            'parsed_arguments': (
-                DEFAULT_CONFIG_FILE,
-                DEFAULT_CONFIG_SECT,
-                ['test_suite'],
-                DONT_APT_UPDATE,
-                NETWORK_NOT_ALREADY_SETUP,
-                VISUALIZATION_OFF,
-                DONT_DIST_UPGRADE,
-                DONT_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
-            )
-        }),
-        ('multiple test suites', {
-            'command_line_arguments': [
-                'test_suite1', 'test_suite2', 'test_suite3'
-            ],
-            'parsed_arguments': (
-                DEFAULT_CONFIG_FILE,
-                DEFAULT_CONFIG_SECT,
-                ['test_suite1', 'test_suite2', 'test_suite3'],
-                DONT_APT_UPDATE, NETWORK_NOT_ALREADY_SETUP, VISUALIZATION_OFF,
-                DONT_DIST_UPGRADE,
-                DONT_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
             )
         }),
         ('with network already setup', {
             'command_line_arguments': ['--network-ready'],
             'parsed_arguments': (
-                DEFAULT_CONFIG_FILE, DEFAULT_CONFIG_SECT, [], DONT_APT_UPDATE,
+                SETUP_CMD,
+                DEFAULT_CONFIG_FILE,
+                DEFAULT_CONFIG_SECT,
                 NETWORK_ALREADY_SETUP,
-                VISUALIZATION_OFF,
-                DONT_DIST_UPGRADE,
-                DONT_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
-            )
-        }),
-        ('with touch visualisation enabled', {
-            'command_line_arguments': ['--touch-visualization'],
-            'parsed_arguments': (
-                DEFAULT_CONFIG_FILE, DEFAULT_CONFIG_SECT, [], DONT_APT_UPDATE,
-                NETWORK_NOT_ALREADY_SETUP,
-                VISUALIZATION_ON,
-                DONT_DIST_UPGRADE,
-                DONT_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
-            )
-        }),
-        ('with install dependencies enabled', {
+                DONT_USE_VERBOSE_LOGGING,
+                DO_USE_CACHE_DEPS,
+                DO_DIST_UPGRADE,
+                DONT_INSTALL_TESTS,
+                DO_APT_UPDATE,
+            )
+        }),
+        ('with dist-upgrade enabled', {
             'command_line_arguments': ['--dist-upgrade'],
             'parsed_arguments': (
-                DEFAULT_CONFIG_FILE, DEFAULT_CONFIG_SECT, [], DONT_APT_UPDATE,
+                SETUP_CMD,
+                DEFAULT_CONFIG_FILE,
+                DEFAULT_CONFIG_SECT,
                 NETWORK_NOT_ALREADY_SETUP,
-                VISUALIZATION_OFF,
+                DONT_USE_VERBOSE_LOGGING,
+                DONT_USE_CACHE_DEPS,
                 DO_DIST_UPGRADE,
                 DONT_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
+                DO_APT_UPDATE,
             )
         }),
         ('with install dependencies enabled', {
             'command_line_arguments': ['--install-tests'],
             'parsed_arguments': (
-                DEFAULT_CONFIG_FILE, DEFAULT_CONFIG_SECT, [], DONT_APT_UPDATE,
+                SETUP_CMD,
+                DEFAULT_CONFIG_FILE,
+                DEFAULT_CONFIG_SECT,
                 NETWORK_NOT_ALREADY_SETUP,
-                VISUALIZATION_OFF,
+                DONT_USE_VERBOSE_LOGGING,
+                DONT_USE_CACHE_DEPS,
                 DONT_DIST_UPGRADE,
                 DO_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
-            )
-        }),
-        ('with silent mode specified', {
-            'command_line_arguments': ['--silent'],
-            'parsed_arguments': (
-                DEFAULT_CONFIG_FILE, DEFAULT_CONFIG_SECT, [], DONT_APT_UPDATE,
-                NETWORK_NOT_ALREADY_SETUP,
-                VISUALIZATION_OFF,
-                DONT_DIST_UPGRADE,
-                DONT_INSTALL_TESTS,
-                DO_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
+                DO_APT_UPDATE,
             )
         }),
         ('with cache deps specified', {
             'command_line_arguments': ['--cache-deps'],
             'parsed_arguments': (
-                DEFAULT_CONFIG_FILE, DEFAULT_CONFIG_SECT, [], DONT_APT_UPDATE,
-                NETWORK_NOT_ALREADY_SETUP,
-                VISUALIZATION_OFF,
-                DONT_DIST_UPGRADE,
-                DONT_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DO_USE_CACHE_DEPS,
-                DONT_USE_VERBOSE_LOGGING,
-            )
-        }),
-        ('with verbose logging specified', {
-            'command_line_arguments': ['--verbose'],
-            'parsed_arguments': (
-                DEFAULT_CONFIG_FILE, DEFAULT_CONFIG_SECT, [], DONT_APT_UPDATE,
-                NETWORK_NOT_ALREADY_SETUP,
-                VISUALIZATION_OFF,
-                DONT_DIST_UPGRADE,
-                DONT_INSTALL_TESTS,
-                DONT_USE_SILENT,
-                DONT_USE_CACHE_DEPS,
-                DO_USE_VERBOSE_LOGGING,
+                SETUP_CMD,
+                DEFAULT_CONFIG_FILE,
+                DEFAULT_CONFIG_SECT,
+                NETWORK_NOT_ALREADY_SETUP,
+                DONT_USE_VERBOSE_LOGGING,
+                DO_USE_CACHE_DEPS,
+                DONT_DIST_UPGRADE,
+                DONT_INSTALL_TESTS,
+                DO_APT_UPDATE,
+            )
+        }),
+        ('with verbose logging specified', {
+            'command_line_arguments': ['--verbose'],
+            'parsed_arguments': (
+                SETUP_CMD,
+                DEFAULT_CONFIG_FILE,
+                DEFAULT_CONFIG_SECT,
+                NETWORK_NOT_ALREADY_SETUP,
+                DO_USE_VERBOSE_LOGGING,
+                DO_USE_CACHE_DEPS,
+                DO_DIST_UPGRADE,
+                DONT_INSTALL_TESTS,
+                DO_APT_UPDATE,
+            )
+        }),
+    ]
+
+    @mock.patch('ubuntu_system_tests.host.command_line.run_autopkg_setup')
+    def test_main_must_pass_command_line_arguments_to_run_system_tests(
+            self, mock_run_autopkg_setup):
+        command_line.main([self.SETUP_CMD] + self.command_line_arguments)
+        params = get_cmdline_setup_params(*self.parsed_arguments)
+        mock_run_autopkg_setup.assert_called_once_with(params)
+
+
+class RunCommandLineArgumentsTestCase(testscenarios.TestWithScenarios):
+
+    RUN_CMD = 'run'
+    DEFAULT_CONFIG_FILE = 'ubuntu-system-tests.conf'
+    DEFAULT_CONFIG_SECT = config.KEY_DEFAULT
+    ALL_SUITES = []
+    NETWORK_ALREADY_SETUP = True
+    NETWORK_NOT_ALREADY_SETUP = False
+    DO_USE_VERBOSE_LOGGING = True
+    DONT_USE_VERBOSE_LOGGING = False
+    DO_USE_TOUCH_VISUALIZATION = True
+    DONT_USE_TOUCH_VISUALIZATION = False
+    DO_USE_SILENT_MODE = True
+    DONT_USE_SILENT_MODE = False
+
+    scenarios = [
+        ('default run arguments', {
+            'command_line_arguments': [],
+            'parsed_arguments': (
+                RUN_CMD,
+                DEFAULT_CONFIG_FILE,
+                DEFAULT_CONFIG_SECT,
+                NETWORK_NOT_ALREADY_SETUP,
+                DONT_USE_VERBOSE_LOGGING,
+                DONT_USE_SILENT_MODE,
+                ALL_SUITES,
+                DONT_USE_TOUCH_VISUALIZATION,
+            )
+        }),
+        ('alternative config', {
+            'command_line_arguments': ['--config=alternative.conf'],
+            'parsed_arguments': (
+                RUN_CMD,
+                'alternative.conf',
+                DEFAULT_CONFIG_SECT,
+                NETWORK_NOT_ALREADY_SETUP,
+                DONT_USE_VERBOSE_LOGGING,
+                DONT_USE_SILENT_MODE,
+                ALL_SUITES,
+                DONT_USE_TOUCH_VISUALIZATION,
+            )
+        }),
+        ('alternative config section', {
+            'command_line_arguments': ['--config-section=custom-section'],
+            'parsed_arguments': (
+                RUN_CMD,
+                DEFAULT_CONFIG_FILE,
+                'custom-section',
+                NETWORK_NOT_ALREADY_SETUP,
+                DONT_USE_VERBOSE_LOGGING,
+                DONT_USE_SILENT_MODE,
+                ALL_SUITES,
+                DONT_USE_TOUCH_VISUALIZATION,
+            )
+        }),
+        ('with network already setup', {
+            'command_line_arguments': ['--network-ready'],
+            'parsed_arguments': (
+                RUN_CMD,
+                DEFAULT_CONFIG_FILE,
+                DEFAULT_CONFIG_SECT,
+                NETWORK_ALREADY_SETUP,
+                DONT_USE_VERBOSE_LOGGING,
+                DONT_USE_SILENT_MODE,
+                ALL_SUITES,
+                DONT_USE_TOUCH_VISUALIZATION,
+            )
+        }),
+        ('with verbose logging specified', {
+            'command_line_arguments': ['--verbose'],
+            'parsed_arguments': (
+                RUN_CMD,
+                DEFAULT_CONFIG_FILE,
+                DEFAULT_CONFIG_SECT,
+                NETWORK_NOT_ALREADY_SETUP,
+                DO_USE_VERBOSE_LOGGING,
+                DONT_USE_SILENT_MODE,
+                ALL_SUITES,
+                DONT_USE_TOUCH_VISUALIZATION,
+            )
+        }),
+        ('single test', {
+            'command_line_arguments': ['suite.tests.test_1'],
+            'parsed_arguments': (
+                RUN_CMD,
+                DEFAULT_CONFIG_FILE,
+                DEFAULT_CONFIG_SECT,
+                NETWORK_NOT_ALREADY_SETUP,
+                DONT_USE_VERBOSE_LOGGING,
+                DONT_USE_SILENT_MODE,
+                ['suite.tests.test_1'],
+                DONT_USE_TOUCH_VISUALIZATION,
+            )
+        }),
+        ('mutliple tests', {
+            'command_line_arguments': [
+                'suite.tests.test_1', 'suite.tests.test_2'],
+            'parsed_arguments': (
+                RUN_CMD,
+                DEFAULT_CONFIG_FILE,
+                DEFAULT_CONFIG_SECT,
+                NETWORK_NOT_ALREADY_SETUP,
+                DONT_USE_VERBOSE_LOGGING,
+                DONT_USE_SILENT_MODE,
+                ['suite.tests.test_1', 'suite.tests.test_2'],
+                DONT_USE_TOUCH_VISUALIZATION,
             )
         }),
     ]
@@ -536,7 +351,6 @@
     @mock.patch('ubuntu_system_tests.host.command_line.run_system_tests')
     def test_main_must_pass_command_line_arguments_to_run_system_tests(
             self, mock_run_system_tests):
-        command_line.main(self.command_line_arguments)
-
-        mock_run_system_tests.assert_called_once_with(
-            *self.parsed_arguments)
+        command_line.main([self.RUN_CMD] + self.command_line_arguments)
+        params = get_cmdline_run_params(*self.parsed_arguments)
+        mock_run_system_tests.assert_called_once_with(params)

=== added file 'ubuntu_system_tests/selftests/test_commands.py'
--- ubuntu_system_tests/selftests/test_commands.py	1970-01-01 00:00:00 +0000
+++ ubuntu_system_tests/selftests/test_commands.py	2016-06-29 08:06:07 +0000
@@ -0,0 +1,239 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu System Tests
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+from unittest import mock
+
+from ubuntu_system_tests import selftests
+from ubuntu_system_tests.host import commands
+from ubuntu_system_tests.common import config
+
+
+def check_any_item_in_list_contains_value(list, value):
+    return any(item.find(value) >= 0 for item in list)
+
+
+def check_any_item_in_list_startswith_value(list, value):
+    return any(item.startswith(value) for item in list)
+
+
+class RunCommandTestCase(selftests.ConfigBaseTestCase):
+
+    def setUp(self):
+        super().setUp()
+        self.config_stack = config.UbuntuSystemTestsConfig()
+
+    def test_setup_network_command_contains_credentials(self):
+        self.set_config_values()
+
+        command = commands._get_device_network_command(
+            'test_ssid', 'test_password')
+
+        self.assertTrue(command.startswith(
+            'nmcli device wifi connect \'test_ssid\' '
+            'password \'test_password\''))
+
+    def test_enable_introspection_with_serial(self):
+        serial = 'test_serial'
+        self.set_config_values(device_serial=serial)
+
+        command = commands.build_enable_autopilot_introspection_command(
+            self.config_stack)
+
+        self.assertEqual(['phablet-config', '-s', 'test_serial'], command[:3])
+
+    def test_enable_introspection_without_serial(self):
+        self.set_config_values(device_serial='')
+
+        command = commands.build_enable_autopilot_introspection_command(
+            self.config_stack)
+
+        self.assertNotIn('-s', command)
+
+    def test_adt_run_must_be_called_with_short_timeout(self):
+        self.set_config_values()
+
+        command = commands.build_adt_run_test_command(
+            self.config_stack, config.KEY_DEFAULT)
+        command = ' '.join(command)
+
+        run_arguments, _ = command.split('---')
+        self.assertIn('--timeout-short=600', run_arguments)
+
+    def test_adt_run_command_no_file_mount(self):
+        self.set_config_values()
+        run_cmd = ' '.join(commands.build_adt_run_test_command(
+            self.config_stack, config.KEY_DEFAULT))
+        self.assertNotIn('mount -o remount,ro /', run_cmd)
+        self.assertNotIn('mount -o remount,rw /', run_cmd)
+
+    def test_adt_run_with_serial(self):
+        serial = 'test_serial'
+        self.set_config_values(device_serial=serial)
+
+        command = commands.build_adt_run_test_command(
+            self.config_stack, config.KEY_DEFAULT)
+        command = ' '.join(command)
+
+        _, adb_arguments = command.split('ssh -s adb --')
+        self.assertIn('-s test_serial', adb_arguments)
+
+    def test_adt_run_without_serial(self):
+        self.set_config_values(device_serial='')
+
+        command = commands.build_adt_run_test_command(
+            self.config_stack, config.KEY_DEFAULT)
+        command = ' '.join(command)
+
+        _, adb_arguments = command.split('ssh -s adb --')
+        self.assertNotIn('-s test_serial', adb_arguments)
+
+    def test_adt_run_command_with_network_ready_creates_no_network(self):
+        self.set_config_values()
+
+        command = commands.build_adt_run_test_command(
+            self.config_stack, config.KEY_DEFAULT, network_ready=True)
+
+        self.assertFalse(check_any_item_in_list_startswith_value(
+            command, 'nmcli device wifi connect'))
+
+    def test_adt_run_command_with_default_parameters_creates_network(self):
+        self.set_config_values()
+
+        command = commands.build_adt_run_test_command(
+            self.config_stack, config.KEY_DEFAULT)
+
+        self.assertTrue(check_any_item_in_list_startswith_value(
+            command, 'nmcli device wifi connect'))
+
+    def test_enabling_verbose_option(self):
+        self.set_config_values()
+        command = commands.build_adt_run_test_command(
+            self.config_stack, config.KEY_DEFAULT, verbose=True)
+        self.assertTrue(check_any_item_in_list_contains_value(command, '-ddd'))
+
+    def test_verbose_disabled_by_default(self):
+        self.set_config_values()
+        command = commands.build_adt_run_test_command(
+            self.config_stack, config.KEY_DEFAULT)
+        self.assertFalse(
+            check_any_item_in_list_contains_value(command, '-ddd'))
+
+
+class SetupCommandTestCase(selftests.ConfigBaseTestCase):
+
+    def setUp(self):
+        super().setUp()
+        self.config_stack = config.UbuntuSystemTestsConfig()
+
+    def test_ppa_setup_commands_set_image_readwrite(self):
+        self.set_config_values()
+        self.assertIn(
+            'mount -o remount,rw /',
+            ' '.join(commands.build_adt_run_setup_command(
+                self.config_stack, config.KEY_DEFAULT))
+        )
+
+    def test_ppa_setup_commands_used_in_adt_setup(self):
+        self.set_config_values()
+        command = commands.build_adt_run_setup_command(
+            self.config_stack, config.KEY_DEFAULT)
+        command = ' '.join(command)
+
+        run_arguments, _ = command.split('---')
+        self.assertIn(
+            'apt-add-repository -y ppa:canonical-platform-qa',
+            run_arguments
+        )
+
+    def test_setup_commands_install_testability_libs(self):
+        self.set_config_values()
+        command = commands.build_adt_run_setup_command(
+            self.config_stack, config.KEY_DEFAULT)
+        self.assertTrue(
+            check_any_item_in_list_contains_value(
+                command,
+                'apt-get -y --no-install-recommends install '
+                'qttestability-autopilot')
+        )
+
+    def test_setting_fs_rw_is_conditional_on_current_state(self):
+        self.assertIn(
+            commands._get_is_fs_ro_command(), commands._get_fs_rw_command())
+
+    def test_setting_fs_ro_is_conditional_on_current_state(self):
+        command = commands._get_fs_ro_setup_command()
+        check_any_item_in_list_contains_value(
+            command, commands._get_is_fs_rw_command())
+
+    def test_enabling_verbose_option(self):
+        self.set_config_values()
+        command = commands.build_adt_run_setup_command(
+            self.config_stack, config.KEY_DEFAULT, verbose=True)
+        self.assertTrue(check_any_item_in_list_contains_value(command, '-ddd'))
+
+    def test_verbose_disabled_by_default(self):
+        self.set_config_values()
+        command = commands.build_adt_run_setup_command(
+            self.config_stack, config.KEY_DEFAULT)
+        self.assertFalse(
+            check_any_item_in_list_contains_value(command, '-ddd'))
+
+    def test_cache_deps_uses_default_control_file(self):
+        self.set_config_values()
+        command = commands.build_adt_run_setup_command(
+            self.config_stack, config.KEY_DEFAULT, cache_deps=True)
+        self.assertFalse(
+            check_any_item_in_list_contains_value(
+                command, '--override-control'))
+
+    def test_no_cache_deps_overrides_control_file(self):
+        self.set_config_values()
+        command = commands.build_adt_run_setup_command(
+            self.config_stack, config.KEY_DEFAULT)
+        self.assertTrue(
+            check_any_item_in_list_contains_value(
+                command, '--override-control'))
+
+    def test_cache_deps_installs_apt_cacher(self):
+        self.set_config_values()
+        command = commands.build_adt_run_setup_command(
+            self.config_stack, config.KEY_DEFAULT, cache_deps=True)
+        self.assertTrue(
+            check_any_item_in_list_contains_value(
+                command,
+                'apt-get -y --no-install-recommends install apt-cacher-ng'))
+
+    def test_default_no_apt_cacher_install(self):
+        self.set_config_values()
+        command = commands.build_adt_run_setup_command(
+            self.config_stack, config.KEY_DEFAULT)
+        self.assertFalse(
+            check_any_item_in_list_contains_value(command, 'apt-cacher-ng'))
+
+    @mock.patch('ubuntu_system_tests.host.testdesc.get_all_dependencies')
+    def test_install_tests_adds_extra_dependencies(
+            self, mock_get_all_dependencies):
+        self.set_config_values()
+        mock_get_all_dependencies.return_value = ['test-dep-1', 'test-dep-2']
+        command = commands.build_adt_run_setup_command(
+            self.config_stack, config.KEY_DEFAULT, install_tests=True)
+        self.assertTrue(
+            check_any_item_in_list_contains_value(
+                command, 'install test-dep-1 test-dep-2'))

=== added file 'ubuntu_system_tests/selftests/test_debian.py'
--- ubuntu_system_tests/selftests/test_debian.py	1970-01-01 00:00:00 +0000
+++ ubuntu_system_tests/selftests/test_debian.py	2016-06-29 08:06:07 +0000
@@ -0,0 +1,125 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu System Tests
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import fixtures
+import json
+import os
+from tempfile import NamedTemporaryFile
+from unittest import mock
+
+from ubuntu_system_tests import selftests
+from ubuntu_system_tests.host import debian
+from ubuntu_system_tests.common import config
+
+
+def check_any_item_in_list_contains_value(list, value):
+    return any(item.find(value) for item in list)
+
+
+def check_any_item_in_list_startswith_value(list, value):
+    return any(item.startswith(value) for item in list)
+
+
+class CommandTestCase(selftests.ConfigBaseTestCase):
+
+    def setUp(self):
+        super().setUp()
+        self.config_stack = config.UbuntuSystemTestsConfig()
+
+    def test_get_extra_dependencies(self):
+        tests = ('app1.tests.test_module.TestClass.test_x '
+                 'app2.tests.test_module.TestClass.test_y '
+                 'app3.tests.test_module.TestClass.test_z')
+        self.set_config_values(tests_to_run=tests)
+        expected_depends = ['dep1',
+                            'dep2']
+        # Mock _read_dependency_files
+        with mock.patch(
+            'ubuntu_system_tests.host.debian._read_dependency_files'
+        ) as mock_read_depends:
+            mock_read_depends.return_value = {
+                'app2.tests': 'dep1',
+                'app3.tests': 'dep2'
+            }
+            extra_depends = debian.get_extra_dependencies(
+                self.config_stack.get('tests_to_run').split()
+            )
+        self.assertCountEqual(extra_depends, expected_depends)
+
+    def test_read_dependency_file(self):
+        with fixtures.TempDir() as deps_dir:
+            deps_file1 = open(os.path.join(deps_dir.path, 'deps1.json'), 'w')
+            deps_file2 = open(os.path.join(deps_dir.path, 'deps2.json'), 'w')
+            deps_json1 = {
+                "test_dependencies": [
+                    {"id": "app1.tests", "depends": "dep1"}
+                ]
+            }
+            deps_json2 = {
+                "test_dependencies": [
+                    {"id": "app2.tests", "depends": "dep2"}
+                ]
+            }
+            deps_file1.write(json.dumps(deps_json1))
+            deps_file2.write(json.dumps(deps_json2))
+            deps_file1.close()
+            deps_file2.close()
+
+            expected = {'app1.tests': 'dep1', 'app2.tests': 'dep2'}
+            actual = debian._read_dependency_files(deps_dir.path)
+            self.assertEqual(expected, actual)
+
+    def test_extra_dependencies_for_test(self):
+        test = 'app1.tests.test_module.TestClass.test_y'
+        dependencies_map = {'app1.tests': 'dep1', 'app2.tests': 'dep2'}
+        extra_depends = debian._extra_dependencies_for_test(
+            test, dependencies_map
+        )
+        self.assertEqual(extra_depends, ['dep1'])
+
+    def test_create_temp_control(self):
+        with NamedTemporaryFile() as control:
+            control.write("""Tests: systemtests
+Depends: dep1,
+         dep2,
+         dep3,
+
+Tests: someothertests
+Depends: other1,
+         other2,
+         other3,""".encode('utf-8'))
+            control.flush()
+            temp_control = debian.create_temp_control_file(
+                ['dep4', 'dep5'], control.name
+            )
+            with open(temp_control) as temp:
+                tmp_control_content = temp.read()
+                expected_content = """Tests: systemtests
+Depends: dep1,
+         dep2,
+         dep3,
+         dep4,
+         dep5,
+
+Tests: someothertests
+Depends: other1,
+         other2,
+         other3,"""
+                self.assertEqual(tmp_control_content, expected_content)

=== modified file 'ubuntu_system_tests/selftests/utils.py'
--- ubuntu_system_tests/selftests/utils.py	2016-06-02 13:08:41 +0000
+++ ubuntu_system_tests/selftests/utils.py	2016-06-29 08:06:07 +0000
@@ -22,6 +22,7 @@
 import shutil
 import tempfile
 import unittest
+from argparse import Namespace
 from unittest import mock
 
 from ubuntu_system_tests.common import config
@@ -60,6 +61,28 @@
         self.config_stack.save()
 
 
+def get_cmdline_run_params(
+        command='run', config=config.DEFAULT_CONF,
+        config_section=config.KEY_DEFAULT, network_ready=False, verbose=False,
+        silent=False, suites=[], touch_visualization=False):
+    return Namespace(
+        command=command, config=config, config_section=config_section,
+        network_ready=network_ready, verbose=verbose, silent=silent,
+        suites=suites, touch_visualization=touch_visualization)
+
+
+def get_cmdline_setup_params(
+        command='setup', config=config.DEFAULT_CONF,
+        config_section=config.KEY_DEFAULT, network_ready=False, verbose=False,
+        cache_deps=False, dist_upgrade=False, install_tests=False,
+        update_apt=False):
+    return Namespace(
+        command=command, config=config, config_section=config_section,
+        network_ready=network_ready, verbose=verbose, cache_deps=cache_deps,
+        dist_upgrade=dist_upgrade, install_tests=install_tests,
+        update_apt=update_apt)
+
+
 def set_uniq_cwd(test, prefix=None):
     """Create a temporary directory an cd into it for the test duration.