Merge lp:~heber013/ubuntu-system-tests/add-install-package-option into lp:ubuntu-system-tests

[Heber Parrucci]

Adding option to install packages as part of the setup

Adding option to setup in order to install one or more packages in the device as part of the setup process:

  $ ubuntu-system-tests setup --install-packages package1,package2

You can also install packages from proposed archive:

  $ ubuntu-system-tests setup --install-packages package1/*release*-proposed

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:582
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1332/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2465/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1332/rebuild

review: Needs Fixing (continuous-integration)

[lp:~heber013/ubuntu-system-tests/add-install-package-option]

583. By Heber Parrucci on 2018-02-16

Fixing flake8 issues

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:583
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1333/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2466/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1333/rebuild

review: Needs Fixing (continuous-integration)

[lp:~heber013/ubuntu-system-tests/add-install-package-option]

584. By Heber Parrucci on 2018-02-16

fixing selftest

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:584
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1334/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2468/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1334/rebuild

review: Approve (continuous-integration)

[lp:~heber013/ubuntu-system-tests/add-install-package-option]

585. By Heber Parrucci on 2018-03-23

Add file in sources.list.d for enabling proposed

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:585
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1335/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2472/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci/1335/rebuild

review: Approve (continuous-integration)

Unmerged revisions

585. By Heber Parrucci on 2018-03-23

Add file in sources.list.d for enabling proposed

584. By Heber Parrucci on 2018-02-16

fixing selftest

583. By Heber Parrucci on 2018-02-16

Fixing flake8 issues

582. By Heber Parrucci on 2018-02-15

Adding option to install packages as part of the setup

=== modified file 'README.rst'
--- README.rst	2017-06-27 20:33:54 +0000
+++ README.rst	2018-03-23 12:23:17 +0000
@@ -596,6 +596,21 @@
   $ ubuntu-system-tests setup --install-tests package1 package2.module2
 
 
+Install packages in the device
+------------------------------
+It is possible to install one or more packages in the device as part of the setup process.
+It can be used by using *--install-packages* from command line:
+
+::
+
+  $ ubuntu-system-tests setup --install-packages package1,package2
+
+You can also install packages from proposed archive:
+::
+
+    $ ubuntu-system-tests setup --install-packages package1/*release*-proposed
+
+
 Upgrade device using dist-upgrade
 ---------------------------------
 

=== modified file 'ubuntu_system_tests/host/command_line.py'
--- ubuntu_system_tests/host/command_line.py	2017-08-16 18:53:17 +0000
+++ ubuntu_system_tests/host/command_line.py	2018-03-23 12:23:17 +0000
@@ -110,6 +110,9 @@
      'of {a} are also supported. The list of tests returned from Practitest '
      'will be used in addition to any tests specified in the '
      'suites parameter.'.format(a=', '.join(FILTER_ALIAS.keys()))})
+OPTION_INSTALL_PACKAGES = Option(
+    ['--install-packages'],
+    {'help': 'Comma separated list of packages to install as part of the setup'})
 OPTION_QEMU_HEADLESS = Option(
     ['--headless', '-l'],
     {'action': 'store_true',
@@ -216,6 +219,7 @@
     OPTION_CACHE_DEPS,
     OPTION_DIST_UPGRADE,
     OPTION_INSTALL_TESTS,
+    OPTION_INSTALL_PACKAGES,
     OPTION_UPDATE_APT,
     OPTION_PPA_LIST,
     OPTION_SETUP_SUITES,
@@ -685,7 +689,7 @@
         status = target.run_setup().status
         if status and count < 1:
             # Error was reported after first setup attempt,
-            # so reboot device and try again.
+            # so reboot device and try again
             print(
                 'Rebooting device after setup completed with '
                 'status {s}.'.format(s=status))
@@ -698,11 +702,21 @@
     """Run post setup actions on the target and return status result."""
     if status:
         print('Setup failed with status: {}.'.format(status))
-    elif args.cache_deps:
-        # Download dependencies using autopkgtest
-        adt_run_cmd = cmds.build_adt_run_test_command(
-            config_stack, args, target)
-        status = subprocess.call(adt_run_cmd)
+    else:
+        if args.install_packages:
+            target.connect()
+            target.push_setup_script(script='enable_proposed.py')
+            command = '/usr/bin/python3 /tmp/enable_proposed.py'
+            target.run_sudo(command)
+            target.run_sudo('apt update')
+            packages = args.install_packages.split(',')
+            for package in packages:
+                target.run_sudo('apt install %s -y' % package)
+        if args.cache_deps:
+            # Download dependencies using autopkgtest
+            adt_run_cmd = cmds.build_adt_run_test_command(
+                config_stack, args, target)
+            status = subprocess.call(adt_run_cmd)
     return status
 
 

=== added file 'ubuntu_system_tests/host/enable_proposed.py'
--- ubuntu_system_tests/host/enable_proposed.py	1970-01-01 00:00:00 +0000
+++ ubuntu_system_tests/host/enable_proposed.py	2018-03-23 12:23:17 +0000
@@ -0,0 +1,58 @@
+#!/usr/bin/env python3
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu System Tests
+# Copyright (C) 2018 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+"""This script is copied to target and executed to perform enable proposed archive."""
+
+import subprocess
+
+import logging
+
+import os
+
+logger = logging.getLogger(__name__)
+logger.setLevel(logging.INFO)
+
+
+def enable_proposed():
+    release_code_name = get_release_code_name()
+    proposed = 'deb http://archive.ubuntu.com/ubuntu/ ' \
+               '{}-proposed restricted main multiverse universe\n'.format(release_code_name)
+    if not os.path.exists('/etc/apt/sources.list.d/proposed.list'):
+        with open('/etc/apt/sources.list.d/proposed.list', 'w') as _f:
+            _f.write(proposed)
+    with open('/etc/apt/preferences.d/proposed-updates', 'w') as _f:
+        _f.write('Package: *\n')
+        _f.write('Pin: release a={}-proposed\n'.format(release_code_name))
+        _f.write('Pin-Priority: 400\n')
+
+
+def get_release_code_name():
+    try:
+        return subprocess.check_output(['lsb_release',
+                                        '-c'],
+                                       universal_newlines=True).strip().split('\t')[1]
+    except subprocess.CalledProcessError:
+        logger.error('Cannot get current release in target')
+        raise
+
+
+if __name__ == '__main__':
+    enable_proposed()

=== modified file 'ubuntu_system_tests/host/targets.py'
--- ubuntu_system_tests/host/targets.py	2017-11-29 17:55:40 +0000
+++ ubuntu_system_tests/host/targets.py	2018-03-23 12:23:17 +0000
@@ -160,10 +160,10 @@
             result.status = 1
         return result
 
-    def _push_setup_script(self):
+    def push_setup_script(self, script='target_setup.py'):
         src = os.path.join(
-            os.path.dirname(os.path.realpath(__file__)), 'target_setup.py')
-        dst = os.path.join('/tmp/target_setup.py')
+            os.path.dirname(os.path.realpath(__file__)), script)
+        dst = os.path.join('/tmp/{}'.format(script))
         self.put(src, dst)
         if self.run_sudo('chmod 744 {}'.format(dst)).status:
             raise RuntimeError(
@@ -178,7 +178,7 @@
             self.put(src, '/tmp/target_config.json')
 
     def _run_target_setup(self):
-        self._push_setup_script()
+        self.push_setup_script()
         self._push_setup_config()
         command = '/usr/bin/python3 /tmp/target_setup.py --config_path /tmp/target_config.json ' \
                   '--username %s' % self.config_stack.get('device_username')

=== modified file 'ubuntu_system_tests/selftests/utils.py'
--- ubuntu_system_tests/selftests/utils.py	2017-03-06 16:56:01 +0000
+++ ubuntu_system_tests/selftests/utils.py	2018-03-23 12:23:17 +0000
@@ -109,14 +109,14 @@
         config_section=config.KEY_DEFAULT, network_ready=False, verbose=False,
         cache_deps=False, dist_upgrade=False, install_tests=False,
         update_apt=False, silent=False, headless=False, img=None,
-        suites=None, filters=None, ppa=None, ip=None, port=None):
+        suites=None, filters=None, ppa=None, ip=None, port=None, install_packages=None):
     suites = suites or []
     return Namespace(
         command=command, config=config, config_section=config_section,
         network_ready=network_ready, verbose=verbose, cache_deps=cache_deps,
         dist_upgrade=dist_upgrade, install_tests=install_tests,
         update_apt=update_apt, silent=silent, suites=suites, filters=filters,
-        ppa=ppa, headless=headless, img=img, ip=ip, port=port)
+        ppa=ppa, headless=headless, img=img, ip=ip, port=port, install_packages=install_packages)
 
 
 def set_uniq_cwd(test, prefix=None):