Merge lp:~doanac/utah/process_runlist_cleanup into lp:utah

[Andy Doan]

This finishes the cleanup of the overly complex process_master_runlist function

[lp:~doanac/utah/process_runlist_cleanup]

927. By Andy Doan on 2013-06-04

fix test-case regression from revno 915.4.8

provisioning.py was fixed, but I failed to fix the test case

928. By Andy Doan on 2013-06-04

fix some broken test cases

These all broke due to recent API changes

929. By Max Brustkern on 2013-06-05

Merged new ssh fixes

[Javier Collado (javier.collado)]

Please could you rebase to fix the merge conflict? The diff here doesn't seem to be properly displayed. For example,
it looks like `process_master_runlist` docstring includes parameters, but looking at the branch I don't see that.

[lp:~doanac/utah/process_runlist_cleanup]

930. By Max Brustkern on 2013-06-06

Merged DefaultValidator removal to fix the build

931. By Andy Doan on 2013-06-06

fix #1185584 retry for bzr failure

932. By Andy Doan on 2013-06-07

more simplifications to the process_master_runlist function

Move logic of getting and validating runlist into its own function.

While fixing I noticed that ContentTooShortError is a subclass of
IOError, so the exception logic for that was un-reachable. This makes
one exception case and tries to include useful error information.

Also while fixing, I noticed the "suites" logic is no longer needed,
because we only support one type of runlist in the schema we validate
against.

This takes us from of mccabe complexity of 16 to 10.

933. By Andy Doan on 2013-06-07

break out more of process_master_runlist

This moves the suite creation logic into its own function

complexity now down to an 8

934. By Andy Doan on 2013-06-07

simplify testsuite constructor: break out "add_test" logic

Unmerged revisions

=== added file 'templates/utah-autorun.sh.jinja2'
--- templates/utah-autorun.sh.jinja2	1970-01-01 00:00:00 +0000
+++ templates/utah-autorun.sh.jinja2	2013-06-07 18:58:24 +0000
@@ -0,0 +1,41 @@
+#!/bin/sh
+
+set -e
+
+# looks under /etc/utah/autorun for scripts to execute at startup
+# as scripts are executed they'll be moved to
+# /var/cache/utah/autorun/inprogress and then to
+# /var/cache/utah/autorun/complete/.
+
+WORKDIR="${WORKDIR-/etc/utah/autorun}"
+RESULTDIR="${RESULTDIR-/var/cache/utah/autorun}"
+INPROGRESS=${RESULTDIR}/inprogress
+COMPLETE=${RESULTDIR}/complete
+
+{% include "log-function.jinja2" %}
+
+if [ ! -d ${WORKDIR} ] ; then
+	log "No work found under ${WORKDIR}"
+	exit 0
+fi
+
+[ -d ${INPROGRESS} ] || mkdir -p ${INPROGRESS}
+[ -d ${COMPLETE} ] || mkdir -p ${COMPLETE}
+
+for p in ${WORKDIR}/* ; do
+	if test -f $p && test -x $p ; then
+		log "running $p"
+		fname=$(basename $p)
+		mv $p ${INPROGRESS}/
+		set +e
+		${INPROGRESS}/$fname
+		rc=$?
+		set -e
+		mv ${INPROGRESS}/$fname ${COMPLETE}/${fname}.$(date +%s).$rc
+		log "completed $p rc=$rc"
+		if [ $rc -ne 0 ] ; then
+			log "exiting autorun due to failure"
+			exit 1
+		fi
+	fi
+done

=== renamed file 'templates/utah-autorun.sh.jinja2' => 'templates/utah-autorun.sh.jinja2.moved'
=== added file 'tests/test_autorun.py'
--- tests/test_autorun.py	1970-01-01 00:00:00 +0000
+++ tests/test_autorun.py	2013-06-07 18:58:24 +0000
@@ -0,0 +1,71 @@
+# Ubuntu Testing Automation Harness
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Unit tests for the utah-autorun.sh script."""
+
+import os
+import re
+import shutil
+import stat
+import subprocess
+import tempfile
+import unittest
+
+from utah import template
+
+
+class TestAutoRun(unittest.TestCase):
+
+    """Tests the functionality of the utah-autorun.sh script."""
+
+    def setUp(self):
+        """Set up a temporary work directory for the script."""
+        self.workdir = tempfile.mkdtemp(prefix='utah-autotest-')
+        # ensure we can safely ignore directories
+        os.mkdir('{}/foo'.format(self.workdir))
+
+        #piggy back the foo directory for the run script location
+        self.script = os.path.join(self.workdir, 'foo', 'utah-autorun.sh')
+        lf = '/dev/null'
+        template.write('utah-autorun.sh.jinja2', self.script, log_file=lf)
+        os.chmod(self.script, stat.S_IRWXU)
+
+        self._write_script('01_foo', 'echo foo running')
+        self._write_script('02_bar', 'echo bar running')
+
+    def tearDown(self):
+        """Clean up the work directory."""
+        shutil.rmtree(self.workdir)
+
+    def _write_script(self, name, contents):
+        with open('{}/{}'.format(self.workdir, name), 'w') as f:
+            f.write('#!/bin/sh\n')
+            f.write(contents)
+            os.fchmod(f.fileno(), stat.S_IRWXU)
+
+    def test_autorun(self):
+        """minimal test to make sure autorun works."""
+        env = {'WORKDIR': self.workdir, 'RESULTDIR': self.workdir}
+        output = subprocess.check_output([self.script], env=env)
+        self.assertEqual(output, "foo running\nbar running\n")
+        entries = sorted(os.listdir(os.path.join(self.workdir, 'complete')))
+        self.assertIsNotNone(re.match(r'01_foo\.\d+', entries[0]))
+        self.assertIsNotNone(re.match(r'02_bar\.\d+', entries[1]))
+        self.assertEqual(len(entries), 2)
+        entries = sorted(os.listdir(self.workdir))
+        self.assertEqual(entries[0], 'complete')
+        self.assertEqual(entries[1], 'foo')
+        self.assertEqual(entries[2], 'inprogress')
+        self.assertEqual(len(entries), 3)

=== renamed file 'tests/test_autorun.py' => 'tests/test_autorun.py.moved'
=== added file 'tests/test_config.py'
--- tests/test_config.py	1970-01-01 00:00:00 +0000
+++ tests/test_config.py	2013-06-07 18:58:24 +0000
@@ -0,0 +1,105 @@
+# Ubuntu Testing Automation Harness
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Test configuration module."""
+
+import io
+import unittest
+
+import json
+import jsonschema
+
+from mock import patch
+
+import utah.config
+
+
+class TestConfig(unittest.TestCase):
+
+    """Test configuration functionality."""
+
+    def test_defaults_validate(self):
+        """Verify that defaults validate against the schema."""
+        config = utah.config._Config()
+        jsonschema.validate(dict(config), config.SCHEMA)
+
+    def test_private_variables_not_in_dict(self):
+        """Verify there are no private variables when converting to a dict."""
+        config = utah.config._Config()
+        config._private = '<value>'
+        dictionary = dict(config)
+        self.assertNotIn('_private', dictionary)
+
+    def test_invalid_json_file(self):
+        """Verify error is printed to stderr on invalid json file."""
+        config = utah.config._Config()
+
+        config_file = '<config_file>'
+        with patch('utah.config.os.path') as path, \
+                patch('__builtin__.open') as open_, \
+                patch('sys.stderr') as stderr:
+            path.is_file.return_value = True
+            path.getsize.return_value = 1
+            open_.return_value = io.StringIO(u'not a valid json file')
+
+            config._process_config_file(config_file)
+
+            stderr.write.assert_called_with(
+                'ERROR: {} is not a valid JSON file\n'.format(config_file))
+
+    def test_invalid_config_file(self):
+        """Verify error is printed to stderr on invalid configuration file."""
+        config = utah.config._Config()
+
+        config_file = '<config_file>'
+        unknown_key = 'unknown_key'
+        with patch('utah.config.os.path') as path, \
+                patch('__builtin__.open') as open_, \
+                patch('sys.stderr') as stderr:
+            path.isfile.return_value = True
+            path.getsize.return_value = 1
+            contents = unicode(json.dumps({unknown_key: 'value'}))
+            open_.return_value = io.StringIO(contents)
+            config._process_config_file(config_file)
+
+            stderr.write.assert_called_with(
+                'ERROR: {} is not a valid configuration file\n'
+                'Detailed information: Additional properties are not allowed '
+                "(u'{}' was unexpected)\n"
+                .format(config_file, unknown_key))
+
+    def test_value_overwritten(self):
+        """Verify warning when configuration value is overwritten."""
+        config = utah.config._Config()
+
+        config_filenames = ['config_file_1', 'config_file_2']
+        option_name = 'arch'
+        option_value = 'amd64'
+        with patch('utah.config.os.path') as path, \
+                patch('__builtin__.open') as open_, \
+                patch('sys.stderr') as stderr:
+            path.isfile.return_value = True
+            path.getsize.return_value = 1
+            contents = unicode(json.dumps({option_name: option_value}))
+            open_.side_effect = (
+                io.StringIO(contents),
+                io.StringIO(contents),
+            )
+            config._process_config_file(config_filenames[0])
+            config._process_config_file(config_filenames[1])
+
+            stderr.write.assert_called_with(
+                'WARNING: {} was already set by {}'
+                .format(option_name, config_filenames[0]))

=== renamed file 'tests/test_config.py' => 'tests/test_config.py.moved'
=== added file 'tests/test_provision_data.py'
--- tests/test_provision_data.py	1970-01-01 00:00:00 +0000
+++ tests/test_provision_data.py	2013-06-07 18:58:24 +0000
@@ -0,0 +1,92 @@
+# Ubuntu Testing Automation Harness
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Unit tests for the utah.provisioning.data module."""
+
+import os
+import shutil
+import subprocess
+import tempfile
+import unittest
+
+from utah.provisioning.data import ProvisionData
+
+
+class TestProvisionData(unittest.TestCase):
+
+    """Tests the functions of the ProvisionData class."""
+
+    def setUp(self):
+        """Create temp files needed for testing."""
+        self.files = []
+        for x in xrange(4):
+            fd, f = tempfile.mkstemp(prefix='utah_provision_data')
+            self.files.append(f)
+            os.write(fd, '{}'.format(x))
+            os.close(fd)
+
+        self.initrd = tempfile.mkdtemp(prefix='utah_initrd')
+
+    def tearDown(self):
+        """Clean up the artifacts create in setUp."""
+        for f in self.files:
+            os.unlink(f)
+        shutil.rmtree(self.initrd)
+
+    def test_add_files(self):
+        """Ensure the add_files API works."""
+        pd = ProvisionData()
+        pd.add_files(self.files[0], '/foo')
+        pd.add_files((self.files[1], self.files[2]), '/bar')
+
+        pd.update_initrd(self.initrd)
+
+        dstdir = os.path.join(self.initrd, 'utah-copy-files')
+        for x in xrange(3):
+            bname = os.path.basename(self.files[x])
+            dstfile = os.path.join(dstdir, bname)
+            self.assertTrue(os.path.isfile(dstfile))
+            with open(dstfile) as f:
+                content = f.read()
+                self.assertEqual(content, str(x))
+
+        script = os.path.join(self.initrd, 'utah-copy-files.sh')
+        self.assertTrue(os.path.isfile(script))
+
+        #update the copy script to copy to our tmpdir so we can run locally
+        escaped_dir = self.initrd.replace('/', '\/')
+        exp = 's/\/target/{}/g'.format(escaped_dir)
+        subprocess.check_call(['sed', '-i', '-e', exp, script])
+
+        escaped_dir = self.initrd.replace('/', '\/')
+        exp = 's/\/utah-copy-files/{}\/utah-copy-files/g'.format(escaped_dir)
+        subprocess.check_call(['sed', '-i', '-e', exp, script])
+
+        subprocess.check_call(['sh', script])
+
+    def test_add_cmds(self):
+        """Ensure the add_autostart_cmd API works."""
+        pd = ProvisionData()
+        for x in xrange(3):
+            pd.add_autostart_cmd('{}_foo'.format(x), str(x))
+
+        pd.update_initrd(self.initrd)
+        for x in xrange(3):
+            fname = os.path.join(
+                self.initrd, 'utah-autorun', '{}_foo'.format(x))
+            self.assertTrue(os.path.isfile(fname))
+            with open(fname) as f:
+                cmd = f.readlines()[-1]
+                self.assertEqual(cmd, '{}\n'.format(x))

=== renamed file 'tests/test_provision_data.py' => 'tests/test_provision_data.py.moved'
=== modified file 'utah/client/runner.py'
--- utah/client/runner.py	2013-06-06 22:52:50 +0000
+++ utah/client/runner.py	2013-06-07 18:58:24 +0000
@@ -432,6 +432,7 @@
             # fetch failure but allow the test runs to proceed.
             if self.result.status != 'PASS':
                 self.result.status = 'PASS'
+<<<<<<< TREE
         return fetch_success
 
     def process_master_runlist(self, runlist=None, resume=False):
@@ -448,17 +449,20 @@
 
         # Download runlist using the URL passed through the commmand line to
         # local filename if needed (it might be a local file already or cached)
+=======
+        return fetch_success
+
+    @staticmethod
+    def _fetch_and_parse(runlist):
+>>>>>>> MERGE-SOURCE
         try:
             local_filename = urllib.urlretrieve(runlist)[0]
-        except IOError:
-            raise exceptions.MissingFile(runlist)
-        except urllib.ContentTooShortError as err:
+        except IOError as err:
             raise exceptions.MissingFile(
-                'Error when downloading {} (probably interrupted): {}'
-                .format(runlist, err))
+                'Error when downloading {}: {}'.format(runlist, err))
 
         data = parse_yaml_file(local_filename)
-        validator = DefaultValidator(self.MASTER_RUNLIST_SCHEMA)
+        validator = DefaultValidator(Runner.MASTER_RUNLIST_SCHEMA)
         try:
             validator.validate(data)
         except jsonschema.ValidationError as exception:
@@ -466,15 +470,56 @@
                 'Master runlist failed to validate: {!r}\n'
                 'Detailed information: {}'
                 .format(local_filename, exception))
+        return data
+
+    @staticmethod
+    def _clean_suite(name):
+        # convert to absolute name to make troubleshooting a little easier
+        name = os.path.abspath(name)
+        try:
+            shutil.rmtree(name)
+        except OSError as e:
+            raise exceptions.UTAHClientError(
+                'Error removing the testsuite {}: {}'.format(name, e))
+
+    def _add_suite(self, suite):
+        name = suite['name']
+
+        if name not in self.fetched_suites:
+            if self._fetch_suite(suite):
+                self.fetched_suites.append(name)
+                suite_runlist = suite.get('runlist', DEFAULT_TSLIST)
+
+                s = TestSuite(name=name,
+                              runlist_file=suite_runlist,
+                              includes=suite.get('include_tests', None),
+                              excludes=suite.get('exclude_tests', None),
+                              result=self.result,
+                              path=self.testsuitedir,
+                              timeout=self.timeout,
+                              battery_measurements=self.battery_measurements,
+                              _save_state_callback=self.save_state,
+                              _reboot_callback=self.reboot)
+                self.add_suite(s)
+            else:
+                self.fetch_errors += 1
+
+    def process_master_runlist(self, runlist=None, resume=False):
+        """Parse a master runlist and build a list of suites from the data.
+
+        :param runlist: URL pointing to a runlist
+        :type rulist: string
+        :param resume: Continue previous execution
+        :type resume: boolean
+
+        """
+        runlist = runlist or self.master_runlist
+        data = self._fetch_and_parse(runlist)
 
         if 'timeout' in data:
             self.timeout = int(data['timeout'])
 
-        if 'name' in data:
-            self.name = data['name']
-        else:
-            self.name = 'unnamed'
-
+        self.name = data.get('name', 'unnamed')
         self.battery_measurements = data['battery_measurements']
         self.repeat_count = data['repeat_count']
 
@@ -482,13 +527,7 @@
 
         orig_dir = os.getcwd()
 
-        if 'testsuites' in data:
-            suites = data['testsuites']
-        elif isinstance(data, list):
-            suites = data
-        else:
-            raise exceptions.BadMasterRunlist(str(data))
-
+        suites = data['testsuites']
         suites = data['testsuites'] if 'testsuites' in data else data
         for suite in suites:
             # Allow the inclusion of other master.run files
@@ -500,11 +539,6 @@
 
             name = suite['name']
 
-            includes = suite.get('include_tests')
-            excludes = suite.get('exclude_tests')
-
-            suite_runlist = suite.get('runlist', DEFAULT_TSLIST)
-
             if name in seen:
                 raise exceptions.BadMasterRunlist(
                     "{} duplicated in runlist".format(name))
@@ -512,38 +546,10 @@
             # Fetch the testsuite. On resume don't remove the testsuite
             # directory.
             if not resume and os.path.exists(name):
-                # Using absolute name makes no difference
-                # except on failures where it's easier
-                # to find troubleshoot permission problems
-                absolute_name = os.path.abspath(name)
-                try:
-                    shutil.rmtree(absolute_name)
-                except OSError as err:
-                    raise exceptions.UTAHClientError(
-                        'Error removing the testsuite directory {}: {}'
-                        .format(absolute_name, err))
+                self._clean_suite(name)
+
             mkdir(name)
-
-            if name not in self.fetched_suites:
-                if not self._fetch_suite(suite):
-                    # If fetch failed move on to the next testsuite.
-                    self.fetch_errors += 1
-                    continue
-
-                self.fetched_suites.append(name)
-
-                # Create a TestSuite
-                s = TestSuite(name=name,
-                              runlist_file=suite_runlist,
-                              includes=includes,
-                              excludes=excludes,
-                              result=self.result,
-                              path=self.testsuitedir,
-                              timeout=self.timeout,
-                              battery_measurements=self.battery_measurements,
-                              _save_state_callback=self.save_state,
-                              _reboot_callback=self.reboot)
-                self.add_suite(s)
+            self._add_suite(suite)
 
     def get_next_suite(self):
         """Return the next suite to be run.

=== added file 'utah/client/tests/test_client.py'
--- utah/client/tests/test_client.py	1970-01-01 00:00:00 +0000
+++ utah/client/tests/test_client.py	2013-06-07 18:58:24 +0000
@@ -0,0 +1,71 @@
+# Ubuntu Testing Automation Harness
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Tests for the client script."""
+
+import os
+import sys
+import tempfile
+import unittest
+
+from mock import patch
+
+import client
+
+from utah.client.common import ReturnCodes
+
+
+class TestClient(unittest.TestCase):
+
+    """Test the client script."""
+
+    @patch('client.sys.stderr')
+    @patch('utah.client.runner.parse_yaml_file')
+    @patch('utah.client.runner.urllib.urlretrieve')
+    @patch('utah.client.runner.os.path.isfile')
+    @patch('client.url_argument')
+    def test_report_not_written_on_invalid_runlist(
+            self, url_argument, isfile, urlretrieve, parse_yaml_file, stderr):
+        """Test report file is not created on runlist validation failure."""
+        output_file = tempfile.mktemp(prefix='utah-', suffix='.yaml')
+
+        # Set mock objects return value
+        url_argument.return_value = '<runlist>'
+        isfile.return_value = True
+        filename = '<local_filename>'
+        urlretrieve.return_value = [filename]
+        parse_yaml_file.return_value = 'invalid_runlist'
+
+        with self.assertRaises(SystemExit):
+            client.main(['-r', '<runlist>', '-o', output_file])
+        self.assertFalse(os.path.exists(output_file),
+                         'Output file was created: {}'.format(output_file))
+        args, kwargs = stderr.write.call_args_list[0]
+        self.assertTrue(
+            args[0].startswith('Master runlist failed to validate: '
+                               "'{}'\n".format(filename)))
+
+    @patch('argparse._sys.stderr')
+    def test_invalid_argument_return_code(self, stderr):
+        """Test return code value on invalid command line arguments."""
+        prog = 'utah'
+        sys.argv[0] = prog
+        with self.assertRaises(SystemExit) as cm:
+            client.main(['--invalid-option'])
+
+        self.assertEqual(cm.exception.code, ReturnCodes.CMD_PARSING_ERROR)
+        stderr.write.assert_called_with(
+            '{}: error: unrecognized arguments: --invalid-option\n'
+            .format(prog))

=== renamed file 'utah/client/tests/test_client.py' => 'utah/client/tests/test_client.py.moved'
=== modified file 'utah/client/tests/test_runner.py'
--- utah/client/tests/test_runner.py	2013-06-03 16:27:13 +0000
+++ utah/client/tests/test_runner.py	2013-06-07 18:58:24 +0000
@@ -18,6 +18,7 @@
 
 import os
 import shutil
+import tempfile
 import unittest
 from mock import patch
 
@@ -59,21 +60,16 @@
                              state_agent=self.state_agent,
                              output=self.output_file)
         self.bad_dir = "/tmp/does_not_exist"
+        _, self.tmpfile = tempfile.mkstemp()
 
         self.assertFalse(os.path.exists(self.bad_dir),
                          "{} shouldn't exist".format(self.bad_dir))
 
     def tearDown(self):
         """Clean up test resources."""
-        # Remove self.bad_dir if it got created.
-        try:
+        os.unlink(self.tmpfile)
+        if os.path.exists(self.bad_dir):
             shutil.rmtree(self.bad_dir)
-        except OSError as e:
-            # ignore 'No such file or directory errors
-            if e.errno == 2:
-                pass
-            else:
-                raise
 
     def test_fetch_failed(self):
         """Test handling of failed fetch commands."""
@@ -238,6 +234,16 @@
         self.assertListEqual(kwargs['excludes'],
                              ['test_1', 'test_2', 'test_3'])
 
+    def test_fetch_runlist(self):
+        """Ensure proper error handling of runlist fetching logic."""
+        with self.assertRaises(exceptions.MissingFile):
+            Runner._fetch_and_parse('this is a bad url')
+
+        with self.assertRaises(exceptions.ValidationError):
+            with open(self.tmpfile, 'w') as f:
+                f.write('andy: blah\n')
+            Runner._fetch_and_parse(self.tmpfile)
+
 
 class TestRunnerMasterRunlistSchema(unittest.TestCase):
 

=== modified file 'utah/client/testsuite.py'
--- utah/client/testsuite.py	2013-06-02 18:08:30 +0000
+++ utah/client/testsuite.py	2013-06-07 18:58:24 +0000
@@ -37,17 +37,24 @@
 from utah.client.testcase import TestCase
 
 
-def parse_runlist_file(runlist_file):
+def _parse_runlist_file(runlist_file):
     """Parse a tslist.run runlist file and check against schema.
 
     :returns: Parsed data from the runlist
     :rtype: dict
 
     """
+    if not os.path.exists(runlist_file):
+        raise exceptions.MissingFile('File not found: {}'.format(runlist_file))
+
     with open(runlist_file, 'r') as fp:
         data = yaml.load(fp)
-
-    jsonschema.validate(data, TestSuite.RUNLIST_SCHEMA)
+    try:
+        jsonschema.validate(data, TestSuite.RUNLIST_SCHEMA)
+    except jsonschema.ValidationError as exception:
+            raise exceptions.ValidationError(
+                'Test suite runlist invalid: {!r}\n'
+                'Detailed information: {}'.format(runlist_file, exception))
 
     return data
 
@@ -147,12 +154,13 @@
             self.save_state_callback = _save_state_callback
 
         self.result = result
-
-        control_data = None
-
-        if _control_data is not None:
-            control_data = _control_data
-        elif self.control_file is not None:
+        self._initialize_control_data(_control_data, control_file)
+
+        for test in self._get_runlist_data(_runlist_data):
+            self._add_test(test, includes, excludes)
+
+    def _initialize_control_data(self, control_data, control_file):
+        if not control_data and self.control_file is not None:
             try:
                 control_data = parse_control_file(self.control_file,
                                                   self.CONTROL_SCHEMA)
@@ -173,57 +181,41 @@
 
             self.__dict__.update(control_data)
 
-        if _runlist_data is not None:
-            runlist_data = _runlist_data
-        elif os.path.exists(self.runlist_file):
-            try:
-                runlist_data = parse_runlist_file(self.runlist_file)
-            except jsonschema.ValidationError as exception:
-                raise exceptions.ValidationError(
-                    '{!r} test suite runlist invalid: {!r}\n'
-                    'Detailed information: {}'
-                    .format(self.name, self.runlist_file, exception))
-        else:
-            raise exceptions.MissingFile(
-                'File not found: {}'.format(self.runlist_file))
-
-        for test in runlist_data:
-            name = test['test']
-            test_path = os.path.join(self.name, name)
-
-            if includes is not None and name not in includes:
-                continue
-
-            if excludes is not None and name in excludes:
-                continue
-
-            command = test.get('command')
-
-            tc = TestCase(name=name,
-                          path=test_path,
-                          command=command,
-                          timeout=self.timeout,
-                          result=self.result,
-                          battery_measurements=self.battery_measurements,
-                          _save_state_callback=self.save_state_callback,
-                          _reboot_callback=self.reboot_callback)
-
-            runlist_properties = self.RUNLIST_SCHEMA['items']['properties']
-            override_properties = \
-                runlist_properties['overrides']['properties'].keys()
-            overrides = {}
-            for property_name in override_properties:
-                if property_name in test:
-                    overrides[property_name] = test[property_name]
-            if 'overrides' in test:
-                if self.timeout is not None:
-                    test['overrides']['timeout'] = self.timeout
-                overrides.update(test['overrides'])
-
-            if overrides:
-                tc.process_overrides(overrides)
-
-            self.tests.append(tc)
+    def _get_runlist_data(self, runlist_data):
+        if runlist_data:
+            return runlist_data
+        return _parse_runlist_file(self.runlist_file)
+
+    def _add_test(self, test, incs, excs):
+        name = test['test']
+
+        if (incs and name not in incs) or (excs and name in excs):
+            return
+
+        tc = TestCase(name=name,
+                      path=os.path.join(self.name, name),
+                      command=test.get('command'),
+                      timeout=self.timeout,
+                      result=self.result,
+                      battery_measurements=self.battery_measurements,
+                      _save_state_callback=self.save_state_callback,
+                      _reboot_callback=self.reboot_callback)
+
+        runlist_properties = self.RUNLIST_SCHEMA['items']['properties']
+        override_properties = \
+            runlist_properties['overrides']['properties'].keys()
+        overrides = {}
+        for property_name in override_properties:
+            if property_name in test:
+                overrides[property_name] = test[property_name]
+        if 'overrides' in test:
+            if self.timeout is not None:
+                test['overrides']['timeout'] = self.timeout
+            overrides.update(test['overrides'])
+
+        tc.process_overrides(overrides)
+
+        self.tests.append(tc)
 
     def __str__(self):
         return "{}: {}".format(self.control_file, self.runlist_file)

=== added file 'utah/experimental.py'
--- utah/experimental.py	1970-01-01 00:00:00 +0000
+++ utah/experimental.py	2013-06-07 18:58:24 +0000
@@ -0,0 +1,48 @@
+# Ubuntu Testing Automation Harness
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Allows conditional inclusion of experimental features."""
+
+import logging
+import os
+
+_features = [
+    'PROVISIONED_AUTORUN',
+]
+
+
+class _feature(object):
+
+    """Builds up a list of what experimental features are enabled.
+
+    Experimental features can be enabled by the existance of an environment
+    variable in the format: UTAH_FEATURE_FOO=<any value>
+
+    """
+
+    def __init__(self):
+        for f in _features:
+            key = 'UTAH_FEATURE_{}'.format(f)
+            setattr(self, f, key in os.environ)
+
+    def log_features(self):
+        """Writes an info logging message with the state of each feature."""
+        buf = 'UTAH Features:'
+        for f in _features:
+            buf += '\n {}: {}'.format(f, getattr(self, f))
+        logging.info(buf)
+
+
+feature = _feature()

=== renamed file 'utah/experimental.py' => 'utah/experimental.py.moved'
=== added file 'utah/provisioning/data.py'
--- utah/provisioning/data.py	1970-01-01 00:00:00 +0000
+++ utah/provisioning/data.py	2013-06-07 18:58:24 +0000
@@ -0,0 +1,80 @@
+# Ubuntu Testing Automation Harness
+# Copyright 2012 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Data structure to help simplify provisioning logic."""
+
+import os
+import shutil
+
+
+class ProvisionData(object):
+
+    """A structure to hold information about image customizations needed.
+
+    This can be used to inform our provisioning logic about extra files and
+    auto-start commands it should set up for the target
+
+    """
+
+    def __init__(self):
+        self._files = []
+        self._cmds = []
+
+    def add_files(self, files, target_dir):
+        """Add a file to get copied to the target system."""
+        if isinstance(files, basestring):
+            files = [files]
+        self._files.append({'files': files, 'target_dir': target_dir})
+
+    def add_autostart_cmd(self, name, cmd):
+        """Add a command to be run at startup."""
+        self._cmds.append({'name': name, 'cmd': cmd})
+
+    def _update_initrd_files(self, path):
+        copy_dir = os.path.join(path, 'utah-copy-files')
+        os.mkdir(copy_dir)
+
+        with open(os.path.join(path, 'utah-copy-files.sh'), 'w') as f:
+            f.write('#!/bin/sh\n')
+            for fileobj in self._files:
+                tgt_dst = '/target{}'.format(fileobj['target_dir'])
+                f.write('[ -d {d} ] || mkdir -p {d}\n'.format(d=tgt_dst))
+                for path in fileobj['files']:
+                    fname = os.path.basename(path)
+                    hst_cpy = os.path.join(copy_dir, fname)
+                    tgt_cpy = os.path.join('/utah-copy-files', fname)
+                    shutil.copyfile(path, hst_cpy)
+                    f.write('cp {} {}/\n'.format(tgt_cpy, tgt_dst))
+
+    def _update_initrd_cmds(self, path):
+        path = os.path.join(path, 'utah-autorun')
+        os.mkdir(path)
+
+        for cmd in self._cmds:
+            with open(os.path.join(path, cmd['name']), 'w') as f:
+                f.write('#!/bin/sh\n')
+                f.write('{}\n'.format(cmd['cmd']))
+
+    def update_initrd(self, path):
+        """Build the initrd file to include needed files commands.
+
+        This copies the files to the initrd, and builds a script that can be
+        run to put them in the proper directory on the installed system. It
+        also adds a set of auto run files that can be copied over to the
+        installed system during the late command.
+
+        """
+        self._update_initrd_files(path)
+        self._update_initrd_cmds(path)

=== renamed file 'utah/provisioning/data.py' => 'utah/provisioning/data.py.moved'