Merge ~powersj/cloud-init:integration-testing into cloud-init:master

[Joshua Powers]

inttests: initial commit of integration test framework

The adds in end-to-end testing of cloud-init. The framework utilizes
LXD and cloud images as a backend to test user-data passed in.
Arbitrary data is then captured from predefined commands specified
by the user. After collection, data verification is completed by
running a series of Python unit tests against the collected data.

Currently only the Ubuntu Trusty, Xenial, Yakkety, and Zesty
releases are supported. Test cases for 50% of the modules is
complete and available.

Additionally a Read the Docs file was created to guide test
writing and execution.

* initial commit of integration test framework
* added tests.rst to Read the Docs and updated index.rst
* updated tox.ini to not run our integration unit test scripts when running unit tests

[Scott Moser (smoser)]

some comments inline

review: Needs Fixing

[Joshua Powers (powersj)]

Updating PPA and other sources.list tests to no longer use my personal
information. This will use the generic cloud-init testing and
development information. Also modifies the tox config to only run the
tests under unittests for Py3. Disabling the SSH tests that utilize my
own key until we have a function to create keys on the fly. Also
removing numerous bashisms.

review: Needs Resubmitting

[Scott Moser (smoser)]

I pulled this into trunk.
I had some issues trying to just 'rebase' and then squash so i just diffed and applied locally.

other changes i made:
 * add vi: on each python file
 * add cloud-init license on each test python file
 * keep tests.rst under 80 chars wide.

[Joshua Powers (powersj)]

Thank you very much!

Is updating read-the-docs something we can do to? Or does that happen
automatically?

Josh

On Thu, Dec 22, 2016 at 3:44 PM, Scott Moser <email address hidden> wrote:

> I pulled this into trunk.
> I had some issues trying to just 'rebase' and then squash so i just diffed
> and applied locally.
>
> other changes i made:
> * add vi: on each python file
> * add cloud-init license on each test python file
> * keep tests.rst under 80 chars wide.
>
> --
> https://code.launchpad.net/~powersj/cloud-init/+git/cloud-
> init/+merge/312830
> You are the owner of ~powersj/cloud-init:integration-testing.
>

[Scott Moser (smoser)]

Should happen automatically. I'll check to make sure tomorrow

On December 22, 2016 6:12:42 PM EST, Joshua Powers <email address hidden> wrote:
>Thank you very much!
>
>Is updating read-the-docs something we can do to? Or does that happen
>automatically?
>
>Josh
>
>On Thu, Dec 22, 2016 at 3:44 PM, Scott Moser <email address hidden> wrote:
>
>> I pulled this into trunk.
>> I had some issues trying to just 'rebase' and then squash so i just
>diffed
>> and applied locally.
>>
>> other changes i made:
>> * add vi: on each python file
>> * add cloud-init license on each test python file
>> * keep tests.rst under 80 chars wide.
>>
>> --
>> https://code.launchpad.net/~powersj/cloud-init/+git/cloud-
>> init/+merge/312830
>> You are the owner of ~powersj/cloud-init:integration-testing.
>>
>
>--
>https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/312830
>You are reviewing the proposed merge of
>~powersj/cloud-init:integration-testing into cloud-init:master.

diff --git a/doc/rtd/index.rst b/doc/rtd/index.rst
index 3caf33f..8f163b6 100644
--- a/doc/rtd/index.rst
+++ b/doc/rtd/index.rst
@@ -41,6 +41,7 @@ initialization of a cloud instance.
    topics/vendordata.rst
    topics/moreinfo.rst
    topics/hacking.rst
+   topics/tests.rst
 
 .. _Cloud-init: https://launchpad.net/cloud-init
 .. vi: textwidth=78
diff --git a/doc/rtd/topics/tests.rst b/doc/rtd/topics/tests.rst
new file mode 100644
index 0000000..d2db140
--- /dev/null
+++ b/doc/rtd/topics/tests.rst
@@ -0,0 +1,238 @@
+.. contents:: Table of Contents
+   :depth: 2
+
+============================
+Test Development
+============================
+
+
+Overview
+--------
+
+The purpose of this page is to describe how to write integration tests for cloud-init. As a test writer you need to develop a test configuration and a verification file:
+
+ * The test configuration specifies a specific cloud-config to be used by cloud-init and a list of arbitrary commands to capture the output of (e.g my_test.yaml)
+
+ * The verification file runs tests on the collected output to determine the result of the test (e.g. my_test.py)
+
+The names must match, however the extensions will of course be different, yaml vs py.
+
+Configuration
+-------------
+
+The test configuration is a YAML file such as *ntp_server.yaml* below:
+
+.. code-block:: yaml
+
+    #
+    # NTP config using specific servers (ntp_server.yaml)
+    #
+    cloud_config: |
+      #cloud-config
+      ntp:
+        servers:
+          - pool.ntp.org
+    collect_scripts:
+      ntp_installed_servers: |
+        #!/bin/bash
+        dpkg -l | grep ntp | wc -l
+      ntp_conf_dist_servers: |
+        #!/bin/bash
+        ls /etc/ntp.conf.dist | wc -l
+      ntp_conf_servers: |
+        #!/bin/bash
+        cat /etc/ntp.conf | grep '^server'
+
+
+There are two keys, 1 required and 1 optional, in the YAML file:
+
+1. The required key is ``cloud_config``. This should be a string of valid YAML that is exactly what would normally be placed in a cloud-config file, including the cloud-config header. This essentially sets up the scenario under test.
+
+2. The optional key is ``collect_scripts``. This key has one or more sub-keys containing strings of arbitrary commands to execute (e.g. ```cat /var/log/cloud-config-output.log```). In the example above the output of dpkg is captured, grep for ntp, and the number of lines reported. The name of the sub-key is important. The sub-key is used by the verification script to recall the output of the commands ran.
+
+Default Collect Scripts
+~~~~~~~~~~~~~~~~~~~~~~~
+
+By default the following files will be collected for every test. There is no need to specify these items:
+
+* ``/var/log/cloud-init.log``
+* ``/var/log/cloud-init-output.log``
+* ``/run/cloud-init/.instance-id``
+* ``/run/cloud-init/result.json``
+* ``/run/cloud-init/status.json``
+* ```dpkg-query -W -f='${Version}' cloud-init```
+
+Verification
+------------
+
+The verification script is a Python file with unit tests like the one, `ntp_server.py`, below:
+
+.. code-block:: python
+
+    """cloud-init Integration Test Verify Script (ntp_server.yaml)"""
+    from tests.cloud_tests.testcases import base
+
+
+    class TestNtpServers(base.CloudTestCase):
+        """Test ntp module"""
+
+        def test_ntp_installed(self):
+            """Test ntp installed"""
+            out = self.get_data_file('ntp_installed_servers')
+            self.assertEqual(1, int(out))
+
+        def test_ntp_dist_entries(self):
+            """Test dist config file has one entry"""
+            out = self.get_data_file('ntp_conf_dist_servers')
+            self.assertEqual(1, int(out))
+
+        def test_ntp_entires(self):
+            """Test config entries"""
+            out = self.get_data_file('ntp_conf_servers')
+            self.assertIn('server pool.ntp.org iburst', out)
+
+
+Here is a breakdown of the unit test file:
+
+* The import statement allows access to the output files.
+
+* The class can be named anything, but must import the ``base.CloudTestCase``
+
+* There can be 1 to N number of functions with any name, however only tests starting with ``test_*`` will be executed.
+
+* Output from the commands can be accessed via ``self.get_data_file('key')`` where key is the sub-key of ``collect_scripts`` above.
+
+Layout
+------
+
+Integration tests are located under the `tests/cloud_tests` directory. Test configurations are placed under `configs` and the test verification scripts under `testcases`:
+
+.. code-block:: bash
+
+    cloud-init$ tree -d tests/cloud_tests/
+    tests/cloud_tests/
+    ├── configs
+    │   ├── bugs
+    │   ├── examples
+    │   ├── main
+    │   └── modules
+    └── testcases
+        ├── bugs
+        ├── examples
+        ├── main
+        └── modules
+
+The sub-folders of bugs, examples, main, and modules help organize the tests. View the README.md in each to understand in more detail each directory.
+
+
+=====================
+Development Checklist
+=====================
+
+* Configuration File
+    * Named 'your_test_here.yaml'
+    * Contains at least a valid cloud-config
+    * Optionally, commands to capture additional output
+    * Valid YAML
+    * Placed in the appropriate sub-folder in the configs directory
+* Verification File
+    * Named 'your_test_here.py'
+    * Valid unit tests validating output collected
+    * Passes pylint & pep8 checks
+    * Placed in the appropriate sub-folder in the testcsaes directory
+* Tested by running the test: 
+
+.. code-block:: bash
+
+    $ python3 -m tests.cloud_tests run -v -n <release of choice> --deb <build of cloud-init> -t tests/cloud_tests/configs/<dir>/your_test_here.yaml
+
+
+=========
+Execution
+=========
+
+Executing tests has three options:
+
+* ``run`` an alias to run both ``collect`` and ``verify``
+
+* ``collect`` deploys on the specified platform and os, patches with the requested deb or rpm, and finally collects output of the arbitrary commands.
+
+* ``verify`` given a directory of test data, run the Python unit tests on it to generate results.
+
+Run
+---
+The first example will provide a complete end-to-end run of data collection and verification. There are additional examples below explaining how to run one or the other independently.
+
+.. code-block:: bash
+
+    $ git clone https://git.launchpad.net/cloud-init
+    $ cd cloud-init
+    $ python3 -m tests.cloud_tests run -v -n trusty -n xenial --deb cloud-init_0.7.8~my_patch_all.deb
+
+The above command will do the following:
+
+* ``-v`` verbose output
+
+* ``run`` both collect output and run tests the output
+
+* ``-n trusty`` on the Ubuntu Trusty release
+
+* ``-n xenial`` on the Ubuntu Xenial release
+
+* ``--deb cloud-init_0.7.8~patch_all.deb`` use this deb as the version of cloud-init to run with
+
+For a more detailed explanation of each option see below.
+
+Collect
+-------
+
+If developing tests it may be necessary to see if cloud-config works as expected and the correct files are pulled down. In this case only a collect can be ran by running:
+
+.. code-block:: bash
+
+    $ python3 -m tests.cloud_tests collect -n xenial -d /tmp/collection --deb cloud-init_0.7.8~my_patch_all.deb 
+
+The above command will run the collection tests on xenial with the provided deb and place all results into `/tmp/collection`.
+
+Verify
+------
+
+When developing tests it is much easier to simply rerun the verify scripts without the more lengthy collect process. This can be done by running:
+
+.. code-block:: bash
+
+    $ python3 -m tests.cloud_tests verify -d /tmp/collection
+
+The above command will run the verify scripts on the data discovered in `/tmp/collection`.
+
+
+============
+Architecture
+============
+
+The following outlines the process flow during a complete end-to-end LXD-backed test.
+
+1. Configuration
+    * The back end and specific OS releases are verified as supported
+    * The test or tests that need to be run are determined either by directory or by individual yaml
+
+2. Image Creation
+    * Acquire the daily LXD image
+    * Install the specified cloud-init package
+    * Clean the image so that it does not appear to have been booted
+    * A snapshot of the image is created and reused by all tests
+
+3. Configuration
+    * For each test, the cloud-config is injected into a copy of the snapshot and booted
+    * The framework waits for ``/var/lib/cloud/instance/boot-finished`` (up to 120 seconds)
+    * All default commands are ran and output collected
+    * Any commands the user specified are executed and output collected
+
+4. Verification
+    * The default commands are checked for any failures, errors, and warnings to validate basic functionality of cloud-init completed successfully
+    * The user generated unit tests are then ran validating against the collected output
+
+5. Results
+    * If any failures were detected the test suite returns a failure
+
+
diff --git a/tests/cloud_tests/__init__.py b/tests/cloud_tests/__init__.py
new file mode 100644
index 0000000..8c1b6d1
--- /dev/null
+++ b/tests/cloud_tests/__init__.py
@@ -0,0 +1,26 @@
+import logging
+import os
+
+BASE_DIR = os.path.dirname(os.path.abspath(__file__))
+TESTCASES_DIR = os.path.join(BASE_DIR, 'testcases')
+TEST_CONF_DIR = os.path.join(BASE_DIR, 'configs')
+
+
+def _initialize_logging():
+    """
+    configure logging for cloud_tests
+    """
+    logger = logging.getLogger(__name__)
+    logger.setLevel(logging.DEBUG)
+    formatter = logging.Formatter(
+        '%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+
+    console = logging.StreamHandler()
+    console.setLevel(logging.DEBUG)
+    console.setFormatter(formatter)
+
+    logger.addHandler(console)
+
+    return logger
+
+LOG = _initialize_logging()
diff --git a/tests/cloud_tests/__main__.py b/tests/cloud_tests/__main__.py
new file mode 100644
index 0000000..dbd4cbd
--- /dev/null
+++ b/tests/cloud_tests/__main__.py
@@ -0,0 +1,89 @@
+import argparse
+import logging
+import shutil
+import sys
+import tempfile
+
+from tests.cloud_tests import (args, collect, manage, verify)
+from tests.cloud_tests import LOG
+
+
+def configure_log(args):
+    """
+    configure logging
+    """
+    level = logging.INFO
+    if args.verbose:
+        level = logging.DEBUG
+    elif args.quiet:
+        level = logging.WARN
+    LOG.setLevel(level)
+
+
+def run(args):
+    """
+    run full test suite
+    """
+    failed = 0
+    args.data_dir = tempfile.mkdtemp(prefix='cloud_test_data_')
+    LOG.debug('using tmpdir %s', args.data_dir)
+    try:
+        failed += collect.collect(args)
+        failed += verify.verify(args)
+    except Exception:
+        failed += 1
+        raise
+    finally:
+        # TODO: make this configurable via environ or cmdline
+        if failed:
+            LOG.warn('some tests failed, leaving data in %s', args.data_dir)
+        else:
+            shutil.rmtree(args.data_dir)
+    return failed
+
+
+def main():
+    """
+    entry point for cloud test suite
+    """
+    # configure parser
+    parser = argparse.ArgumentParser(prog='cloud_tests')
+    subparsers = parser.add_subparsers(dest="subcmd")
+    subparsers.required = True
+
+    def add_subparser(name, description, arg_sets):
+        """
+        add arguments to subparser
+        """
+        subparser = subparsers.add_parser(name, help=description)
+        for (_args, _kwargs) in (a for arg_set in arg_sets for a in arg_set):
+            subparser.add_argument(*_args, **_kwargs)
+
+    # configure subparsers
+    for (name, (description, arg_sets)) in args.SUBCMDS.items():
+        add_subparser(name, description,
+                      [args.ARG_SETS[arg_set] for arg_set in arg_sets])
+
+    # parse arguments
+    parsed = parser.parse_args()
+
+    # process arguments
+    configure_log(parsed)
+    (_, arg_sets) = args.SUBCMDS[parsed.subcmd]
+    for normalizer in [args.NORMALIZERS[arg_set] for arg_set in arg_sets]:
+        parsed = normalizer(parsed)
+        if not parsed:
+            return -1
+
+    # run handler
+    LOG.debug('running with args: %s\n', parsed)
+    return {
+        'collect': collect.collect,
+        'create': manage.create,
+        'run': run,
+        'verify': verify.verify,
+    }[parsed.subcmd](parsed)
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/tests/cloud_tests/args.py b/tests/cloud_tests/args.py
new file mode 100644
index 0000000..6a5c722
--- /dev/null
+++ b/tests/cloud_tests/args.py
@@ -0,0 +1,217 @@
+import os
+
+from tests.cloud_tests import config, util
+from tests.cloud_tests import LOG
+
+ARG_SETS = {
+    'COLLECT': (
+        (('-p', '--platform'),
+         {'help': 'platform(s) to run tests on', 'metavar': 'PLATFORM',
+          'action': 'append', 'choices': config.list_enabled_platforms(),
+          'default': []}),
+        (('-n', '--os-name'),
+         {'help': 'the name(s) of the OS(s) to test', 'metavar': 'NAME',
+          'action': 'append', 'choices': config.list_enabled_distros(),
+          'default': []}),
+        (('-t', '--test-config'),
+         {'help': 'test config file(s) to use', 'metavar': 'FILE',
+          'action': 'append', 'default': []}),),
+    'CREATE': (
+        (('-c', '--config'),
+         {'help': 'cloud-config yaml for testcase', 'metavar': 'DATA',
+          'action': 'store', 'required': False, 'default': None}),
+        (('-e', '--enable'),
+         {'help': 'enable testcase', 'required': False, 'default': False,
+          'action': 'store_true'}),
+        (('name',),
+         {'help': 'testcase name, in format "<category>/<test>"',
+          'action': 'store'}),
+        (('-d', '--description'),
+         {'help': 'description of testcase', 'required': False}),
+        (('-f', '--force'),
+         {'help': 'overwrite already existing test', 'required': False,
+          'action': 'store_true', 'default': False}),),
+    'INTERFACE': (
+        (('-v', '--verbose'),
+         {'help': 'verbose output', 'action': 'store_true', 'default': False}),
+        (('-q', '--quiet'),
+         {'help': 'quiet output', 'action': 'store_true', 'default': False}),),
+    'OUTPUT': (
+        (('-d', '--data-dir'),
+         {'help': 'directory to store test data in',
+          'action': 'store', 'metavar': 'DIR', 'required': True}),),
+    'RESULT': (
+        (('-r', '--result'),
+         {'help': 'file to write results to',
+          'action': 'store', 'metavar': 'FILE'}),),
+    'SETUP': (
+        (('--deb',),
+         {'help': 'install deb', 'metavar': 'FILE', 'action': 'store'}),
+        (('--rpm',),
+         {'help': 'install rpm', 'metavar': 'FILE', 'action': 'store'}),
+        (('--script',),
+         {'help': 'script to set up image', 'metavar': 'DATA',
+          'action': 'store'}),
+        (('--repo',),
+         {'help': 'repo to enable (implies -u)', 'metavar': 'NAME',
+          'action': 'store'}),
+        (('--ppa',),
+         {'help': 'ppa to enable (implies -u)', 'metavar': 'NAME',
+          'action': 'store'}),
+        (('-u', '--upgrade'),
+         {'help': 'upgrade before starting tests', 'action': 'store_true',
+          'default': False}),),
+}
+
+SUBCMDS = {
+    'collect': ('collect test data',
+                ('COLLECT', 'INTERFACE', 'OUTPUT', 'RESULT', 'SETUP')),
+    'create': ('create new test case', ('CREATE', 'INTERFACE')),
+    'run': ('run test suite', ('COLLECT', 'INTERFACE', 'RESULT', 'SETUP')),
+    'verify': ('verify test data', ('INTERFACE', 'OUTPUT', 'RESULT')),
+}
+
+
+def _empty_normalizer(args):
+    """
+    do not normalize arguments
+    """
+    return args
+
+
+def normalize_create_args(args):
+    """
+    normalize CREATE arguments
+    args: parsed args
+    return_value: updated args, or None if errors occurred
+    """
+    # ensure valid name for new test
+    if len(args.name.split('/')) != 2:
+        LOG.error('invalid test name: %s', args.name)
+        return None
+    if os.path.exists(config.name_to_path(args.name)):
+        msg = 'test: {} already exists'.format(args.name)
+        if args.force:
+            LOG.warn('%s but ignoring due to --force', msg)
+        else:
+            LOG.error(msg)
+            return None
+
+    # ensure test config valid if specified
+    if isinstance(args.config, str) and len(args.config) == 0:
+        LOG.error('test config cannot be empty if specified')
+        return None
+
+    # ensure description valid if specified
+    if (isinstance(args.description, str) and
+            (len(args.description) > 70 or len(args.description) == 0)):
+        LOG.error('test description must be between 1 and 70 characters')
+        return None
+
+    return args
+
+
+def normalize_collect_args(args):
+    """
+    normalize COLLECT arguments
+    args: parsed args
+    return_value: updated args, or None if errors occurred
+    """
+    # platform should default to all supported
+    if len(args.platform) == 0:
+        args.platform = config.list_enabled_platforms()
+    args.platform = util.sorted_unique(args.platform)
+
+    # os name should default to all enabled
+    # if os name is provided ensure that all provided are supported
+    if len(args.os_name) == 0:
+        args.os_name = config.list_enabled_distros()
+    else:
+        supported = config.list_enabled_distros()
+        invalid = [os_name for os_name in args.os_name
+                   if os_name not in supported]
+        if len(invalid) != 0:
+            LOG.error('invalid os name(s): %s', invalid)
+            return None
+    args.os_name = util.sorted_unique(args.os_name)
+
+    # test configs should default to all enabled
+    # if test configs are provided, ensure that all provided are valid
+    if len(args.test_config) == 0:
+        args.test_config = config.list_test_configs()
+    else:
+        valid = []
+        invalid = []
+        for name in args.test_config:
+            if os.path.exists(name):
+                valid.append(name)
+            elif os.path.exists(config.name_to_path(name)):
+                valid.append(config.name_to_path(name))
+            else:
+                invalid.append(name)
+        if len(invalid) != 0:
+            LOG.error('invalid test config(s): %s', invalid)
+            return None
+        else:
+            args.test_config = valid
+    args.test_config = util.sorted_unique(args.test_config)
+
+    return args
+
+
+def normalize_output_args(args):
+    """
+    normalize OUTPUT arguments
+    args: parsed args
+    return_value: updated args, or None if errors occurred
+    """
+    if not args.data_dir:
+        LOG.error('--data-dir must be specified')
+        return None
+
+    # ensure clean output dir if collect
+    # ensure data exists if verify
+    if args.subcmd == 'collect':
+        if not util.is_clean_writable_dir(args.data_dir):
+            LOG.error('data_dir must be empty/new and must be writable')
+            return None
+    elif args.subcmd == 'verify':
+        if not os.path.exists(args.data_dir):
+            LOG.error('data_dir %s does not exist', args.data_dir)
+            return None
+
+    return args
+
+
+def normalize_setup_args(args):
+    """
+    normalize SETUP arguments
+    args: parsed args
+    return_value: updated_args, or None if errors occurred
+    """
+    # ensure deb or rpm valid if specified
+    for pkg in (args.deb, args.rpm):
+        if pkg is not None and not os.path.exists(pkg):
+            LOG.error('cannot find package: %s', pkg)
+            return None
+
+    # if repo or ppa to be enabled run upgrade
+    if args.repo or args.ppa:
+        args.upgrade = True
+
+    # if ppa is specified, remove leading 'ppa:' if any
+    _ppa_header = 'ppa:'
+    if args.ppa and args.ppa.startswith(_ppa_header):
+        args.ppa = args.ppa[len(_ppa_header):]
+
+    return args
+
+
+NORMALIZERS = {
+    'COLLECT': normalize_collect_args,
+    'CREATE': normalize_create_args,
+    'INTERFACE': _empty_normalizer,
+    'OUTPUT': normalize_output_args,
+    'RESULT': _empty_normalizer,
+    'SETUP': normalize_setup_args,
+}
diff --git a/tests/cloud_tests/collect.py b/tests/cloud_tests/collect.py
new file mode 100644
index 0000000..f2767e6
--- /dev/null
+++ b/tests/cloud_tests/collect.py
@@ -0,0 +1,157 @@
+from tests.cloud_tests import (config, LOG, setup_image, util)
+from tests.cloud_tests.stage import (PlatformComponent, run_stage, run_single)
+from tests.cloud_tests import (platforms, images, snapshots, instances)
+
+from functools import partial
+import os
+
+
+def collect_script(instance, base_dir, script, script_name):
+    """
+    collect script data
+    instance: instance to run script on
+    base_dir: base directory for output data
+    script: script contents
+    script_name: name of script to run
+    return_value: None, may raise errors
+    """
+    LOG.debug('running collect script: %s', script_name)
+    util.write_file(os.path.join(base_dir, script_name),
+                    instance.run_script(script))
+
+
+def collect_test_data(args, snapshot, os_name, test_name):
+    """
+    collect data for test case
+    args: cmdline arguments
+    snapshot: instantiated snapshot
+    test_name: name or path of test to run
+    return_value: tuple of results and fail count
+    """
+    res = ({}, 1)
+
+    # load test config
+    test_name = config.path_to_name(test_name)
+    test_config = config.load_test_config(test_name)
+    user_data = test_config['cloud_config']
+    test_scripts = test_config['collect_scripts']
+    test_output_dir = os.sep.join(
+        (args.data_dir, snapshot.platform_name, os_name, test_name))
+    boot_timeout = (test_config.get('boot_timeout')
+                    if isinstance(test_config.get('boot_timeout'), int) else
+                    snapshot.config.get('timeout'))
+
+    # if test is not enabled, skip and return 0 failures
+    if not test_config.get('enabled', False):
+        LOG.warn('test config %s is not enabled, skipping', test_name)
+        return ({}, 0)
+
+    # create test instance
+    component = PlatformComponent(
+        partial(instances.get_instance, snapshot, user_data,
+                block=True, start=False, use_desc=test_name))
+
+    LOG.info('collecting test data for test: %s', test_name)
+    with component as instance:
+        start_call = partial(run_single, 'boot instance', partial(
+            instance.start, wait=True, wait_time=boot_timeout))
+        collect_calls = [partial(run_single, 'script {}'.format(script_name),
+                                 partial(collect_script, instance,
+                                         test_output_dir, script, script_name))
+                         for script_name, script in test_scripts.items()]
+
+        res = run_stage('collect for test: {}'.format(test_name),
+                        [start_call] + collect_calls)
+
+    return res
+
+
+def collect_snapshot(args, image, os_name):
+    """
+    collect data for snapshot of image
+    args: cmdline arguments
+    image: instantiated image with set up complete
+    return_value tuple of results and fail count
+    """
+    res = ({}, 1)
+
+    component = PlatformComponent(partial(snapshots.get_snapshot, image))
+
+    LOG.debug('creating snapshot for %s', os_name)
+    with component as snapshot:
+        LOG.info('collecting test data for os: %s', os_name)
+        res = run_stage(
+            'collect test data for {}'.format(os_name),
+            [partial(collect_test_data, args, snapshot, os_name, test_name)
+             for test_name in args.test_config])
+
+    return res
+
+
+def collect_image(args, platform, os_name):
+    """
+    collect data for image
+    args: cmdline arguments
+    platform: instantiated platform
+    os_name: name of distro to collect for
+    return_value: tuple of results and fail count
+    """
+    res = ({}, 1)
+
+    os_config = config.load_os_config(os_name)
+    if not os_config.get('enabled'):
+        raise ValueError('OS {} not enabled'.format(os_name))
+
+    component = PlatformComponent(
+        partial(images.get_image, platform, os_config))
+
+    LOG.info('acquiring image for os: %s', os_name)
+    with component as image:
+        res = run_stage('set up and collect data for os: {}'.format(os_name),
+                        [partial(setup_image.setup_image, args, image)] +
+                        [partial(collect_snapshot, args, image, os_name)],
+                        continue_after_error=False)
+
+    return res
+
+
+def collect_platform(args, platform_name):
+    """
+    collect data for platform
+    args: cmdline arguments
+    platform_name: platform to collect for
+    return_value: tuple of results and fail count
+    """
+    res = ({}, 1)
+
+    platform_config = config.load_platform_config(platform_name)
+    if not platform_config.get('enabled'):
+        raise ValueError('Platform {} not enabled'.format(platform_name))
+
+    component = PlatformComponent(
+        partial(platforms.get_platform, platform_name, platform_config))
+
+    LOG.info('setting up platform: %s', platform_name)
+    with component as platform:
+        res = run_stage('collect for platform: {}'.format(platform_name),
+                        [partial(collect_image, args, platform, os_name)
+                         for os_name in args.os_name])
+
+    return res
+
+
+def collect(args):
+    """
+    entry point for collection
+    args: cmdline arguments
+    return_value: fail count
+    """
+    (res, failed) = run_stage(
+        'collect data', [partial(collect_platform, args, platform_name)
+                         for platform_name in args.platform])
+
+    LOG.debug('collect stages: %s', res)
+    if args.result:
+        util.merge_results({'collect_stages': res}, args.result)
+
+    return failed
diff --git a/tests/cloud_tests/config.py b/tests/cloud_tests/config.py
new file mode 100644
index 0000000..d37eb75
--- /dev/null
+++ b/tests/cloud_tests/config.py
@@ -0,0 +1,109 @@
+import glob
+import os
+
+from cloudinit import util as c_util
+from tests.cloud_tests import (BASE_DIR, TEST_CONF_DIR)
+
+# conf files
+CONF_EXT = '.yaml'
+VERIFY_EXT = '.py'
+PLATFORM_CONF = os.path.join(BASE_DIR, 'platforms.yaml')
+RELEASES_CONF = os.path.join(BASE_DIR, 'releases.yaml')
+TESTCASE_CONF = os.path.join(BASE_DIR, 'testcases.yaml')
+
+
+def path_to_name(path):
+    """
+    convert abs or rel path to test config to path under configs/
+    if already a test name, do nothing
+    """
+    dir_path, file_name = os.path.split(os.path.normpath(path))
+    name = os.path.splitext(file_name)[0]
+    return os.sep.join((os.path.basename(dir_path), name))
+
+
+def name_to_path(name):
+    """
+    convert test config path under configs/ to full config path,
+    if already a full path, do nothing
+    """
+    name = os.path.normpath(name)
+    if not name.endswith(CONF_EXT):
+        name = name + CONF_EXT
+    return name if os.path.isabs(name) else os.path.join(TEST_CONF_DIR, name)
+
+
+def name_sanatize(name):
+    """
+    sanatize test name to be used as a module name
+    """
+    return name.replace('-', '_')
+
+
+def name_to_module(name):
+    """
+    convert test name to a loadable module name under testcases/
+    """
+    name = name_sanatize(path_to_name(name))
+    return name.replace(os.path.sep, '.')
+
+
+def merge_config(base, override):
+    """
+    merge config and base
+    """
+    res = base.copy()
+    res.update(override)
+    res.update({k: merge_config(base.get(k, {}), v)
+                for k, v in override.items() if isinstance(v, dict)})
+    return res
+
+
+def load_platform_config(platform):
+    """
+    load configuration for platform
+    """
+    main_conf = c_util.read_conf(PLATFORM_CONF)
+    return merge_config(main_conf.get('default_platform_config'),
+                        main_conf.get('platforms')[platform])
+
+
+def load_os_config(os_name):
+    """
+    load configuration for os
+    """
+    main_conf = c_util.read_conf(RELEASES_CONF)
+    return merge_config(main_conf.get('default_release_config'),
+                        main_conf.get('releases')[os_name])
+
+
+def load_test_config(path):
+    """
+    load a test config file by either abs path or rel path
+    """
+    return merge_config(c_util.read_conf(TESTCASE_CONF)['base_test_data'],
+                        c_util.read_conf(name_to_path(path)))
+
+
+def list_enabled_platforms():
+    """
+    list all platforms enabled for testing
+    """
+    platforms = c_util.read_conf(PLATFORM_CONF).get('platforms')
+    return [k for k, v in platforms.items() if v.get('enabled')]
+
+
+def list_enabled_distros():
+    """
+    list all distros enabled for testing
+    """
+    releases = c_util.read_conf(RELEASES_CONF).get('releases')
+    return [k for k, v in releases.items() if v.get('enabled')]
+
+
+def list_test_configs():
+    """
+    list all available test config files by abspath
+    """
+    return [os.path.abspath(f) for f in
+            glob.glob(os.sep.join((TEST_CONF_DIR, '*', '*.yaml')))]
diff --git a/tests/cloud_tests/configs/bugs/README.md b/tests/cloud_tests/configs/bugs/README.md
new file mode 100644
index 0000000..269059a
--- /dev/null
+++ b/tests/cloud_tests/configs/bugs/README.md
@@ -0,0 +1,11 @@
+# Bug Test Configs
+
+## purpose
+Configs that reproduce bugs filed against cloud-init. Having test configs for
+cloud-init bugs ensures that the fixes do not break in the future, and makes it
+easy to see how many systems and platforms are effected by a new bug.
+
+## structure
+Should have one test config for most bugs filed. The name of the test should
+contain ``lp`` followed by the bug number. It may also be useful to add a
+comment to each bug config with a summary copied from the bug report.
diff --git a/tests/cloud_tests/configs/bugs/lp1511485.yaml b/tests/cloud_tests/configs/bugs/lp1511485.yaml
new file mode 100644
index 0000000..846162f
--- /dev/null
+++ b/tests/cloud_tests/configs/bugs/lp1511485.yaml
@@ -0,0 +1,9 @@
+#
+# LP Bug 1511485: final_message is silent on ubuntu-12.04.5 / cloud-init 0.6.3
+#
+# 2016-11-17: Disabled as covered by module based tests
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  final_message: "Final message from cloud-config"
diff --git a/tests/cloud_tests/configs/bugs/lp1611074.yaml b/tests/cloud_tests/configs/bugs/lp1611074.yaml
new file mode 100644
index 0000000..94005e8
--- /dev/null
+++ b/tests/cloud_tests/configs/bugs/lp1611074.yaml
@@ -0,0 +1,6 @@
+#
+# LP Bug 1611074: Reformatting of ephemeral drive fails on resize of Azure VM
+#
+# 2016-11-18: Disabled until test written
+#
+enabled: False
diff --git a/tests/cloud_tests/configs/bugs/lp1628337.yaml b/tests/cloud_tests/configs/bugs/lp1628337.yaml
new file mode 100644
index 0000000..251662a
--- /dev/null
+++ b/tests/cloud_tests/configs/bugs/lp1628337.yaml
@@ -0,0 +1,18 @@
+#
+# LP Bug 1628337: cloud-init tries to install NTP before even configuring the archives
+#
+cloud_config: |
+  #cloud-config
+  ntp:
+    servers: ['ntp.ubuntu.com']
+  apt:
+    primary:
+      - arches: [default]
+        uri: http://us.archive.ubuntu.com/ubuntu/
+collect_sciprts:
+  ntp.conf: |
+    #!/bin/bash
+    cat /etc/ntp.conf
+  sources.list: |
+    #!/bin/bash
+    cat /etc/apt/sources.list
diff --git a/tests/cloud_tests/configs/examples/README.md b/tests/cloud_tests/configs/examples/README.md
new file mode 100644
index 0000000..73b8fce
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/README.md
@@ -0,0 +1,10 @@
+# Example Test Configs
+
+## Purpose
+This folder contains example cloud configs found on
+[cloudinit.readthedocs.io](https://cloudinit.readthedocs.io/en/latest/topics/examples.html).
+Examples covered by other tests, like modules, are excluded from tests here
+to prevent duplication and reduce test time.
+
+## Structure
+One test per example test config on cloudinit.readthedocs.io
diff --git a/tests/cloud_tests/configs/examples/TODO.md b/tests/cloud_tests/configs/examples/TODO.md
new file mode 100644
index 0000000..66404cb
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/TODO.md
@@ -0,0 +1,13 @@
+# Missing Examples
+
+Below lists each of the issing examples and why it is not currently added.
+
+ - Chef (takes > 60 seconds to run)
+ - Puppet (takes > 60 seconds to run)
+ - Manage resolve.conf (lxd backend overrides changes)
+ - Adding a yum repository (need centos system)
+ - Register RedHat Subscription (need centos system + subscription)
+ - Adjust mount points mounted (need multiple disks)
+ - Call a url when finished (need end point)
+ - Reboot/poweroff when finished (how to test)
+ - Disk setup (need multiple disks)
diff --git a/tests/cloud_tests/configs/examples/add_apt_repositories.yaml b/tests/cloud_tests/configs/examples/add_apt_repositories.yaml
new file mode 100644
index 0000000..91ca1a6
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/add_apt_repositories.yaml
@@ -0,0 +1,19 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as covered by module based tests
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  apt:
+    primary:
+      - arches: [default]
+        uri: "http://www.gtlib.gatech.edu/pub/ubuntu-releases/"
+collect_scripts:
+  ubuntu.sources.list: |
+    #!/bin/bash
+    cat /etc/apt/sources.list | grep -v '^#' | sed '/^\s*$/d' | grep archive.ubuntu.com | wc -l
+  gatech.sources.list: |
+    #!/bin/bash
+    cat /etc/apt/sources.list | grep -v '^#' | sed '/^\s*$/d' | grep gtlib.gatech.edu | wc -l
diff --git a/tests/cloud_tests/configs/examples/alter_completion_message.yaml b/tests/cloud_tests/configs/examples/alter_completion_message.yaml
new file mode 100644
index 0000000..a5aa44b
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/alter_completion_message.yaml
@@ -0,0 +1,14 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as covered by module based tests
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  final_message: |
+    This is my final message!
+    $version
+    $timestamp
+    $datasource
+    $uptime
diff --git a/tests/cloud_tests/configs/examples/configure_instance_trusted_ca_certificates.yaml b/tests/cloud_tests/configs/examples/configure_instance_trusted_ca_certificates.yaml
new file mode 100644
index 0000000..d0f7794
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/configure_instance_trusted_ca_certificates.yaml
@@ -0,0 +1,39 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as covered by module based tests
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+    ca-certs:
+      # If present and set to True, the 'remove-defaults' parameter will remove
+      # all the default trusted CA certificates that are normally shipped with
+      # Ubuntu.
+      # This is mainly for paranoid admins - most users will not need this
+      # functionality.
+      remove-defaults: true
+
+      # If present, the 'trusted' parameter should contain a certificate (or list
+      # of certificates) to add to the system as trusted CA certificates.
+      # Pay close attention to the YAML multiline list syntax.  The example shown
+      # here is for a list of multiline certificates.
+      trusted:
+      - |
+       -----BEGIN CERTIFICATE-----
+       YOUR-ORGS-TRUSTED-CA-CERT-HERE
+       -----END CERTIFICATE-----
+      - |
+       -----BEGIN CERTIFICATE-----
+       YOUR-ORGS-TRUSTED-CA-CERT-HERE
+       -----END CERTIFICATE-----
+collect_scripts:
+  cloudinit_certs: |
+    #!/bin/bash
+    cat /etc/ssl/certs/cloud-init-ca-certs.pem
+  cert_count_ca: |
+    #!/bin/bash
+    wc -l /etc/ssl/certs/ca-certificates.crt
+  cert_count_cloudinit: |
+    #!/bin/bash
+    wc -l /etc/ssl/certs/cloud-init-ca-certs.pem
diff --git a/tests/cloud_tests/configs/examples/configure_instances_ssh_keys.yaml b/tests/cloud_tests/configs/examples/configure_instances_ssh_keys.yaml
new file mode 100644
index 0000000..c2f03be
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/configure_instances_ssh_keys.yaml
@@ -0,0 +1,61 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as covered by module based tests
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  ssh_authorized_keys:
+    - ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAGEA3FSyQwBI6Z+nCSjUUk8EEAnnkhXlukKoUPND/RRClWz2s5TCzIkd3Ou5+Cyz71X0XmazM3l5WgeErvtIwQMyT1KjNoMhoJMrJnWqQPOt5Q8zWd9qG7PBl9+eiH5qV7NZ mykey@host
+    - ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA3I7VUf2l5gSn5uavROsc5HRDpZdQueUq5ozemNSj8T7enqKHOEaFoU2VoPgGEWC9RyzSQVeyD6s7APMcE82EtmW4skVEgEGSbDc1pvxzxtchBj78hJP6Cf5TCMFSXw+Fz5rF1dR23QDbN1mkHs7adr8GW4kSWqU7Q7NDwfIrJJtO7Hi42GyXtvEONHbiRPOe8stqUly7MvUoN+5kfjBM8Qqpfl2+FNhTYWpMfYdPUnE7u536WqzFmsaqJctz3gBxH9Ex7dFtrxR4qiqEr9Qtlu3xGn7Bw07/+i1D+ey3ONkZLN+LQ714cgj8fRS4Hj29SCmXp5Kt5/82cD/VN3NtHw== smoser@brickies
+
+  # Send pre-generated ssh private keys to the server
+  # If these are present, they will be written to /etc/ssh and
+  # new random keys will not be generated
+  #  in addition to 'rsa' and 'dsa' as shown below, 'ecdsa' is also supported
+  ssh_keys:
+    rsa_private: |
+      -----BEGIN RSA PRIVATE KEY-----
+      MIIBxwIBAAJhAKD0YSHy73nUgysO13XsJmd4fHiFyQ+00R7VVu2iV9Qcon2LZS/x
+      1cydPZ4pQpfjEha6WxZ6o8ci/Ea/w0n+0HGPwaxlEG2Z9inNtj3pgFrYcRztfECb
+      1j6HCibZbAzYtwIBIwJgO8h72WjcmvcpZ8OvHSvTwAguO2TkR6mPgHsgSaKy6GJo
+      PUJnaZRWuba/HX0KGyhz19nPzLpzG5f0fYahlMJAyc13FV7K6kMBPXTRR6FxgHEg
+      L0MPC7cdqAwOVNcPY6A7AjEA1bNaIjOzFN2sfZX0j7OMhQuc4zP7r80zaGc5oy6W
+      p58hRAncFKEvnEq2CeL3vtuZAjEAwNBHpbNsBYTRPCHM7rZuG/iBtwp8Rxhc9I5w
+      ixvzMgi+HpGLWzUIBS+P/XhekIjPAjA285rVmEP+DR255Ls65QbgYhJmTzIXQ2T9
+      luLvcmFBC6l35Uc4gTgg4ALsmXLn71MCMGMpSWspEvuGInayTCL+vEjmNBT+FAdO
+      W7D4zCpI43jRS9U06JVOeSc9CDk2lwiA3wIwCTB/6uc8Cq85D9YqpM10FuHjKpnP
+      REPPOyrAspdeOAV+6VKRavstea7+2DZmSUgE
+      -----END RSA PRIVATE KEY-----
+
+    rsa_public: ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAGEAoPRhIfLvedSDKw7XdewmZ3h8eIXJD7TRHtVW7aJX1ByifYtlL/HVzJ09nilCl+MSFrpbFnqjxyL8Rr/DSf7QcY/BrGUQbZn2Kc22PemAWthxHO18QJvWPocKJtlsDNi3 smoser@localhost
+
+    dsa_private: |
+      -----BEGIN DSA PRIVATE KEY-----
+      MIIBuwIBAAKBgQDP2HLu7pTExL89USyM0264RCyWX/CMLmukxX0Jdbm29ax8FBJT
+      pLrO8TIXVY5rPAJm1dTHnpuyJhOvU9G7M8tPUABtzSJh4GVSHlwaCfycwcpLv9TX
+      DgWIpSj+6EiHCyaRlB1/CBp9RiaB+10QcFbm+lapuET+/Au6vSDp9IRtlQIVAIMR
+      8KucvUYbOEI+yv+5LW9u3z/BAoGBAI0q6JP+JvJmwZFaeCMMVxXUbqiSko/P1lsa
+      LNNBHZ5/8MOUIm8rB2FC6ziidfueJpqTMqeQmSAlEBCwnwreUnGfRrKoJpyPNENY
+      d15MG6N5J+z81sEcHFeprryZ+D3Ge9VjPq3Tf3NhKKwCDQ0240aPezbnjPeFm4mH
+      bYxxcZ9GAoGAXmLIFSQgiAPu459rCKxT46tHJtM0QfnNiEnQLbFluefZ/yiI4DI3
+      8UzTCOXLhUA7ybmZha+D/csj15Y9/BNFuO7unzVhikCQV9DTeXX46pG4s1o23JKC
+      /QaYWNMZ7kTRv+wWow9MhGiVdML4ZN4XnifuO5krqAybngIy66PMEoQCFEIsKKWv
+      99iziAH0KBMVbxy03Trz
+      -----END DSA PRIVATE KEY-----
+
+    dsa_public: ssh-dsa AAAAB3NzaC1kc3MAAACBAM/Ycu7ulMTEvz1RLIzTbrhELJZf8Iwua6TFfQl1ubb1rHwUElOkus7xMhdVjms8AmbV1Meem7ImE69T0bszy09QAG3NImHgZVIeXBoJ/JzByku/1NcOBYilKP7oSIcLJpGUHX8IGn1GJoH7XRBwVub6Vqm4RP78C7q9IOn0hG2VAAAAFQCDEfCrnL1GGzhCPsr/uS1vbt8/wQAAAIEAjSrok/4m8mbBkVp4IwxXFdRuqJKSj8/WWxos00Ednn/ww5QibysHYULrOKJ1+54mmpMyp5CZICUQELCfCt5ScZ9GsqgmnI80Q1h3Xkwbo3kn7PzWwRwcV6muvJn4PcZ71WM+rdN/c2EorAINDTbjRo97NueM94WbiYdtjHFxn0YAAACAXmLIFSQgiAPu459rCKxT46tHJtM0QfnNiEnQLbFluefZ/yiI4DI38UzTCOXLhUA7ybmZha+D/csj15Y9/BNFuO7unzVhikCQV9DTeXX46pG4s1o23JKC/QaYWNMZ7kTRv+wWow9MhGiVdML4ZN4XnifuO5krqAybngIy66PMEoQ= smoser@localhost
+collect_scripts:
+  cert_count: |
+    #!/bin/bash
+    ls | wc -l
+  dsa_public: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_dsa_key.pub
+  rsa_public: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_rsa_key.pub
+  auth_keys: |
+    #!/bin/bash
+    cat /home/ubuntu/.ssh/authorized_keys
diff --git a/tests/cloud_tests/configs/examples/including_user_groups.yaml b/tests/cloud_tests/configs/examples/including_user_groups.yaml
new file mode 100644
index 0000000..67e4fea
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/including_user_groups.yaml
@@ -0,0 +1,51 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as covered by module based tests
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  # Add groups to the system
+  groups:
+    - secret: [foobar,barfoo]
+    - cloud-users
+
+  # Add users to the system. Users are added after groups are added.
+  users:
+    - default
+    - name: foobar
+      gecos: Foo B. Bar
+      primary-group: foobar
+      groups: users
+      expiredate: 2038-01-19
+      lock_passwd: false
+      passwd: $6$j212wezy$7H/1LT4f9/N3wpgNunhsIqtMj62OKiS3nyNwuizouQc3u7MbYCarYeAHWYPYb2FT.lbioDm2RrkJPb9BZMN1O/
+    - name: barfoo
+      gecos: Bar B. Foo
+      sudo: ALL=(ALL) NOPASSWD:ALL
+      groups: cloud-users
+      lock_passwd: true
+    - name: cloudy
+      gecos: Magic Cloud App Daemon User
+      inactive: true
+      system: true
+collect_scripts:
+  group_ubuntu: |
+    #!/bin/bash
+    getent group ubuntu
+  group_cloud_users: |
+    #!/bin/bash
+    getent group cloud-users
+  user_ubuntu: |
+    #!/bin/bash
+    getent passwd ubuntu
+  user_foobar: |
+    #!/bin/bash
+    getent passwd foobar
+  user_barfoo: |
+    #!/bin/bash
+    getent passwd barfoo
+  user_cloudy: |
+    #!/bin/bash
+    getent passwd cloudy
diff --git a/tests/cloud_tests/configs/examples/install_arbitrary_packages.yaml b/tests/cloud_tests/configs/examples/install_arbitrary_packages.yaml
new file mode 100644
index 0000000..65ed649
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/install_arbitrary_packages.yaml
@@ -0,0 +1,18 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as covered by module based tests
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  packages:
+   - htop
+   - tree
+collect_scripts:
+  htop: |
+    #!/bin/bash
+    dpkg -l | grep htop | wc -l
+  tree: |
+    #!/bin/bash
+    dpkg -l | grep tree | wc -l
diff --git a/tests/cloud_tests/configs/examples/install_run_chef_recipes.yaml b/tests/cloud_tests/configs/examples/install_run_chef_recipes.yaml
new file mode 100644
index 0000000..7b1622b
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/install_run_chef_recipes.yaml
@@ -0,0 +1,92 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as test suite fails this long running test currently
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  # Key from http://apt.opscode.com/[e-mail]
+  apt:
+    sources:
+     - source: "deb http://apt.opscode.com/ $RELEASE-0.10 main"
+       key: |
+         -----BEGIN PGP PUBLIC KEY BLOCK-----
+         Version: GnuPG v1.4.9 (GNU/Linux)
+
+         mQGiBEppC7QRBADfsOkZU6KZK+YmKw4wev5mjKJEkVGlus+NxW8wItX5sGa6kdUu
+         twAyj7Yr92rF+ICFEP3gGU6+lGo0Nve7KxkN/1W7/m3G4zuk+ccIKmjp8KS3qn99
+         dxy64vcji9jIllVa+XXOGIp0G8GEaj7mbkixL/bMeGfdMlv8Gf2XPpp9vwCgn/GC
+         JKacfnw7MpLKUHOYSlb//JsEAJqao3ViNfav83jJKEkD8cf59Y8xKia5OpZqTK5W
+         ShVnNWS3U5IVQk10ZDH97Qn/YrK387H4CyhLE9mxPXs/ul18ioiaars/q2MEKU2I
+         XKfV21eMLO9LYd6Ny/Kqj8o5WQK2J6+NAhSwvthZcIEphcFignIuobP+B5wNFQpe
+         DbKfA/0WvN2OwFeWRcmmd3Hz7nHTpcnSF+4QX6yHRF/5BgxkG6IqBIACQbzPn6Hm
+         sMtm/SVf11izmDqSsQptCrOZILfLX/mE+YOl+CwWSHhl+YsFts1WOuh1EhQD26aO
+         Z84HuHV5HFRWjDLw9LriltBVQcXbpfSrRP5bdr7Wh8vhqJTPjrQnT3BzY29kZSBQ
+         YWNrYWdlcyA8cGFja2FnZXNAb3BzY29kZS5jb20+iGAEExECACAFAkppC7QCGwMG
+         CwkIBwMCBBUCCAMEFgIDAQIeAQIXgAAKCRApQKupg++Caj8sAKCOXmdG36gWji/K
+         +o+XtBfvdMnFYQCfTCEWxRy2BnzLoBBFCjDSK6sJqCu5Ag0ESmkLtBAIAIO2SwlR
+         lU5i6gTOp42RHWW7/pmW78CwUqJnYqnXROrt3h9F9xrsGkH0Fh1FRtsnncgzIhvh
+         DLQnRHnkXm0ws0jV0PF74ttoUT6BLAUsFi2SPP1zYNJ9H9fhhK/pjijtAcQwdgxu
+         wwNJ5xCEscBZCjhSRXm0d30bK1o49Cow8ZIbHtnXVP41c9QWOzX/LaGZsKQZnaMx
+         EzDk8dyyctR2f03vRSVyTFGgdpUcpbr9eTFVgikCa6ODEBv+0BnCH6yGTXwBid9g
+         w0o1e/2DviKUWCC+AlAUOubLmOIGFBuI4UR+rux9affbHcLIOTiKQXv79lW3P7W8
+         AAfniSQKfPWXrrcAAwUH/2XBqD4Uxhbs25HDUUiM/m6Gnlj6EsStg8n0nMggLhuN
+         QmPfoNByMPUqvA7sULyfr6xCYzbzRNxABHSpf85FzGQ29RF4xsA4vOOU8RDIYQ9X
+         Q8NqqR6pydprRFqWe47hsAN7BoYuhWqTtOLSBmnAnzTR5pURoqcquWYiiEavZixJ
+         3ZRAq/HMGioJEtMFrvsZjGXuzef7f0ytfR1zYeLVWnL9Bd32CueBlI7dhYwkFe+V
+         Ep5jWOCj02C1wHcwt+uIRDJV6TdtbIiBYAdOMPk15+VBdweBXwMuYXr76+A7VeDL
+         zIhi7tKFo6WiwjKZq0dzctsJJjtIfr4K4vbiD9Ojg1iISQQYEQIACQUCSmkLtAIb
+         DAAKCRApQKupg++CauISAJ9CxYPOKhOxalBnVTLeNUkAHGg2gACeIsbobtaD4ZHG
+         0GLl8EkfA8uhluM=
+         =zKAm
+         -----END PGP PUBLIC KEY BLOCK-----
+
+  chef:
+
+   # Valid values are 'gems' and 'packages' and 'omnibus'
+   install_type: "packages"
+
+   # Boolean: run 'install_type' code even if chef-client
+   #          appears already installed.
+   force_install: false
+
+   # Chef settings
+   server_url: "https://chef.yourorg.com:4000"
+
+   # Node Name
+   # Defaults to the instance-id if not present
+   node_name: "your-node-name"
+
+   # Environment
+   # Defaults to '_default' if not present
+   environment: "production"
+
+   # Default validation name is chef-validator
+   validation_name: "yourorg-validator"
+   # if validation_cert's value is "system" then it is expected
+   # that the file already exists on the system.
+   validation_cert: |
+       -----BEGIN RSA PRIVATE KEY-----
+       YOUR-ORGS-VALIDATION-KEY-HERE
+       -----END RSA PRIVATE KEY-----
+
+   # A run list for a first boot json
+   run_list:
+    - "recipe[apache2]"
+    - "role[db]"
+
+   # Specify a list of initial attributes used by the cookbooks
+   initial_attributes:
+      apache:
+        prefork:
+          maxclients: 100
+        keepalive: "off"
+
+   # if install_type is 'omnibus', change the url to download
+   omnibus_url: "https://www.opscode.com/chef/install.sh"
+
+
+  # Capture all subprocess output into a logfile
+  # Useful for troubleshooting cloud-init issues
+  output: {all: '| tee -a /var/log/cloud-init-output.log'}
diff --git a/tests/cloud_tests/configs/examples/run_apt_upgrade.yaml b/tests/cloud_tests/configs/examples/run_apt_upgrade.yaml
new file mode 100644
index 0000000..b7aa53f
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/run_apt_upgrade.yaml
@@ -0,0 +1,9 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as covered by module based tests
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  package_upgrade: true
diff --git a/tests/cloud_tests/configs/examples/run_commands.yaml b/tests/cloud_tests/configs/examples/run_commands.yaml
new file mode 100644
index 0000000..7305048
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/run_commands.yaml
@@ -0,0 +1,14 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as covered by module based tests
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  runcmd:
+   - echo cloud-init run cmd test > /tmp/run_cmd
+collect_scripts:
+  run_cmd: |
+    #!/bin/bash
+    cat /tmp/run_cmd
diff --git a/tests/cloud_tests/configs/examples/run_commands_first_boot.yaml b/tests/cloud_tests/configs/examples/run_commands_first_boot.yaml
new file mode 100644
index 0000000..8a8467e
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/run_commands_first_boot.yaml
@@ -0,0 +1,14 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as covered by module based tests
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  bootcmd:
+   - echo 192.168.1.130 us.archive.ubuntu.com > /etc/hosts
+collect_scripts:
+  hosts: |
+    #!/bin/bash
+    cat /etc/hosts
diff --git a/tests/cloud_tests/configs/examples/setup_run_puppet.yaml b/tests/cloud_tests/configs/examples/setup_run_puppet.yaml
new file mode 100644
index 0000000..abd2177
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/setup_run_puppet.yaml
@@ -0,0 +1,53 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as test suite fails this long running test currently
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  puppet:
+   # Every key present in the conf object will be added to puppet.conf:
+   # [name]
+   # subkey=value
+   #
+   # For example the configuration below will have the following section
+   # added to puppet.conf:
+   # [puppetd]
+   # server=puppetmaster.example.org
+   # certname=i-0123456.ip-X-Y-Z.cloud.internal
+   #
+   # The puppmaster ca certificate will be available in
+   # /var/lib/puppet/ssl/certs/ca.pem
+   conf:
+     agent:
+       server: "puppetmaster.example.org"
+       # certname supports substitutions at runtime:
+       #   %i: instanceid
+       #       Example: i-0123456
+       #   %f: fqdn of the machine
+       #       Example: ip-X-Y-Z.cloud.internal
+       #
+       # NB: the certname will automatically be lowercased as required by puppet
+       certname: "%i.%f"
+     # ca_cert is a special case. It won't be added to puppet.conf.
+     # It holds the puppetmaster certificate in pem format.
+     # It should be a multi-line string (using the | yaml notation for
+     # multi-line strings).
+     # The puppetmaster certificate is located in
+     # /var/lib/puppet/ssl/ca/ca_crt.pem on the puppetmaster host.
+     #
+     ca_cert: |
+       -----BEGIN CERTIFICATE-----
+       MIICCTCCAXKgAwIBAgIBATANBgkqhkiG9w0BAQUFADANMQswCQYDVQQDDAJjYTAe
+       Fw0xMDAyMTUxNzI5MjFaFw0xNTAyMTQxNzI5MjFaMA0xCzAJBgNVBAMMAmNhMIGf
+       MA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCu7Q40sm47/E1Pf+r8AYb/V/FWGPgc
+       b014OmNoX7dgCxTDvps/h8Vw555PdAFsW5+QhsGr31IJNI3kSYprFQcYf7A8tNWu
+       1MASW2CfaEiOEi9F1R3R4Qlz4ix+iNoHiUDTjazw/tZwEdxaQXQVLwgTGRwVa+aA
+       qbutJKi93MILLwIDAQABo3kwdzA4BglghkgBhvhCAQ0EKxYpUHVwcGV0IFJ1Ynkv
+       T3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwDwYDVR0TAQH/BAUwAwEB/zAd
+       BgNVHQ4EFgQUu4+jHB+GYE5Vxo+ol1OAhevspjAwCwYDVR0PBAQDAgEGMA0GCSqG
+       SIb3DQEBBQUAA4GBAH/rxlUIjwNb3n7TXJcDJ6MMHUlwjr03BDJXKb34Ulndkpaf
+       +GAlzPXWa7bO908M9I8RnPfvtKnteLbvgTK+h+zX1XCty+S2EQWk29i2AdoqOTxb
+       hppiGMp0tT5Havu4aceCXiy2crVcudj3NFciy8X66SoECemW9UYDCb9T5D0d
+       -----END CERTIFICATE-----
diff --git a/tests/cloud_tests/configs/examples/writing_out_arbitrary_files.yaml b/tests/cloud_tests/configs/examples/writing_out_arbitrary_files.yaml
new file mode 100644
index 0000000..37c5fb8
--- /dev/null
+++ b/tests/cloud_tests/configs/examples/writing_out_arbitrary_files.yaml
@@ -0,0 +1,43 @@
+#
+# From cloud config examples on cloudinit.readthedocs.io
+#
+# 2016-11-17: Disabled as covered by module based tests
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  write_files:
+  -   encoding: b64
+      content: CiMgVGhpcyBmaWxlIGNvbnRyb2xzIHRoZSBzdGF0ZSBvZiBTRUxpbnV4
+      owner: root:root
+      path: /root/file_b64
+      permissions: '0644'
+  -   content: |
+          # My new /root/file_text
+
+          SMBDOPTIONS="-D"
+      path: /root/file_text
+  -   content: !!binary |
+          f0VMRgIBAQAAAAAAAAAAAAIAPgABAAAAwARAAAAAAABAAAAAAAAAAJAVAAAAAAAAAAAAAEAAOAAI
+          AEAAHgAdAAYAAAAFAAAAQAAAAAAAAABAAEAAAAAAAEAAQAAAAAAAwAEAAAAAAADAAQAAAAAAAAgA
+          AAAAAAAAAwAAAAQAAAAAAgAAAAAAAAACQAAAAAAAAAJAAAAAAAAcAAAAAAAAABwAAAAAAAAAAQAA
+      path: /root/file_binary
+      permissions: '0555'
+  -   encoding: gzip
+      content: !!binary |
+          H4sIAIDb/U8C/1NW1E/KzNMvzuBKTc7IV8hIzcnJVyjPL8pJ4QIA6N+MVxsAAAA=
+      path: /root/file_gzip
+      permissions: '0755'
+collect_scripts:
+  file_b64: |
+    #!/bin/bash
+    file /root/file_b64
+  file_text: |
+    #!/bin/bash
+    file /root/file_text
+  file_binary: |
+    #!/bin/bash
+    file /root/file_binary
+  file_gzip: |
+    #!/bin/bash
+    file /root/file_gzip
diff --git a/tests/cloud_tests/configs/main/README.md b/tests/cloud_tests/configs/main/README.md
new file mode 100644
index 0000000..1418e7f
--- /dev/null
+++ b/tests/cloud_tests/configs/main/README.md
@@ -0,0 +1,9 @@
+# Main Functionality Test Configs
+
+## purpose
+Test main features and config options of cloud-init such as logging, output
+redirection, early init and integration with init system
+
+## structure
+Should have one or more test configs for all main cloud-init output and logging
+options, and basic functionality test cases
diff --git a/tests/cloud_tests/configs/main/command_output_simple.yaml b/tests/cloud_tests/configs/main/command_output_simple.yaml
new file mode 100644
index 0000000..e418ac4
--- /dev/null
+++ b/tests/cloud_tests/configs/main/command_output_simple.yaml
@@ -0,0 +1,11 @@
+#
+# Test functionality of simple output redirection
+#
+cloud_config: |
+    #cloud-config
+    output: { all: "| tee -a /var/log/cloud-init-test-output" }
+    final_message: "should be last line in cloud-init-test-output file"
+collect_scripts:
+    cloud-init-test-output: |
+        #!/bin/bash
+        cat /var/log/cloud-init-test-output
diff --git a/tests/cloud_tests/configs/modules/README.md b/tests/cloud_tests/configs/modules/README.md
new file mode 100644
index 0000000..91d2964
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/README.md
@@ -0,0 +1,10 @@
+# Module Test Configs
+
+## Purpose
+Test functionality of cloud config modules. See
+[here](https://cloudinit.readthedocs.io/en/latest/topics/modules.html) for
+a full list.
+
+## Structure
+Should have one or more test configs for each module in cloudinit/config/. The
+name of the test should indicate which module the config is verifying.
diff --git a/tests/cloud_tests/configs/modules/TODO.md b/tests/cloud_tests/configs/modules/TODO.md
new file mode 100644
index 0000000..8a9e162
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/TODO.md
@@ -0,0 +1,98 @@
+# TODO
+
+The following lists complete or partially misisng modules. If a module is
+listed with nothing below it indicates that no work is completed on that
+module. If there is a list below the module name that is the remainig
+identified work.
+
+## apt_configure
+
+  * apt_get_wrapper
+    * What does this do? How to use it?
+  * apt_get_command
+    * To specify a different 'apt-get' command, set 'apt_get_command'.
+    This must be a list, and the subcommand (update, upgrade) is appended to it.
+    * Modify default and verify the options got passed correctly.
+  * preserve sources
+    * TBD
+
+## chef
[phone]: Tests took > 60 seconds and test framework times out currently.
+
+## disable EC2 metadata
+
+## disk setup
+
+## emit upstart
+
+## fan
+
+## growpart
+
+## grub dpkg
+
+## landscape
[phone]: Module is not working
+
+## lxd
[phone]: Need a zfs backed test written
+
+## mcollective
+
+## migrator
+
+## mounts
+
+## phone home
+
+## power state change
+
+## puppet
[phone]: Tests took > 60 seconds and test framework times out currently.
+
+## resizefs
+
+## resolv conf
[phone]: Issues with changing resolv.conf and lxc backend.
+
+## redhat subscription
[phone]: Need RH support in test framework.
+
+## rightscale userdata
[phone]: Specific to RightScale cloud enviornment.
+
+## rsyslog
+
+## scripts per boot
+Not applicable to write a test for this as it specifies when something should be run.
+
+## scripts per instance
+Not applicable to write a test for this as it specifies when something should be run.
+
+## scripts per once
+Not applicable to write a test for this as it specifies when something should be run.
+
+## scripts user
+Not applicable to write a test for this as it specifies when something should be run.
+
+## scripts vendor
+Not applicable to write a test for this as it specifies when something should be run.
+
+## snappy
[phone]: Need test to install snaps from store
+
+## snap-config
[phone]: Need to investigate
+
+## spacewalk
+
+## ssh authkey fingerprints
+The authkey_hash key does not appear to work. In fact the default claims to be md5, however syslog only shows sha256
+
+## ubuntu init switch
+
+## update etc hosts
[phone]: Issues with changing /etc/hosts and lxc backend.
+
+## yum add repo
[phone]: Need RH support in test framework.
diff --git a/tests/cloud_tests/configs/modules/apt_configure_conf.yaml b/tests/cloud_tests/configs/modules/apt_configure_conf.yaml
new file mode 100644
index 0000000..222c24b
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/apt_configure_conf.yaml
@@ -0,0 +1,17 @@
+#
+# Provide a configuration for APT
+#
+cloud_config: |
+  #cloud-config
+  apt:
+    conf: |
+      APT {
+          Get {
+              Assume-Yes "true";
+              Fix-Broken "true";
+          }
+      }
+collect_scripts:
+  94cloud-init-config: |
+    #!/bin/bash
+    cat /etc/apt/apt.conf.d/94cloud-init-config
diff --git a/tests/cloud_tests/configs/modules/apt_configure_disable_suites.yaml b/tests/cloud_tests/configs/modules/apt_configure_disable_suites.yaml
new file mode 100644
index 0000000..51ce43e
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/apt_configure_disable_suites.yaml
@@ -0,0 +1,15 @@
+#
+# Disables everything in sources.list
+#
+cloud_config: |
+  #cloud-config
+  apt:
+    disable_suites:
+      - $RELEASE
+      - $RELEASE-updates
+      - $RELEASE-backports
+      - $RELEASE-security
+collect_scripts:
+  sources.list: |
+    #!/bin/bash
+    grep -v '^#' /etc/apt/sources.list | sed '/^\s*$/d'
diff --git a/tests/cloud_tests/configs/modules/apt_configure_primary.yaml b/tests/cloud_tests/configs/modules/apt_configure_primary.yaml
new file mode 100644
index 0000000..ff7cb70
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/apt_configure_primary.yaml
@@ -0,0 +1,17 @@
+#
+# Setup a custome primary sources.list
+#
+cloud_config: |
+  #cloud-config
+  apt:
+    primary:
+      - arches:
+          - default
+        uri: "http://www.gtlib.gatech.edu/pub/ubuntu-releases/"
+collect_scripts:
+  ubuntu.sources.list: |
+    #!/bin/bash
+    grep -v '^#' /etc/apt/sources.list | sed '/^\s*$/d' | grep -c archive.ubuntu.com
+  gatech.sources.list: |
+    #!/bin/bash
+    grep -v '^#' /etc/apt/sources.list | sed '/^\s*$/d' | grep -c gtlib.gatech.edu
diff --git a/tests/cloud_tests/configs/modules/apt_configure_proxy.yaml b/tests/cloud_tests/configs/modules/apt_configure_proxy.yaml
new file mode 100644
index 0000000..e73778f
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/apt_configure_proxy.yaml
@@ -0,0 +1,14 @@
+#
+# Set apt proxy
+#
+cloud_config: |
+  #cloud-config
+  apt:
+    proxy: "http://squid.internal:3128"
+    http_proxy: "http://squid.internal:3128"
+    ftp_proxy: "ftp://squid.internal:3128"
+    https_proxy: "https://squid.internal:3128"
+collect_scripts:
+  90cloud-init-aptproxy: |
+    #!/bin/bash
+    cat /etc/apt/apt.conf.d/90cloud-init-aptproxy
diff --git a/tests/cloud_tests/configs/modules/apt_configure_security.yaml b/tests/cloud_tests/configs/modules/apt_configure_security.yaml
new file mode 100644
index 0000000..933e875
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/apt_configure_security.yaml
@@ -0,0 +1,13 @@
+#
+# Add security to sources.list
+#
+cloud_config: |
+  #cloud-config
+  apt:
+    security:
+      - arches:
+        - default
+collect_scripts:
+  sources.list: |
+    #!/bin/bash
+    grep -c security.ubuntu.com /etc/apt/sources.list
diff --git a/tests/cloud_tests/configs/modules/apt_configure_sources_key.yaml b/tests/cloud_tests/configs/modules/apt_configure_sources_key.yaml
new file mode 100644
index 0000000..02fb006
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/apt_configure_sources_key.yaml
@@ -0,0 +1,45 @@
+#
+# Add a sources.list entry with a given key (Debian Jessie)
+#
+cloud_config: |
+  #cloud-config
+  apt:
+    sources:
+      source1:
+        source: "deb http://ppa.launchpad.net/cloud-init-dev/test-archive/ubuntu $RELEASE main"
+        key: |
+          -----BEGIN PGP PUBLIC KEY BLOCK-----
+          Version: SKS 1.1.6
+          Comment: Hostname: keyserver.ubuntu.com
+
+          mQINBFbZRUIBEAC+A0PIKYBP9kLC4hQtRrffRS11uLo8/BdtmOdrlW0hpPHzCfKnjR3tvSEI
+          lqPHG1QrrjAXKZDnZMRz+h/px7lUztvytGzHPSJd5ARUzAyjyRezUhoJ3VSCxrPqx62avuWf
+          RfoJaIeHfDehL5/dTVkyiWxfVZ369ZX6JN2AgLsQTeybTQ75+2z0xPrrhnGmgh6g0qTYcAaq
+          M5ONOGiqeSBX/Smjh6ALy5XkhUiFGLsI7Yluf6XSICY/x7gd6RAfgSIQrUTNMoS1sqhT4aot
+          +xvOfQy8ySkfAK4NddXql6E/+ZqTmBY/Lr0YklFBy8jGT+UysfiIznPMIwbmgq5Li7BtDDtX
+          b8Uyi4edPpjtextezfXYn4NVIpPL5dPZS/FXh4HpzyH0pYCfrH4QDGA7i52AGmhpiOFjJMo6
+          N33sdjZHOH/2Vyp+QZaQnsdUAi1N4M6c33tQbpIScn1SY+El8z5JDA4PBzkw8HpLCi1gGoa6
+          V4kfbWqXXbGAJFkLkP/vc4+pY9axOlmCkJg7xCPwhI75y1cONgovhz+BEXOzolh5KZuGbGbj
+          xe0wva5DLBeIg7EQFf+99pOS7Syby3Xpm6ZbswEFV0cllK4jf/QMjtfInxobuMoI0GV0bE5l
+          WlRtPCK5FnbHwxi0wPNzB/5fwzJ77r6HgPrR0OkT0lWmbUyoOQARAQABtC1MYXVuY2hwYWQg
+          UFBBIGZvciBjbG91ZCBpbml0IGRldmVsb3BtZW50IHRlYW2JAjgEEwECACIFAlbZRUICGwMG
+          CwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEAg9Bvvk0wTfHfcP/REK5N2s1JYc69qEa9ZN
+          o6oi+A7l6AYw+ZY88O5TJe7F9otv5VXCIKSUT0Vsepjgf0mtXAgf/sb2lsJn/jp7tzgov3YH
+          vSrkTkRydz8xcA87gwQKePuvTLxQpftF4flrBxgSueIn5O/tPrBOxLz7EVYBc78SKg9aj9L2
+          yUp+YuNevlwfZCTYeBb9r3FHaab2HcgkwqYch66+nKYfwiLuQ9NzXXm0Wn0JcEQ6pWvJscbj
+          C9BdawWovfvMK5/YLfI6Btm7F4mIpQBdhSOUp/YXKmdvHpmwxMCN2QhqYK49SM7qE9aUDbJL
+          arppSEBtlCLWhRBZYLTUna+BkuQ1bHz4St++XTR49Qd7vDERALpApDjB2dxPfMiBzCMwQQyq
+          uy13exU8o2ETLg+dZSLfDTzrBNsBFmXlw8WW17nTISYdKeGKL+QdlUjpzdwUMMzHhAO8SmMH
+          zjeSlDSRMXBJFAFSbCl7EwmMKa3yVX0zInT91fNllZ3iatAmtVdqVH/BFQfTIMH2ET7A8WzJ
+          ZzVSuMRhqoKdr5AMcHuJGPUoVkVJHQA+NNvEiXSysF3faL7jmKapmUwrhpYYX2H8pf+VMu2e
+          cLflKTI28dl+ZQ4Pl/aVsxrti/pzhdYy05Sn5ddtySyIkvo8L1cU5MWpbvSlFPkTstBUDLBf
+          pb0uBy+g0oxJQg15
+          =uy53
+          -----END PGP PUBLIC KEY BLOCK-----
+collect_scripts:
+  sources.list: |
+    #!/bin/bash
+    cat /etc/apt/sources.list.d/source1.list
+  apt_key_list: |
+    #!/bin/bash
+    apt-key finger
diff --git a/tests/cloud_tests/configs/modules/apt_configure_sources_keyserver.yaml b/tests/cloud_tests/configs/modules/apt_configure_sources_keyserver.yaml
new file mode 100644
index 0000000..323c255
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/apt_configure_sources_keyserver.yaml
@@ -0,0 +1,18 @@
+#
+# Add a sources.list entry with a key from a keyserver
+#
+cloud_config: |
+  #cloud-config
+  apt:
+    sources:
+      source1:
+        keyid: 0165013E
+        keyserver: keyserver.ubuntu.com
+        source: "deb http://ppa.launchpad.net/cloud-init-dev/test-archive/ubuntu $RELEASE main"
+collect_scripts:
+  sources.list: |
+    #!/bin/bash
+    cat /etc/apt/sources.list.d/source1.list
+  apt_key_list: |
+    #!/bin/bash
+    apt-key finger
diff --git a/tests/cloud_tests/configs/modules/apt_configure_sources_list.yaml b/tests/cloud_tests/configs/modules/apt_configure_sources_list.yaml
new file mode 100644
index 0000000..94a2101
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/apt_configure_sources_list.yaml
@@ -0,0 +1,17 @@
+#
+# Generate a sources.list
+#
+cloud_config: |
+  #cloud-config
+  apt:
+    sources_list: |
+      deb $MIRROR $RELEASE main restricted
+      deb-src $MIRROR $RELEASE main restricted
+      deb $PRIMARY $RELEASE universe restricted
+      deb-src $PRIMARY $RELEASE universe restricted
+      deb $SECURITY $RELEASE-security multiverse
+      deb-src $SECURITY $RELEASE-security multiverse
+collect_scripts:
+  sources.list: |
+    #/bin/bash
+    cat /etc/apt/sources.list
diff --git a/tests/cloud_tests/configs/modules/apt_configure_sources_ppa.yaml b/tests/cloud_tests/configs/modules/apt_configure_sources_ppa.yaml
new file mode 100644
index 0000000..4045ee4
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/apt_configure_sources_ppa.yaml
@@ -0,0 +1,18 @@
+#
+# Add a PPA to source.list
+#
+cloud_config: |
+  #cloud-config
+  apt:
+    sources:
+      source1:
+        keyid: 0165013E
+        keyserver: keyserver.ubuntu.com
+        source: "ppa:curtin-dev/test-archive"
+collect_scripts:
+  sources.list: |
+    #!/bin/bash
+    cat /etc/apt/sources.list.d/curtin-dev-ubuntu-test-archive-*.list
+  apt-key: |
+    #!/bin/bash
+    apt-key finger
diff --git a/tests/cloud_tests/configs/modules/apt_pipelining_disable.yaml b/tests/cloud_tests/configs/modules/apt_pipelining_disable.yaml
new file mode 100644
index 0000000..25545ab
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/apt_pipelining_disable.yaml
@@ -0,0 +1,11 @@
+#
+# Disable apt pipelining value
+#
+cloud_config: |
+  #cloud-config
+  apt:
+    apt_pipelining: false
+collect_scripts:
+  90cloud-init-pipelining: |
+    #!/bin/bash
+    cat /etc/apt/apt.conf.d/90cloud-init-pipelining
diff --git a/tests/cloud_tests/configs/modules/apt_pipelining_os.yaml b/tests/cloud_tests/configs/modules/apt_pipelining_os.yaml
new file mode 100644
index 0000000..16ed3cb
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/apt_pipelining_os.yaml
@@ -0,0 +1,11 @@
+#
+# Set apt pipelining value to OS
+#
+cloud_config: |
+  #cloud-config
+  apt:
+    apt_pipelining: os
+collect_scripts:
+  90cloud-init-pipelining: |
+    #!/bin/bash
+    cat /etc/apt/apt.conf.d/90cloud-init-pipelining
diff --git a/tests/cloud_tests/configs/modules/bootcmd.yaml b/tests/cloud_tests/configs/modules/bootcmd.yaml
new file mode 100644
index 0000000..4767d75
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/bootcmd.yaml
@@ -0,0 +1,11 @@
+#
+# Early boot command
+#
+cloud_config: |
+  #cloud-config
+  bootcmd:
+   - echo 192.168.1.130 us.archive.ubuntu.com > /etc/hosts
+collect_scripts:
+  hosts: |
+    #!/bin/bash
+    cat /etc/hosts
diff --git a/tests/cloud_tests/configs/modules/byobu.yaml b/tests/cloud_tests/configs/modules/byobu.yaml
new file mode 100644
index 0000000..6f9e4f8
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/byobu.yaml
@@ -0,0 +1,16 @@
+#
+# Install and enable byobu system wide and default user
+#
+cloud_config: |
+  #cloud-config
+  byobu_by_default: enable
+collect_scripts:
+  byobu_installed: |
+    #!/bin/bash
+    which byobu
+  byobu_profile_enabled: |
+    #!/bin/bash
+    ls /etc/profile.d/Z97-byobu.sh
+  byobu_launch_exists: |
+    #!/bin/bash
+    which /usr/bin/byobu-launch
diff --git a/tests/cloud_tests/configs/modules/ca_certs.yaml b/tests/cloud_tests/configs/modules/ca_certs.yaml
new file mode 100644
index 0000000..5e386ae
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/ca_certs.yaml
@@ -0,0 +1,51 @@
+#
+# Remove existing ca_certs and install custom ca-cert
+#
+cloud_config: |
+  #cloud-config
+  ca-certs:
+    remove-defaults: true
+    trusted:
+      - |
+        -----BEGIN CERTIFICATE-----
+        MIIGJzCCBA+gAwIBAgIBATANBgkqhkiG9w0BAQUFADCBsjELMAkGA1UEBhMCRlIx
+        DzANBgNVBAgMBkFsc2FjZTETMBEGA1UEBwwKU3RyYXNib3VyZzEYMBYGA1UECgwP
+        d3d3LmZyZWVsYW4ub3JnMRAwDgYDVQQLDAdmcmVlbGFuMS0wKwYDVQQDDCRGcmVl
+        bGFuIFNhbXBsZSBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxIjAgBgkqhkiG9w0BCQEW
+        E2NvbnRhY3RAZnJlZWxhbi5vcmcwHhcNMTIwNDI3MTAzMTE4WhcNMjIwNDI1MTAz
+        MTE4WjB+MQswCQYDVQQGEwJGUjEPMA0GA1UECAwGQWxzYWNlMRgwFgYDVQQKDA93
+        d3cuZnJlZWxhbi5vcmcxEDAOBgNVBAsMB2ZyZWVsYW4xDjAMBgNVBAMMBWFsaWNl
+        MSIwIAYJKoZIhvcNAQkBFhNjb250YWN0QGZyZWVsYW4ub3JnMIICIjANBgkqhkiG
+        9w0BAQEFAAOCAg8AMIICCgKCAgEA3W29+ID6194bH6ejLrIC4hb2Ugo8v6ZC+Mrc
+        k2dNYMNPjcOKABvxxEtBamnSaeU/IY7FC/giN622LEtV/3oDcrua0+yWuVafyxmZ
+        yTKUb4/GUgafRQPf/eiX9urWurtIK7XgNGFNUjYPq4dSJQPPhwCHE/LKAykWnZBX
+        RrX0Dq4XyApNku0IpjIjEXH+8ixE12wH8wt7DEvdO7T3N3CfUbaITl1qBX+Nm2Z6
+        q4Ag/u5rl8NJfXg71ZmXA3XOj7zFvpyapRIZcPmkvZYn7SMCp8dXyXHPdpSiIWL2
+        uB3KiO4JrUYvt2GzLBUThp+lNSZaZ/Q3yOaAAUkOx+1h08285Pi+P8lO+H2Xic4S
+        vMq1xtLg2bNoPC5KnbRfuFPuUD2/3dSiiragJ6uYDLOyWJDivKGt/72OVTEPAL9o
+        6T2pGZrwbQuiFGrGTMZOvWMSpQtNl+tCCXlT4mWqJDRwuMGrI4DnnGzt3IKqNwS4
+        Qyo9KqjMIPwnXZAmWPm3FOKe4sFwc5fpawKO01JZewDsYTDxVj+cwXwFxbE2yBiF
+        z2FAHwfopwaH35p3C6lkcgP2k/zgAlnBluzACUI+MKJ/G0gv/uAhj1OHJQ3L6kn1
+        SpvQ41/ueBjlunExqQSYD7GtZ1Kg8uOcq2r+WISE3Qc9MpQFFkUVllmgWGwYDuN3
+        Zsez95kCAwEAAaN7MHkwCQYDVR0TBAIwADAsBglghkgBhvhCAQ0EHxYdT3BlblNT
+        TCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFFlfyRO6G8y5qEFKikl5
+        ajb2fT7XMB8GA1UdIwQYMBaAFCNsLT0+KV14uGw+quK7Lh5sh/JTMA0GCSqGSIb3
+        DQEBBQUAA4ICAQAT5wJFPqervbja5+90iKxi1d0QVtVGB+z6aoAMuWK+qgi0vgvr
+        mu9ot2lvTSCSnRhjeiP0SIdqFMORmBtOCFk/kYDp9M/91b+vS+S9eAlxrNCB5VOf
+        PqxEPp/wv1rBcE4GBO/c6HcFon3F+oBYCsUQbZDKSSZxhDm3mj7pb67FNbZbJIzJ
+        70HDsRe2O04oiTx+h6g6pW3cOQMgIAvFgKN5Ex727K4230B0NIdGkzuj4KSML0NM
+        slSAcXZ41OoSKNjy44BVEZv0ZdxTDrRM4EwJtNyggFzmtTuV02nkUj1bYYYC5f0L
+        ADr6s0XMyaNk8twlWYlYDZ5uKDpVRVBfiGcq0uJIzIvemhuTrofh8pBQQNkPRDFT
+        Rq1iTo1Ihhl3/Fl1kXk1WR3jTjNb4jHX7lIoXwpwp767HAPKGhjQ9cFbnHMEtkro
+        RlJYdtRq5mccDtwT0GFyoJLLBZdHHMHJz0F9H7FNk2tTQQMhK5MVYwg+LIaee586
+        CQVqfbscp7evlgjLW98H+5zylRHAgoH2G79aHljNKMp9BOuq6SnEglEsiWGVtu2l
+        hnx8SB3sVJZHeer8f/UQQwqbAO+Kdy70NmbSaqaVtp8jOxLiidWkwSyRTsuU6D8i
+        DiH5uEqBXExjrj0FslxcVKdVj5glVcSmkLwZKbEU1OKwleT/iXFhvooWhQ==
+        -----END CERTIFICATE-----
+collect_scripts:
+  cert_count: |
+    #!/bin/bash
+    ls -l /etc/ssl/certs | wc -l
+  cert: |
+    #!/bin/bash
+    md5sum /etc/ssl/certs/ca-certificates.crt
\ No newline at end of file
diff --git a/tests/cloud_tests/configs/modules/debug_disable.yaml b/tests/cloud_tests/configs/modules/debug_disable.yaml
new file mode 100644
index 0000000..cf94910
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/debug_disable.yaml
@@ -0,0 +1,7 @@
+#
+# Do not run in debug mode
+#
+cloud_config: |
+  #cloud-config
+  debug:
+    verbose: False
diff --git a/tests/cloud_tests/configs/modules/debug_enable.yaml b/tests/cloud_tests/configs/modules/debug_enable.yaml
new file mode 100644
index 0000000..ba8080f
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/debug_enable.yaml
@@ -0,0 +1,7 @@
+#
+# Run in debug mode
+#
+cloud_config: |
+  #cloud-config
+  debug:
+    verbose: True
diff --git a/tests/cloud_tests/configs/modules/final_message.yaml b/tests/cloud_tests/configs/modules/final_message.yaml
new file mode 100644
index 0000000..d2a760e
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/final_message.yaml
@@ -0,0 +1,11 @@
+#
+# Print a final message with various predefined variables
+#
+cloud_config: |
+  #cloud-config
+  final_message: |
+    This is my final message!
+    $version
+    $timestamp
+    $datasource
+    $uptime
diff --git a/tests/cloud_tests/configs/modules/keys_to_console.yaml b/tests/cloud_tests/configs/modules/keys_to_console.yaml
new file mode 100644
index 0000000..504a127
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/keys_to_console.yaml
@@ -0,0 +1,11 @@
+#
+# Hide printing of ssh key and fingerprints for specific keys
+#
+cloud_config: |
+  #cloud-config
+  ssh_fp_console_blacklist: [ssh-dss, ssh-dsa, ecdsa-sha2-nistp256]
+  ssh_key_console_blacklist: [ssh-dss, ssh-dsa, ecdsa-sha2-nistp256]
+collect_scripts:
+  syslog: |
+    #!/bin/bash
+    cat /var/log/syslog
diff --git a/tests/cloud_tests/configs/modules/landscape.yaml b/tests/cloud_tests/configs/modules/landscape.yaml
new file mode 100644
index 0000000..6eb14a0
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/landscape.yaml
@@ -0,0 +1,24 @@
+#
+# Setup landscape client settings
+#
+# 2016-11-17: Disabled due to this not working
+#
+enabled: false
+cloud_config: |
+  #cloud-conifg
+  landscape:
+    client:
+      log_level: "info"
+      url: "https://landscape.canonical.com/message-system"
+      ping_url: "http://landscape.canonical.com/ping"
+      data_path: "/var/lib/landscape/client"
+      http_proxy: "http://my.proxy.com/foobar"
+      https_proxy: "https://my.proxy.com/foobar"
+      tags: "server,cloud"
+      computer_title: "footitle"
+      registration_key: "fookey"
+      account_name: "fooaccount"
+collect_scripts:
+  client.conf: |
+    #!/bin/bash
+    cat /etc/landscape/client.conf
diff --git a/tests/cloud_tests/configs/modules/locale.yaml b/tests/cloud_tests/configs/modules/locale.yaml
new file mode 100644
index 0000000..be3c747
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/locale.yaml
@@ -0,0 +1,17 @@
+#
+# Set locale to non-default option and verify
+#
+cloud_config: |
+  #cloud-config
+  locale: en_GB.UTF-8
+  locale_configfile: /etc/default/locale
+collect_scripts:
+  locale_default: |
+    #!/bin/bash
+    cat /etc/default/locale
+  locale_a: |
+    #!/bin/bash
+    locale -a
+  locale_gen: |
+    #!/bin/bash
+    cat /etc/locale.gen | grep -v '^#' | uniq
diff --git a/tests/cloud_tests/configs/modules/lxd_bridge.yaml b/tests/cloud_tests/configs/modules/lxd_bridge.yaml
new file mode 100644
index 0000000..956d888
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/lxd_bridge.yaml
@@ -0,0 +1,28 @@
+#
+# LXD configured with directory backend and IPv4 bridge
+#
+cloud_config: |
+  #cloud-config
+  lxd:
+    init:
+      storage_backend: dir
+    bridge:
+      mode: new
+      name: lxdbr0
+      ipv4_address: 10.100.100.1
+      ipv4_netmask: 24
+      ipv4_dhcp_first: 10.100.100.100
+      ipv4_dhcp_last: 10.100.100.200
+      ipv4_nat: true
+      domain: lxd
+collect_scripts:
+  lxc: |
+    #!/bin/bash
+    which lxc
+  lxd: |
+    #!/bin/bash
+    which lxd
+  lxc-bridge: |
+    #!/bin/bash
+    ip addr show lxdbr0
+    cat /etc/default/lxd-bridge 2>/dev/null | grep -v ^#  | sort -u
diff --git a/tests/cloud_tests/configs/modules/lxd_dir.yaml b/tests/cloud_tests/configs/modules/lxd_dir.yaml
new file mode 100644
index 0000000..1e24670
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/lxd_dir.yaml
@@ -0,0 +1,15 @@
+#
+# LXD configured with directory backend
+#
+cloud_config: |
+  #cloud-config
+  lxd:
+    init:
+      storage_backend: dir
+collect_scripts:
+  lxc: |
+    #!/bin/bash
+    which lxc
+  lxd: |
+    #!/bin/bash
+    which lxd
diff --git a/tests/cloud_tests/configs/modules/ntp.yaml b/tests/cloud_tests/configs/modules/ntp.yaml
new file mode 100644
index 0000000..94243c7
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/ntp.yaml
@@ -0,0 +1,18 @@
+#
+# Emtpy NTP config to setup using defaults
+#
+cloud_config: |
+  #cloud-config
+  ntp:
+    pools: {}
+    servers: {}
+collect_scripts:
+  ntp_installed_empty: |
+    #!/bin/bash
+    dpkg -l | grep ntp | wc -l
+  ntp_conf_dist_empty: |
+    #!/bin/bash
+    ls /etc/ntp.conf.dist | wc -l
+  ntp_conf_empty: |
+    #!/bin/bash
+    grep '^pool' /etc/ntp.conf
diff --git a/tests/cloud_tests/configs/modules/ntp_pools.yaml b/tests/cloud_tests/configs/modules/ntp_pools.yaml
new file mode 100644
index 0000000..394077d
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/ntp_pools.yaml
@@ -0,0 +1,21 @@
+#
+# NTP config using specific pools
+#
+cloud_config: |
+  #cloud-config
+  ntp:
+    pools:
+        - 0.pool.ntp.org
+        - 1.pool.ntp.org
+        - 2.pool.ntp.org
+        - 3.pool.ntp.org
+collect_scripts:
+  ntp_installed_pools: |
+    #!/bin/bash
+    dpkg -l | grep ntp | wc -l
+  ntp_conf_dist_pools: |
+    #!/bin/bash
+    ls /etc/ntp.conf.dist | wc -l
+  ntp_conf_pools: |
+    #!/bin/bash
+    grep '^pool' /etc/ntp.conf
diff --git a/tests/cloud_tests/configs/modules/ntp_servers.yaml b/tests/cloud_tests/configs/modules/ntp_servers.yaml
new file mode 100644
index 0000000..b041526
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/ntp_servers.yaml
@@ -0,0 +1,18 @@
+#
+# NTP config using specific servers
+#
+cloud_config: |
+  #cloud-config
+  ntp:
+    servers:
+        - pool.ntp.org
+collect_scripts:
+  ntp_installed_servers: |
+    #!/bin/bash
+    dpkg -l | grep ntp | wc -l
+  ntp_conf_dist_servers: |
+    #!/bin/bash
+    ls /etc/ntp.conf.dist | wc -l
+  ntp_conf_servers: |
+    #!/bin/bash
+    grep '^server' /etc/ntp.conf
diff --git a/tests/cloud_tests/configs/modules/package_update_upgrade_install.yaml b/tests/cloud_tests/configs/modules/package_update_upgrade_install.yaml
new file mode 100644
index 0000000..7fe8bd2
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/package_update_upgrade_install.yaml
@@ -0,0 +1,20 @@
+#
+# Update/upgrade via apt and then install a pair of packages
+#
+cloud_config: |
+  #cloud-config
+  packages:
+    - htop
+    - tree
+  package_update: true
+  package_upgrade: true
+collect_scripts:
+  apt_history_cmdline: |
+    #!/bin/bash
+    grep ^Commandline: /var/log/apt/history.log
+  dpkg_htop: |
+    #!/bin/bash
+    dpkg -l | grep htop | wc -l
+  dpkg_tree: |
+    #!/bin/bash
+    dpkg -l | grep tree | wc -l
diff --git a/tests/cloud_tests/configs/modules/runcmd.yaml b/tests/cloud_tests/configs/modules/runcmd.yaml
new file mode 100644
index 0000000..fc31657
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/runcmd.yaml
@@ -0,0 +1,11 @@
+#
+# Run a simple command
+#
+cloud_config: |
+  #cloud-config
+  runcmd:
+   - echo cloud-init run cmd test > /tmp/run_cmd
+collect_scripts:
+  run_cmd: |
+    #!/bin/bash
+    cat /tmp/run_cmd
diff --git a/tests/cloud_tests/configs/modules/salt_minion.yaml b/tests/cloud_tests/configs/modules/salt_minion.yaml
new file mode 100644
index 0000000..ea091a3
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/salt_minion.yaml
@@ -0,0 +1,32 @@
+#
+# Create config for a salt minion
+#
+# 2016-11-17: Currently takes >60 seconds results in test failure
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  salt_minion:
+      conf:
+          master: salt.mydomain.com
+      public_key: |
+          ------BEGIN PUBLIC KEY-------
+          <key data>
+          ------END PUBLIC KEY-------
+      private_key: |
+          ------BEGIN PRIVATE KEY------
+          <key data>
+          ------END PRIVATE KEY-------
+collect_scripts:
+  minion: |
+    #!/bin/bash
+    cat /etc/salt/minion
+  minion_id: |
+    #!/bin/bash
+    cat /etc/salt/minion_id
+  minion.pem: |
+    #!/bin/bash
+    cat /etc/salt/pki/minion/minion.pem
+  minion.pub: |
+    #!/bin/bash
+    cat /etc/salt/pki/minion/minion.pub
diff --git a/tests/cloud_tests/configs/modules/seed_random_command.yaml b/tests/cloud_tests/configs/modules/seed_random_command.yaml
new file mode 100644
index 0000000..aa2c681
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/seed_random_command.yaml
@@ -0,0 +1,16 @@
+#
+# Use uuid to create a random string
+#
+# 2016-11-15 Disabled as this is not working currently
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  random_seed:
+    command: ["cat", "/proc/sys/kernel/random/uuid"]
+    command_required: true
+    file: /root/seed
+collect_scripts:
+  seed_data: |
+    #!/bin/bash
+    cat /root/seed
diff --git a/tests/cloud_tests/configs/modules/seed_random_data.yaml b/tests/cloud_tests/configs/modules/seed_random_data.yaml
new file mode 100644
index 0000000..6f8fd4a
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/seed_random_data.yaml
@@ -0,0 +1,13 @@
+#
+# Push in random raw string to set as seed
+#
+cloud_config: |
+  #cloud-config
+  random_seed:
+    data: 'MYUb34023nD:LFDK10913jk;dfnk:Df'
+    encoding: raw
+    file: /root/seed
+collect_scripts:
+  seed_data: |
+    #!/bin/bash
+    cat /root/seed
diff --git a/tests/cloud_tests/configs/modules/set_hostname.yaml b/tests/cloud_tests/configs/modules/set_hostname.yaml
new file mode 100644
index 0000000..35987fd
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/set_hostname.yaml
@@ -0,0 +1,16 @@
+#
+# Set the hostname and update /etc/hosts
+#
+cloud_config: |
+  #cloud-config
+  hostname: myhostname
+collect_scripts:
+  hosts: |
+    #!/bin/bash
+    grep ^127 /etc/hosts
+  hostname: |
+    #!/bin/bash
+    hostname
+  fqdn: |
+    #!/bin/bash
+    hostname --fqdn
diff --git a/tests/cloud_tests/configs/modules/set_hostname_fqdn.yaml b/tests/cloud_tests/configs/modules/set_hostname_fqdn.yaml
new file mode 100644
index 0000000..5c62895
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/set_hostname_fqdn.yaml
@@ -0,0 +1,18 @@
+#
+# Set the hostname and update /etc/hosts
+#
+cloud_config: |
+  #cloud-config
+  manage_etc_hosts: true
+  hostname: myhostname
+  fqdn: host.myorg.com
+collect_scripts:
+  hosts: |
+    #!/bin/bash
+    grep ^127 /etc/hosts
+  hostname: |
+    #!/bin/bash
+    hostname
+  fqdn: |
+    #!/bin/bash
+    hostname --fqdn
diff --git a/tests/cloud_tests/configs/modules/set_password.yaml b/tests/cloud_tests/configs/modules/set_password.yaml
new file mode 100644
index 0000000..6403d4c
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/set_password.yaml
@@ -0,0 +1,15 @@
+#
+# Set password of default user
+#
+cloud_config: |
+  #cloud-config
+  password: password
+  chpasswd: { expire: False }
+  ssh_pwauth: True
+collect_scripts:
+  shadow: |
+    #!/bin/bash
+    cat /etc/shadow
+  sshd_config: |
+    #!/bin/bash
+    grep '^PasswordAuth' /etc/ssh/sshd_config
diff --git a/tests/cloud_tests/configs/modules/set_password_expire.yaml b/tests/cloud_tests/configs/modules/set_password_expire.yaml
new file mode 100644
index 0000000..6b8ae18
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/set_password_expire.yaml
@@ -0,0 +1,26 @@
+#
+# Expire password for all users
+#
+cloud_config: |
+  #cloud-config
+  chpasswd: { expire: True }
+  users:
+    - name: tom
+      password: $1$xyz$sPMsLNmf66Ohl.ol6JvzE.
+      lock_passwd: false
+    - name: dick
+      password: $1$xyz$sPMsLNmf66Ohl.ol6JvzE.
+      lock_passwd: false
+    - name: harry
+      password: $1$xyz$sPMsLNmf66Ohl.ol6JvzE.
+      lock_passwd: false
+    - name: jane
+      password: $1$xyz$sPMsLNmf66Ohl.ol6JvzE.
+      lock_passwd: false
+collect_scripts:
+  shadow: |
+    #!/bin/bash
+    cat /etc/shadow
+  sshd_config: |
+    #!/bin/bash
+    grep '^PasswordAuth' /etc/ssh/sshd_config
diff --git a/tests/cloud_tests/configs/modules/set_password_list.yaml b/tests/cloud_tests/configs/modules/set_password_list.yaml
new file mode 100644
index 0000000..6b6e4c7
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/set_password_list.yaml
@@ -0,0 +1,31 @@
+#
+# Set password of list of users
+#
+cloud_config: |
+  #cloud-config
+  ssh_pwauth: yes
+  users:
+    - name: tom
+      password: $1$xyz$sPMsLNmf66Ohl.ol6JvzE.
+      lock_passwd: false
+    - name: dick
+      password: $1$xyz$sPMsLNmf66Ohl.ol6JvzE.
+      lock_passwd: false
+    - name: harry
+      password: $1$xyz$sPMsLNmf66Ohl.ol6JvzE.
+      lock_passwd: false
+    - name: jane
+      password: $1$xyz$sPMsLNmf66Ohl.ol6JvzE.
+      lock_passwd: false
+  chpasswd:
+    list: |
+      tom:mypassword123!
+      dick:R
+      harry:Random
+collect_scripts:
+  shadow: |
+    #!/bin/bash
+    cat /etc/shadow
+  sshd_config: |
+    #!/bin/bash
+    grep '^PasswordAuth' /etc/ssh/sshd_config
diff --git a/tests/cloud_tests/configs/modules/snappy.yaml b/tests/cloud_tests/configs/modules/snappy.yaml
new file mode 100644
index 0000000..2e0b0d9
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/snappy.yaml
@@ -0,0 +1,11 @@
+#
+# Install snappy
+#
+cloud_config: |
+  #cloud-config
+  snappy:
+    system_snappy: auto
+collect_scripts:
+  snap_version: |
+    #!/bin/bash
+    snap --version
diff --git a/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_disable.yaml b/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_disable.yaml
new file mode 100644
index 0000000..715331a
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_disable.yaml
@@ -0,0 +1,11 @@
+#
+# Disable fingerprint printing
+#
+cloud_config: |
+  #cloud-config
+  ssh_genkeytypes: []
+  no_ssh_fingerprints: true
+collect_scripts:
+  syslog: |
+    #!/bin/bash
+    cat /var/log/syslog
diff --git a/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_enable.yaml b/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_enable.yaml
new file mode 100644
index 0000000..9d37723
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_enable.yaml
@@ -0,0 +1,14 @@
+#
+# Print auth keys with different hash than md5
+#
+cloud_config: |
+  #cloud-config
+  ssh_genkeytypes:
+  - ecdsa
+  - ed25519
+  ssh_authorized_keys:
+    - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDXW9Gg5H7ehjdSc6qDzwNtgCy94XYHhEYlXZMO2+FJrH3wfHGiMfCwOHxcOMt2QiXItULthdeQWS9QjBSSjVRXf6731igFrqPFyS9qBlOQ5D29C4HBXFnQggGVpBNJ82IRJv7szbbe/vpgLBP4kttUza9Dr4e1YM1ln4PRnjfXea6T0m+m1ixNb5432pTXlqYOnNOxSIm1gHgMLxPuDrJvQERDKrSiKSjIdyC9Jd8t2e1tkNLY0stmckVRbhShmcJvlyofHWbc2Ca1mmtP7MlS1VQnfLkvU1IrFwkmaQmaggX6WR6coRJ6XFXdWcq/AI2K6GjSnl1dnnCxE8VCEXBlXgFzad+PMSG4yiL5j8Oo1ZVpkTdgBnw4okGqTYCXyZg6X00As9IBNQfZMFlQXlIo4FiWgj3CO5QHQOyOX6FuEumaU13GnERrSSdp9tCs1Qm3/DG2RSCQBWTfcgMcStIvKqvJ3IjFn0vGLvI3Ampnq9q1SHwmmzAPSdzcMA76HyMUA5VWaBvWHlUxzIM6unxZASnwvuCzpywSEB5J2OF+p6H+cStJwQ32XwmOG8pLp1srlVWpqZI58Du/lzrkPqONphoZx0LDV86w7RUz1ksDzAdcm0tvmNRFMN1a0frDs506oA3aWK0oDk4Nmvk8sXGTYYw3iQSkOvDUUlIsqdaO+w==
+collect_scripts:
+  syslog: |
+    #!/bin/bash
+    cat /var/log/syslog
diff --git a/tests/cloud_tests/configs/modules/ssh_import_id.yaml b/tests/cloud_tests/configs/modules/ssh_import_id.yaml
new file mode 100644
index 0000000..6e08a4c
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/ssh_import_id.yaml
@@ -0,0 +1,12 @@
+#
+# Import a user's ssh key via gh or lp
+#
+cloud_config: |
+  #cloud-config
+  ssh_import_id:
+    - gh:powersj
+    - lp:smoser
+collect_scripts:
+  auth_keys_ubuntu: |
+    #!/bin/bash
+    cat /home/ubuntu/.ssh/authorized_keys
diff --git a/tests/cloud_tests/configs/modules/ssh_keys_generate.yaml b/tests/cloud_tests/configs/modules/ssh_keys_generate.yaml
new file mode 100644
index 0000000..3c74ad5
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/ssh_keys_generate.yaml
@@ -0,0 +1,40 @@
+#
+# SSH keys generated using cloud-init
+#
+cloud_config: |
+  #cloud-config
+  ssh_genkeytypes:
+  - ecdsa
+  - ed25519
+  authkey_hash: sha512
+collect_scripts:
+  auth_keys_root: |
+    #!/bin/bash
+    cat /root/.ssh/authorized_keys
+  auth_keys_ubuntu: |
+    #!/bin/bash
+    cat /home/ubuntu/ssh/authorized_keys
+  dsa_public: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_dsa_key.pub
+  dsa_private: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_dsa_key
+  rsa_public: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_rsa_key.pub
+  rsa_private: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_rsa_key
+  ecdsa_public: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_ecdsa_key.pub
+  ecdsa_private: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_ecdsa_key
+  ed25519_public: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_ed25519_key.pub
+  ed25519_private: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_ed25519_key
diff --git a/tests/cloud_tests/configs/modules/ssh_keys_provided.yaml b/tests/cloud_tests/configs/modules/ssh_keys_provided.yaml
new file mode 100644
index 0000000..2590630
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/ssh_keys_provided.yaml
@@ -0,0 +1,100 @@
+#
+# SSH keys provided via cloud config
+#
+enabled: False
+cloud_config: |
+  #cloud-config
+  disable_root: false
+  ssh_authorized_keys:
+    - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDXW9Gg5H7ehjdSc6qDzwNtgCy94XYHhEYlXZMO2+FJrH3wfHGiMfCwOHxcOMt2QiXItULthdeQWS9QjBSSjVRXf6731igFrqPFyS9qBlOQ5D29C4HBXFnQggGVpBNJ82IRJv7szbbe/vpgLBP4kttUza9Dr4e1YM1ln4PRnjfXea6T0m+m1ixNb5432pTXlqYOnNOxSIm1gHgMLxPuDrJvQERDKrSiKSjIdyC9Jd8t2e1tkNLY0stmckVRbhShmcJvlyofHWbc2Ca1mmtP7MlS1VQnfLkvU1IrFwkmaQmaggX6WR6coRJ6XFXdWcq/AI2K6GjSnl1dnnCxE8VCEXBlXgFzad+PMSG4yiL5j8Oo1ZVpkTdgBnw4okGqTYCXyZg6X00As9IBNQfZMFlQXlIo4FiWgj3CO5QHQOyOX6FuEumaU13GnERrSSdp9tCs1Qm3/DG2RSCQBWTfcgMcStIvKqvJ3IjFn0vGLvI3Ampnq9q1SHwmmzAPSdzcMA76HyMUA5VWaBvWHlUxzIM6unxZASnwvuCzpywSEB5J2OF+p6H+cStJwQ32XwmOG8pLp1srlVWpqZI58Du/lzrkPqONphoZx0LDV86w7RUz1ksDzAdcm0tvmNRFMN1a0frDs506oA3aWK0oDk4Nmvk8sXGTYYw3iQSkOvDUUlIsqdaO+w==
+  ssh_keys:
+    rsa_private: |
+      -----BEGIN RSA PRIVATE KEY-----
+      MIIEowIBAAKCAQEAtPx6PqN3iSEsnTtibyIEy52Tra8T5fn0ryXyg46Di2NBwdnj
+      o8trNv9jenfV/UhmePl58lXjT43wV8OCMl6KsYXyBdegM35NNtono4I4mLLKFMR9
+      9TOtDn6iYcaNenVhF3ZCj9Z2nNOlTrdc0uchHqKMrxLjCRCUrL91Uf+xioTF901Y
+      RM+ZqC5lT92yAL76F4qPF+Lq1QtUfNfUIwwvOp5ccDZLPxij0YvyBzubYye9hJHu
+      yjbJv78R4JHV+L2WhzSoX3W/6WrxVzeXqFGqH894ccOaC/7tnqSP6V8lIQ6fE2+c
+      DurJcpM3CJRgkndGHjtU55Y71YkcdLksSMvezQIDAQABAoIBAQCrU4IJP8dNeaj5
+      IpkY6NQvR/jfZqfogYi+MKb1IHin/4rlDfUvPcY9pt8ttLlObjYK+OcWn3Vx/sRw
+      4DOkqNiUGl80Zp1RgZNohHUXlJMtAbrIlAVEk+mTmg7vjfyp2unRQvLZpMRdywBm
+      lq95OrCghnG03aUsFJUZPpi5ydnwbA12ma+KHkG0EzaVlhA7X9N6z0K6U+zue2gl
+      goMLt/MH0rsYawkHrwiwXaIFQeyV4MJP0vmrZLbFk1bycu9X/xPtTYotWyWo4eKA
+      cb05uu04qwexkKHDM0KXtT0JecbTo2rOefFo8Uuab6uJY+fEHNocZ+v1vLA4aOxJ
+      ovp1JuXlAoGBAOWYNgKrlTfy5n0sKsNk+1RuL2jHJZJ3HMd0EIt7/fFQN3Fi08Hu
+      jtntqD30Wj+DJK8b8Lrt66FruxyEJm5VhVmwkukrLR5ige2f6ftZnoFCmdyy+0zP
+      dnPZSUe2H5ZPHa+qthJgHLn+al2P04tGh+1fGHC2PbP+e0Co+/ZRIOxrAoGBAMnN
+      IEen9/FRsqvnDd36I8XnJGskVRTZNjylxBmbKcuMWm+gNhOI7gsCAcqzD4BYZjjW
+      pLhrt/u9p+l4MOJy6OUUdM/okg12SnJEGryysOcVBcXyrvOfklWnANG4EAH5jt1N
+      ftTb1XTxzvWVuR/WJK0B5MZNYM71cumBdUDtPi+nAoGAYmoIXMSnxb+8xNL10aOr
+      h9ljQQp8NHgSQfyiSufvRk0YNuYh1vMnEIsqnsPrG2Zfhx/25GmvoxXGssaCorDN
+      5FAn6QK06F1ZTD5L0Y3sv4OI6G1gAuC66ZWuL6sFhyyKkQ4f1WiVZ7SCa3CHQSAO
+      i9VDaKz1bf4bXvAQcNj9v9kCgYACSOZCqW4vN0OUmqsXhkt9ZB6Pb/veno70pNPR
+      jmYsvcwQU3oJQpWfXkhy6RAV3epaXmPDCsUsfns2M3wqNC7a2R5xdCqjKGGzZX4A
+      AO3rz9se4J6Gd5oKijeCKFlWDGNHsibrdgm2pz42nZlY+O21X74dWKbt8O16I1MW
+      hxkbJQKBgAXfuen/srVkJgPuqywUYag90VWCpHsuxdn+fZJa50SyZADr+RbiDfH2
+      vek8Uo8ap8AEsv4Rfs9opUcUZevLp3g2741eOaidHVLm0l4iLIVl03otGOqvSzs+
+      A3tFPEOxauXpzCt8f8eXsz0WQXAgIKW2h8zu5QHjomioU3i27mtE
+      -----END RSA PRIVATE KEY-----
+    rsa_public: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0/Ho+o3eJISydO2JvIgTLnZOtrxPl+fSvJfKDjoOLY0HB2eOjy2s2/2N6d9X9SGZ4+XnyVeNPjfBXw4IyXoqxhfIF16Azfk022iejgjiYssoUxH31M60OfqJhxo16dWEXdkKP1nac06VOt1zS5yEeooyvEuMJEJSsv3VR/7GKhMX3TVhEz5moLmVP3bIAvvoXio8X4urVC1R819QjDC86nlxwNks/GKPRi/IHO5tjJ72Eke7KNsm/vxHgkdX4vZaHNKhfdb/pavFXN5eoUaofz3hxw5oL/u2epI/pXyUhDp8Tb5wO6slykzcIlGCSd0YeO1TnljvViRx0uSxIy97N root@xenial-lxd
+    dsa_private: |
+      -----BEGIN DSA PRIVATE KEY-----
+      MIIBuwIBAAKBgQD5Fstc23IVSDe6k4DNP8smPKuEWUvHDTOGaXrhOVAfzZ6+jklP
+      55mzvC7jO53PWWC31hq10xBoWdev0WtcNF9Tv+4bAa1263y51Rqo4GI7xx+xic1d
+      mLqqfYijBT9k48J/1tV0cs1Wjs6FP/IJTD/kYVC930JjYQMi722lBnUxsQIVAL7i
+      z3fTGKTvSzvW0wQlwnYpS2QFAoGANp+KdyS9V93HgxGQEN1rlj/TSv/a3EVdCKtE
+      nQf55aPHxDAVDVw5JtRh4pZbbRV4oGRPc9KOdjo5BU28vSM3Lmhkb+UaaDXwHkgI
+      nK193o74DKjADWZxuLyyiKHiMOhxozoxDfjWxs8nz6uqvSW0pr521EwIY6RajbED
+      nZ2a3GkCgYEAyoUomNRB6bmpsIfzt8zdtqLP5umIj2uhr9MVPL8/QdbxmJ72Z7pf
+      Q2z1B7QAdIBGOlqJXtlau7ABhWK29Efe+99ObyTSSdDc6RCDeAwUmBAiPRQhDH2E
+      wExw3doDSCUb28L1B50wBzQ8mC3KXp6C7IkBXWspb16DLHUHFSI8bkICFA5kVUcW
+      nCPOXEQsayANi8+Cb7BH
+      -----END DSA PRIVATE KEY-----
+    dsa_public: ssh-dss AAAAB3NzaC1kc3MAAACBAPkWy1zbchVIN7qTgM0/yyY8q4RZS8cNM4ZpeuE5UB/Nnr6OSU/nmbO8LuM7nc9ZYLfWGrXTEGhZ16/Ra1w0X1O/7hsBrXbrfLnVGqjgYjvHH7GJzV2Yuqp9iKMFP2Tjwn/W1XRyzVaOzoU/8glMP+RhUL3fQmNhAyLvbaUGdTGxAAAAFQC+4s930xik70s71tMEJcJ2KUtkBQAAAIA2n4p3JL1X3ceDEZAQ3WuWP9NK/9rcRV0Iq0SdB/nlo8fEMBUNXDkm1GHillttFXigZE9z0o52OjkFTby9IzcuaGRv5RpoNfAeSAicrX3ejvgMqMANZnG4vLKIoeIw6HGjOjEN+NbGzyfPq6q9JbSmvnbUTAhjpFqNsQOdnZrcaQAAAIEAyoUomNRB6bmpsIfzt8zdtqLP5umIj2uhr9MVPL8/QdbxmJ72Z7pfQ2z1B7QAdIBGOlqJXtlau7ABhWK29Efe+99ObyTSSdDc6RCDeAwUmBAiPRQhDH2EwExw3doDSCUb28L1B50wBzQ8mC3KXp6C7IkBXWspb16DLHUHFSI8bkI= root@xenial-lxd
+    ed25519_private: |
+      -----BEGIN OPENSSH PRIVATE KEY-----
+      b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW
+      QyNTUxOQAAACDbnQGUruL42aVVsyHeaV5mYNTOhteXao0Nl5DVThJ2+QAAAJgwt+lcMLfp
+      XAAAAAtzc2gtZWQyNTUxOQAAACDbnQGUruL42aVVsyHeaV5mYNTOhteXao0Nl5DVThJ2+Q
+      AAAEDQlFZpz9q8+/YJHS9+jPAqy2ZT6cGEv8HTB6RZtTjd/dudAZSu4vjZpVWzId5pXmZg
+      1M6G15dqjQ2XkNVOEnb5AAAAD3Jvb3RAeGVuaWFsLWx4ZAECAwQFBg==
+      -----END OPENSSH PRIVATE KEY-----
+    ed25519_public: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINudAZSu4vjZpVWzId5pXmZg1M6G15dqjQ2XkNVOEnb5 root@xenial-lxd
+    ecdsa_private: |
+      -----BEGIN EC PRIVATE KEY-----
+      MHcCAQEEIDuK+QFc1wmyJY8uDqQVa1qHte30Rk/fdLxGIBkwJAyOoAoGCCqGSM49
+      AwEHoUQDQgAEWxLlO+TL8gL91eET9p/HFQbqR1A691AkJgZk3jY5mpZqxgX4vcgb
+      7f/CtXuM6s2svcDJqAeXr6Wk8OJJcMxylA==
+      -----END EC PRIVATE KEY-----
+    ecdsa_public: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFsS5Tvky/IC/dXhE/afxxUG6kdQOvdQJCYGZN42OZqWasYF+L3IG+3/wrV7jOrNrL3AyagHl6+lpPDiSXDMcpQ= root@xenial-lxd
+collect_scripts:
+  auth_keys_root: |
+    #!/bin/bash
+    cat /root/.ssh/authorized_keys
+  auth_keys_ubuntu: |
+    #!/bin/bash
+    cat /home/ubuntu/ssh/authorized_keys
+  dsa_public: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_dsa_key.pub
+  dsa_private: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_dsa_key
+  rsa_public: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_rsa_key.pub
+  rsa_private: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_rsa_key
+  ecdsa_public: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_ecdsa_key.pub
+  ecdsa_private: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_ecdsa_key
+  ed25519_public: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_ed25519_key.pub
+  ed25519_private: |
+    #!/bin/bash
+    cat /etc/ssh/ssh_host_ed25519_key
diff --git a/tests/cloud_tests/configs/modules/timezone.yaml b/tests/cloud_tests/configs/modules/timezone.yaml
new file mode 100644
index 0000000..b0f7413
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/timezone.yaml
@@ -0,0 +1,10 @@
+#
+# Set system timezone
+#
+cloud_config: |
+  #cloud-config
+  timezone: US/Aleutian
+collect_scripts:
+  timezone: |
+    #!/bin/bash
+    date +%Z
diff --git a/tests/cloud_tests/configs/modules/user_groups.yaml b/tests/cloud_tests/configs/modules/user_groups.yaml
new file mode 100644
index 0000000..1356efb
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/user_groups.yaml
@@ -0,0 +1,48 @@
+#
+# Create groups and users with various options
+#
+cloud_config: |
+  #cloud-config
+  # Add groups to the system
+  groups:
+    - secret: [foobar,barfoo]
+    - cloud-users
+
+  # Add users to the system. Users are added after groups are added.
+  users:
+    - default
+    - name: foobar
+      gecos: Foo B. Bar
+      primary-group: foobar
+      groups: users
+      expiredate: 2038-01-19
+      lock_passwd: false
+      passwd: $6$j212wezy$7H/1LT4f9/N3wpgNunhsIqtMj62OKiS3nyNwuizouQc3u7MbYCarYeAHWYPYb2FT.lbioDm2RrkJPb9BZMN1O/
+    - name: barfoo
+      gecos: Bar B. Foo
+      sudo: ALL=(ALL) NOPASSWD:ALL
+      groups: cloud-users
+      lock_passwd: true
+    - name: cloudy
+      gecos: Magic Cloud App Daemon User
+      inactive: true
+      system: true
+collect_scripts:
+  group_ubuntu: |
+    #!/bin/bash
+    getent group ubuntu
+  group_cloud_users: |
+    #!/bin/bash
+    getent group cloud-users
+  user_ubuntu: |
+    #!/bin/bash
+    getent passwd ubuntu
+  user_foobar: |
+    #!/bin/bash
+    getent passwd foobar
+  user_barfoo: |
+    #!/bin/bash
+    getent passwd barfoo
+  user_cloudy: |
+    #!/bin/bash
+    getent passwd cloudy
diff --git a/tests/cloud_tests/configs/modules/write_files.yaml b/tests/cloud_tests/configs/modules/write_files.yaml
new file mode 100644
index 0000000..3a3bbce
--- /dev/null
+++ b/tests/cloud_tests/configs/modules/write_files.yaml
@@ -0,0 +1,40 @@
+#
+# Write various file types
+#
+cloud_config: |
+  #cloud-config
+  write_files:
+  -   encoding: b64
+      content: CiMgVGhpcyBmaWxlIGNvbnRyb2xzIHRoZSBzdGF0ZSBvZiBTRUxpbnV4
+      owner: root:root
+      path: /root/file_b64
+      permissions: '0644'
+  -   content: |
+          # My new /root/file_text
+
+          SMBDOPTIONS="-D"
+      path: /root/file_text
+  -   content: !!binary |
+          f0VMRgIBAQAAAAAAAAAAAAIAPgABAAAAwARAAAAAAABAAAAAAAAAAJAVAAAAAAAAAAAAAEAAOAAI
+          AEAAHgAdAAYAAAAFAAAAQAAAAAAAAABAAEAAAAAAAEAAQAAAAAAAwAEAAAAAAADAAQAAAAAAAAgA
+          AAAAAAAAAwAAAAQAAAAAAgAAAAAAAAACQAAAAAAAAAJAAAAAAAAcAAAAAAAAABwAAAAAAAAAAQAA
+      path: /root/file_binary
+      permissions: '0555'
+  -   encoding: gzip
+      content: !!binary |
+          H4sIAIDb/U8C/1NW1E/KzNMvzuBKTc7IV8hIzcnJVyjPL8pJ4QIA6N+MVxsAAAA=
+      path: /root/file_gzip
+      permissions: '0755'
+collect_scripts:
+  file_b64: |
+    #!/bin/bash
+    file /root/file_b64
+  file_text: |
+    #!/bin/bash
+    file /root/file_text
+  file_binary: |
+    #!/bin/bash
+    file /root/file_binary
+  file_gzip: |
+    #!/bin/bash
+    file /root/file_gzip
diff --git a/tests/cloud_tests/images/__init__.py b/tests/cloud_tests/images/__init__.py
new file mode 100644
index 0000000..997865e
--- /dev/null
+++ b/tests/cloud_tests/images/__init__.py
@@ -0,0 +1,6 @@
+def get_image(platform, config):
+    """
+    get image from platform object using os_name, looking up img_conf in main
+    config file
+    """
+    return platform.get_image(config)
diff --git a/tests/cloud_tests/images/base.py b/tests/cloud_tests/images/base.py
new file mode 100644
index 0000000..02d928c
--- /dev/null
+++ b/tests/cloud_tests/images/base.py
@@ -0,0 +1,60 @@
+class Image(object):
+    """
+    Base class for images
+    """
+    platform_name = None
+
+    def __init__(self, name, config, platform):
+        """
+        setup
+        """
+        self.name = name
+        self.config = config
+        self.platform = platform
+
+    def __str__(self):
+        """
+        a brief description of the image
+        """
+        return '-'.join((self.properties['os'], self.properties['release']))
+
+    @property
+    def properties(self):
+        """
+        {} containing: 'arch', 'os', 'version', 'release'
+        """
+        raise NotImplementedError
+
+    # FIXME: instead of having execute and push_file and other instance methods
+    #        here which pass through to a hidden instance, it might be better
+    #        to expose an instance that the image can be modified through
+    def execute(self, command, stdin=None, stdout=None, stderr=None, env={}):
+        """
+        execute command in image, modifying image
+        """
+        raise NotImplementedError
+
+    def push_file(self, local_path, remote_path):
+        """
+        copy file at 'local_path' to instance at 'remote_path', modifying image
+        """
+        raise NotImplementedError
+
+    def run_script(self, script):
+        """
+        run script in image, modifying image
+        return_value: script output
+        """
+        raise NotImplementedError
+
+    def snapshot(self):
+        """
+        create snapshot of image, block until done
+        """
+        raise NotImplementedError
+
+    def destroy(self):
+        """
+        clean up data associated with image
+        """
+        pass
diff --git a/tests/cloud_tests/images/lxd.py b/tests/cloud_tests/images/lxd.py
new file mode 100644
index 0000000..b06c980
--- /dev/null
+++ b/tests/cloud_tests/images/lxd.py
@@ -0,0 +1,88 @@
+from tests.cloud_tests.images import base
+from tests.cloud_tests.snapshots import lxd as lxd_snapshot
+
+
+class LXDImage(base.Image):
+    """
+    LXD backed image
+    """
+    platform_name = "lxd"
+
+    def __init__(self, name, config, platform, pylxd_image):
+        """
+        setup
+        """
+        self.platform = platform
+        self._pylxd_image = pylxd_image
+        self._instance = None
+        super(LXDImage, self).__init__(name, config, platform)
+
+    @property
+    def pylxd_image(self):
+        self._pylxd_image.sync()
+        return self._pylxd_image
+
+    @property
+    def instance(self):
+        if not self._instance:
+            self._instance = self.platform.launch_container(
+                image=self.pylxd_image.fingerprint,
+                image_desc=str(self), use_desc='image-modification')
+        self._instance.start(wait=True, wait_time=self.config.get('timeout'))
+        return self._instance
+
+    @property
+    def properties(self):
+        """
+        {} containing: 'arch', 'os', 'version', 'release'
+        """
+        properties = self.pylxd_image.properties
+        return {
+            'arch': properties.get('architecture'),
+            'os': properties.get('os'),
+            'version': properties.get('version'),
+            'release': properties.get('release'),
+        }
+
+    def execute(self, *args, **kwargs):
+        """
+        execute command in image, modifying image
+        """
+        return self.instance.execute(*args, **kwargs)
+
+    def push_file(self, local_path, remote_path):
+        """
+        copy file at 'local_path' to instance at 'remote_path', modifying image
+        """
+        return self.instance.push_file(local_path, remote_path)
+
+    def run_script(self, script):
+        """
+        run script in image, modifying image
+        return_value: script output
+        """
+        return self.instance.run_script(script)
+
+    def snapshot(self):
+        """
+        create snapshot of image, block until done
+        """
+        # clone current instance, start and freeze clone
+        instance = self.platform.launch_container(
+            container=self.instance.name, image_desc=str(self),
+            use_desc='snapshot')
+        instance.start(wait=True, wait_time=self.config.get('timeout'))
+        if self.config.get('boot_clean_script'):
+            instance.run_script(self.config.get('boot_clean_script'))
+        instance.freeze()
+        return lxd_snapshot.LXDSnapshot(
+            self.properties, self.config, self.platform, instance)
+
+    def destroy(self):
+        """
+        clean up data associated with image
+        """
+        if self._instance:
+            self._instance.destroy()
+        self.pylxd_image.delete(wait=True)
+        super(LXDImage, self).destroy()
diff --git a/tests/cloud_tests/instances/__init__.py b/tests/cloud_tests/instances/__init__.py
new file mode 100644
index 0000000..fd9cbaf
--- /dev/null
+++ b/tests/cloud_tests/instances/__init__.py
@@ -0,0 +1,5 @@
+def get_instance(snapshot, *args, **kwargs):
+    """
+    get instance from snapshot
+    """
+    return snapshot.launch(*args, **kwargs)
diff --git a/tests/cloud_tests/instances/base.py b/tests/cloud_tests/instances/base.py
new file mode 100644
index 0000000..fd5bd41
--- /dev/null
+++ b/tests/cloud_tests/instances/base.py
@@ -0,0 +1,116 @@
+import os
+import uuid
+
+
+class Instance(object):
+    """
+    Base instance object
+    """
+    platform_name = None
+
+    def __init__(self, name):
+        """
+        setup
+        """
+        self.name = name
+
+    def execute(self, command, stdin=None, stdout=None, stderr=None, env={}):
+        """
+        command: the command to execute as root inside the image
+        stdin, stderr, stdout: file handles
+        env: environment variables
+
+        Execute assumes functional networking and execution as root with the
+        target filesystem being available at /.
+
+        return_value: tuple containing stdout data, stderr data, exit code
+        """
+        raise NotImplementedError
+
+    def read_data(self, remote_path, encode=False):
+        """
+        read_data from instance filesystem
+        remote_path: path in instance
+        decode: return as string
+        return_value: data as str or bytes
+        """
+        raise NotImplementedError
+
+    def write_data(self, remote_path, data):
+        """
+        write data to instance filesystem
+        remote_path: path in instance
+        data: data to write, either str or bytes
+        """
+        raise NotImplementedError
+
+    def pull_file(self, remote_path, local_path):
+        """
+        copy file at 'remote_path', from instance to 'local_path'
+        """
+        with open(local_path, 'wb') as fp:
+            fp.write(self.read_data(remote_path), encode=True)
+
+    def push_file(self, local_path, remote_path):
+        """
+        copy file at 'local_path' to instance at 'remote_path'
+        """
+        with open(local_path, 'rb') as fp:
+            self.write_data(remote_path, fp.read())
+
+    def run_script(self, script):
+        """
+        run script in target and return stdout
+        """
+        script_path = os.path.join('/tmp', str(uuid.uuid1()))
+        self.write_data(script_path, script)
+        (out, err, exit_code) = self.execute(['/bin/bash', script_path])
+        return out
+
+    def console_log(self):
+        """
+        return_value: bytes of this instance’s console
+        """
+        raise NotImplementedError
+
+    def reboot(self, wait=True):
+        """
+        reboot instance
+        """
+        raise NotImplementedError
+
+    def shutdown(self, wait=True):
+        """
+        shutdown instance
+        """
+        raise NotImplementedError
+
+    def start(self, wait=True):
+        """
+        start instance
+        """
+        raise NotImplementedError
+
+    def destroy(self):
+        """
+        clean up instance
+        """
+        pass
+
+    def _wait_for_cloud_init(self, wait_time):
+        """
+        wait until system has fully booted and cloud-init has finished
+        """
+        if not wait_time:
+            return
+
+        found_msg = 'found'
+        cmd = ('for ((i=0;i<{wait};i++)); do [ -f "{file}" ] && '
+               '{{ echo "{msg}";break; }} || sleep 1; done').format(
+            file='/run/cloud-init/result.json',
+            wait=wait_time, msg=found_msg)
+
+        (out, err, exit) = self.execute(['/bin/bash', '-c', cmd])
+        if out.strip() != found_msg:
+            raise OSError('timeout: after {}s, cloud-init has not started'
+                          .format(wait_time))
diff --git a/tests/cloud_tests/instances/lxd.py b/tests/cloud_tests/instances/lxd.py
new file mode 100644
index 0000000..39bd5e5
--- /dev/null
+++ b/tests/cloud_tests/instances/lxd.py
@@ -0,0 +1,117 @@
+from tests.cloud_tests.instances import base
+
+
+class LXDInstance(base.Instance):
+    """
+    LXD container backed instance
+    """
+    platform_name = "lxd"
+
+    def __init__(self, name, platform, pylxd_container):
+        """
+        setup
+        """
+        self.platform = platform
+        self._pylxd_container = pylxd_container
+        super(LXDInstance, self).__init__(name)
+
+    @property
+    def pylxd_container(self):
+        self._pylxd_container.sync()
+        return self._pylxd_container
+
+    def execute(self, command, stdin=None, stdout=None, stderr=None, env={}):
+        """
+        command: the command to execute as root inside the image
+        stdin, stderr, stdout: file handles
+        env: environment variables
+
+        Execute assumes functional networking and execution as root with the
+        target filesystem being available at /.
+
+        return_value: tuple containing stdout data, stderr data, exit code
+        """
+        # TODO: the pylxd api handler for container.execute needs to be
+        #       extended to properly pass in stdin
+        # TODO: the pylxd api handler for container.execute needs to be
+        #       extended to get the return code, for now just use 0
+        self.start()
+        if stdin:
+            raise NotImplementedError
+        res = self.pylxd_container.execute(command, environment=env)
+        for (f, data) in (i for i in zip((stdout, stderr), res) if i[0]):
+            f.write(data)
+        return res + (0,)
+
+    def read_data(self, remote_path, decode=False):
+        """
+        read data from instance filesystem
+        remote_path: path in instance
+        decode: return as string
+        return_value: data as str or bytes
+        """
+        data = self.pylxd_container.files.get(remote_path)
+        return data.decode() if decode and isinstance(data, bytes) else data
+
+    def write_data(self, remote_path, data):
+        """
+        write data to instance filesystem
+        remote_path: path in instance
+        data: data to write, either str or bytes
+        """
+        self.pylxd_container.files.put(remote_path, data)
+
+    def console_log(self):
+        """
+        return_value: bytes of this instance’s console
+        """
+        raise NotImplementedError
+
+    def reboot(self, wait=True):
+        """
+        reboot instance
+        """
+        self.shutdown(wait=wait)
+        self.start(wait=wait)
+
+    def shutdown(self, wait=True):
+        """
+        shutdown instance
+        """
+        if self.pylxd_container.status != 'Stopped':
+            self.pylxd_container.stop(wait=wait)
+
+    def start(self, wait=True, wait_time=None):
+        """
+        start instance
+        """
+        if self.pylxd_container.status != 'Running':
+            self.pylxd_container.start(wait=wait)
+            if wait and isinstance(wait_time, int):
+                self._wait_for_cloud_init(wait_time)
+
+    def freeze(self):
+        """
+        freeze instance
+        """
+        if self.pylxd_container.status != 'Frozen':
+            self.pylxd_container.freeze(wait=True)
+
+    def unfreeze(self):
+        """
+        unfreeze instance
+        """
+        if self.pylxd_container.status == 'Frozen':
+            self.pylxd_container.unfreeze(wait=True)
+
+    def destroy(self):
+        """
+        clean up instance
+        """
+        self.unfreeze()
+        self.shutdown()
+        self.pylxd_container.delete(wait=True)
+        if self.platform.container_exists(self.name):
+            raise OSError('container {} was not properly removed'
+                          .format(self.name))
+        super(LXDInstance, self).destroy()
diff --git a/tests/cloud_tests/manage.py b/tests/cloud_tests/manage.py
new file mode 100644
index 0000000..55d7dd9
--- /dev/null
+++ b/tests/cloud_tests/manage.py
@@ -0,0 +1,71 @@
+from tests.cloud_tests.config import VERIFY_EXT
+from tests.cloud_tests import (config, util)
+from tests.cloud_tests import TESTCASES_DIR
+
+import os
+import textwrap
+
+_verifier_fmt = textwrap.dedent(
+    """
+    \"\"\"cloud-init Integration Test Verify Script\"\"\"
+    from tests.cloud_tests.testcases import base
+
+
+    class {test_class}(base.CloudTestCase):
+        \"\"\"
+        Name: {test_name}
+        Category: {test_category}
+        Description: {test_description}
+        \"\"\"
+        pass
+    """
+).lstrip()
+_config_fmt = textwrap.dedent(
+    """
+    #
+    # Name: {test_name}
+    # Category: {test_category}
+    # Description: {test_description}
+    #
+    {config}
+    """
+).strip()
+
+
+def write_testcase_config(args, fmt_args, testcase_file):
+    """
+    write the testcase config file
+    """
+    testcase_config = {'enabled': args.enable, 'collect_scripts': {}}
+    if args.config:
+        testcase_config['cloud_config'] = args.config
+    fmt_args['config'] = util.yaml_format(testcase_config)
+    util.write_file(testcase_file, _config_fmt.format(**fmt_args), omode='w')
+
+
+def write_verifier(args, fmt_args, verifier_file):
+    """
+    write the verifier script
+    """
+    fmt_args['test_class'] = 'Test{}'.format(
+        config.name_sanatize(fmt_args['test_name']).title())
+    util.write_file(verifier_file, _verifier_fmt.format(**fmt_args), omode='w')
+
+
+def create(args):
+    """
+    create a new testcase
+    """
+    (test_category, test_name) = args.name.split('/')
+    fmt_args = {'test_name': test_name, 'test_category': test_category,
+                'test_description': str(args.description)}
+
+    testcase_file = config.name_to_path(args.name)
+    verifier_file = os.path.join(
+        TESTCASES_DIR, test_category,
+        config.name_sanatize(test_name) + VERIFY_EXT)
+
+    write_testcase_config(args, fmt_args, testcase_file)
+    write_verifier(args, fmt_args, verifier_file)
+
+    return 0
diff --git a/tests/cloud_tests/platforms.yaml b/tests/cloud_tests/platforms.yaml
new file mode 100644
index 0000000..ffed29a
--- /dev/null
+++ b/tests/cloud_tests/platforms.yaml
@@ -0,0 +1,15 @@
+# ============================= Platform Config ===============================
+default_platform_config:
+    # all disabled by default
+    enabled: false
+    # maximum time to retrieve image
+    get_image_timeout: 300
+    # maximum time to create instance (before waiting for cloud-init)
+    create_instance_timeout: 60
+
+platforms:
+    lxd:
+        enabled: true
+        get_image_timeout: 600
+    ec2: {}
+    azure: {}
diff --git a/tests/cloud_tests/platforms/__init__.py b/tests/cloud_tests/platforms/__init__.py
new file mode 100644
index 0000000..52e453f
--- /dev/null
+++ b/tests/cloud_tests/platforms/__init__.py
@@ -0,0 +1,15 @@
+from tests.cloud_tests.platforms import lxd
+
+PLATFORMS = {
+    'lxd': lxd.LXDPlatform,
+}
+
+
+def get_platform(platform_name, config):
+    """
+    Get the platform object for 'platform_name' and init
+    """
+    platform_cls = PLATFORMS.get(platform_name)
+    if not platform_cls:
+        raise ValueError('invalid platform name: {}'.format(platform_name))
+    return platform_cls(config)
diff --git a/tests/cloud_tests/platforms/base.py b/tests/cloud_tests/platforms/base.py
new file mode 100644
index 0000000..3679e85
--- /dev/null
+++ b/tests/cloud_tests/platforms/base.py
@@ -0,0 +1,48 @@
+class Platform(object):
+    """
+    Base class for platforms
+    """
+    platform_name = None
+
+    def __init__(self, config):
+        """
+        Set up platform
+        """
+        self.config = config
+
+    def get_image(self, img_conf):
+        """
+        Get image using 'img_conf', where img_conf is a dict containing all
+        image configuration parameters
+
+        in this dict there must be a 'platform_ident' key containing
+        configuration for identifying each image on a per platform basis
+
+        see implementations for get_image() for details about the contents
+        of the platform's config entry
+
+        note: see 'releases' main_config.yaml for example entries
+
+        img_conf: configuration for image
+        return_value: cloud_tests.images instance
+        """
+        raise NotImplementedError
+
+    def destroy(self):
+        """
+        Clean up platform data
+        """
+        pass
+
+    def _extract_img_platform_config(self, img_conf):
+        """
+        extract platform configuration for current platform from img_conf
+        """
+        platform_ident = img_conf.get('platform_ident')
+        if not platform_ident:
+            raise ValueError('invalid img_conf, missing \'platform_ident\'')
+        ident = platform_ident.get(self.platform_name)
+        if not ident:
+            raise ValueError('img_conf: {} missing config for platform {}'
+                             .format(img_conf, self.platform_name))
+        return ident
diff --git a/tests/cloud_tests/platforms/lxd.py b/tests/cloud_tests/platforms/lxd.py
new file mode 100644
index 0000000..414a162
--- /dev/null
+++ b/tests/cloud_tests/platforms/lxd.py
@@ -0,0 +1,93 @@
+from pylxd import (Client, exceptions)
+
+from tests.cloud_tests.images import lxd as lxd_image
+from tests.cloud_tests.instances import lxd as lxd_instance
+from tests.cloud_tests.platforms import base
+from tests.cloud_tests import util
+
+DEFAULT_SSTREAMS_SERVER = "https://images.linuxcontainers.org:8443"
+
+
+class LXDPlatform(base.Platform):
+    """
+    Lxd test platform
+    """
+    platform_name = 'lxd'
+
+    def __init__(self, config):
+        """
+        Set up platform
+        """
+        super(LXDPlatform, self).__init__(config)
+        # TODO: allow configuration of remote lxd host via env variables
+        # set up lxd connection
+        self.client = Client()
+
+    def get_image(self, img_conf):
+        """
+        Get image
+        img_conf: dict containing config for image. platform_ident must have:
+            alias: alias to use for simplestreams server
+            sstreams_server: simplestreams server to use, or None for default
+        return_value: cloud_tests.images instance
+        """
+        lxd_conf = self._extract_img_platform_config(img_conf)
+        image = self.client.images.create_from_simplestreams(
+            lxd_conf.get('sstreams_server', DEFAULT_SSTREAMS_SERVER),
+            lxd_conf['alias'])
+        return lxd_image.LXDImage(
+            image.properties['description'], img_conf, self, image)
+
+    def launch_container(self, image=None, container=None, ephemeral=False,
+                         config=None, block=True,
+                         image_desc=None, use_desc=None):
+        """
+        launch a container
+        image: image fingerprint to launch from
+        container: container to copy
+        ephemeral: delete image after first shutdown
+        config: config options for instance as dict
+        block: wait until container created
+        image_desc: description of image being launched
+        use_desc: description of container's use
+        return_value: cloud_tests.instances instance
+        """
+        if not (image or container):
+            raise ValueError("either image or container must be specified")
+        container = self.client.containers.create({
+            'name': util.gen_instance_name(image_desc=image_desc,
+                                           use_desc=use_desc,
+                                           used_list=self.list_containers()),
+            'ephemeral': bool(ephemeral),
+            'config': config if isinstance(config, dict) else {},
+            'source': ({'type': 'image', 'fingerprint': image} if image else
+                       {'type': 'copy', 'source': container})
+        }, wait=block)
+        return lxd_instance.LXDInstance(container.name, self, container)
+
+    def container_exists(self, container_name):
+        """
+        check if container with name 'container_name' exists
+        return_value: True if exists else False
+        """
+        res = True
+        try:
+            self.client.containers.get(container_name)
+        except exceptions.LXDAPIException as e:
+            res = False
+            if e.response.status_code != 404:
+                raise
+        return res
+
+    def list_containers(self):
+        """
+        list names of all containers
+        return_value: list of names
+        """
+        return [container.name for container in self.client.containers.all()]
+
+    def destroy(self):
+        """
+        Clean up platform data
+        """
+        super(LXDPlatform, self).destroy()
diff --git a/tests/cloud_tests/releases.yaml b/tests/cloud_tests/releases.yaml
new file mode 100644
index 0000000..a3a6e4d
--- /dev/null
+++ b/tests/cloud_tests/releases.yaml
@@ -0,0 +1,77 @@
+# ============================= Release Config ================================
+default_release_config:
+    # all are disabled by default
+    enabled: false
+    # timeout for booting image and running cloud init
+    timeout: 120
+    # platform_ident values for the image, with data to identify the image
+    # on that platform. see platforms.base for more information
+    platform_ident: {}
+    # a script to run after a boot that is used to modify an image, before
+    # making a snapshot of the image. may be useful for removing data left
+    # behind from cloud-init booting, such as logs, to ensure that data from
+    # snapshot.launch() will not include a cloud-init.log from a boot used to
+    # create the snapshot, if cloud-init has not run
+    boot_clean_script: |
+        #!/bin/bash
+        rm -rf /var/log/cloud-init.log /var/log/cloud-init-output.log \
+            /var/lib/cloud/ /run/cloud-init/ /var/log/syslog
+
+releases:
+    trusty:
+        enabled: true
+        platform_ident:
+            lxd:
+                # if sstreams_server is omitted, default is used, defined in
+                # tests.cloud_tests.platforms.lxd.DEFAULT_SSTREAMS_SERVER as:
+                # sstreams_server: https://us.images.linuxcontainers.org:8443
+                #alias: ubuntu/trusty/default
+                alias: t
+                sstreams_server: https://cloud-images.ubuntu.com/daily
+    xenial:
+        enabled: true
+        platform_ident:
+            lxd:
+                #alias: ubuntu/xenial/default
+                alias: x
+                sstreams_server: https://cloud-images.ubuntu.com/daily
+    yakkety:
+        enabled: true
+        platform_ident:
+            lxd:
+                #alias: ubuntu/yakkety/default
+                alias: y
+                sstreams_server: https://cloud-images.ubuntu.com/daily
+    zesty:
+        enabled: true
+        platform_ident:
+            lxd:
+                #alias: ubuntu/zesty/default
+                alias: z
+                sstreams_server: https://cloud-images.ubuntu.com/daily
+    jessie:
+        platform_ident:
+            lxd:
+                alias: debian/jessie/default
+    sid:
+        platform_ident:
+            lxd:
+                alias: debian/sid/default
+    stretch:
+        platform_ident:
+            lxd:
+                alias: debian/stretch/default
+    wheezy:
+        platform_ident:
+            lxd:
+                alias: debian/wheezy/default
+    centos70:
+        timeout: 180
+        platform_ident:
+            lxd:
+                alias: centos/7/default
+    centos66:
+        timeout: 180
+        platform_ident:
+            lxd:
+                alias: centos/6/default
diff --git a/tests/cloud_tests/setup_image.py b/tests/cloud_tests/setup_image.py
new file mode 100644
index 0000000..488c7fa
--- /dev/null
+++ b/tests/cloud_tests/setup_image.py
@@ -0,0 +1,191 @@
+from tests.cloud_tests import LOG
+from tests.cloud_tests import stage, util
+
+from functools import partial
+import os
+
+
+def install_deb(args, image):
+    """
+    install deb into image
+    args: cmdline arguments, must contain --deb
+    image: cloud_tests.images instance to operate on
+    return_value: None, may raise errors
+    """
+    # ensure system is compatible with package format
+    os_family = util.get_os_family(image.properties['os'])
+    if os_family != 'debian':
+        raise NotImplementedError('install deb: {} not supported on os '
+                                  'family: {}'.format(args.deb, os_family))
+
+    # install deb
+    LOG.debug('installing deb: %s into target', args.deb)
+    remote_path = os.path.join('/tmp', os.path.basename(args.deb))
+    image.push_file(args.deb, remote_path)
+    (out, err, exit) = image.execute(['dpkg', '-i', remote_path])
+    if exit != 0:
+        raise OSError('failed install deb: {}\n\tstdout: {}\n\tstderr: {}'
+                      .format(args.deb, out, err))
+
+    # check installed deb version matches package
+    fmt = ['-W', "--showformat='${Version}'"]
+    (out, err, exit) = image.execute(['dpkg-deb'] + fmt + [remote_path])
+    expected_version = out.strip()
+    (out, err, exit) = image.execute(['dpkg-query'] + fmt + ['cloud-init'])
+    found_version = out.strip()
+    if expected_version != found_version:
+        raise OSError('install deb version "{}" does not match expected "{}"'
+                      .format(found_version, expected_version))
+
+    LOG.debug('successfully installed: %s, version: %s', args.deb,
+              found_version)
+
+
+def install_rpm(args, image):
+    """
+    install rpm into image
+    args: cmdline arguments, must contain --rpm
+    image: cloud_tests.images instance to operate on
+    return_value: None, may raise errors
+    """
+    # ensure system is compatible with package format
+    os_family = util.get_os_family(image.properties['os'])
+    if os_family not in ['redhat', 'sles']:
+        raise NotImplementedError('install rpm: {} not supported on os '
+                                  'family: {}'.format(args.rpm, os_family))
+
+    # install rpm
+    LOG.debug('installing rpm: %s into target', args.rpm)
+    remote_path = os.path.join('/tmp', os.path.basename(args.rpm))
+    image.push_file(args.rpm, remote_path)
+    (out, err, exit) = image.execute(['rpm', '-U', remote_path])
+    if exit != 0:
+        raise OSError('failed to install rpm: {}\n\tstdout: {}\n\tstderr: {}'
+                      .format(args.rpm, out, err))
+
+    fmt = ['--queryformat', '"%{VERSION}"']
+    (out, err, exit) = image.execute(['rpm', '-q'] + fmt + [remote_path])
+    expected_version = out.strip()
+    (out, err, exit) = image.execute(['rpm', '-q'] + fmt + ['cloud-init'])
+    found_version = out.strip()
+    if expected_version != found_version:
+        raise OSError('install rpm version "{}" does not match expected "{}"'
+                      .format(found_version, expected_version))
+
+    LOG.debug('successfully installed: %s, version %s', args.rpm,
+              found_version)
+
+
+def upgrade(args, image):
+    """
+    run the system's upgrade command
+    args: cmdline arguments
+    image: cloud_tests.images instance to operate on
+    return_value: None, may raise errors
+    """
+    # determine appropriate upgrade command for os_family
+    # TODO: maybe use cloudinit.distros for this?
+    os_family = util.get_os_family(image.properties['os'])
+    if os_family == 'debian':
+        cmd = 'apt-get update && apt-get upgrade --yes'
+    elif os_family == 'redhat':
+        cmd = 'yum upgrade --assumeyes'
+    else:
+        raise NotImplementedError('upgrade command not configured for distro '
+                                  'from family: {}'.format(os_family))
+
+    # upgrade system
+    LOG.debug('upgrading system')
+    (out, err, exit) = image.execute(['/bin/sh', '-c', cmd])
+    if exit != 0:
+        raise OSError('failed to upgrade system\n\tstdout: {}\n\tstderr:{}'
+                      .format(out, err))
+
+
+def run_script(args, image):
+    """
+    run a script in the target image
+    args: cmdline arguments, must contain --script
+    image: cloud_tests.images instance to operate on
+    return_value: None, may raise errors
+    """
+    # TODO: get exit status back from script and add error handling here
+    LOG.debug('running setup image script in target image')
+    image.run_script(args.script)
+
+
+def enable_ppa(args, image):
+    """
+    enable a ppa in the target image
+    args: cmdline arguments, must contain --ppa
+    image: cloud_tests.image instance to operate on
+    return_value: None, may raise errors
+    """
+    # ppa only supported on ubuntu (maybe debian?)
+    if image.properties['os'] != 'ubuntu':
+        raise NotImplementedError('enabling a ppa is only available on ubuntu')
+
+    # add ppa with add-apt-repository and update
+    ppa = 'ppa:{}'.format(args.ppa)
+    LOG.debug('enabling %s', ppa)
+    cmd = 'add-apt-repository --yes {} && apt-get update'.format(ppa)
+    (out, err, exit) = image.execute(['/bin/sh', '-c', cmd])
+    if exit != 0:
+        raise OSError('enable ppa for {} failed\n\tstdout: {}\n\tstderr: {}'
+                      .format(ppa, out, err))
+
+
+def enable_repo(args, image):
+    """
+    enable a repository in the target image
+    args: cmdline arguments, must contain --repo
+    image: cloud_tests.image instance to operate on
+    return_value: None, may raise errors
+    """
+    # find enable repo command for the distro
+    os_family = util.get_os_family(image.properties['os'])
+    if os_family == 'debian':
+        cmd = ('echo "{}" >> "/etc/apt/sources.list" '.format(args.repo) +
+               '&& apt-get update')
+    elif os_family == 'centos':
+        cmd = 'yum-config-manager --add-repo="{}"'.format(args.repo)
+    else:
+        raise NotImplementedError('enable repo command not configured for '
+                                  'distro from family: {}'.format(os_family))
+
+    LOG.debug('enabling repo: "%s"', args.repo)
+    (out, err, exit) = image.execute(['/bin/sh', '-c', cmd])
+    if exit != 0:
+        raise OSError('enable repo {} failed\n\tstdout: {}\n\tstderr: {}'
+                      .format(args.repo, out, err))
+
+
+def setup_image(args, image):
+    """
+    set up image as specified in args
+    args: cmdline arguments
+    image: cloud_tests.image instance to operate on
+    return_value: tuple of results and fail count
+    """
+    # mapping of setup cmdline arg name to setup function
+    # represented as a tuple rather than a dict or odict as lookup by name not
+    # needed, and order is important as --script and --upgrade go at the end
+    handlers = (
+        # arg   handler     description
+        ('deb', install_deb, 'setup func for --deb, install deb'),
+        ('rpm', install_rpm, 'setup func for --rpm, install rpm'),
+        ('repo', enable_repo, 'setup func for --repo, enable repo'),
+        ('ppa', enable_ppa, 'setup func for --ppa, enable ppa'),
+        ('script', run_script, 'setup func for --script, run script'),
+        ('upgrade', upgrade, 'setup func for --upgrade, upgrade pkgs'),
+    )
+
+    # determine which setup functions needed
+    calls = [partial(stage.run_single, desc, partial(func, args, image))
+             for name, func, desc in handlers if getattr(args, name, None)]
+
+    image_name = 'image: distro={}, release={}'.format(
+        image.properties['os'], image.properties['release'])
+    LOG.info('setting up %s', image_name)
+    return stage.run_stage('set up for {}'.format(image_name), calls,
+                           continue_after_error=False)
diff --git a/tests/cloud_tests/snapshots/__init__.py b/tests/cloud_tests/snapshots/__init__.py
new file mode 100644
index 0000000..328275c
--- /dev/null
+++ b/tests/cloud_tests/snapshots/__init__.py
@@ -0,0 +1,5 @@
+def get_snapshot(image):
+    """
+    get snapshot from image
+    """
+    return image.snapshot()
diff --git a/tests/cloud_tests/snapshots/base.py b/tests/cloud_tests/snapshots/base.py
new file mode 100644
index 0000000..5c4a6ce
--- /dev/null
+++ b/tests/cloud_tests/snapshots/base.py
@@ -0,0 +1,39 @@
+class Snapshot(object):
+    """
+    Base class for snapshots
+    """
+    platform_name = None
+
+    def __init__(self, properties, config):
+        """
+        Set up snapshot
+        """
+        self.properties = properties
+        self.config = config
+
+    def __str__(self):
+        """
+        a brief description of the snapshot
+        """
+        return '-'.join((self.properties['os'], self.properties['release']))
+
+    def launch(self, user_data, meta_data=None, block=True, start=True,
+               use_desc=None):
+        """
+        launch instance
+
+        user_data: user-data for the instance
+        instance_id: instance-id for the instance
+        block: wait until instance is created
+        start: start instance and wait until fully started
+        use_desc: description of snapshot instance use
+
+        return_value: an Instance
+        """
+        raise NotImplementedError
+
+    def destroy(self):
+        """
+        Clean up snapshot data
+        """
+        pass
diff --git a/tests/cloud_tests/snapshots/lxd.py b/tests/cloud_tests/snapshots/lxd.py
new file mode 100644
index 0000000..dd71b20
--- /dev/null
+++ b/tests/cloud_tests/snapshots/lxd.py
@@ -0,0 +1,46 @@
+from tests.cloud_tests.snapshots import base
+
+
+class LXDSnapshot(base.Snapshot):
+    """
+    LXD image copy backed snapshot
+    """
+    platform_name = "lxd"
+
+    def __init__(self, properties, config, platform, pylxd_frozen_instance):
+        """
+        Set up snapshot
+        """
+        self.platform = platform
+        self.pylxd_frozen_instance = pylxd_frozen_instance
+        super(LXDSnapshot, self).__init__(properties, config)
+
+    def launch(self, user_data, meta_data=None, block=True, start=True,
+               use_desc=None):
+        """
+        launch instance
+
+        user_data: user-data for the instance
+        instance_id: instance-id for the instance
+        block: wait until instance is created
+        start: start instance and wait until fully started
+        use_desc: description of snapshot instance use
+
+        return_value: an Instance
+        """
+        inst_config = {'user.user-data': user_data}
+        if meta_data:
+            inst_config['user.meta-data'] = meta_data
+        instance = self.platform.launch_container(
+            container=self.pylxd_frozen_instance.name, config=inst_config,
+            block=block, image_desc=str(self), use_desc=use_desc)
+        if start:
+            instance.start(wait=True, wait_time=self.config.get('timeout'))
+        return instance
+
+    def destroy(self):
+        """
+        Clean up snapshot data
+        """
+        self.pylxd_frozen_instance.destroy()
+        super(LXDSnapshot, self).destroy()
diff --git a/tests/cloud_tests/stage.py b/tests/cloud_tests/stage.py
new file mode 100644
index 0000000..2cf7f82
--- /dev/null
+++ b/tests/cloud_tests/stage.py
@@ -0,0 +1,109 @@
+import sys
+import time
+import traceback
+
+from tests.cloud_tests import LOG
+
+
+class PlatformComponent(object):
+    """
+    context manager to safely handle platform components, ensuring that
+    .destroy() is called
+    """
+
+    def __init__(self, get_func):
+        """
+        store get_<platform component> function as partial taking no args
+        """
+        self.get_func = get_func
+
+    def __enter__(self):
+        """
+        create instance of platform component
+        """
+        self.instance = self.get_func()
+        return self.instance
+
+    def __exit__(self, etype, value, trace):
+        """
+        destroy instance
+        """
+        if self.instance is not None:
+            self.instance.destroy()
+
+
+def run_single(name, call):
+    """
+    run a single function, keeping track of results and failures and time
+    name: name of part
+    call: call to make
+    return_value: a tuple of result and fail count
+    """
+    res = {
+        'name': name,
+        'time': 0,
+        'errors': [],
+        'success': False
+    }
+    failed = 0
+    start_time = time.time()
+
+    try:
+        call()
+    except Exception as e:
+        failed += 1
+        res['errors'].append(str(e))
+        LOG.error('stage part: %s encountered error: %s', name, str(e))
+        trace = traceback.extract_tb(sys.exc_info()[-1])
+        LOG.error('traceback:\n%s', ''.join(traceback.format_list(trace)))
+
+    res['time'] = time.time() - start_time
+    if failed == 0:
+        res['success'] = True
+
+    return res, failed
+
+
+def run_stage(parent_name, calls, continue_after_error=True):
+    """
+    run a stage of collection, keeping track of results and failures
+    parent_name: name of stage calls are under
+    calls: list of function call taking no params. must return a tuple
+           of results and failures. may raise exceptions
+    continue_after_error: whether or not to proceed to the next call after
+                          catching an exception or recording a failure
+    return_value: a tuple of results and failures, with result containing
+                  results from the function call under 'stages', and a list
+                  of errors (if any on this level), and elapsed time
+                  running stage, and the name
+    """
+    res = {
+        'name': parent_name,
+        'time': 0,
+        'errors': [],
+        'stages': [],
+        'success': False,
+    }
+    failed = 0
+    start_time = time.time()
+
+    for call in calls:
+        try:
+            (call_res, call_failed) = call()
+            res['stages'].append(call_res)
+        except Exception as e:
+            call_failed = 1
+            res['errors'].append(str(e))
+            LOG.error('stage: %s encountered error: %s', parent_name, str(e))
+            trace = traceback.extract_tb(sys.exc_info()[-1])
+            LOG.error('traceback:\n%s', ''.join(traceback.format_list(trace)))
+
+        failed += call_failed
+        if call_failed and not continue_after_error:
+            break
+
+    res['time'] = time.time() - start_time
+    if not failed:
+        res['success'] = True
+
+    return (res, failed)
diff --git a/tests/cloud_tests/testcases.yaml b/tests/cloud_tests/testcases.yaml
new file mode 100644
index 0000000..02701bd
--- /dev/null
+++ b/tests/cloud_tests/testcases.yaml
@@ -0,0 +1,25 @@
+# ============================= Base Test Config ==============================
+base_test_data:
+    script_timeout: 20
+    enabled: True
+    cloud_config: |
+        #cloud-config
+    collect_scripts:
+        cloud-init.log: |
+            #!/bin/bash
+            cat /var/log/cloud-init.log
+        cloud-init-output.log: |
+            #!/bin/bash
+            cat /var/log/cloud-init-output.log
+        instance-id: |
+            #!/bin/bash
+            cat /run/cloud-init/.instance-id
+        result.json: |
+            #!/bin/bash
+            cat /run/cloud-init/result.json
+        status.json: |
+            #!/bin/bash
+            cat /run/cloud-init/status.json
+        cloud-init-version: |
+            #!/bin/bash
+            dpkg-query -W -f='${Version}' cloud-init
diff --git a/tests/cloud_tests/testcases/__init__.py b/tests/cloud_tests/testcases/__init__.py
new file mode 100644
index 0000000..f5cb0fd
--- /dev/null
+++ b/tests/cloud_tests/testcases/__init__.py
@@ -0,0 +1,43 @@
+import importlib
+import inspect
+import unittest
+
+from tests.cloud_tests import config
+from tests.cloud_tests.testcases.base import CloudTestCase as base_test
+
+
+def discover_tests(test_name):
+    """
+    discover tests in test file for 'testname'
+    return_value: list of test classes
+    """
+    testmod_name = 'tests.cloud_tests.testcases.{}'.format(
+        config.name_sanatize(test_name))
+    try:
+        testmod = importlib.import_module(testmod_name)
+    except NameError:
+        raise ValueError('no test verifier found at: {}'.format(testmod_name))
+
+    return [mod for name, mod in inspect.getmembers(testmod)
+            if inspect.isclass(mod) and base_test in mod.__bases__ and
+            getattr(mod, '__test__', True)]
+
+
+def get_suite(test_name, data, conf):
+    """
+    get test suite with all tests for 'testname'
+    return_value: a test suite
+    """
+    suite = unittest.TestSuite()
+    for test_class in discover_tests(test_name):
+
+        class tmp(test_class):
+
+            @classmethod
+            def setUpClass(cls):
+                cls.data = data
+                cls.conf = conf
+
+        suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(tmp))
+
+    return suite
diff --git a/tests/cloud_tests/testcases/base.py b/tests/cloud_tests/testcases/base.py
new file mode 100644
index 0000000..6d6c282
--- /dev/null
+++ b/tests/cloud_tests/testcases/base.py
@@ -0,0 +1,77 @@
+from cloudinit import util as c_util
+
+import json
+import unittest
+
+
+class CloudTestCase(unittest.TestCase):
+    """
+    base test class for verifiers
+    """
+    data = None
+    conf = None
+    _cloud_config = None
+
+    @property
+    def cloud_config(self):
+        """
+        get the cloud-config used by the test
+        """
+        if not self._cloud_config:
+            self._cloud_config = c_util.load_yaml(self.conf)
+        return self._cloud_config
+
+    def get_config_entry(self, name):
+        """
+        get a config entry from cloud-config ensuring that it is present
+        """
+        if name not in self.cloud_config:
+            raise AssertionError('Key "{}" not in cloud config'.format(name))
+        return self.cloud_config[name]
+
+    def get_data_file(self, name):
+        """
+        get data file failing test if it is not present
+        """
+        if name not in self.data:
+            raise AssertionError('File "{}" missing from collect data'
+                                 .format(name))
+        return self.data[name]
+
+    def get_instance_id(self):
+        """
+        get recorded instance id
+        """
+        return self.get_data_file('instance-id').strip()
+
+    def get_status_data(self, data, version=None):
+        """
+        parse result.json and status.json like data files
+        data: data to load
+        version: cloud-init output version, defaults to 'v1'
+        return_value: dict of data or None if missing
+        """
+        if not version:
+            version = 'v1'
+        data = json.loads(data)
+        return data.get(version)
+
+    def get_datasource(self):
+        """
+        get datasource name
+        """
+        data = self.get_status_data(self.get_data_file('result.json'))
+        return data.get('datasource')
+
+    def test_no_stages_errors(self):
+        """
+        ensure that there were no errors in any stage
+        """
+        status = self.get_status_data(self.get_data_file('status.json'))
+        for stage in ('init', 'init-local', 'modules-config', 'modules-final'):
+            self.assertIn(stage, status)
+            self.assertEqual(len(status[stage]['errors']), 0,
+                             'errors {} were encountered in stage {}'
+                             .format(status[stage]['errors'], stage))
+        result = self.get_status_data(self.get_data_file('result.json'))
+        self.assertEqual(len(result['errors']), 0)
diff --git a/tests/cloud_tests/testcases/bugs/__init__.py b/tests/cloud_tests/testcases/bugs/__init__.py
new file mode 100644
index 0000000..6c078a5
--- /dev/null
+++ b/tests/cloud_tests/testcases/bugs/__init__.py
@@ -0,0 +1,4 @@
+"""
+Test verifiers for cloud-init bugs
+See configs/bugs/README.md for more information
+"""
diff --git a/tests/cloud_tests/testcases/bugs/lp1511485.py b/tests/cloud_tests/testcases/bugs/lp1511485.py
new file mode 100644
index 0000000..962215b
--- /dev/null
+++ b/tests/cloud_tests/testcases/bugs/lp1511485.py
@@ -0,0 +1,11 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestLP1511485(base.CloudTestCase):
+    """Test LP# 1511485"""
+
+    def test_final_message(self):
+        """Test final message exists"""
+        out = self.get_data_file('cloud-init-output.log')
+        self.assertIn('Final message from cloud-config', out)
diff --git a/tests/cloud_tests/testcases/bugs/lp1628337.py b/tests/cloud_tests/testcases/bugs/lp1628337.py
new file mode 100644
index 0000000..f9ae7ae
--- /dev/null
+++ b/tests/cloud_tests/testcases/bugs/lp1628337.py
@@ -0,0 +1,19 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestLP1628337(base.CloudTestCase):
+    """Test LP# 1511485"""
+
+    def test_fetch_indices(self):
+        """Verify no apt errors"""
+        out = self.get_data_file('cloud-init-output.log')
+        self.assertNotIn('W: Failed to fetch', out)
+        self.assertNotIn('W: Some index files failed to download. '
+                         'They have been ignored, or old ones used instead.',
+                         out)
+
+    def test_ntp(self):
+        """Verify can find ntp and install it"""
+        out = self.get_data_file('cloud-init-output.log')
+        self.assertNotIn('E: Unable to locate package ntp', out)
diff --git a/tests/cloud_tests/testcases/examples/__init__.py b/tests/cloud_tests/testcases/examples/__init__.py
new file mode 100644
index 0000000..8d047ee
--- /dev/null
+++ b/tests/cloud_tests/testcases/examples/__init__.py
@@ -0,0 +1,4 @@
+"""
+Test verifiers for cloud-init examples
+See configs/examples/README.md for more information
+"""
diff --git a/tests/cloud_tests/testcases/examples/add_apt_repositories.py b/tests/cloud_tests/testcases/examples/add_apt_repositories.py
new file mode 100644
index 0000000..f45eaa7
--- /dev/null
+++ b/tests/cloud_tests/testcases/examples/add_apt_repositories.py
@@ -0,0 +1,16 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptconfigurePrimary(base.CloudTestCase):
+    """Example cloud-config test"""
+
+    def test_ubuntu_sources(self):
+        """Test no default Ubuntu entries exist"""
+        out = self.get_data_file('ubuntu.sources.list')
+        self.assertEqual(0, int(out))
+
+    def test_gatech_sources(self):
+        """Test GaTech entires exist"""
+        out = self.get_data_file('gatech.sources.list')
+        self.assertEqual(20, int(out))
diff --git a/tests/cloud_tests/testcases/examples/alter_completion_message.py b/tests/cloud_tests/testcases/examples/alter_completion_message.py
new file mode 100644
index 0000000..21e3536
--- /dev/null
+++ b/tests/cloud_tests/testcases/examples/alter_completion_message.py
@@ -0,0 +1,45 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestFinalMessage(base.CloudTestCase):
+    """
+    test cloud init module `cc_final_message`
+    """
+    subs_char = '$'
+
+    def get_final_message_config(self):
+        """
+        get config for final message
+        """
+        self.assertIn('final_message', self.cloud_config)
+        return self.cloud_config['final_message']
+
+    def get_final_message(self):
+        """
+        get final message from log
+        """
+        out = self.get_data_file('cloud-init-output.log')
+        lines = len(self.get_final_message_config().splitlines())
+        return '\n'.join(out.splitlines()[-1 * lines:])
+
+    def test_final_message_string(self):
+        """
+        ensure final handles regular strings
+        """
+        for actual, config in zip(
+                self.get_final_message().splitlines(),
+                self.get_final_message_config().splitlines()):
+            if self.subs_char not in config:
+                self.assertEqual(actual, config)
+
+    def test_final_message_subs(self):
+        """
+        test variable substitution in final message
+        """
+        # TODO: add verification of other substitutions
+        patterns = {'$datasource': self.get_datasource()}
+        for key, expected in patterns.items():
+            index = self.get_final_message_config().splitlines().index(key)
+            actual = self.get_final_message().splitlines()[index]
+            self.assertEqual(actual, expected)
diff --git a/tests/cloud_tests/testcases/examples/configure_instance_trusted_ca_certificates.py b/tests/cloud_tests/testcases/examples/configure_instance_trusted_ca_certificates.py
new file mode 100644
index 0000000..c50277c
--- /dev/null
+++ b/tests/cloud_tests/testcases/examples/configure_instance_trusted_ca_certificates.py
@@ -0,0 +1,23 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestTrustedCA(base.CloudTestCase):
+    """Example cloud-config test"""
+
+    def test_cert_count_ca(self):
+        """Test correct count of CAs in .crt"""
+        out = self.get_data_file('cert_count_ca')
+        self.assertIn('7 /etc/ssl/certs/ca-certificates.crt', out)
+
+    def test_cert_count_cloudinit(self):
+        """Test correct count of CAs in .pem"""
+        out = self.get_data_file('cert_count_cloudinit')
+        self.assertIn('7 /etc/ssl/certs/cloud-init-ca-certs.pem', out)
+
+    def test_cloudinit_certs(self):
+        """Test text of cert"""
+        out = self.get_data_file('cloudinit_certs')
+        self.assertIn('-----BEGIN CERTIFICATE-----', out)
+        self.assertIn('YOUR-ORGS-TRUSTED-CA-CERT-HERE', out)
+        self.assertIn('-----END CERTIFICATE-----', out)
diff --git a/tests/cloud_tests/testcases/examples/configure_instances_ssh_keys.py b/tests/cloud_tests/testcases/examples/configure_instances_ssh_keys.py
new file mode 100644
index 0000000..f5afbe7
--- /dev/null
+++ b/tests/cloud_tests/testcases/examples/configure_instances_ssh_keys.py
@@ -0,0 +1,27 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestSSHKeys(base.CloudTestCase):
+    """Example cloud-config test"""
+
+    def test_cert_count(self):
+        """Test cert count"""
+        out = self.get_data_file('cert_count')
+        self.assertEqual(20, int(out))
+
+    def test_dsa_public(self):
+        """Test DSA key has ending"""
+        out = self.get_data_file('dsa_public')
+        self.assertIn('ZN4XnifuO5krqAybngIy66PMEoQ= smoser@localhost', out)
+
+    def test_rsa_public(self):
+        """Test RSA key has specific ending"""
+        out = self.get_data_file('rsa_public')
+        self.assertIn('PemAWthxHO18QJvWPocKJtlsDNi3 smoser@localhost', out)
+
+    def test_auth_keys(self):
+        """Test authorized keys has specific ending"""
+        out = self.get_data_file('auth_keys')
+        self.assertIn('QPOt5Q8zWd9qG7PBl9+eiH5qV7NZ mykey@host', out)
+        self.assertIn('Hj29SCmXp5Kt5/82cD/VN3NtHw== smoser@brickies', out)
diff --git a/tests/cloud_tests/testcases/examples/including_user_groups.py b/tests/cloud_tests/testcases/examples/including_user_groups.py
new file mode 100644
index 0000000..704f699
--- /dev/null
+++ b/tests/cloud_tests/testcases/examples/including_user_groups.py
@@ -0,0 +1,39 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestUserGroups(base.CloudTestCase):
+    """Example cloud-config test"""
+
+    def test_group_ubuntu(self):
+        """Test ubuntu group exists"""
+        out = self.get_data_file('group_ubuntu')
+        self.assertRegex(out, r'ubuntu:x:[0-9]{4}:')
+
+    def test_group_cloud_users(self):
+        """Test cloud users group exists"""
+        out = self.get_data_file('group_cloud_users')
+        self.assertRegex(out, r'cloud-users:x:[0-9]{4}:barfoo')
+
+    def test_user_ubuntu(self):
+        """Test ubuntu user exists"""
+        out = self.get_data_file('user_ubuntu')
+        self.assertRegex(
+            out, r'ubuntu:x:[0-9]{4}:[0-9]{4}:Ubuntu:/home/ubuntu:/bin/bash')
+
+    def test_user_foobar(self):
+        """Test foobar user exists"""
+        out = self.get_data_file('user_foobar')
+        self.assertRegex(
+            out, r'foobar:x:[0-9]{4}:[0-9]{4}:Foo B. Bar:/home/foobar:')
+
+    def test_user_barfoo(self):
+        """Test barfoo user exists"""
+        out = self.get_data_file('user_barfoo')
+        self.assertRegex(
+            out, r'barfoo:x:[0-9]{4}:[0-9]{4}:Bar B. Foo:/home/barfoo:')
+
+    def test_user_cloudy(self):
+        """Test cloudy user exists"""
+        out = self.get_data_file('user_cloudy')
+        self.assertRegex(out, r'cloudy:x:[0-9]{3,4}:')
diff --git a/tests/cloud_tests/testcases/examples/install_arbitrary_packages.py b/tests/cloud_tests/testcases/examples/install_arbitrary_packages.py
new file mode 100644
index 0000000..08599be
--- /dev/null
+++ b/tests/cloud_tests/testcases/examples/install_arbitrary_packages.py
@@ -0,0 +1,16 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestInstall(base.CloudTestCase):
+    """Example cloud-config test"""
+
+    def test_htop(self):
+        """Verify htop installed"""
+        out = self.get_data_file('htop')
+        self.assertEqual(1, int(out))
+
+    def test_tree(self):
+        """Verify tree installed"""
+        out = self.get_data_file('treeutils')
+        self.assertEqual(1, int(out))
diff --git a/tests/cloud_tests/testcases/examples/run_apt_upgrade.py b/tests/cloud_tests/testcases/examples/run_apt_upgrade.py
new file mode 100644
index 0000000..6c07e32
--- /dev/null
+++ b/tests/cloud_tests/testcases/examples/run_apt_upgrade.py
@@ -0,0 +1,15 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestUpgrade(base.CloudTestCase):
+    """Example cloud-config test"""
+
+    def test_upgrade(self):
+        """Test upgrade exists in apt history"""
+        out = self.get_data_file('cloud-init.log')
+        self.assertIn(
+            '[CLOUDINIT] util.py[DEBUG]: apt-upgrade '
+            '[eatmydata apt-get --option=Dpkg::Options::=--force-confold '
+            '--option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet '
+            'dist-upgrade] took', out)
diff --git a/tests/cloud_tests/testcases/examples/run_commands.py b/tests/cloud_tests/testcases/examples/run_commands.py
new file mode 100644
index 0000000..b4d311f
--- /dev/null
+++ b/tests/cloud_tests/testcases/examples/run_commands.py
@@ -0,0 +1,11 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestRunCmd(base.CloudTestCase):
+    """Example cloud-config test"""
+
+    def test_run_cmd(self):
+        """Test run command worked"""
+        out = self.get_data_file('run_cmd')
+        self.assertIn('cloud-init run cmd test', out)
diff --git a/tests/cloud_tests/testcases/examples/run_commands_first_boot.py b/tests/cloud_tests/testcases/examples/run_commands_first_boot.py
new file mode 100644
index 0000000..a7577ae
--- /dev/null
+++ b/tests/cloud_tests/testcases/examples/run_commands_first_boot.py
@@ -0,0 +1,11 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestBootCmd(base.CloudTestCase):
+    """Example cloud-config test"""
+
+    def test_bootcmd_host(self):
+        """Test boot command worked"""
+        out = self.get_data_file('hosts')
+        self.assertIn('192.168.1.130 us.archive.ubuntu.com', out)
diff --git a/tests/cloud_tests/testcases/examples/writing_out_arbitrary_files.py b/tests/cloud_tests/testcases/examples/writing_out_arbitrary_files.py
new file mode 100644
index 0000000..97b6d2c
--- /dev/null
+++ b/tests/cloud_tests/testcases/examples/writing_out_arbitrary_files.py
@@ -0,0 +1,26 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestWriteFiles(base.CloudTestCase):
+    """Example cloud-config test"""
+
+    def test_b64(self):
+        """Test b64 encoded file reads as ascii"""
+        out = self.get_data_file('file_b64')
+        self.assertIn('ASCII text', out)
+
+    def test_binary(self):
+        """Test binary file reads as executable"""
+        out = self.get_data_file('file_binary')
+        self.assertIn('ELF 64-bit LSB executable, x86-64, version 1', out)
+
+    def test_gzip(self):
+        """Test gzip file shows up as a shell script"""
+        out = self.get_data_file('file_gzip')
+        self.assertIn('POSIX shell script, ASCII text executable', out)
+
+    def test_text(self):
+        """Test text shows up as ASCII text"""
+        out = self.get_data_file('file_text')
+        self.assertIn('ASCII text', out)
diff --git a/tests/cloud_tests/testcases/main/__init__.py b/tests/cloud_tests/testcases/main/__init__.py
new file mode 100644
index 0000000..460550a
--- /dev/null
+++ b/tests/cloud_tests/testcases/main/__init__.py
@@ -0,0 +1,4 @@
+"""
+Test verifiers for cloud-init main features
+See configs/main/README.md for more information
+"""
diff --git a/tests/cloud_tests/testcases/main/command_output_simple.py b/tests/cloud_tests/testcases/main/command_output_simple.py
new file mode 100644
index 0000000..281f221
--- /dev/null
+++ b/tests/cloud_tests/testcases/main/command_output_simple.py
@@ -0,0 +1,17 @@
+from tests.cloud_tests.testcases import base
+
+
+class TestCommandOutputSimple(base.CloudTestCase):
+    """
+    test functionality of simple output redirection
+    """
+
+    def test_output_file(self):
+        """
+        ensure that the output file is not empty and has all stages
+        """
+        data = self.get_data_file('cloud-init-test-output')
+        self.assertNotEqual(len(data), 0, "specified log empty")
+        self.assertEqual(self.get_config_entry('final_message'),
+                         data.splitlines()[-1].strip())
+        # TODO: need to test that all stages redirected here
diff --git a/tests/cloud_tests/testcases/modules/__init__.py b/tests/cloud_tests/testcases/modules/__init__.py
new file mode 100644
index 0000000..d01fa9e
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/__init__.py
@@ -0,0 +1,4 @@
+"""
+Test verifiers for cloud-init cc modules
+See configs/modules/README.md for more information
+"""
diff --git a/tests/cloud_tests/testcases/modules/apt_configure_conf.py b/tests/cloud_tests/testcases/modules/apt_configure_conf.py
new file mode 100644
index 0000000..06022f1
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/apt_configure_conf.py
@@ -0,0 +1,16 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptconfigureConf(base.CloudTestCase):
+    """Test apt-configure module"""
+
+    def test_apt_conf_assumeyes(self):
+        """Test config assumes true"""
+        out = self.get_data_file('94cloud-init-config')
+        self.assertIn('Assume-Yes "true";', out)
+
+    def test_apt_conf_fixbroken(self):
+        """Test config fixes broken"""
+        out = self.get_data_file('94cloud-init-config')
+        self.assertIn('Fix-Broken "true";', out)
diff --git a/tests/cloud_tests/testcases/modules/apt_configure_disable_suites.py b/tests/cloud_tests/testcases/modules/apt_configure_disable_suites.py
new file mode 100644
index 0000000..e5fe979
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/apt_configure_disable_suites.py
@@ -0,0 +1,11 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptconfigureDisableSuites(base.CloudTestCase):
+    """Test apt-configure module"""
+
+    def test_empty_sourcelist(self):
+        """Test source list is empty"""
+        out = self.get_data_file('sources.list')
+        self.assertEqual('', out)
diff --git a/tests/cloud_tests/testcases/modules/apt_configure_primary.py b/tests/cloud_tests/testcases/modules/apt_configure_primary.py
new file mode 100644
index 0000000..4ec730b
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/apt_configure_primary.py
@@ -0,0 +1,16 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptconfigurePrimary(base.CloudTestCase):
+    """Test apt-configure module"""
+
+    def test_ubuntu_sources(self):
+        """Test no default Ubuntu entries exist"""
+        out = self.get_data_file('ubuntu.sources.list')
+        self.assertEqual(0, int(out))
+
+    def test_gatech_sources(self):
+        """Test GaTech entires exist"""
+        out = self.get_data_file('gatech.sources.list')
+        self.assertEqual(20, int(out))
diff --git a/tests/cloud_tests/testcases/modules/apt_configure_proxy.py b/tests/cloud_tests/testcases/modules/apt_configure_proxy.py
new file mode 100644
index 0000000..3079b9f
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/apt_configure_proxy.py
@@ -0,0 +1,18 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptconfigureProxy(base.CloudTestCase):
+    """Test apt-configure module"""
+
+    def test_proxy_config(self):
+        """Test proxy options added to apt config"""
+        out = self.get_data_file('90cloud-init-aptproxy')
+        self.assertIn(
+            'Acquire::http::Proxy "http://squid.internal:3128";', out)
+        self.assertIn(
+            'Acquire::http::Proxy "http://squid.internal:3128";', out)
+        self.assertIn(
+            'Acquire::ftp::Proxy "ftp://squid.internal:3128";', out)
+        self.assertIn(
+            'Acquire::https::Proxy "https://squid.internal:3128";', out)
diff --git a/tests/cloud_tests/testcases/modules/apt_configure_security.py b/tests/cloud_tests/testcases/modules/apt_configure_security.py
new file mode 100644
index 0000000..a74d07f
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/apt_configure_security.py
@@ -0,0 +1,11 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptconfigureSecurity(base.CloudTestCase):
+    """Test apt-configure module"""
+
+    def test_security_mirror(self):
+        """Test security lines added and uncommented in source.list"""
+        out = self.get_data_file('sources.list')
+        self.assertEqual(6, int(out))
diff --git a/tests/cloud_tests/testcases/modules/apt_configure_sources_key.py b/tests/cloud_tests/testcases/modules/apt_configure_sources_key.py
new file mode 100644
index 0000000..2517e31
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/apt_configure_sources_key.py
@@ -0,0 +1,19 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptconfigureSourcesKey(base.CloudTestCase):
+    """Test apt-configure module"""
+
+    def test_apt_key_list(self):
+        """Test key list updated"""
+        out = self.get_data_file('apt_key_list')
+        self.assertIn(
+            '1FF0 D853 5EF7 E719 E5C8  1B9C 083D 06FB E4D3 04DF', out)
+        self.assertIn('Launchpad PPA for cloud init development team', out)
+
+    def test_source_list(self):
+        """Test source.list updated"""
+        out = self.get_data_file('sources.list')
+        self.assertIn(
+            'http://ppa.launchpad.net/cloud-init-dev/test-archive/ubuntu', out)
diff --git a/tests/cloud_tests/testcases/modules/apt_configure_sources_keyserver.py b/tests/cloud_tests/testcases/modules/apt_configure_sources_keyserver.py
new file mode 100644
index 0000000..68b9308
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/apt_configure_sources_keyserver.py
@@ -0,0 +1,19 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptconfigureSourcesKeyserver(base.CloudTestCase):
+    """Test apt-configure module"""
+
+    def test_apt_key_list(self):
+        """Test specific key added"""
+        out = self.get_data_file('apt_key_list')
+        self.assertIn(
+            '1BC3 0F71 5A3B 8612 47A8  1A5E 55FE 7C8C 0165 013E', out)
+        self.assertIn('Launchpad PPA for curtin developers', out)
+
+    def test_source_list(self):
+        """Test source.list updated"""
+        out = self.get_data_file('sources.list')
+        self.assertIn(
+            'http://ppa.launchpad.net/cloud-init-dev/test-archive/ubuntu', out)
diff --git a/tests/cloud_tests/testcases/modules/apt_configure_sources_list.py b/tests/cloud_tests/testcases/modules/apt_configure_sources_list.py
new file mode 100644
index 0000000..aef4645
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/apt_configure_sources_list.py
@@ -0,0 +1,22 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptconfigureSourcesList(base.CloudTestCase):
+    """Test apt-configure module"""
+
+    def test_sources_list(self):
+        """Test sources.list includes sources"""
+        out = self.get_data_file('sources.list')
+        self.assertRegex(out, r'deb http:\/\/archive.ubuntu.com\/ubuntu '
+                         '[a-z].* main restricted')
+        self.assertRegex(out, r'deb-src http:\/\/archive.ubuntu.com\/ubuntu '
+                         '[a-z].* main restricted')
+        self.assertRegex(out, r'deb http:\/\/archive.ubuntu.com\/ubuntu '
+                         '[a-z].* universe restricted')
+        self.assertRegex(out, r'deb-src http:\/\/archive.ubuntu.com\/ubuntu '
+                         '[a-z].* universe restricted')
+        self.assertRegex(out, r'deb http:\/\/security.ubuntu.com\/ubuntu '
+                         '[a-z].*security multiverse')
+        self.assertRegex(out, r'deb-src http:\/\/security.ubuntu.com\/ubuntu '
+                         '[a-z].*security multiverse')
diff --git a/tests/cloud_tests/testcases/modules/apt_configure_sources_ppa.py b/tests/cloud_tests/testcases/modules/apt_configure_sources_ppa.py
new file mode 100644
index 0000000..a44a565
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/apt_configure_sources_ppa.py
@@ -0,0 +1,19 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptconfigureSourcesPPA(base.CloudTestCase):
+    """Test apt-configure module"""
+
+    def test_ppa(self):
+        """test specific ppa added"""
+        out = self.get_data_file('sources.list')
+        self.assertIn(
+            'http://ppa.launchpad.net/curtin-dev/test-archive/ubuntu', out)
+
+    def test_ppa_key(self):
+        """test ppa key added"""
+        out = self.get_data_file('apt-key')
+        self.assertIn(
+            '1BC3 0F71 5A3B 8612 47A8  1A5E 55FE 7C8C 0165 013E', out)
+        self.assertIn('Launchpad PPA for curtin developers', out)
diff --git a/tests/cloud_tests/testcases/modules/apt_pipelining_disable.py b/tests/cloud_tests/testcases/modules/apt_pipelining_disable.py
new file mode 100644
index 0000000..255c398
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/apt_pipelining_disable.py
@@ -0,0 +1,11 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptPipeliningDisable(base.CloudTestCase):
+    """Test apt-pipelining module"""
+
+    def test_disable_pipelining(self):
+        """Test pipelining disabled"""
+        out = self.get_data_file('90cloud-init-pipelining')
+        self.assertIn('Acquire::http::Pipeline-Depth "0";', out)
diff --git a/tests/cloud_tests/testcases/modules/apt_pipelining_os.py b/tests/cloud_tests/testcases/modules/apt_pipelining_os.py
new file mode 100644
index 0000000..301f96c
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/apt_pipelining_os.py
@@ -0,0 +1,11 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestAptPipeliningOS(base.CloudTestCase):
+    """Test apt-pipelining module"""
+
+    def test_os_pipelining(self):
+        """Test pipelining set to os"""
+        out = self.get_data_file('90cloud-init-pipelining')
+        self.assertIn('Acquire::http::Pipeline-Depth "0";', out)
diff --git a/tests/cloud_tests/testcases/modules/bootcmd.py b/tests/cloud_tests/testcases/modules/bootcmd.py
new file mode 100644
index 0000000..ca6eaaa
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/bootcmd.py
@@ -0,0 +1,11 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestBootCmd(base.CloudTestCase):
+    """Test bootcmd module"""
+
+    def test_bootcmd_host(self):
+        """Test boot cmd worked"""
+        out = self.get_data_file('hosts')
+        self.assertIn('192.168.1.130 us.archive.ubuntu.com', out)
diff --git a/tests/cloud_tests/testcases/modules/byobu.py b/tests/cloud_tests/testcases/modules/byobu.py
new file mode 100644
index 0000000..59169a3
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/byobu.py
@@ -0,0 +1,21 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestByobu(base.CloudTestCase):
+    """Test Byobu module"""
+
+    def test_byobu_installed(self):
+        """Test byobu installed"""
+        out = self.get_data_file('byobu_installed')
+        self.assertIn('/usr/bin/byobu', out)
+
+    def test_byobu_profile_enabled(self):
+        """Test byobu profile.d file exists"""
+        out = self.get_data_file('byobu_profile_enabled')
+        self.assertIn('/etc/profile.d/Z97-byobu.sh', out)
+
+    def test_byobu_launch_exists(self):
+        """Test byobu-launch exists"""
+        out = self.get_data_file('byobu_launch_exists')
+        self.assertIn('/usr/bin/byobu-launch', out)
diff --git a/tests/cloud_tests/testcases/modules/ca_certs.py b/tests/cloud_tests/testcases/modules/ca_certs.py
new file mode 100644
index 0000000..f734240
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/ca_certs.py
@@ -0,0 +1,16 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestCaCerts(base.CloudTestCase):
+    """Test ca certs module"""
+
+    def test_cert_count(self):
+        """Test the count is proper"""
+        out = self.get_data_file('cert_count')
+        self.assertEqual(5, int(out))
+
+    def test_cert_installed(self):
+        """Test line from our cert exists"""
+        out = self.get_data_file('cert')
+        self.assertIn('a36c744454555024e7f82edc420fd2c8', out)
diff --git a/tests/cloud_tests/testcases/modules/debug_disable.py b/tests/cloud_tests/testcases/modules/debug_disable.py
new file mode 100644
index 0000000..ec3a46e
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/debug_disable.py
@@ -0,0 +1,12 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestDebugDisable(base.CloudTestCase):
+    """Disable debug messages"""
+
+    def test_debug_disable(self):
+        """Test verbose output missing from logs"""
+        out = self.get_data_file('cloud-init.log')
+        self.assertNotIn(
+            out, r'Skipping module named [a-z].* verbose printing disabled')
diff --git a/tests/cloud_tests/testcases/modules/debug_enable.py b/tests/cloud_tests/testcases/modules/debug_enable.py
new file mode 100644
index 0000000..b33804d
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/debug_enable.py
@@ -0,0 +1,11 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestDebugEnable(base.CloudTestCase):
+    """Test debug messages"""
+
+    def test_debug_enable(self):
+        """Test debug messages in cloud-init log"""
+        out = self.get_data_file('cloud-init.log')
+        self.assertIn('[DEBUG]', out)
diff --git a/tests/cloud_tests/testcases/modules/final_message.py b/tests/cloud_tests/testcases/modules/final_message.py
new file mode 100644
index 0000000..21e3536
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/final_message.py
@@ -0,0 +1,45 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestFinalMessage(base.CloudTestCase):
+    """
+    test cloud init module `cc_final_message`
+    """
+    subs_char = '$'
+
+    def get_final_message_config(self):
+        """
+        get config for final message
+        """
+        self.assertIn('final_message', self.cloud_config)
+        return self.cloud_config['final_message']
+
+    def get_final_message(self):
+        """
+        get final message from log
+        """
+        out = self.get_data_file('cloud-init-output.log')
+        lines = len(self.get_final_message_config().splitlines())
+        return '\n'.join(out.splitlines()[-1 * lines:])
+
+    def test_final_message_string(self):
+        """
+        ensure final handles regular strings
+        """
+        for actual, config in zip(
+                self.get_final_message().splitlines(),
+                self.get_final_message_config().splitlines()):
+            if self.subs_char not in config:
+                self.assertEqual(actual, config)
+
+    def test_final_message_subs(self):
+        """
+        test variable substitution in final message
+        """
+        # TODO: add verification of other substitutions
+        patterns = {'$datasource': self.get_datasource()}
+        for key, expected in patterns.items():
+            index = self.get_final_message_config().splitlines().index(key)
+            actual = self.get_final_message().splitlines()[index]
+            self.assertEqual(actual, expected)
diff --git a/tests/cloud_tests/testcases/modules/keys_to_console.py b/tests/cloud_tests/testcases/modules/keys_to_console.py
new file mode 100644
index 0000000..bee8810
--- /dev/null
+++ b/tests/cloud_tests/testcases/modules/keys_to_console.py
@@ -0,0 +1,18 @@
+"""cloud-init Integration Test Verify Script"""
+from tests.cloud_tests.testcases import base
+
+
+class TestKeysToConsole(base.CloudTestCase):