Merge ~chad.smith/cloud-init:feature/schema-resizefs-bootcmd into cloud-init:master

[Chad Smith]

schema and docs: Add jsonschema to resizefs and bootcmd modules

Add schema definitions to both cc_resizefs and cc_bootcmd modules. Extend
schema.py to parse and document enumerated json types. Schema definitions
are used to generate module documention and log warnings for schema
infractions.

This branch also does the following:
  - drops vestigial 'resize_rootfs_tmp' option from cc_resizefs. That
    option only created the specified directory and didn't make use of
that directory for any resize operations.
  - Drop yaml.dumps calls from schema documentation generation to avoid
    yaml import costs on module load
  - Add __doc__ = get_schema_doc(schema) definitions it each module to
    supplement python help() calls for cc_runcmd, cc_bootcmd, cc_ntp and
cc_resizefs
  - Add a SCHEMA_EXAMPLES_SPACER_TEMPLATE string to docs for modules which
    contain more than one example

to test:
$ tox -e doc; xdg-open doc/rtd_html/topics/modules.html
$ python3 -m cloudinit.cmd.main devel schema --doc
$ cat > test.cfg <<EOF
#cloud-config
bootcmd: junk
resize_rootfs: invalid
EOF
$ python3 -m cloudinit.cmd.main devel schema -c test.cfg --annotate

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:07195877d396a5a36583b3bd277a91561a606f29
https://jenkins.ubuntu.com/server/job/cloud-init-ci/257/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    FAILED: MAAS Compatability Testing

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/257/rebuild

review: Needs Fixing (continuous-integration)

5562e81... by Chad Smith on 2017-09-05

add unit test for documentation of jsonschema enum type

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:5562e81e085038086f605489a59e7708f9bf3a43
https://jenkins.ubuntu.com/server/job/cloud-init-ci/258/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    FAILED: MAAS Compatability Testing

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/258/rebuild

review: Needs Fixing (continuous-integration)

7b8bdd7... by Chad Smith on 2017-09-05

add a simple resizefs example

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:7b8bdd7a0c3b7d7c2806d0ae57af1bc73e785298
https://jenkins.ubuntu.com/server/job/cloud-init-ci/259/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/259/rebuild

review: Needs Fixing (continuous-integration)

f86c892... by Chad Smith on 2017-09-06

flakes

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:f86c8927fd409eb72ddfddfc290f749c73cfd63d
https://jenkins.ubuntu.com/server/job/cloud-init-ci/260/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    FAILED: Ubuntu LTS: Integration

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/260/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:f86c8927fd409eb72ddfddfc290f749c73cfd63d
https://jenkins.ubuntu.com/server/job/cloud-init-ci/262/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    FAILED: MAAS Compatability Testing

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/262/rebuild

review: Needs Fixing (continuous-integration)

00cf0f2... by Chad Smith on 2017-09-06

tempfile perms are different in different environments (CentOS sets mode 644 ubuntu 664. drop this aspect of the unit test as we aren't explicitly setting the mode in the module either way

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:00cf0f29c981f6387ba9c115abd6d95bdf052758
https://jenkins.ubuntu.com/server/job/cloud-init-ci/263/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/263/rebuild

review: Approve (continuous-integration)

1a7aca9... by Chad Smith on 2017-09-08

pull in smoser's patch for non-yaml.dumps in documentation creation. Update unit tests, fix cc_runcmd, cc_bootcmd, cc_resizefs and cc_ntp modules to set __doc__ = get_schema_doc() for python's builtin help

[Chad Smith (chad.smith)]

Good suggestions Scott, I pulled in your changes and adapted cc_runcmd, cc_ntp, cc_resizefs and cc_bootcmd to all set __doc__ through get_schema_doc().

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:1a7aca930109d21a1e6b26f7456ceafe6e713447
https://jenkins.ubuntu.com/server/job/cloud-init-ci/275/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/275/rebuild

review: Approve (continuous-integration)

diff --git a/cloudinit/config/cc_bootcmd.py b/cloudinit/config/cc_bootcmd.py
index 9c0476a..d05dd8f 100644
--- a/cloudinit/config/cc_bootcmd.py
+++ b/cloudinit/config/cc_bootcmd.py
@@ -3,45 +3,73 @@
 #
 # Author: Scott Moser <[e-mail]>
 # Author: Juerg Haefliger <[e-mail]>
+# Author: Chad Smith <[e-mail]>
 #
 # This file is part of cloud-init. See LICENSE file for license information.
 
-"""
-Bootcmd
--------
-**Summary:** run commands early in boot process
-
-This module runs arbitrary commands very early in the boot process,
-only slightly after a boothook would run. This is very similar to a
-boothook, but more user friendly. The environment variable ``INSTANCE_ID``
-will be set to the current instance id for all run commands. Commands can be
-specified either as lists or strings. For invocation details, see ``runcmd``.
-
-.. note::
-    bootcmd should only be used for things that could not be done later in the
-    boot process.
-
-**Internal name:** ``cc_bootcmd``
-
-**Module frequency:** per always
-
-**Supported distros:** all
-
-**Config keys**::
-
-    bootcmd:
-        - echo 192.168.1.130 us.archive.ubuntu.com > /etc/hosts
-        - [ cloud-init-per, once, mymkfs, mkfs, /dev/vdb ]
-"""
+"""Bootcmd: run arbitrary commands early in the boot process."""
 
 import os
+from textwrap import dedent
 
+from cloudinit.config.schema import (
+    get_schema_doc, validate_cloudconfig_schema)
 from cloudinit.settings import PER_ALWAYS
 from cloudinit import temp_utils
 from cloudinit import util
 
 frequency = PER_ALWAYS
 
+# The schema definition for each cloud-config module is a strict contract for
+# describing supported configuration parameters for each cloud-config section.
+# It allows cloud-config to validate and alert users to invalid or ignored
+# configuration options before actually attempting to deploy with said
+# configuration.
+
+distros = ['all']
+
+schema = {
+    'id': 'cc_bootcmd',
+    'name': 'Bootcmd',
+    'title': 'Run arbitrary commands early in the boot process',
+    'description': dedent("""\
+        This module runs arbitrary commands very early in the boot process,
+        only slightly after a boothook would run. This is very similar to a
+        boothook, but more user friendly. The environment variable
+        ``INSTANCE_ID`` will be set to the current instance id for all run
+        commands. Commands can be specified either as lists or strings. For
+        invocation details, see ``runcmd``.
+
+        .. note::
+            bootcmd should only be used for things that could not be done later
+            in the boot process."""),
+    'distros': distros,
+    'examples': [dedent("""\
+        bootcmd:
+            - echo 192.168.1.130 us.archive.ubuntu.com > /etc/hosts
+            - [ cloud-init-per, once, mymkfs, mkfs, /dev/vdb ]
+    """)],
+    'frequency': PER_ALWAYS,
+    'type': 'object',
+    'properties': {
+        'bootcmd': {
+            'type': 'array',
+            'items': {
+                'oneOf': [
+                    {'type': 'array', 'items': {'type': 'string'}},
+                    {'type': 'string'}]
+            },
+            'additionalItems': False,  # Reject items of non-string non-list
+            'additionalProperties': False,
+            'minItems': 1,
+            'required': [],
+            'uniqueItems': True
+        }
+    }
+}
+
+__doc__ = get_schema_doc(schema)  # Supplement python help()
+
 
 def handle(name, cfg, cloud, log, _args):
 
@@ -50,13 +78,18 @@ def handle(name, cfg, cloud, log, _args):
                    " no 'bootcmd' key in configuration"), name)
         return
 
+<<<<<<< cloudinit/config/cc_bootcmd.py
     with temp_utils.ExtendedTemporaryFile(suffix=".sh") as tmpf:
+=======
+    validate_cloudconfig_schema(cfg, schema)
+    with util.ExtendedTemporaryFile(suffix=".sh") as tmpf:
+>>>>>>> cloudinit/config/cc_bootcmd.py
         try:
             content = util.shellify(cfg["bootcmd"])
             tmpf.write(util.encode_text(content))
             tmpf.flush()
-        except Exception:
-            util.logexc(log, "Failed to shellify bootcmd")
+        except Exception as e:
+            util.logexc(log, "Failed to shellify bootcmd: %s", str(e))
             raise
 
         try:
diff --git a/cloudinit/config/cc_ntp.py b/cloudinit/config/cc_ntp.py
index a02b4bf..15ae1ec 100644
--- a/cloudinit/config/cc_ntp.py
+++ b/cloudinit/config/cc_ntp.py
@@ -4,39 +4,10 @@
 #
 # This file is part of cloud-init. See LICENSE file for license information.
 
-"""
-NTP
----
-**Summary:** enable and configure ntp
-
-Handle ntp configuration. If ntp is not installed on the system and ntp
-configuration is specified, ntp will be installed. If there is a default ntp
-config file in the image or one is present in the distro's ntp package, it will
-be copied to ``/etc/ntp.conf.dist`` before any changes are made. A list of ntp
-pools and ntp servers can be provided under the ``ntp`` config key. If no ntp
-servers or pools are provided, 4 pools will be used in the format
-``{0-3}.{distro}.pool.ntp.org``.
-
-**Internal name:** ``cc_ntp``
-
-**Module frequency:** per instance
-
-**Supported distros:** centos, debian, fedora, opensuse, ubuntu
-
-**Config keys**::
-
-    ntp:
-        pools:
-            - 0.company.pool.ntp.org
-            - 1.company.pool.ntp.org
-            - ntp.myorg.org
-        servers:
-            - my.ntp.server.local
-            - ntp.ubuntu.com
-            - 192.168.23.2
-"""
+"""NTP: enable and configure ntp"""
 
-from cloudinit.config.schema import validate_cloudconfig_schema
+from cloudinit.config.schema import (
+    get_schema_doc, validate_cloudconfig_schema)
 from cloudinit import log as logging
 from cloudinit.settings import PER_INSTANCE
 from cloudinit import templater
@@ -76,10 +47,13 @@ schema = {
         ``{0-3}.{distro}.pool.ntp.org``."""),
     'distros': distros,
     'examples': [
-        {'ntp': {'pools': ['0.company.pool.ntp.org', '1.company.pool.ntp.org',
-                           'ntp.myorg.org'],
-                 'servers': ['my.ntp.server.local', 'ntp.ubuntu.com',
-                             '192.168.23.2']}}],
+        dedent("""\
+        ntp:
+          pools: [0.int.pool.ntp.org, 1.int.pool.ntp.org, ntp.myorg.org]
+          servers:
+            - ntp.server.local
+            - ntp.ubuntu.com
+            - 192.168.23.2""")],
     'frequency': PER_INSTANCE,
     'type': 'object',
     'properties': {
@@ -117,6 +91,8 @@ schema = {
     }
 }
 
+__doc__ = get_schema_doc(schema)  # Supplement python help()
+
 
 def handle(name, cfg, cloud, log, _args):
     """Enable and configure ntp."""
diff --git a/cloudinit/config/cc_resizefs.py b/cloudinit/config/cc_resizefs.py
index ceee952..f14d383 100644
--- a/cloudinit/config/cc_resizefs.py
+++ b/cloudinit/config/cc_resizefs.py
@@ -6,31 +6,8 @@
 #
 # This file is part of cloud-init. See LICENSE file for license information.
 
-"""
-Resizefs
---------
-**Summary:** resize filesystem
+"""Resizefs: cloud-config module which resizes the filesystem"""
 
-Resize a filesystem to use all avaliable space on partition. This module is
-useful along with ``cc_growpart`` and will ensure that if the root partition
-has been resized the root filesystem will be resized along with it. By default,
-``cc_resizefs`` will resize the root partition and will block the boot process
-while the resize command is running. Optionally, the resize operation can be
-performed in the background while cloud-init continues running modules. This
-can be enabled by setting ``resize_rootfs`` to ``true``. This module can be
-disabled altogether by setting ``resize_rootfs`` to ``false``.
-
-**Internal name:** ``cc_resizefs``
-
-**Module frequency:** per always
-
-**Supported distros:** all
-
-**Config keys**::
-
-    resize_rootfs: <true/false/"noblock">
-    resize_rootfs_tmp: <directory>
-"""
 
 import errno
 import getopt
@@ -38,11 +15,47 @@ import os
 import re
 import shlex
 import stat
+from textwrap import dedent
 
+from cloudinit.config.schema import (
+    get_schema_doc, validate_cloudconfig_schema)
 from cloudinit.settings import PER_ALWAYS
 from cloudinit import util
 
+NOBLOCK = "noblock"
+
 frequency = PER_ALWAYS
+distros = ['all']
+
+schema = {
+    'id': 'cc_resizefs',
+    'name': 'Resizefs',
+    'title': 'Resize filesystem',
+    'description': dedent("""\
+        Resize a filesystem to use all avaliable space on partition. This
+        module is useful along with ``cc_growpart`` and will ensure that if the
+        root partition has been resized the root filesystem will be resized
+        along with it. By default, ``cc_resizefs`` will resize the root
+        partition and will block the boot process while the resize command is
+        running. Optionally, the resize operation can be performed in the
+        background while cloud-init continues running modules. This can be
+        enabled by setting ``resize_rootfs`` to ``true``. This module can be
+        disabled altogether by setting ``resize_rootfs`` to ``false``."""),
+    'distros': distros,
+    'examples': [
+        'resize_rootfs: false  # disable root filesystem resize operation'],
+    'frequency': PER_ALWAYS,
+    'type': 'object',
+    'properties': {
+        'resize_rootfs': {
+            'enum': [True, False, NOBLOCK],
+            'description': dedent("""\
+                Whether to resize the root partition. Default: 'true'""")
+        }
+    }
+}
+
+__doc__ = get_schema_doc(schema)  # Supplement python help()
 
 
 def _resize_btrfs(mount_point, devpth):
@@ -131,8 +144,6 @@ RESIZE_FS_PRECHECK_CMDS = {
     'ufs': _can_skip_resize_ufs
 }
 
-NOBLOCK = "noblock"
-
 
 def rootdev_from_cmdline(cmdline):
     found = None
@@ -161,71 +172,81 @@ def can_skip_resize(fs_type, resize_what, devpth):
     return False
 
 
-def handle(name, cfg, _cloud, log, args):
-    if len(args) != 0:
-        resize_root = args[0]
-    else:
-        resize_root = util.get_cfg_option_str(cfg, "resize_rootfs", True)
-
-    if not util.translate_bool(resize_root, addons=[NOBLOCK]):
-        log.debug("Skipping module named %s, resizing disabled", name)
-        return
-
-    # TODO(harlowja) is the directory ok to be used??
-    resize_root_d = util.get_cfg_option_str(cfg, "resize_rootfs_tmp", "/run")
-    util.ensure_dir(resize_root_d)
+def is_device_path_writable_block(devpath, info, log):
+    """Return True if devpath is a writable block device.
 
-    # TODO(harlowja): allow what is to be resized to be configurable??
-    resize_what = "/"
-    result = util.get_mount_info(resize_what, log)
-    if not result:
-        log.warn("Could not determine filesystem type of %s", resize_what)
-        return
-
-    (devpth, fs_type, mount_point) = result
-
-    info = "dev=%s mnt_point=%s path=%s" % (devpth, mount_point, resize_what)
-    log.debug("resize_info: %s" % info)
+    @param devpath: Path to the root device we want to resize.
+    @param info: String representing information about the requested device.
+    @param log: Logger to which logs will be added upon error.
 
+    @returns Boolean True if block device is writable
+    """
     container = util.is_container()
 
     # Ensure the path is a block device.
-    if (devpth == "/dev/root" and not os.path.exists(devpth) and
+    if (devpath == "/dev/root" and not os.path.exists(devpath) and
             not container):
-        devpth = util.rootdev_from_cmdline(util.get_cmdline())
-        if devpth is None:
+        devpath = util.rootdev_from_cmdline(util.get_cmdline())
+        if devpath is None:
             log.warn("Unable to find device '/dev/root'")
-            return
-        log.debug("Converted /dev/root to '%s' per kernel cmdline", devpth)
+            return False
+        log.debug("Converted /dev/root to '%s' per kernel cmdline", devpath)
 
     try:
-        statret = os.stat(devpth)
+        statret = os.stat(devpath)
     except OSError as exc:
         if container and exc.errno == errno.ENOENT:
             log.debug("Device '%s' did not exist in container. "
-                      "cannot resize: %s", devpth, info)
+                      "cannot resize: %s", devpath, info)
         elif exc.errno == errno.ENOENT:
             log.warn("Device '%s' did not exist. cannot resize: %s",
-                     devpth, info)
+                     devpath, info)
         else:
             raise exc
-        return
+        return False
 
-    if not os.access(devpth, os.W_OK):
+    if not os.access(devpath, os.W_OK):
         if container:
             log.debug("'%s' not writable in container. cannot resize: %s",
-                      devpth, info)
+                      devpath, info)
         else:
-            log.warn("'%s' not writable. cannot resize: %s", devpth, info)
+            log.warn("'%s' not writable. cannot resize: %s", devpath, info)
         return
 
     if not stat.S_ISBLK(statret.st_mode) and not stat.S_ISCHR(statret.st_mode):
         if container:
             log.debug("device '%s' not a block device in container."
-                      " cannot resize: %s" % (devpth, info))
+                      " cannot resize: %s" % (devpath, info))
         else:
             log.warn("device '%s' not a block device. cannot resize: %s" %
-                     (devpth, info))
+                     (devpath, info))
+        return False
+    return True
+
+
+def handle(name, cfg, _cloud, log, args):
+    if len(args) != 0:
+        resize_root = args[0]
+    else:
+        resize_root = util.get_cfg_option_str(cfg, "resize_rootfs", True)
+    validate_cloudconfig_schema(cfg, schema)
+    if not util.translate_bool(resize_root, addons=[NOBLOCK]):
+        log.debug("Skipping module named %s, resizing disabled", name)
+        return
+
+    # TODO(harlowja): allow what is to be resized to be configurable??
+    resize_what = "/"
+    result = util.get_mount_info(resize_what, log)
+    if not result:
+        log.warn("Could not determine filesystem type of %s", resize_what)
+        return
+
+    (devpth, fs_type, mount_point) = result
+
+    info = "dev=%s mnt_point=%s path=%s" % (devpth, mount_point, resize_what)
+    log.debug("resize_info: %s" % info)
+
+    if not is_device_path_writable_block(devpth, info, log):
         return
 
     resizer = None
diff --git a/cloudinit/config/cc_runcmd.py b/cloudinit/config/cc_runcmd.py
index 7c3ccd4..7f99569 100644
--- a/cloudinit/config/cc_runcmd.py
+++ b/cloudinit/config/cc_runcmd.py
@@ -8,7 +8,8 @@
 
 """Runcmd: run arbitrary commands at rc.local with output to the console"""
 
-from cloudinit.config.schema import validate_cloudconfig_schema
+from cloudinit.config.schema import (
+    get_schema_doc, validate_cloudconfig_schema)
 from cloudinit.settings import PER_INSTANCE
 from cloudinit import util
 
@@ -67,6 +68,8 @@ schema = {
     }
 }
 
+__doc__ = get_schema_doc(schema)  # Supplement python help()
+
 
 def handle(name, cfg, cloud, log, _args):
     if "runcmd" not in cfg:
diff --git a/cloudinit/config/schema.py b/cloudinit/config/schema.py
index 73dd5c2..c17d973 100644
--- a/cloudinit/config/schema.py
+++ b/cloudinit/config/schema.py
@@ -14,6 +14,7 @@ import re
 import sys
 import yaml
 
+_YAML_MAP = {True: 'true', False: 'false', None: 'null'}
 SCHEMA_UNDEFINED = b'UNDEFINED'
 CLOUD_CONFIG_HEADER = b'#cloud-config'
 SCHEMA_DOC_TMPL = """
@@ -34,6 +35,8 @@ SCHEMA_DOC_TMPL = """
 {examples}
 """
 SCHEMA_PROPERTY_TMPL = '{prefix}**{prop_name}:** ({type}) {description}'
+SCHEMA_EXAMPLES_HEADER = '\n**Examples**::\n\n'
+SCHEMA_EXAMPLES_SPACER_TEMPLATE = '\n    # --- Example{0} ---'
 
 
 class SchemaValidationError(ValueError):
@@ -212,6 +215,9 @@ def _schemapath_for_cloudconfig(config, original_content):
 def _get_property_type(property_dict):
     """Return a string representing a property type from a given jsonschema."""
     property_type = property_dict.get('type', SCHEMA_UNDEFINED)
+    if property_type == SCHEMA_UNDEFINED and property_dict.get('enum'):
+        property_type = [
+            str(_YAML_MAP.get(k, k)) for k in property_dict['enum']]
     if isinstance(property_type, list):
         property_type = '/'.join(property_type)
     items = property_dict.get('items', {})
@@ -249,15 +255,14 @@ def _get_schema_examples(schema, prefix=''):
     examples = schema.get('examples')
     if not examples:
         return ''
-    rst_content = '\n**Examples**::\n\n'
-    for example in examples:
-        if isinstance(example, str):
-            example_content = example
-        else:
-            example_content = yaml.dump(example, default_flow_style=False)
+    rst_content = SCHEMA_EXAMPLES_HEADER
+    for count, example in enumerate(examples):
         # Python2.6 is missing textwrapper.indent
-        lines = example_content.split('\n')
+        lines = example.split('\n')
         indented_lines = ['    {0}'.format(line) for line in lines]
+        if rst_content != SCHEMA_EXAMPLES_HEADER:
+            indented_lines.insert(
+                0, SCHEMA_EXAMPLES_SPACER_TEMPLATE.format(count + 1))
         rst_content += '\n'.join(indented_lines)
     return rst_content
 
diff --git a/tests/unittests/test_handler/test_handler_bootcmd.py b/tests/unittests/test_handler/test_handler_bootcmd.py
new file mode 100644
index 0000000..11b8699
--- /dev/null
+++ b/tests/unittests/test_handler/test_handler_bootcmd.py
@@ -0,0 +1,157 @@
+# This file is part of cloud-init. See LICENSE file for license information.
+
+from cloudinit.config import cc_bootcmd
+from cloudinit.sources import DataSourceNone
+from cloudinit import (distros, helpers, cloud, util)
+from ..helpers import CiTestCase, mock, skipIf
+
+import logging
+
+try:
+    import jsonschema
+    assert jsonschema  # avoid pyflakes error F401: import unused
+    _missing_jsonschema_dep = False
+except ImportError:
+    _missing_jsonschema_dep = True
+
+LOG = logging.getLogger(__name__)
+
+
+class TestBootcmd(CiTestCase):
+
+    with_logs = True
+
+    def setUp(self):
+        super(TestBootcmd, self).setUp()
+        self.subp = util.subp
+        self.new_root = self.tmp_dir()
+
+    def _get_cloud(self, distro):
+        paths = helpers.Paths({})
+        cls = distros.fetch(distro)
+        mydist = cls(distro, {}, paths)
+        myds = DataSourceNone.DataSourceNone({}, mydist, paths)
+        paths.datasource = myds
+        return cloud.Cloud(myds, paths, {}, mydist, None)
+
+    def test_handler_skip_if_no_bootcmd(self):
+        """When the provided config doesn't contain bootcmd, skip it."""
+        cfg = {}
+        mycloud = self._get_cloud('ubuntu')
+        cc_bootcmd.handle('notimportant', cfg, mycloud, LOG, None)
+        self.assertIn(
+            "Skipping module named notimportant, no 'bootcmd' key",
+            self.logs.getvalue())
+
+    def test_handler_invalid_command_set(self):
+        """Commands which can't be converted to shell will raise errors."""
+        invalid_config = {'bootcmd': 1}
+        cc = self._get_cloud('ubuntu')
+        with self.assertRaises(TypeError) as context_manager:
+            cc_bootcmd.handle('cc_bootcmd', invalid_config, cc, LOG, [])
+        self.assertIn('Failed to shellify bootcmd', self.logs.getvalue())
+        self.assertEqual(
+            "'int' object is not iterable",
+            str(context_manager.exception))
+
+    @skipIf(_missing_jsonschema_dep, "No python-jsonschema dependency")
+    def test_handler_schema_validation_warns_non_array_type(self):
+        """Schema validation warns of non-array type for bootcmd key.
+
+        Schema validation is not strict, so bootcmd attempts to shellify the
+        invalid content.
+        """
+        invalid_config = {'bootcmd': 1}
+        cc = self._get_cloud('ubuntu')
+        with self.assertRaises(TypeError):
+            cc_bootcmd.handle('cc_bootcmd', invalid_config, cc, LOG, [])
+        self.assertIn(
+            'Invalid config:\nbootcmd: 1 is not of type \'array\'',
+            self.logs.getvalue())
+        self.assertIn('Failed to shellify', self.logs.getvalue())
+
+    @skipIf(_missing_jsonschema_dep, 'No python-jsonschema dependency')
+    def test_handler_schema_validation_warns_non_array_item_type(self):
+        """Schema validation warns of non-array or string bootcmd items.
+
+        Schema validation is not strict, so bootcmd attempts to shellify the
+        invalid content.
+        """
+        invalid_config = {
+            'bootcmd': ['ls /', 20, ['wget', 'http://stuff/blah'], {'a': 'n'}]}
+        cc = self._get_cloud('ubuntu')
+        with self.assertRaises(RuntimeError) as context_manager:
+            cc_bootcmd.handle('cc_bootcmd', invalid_config, cc, LOG, [])
+        expected_warnings = [
+            'bootcmd.1: 20 is not valid under any of the given schemas',
+            'bootcmd.3: {\'a\': \'n\'} is not valid under any of the given'
+            ' schema'
+        ]
+        logs = self.logs.getvalue()
+        for warning in expected_warnings:
+            self.assertIn(warning, logs)
+        self.assertIn('Failed to shellify', logs)
+        self.assertEqual(
+            'Unable to shellify type int which is not a list or string',
+            str(context_manager.exception))
+
+    def test_handler_creates_and_runs_bootcmd_script_with_instance_id(self):
+        """Valid schema runs a bootcmd script with INSTANCE_ID in the env."""
+        cc = self._get_cloud('ubuntu')
+        bootcmd_file = self.tmp_path('bootcmd.sh', self.new_root)
+        out_file = self.tmp_path('bootcmd.out', self.new_root)
+        valid_config = {'bootcmd': [
+            'echo $INSTANCE_ID > {0}'.format(out_file)]}
+
+        class FakeExtendedTempFile(object):
+            def __init__(_self, suffix):
+                _self.suffix = suffix
+                path = self.tmp_path('bootcmd' + _self.suffix, self.new_root)
+                _self.handle = open(path, 'w+b')
+
+            def __enter__(_self):
+                return _self.handle
+
+            def __exit__(_self, exc_type, exc_value, traceback):
+                _self.handle.close()
+
+        mock_path = 'cloudinit.config.cc_bootcmd.util.ExtendedTemporaryFile'
+        with mock.patch(mock_path, FakeExtendedTempFile):
+            cc_bootcmd.handle('cc_bootcmd', valid_config, cc, LOG, [])
+        self.assertEqual(
+            '#!/bin/sh\necho $INSTANCE_ID > {0}\n'.format(out_file),
+            util.load_file(bootcmd_file))
+        self.assertEqual('iid-datasource-none\n', util.load_file(out_file))
+
+    def test_handler_runs_bootcmd_script_with_error(self):
+        """When a valid script generates an error, that error is raised."""
+        cc = self._get_cloud('ubuntu')
+        bootcmd_file = self.tmp_path('bootcmd.sh', self.new_root)
+        valid_config = {'bootcmd': ['exit 1']}  # Script with error
+
+        class FakeExtendedTempFile(object):
+            def __init__(_self, suffix):
+                _self.suffix = suffix
+                path = self.tmp_path('bootcmd' + _self.suffix, self.new_root)
+                _self.handle = open(path, 'w+b')
+
+            def __enter__(_self):
+                return _self.handle
+
+            def __exit__(_self, exc_type, exc_value, traceback):
+                _self.handle.close()
+
+        mock_path = 'cloudinit.config.cc_bootcmd.util.ExtendedTemporaryFile'
+        with mock.patch(mock_path, FakeExtendedTempFile):
+            with self.assertRaises(util.ProcessExecutionError) as ctxt_manager:
+                cc_bootcmd.handle('does-not-matter', valid_config, cc, LOG, [])
+        self.assertIn(
+            'Unexpected error while running command.\n'
+            "Command: ['/bin/sh', '{0}']".format(bootcmd_file),
+            str(ctxt_manager.exception))
+        self.assertIn(
+            'Failed to run bootcmd module does-not-matter',
+            self.logs.getvalue())
+
+
+# vi: ts=4 expandtab
diff --git a/tests/unittests/test_handler/test_handler_resizefs.py b/tests/unittests/test_handler/test_handler_resizefs.py
index 52591b8..7ce7a7d 100644
--- a/tests/unittests/test_handler/test_handler_resizefs.py
+++ b/tests/unittests/test_handler/test_handler_resizefs.py
@@ -1,17 +1,29 @@
 # This file is part of cloud-init. See LICENSE file for license information.
 
-from cloudinit.config import cc_resizefs
+from cloudinit.config.cc_resizefs import (
+    can_skip_resize, handle, is_device_path_writable_block,
+    rootdev_from_cmdline)
 
+import logging
 import textwrap
-import unittest
+
+from ..helpers import CiTestCase, mock, skipIf, util, wrap_and_call
+
+
+LOG = logging.getLogger(__name__)
+
 
 try:
-    from unittest import mock
+    import jsonschema
+    assert jsonschema  # avoid pyflakes error F401: import unused
+    _missing_jsonschema_dep = False
 except ImportError:
-    import mock
+    _missing_jsonschema_dep = True
+
 
+class TestResizefs(CiTestCase):
+    with_logs = True
 
-class TestResizefs(unittest.TestCase):
     def setUp(self):
         super(TestResizefs, self).setUp()
         self.name = "resizefs"
@@ -34,7 +46,7 @@ class TestResizefs(unittest.TestCase):
               58720296   3145728    3  freebsd-swap  (1.5G)
               61866024   1048496       - free -  (512M)
             """)
-        res = cc_resizefs.can_skip_resize(fs_type, resize_what, devpth)
+        res = can_skip_resize(fs_type, resize_what, devpth)
         self.assertTrue(res)
 
     @mock.patch('cloudinit.config.cc_resizefs._get_dumpfs_output')
@@ -52,8 +64,241 @@ class TestResizefs(unittest.TestCase):
             =>      34  297086  da0  GPT  (145M)
                     34  297086    1  freebsd-ufs  (145M)
             """)
-        res = cc_resizefs.can_skip_resize(fs_type, resize_what, devpth)
+        res = can_skip_resize(fs_type, resize_what, devpth)
         self.assertTrue(res)
 
+    def test_handle_noops_on_disabled(self):
+        """The handle function logs when the configuration disables resize."""
+        cfg = {'resize_rootfs': False}
+        handle('cc_resizefs', cfg, _cloud=None, log=LOG, args=[])
+        self.assertIn(
+            'DEBUG: Skipping module named cc_resizefs, resizing disabled\n',
+            self.logs.getvalue())
+
+    @skipIf(_missing_jsonschema_dep, "No python-jsonschema dependency")
+    def test_handle_schema_validation_logs_invalid_resize_rootfs_value(self):
+        """The handle reports json schema violations as a warning.
+
+        Invalid values for resize_rootfs result in disabling the module.
+        """
+        cfg = {'resize_rootfs': 'junk'}
+        handle('cc_resizefs', cfg, _cloud=None, log=LOG, args=[])
+        logs = self.logs.getvalue()
+        self.assertIn(
+            "WARNING: Invalid config:\nresize_rootfs: 'junk' is not one of"
+            " [True, False, 'noblock']",
+            logs)
+        self.assertIn(
+            'DEBUG: Skipping module named cc_resizefs, resizing disabled\n',
+            logs)
+
+    @mock.patch('cloudinit.config.cc_resizefs.util.get_mount_info')
+    def test_handle_warns_on_unknown_mount_info(self, m_get_mount_info):
+        """handle warns when get_mount_info sees unknown filesystem for /."""
+        m_get_mount_info.return_value = None
+        cfg = {'resize_rootfs': True}
+        handle('cc_resizefs', cfg, _cloud=None, log=LOG, args=[])
+        logs = self.logs.getvalue()
+        self.assertNotIn("WARNING: Invalid config:\nresize_rootfs:", logs)
+        self.assertIn(
+            'WARNING: Could not determine filesystem type of /\n',
+            logs)
+        self.assertEqual(
+            [mock.call('/', LOG)],
+            m_get_mount_info.call_args_list)
+
+    def test_handle_warns_on_undiscoverable_root_path_in_commandline(self):
+        """handle noops when the root path is not found on the commandline."""
+        cfg = {'resize_rootfs': True}
+        exists_mock_path = 'cloudinit.config.cc_resizefs.os.path.exists'
+
+        def fake_mount_info(path, log):
+            self.assertEqual('/', path)
+            self.assertEqual(LOG, log)
+            return ('/dev/root', 'ext4', '/')
+
+        with mock.patch(exists_mock_path) as m_exists:
+            m_exists.return_value = False
+            wrap_and_call(
+                'cloudinit.config.cc_resizefs.util',
+                {'is_container': {'return_value': False},
+                 'get_mount_info': {'side_effect': fake_mount_info},
+                 'get_cmdline': {'return_value': 'BOOT_IMAGE=/vmlinuz.efi'}},
+                handle, 'cc_resizefs', cfg, _cloud=None, log=LOG,
+                args=[])
+        logs = self.logs.getvalue()
+        self.assertIn("WARNING: Unable to find device '/dev/root'", logs)
+
+
+class TestRootDevFromCmdline(CiTestCase):
+
+    def test_rootdev_from_cmdline_with_no_root(self):
+        """Return None from rootdev_from_cmdline when root is not present."""
+        invalid_cases = [
+            'BOOT_IMAGE=/adsf asdfa werasef  root adf', 'BOOT_IMAGE=/adsf', '']
+        for case in invalid_cases:
+            self.assertIsNone(rootdev_from_cmdline(case))
+
+    def test_rootdev_from_cmdline_with_root_startswith_dev(self):
+        """Return the cmdline root when the path starts with /dev."""
+        self.assertEqual(
+            '/dev/this',
+            rootdev_from_cmdline('asdf root=/dev/this'))
+
+    def test_rootdev_from_cmdline_with_root_without_dev_prefix(self):
+        """Add /dev prefix to cmdline root when the path lacks the prefix."""
+        self.assertEqual(
+            '/dev/this',
+            rootdev_from_cmdline('asdf root=this'))
+
+    def test_rootdev_from_cmdline_with_root_with_label(self):
+        """When cmdline root contains a LABEL, our root is disk/by-label."""
+        self.assertEqual(
+            '/dev/disk/by-label/unique',
+            rootdev_from_cmdline('asdf root=LABEL=unique'))
+
+    def test_rootdev_from_cmdline_with_root_with_uuid(self):
+        """When cmdline root contains a UUID, our root is disk/by-uuid."""
+        self.assertEqual(
+            '/dev/disk/by-uuid/adsfdsaf-adsf',
+            rootdev_from_cmdline('asdf root=UUID=adsfdsaf-adsf'))
+
+
+class TestIsDevicePathWritableBlock(CiTestCase):
+
+    with_logs = True
+
+    def test_is_device_path_writable_block_warns_missing_cmdline_root(self):
+        """When root does not exist isn't in the cmdline, log warning."""
+        info = 'does not matter'
+
+        def fake_mount_info(path, log):
+            self.assertEqual('/', path)
+            self.assertEqual(LOG, log)
+            return ('/dev/root', 'ext4', '/')
+
+        exists_mock_path = 'cloudinit.config.cc_resizefs.os.path.exists'
+        with mock.patch(exists_mock_path) as m_exists:
+            m_exists.return_value = False
+            is_valid = wrap_and_call(
+                'cloudinit.config.cc_resizefs.util',
+                {'is_container': {'return_value': False},
+                 'get_mount_info': {'side_effect': fake_mount_info},
+                 'get_cmdline': {'return_value': 'BOOT_IMAGE=/vmlinuz.efi'}},
+                is_device_path_writable_block, '/dev/root', info, LOG)
+        self.assertFalse(is_valid)
+        logs = self.logs.getvalue()
+        self.assertIn("WARNING: Unable to find device '/dev/root'", logs)
+
+    def test_is_device_path_writable_block_does_not_exist(self):
+        """When devpath does not exist, a warning is logged."""
+        info = 'dev=/I/dont/exist mnt_point=/ path=/dev/none'
+        is_valid = wrap_and_call(
+            'cloudinit.config.cc_resizefs.util',
+            {'is_container': {'return_value': False}},
+            is_device_path_writable_block, '/I/dont/exist', info, LOG)
+        self.assertFalse(is_valid)
+        self.assertIn(
+            "WARNING: Device '/I/dont/exist' did not exist."
+            ' cannot resize: %s' % info,
+            self.logs.getvalue())
+
+    def test_is_device_path_writable_block_does_not_exist_in_container(self):
+        """When devpath does not exist in a container, log a debug message."""
+        info = 'dev=/I/dont/exist mnt_point=/ path=/dev/none'
+        is_valid = wrap_and_call(
+            'cloudinit.config.cc_resizefs.util',
+            {'is_container': {'return_value': True}},
+            is_device_path_writable_block, '/I/dont/exist', info, LOG)
+        self.assertFalse(is_valid)
+        self.assertIn(
+            "DEBUG: Device '/I/dont/exist' did not exist in container."
+            ' cannot resize: %s' % info,
+            self.logs.getvalue())
+
+    def test_is_device_path_writable_block_raises_oserror(self):
+        """When unexpected OSError is raises by os.stat it is reraised."""
+        info = 'dev=/I/dont/exist mnt_point=/ path=/dev/none'
+        with self.assertRaises(OSError) as context_manager:
+            wrap_and_call(
+                'cloudinit.config.cc_resizefs',
+                {'util.is_container': {'return_value': True},
+                 'os.stat': {'side_effect': OSError('Something unexpected')}},
+                is_device_path_writable_block, '/I/dont/exist', info, LOG)
+        self.assertEqual(
+            'Something unexpected', str(context_manager.exception))
+
+    def test_is_device_path_writable_block_readonly(self):
+        """When root device is readonly, emit a warning and return False."""
+        fake_devpath = self.tmp_path('dev/readonly')
+        util.write_file(fake_devpath, '', mode=0o400)  # read-only
+        info = 'dev=/dev/root mnt_point=/ path={0}'.format(fake_devpath)
+
+        exists_mock_path = 'cloudinit.config.cc_resizefs.os.path.exists'
+        with mock.patch(exists_mock_path) as m_exists:
+            m_exists.return_value = False
+            is_valid = wrap_and_call(
+                'cloudinit.config.cc_resizefs.util',
+                {'is_container': {'return_value': False},
+                 'rootdev_from_cmdline': {'return_value': fake_devpath}},
+                is_device_path_writable_block, '/dev/root', info, LOG)
+        self.assertFalse(is_valid)
+        logs = self.logs.getvalue()
+        self.assertIn(
+            "Converted /dev/root to '{0}' per kernel cmdline".format(
+                fake_devpath),
+            logs)
+        self.assertIn(
+            "WARNING: '{0}' not writable. cannot resize".format(fake_devpath),
+            logs)
+
+    def test_is_device_path_writable_block_readonly_in_container(self):
+        """When root device is readonly, emit debug log and return False."""
+        fake_devpath = self.tmp_path('dev/readonly')
+        util.write_file(fake_devpath, '', mode=0o400)  # read-only
+        info = 'dev=/dev/root mnt_point=/ path={0}'.format(fake_devpath)
+
+        is_valid = wrap_and_call(
+            'cloudinit.config.cc_resizefs.util',
+            {'is_container': {'return_value': True}},
+            is_device_path_writable_block, fake_devpath, info, LOG)
+        self.assertFalse(is_valid)
+        self.assertIn(
+            "DEBUG: '{0}' not writable in container. cannot resize".format(
+                fake_devpath),
+            self.logs.getvalue())
+
+    def test_is_device_path_writable_block_non_block(self):
+        """When device is not a block device, emit warning return False."""
+        fake_devpath = self.tmp_path('dev/readwrite')
+        util.write_file(fake_devpath, '', mode=0o600)  # read-write
+        info = 'dev=/dev/root mnt_point=/ path={0}'.format(fake_devpath)
+
+        is_valid = wrap_and_call(
+            'cloudinit.config.cc_resizefs.util',
+            {'is_container': {'return_value': False}},
+            is_device_path_writable_block, fake_devpath, info, LOG)
+        self.assertFalse(is_valid)
+        self.assertIn(
+            "WARNING: device '{0}' not a block device. cannot resize".format(
+                fake_devpath),
+            self.logs.getvalue())
+
+    def test_is_device_path_writable_block_non_block_on_container(self):
+        """When device is non-block device in container, emit debug log."""
+        fake_devpath = self.tmp_path('dev/readwrite')
+        util.write_file(fake_devpath, '', mode=0o600)  # read-write
+        info = 'dev=/dev/root mnt_point=/ path={0}'.format(fake_devpath)
+
+        is_valid = wrap_and_call(
+            'cloudinit.config.cc_resizefs.util',
+            {'is_container': {'return_value': True}},
+            is_device_path_writable_block, fake_devpath, info, LOG)
+        self.assertFalse(is_valid)
+        self.assertIn(
+            "DEBUG: device '{0}' not a block device in container."
+            ' cannot resize'.format(fake_devpath),
+            self.logs.getvalue())
+
 
 # vi: ts=4 expandtab
diff --git a/tests/unittests/test_handler/test_schema.py b/tests/unittests/test_handler/test_schema.py
index 6137e3c..745bb0f 100644
--- a/tests/unittests/test_handler/test_schema.py
+++ b/tests/unittests/test_handler/test_schema.py
@@ -27,7 +27,7 @@ class GetSchemaTest(CiTestCase):
         """Every cloudconfig module with schema is listed in allOf keyword."""
         schema = get_schema()
         self.assertItemsEqual(
-            ['cc_ntp', 'cc_runcmd'],
+            ['cc_bootcmd', 'cc_ntp', 'cc_resizefs', 'cc_runcmd'],
             [subschema['id'] for subschema in schema['allOf']])
         self.assertEqual('cloud-config-schema', schema['id'])
         self.assertEqual(
@@ -205,6 +205,17 @@ class GetSchemaDocTest(CiTestCase):
             '**prop1:** (string/integer) prop-description',
             get_schema_doc(full_schema))
 
+    def test_get_schema_doc_handles_enum_types(self):
+        """get_schema_doc converts enum types to yaml and delimits with '/'."""
+        full_schema = copy(self.required_schema)
+        full_schema.update(
+            {'properties': {
+                'prop1': {'enum': [True, False, 'stuff'],
+                          'description': 'prop-description'}}})
+        self.assertIn(
+            '**prop1:** (true/false/stuff) prop-description',
+            get_schema_doc(full_schema))
+
     def test_get_schema_doc_handles_nested_oneof_property_types(self):
         """get_schema_doc describes array items oneOf declarations in type."""
         full_schema = copy(self.required_schema)
@@ -219,29 +230,11 @@ class GetSchemaDocTest(CiTestCase):
             '**prop1:** (array of (string)/(integer)) prop-description',
             get_schema_doc(full_schema))
 
-    def test_get_schema_doc_returns_restructured_text_with_examples(self):
-        """get_schema_doc returns indented examples when present in schema."""
-        full_schema = copy(self.required_schema)
-        full_schema.update(
-            {'examples': [{'ex1': [1, 2, 3]}],
-             'properties': {
-                'prop1': {'type': 'array', 'description': 'prop-description',
-                          'items': {'type': 'integer'}}}})
-        self.assertIn(
-            dedent("""
-                **Config schema**:
-                    **prop1:** (array of integer) prop-description
-
-                **Examples**::
-
-                    ex1"""),
-            get_schema_doc(full_schema))
-
-    def test_get_schema_doc_handles_unstructured_examples(self):
-        """get_schema_doc properly indented examples which as just strings."""
+    def test_get_schema_doc_handles_string_examples(self):
+        """get_schema_doc properly indented examples as a list of strings."""
         full_schema = copy(self.required_schema)
         full_schema.update(
-            {'examples': ['My example:\n    [don\'t, expand, "this"]'],
+            {'examples': ['ex1:\n    [don\'t, expand, "this"]', 'ex2: true'],
              'properties': {
                 'prop1': {'type': 'array', 'description': 'prop-description',
                           'items': {'type': 'integer'}}}})
@@ -252,8 +245,11 @@ class GetSchemaDocTest(CiTestCase):
 
                 **Examples**::
 
-                    My example:
-                        [don't, expand, "this"]"""),
+                    ex1:
+                        [don't, expand, "this"]
+                    # --- Example2 ---
+                    ex2: true
+            """),
             get_schema_doc(full_schema))
 
     def test_get_schema_doc_raises_key_errors(self):