Merge ~raharper/cloud-init:feature/cloud-init-hotplug-handler into cloud-init:master

[Scott Moser]

wip fixme

[Scott Moser (smoser)]

i'm not all the way through, but here are some comments.

725b992... by Ryan Harper on 2018-10-04

Add unittests, remove hard coded scope, fix tox

abaa5b1... by Ryan Harper on 2018-10-04

Add tests for event module

[Ryan Harper (raharper)]

Thanks for the review. I've pushed some of your changes already. Doing another round now.

a2eb198... by Ryan Harper on 2018-10-04

Address review comments

hotplug:
  - Capture update result, log on error
  - Add more error details when cloud-init is not configured for hotplug
  - Use RuntimeError instead of Exception, add more detail in message
tools/hook-hotplug
  - Drop trailing slash
  - Quote environment variables when creating array

97dc6e6... by Ryan Harper on 2018-10-04

EventType.UDEV -> EventType.HOTPLUG and udev -> hotplug

2d16ce8... by Ryan Harper on 2018-10-04

Drop extra || return 1

5b0d4cf... by Ryan Harper on 2018-10-04

Fix logic thinko if not allowed return

94d37e1... by Ryan Harper on 2018-10-12

docs: add documentation for events and hotplug support

[Si-Wei Liu (siwliu)]

Any plan to merge this into master soon before the next quarterly cloud-init release? I see no progress had been made since November last year, wondering what happened to this enhancement? Oracle is looking forward to adopting and leveraging this feature in its Ubuntu cloud images.

Unmerged commits

94d37e1... by Ryan Harper on 2018-10-12

docs: add documentation for events and hotplug support

5b0d4cf... by Ryan Harper on 2018-10-04

Fix logic thinko if not allowed return

2d16ce8... by Ryan Harper on 2018-10-04

Drop extra || return 1

97dc6e6... by Ryan Harper on 2018-10-04

EventType.UDEV -> EventType.HOTPLUG and udev -> hotplug

a2eb198... by Ryan Harper on 2018-10-04

Address review comments

hotplug:
  - Capture update result, log on error
  - Add more error details when cloud-init is not configured for hotplug
  - Use RuntimeError instead of Exception, add more detail in message
tools/hook-hotplug
  - Drop trailing slash
  - Quote environment variables when creating array

abaa5b1... by Ryan Harper on 2018-10-04

Add tests for event module

725b992... by Ryan Harper on 2018-10-04

Add unittests, remove hard coded scope, fix tox

af4ca37... by Ryan Harper on 2018-10-04

Add debug, copy dictionaries before merging

82287a4... by Ryan Harper on 2018-10-04

Reorder updates config merging with datasource capabilities

System config needs to filter datasource capabilities.

809bb1a... by Ryan Harper on 2018-10-03

Retain apply_network_config, apply names but not config in some cases

diff --git a/bash_completion/cloud-init b/bash_completion/cloud-init
index 8c25032..0eab40c 100644
--- a/bash_completion/cloud-init
+++ b/bash_completion/cloud-init
@@ -28,7 +28,7 @@ _cloudinit_complete()
                     COMPREPLY=($(compgen -W "--help --tarfile --include-userdata" -- $cur_word))
                     ;;
                 devel)
-                    COMPREPLY=($(compgen -W "--help schema net-convert" -- $cur_word))
+                    COMPREPLY=($(compgen -W "--help hotplug-hook net-convert schema" -- $cur_word))
                     ;;
                 dhclient-hook|features)
                     COMPREPLY=($(compgen -W "--help" -- $cur_word))
@@ -61,6 +61,9 @@ _cloudinit_complete()
                 --frequency)
                     COMPREPLY=($(compgen -W "--help instance always once" -- $cur_word))
                     ;;
+                hotplug-hook)
+                    COMPREPLY=($(compgen -W "--help" -- $cur_word))
+                    ;;
                 net-convert)
                     COMPREPLY=($(compgen -W "--help --network-data --kind --directory --output-kind" -- $cur_word))
                     ;;
diff --git a/cloudinit/cmd/devel/hotplug_hook.py b/cloudinit/cmd/devel/hotplug_hook.py
new file mode 100644
index 0000000..ae23d07
--- /dev/null
+++ b/cloudinit/cmd/devel/hotplug_hook.py
@@ -0,0 +1,210 @@
+# This file is part of cloud-init. See LICENSE file for license information.
+
+"""Handle reconfiguration on hotplug events"""
+import abc
+import argparse
+import os
+import six
+import sys
+import time
+
+from cloudinit.event import EventType
+from cloudinit import log
+from cloudinit import reporting
+from cloudinit.reporting import events
+from cloudinit import sources
+from cloudinit.stages import Init
+from cloudinit.net import read_sys_net_safe
+from cloudinit.net.network_state import parse_net_config_data
+
+
+LOG = log.getLogger(__name__)
+NAME = 'hotplug-hook'
+
+
+def get_parser(parser=None):
+    """Build or extend and arg parser for hotplug-hook utility.
+
+    @param parser: Optional existing ArgumentParser instance representing the
+        subcommand which will be extended to support the args of this utility.
+
+    @returns: ArgumentParser with proper argument configuration.
+    """
+    if not parser:
+        parser = argparse.ArgumentParser(prog=NAME, description=__doc__)
+
+    parser.add_argument("-d", "--devpath",
+                        metavar="PATH",
+                        help="sysfs path to hotplugged device")
+    parser.add_argument("--hotplug-debug", action='store_true',
+                        help='enable debug logging to stderr.')
+    parser.add_argument("-s", "--subsystem",
+                        choices=['net', 'block'])
+    parser.add_argument("-u", "--udevaction",
+                        choices=['add', 'change', 'remove'])
+
+    return parser
+
+
+def log_console(msg):
+    """Log messages to stderr console and configured logging."""
+    sys.stderr.write(msg + '\n')
+    sys.stderr.flush()
+    LOG.debug(msg)
+
+
+def devpath_to_macaddr(devpath):
+    macaddr = read_sys_net_safe(os.path.basename(devpath), 'address')
+    return macaddr
+
+
+def in_netconfig(unique_id, netconfig):
+    netstate = parse_net_config_data(netconfig)
+    found = [iface
+             for iface in netstate.iter_interfaces()
+             if iface.get('mac_address') == unique_id]
+    log_console('Ifaces with ID=%s : %s' % (unique_id, found))
+    return len(found) > 0
+
+
[e-mail]_metaclass(abc.ABCMeta)
+class UeventHandler(object):
+    def __init__(self, ds, devpath, success_fn):
+        self.datasource = ds
+        self.devpath = devpath
+        self.success_fn = success_fn
+
+    @abc.abstractmethod
+    def apply(self):
+        raise NotImplementedError()
+
+    @property
+    @abc.abstractmethod
+    def config(self):
+        raise NotImplementedError()
+
+    @abc.abstractmethod
+    def detect(self, action):
+        raise NotImplementedError()
+
+    def success(self):
+        return self.success_fn()
+
+    def update(self):
+        result = self.datasource.update_metadata([EventType.HOTPLUG])
+        if not result:
+            log_console('Datasource %s not updated for '
+                        ' event %s' % (self.datasource, EventType.HOTPLUG))
+        return result
+
+
+class NetHandler(UeventHandler):
+    def __init__(self, ds, devpath, success_fn):
+        super(NetHandler, self).__init__(ds, devpath, success_fn)
+        self.id = devpath_to_macaddr(self.devpath)
+
+    def apply(self):
+        return self.datasource.distro.apply_network_config(self.config,
+                                                           bring_up=True)
+
+    @property
+    def config(self):
+        return self.datasource.network_config
+
+    def detect(self, action):
+        detect_presence = None
+        if action == 'add':
+            detect_presence = True
+        elif action == 'remove':
+            detect_presence = False
+        else:
+            raise ValueError('Cannot detect unknown action: %s' % action)
+
+        return detect_presence == in_netconfig(self.id, self.config)
+
+
+UEVENT_HANDLERS = {
+    'net': NetHandler,
+}
+
+SUBSYSTEM_TO_EVENT = {
+    'net': 'network',
+    'block': 'storage',
+}
+
+
+def handle_args(name, args):
+    log_console('%s called with args=%s' % (NAME, args))
+    hotplug_reporter = events.ReportEventStack(NAME, __doc__,
+                                               reporting_enabled=True)
+    with hotplug_reporter:
+        # only handling net udev events for now
+        event_handler_cls = UEVENT_HANDLERS.get(args.subsystem)
+        if not event_handler_cls:
+            log_console('hotplug-hook: cannot handle events for subsystem: '
+                        '"%s"' % args.subsystem)
+            return 1
+
+        log_console('Reading cloud-init configation')
+        hotplug_init = Init(ds_deps=[], reporter=hotplug_reporter)
+        hotplug_init.read_cfg()
+
+        log_console('Configuring logging')
+        log.setupLogging(hotplug_init.cfg)
+        if 'reporting' in hotplug_init.cfg:
+            reporting.update_configuration(hotplug_init.cfg.get('reporting'))
+
+        log_console('Fetching datasource')
+        try:
+            ds = hotplug_init.fetch(existing="trust")
+        except sources.DataSourceNotFoundException:
+            log_console('No Ds found')
+            return 1
+
+        subevent = SUBSYSTEM_TO_EVENT.get(args.subsystem)
+        if not hotplug_init.update_event_allowed(EventType.HOTPLUG,
+                                                 scope=subevent):
+            log_console('cloud-init not configured to handle hotplug event'
+                        ' of type %s' % subevent)
+            return
+
+        log_console('Creating %s event handler' % args.subsystem)
+        event_handler = event_handler_cls(ds, args.devpath,
+                                          hotplug_init._write_to_cache)
+        retries = [1, 1, 1, 3, 5]
+        for attempt, wait in enumerate(retries):
+            log_console('subsystem=%s update attempt %s/%s' % (args.subsystem,
+                                                               attempt,
+                                                               len(retries)))
+            try:
+                log_console('Refreshing metadata')
+                result = event_handler.update()
+                if not result:
+                    raise RuntimeError(
+                        'Event handler %s update failed' % event_handler_cls)
+
+                if event_handler.detect(action=args.udevaction):
+                    log_console('Detected update, apply config change')
+                    event_handler.apply()
+                    log_console('Updating cache')
+                    event_handler.success()
+                    break
+                else:
+                    raise RuntimeError('Failed to detect %s in updated '
+                                       'metadata' % event_handler.id)
+
+            except Exception as e:
+                if attempt + 1 >= len(retries):
+                    raise
+                log_console('Exception while processing hotplug event. %s' % e)
+                time.sleep(wait)
+
+        log_console('Exiting hotplug handler')
+        reporting.flush_events()
+
+
+if __name__ == '__main__':
+    args = get_parser().parse_args()
+    handle_args(NAME, args)
+
+# vi: ts=4 expandtab
diff --git a/cloudinit/cmd/devel/parser.py b/cloudinit/cmd/devel/parser.py
index 99a234c..3ad09b3 100644
--- a/cloudinit/cmd/devel/parser.py
+++ b/cloudinit/cmd/devel/parser.py
@@ -6,7 +6,7 @@
 
 import argparse
 from cloudinit.config import schema
-
+from . import hotplug_hook
 from . import net_convert
 from . import render
 
@@ -20,6 +20,8 @@ def get_parser(parser=None):
     subparsers.required = True
 
     subcmds = [
+        (hotplug_hook.NAME, hotplug_hook.__doc__,
+         hotplug_hook.get_parser, hotplug_hook.handle_args),
         ('schema', 'Validate cloud-config files for document schema',
          schema.get_parser, schema.handle_schema_args),
         (net_convert.NAME, net_convert.__doc__,
diff --git a/cloudinit/distros/__init__.py b/cloudinit/distros/__init__.py
index ef618c2..92285f5 100644
--- a/cloudinit/distros/__init__.py
+++ b/cloudinit/distros/__init__.py
@@ -69,6 +69,7 @@ class Distro(object):
         self._paths = paths
         self._cfg = cfg
         self.name = name
+        self.net_renderer = None
 
     @abc.abstractmethod
     def install_packages(self, pkglist):
@@ -89,9 +90,8 @@ class Distro(object):
         name, render_cls = renderers.select(priority=priority)
         LOG.debug("Selected renderer '%s' from priority list: %s",
                   name, priority)
-        renderer = render_cls(config=self.renderer_configs.get(name))
-        renderer.render_network_config(network_config)
-        return []
+        self.net_renderer = render_cls(config=self.renderer_configs.get(name))
+        return self.net_renderer.render_network_config(network_config)
 
     def _find_tz_file(self, tz):
         tz_file = os.path.join(self.tz_zone_dir, str(tz))
@@ -176,6 +176,7 @@ class Distro(object):
         # a much less complete network config format (interfaces(5)).
         try:
             dev_names = self._write_network_config(netconfig)
+            LOG.debug('Network config found dev names: %s', dev_names)
         except NotImplementedError:
             # backwards compat until all distros have apply_network_config
             return self._apply_network_from_network_config(
diff --git a/cloudinit/distros/debian.py b/cloudinit/distros/debian.py
index d517fb8..4f1e6a9 100644
--- a/cloudinit/distros/debian.py
+++ b/cloudinit/distros/debian.py
@@ -114,14 +114,23 @@ class Distro(distros.Distro):
         return self._supported_write_network_config(netconfig)
 
     def _bring_up_interfaces(self, device_names):
-        use_all = False
-        for d in device_names:
-            if d == 'all':
-                use_all = True
-        if use_all:
-            return distros.Distro._bring_up_interface(self, '--all')
+        render_name = self.net_renderer.name
+        if render_name == 'eni':
+            LOG.debug('Bringing up interfaces with eni/ifup')
+            use_all = False
+            for d in device_names:
+                if d == 'all':
+                    use_all = True
+            if use_all:
+                return distros.Distro._bring_up_interface(self, '--all')
+            else:
+                return distros.Distro._bring_up_interfaces(self, device_names)
+        elif render_name == 'netplan':
+            LOG.debug('Bringing up interfaces with netplan apply')
+            util.subp(['netplan', 'apply'])
         else:
-            return distros.Distro._bring_up_interfaces(self, device_names)
+            LOG.warning('Cannot bring up interfaces, unknown renderer: "%s"',
+                        render_name)
 
     def _write_hostname(self, your_hostname, out_fn):
         conf = None
diff --git a/cloudinit/event.py b/cloudinit/event.py
index f7b311f..3a18ef9 100644
--- a/cloudinit/event.py
+++ b/cloudinit/event.py
@@ -2,16 +2,74 @@
 
 """Classes and functions related to event handling."""
 
+import copy
+
+from cloudinit import log as logging
+from cloudinit import util
+
+
+LOG = logging.getLogger(__name__)
+
 
 # Event types which can generate maintenance requests for cloud-init.
 class EventType(object):
     BOOT = "System boot"
     BOOT_NEW_INSTANCE = "New instance first boot"
+    HOTPLUG = "Hotplug add event"
 
     # TODO: Cloud-init will grow support for the follow event types:
-    # UDEV
     # METADATA_CHANGE
     # USER_REQUEST
 
 
+EventTypeMap = {
+    'boot': EventType.BOOT,
+    'boot-new-instance': EventType.BOOT_NEW_INSTANCE,
+    'hotplug': EventType.HOTPLUG,
+}
+
+
+# inverted mapping
+EventNameMap = {v: k for k, v in EventTypeMap.items()}
+
+
+def get_allowed_events(sys_events, ds_events):
+    '''Merge datasource capabilties with system config to determine which
+       update events are allowed.'''
+
+    # updates:
+    #   policy-version: 1
+    #   network:
+    #     when: [boot-new-instance, boot, hotplug]
+    #   storage:
+    #     when: [boot-new-instance, hotplug]
+    #     watch: http://169.254.169.254/metadata/storage_config/
+
+    LOG.debug('updates: system   cfg: %s', sys_events)
+    LOG.debug('updates: datasrc caps: %s', ds_events)
+
+    updates = util.mergemanydict([copy.deepcopy(sys_events),
+                                  copy.deepcopy(ds_events)])
+    LOG.debug('updates: merged  cfg: %s', updates)
+
+    events = {}
+    for etype in [scope for scope, val in sys_events.items()
+                  if type(val) == dict and 'when' in val]:
+        events[etype] = (
+            set([EventTypeMap.get(evt)
+                 for evt in updates.get(etype, {}).get('when', [])
+                 if evt in EventTypeMap]))
+
+    LOG.debug('updates: allowed events: %s', events)
+    return events
+
+
+def get_update_events_config(update_events):
+    '''Return a dictionary of updates config'''
+    evt_cfg = {'policy-version': 1}
+    for scope, events in update_events.items():
+        evt_cfg[scope] = {'when': [EventNameMap[evt] for evt in events]}
+
+    return evt_cfg
+
 # vi: ts=4 expandtab
diff --git a/cloudinit/net/eni.py b/cloudinit/net/eni.py
index c6f631a..3d8dcfb 100644
--- a/cloudinit/net/eni.py
+++ b/cloudinit/net/eni.py
@@ -338,6 +338,8 @@ def _ifaces_to_net_config_data(ifaces):
 class Renderer(renderer.Renderer):
     """Renders network information in a /etc/network/interfaces format."""
 
+    name = 'eni'
+
     def __init__(self, config=None):
         if not config:
             config = {}
diff --git a/cloudinit/net/netplan.py b/cloudinit/net/netplan.py
index bc1087f..08c9d05 100644
--- a/cloudinit/net/netplan.py
+++ b/cloudinit/net/netplan.py
@@ -178,6 +178,8 @@ def _clean_default(target=None):
 class Renderer(renderer.Renderer):
     """Renders network information in a /etc/netplan/network.yaml format."""
 
+    name = 'netplan'
+
     NETPLAN_GENERATE = ['netplan', 'generate']
 
     def __init__(self, config=None):
diff --git a/cloudinit/net/renderer.py b/cloudinit/net/renderer.py
index 5f32e90..88a1221 100644
--- a/cloudinit/net/renderer.py
+++ b/cloudinit/net/renderer.py
@@ -44,6 +44,10 @@ class Renderer(object):
                                                  driver=driver))
         return content.getvalue()
 
+    @staticmethod
+    def get_interface_names(network_state):
+        return [cfg.get('name') for cfg in network_state.iter_interfaces()]
+
     @abc.abstractmethod
     def render_network_state(self, network_state, templates=None,
                              target=None):
@@ -51,8 +55,9 @@ class Renderer(object):
 
     def render_network_config(self, network_config, templates=None,
                               target=None):
-        return self.render_network_state(
-            network_state=parse_net_config_data(network_config),
-            templates=templates, target=target)
+        network_state = parse_net_config_data(network_config)
+        self.render_network_state(network_state=network_state,
+                                  templates=templates, target=target)
+        return self.get_interface_names(network_state)
 
 # vi: ts=4 expandtab
diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index 9c16d3a..d502268 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -232,6 +232,8 @@ class NetInterface(ConfigMap):
 class Renderer(renderer.Renderer):
     """Renders network information in a /etc/sysconfig format."""
 
+    name = 'sysconfig'
+
     # See: https://access.redhat.com/documentation/en-US/\
     #      Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/\
     #      s1-networkscripts-interfaces.html (or other docs for
diff --git a/cloudinit/sources/DataSourceOpenStack.py b/cloudinit/sources/DataSourceOpenStack.py
index 4a01524..b777c65 100644
--- a/cloudinit/sources/DataSourceOpenStack.py
+++ b/cloudinit/sources/DataSourceOpenStack.py
@@ -7,7 +7,9 @@
 import time
 
 from cloudinit import log as logging
+from cloudinit.event import EventType
 from cloudinit.net.dhcp import EphemeralDHCPv4, NoDHCPLeaseError
+from cloudinit.net import is_up
 from cloudinit import sources
 from cloudinit import url_helper
 from cloudinit import util
@@ -93,6 +95,15 @@ class DataSourceOpenStack(openstack.SourceMixin, sources.DataSource):
         return sources.instance_id_matches_system_uuid(self.get_instance_id())
 
     @property
+    def update_events(self):
+        events = {'network': set([EventType.BOOT_NEW_INSTANCE,
+                                  EventType.BOOT,
+                                  EventType.HOTPLUG]),
+                  'storage': set([])}
+        LOG.debug('OpenStack update events: %s', events)
+        return events
+
+    @property
     def network_config(self):
         """Return a network config dict for rendering ENI or netplan files."""
         if self._network_config != sources.UNSET:
@@ -122,11 +133,13 @@ class DataSourceOpenStack(openstack.SourceMixin, sources.DataSource):
             False when unable to contact metadata service or when metadata
             format is invalid or disabled.
         """
-        oracle_considered = 'Oracle' in self.sys_cfg.get('datasource_list')
+        oracle_considered = 'Oracle' in self.sys_cfg.get('datasource_list',
+                                                         {})
         if not detect_openstack(accept_oracle=not oracle_considered):
             return False
 
-        if self.perform_dhcp_setup:  # Setup networking in init-local stage.
+        if self.perform_dhcp_setup and not is_up(self.fallback_interface):
+            # Setup networking in init-local stage.
             try:
                 with EphemeralDHCPv4(self.fallback_interface):
                     results = util.log_time(
diff --git a/cloudinit/sources/__init__.py b/cloudinit/sources/__init__.py
index e6966b3..00b88d4 100644
--- a/cloudinit/sources/__init__.py
+++ b/cloudinit/sources/__init__.py
@@ -158,15 +158,6 @@ class DataSource(object):
     url_timeout = 10    # timeout for each metadata url read attempt
     url_retries = 5     # number of times to retry url upon 404
 
-    # The datasource defines a set of supported EventTypes during which
-    # the datasource can react to changes in metadata and regenerate
-    # network configuration on metadata changes.
-    # A datasource which supports writing network config on each system boot
-    # would call update_events['network'].add(EventType.BOOT).
-
-    # Default: generate network config on new instance id (first boot).
-    update_events = {'network': set([EventType.BOOT_NEW_INSTANCE])}
-
     # N-tuple listing default values for any metadata-related class
     # attributes cached on an instance by a process_data runs. These attribute
     # values are reset via clear_cached_attrs during any update_metadata call.
@@ -176,6 +167,7 @@ class DataSource(object):
         ('vendordata', None), ('vendordata_raw', None))
 
     _dirty_cache = False
+    _update_events = {}
 
     # N-tuple of keypaths or keynames redact from instance-data.json for
     # non-root users
@@ -589,6 +581,24 @@ class DataSource(object):
     def get_package_mirror_info(self):
         return self.distro.get_package_mirror_info(data_source=self)
 
+    # The datasource defines a set of supported EventTypes during which
+    # the datasource can react to changes in metadata and regenerate
+    # network configuration on metadata changes.
+    # A datasource which supports writing network config on each system boot
+    # would call update_events['network'].add(EventType.BOOT).
+
+    # Default: generate network config on new instance id (first boot).
+    @property
+    def update_events(self):
+        if not self._update_events:
+            self._update_events = {'network':
+                                   set([EventType.BOOT_NEW_INSTANCE])}
+        return self._update_events
+
+    @update_events.setter
+    def update_events(self, events):
+        self._update_events.update(events)
+
     def update_metadata(self, source_event_types):
         """Refresh cached metadata if the datasource supports this event.
 
diff --git a/cloudinit/stages.py b/cloudinit/stages.py
index 8a06412..90c5c76 100644
--- a/cloudinit/stages.py
+++ b/cloudinit/stages.py
@@ -22,9 +22,8 @@ from cloudinit.handlers.cloud_config import CloudConfigPartHandler
 from cloudinit.handlers.jinja_template import JinjaTemplatePartHandler
 from cloudinit.handlers.shell_script import ShellScriptPartHandler
 from cloudinit.handlers.upstart_job import UpstartJobPartHandler
-
-from cloudinit.event import EventType
-
+from cloudinit.event import (
+    EventType, get_allowed_events, get_update_events_config)
 from cloudinit import cloud
 from cloudinit import config
 from cloudinit import distros
@@ -644,7 +643,48 @@ class Init(object):
                 return (ncfg, loc)
         return (self.distro.generate_fallback_config(), "fallback")
 
+    def update_event_allowed(self, event_source_type, scope=None):
+        # convert ds events to config
+        ds_config = get_update_events_config(self.datasource.update_events)
+        LOG.debug('Datasource updates cfg: %s', ds_config)
+
+        sys_config = self.cfg.get('updates', {})
+        LOG.debug('System updates cfg: %s', sys_config)
+
+        allowed = get_allowed_events(sys_config, ds_config)
+        LOG.debug('Allowable update events: %s', allowed)
+
+        if not scope:
+            scopes = [allowed.keys()]
+        else:
+            scopes = [scope]
+        LOG.debug('Possible scopes for this event: %s', scopes)
+
+        for evt_scope in scopes:
+            if event_source_type in allowed.get(evt_scope, []):
+                LOG.debug('Event Allowed: scope=%s EventType=%s',
+                          evt_scope, event_source_type)
+                return True
+
+        LOG.debug('Event Denied: scopes=%s EventType=%s',
+                  scopes, event_source_type)
+        return False
+
     def apply_network_config(self, bring_up):
+        apply_network = True
+        if self.datasource is not NULL_DATA_SOURCE:
+            if not self.is_new_instance():
+                if self.update_event_allowed(EventType.BOOT, scope='network'):
+                    if not self.datasource.update_metadata([EventType.BOOT]):
+                        LOG.debug(
+                            "No network config applied. Datasource failed"
+                            " update metadata on '%s' event", EventType.BOOT)
+                        apply_network = False
+                else:
+                    LOG.debug("No network config applied. "
+                              "'%s' event not allowed", EventType.BOOT)
+                    apply_network = False
+
         netcfg, src = self._find_networking_config()
         if netcfg is None:
             LOG.info("network config is disabled by %s", src)
@@ -656,14 +696,8 @@ class Init(object):
         except Exception as e:
             LOG.warning("Failed to rename devices: %s", e)
 
-        if self.datasource is not NULL_DATA_SOURCE:
-            if not self.is_new_instance():
-                if not self.datasource.update_metadata([EventType.BOOT]):
-                    LOG.debug(
-                        "No network config applied. Neither a new instance"
-                        " nor datasource network update on '%s' event",
-                        EventType.BOOT)
-                    return
+        if not apply_network:
+            return
 
         LOG.info("Applying network configuration from %s bringup=%s: %s",
                  src, bring_up, netcfg)
diff --git a/cloudinit/tests/test_event.py b/cloudinit/tests/test_event.py
new file mode 100644
index 0000000..ad81deb
--- /dev/null
+++ b/cloudinit/tests/test_event.py
@@ -0,0 +1,75 @@
+# This file is part of cloud-init. See LICENSE file for license information.
+
+"""Tests related to cloudinit.event module."""
+
+import copy
+import random
+import string
+
+from cloudinit.event import (EventType,
+                             EventNameMap,
+                             get_allowed_events,
+                             get_update_events_config)
+
+from cloudinit.tests.helpers import CiTestCase
+
+
+def rand_event_names():
+    return [random.choice(list(EventNameMap.keys()))
+            for x in range(len(EventNameMap.keys()))]
+
+
+def rand_string(size=6, chars=string.ascii_lowercase):
+    return ''.join(random.choice(chars) for x in range(size))
+
+
+class TestEvent(CiTestCase):
+    with_logs = True
+
+    DEFAULT_UPDATE_EVENTS = {'network': set([EventType.BOOT_NEW_INSTANCE]),
+                             'storage': set([])}
+    DEFAULT_UPDATES_CONFIG = {'policy-version': 1,
+                              'network': {'when': ['boot-new-instance']},
+                              'storage': {'when': []}}
+
+    def test_events_to_config(self):
+        """validate default update_events dictionary maps to default policy"""
+        events = copy.deepcopy(self.DEFAULT_UPDATE_EVENTS)
+        config = get_update_events_config(events)
+
+        for scope, events in events.items():
+            self.assertIn(scope, config)
+            for evt in events:
+                self.assertIn(evt, EventNameMap)
+                self.assertIn(EventNameMap.get(evt),
+                              config.get(scope).get('when'))
+
+        self.assertEqual(sorted(config),
+                         sorted(self.DEFAULT_UPDATES_CONFIG))
+
+    def test_get_allowed_events_defaults_filter_datasource(self):
+        ds_config = {
+            'policy-version': 1,
+            'network': {'when': rand_event_names()},
+            'storage': {'when': rand_event_names()},
+        }
+        allowed = get_allowed_events(self.DEFAULT_UPDATES_CONFIG, ds_config)
+
+        # system config filters out ds capabilities
+        self.assertEqual(sorted(allowed), sorted(self.DEFAULT_UPDATE_EVENTS))
+
+    def test_get_allowed_events_uses_system_config_scopes(self):
+        ds_config = {
+            'policy-version': 1,
+            'network': {'when': rand_event_names()},
+            'storage': {'when': rand_event_names()},
+        }
+        rand_scope = rand_string()
+        rand_events = rand_event_names()
+        sys_config = {'policy-version': 1, rand_scope: {'when': rand_events}}
+
+        self.assertNotIn(rand_scope, ds_config)
+        allowed = get_allowed_events(sys_config, ds_config)
+        self.assertIn(rand_scope, allowed)
+
+# vi: ts=4 expandtab
diff --git a/cloudinit/tests/test_stages.py b/cloudinit/tests/test_stages.py
index 94b6b25..037ca2f 100644
--- a/cloudinit/tests/test_stages.py
+++ b/cloudinit/tests/test_stages.py
@@ -47,6 +47,10 @@ class TestInit(CiTestCase):
             'distro': 'ubuntu', 'paths': {'cloud_dir': self.tmpdir,
                                           'run_dir': self.tmpdir}}}
         self.init.datasource = FakeDataSource(paths=self.init.paths)
+        self.init.datasource.update_events = {
+            'network': set([EventType.BOOT_NEW_INSTANCE])}
+        self.add_patch('cloudinit.stages.get_allowed_events', 'mock_allowed',
+                       return_value=self.init.datasource.update_events)
 
     def test_wb__find_networking_config_disabled(self):
         """find_networking_config returns no config when disabled."""
@@ -200,11 +204,10 @@ class TestInit(CiTestCase):
         self.init._find_networking_config = fake_network_config
         self.init.apply_network_config(True)
         self.init.distro.apply_network_config_names.assert_called_with(net_cfg)
+        self.assertIn("No network config applied. "
+                      "'%s' event not allowed" % EventType.BOOT,
+                      self.logs.getvalue())
         self.init.distro.apply_network_config.assert_not_called()
-        self.assertIn(
-            'No network config applied. Neither a new instance'
-            " nor datasource network update on '%s' event" % EventType.BOOT,
-            self.logs.getvalue())
 
     @mock.patch('cloudinit.distros.ubuntu.Distro')
     def test_apply_network_on_datasource_allowed_event(self, m_ubuntu):
@@ -222,10 +225,28 @@ class TestInit(CiTestCase):
 
         self.init._find_networking_config = fake_network_config
         self.init.datasource = FakeDataSource(paths=self.init.paths)
-        self.init.datasource.update_events = {'network': [EventType.BOOT]}
+        self.init.datasource.update_events = {'network': set([EventType.BOOT])}
         self.init.apply_network_config(True)
         self.init.distro.apply_network_config_names.assert_called_with(net_cfg)
         self.init.distro.apply_network_config.assert_called_with(
             net_cfg, bring_up=True)
 
+    def test_update_events_allowed_true_on_supported_event(self):
+        """Update Event Allowed True if source_event supported"""
+        self.assertTrue(
+            self.init.update_event_allowed(EventType.BOOT_NEW_INSTANCE,
+                                           scope='network'))
+
+    def test_update_events_allowed_false_on_unsupported_event(self):
+        """Update Event Allowed False if source_event supported"""
+        self.assertFalse(
+            self.init.update_event_allowed(EventType.BOOT,
+                                           scope='network'))
+
+    def test_update_events_allowed_false_on_unknown_scope(self):
+        """Update Event Allowed False if source_event supported"""
+        self.assertFalse(
+            self.init.update_event_allowed(EventType.BOOT, scope='xkcd'))
+
+
 # vi: ts=4 expandtab
diff --git a/cloudinit/version.py b/cloudinit/version.py
index 844a02e..1993983 100644
--- a/cloudinit/version.py
+++ b/cloudinit/version.py
@@ -12,6 +12,8 @@ FEATURES = [
     'NETWORK_CONFIG_V1',
     # supports network config version 2 (netplan)
     'NETWORK_CONFIG_V2',
+    # supports network config update via hotplug
+    'NETWORK_HOTPLUG',
 ]
 
 
diff --git a/config/cloud.cfg.d/10_updates_policy.cfg b/config/cloud.cfg.d/10_updates_policy.cfg
new file mode 100644
index 0000000..245a2d8
--- /dev/null
+++ b/config/cloud.cfg.d/10_updates_policy.cfg
@@ -0,0 +1,6 @@
+# default policy for cloud-init for when to update system config
+# such as network and storage configurations
+updates:
+  policy-version: 1
+  network:
+    when: ['boot-new-instance']
diff --git a/doc/rtd/index.rst b/doc/rtd/index.rst
index 20a99a3..23b848f 100644
--- a/doc/rtd/index.rst
+++ b/doc/rtd/index.rst
@@ -36,6 +36,7 @@ initialization of a cloud instance.
    topics/examples.rst
    topics/boot.rst
    topics/datasources.rst
+   topics/events.rst
    topics/logging.rst
    topics/modules.rst
    topics/merging.rst
diff --git a/doc/rtd/topics/capabilities.rst b/doc/rtd/topics/capabilities.rst
index 0d8b894..a40c998 100644
--- a/doc/rtd/topics/capabilities.rst
+++ b/doc/rtd/topics/capabilities.rst
@@ -41,6 +41,8 @@ Currently defined feature names include:
    see :ref:`network_config_v1` documentation for examples.
  - ``NETWORK_CONFIG_V2`` support for v2 networking configuration,
    see :ref:`network_config_v2` documentation for examples.
+ - ``NETWORK_HOTPLUG`` support for configuring network on hotplug,
+   see :ref:`events` documentation for more information.
 
 
 CLI Interface
@@ -237,6 +239,10 @@ likely be promoted to top-level subcommands when stable.
    containing  the jinja template header ``## template: jinja`` and renders
    that content with any instance-data.json variables present.
 
+ * ``cloud-init devel hotplug-hook``: Command called to handle when a new
+   device has been added to the system dynamically.  Typically called from
+   a Linux udev rule which provides device specific values which are passed
+   as command line options.
 
 .. _cli_clean:
 
diff --git a/doc/rtd/topics/datasources/smartos.rst b/doc/rtd/topics/datasources/smartos.rst
index cb9a128..42841f6 100644
--- a/doc/rtd/topics/datasources/smartos.rst
+++ b/doc/rtd/topics/datasources/smartos.rst
@@ -1,7 +1,7 @@
 .. _datasource_smartos:
 
-SmartOS Datasource
-==================
+SmartOS
+=======
 
 This datasource finds metadata and user-data from the SmartOS virtualization
 platform (i.e. Joyent).
diff --git a/doc/rtd/topics/events.rst b/doc/rtd/topics/events.rst
new file mode 100644
index 0000000..db8f657
--- /dev/null
+++ b/doc/rtd/topics/events.rst
@@ -0,0 +1,150 @@
+******************
+Events and Updates
+******************
+
+- Events
+- Datasource Event Support
+- Configuring Event Updates
+- Examples
+
+.. _events:
+
+Events
+======
+
+`Cloud-init`_ 's will fetch and apply cloud and user data configuration
+upon serveral event types.  The two most common events for `Cloud-init`_
+are when an instance first boots and any subsequent boot thereafter (reboot).
+In addition to boot events, `Cloud-init`_ users and vendors are interested
+in when devices are added.  `Cloud-init`_ currently supports the following
+event types:
+
+- **BOOT_NEW_INSTANCE**: ``New instance first boot``
+- **BOOT**: ``Any system boot other than 'BOOT_NEW_INSTANCE'``
+- **HOTPLUG**: ``Dynamic add of a system device``
+
+Future work will include infrastructure and support for the following
+events:
+
+- **METADATA_CHANGE**: ``An instance's metadata has change``
+- **USER_REQUEST**: ``Directed request to update``
+
+Datasource Event Support
+========================
+
+All :ref:`datasources` by default support the ``BOOT_NEW_INSTANCE`` event.
+Each Datasource will provide a set of events that it is capable of handling.
+Datasources may not support all event types.  In some cases a system
+may be configured to allow a particular event but may be running on
+a platform who's datasource cannot support the event.
+
+.. table::
+  :widths: auto
+
+  +-----------------------------+------------------+
+  | Datasource                  | Supported Events |
+  +=============================+==================+
+  | :ref:`datasource_azure`     | BOOT             |
+  +-----------------------------+------------------+
+  | :ref:`datasource_openstack` | BOOT, HOTPLUG    |
+  +-----------------------------+------------------+
+  | :ref:`datasource_smartos`   | BOOT             |
+  +-----------------------------+------------------+
+
+
+Configuring Event Updates
+=========================
+
+`Cloud-init`_ has a default updates policy to handle new instance
+events always.  Vendors may want an instance to handle additional
+events.  Users have the final say and may provide update configuration
+which can be used to enable or disable handling of specific events.
+
+updates
+~~~~~~~
+Specify update policy configuration for cloud-init to define which
+events are allowed to be handled.  This is separate from whether a
+particular platform or datasource has the capability for such events.
+
+**policy-version**: *<Latest policy version, currently 1>*
+
+The ``policy-version`` value specifies the updates configuration
+version number.  Current version is 1, future versions may modify
+the configuation structure.
+
+**scope**:  *<name of the scope for event policy>*
+
+The ``scope`` value is a string which defines under which domain do the
+event occur.  Currently there are two known scopes: ``network`` and
+``storage``.  Scopes are defined by convention but arbitrary values
+can be used.
+
+**when**: *<list of events to handle for a particular scope>*
+
+Each ``scope`` requires a ``when`` element to specify which events
+are to allowed to be handled.
+
+
+Examples
+========
+
+default
+~~~~~~~
+
+The default policy for handling new instances is found in
+/etc/cloud/cloud.cfg.d/10_updates_policy.cfg
+
+.. code-block:: shell-session
+
+ # default policy for cloud-init for when to update system config
+ # such as network and storage configurations
+ updates:
+   policy-version: 1
+   network:
+     when: ['boot-new-instance']
+
+This default policy indicates that whenever cloud-init generates a
+``BOOT_NEW_INSTANCE`` event that the ``network`` scope will be updated.
+This results in cloud-init applying network configuration when booting
+a new instance.
+
+.. note::
+  Removing 'boot-new-instance' from the policy will cause issues when
+  capturing images and booting them else where as the network config
+  will remain static.
+
+apply network config every boot
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+On each firsboot and every boot cloud-init will apply network configuration
+found in the datasource.
+
+.. code-block:: shell-session
+
+ # apply network config on every boot
+ updates:
+   policy-version: 1
+   network:
+     when: ['boot-new-instance', 'boot']
+
+apply network config on hotplug
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Apply network configuration from the datasource on first boot, each boot
+thereafter and when new network devices are dynamically added.
+
+
+.. code-block:: shell-session
+
+ # apply network config on every boot and hotplug
+ updates:
+   policy-version: 1
+   network:
+     when: ['boot-new-instance', 'boot', 'hotplug']
+
+.. note::
+   When enabling hotplug, it's best practice to also enable the boot event.
+   In the case of a device removal, the network configuration will be
+   reconfigure on the very next boot.
+
+
+.. _Cloud-init: https://launchpad.net/cloud-init
+.. vi: textwidth=78
diff --git a/setup.py b/setup.py
index ea37efc..7b76bc4 100755
--- a/setup.py
+++ b/setup.py
@@ -138,6 +138,7 @@ INITSYS_FILES = {
     'systemd': [render_tmpl(f)
                 for f in (glob('systemd/*.tmpl') +
                           glob('systemd/*.service') +
+                          glob('systemd/*.socket') +
                           glob('systemd/*.target')) if is_f(f)],
     'systemd.generators': [f for f in glob('systemd/*-generator') if is_f(f)],
     'upstart': [f for f in glob('upstart/*') if is_f(f)],
@@ -243,6 +244,7 @@ data_files = [
     (ETC + '/cloud/cloud.cfg.d', glob('config/cloud.cfg.d/*')),
     (ETC + '/cloud/templates', glob('templates/*')),
     (USR_LIB_EXEC + '/cloud-init', ['tools/ds-identify',
+                                    'tools/hook-hotplug',
                                     'tools/uncloud-init',
                                     'tools/write-ssh-key-fingerprints']),
     (USR + '/share/doc/cloud-init', [f for f in glob('doc/*') if is_f(f)]),
diff --git a/systemd/cloud-init-hotplugd.service b/systemd/cloud-init-hotplugd.service
new file mode 100644
index 0000000..6f231cd
--- /dev/null
+++ b/systemd/cloud-init-hotplugd.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=cloud-init hotplug hook daemon
+After=cloud-init-hotplugd.socket
+
+[Service]
+Type=simple
+ExecStart=/bin/bash -c 'read args <&3; echo "args=$args"; \
+                        exec /usr/bin/cloud-init devel hotplug-hook $args; \
+                        exit 0'
+SyslogIdentifier=cloud-init-hotplugd
+TimeoutStopSec=5
diff --git a/systemd/cloud-init-hotplugd.socket b/systemd/cloud-init-hotplugd.socket
new file mode 100644
index 0000000..f8f1048
--- /dev/null
+++ b/systemd/cloud-init-hotplugd.socket
@@ -0,0 +1,8 @@
+[Unit]
+Description=cloud-init hotplug hook socket
+
+[Socket]
+ListenFIFO=/run/cloud-init/hook-hotplug-cmd
+
+[Install]
+WantedBy=cloud-init.target
diff --git a/tools/hook-hotplug b/tools/hook-hotplug
new file mode 100755
index 0000000..469c45b
--- /dev/null
+++ b/tools/hook-hotplug
@@ -0,0 +1,26 @@
+#!/bin/bash
+# This file is part of cloud-init. See LICENSE file for license information.
+
+# This script checks if cloud-init has hotplug hooked and if
+# cloud-init has finished; if so invoke cloud-init hotplug-hook
+
+is_finished() {
+    [ -e /run/cloud-init/result.json ]
+}
+
+if is_finished; then
+    # only hook pci devices at this time
+    case "${DEVPATH}" in
+        /devices/pci*)
+            # open cloud-init's hotplug-hook fifo rw
+            exec 3<>/run/cloud-init/hook-hotplug-cmd
+            env_params=(
+                --devpath="${DEVPATH}"
+                --subsystem="${SUBSYSTEM}"
+                --udevaction="${ACTION}"
+            )
+            # write params to cloud-init's hotplug-hook fifo
+            echo "--hotplug-debug ${env_params[@]}" >&3
+            ;;
+    esac
+fi
diff --git a/udev/10-cloud-init-hook-hotplug.rules b/udev/10-cloud-init-hook-hotplug.rules
new file mode 100644
index 0000000..74324f4
--- /dev/null
+++ b/udev/10-cloud-init-hook-hotplug.rules
@@ -0,0 +1,5 @@
+# Handle device adds only
+ACTION!="add", GOTO="cloudinit_end"
+LABEL="cloudinit_hook"
+SUBSYSTEM=="net|block", RUN+="/usr/lib/cloud-init/hook-hotplug"
+LABEL="cloudinit_end"