Merge lp:~cloud-init-dev/cloud-init/trunk into lp:~jbauer/cloud-init/salt

[Jeff Bauer]

fix launchpad bug #996166, installs wrong salt pkg

Fixes: https://bugs.launchpad.net/cloud-init/+bug/996166

installs wrong package in cc_salt_minion.py

I'm not sure if I've got the bzr merge correct, but it's only a one line modification:

=== modified file 'cloudinit/CloudConfig/cc_salt_minion.py'
--- cloudinit/CloudConfig/cc_salt_minion.py 2012-05-08 17:18:53 +0000
+++ cloudinit/CloudConfig/cc_salt_minion.py 2012-05-08 17:33:32 +0000
@@ -27,7 +27,7 @@
         return
     salt_cfg = cfg['salt_minion']
     # Start by installing the salt package ...
- cc.install_packages(("salt",))
+ cc.install_packages(("salt-minion",))
     config_dir = '/etc/salt'
     if not os.path.isdir(config_dir):
         os.makedirs(config_dir)

[lp:~cloud-init-dev/cloud-init/trunk]

558. By Scott Moser on 2012-05-22

support relative path in AuthorizedKeysFile

559. By Scott Moser on 2012-05-22

remove usage of subprocess.check_output

in order to work on python 2.6, replace usage of check_output with util.subp.

560. By Scott Moser on 2012-06-13

Use --quiet when running apt-get

Use the --quiet switch when running apt-get to get output suitable for
logging, rather than with pretty progress updates designed for interactive
use. This makes the log, as returned by GetConsoleOutput for instance, a
little shorter and easier to read. Some action completion notices are also
missed, but it's pretty clear still as no error output appears before
cloud-init goes on to the next thing.

Mer apt-get man page:
  Quiet; produces output suitable for logging, omitting progress indicators.

561. By Scott Moser on 2012-06-20

cc_salt_minion: install package salt-minion rather than salt

=== modified file 'ChangeLog'
--- ChangeLog	2012-01-30 14:24:41 +0000
+++ ChangeLog	2012-06-13 13:16:19 +0000
@@ -1,3 +1,6 @@
+0.6.4:
+ - support relative path in AuthorizedKeysFile (LP: #970071).
+ - make apt-get update run with --quiet (suitable for logging) (LP: #1012613)
 0.6.3:
  - add sample systemd config files [Garrett Holmstrom]
  - add Fedora support [Garrent Holstrom] (LP: #883286)
@@ -8,7 +11,8 @@
  - support setting of Acquire::HTTP::Proxy via 'apt_proxy'
  - DataSourceEc2: more resilliant to slow metadata service
    - config change: 'retries' dropped, 'max_wait' added, timeout increased
- - close stdin in all cloud-init programs that are launched at boot (LP: #903993)
+ - close stdin in all cloud-init programs that are launched at boot 
+   (LP: #903993)
  - revert management of /etc/hosts to 0.6.1 style (LP: #890501, LP: #871966)
  - write full ssh keys to console for easy machine consumption (LP: #893400)
  - put INSTANCE_ID environment variable in bootcmd scripts
@@ -19,9 +23,33 @@
    in the payload parameter. (LP: #874342)
  - add test case framework [Mike Milner] (LP: #890851)
  - fix pylint warnings [Juerg Haefliger] (LP: #914739)
- - add support for adding and deleting CA Certificates [Mike Milner] (LP: #915232)
+ - add support for adding and deleting CA Certificates [Mike Milner] 
+   (LP: #915232)
  - in ci-info lines, use '.' to indicate empty field for easier machine reading
  - support empty lines in "#include" files (LP: #923043)
+ - support configuration of salt minions (Jeff Bauer) (LP: #927795)
+ - DataSourceOVF: only search for OVF data on ISO9660 filesystems (LP: #898373)
+ - DataSourceConfigDrive: support getting data from openstack config drive
+   (LP: #857378)
+ - DataSourceNoCloud: support seed from external disk of ISO or vfat
+   (LP: #857378)
+ - DataSourceNoCloud: support inserting /etc/network/interfaces
+ - DataSourceMaaS: add data source for Ubuntu Machines as a Service (MaaS)
+   (LP: #942061)
+ - DataSourceCloudStack: add support for CloudStack datasource [Cosmin Luta]
+ - add option 'apt_pipelining' to address issue with S3 mirrors 
+   (LP: #948461) [Ben Howard]
+ - warn on non-multipart, non-handled user-data [Martin Packman]
+ - run resizefs in the background in order to not block boot (LP: #961226)
+ - Fix bug in Chef support where validation_key was present in config, but
+   'validation_cert' was not (LP: #960547)
+ - Provide user friendly message when an invalid locale is set 
+   [Ben Howard] (LP: #859814)
+ - Support reading cloud-config from kernel command line parameter and
+   populating local file with it, which can then provide data for DataSources
+ - improve chef examples for working configurations on 11.10 and 12.04 
+   [Lorin Hochstein] (LP: #960564)
+
 0.6.2:
  - fix bug where update was not done unless update was explicitly set.
    It would not be run if 'upgrade' or packages were set to be installed
@@ -59,18 +87,20 @@
  - support multiple staticly configured network devices, as long as
    all of them come up early (LP: #810044)
  - Changes to handling user data mean that:
-   * boothooks will now run more than once as they were intended (and as bootcmd
-     commands do)
+   * boothooks will now run more than once as they were intended (and as
+     bootcmd commands do)
    * cloud-config and user-scripts will be updated from user data every boot
  - Fix issue where 'isatty' would return true for apt-add-repository.
    apt-add-repository would get stdin which was attached to a terminal
-  (/dev/console) and would thus hang when running during boot.  (LP: 831505)
-   This was done by changing all users of util.subp to have None input unless specified
+   (/dev/console) and would thus hang when running during boot.  (LP: 831505)
+   This was done by changing all users of util.subp to have None input unless
+   specified
  - Add some debug info to the console when cloud-init runs.
-   This is useful if debugging, IP and route information is printed to the console.
+   This is useful if debugging, IP and route information is printed to the 
+   console.
  - change the mechanism for handling .ssh/authorized_keys, to update entries
-   rather than appending.  This ensures that the authorized_keys that are being
-   inserted actually do something (LP: #434076, LP: #833499)
+   rather than appending.  This ensures that the authorized_keys that are
+   being inserted actually do something (LP: #434076, LP: #833499)
  - log warning on failure to set hostname (LP: #832175)
  - upstart/cloud-init-nonet.conf: wait for all network interfaces to be up
    allow for the possibility of /var/run != /run.
 
=== modified file 'cloud-init.py'
--- cloud-init.py	2012-01-18 14:07:33 +0000
+++ cloud-init.py	2012-06-13 13:16:19 +0000
@@ -28,6 +28,7 @@
 import cloudinit.DataSource as ds
 import cloudinit.netinfo as netinfo
 import time
+import traceback
 import logging
 import errno
 import os
@@ -67,6 +68,30 @@
         warn("unable to open /proc/uptime\n")
         uptime = "na"
 
+    cmdline_msg = None
+    cmdline_exc = None
+    if cmd == "start":
+        target = "%s.d/%s" % (cloudinit.system_config,
+            "91_kernel_cmdline_url.cfg")
+        if os.path.exists(target):
+            cmdline_msg = "cmdline: %s existed" % target
+        else:
+            cmdline = util.get_cmdline()
+            try:
+                (key, url, content) = cloudinit.get_cmdline_url(
+                    cmdline=cmdline)
+                if key and content:
+                    util.write_file(target, content, mode=0600)
+                    cmdline_msg = ("cmdline: wrote %s from %s, %s" %
+                        (target, key, url))
+                elif key:
+                    cmdline_msg = ("cmdline: %s, %s had no cloud-config" %
+                        (key, url))
+            except Exception:
+                cmdline_exc = ("cmdline: '%s' raised exception\n%s" %
+                    (cmdline, traceback.format_exc()))
+                warn(cmdline_exc)
+
     try:
         cfg = cloudinit.get_base_cfg(cfg_path)
     except Exception as e:
@@ -86,6 +111,11 @@
     cloudinit.logging_set_from_cfg(cfg)
     log = logging.getLogger()
 
+    if cmdline_exc:
+        log.debug(cmdline_exc)
+    elif cmdline_msg:
+        log.debug(cmdline_msg)
+
     try:
         cloudinit.initfs()
     except Exception as e:
@@ -136,7 +166,7 @@
         cloud.get_data_source()
     except cloudinit.DataSourceNotFoundException as e:
         sys.stderr.write("no instance data found in %s\n" % cmd)
-        sys.exit(1)
+        sys.exit(0)
 
     # set this as the current instance
     cloud.set_cur_instance()
 
=== modified file 'cloudinit/CloudConfig/__init__.py'
--- cloudinit/CloudConfig/__init__.py	2012-01-18 14:07:33 +0000
+++ cloudinit/CloudConfig/__init__.py	2012-06-13 13:16:19 +0000
@@ -260,7 +260,7 @@
     e = os.environ.copy()
     e['DEBIAN_FRONTEND'] = 'noninteractive'
     cmd = ['apt-get', '--option', 'Dpkg::Options::=--force-confold',
-           '--assume-yes', tlc]
+           '--assume-yes', '--quiet', tlc]
     cmd.extend(args)
     subprocess.check_call(cmd, env=e)
 
 
=== added file 'cloudinit/CloudConfig/cc_apt_pipelining.py'
--- cloudinit/CloudConfig/cc_apt_pipelining.py	1970-01-01 00:00:00 +0000
+++ cloudinit/CloudConfig/cc_apt_pipelining.py	2012-06-13 13:16:19 +0000
@@ -0,0 +1,53 @@
+# vi: ts=4 expandtab
+#
+#    Copyright (C) 2011 Canonical Ltd.
+#
+#    Author: Ben Howard <[e-mail]>
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU General Public License version 3, as
+#    published by the Free Software Foundation.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+#
+#    You should have received a copy of the GNU General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import cloudinit.util as util
+from cloudinit.CloudConfig import per_instance
+
+frequency = per_instance
+default_file = "/etc/apt/apt.conf.d/90cloud-init-pipelining"
+
+
+def handle(_name, cfg, _cloud, log, _args):
+
+    apt_pipe_value = util.get_cfg_option_str(cfg, "apt_pipelining", False)
+    apt_pipe_value = str(apt_pipe_value).lower()
+
+    if apt_pipe_value == "false":
+        write_apt_snippet("0", log)
+
+    elif apt_pipe_value in ("none", "unchanged", "os"):
+        return
+
+    elif apt_pipe_value in str(range(0, 6)):
+        write_apt_snippet(apt_pipe_value, log)
+
+    else:
+        log.warn("Invalid option for apt_pipeling: %s" % apt_pipe_value)
+
+
+def write_apt_snippet(setting, log, f_name=default_file):
+    """ Writes f_name with apt pipeline depth 'setting' """
+
+    acquire_pipeline_depth = 'Acquire::http::Pipeline-Depth "%s";\n'
+    file_contents = ("//Written by cloud-init per 'apt_pipelining'\n"
+                     + (acquire_pipeline_depth % setting))
+
+    util.write_file(f_name, file_contents)
+
+    log.debug("Wrote %s with APT pipeline setting" % f_name)
 
=== modified file 'cloudinit/CloudConfig/cc_ca_certs.py'
--- cloudinit/CloudConfig/cc_ca_certs.py	2012-01-17 21:38:01 +0000
+++ cloudinit/CloudConfig/cc_ca_certs.py	2012-06-13 13:16:19 +0000
@@ -16,7 +16,7 @@
 import os
 from subprocess import check_call
 from cloudinit.util import (write_file, get_cfg_option_list_or_str,
-                            delete_dir_contents)
+                            delete_dir_contents, subp)
 
 CA_CERT_PATH = "/usr/share/ca-certificates/"
 CA_CERT_FILENAME = "cloud-init-ca-certs.crt"
@@ -54,6 +54,8 @@
     delete_dir_contents(CA_CERT_PATH)
     delete_dir_contents(CA_CERT_SYSTEM_PATH)
     write_file(CA_CERT_CONFIG, "", mode=0644)
+    debconf_sel = "ca-certificates ca-certificates/trust_new_crts select no"
+    subp(('debconf-set-selections', '-'), debconf_sel)
 
 
 def handle(_name, cfg, _cloud, log, _args):
 
=== modified file 'cloudinit/CloudConfig/cc_chef.py'
--- cloudinit/CloudConfig/cc_chef.py	2012-01-18 14:07:33 +0000
+++ cloudinit/CloudConfig/cc_chef.py	2012-06-13 13:16:19 +0000
@@ -40,11 +40,11 @@
     # set the validation key based on the presence of either 'validation_key'
     # or 'validation_cert'. In the case where both exist, 'validation_key'
     # takes precedence
-    if ('validation_key' in chef_cfg or 'validation_cert' in chef_cfg):
-        validation_key = util.get_cfg_option_str(chef_cfg, 'validation_key',
-                                                 chef_cfg['validation_cert'])
-        with open('/etc/chef/validation.pem', 'w') as validation_key_fh:
-            validation_key_fh.write(validation_key)
+    for key in ('validation_key', 'validation_cert'):
+        if key in chef_cfg and chef_cfg[key]:
+            with open('/etc/chef/validation.pem', 'w') as validation_key_fh:
+                validation_key_fh.write(chef_cfg[key])
+            break
 
     # create the chef config from template
     util.render_to_file('chef_client.rb', '/etc/chef/client.rb',
 
=== modified file 'cloudinit/CloudConfig/cc_landscape.py'
--- cloudinit/CloudConfig/cc_landscape.py	2012-01-18 14:07:33 +0000
+++ cloudinit/CloudConfig/cc_landscape.py	2012-06-13 13:16:19 +0000
@@ -18,6 +18,8 @@
 #    You should have received a copy of the GNU General Public License
 #    along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+import os
+import os.path
 from cloudinit.CloudConfig import per_instance
 from configobj import ConfigObj
 
@@ -50,6 +52,9 @@
 
     merged = mergeTogether([lsc_builtincfg, lsc_client_cfg_file, ls_cloudcfg])
 
+    if not os.path.isdir(os.path.dirname(lsc_client_cfg_file)):
+        os.makedirs(os.path.dirname(lsc_client_cfg_file))
+
     with open(lsc_client_cfg_file, "w") as fp:
         merged.write(fp)
 
 
=== modified file 'cloudinit/CloudConfig/cc_resizefs.py'
--- cloudinit/CloudConfig/cc_resizefs.py	2012-01-18 14:07:33 +0000
+++ cloudinit/CloudConfig/cc_resizefs.py	2012-06-13 13:16:19 +0000
@@ -22,6 +22,8 @@
 import subprocess
 import os
 import stat
+import sys
+import time
 import tempfile
 from cloudinit.CloudConfig import per_always
 
@@ -34,23 +36,22 @@
         if str(args[0]).lower() in ['true', '1', 'on', 'yes']:
             resize_root = True
     else:
-        resize_root = util.get_cfg_option_bool(cfg, "resize_rootfs", True)
+        resize_root = util.get_cfg_option_str(cfg, "resize_rootfs", True)
 
-    if not resize_root:
+    if str(resize_root).lower() in ['false', '0']:
         return
 
-    # this really only uses the filename from mktemp, then we mknod into it
-    (fd, devpth) = tempfile.mkstemp()
-    os.unlink(devpth)
-    os.close(fd)
+    # we use mktemp rather than mkstemp because early in boot nothing
+    # else should be able to race us for this, and we need to mknod.
+    devpth = tempfile.mktemp(prefix="cloudinit.resizefs.", dir="/run")
 
     try:
         st_dev = os.stat("/").st_dev
         dev = os.makedev(os.major(st_dev), os.minor(st_dev))
         os.mknod(devpth, 0400 | stat.S_IFBLK, dev)
     except:
-        if util.islxc():
-            log.debug("inside lxc, ignoring mknod failure in resizefs")
+        if util.is_container():
+            log.debug("inside container, ignoring mknod failure in resizefs")
             return
         log.warn("Failed to make device node to resize /")
         raise
@@ -65,9 +66,6 @@
         os.unlink(devpth)
         raise
 
-    log.debug("resizing root filesystem (type=%s, maj=%i, min=%i)" %
-        (str(fstype).rstrip("\n"), os.major(st_dev), os.minor(st_dev)))
-
     if str(fstype).startswith("ext"):
         resize_cmd = ['resize2fs', devpth]
     elif fstype == "xfs":
@@ -77,7 +75,28 @@
         log.debug("not resizing unknown filesystem %s" % fstype)
         return
 
+    if resize_root == "noblock":
+        fid = os.fork()
+        if fid == 0:
+            try:
+                do_resize(resize_cmd, devpth, log)
+                os._exit(0)  # pylint: disable=W0212
+            except Exception as exc:
+                sys.stderr.write("Failed: %s" % exc)
+                os._exit(1)  # pylint: disable=W0212
+    else:
+        do_resize(resize_cmd, devpth, log)
+
+    log.debug("resizing root filesystem (type=%s, maj=%i, min=%i, val=%s)" %
+        (str(fstype).rstrip("\n"), os.major(st_dev), os.minor(st_dev),
+         resize_root))
+
+    return
+
+
+def do_resize(resize_cmd, devpth, log):
     try:
+        start = time.time()
         util.subp(resize_cmd)
     except subprocess.CalledProcessError as e:
         log.warn("Failed to resize filesystem (%s)" % resize_cmd)
@@ -86,4 +105,4 @@
         raise
 
     os.unlink(devpth)
-    return
+    log.debug("resize took %s seconds" % (time.time() - start))
 
=== modified file 'cloudinit/CloudConfig/cc_salt_minion.py'
--- cloudinit/CloudConfig/cc_salt_minion.py	2012-02-11 15:27:14 +0000
+++ cloudinit/CloudConfig/cc_salt_minion.py	2012-06-13 13:16:19 +0000
@@ -20,7 +20,8 @@
 import cloudinit.CloudConfig as cc
 import yaml
 
-def handle(_name, cfg, cloud, log, _args):
+
+def handle(_name, cfg, _cloud, _log, _args):
     # If there isn't a salt key in the configuration don't do anything
     if 'salt_minion' not in cfg:
         return
 
=== modified file 'cloudinit/CloudConfig/cc_update_etc_hosts.py'
--- cloudinit/CloudConfig/cc_update_etc_hosts.py	2012-01-18 14:07:33 +0000
+++ cloudinit/CloudConfig/cc_update_etc_hosts.py	2012-06-13 13:16:19 +0000
@@ -28,7 +28,7 @@
 def handle(_name, cfg, cloud, log, _args):
     (hostname, fqdn) = util.get_hostname_fqdn(cfg, cloud)
 
-    manage_hosts = util.get_cfg_option_bool(cfg, "manage_etc_hosts", False)
+    manage_hosts = util.get_cfg_option_str(cfg, "manage_etc_hosts", False)
     if manage_hosts in ("True", "true", True, "template"):
         # render from template file
         try:
 
=== modified file 'cloudinit/DataSource.py'
--- cloudinit/DataSource.py	2012-01-18 14:07:33 +0000
+++ cloudinit/DataSource.py	2012-06-13 13:16:19 +0000
@@ -70,7 +70,10 @@
             return([])
 
         if isinstance(self.metadata['public-keys'], str):
-            return([self.metadata['public-keys'], ])
+            return(str(self.metadata['public-keys']).splitlines())
+
+        if isinstance(self.metadata['public-keys'], list):
+            return(self.metadata['public-keys'])
 
         for _keyname, klist in self.metadata['public-keys'].items():
             # lp:506332 uec metadata service responds with
 
=== added file 'cloudinit/DataSourceCloudStack.py'
--- cloudinit/DataSourceCloudStack.py	1970-01-01 00:00:00 +0000
+++ cloudinit/DataSourceCloudStack.py	2012-06-13 13:16:19 +0000
@@ -0,0 +1,92 @@
+# vi: ts=4 expandtab
+#
+#    Copyright (C) 2012 Canonical Ltd.
+#    Copyright (C) 2012 Cosmin Luta
+#
+#    Author: Cosmin Luta <[e-mail]>
+#    Author: Scott Moser <[e-mail]>
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU General Public License version 3, as
+#    published by the Free Software Foundation.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+#
+#    You should have received a copy of the GNU General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import cloudinit.DataSource as DataSource
+
+from cloudinit import seeddir as base_seeddir
+from cloudinit import log
+import cloudinit.util as util
+from socket import inet_ntoa
+import time
+import boto.utils as boto_utils
+from struct import pack
+
+
+class DataSourceCloudStack(DataSource.DataSource):
+    api_ver = 'latest'
+    seeddir = base_seeddir + '/cs'
+    metadata_address = None
+
+    def __init__(self, sys_cfg=None):
+        DataSource.DataSource.__init__(self, sys_cfg)
+        # Cloudstack has its metadata/userdata URLs located at
+        # http://<default-gateway-ip>/latest/
+        self.metadata_address = "http://%s/" % self.get_default_gateway()
+
+    def get_default_gateway(self):
+        """ Returns the default gateway ip address in the dotted format
+        """
+        with open("/proc/net/route", "r") as f:
+            for line in f.readlines():
+                items = line.split("\t")
+                if items[1] == "00000000":
+                    # found the default route, get the gateway
+                    gw = inet_ntoa(pack("<L", int(items[2], 16)))
+                    log.debug("found default route, gateway is %s" % gw)
+                    return gw
+
+    def __str__(self):
+        return "DataSourceCloudStack"
+
+    def get_data(self):
+        seedret = {}
+        if util.read_optional_seed(seedret, base=self.seeddir + "/"):
+            self.userdata_raw = seedret['user-data']
+            self.metadata = seedret['meta-data']
+            log.debug("using seeded cs data in %s" % self.seeddir)
+            return True
+
+        try:
+            start = time.time()
+            self.userdata_raw = boto_utils.get_instance_userdata(self.api_ver,
+                None, self.metadata_address)
+            self.metadata = boto_utils.get_instance_metadata(self.api_ver,
+                self.metadata_address)
+            log.debug("crawl of metadata service took %ds" %
+                (time.time() - start))
+            return True
+        except Exception as e:
+            log.exception(e)
+            return False
+
+    def get_instance_id(self):
+        return self.metadata['instance-id']
+
+    def get_availability_zone(self):
+        return self.metadata['availability-zone']
+
+datasources = [
+  (DataSourceCloudStack, (DataSource.DEP_FILESYSTEM, DataSource.DEP_NETWORK)),
+]
+
+
+# return a list of data sources that match this set of dependencies
+def get_datasource_list(depends):
+    return DataSource.list_from_depends(depends, datasources)
 
=== added file 'cloudinit/DataSourceConfigDrive.py'
--- cloudinit/DataSourceConfigDrive.py	1970-01-01 00:00:00 +0000
+++ cloudinit/DataSourceConfigDrive.py	2012-06-13 13:16:19 +0000
@@ -0,0 +1,231 @@
+#    Copyright (C) 2012 Canonical Ltd.
+#
+#    Author: Scott Moser <[e-mail]>
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU General Public License version 3, as
+#    published by the Free Software Foundation.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+#
+#    You should have received a copy of the GNU General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import cloudinit.DataSource as DataSource
+
+from cloudinit import seeddir as base_seeddir
+from cloudinit import log
+import cloudinit.util as util
+import os.path
+import os
+import json
+import subprocess
+
+DEFAULT_IID = "iid-dsconfigdrive"
+
+
+class DataSourceConfigDrive(DataSource.DataSource):
+    seed = None
+    seeddir = base_seeddir + '/config_drive'
+    cfg = {}
+    userdata_raw = None
+    metadata = None
+    dsmode = "local"
+
+    def __str__(self):
+        mstr = "DataSourceConfigDrive[%s]" % self.dsmode
+        mstr = mstr + " [seed=%s]" % self.seed
+        return(mstr)
+
+    def get_data(self):
+        found = None
+        md = {}
+        ud = ""
+
+        defaults = {"instance-id": DEFAULT_IID, "dsmode": "pass"}
+
+        if os.path.isdir(self.seeddir):
+            try:
+                (md, ud) = read_config_drive_dir(self.seeddir)
+                found = self.seeddir
+            except nonConfigDriveDir:
+                pass
+
+        if not found:
+            dev = cfg_drive_device()
+            if dev:
+                try:
+                    (md, ud) = util.mount_callback_umount(dev,
+                        read_config_drive_dir)
+                    found = dev
+                except (nonConfigDriveDir, util.mountFailedError):
+                    pass
+
+        if not found:
+            return False
+
+        if 'dsconfig' in md:
+            self.cfg = md['dscfg']
+
+        md = util.mergedict(md, defaults)
+
+        # update interfaces and ifup only on the local datasource
+        # this way the DataSourceConfigDriveNet doesn't do it also.
+        if 'network-interfaces' in md and self.dsmode == "local":
+            if md['dsmode'] == "pass":
+                log.info("updating network interfaces from configdrive")
+            else:
+                log.debug("updating network interfaces from configdrive")
+
+            util.write_file("/etc/network/interfaces",
+                md['network-interfaces'])
+            try:
+                (out, err) = util.subp(['ifup', '--all'])
+                if len(out) or len(err):
+                    log.warn("ifup --all had stderr: %s" % err)
+
+            except subprocess.CalledProcessError as exc:
+                log.warn("ifup --all failed: %s" % (exc.output[1]))
+
+        self.seed = found
+        self.metadata = md
+        self.userdata_raw = ud
+
+        if md['dsmode'] == self.dsmode:
+            return True
+
+        log.debug("%s: not claiming datasource, dsmode=%s" %
+            (self, md['dsmode']))
+        return False
+
+    def get_public_ssh_keys(self):
+        if not 'public-keys' in self.metadata:
+            return([])
+        return(self.metadata['public-keys'])
+
+    # the data sources' config_obj is a cloud-config formated
+    # object that came to it from ways other than cloud-config
+    # because cloud-config content would be handled elsewhere
+    def get_config_obj(self):
+        return(self.cfg)
+
+
+class DataSourceConfigDriveNet(DataSourceConfigDrive):
+    dsmode = "net"
+
+
+class nonConfigDriveDir(Exception):
+    pass
+
+
+def cfg_drive_device():
+    """ get the config drive device.  return a string like '/dev/vdb'
+        or None (if there is no non-root device attached). This does not
+        check the contents, only reports that if there *were* a config_drive
+        attached, it would be this device.
+        per config_drive documentation, this is
+         "associated as the last available disk on the instance"
+    """
+
+    if 'CLOUD_INIT_CONFIG_DRIVE_DEVICE' in os.environ:
+        return(os.environ['CLOUD_INIT_CONFIG_DRIVE_DEVICE'])
+
+    # we are looking for a raw block device (sda, not sda1) with a vfat
+    # filesystem on it.
+
+    letters = "abcdefghijklmnopqrstuvwxyz"
+    devs = util.find_devs_with("TYPE=vfat")
+
+    # filter out anything not ending in a letter (ignore partitions)
+    devs = [f for f in devs if f[-1] in letters]
+
+    # sort them in reverse so "last" device is first
+    devs.sort(reverse=True)
+
+    if len(devs):
+        return(devs[0])
+
+    return(None)
+
+
+def read_config_drive_dir(source_dir):
+    """
+    read_config_drive_dir(source_dir):
+       read source_dir, and return a tuple with metadata dict and user-data
+       string populated.  If not a valid dir, raise a nonConfigDriveDir
+    """
+    md = {}
+    ud = ""
+
+    flist = ("etc/network/interfaces", "root/.ssh/authorized_keys", "meta.js")
+    found = [f for f in flist if os.path.isfile("%s/%s" % (source_dir, f))]
+    keydata = ""
+
+    if len(found) == 0:
+        raise nonConfigDriveDir("%s: %s" % (source_dir, "no files found"))
+
+    if "etc/network/interfaces" in found:
+        with open("%s/%s" % (source_dir, "/etc/network/interfaces")) as fp:
+            md['network-interfaces'] = fp.read()
+
+    if "root/.ssh/authorized_keys" in found:
+        with open("%s/%s" % (source_dir, "root/.ssh/authorized_keys")) as fp:
+            keydata = fp.read()
+
+    meta_js = {}
+
+    if "meta.js" in found:
+        content = ''
+        with open("%s/%s" % (source_dir, "meta.js")) as fp:
+            content = fp.read()
+        md['meta_js'] = content
+        try:
+            meta_js = json.loads(content)
+        except ValueError:
+            raise nonConfigDriveDir("%s: %s" %
+                (source_dir, "invalid json in meta.js"))
+
+    keydata = meta_js.get('public-keys', keydata)
+
+    if keydata:
+        lines = keydata.splitlines()
+        md['public-keys'] = [l for l in lines
+            if len(l) and not l.startswith("#")]
+
+    for copy in ('dsmode', 'instance-id', 'dscfg'):
+        if copy in meta_js:
+            md[copy] = meta_js[copy]
+
+    if 'user-data' in meta_js:
+        ud = meta_js['user-data']
+
+    return(md, ud)
+
+datasources = (
+  (DataSourceConfigDrive, (DataSource.DEP_FILESYSTEM, )),
+  (DataSourceConfigDriveNet,
+    (DataSource.DEP_FILESYSTEM, DataSource.DEP_NETWORK)),
+)
+
+
+# return a list of data sources that match this set of dependencies
+def get_datasource_list(depends):
+    return(DataSource.list_from_depends(depends, datasources))
+
+if __name__ == "__main__":
+    def main():
+        import sys
+        import pprint
+        print cfg_drive_device()
+        (md, ud) = read_config_drive_dir(sys.argv[1])
+        print "=== md ==="
+        pprint.pprint(md)
+        print "=== ud ==="
+        print(ud)
+
+    main()
+
+# vi: ts=4 expandtab
 
=== modified file 'cloudinit/DataSourceEc2.py'
--- cloudinit/DataSourceEc2.py	2012-01-18 14:07:33 +0000
+++ cloudinit/DataSourceEc2.py	2012-06-13 13:16:19 +0000
@@ -24,7 +24,6 @@
 from cloudinit import log
 import cloudinit.util as util
 import socket
-import urllib2
 import time
 import boto.utils as boto_utils
 import os.path
@@ -134,8 +133,8 @@
             url2base[cur] = url
 
         starttime = time.time()
-        url = wait_for_metadata_service(urls=urls, max_wait=max_wait,
-                  timeout=timeout, status_cb=log.warn)
+        url = util.wait_for_url(urls=urls, max_wait=max_wait,
+                                timeout=timeout, status_cb=log.warn)
 
         if url:
             log.debug("Using metadata source: '%s'" % url2base[url])
@@ -208,87 +207,6 @@
         return False
 
 
-def wait_for_metadata_service(urls, max_wait=None, timeout=None,
-                              status_cb=None):
-    """
-    urls:      a list of urls to try
-    max_wait:  roughly the maximum time to wait before giving up
-               The max time is *actually* len(urls)*timeout as each url will
-               be tried once and given the timeout provided.
-    timeout:   the timeout provided to urllib2.urlopen
-    status_cb: call method with string message when a url is not available
-
-    the idea of this routine is to wait for the EC2 metdata service to
-    come up.  On both Eucalyptus and EC2 we have seen the case where
-    the instance hit the MD before the MD service was up.  EC2 seems
-    to have permenantely fixed this, though.
-
-    In openstack, the metadata service might be painfully slow, and
-    unable to avoid hitting a timeout of even up to 10 seconds or more
-    (LP: #894279) for a simple GET.
-
-    Offset those needs with the need to not hang forever (and block boot)
-    on a system where cloud-init is configured to look for EC2 Metadata
-    service but is not going to find one.  It is possible that the instance
-    data host (169.254.169.254) may be firewalled off Entirely for a sytem,
-    meaning that the connection will block forever unless a timeout is set.
-    """
-    starttime = time.time()
-
-    sleeptime = 1
-
-    def nullstatus_cb(msg):
-        return
-
-    if status_cb == None:
-        status_cb = nullstatus_cb
-
-    def timeup(max_wait, starttime):
-        return((max_wait <= 0 or max_wait == None) or
-               (time.time() - starttime > max_wait))
-
-    loop_n = 0
-    while True:
-        sleeptime = int(loop_n / 5) + 1
-        for url in urls:
-            now = time.time()
-            if loop_n != 0:
-                if timeup(max_wait, starttime):
-                    break
-                if timeout and (now + timeout > (starttime + max_wait)):
-                    # shorten timeout to not run way over max_time
-                    timeout = int((starttime + max_wait) - now)
-
-            reason = ""
-            try:
-                req = urllib2.Request(url)
-                resp = urllib2.urlopen(req, timeout=timeout)
-                if resp.read() != "":
-                    return url
-                reason = "empty data [%s]" % resp.getcode()
-            except urllib2.HTTPError as e:
-                reason = "http error [%s]" % e.code
-            except urllib2.URLError as e:
-                reason = "url error [%s]" % e.reason
-            except socket.timeout as e:
-                reason = "socket timeout [%s]" % e
-            except Exception as e:
-                reason = "unexpected error [%s]" % e
-
-            if log:
-                status_cb("'%s' failed [%s/%ss]: %s" %
-                          (url, int(time.time() - starttime), max_wait,
-                           reason))
-
-        if timeup(max_wait, starttime):
-            break
-
-        loop_n = loop_n + 1
-        time.sleep(sleeptime)
-
-    return False
-
-
 datasources = [
   (DataSourceEc2, (DataSource.DEP_FILESYSTEM, DataSource.DEP_NETWORK)),
 ]
 
=== added file 'cloudinit/DataSourceMAAS.py'
--- cloudinit/DataSourceMAAS.py	1970-01-01 00:00:00 +0000
+++ cloudinit/DataSourceMAAS.py	2012-06-13 13:16:19 +0000
@@ -0,0 +1,345 @@
+# vi: ts=4 expandtab
+#
+#    Copyright (C) 2012 Canonical Ltd.
+#
+#    Author: Scott Moser <[e-mail]>
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU General Public License version 3, as
+#    published by the Free Software Foundation.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+#
+#    You should have received a copy of the GNU General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import cloudinit.DataSource as DataSource
+
+from cloudinit import seeddir as base_seeddir
+from cloudinit import log
+import cloudinit.util as util
+import errno
+import oauth.oauth as oauth
+import os.path
+import urllib2
+import time
+
+
+MD_VERSION = "2012-03-01"
+
+
+class DataSourceMAAS(DataSource.DataSource):
+    """
+    DataSourceMAAS reads instance information from MAAS.
+    Given a config metadata_url, and oauth tokens, it expects to find
+    files under the root named:
+      instance-id
+      user-data
+      hostname
+    """
+    seeddir = base_seeddir + '/maas'
+    baseurl = None
+
+    def __str__(self):
+        return("DataSourceMAAS[%s]" % self.baseurl)
+
+    def get_data(self):
+        mcfg = self.ds_cfg
+
+        try:
+            (userdata, metadata) = read_maas_seed_dir(self.seeddir)
+            self.userdata_raw = userdata
+            self.metadata = metadata
+            self.baseurl = self.seeddir
+            return True
+        except MAASSeedDirNone:
+            pass
+        except MAASSeedDirMalformed as exc:
+            log.warn("%s was malformed: %s\n" % (self.seeddir, exc))
+            raise
+
+        try:
+            # if there is no metadata_url, then we're not configured
+            url = mcfg.get('metadata_url', None)
+            if url == None:
+                return False
+
+            if not self.wait_for_metadata_service(url):
+                return False
+
+            self.baseurl = url
+
+            (userdata, metadata) = read_maas_seed_url(self.baseurl,
+                self.md_headers)
+            self.userdata_raw = userdata
+            self.metadata = metadata
+            return True
+        except Exception:
+            util.logexc(log)
+            return False
+
+    def md_headers(self, url):
+        mcfg = self.ds_cfg
+
+        # if we are missing token_key, token_secret or consumer_key
+        # then just do non-authed requests
+        for required in ('token_key', 'token_secret', 'consumer_key'):
+            if required not in mcfg:
+                return({})
+
+        consumer_secret = mcfg.get('consumer_secret', "")
+
+        return(oauth_headers(url=url, consumer_key=mcfg['consumer_key'],
+            token_key=mcfg['token_key'], token_secret=mcfg['token_secret'],
+            consumer_secret=consumer_secret))
+
+    def wait_for_metadata_service(self, url):
+        mcfg = self.ds_cfg
+
+        max_wait = 120
+        try:
+            max_wait = int(mcfg.get("max_wait", max_wait))
+        except Exception:
+            util.logexc(log)
+            log.warn("Failed to get max wait. using %s" % max_wait)
+
+        if max_wait == 0:
+            return False
+
+        timeout = 50
+        try:
+            timeout = int(mcfg.get("timeout", timeout))
+        except Exception:
+            util.logexc(log)
+            log.warn("Failed to get timeout, using %s" % timeout)
+
+        starttime = time.time()
+        check_url = "%s/%s/meta-data/instance-id" % (url, MD_VERSION)
+        url = util.wait_for_url(urls=[check_url], max_wait=max_wait,
+            timeout=timeout, status_cb=log.warn,
+            headers_cb=self.md_headers)
+
+        if url:
+            log.debug("Using metadata source: '%s'" % url)
+        else:
+            log.critical("giving up on md after %i seconds\n" %
+                         int(time.time() - starttime))
+
+        return (bool(url))
+
+
+def read_maas_seed_dir(seed_d):
+    """
+    Return user-data and metadata for a maas seed dir in seed_d.
+    Expected format of seed_d are the following files:
+      * instance-id
+      * local-hostname
+      * user-data
+    """
+    files = ('local-hostname', 'instance-id', 'user-data', 'public-keys')
+    md = {}
+
+    if not os.path.isdir(seed_d):
+        raise MAASSeedDirNone("%s: not a directory")
+
+    for fname in files:
+        try:
+            with open(os.path.join(seed_d, fname)) as fp:
+                md[fname] = fp.read()
+                fp.close()
+        except IOError as e:
+            if e.errno != errno.ENOENT:
+                raise
+
+    return(check_seed_contents(md, seed_d))
+
+
+def read_maas_seed_url(seed_url, header_cb=None, timeout=None,
+    version=MD_VERSION):
+    """
+    Read the maas datasource at seed_url.
+    header_cb is a method that should return a headers dictionary that will
+    be given to urllib2.Request()
+
+    Expected format of seed_url is are the following files:
+      * <seed_url>/<version>/meta-data/instance-id
+      * <seed_url>/<version>/meta-data/local-hostname
+      * <seed_url>/<version>/user-data
+    """
+    files = ('meta-data/local-hostname',
+             'meta-data/instance-id',
+             'meta-data/public-keys',
+             'user-data')
+
+    base_url = "%s/%s" % (seed_url, version)
+    md = {}
+    for fname in files:
+        url = "%s/%s" % (base_url, fname)
+        if header_cb:
+            headers = header_cb(url)
+        else:
+            headers = {}
+
+        try:
+            req = urllib2.Request(url, data=None, headers=headers)
+            resp = urllib2.urlopen(req, timeout=timeout)
+            md[os.path.basename(fname)] = resp.read()
+        except urllib2.HTTPError as e:
+            if e.code != 404:
+                raise
+
+    return(check_seed_contents(md, seed_url))
+
+
+def check_seed_contents(content, seed):
+    """Validate if content is Is the content a dict that is valid as a
+       return for a datasource.
+       Either return a (userdata, metadata) tuple or
+       Raise MAASSeedDirMalformed or MAASSeedDirNone
+    """
+    md_required = ('instance-id', 'local-hostname')
+    found = content.keys()
+
+    if len(content) == 0:
+        raise MAASSeedDirNone("%s: no data files found" % seed)
+
+    missing = [k for k in md_required if k not in found]
+    if len(missing):
+        raise MAASSeedDirMalformed("%s: missing files %s" % (seed, missing))
+
+    userdata = content.get('user-data', "")
+    md = {}
+    for (key, val) in content.iteritems():
+        if key == 'user-data':
+            continue
+        md[key] = val
+
+    return(userdata, md)
+
+
+def oauth_headers(url, consumer_key, token_key, token_secret, consumer_secret):
+    consumer = oauth.OAuthConsumer(consumer_key, consumer_secret)
+    token = oauth.OAuthToken(token_key, token_secret)
+    params = {
+        'oauth_version': "1.0",
+        'oauth_nonce': oauth.generate_nonce(),
+        'oauth_timestamp': int(time.time()),
+        'oauth_token': token.key,
+        'oauth_consumer_key': consumer.key,
+    }
+    req = oauth.OAuthRequest(http_url=url, parameters=params)
+    req.sign_request(oauth.OAuthSignatureMethod_PLAINTEXT(),
+        consumer, token)
+    return(req.to_header())
+
+
+class MAASSeedDirNone(Exception):
+    pass
+
+
+class MAASSeedDirMalformed(Exception):
+    pass
+
+
+datasources = [
+  (DataSourceMAAS, (DataSource.DEP_FILESYSTEM, DataSource.DEP_NETWORK)),
+]
+
+
+# return a list of data sources that match this set of dependencies
+def get_datasource_list(depends):
+    return(DataSource.list_from_depends(depends, datasources))
+
+
+if __name__ == "__main__":
+    def main():
+        """
+        Call with single argument of directory or http or https url.
+        If url is given additional arguments are allowed, which will be
+        interpreted as consumer_key, token_key, token_secret, consumer_secret
+        """
+        import argparse
+        import pprint
+
+        parser = argparse.ArgumentParser(description='Interact with MAAS DS')
+        parser.add_argument("--config", metavar="file",
+            help="specify DS config file", default=None)
+        parser.add_argument("--ckey", metavar="key",
+            help="the consumer key to auth with", default=None)
+        parser.add_argument("--tkey", metavar="key",
+            help="the token key to auth with", default=None)
+        parser.add_argument("--csec", metavar="secret",
+            help="the consumer secret (likely '')", default="")
+        parser.add_argument("--tsec", metavar="secret",
+            help="the token secret to auth with", default=None)
+        parser.add_argument("--apiver", metavar="version",
+            help="the apiver to use ("" can be used)", default=MD_VERSION)
+
+        subcmds = parser.add_subparsers(title="subcommands", dest="subcmd")
+        subcmds.add_parser('crawl', help="crawl the datasource")
+        subcmds.add_parser('get', help="do a single GET of provided url")
+        subcmds.add_parser('check-seed', help="read andn verify seed at url")
+
+        parser.add_argument("url", help="the data source to query")
+
+        args = parser.parse_args()
+
+        creds = {'consumer_key': args.ckey, 'token_key': args.tkey,
+            'token_secret': args.tsec, 'consumer_secret': args.csec}
+
+        if args.config:
+            import yaml
+            with open(args.config) as fp:
+                cfg = yaml.load(fp)
+            if 'datasource' in cfg:
+                cfg = cfg['datasource']['MAAS']
+            for key in creds.keys():
+                if key in cfg and creds[key] == None:
+                    creds[key] = cfg[key]
+
+        def geturl(url, headers_cb):
+            req = urllib2.Request(url, data=None, headers=headers_cb(url))
+            return(urllib2.urlopen(req).read())
+
+        def printurl(url, headers_cb):
+            print "== %s ==\n%s\n" % (url, geturl(url, headers_cb))
+
+        def crawl(url, headers_cb=None):
+            if url.endswith("/"):
+                for line in geturl(url, headers_cb).splitlines():
+                    if line.endswith("/"):
+                        crawl("%s%s" % (url, line), headers_cb)
+                    else:
+                        printurl("%s%s" % (url, line), headers_cb)
+            else:
+                printurl(url, headers_cb)
+
+        def my_headers(url):
+            headers = {}
+            if creds.get('consumer_key', None) != None:
+                headers = oauth_headers(url, **creds)
+            return headers
+
+        if args.subcmd == "check-seed":
+            if args.url.startswith("http"):
+                (userdata, metadata) = read_maas_seed_url(args.url,
+                    header_cb=my_headers, version=args.apiver)
+            else:
+                (userdata, metadata) = read_maas_seed_url(args.url)
+            print "=== userdata ==="
+            print userdata
+            print "=== metadata ==="
+            pprint.pprint(metadata)
+
+        elif args.subcmd == "get":
+            printurl(args.url, my_headers)
+
+        elif args.subcmd == "crawl":
+            if not args.url.endswith("/"):
+                args.url = "%s/" % args.url
+            crawl(args.url, my_headers)
+
+    main()
 
=== modified file 'cloudinit/DataSourceNoCloud.py'
--- cloudinit/DataSourceNoCloud.py	2012-01-18 14:07:33 +0000
+++ cloudinit/DataSourceNoCloud.py	2012-06-13 13:16:19 +0000
@@ -23,6 +23,8 @@
 from cloudinit import seeddir as base_seeddir
 from cloudinit import log
 import cloudinit.util as util
+import errno
+import subprocess
 
 
 class DataSourceNoCloud(DataSource.DataSource):
@@ -30,6 +32,7 @@
     userdata = None
     userdata_raw = None
     supported_seed_starts = ("/", "file://")
+    dsmode = "local"
     seed = None
     cmdline_id = "ds=nocloud"
     seeddir = base_seeddir + '/nocloud'
@@ -41,7 +44,7 @@
 
     def get_data(self):
         defaults = {
-            "instance-id": "nocloud"
+            "instance-id": "nocloud", "dsmode": self.dsmode
         }
 
         found = []
@@ -64,13 +67,54 @@
             found.append(self.seeddir)
             log.debug("using seeded cache data in %s" % self.seeddir)
 
+        # if the datasource config had a 'seedfrom' entry, then that takes
+        # precedence over a 'seedfrom' that was found in a filesystem
+        # but not over external medi
+        if 'seedfrom' in self.ds_cfg and self.ds_cfg['seedfrom']:
+            found.append("ds_config")
+            md["seedfrom"] = self.ds_cfg['seedfrom']
+
+        fslist = util.find_devs_with("TYPE=vfat")
+        fslist.extend(util.find_devs_with("TYPE=iso9660"))
+
+        label_list = util.find_devs_with("LABEL=cidata")
+        devlist = list(set(fslist) & set(label_list))
+        devlist.sort(reverse=True)
+
+        for dev in devlist:
+            try:
+                (newmd, newud) = util.mount_callback_umount(dev,
+                    util.read_seeded)
+                md = util.mergedict(newmd, md)
+                ud = newud
+
+                # for seed from a device, the default mode is 'net'.
+                # that is more likely to be what is desired.
+                # If they want dsmode of local, then they must
+                # specify that.
+                if 'dsmode' not in md:
+                    md['dsmode'] = "net"
+
+                log.debug("using data from %s" % dev)
+                found.append(dev)
+                break
+            except OSError, e:
+                if e.errno != errno.ENOENT:
+                    raise
+            except util.mountFailedError:
+                log.warn("Failed to mount %s when looking for seed" % dev)
+
         # there was no indication on kernel cmdline or data
         # in the seeddir suggesting this handler should be used.
         if len(found) == 0:
             return False
 
+        seeded_interfaces = None
+
         # the special argument "seedfrom" indicates we should
         # attempt to seed the userdata / metadata from its value
+        # its primarily value is in allowing the user to type less
+        # on the command line, ie: ds=nocloud;s=http://bit.ly/abcdefg
         if "seedfrom" in md:
             seedfrom = md["seedfrom"]
             seedfound = False
@@ -83,6 +127,9 @@
                     (seedfrom, self.__class__))
                 return False
 
+            if 'network-interfaces' in md:
+                seeded_interfaces = self.dsmode
+
             # this could throw errors, but the user told us to do it
             # so if errors are raised, let them raise
             (md_seed, ud) = util.read_seeded(seedfrom, timeout=None)
@@ -93,10 +140,35 @@
             found.append(seedfrom)
 
         md = util.mergedict(md, defaults)
+
+        # update the network-interfaces if metadata had 'network-interfaces'
+        # entry and this is the local datasource, or 'seedfrom' was used
+        # and the source of the seed was self.dsmode
+        # ('local' for NoCloud, 'net' for NoCloudNet')
+        if ('network-interfaces' in md and
+            (self.dsmode in ("local", seeded_interfaces))):
+            log.info("updating network interfaces from nocloud")
+
+            util.write_file("/etc/network/interfaces",
+                md['network-interfaces'])
+            try:
+                (out, err) = util.subp(['ifup', '--all'])
+                if len(out) or len(err):
+                    log.warn("ifup --all had stderr: %s" % err)
+
+            except subprocess.CalledProcessError as exc:
+                log.warn("ifup --all failed: %s" % (exc.output[1]))
+
         self.seed = ",".join(found)
         self.metadata = md
         self.userdata_raw = ud
-        return True
+
+        if md['dsmode'] == self.dsmode:
+            return True
+
+        log.debug("%s: not claiming datasource, dsmode=%s" %
+            (self, md['dsmode']))
+        return False
 
 
 # returns true or false indicating if cmdline indicated
@@ -145,6 +217,7 @@
     cmdline_id = "ds=nocloud-net"
     supported_seed_starts = ("http://", "https://", "ftp://")
     seeddir = base_seeddir + '/nocloud-net'
+    dsmode = "net"
 
 
 datasources = (
 
=== modified file 'cloudinit/DataSourceOVF.py'
--- cloudinit/DataSourceOVF.py	2012-01-18 14:07:33 +0000
+++ cloudinit/DataSourceOVF.py	2012-06-13 13:16:19 +0000
@@ -162,7 +162,7 @@
 
 # transport functions take no input and return
 # a 3 tuple of content, path, filename
-def transport_iso9660(require_iso=False):
+def transport_iso9660(require_iso=True):
 
     # default_regex matches values in
     # /lib/udev/rules.d/60-cdrom_id.rules
 
=== modified file 'cloudinit/SshUtil.py'
--- cloudinit/SshUtil.py	2012-01-18 14:07:33 +0000
+++ cloudinit/SshUtil.py	2012-06-13 13:16:19 +0000
@@ -155,6 +155,8 @@
         akeys = ssh_cfg.get("AuthorizedKeysFile", "%h/.ssh/authorized_keys")
         akeys = akeys.replace("%h", pwent.pw_dir)
         akeys = akeys.replace("%u", user)
+        if not akeys.startswith('/'):
+            akeys = os.path.join(pwent.pw_dir, akeys)
         authorized_keys = akeys
     except Exception:
         authorized_keys = '%s/.ssh/authorized_keys' % pwent.pw_dir
 
=== modified file 'cloudinit/UserDataHandler.py'
--- cloudinit/UserDataHandler.py	2012-01-30 14:24:41 +0000
+++ cloudinit/UserDataHandler.py	2012-06-13 13:16:19 +0000
@@ -180,7 +180,7 @@
 
         payload = part.get_payload(decode=True)
 
-        if ctype_orig == "text/plain":
+        if ctype_orig in ("text/plain", "text/x-not-multipart"):
             ctype = type_from_startswith(payload)
 
         if ctype is None:
@@ -213,7 +213,7 @@
             else:
                 msg[key] = val
     else:
-        mtype = headers.get("Content-Type", "text/plain")
+        mtype = headers.get("Content-Type", "text/x-not-multipart")
         maintype, subtype = mtype.split("/", 1)
         msg = MIMEBase(maintype, subtype, *headers)
         msg.set_payload(data)
 
=== modified file 'cloudinit/__init__.py'
--- cloudinit/__init__.py	2012-01-18 14:07:33 +0000
+++ cloudinit/__init__.py	2012-06-13 13:16:19 +0000
@@ -29,7 +29,7 @@
 
 cfg_builtin = """
 log_cfgs: []
-datasource_list: ["NoCloud", "OVF", "Ec2"]
+datasource_list: ["NoCloud", "ConfigDrive", "OVF", "MAAS", "Ec2", "CloudStack"]
 def_log_file: /var/log/cloud-init.log
 syslog_fix_perms: syslog:adm
 """
@@ -60,7 +60,6 @@
 import sys
 import os.path
 import errno
-import pwd
 import subprocess
 import yaml
 import logging
@@ -138,7 +137,9 @@
 
         if ds_deps != None:
             self.ds_deps = ds_deps
+
         self.sysconfig = sysconfig
+
         self.cfg = self.read_cfg()
 
     def read_cfg(self):
@@ -572,10 +573,14 @@
     if not (modfreq == per_always or
             (frequency == per_instance and modfreq == per_instance)):
         return
-    if mod.handler_version == 1:
-        mod.handle_part(data, ctype, filename, payload)
-    else:
-        mod.handle_part(data, ctype, filename, payload, frequency)
+    try:
+        if mod.handler_version == 1:
+            mod.handle_part(data, ctype, filename, payload)
+        else:
+            mod.handle_part(data, ctype, filename, payload, frequency)
+    except:
+        util.logexc(log)
+        traceback.print_exc(file=sys.stderr)
 
 
 def partwalker_handle_handler(pdata, _ctype, _filename, payload):
@@ -586,15 +591,13 @@
     modfname = modname + ".py"
     util.write_file("%s/%s" % (pdata['handlerdir'], modfname), payload, 0600)
 
-    pdata['handlercount'] = curcount + 1
-
     try:
         mod = __import__(modname)
         handler_register(mod, pdata['handlers'], pdata['data'], frequency)
+        pdata['handlercount'] = curcount + 1
     except:
         util.logexc(log)
         traceback.print_exc(file=sys.stderr)
-        return
 
 
 def partwalker_callback(pdata, ctype, filename, payload):
@@ -605,6 +608,14 @@
         partwalker_handle_handler(pdata, ctype, filename, payload)
         return
     if ctype not in pdata['handlers']:
+        if ctype == "text/x-not-multipart":
+            # Extract the first line or 24 bytes for displaying in the log
+            start = payload.split("\n", 1)[0][:24]
+            if start < payload:
+                details = "starting '%s...'" % start.encode("string-escape")
+            else:
+                details = repr(payload)
+            log.warning("Unhandled non-multipart userdata %s", details)
         return
     handler_handle_part(pdata['handlers'][ctype], pdata['data'],
         ctype, filename, payload, pdata['frequency'])
@@ -630,3 +641,27 @@
 
     def handle_part(self, data, ctype, filename, payload, frequency):
         return(self.handler(data, ctype, filename, payload, frequency))
+
+
+def get_cmdline_url(names=('cloud-config-url', 'url'),
+                    starts="#cloud-config", cmdline=None):
+
+    if cmdline == None:
+        cmdline = util.get_cmdline()
+
+    data = util.keyval_str_to_dict(cmdline)
+    url = None
+    key = None
+    for key in names:
+        if key in data:
+            url = data[key]
+            break
+    if url == None:
+        return (None, None, None)
+
+    contents = util.readurl(url)
+
+    if contents.startswith(starts):
+        return (key, url, contents)
+
+    return (key, url, None)
 
=== modified file 'cloudinit/netinfo.py'
--- cloudinit/netinfo.py	2012-01-30 14:24:30 +0000
+++ cloudinit/netinfo.py	2012-06-13 13:16:19 +0000
@@ -19,14 +19,14 @@
 #    You should have received a copy of the GNU General Public License
 #    along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-import subprocess
+import cloudinit.util as util
 
 
 def netdev_info(empty=""):
     fields = ("hwaddr", "addr", "bcast", "mask")
-    ifcfg_out = str(subprocess.check_output(["ifconfig", "-a"]))
+    (ifcfg_out, _err) = util.subp(["ifconfig", "-a"])
     devs = {}
-    for line in ifcfg_out.splitlines():
+    for line in str(ifcfg_out).splitlines():
         if len(line) == 0:
             continue
         if line[0] not in ("\t", " "):
@@ -70,9 +70,9 @@
 
 
 def route_info():
-    route_out = str(subprocess.check_output(["route", "-n"]))
+    (route_out, _err) = util.subp(["route", "-n"])
     routes = []
-    for line in route_out.splitlines()[1:]:
+    for line in str(route_out).splitlines()[1:]:
         if not line:
             continue
         toks = line.split()
 
=== modified file 'cloudinit/util.py'
--- cloudinit/util.py	2012-01-18 14:07:33 +0000
+++ cloudinit/util.py	2012-06-13 13:16:19 +0000
@@ -32,6 +32,7 @@
 import socket
 import sys
 import time
+import tempfile
 import traceback
 import urlparse
 
@@ -208,16 +209,18 @@
     if skip_no_exist and not os.path.isdir(dirp):
         return
 
-    # per bug 857926, Fedora's run-parts will exit failure on empty dir
-    if os.path.isdir(dirp) and os.listdir(dirp) == []:
-        return
-
-    cmd = ['run-parts', '--regex', '.*', dirp]
-    sp = subprocess.Popen(cmd)
-    sp.communicate()
-    if sp.returncode is not 0:
-        raise subprocess.CalledProcessError(sp.returncode, cmd)
-    return
+    failed = 0
+    for exe_name in sorted(os.listdir(dirp)):
+        exe_path = os.path.join(dirp, exe_name)
+        if os.path.isfile(exe_path) and os.access(exe_path, os.X_OK):
+            popen = subprocess.Popen([exe_path])
+            popen.communicate()
+            if popen.returncode is not 0:
+                failed += 1
+                sys.stderr.write("failed: %s [%i]\n" %
+                    (exe_path, popen.returncode))
+    if failed:
+        raise RuntimeError('runparts: %i failures' % failed)
 
 
 def subp(args, input_=None):
@@ -515,30 +518,70 @@
     return(string.replace('\r\n', '\n'))
 
 
-def islxc():
-    # is this host running lxc?
-    try:
-        with open("/proc/1/cgroup") as f:
-            if f.read() == "/":
-                return True
-    except IOError as e:
-        if e.errno != errno.ENOENT:
-            raise
-
-    try:
-        # try to run a program named 'lxc-is-container'. if it returns true,
-        # then we're inside a container. otherwise, no
-        sp = subprocess.Popen(['lxc-is-container'], stdout=subprocess.PIPE,
-                              stderr=subprocess.PIPE)
-        sp.communicate(None)
-        return(sp.returncode == 0)
-    except OSError as e:
-        if e.errno != errno.ENOENT:
-            raise
+def is_container():
+    # is this code running in a container of some sort
+
+    for helper in ('running-in-container', 'lxc-is-container'):
+        try:
+            # try to run a helper program. if it returns true
+            # then we're inside a container. otherwise, no
+            sp = subprocess.Popen(helper, stdout=subprocess.PIPE,
+                                  stderr=subprocess.PIPE)
+            sp.communicate(None)
+            return(sp.returncode == 0)
+        except OSError as e:
+            if e.errno != errno.ENOENT:
+                raise
+
+    # this code is largely from the logic in
+    # ubuntu's /etc/init/container-detect.conf
+    try:
+        # Detect old-style libvirt
+        # Detect OpenVZ containers
+        pid1env = get_proc_env(1)
+        if "container" in pid1env:
+            return True
+
+        if "LIBVIRT_LXC_UUID" in pid1env:
+            return True
+
+    except IOError as e:
+        if e.errno != errno.ENOENT:
+            pass
+
+    # Detect OpenVZ containers
+    if os.path.isdir("/proc/vz") and not os.path.isdir("/proc/bc"):
+        return True
+
+    try:
+        # Detect Vserver containers
+        with open("/proc/self/status") as fp:
+            lines = fp.read().splitlines()
+            for line in lines:
+                if line.startswith("VxID:"):
+                    (_key, val) = line.strip().split(":", 1)
+                    if val != "0":
+                        return True
+    except IOError as e:
+        if e.errno != errno.ENOENT:
+            pass
 
     return False
 
 
+def get_proc_env(pid):
+    # return the environment in a dict that a given process id was started with
+    env = {}
+    with open("/proc/%s/environ" % pid) as fp:
+        toks = fp.read().split("\0")
+        for tok in toks:
+            if tok == "":
+                continue
+            (name, val) = tok.split("=", 1)
+            env[name] = val
+    return env
+
+
 def get_hostname_fqdn(cfg, cloud):
     # return the hostname and fqdn from 'cfg'.  If not found in cfg,
     # then fall back to data from cloud
@@ -630,3 +673,183 @@
         return
     with open(os.devnull) as fp:
         os.dup2(fp.fileno(), sys.stdin.fileno())
+
+
+def find_devs_with(criteria):
+    """
+    find devices matching given criteria (via blkid)
+    criteria can be *one* of:
+      TYPE=<filesystem>
+      LABEL=<label>
+      UUID=<uuid>
+    """
+    try:
+        (out, _err) = subp(['blkid', '-t%s' % criteria, '-odevice'])
+    except subprocess.CalledProcessError:
+        return([])
+    return(str(out).splitlines())
+
+
+class mountFailedError(Exception):
+    pass
+
+
+def mount_callback_umount(device, callback, data=None):
+    """
+    mount the device, call method 'callback' passing the directory
+    in which it was mounted, then unmount.  Return whatever 'callback'
+    returned.  If data != None, also pass data to callback.
+    """
+
+    def _cleanup(umount, tmpd):
+        if umount:
+            try:
+                subp(["umount", '-l', umount])
+            except subprocess.CalledProcessError:
+                raise
+        if tmpd:
+            os.rmdir(tmpd)
+
+    # go through mounts to see if it was already mounted
+    fp = open("/proc/mounts")
+    mounts = fp.readlines()
+    fp.close()
+
+    tmpd = None
+
+    mounted = {}
+    for mpline in mounts:
+        (dev, mp, fstype, _opts, _freq, _passno) = mpline.split()
+        mp = mp.replace("\\040", " ")
+        mounted[dev] = (dev, fstype, mp, False)
+
+    umount = False
+    if device in mounted:
+        mountpoint = "%s/" % mounted[device][2]
+    else:
+        tmpd = tempfile.mkdtemp()
+
+        mountcmd = ["mount", "-o", "ro", device, tmpd]
+
+        try:
+            (_out, _err) = subp(mountcmd)
+            umount = tmpd
+        except subprocess.CalledProcessError as exc:
+            _cleanup(umount, tmpd)
+            raise mountFailedError(exc.output[1])
+
+        mountpoint = "%s/" % tmpd
+
+    try:
+        if data == None:
+            ret = callback(mountpoint)
+        else:
+            ret = callback(mountpoint, data)
+
+    except Exception as exc:
+        _cleanup(umount, tmpd)
+        raise exc
+
+    _cleanup(umount, tmpd)
+
+    return(ret)
+
+
+def wait_for_url(urls, max_wait=None, timeout=None,
+                 status_cb=None, headers_cb=None):
+    """
+    urls:      a list of urls to try
+    max_wait:  roughly the maximum time to wait before giving up
+               The max time is *actually* len(urls)*timeout as each url will
+               be tried once and given the timeout provided.
+    timeout:   the timeout provided to urllib2.urlopen
+    status_cb: call method with string message when a url is not available
+    headers_cb: call method with single argument of url to get headers
+                for request.
+
+    the idea of this routine is to wait for the EC2 metdata service to
+    come up.  On both Eucalyptus and EC2 we have seen the case where
+    the instance hit the MD before the MD service was up.  EC2 seems
+    to have permenantely fixed this, though.
+
+    In openstack, the metadata service might be painfully slow, and
+    unable to avoid hitting a timeout of even up to 10 seconds or more
+    (LP: #894279) for a simple GET.
+
+    Offset those needs with the need to not hang forever (and block boot)
+    on a system where cloud-init is configured to look for EC2 Metadata
+    service but is not going to find one.  It is possible that the instance
+    data host (169.254.169.254) may be firewalled off Entirely for a sytem,
+    meaning that the connection will block forever unless a timeout is set.
+    """
+    starttime = time.time()
+
+    sleeptime = 1
+
+    def nullstatus_cb(msg):
+        return
+
+    if status_cb == None:
+        status_cb = nullstatus_cb
+
+    def timeup(max_wait, starttime):
+        return((max_wait <= 0 or max_wait == None) or
+               (time.time() - starttime > max_wait))
+
+    loop_n = 0
+    while True:
+        sleeptime = int(loop_n / 5) + 1
+        for url in urls:
+            now = time.time()
+            if loop_n != 0:
+                if timeup(max_wait, starttime):
+                    break
+                if timeout and (now + timeout > (starttime + max_wait)):
+                    # shorten timeout to not run way over max_time
+                    timeout = int((starttime + max_wait) - now)
+
+            reason = ""
+            try:
+                if headers_cb != None:
+                    headers = headers_cb(url)
+                else:
+                    headers = {}
+
+                req = urllib2.Request(url, data=None, headers=headers)
+                resp = urllib2.urlopen(req, timeout=timeout)
+                if resp.read() != "":
+                    return url
+                reason = "empty data [%s]" % resp.getcode()
+            except urllib2.HTTPError as e:
+                reason = "http error [%s]" % e.code
+            except urllib2.URLError as e:
+                reason = "url error [%s]" % e.reason
+            except socket.timeout as e:
+                reason = "socket timeout [%s]" % e
+            except Exception as e:
+                reason = "unexpected error [%s]" % e
+
+            status_cb("'%s' failed [%s/%ss]: %s" %
+                      (url, int(time.time() - starttime), max_wait,
+                       reason))
+
+        if timeup(max_wait, starttime):
+            break
+
+        loop_n = loop_n + 1
+        time.sleep(sleeptime)
+
+    return False
+
+
+def keyval_str_to_dict(kvstring):
+    ret = {}
+    for tok in kvstring.split():
+        try:
+            (key, val) = tok.split("=", 1)
+        except ValueError:
+            key = tok
+            val = True
+        ret[key] = val
+
+    return(ret)
 
=== modified file 'config/cloud.cfg'
--- config/cloud.cfg	2012-01-17 17:46:44 +0000
+++ config/cloud.cfg	2012-06-13 13:16:19 +0000
@@ -1,7 +1,7 @@
 user: ubuntu
 disable_root: 1
 preserve_hostname: False
-# datasource_list: [ "NoCloud", "OVF", "Ec2" ]
+# datasource_list: ["NoCloud", "ConfigDrive", "OVF", "MAAS", "Ec2", "CloudStack"]
 
 cloud_init_modules:
  - bootcmd
@@ -19,11 +19,13 @@
  - locale
  - set-passwords
  - grub-dpkg
+ - apt-pipelining
  - apt-update-upgrade
  - landscape
  - timezone
  - puppet
  - chef
+ - salt-minion
  - mcollective
  - disable-ec2-metadata
  - runcmd
 
=== modified file 'debian.trunk/control'
--- debian.trunk/control	2012-01-12 17:51:48 +0000
+++ debian.trunk/control	2012-06-13 13:16:19 +0000
@@ -20,6 +20,7 @@
          python-boto (>=2.0),
          python-cheetah,
          python-configobj,
+         python-oauth,
          python-software-properties,
          python-yaml,
          ${misc:Depends},
 
=== added directory 'doc/configdrive'
=== added file 'doc/configdrive/README'
--- doc/configdrive/README	1970-01-01 00:00:00 +0000
+++ doc/configdrive/README	2012-06-13 13:16:19 +0000
@@ -0,0 +1,118 @@
+The 'ConfigDrive' DataSource supports the OpenStack configdrive disk.
+See doc/source/api_ext/ext_config_drive.rst in the nova source code for
+more information on config drive.
+
+The following criteria are required to be identified by
+DataSourceConfigDrive as a config drive:
+ * must be formated with vfat filesystem
+ * must be a un-partitioned block device (/dev/vdb, not /dev/vdb1)
+ * must contain one of the following files:
+    * etc/network/interfaces
+    * root/.ssh/authorized_keys
+    * meta.js
+
+By default, cloud-init does not consider this source to be a full-fledged
+datasource.  Instead, the default behavior is to assume it is really only
+present to provide networking information.  Cloud-init will copy off the
+network information, apply it to the system, and then continue on.  The
+"full" datasource would then be found in the EC2 metadata service.
+
+== Content of config-drive ==
+  * etc/network/interfaces
+    This file is laid down by nova in order to pass static networking
+    information to the guest.  Cloud-init will copy it off of the config-drive
+    and into /etc/network/interfaces as soon as it can, and then attempt to
+    bring up all network interfaces.
+
+  * root/.ssh/authorized_keys
+    This file is laid down by nova, and contains the keys that were
+    provided to it on instance creation (nova-boot --key ....)
+
+    Cloud-init will copy those keys and put them into the configured user
+    ('ubuntu') .ssh/authorized_keys.
+
+  * meta.js
+    meta.js is populated on the config-drive in response to the user passing
+    "meta flags" (nova boot --meta key=value ...).  It is expected to be json
+    formated.
+
+== Configuration ==
+Cloud-init's behavior can be modified by keys found in the meta.js file in
+the following ways:
+ * dsmode:  
+   values: local, net, pass
+   default: pass
+
+   This is what indicates if configdrive is a final data source or not.
+   By default it is 'pass', meaning this datasource should not be read.
+   Set it to 'local' or 'net' to stop cloud-init from continuing on to
+   search for other data sources after network config.
+
+   The difference between 'local' and 'net' is that local will not require
+   networking to be up before user-data actions (or boothooks) are run.
+   
+ * instance-id:
+   default: iid-dsconfigdrive
+   This is utilized as the metadata's instance-id.  It should generally
+   be unique, as it is what is used to determine "is this a new instance".
+
+ * public-keys:
+   default: None
+   if present, these keys will be used as the public keys for the
+   instance.  This value overrides the content in authorized_keys.
+   Note: it is likely preferable to provide keys via user-data
+
+ * user-data:
+   default: None
+   This provides cloud-init user-data.  See other documentation for what
+   all can be present here.
+
+== Example ==
+Here is an example using the nova client (python-novaclien)
+
+Assuming the following variables set up:
+ * img_id : set to the nova image id (uuid from image-list)
+ * flav_id : set to numeric flavor_id (nova flavor-list)
+ * keyname : set to name of key for this instance (nova keypair-list)
+
+$ cat my-user-data 
+#!/bin/sh
+echo ==== USER_DATA FROM EC2 MD ==== | tee /ud.log
+
+$ ud_value=$(sed 's,EC2 MD,META KEY,')
+
+## Now, 'ud_value' has same content of my-user-data file, but
+## with the string "USER_DATA FROM META KEY"
+
+## launch an instance with dsmode=pass
+## This will really not use the configdrive for anything as the mode
+## for the datasource is 'pass', meaning it will still expect some
+## other data source (DataSourceEc2).
+
+$ nova boot --image=$img_id --config-drive=1 --flavor=$flav_id \
+   --key_name=$keyname \
+   --user_data=my-user-data \
+   "--meta=instance-id=iid-001 \
+   "--meta=user-data=${ud_keyval}" \
+   "--meta=dsmode=pass" cfgdrive-dsmode-pass
+
+$ euca-get-console-output i-0000001 | grep USER_DATA
+echo ==== USER_DATA FROM EC2 MD ==== | tee /ud.log
+
+## Now, launch an instance with dsmode=local
+## This time, the only metadata and userdata available to cloud-init
+## are on the config-drive
+$ nova boot --image=$img_id --config-drive=1 --flavor=$flav_id \
+   --key_name=$keyname \
+   --user_data=my-user-data \
+   "--meta=instance-id=iid-001 \
+   "--meta=user-data=${ud_keyval}" \
+   "--meta=dsmode=local" cfgdrive-dsmode-local
+
+$ euca-get-console-output i-0000002 | grep USER_DATA
+echo ==== USER_DATA FROM META KEY ==== | tee /ud.log
+
+--
+[1] https://github.com/openstack/nova/blob/master/doc/source/api_ext/ext_config_drive.rst for more if
+
+
 
=== added file 'doc/examples/cloud-config-chef-oneiric.txt'
--- doc/examples/cloud-config-chef-oneiric.txt	1970-01-01 00:00:00 +0000
+++ doc/examples/cloud-config-chef-oneiric.txt	2012-06-13 13:16:19 +0000
@@ -0,0 +1,90 @@
+#cloud-config
+#
+# This is an example file to automatically install chef-client and run a 
+# list of recipes when the instance boots for the first time.
+# Make sure that this file is valid yaml before starting instances.
+# It should be passed as user-data when starting the instance.
+#
+# This example assumes the instance is 11.10 (oneiric)
+
+
+# The default is to install from packages. 
+
+# Key from http://apt.opscode.com/[e-mail]
+apt_sources:
+ - source: "deb http://apt.opscode.com/ $RELEASE-0.10 main"
+   key: |
+     -----BEGIN PGP PUBLIC KEY BLOCK-----
+     Version: GnuPG v1.4.9 (GNU/Linux)
+     
+     mQGiBEppC7QRBADfsOkZU6KZK+YmKw4wev5mjKJEkVGlus+NxW8wItX5sGa6kdUu
+     twAyj7Yr92rF+ICFEP3gGU6+lGo0Nve7KxkN/1W7/m3G4zuk+ccIKmjp8KS3qn99
+     dxy64vcji9jIllVa+XXOGIp0G8GEaj7mbkixL/bMeGfdMlv8Gf2XPpp9vwCgn/GC
+     JKacfnw7MpLKUHOYSlb//JsEAJqao3ViNfav83jJKEkD8cf59Y8xKia5OpZqTK5W
+     ShVnNWS3U5IVQk10ZDH97Qn/YrK387H4CyhLE9mxPXs/ul18ioiaars/q2MEKU2I
+     XKfV21eMLO9LYd6Ny/Kqj8o5WQK2J6+NAhSwvthZcIEphcFignIuobP+B5wNFQpe
+     DbKfA/0WvN2OwFeWRcmmd3Hz7nHTpcnSF+4QX6yHRF/5BgxkG6IqBIACQbzPn6Hm
+     sMtm/SVf11izmDqSsQptCrOZILfLX/mE+YOl+CwWSHhl+YsFts1WOuh1EhQD26aO
+     Z84HuHV5HFRWjDLw9LriltBVQcXbpfSrRP5bdr7Wh8vhqJTPjrQnT3BzY29kZSBQ
+     YWNrYWdlcyA8cGFja2FnZXNAb3BzY29kZS5jb20+iGAEExECACAFAkppC7QCGwMG
+     CwkIBwMCBBUCCAMEFgIDAQIeAQIXgAAKCRApQKupg++Caj8sAKCOXmdG36gWji/K
+     +o+XtBfvdMnFYQCfTCEWxRy2BnzLoBBFCjDSK6sJqCu5Ag0ESmkLtBAIAIO2SwlR
+     lU5i6gTOp42RHWW7/pmW78CwUqJnYqnXROrt3h9F9xrsGkH0Fh1FRtsnncgzIhvh
+     DLQnRHnkXm0ws0jV0PF74ttoUT6BLAUsFi2SPP1zYNJ9H9fhhK/pjijtAcQwdgxu
+     wwNJ5xCEscBZCjhSRXm0d30bK1o49Cow8ZIbHtnXVP41c9QWOzX/LaGZsKQZnaMx
+     EzDk8dyyctR2f03vRSVyTFGgdpUcpbr9eTFVgikCa6ODEBv+0BnCH6yGTXwBid9g
+     w0o1e/2DviKUWCC+AlAUOubLmOIGFBuI4UR+rux9affbHcLIOTiKQXv79lW3P7W8
+     AAfniSQKfPWXrrcAAwUH/2XBqD4Uxhbs25HDUUiM/m6Gnlj6EsStg8n0nMggLhuN
+     QmPfoNByMPUqvA7sULyfr6xCYzbzRNxABHSpf85FzGQ29RF4xsA4vOOU8RDIYQ9X
+     Q8NqqR6pydprRFqWe47hsAN7BoYuhWqTtOLSBmnAnzTR5pURoqcquWYiiEavZixJ
+     3ZRAq/HMGioJEtMFrvsZjGXuzef7f0ytfR1zYeLVWnL9Bd32CueBlI7dhYwkFe+V
+     Ep5jWOCj02C1wHcwt+uIRDJV6TdtbIiBYAdOMPk15+VBdweBXwMuYXr76+A7VeDL
+     zIhi7tKFo6WiwjKZq0dzctsJJjtIfr4K4vbiD9Ojg1iISQQYEQIACQUCSmkLtAIb
+     DAAKCRApQKupg++CauISAJ9CxYPOKhOxalBnVTLeNUkAHGg2gACeIsbobtaD4ZHG
+     0GLl8EkfA8uhluM=
+     =zKAm
+     -----END PGP PUBLIC KEY BLOCK-----
+
+chef:
+
+ # 11.10 will fail if install_type is "gems" (LP: #960576)
+ install_type: "packages"
+
+ # Chef settings
+ server_url: "https://chef.yourorg.com:4000"
+
+ # Node Name
+ # Defaults to the instance-id if not present
+ node_name: "your-node-name"
+
+ # Environment
+ # Defaults to '_default' if not present
+ environment: "production"
+
+ # Default validation name is chef-validator
+ validation_name: "yourorg-validator"
+
+ # value of validation_cert is not used if validation_key defined,
+ # but variable needs to be defined (LP: #960547)
+ validation_cert: "unused"
+ validation_key: |
+     -----BEGIN RSA PRIVATE KEY-----
+     YOUR-ORGS-VALIDATION-KEY-HERE
+     -----END RSA PRIVATE KEY-----
+ 
+ # A run list for a first boot json
+ run_list:
+  - "recipe[apache2]"
+  - "role[db]"
+
+ # Specify a list of initial attributes used by the cookbooks
+ initial_attributes:
+    apache:
+      prefork:
+        maxclients: 100
+      keepalive: "off"
+
+
+# Capture all subprocess output into a logfile
+# Useful for troubleshooting cloud-init issues
+output: {all: '| tee -a /var/log/cloud-init-output.log'}
 
=== modified file 'doc/examples/cloud-config-chef.txt'
--- doc/examples/cloud-config-chef.txt	2012-01-20 20:10:28 +0000
+++ doc/examples/cloud-config-chef.txt	2012-06-13 13:16:19 +0000
@@ -1,17 +1,54 @@
 #cloud-config
 #
-# This is an example file to automatically setup chef and run a list of recipes
-# when the instance boots for the first time.
+# This is an example file to automatically install chef-client and run a 
+# list of recipes when the instance boots for the first time.
 # Make sure that this file is valid yaml before starting instances.
 # It should be passed as user-data when starting the instance.
-
-# The default is to install from packages. If you want the latest packages from Opscode, be sure to add their repo:
-apt_mirror: http://apt.opscode.com/
+#
+# This example assumes the instance is 12.04 (precise)
+
+
+# The default is to install from packages. 
+
+# Key from http://apt.opscode.com/[e-mail]
+apt_sources:
+ - source: "deb http://apt.opscode.com/ $RELEASE-0.10 main"
+   key: |
+     -----BEGIN PGP PUBLIC KEY BLOCK-----
+     Version: GnuPG v1.4.9 (GNU/Linux)
+     
+     mQGiBEppC7QRBADfsOkZU6KZK+YmKw4wev5mjKJEkVGlus+NxW8wItX5sGa6kdUu
+     twAyj7Yr92rF+ICFEP3gGU6+lGo0Nve7KxkN/1W7/m3G4zuk+ccIKmjp8KS3qn99
+     dxy64vcji9jIllVa+XXOGIp0G8GEaj7mbkixL/bMeGfdMlv8Gf2XPpp9vwCgn/GC
+     JKacfnw7MpLKUHOYSlb//JsEAJqao3ViNfav83jJKEkD8cf59Y8xKia5OpZqTK5W
+     ShVnNWS3U5IVQk10ZDH97Qn/YrK387H4CyhLE9mxPXs/ul18ioiaars/q2MEKU2I
+     XKfV21eMLO9LYd6Ny/Kqj8o5WQK2J6+NAhSwvthZcIEphcFignIuobP+B5wNFQpe
+     DbKfA/0WvN2OwFeWRcmmd3Hz7nHTpcnSF+4QX6yHRF/5BgxkG6IqBIACQbzPn6Hm
+     sMtm/SVf11izmDqSsQptCrOZILfLX/mE+YOl+CwWSHhl+YsFts1WOuh1EhQD26aO
+     Z84HuHV5HFRWjDLw9LriltBVQcXbpfSrRP5bdr7Wh8vhqJTPjrQnT3BzY29kZSBQ
+     YWNrYWdlcyA8cGFja2FnZXNAb3BzY29kZS5jb20+iGAEExECACAFAkppC7QCGwMG
+     CwkIBwMCBBUCCAMEFgIDAQIeAQIXgAAKCRApQKupg++Caj8sAKCOXmdG36gWji/K
+     +o+XtBfvdMnFYQCfTCEWxRy2BnzLoBBFCjDSK6sJqCu5Ag0ESmkLtBAIAIO2SwlR
+     lU5i6gTOp42RHWW7/pmW78CwUqJnYqnXROrt3h9F9xrsGkH0Fh1FRtsnncgzIhvh
+     DLQnRHnkXm0ws0jV0PF74ttoUT6BLAUsFi2SPP1zYNJ9H9fhhK/pjijtAcQwdgxu
+     wwNJ5xCEscBZCjhSRXm0d30bK1o49Cow8ZIbHtnXVP41c9QWOzX/LaGZsKQZnaMx
+     EzDk8dyyctR2f03vRSVyTFGgdpUcpbr9eTFVgikCa6ODEBv+0BnCH6yGTXwBid9g
+     w0o1e/2DviKUWCC+AlAUOubLmOIGFBuI4UR+rux9affbHcLIOTiKQXv79lW3P7W8
+     AAfniSQKfPWXrrcAAwUH/2XBqD4Uxhbs25HDUUiM/m6Gnlj6EsStg8n0nMggLhuN
+     QmPfoNByMPUqvA7sULyfr6xCYzbzRNxABHSpf85FzGQ29RF4xsA4vOOU8RDIYQ9X
+     Q8NqqR6pydprRFqWe47hsAN7BoYuhWqTtOLSBmnAnzTR5pURoqcquWYiiEavZixJ
+     3ZRAq/HMGioJEtMFrvsZjGXuzef7f0ytfR1zYeLVWnL9Bd32CueBlI7dhYwkFe+V
+     Ep5jWOCj02C1wHcwt+uIRDJV6TdtbIiBYAdOMPk15+VBdweBXwMuYXr76+A7VeDL
+     zIhi7tKFo6WiwjKZq0dzctsJJjtIfr4K4vbiD9Ojg1iISQQYEQIACQUCSmkLtAIb
+     DAAKCRApQKupg++CauISAJ9CxYPOKhOxalBnVTLeNUkAHGg2gACeIsbobtaD4ZHG
+     0GLl8EkfA8uhluM=
+     =zKAm
+     -----END PGP PUBLIC KEY BLOCK-----
 
 chef:
 
  # Valid values are 'gems' and 'packages'
- install_type: "gems"
+ install_type: "packages"
 
  # Chef settings
  server_url: "https://chef.yourorg.com:4000"
@@ -42,3 +79,8 @@
       prefork:
         maxclients: 100
       keepalive: "off"
+
+
+# Capture all subprocess output into a logfile
+# Useful for troubleshooting cloud-init issues
+output: {all: '| tee -a /var/log/cloud-init-output.log'}
 
=== modified file 'doc/examples/cloud-config-datasources.txt'
--- doc/examples/cloud-config-datasources.txt	2011-12-19 17:00:48 +0000
+++ doc/examples/cloud-config-datasources.txt	2012-06-13 13:16:19 +0000
@@ -13,3 +13,21 @@
     metadata_urls:
      - http://169.254.169.254:80
      - http://instance-data:8773
+
+  MAAS:
+    timeout : 50
+    max_wait : 120
+
+    # there are no default values for metadata_url or oauth credentials
+    # If no credentials are present, non-authed attempts will be made.
+    metadata_url: http://mass-host.localdomain/source
+    consumer_key: Xh234sdkljf
+    token_key: kjfhgb3n
+    token_secret: 24uysdfx1w4
+
+  NoCloud:
+    # default seedfrom is None
+    # if found, then it should contain a url with:
+    #    <url>/user-data and <url>/meta-data
+    # seedfrom: http://my.example.com/i-abcde
+    seedfrom: None
 
=== modified file 'doc/examples/cloud-config.txt'
--- doc/examples/cloud-config.txt	2011-12-20 16:40:51 +0000
+++ doc/examples/cloud-config.txt	2012-06-13 13:16:19 +0000
@@ -45,6 +45,15 @@
 # apt_proxy (configure Acquire::HTTP::Proxy)
 apt_proxy: http://my.apt.proxy:3128
 
+# apt_pipelining (configure Acquire::http::Pipeline-Depth)
+# Default: disables HTTP pipelining. Certain web servers, such
+# as S3 do not pipeline properly (LP: #948461).
+# Valid options:
+#   False/default: Disables pipelining for APT
+#   None/Unchanged: Use OS default
+#   Number: Set pipelining to some number (not recommended)
+apt_pipelining: False
+
 # Preserve existing /etc/apt/sources.list
 # Default: overwrite sources_list with mirror.  If this is true
 # then apt_mirror above will have no effect
@@ -342,6 +351,8 @@
 # this allows you to launch an instance with a larger disk / partition
 # and have the instance automatically grow / to accomoddate it
 # set to 'False' to disable
+# by default, the resizefs is done early in boot, and blocks
+# if resize_rootfs is set to 'noblock', then it will be run in parallel
 resize_rootfs: True
 
 ## hostname and /etc/hosts management
 
=== added file 'doc/kernel-cmdline.txt'
--- doc/kernel-cmdline.txt	1970-01-01 00:00:00 +0000
+++ doc/kernel-cmdline.txt	2012-06-13 13:16:19 +0000
@@ -0,0 +1,48 @@
+In order to allow an ephemeral, or otherwise pristine image to
+receive some configuration, cloud-init will read a url directed by
+the kernel command line and proceed as if its data had previously existed.
+
+This allows for configuring a meta-data service, or some other data.
+
+Note, that usage of the kernel command line is somewhat of a last resort,
+as it requires knowing in advance the correct command line or modifying
+the boot loader to append data.
+
+For example, when 'cloud-init start' runs, it will check to
+see if if one of 'cloud-config-url' or 'url' appear in key/value fashion
+in the kernel command line as in:
+   root=/dev/sda ro url=http://foo.bar.zee/abcde
+
+Cloud-init will then read the contents of the given url. 
+If the content starts with '#cloud-config', it will store
+that data to the local filesystem in a static filename 
+'/etc/cloud/cloud.cfg.d/91_kernel_cmdline_url.cfg', and consider it as
+part of the config from that point forward.
+
+If that file exists already, it will not be overwritten, and the url parameters
+completely ignored.
+
+Then, when the DataSource runs, it will find that config already available.
+
+So, in able to configure the MAAS DataSource by controlling the kernel
+command line from outside the image, you can append:
+   url=http://your.url.here/abcdefg
+or
+   cloud-config-url=http://your.url.here/abcdefg
+
+Then, have the following content at that url:
+   #cloud-config
+   datasource:
+    MAAS:
+      metadata_url: http://mass-host.localdomain/source
+      consumer_key: Xh234sdkljf
+      token_key: kjfhgb3n
+      token_secret: 24uysdfx1w4
+
+Notes:
+ * Because 'url=' is so very generic, in order to avoid false positives,
+   cloud-init requires the content to start with '#cloud-config' in order
+   for it to be considered.
+ * The url= is un-authed http GET, and contains credentials
+   It could be set up to be randomly generated and also check source 
+   address in order to be more secure
 
=== added directory 'doc/nocloud'
=== added file 'doc/nocloud/README'
--- doc/nocloud/README	1970-01-01 00:00:00 +0000
+++ doc/nocloud/README	2012-06-13 13:16:19 +0000
@@ -0,0 +1,55 @@
+The data source 'NoCloud' and 'NoCloudNet' allow the user to provide user-data
+and meta-data to the instance without running a network service (or even without
+having a network at all)
+
+You can provide meta-data and user-data to a local vm boot via files on a vfat
+or iso9660 filesystem.  These user-data and meta-data files are expected to be
+in the format described in doc/example/seed/README .  Basically, user-data is
+simply user-data and meta-data is a yaml formated file representing what you'd
+find in the EC2 metadata service.
+
+Given a disk 12.04 cloud image in 'disk.img', you can create a sufficient disk
+by following the example below.
+
+## create user-data and meta-data files that will be used
+## to modify image on first boot
+$ { echo instance-id: iid-local01; echo local-hostname: cloudimg; } > meta-data
+
+$ printf "#cloud-config\npassword: passw0rd\nchpasswd: { expire: False }\nssh_pwauth: True\n" > user-data
+
+## create a disk to attach with some user-data and meta-data
+$ genisoimage  -output seed.iso -volid cidata -joliet -rock user-data meta-data
+
+## alternatively, create a vfat filesystem with same files
+## $ truncate --size 2M seed.img
+## $ mkfs.vfat -n cidata seed.img
+## $ mcopy -oi seed.img user-data meta-data ::
+
+## create a new qcow image to boot, backed by your original image
+$ qemu-img create -f qcow2 -b disk.img boot-disk.img
+
+## boot the image and login as 'ubuntu' with password 'passw0rd'
+## note, passw0rd was set as password through the user-data above,
+## there is no password set on these images.
+$ kvm -m 256 \
+   -net nic -net user,hostfwd=tcp::2222-:22 \
+   -drive file=boot-disk.img,if=virtio \
+   -drive file=seed.iso,if=virtio
+
+Note, that the instance-id provided ('iid-local01' above) is what is used to
+determine if this is "first boot".  So if you are making updates to user-data
+you will also have to change that, or start the disk fresh.
+
+
+Also, you can inject an /etc/network/interfaces file by providing the content
+for that file in the 'network-interfaces' field of metadata.  Example metadata:
+  instance-id: iid-abcdefg
+  network-interfaces: |
+    iface eth0 inet static
+    address 192.168.1.10
+    network 192.168.1.0
+    netmask 255.255.255.0
+    broadcast 192.168.1.255
+    gateway 192.168.1.254
+  hostname: myhost
+
 
=== modified file 'setup.py'
--- setup.py	2011-12-20 16:39:46 +0000
+++ setup.py	2012-06-13 13:16:19 +0000
@@ -47,5 +47,6 @@
                   ('/usr/share/doc/cloud-init', filter(is_f,glob('doc/*'))),
                   ('/usr/share/doc/cloud-init/examples', filter(is_f,glob('doc/examples/*'))),
                   ('/usr/share/doc/cloud-init/examples/seed', filter(is_f,glob('doc/examples/seed/*'))),
+                  ('/etc/profile.d', ['tools/Z99-cloud-locale-test.sh']),
                   ],
       )
 
=== added file 'tests/unittests/test__init__.py'
--- tests/unittests/test__init__.py	1970-01-01 00:00:00 +0000
+++ tests/unittests/test__init__.py	2012-06-13 13:16:19 +0000
@@ -0,0 +1,242 @@
+from mocker import MockerTestCase, ANY, ARGS, KWARGS
+import os
+
+from cloudinit import (partwalker_handle_handler, handler_handle_part,
+                       handler_register, get_cmdline_url)
+from cloudinit.util import write_file, logexc, readurl
+
+
+class TestPartwalkerHandleHandler(MockerTestCase):
+    def setUp(self):
+        self.data = {
+            "handlercount": 0,
+            "frequency": "?",
+            "handlerdir": "?",
+            "handlers": [],
+            "data": None}
+
+        self.expected_module_name = "part-handler-%03d" % (
+            self.data["handlercount"],)
+        expected_file_name = "%s.py" % self.expected_module_name
+        expected_file_fullname = os.path.join(self.data["handlerdir"],
+                                              expected_file_name)
+        self.module_fake = "fake module handle"
+        self.ctype = None
+        self.filename = None
+        self.payload = "dummy payload"
+
+        # Mock the write_file function
+        write_file_mock = self.mocker.replace(write_file, passthrough=False)
+        write_file_mock(expected_file_fullname, self.payload, 0600)
+
+    def test_no_errors(self):
+        """Payload gets written to file and added to C{pdata}."""
+        # Mock the __import__ builtin
+        import_mock = self.mocker.replace("__builtin__.__import__")
+        import_mock(self.expected_module_name)
+        self.mocker.result(self.module_fake)
+        # Mock the handle_register function
+        handle_reg_mock = self.mocker.replace(handler_register,
+                                              passthrough=False)
+        handle_reg_mock(self.module_fake, self.data["handlers"],
+                        self.data["data"], self.data["frequency"])
+        # Activate mocks
+        self.mocker.replay()
+
+        partwalker_handle_handler(self.data, self.ctype, self.filename,
+                                  self.payload)
+
+        self.assertEqual(1, self.data["handlercount"])
+
+    def test_import_error(self):
+        """Module import errors are logged. No handler added to C{pdata}"""
+        # Mock the __import__ builtin
+        import_mock = self.mocker.replace("__builtin__.__import__")
+        import_mock(self.expected_module_name)
+        self.mocker.throw(ImportError())
+        # Mock log function
+        logexc_mock = self.mocker.replace(logexc, passthrough=False)
+        logexc_mock(ANY)
+        # Mock the print_exc function
+        print_exc_mock = self.mocker.replace("traceback.print_exc",
+                                             passthrough=False)
+        print_exc_mock(ARGS, KWARGS)
+        # Activate mocks
+        self.mocker.replay()
+
+        partwalker_handle_handler(self.data, self.ctype, self.filename,
+                                  self.payload)
+
+        self.assertEqual(0, self.data["handlercount"])
+
+    def test_attribute_error(self):
+        """Attribute errors are logged. No handler added to C{pdata}"""
+        # Mock the __import__ builtin
+        import_mock = self.mocker.replace("__builtin__.__import__")
+        import_mock(self.expected_module_name)
+        self.mocker.result(self.module_fake)
+        # Mock the handle_register function
+        handle_reg_mock = self.mocker.replace(handler_register,
+                                              passthrough=False)
+        handle_reg_mock(self.module_fake, self.data["handlers"],
+                        self.data["data"], self.data["frequency"])
+        self.mocker.throw(AttributeError())
+        # Mock log function
+        logexc_mock = self.mocker.replace(logexc, passthrough=False)
+        logexc_mock(ANY)
+        # Mock the print_exc function
+        print_exc_mock = self.mocker.replace("traceback.print_exc",
+                                             passthrough=False)
+        print_exc_mock(ARGS, KWARGS)
+        # Activate mocks
+        self.mocker.replay()
+
+        partwalker_handle_handler(self.data, self.ctype, self.filename,
+                                  self.payload)
+
+        self.assertEqual(0, self.data["handlercount"])
+
+
+class TestHandlerHandlePart(MockerTestCase):
+    def setUp(self):
+        self.data = "fake data"
+        self.ctype = "fake ctype"
+        self.filename = "fake filename"
+        self.payload = "fake payload"
+        self.frequency = "once-per-instance"
+
+    def test_normal_version_1(self):
+        """
+        C{handle_part} is called without C{frequency} for
+        C{handler_version} == 1.
+        """
+        # Build a mock part-handler module
+        mod_mock = self.mocker.mock()
+        getattr(mod_mock, "frequency")
+        self.mocker.result("once-per-instance")
+        getattr(mod_mock, "handler_version")
+        self.mocker.result(1)
+        mod_mock.handle_part(self.data, self.ctype, self.filename,
+                             self.payload)
+        self.mocker.replay()
+
+        handler_handle_part(mod_mock, self.data, self.ctype, self.filename,
+                            self.payload, self.frequency)
+
+    def test_normal_version_2(self):
+        """
+        C{handle_part} is called with C{frequency} for
+        C{handler_version} == 2.
+        """
+        # Build a mock part-handler module
+        mod_mock = self.mocker.mock()
+        getattr(mod_mock, "frequency")
+        self.mocker.result("once-per-instance")
+        getattr(mod_mock, "handler_version")
+        self.mocker.result(2)
+        mod_mock.handle_part(self.data, self.ctype, self.filename,
+                             self.payload, self.frequency)
+        self.mocker.replay()
+
+        handler_handle_part(mod_mock, self.data, self.ctype, self.filename,
+                            self.payload, self.frequency)
+
+    def test_modfreq_per_always(self):
+        """
+        C{handle_part} is called regardless of frequency if nofreq is always.
+        """
+        self.frequency = "once"
+        # Build a mock part-handler module
+        mod_mock = self.mocker.mock()
+        getattr(mod_mock, "frequency")
+        self.mocker.result("always")
+        getattr(mod_mock, "handler_version")
+        self.mocker.result(1)
+        mod_mock.handle_part(self.data, self.ctype, self.filename,
+                             self.payload)
+        self.mocker.replay()
+
+        handler_handle_part(mod_mock, self.data, self.ctype, self.filename,
+                            self.payload, self.frequency)
+
+    def test_no_handle_when_modfreq_once(self):
+        """C{handle_part} is not called if frequency is once"""
+        self.frequency = "once"
+        # Build a mock part-handler module
+        mod_mock = self.mocker.mock()
+        getattr(mod_mock, "frequency")
+        self.mocker.result("once-per-instance")
+        self.mocker.replay()
+
+        handler_handle_part(mod_mock, self.data, self.ctype, self.filename,
+                            self.payload, self.frequency)
+
+    def test_exception_is_caught(self):
+        """Exceptions within C{handle_part} are caught and logged."""
+        # Build a mock part-handler module
+        mod_mock = self.mocker.mock()
+        getattr(mod_mock, "frequency")
+        self.mocker.result("once-per-instance")
+        getattr(mod_mock, "handler_version")
+        self.mocker.result(1)
+        mod_mock.handle_part(self.data, self.ctype, self.filename,
+                             self.payload)
+        self.mocker.throw(Exception())
+        # Mock log function
+        logexc_mock = self.mocker.replace(logexc, passthrough=False)
+        logexc_mock(ANY)
+        # Mock the print_exc function
+        print_exc_mock = self.mocker.replace("traceback.print_exc",
+                                             passthrough=False)
+        print_exc_mock(ARGS, KWARGS)
+        self.mocker.replay()
+
+        handler_handle_part(mod_mock, self.data, self.ctype, self.filename,
+                            self.payload, self.frequency)
+
+
+class TestCmdlineUrl(MockerTestCase):
+    def test_invalid_content(self):
+        url = "http://example.com/foo"
+        key = "mykey"
+        payload = "0"
+        cmdline = "ro %s=%s bar=1" % (key, url)
+
+        mock_readurl = self.mocker.replace(readurl, passthrough=False)
+        mock_readurl(url)
+        self.mocker.result(payload)
+
+        self.mocker.replay()
+
+        self.assertEqual((key, url, None),
+            get_cmdline_url(names=[key], starts="xxxxxx", cmdline=cmdline))
+
+    def test_valid_content(self):
+        url = "http://example.com/foo"
+        key = "mykey"
+        payload = "xcloud-config\nmydata: foo\nbar: wark\n"
+        cmdline = "ro %s=%s bar=1" % (key, url)
+
+        mock_readurl = self.mocker.replace(readurl, passthrough=False)
+        mock_readurl(url)
+        self.mocker.result(payload)
+
+        self.mocker.replay()
+
+        self.assertEqual((key, url, payload),
+            get_cmdline_url(names=[key], starts="xcloud-config",
+                            cmdline=cmdline))
+
+    def test_no_key_found(self):
+        url = "http://example.com/foo"
+        key = "mykey"
+        cmdline = "ro %s=%s bar=1" % (key, url)
+
+        self.mocker.replace(readurl, passthrough=False)
+        self.mocker.replay()
+
+        self.assertEqual((None, None, None),
+            get_cmdline_url(names=["does-not-appear"],
+                starts="#cloud-config", cmdline=cmdline))
+
+# vi: ts=4 expandtab
 
=== added directory 'tests/unittests/test_datasource'
=== added file 'tests/unittests/test_datasource/test_maas.py'
--- tests/unittests/test_datasource/test_maas.py	1970-01-01 00:00:00 +0000
+++ tests/unittests/test_datasource/test_maas.py	2012-06-13 13:16:19 +0000
@@ -0,0 +1,153 @@
+from tempfile import mkdtemp
+from shutil import rmtree
+import os
+from StringIO import StringIO
+from copy import copy
+from cloudinit.DataSourceMAAS import (
+    MAASSeedDirNone,
+    MAASSeedDirMalformed,
+    read_maas_seed_dir,
+    read_maas_seed_url,
+)
+from mocker import MockerTestCase
+
+
+class TestMAASDataSource(MockerTestCase):
+
+    def setUp(self):
+        super(TestMAASDataSource, self).setUp()
+        # Make a temp directoy for tests to use.
+        self.tmp = mkdtemp(prefix="unittest_")
+
+    def tearDown(self):
+        super(TestMAASDataSource, self).tearDown()
+        # Clean up temp directory
+        rmtree(self.tmp)
+
+    def test_seed_dir_valid(self):
+        """Verify a valid seeddir is read as such"""
+
+        data = {'instance-id': 'i-valid01',
+            'local-hostname': 'valid01-hostname',
+            'user-data': 'valid01-userdata',
+            'public-keys': 'ssh-rsa AAAAB3Nz...aC1yc2E= keyname'}
+
+        my_d = os.path.join(self.tmp, "valid")
+        populate_dir(my_d, data)
+
+        (userdata, metadata) = read_maas_seed_dir(my_d)
+
+        self.assertEqual(userdata, data['user-data'])
+        for key in ('instance-id', 'local-hostname'):
+            self.assertEqual(data[key], metadata[key])
+
+        # verify that 'userdata' is not returned as part of the metadata
+        self.assertFalse(('user-data' in metadata))
+
+    def test_seed_dir_valid_extra(self):
+        """Verify extra files do not affect seed_dir validity """
+
+        data = {'instance-id': 'i-valid-extra',
+            'local-hostname': 'valid-extra-hostname',
+            'user-data': 'valid-extra-userdata', 'foo': 'bar'}
+
+        my_d = os.path.join(self.tmp, "valid_extra")
+        populate_dir(my_d, data)
+
+        (userdata, metadata) = read_maas_seed_dir(my_d)
+
+        self.assertEqual(userdata, data['user-data'])
+        for key in ('instance-id', 'local-hostname'):
+            self.assertEqual(data[key], metadata[key])
+
+        # additional files should not just appear as keys in metadata atm
+        self.assertFalse(('foo' in metadata))
+
+    def test_seed_dir_invalid(self):
+        """Verify that invalid seed_dir raises MAASSeedDirMalformed"""
+
+        valid = {'instance-id': 'i-instanceid',
+            'local-hostname': 'test-hostname', 'user-data': ''}
+
+        my_based = os.path.join(self.tmp, "valid_extra")
+
+        # missing 'userdata' file
+        my_d = "%s-01" % my_based
+        invalid_data = copy(valid)
+        del invalid_data['local-hostname']
+        populate_dir(my_d, invalid_data)
+        self.assertRaises(MAASSeedDirMalformed, read_maas_seed_dir, my_d)
+
+        # missing 'instance-id'
+        my_d = "%s-02" % my_based
+        invalid_data = copy(valid)
+        del invalid_data['instance-id']
+        populate_dir(my_d, invalid_data)
+        self.assertRaises(MAASSeedDirMalformed, read_maas_seed_dir, my_d)
+
+    def test_seed_dir_none(self):
+        """Verify that empty seed_dir raises MAASSeedDirNone"""
+
+        my_d = os.path.join(self.tmp, "valid_empty")
+        self.assertRaises(MAASSeedDirNone, read_maas_seed_dir, my_d)
+
+    def test_seed_dir_missing(self):
+        """Verify that missing seed_dir raises MAASSeedDirNone"""
+        self.assertRaises(MAASSeedDirNone, read_maas_seed_dir,
+            os.path.join(self.tmp, "nonexistantdirectory"))
+
+    def test_seed_url_valid(self):
+        """Verify that valid seed_url is read as such"""
+        valid = {'meta-data/instance-id': 'i-instanceid',
+            'meta-data/local-hostname': 'test-hostname',
+            'meta-data/public-keys': 'test-hostname',
+            'user-data': 'foodata'}
+
+        my_seed = "http://example.com/xmeta"
+        my_ver = "1999-99-99"
+        my_headers = {'header1': 'value1', 'header2': 'value2'}
+
+        def my_headers_cb(url):
+            return(my_headers)
+
+        mock_request = self.mocker.replace("urllib2.Request",
+            passthrough=False)
+        mock_urlopen = self.mocker.replace("urllib2.urlopen",
+            passthrough=False)
+
+        for (key, val) in valid.iteritems():
+            mock_request("%s/%s/%s" % (my_seed, my_ver, key),
+                data=None, headers=my_headers)
+            self.mocker.nospec()
+            self.mocker.result("fake-request-%s" % key)
+            mock_urlopen("fake-request-%s" % key, timeout=None)
+            self.mocker.result(StringIO(val))
+
+        self.mocker.replay()
+
+        (userdata, metadata) = read_maas_seed_url(my_seed,
+            header_cb=my_headers_cb, version=my_ver)
+
+        self.assertEqual("foodata", userdata)
+        self.assertEqual(metadata['instance-id'],
+            valid['meta-data/instance-id'])
+        self.assertEqual(metadata['local-hostname'],
+            valid['meta-data/local-hostname'])
+
+    def test_seed_url_invalid(self):
+        """Verify that invalid seed_url raises MAASSeedDirMalformed"""
+        pass
+
+    def test_seed_url_missing(self):
+        """Verify seed_url with no found entries raises MAASSeedDirNone"""
+        pass
+
+
+def populate_dir(seed_dir, files):
+    os.mkdir(seed_dir)
+    for (name, content) in files.iteritems():
+        with open(os.path.join(seed_dir, name), "w") as fp:
+            fp.write(content)
+            fp.close()
+
+# vi: ts=4 expandtab
 
=== added directory 'tests/unittests/test_handler'
=== renamed file 'tests/unittests/test_handler_ca_certs.py' => 'tests/unittests/test_handler/test_handler_ca_certs.py'
--- tests/unittests/test_handler_ca_certs.py	2012-01-17 21:38:01 +0000
+++ tests/unittests/test_handler/test_handler_ca_certs.py	2012-06-13 13:16:19 +0000
@@ -169,10 +169,14 @@
         mock_delete_dir_contents = self.mocker.replace(delete_dir_contents,
                                                        passthrough=False)
         mock_write = self.mocker.replace(write_file, passthrough=False)
+        mock_subp = self.mocker.replace("cloudinit.util.subp",
+                                        passthrough=False)
 
         mock_delete_dir_contents("/usr/share/ca-certificates/")
         mock_delete_dir_contents("/etc/ssl/certs/")
         mock_write("/etc/ca-certificates.conf", "", mode=0644)
+        mock_subp(('debconf-set-selections', '-'),
+                  "ca-certificates ca-certificates/trust_new_crts select no")
         self.mocker.replay()
 
         remove_default_ca_certs()
 
=== added file 'tests/unittests/test_userdata.py'
--- tests/unittests/test_userdata.py	1970-01-01 00:00:00 +0000
+++ tests/unittests/test_userdata.py	2012-06-13 13:16:19 +0000
@@ -0,0 +1,107 @@
+"""Tests for handling of userdata within cloud init"""
+
+import logging
+import StringIO
+
+from email.mime.base import MIMEBase
+
+from mocker import MockerTestCase
+
+import cloudinit
+from cloudinit.DataSource import DataSource
+
+
+instance_id = "i-testing"
+
+
+class FakeDataSource(DataSource):
+
+    def __init__(self, userdata):
+        DataSource.__init__(self)
+        self.metadata = {'instance-id': instance_id}
+        self.userdata_raw = userdata
+
+
+class TestConsumeUserData(MockerTestCase):
+
+    _log_handler = None
+    _log = None
+    log_file = None
+
+    def setUp(self):
+        self.mock_write = self.mocker.replace("cloudinit.util.write_file",
+            passthrough=False)
+        self.mock_write(self.get_ipath("cloud_config"), "", 0600)
+        self.capture_log()
+
+    def tearDown(self):
+        self._log.removeHandler(self._log_handler)
+
+    @staticmethod
+    def get_ipath(name):
+        return "%s/instances/%s%s" % (cloudinit.varlibdir, instance_id,
+            cloudinit.pathmap[name])
+
+    def capture_log(self):
+        self.log_file = StringIO.StringIO()
+        self._log_handler = logging.StreamHandler(self.log_file)
+        self._log_handler.setLevel(logging.DEBUG)
+        self._log = logging.getLogger(cloudinit.logger_name)
+        self._log.addHandler(self._log_handler)
+
+    def test_unhandled_type_warning(self):
+        """Raw text without magic is ignored but shows warning"""
+        self.mocker.replay()
+        ci = cloudinit.CloudInit()
+        ci.datasource = FakeDataSource("arbitrary text\n")
+        ci.consume_userdata()
+        self.assertEqual(
+            "Unhandled non-multipart userdata starting 'arbitrary text...'\n",
+            self.log_file.getvalue())
+
+    def test_mime_text_plain(self):
+        """Mime message of type text/plain is ignored without warning"""
+        self.mocker.replay()
+        ci = cloudinit.CloudInit()
+        message = MIMEBase("text", "plain")
+        message.set_payload("Just text")
+        ci.datasource = FakeDataSource(message.as_string())
+        ci.consume_userdata()
+        self.assertEqual("", self.log_file.getvalue())
+
+    def test_shellscript(self):
+        """Raw text starting #!/bin/sh is treated as script"""
+        script = "#!/bin/sh\necho hello\n"
+        outpath = cloudinit.get_ipath_cur("scripts") + "/part-001"
+        self.mock_write(outpath, script, 0700)
+        self.mocker.replay()
+        ci = cloudinit.CloudInit()
+        ci.datasource = FakeDataSource(script)
+        ci.consume_userdata()
+        self.assertEqual("", self.log_file.getvalue())
+
+    def test_mime_text_x_shellscript(self):
+        """Mime message of type text/x-shellscript is treated as script"""
+        script = "#!/bin/sh\necho hello\n"
+        outpath = cloudinit.get_ipath_cur("scripts") + "/part-001"
+        self.mock_write(outpath, script, 0700)
+        self.mocker.replay()
+        ci = cloudinit.CloudInit()
+        message = MIMEBase("text", "x-shellscript")
+        message.set_payload(script)
+        ci.datasource = FakeDataSource(message.as_string())
+        ci.consume_userdata()
+        self.assertEqual("", self.log_file.getvalue())
+
+    def test_mime_text_plain_shell(self):
+        """Mime type text/plain starting #!/bin/sh is treated as script"""
+        script = "#!/bin/sh\necho hello\n"
+        outpath = cloudinit.get_ipath_cur("scripts") + "/part-001"
+        self.mock_write(outpath, script, 0700)
+        self.mocker.replay()
+        ci = cloudinit.CloudInit()
+        message = MIMEBase("text", "plain")
+        message.set_payload(script)
+        ci.datasource = FakeDataSource(message.as_string())
+        ci.consume_userdata()
+        self.assertEqual("", self.log_file.getvalue())
 
=== modified file 'tests/unittests/test_util.py'
--- tests/unittests/test_util.py	2012-01-17 17:35:31 +0000
+++ tests/unittests/test_util.py	2012-06-13 13:16:19 +0000
@@ -6,7 +6,8 @@
 import stat
 
 from cloudinit.util import (mergedict, get_cfg_option_list_or_str, write_file,
-                            delete_dir_contents)
+                            delete_dir_contents, get_cmdline,
+                            keyval_str_to_dict)
 
 
 class TestMergeDict(TestCase):
@@ -28,7 +29,7 @@
     def test_merge_does_not_override(self):
         """Test that candidate doesn't override source."""
         source = {"key1": "value1", "key2": "value2"}
-        candidate = {"key2": "value2", "key2": "NEW VALUE"}
+        candidate = {"key1": "value2", "key2": "NEW VALUE"}
         result = mergedict(source, candidate)
         self.assertEqual(source, result)
 
@@ -248,3 +249,18 @@
         delete_dir_contents(self.tmp)
 
         self.assertDirEmpty(self.tmp)
+
+
+class TestKeyValStrings(TestCase):
+    def test_keyval_str_to_dict(self):
+        expected = {'1': 'one', '2': 'one+one', 'ro': True}
+        cmdline = "1=one ro 2=one+one"
+        self.assertEqual(expected, keyval_str_to_dict(cmdline))
+
+
+class TestGetCmdline(TestCase):
+    def test_cmdline_reads_debug_env(self):
+        os.environ['DEBUG_PROC_CMDLINE'] = 'abcd 123'
+        self.assertEqual(os.environ['DEBUG_PROC_CMDLINE'], get_cmdline())
+
+# vi: ts=4 expandtab
 
=== added file 'tools/Z99-cloud-locale-test.sh'
--- tools/Z99-cloud-locale-test.sh	1970-01-01 00:00:00 +0000
+++ tools/Z99-cloud-locale-test.sh	2012-06-13 13:16:19 +0000
@@ -0,0 +1,92 @@
+#!/bin/sh
+# vi: ts=4 noexpandtab
+#
+# Author: Ben Howard <[e-mail]>
+# Author: Scott Moser <[e-mail]>
+# (c) 2012, Canonical Group, Ltd.
+#
+# Purpose: Detect invalid locale settings and inform the user
+#  of how to fix them.
+#
+
+locale_warn() {
+	local cr="
+"
+	local bad_names="" bad_lcs="" key="" value="" var=""
+	local w1 w2 w3 w4 remain
+	# locale is expected to output either:
+	# VARIABLE=
+	# VARIABLE="value"
+	# locale: Cannot set LC_SOMETHING to default locale
+	while read -r w1 w2 w3 w4 remain; do
+		case "$w1" in
+			locale:) bad_names="${bad_names} ${w4}";;
+			*)
+				key=${w1%%=*}
+				val=${w1#*=}
+				val=${val#\"}
+				val=${val%\"}
+				vars="${vars} $key=$val";;
+		esac
+	done
+	for bad in $bad_names; do
+		for var in ${vars}; do
+			[ "${bad}" = "${var%=*}" ] || continue
+			value=${var#*=}
+			[ "${bad_lcs#* ${value}}" = "${bad_lcs}" ] &&
+        			bad_lcs="${bad_lcs} ${value}"
+			break
+		done
+	done
+	bad_lcs=${bad_lcs# }
+	[ -n "$bad_lcs" ] || return 0
+
+	printf "_____________________________________________________________________\n"
+	printf "WARNING! Your environment specifies an invalid locale.\n"
+	printf " This can affect your user experience significantly, including the\n"
+	printf " ability to manage packages. You may install the locales by running:\n\n"
+
+	local bad invalid="" to_gen="" sfile="/usr/share/i18n/SUPPORTED"
+	local pkgs=""
+	if [ -e "$sfile" ]; then
+		for bad in ${bad_lcs}; do
+			grep -q -i "${bad}" "$sfile" &&
+				to_gen="${to_gen} ${bad}" ||
+				invalid="${invalid} ${bad}"
+		done
+	else
+		printf "  sudo apt-get install locales\n"
+		to_gen=$bad_lcs
+	fi
+	to_gen=${to_gen# }
+
+	local pkgs=""
+	for bad in ${to_gen}; do
+		pkgs="${pkgs} language-pack-${bad%%_*}"
+	done
+	pkgs=${pkgs# }
+
+	if [ -n "${pkgs}" ]; then
+		printf "   sudo apt-get install ${pkgs# }\n"
+		printf "     or\n"
+		printf "   sudo locale-gen ${to_gen# }\n"
+		printf "\n"
+	fi
+	for bad in ${invalid}; do
+        	printf "WARNING: '${bad}' is an invalid locale\n"
+	done
+
+	printf "To see all available language packs, run:\n"
+	printf "   apt-cache search \"^language-pack-[a-z][a-z]$\"\n"
+	printf "To disable this message for all users, run:\n"
+	printf "   sudo touch /var/lib/cloud/instance/locale-check.skip\n"
+	printf "_____________________________________________________________________\n\n"
+
+	# only show the message once
+	: > ~/.cloud-locale-test.skip 2>/dev/null || :
+}
+
+[ -f ~/.cloud-locale-test.skip -o -f /var/lib/cloud/instance/locale-check.skip ] ||
+	locale 2>&1 | locale_warn
+
+unset locale_warn
 
=== modified file 'tools/run-pylint'
--- tools/run-pylint	2012-01-17 20:59:21 +0000
+++ tools/run-pylint	2012-06-13 13:16:19 +0000
@@ -1,6 +1,8 @@
 #!/bin/bash
 
-def_files='cloud*.py cloudinit/*.py cloudinit/CloudConfig/*.py'
+ci_files='cloud*.py cloudinit/*.py cloudinit/CloudConfig/*.py'
+test_files=$(find tests -name "*.py")
+def_files="$ci_files $test_files"
 
 if [ $# -eq 0 ]; then
    files=( )