Merge lp:~1chb1n/charms/trusty/ubuntu/simplify-and-test into lp:charms/trusty/ubuntu

[Ryan Beisner]

Revert charm to have no hooks and no config options; add functional tests for all currently-supported Ubuntu releases; add bundletester usage examples.

Charm is now compatible with both 'juju test' and bundletester.

Revert charm to have no hooks and no config options; add functional tests for all currently-supported Ubuntu releases; add bundletester usage examples.

Charm is now compatible with both 'juju test' and bundletester.

[Ryan Beisner (1chb1n)]

Tested via bundletester / amulet on Precise, Trusty, Utopic, Vivid. Ready for review.

Results: http://paste.ubuntu.com/10952530/

[Marco Ceppi (marcoceppi)]

See below

[Ryan Beisner (1chb1n)]

Thank you for the review. The bundle file is an example which won't be exercised in automation unless explicitly called. It accompanies the bundletester usage examples in the Makefile, which a user or charmer can optionally fire.

The juju-gui is present as a general use example, and as a handy way to fire up the most basic juju canvas. I think the Ubuntu charm is a good place for that, as well as these other usage examples which encourage easily adding functional test coverage. ;-)

To clarify, this MP aims to address 3 things:

1) Return the charm to a functional state on all supported releases.

2) Add test coverage for all supported releases, detect future breakage re: bug 1447765.

3) Add bundletester usage examples since we don't have that in one place anywhere else.

[Tim Van Steenburgh (tvansteenburgh)]

+1 LGTM, thanks Ryan.

review: Approve

=== added file 'Makefile'
--- Makefile	1970-01-01 00:00:00 +0000
+++ Makefile	2015-04-30 13:14:23 +0000
@@ -0,0 +1,57 @@
+#!/usr/bin/make
+PYTHON := /usr/bin/env python
+
+virtualenv:
+	@echo Setting up python virtual env...
+	virtualenv .venv
+	.venv/bin/pip install flake8 charm-tools bundletester
+
+lint: virtualenv
+	@echo Lint inspections and charm proof...
+	.venv/bin/flake8 --exclude hooks/charmhelpers tests
+	.venv/bin/charm proof
+
+test: virtualenv
+	@echo No unit tests defined.  This is an example placeholder.
+
+functional_test: virtualenv
+	# Consume ./tests/tests.yaml to determine the tests to run,
+	# in addition to functional tests in the ./tests dir.
+	@echo Starting functional, lint and unit tests...
+	.venv/bin/bundletester -v -F -l DEBUG -r dot -o results-all.dot
+
+test_example: virtualenv
+	# A bundletester usage example to run only the matching tests.
+	@echo Starting a subset of tests...
+	.venv/bin/bundletester -v -F -l DEBUG -r json -o results-ex.json \
+        --test-pattern 02*
+
+test_example2: virtualenv
+	# A bundletester usage example to run only the specified tests,
+    # with a different output format.
+	@echo Starting a subset of tests...
+	.venv/bin/bundletester -v -F -l DEBUG -r spec -o results-ex2.spec \
+        010_basic_precise 015_basic_trusty
+
+test_example3: virtualenv
+	# A bundletester bundle usage example.
+	@echo Starting a subset of tests...
+	.venv/bin/bundletester -v -F -l DEBUG -r dot -o results-ex3.dot \
+        -b files/bundle-example.yaml
+
+bin/charm_helpers_sync.py:
+	@mkdir -p bin
+	@bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \
+        > bin/charm_helpers_sync.py
+
+sync: bin/charm_helpers_sync.py
+	@echo Syncing charm helpers for functional tests...
+	@$(PYTHON) bin/charm_helpers_sync.py -c charm-helpers-tests.yaml
+
+publish: clean lint
+	bzr push lp:charms/trusty/ubuntu
+
+clean:
+	@ echo Cleaning up venvs and pyc files...
+	rm -rf .venv
+	find -name *.pyc -delete
 
=== modified file 'README.md'
--- README.md	2015-03-25 19:59:09 +0000
+++ README.md	2015-04-30 13:14:23 +0000
@@ -25,32 +25,16 @@
     juju deploy ubuntu mytestmachine1
     juju deploy ubuntu mytestmachine2
 
-and so on.
-
-# Configuration
-
-A configuration option to create a new network for LXC containers is provided via a boolean config option, 'new-lxc-network.'
-The default is set to false indicating the same network of the node will also be used for LXC containers.
-
-To use a new network for LXC containers post juju deploy, issue:
-
-    juju set ubuntu new-lxc-network=true
-
-If you want to have a new network for LXC containers on deploy set up a lxc-config.yaml:
-
-    ubuntu:
-      new-lxc-network: true
-
-and pass that as an arugment to deploy:
-
-    juju deploy --config lxc-config.yaml ubuntu
-
-For further information on setting configuration with Juju see https://jujucharms.com/docs/latest/charms-config
+and so on. 
 
 ## Known Limitations and Issues
 
 This charm does not provide anything other than a blank server, so it does not relate to other charms.
 
+# Configuration
+
+This charm has no configuration options.
+
 # Contact Information
 
 
 
=== removed file 'charm-helpers-hooks.yaml'
--- charm-helpers-hooks.yaml	2015-02-19 16:21:31 +0000
+++ charm-helpers-hooks.yaml	1970-01-01 00:00:00 +0000
@@ -1,5 +0,0 @@
-branch: lp:charm-helpers
-destination: hooks/charmhelpers
-include:
-    - core
-    - fetch
 
=== added file 'charm-helpers-tests.yaml'
--- charm-helpers-tests.yaml	1970-01-01 00:00:00 +0000
+++ charm-helpers-tests.yaml	2015-04-30 13:14:23 +0000
@@ -0,0 +1,4 @@
+branch: lp:charm-helpers
+destination: tests/charmhelpers
+include:
+    - contrib.amulet
 
=== removed file 'config.yaml'
--- config.yaml	2015-03-25 19:59:09 +0000
+++ config.yaml	1970-01-01 00:00:00 +0000
@@ -1,8 +0,0 @@
-options:
-  new-lxc-network:
-    type: boolean
-    default: False 
-    description: |
-      If True creates new network for lxc containers, otherwise using the
-      same network of the node for lxc containers.
-
 
=== added directory 'files'
=== added file 'files/bundle-example.yaml'
--- files/bundle-example.yaml	1970-01-01 00:00:00 +0000
+++ files/bundle-example.yaml	2015-04-30 13:14:23 +0000
@@ -0,0 +1,14 @@
+"ubuntu-example-bundle":
+  services:
+    "ubuntu":
+      charm: cs:trusty/ubuntu
+      num_units: 1
+      annotations:
+        "gui-x": "0"
+        "gui-y": "0"
+    "juju-gui":
+      charm: "cs:trusty/juju-gui"
+      num_units: 1
+      annotations:
+        "gui-x": "500"
+        "gui-y": "0"
 
=== removed directory 'hooks'
=== removed file 'hooks/__init__.py'
=== removed directory 'hooks/charmhelpers'
=== removed file 'hooks/charmhelpers/__init__.py'
=== removed directory 'hooks/charmhelpers/core'
=== removed file 'hooks/charmhelpers/core/__init__.py'
--- hooks/charmhelpers/core/__init__.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,15 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
 
=== removed file 'hooks/charmhelpers/core/decorators.py'
--- hooks/charmhelpers/core/decorators.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/decorators.py	1970-01-01 00:00:00 +0000
@@ -1,57 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-#
-# Copyright 2014 Canonical Ltd.
-#
-# Authors:
-#  Edward Hope-Morley <[e-mail]>
-#
-
-import time
-
-from charmhelpers.core.hookenv import (
-    log,
-    INFO,
-)
-
-
-def retry_on_exception(num_retries, base_delay=0, exc_type=Exception):
-    """If the decorated function raises exception exc_type, allow num_retries
-    retry attempts before raise the exception.
-    """
-    def _retry_on_exception_inner_1(f):
-        def _retry_on_exception_inner_2(*args, **kwargs):
-            retries = num_retries
-            multiplier = 1
-            while True:
-                try:
-                    return f(*args, **kwargs)
-                except exc_type:
-                    if not retries:
-                        raise
-
-                delay = base_delay * multiplier
-                multiplier += 1
-                log("Retrying '%s' %d more times (delay=%s)" %
-                    (f.__name__, retries, delay), level=INFO)
-                retries -= 1
-                if delay:
-                    time.sleep(delay)
-
-        return _retry_on_exception_inner_2
-
-    return _retry_on_exception_inner_1
 
=== removed file 'hooks/charmhelpers/core/fstab.py'
--- hooks/charmhelpers/core/fstab.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/fstab.py	1970-01-01 00:00:00 +0000
@@ -1,134 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import io
-import os
-
-__author__ = 'Jorge Niedbalski R. <[e-mail]>'
-
-
-class Fstab(io.FileIO):
-    """This class extends file in order to implement a file reader/writer
-    for file `/etc/fstab`
-    """
-
-    class Entry(object):
-        """Entry class represents a non-comment line on the `/etc/fstab` file
-        """
-        def __init__(self, device, mountpoint, filesystem,
-                     options, d=0, p=0):
-            self.device = device
-            self.mountpoint = mountpoint
-            self.filesystem = filesystem
-
-            if not options:
-                options = "defaults"
-
-            self.options = options
-            self.d = int(d)
-            self.p = int(p)
-
-        def __eq__(self, o):
-            return str(self) == str(o)
-
-        def __str__(self):
-            return "{} {} {} {} {} {}".format(self.device,
-                                              self.mountpoint,
-                                              self.filesystem,
-                                              self.options,
-                                              self.d,
-                                              self.p)
-
-    DEFAULT_PATH = os.path.join(os.path.sep, 'etc', 'fstab')
-
-    def __init__(self, path=None):
-        if path:
-            self._path = path
-        else:
-            self._path = self.DEFAULT_PATH
-        super(Fstab, self).__init__(self._path, 'rb+')
-
-    def _hydrate_entry(self, line):
-        # NOTE: use split with no arguments to split on any
-        #       whitespace including tabs
-        return Fstab.Entry(*filter(
-            lambda x: x not in ('', None),
-            line.strip("\n").split()))
-
-    @property
-    def entries(self):
-        self.seek(0)
-        for line in self.readlines():
-            line = line.decode('us-ascii')
-            try:
-                if line.strip() and not line.strip().startswith("#"):
-                    yield self._hydrate_entry(line)
-            except ValueError:
-                pass
-
-    def get_entry_by_attr(self, attr, value):
-        for entry in self.entries:
-            e_attr = getattr(entry, attr)
-            if e_attr == value:
-                return entry
-        return None
-
-    def add_entry(self, entry):
-        if self.get_entry_by_attr('device', entry.device):
-            return False
-
-        self.write((str(entry) + '\n').encode('us-ascii'))
-        self.truncate()
-        return entry
-
-    def remove_entry(self, entry):
-        self.seek(0)
-
-        lines = [l.decode('us-ascii') for l in self.readlines()]
-
-        found = False
-        for index, line in enumerate(lines):
-            if line.strip() and not line.strip().startswith("#"):
-                if self._hydrate_entry(line) == entry:
-                    found = True
-                    break
-
-        if not found:
-            return False
-
-        lines.remove(line)
-
-        self.seek(0)
-        self.write(''.join(lines).encode('us-ascii'))
-        self.truncate()
-        return True
-
-    @classmethod
-    def remove_by_mountpoint(cls, mountpoint, path=None):
-        fstab = cls(path=path)
-        entry = fstab.get_entry_by_attr('mountpoint', mountpoint)
-        if entry:
-            return fstab.remove_entry(entry)
-        return False
-
-    @classmethod
-    def add(cls, device, mountpoint, filesystem, options=None, path=None):
-        return cls(path=path).add_entry(Fstab.Entry(device,
-                                                    mountpoint, filesystem,
-                                                    options=options))
 
=== removed file 'hooks/charmhelpers/core/hookenv.py'
--- hooks/charmhelpers/core/hookenv.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/hookenv.py	1970-01-01 00:00:00 +0000
@@ -1,568 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-"Interactions with the Juju environment"
-# Copyright 2013 Canonical Ltd.
-#
-# Authors:
-#  Charm Helpers Developers <[e-mail]>
-
-import os
-import json
-import yaml
-import subprocess
-import sys
-from subprocess import CalledProcessError
-
-import six
-if not six.PY3:
-    from UserDict import UserDict
-else:
-    from collections import UserDict
-
-CRITICAL = "CRITICAL"
-ERROR = "ERROR"
-WARNING = "WARNING"
-INFO = "INFO"
-DEBUG = "DEBUG"
-MARKER = object()
-
-cache = {}
-
-
-def cached(func):
-    """Cache return values for multiple executions of func + args
-
-    For example::
-
-        @cached
-        def unit_get(attribute):
-            pass
-
-        unit_get('test')
-
-    will cache the result of unit_get + 'test' for future calls.
-    """
-    def wrapper(*args, **kwargs):
-        global cache
-        key = str((func, args, kwargs))
-        try:
-            return cache[key]
-        except KeyError:
-            res = func(*args, **kwargs)
-            cache[key] = res
-            return res
-    return wrapper
-
-
-def flush(key):
-    """Flushes any entries from function cache where the
-    key is found in the function+args """
-    flush_list = []
-    for item in cache:
-        if key in item:
-            flush_list.append(item)
-    for item in flush_list:
-        del cache[item]
-
-
-def log(message, level=None):
-    """Write a message to the juju log"""
-    command = ['juju-log']
-    if level:
-        command += ['-l', level]
-    if not isinstance(message, six.string_types):
-        message = repr(message)
-    command += [message]
-    subprocess.call(command)
-
-
-class Serializable(UserDict):
-    """Wrapper, an object that can be serialized to yaml or json"""
-
-    def __init__(self, obj):
-        # wrap the object
-        UserDict.__init__(self)
-        self.data = obj
-
-    def __getattr__(self, attr):
-        # See if this object has attribute.
-        if attr in ("json", "yaml", "data"):
-            return self.__dict__[attr]
-        # Check for attribute in wrapped object.
-        got = getattr(self.data, attr, MARKER)
-        if got is not MARKER:
-            return got
-        # Proxy to the wrapped object via dict interface.
-        try:
-            return self.data[attr]
-        except KeyError:
-            raise AttributeError(attr)
-
-    def __getstate__(self):
-        # Pickle as a standard dictionary.
-        return self.data
-
-    def __setstate__(self, state):
-        # Unpickle into our wrapper.
-        self.data = state
-
-    def json(self):
-        """Serialize the object to json"""
-        return json.dumps(self.data)
-
-    def yaml(self):
-        """Serialize the object to yaml"""
-        return yaml.dump(self.data)
-
-
-def execution_environment():
-    """A convenient bundling of the current execution context"""
-    context = {}
-    context['conf'] = config()
-    if relation_id():
-        context['reltype'] = relation_type()
-        context['relid'] = relation_id()
-        context['rel'] = relation_get()
-    context['unit'] = local_unit()
-    context['rels'] = relations()
-    context['env'] = os.environ
-    return context
-
-
-def in_relation_hook():
-    """Determine whether we're running in a relation hook"""
-    return 'JUJU_RELATION' in os.environ
-
-
-def relation_type():
-    """The scope for the current relation hook"""
-    return os.environ.get('JUJU_RELATION', None)
-
-
-def relation_id():
-    """The relation ID for the current relation hook"""
-    return os.environ.get('JUJU_RELATION_ID', None)
-
-
-def local_unit():
-    """Local unit ID"""
-    return os.environ['JUJU_UNIT_NAME']
-
-
-def remote_unit():
-    """The remote unit for the current relation hook"""
-    return os.environ['JUJU_REMOTE_UNIT']
-
-
-def service_name():
-    """The name service group this unit belongs to"""
-    return local_unit().split('/')[0]
-
-
-def hook_name():
-    """The name of the currently executing hook"""
-    return os.path.basename(sys.argv[0])
-
-
-class Config(dict):
-    """A dictionary representation of the charm's config.yaml, with some
-    extra features:
-
-    - See which values in the dictionary have changed since the previous hook.
-    - For values that have changed, see what the previous value was.
-    - Store arbitrary data for use in a later hook.
-
-    NOTE: Do not instantiate this object directly - instead call
-    ``hookenv.config()``, which will return an instance of :class:`Config`.
-
-    Example usage::
-
-        >>> # inside a hook
-        >>> from charmhelpers.core import hookenv
-        >>> config = hookenv.config()
-        >>> config['foo']
-        'bar'
-        >>> # store a new key/value for later use
-        >>> config['mykey'] = 'myval'
-
-
-        >>> # user runs `juju set mycharm foo=baz`
-        >>> # now we're inside subsequent config-changed hook
-        >>> config = hookenv.config()
-        >>> config['foo']
-        'baz'
-        >>> # test to see if this val has changed since last hook
-        >>> config.changed('foo')
-        True
-        >>> # what was the previous value?
-        >>> config.previous('foo')
-        'bar'
-        >>> # keys/values that we add are preserved across hooks
-        >>> config['mykey']
-        'myval'
-
-    """
-    CONFIG_FILE_NAME = '.juju-persistent-config'
-
-    def __init__(self, *args, **kw):
-        super(Config, self).__init__(*args, **kw)
-        self.implicit_save = True
-        self._prev_dict = None
-        self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME)
-        if os.path.exists(self.path):
-            self.load_previous()
-
-    def __getitem__(self, key):
-        """For regular dict lookups, check the current juju config first,
-        then the previous (saved) copy. This ensures that user-saved values
-        will be returned by a dict lookup.
-
-        """
-        try:
-            return dict.__getitem__(self, key)
-        except KeyError:
-            return (self._prev_dict or {})[key]
-
-    def keys(self):
-        prev_keys = []
-        if self._prev_dict is not None:
-            prev_keys = self._prev_dict.keys()
-        return list(set(prev_keys + list(dict.keys(self))))
-
-    def load_previous(self, path=None):
-        """Load previous copy of config from disk.
-
-        In normal usage you don't need to call this method directly - it
-        is called automatically at object initialization.
-
-        :param path:
-
-            File path from which to load the previous config. If `None`,
-            config is loaded from the default location. If `path` is
-            specified, subsequent `save()` calls will write to the same
-            path.
-
-        """
-        self.path = path or self.path
-        with open(self.path) as f:
-            self._prev_dict = json.load(f)
-
-    def changed(self, key):
-        """Return True if the current value for this key is different from
-        the previous value.
-
-        """
-        if self._prev_dict is None:
-            return True
-        return self.previous(key) != self.get(key)
-
-    def previous(self, key):
-        """Return previous value for this key, or None if there
-        is no previous value.
-
-        """
-        if self._prev_dict:
-            return self._prev_dict.get(key)
-        return None
-
-    def save(self):
-        """Save this config to disk.
-
-        If the charm is using the :mod:`Services Framework <services.base>`
-        or :meth:'@hook <Hooks.hook>' decorator, this
-        is called automatically at the end of successful hook execution.
-        Otherwise, it should be called directly by user code.
-
-        To disable automatic saves, set ``implicit_save=False`` on this
-        instance.
-
-        """
-        if self._prev_dict:
-            for k, v in six.iteritems(self._prev_dict):
-                if k not in self:
-                    self[k] = v
-        with open(self.path, 'w') as f:
-            json.dump(self, f)
-
-
-@cached
-def config(scope=None):
-    """Juju charm configuration"""
-    config_cmd_line = ['config-get']
-    if scope is not None:
-        config_cmd_line.append(scope)
-    config_cmd_line.append('--format=json')
-    try:
-        config_data = json.loads(
-            subprocess.check_output(config_cmd_line).decode('UTF-8'))
-        if scope is not None:
-            return config_data
-        return Config(config_data)
-    except ValueError:
-        return None
-
-
-@cached
-def relation_get(attribute=None, unit=None, rid=None):
-    """Get relation information"""
-    _args = ['relation-get', '--format=json']
-    if rid:
-        _args.append('-r')
-        _args.append(rid)
-    _args.append(attribute or '-')
-    if unit:
-        _args.append(unit)
-    try:
-        return json.loads(subprocess.check_output(_args).decode('UTF-8'))
-    except ValueError:
-        return None
-    except CalledProcessError as e:
-        if e.returncode == 2:
-            return None
-        raise
-
-
-def relation_set(relation_id=None, relation_settings=None, **kwargs):
-    """Set relation information for the current unit"""
-    relation_settings = relation_settings if relation_settings else {}
-    relation_cmd_line = ['relation-set']
-    if relation_id is not None:
-        relation_cmd_line.extend(('-r', relation_id))
-    for k, v in (list(relation_settings.items()) + list(kwargs.items())):
-        if v is None:
-            relation_cmd_line.append('{}='.format(k))
-        else:
-            relation_cmd_line.append('{}={}'.format(k, v))
-    subprocess.check_call(relation_cmd_line)
-    # Flush cache of any relation-gets for local unit
-    flush(local_unit())
-
-
-@cached
-def relation_ids(reltype=None):
-    """A list of relation_ids"""
-    reltype = reltype or relation_type()
-    relid_cmd_line = ['relation-ids', '--format=json']
-    if reltype is not None:
-        relid_cmd_line.append(reltype)
-        return json.loads(
-            subprocess.check_output(relid_cmd_line).decode('UTF-8')) or []
-    return []
-
-
-@cached
-def related_units(relid=None):
-    """A list of related units"""
-    relid = relid or relation_id()
-    units_cmd_line = ['relation-list', '--format=json']
-    if relid is not None:
-        units_cmd_line.extend(('-r', relid))
-    return json.loads(
-        subprocess.check_output(units_cmd_line).decode('UTF-8')) or []
-
-
-@cached
-def relation_for_unit(unit=None, rid=None):
-    """Get the json represenation of a unit's relation"""
-    unit = unit or remote_unit()
-    relation = relation_get(unit=unit, rid=rid)
-    for key in relation:
-        if key.endswith('-list'):
-            relation[key] = relation[key].split()
-    relation['__unit__'] = unit
-    return relation
-
-
-@cached
-def relations_for_id(relid=None):
-    """Get relations of a specific relation ID"""
-    relation_data = []
-    relid = relid or relation_ids()
-    for unit in related_units(relid):
-        unit_data = relation_for_unit(unit, relid)
-        unit_data['__relid__'] = relid
-        relation_data.append(unit_data)
-    return relation_data
-
-
-@cached
-def relations_of_type(reltype=None):
-    """Get relations of a specific type"""
-    relation_data = []
-    reltype = reltype or relation_type()
-    for relid in relation_ids(reltype):
-        for relation in relations_for_id(relid):
-            relation['__relid__'] = relid
-            relation_data.append(relation)
-    return relation_data
-
-
-@cached
-def metadata():
-    """Get the current charm metadata.yaml contents as a python object"""
-    with open(os.path.join(charm_dir(), 'metadata.yaml')) as md:
-        return yaml.safe_load(md)
-
-
-@cached
-def relation_types():
-    """Get a list of relation types supported by this charm"""
-    rel_types = []
-    md = metadata()
-    for key in ('provides', 'requires', 'peers'):
-        section = md.get(key)
-        if section:
-            rel_types.extend(section.keys())
-    return rel_types
-
-
-@cached
-def charm_name():
-    """Get the name of the current charm as is specified on metadata.yaml"""
-    return metadata().get('name')
-
-
-@cached
-def relations():
-    """Get a nested dictionary of relation data for all related units"""
-    rels = {}
-    for reltype in relation_types():
-        relids = {}
-        for relid in relation_ids(reltype):
-            units = {local_unit(): relation_get(unit=local_unit(), rid=relid)}
-            for unit in related_units(relid):
-                reldata = relation_get(unit=unit, rid=relid)
-                units[unit] = reldata
-            relids[relid] = units
-        rels[reltype] = relids
-    return rels
-
-
-@cached
-def is_relation_made(relation, keys='private-address'):
-    '''
-    Determine whether a relation is established by checking for
-    presence of key(s).  If a list of keys is provided, they
-    must all be present for the relation to be identified as made
-    '''
-    if isinstance(keys, str):
-        keys = [keys]
-    for r_id in relation_ids(relation):
-        for unit in related_units(r_id):
-            context = {}
-            for k in keys:
-                context[k] = relation_get(k, rid=r_id,
-                                          unit=unit)
-            if None not in context.values():
-                return True
-    return False
-
-
-def open_port(port, protocol="TCP"):
-    """Open a service network port"""
-    _args = ['open-port']
-    _args.append('{}/{}'.format(port, protocol))
-    subprocess.check_call(_args)
-
-
-def close_port(port, protocol="TCP"):
-    """Close a service network port"""
-    _args = ['close-port']
-    _args.append('{}/{}'.format(port, protocol))
-    subprocess.check_call(_args)
-
-
-@cached
-def unit_get(attribute):
-    """Get the unit ID for the remote unit"""
-    _args = ['unit-get', '--format=json', attribute]
-    try:
-        return json.loads(subprocess.check_output(_args).decode('UTF-8'))
-    except ValueError:
-        return None
-
-
-def unit_private_ip():
-    """Get this unit's private IP address"""
-    return unit_get('private-address')
-
-
-class UnregisteredHookError(Exception):
-    """Raised when an undefined hook is called"""
-    pass
-
-
-class Hooks(object):
-    """A convenient handler for hook functions.
-
-    Example::
-
-        hooks = Hooks()
-
-        # register a hook, taking its name from the function name
-        @hooks.hook()
-        def install():
-            pass  # your code here
-
-        # register a hook, providing a custom hook name
-        @hooks.hook("config-changed")
-        def config_changed():
-            pass  # your code here
-
-        if __name__ == "__main__":
-            # execute a hook based on the name the program is called by
-            hooks.execute(sys.argv)
-    """
-
-    def __init__(self, config_save=True):
-        super(Hooks, self).__init__()
-        self._hooks = {}
-        self._config_save = config_save
-
-    def register(self, name, function):
-        """Register a hook"""
-        self._hooks[name] = function
-
-    def execute(self, args):
-        """Execute a registered hook based on args[0]"""
-        hook_name = os.path.basename(args[0])
-        if hook_name in self._hooks:
-            self._hooks[hook_name]()
-            if self._config_save:
-                cfg = config()
-                if cfg.implicit_save:
-                    cfg.save()
-        else:
-            raise UnregisteredHookError(hook_name)
-
-    def hook(self, *hook_names):
-        """Decorator, registering them as hooks"""
-        def wrapper(decorated):
-            for hook_name in hook_names:
-                self.register(hook_name, decorated)
-            else:
-                self.register(decorated.__name__, decorated)
-                if '_' in decorated.__name__:
-                    self.register(
-                        decorated.__name__.replace('_', '-'), decorated)
-            return decorated
-        return wrapper
-
-
-def charm_dir():
-    """Return the root directory of the current charm"""
-    return os.environ.get('CHARM_DIR')
 
=== removed file 'hooks/charmhelpers/core/host.py'
--- hooks/charmhelpers/core/host.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/host.py	1970-01-01 00:00:00 +0000
@@ -1,446 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-"""Tools for working with the host system"""
-# Copyright 2012 Canonical Ltd.
-#
-# Authors:
-#  Nick Moffitt <[e-mail]>
-#  Matthew Wedgwood <[e-mail]>
-
-import os
-import re
-import pwd
-import grp
-import random
-import string
-import subprocess
-import hashlib
-from contextlib import contextmanager
-from collections import OrderedDict
-
-import six
-
-from .hookenv import log
-from .fstab import Fstab
-
-
-def service_start(service_name):
-    """Start a system service"""
-    return service('start', service_name)
-
-
-def service_stop(service_name):
-    """Stop a system service"""
-    return service('stop', service_name)
-
-
-def service_restart(service_name):
-    """Restart a system service"""
-    return service('restart', service_name)
-
-
-def service_reload(service_name, restart_on_failure=False):
-    """Reload a system service, optionally falling back to restart if
-    reload fails"""
-    service_result = service('reload', service_name)
-    if not service_result and restart_on_failure:
-        service_result = service('restart', service_name)
-    return service_result
-
-
-def service(action, service_name):
-    """Control a system service"""
-    cmd = ['service', service_name, action]
-    return subprocess.call(cmd) == 0
-
-
-def service_running(service):
-    """Determine whether a system service is running"""
-    try:
-        output = subprocess.check_output(
-            ['service', service, 'status'],
-            stderr=subprocess.STDOUT).decode('UTF-8')
-    except subprocess.CalledProcessError:
-        return False
-    else:
-        if ("start/running" in output or "is running" in output):
-            return True
-        else:
-            return False
-
-
-def service_available(service_name):
-    """Determine whether a system service is available"""
-    try:
-        subprocess.check_output(
-            ['service', service_name, 'status'],
-            stderr=subprocess.STDOUT).decode('UTF-8')
-    except subprocess.CalledProcessError as e:
-        return 'unrecognized service' not in e.output
-    else:
-        return True
-
-
-def adduser(username, password=None, shell='/bin/bash', system_user=False):
-    """Add a user to the system"""
-    try:
-        user_info = pwd.getpwnam(username)
-        log('user {0} already exists!'.format(username))
-    except KeyError:
-        log('creating user {0}'.format(username))
-        cmd = ['useradd']
-        if system_user or password is None:
-            cmd.append('--system')
-        else:
-            cmd.extend([
-                '--create-home',
-                '--shell', shell,
-                '--password', password,
-            ])
-        cmd.append(username)
-        subprocess.check_call(cmd)
-        user_info = pwd.getpwnam(username)
-    return user_info
-
-
-def add_group(group_name, system_group=False):
-    """Add a group to the system"""
-    try:
-        group_info = grp.getgrnam(group_name)
-        log('group {0} already exists!'.format(group_name))
-    except KeyError:
-        log('creating group {0}'.format(group_name))
-        cmd = ['addgroup']
-        if system_group:
-            cmd.append('--system')
-        else:
-            cmd.extend([
-                '--group',
-            ])
-        cmd.append(group_name)
-        subprocess.check_call(cmd)
-        group_info = grp.getgrnam(group_name)
-    return group_info
-
-
-def add_user_to_group(username, group):
-    """Add a user to a group"""
-    cmd = [
-        'gpasswd', '-a',
-        username,
-        group
-    ]
-    log("Adding user {} to group {}".format(username, group))
-    subprocess.check_call(cmd)
-
-
-def rsync(from_path, to_path, flags='-r', options=None):
-    """Replicate the contents of a path"""
-    options = options or ['--delete', '--executability']
-    cmd = ['/usr/bin/rsync', flags]
-    cmd.extend(options)
-    cmd.append(from_path)
-    cmd.append(to_path)
-    log(" ".join(cmd))
-    return subprocess.check_output(cmd).decode('UTF-8').strip()
-
-
-def symlink(source, destination):
-    """Create a symbolic link"""
-    log("Symlinking {} as {}".format(source, destination))
-    cmd = [
-        'ln',
-        '-sf',
-        source,
-        destination,
-    ]
-    subprocess.check_call(cmd)
-
-
-def mkdir(path, owner='root', group='root', perms=0o555, force=False):
-    """Create a directory"""
-    log("Making dir {} {}:{} {:o}".format(path, owner, group,
-                                          perms))
-    uid = pwd.getpwnam(owner).pw_uid
-    gid = grp.getgrnam(group).gr_gid
-    realpath = os.path.abspath(path)
-    path_exists = os.path.exists(realpath)
-    if path_exists and force:
-        if not os.path.isdir(realpath):
-            log("Removing non-directory file {} prior to mkdir()".format(path))
-            os.unlink(realpath)
-            os.makedirs(realpath, perms)
-    elif not path_exists:
-        os.makedirs(realpath, perms)
-    os.chown(realpath, uid, gid)
-    os.chmod(realpath, perms)
-
-
-def write_file(path, content, owner='root', group='root', perms=0o444):
-    """Create or overwrite a file with the contents of a byte string."""
-    log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
-    uid = pwd.getpwnam(owner).pw_uid
-    gid = grp.getgrnam(group).gr_gid
-    with open(path, 'wb') as target:
-        os.fchown(target.fileno(), uid, gid)
-        os.fchmod(target.fileno(), perms)
-        target.write(content)
-
-
-def fstab_remove(mp):
-    """Remove the given mountpoint entry from /etc/fstab
-    """
-    return Fstab.remove_by_mountpoint(mp)
-
-
-def fstab_add(dev, mp, fs, options=None):
-    """Adds the given device entry to the /etc/fstab file
-    """
-    return Fstab.add(dev, mp, fs, options=options)
-
-
-def mount(device, mountpoint, options=None, persist=False, filesystem="ext3"):
-    """Mount a filesystem at a particular mountpoint"""
-    cmd_args = ['mount']
-    if options is not None:
-        cmd_args.extend(['-o', options])
-    cmd_args.extend([device, mountpoint])
-    try:
-        subprocess.check_output(cmd_args)
-    except subprocess.CalledProcessError as e:
-        log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output))
-        return False
-
-    if persist:
-        return fstab_add(device, mountpoint, filesystem, options=options)
-    return True
-
-
-def umount(mountpoint, persist=False):
-    """Unmount a filesystem"""
-    cmd_args = ['umount', mountpoint]
-    try:
-        subprocess.check_output(cmd_args)
-    except subprocess.CalledProcessError as e:
-        log('Error unmounting {}\n{}'.format(mountpoint, e.output))
-        return False
-
-    if persist:
-        return fstab_remove(mountpoint)
-    return True
-
-
-def mounts():
-    """Get a list of all mounted volumes as [[mountpoint,device],[...]]"""
-    with open('/proc/mounts') as f:
-        # [['/mount/point','/dev/path'],[...]]
-        system_mounts = [m[1::-1] for m in [l.strip().split()
-                                            for l in f.readlines()]]
-    return system_mounts
-
-
-def file_hash(path, hash_type='md5'):
-    """
-    Generate a hash checksum of the contents of 'path' or None if not found.
-
-    :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,
-                          such as md5, sha1, sha256, sha512, etc.
-    """
-    if os.path.exists(path):
-        h = getattr(hashlib, hash_type)()
-        with open(path, 'rb') as source:
-            h.update(source.read())
-        return h.hexdigest()
-    else:
-        return None
-
-
-def check_hash(path, checksum, hash_type='md5'):
-    """
-    Validate a file using a cryptographic checksum.
-
-    :param str checksum: Value of the checksum used to validate the file.
-    :param str hash_type: Hash algorithm used to generate `checksum`.
-        Can be any hash alrgorithm supported by :mod:`hashlib`,
-        such as md5, sha1, sha256, sha512, etc.
-    :raises ChecksumError: If the file fails the checksum
-
-    """
-    actual_checksum = file_hash(path, hash_type)
-    if checksum != actual_checksum:
-        raise ChecksumError("'%s' != '%s'" % (checksum, actual_checksum))
-
-
-class ChecksumError(ValueError):
-    pass
-
-
-def restart_on_change(restart_map, stopstart=False):
-    """Restart services based on configuration files changing
-
-    This function is used a decorator, for example::
-
-        @restart_on_change({
-            '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]
-            })
-        def ceph_client_changed():
-            pass  # your code here
-
-    In this example, the cinder-api and cinder-volume services
-    would be restarted if /etc/ceph/ceph.conf is changed by the
-    ceph_client_changed function.
-    """
-    def wrap(f):
-        def wrapped_f(*args, **kwargs):
-            checksums = {}
-            for path in restart_map:
-                checksums[path] = file_hash(path)
-            f(*args, **kwargs)
-            restarts = []
-            for path in restart_map:
-                if checksums[path] != file_hash(path):
-                    restarts += restart_map[path]
-            services_list = list(OrderedDict.fromkeys(restarts))
-            if not stopstart:
-                for service_name in services_list:
-                    service('restart', service_name)
-            else:
-                for action in ['stop', 'start']:
-                    for service_name in services_list:
-                        service(action, service_name)
-        return wrapped_f
-    return wrap
-
-
-def lsb_release():
-    """Return /etc/lsb-release in a dict"""
-    d = {}
-    with open('/etc/lsb-release', 'r') as lsb:
-        for l in lsb:
-            k, v = l.split('=')
-            d[k.strip()] = v.strip()
-    return d
-
-
-def pwgen(length=None):
-    """Generate a random pasword."""
-    if length is None:
-        length = random.choice(range(35, 45))
-    alphanumeric_chars = [
-        l for l in (string.ascii_letters + string.digits)
-        if l not in 'l0QD1vAEIOUaeiou']
-    random_chars = [
-        random.choice(alphanumeric_chars) for _ in range(length)]
-    return(''.join(random_chars))
-
-
-def list_nics(nic_type):
-    '''Return a list of nics of given type(s)'''
-    if isinstance(nic_type, six.string_types):
-        int_types = [nic_type]
-    else:
-        int_types = nic_type
-    interfaces = []
-    for int_type in int_types:
-        cmd = ['ip', 'addr', 'show', 'label', int_type + '*']
-        ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
-        ip_output = (line for line in ip_output if line)
-        for line in ip_output:
-            if line.split()[1].startswith(int_type):
-                matched = re.search('.*: (' + int_type + r'[0-9]+\.[0-9]+)@.*', line)
-                if matched:
-                    interface = matched.groups()[0]
-                else:
-                    interface = line.split()[1].replace(":", "")
-                interfaces.append(interface)
-
-    return interfaces
-
-
-def set_nic_mtu(nic, mtu):
-    '''Set MTU on a network interface'''
-    cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
-    subprocess.check_call(cmd)
-
-
-def get_nic_mtu(nic):
-    cmd = ['ip', 'addr', 'show', nic]
-    ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
-    mtu = ""
-    for line in ip_output:
-        words = line.split()
-        if 'mtu' in words:
-            mtu = words[words.index("mtu") + 1]
-    return mtu
-
-
-def get_nic_hwaddr(nic):
-    cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
-    ip_output = subprocess.check_output(cmd).decode('UTF-8')
-    hwaddr = ""
-    words = ip_output.split()
-    if 'link/ether' in words:
-        hwaddr = words[words.index('link/ether') + 1]
-    return hwaddr
-
-
-def cmp_pkgrevno(package, revno, pkgcache=None):
-    '''Compare supplied revno with the revno of the installed package
-
-    *  1 => Installed revno is greater than supplied arg
-    *  0 => Installed revno is the same as supplied arg
-    * -1 => Installed revno is less than supplied arg
-
-    This function imports apt_cache function from charmhelpers.fetch if
-    the pkgcache argument is None. Be sure to add charmhelpers.fetch if
-    you call this function, or pass an apt_pkg.Cache() instance.
-    '''
-    import apt_pkg
-    if not pkgcache:
-        from charmhelpers.fetch import apt_cache
-        pkgcache = apt_cache()
-    pkg = pkgcache[package]
-    return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
-
-
-@contextmanager
-def chdir(d):
-    cur = os.getcwd()
-    try:
-        yield os.chdir(d)
-    finally:
-        os.chdir(cur)
-
-
-def chownr(path, owner, group, follow_links=True):
-    uid = pwd.getpwnam(owner).pw_uid
-    gid = grp.getgrnam(group).gr_gid
-    if follow_links:
-        chown = os.chown
-    else:
-        chown = os.lchown
-
-    for root, dirs, files in os.walk(path):
-        for name in dirs + files:
-            full = os.path.join(root, name)
-            broken_symlink = os.path.lexists(full) and not os.path.exists(full)
-            if not broken_symlink:
-                chown(full, uid, gid)
-
-
-def lchownr(path, owner, group):
-    chownr(path, owner, group, follow_links=False)
 
=== removed directory 'hooks/charmhelpers/core/services'
=== removed file 'hooks/charmhelpers/core/services/__init__.py'
--- hooks/charmhelpers/core/services/__init__.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/services/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,18 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from .base import *  # NOQA
-from .helpers import *  # NOQA
 
=== removed file 'hooks/charmhelpers/core/services/base.py'
--- hooks/charmhelpers/core/services/base.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/services/base.py	1970-01-01 00:00:00 +0000
@@ -1,329 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-import re
-import json
-from collections import Iterable
-
-from charmhelpers.core import host
-from charmhelpers.core import hookenv
-
-
-__all__ = ['ServiceManager', 'ManagerCallback',
-           'PortManagerCallback', 'open_ports', 'close_ports', 'manage_ports',
-           'service_restart', 'service_stop']
-
-
-class ServiceManager(object):
-    def __init__(self, services=None):
-        """
-        Register a list of services, given their definitions.
-
-        Service definitions are dicts in the following formats (all keys except
-        'service' are optional)::
-
-            {
-                "service": <service name>,
-                "required_data": <list of required data contexts>,
-                "provided_data": <list of provided data contexts>,
-                "data_ready": <one or more callbacks>,
-                "data_lost": <one or more callbacks>,
-                "start": <one or more callbacks>,
-                "stop": <one or more callbacks>,
-                "ports": <list of ports to manage>,
-            }
-
-        The 'required_data' list should contain dicts of required data (or
-        dependency managers that act like dicts and know how to collect the data).
-        Only when all items in the 'required_data' list are populated are the list
-        of 'data_ready' and 'start' callbacks executed.  See `is_ready()` for more
-        information.
-
-        The 'provided_data' list should contain relation data providers, most likely
-        a subclass of :class:`charmhelpers.core.services.helpers.RelationContext`,
-        that will indicate a set of data to set on a given relation.
-
-        The 'data_ready' value should be either a single callback, or a list of
-        callbacks, to be called when all items in 'required_data' pass `is_ready()`.
-        Each callback will be called with the service name as the only parameter.
-        After all of the 'data_ready' callbacks are called, the 'start' callbacks
-        are fired.
-
-        The 'data_lost' value should be either a single callback, or a list of
-        callbacks, to be called when a 'required_data' item no longer passes
-        `is_ready()`.  Each callback will be called with the service name as the
-        only parameter.  After all of the 'data_lost' callbacks are called,
-        the 'stop' callbacks are fired.
-
-        The 'start' value should be either a single callback, or a list of
-        callbacks, to be called when starting the service, after the 'data_ready'
-        callbacks are complete.  Each callback will be called with the service
-        name as the only parameter.  This defaults to
-        `[host.service_start, services.open_ports]`.
-
-        The 'stop' value should be either a single callback, or a list of
-        callbacks, to be called when stopping the service.  If the service is
-        being stopped because it no longer has all of its 'required_data', this
-        will be called after all of the 'data_lost' callbacks are complete.
-        Each callback will be called with the service name as the only parameter.
-        This defaults to `[services.close_ports, host.service_stop]`.
-
-        The 'ports' value should be a list of ports to manage.  The default
-        'start' handler will open the ports after the service is started,
-        and the default 'stop' handler will close the ports prior to stopping
-        the service.
-
-
-        Examples:
-
-        The following registers an Upstart service called bingod that depends on
-        a mongodb relation and which runs a custom `db_migrate` function prior to
-        restarting the service, and a Runit service called spadesd::
-
-            manager = services.ServiceManager([
-                {
-                    'service': 'bingod',
-                    'ports': [80, 443],
-                    'required_data': [MongoRelation(), config(), {'my': 'data'}],
-                    'data_ready': [
-                        services.template(source='bingod.conf'),
-                        services.template(source='bingod.ini',
-                                          target='/etc/bingod.ini',
-                                          owner='bingo', perms=0400),
-                    ],
-                },
-                {
-                    'service': 'spadesd',
-                    'data_ready': services.template(source='spadesd_run.j2',
-                                                    target='/etc/sv/spadesd/run',
-                                                    perms=0555),
-                    'start': runit_start,
-                    'stop': runit_stop,
-                },
-            ])
-            manager.manage()
-        """
-        self._ready_file = os.path.join(hookenv.charm_dir(), 'READY-SERVICES.json')
-        self._ready = None
-        self.services = {}
-        for service in services or []:
-            service_name = service['service']
-            self.services[service_name] = service
-
-    def manage(self):
-        """
-        Handle the current hook by doing The Right Thing with the registered services.
-        """
-        hook_name = hookenv.hook_name()
-        if hook_name == 'stop':
-            self.stop_services()
-        else:
-            self.provide_data()
-            self.reconfigure_services()
-        cfg = hookenv.config()
-        if cfg.implicit_save:
-            cfg.save()
-
-    def provide_data(self):
-        """
-        Set the relation data for each provider in the ``provided_data`` list.
-
-        A provider must have a `name` attribute, which indicates which relation
-        to set data on, and a `provide_data()` method, which returns a dict of
-        data to set.
-        """
-        hook_name = hookenv.hook_name()
-        for service in self.services.values():
-            for provider in service.get('provided_data', []):
-                if re.match(r'{}-relation-(joined|changed)'.format(provider.name), hook_name):
-                    data = provider.provide_data()
-                    _ready = provider._is_ready(data) if hasattr(provider, '_is_ready') else data
-                    if _ready:
-                        hookenv.relation_set(None, data)
-
-    def reconfigure_services(self, *service_names):
-        """
-        Update all files for one or more registered services, and,
-        if ready, optionally restart them.
-
-        If no service names are given, reconfigures all registered services.
-        """
-        for service_name in service_names or self.services.keys():
-            if self.is_ready(service_name):
-                self.fire_event('data_ready', service_name)
-                self.fire_event('start', service_name, default=[
-                    service_restart,
-                    manage_ports])
-                self.save_ready(service_name)
-            else:
-                if self.was_ready(service_name):
-                    self.fire_event('data_lost', service_name)
-                self.fire_event('stop', service_name, default=[
-                    manage_ports,
-                    service_stop])
-                self.save_lost(service_name)
-
-    def stop_services(self, *service_names):
-        """
-        Stop one or more registered services, by name.
-
-        If no service names are given, stops all registered services.
-        """
-        for service_name in service_names or self.services.keys():
-            self.fire_event('stop', service_name, default=[
-                manage_ports,
-                service_stop])
-
-    def get_service(self, service_name):
-        """
-        Given the name of a registered service, return its service definition.
-        """
-        service = self.services.get(service_name)
-        if not service:
-            raise KeyError('Service not registered: %s' % service_name)
-        return service
-
-    def fire_event(self, event_name, service_name, default=None):
-        """
-        Fire a data_ready, data_lost, start, or stop event on a given service.
-        """
-        service = self.get_service(service_name)
-        callbacks = service.get(event_name, default)
-        if not callbacks:
-            return
-        if not isinstance(callbacks, Iterable):
-            callbacks = [callbacks]
-        for callback in callbacks:
-            if isinstance(callback, ManagerCallback):
-                callback(self, service_name, event_name)
-            else:
-                callback(service_name)
-
-    def is_ready(self, service_name):
-        """
-        Determine if a registered service is ready, by checking its 'required_data'.
-
-        A 'required_data' item can be any mapping type, and is considered ready
-        if `bool(item)` evaluates as True.
-        """
-        service = self.get_service(service_name)
-        reqs = service.get('required_data', [])
-        return all(bool(req) for req in reqs)
-
-    def _load_ready_file(self):
-        if self._ready is not None:
-            return
-        if os.path.exists(self._ready_file):
-            with open(self._ready_file) as fp:
-                self._ready = set(json.load(fp))
-        else:
-            self._ready = set()
-
-    def _save_ready_file(self):
-        if self._ready is None:
-            return
-        with open(self._ready_file, 'w') as fp:
-            json.dump(list(self._ready), fp)
-
-    def save_ready(self, service_name):
-        """
-        Save an indicator that the given service is now data_ready.
-        """
-        self._load_ready_file()
-        self._ready.add(service_name)
-        self._save_ready_file()
-
-    def save_lost(self, service_name):
-        """
-        Save an indicator that the given service is no longer data_ready.
-        """
-        self._load_ready_file()
-        self._ready.discard(service_name)
-        self._save_ready_file()
-
-    def was_ready(self, service_name):
-        """
-        Determine if the given service was previously data_ready.
-        """
-        self._load_ready_file()
-        return service_name in self._ready
-
-
-class ManagerCallback(object):
-    """
-    Special case of a callback that takes the `ServiceManager` instance
-    in addition to the service name.
-
-    Subclasses should implement `__call__` which should accept three parameters:
-
-        * `manager`       The `ServiceManager` instance
-        * `service_name`  The name of the service it's being triggered for
-        * `event_name`    The name of the event that this callback is handling
-    """
-    def __call__(self, manager, service_name, event_name):
-        raise NotImplementedError()
-
-
-class PortManagerCallback(ManagerCallback):
-    """
-    Callback class that will open or close ports, for use as either
-    a start or stop action.
-    """
-    def __call__(self, manager, service_name, event_name):
-        service = manager.get_service(service_name)
-        new_ports = service.get('ports', [])
-        port_file = os.path.join(hookenv.charm_dir(), '.{}.ports'.format(service_name))
-        if os.path.exists(port_file):
-            with open(port_file) as fp:
-                old_ports = fp.read().split(',')
-            for old_port in old_ports:
-                if bool(old_port):
-                    old_port = int(old_port)
-                    if old_port not in new_ports:
-                        hookenv.close_port(old_port)
-        with open(port_file, 'w') as fp:
-            fp.write(','.join(str(port) for port in new_ports))
-        for port in new_ports:
-            if event_name == 'start':
-                hookenv.open_port(port)
-            elif event_name == 'stop':
-                hookenv.close_port(port)
-
-
-def service_stop(service_name):
-    """
-    Wrapper around host.service_stop to prevent spurious "unknown service"
-    messages in the logs.
-    """
-    if host.service_running(service_name):
-        host.service_stop(service_name)
-
-
-def service_restart(service_name):
-    """
-    Wrapper around host.service_restart to prevent spurious "unknown service"
-    messages in the logs.
-    """
-    if host.service_available(service_name):
-        if host.service_running(service_name):
-            host.service_restart(service_name)
-        else:
-            host.service_start(service_name)
-
-
-# Convenience aliases
-open_ports = close_ports = manage_ports = PortManagerCallback()
 
=== removed file 'hooks/charmhelpers/core/services/helpers.py'
--- hooks/charmhelpers/core/services/helpers.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/services/helpers.py	1970-01-01 00:00:00 +0000
@@ -1,259 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-import yaml
-from charmhelpers.core import hookenv
-from charmhelpers.core import templating
-
-from charmhelpers.core.services.base import ManagerCallback
-
-
-__all__ = ['RelationContext', 'TemplateCallback',
-           'render_template', 'template']
-
-
-class RelationContext(dict):
-    """
-    Base class for a context generator that gets relation data from juju.
-
-    Subclasses must provide the attributes `name`, which is the name of the
-    interface of interest, `interface`, which is the type of the interface of
-    interest, and `required_keys`, which is the set of keys required for the
-    relation to be considered complete.  The data for all interfaces matching
-    the `name` attribute that are complete will used to populate the dictionary
-    values (see `get_data`, below).
-
-    The generated context will be namespaced under the relation :attr:`name`,
-    to prevent potential naming conflicts.
-
-    :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
-    :param list additional_required_keys: Extend the list of :attr:`required_keys`
-    """
-    name = None
-    interface = None
-    required_keys = []
-
-    def __init__(self, name=None, additional_required_keys=None):
-        if name is not None:
-            self.name = name
-        if additional_required_keys is not None:
-            self.required_keys.extend(additional_required_keys)
-        self.get_data()
-
-    def __bool__(self):
-        """
-        Returns True if all of the required_keys are available.
-        """
-        return self.is_ready()
-
-    __nonzero__ = __bool__
-
-    def __repr__(self):
-        return super(RelationContext, self).__repr__()
-
-    def is_ready(self):
-        """
-        Returns True if all of the `required_keys` are available from any units.
-        """
-        ready = len(self.get(self.name, [])) > 0
-        if not ready:
-            hookenv.log('Incomplete relation: {}'.format(self.__class__.__name__), hookenv.DEBUG)
-        return ready
-
-    def _is_ready(self, unit_data):
-        """
-        Helper method that tests a set of relation data and returns True if
-        all of the `required_keys` are present.
-        """
-        return set(unit_data.keys()).issuperset(set(self.required_keys))
-
-    def get_data(self):
-        """
-        Retrieve the relation data for each unit involved in a relation and,
-        if complete, store it in a list under `self[self.name]`.  This
-        is automatically called when the RelationContext is instantiated.
-
-        The units are sorted lexographically first by the service ID, then by
-        the unit ID.  Thus, if an interface has two other services, 'db:1'
-        and 'db:2', with 'db:1' having two units, 'wordpress/0' and 'wordpress/1',
-        and 'db:2' having one unit, 'mediawiki/0', all of which have a complete
-        set of data, the relation data for the units will be stored in the
-        order: 'wordpress/0', 'wordpress/1', 'mediawiki/0'.
-
-        If you only care about a single unit on the relation, you can just
-        access it as `{{ interface[0]['key'] }}`.  However, if you can at all
-        support multiple units on a relation, you should iterate over the list,
-        like::
-
-            {% for unit in interface -%}
-                {{ unit['key'] }}{% if not loop.last %},{% endif %}
-            {%- endfor %}
-
-        Note that since all sets of relation data from all related services and
-        units are in a single list, if you need to know which service or unit a
-        set of data came from, you'll need to extend this class to preserve
-        that information.
-        """
-        if not hookenv.relation_ids(self.name):
-            return
-
-        ns = self.setdefault(self.name, [])
-        for rid in sorted(hookenv.relation_ids(self.name)):
-            for unit in sorted(hookenv.related_units(rid)):
-                reldata = hookenv.relation_get(rid=rid, unit=unit)
-                if self._is_ready(reldata):
-                    ns.append(reldata)
-
-    def provide_data(self):
-        """
-        Return data to be relation_set for this interface.
-        """
-        return {}
-
-
-class MysqlRelation(RelationContext):
-    """
-    Relation context for the `mysql` interface.
-
-    :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
-    :param list additional_required_keys: Extend the list of :attr:`required_keys`
-    """
-    name = 'db'
-    interface = 'mysql'
-    required_keys = ['host', 'user', 'password', 'database']
-
-
-class HttpRelation(RelationContext):
-    """
-    Relation context for the `http` interface.
-
-    :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
-    :param list additional_required_keys: Extend the list of :attr:`required_keys`
-    """
-    name = 'website'
-    interface = 'http'
-    required_keys = ['host', 'port']
-
-    def provide_data(self):
-        return {
-            'host': hookenv.unit_get('private-address'),
-            'port': 80,
-        }
-
-
-class RequiredConfig(dict):
-    """
-    Data context that loads config options with one or more mandatory options.
-
-    Once the required options have been changed from their default values, all
-    config options will be available, namespaced under `config` to prevent
-    potential naming conflicts (for example, between a config option and a
-    relation property).
-
-    :param list *args: List of options that must be changed from their default values.
-    """
-
-    def __init__(self, *args):
-        self.required_options = args
-        self['config'] = hookenv.config()
-        with open(os.path.join(hookenv.charm_dir(), 'config.yaml')) as fp:
-            self.config = yaml.load(fp).get('options', {})
-
-    def __bool__(self):
-        for option in self.required_options:
-            if option not in self['config']:
-                return False
-            current_value = self['config'][option]
-            default_value = self.config[option].get('default')
-            if current_value == default_value:
-                return False
-            if current_value in (None, '') and default_value in (None, ''):
-                return False
-        return True
-
-    def __nonzero__(self):
-        return self.__bool__()
-
-
-class StoredContext(dict):
-    """
-    A data context that always returns the data that it was first created with.
-
-    This is useful to do a one-time generation of things like passwords, that
-    will thereafter use the same value that was originally generated, instead
-    of generating a new value each time it is run.
-    """
-    def __init__(self, file_name, config_data):
-        """
-        If the file exists, populate `self` with the data from the file.
-        Otherwise, populate with the given data and persist it to the file.
-        """
-        if os.path.exists(file_name):
-            self.update(self.read_context(file_name))
-        else:
-            self.store_context(file_name, config_data)
-            self.update(config_data)
-
-    def store_context(self, file_name, config_data):
-        if not os.path.isabs(file_name):
-            file_name = os.path.join(hookenv.charm_dir(), file_name)
-        with open(file_name, 'w') as file_stream:
-            os.fchmod(file_stream.fileno(), 0o600)
-            yaml.dump(config_data, file_stream)
-
-    def read_context(self, file_name):
-        if not os.path.isabs(file_name):
-            file_name = os.path.join(hookenv.charm_dir(), file_name)
-        with open(file_name, 'r') as file_stream:
-            data = yaml.load(file_stream)
-            if not data:
-                raise OSError("%s is empty" % file_name)
-            return data
-
-
-class TemplateCallback(ManagerCallback):
-    """
-    Callback class that will render a Jinja2 template, for use as a ready
-    action.
-
-    :param str source: The template source file, relative to
-    `$CHARM_DIR/templates`
-
-    :param str target: The target to write the rendered template to
-    :param str owner: The owner of the rendered file
-    :param str group: The group of the rendered file
-    :param int perms: The permissions of the rendered file
-    """
-    def __init__(self, source, target,
-                 owner='root', group='root', perms=0o444):
-        self.source = source
-        self.target = target
-        self.owner = owner
-        self.group = group
-        self.perms = perms
-
-    def __call__(self, manager, service_name, event_name):
-        service = manager.get_service(service_name)
-        context = {}
-        for ctx in service.get('required_data', []):
-            context.update(ctx)
-        templating.render(self.source, self.target, context,
-                          self.owner, self.group, self.perms)
-
-
-# Convenience aliases for templates
-render_template = template = TemplateCallback
 
=== removed file 'hooks/charmhelpers/core/strutils.py'
--- hooks/charmhelpers/core/strutils.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/strutils.py	1970-01-01 00:00:00 +0000
@@ -1,42 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import six
-
-
-def bool_from_string(value):
-    """Interpret string value as boolean.
-
-    Returns True if value translates to True otherwise False.
-    """
-    if isinstance(value, six.string_types):
-        value = six.text_type(value)
-    else:
-        msg = "Unable to interpret non-string value '%s' as boolean" % (value)
-        raise ValueError(msg)
-
-    value = value.strip().lower()
-
-    if value in ['y', 'yes', 'true', 't']:
-        return True
-    elif value in ['n', 'no', 'false', 'f']:
-        return False
-
-    msg = "Unable to interpret string value '%s' as boolean" % (value)
-    raise ValueError(msg)
 
=== removed file 'hooks/charmhelpers/core/sysctl.py'
--- hooks/charmhelpers/core/sysctl.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/sysctl.py	1970-01-01 00:00:00 +0000
@@ -1,56 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import yaml
-
-from subprocess import check_call
-
-from charmhelpers.core.hookenv import (
-    log,
-    DEBUG,
-    ERROR,
-)
-
-__author__ = 'Jorge Niedbalski R. <[e-mail]>'
-
-
-def create(sysctl_dict, sysctl_file):
-    """Creates a sysctl.conf file from a YAML associative array
-
-    :param sysctl_dict: a YAML-formatted string of sysctl options eg "{ 'kernel.max_pid': 1337 }"
-    :type sysctl_dict: str
-    :param sysctl_file: path to the sysctl file to be saved
-    :type sysctl_file: str or unicode
-    :returns: None
-    """
-    try:
-        sysctl_dict_parsed = yaml.safe_load(sysctl_dict)
-    except yaml.YAMLError:
-        log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict),
-            level=ERROR)
-        return
-
-    with open(sysctl_file, "w") as fd:
-        for key, value in sysctl_dict_parsed.items():
-            fd.write("{}={}\n".format(key, value))
-
-    log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict_parsed),
-        level=DEBUG)
-
-    check_call(["sysctl", "-p", sysctl_file])
 
=== removed file 'hooks/charmhelpers/core/templating.py'
--- hooks/charmhelpers/core/templating.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/templating.py	1970-01-01 00:00:00 +0000
@@ -1,68 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-
-from charmhelpers.core import host
-from charmhelpers.core import hookenv
-
-
-def render(source, target, context, owner='root', group='root',
-           perms=0o444, templates_dir=None, encoding='UTF-8'):
-    """
-    Render a template.
-
-    The `source` path, if not absolute, is relative to the `templates_dir`.
-
-    The `target` path should be absolute.
-
-    The context should be a dict containing the values to be replaced in the
-    template.
-
-    The `owner`, `group`, and `perms` options will be passed to `write_file`.
-
-    If omitted, `templates_dir` defaults to the `templates` folder in the charm.
-
-    Note: Using this requires python-jinja2; if it is not installed, calling
-    this will attempt to use charmhelpers.fetch.apt_install to install it.
-    """
-    try:
-        from jinja2 import FileSystemLoader, Environment, exceptions
-    except ImportError:
-        try:
-            from charmhelpers.fetch import apt_install
-        except ImportError:
-            hookenv.log('Could not import jinja2, and could not import '
-                        'charmhelpers.fetch to install it',
-                        level=hookenv.ERROR)
-            raise
-        apt_install('python-jinja2', fatal=True)
-        from jinja2 import FileSystemLoader, Environment, exceptions
-
-    if templates_dir is None:
-        templates_dir = os.path.join(hookenv.charm_dir(), 'templates')
-    loader = Environment(loader=FileSystemLoader(templates_dir))
-    try:
-        source = source
-        template = loader.get_template(source)
-    except exceptions.TemplateNotFound as e:
-        hookenv.log('Could not load template %s from %s.' %
-                    (source, templates_dir),
-                    level=hookenv.ERROR)
-        raise e
-    content = template.render(context)
-    host.mkdir(os.path.dirname(target), owner, group, perms=0o755)
-    host.write_file(target, content.encode(encoding), owner, group, perms)
 
=== removed file 'hooks/charmhelpers/core/unitdata.py'
--- hooks/charmhelpers/core/unitdata.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/core/unitdata.py	1970-01-01 00:00:00 +0000
@@ -1,477 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-#
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-#
-#
-# Authors:
-#  Kapil Thangavelu <[e-mail]>
-#
-"""
-Intro
------
-
-A simple way to store state in units. This provides a key value
-storage with support for versioned, transactional operation,
-and can calculate deltas from previous values to simplify unit logic
-when processing changes.
-
-
-Hook Integration
-----------------
-
-There are several extant frameworks for hook execution, including
-
- - charmhelpers.core.hookenv.Hooks
- - charmhelpers.core.services.ServiceManager
-
-The storage classes are framework agnostic, one simple integration is
-via the HookData contextmanager. It will record the current hook
-execution environment (including relation data, config data, etc.),
-setup a transaction and allow easy access to the changes from
-previously seen values. One consequence of the integration is the
-reservation of particular keys ('rels', 'unit', 'env', 'config',
-'charm_revisions') for their respective values.
-
-Here's a fully worked integration example using hookenv.Hooks::
-
-       from charmhelper.core import hookenv, unitdata
-
-       hook_data = unitdata.HookData()
-       db = unitdata.kv()
-       hooks = hookenv.Hooks()
-
-       @hooks.hook
-       def config_changed():
-           # Print all changes to configuration from previously seen
-           # values.
-           for changed, (prev, cur) in hook_data.conf.items():
-               print('config changed', changed,
-                     'previous value', prev,
-                     'current value',  cur)
-
-           # Get some unit specific bookeeping
-           if not db.get('pkg_key'):
-               key = urllib.urlopen('https://example.com/pkg_key').read()
-               db.set('pkg_key', key)
-
-           # Directly access all charm config as a mapping.
-           conf = db.getrange('config', True)
-
-           # Directly access all relation data as a mapping
-           rels = db.getrange('rels', True)
-
-       if __name__ == '__main__':
-           with hook_data():
-               hook.execute()
-
-
-A more basic integration is via the hook_scope context manager which simply
-manages transaction scope (and records hook name, and timestamp)::
-
-  >>> from unitdata import kv
-  >>> db = kv()
-  >>> with db.hook_scope('install'):
-  ...    # do work, in transactional scope.
-  ...    db.set('x', 1)
-  >>> db.get('x')
-  1
-
-
-Usage
------
-
-Values are automatically json de/serialized to preserve basic typing
-and complex data struct capabilities (dicts, lists, ints, booleans, etc).
-
-Individual values can be manipulated via get/set::
-
-   >>> kv.set('y', True)
-   >>> kv.get('y')
-   True
-
-   # We can set complex values (dicts, lists) as a single key.
-   >>> kv.set('config', {'a': 1, 'b': True'})
-
-   # Also supports returning dictionaries as a record which
-   # provides attribute access.
-   >>> config = kv.get('config', record=True)
-   >>> config.b
-   True
-
-
-Groups of keys can be manipulated with update/getrange::
-
-   >>> kv.update({'z': 1, 'y': 2}, prefix="gui.")
-   >>> kv.getrange('gui.', strip=True)
-   {'z': 1, 'y': 2}
-
-When updating values, its very helpful to understand which values
-have actually changed and how have they changed. The storage
-provides a delta method to provide for this::
-
-   >>> data = {'debug': True, 'option': 2}
-   >>> delta = kv.delta(data, 'config.')
-   >>> delta.debug.previous
-   None
-   >>> delta.debug.current
-   True
-   >>> delta
-   {'debug': (None, True), 'option': (None, 2)}
-
-Note the delta method does not persist the actual change, it needs to
-be explicitly saved via 'update' method::
-
-   >>> kv.update(data, 'config.')
-
-Values modified in the context of a hook scope retain historical values
-associated to the hookname.
-
-   >>> with db.hook_scope('config-changed'):
-   ...      db.set('x', 42)
-   >>> db.gethistory('x')
-   [(1, u'x', 1, u'install', u'2015-01-21T16:49:30.038372'),
-    (2, u'x', 42, u'config-changed', u'2015-01-21T16:49:30.038786')]
-
-"""
-
-import collections
-import contextlib
-import datetime
-import json
-import os
-import pprint
-import sqlite3
-import sys
-
-__author__ = 'Kapil Thangavelu <[e-mail]>'
-
-
-class Storage(object):
-    """Simple key value database for local unit state within charms.
-
-    Modifications are automatically committed at hook exit. That's
-    currently regardless of exit code.
-
-    To support dicts, lists, integer, floats, and booleans values
-    are automatically json encoded/decoded.
-    """
-    def __init__(self, path=None):
-        self.db_path = path
-        if path is None:
-            self.db_path = os.path.join(
-                os.environ.get('CHARM_DIR', ''), '.unit-state.db')
-        self.conn = sqlite3.connect('%s' % self.db_path)
-        self.cursor = self.conn.cursor()
-        self.revision = None
-        self._closed = False
-        self._init()
-
-    def close(self):
-        if self._closed:
-            return
-        self.flush(False)
-        self.cursor.close()
-        self.conn.close()
-        self._closed = True
-
-    def _scoped_query(self, stmt, params=None):
-        if params is None:
-            params = []
-        return stmt, params
-
-    def get(self, key, default=None, record=False):
-        self.cursor.execute(
-            *self._scoped_query(
-                'select data from kv where key=?', [key]))
-        result = self.cursor.fetchone()
-        if not result:
-            return default
-        if record:
-            return Record(json.loads(result[0]))
-        return json.loads(result[0])
-
-    def getrange(self, key_prefix, strip=False):
-        stmt = "select key, data from kv where key like '%s%%'" % key_prefix
-        self.cursor.execute(*self._scoped_query(stmt))
-        result = self.cursor.fetchall()
-
-        if not result:
-            return None
-        if not strip:
-            key_prefix = ''
-        return dict([
-            (k[len(key_prefix):], json.loads(v)) for k, v in result])
-
-    def update(self, mapping, prefix=""):
-        for k, v in mapping.items():
-            self.set("%s%s" % (prefix, k), v)
-
-    def unset(self, key):
-        self.cursor.execute('delete from kv where key=?', [key])
-        if self.revision and self.cursor.rowcount:
-            self.cursor.execute(
-                'insert into kv_revisions values (?, ?, ?)',
-                [key, self.revision, json.dumps('DELETED')])
-
-    def set(self, key, value):
-        serialized = json.dumps(value)
-
-        self.cursor.execute(
-            'select data from kv where key=?', [key])
-        exists = self.cursor.fetchone()
-
-        # Skip mutations to the same value
-        if exists:
-            if exists[0] == serialized:
-                return value
-
-        if not exists:
-            self.cursor.execute(
-                'insert into kv (key, data) values (?, ?)',
-                (key, serialized))
-        else:
-            self.cursor.execute('''
-            update kv
-            set data = ?
-            where key = ?''', [serialized, key])
-
-        # Save
-        if not self.revision:
-            return value
-
-        self.cursor.execute(
-            'select 1 from kv_revisions where key=? and revision=?',
-            [key, self.revision])
-        exists = self.cursor.fetchone()
-
-        if not exists:
-            self.cursor.execute(
-                '''insert into kv_revisions (
-                revision, key, data) values (?, ?, ?)''',
-                (self.revision, key, serialized))
-        else:
-            self.cursor.execute(
-                '''
-                update kv_revisions
-                set data = ?
-                where key = ?
-                and   revision = ?''',
-                [serialized, key, self.revision])
-
-        return value
-
-    def delta(self, mapping, prefix):
-        """
-        return a delta containing values that have changed.
-        """
-        previous = self.getrange(prefix, strip=True)
-        if not previous:
-            pk = set()
-        else:
-            pk = set(previous.keys())
-        ck = set(mapping.keys())
-        delta = DeltaSet()
-
-        # added
-        for k in ck.difference(pk):
-            delta[k] = Delta(None, mapping[k])
-
-        # removed
-        for k in pk.difference(ck):
-            delta[k] = Delta(previous[k], None)
-
-        # changed
-        for k in pk.intersection(ck):
-            c = mapping[k]
-            p = previous[k]
-            if c != p:
-                delta[k] = Delta(p, c)
-
-        return delta
-
-    @contextlib.contextmanager
-    def hook_scope(self, name=""):
-        """Scope all future interactions to the current hook execution
-        revision."""
-        assert not self.revision
-        self.cursor.execute(
-            'insert into hooks (hook, date) values (?, ?)',
-            (name or sys.argv[0],
-             datetime.datetime.utcnow().isoformat()))
-        self.revision = self.cursor.lastrowid
-        try:
-            yield self.revision
-            self.revision = None
-        except:
-            self.flush(False)
-            self.revision = None
-            raise
-        else:
-            self.flush()
-
-    def flush(self, save=True):
-        if save:
-            self.conn.commit()
-        elif self._closed:
-            return
-        else:
-            self.conn.rollback()
-
-    def _init(self):
-        self.cursor.execute('''
-            create table if not exists kv (
-               key text,
-               data text,
-               primary key (key)
-               )''')
-        self.cursor.execute('''
-            create table if not exists kv_revisions (
-               key text,
-               revision integer,
-               data text,
-               primary key (key, revision)
-               )''')
-        self.cursor.execute('''
-            create table if not exists hooks (
-               version integer primary key autoincrement,
-               hook text,
-               date text
-               )''')
-        self.conn.commit()
-
-    def gethistory(self, key, deserialize=False):
-        self.cursor.execute(
-            '''
-            select kv.revision, kv.key, kv.data, h.hook, h.date
-            from kv_revisions kv,
-                 hooks h
-            where kv.key=?
-             and kv.revision = h.version
-            ''', [key])
-        if deserialize is False:
-            return self.cursor.fetchall()
-        return map(_parse_history, self.cursor.fetchall())
-
-    def debug(self, fh=sys.stderr):
-        self.cursor.execute('select * from kv')
-        pprint.pprint(self.cursor.fetchall(), stream=fh)
-        self.cursor.execute('select * from kv_revisions')
-        pprint.pprint(self.cursor.fetchall(), stream=fh)
-
-
-def _parse_history(d):
-    return (d[0], d[1], json.loads(d[2]), d[3],
-            datetime.datetime.strptime(d[-1], "%Y-%m-%dT%H:%M:%S.%f"))
-
-
-class HookData(object):
-    """Simple integration for existing hook exec frameworks.
-
-    Records all unit information, and stores deltas for processing
-    by the hook.
-
-    Sample::
-
-       from charmhelper.core import hookenv, unitdata
-
-       changes = unitdata.HookData()
-       db = unitdata.kv()
-       hooks = hookenv.Hooks()
-
-       @hooks.hook
-       def config_changed():
-           # View all changes to configuration
-           for changed, (prev, cur) in changes.conf.items():
-               print('config changed', changed,
-                     'previous value', prev,
-                     'current value',  cur)
-
-           # Get some unit specific bookeeping
-           if not db.get('pkg_key'):
-               key = urllib.urlopen('https://example.com/pkg_key').read()
-               db.set('pkg_key', key)
-
-       if __name__ == '__main__':
-           with changes():
-               hook.execute()
-
-    """
-    def __init__(self):
-        self.kv = kv()
-        self.conf = None
-        self.rels = None
-
-    @contextlib.contextmanager
-    def __call__(self):
-        from charmhelpers.core import hookenv
-        hook_name = hookenv.hook_name()
-
-        with self.kv.hook_scope(hook_name):
-            self._record_charm_version(hookenv.charm_dir())
-            delta_config, delta_relation = self._record_hook(hookenv)
-            yield self.kv, delta_config, delta_relation
-
-    def _record_charm_version(self, charm_dir):
-        # Record revisions.. charm revisions are meaningless
-        # to charm authors as they don't control the revision.
-        # so logic dependnent on revision is not particularly
-        # useful, however it is useful for debugging analysis.
-        charm_rev = open(
-            os.path.join(charm_dir, 'revision')).read().strip()
-        charm_rev = charm_rev or '0'
-        revs = self.kv.get('charm_revisions', [])
-        if charm_rev not in revs:
-            revs.append(charm_rev.strip() or '0')
-            self.kv.set('charm_revisions', revs)
-
-    def _record_hook(self, hookenv):
-        data = hookenv.execution_environment()
-        self.conf = conf_delta = self.kv.delta(data['conf'], 'config')
-        self.rels = rels_delta = self.kv.delta(data['rels'], 'rels')
-        self.kv.set('env', data['env'])
-        self.kv.set('unit', data['unit'])
-        self.kv.set('relid', data.get('relid'))
-        return conf_delta, rels_delta
-
-
-class Record(dict):
-
-    __slots__ = ()
-
-    def __getattr__(self, k):
-        if k in self:
-            return self[k]
-        raise AttributeError(k)
-
-
-class DeltaSet(Record):
-
-    __slots__ = ()
-
-
-Delta = collections.namedtuple('Delta', ['previous', 'current'])
-
-
-_KV = None
-
-
-def kv():
-    global _KV
-    if _KV is None:
-        _KV = Storage()
-    return _KV
 
=== removed directory 'hooks/charmhelpers/fetch'
=== removed file 'hooks/charmhelpers/fetch/__init__.py'
--- hooks/charmhelpers/fetch/__init__.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/fetch/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,439 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import importlib
-from tempfile import NamedTemporaryFile
-import time
-from yaml import safe_load
-from charmhelpers.core.host import (
-    lsb_release
-)
-import subprocess
-from charmhelpers.core.hookenv import (
-    config,
-    log,
-)
-import os
-
-import six
-if six.PY3:
-    from urllib.parse import urlparse, urlunparse
-else:
-    from urlparse import urlparse, urlunparse
-
-
-CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
-deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
-"""
-PROPOSED_POCKET = """# Proposed
-deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
-"""
-CLOUD_ARCHIVE_POCKETS = {
-    # Folsom
-    'folsom': 'precise-updates/folsom',
-    'precise-folsom': 'precise-updates/folsom',
-    'precise-folsom/updates': 'precise-updates/folsom',
-    'precise-updates/folsom': 'precise-updates/folsom',
-    'folsom/proposed': 'precise-proposed/folsom',
-    'precise-folsom/proposed': 'precise-proposed/folsom',
-    'precise-proposed/folsom': 'precise-proposed/folsom',
-    # Grizzly
-    'grizzly': 'precise-updates/grizzly',
-    'precise-grizzly': 'precise-updates/grizzly',
-    'precise-grizzly/updates': 'precise-updates/grizzly',
-    'precise-updates/grizzly': 'precise-updates/grizzly',
-    'grizzly/proposed': 'precise-proposed/grizzly',
-    'precise-grizzly/proposed': 'precise-proposed/grizzly',
-    'precise-proposed/grizzly': 'precise-proposed/grizzly',
-    # Havana
-    'havana': 'precise-updates/havana',
-    'precise-havana': 'precise-updates/havana',
-    'precise-havana/updates': 'precise-updates/havana',
-    'precise-updates/havana': 'precise-updates/havana',
-    'havana/proposed': 'precise-proposed/havana',
-    'precise-havana/proposed': 'precise-proposed/havana',
-    'precise-proposed/havana': 'precise-proposed/havana',
-    # Icehouse
-    'icehouse': 'precise-updates/icehouse',
-    'precise-icehouse': 'precise-updates/icehouse',
-    'precise-icehouse/updates': 'precise-updates/icehouse',
-    'precise-updates/icehouse': 'precise-updates/icehouse',
-    'icehouse/proposed': 'precise-proposed/icehouse',
-    'precise-icehouse/proposed': 'precise-proposed/icehouse',
-    'precise-proposed/icehouse': 'precise-proposed/icehouse',
-    # Juno
-    'juno': 'trusty-updates/juno',
-    'trusty-juno': 'trusty-updates/juno',
-    'trusty-juno/updates': 'trusty-updates/juno',
-    'trusty-updates/juno': 'trusty-updates/juno',
-    'juno/proposed': 'trusty-proposed/juno',
-    'trusty-juno/proposed': 'trusty-proposed/juno',
-    'trusty-proposed/juno': 'trusty-proposed/juno',
-    # Kilo
-    'kilo': 'trusty-updates/kilo',
-    'trusty-kilo': 'trusty-updates/kilo',
-    'trusty-kilo/updates': 'trusty-updates/kilo',
-    'trusty-updates/kilo': 'trusty-updates/kilo',
-    'kilo/proposed': 'trusty-proposed/kilo',
-    'trusty-kilo/proposed': 'trusty-proposed/kilo',
-    'trusty-proposed/kilo': 'trusty-proposed/kilo',
-}
-
-# The order of this list is very important. Handlers should be listed in from
-# least- to most-specific URL matching.
-FETCH_HANDLERS = (
-    'charmhelpers.fetch.archiveurl.ArchiveUrlFetchHandler',
-    'charmhelpers.fetch.bzrurl.BzrUrlFetchHandler',
-    'charmhelpers.fetch.giturl.GitUrlFetchHandler',
-)
-
-APT_NO_LOCK = 100  # The return code for "couldn't acquire lock" in APT.
-APT_NO_LOCK_RETRY_DELAY = 10  # Wait 10 seconds between apt lock checks.
-APT_NO_LOCK_RETRY_COUNT = 30  # Retry to acquire the lock X times.
-
-
-class SourceConfigError(Exception):
-    pass
-
-
-class UnhandledSource(Exception):
-    pass
-
-
-class AptLockError(Exception):
-    pass
-
-
-class BaseFetchHandler(object):
-
-    """Base class for FetchHandler implementations in fetch plugins"""
-
-    def can_handle(self, source):
-        """Returns True if the source can be handled. Otherwise returns
-        a string explaining why it cannot"""
-        return "Wrong source type"
-
-    def install(self, source):
-        """Try to download and unpack the source. Return the path to the
-        unpacked files or raise UnhandledSource."""
-        raise UnhandledSource("Wrong source type {}".format(source))
-
-    def parse_url(self, url):
-        return urlparse(url)
-
-    def base_url(self, url):
-        """Return url without querystring or fragment"""
-        parts = list(self.parse_url(url))
-        parts[4:] = ['' for i in parts[4:]]
-        return urlunparse(parts)
-
-
-def filter_installed_packages(packages):
-    """Returns a list of packages that require installation"""
-    cache = apt_cache()
-    _pkgs = []
-    for package in packages:
-        try:
-            p = cache[package]
-            p.current_ver or _pkgs.append(package)
-        except KeyError:
-            log('Package {} has no installation candidate.'.format(package),
-                level='WARNING')
-            _pkgs.append(package)
-    return _pkgs
-
-
-def apt_cache(in_memory=True):
-    """Build and return an apt cache"""
-    import apt_pkg
-    apt_pkg.init()
-    if in_memory:
-        apt_pkg.config.set("Dir::Cache::pkgcache", "")
-        apt_pkg.config.set("Dir::Cache::srcpkgcache", "")
-    return apt_pkg.Cache()
-
-
-def apt_install(packages, options=None, fatal=False):
-    """Install one or more packages"""
-    if options is None:
-        options = ['--option=Dpkg::Options::=--force-confold']
-
-    cmd = ['apt-get', '--assume-yes']
-    cmd.extend(options)
-    cmd.append('install')
-    if isinstance(packages, six.string_types):
-        cmd.append(packages)
-    else:
-        cmd.extend(packages)
-    log("Installing {} with options: {}".format(packages,
-                                                options))
-    _run_apt_command(cmd, fatal)
-
-
-def apt_upgrade(options=None, fatal=False, dist=False):
-    """Upgrade all packages"""
-    if options is None:
-        options = ['--option=Dpkg::Options::=--force-confold']
-
-    cmd = ['apt-get', '--assume-yes']
-    cmd.extend(options)
-    if dist:
-        cmd.append('dist-upgrade')
-    else:
-        cmd.append('upgrade')
-    log("Upgrading with options: {}".format(options))
-    _run_apt_command(cmd, fatal)
-
-
-def apt_update(fatal=False):
-    """Update local apt cache"""
-    cmd = ['apt-get', 'update']
-    _run_apt_command(cmd, fatal)
-
-
-def apt_purge(packages, fatal=False):
-    """Purge one or more packages"""
-    cmd = ['apt-get', '--assume-yes', 'purge']
-    if isinstance(packages, six.string_types):
-        cmd.append(packages)
-    else:
-        cmd.extend(packages)
-    log("Purging {}".format(packages))
-    _run_apt_command(cmd, fatal)
-
-
-def apt_hold(packages, fatal=False):
-    """Hold one or more packages"""
-    cmd = ['apt-mark', 'hold']
-    if isinstance(packages, six.string_types):
-        cmd.append(packages)
-    else:
-        cmd.extend(packages)
-    log("Holding {}".format(packages))
-
-    if fatal:
-        subprocess.check_call(cmd)
-    else:
-        subprocess.call(cmd)
-
-
-def add_source(source, key=None):
-    """Add a package source to this system.
-
-    @param source: a URL or sources.list entry, as supported by
-    add-apt-repository(1). Examples::
-
-        ppa:charmers/example
-        deb https://stub:[e-mail]/ubuntu trusty main
-
-    In addition:
-        'proposed:' may be used to enable the standard 'proposed'
-        pocket for the release.
-        'cloud:' may be used to activate official cloud archive pockets,
-        such as 'cloud:icehouse'
-        'distro' may be used as a noop
-
-    @param key: A key to be added to the system's APT keyring and used
-    to verify the signatures on packages. Ideally, this should be an
-    ASCII format GPG public key including the block headers. A GPG key
-    id may also be used, but be aware that only insecure protocols are
-    available to retrieve the actual public key from a public keyserver
-    placing your Juju environment at risk. ppa and cloud archive keys
-    are securely added automtically, so sould not be provided.
-    """
-    if source is None:
-        log('Source is not present. Skipping')
-        return
-
-    if (source.startswith('ppa:') or
-        source.startswith('http') or
-        source.startswith('deb ') or
-            source.startswith('cloud-archive:')):
-        subprocess.check_call(['add-apt-repository', '--yes', source])
-    elif source.startswith('cloud:'):
-        apt_install(filter_installed_packages(['ubuntu-cloud-keyring']),
-                    fatal=True)
-        pocket = source.split(':')[-1]
-        if pocket not in CLOUD_ARCHIVE_POCKETS:
-            raise SourceConfigError(
-                'Unsupported cloud: source option %s' %
-                pocket)
-        actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket]
-        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
-            apt.write(CLOUD_ARCHIVE.format(actual_pocket))
-    elif source == 'proposed':
-        release = lsb_release()['DISTRIB_CODENAME']
-        with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
-            apt.write(PROPOSED_POCKET.format(release))
-    elif source == 'distro':
-        pass
-    else:
-        log("Unknown source: {!r}".format(source))
-
-    if key:
-        if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
-            with NamedTemporaryFile('w+') as key_file:
-                key_file.write(key)
-                key_file.flush()
-                key_file.seek(0)
-                subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file)
-        else:
-            # Note that hkp: is in no way a secure protocol. Using a
-            # GPG key id is pointless from a security POV unless you
-            # absolutely trust your network and DNS.
-            subprocess.check_call(['apt-key', 'adv', '--keyserver',
-                                   'hkp://keyserver.ubuntu.com:80', '--recv',
-                                   key])
-
-
-def configure_sources(update=False,
-                      sources_var='install_sources',
-                      keys_var='install_keys'):
-    """
-    Configure multiple sources from charm configuration.
-
-    The lists are encoded as yaml fragments in the configuration.
-    The frament needs to be included as a string. Sources and their
-    corresponding keys are of the types supported by add_source().
-
-    Example config:
-        install_sources: |
-          - "ppa:foo"
-          - "http://example.com/repo precise main"
-        install_keys: |
-          - null
-          - "a1b2c3d4"
-
-    Note that 'null' (a.k.a. None) should not be quoted.
-    """
-    sources = safe_load((config(sources_var) or '').strip()) or []
-    keys = safe_load((config(keys_var) or '').strip()) or None
-
-    if isinstance(sources, six.string_types):
-        sources = [sources]
-
-    if keys is None:
-        for source in sources:
-            add_source(source, None)
-    else:
-        if isinstance(keys, six.string_types):
-            keys = [keys]
-
-        if len(sources) != len(keys):
-            raise SourceConfigError(
-                'Install sources and keys lists are different lengths')
-        for source, key in zip(sources, keys):
-            add_source(source, key)
-    if update:
-        apt_update(fatal=True)
-
-
-def install_remote(source, *args, **kwargs):
-    """
-    Install a file tree from a remote source
-
-    The specified source should be a url of the form:
-        scheme://[host]/path[#[option=value][&...]]
-
-    Schemes supported are based on this modules submodules.
-    Options supported are submodule-specific.
-    Additional arguments are passed through to the submodule.
-
-    For example::
-
-        dest = install_remote('http://example.com/archive.tgz',
-                              checksum='deadbeef',
-                              hash_type='sha1')
-
-    This will download `archive.tgz`, validate it using SHA1 and, if
-    the file is ok, extract it and return the directory in which it
-    was extracted.  If the checksum fails, it will raise
-    :class:`charmhelpers.core.host.ChecksumError`.
-    """
-    # We ONLY check for True here because can_handle may return a string
-    # explaining why it can't handle a given source.
-    handlers = [h for h in plugins() if h.can_handle(source) is True]
-    installed_to = None
-    for handler in handlers:
-        try:
-            installed_to = handler.install(source, *args, **kwargs)
-        except UnhandledSource:
-            pass
-    if not installed_to:
-        raise UnhandledSource("No handler found for source {}".format(source))
-    return installed_to
-
-
-def install_from_config(config_var_name):
-    charm_config = config()
-    source = charm_config[config_var_name]
-    return install_remote(source)
-
-
-def plugins(fetch_handlers=None):
-    if not fetch_handlers:
-        fetch_handlers = FETCH_HANDLERS
-    plugin_list = []
-    for handler_name in fetch_handlers:
-        package, classname = handler_name.rsplit('.', 1)
-        try:
-            handler_class = getattr(
-                importlib.import_module(package),
-                classname)
-            plugin_list.append(handler_class())
-        except (ImportError, AttributeError):
-            # Skip missing plugins so that they can be ommitted from
-            # installation if desired
-            log("FetchHandler {} not found, skipping plugin".format(
-                handler_name))
-    return plugin_list
-
-
-def _run_apt_command(cmd, fatal=False):
-    """
-    Run an APT command, checking output and retrying if the fatal flag is set
-    to True.
-
-    :param: cmd: str: The apt command to run.
-    :param: fatal: bool: Whether the command's output should be checked and
-        retried.
-    """
-    env = os.environ.copy()
-
-    if 'DEBIAN_FRONTEND' not in env:
-        env['DEBIAN_FRONTEND'] = 'noninteractive'
-
-    if fatal:
-        retry_count = 0
-        result = None
-
-        # If the command is considered "fatal", we need to retry if the apt
-        # lock was not acquired.
-
-        while result is None or result == APT_NO_LOCK:
-            try:
-                result = subprocess.check_call(cmd, env=env)
-            except subprocess.CalledProcessError as e:
-                retry_count = retry_count + 1
-                if retry_count > APT_NO_LOCK_RETRY_COUNT:
-                    raise
-                result = e.returncode
-                log("Couldn't acquire DPKG lock. Will retry in {} seconds."
-                    "".format(APT_NO_LOCK_RETRY_DELAY))
-                time.sleep(APT_NO_LOCK_RETRY_DELAY)
-
-    else:
-        subprocess.call(cmd, env=env)
 
=== removed file 'hooks/charmhelpers/fetch/archiveurl.py'
--- hooks/charmhelpers/fetch/archiveurl.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/fetch/archiveurl.py	1970-01-01 00:00:00 +0000
@@ -1,161 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-import hashlib
-import re
-
-from charmhelpers.fetch import (
-    BaseFetchHandler,
-    UnhandledSource
-)
-from charmhelpers.payload.archive import (
-    get_archive_handler,
-    extract,
-)
-from charmhelpers.core.host import mkdir, check_hash
-
-import six
-if six.PY3:
-    from urllib.request import (
-        build_opener, install_opener, urlopen, urlretrieve,
-        HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
-    )
-    from urllib.parse import urlparse, urlunparse, parse_qs
-    from urllib.error import URLError
-else:
-    from urllib import urlretrieve
-    from urllib2 import (
-        build_opener, install_opener, urlopen,
-        HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
-        URLError
-    )
-    from urlparse import urlparse, urlunparse, parse_qs
-
-
-def splituser(host):
-    '''urllib.splituser(), but six's support of this seems broken'''
-    _userprog = re.compile('^(.*)@(.*)$')
-    match = _userprog.match(host)
-    if match:
-        return match.group(1, 2)
-    return None, host
-
-
-def splitpasswd(user):
-    '''urllib.splitpasswd(), but six's support of this is missing'''
-    _passwdprog = re.compile('^([^:]*):(.*)$', re.S)
-    match = _passwdprog.match(user)
-    if match:
-        return match.group(1, 2)
-    return user, None
-
-
-class ArchiveUrlFetchHandler(BaseFetchHandler):
-    """
-    Handler to download archive files from arbitrary URLs.
-
-    Can fetch from http, https, ftp, and file URLs.
-
-    Can install either tarballs (.tar, .tgz, .tbz2, etc) or zip files.
-
-    Installs the contents of the archive in $CHARM_DIR/fetched/.
-    """
-    def can_handle(self, source):
-        url_parts = self.parse_url(source)
-        if url_parts.scheme not in ('http', 'https', 'ftp', 'file'):
-            return "Wrong source type"
-        if get_archive_handler(self.base_url(source)):
-            return True
-        return False
-
-    def download(self, source, dest):
-        """
-        Download an archive file.
-
-        :param str source: URL pointing to an archive file.
-        :param str dest: Local path location to download archive file to.
-        """
-        # propogate all exceptions
-        # URLError, OSError, etc
-        proto, netloc, path, params, query, fragment = urlparse(source)
-        if proto in ('http', 'https'):
-            auth, barehost = splituser(netloc)
-            if auth is not None:
-                source = urlunparse((proto, barehost, path, params, query, fragment))
-                username, password = splitpasswd(auth)
-                passman = HTTPPasswordMgrWithDefaultRealm()
-                # Realm is set to None in add_password to force the username and password
-                # to be used whatever the realm
-                passman.add_password(None, source, username, password)
-                authhandler = HTTPBasicAuthHandler(passman)
-                opener = build_opener(authhandler)
-                install_opener(opener)
-        response = urlopen(source)
-        try:
-            with open(dest, 'w') as dest_file:
-                dest_file.write(response.read())
-        except Exception as e:
-            if os.path.isfile(dest):
-                os.unlink(dest)
-            raise e
-
-    # Mandatory file validation via Sha1 or MD5 hashing.
-    def download_and_validate(self, url, hashsum, validate="sha1"):
-        tempfile, headers = urlretrieve(url)
-        check_hash(tempfile, hashsum, validate)
-        return tempfile
-
-    def install(self, source, dest=None, checksum=None, hash_type='sha1'):
-        """
-        Download and install an archive file, with optional checksum validation.
-
-        The checksum can also be given on the `source` URL's fragment.
-        For example::
-
-            handler.install('http://example.com/file.tgz#sha1=deadbeef')
-
-        :param str source: URL pointing to an archive file.
-        :param str dest: Local destination path to install to. If not given,
-            installs to `$CHARM_DIR/archives/archive_file_name`.
-        :param str checksum: If given, validate the archive file after download.
-        :param str hash_type: Algorithm used to generate `checksum`.
-            Can be any hash alrgorithm supported by :mod:`hashlib`,
-            such as md5, sha1, sha256, sha512, etc.
-
-        """
-        url_parts = self.parse_url(source)
-        dest_dir = os.path.join(os.environ.get('CHARM_DIR'), 'fetched')
-        if not os.path.exists(dest_dir):
-            mkdir(dest_dir, perms=0o755)
-        dld_file = os.path.join(dest_dir, os.path.basename(url_parts.path))
-        try:
-            self.download(source, dld_file)
-        except URLError as e:
-            raise UnhandledSource(e.reason)
-        except OSError as e:
-            raise UnhandledSource(e.strerror)
-        options = parse_qs(url_parts.fragment)
-        for key, value in options.items():
-            if not six.PY3:
-                algorithms = hashlib.algorithms
-            else:
-                algorithms = hashlib.algorithms_available
-            if key in algorithms:
-                check_hash(dld_file, value, key)
-        if checksum:
-            check_hash(dld_file, checksum, hash_type)
-        return extract(dld_file, dest)
 
=== removed file 'hooks/charmhelpers/fetch/bzrurl.py'
--- hooks/charmhelpers/fetch/bzrurl.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/fetch/bzrurl.py	1970-01-01 00:00:00 +0000
@@ -1,78 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-from charmhelpers.fetch import (
-    BaseFetchHandler,
-    UnhandledSource
-)
-from charmhelpers.core.host import mkdir
-
-import six
-if six.PY3:
-    raise ImportError('bzrlib does not support Python3')
-
-try:
-    from bzrlib.branch import Branch
-    from bzrlib import bzrdir, workingtree, errors
-except ImportError:
-    from charmhelpers.fetch import apt_install
-    apt_install("python-bzrlib")
-    from bzrlib.branch import Branch
-    from bzrlib import bzrdir, workingtree, errors
-
-
-class BzrUrlFetchHandler(BaseFetchHandler):
-    """Handler for bazaar branches via generic and lp URLs"""
-    def can_handle(self, source):
-        url_parts = self.parse_url(source)
-        if url_parts.scheme not in ('bzr+ssh', 'lp'):
-            return False
-        else:
-            return True
-
-    def branch(self, source, dest):
-        url_parts = self.parse_url(source)
-        # If we use lp:branchname scheme we need to load plugins
-        if not self.can_handle(source):
-            raise UnhandledSource("Cannot handle {}".format(source))
-        if url_parts.scheme == "lp":
-            from bzrlib.plugin import load_plugins
-            load_plugins()
-        try:
-            local_branch = bzrdir.BzrDir.create_branch_convenience(dest)
-        except errors.AlreadyControlDirError:
-            local_branch = Branch.open(dest)
-        try:
-            remote_branch = Branch.open(source)
-            remote_branch.push(local_branch)
-            tree = workingtree.WorkingTree.open(dest)
-            tree.update()
-        except Exception as e:
-            raise e
-
-    def install(self, source):
-        url_parts = self.parse_url(source)
-        branch_name = url_parts.path.strip("/").split("/")[-1]
-        dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
-                                branch_name)
-        if not os.path.exists(dest_dir):
-            mkdir(dest_dir, perms=0o755)
-        try:
-            self.branch(source, dest_dir)
-        except OSError as e:
-            raise UnhandledSource(e.strerror)
-        return dest_dir
 
=== removed file 'hooks/charmhelpers/fetch/giturl.py'
--- hooks/charmhelpers/fetch/giturl.py	2015-02-19 16:21:31 +0000
+++ hooks/charmhelpers/fetch/giturl.py	1970-01-01 00:00:00 +0000
@@ -1,71 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-from charmhelpers.fetch import (
-    BaseFetchHandler,
-    UnhandledSource
-)
-from charmhelpers.core.host import mkdir
-
-import six
-if six.PY3:
-    raise ImportError('GitPython does not support Python 3')
-
-try:
-    from git import Repo
-except ImportError:
-    from charmhelpers.fetch import apt_install
-    apt_install("python-git")
-    from git import Repo
-
-from git.exc import GitCommandError  # noqa E402
-
-
-class GitUrlFetchHandler(BaseFetchHandler):
-    """Handler for git branches via generic and github URLs"""
-    def can_handle(self, source):
-        url_parts = self.parse_url(source)
-        # TODO (mattyw) no support for ssh git@ yet
-        if url_parts.scheme not in ('http', 'https', 'git'):
-            return False
-        else:
-            return True
-
-    def clone(self, source, dest, branch):
-        if not self.can_handle(source):
-            raise UnhandledSource("Cannot handle {}".format(source))
-
-        repo = Repo.clone_from(source, dest)
-        repo.git.checkout(branch)
-
-    def install(self, source, branch="master", dest=None):
-        url_parts = self.parse_url(source)
-        branch_name = url_parts.path.strip("/").split("/")[-1]
-        if dest:
-            dest_dir = os.path.join(dest, branch_name)
-        else:
-            dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
-                                    branch_name)
-        if not os.path.exists(dest_dir):
-            mkdir(dest_dir, perms=0o755)
-        try:
-            self.clone(source, dest_dir, branch)
-        except GitCommandError as e:
-            raise UnhandledSource(e.message)
-        except OSError as e:
-            raise UnhandledSource(e.strerror)
-        return dest_dir
 
=== removed file 'hooks/config-changed'
--- hooks/config-changed	2015-02-19 21:19:01 +0000
+++ hooks/config-changed	1970-01-01 00:00:00 +0000
@@ -1,43 +0,0 @@
-#!/usr/bin/python
-
-#
-# Copyright 2015 Canonical Ltd.
-#
-# Authors:
-#  Liang Chen <[e-mail]>
-#
-import subprocess
-import sys
-
-from charmhelpers.core.hookenv import (
-    config,
-    Hooks, UnregisteredHookError,
-    log, ERROR,
-)
-
-from utils import render_template
-
-hooks = Hooks()
-
-
-def emit_lxc_br_conf():
-    lxc_context = {}
-    if config('new-lxc-network'):
-        lxc_context['new_network'] = True
-
-    lxc_bridge_conf = "/etc/network/interfaces.d/lxcbr0.cfg"
-    with open(lxc_bridge_conf, 'w') as lxc_br_conf:
-        lxc_br_conf.write(render_template('lxc-bridge.conf', lxc_context))
-
-
[e-mail]('config-changed')
-def config_changed():
-    emit_lxc_br_conf()
-    cmd = ['ifup', 'lxcbr0']
-    subprocess.check_call(cmd)
-
-if __name__ == '__main__':
-    try:
-        hooks.execute(sys.argv)
-    except UnregisteredHookError as e:
-        log('Unknown hook {} - skipping.'.format(e))
 
=== removed file 'hooks/hooks.py'
--- hooks/hooks.py	2015-02-19 21:19:01 +0000
+++ hooks/hooks.py	1970-01-01 00:00:00 +0000
@@ -1,45 +0,0 @@
-#!/usr/bin/python
-
-#
-# Copyright 2015 Canonical Ltd.
-#
-# Authors:
-#  Liang Chen <[e-mail]>
-#
-
-import subprocess
-import sys
-
-from charmhelpers.core.hookenv import (
-    config,
-    Hooks, UnregisteredHookError,
-    log,
-)
-
-from utils import render_template
-
-hooks = Hooks()
-
-
-def emit_lxc_br_conf():
-    lxc_context = {}
-    if config('new-lxc-network'):
-        lxc_context['new_network'] = True
-
-    lxc_bridge_conf = "/etc/network/interfaces.d/lxcbr0.cfg"
-    with open(lxc_bridge_conf, 'w') as lxc_br_conf:
-        lxc_br_conf.write(render_template('lxc-bridge.conf', lxc_context))
-
-
[e-mail]('config-changed')
-def config_changed():
-    emit_lxc_br_conf()
-    cmd = ['ifup', 'lxcbr0']
-    subprocess.check_call(cmd)
-
-
-if __name__ == '__main__':
-    try:
-        hooks.execute(sys.argv)
-    except UnregisteredHookError as e:
-        log('Unknown hook {} - skipping.'.format(e))
 
=== removed file 'hooks/install'
--- hooks/install	2012-05-03 22:17:19 +0000
+++ hooks/install	1970-01-01 00:00:00 +0000
@@ -1,2 +0,0 @@
-#!/bin/bash
-# Does nothing. - charm proof requires an install hook.
 
=== removed file 'hooks/utils.py'
--- hooks/utils.py	2015-02-19 21:19:01 +0000
+++ hooks/utils.py	1970-01-01 00:00:00 +0000
@@ -1,27 +0,0 @@
-#
-# Copyright 2015 Canonical Ltd.
-#
-# Authors:
-#  Liang Chen <[e-mail]>
-#
-
-TEMPLATES_DIR = 'templates'
-
-from charmhelpers.fetch import (
-    apt_install,
-    filter_installed_packages
-)
-
-try:
-    import jinja2
-except ImportError:
-    apt_install(filter_installed_packages(['python-jinja2']),
-                fatal=True)
-    import jinja2
-
-
-def render_template(template_name, context, template_dir=TEMPLATES_DIR):
-    templates = jinja2.Environment(
-        loader=jinja2.FileSystemLoader(template_dir))
-    template = templates.get_template(template_name)
-    return template.render(context)
 
=== modified file 'metadata.yaml'
--- metadata.yaml	2013-12-11 21:28:41 +0000
+++ metadata.yaml	2015-04-30 13:14:23 +0000
@@ -3,5 +3,5 @@
 maintainer: Charmers <[e-mail]>
 description: |
   This simply deploys Ubuntu Server.
-categories:
+tags:
      - misc
 
=== removed directory 'templates'
=== removed file 'templates/lxc-bridge.conf'
--- templates/lxc-bridge.conf	2015-02-19 21:19:01 +0000
+++ templates/lxc-bridge.conf	1970-01-01 00:00:00 +0000
@@ -1,10 +0,0 @@
-    
-{% if new_network -%}
-auto eth0
-iface eth0 inet manual
- 
-auto lxcbr0
-iface lxcbr0 inet dhcp
-    bridge_ports eth0
- 
-{% endif -%}
 
=== modified file 'tests/00-setup.sh'
--- tests/00-setup.sh	2015-02-19 17:29:32 +0000
+++ tests/00-setup.sh	2015-04-30 13:14:23 +0000
@@ -1,15 +1,9 @@
-#!/bin/bash
-
-# This script runs first to set up the environment for the tests.
-
-set -x 
-
-# Check if amulet is installed before adding repository and updating apt-get.
+#!/bin/bash -x
+
+# Install amulet if not already installed.
 dpkg -s amulet
 if [ $? -ne 0 ]; then
     sudo add-apt-repository -y ppa:juju/stable
-    sudo apt-get update -qq 
+    sudo apt-get update -qq
     sudo apt-get install -y amulet
 fi
-# Install any additional packages needed for tests here.
-
 
=== added file 'tests/010_basic_precise'
--- tests/010_basic_precise	1970-01-01 00:00:00 +0000
+++ tests/010_basic_precise	2015-04-30 13:14:23 +0000
@@ -0,0 +1,7 @@
+#!/usr/bin/python
+"""Amulet tests on a basic ubuntu charm deployment on precise."""
+
+from basic_deployment import ubuntu_basic_deployment
+
+if __name__ == '__main__':
+    ubuntu_basic_deployment(series='precise')
 
=== added file 'tests/015_basic_trusty'
--- tests/015_basic_trusty	1970-01-01 00:00:00 +0000
+++ tests/015_basic_trusty	2015-04-30 13:14:23 +0000
@@ -0,0 +1,7 @@
+#!/usr/bin/python
+"""Amulet tests on a basic ubuntu charm deployment on trusty."""
+
+from basic_deployment import ubuntu_basic_deployment
+
+if __name__ == '__main__':
+    ubuntu_basic_deployment(series='trusty')
 
=== added file 'tests/020_basic_utopic'
--- tests/020_basic_utopic	1970-01-01 00:00:00 +0000
+++ tests/020_basic_utopic	2015-04-30 13:14:23 +0000
@@ -0,0 +1,7 @@
+#!/usr/bin/python
+"""Amulet tests on a basic ubuntu charm deployment on utopic."""
+
+from basic_deployment import ubuntu_basic_deployment
+
+if __name__ == '__main__':
+    ubuntu_basic_deployment(series='utopic')
 
=== added file 'tests/025_basic_vivid'
--- tests/025_basic_vivid	1970-01-01 00:00:00 +0000
+++ tests/025_basic_vivid	2015-04-30 13:14:23 +0000
@@ -0,0 +1,7 @@
+#!/usr/bin/python
+"""Amulet tests on a basic ubuntu charm deployment on trusty."""
+
+from basic_deployment import ubuntu_basic_deployment
+
+if __name__ == '__main__':
+    ubuntu_basic_deployment(series='vivid')
 
=== removed file 'tests/10-deploy-test.py'
--- tests/10-deploy-test.py	2015-03-27 15:26:53 +0000
+++ tests/10-deploy-test.py	1970-01-01 00:00:00 +0000
@@ -1,90 +0,0 @@
-#!/usr/bin/python3
-
-# This Amulet based tests
-# The goal is to ensure the Ubuntu charm
-# sucessfully deploys and can be accessed.
-# Note the Ubuntu charm does not have any
-# relations or config options.
-
-import amulet
-
-# Timeout value, in seconds to deploy the environment
-seconds = 900
-series = 'trusty'
-
-# Set up the deployer module to interact and set up the environment.
-d = amulet.Deployment(series=series)
-
-# Define the environment in terms of charms, their config, and relations.
-
-# Add the Ubuntu charm to the deployment.
-d.add('ubuntu')
-
-# Deploy the environment currently defined
-try:
-    # Wait the defined about amount of time to deploy the environment.
-    # Setup makes sure the services are deployed, related, and in a
-    # "started" state.
-    d.setup(timeout=seconds)
-    # Use a sentry to ensure there are no remaining hooks being execute
-    # on any of the nodes
-except amulet.helpers.TimeoutError:
-    # Pending the configuration the test will fail or be skipped
-    # if not deployed properly.
-    error_message = 'The environment did not deploy in %d seconds.' % seconds
-    amulet.raise_status(amulet.SKIP, msg=error_message)
-except:
-    # Something else has gone wrong, raise the error so we can see it and this
-    # will automatically "FAIL" the test.
-    raise
-
-# Access the Ubuntu instance to ensure it has been deployed correctly
-
-# Define the commands to be ran
-lsb_command = 'lsb_release -cs'
-uname_command = 'uname -a'
-lxc_interface_command = 'ifconfig -s lxcbr0'
-
-print(lsb_command)
-# Print the release information
-output, code = d.sentry.unit['ubuntu/0'].run(lsb_command)
-print(output)
-# Confirm the lsb_release command ran successfully
-if (code != 0):
-    message = 'The ' + lsb_command + ' did not return the expected return code of 0.'
-    amulet.raise_status(amulet.FAIL, msg=message)
-else:
-    if series in output:
-        print('The series is correct.')
-    else:
-        message = 'The series is not correct.'
-        print(message)
-        amulet.raise_status(amulet.FAIL, msg=message)
-
-print(uname_command)
-
-# Get the uname -a output
-output, code = d.sentry.unit['ubuntu/0'].run(uname_command)
-print(output)
-# Confirm the uname command was ran successfully
-if (code != 0):
-    message = 'The ' + uname_command + ' did not return the expected return code of 0.'
-    amulet.raise_status(amulet.FAIL, msg=message)
-else:
-    message = 'The ' + uname_command + ' successfully executed.'
-    print(output)
-    print(message)
-
-# Test creating a new network for LXC via new-lxc-network config option.
-# Set the new-lxc-network config option to create a new network for LXC.
-d.configure('ubuntu', {'new-lxc-network': 'true'})
-# Confirm the the lxc bridge interface was sucessfully created
-output, code = d.sentry.unit['ubuntu/0'].run(lxc_interface_command)
-if (code != 0):
-    error_message = 'The lxcbr0 interface was not created.'
-    print(output)
-    amulet.raise_status(amulet.FAIL, msg=error_message)
-else:
-    message = 'The lxcbr0 interface sucessfully created.'
-    print(output)
-    print(message)
 
=== added file 'tests/basic_deployment.py'
--- tests/basic_deployment.py	1970-01-01 00:00:00 +0000
+++ tests/basic_deployment.py	2015-04-30 13:14:23 +0000
@@ -0,0 +1,49 @@
+#!/usr/bin/python3
+"""
+Ubuntu charm functional test.  Take note that the Ubuntu
+charm does not have any relations or config options
+to exercise.
+"""
+
+import amulet
+from charmhelpers.contrib.amulet.utils import AmuletUtils
+import logging
+
+
+def ubuntu_basic_deployment(series):
+    """ Common test routines to run per-series. """
+
+    # Initialize
+    seconds = 900
+    u = AmuletUtils(logging.DEBUG)
+    d = amulet.Deployment(series=series)
+    d.add('ubuntu')
+    unit = 'ubuntu/0'
+    lsb_command = 'lsb_release -cs'
+
+    # Deploy services, wait for started state.  Fail or skip on timeout.
+    try:
+        d.setup(timeout=seconds)
+    except amulet.helpers.TimeoutError:
+        message = 'Deployment timed out ({}s)'.format(seconds)
+        amulet.raise_status(amulet.SKIP, msg=message)
+    except:
+        raise
+
+    # Confirm Ubuntu release name from the unit.
+    u.log.debug('Command:  {}'.format(lsb_command))
+    output, code = d.sentry.unit[unit].run(lsb_command)
+    u.log.debug('Output:  {}'.format(output))
+
+    if (code != 0):
+        message = 'Command FAIL:  {}'.format(lsb_command)
+        u.log.error(message)
+        amulet.raise_status(amulet.FAIL, msg=message)
+    else:
+        if series in output:
+            message = 'Series:  OK'
+            u.log.info(message)
+        else:
+            message = 'Series:  FAIL ({})'.format(output)
+            u.log.error(message)
+            amulet.raise_status(amulet.FAIL, msg=message)
 
=== added directory 'tests/charmhelpers'
=== added file 'tests/charmhelpers/__init__.py'
--- tests/charmhelpers/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/__init__.py	2015-04-30 13:14:23 +0000
@@ -0,0 +1,38 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+# Bootstrap charm-helpers, installing its dependencies if necessary using
+# only standard libraries.
+import subprocess
+import sys
+
+try:
+    import six  # flake8: noqa
+except ImportError:
+    if sys.version_info.major == 2:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python-six'])
+    else:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python3-six'])
+    import six  # flake8: noqa
+
+try:
+    import yaml  # flake8: noqa
+except ImportError:
+    if sys.version_info.major == 2:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml'])
+    else:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml'])
+    import yaml  # flake8: noqa
 
=== added directory 'tests/charmhelpers/contrib'
=== added file 'tests/charmhelpers/contrib/__init__.py'
--- tests/charmhelpers/contrib/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/__init__.py	2015-04-30 13:14:23 +0000
@@ -0,0 +1,15 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
 
=== added directory 'tests/charmhelpers/contrib/amulet'
=== added file 'tests/charmhelpers/contrib/amulet/__init__.py'
--- tests/charmhelpers/contrib/amulet/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/amulet/__init__.py	2015-04-30 13:14:23 +0000
@@ -0,0 +1,15 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
 
=== added file 'tests/charmhelpers/contrib/amulet/deployment.py'
--- tests/charmhelpers/contrib/amulet/deployment.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/amulet/deployment.py	2015-04-30 13:14:23 +0000
@@ -0,0 +1,93 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import amulet
+import os
+import six
+
+
+class AmuletDeployment(object):
+    """Amulet deployment.
+
+       This class provides generic Amulet deployment and test runner
+       methods.
+       """
+
+    def __init__(self, series=None):
+        """Initialize the deployment environment."""
+        self.series = None
+
+        if series:
+            self.series = series
+            self.d = amulet.Deployment(series=self.series)
+        else:
+            self.d = amulet.Deployment()
+
+    def _add_services(self, this_service, other_services):
+        """Add services.
+
+           Add services to the deployment where this_service is the local charm
+           that we're testing and other_services are the other services that
+           are being used in the local amulet tests.
+           """
+        if this_service['name'] != os.path.basename(os.getcwd()):
+            s = this_service['name']
+            msg = "The charm's root directory name needs to be {}".format(s)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        if 'units' not in this_service:
+            this_service['units'] = 1
+
+        self.d.add(this_service['name'], units=this_service['units'])
+
+        for svc in other_services:
+            if 'location' in svc:
+                branch_location = svc['location']
+            elif self.series:
+                branch_location = 'cs:{}/{}'.format(self.series, svc['name']),
+            else:
+                branch_location = None
+
+            if 'units' not in svc:
+                svc['units'] = 1
+
+            self.d.add(svc['name'], charm=branch_location, units=svc['units'])
+
+    def _add_relations(self, relations):
+        """Add all of the relations for the services."""
+        for k, v in six.iteritems(relations):
+            self.d.relate(k, v)
+
+    def _configure_services(self, configs):
+        """Configure all of the services."""
+        for service, config in six.iteritems(configs):
+            self.d.configure(service, config)
+
+    def _deploy(self):
+        """Deploy environment and wait for all hooks to finish executing."""
+        try:
+            self.d.setup(timeout=900)
+            self.d.sentry.wait(timeout=900)
+        except amulet.helpers.TimeoutError:
+            amulet.raise_status(amulet.FAIL, msg="Deployment timed out")
+        except Exception:
+            raise
+
+    def run_tests(self):
+        """Run all of the methods that are prefixed with 'test_'."""
+        for test in dir(self):
+            if test.startswith('test_'):
+                getattr(self, test)()
 
=== added file 'tests/charmhelpers/contrib/amulet/utils.py'
--- tests/charmhelpers/contrib/amulet/utils.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/amulet/utils.py	2015-04-30 13:14:23 +0000
@@ -0,0 +1,323 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import ConfigParser
+import io
+import logging
+import re
+import sys
+import time
+
+import six
+
+
+class AmuletUtils(object):
+    """Amulet utilities.
+
+       This class provides common utility functions that are used by Amulet
+       tests.
+       """
+
+    def __init__(self, log_level=logging.ERROR):
+        self.log = self.get_logger(level=log_level)
+
+    def get_logger(self, name="amulet-logger", level=logging.DEBUG):
+        """Get a logger object that will log to stdout."""
+        log = logging
+        logger = log.getLogger(name)
+        fmt = log.Formatter("%(asctime)s %(funcName)s "
+                            "%(levelname)s: %(message)s")
+
+        handler = log.StreamHandler(stream=sys.stdout)
+        handler.setLevel(level)
+        handler.setFormatter(fmt)
+
+        logger.addHandler(handler)
+        logger.setLevel(level)
+
+        return logger
+
+    def valid_ip(self, ip):
+        if re.match(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$", ip):
+            return True
+        else:
+            return False
+
+    def valid_url(self, url):
+        p = re.compile(
+            r'^(?:http|ftp)s?://'
+            r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}\.?)|'  # noqa
+            r'localhost|'
+            r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'
+            r'(?::\d+)?'
+            r'(?:/?|[/?]\S+)$',
+            re.IGNORECASE)
+        if p.match(url):
+            return True
+        else:
+            return False
+
+    def validate_services(self, commands):
+        """Validate services.
+
+           Verify the specified services are running on the corresponding
+           service units.
+           """
+        for k, v in six.iteritems(commands):
+            for cmd in v:
+                output, code = k.run(cmd)
+                self.log.debug('{} `{}` returned '
+                               '{}'.format(k.info['unit_name'],
+                                           cmd, code))
+                if code != 0:
+                    return "command `{}` returned {}".format(cmd, str(code))
+        return None
+
+    def _get_config(self, unit, filename):
+        """Get a ConfigParser object for parsing a unit's config file."""
+        file_contents = unit.file_contents(filename)
+
+        # NOTE(beisner):  by default, ConfigParser does not handle options
+        # with no value, such as the flags used in the mysql my.cnf file.
+        # https://bugs.python.org/issue7005
+        config = ConfigParser.ConfigParser(allow_no_value=True)
+        config.readfp(io.StringIO(file_contents))
+        return config
+
+    def validate_config_data(self, sentry_unit, config_file, section,
+                             expected):
+        """Validate config file data.
+
+           Verify that the specified section of the config file contains
+           the expected option key:value pairs.
+           """
+        config = self._get_config(sentry_unit, config_file)
+
+        if section != 'DEFAULT' and not config.has_section(section):
+            return "section [{}] does not exist".format(section)
+
+        for k in expected.keys():
+            if not config.has_option(section, k):
+                return "section [{}] is missing option {}".format(section, k)
+            if config.get(section, k) != expected[k]:
+                return "section [{}] {}:{} != expected {}:{}".format(
+                       section, k, config.get(section, k), k, expected[k])
+        return None
+
+    def _validate_dict_data(self, expected, actual):
+        """Validate dictionary data.
+
+           Compare expected dictionary data vs actual dictionary data.
+           The values in the 'expected' dictionary can be strings, bools, ints,
+           longs, or can be a function that evaluate a variable and returns a
+           bool.
+           """
+        self.log.debug('actual: {}'.format(repr(actual)))
+        self.log.debug('expected: {}'.format(repr(expected)))
+
+        for k, v in six.iteritems(expected):
+            if k in actual:
+                if (isinstance(v, six.string_types) or
+                        isinstance(v, bool) or
+                        isinstance(v, six.integer_types)):
+                    if v != actual[k]:
+                        return "{}:{}".format(k, actual[k])
+                elif not v(actual[k]):
+                    return "{}:{}".format(k, actual[k])
+            else:
+                return "key '{}' does not exist".format(k)
+        return None
+
+    def validate_relation_data(self, sentry_unit, relation, expected):
+        """Validate actual relation data based on expected relation data."""
+        actual = sentry_unit.relation(relation[0], relation[1])
+        return self._validate_dict_data(expected, actual)
+
+    def _validate_list_data(self, expected, actual):
+        """Compare expected list vs actual list data."""
+        for e in expected:
+            if e not in actual:
+                return "expected item {} not found in actual list".format(e)
+        return None
+
+    def not_null(self, string):
+        if string is not None:
+            return True
+        else:
+            return False
+
+    def _get_file_mtime(self, sentry_unit, filename):
+        """Get last modification time of file."""
+        return sentry_unit.file_stat(filename)['mtime']
+
+    def _get_dir_mtime(self, sentry_unit, directory):
+        """Get last modification time of directory."""
+        return sentry_unit.directory_stat(directory)['mtime']
+
+    def _get_proc_start_time(self, sentry_unit, service, pgrep_full=False):
+        """Get process' start time.
+
+           Determine start time of the process based on the last modification
+           time of the /proc/pid directory. If pgrep_full is True, the process
+           name is matched against the full command line.
+           """
+        if pgrep_full:
+            cmd = 'pgrep -o -f {}'.format(service)
+        else:
+            cmd = 'pgrep -o {}'.format(service)
+        cmd = cmd + '  | grep  -v pgrep || exit 0'
+        cmd_out = sentry_unit.run(cmd)
+        self.log.debug('CMDout: ' + str(cmd_out))
+        if cmd_out[0]:
+            self.log.debug('Pid for %s %s' % (service, str(cmd_out[0])))
+            proc_dir = '/proc/{}'.format(cmd_out[0].strip())
+            return self._get_dir_mtime(sentry_unit, proc_dir)
+
+    def service_restarted(self, sentry_unit, service, filename,
+                          pgrep_full=False, sleep_time=20):
+        """Check if service was restarted.
+
+           Compare a service's start time vs a file's last modification time
+           (such as a config file for that service) to determine if the service
+           has been restarted.
+           """
+        time.sleep(sleep_time)
+        if (self._get_proc_start_time(sentry_unit, service, pgrep_full) >=
+                self._get_file_mtime(sentry_unit, filename)):
+            return True
+        else:
+            return False
+
+    def service_restarted_since(self, sentry_unit, mtime, service,
+                                pgrep_full=False, sleep_time=20,
+                                retry_count=2):
+        """Check if service was been started after a given time.
+
+        Args:
+          sentry_unit (sentry): The sentry unit to check for the service on
+          mtime (float): The epoch time to check against
+          service (string): service name to look for in process table
+          pgrep_full (boolean): Use full command line search mode with pgrep
+          sleep_time (int): Seconds to sleep before looking for process
+          retry_count (int): If service is not found, how many times to retry
+
+        Returns:
+          bool: True if service found and its start time it newer than mtime,
+                False if service is older than mtime or if service was
+                not found.
+        """
+        self.log.debug('Checking %s restarted since %s' % (service, mtime))
+        time.sleep(sleep_time)
+        proc_start_time = self._get_proc_start_time(sentry_unit, service,
+                                                    pgrep_full)
+        while retry_count > 0 and not proc_start_time:
+            self.log.debug('No pid file found for service %s, will retry %i '
+                           'more times' % (service, retry_count))
+            time.sleep(30)
+            proc_start_time = self._get_proc_start_time(sentry_unit, service,
+                                                        pgrep_full)
+            retry_count = retry_count - 1
+
+        if not proc_start_time:
+            self.log.warn('No proc start time found, assuming service did '
+                          'not start')
+            return False
+        if proc_start_time >= mtime:
+            self.log.debug('proc start time is newer than provided mtime'
+                           '(%s >= %s)' % (proc_start_time, mtime))
+            return True
+        else:
+            self.log.warn('proc start time (%s) is older than provided mtime '
+                          '(%s), service did not restart' % (proc_start_time,
+                                                             mtime))
+            return False
+
+    def config_updated_since(self, sentry_unit, filename, mtime,
+                             sleep_time=20):
+        """Check if file was modified after a given time.
+
+        Args:
+          sentry_unit (sentry): The sentry unit to check the file mtime on
+          filename (string): The file to check mtime of
+          mtime (float): The epoch time to check against
+          sleep_time (int): Seconds to sleep before looking for process
+
+        Returns:
+          bool: True if file was modified more recently than mtime, False if
+                file was modified before mtime,
+        """
+        self.log.debug('Checking %s updated since %s' % (filename, mtime))
+        time.sleep(sleep_time)
+        file_mtime = self._get_file_mtime(sentry_unit, filename)
+        if file_mtime >= mtime:
+            self.log.debug('File mtime is newer than provided mtime '
+                           '(%s >= %s)' % (file_mtime, mtime))
+            return True
+        else:
+            self.log.warn('File mtime %s is older than provided mtime %s'
+                          % (file_mtime, mtime))
+            return False
+
+    def validate_service_config_changed(self, sentry_unit, mtime, service,
+                                        filename, pgrep_full=False,
+                                        sleep_time=20, retry_count=2):
+        """Check service and file were updated after mtime
+
+        Args:
+          sentry_unit (sentry): The sentry unit to check for the service on
+          mtime (float): The epoch time to check against
+          service (string): service name to look for in process table
+          filename (string): The file to check mtime of
+          pgrep_full (boolean): Use full command line search mode with pgrep
+          sleep_time (int): Seconds to sleep before looking for process
+          retry_count (int): If service is not found, how many times to retry
+
+        Typical Usage:
+            u = OpenStackAmuletUtils(ERROR)
+            ...
+            mtime = u.get_sentry_time(self.cinder_sentry)
+            self.d.configure('cinder', {'verbose': 'True', 'debug': 'True'})
+            if not u.validate_service_config_changed(self.cinder_sentry,
+                                                     mtime,
+                                                     'cinder-api',
+                                                     '/etc/cinder/cinder.conf')
+                amulet.raise_status(amulet.FAIL, msg='update failed')
+        Returns:
+          bool: True if both service and file where updated/restarted after
+                mtime, False if service is older than mtime or if service was
+                not found or if filename was modified before mtime.
+        """
+        self.log.debug('Checking %s restarted since %s' % (service, mtime))
+        time.sleep(sleep_time)
+        service_restart = self.service_restarted_since(sentry_unit, mtime,
+                                                       service,
+                                                       pgrep_full=pgrep_full,
+                                                       sleep_time=0,
+                                                       retry_count=retry_count)
+        config_update = self.config_updated_since(sentry_unit, filename, mtime,
+                                                  sleep_time=0)
+        return service_restart and config_update
+
+    def get_sentry_time(self, sentry_unit):
+        """Return current epoch time on a sentry"""
+        cmd = "date +'%s'"
+        return float(sentry_unit.run(cmd)[0])
+
+    def relation_error(self, name, data):
+        return 'unexpected relation data in {} - {}'.format(name, data)
+
+    def endpoint_error(self, name, data):
+        return 'unexpected endpoint data in {} - {}'.format(name, data)
 
=== added file 'tests/tests.yaml'
--- tests/tests.yaml	1970-01-01 00:00:00 +0000
+++ tests/tests.yaml	2015-04-30 13:14:23 +0000
@@ -0,0 +1,9 @@
+bootstrap: true
+reset: true
+virtualenv: true
+makefile:
+    - lint
+sources:
+    - ppa:juju/stable
+packages:
+    - amulet