Merge lp:~freyes/charms/trusty/memcached/python-rewrite into lp:charms/trusty/memcached

[Felipe Reyes]

Dear Charmers,

This MP is a rewrite in python to leverage charmhelpers and secure memcached service using ufw rules to only allow access from machines related to the service.

Summary of changes:

* Drop all the bash in favor of python
* Added unit tests
* Changed config type to boolean of following keys:
  * disable-auto-cleanup
  * disable-cas
  * disable-large-pages
* Use ufw to secure memcached[0]

[0] root@ubuntu:~# ufw status # before relating memcache with mediawiki
Status: active

To Action From
-- ------ ----
22 ALLOW Anywhere
4949 ALLOW Anywhere
22 (v6) ALLOW Anywhere (v6)
4949 (v6) ALLOW Anywhere (v6)
21/tcp ALLOW ::1

root@ubuntu:~# ufw status # after juju add-relation memcached mediawiki
Status: active

To Action From
-- ------ ----
22 ALLOW Anywhere
4949 ALLOW Anywhere
11211/tcp ALLOW 192.168.0.120
22 (v6) ALLOW Anywhere (v6)
4949 (v6) ALLOW Anywhere (v6)

[Jorge Niedbalski (niedbalski)]

Hello Felipe,

Thanks for this contribution. I made some inline comments, required to be fixed. Also i have a few suggestions for improving.

On the lint side,

- Remove the 'tags' section on the metadata.yaml file.
- Add the categories: ["system"] on the metadata.yaml file.

On the test side,

- Remove the pre_install_hooks code.
- Add test coverage for the method cache_relation_departed
- Add test coverage for the method upgrade_charm

On the memcache_utils file.

- Please instead of create your own template rendering method, please see:
hooks/charmhelpers/core/templating.py
- Add test coverage for method revoke_access

review: Needs Fixing

[Jorge Niedbalski (niedbalski)]

Hello Felipe,

The 'make test' target doesn't works properly, in fact, it references to files that doesn't longer exists on the project.

Please fix them and OTT LGTM

review: Needs Fixing

[Felipe Reyes (freyes)]

@niedbalski, I fixed the amulet test, please try running 'make test' again. Thanks.

[Review Queue (review-queue)]

This items has failed automated testing! Results available here http://reports.vapour.ws/charm-tests/charm-bundle-test-10637-results

review: Needs Fixing (automated testing)

[Jorge Niedbalski (niedbalski)]

Felipe,

Your makefile is assuming you already have nosetests installed,
Could you please modify your Makefile for making it looking like this one https://pastebin.canonical.com/122033/ ?

review: Needs Fixing

[Jorge Niedbalski (niedbalski)]

Felipe,

Everything looks OK. Tests/Lint passing and you followed the overall suggestions.

Thanks

review: Approve

[Review Queue (review-queue)]

This items has failed automated testing! Results available here http://reports.vapour.ws/charm-tests/charm-bundle-test-10646-results

review: Needs Fixing (automated testing)

=== added file '.bzrignore'
--- .bzrignore	1970-01-01 00:00:00 +0000
+++ .bzrignore	2014-12-09 22:04:35 +0000
@@ -0,0 +1,5 @@
+*.py[co]
+*~
+bin/
+.coverage
+.venv

=== added file 'Makefile'
--- Makefile	1970-01-01 00:00:00 +0000
+++ Makefile	2014-12-09 22:04:35 +0000
@@ -0,0 +1,39 @@
+#!/usr/bin/make
+PYTHON := /usr/bin/env python
+EXTRA  := 
+
+
+clean:
+	rm -f .coverage
+	find . -name '*.pyc' -delete
+	rm -rf .venv
+	(which dh_clean && dh_clean) || true
+
+.venv:
+	sudo apt-get install -y gcc python-dev python-virtualenv python-apt
+	virtualenv .venv --system-site-packages
+	.venv/bin/pip install -I -r test_requirements.txt
+
+lint:   .venv
+	.venv/bin/flake8 --exclude hooks/charmhelpers hooks tests unit_tests
+	@charm proof
+
+test:  clean .venv
+	@echo Starting unit tests...
+	.venv/bin/nosetests -s --nologcapture --with-coverage $(EXTRA) unit_tests/
+
+functional_test:
+	@echo Starting amulet tests...
+	@juju test -v -p AMULET_HTTP_PROXY --timeout 900
+
+bin/charm_helpers_sync.py:
+	@mkdir -p bin
+	@bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \
+        > bin/charm_helpers_sync.py
+
+sync: bin/charm_helpers_sync.py
+	@$(PYTHON) bin/charm_helpers_sync.py -c charm-helpers.yaml
+
+publish: lint unit_test
+	bzr push lp:charms/memcached
+	bzr push lp:charms/trusty/memcached

=== added file 'charm-helpers.yaml'
--- charm-helpers.yaml	1970-01-01 00:00:00 +0000
+++ charm-helpers.yaml	2014-12-09 22:04:35 +0000
@@ -0,0 +1,6 @@
+destination: hooks/charmhelpers
+branch: lp:charm-helpers
+include:
+    - fetch
+    - core
+    - contrib.network

=== modified file 'config.yaml'
--- config.yaml	2013-07-30 12:19:47 +0000
+++ config.yaml	2014-12-09 22:04:35 +0000
@@ -28,8 +28,8 @@
         description: |
             do not remove things automatically from the cache on OOM
             (memcached option -M)
-        default: "no"
-        type: string
+        default: False
+        type: boolean
     factor:
         description: |
             Use <factor> as the multiplier for computing the sizes of memory
@@ -52,8 +52,8 @@
         default: -1
     disable-cas:
         description: disable use of CAS (and reduce the per-item size by 8 bytes)
-        default: "no"
-        type: string
+        default: False
+        type: boolean
     slab-page-size:
         description: |
             Override the size of each slab page in bytes. In mundane
@@ -72,8 +72,8 @@
         type: int
     disable-large-pages:
         description: The charm will will try to use large pages if given more than 2GB of RAM. You may want to disable this behavior. (memcached option -L)
-        type: string
-        default: "no"
+        type: boolean
+        default: False
     extra-options:
         description: memcached has many other options documented in its man page. You may pass them here as a string which will be appended to memcached's execution.
         type: string

=== added symlink 'hooks/cache-relation-departed'
=== target is u'memcached_hooks.py'
=== modified file 'hooks/cache-relation-joined'
--- hooks/cache-relation-joined	2012-09-20 11:43:24 +0000
+++ hooks/cache-relation-joined	1970-01-01 00:00:00 +0000
@@ -1,9 +0,0 @@
-#!/bin/sh
-if [ -z "$JUJU_RELATION_ID" ] ; then
-    rids=$(relation-ids cache)
-else
-    rids=$JUJU_RELATION_ID
-fi
-for rid in $rids ; do
-    relation-set -r $rid host=`unit-get private-address` port=`config-get tcp-port` udp-port=`config-get udp-port`
-done

=== target is u'memcached_hooks.py'
=== added directory 'hooks/charmhelpers'
=== added file 'hooks/charmhelpers/__init__.py'
=== added directory 'hooks/charmhelpers/contrib'
=== added file 'hooks/charmhelpers/contrib/__init__.py'
=== added directory 'hooks/charmhelpers/contrib/network'
=== added file 'hooks/charmhelpers/contrib/network/__init__.py'
=== added file 'hooks/charmhelpers/contrib/network/ip.py'
--- hooks/charmhelpers/contrib/network/ip.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/network/ip.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,351 @@
+import glob
+import re
+import subprocess
+
+from functools import partial
+
+from charmhelpers.core.hookenv import unit_get
+from charmhelpers.fetch import apt_install
+from charmhelpers.core.hookenv import (
+    log
+)
+
+try:
+    import netifaces
+except ImportError:
+    apt_install('python-netifaces')
+    import netifaces
+
+try:
+    import netaddr
+except ImportError:
+    apt_install('python-netaddr')
+    import netaddr
+
+
+def _validate_cidr(network):
+    try:
+        netaddr.IPNetwork(network)
+    except (netaddr.core.AddrFormatError, ValueError):
+        raise ValueError("Network (%s) is not in CIDR presentation format" %
+                         network)
+
+
+def no_ip_found_error_out(network):
+    errmsg = ("No IP address found in network: %s" % network)
+    raise ValueError(errmsg)
+
+
+def get_address_in_network(network, fallback=None, fatal=False):
+    """Get an IPv4 or IPv6 address within the network from the host.
+
+    :param network (str): CIDR presentation format. For example,
+        '192.168.1.0/24'.
+    :param fallback (str): If no address is found, return fallback.
+    :param fatal (boolean): If no address is found, fallback is not
+        set and fatal is True then exit(1).
+    """
+    if network is None:
+        if fallback is not None:
+            return fallback
+
+        if fatal:
+            no_ip_found_error_out(network)
+        else:
+            return None
+
+    _validate_cidr(network)
+    network = netaddr.IPNetwork(network)
+    for iface in netifaces.interfaces():
+        addresses = netifaces.ifaddresses(iface)
+        if network.version == 4 and netifaces.AF_INET in addresses:
+            addr = addresses[netifaces.AF_INET][0]['addr']
+            netmask = addresses[netifaces.AF_INET][0]['netmask']
+            cidr = netaddr.IPNetwork("%s/%s" % (addr, netmask))
+            if cidr in network:
+                return str(cidr.ip)
+
+        if network.version == 6 and netifaces.AF_INET6 in addresses:
+            for addr in addresses[netifaces.AF_INET6]:
+                if not addr['addr'].startswith('fe80'):
+                    cidr = netaddr.IPNetwork("%s/%s" % (addr['addr'],
+                                                        addr['netmask']))
+                    if cidr in network:
+                        return str(cidr.ip)
+
+    if fallback is not None:
+        return fallback
+
+    if fatal:
+        no_ip_found_error_out(network)
+
+    return None
+
+
+def is_ipv6(address):
+    """Determine whether provided address is IPv6 or not."""
+    try:
+        address = netaddr.IPAddress(address)
+    except netaddr.AddrFormatError:
+        # probably a hostname - so not an address at all!
+        return False
+
+    return address.version == 6
+
+
+def is_address_in_network(network, address):
+    """
+    Determine whether the provided address is within a network range.
+
+    :param network (str): CIDR presentation format. For example,
+        '192.168.1.0/24'.
+    :param address: An individual IPv4 or IPv6 address without a net
+        mask or subnet prefix. For example, '192.168.1.1'.
+    :returns boolean: Flag indicating whether address is in network.
+    """
+    try:
+        network = netaddr.IPNetwork(network)
+    except (netaddr.core.AddrFormatError, ValueError):
+        raise ValueError("Network (%s) is not in CIDR presentation format" %
+                         network)
+
+    try:
+        address = netaddr.IPAddress(address)
+    except (netaddr.core.AddrFormatError, ValueError):
+        raise ValueError("Address (%s) is not in correct presentation format" %
+                         address)
+
+    if address in network:
+        return True
+    else:
+        return False
+
+
+def _get_for_address(address, key):
+    """Retrieve an attribute of or the physical interface that
+    the IP address provided could be bound to.
+
+    :param address (str): An individual IPv4 or IPv6 address without a net
+        mask or subnet prefix. For example, '192.168.1.1'.
+    :param key: 'iface' for the physical interface name or an attribute
+        of the configured interface, for example 'netmask'.
+    :returns str: Requested attribute or None if address is not bindable.
+    """
+    address = netaddr.IPAddress(address)
+    for iface in netifaces.interfaces():
+        addresses = netifaces.ifaddresses(iface)
+        if address.version == 4 and netifaces.AF_INET in addresses:
+            addr = addresses[netifaces.AF_INET][0]['addr']
+            netmask = addresses[netifaces.AF_INET][0]['netmask']
+            network = netaddr.IPNetwork("%s/%s" % (addr, netmask))
+            cidr = network.cidr
+            if address in cidr:
+                if key == 'iface':
+                    return iface
+                else:
+                    return addresses[netifaces.AF_INET][0][key]
+
+        if address.version == 6 and netifaces.AF_INET6 in addresses:
+            for addr in addresses[netifaces.AF_INET6]:
+                if not addr['addr'].startswith('fe80'):
+                    network = netaddr.IPNetwork("%s/%s" % (addr['addr'],
+                                                           addr['netmask']))
+                    cidr = network.cidr
+                    if address in cidr:
+                        if key == 'iface':
+                            return iface
+                        elif key == 'netmask' and cidr:
+                            return str(cidr).split('/')[1]
+                        else:
+                            return addr[key]
+
+    return None
+
+
+get_iface_for_address = partial(_get_for_address, key='iface')
+
+
+get_netmask_for_address = partial(_get_for_address, key='netmask')
+
+
+def format_ipv6_addr(address):
+    """If address is IPv6, wrap it in '[]' otherwise return None.
+
+    This is required by most configuration files when specifying IPv6
+    addresses.
+    """
+    if is_ipv6(address):
+        return "[%s]" % address
+
+    return None
+
+
+def get_iface_addr(iface='eth0', inet_type='AF_INET', inc_aliases=False,
+                   fatal=True, exc_list=None):
+    """Return the assigned IP address for a given interface, if any."""
+    # Extract nic if passed /dev/ethX
+    if '/' in iface:
+        iface = iface.split('/')[-1]
+
+    if not exc_list:
+        exc_list = []
+
+    try:
+        inet_num = getattr(netifaces, inet_type)
+    except AttributeError:
+        raise Exception("Unknown inet type '%s'" % str(inet_type))
+
+    interfaces = netifaces.interfaces()
+    if inc_aliases:
+        ifaces = []
+        for _iface in interfaces:
+            if iface == _iface or _iface.split(':')[0] == iface:
+                ifaces.append(_iface)
+
+        if fatal and not ifaces:
+            raise Exception("Invalid interface '%s'" % iface)
+
+        ifaces.sort()
+    else:
+        if iface not in interfaces:
+            if fatal:
+                raise Exception("Interface '%s' not found " % (iface))
+            else:
+                return []
+
+        else:
+            ifaces = [iface]
+
+    addresses = []
+    for netiface in ifaces:
+        net_info = netifaces.ifaddresses(netiface)
+        if inet_num in net_info:
+            for entry in net_info[inet_num]:
+                if 'addr' in entry and entry['addr'] not in exc_list:
+                    addresses.append(entry['addr'])
+
+    if fatal and not addresses:
+        raise Exception("Interface '%s' doesn't have any %s addresses." %
+                        (iface, inet_type))
+
+    return sorted(addresses)
+
+
+get_ipv4_addr = partial(get_iface_addr, inet_type='AF_INET')
+
+
+def get_iface_from_addr(addr):
+    """Work out on which interface the provided address is configured."""
+    for iface in netifaces.interfaces():
+        addresses = netifaces.ifaddresses(iface)
+        for inet_type in addresses:
+            for _addr in addresses[inet_type]:
+                _addr = _addr['addr']
+                # link local
+                ll_key = re.compile("(.+)%.*")
+                raw = re.match(ll_key, _addr)
+                if raw:
+                    _addr = raw.group(1)
+
+                if _addr == addr:
+                    log("Address '%s' is configured on iface '%s'" %
+                        (addr, iface))
+                    return iface
+
+    msg = "Unable to infer net iface on which '%s' is configured" % (addr)
+    raise Exception(msg)
+
+
+def sniff_iface(f):
+    """Ensure decorated function is called with a value for iface.
+
+    If no iface provided, inject net iface inferred from unit private address.
+    """
+    def iface_sniffer(*args, **kwargs):
+        if not kwargs.get('iface', None):
+            kwargs['iface'] = get_iface_from_addr(unit_get('private-address'))
+
+        return f(*args, **kwargs)
+
+    return iface_sniffer
+
+
+@sniff_iface
+def get_ipv6_addr(iface=None, inc_aliases=False, fatal=True, exc_list=None,
+                  dynamic_only=True):
+    """Get assigned IPv6 address for a given interface.
+
+    Returns list of addresses found. If no address found, returns empty list.
+
+    If iface is None, we infer the current primary interface by doing a reverse
+    lookup on the unit private-address.
+
+    We currently only support scope global IPv6 addresses i.e. non-temporary
+    addresses. If no global IPv6 address is found, return the first one found
+    in the ipv6 address list.
+    """
+    addresses = get_iface_addr(iface=iface, inet_type='AF_INET6',
+                               inc_aliases=inc_aliases, fatal=fatal,
+                               exc_list=exc_list)
+
+    if addresses:
+        global_addrs = []
+        for addr in addresses:
+            key_scope_link_local = re.compile("^fe80::..(.+)%(.+)")
+            m = re.match(key_scope_link_local, addr)
+            if m:
+                eui_64_mac = m.group(1)
+                iface = m.group(2)
+            else:
+                global_addrs.append(addr)
+
+        if global_addrs:
+            # Make sure any found global addresses are not temporary
+            cmd = ['ip', 'addr', 'show', iface]
+            out = subprocess.check_output(cmd).decode('UTF-8')
+            if dynamic_only:
+                key = re.compile("inet6 (.+)/[0-9]+ scope global dynamic.*")
+            else:
+                key = re.compile("inet6 (.+)/[0-9]+ scope global.*")
+
+            addrs = []
+            for line in out.split('\n'):
+                line = line.strip()
+                m = re.match(key, line)
+                if m and 'temporary' not in line:
+                    # Return the first valid address we find
+                    for addr in global_addrs:
+                        if m.group(1) == addr:
+                            if not dynamic_only or \
+                                    m.group(1).endswith(eui_64_mac):
+                                addrs.append(addr)
+
+            if addrs:
+                return addrs
+
+    if fatal:
+        raise Exception("Interface '%s' does not have a scope global "
+                        "non-temporary ipv6 address." % iface)
+
+    return []
+
+
+def get_bridges(vnic_dir='/sys/devices/virtual/net'):
+    """Return a list of bridges on the system."""
+    b_regex = "%s/*/bridge" % vnic_dir
+    return [x.replace(vnic_dir, '').split('/')[1] for x in glob.glob(b_regex)]
+
+
+def get_bridge_nics(bridge, vnic_dir='/sys/devices/virtual/net'):
+    """Return a list of nics comprising a given bridge on the system."""
+    brif_regex = "%s/%s/brif/*" % (vnic_dir, bridge)
+    return [x.split('/')[-1] for x in glob.glob(brif_regex)]
+
+
+def is_bridge_member(nic):
+    """Check if a given nic is a member of a bridge."""
+    for bridge in get_bridges():
+        if nic in get_bridge_nics(bridge):
+            return True
+
+    return False

=== added directory 'hooks/charmhelpers/contrib/network/ovs'
=== added file 'hooks/charmhelpers/contrib/network/ovs/__init__.py'
--- hooks/charmhelpers/contrib/network/ovs/__init__.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/network/ovs/__init__.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,80 @@
+''' Helpers for interacting with OpenvSwitch '''
+import subprocess
+import os
+from charmhelpers.core.hookenv import (
+    log, WARNING
+)
+from charmhelpers.core.host import (
+    service
+)
+
+
+def add_bridge(name):
+    ''' Add the named bridge to openvswitch '''
+    log('Creating bridge {}'.format(name))
+    subprocess.check_call(["ovs-vsctl", "--", "--may-exist", "add-br", name])
+
+
+def del_bridge(name):
+    ''' Delete the named bridge from openvswitch '''
+    log('Deleting bridge {}'.format(name))
+    subprocess.check_call(["ovs-vsctl", "--", "--if-exists", "del-br", name])
+
+
+def add_bridge_port(name, port, promisc=False):
+    ''' Add a port to the named openvswitch bridge '''
+    log('Adding port {} to bridge {}'.format(port, name))
+    subprocess.check_call(["ovs-vsctl", "--", "--may-exist", "add-port",
+                           name, port])
+    subprocess.check_call(["ip", "link", "set", port, "up"])
+    if promisc:
+        subprocess.check_call(["ip", "link", "set", port, "promisc", "on"])
+    else:
+        subprocess.check_call(["ip", "link", "set", port, "promisc", "off"])
+
+
+def del_bridge_port(name, port):
+    ''' Delete a port from the named openvswitch bridge '''
+    log('Deleting port {} from bridge {}'.format(port, name))
+    subprocess.check_call(["ovs-vsctl", "--", "--if-exists", "del-port",
+                           name, port])
+    subprocess.check_call(["ip", "link", "set", port, "down"])
+    subprocess.check_call(["ip", "link", "set", port, "promisc", "off"])
+
+
+def set_manager(manager):
+    ''' Set the controller for the local openvswitch '''
+    log('Setting manager for local ovs to {}'.format(manager))
+    subprocess.check_call(['ovs-vsctl', 'set-manager',
+                           'ssl:{}'.format(manager)])
+
+
+CERT_PATH = '/etc/openvswitch/ovsclient-cert.pem'
+
+
+def get_certificate():
+    ''' Read openvswitch certificate from disk '''
+    if os.path.exists(CERT_PATH):
+        log('Reading ovs certificate from {}'.format(CERT_PATH))
+        with open(CERT_PATH, 'r') as cert:
+            full_cert = cert.read()
+            begin_marker = "-----BEGIN CERTIFICATE-----"
+            end_marker = "-----END CERTIFICATE-----"
+            begin_index = full_cert.find(begin_marker)
+            end_index = full_cert.rfind(end_marker)
+            if end_index == -1 or begin_index == -1:
+                raise RuntimeError("Certificate does not contain valid begin"
+                                   " and end markers.")
+            full_cert = full_cert[begin_index:(end_index + len(end_marker))]
+            return full_cert
+    else:
+        log('Certificate not found', level=WARNING)
+        return None
+
+
+def full_restart():
+    ''' Full restart and reload of openvswitch '''
+    if os.path.exists('/etc/init/openvswitch-force-reload-kmod.conf'):
+        service('start', 'openvswitch-force-reload-kmod')
+    else:
+        service('force-reload-kmod', 'openvswitch-switch')

=== added file 'hooks/charmhelpers/contrib/network/ufw.py'
--- hooks/charmhelpers/contrib/network/ufw.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/network/ufw.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,189 @@
+"""
+This module contains helpers to add and remove ufw rules.
+
+Examples:
+
+- open SSH port for subnet 10.0.3.0/24:
+
+  >>> from charmhelpers.contrib.network import ufw
+  >>> ufw.enable()
+  >>> ufw.grant_access(src='10.0.3.0/24', dst='any', port='22', proto='tcp')
+
+- open service by name as defined in /etc/services:
+
+  >>> from charmhelpers.contrib.network import ufw
+  >>> ufw.enable()
+  >>> ufw.service('ssh', 'open')
+
+- close service by port number:
+
+  >>> from charmhelpers.contrib.network import ufw
+  >>> ufw.enable()
+  >>> ufw.service('4949', 'close')  # munin
+"""
+
+__author__ = "Felipe Reyes <[e-mail]>"
+
+import re
+import os
+import subprocess
+from charmhelpers.core import hookenv
+
+
+def is_enabled():
+    """
+    Check if `ufw` is enabled
+
+    :returns: True if ufw is enabled
+    """
+    output = subprocess.check_output(['ufw', 'status'],
+                                     env={'LANG': 'en_US',
+                                          'PATH': os.environ['PATH']})
+
+    m = re.findall(r'^Status: active\n', output, re.M)
+
+    return len(m) >= 1
+
+
+def enable():
+    """
+    Enable ufw
+
+    :returns: True if ufw is successfully enabled
+    """
+    if is_enabled():
+        return True
+
+    output = subprocess.check_output(['ufw', 'enable'],
+                                     env={'LANG': 'en_US',
+                                          'PATH': os.environ['PATH']})
+
+    m = re.findall('^Firewall is active and enabled on system startup\n',
+                   output, re.M)
+    hookenv.log(output, level='DEBUG')
+
+    if len(m) == 0:
+        hookenv.log("ufw couldn't be enabled", level='WARN')
+        return False
+    else:
+        hookenv.log("ufw enabled", level='INFO')
+        return True
+
+
+def disable():
+    """
+    Disable ufw
+
+    :returns: True if ufw is successfully disabled
+    """
+    if not is_enabled():
+        return True
+
+    output = subprocess.check_output(['ufw', 'disable'],
+                                     env={'LANG': 'en_US',
+                                          'PATH': os.environ['PATH']})
+
+    m = re.findall(r'^Firewall stopped and disabled on system startup\n',
+                   output, re.M)
+    hookenv.log(output, level='DEBUG')
+
+    if len(m) == 0:
+        hookenv.log("ufw couldn't be disabled", level='WARN')
+        return False
+    else:
+        hookenv.log("ufw disabled", level='INFO')
+        return True
+
+
+def modify_access(src, dst='any', port=None, proto=None, action='allow'):
+    """
+    Grant access to an address or subnet
+
+    :param src: address (e.g. 192.168.1.234) or subnet
+                (e.g. 192.168.1.0/24).
+    :param dst: destiny of the connection, if the machine has multiple IPs and
+                connections to only one of those have to accepted this is the
+                field has to be set.
+    :param port: destiny port
+    :param proto: protocol (tcp or udp)
+    :param action: `allow` or `delete`
+    """
+    if not is_enabled():
+        hookenv.log('ufw is disabled, skipping modify_access()', level='WARN')
+        return
+
+    if action == 'delete':
+        cmd = ['ufw', 'delete', 'allow']
+    else:
+        cmd = ['ufw', action]
+
+    if src is not None:
+        cmd += ['from', src]
+
+    if dst is not None:
+        cmd += ['to', dst]
+
+    if port is not None:
+        cmd += ['port', port]
+
+    if proto is not None:
+        cmd += ['proto', proto]
+
+    hookenv.log('ufw {}: {}'.format(action, ' '.join(cmd)), level='DEBUG')
+    p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
+    (stdout, stderr) = p.communicate()
+
+    hookenv.log(stdout, level='INFO')
+
+    if p.returncode != 0:
+        hookenv.log(stderr, level='ERROR')
+        hookenv.log('Error running: {}, exit code: {}'.format(' '.join(cmd),
+                                                              p.returncode),
+                    level='ERROR')
+
+
+def grant_access(src, dst='any', port=None, proto=None):
+    """
+    Grant access to an address or subnet
+
+    :param src: address (e.g. 192.168.1.234) or subnet
+                (e.g. 192.168.1.0/24).
+    :param dst: destiny of the connection, if the machine has multiple IPs and
+                connections to only one of those have to accepted this is the
+                field has to be set.
+    :param port: destiny port
+    :param proto: protocol (tcp or udp)
+    """
+    return modify_access(src, dst=dst, port=port, proto=proto, action='allow')
+
+
+def revoke_access(src, dst='any', port=None, proto=None):
+    """
+    Revoke access to an address or subnet
+
+    :param src: address (e.g. 192.168.1.234) or subnet
+                (e.g. 192.168.1.0/24).
+    :param dst: destiny of the connection, if the machine has multiple IPs and
+                connections to only one of those have to accepted this is the
+                field has to be set.
+    :param port: destiny port
+    :param proto: protocol (tcp or udp)
+    """
+    return modify_access(src, dst=dst, port=port, proto=proto, action='delete')
+
+
+def service(name, action):
+    """
+    Open/close access to a service
+
+    :param name: could be a service name defined in `/etc/services` or a port
+                 number.
+    :param action: `open` or `close`
+    """
+    if action == 'open':
+        subprocess.check_output(['ufw', 'allow', name])
+    elif action == 'close':
+        subprocess.check_output(['ufw', 'delete', 'allow', name])
+    else:
+        raise Exception(("'{}' not supported, use 'allow' "
+                         "or 'delete'").format(action))

=== added directory 'hooks/charmhelpers/core'
=== added file 'hooks/charmhelpers/core/__init__.py'
=== added file 'hooks/charmhelpers/core/fstab.py'
--- hooks/charmhelpers/core/fstab.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/fstab.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,118 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+__author__ = 'Jorge Niedbalski R. <[e-mail]>'
+
+import io
+import os
+
+
+class Fstab(io.FileIO):
+    """This class extends file in order to implement a file reader/writer
+    for file `/etc/fstab`
+    """
+
+    class Entry(object):
+        """Entry class represents a non-comment line on the `/etc/fstab` file
+        """
+        def __init__(self, device, mountpoint, filesystem,
+                     options, d=0, p=0):
+            self.device = device
+            self.mountpoint = mountpoint
+            self.filesystem = filesystem
+
+            if not options:
+                options = "defaults"
+
+            self.options = options
+            self.d = int(d)
+            self.p = int(p)
+
+        def __eq__(self, o):
+            return str(self) == str(o)
+
+        def __str__(self):
+            return "{} {} {} {} {} {}".format(self.device,
+                                              self.mountpoint,
+                                              self.filesystem,
+                                              self.options,
+                                              self.d,
+                                              self.p)
+
+    DEFAULT_PATH = os.path.join(os.path.sep, 'etc', 'fstab')
+
+    def __init__(self, path=None):
+        if path:
+            self._path = path
+        else:
+            self._path = self.DEFAULT_PATH
+        super(Fstab, self).__init__(self._path, 'rb+')
+
+    def _hydrate_entry(self, line):
+        # NOTE: use split with no arguments to split on any
+        #       whitespace including tabs
+        return Fstab.Entry(*filter(
+            lambda x: x not in ('', None),
+            line.strip("\n").split()))
+
+    @property
+    def entries(self):
+        self.seek(0)
+        for line in self.readlines():
+            line = line.decode('us-ascii')
+            try:
+                if line.strip() and not line.startswith("#"):
+                    yield self._hydrate_entry(line)
+            except ValueError:
+                pass
+
+    def get_entry_by_attr(self, attr, value):
+        for entry in self.entries:
+            e_attr = getattr(entry, attr)
+            if e_attr == value:
+                return entry
+        return None
+
+    def add_entry(self, entry):
+        if self.get_entry_by_attr('device', entry.device):
+            return False
+
+        self.write((str(entry) + '\n').encode('us-ascii'))
+        self.truncate()
+        return entry
+
+    def remove_entry(self, entry):
+        self.seek(0)
+
+        lines = [l.decode('us-ascii') for l in self.readlines()]
+
+        found = False
+        for index, line in enumerate(lines):
+            if not line.startswith("#"):
+                if self._hydrate_entry(line) == entry:
+                    found = True
+                    break
+
+        if not found:
+            return False
+
+        lines.remove(line)
+
+        self.seek(0)
+        self.write(''.join(lines).encode('us-ascii'))
+        self.truncate()
+        return True
+
+    @classmethod
+    def remove_by_mountpoint(cls, mountpoint, path=None):
+        fstab = cls(path=path)
+        entry = fstab.get_entry_by_attr('mountpoint', mountpoint)
+        if entry:
+            return fstab.remove_entry(entry)
+        return False
+
+    @classmethod
+    def add(cls, device, mountpoint, filesystem, options=None, path=None):
+        return cls(path=path).add_entry(Fstab.Entry(device,
+                                                    mountpoint, filesystem,
+                                                    options=options))

=== added file 'hooks/charmhelpers/core/hookenv.py'
--- hooks/charmhelpers/core/hookenv.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/hookenv.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,542 @@
+"Interactions with the Juju environment"
+# Copyright 2013 Canonical Ltd.
+#
+# Authors:
+#  Charm Helpers Developers <[e-mail]>
+
+import os
+import json
+import yaml
+import subprocess
+import sys
+from subprocess import CalledProcessError
+
+import six
+if not six.PY3:
+    from UserDict import UserDict
+else:
+    from collections import UserDict
+
+CRITICAL = "CRITICAL"
+ERROR = "ERROR"
+WARNING = "WARNING"
+INFO = "INFO"
+DEBUG = "DEBUG"
+MARKER = object()
+
+cache = {}
+
+
+def cached(func):
+    """Cache return values for multiple executions of func + args
+
+    For example::
+
+        @cached
+        def unit_get(attribute):
+            pass
+
+        unit_get('test')
+
+    will cache the result of unit_get + 'test' for future calls.
+    """
+    def wrapper(*args, **kwargs):
+        global cache
+        key = str((func, args, kwargs))
+        try:
+            return cache[key]
+        except KeyError:
+            res = func(*args, **kwargs)
+            cache[key] = res
+            return res
+    return wrapper
+
+
+def flush(key):
+    """Flushes any entries from function cache where the
+    key is found in the function+args """
+    flush_list = []
+    for item in cache:
+        if key in item:
+            flush_list.append(item)
+    for item in flush_list:
+        del cache[item]
+
+
+def log(message, level=None):
+    """Write a message to the juju log"""
+    command = ['juju-log']
+    if level:
+        command += ['-l', level]
+    if not isinstance(message, six.string_types):
+        message = repr(message)
+    command += [message]
+    subprocess.call(command)
+
+
+class Serializable(UserDict):
+    """Wrapper, an object that can be serialized to yaml or json"""
+
+    def __init__(self, obj):
+        # wrap the object
+        UserDict.__init__(self)
+        self.data = obj
+
+    def __getattr__(self, attr):
+        # See if this object has attribute.
+        if attr in ("json", "yaml", "data"):
+            return self.__dict__[attr]
+        # Check for attribute in wrapped object.
+        got = getattr(self.data, attr, MARKER)
+        if got is not MARKER:
+            return got
+        # Proxy to the wrapped object via dict interface.
+        try:
+            return self.data[attr]
+        except KeyError:
+            raise AttributeError(attr)
+
+    def __getstate__(self):
+        # Pickle as a standard dictionary.
+        return self.data
+
+    def __setstate__(self, state):
+        # Unpickle into our wrapper.
+        self.data = state
+
+    def json(self):
+        """Serialize the object to json"""
+        return json.dumps(self.data)
+
+    def yaml(self):
+        """Serialize the object to yaml"""
+        return yaml.dump(self.data)
+
+
+def execution_environment():
+    """A convenient bundling of the current execution context"""
+    context = {}
+    context['conf'] = config()
+    if relation_id():
+        context['reltype'] = relation_type()
+        context['relid'] = relation_id()
+        context['rel'] = relation_get()
+    context['unit'] = local_unit()
+    context['rels'] = relations()
+    context['env'] = os.environ
+    return context
+
+
+def in_relation_hook():
+    """Determine whether we're running in a relation hook"""
+    return 'JUJU_RELATION' in os.environ
+
+
+def relation_type():
+    """The scope for the current relation hook"""
+    return os.environ.get('JUJU_RELATION', None)
+
+
+def relation_id():
+    """The relation ID for the current relation hook"""
+    return os.environ.get('JUJU_RELATION_ID', None)
+
+
+def local_unit():
+    """Local unit ID"""
+    return os.environ['JUJU_UNIT_NAME']
+
+
+def remote_unit():
+    """The remote unit for the current relation hook"""
+    return os.environ['JUJU_REMOTE_UNIT']
+
+
+def service_name():
+    """The name service group this unit belongs to"""
+    return local_unit().split('/')[0]
+
+
+def hook_name():
+    """The name of the currently executing hook"""
+    return os.path.basename(sys.argv[0])
+
+
+class Config(dict):
+    """A dictionary representation of the charm's config.yaml, with some
+    extra features:
+
+    - See which values in the dictionary have changed since the previous hook.
+    - For values that have changed, see what the previous value was.
+    - Store arbitrary data for use in a later hook.
+
+    NOTE: Do not instantiate this object directly - instead call
+    ``hookenv.config()``, which will return an instance of :class:`Config`.
+
+    Example usage::
+
+        >>> # inside a hook
+        >>> from charmhelpers.core import hookenv
+        >>> config = hookenv.config()
+        >>> config['foo']
+        'bar'
+        >>> # store a new key/value for later use
+        >>> config['mykey'] = 'myval'
+
+
+        >>> # user runs `juju set mycharm foo=baz`
+        >>> # now we're inside subsequent config-changed hook
+        >>> config = hookenv.config()
+        >>> config['foo']
+        'baz'
+        >>> # test to see if this val has changed since last hook
+        >>> config.changed('foo')
+        True
+        >>> # what was the previous value?
+        >>> config.previous('foo')
+        'bar'
+        >>> # keys/values that we add are preserved across hooks
+        >>> config['mykey']
+        'myval'
+
+    """
+    CONFIG_FILE_NAME = '.juju-persistent-config'
+
+    def __init__(self, *args, **kw):
+        super(Config, self).__init__(*args, **kw)
+        self.implicit_save = True
+        self._prev_dict = None
+        self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME)
+        if os.path.exists(self.path):
+            self.load_previous()
+
+    def __getitem__(self, key):
+        """For regular dict lookups, check the current juju config first,
+        then the previous (saved) copy. This ensures that user-saved values
+        will be returned by a dict lookup.
+
+        """
+        try:
+            return dict.__getitem__(self, key)
+        except KeyError:
+            return (self._prev_dict or {})[key]
+
+    def keys(self):
+        prev_keys = []
+        if self._prev_dict is not None:
+            prev_keys = self._prev_dict.keys()
+        return list(set(prev_keys + list(dict.keys(self))))
+
+    def load_previous(self, path=None):
+        """Load previous copy of config from disk.
+
+        In normal usage you don't need to call this method directly - it
+        is called automatically at object initialization.
+
+        :param path:
+
+            File path from which to load the previous config. If `None`,
+            config is loaded from the default location. If `path` is
+            specified, subsequent `save()` calls will write to the same
+            path.
+
+        """
+        self.path = path or self.path
+        with open(self.path) as f:
+            self._prev_dict = json.load(f)
+
+    def changed(self, key):
+        """Return True if the current value for this key is different from
+        the previous value.
+
+        """
+        if self._prev_dict is None:
+            return True
+        return self.previous(key) != self.get(key)
+
+    def previous(self, key):
+        """Return previous value for this key, or None if there
+        is no previous value.
+
+        """
+        if self._prev_dict:
+            return self._prev_dict.get(key)
+        return None
+
+    def save(self):
+        """Save this config to disk.
+
+        If the charm is using the :mod:`Services Framework <services.base>`
+        or :meth:'@hook <Hooks.hook>' decorator, this
+        is called automatically at the end of successful hook execution.
+        Otherwise, it should be called directly by user code.
+
+        To disable automatic saves, set ``implicit_save=False`` on this
+        instance.
+
+        """
+        if self._prev_dict:
+            for k, v in six.iteritems(self._prev_dict):
+                if k not in self:
+                    self[k] = v
+        with open(self.path, 'w') as f:
+            json.dump(self, f)
+
+
+@cached
+def config(scope=None):
+    """Juju charm configuration"""
+    config_cmd_line = ['config-get']
+    if scope is not None:
+        config_cmd_line.append(scope)
+    config_cmd_line.append('--format=json')
+    try:
+        config_data = json.loads(
+            subprocess.check_output(config_cmd_line).decode('UTF-8'))
+        if scope is not None:
+            return config_data
+        return Config(config_data)
+    except ValueError:
+        return None
+
+
+@cached
+def relation_get(attribute=None, unit=None, rid=None):
+    """Get relation information"""
+    _args = ['relation-get', '--format=json']
+    if rid:
+        _args.append('-r')
+        _args.append(rid)
+    _args.append(attribute or '-')
+    if unit:
+        _args.append(unit)
+    try:
+        return json.loads(subprocess.check_output(_args).decode('UTF-8'))
+    except ValueError:
+        return None
+    except CalledProcessError as e:
+        if e.returncode == 2:
+            return None
+        raise
+
+
+def relation_set(relation_id=None, relation_settings=None, **kwargs):
+    """Set relation information for the current unit"""
+    relation_settings = relation_settings if relation_settings else {}
+    relation_cmd_line = ['relation-set']
+    if relation_id is not None:
+        relation_cmd_line.extend(('-r', relation_id))
+    for k, v in (list(relation_settings.items()) + list(kwargs.items())):
+        if v is None:
+            relation_cmd_line.append('{}='.format(k))
+        else:
+            relation_cmd_line.append('{}={}'.format(k, v))
+    subprocess.check_call(relation_cmd_line)
+    # Flush cache of any relation-gets for local unit
+    flush(local_unit())
+
+
+@cached
+def relation_ids(reltype=None):
+    """A list of relation_ids"""
+    reltype = reltype or relation_type()
+    relid_cmd_line = ['relation-ids', '--format=json']
+    if reltype is not None:
+        relid_cmd_line.append(reltype)
+        return json.loads(
+            subprocess.check_output(relid_cmd_line).decode('UTF-8')) or []
+    return []
+
+
+@cached
+def related_units(relid=None):
+    """A list of related units"""
+    relid = relid or relation_id()
+    units_cmd_line = ['relation-list', '--format=json']
+    if relid is not None:
+        units_cmd_line.extend(('-r', relid))
+    return json.loads(
+        subprocess.check_output(units_cmd_line).decode('UTF-8')) or []
+
+
+@cached
+def relation_for_unit(unit=None, rid=None):
+    """Get the json represenation of a unit's relation"""
+    unit = unit or remote_unit()
+    relation = relation_get(unit=unit, rid=rid)
+    for key in relation:
+        if key.endswith('-list'):
+            relation[key] = relation[key].split()
+    relation['__unit__'] = unit
+    return relation
+
+
+@cached
+def relations_for_id(relid=None):
+    """Get relations of a specific relation ID"""
+    relation_data = []
+    relid = relid or relation_ids()
+    for unit in related_units(relid):
+        unit_data = relation_for_unit(unit, relid)
+        unit_data['__relid__'] = relid
+        relation_data.append(unit_data)
+    return relation_data
+
+
+@cached
+def relations_of_type(reltype=None):
+    """Get relations of a specific type"""
+    relation_data = []
+    reltype = reltype or relation_type()
+    for relid in relation_ids(reltype):
+        for relation in relations_for_id(relid):
+            relation['__relid__'] = relid
+            relation_data.append(relation)
+    return relation_data
+
+
+@cached
+def relation_types():
+    """Get a list of relation types supported by this charm"""
+    charmdir = os.environ.get('CHARM_DIR', '')
+    mdf = open(os.path.join(charmdir, 'metadata.yaml'))
+    md = yaml.safe_load(mdf)
+    rel_types = []
+    for key in ('provides', 'requires', 'peers'):
+        section = md.get(key)
+        if section:
+            rel_types.extend(section.keys())
+    mdf.close()
+    return rel_types
+
+
+@cached
+def relations():
+    """Get a nested dictionary of relation data for all related units"""
+    rels = {}
+    for reltype in relation_types():
+        relids = {}
+        for relid in relation_ids(reltype):
+            units = {local_unit(): relation_get(unit=local_unit(), rid=relid)}
+            for unit in related_units(relid):
+                reldata = relation_get(unit=unit, rid=relid)
+                units[unit] = reldata
+            relids[relid] = units
+        rels[reltype] = relids
+    return rels
+
+
+@cached
+def is_relation_made(relation, keys='private-address'):
+    '''
+    Determine whether a relation is established by checking for
+    presence of key(s).  If a list of keys is provided, they
+    must all be present for the relation to be identified as made
+    '''
+    if isinstance(keys, str):
+        keys = [keys]
+    for r_id in relation_ids(relation):
+        for unit in related_units(r_id):
+            context = {}
+            for k in keys:
+                context[k] = relation_get(k, rid=r_id,
+                                          unit=unit)
+            if None not in context.values():
+                return True
+    return False
+
+
+def open_port(port, protocol="TCP"):
+    """Open a service network port"""
+    _args = ['open-port']
+    _args.append('{}/{}'.format(port, protocol))
+    subprocess.check_call(_args)
+
+
+def close_port(port, protocol="TCP"):
+    """Close a service network port"""
+    _args = ['close-port']
+    _args.append('{}/{}'.format(port, protocol))
+    subprocess.check_call(_args)
+
+
+@cached
+def unit_get(attribute):
+    """Get the unit ID for the remote unit"""
+    _args = ['unit-get', '--format=json', attribute]
+    try:
+        return json.loads(subprocess.check_output(_args).decode('UTF-8'))
+    except ValueError:
+        return None
+
+
+def unit_private_ip():
+    """Get this unit's private IP address"""
+    return unit_get('private-address')
+
+
+class UnregisteredHookError(Exception):
+    """Raised when an undefined hook is called"""
+    pass
+
+
+class Hooks(object):
+    """A convenient handler for hook functions.
+
+    Example::
+
+        hooks = Hooks()
+
+        # register a hook, taking its name from the function name
+        @hooks.hook()
+        def install():
+            pass  # your code here
+
+        # register a hook, providing a custom hook name
+        @hooks.hook("config-changed")
+        def config_changed():
+            pass  # your code here
+
+        if __name__ == "__main__":
+            # execute a hook based on the name the program is called by
+            hooks.execute(sys.argv)
+    """
+
+    def __init__(self, config_save=True):
+        super(Hooks, self).__init__()
+        self._hooks = {}
+        self._config_save = config_save
+
+    def register(self, name, function):
+        """Register a hook"""
+        self._hooks[name] = function
+
+    def execute(self, args):
+        """Execute a registered hook based on args[0]"""
+        hook_name = os.path.basename(args[0])
+        if hook_name in self._hooks:
+            self._hooks[hook_name]()
+            if self._config_save:
+                cfg = config()
+                if cfg.implicit_save:
+                    cfg.save()
+        else:
+            raise UnregisteredHookError(hook_name)
+
+    def hook(self, *hook_names):
+        """Decorator, registering them as hooks"""
+        def wrapper(decorated):
+            for hook_name in hook_names:
+                self.register(hook_name, decorated)
+            else:
+                self.register(decorated.__name__, decorated)
+                if '_' in decorated.__name__:
+                    self.register(
+                        decorated.__name__.replace('_', '-'), decorated)
+            return decorated
+        return wrapper
+
+
+def charm_dir():
+    """Return the root directory of the current charm"""
+    return os.environ.get('CHARM_DIR')

=== added file 'hooks/charmhelpers/core/host.py'
--- hooks/charmhelpers/core/host.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/host.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,416 @@
+"""Tools for working with the host system"""
+# Copyright 2012 Canonical Ltd.
+#
+# Authors:
+#  Nick Moffitt <[e-mail]>
+#  Matthew Wedgwood <[e-mail]>
+
+import os
+import re
+import pwd
+import grp
+import random
+import string
+import subprocess
+import hashlib
+from contextlib import contextmanager
+from collections import OrderedDict
+
+import six
+
+from .hookenv import log
+from .fstab import Fstab
+
+
+def service_start(service_name):
+    """Start a system service"""
+    return service('start', service_name)
+
+
+def service_stop(service_name):
+    """Stop a system service"""
+    return service('stop', service_name)
+
+
+def service_restart(service_name):
+    """Restart a system service"""
+    return service('restart', service_name)
+
+
+def service_reload(service_name, restart_on_failure=False):
+    """Reload a system service, optionally falling back to restart if
+    reload fails"""
+    service_result = service('reload', service_name)
+    if not service_result and restart_on_failure:
+        service_result = service('restart', service_name)
+    return service_result
+
+
+def service(action, service_name):
+    """Control a system service"""
+    cmd = ['service', service_name, action]
+    return subprocess.call(cmd) == 0
+
+
+def service_running(service):
+    """Determine whether a system service is running"""
+    try:
+        output = subprocess.check_output(
+            ['service', service, 'status'],
+            stderr=subprocess.STDOUT).decode('UTF-8')
+    except subprocess.CalledProcessError:
+        return False
+    else:
+        if ("start/running" in output or "is running" in output):
+            return True
+        else:
+            return False
+
+
+def service_available(service_name):
+    """Determine whether a system service is available"""
+    try:
+        subprocess.check_output(
+            ['service', service_name, 'status'],
+            stderr=subprocess.STDOUT).decode('UTF-8')
+    except subprocess.CalledProcessError as e:
+        return 'unrecognized service' not in e.output
+    else:
+        return True
+
+
+def adduser(username, password=None, shell='/bin/bash', system_user=False):
+    """Add a user to the system"""
+    try:
+        user_info = pwd.getpwnam(username)
+        log('user {0} already exists!'.format(username))
+    except KeyError:
+        log('creating user {0}'.format(username))
+        cmd = ['useradd']
+        if system_user or password is None:
+            cmd.append('--system')
+        else:
+            cmd.extend([
+                '--create-home',
+                '--shell', shell,
+                '--password', password,
+            ])
+        cmd.append(username)
+        subprocess.check_call(cmd)
+        user_info = pwd.getpwnam(username)
+    return user_info
+
+
+def add_group(group_name, system_group=False):
+    """Add a group to the system"""
+    try:
+        group_info = grp.getgrnam(group_name)
+        log('group {0} already exists!'.format(group_name))
+    except KeyError:
+        log('creating group {0}'.format(group_name))
+        cmd = ['addgroup']
+        if system_group:
+            cmd.append('--system')
+        else:
+            cmd.extend([
+                '--group',
+            ])
+        cmd.append(group_name)
+        subprocess.check_call(cmd)
+        group_info = grp.getgrnam(group_name)
+    return group_info
+
+
+def add_user_to_group(username, group):
+    """Add a user to a group"""
+    cmd = [
+        'gpasswd', '-a',
+        username,
+        group
+    ]
+    log("Adding user {} to group {}".format(username, group))
+    subprocess.check_call(cmd)
+
+
+def rsync(from_path, to_path, flags='-r', options=None):
+    """Replicate the contents of a path"""
+    options = options or ['--delete', '--executability']
+    cmd = ['/usr/bin/rsync', flags]
+    cmd.extend(options)
+    cmd.append(from_path)
+    cmd.append(to_path)
+    log(" ".join(cmd))
+    return subprocess.check_output(cmd).decode('UTF-8').strip()
+
+
+def symlink(source, destination):
+    """Create a symbolic link"""
+    log("Symlinking {} as {}".format(source, destination))
+    cmd = [
+        'ln',
+        '-sf',
+        source,
+        destination,
+    ]
+    subprocess.check_call(cmd)
+
+
+def mkdir(path, owner='root', group='root', perms=0o555, force=False):
+    """Create a directory"""
+    log("Making dir {} {}:{} {:o}".format(path, owner, group,
+                                          perms))
+    uid = pwd.getpwnam(owner).pw_uid
+    gid = grp.getgrnam(group).gr_gid
+    realpath = os.path.abspath(path)
+    if os.path.exists(realpath):
+        if force and not os.path.isdir(realpath):
+            log("Removing non-directory file {} prior to mkdir()".format(path))
+            os.unlink(realpath)
+    else:
+        os.makedirs(realpath, perms)
+    os.chown(realpath, uid, gid)
+
+
+def write_file(path, content, owner='root', group='root', perms=0o444):
+    """Create or overwrite a file with the contents of a string"""
+    log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
+    uid = pwd.getpwnam(owner).pw_uid
+    gid = grp.getgrnam(group).gr_gid
+    with open(path, 'w') as target:
+        os.fchown(target.fileno(), uid, gid)
+        os.fchmod(target.fileno(), perms)
+        target.write(content)
+
+
+def fstab_remove(mp):
+    """Remove the given mountpoint entry from /etc/fstab
+    """
+    return Fstab.remove_by_mountpoint(mp)
+
+
+def fstab_add(dev, mp, fs, options=None):
+    """Adds the given device entry to the /etc/fstab file
+    """
+    return Fstab.add(dev, mp, fs, options=options)
+
+
+def mount(device, mountpoint, options=None, persist=False, filesystem="ext3"):
+    """Mount a filesystem at a particular mountpoint"""
+    cmd_args = ['mount']
+    if options is not None:
+        cmd_args.extend(['-o', options])
+    cmd_args.extend([device, mountpoint])
+    try:
+        subprocess.check_output(cmd_args)
+    except subprocess.CalledProcessError as e:
+        log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output))
+        return False
+
+    if persist:
+        return fstab_add(device, mountpoint, filesystem, options=options)
+    return True
+
+
+def umount(mountpoint, persist=False):
+    """Unmount a filesystem"""
+    cmd_args = ['umount', mountpoint]
+    try:
+        subprocess.check_output(cmd_args)
+    except subprocess.CalledProcessError as e:
+        log('Error unmounting {}\n{}'.format(mountpoint, e.output))
+        return False
+
+    if persist:
+        return fstab_remove(mountpoint)
+    return True
+
+
+def mounts():
+    """Get a list of all mounted volumes as [[mountpoint,device],[...]]"""
+    with open('/proc/mounts') as f:
+        # [['/mount/point','/dev/path'],[...]]
+        system_mounts = [m[1::-1] for m in [l.strip().split()
+                                            for l in f.readlines()]]
+    return system_mounts
+
+
+def file_hash(path, hash_type='md5'):
+    """
+    Generate a hash checksum of the contents of 'path' or None if not found.
+
+    :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,
+                          such as md5, sha1, sha256, sha512, etc.
+    """
+    if os.path.exists(path):
+        h = getattr(hashlib, hash_type)()
+        with open(path, 'rb') as source:
+            h.update(source.read())
+        return h.hexdigest()
+    else:
+        return None
+
+
+def check_hash(path, checksum, hash_type='md5'):
+    """
+    Validate a file using a cryptographic checksum.
+
+    :param str checksum: Value of the checksum used to validate the file.
+    :param str hash_type: Hash algorithm used to generate `checksum`.
+        Can be any hash alrgorithm supported by :mod:`hashlib`,
+        such as md5, sha1, sha256, sha512, etc.
+    :raises ChecksumError: If the file fails the checksum
+
+    """
+    actual_checksum = file_hash(path, hash_type)
+    if checksum != actual_checksum:
+        raise ChecksumError("'%s' != '%s'" % (checksum, actual_checksum))
+
+
+class ChecksumError(ValueError):
+    pass
+
+
+def restart_on_change(restart_map, stopstart=False):
+    """Restart services based on configuration files changing
+
+    This function is used a decorator, for example::
+
+        @restart_on_change({
+            '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]
+            })
+        def ceph_client_changed():
+            pass  # your code here
+
+    In this example, the cinder-api and cinder-volume services
+    would be restarted if /etc/ceph/ceph.conf is changed by the
+    ceph_client_changed function.
+    """
+    def wrap(f):
+        def wrapped_f(*args):
+            checksums = {}
+            for path in restart_map:
+                checksums[path] = file_hash(path)
+            f(*args)
+            restarts = []
+            for path in restart_map:
+                if checksums[path] != file_hash(path):
+                    restarts += restart_map[path]
+            services_list = list(OrderedDict.fromkeys(restarts))
+            if not stopstart:
+                for service_name in services_list:
+                    service('restart', service_name)
+            else:
+                for action in ['stop', 'start']:
+                    for service_name in services_list:
+                        service(action, service_name)
+        return wrapped_f
+    return wrap
+
+
+def lsb_release():
+    """Return /etc/lsb-release in a dict"""
+    d = {}
+    with open('/etc/lsb-release', 'r') as lsb:
+        for l in lsb:
+            k, v = l.split('=')
+            d[k.strip()] = v.strip()
+    return d
+
+
+def pwgen(length=None):
+    """Generate a random pasword."""
+    if length is None:
+        length = random.choice(range(35, 45))
+    alphanumeric_chars = [
+        l for l in (string.ascii_letters + string.digits)
+        if l not in 'l0QD1vAEIOUaeiou']
+    random_chars = [
+        random.choice(alphanumeric_chars) for _ in range(length)]
+    return(''.join(random_chars))
+
+
+def list_nics(nic_type):
+    '''Return a list of nics of given type(s)'''
+    if isinstance(nic_type, six.string_types):
+        int_types = [nic_type]
+    else:
+        int_types = nic_type
+    interfaces = []
+    for int_type in int_types:
+        cmd = ['ip', 'addr', 'show', 'label', int_type + '*']
+        ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
+        ip_output = (line for line in ip_output if line)
+        for line in ip_output:
+            if line.split()[1].startswith(int_type):
+                matched = re.search('.*: (bond[0-9]+\.[0-9]+)@.*', line)
+                if matched:
+                    interface = matched.groups()[0]
+                else:
+                    interface = line.split()[1].replace(":", "")
+                interfaces.append(interface)
+
+    return interfaces
+
+
+def set_nic_mtu(nic, mtu):
+    '''Set MTU on a network interface'''
+    cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
+    subprocess.check_call(cmd)
+
+
+def get_nic_mtu(nic):
+    cmd = ['ip', 'addr', 'show', nic]
+    ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
+    mtu = ""
+    for line in ip_output:
+        words = line.split()
+        if 'mtu' in words:
+            mtu = words[words.index("mtu") + 1]
+    return mtu
+
+
+def get_nic_hwaddr(nic):
+    cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
+    ip_output = subprocess.check_output(cmd).decode('UTF-8')
+    hwaddr = ""
+    words = ip_output.split()
+    if 'link/ether' in words:
+        hwaddr = words[words.index('link/ether') + 1]
+    return hwaddr
+
+
+def cmp_pkgrevno(package, revno, pkgcache=None):
+    '''Compare supplied revno with the revno of the installed package
+
+    *  1 => Installed revno is greater than supplied arg
+    *  0 => Installed revno is the same as supplied arg
+    * -1 => Installed revno is less than supplied arg
+
+    '''
+    import apt_pkg
+    if not pkgcache:
+        from charmhelpers.fetch import apt_cache
+        pkgcache = apt_cache()
+    pkg = pkgcache[package]
+    return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
+
+
+@contextmanager
+def chdir(d):
+    cur = os.getcwd()
+    try:
+        yield os.chdir(d)
+    finally:
+        os.chdir(cur)
+
+
+def chownr(path, owner, group):
+    uid = pwd.getpwnam(owner).pw_uid
+    gid = grp.getgrnam(group).gr_gid
+
+    for root, dirs, files in os.walk(path):
+        for name in dirs + files:
+            full = os.path.join(root, name)
+            broken_symlink = os.path.lexists(full) and not os.path.exists(full)
+            if not broken_symlink:
+                os.chown(full, uid, gid)

=== added directory 'hooks/charmhelpers/core/services'
=== added file 'hooks/charmhelpers/core/services/__init__.py'
--- hooks/charmhelpers/core/services/__init__.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/services/__init__.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,2 @@
+from .base import *  # NOQA
+from .helpers import *  # NOQA

=== added file 'hooks/charmhelpers/core/services/base.py'
--- hooks/charmhelpers/core/services/base.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/services/base.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,313 @@
+import os
+import re
+import json
+from collections import Iterable
+
+from charmhelpers.core import host
+from charmhelpers.core import hookenv
+
+
+__all__ = ['ServiceManager', 'ManagerCallback',
+           'PortManagerCallback', 'open_ports', 'close_ports', 'manage_ports',
+           'service_restart', 'service_stop']
+
+
+class ServiceManager(object):
+    def __init__(self, services=None):
+        """
+        Register a list of services, given their definitions.
+
+        Service definitions are dicts in the following formats (all keys except
+        'service' are optional)::
+
+            {
+                "service": <service name>,
+                "required_data": <list of required data contexts>,
+                "provided_data": <list of provided data contexts>,
+                "data_ready": <one or more callbacks>,
+                "data_lost": <one or more callbacks>,
+                "start": <one or more callbacks>,
+                "stop": <one or more callbacks>,
+                "ports": <list of ports to manage>,
+            }
+
+        The 'required_data' list should contain dicts of required data (or
+        dependency managers that act like dicts and know how to collect the data).
+        Only when all items in the 'required_data' list are populated are the list
+        of 'data_ready' and 'start' callbacks executed.  See `is_ready()` for more
+        information.
+
+        The 'provided_data' list should contain relation data providers, most likely
+        a subclass of :class:`charmhelpers.core.services.helpers.RelationContext`,
+        that will indicate a set of data to set on a given relation.
+
+        The 'data_ready' value should be either a single callback, or a list of
+        callbacks, to be called when all items in 'required_data' pass `is_ready()`.
+        Each callback will be called with the service name as the only parameter.
+        After all of the 'data_ready' callbacks are called, the 'start' callbacks
+        are fired.
+
+        The 'data_lost' value should be either a single callback, or a list of
+        callbacks, to be called when a 'required_data' item no longer passes
+        `is_ready()`.  Each callback will be called with the service name as the
+        only parameter.  After all of the 'data_lost' callbacks are called,
+        the 'stop' callbacks are fired.
+
+        The 'start' value should be either a single callback, or a list of
+        callbacks, to be called when starting the service, after the 'data_ready'
+        callbacks are complete.  Each callback will be called with the service
+        name as the only parameter.  This defaults to
+        `[host.service_start, services.open_ports]`.
+
+        The 'stop' value should be either a single callback, or a list of
+        callbacks, to be called when stopping the service.  If the service is
+        being stopped because it no longer has all of its 'required_data', this
+        will be called after all of the 'data_lost' callbacks are complete.
+        Each callback will be called with the service name as the only parameter.
+        This defaults to `[services.close_ports, host.service_stop]`.
+
+        The 'ports' value should be a list of ports to manage.  The default
+        'start' handler will open the ports after the service is started,
+        and the default 'stop' handler will close the ports prior to stopping
+        the service.
+
+
+        Examples:
+
+        The following registers an Upstart service called bingod that depends on
+        a mongodb relation and which runs a custom `db_migrate` function prior to
+        restarting the service, and a Runit service called spadesd::
+
+            manager = services.ServiceManager([
+                {
+                    'service': 'bingod',
+                    'ports': [80, 443],
+                    'required_data': [MongoRelation(), config(), {'my': 'data'}],
+                    'data_ready': [
+                        services.template(source='bingod.conf'),
+                        services.template(source='bingod.ini',
+                                          target='/etc/bingod.ini',
+                                          owner='bingo', perms=0400),
+                    ],
+                },
+                {
+                    'service': 'spadesd',
+                    'data_ready': services.template(source='spadesd_run.j2',
+                                                    target='/etc/sv/spadesd/run',
+                                                    perms=0555),
+                    'start': runit_start,
+                    'stop': runit_stop,
+                },
+            ])
+            manager.manage()
+        """
+        self._ready_file = os.path.join(hookenv.charm_dir(), 'READY-SERVICES.json')
+        self._ready = None
+        self.services = {}
+        for service in services or []:
+            service_name = service['service']
+            self.services[service_name] = service
+
+    def manage(self):
+        """
+        Handle the current hook by doing The Right Thing with the registered services.
+        """
+        hook_name = hookenv.hook_name()
+        if hook_name == 'stop':
+            self.stop_services()
+        else:
+            self.provide_data()
+            self.reconfigure_services()
+        cfg = hookenv.config()
+        if cfg.implicit_save:
+            cfg.save()
+
+    def provide_data(self):
+        """
+        Set the relation data for each provider in the ``provided_data`` list.
+
+        A provider must have a `name` attribute, which indicates which relation
+        to set data on, and a `provide_data()` method, which returns a dict of
+        data to set.
+        """
+        hook_name = hookenv.hook_name()
+        for service in self.services.values():
+            for provider in service.get('provided_data', []):
+                if re.match(r'{}-relation-(joined|changed)'.format(provider.name), hook_name):
+                    data = provider.provide_data()
+                    _ready = provider._is_ready(data) if hasattr(provider, '_is_ready') else data
+                    if _ready:
+                        hookenv.relation_set(None, data)
+
+    def reconfigure_services(self, *service_names):
+        """
+        Update all files for one or more registered services, and,
+        if ready, optionally restart them.
+
+        If no service names are given, reconfigures all registered services.
+        """
+        for service_name in service_names or self.services.keys():
+            if self.is_ready(service_name):
+                self.fire_event('data_ready', service_name)
+                self.fire_event('start', service_name, default=[
+                    service_restart,
+                    manage_ports])
+                self.save_ready(service_name)
+            else:
+                if self.was_ready(service_name):
+                    self.fire_event('data_lost', service_name)
+                self.fire_event('stop', service_name, default=[
+                    manage_ports,
+                    service_stop])
+                self.save_lost(service_name)
+
+    def stop_services(self, *service_names):
+        """
+        Stop one or more registered services, by name.
+
+        If no service names are given, stops all registered services.
+        """
+        for service_name in service_names or self.services.keys():
+            self.fire_event('stop', service_name, default=[
+                manage_ports,
+                service_stop])
+
+    def get_service(self, service_name):
+        """
+        Given the name of a registered service, return its service definition.
+        """
+        service = self.services.get(service_name)
+        if not service:
+            raise KeyError('Service not registered: %s' % service_name)
+        return service
+
+    def fire_event(self, event_name, service_name, default=None):
+        """
+        Fire a data_ready, data_lost, start, or stop event on a given service.
+        """
+        service = self.get_service(service_name)
+        callbacks = service.get(event_name, default)
+        if not callbacks:
+            return
+        if not isinstance(callbacks, Iterable):
+            callbacks = [callbacks]
+        for callback in callbacks:
+            if isinstance(callback, ManagerCallback):
+                callback(self, service_name, event_name)
+            else:
+                callback(service_name)
+
+    def is_ready(self, service_name):
+        """
+        Determine if a registered service is ready, by checking its 'required_data'.
+
+        A 'required_data' item can be any mapping type, and is considered ready
+        if `bool(item)` evaluates as True.
+        """
+        service = self.get_service(service_name)
+        reqs = service.get('required_data', [])
+        return all(bool(req) for req in reqs)
+
+    def _load_ready_file(self):
+        if self._ready is not None:
+            return
+        if os.path.exists(self._ready_file):
+            with open(self._ready_file) as fp:
+                self._ready = set(json.load(fp))
+        else:
+            self._ready = set()
+
+    def _save_ready_file(self):
+        if self._ready is None:
+            return
+        with open(self._ready_file, 'w') as fp:
+            json.dump(list(self._ready), fp)
+
+    def save_ready(self, service_name):
+        """
+        Save an indicator that the given service is now data_ready.
+        """
+        self._load_ready_file()
+        self._ready.add(service_name)
+        self._save_ready_file()
+
+    def save_lost(self, service_name):
+        """
+        Save an indicator that the given service is no longer data_ready.
+        """
+        self._load_ready_file()
+        self._ready.discard(service_name)
+        self._save_ready_file()
+
+    def was_ready(self, service_name):
+        """
+        Determine if the given service was previously data_ready.
+        """
+        self._load_ready_file()
+        return service_name in self._ready
+
+
+class ManagerCallback(object):
+    """
+    Special case of a callback that takes the `ServiceManager` instance
+    in addition to the service name.
+
+    Subclasses should implement `__call__` which should accept three parameters:
+
+        * `manager`       The `ServiceManager` instance
+        * `service_name`  The name of the service it's being triggered for
+        * `event_name`    The name of the event that this callback is handling
+    """
+    def __call__(self, manager, service_name, event_name):
+        raise NotImplementedError()
+
+
+class PortManagerCallback(ManagerCallback):
+    """
+    Callback class that will open or close ports, for use as either
+    a start or stop action.
+    """
+    def __call__(self, manager, service_name, event_name):
+        service = manager.get_service(service_name)
+        new_ports = service.get('ports', [])
+        port_file = os.path.join(hookenv.charm_dir(), '.{}.ports'.format(service_name))
+        if os.path.exists(port_file):
+            with open(port_file) as fp:
+                old_ports = fp.read().split(',')
+            for old_port in old_ports:
+                if bool(old_port):
+                    old_port = int(old_port)
+                    if old_port not in new_ports:
+                        hookenv.close_port(old_port)
+        with open(port_file, 'w') as fp:
+            fp.write(','.join(str(port) for port in new_ports))
+        for port in new_ports:
+            if event_name == 'start':
+                hookenv.open_port(port)
+            elif event_name == 'stop':
+                hookenv.close_port(port)
+
+
+def service_stop(service_name):
+    """
+    Wrapper around host.service_stop to prevent spurious "unknown service"
+    messages in the logs.
+    """
+    if host.service_running(service_name):
+        host.service_stop(service_name)
+
+
+def service_restart(service_name):
+    """
+    Wrapper around host.service_restart to prevent spurious "unknown service"
+    messages in the logs.
+    """
+    if host.service_available(service_name):
+        if host.service_running(service_name):
+            host.service_restart(service_name)
+        else:
+            host.service_start(service_name)
+
+
+# Convenience aliases
+open_ports = close_ports = manage_ports = PortManagerCallback()

=== added file 'hooks/charmhelpers/core/services/helpers.py'
--- hooks/charmhelpers/core/services/helpers.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/services/helpers.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,243 @@
+import os
+import yaml
+from charmhelpers.core import hookenv
+from charmhelpers.core import templating
+
+from charmhelpers.core.services.base import ManagerCallback
+
+
+__all__ = ['RelationContext', 'TemplateCallback',
+           'render_template', 'template']
+
+
+class RelationContext(dict):
+    """
+    Base class for a context generator that gets relation data from juju.
+
+    Subclasses must provide the attributes `name`, which is the name of the
+    interface of interest, `interface`, which is the type of the interface of
+    interest, and `required_keys`, which is the set of keys required for the
+    relation to be considered complete.  The data for all interfaces matching
+    the `name` attribute that are complete will used to populate the dictionary
+    values (see `get_data`, below).
+
+    The generated context will be namespaced under the relation :attr:`name`,
+    to prevent potential naming conflicts.
+
+    :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
+    :param list additional_required_keys: Extend the list of :attr:`required_keys`
+    """
+    name = None
+    interface = None
+    required_keys = []
+
+    def __init__(self, name=None, additional_required_keys=None):
+        if name is not None:
+            self.name = name
+        if additional_required_keys is not None:
+            self.required_keys.extend(additional_required_keys)
+        self.get_data()
+
+    def __bool__(self):
+        """
+        Returns True if all of the required_keys are available.
+        """
+        return self.is_ready()
+
+    __nonzero__ = __bool__
+
+    def __repr__(self):
+        return super(RelationContext, self).__repr__()
+
+    def is_ready(self):
+        """
+        Returns True if all of the `required_keys` are available from any units.
+        """
+        ready = len(self.get(self.name, [])) > 0
+        if not ready:
+            hookenv.log('Incomplete relation: {}'.format(self.__class__.__name__), hookenv.DEBUG)
+        return ready
+
+    def _is_ready(self, unit_data):
+        """
+        Helper method that tests a set of relation data and returns True if
+        all of the `required_keys` are present.
+        """
+        return set(unit_data.keys()).issuperset(set(self.required_keys))
+
+    def get_data(self):
+        """
+        Retrieve the relation data for each unit involved in a relation and,
+        if complete, store it in a list under `self[self.name]`.  This
+        is automatically called when the RelationContext is instantiated.
+
+        The units are sorted lexographically first by the service ID, then by
+        the unit ID.  Thus, if an interface has two other services, 'db:1'
+        and 'db:2', with 'db:1' having two units, 'wordpress/0' and 'wordpress/1',
+        and 'db:2' having one unit, 'mediawiki/0', all of which have a complete
+        set of data, the relation data for the units will be stored in the
+        order: 'wordpress/0', 'wordpress/1', 'mediawiki/0'.
+
+        If you only care about a single unit on the relation, you can just
+        access it as `{{ interface[0]['key'] }}`.  However, if you can at all
+        support multiple units on a relation, you should iterate over the list,
+        like::
+
+            {% for unit in interface -%}
+                {{ unit['key'] }}{% if not loop.last %},{% endif %}
+            {%- endfor %}
+
+        Note that since all sets of relation data from all related services and
+        units are in a single list, if you need to know which service or unit a
+        set of data came from, you'll need to extend this class to preserve
+        that information.
+        """
+        if not hookenv.relation_ids(self.name):
+            return
+
+        ns = self.setdefault(self.name, [])
+        for rid in sorted(hookenv.relation_ids(self.name)):
+            for unit in sorted(hookenv.related_units(rid)):
+                reldata = hookenv.relation_get(rid=rid, unit=unit)
+                if self._is_ready(reldata):
+                    ns.append(reldata)
+
+    def provide_data(self):
+        """
+        Return data to be relation_set for this interface.
+        """
+        return {}
+
+
+class MysqlRelation(RelationContext):
+    """
+    Relation context for the `mysql` interface.
+
+    :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
+    :param list additional_required_keys: Extend the list of :attr:`required_keys`
+    """
+    name = 'db'
+    interface = 'mysql'
+    required_keys = ['host', 'user', 'password', 'database']
+
+
+class HttpRelation(RelationContext):
+    """
+    Relation context for the `http` interface.
+
+    :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
+    :param list additional_required_keys: Extend the list of :attr:`required_keys`
+    """
+    name = 'website'
+    interface = 'http'
+    required_keys = ['host', 'port']
+
+    def provide_data(self):
+        return {
+            'host': hookenv.unit_get('private-address'),
+            'port': 80,
+        }
+
+
+class RequiredConfig(dict):
+    """
+    Data context that loads config options with one or more mandatory options.
+
+    Once the required options have been changed from their default values, all
+    config options will be available, namespaced under `config` to prevent
+    potential naming conflicts (for example, between a config option and a
+    relation property).
+
+    :param list *args: List of options that must be changed from their default values.
+    """
+
+    def __init__(self, *args):
+        self.required_options = args
+        self['config'] = hookenv.config()
+        with open(os.path.join(hookenv.charm_dir(), 'config.yaml')) as fp:
+            self.config = yaml.load(fp).get('options', {})
+
+    def __bool__(self):
+        for option in self.required_options:
+            if option not in self['config']:
+                return False
+            current_value = self['config'][option]
+            default_value = self.config[option].get('default')
+            if current_value == default_value:
+                return False
+            if current_value in (None, '') and default_value in (None, ''):
+                return False
+        return True
+
+    def __nonzero__(self):
+        return self.__bool__()
+
+
+class StoredContext(dict):
+    """
+    A data context that always returns the data that it was first created with.
+
+    This is useful to do a one-time generation of things like passwords, that
+    will thereafter use the same value that was originally generated, instead
+    of generating a new value each time it is run.
+    """
+    def __init__(self, file_name, config_data):
+        """
+        If the file exists, populate `self` with the data from the file.
+        Otherwise, populate with the given data and persist it to the file.
+        """
+        if os.path.exists(file_name):
+            self.update(self.read_context(file_name))
+        else:
+            self.store_context(file_name, config_data)
+            self.update(config_data)
+
+    def store_context(self, file_name, config_data):
+        if not os.path.isabs(file_name):
+            file_name = os.path.join(hookenv.charm_dir(), file_name)
+        with open(file_name, 'w') as file_stream:
+            os.fchmod(file_stream.fileno(), 0o600)
+            yaml.dump(config_data, file_stream)
+
+    def read_context(self, file_name):
+        if not os.path.isabs(file_name):
+            file_name = os.path.join(hookenv.charm_dir(), file_name)
+        with open(file_name, 'r') as file_stream:
+            data = yaml.load(file_stream)
+            if not data:
+                raise OSError("%s is empty" % file_name)
+            return data
+
+
+class TemplateCallback(ManagerCallback):
+    """
+    Callback class that will render a Jinja2 template, for use as a ready
+    action.
+
+    :param str source: The template source file, relative to
+    `$CHARM_DIR/templates`
+
+    :param str target: The target to write the rendered template to
+    :param str owner: The owner of the rendered file
+    :param str group: The group of the rendered file
+    :param int perms: The permissions of the rendered file
+    """
+    def __init__(self, source, target,
+                 owner='root', group='root', perms=0o444):
+        self.source = source
+        self.target = target
+        self.owner = owner
+        self.group = group
+        self.perms = perms
+
+    def __call__(self, manager, service_name, event_name):
+        service = manager.get_service(service_name)
+        context = {}
+        for ctx in service.get('required_data', []):
+            context.update(ctx)
+        templating.render(self.source, self.target, context,
+                          self.owner, self.group, self.perms)
+
+
+# Convenience aliases for templates
+render_template = template = TemplateCallback

=== added file 'hooks/charmhelpers/core/sysctl.py'
--- hooks/charmhelpers/core/sysctl.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/sysctl.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,34 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+__author__ = 'Jorge Niedbalski R. <[e-mail]>'
+
+import yaml
+
+from subprocess import check_call
+
+from charmhelpers.core.hookenv import (
+    log,
+    DEBUG,
+)
+
+
+def create(sysctl_dict, sysctl_file):
+    """Creates a sysctl.conf file from a YAML associative array
+
+    :param sysctl_dict: a dict of sysctl options eg { 'kernel.max_pid': 1337 }
+    :type sysctl_dict: dict
+    :param sysctl_file: path to the sysctl file to be saved
+    :type sysctl_file: str or unicode
+    :returns: None
+    """
+    sysctl_dict = yaml.load(sysctl_dict)
+
+    with open(sysctl_file, "w") as fd:
+        for key, value in sysctl_dict.items():
+            fd.write("{}={}\n".format(key, value))
+
+    log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict),
+        level=DEBUG)
+
+    check_call(["sysctl", "-p", sysctl_file])

=== added file 'hooks/charmhelpers/core/templating.py'
--- hooks/charmhelpers/core/templating.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/templating.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,52 @@
+import os
+
+from charmhelpers.core import host
+from charmhelpers.core import hookenv
+
+
+def render(source, target, context, owner='root', group='root',
+           perms=0o444, templates_dir=None):
+    """
+    Render a template.
+
+    The `source` path, if not absolute, is relative to the `templates_dir`.
+
+    The `target` path should be absolute.
+
+    The context should be a dict containing the values to be replaced in the
+    template.
+
+    The `owner`, `group`, and `perms` options will be passed to `write_file`.
+
+    If omitted, `templates_dir` defaults to the `templates` folder in the charm.
+
+    Note: Using this requires python-jinja2; if it is not installed, calling
+    this will attempt to use charmhelpers.fetch.apt_install to install it.
+    """
+    try:
+        from jinja2 import FileSystemLoader, Environment, exceptions
+    except ImportError:
+        try:
+            from charmhelpers.fetch import apt_install
+        except ImportError:
+            hookenv.log('Could not import jinja2, and could not import '
+                        'charmhelpers.fetch to install it',
+                        level=hookenv.ERROR)
+            raise
+        apt_install('python-jinja2', fatal=True)
+        from jinja2 import FileSystemLoader, Environment, exceptions
+
+    if templates_dir is None:
+        templates_dir = os.path.join(hookenv.charm_dir(), 'templates')
+    loader = Environment(loader=FileSystemLoader(templates_dir))
+    try:
+        source = source
+        template = loader.get_template(source)
+    except exceptions.TemplateNotFound as e:
+        hookenv.log('Could not load template %s from %s.' %
+                    (source, templates_dir),
+                    level=hookenv.ERROR)
+        raise e
+    content = template.render(context)
+    host.mkdir(os.path.dirname(target))
+    host.write_file(target, content, owner, group, perms)

=== added directory 'hooks/charmhelpers/fetch'
=== added file 'hooks/charmhelpers/fetch/__init__.py'
--- hooks/charmhelpers/fetch/__init__.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/__init__.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,416 @@
+import importlib
+from tempfile import NamedTemporaryFile
+import time
+from yaml import safe_load
+from charmhelpers.core.host import (
+    lsb_release
+)
+import subprocess
+from charmhelpers.core.hookenv import (
+    config,
+    log,
+)
+import os
+
+import six
+if six.PY3:
+    from urllib.parse import urlparse, urlunparse
+else:
+    from urlparse import urlparse, urlunparse
+
+
+CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
+deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
+"""
+PROPOSED_POCKET = """# Proposed
+deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
+"""
+CLOUD_ARCHIVE_POCKETS = {
+    # Folsom
+    'folsom': 'precise-updates/folsom',
+    'precise-folsom': 'precise-updates/folsom',
+    'precise-folsom/updates': 'precise-updates/folsom',
+    'precise-updates/folsom': 'precise-updates/folsom',
+    'folsom/proposed': 'precise-proposed/folsom',
+    'precise-folsom/proposed': 'precise-proposed/folsom',
+    'precise-proposed/folsom': 'precise-proposed/folsom',
+    # Grizzly
+    'grizzly': 'precise-updates/grizzly',
+    'precise-grizzly': 'precise-updates/grizzly',
+    'precise-grizzly/updates': 'precise-updates/grizzly',
+    'precise-updates/grizzly': 'precise-updates/grizzly',
+    'grizzly/proposed': 'precise-proposed/grizzly',
+    'precise-grizzly/proposed': 'precise-proposed/grizzly',
+    'precise-proposed/grizzly': 'precise-proposed/grizzly',
+    # Havana
+    'havana': 'precise-updates/havana',
+    'precise-havana': 'precise-updates/havana',
+    'precise-havana/updates': 'precise-updates/havana',
+    'precise-updates/havana': 'precise-updates/havana',
+    'havana/proposed': 'precise-proposed/havana',
+    'precise-havana/proposed': 'precise-proposed/havana',
+    'precise-proposed/havana': 'precise-proposed/havana',
+    # Icehouse
+    'icehouse': 'precise-updates/icehouse',
+    'precise-icehouse': 'precise-updates/icehouse',
+    'precise-icehouse/updates': 'precise-updates/icehouse',
+    'precise-updates/icehouse': 'precise-updates/icehouse',
+    'icehouse/proposed': 'precise-proposed/icehouse',
+    'precise-icehouse/proposed': 'precise-proposed/icehouse',
+    'precise-proposed/icehouse': 'precise-proposed/icehouse',
+    # Juno
+    'juno': 'trusty-updates/juno',
+    'trusty-juno': 'trusty-updates/juno',
+    'trusty-juno/updates': 'trusty-updates/juno',
+    'trusty-updates/juno': 'trusty-updates/juno',
+    'juno/proposed': 'trusty-proposed/juno',
+    'juno/proposed': 'trusty-proposed/juno',
+    'trusty-juno/proposed': 'trusty-proposed/juno',
+    'trusty-proposed/juno': 'trusty-proposed/juno',
+}
+
+# The order of this list is very important. Handlers should be listed in from
+# least- to most-specific URL matching.
+FETCH_HANDLERS = (
+    'charmhelpers.fetch.archiveurl.ArchiveUrlFetchHandler',
+    'charmhelpers.fetch.bzrurl.BzrUrlFetchHandler',
+    'charmhelpers.fetch.giturl.GitUrlFetchHandler',
+)
+
+APT_NO_LOCK = 100  # The return code for "couldn't acquire lock" in APT.
+APT_NO_LOCK_RETRY_DELAY = 10  # Wait 10 seconds between apt lock checks.
+APT_NO_LOCK_RETRY_COUNT = 30  # Retry to acquire the lock X times.
+
+
+class SourceConfigError(Exception):
+    pass
+
+
+class UnhandledSource(Exception):
+    pass
+
+
+class AptLockError(Exception):
+    pass
+
+
+class BaseFetchHandler(object):
+
+    """Base class for FetchHandler implementations in fetch plugins"""
+
+    def can_handle(self, source):
+        """Returns True if the source can be handled. Otherwise returns
+        a string explaining why it cannot"""
+        return "Wrong source type"
+
+    def install(self, source):
+        """Try to download and unpack the source. Return the path to the
+        unpacked files or raise UnhandledSource."""
+        raise UnhandledSource("Wrong source type {}".format(source))
+
+    def parse_url(self, url):
+        return urlparse(url)
+
+    def base_url(self, url):
+        """Return url without querystring or fragment"""
+        parts = list(self.parse_url(url))
+        parts[4:] = ['' for i in parts[4:]]
+        return urlunparse(parts)
+
+
+def filter_installed_packages(packages):
+    """Returns a list of packages that require installation"""
+    cache = apt_cache()
+    _pkgs = []
+    for package in packages:
+        try:
+            p = cache[package]
+            p.current_ver or _pkgs.append(package)
+        except KeyError:
+            log('Package {} has no installation candidate.'.format(package),
+                level='WARNING')
+            _pkgs.append(package)
+    return _pkgs
+
+
+def apt_cache(in_memory=True):
+    """Build and return an apt cache"""
+    import apt_pkg
+    apt_pkg.init()
+    if in_memory:
+        apt_pkg.config.set("Dir::Cache::pkgcache", "")
+        apt_pkg.config.set("Dir::Cache::srcpkgcache", "")
+    return apt_pkg.Cache()
+
+
+def apt_install(packages, options=None, fatal=False):
+    """Install one or more packages"""
+    if options is None:
+        options = ['--option=Dpkg::Options::=--force-confold']
+
+    cmd = ['apt-get', '--assume-yes']
+    cmd.extend(options)
+    cmd.append('install')
+    if isinstance(packages, six.string_types):
+        cmd.append(packages)
+    else:
+        cmd.extend(packages)
+    log("Installing {} with options: {}".format(packages,
+                                                options))
+    _run_apt_command(cmd, fatal)
+
+
+def apt_upgrade(options=None, fatal=False, dist=False):
+    """Upgrade all packages"""
+    if options is None:
+        options = ['--option=Dpkg::Options::=--force-confold']
+
+    cmd = ['apt-get', '--assume-yes']
+    cmd.extend(options)
+    if dist:
+        cmd.append('dist-upgrade')
+    else:
+        cmd.append('upgrade')
+    log("Upgrading with options: {}".format(options))
+    _run_apt_command(cmd, fatal)
+
+
+def apt_update(fatal=False):
+    """Update local apt cache"""
+    cmd = ['apt-get', 'update']
+    _run_apt_command(cmd, fatal)
+
+
+def apt_purge(packages, fatal=False):
+    """Purge one or more packages"""
+    cmd = ['apt-get', '--assume-yes', 'purge']
+    if isinstance(packages, six.string_types):
+        cmd.append(packages)
+    else:
+        cmd.extend(packages)
+    log("Purging {}".format(packages))
+    _run_apt_command(cmd, fatal)
+
+
+def apt_hold(packages, fatal=False):
+    """Hold one or more packages"""
+    cmd = ['apt-mark', 'hold']
+    if isinstance(packages, six.string_types):
+        cmd.append(packages)
+    else:
+        cmd.extend(packages)
+    log("Holding {}".format(packages))
+
+    if fatal:
+        subprocess.check_call(cmd)
+    else:
+        subprocess.call(cmd)
+
+
+def add_source(source, key=None):
+    """Add a package source to this system.
+
+    @param source: a URL or sources.list entry, as supported by
+    add-apt-repository(1). Examples::
+
+        ppa:charmers/example
+        deb https://stub:[e-mail]/ubuntu trusty main
+
+    In addition:
+        'proposed:' may be used to enable the standard 'proposed'
+        pocket for the release.
+        'cloud:' may be used to activate official cloud archive pockets,
+        such as 'cloud:icehouse'
+        'distro' may be used as a noop
+
+    @param key: A key to be added to the system's APT keyring and used
+    to verify the signatures on packages. Ideally, this should be an
+    ASCII format GPG public key including the block headers. A GPG key
+    id may also be used, but be aware that only insecure protocols are
+    available to retrieve the actual public key from a public keyserver
+    placing your Juju environment at risk. ppa and cloud archive keys
+    are securely added automtically, so sould not be provided.
+    """
+    if source is None:
+        log('Source is not present. Skipping')
+        return
+
+    if (source.startswith('ppa:') or
+        source.startswith('http') or
+        source.startswith('deb ') or
+            source.startswith('cloud-archive:')):
+        subprocess.check_call(['add-apt-repository', '--yes', source])
+    elif source.startswith('cloud:'):
+        apt_install(filter_installed_packages(['ubuntu-cloud-keyring']),
+                    fatal=True)
+        pocket = source.split(':')[-1]
+        if pocket not in CLOUD_ARCHIVE_POCKETS:
+            raise SourceConfigError(
+                'Unsupported cloud: source option %s' %
+                pocket)
+        actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket]
+        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
+            apt.write(CLOUD_ARCHIVE.format(actual_pocket))
+    elif source == 'proposed':
+        release = lsb_release()['DISTRIB_CODENAME']
+        with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
+            apt.write(PROPOSED_POCKET.format(release))
+    elif source == 'distro':
+        pass
+    else:
+        log("Unknown source: {!r}".format(source))
+
+    if key:
+        if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
+            with NamedTemporaryFile('w+') as key_file:
+                key_file.write(key)
+                key_file.flush()
+                key_file.seek(0)
+                subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file)
+        else:
+            # Note that hkp: is in no way a secure protocol. Using a
+            # GPG key id is pointless from a security POV unless you
+            # absolutely trust your network and DNS.
+            subprocess.check_call(['apt-key', 'adv', '--keyserver',
+                                   'hkp://keyserver.ubuntu.com:80', '--recv',
+                                   key])
+
+
+def configure_sources(update=False,
+                      sources_var='install_sources',
+                      keys_var='install_keys'):
+    """
+    Configure multiple sources from charm configuration.
+
+    The lists are encoded as yaml fragments in the configuration.
+    The frament needs to be included as a string. Sources and their
+    corresponding keys are of the types supported by add_source().
+
+    Example config:
+        install_sources: |
+          - "ppa:foo"
+          - "http://example.com/repo precise main"
+        install_keys: |
+          - null
+          - "a1b2c3d4"
+
+    Note that 'null' (a.k.a. None) should not be quoted.
+    """
+    sources = safe_load((config(sources_var) or '').strip()) or []
+    keys = safe_load((config(keys_var) or '').strip()) or None
+
+    if isinstance(sources, six.string_types):
+        sources = [sources]
+
+    if keys is None:
+        for source in sources:
+            add_source(source, None)
+    else:
+        if isinstance(keys, six.string_types):
+            keys = [keys]
+
+        if len(sources) != len(keys):
+            raise SourceConfigError(
+                'Install sources and keys lists are different lengths')
+        for source, key in zip(sources, keys):
+            add_source(source, key)
+    if update:
+        apt_update(fatal=True)
+
+
+def install_remote(source, *args, **kwargs):
+    """
+    Install a file tree from a remote source
+
+    The specified source should be a url of the form:
+        scheme://[host]/path[#[option=value][&...]]
+
+    Schemes supported are based on this modules submodules.
+    Options supported are submodule-specific.
+    Additional arguments are passed through to the submodule.
+
+    For example::
+
+        dest = install_remote('http://example.com/archive.tgz',
+                              checksum='deadbeef',
+                              hash_type='sha1')
+
+    This will download `archive.tgz`, validate it using SHA1 and, if
+    the file is ok, extract it and return the directory in which it
+    was extracted.  If the checksum fails, it will raise
+    :class:`charmhelpers.core.host.ChecksumError`.
+    """
+    # We ONLY check for True here because can_handle may return a string
+    # explaining why it can't handle a given source.
+    handlers = [h for h in plugins() if h.can_handle(source) is True]
+    installed_to = None
+    for handler in handlers:
+        try:
+            installed_to = handler.install(source, *args, **kwargs)
+        except UnhandledSource:
+            pass
+    if not installed_to:
+        raise UnhandledSource("No handler found for source {}".format(source))
+    return installed_to
+
+
+def install_from_config(config_var_name):
+    charm_config = config()
+    source = charm_config[config_var_name]
+    return install_remote(source)
+
+
+def plugins(fetch_handlers=None):
+    if not fetch_handlers:
+        fetch_handlers = FETCH_HANDLERS
+    plugin_list = []
+    for handler_name in fetch_handlers:
+        package, classname = handler_name.rsplit('.', 1)
+        try:
+            handler_class = getattr(
+                importlib.import_module(package),
+                classname)
+            plugin_list.append(handler_class())
+        except (ImportError, AttributeError):
+            # Skip missing plugins so that they can be ommitted from
+            # installation if desired
+            log("FetchHandler {} not found, skipping plugin".format(
+                handler_name))
+    return plugin_list
+
+
+def _run_apt_command(cmd, fatal=False):
+    """
+    Run an APT command, checking output and retrying if the fatal flag is set
+    to True.
+
+    :param: cmd: str: The apt command to run.
+    :param: fatal: bool: Whether the command's output should be checked and
+        retried.
+    """
+    env = os.environ.copy()
+
+    if 'DEBIAN_FRONTEND' not in env:
+        env['DEBIAN_FRONTEND'] = 'noninteractive'
+
+    if fatal:
+        retry_count = 0
+        result = None
+
+        # If the command is considered "fatal", we need to retry if the apt
+        # lock was not acquired.
+
+        while result is None or result == APT_NO_LOCK:
+            try:
+                result = subprocess.check_call(cmd, env=env)
+            except subprocess.CalledProcessError as e:
+                retry_count = retry_count + 1
+                if retry_count > APT_NO_LOCK_RETRY_COUNT:
+                    raise
+                result = e.returncode
+                log("Couldn't acquire DPKG lock. Will retry in {} seconds."
+                    "".format(APT_NO_LOCK_RETRY_DELAY))
+                time.sleep(APT_NO_LOCK_RETRY_DELAY)
+
+    else:
+        subprocess.call(cmd, env=env)

=== added file 'hooks/charmhelpers/fetch/archiveurl.py'
--- hooks/charmhelpers/fetch/archiveurl.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/archiveurl.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,145 @@
+import os
+import hashlib
+import re
+
+import six
+if six.PY3:
+    from urllib.request import (
+        build_opener, install_opener, urlopen, urlretrieve,
+        HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
+    )
+    from urllib.parse import urlparse, urlunparse, parse_qs
+    from urllib.error import URLError
+else:
+    from urllib import urlretrieve
+    from urllib2 import (
+        build_opener, install_opener, urlopen,
+        HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
+        URLError
+    )
+    from urlparse import urlparse, urlunparse, parse_qs
+
+from charmhelpers.fetch import (
+    BaseFetchHandler,
+    UnhandledSource
+)
+from charmhelpers.payload.archive import (
+    get_archive_handler,
+    extract,
+)
+from charmhelpers.core.host import mkdir, check_hash
+
+
+def splituser(host):
+    '''urllib.splituser(), but six's support of this seems broken'''
+    _userprog = re.compile('^(.*)@(.*)$')
+    match = _userprog.match(host)
+    if match:
+        return match.group(1, 2)
+    return None, host
+
+
+def splitpasswd(user):
+    '''urllib.splitpasswd(), but six's support of this is missing'''
+    _passwdprog = re.compile('^([^:]*):(.*)$', re.S)
+    match = _passwdprog.match(user)
+    if match:
+        return match.group(1, 2)
+    return user, None
+
+
+class ArchiveUrlFetchHandler(BaseFetchHandler):
+    """
+    Handler to download archive files from arbitrary URLs.
+
+    Can fetch from http, https, ftp, and file URLs.
+
+    Can install either tarballs (.tar, .tgz, .tbz2, etc) or zip files.
+
+    Installs the contents of the archive in $CHARM_DIR/fetched/.
+    """
+    def can_handle(self, source):
+        url_parts = self.parse_url(source)
+        if url_parts.scheme not in ('http', 'https', 'ftp', 'file'):
+            return "Wrong source type"
+        if get_archive_handler(self.base_url(source)):
+            return True
+        return False
+
+    def download(self, source, dest):
+        """
+        Download an archive file.
+
+        :param str source: URL pointing to an archive file.
+        :param str dest: Local path location to download archive file to.
+        """
+        # propogate all exceptions
+        # URLError, OSError, etc
+        proto, netloc, path, params, query, fragment = urlparse(source)
+        if proto in ('http', 'https'):
+            auth, barehost = splituser(netloc)
+            if auth is not None:
+                source = urlunparse((proto, barehost, path, params, query, fragment))
+                username, password = splitpasswd(auth)
+                passman = HTTPPasswordMgrWithDefaultRealm()
+                # Realm is set to None in add_password to force the username and password
+                # to be used whatever the realm
+                passman.add_password(None, source, username, password)
+                authhandler = HTTPBasicAuthHandler(passman)
+                opener = build_opener(authhandler)
+                install_opener(opener)
+        response = urlopen(source)
+        try:
+            with open(dest, 'w') as dest_file:
+                dest_file.write(response.read())
+        except Exception as e:
+            if os.path.isfile(dest):
+                os.unlink(dest)
+            raise e
+
+    # Mandatory file validation via Sha1 or MD5 hashing.
+    def download_and_validate(self, url, hashsum, validate="sha1"):
+        tempfile, headers = urlretrieve(url)
+        check_hash(tempfile, hashsum, validate)
+        return tempfile
+
+    def install(self, source, dest=None, checksum=None, hash_type='sha1'):
+        """
+        Download and install an archive file, with optional checksum validation.
+
+        The checksum can also be given on the `source` URL's fragment.
+        For example::
+
+            handler.install('http://example.com/file.tgz#sha1=deadbeef')
+
+        :param str source: URL pointing to an archive file.
+        :param str dest: Local destination path to install to. If not given,
+            installs to `$CHARM_DIR/archives/archive_file_name`.
+        :param str checksum: If given, validate the archive file after download.
+        :param str hash_type: Algorithm used to generate `checksum`.
+            Can be any hash alrgorithm supported by :mod:`hashlib`,
+            such as md5, sha1, sha256, sha512, etc.
+
+        """
+        url_parts = self.parse_url(source)
+        dest_dir = os.path.join(os.environ.get('CHARM_DIR'), 'fetched')
+        if not os.path.exists(dest_dir):
+            mkdir(dest_dir, perms=0o755)
+        dld_file = os.path.join(dest_dir, os.path.basename(url_parts.path))
+        try:
+            self.download(source, dld_file)
+        except URLError as e:
+            raise UnhandledSource(e.reason)
+        except OSError as e:
+            raise UnhandledSource(e.strerror)
+        options = parse_qs(url_parts.fragment)
+        for key, value in options.items():
+            if not six.PY3:
+                algorithms = hashlib.algorithms
+            else:
+                algorithms = hashlib.algorithms_available
+            if key in algorithms:
+                check_hash(dld_file, value, key)
+        if checksum:
+            check_hash(dld_file, checksum, hash_type)
+        return extract(dld_file, dest)

=== added file 'hooks/charmhelpers/fetch/bzrurl.py'
--- hooks/charmhelpers/fetch/bzrurl.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/bzrurl.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,54 @@
+import os
+from charmhelpers.fetch import (
+    BaseFetchHandler,
+    UnhandledSource
+)
+from charmhelpers.core.host import mkdir
+
+import six
+if six.PY3:
+    raise ImportError('bzrlib does not support Python3')
+
+try:
+    from bzrlib.branch import Branch
+except ImportError:
+    from charmhelpers.fetch import apt_install
+    apt_install("python-bzrlib")
+    from bzrlib.branch import Branch
+
+
+class BzrUrlFetchHandler(BaseFetchHandler):
+    """Handler for bazaar branches via generic and lp URLs"""
+    def can_handle(self, source):
+        url_parts = self.parse_url(source)
+        if url_parts.scheme not in ('bzr+ssh', 'lp'):
+            return False
+        else:
+            return True
+
+    def branch(self, source, dest):
+        url_parts = self.parse_url(source)
+        # If we use lp:branchname scheme we need to load plugins
+        if not self.can_handle(source):
+            raise UnhandledSource("Cannot handle {}".format(source))
+        if url_parts.scheme == "lp":
+            from bzrlib.plugin import load_plugins
+            load_plugins()
+        try:
+            remote_branch = Branch.open(source)
+            remote_branch.bzrdir.sprout(dest).open_branch()
+        except Exception as e:
+            raise e
+
+    def install(self, source):
+        url_parts = self.parse_url(source)
+        branch_name = url_parts.path.strip("/").split("/")[-1]
+        dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
+                                branch_name)
+        if not os.path.exists(dest_dir):
+            mkdir(dest_dir, perms=0o755)
+        try:
+            self.branch(source, dest_dir)
+        except OSError as e:
+            raise UnhandledSource(e.strerror)
+        return dest_dir

=== added file 'hooks/charmhelpers/fetch/giturl.py'
--- hooks/charmhelpers/fetch/giturl.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/giturl.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,51 @@
+import os
+from charmhelpers.fetch import (
+    BaseFetchHandler,
+    UnhandledSource
+)
+from charmhelpers.core.host import mkdir
+
+import six
+if six.PY3:
+    raise ImportError('GitPython does not support Python 3')
+
+try:
+    from git import Repo
+except ImportError:
+    from charmhelpers.fetch import apt_install
+    apt_install("python-git")
+    from git import Repo
+
+
+class GitUrlFetchHandler(BaseFetchHandler):
+    """Handler for git branches via generic and github URLs"""
+    def can_handle(self, source):
+        url_parts = self.parse_url(source)
+        # TODO (mattyw) no support for ssh git@ yet
+        if url_parts.scheme not in ('http', 'https', 'git'):
+            return False
+        else:
+            return True
+
+    def clone(self, source, dest, branch):
+        if not self.can_handle(source):
+            raise UnhandledSource("Cannot handle {}".format(source))
+
+        repo = Repo.clone_from(source, dest)
+        repo.git.checkout(branch)
+
+    def install(self, source, branch="master", dest=None):
+        url_parts = self.parse_url(source)
+        branch_name = url_parts.path.strip("/").split("/")[-1]
+        if dest:
+            dest_dir = os.path.join(dest, branch_name)
+        else:
+            dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
+                                    branch_name)
+        if not os.path.exists(dest_dir):
+            mkdir(dest_dir, perms=0o755)
+        try:
+            self.clone(source, dest_dir, branch)
+        except OSError as e:
+            raise UnhandledSource(e.strerror)
+        return dest_dir

=== modified file 'hooks/config-changed'
--- hooks/config-changed	2012-09-13 18:43:22 +0000
+++ hooks/config-changed	1970-01-01 00:00:00 +0000
@@ -1,155 +0,0 @@
-#!/bin/sh
-
-set -ue
-
-tfile=`mktemp /etc/.memcached.conf.XXXXXXX`
-cat > $tfile <<EOF
-################################
-#
-# This config file generated by the juju memcached charm. Changes
-# may not be preserved.
-#
-################################
-#
-# memcached default config file
-# 2003 - Jay Bonci <[e-mail]>
-# This configuration file is read by the start-memcached script provided as
-# part of the Debian GNU/Linux distribution. 
-
-# Run memcached as a daemon. This command is implied, and is not needed for the
-# daemon to run. See the README.Debian that comes with this package for more
-# information.
--d
-
-# Log memcached's output to /var/log/memcached
-logfile /var/log/memcached.log
-
-# Be verbose
-# -v
-
-# Be even more verbose (print client commands as well)
-# -vv
-
-# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
-# Note that the daemon will grow to this size, but does not start out holding this much
-# memory
--m `config-get size`
-
-# Default connection port is 11211
--p `config-get tcp-port`
-
-# Run the daemon as root. The start-memcached will default to running as root if no
-# -u command is present in this config file
--u memcache
-
-# Specify which IP address to listen on. The default is to listen on all IP addresses
-# This parameter is one of the only security measures that memcached has, so make sure
-# it's listening on a firewalled interface.
--l 0.0.0.0
-
-# Limit the number of simultaneous incoming connections. The daemon default is 1024
--c `config-get connection-limit`
-
-# Lock down all paged memory. Consult with the README and homepage before you do this
-# -k
-
-# Return error when memory is exhausted (rather than removing items)
-# -M
-
-# Maximize core file limit
-# -r
-EOF
-
-append_numeric() {
-    local name=$1
-    local opt=$2
-    local target=$3
-    val=`config-get $name`
-    if [ "$val" != "-1" ] ; then
-        cat >> $target <<EOF
-# $name
-$opt $val
-EOF
-    fi
-}
-
-append_bool() {
-    local name=$1
-    local opt=$2
-    local target=$3
-    val=`config-get $name`
-    if [ "$val" = "yes" ] || [  "$val" = "Y" ] ; then
-        cat >> $target <<EOF
-# $name
-$opt
-EOF
-    fi
-}
-
-append_numeric "request-limit" "-R" $tfile
-append_numeric "min-item-size" "-n" $tfile
-append_numeric "slab-page-size" "-I" $tfile
-append_numeric "threads" "-t" $tfile
-append_bool "disable-auto-cleanup" "-M" $tfile
-append_bool "disable-cas" "-C" $tfile
-
-factor=`config-get factor`
-if [ "$factor" != "-1.0" ] ; then
-    cat >> $tfile <<EOF
-# factor
--f $factor
-EOF
-fi
-
-size=`config-get size`
-if [ $size -ge 2048 ] ; then
-    disableLP=`config-get disable-large-pages`
-    if [ "$disableLP" != "yes" ] && [ "$disableLP" != "Y" ] ; then
-        juju-log "Memory >= 2GB, using large-pages to speed up access"
-        # First try to reserve them
-        save_hugepages=`sysctl -n vm.nr_hugepages`
-        npages=$(($size/2))
-        sysctl -w vm.nr_hugepages=$npages
-        allocated=`sysctl -n vm.nr_hugepages`
-        if [ $allocated -lt $npages ] ; then
-            juju-log -l WARNING "Cannot allocate $npages contiguous pages for huge page allocation"
-            juju-log -l WARNING "Setting vm.nr_hugepages back to $save_hugepages"
-            sysctl -w vm.nr_hugepages=$save_hugepages
-            juju-log -l WARNING "Will disable large pages and fall back to regular memory"
-        else
-            juju-log "Allocated $npages contiguous blocks only for huge pages."
-            cat >> $tfile <<EOF
-# large pages can be disabled with disable-large-pages
--L
-EOF
-        fi
-    fi
-fi
-
-new_hash=`md5sum $tfile|cut -d' ' -f1`
-old_hash=`md5sum /etc/memcached.conf|cut -d' ' -f1`
-if [ "$new_hash" != "$old_hash" ] ; then
-    juju-log "New config generated. hash=$new_hash"
-    mv -f /etc/memcached.conf /etc/memcached.conf.$old_hash
-    mv -f $tfile /etc/memcached.conf
-    service memcached restart
-else
-    juju-log "Config not changed. hash=$new_hash"
-fi
-
-tcp_port=`config-get tcp-port`
-udp_port=`config-get udp-port`
-
-# Work around http://pad.lv/900517
-[ -n "$tcp_port" ] || tcp_port=0
-[ -n "$udp_port" ] || udp_port=0
-
-if [ $tcp_port -gt 0 ] ; then
-    open-port $tcp_port/TCP
-fi
-
-if [ $udp_port -gt 0 ] ; then
-    open-port $udp_port/UDP
-fi
-# In case port changed, inform consumers
-hooks/cache-relation-joined

=== target is u'memcached_hooks.py'
=== modified file 'hooks/install'
--- hooks/install	2013-04-17 17:04:41 +0000
+++ hooks/install	1970-01-01 00:00:00 +0000
@@ -1,20 +0,0 @@
-#!/bin/bash
-
-set -e
-if [[ -d exec.d ]]; then
-    shopt -s nullglob
-    for f in exec.d/*/charm-pre-install; do 
-        [[ -x "$f" ]] || continue
-        ${SHELL} -c "$f"|| {
-            ## bail out if anyone fails
-            juju-log -l ERROR "$f: returned exit_status=$? "
-            exit 1
-        }
-    done
-fi
-
-DEBIAN_FRONTEND=noninteractive apt-get -y install -qq memcached python-cheetah python-memcache
-
-cat > /etc/default/memcached <<EOF
-ENABLE_MEMCACHED=yes
-EOF

=== target is u'memcached_hooks.py'
=== added file 'hooks/memcached_hooks.py'
--- hooks/memcached_hooks.py	1970-01-01 00:00:00 +0000
+++ hooks/memcached_hooks.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,217 @@
+#!/usr/bin/env python
+__author__ = 'Felipe Reyes <[e-mail]>'
+
+import re
+import os
+import shutil
+import subprocess
+import sys
+
+from charmhelpers.core.hookenv import (
+    config,
+    local_unit,
+    log,
+    relation_ids,
+    relation_get,
+    relation_set,
+    unit_get,
+    Hooks,
+    UnregisteredHookError,
+)
+from charmhelpers.core import templating
+from charmhelpers.core.host import (
+    restart_on_change,
+    service_reload,
+    service_start,
+    service_stop,
+)
+from charmhelpers.fetch import apt_install
+from charmhelpers.contrib.network import ufw
+import memcached_utils
+
+
+DOT = os.path.dirname(os.path.abspath(__file__))
+ETC_DEFAULT_MEMCACHED = '/etc/default/memcached'
+ETC_MEMCACHED_CONF = '/etc/memcached.conf'
+ETC_MUNIN_NODE_CONF = '/etc/munin/munin-node.conf'
+LOCAL_NAGIOS_PLUGINS = '/usr/local/lib/nagios/plugins'
+MEMCACHE_NAGIOS_PLUGIN = '/usr/local/lib/nagios/plugins/check_memcache.py'
+RESTART_MAP = {ETC_MEMCACHED_CONF: ['memcached'],
+               ETC_MUNIN_NODE_CONF: ['munin-node']}
+hooks = Hooks()
+
+
+class MemcachedError(Exception):
+    pass
+
+
[e-mail]('install')
+def install():
+    apt_install(["memcached", "python-cheetah", "python-memcache"], fatal=True)
+
+    with open(ETC_DEFAULT_MEMCACHED, 'w') as f:
+        f.write('ENABLE_MEMCACHED=yes\n')
+
+    ufw.enable()
+    ufw.service('ssh', 'open')
+
+
[e-mail]('start')
+def start():
+    service_start('memcached')
+
+
[e-mail]('stop')
+def stop():
+    service_stop('memcached')
+
+
[e-mail]('cache-relation-joined')
+def cache_relation_joined():
+
+    settings = {'host': unit_get('private-address'),
+                'port': config('tcp-port'),
+                'udp-port': config('udp-port')}
+
+    for rid in relation_ids('cache'):
+        relation_set(rid, **settings)
+
+    addr = relation_get('private-address')
+    if addr:
+        log('Granting memcached access to {}'.format(addr), level='INFO')
+        memcached_utils.grant_access(addr)
+
+
[e-mail]('cache-relation-departed')
+def cache_relation_departed():
+    addr = relation_get('private-address')
+    log('Revoking memcached access to {}'.format(addr))
+    memcached_utils.revoke_access(addr)
+
+
[e-mail]('config-changed')
+@restart_on_change(RESTART_MAP)
+def config_changed():
+    mem_size = config('size')
+
+    if mem_size == 0:
+        output = subprocess.check_output(['free', '-m'])
+        mem_size = int(re.findall(r'\d+', output.split('\n')[2])[1])
+        mem_size = int(mem_size * 0.9)
+
+    configs = {'mem_size': mem_size,
+               'large_pages_enabled': False}
+
+    for key in ['request-limit', 'min-item-size', 'slab-page-size', 'threads',
+                'disable-auto-cleanup', 'disable-cas', 'factor',
+                'connection-limit', 'tcp-port', 'udp-port']:
+        configs[key.replace('-', '_')] = config(key)
+        log('Config: {} = {}'.format(key, configs[key.replace('-', '_')]),
+            level='DEBUG')
+
+    if mem_size >= 2048:
+        disable_lp = config('disable-large-pages')
+        if not disable_lp:
+            log('Memory >= 2GB, using large-pages to speed up access',
+                level='INFO')
+            save_hp = int(subprocess.check_output(['sysctl', '-n',
+                                                   'vm.nr_hugepages']))
+
+            npages = mem_size / 2
+            subprocess.check_output(['sysctl', '-w',
+                                     'vm.nr_hugepages={}'.format(npages)])
+            allocated = int(subprocess.check_output(['sysctl', '-n',
+                                                     'vm.nr_hugepages']))
+
+            if allocated < npages:
+                log(('Cannot allocate $npages contiguous pages for huge '
+                     'page allocation'), level='WARNING')
+                log('Setting vm.nr_hugepages back to {}'.format(save_hp),
+                    level='WARNING')
+                subprocess.check_output(['sysctl', '-w',
+                                         'vm.nr_hugepages={}'.format(save_hp)])
+                log('Will disable large pages and fall back to regular memory',
+                    level='WARNING')
+                configs['large_pages_enabled'] = False
+            else:
+                log(('Allocated $npages contiguous blocks only '
+                     'for huge pages.'.format(npages)), level='WARNING')
+                configs['large_pages_enabled'] = True
+
+    templating.render('memcached.conf', ETC_MEMCACHED_CONF, configs)
+
+    # In case port changed, inform consumers
+    cache_relation_joined()
+
+
[e-mail]('munin-relation-changed')
+@restart_on_change(RESTART_MAP)
+def munin_relation_changed():
+    remote_ip = relation_get('private-address')
+
+    if not remote_ip:
+        log('Remote node must provide IP', level='INFO')
+        sys.exit(0)
+
+    munin_server_ip = memcached_utils.munin_format_ip(remote_ip)
+
+    # make sure munin port is open, the access is restricted by munin
+    ufw.service('4949', 'open')
+
+    # make sure it's instaled
+    apt_install(['munin-node'], fatal=True)
+    configs = {'munin_server': munin_server_ip}
+    templating.render('munin-node.conf', ETC_MUNIN_NODE_CONF, configs)
+    relation_set(ip=unit_get('private-address'))
+
+
[e-mail]('nrpe-external-master-relation-changed')
+@restart_on_change(RESTART_MAP)
+def nrpe_external_master_relation_changed():
+
+    # make sure it's installed
+    apt_install(['nagios-nrpe-server'], fatal=True)
+    ufw.service('5666', 'open')
+
+    if not os.path.isdir(LOCAL_NAGIOS_PLUGINS):
+        os.makedirs(LOCAL_NAGIOS_PLUGINS)
+
+    if not os.path.isfile(MEMCACHE_NAGIOS_PLUGIN):
+        shutil.copy(os.path.join(DOT, "..", "files", "nrpe-external-master",
+                                 "check_memcache.py"),
+                    os.path.join(LOCAL_NAGIOS_PLUGINS, "check_memcache.py"))
+        os.chmod(os.path.join(LOCAL_NAGIOS_PLUGINS, "check_memcache.py"),
+                 0o555)
+
+    nagios_hostname = "{}-{}".format(config('nagios_context'),
+                                     local_unit().replace('/', '-'))
+    configs = {'tcp_port': config('tcp_port'),
+               'servicegroup': config('nagios-context'),
+               'hostname': nagios_hostname}
+
+    templating.render('nrpe_check_memcached.cfg.tmpl',
+                      '/etc/nagios/nrpe.d/check_memcached.cfg',
+                      configs)
+    templating.render('nrpe_export.cfg.tmpl',
+                      ('/var/lib/nagios/export/service__{}_check_'
+                       'memcached.cfg').format(nagios_hostname),
+                      configs)
+
+    service_reload('nagios-nrpe-server')
+
+
[e-mail]('upgrade-charm')
+def upgrade_charm():
+    install()
+    config_changed()
+
+
+def main():
+    try:
+        hooks.execute(sys.argv)
+    except UnregisteredHookError as e:
+        log('Unknown hook {} - skipping.'.format(e))
+
+
+if __name__ == '__main__':
+    main()

=== added file 'hooks/memcached_utils.py'
--- hooks/memcached_utils.py	1970-01-01 00:00:00 +0000
+++ hooks/memcached_utils.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,27 @@
+__author__ = 'Felipe Reyes <[e-mail]>'
+
+from charmhelpers.contrib.network import ufw
+from charmhelpers.core.hookenv import (
+    config,
+    log,
+)
+
+
+def munin_format_ip(ip):
+    return "^{}$".format(ip.replace('.', '\\.'))
+
+
+def grant_access(address):
+    log('granting access: {}'.format(address), level='DEBUG')
+    ufw.grant_access(address, port=str(config('tcp-port')), proto='tcp')
+
+    if config('udp-port') > 0:
+        ufw.grant_access(address, port=str(config('udp-port')), proto='udp')
+
+
+def revoke_access(address):
+    log('revoking access: {}'.format(address), level='DEBUG')
+    ufw.revoke_access(address, port=str(config('tcp-port')), proto='tcp')
+
+    if config('udp-port') > 0:
+        ufw.revoke_access(address, port=str(config('udp-port')), proto='udp')

=== modified file 'hooks/munin-relation-changed'
--- hooks/munin-relation-changed	2011-06-01 06:33:38 +0000
+++ hooks/munin-relation-changed	1970-01-01 00:00:00 +0000
@@ -1,29 +0,0 @@
-#!/bin/sh
-
-set -ue
-
-ip=`relation-get ip`
-
-if [ -z "$ip" ] ; then
-  echo "Remote node must provide IP"
-  exit 0
-fi
-
-reip="^`echo $ip | sed -e 's,\.,\\.,g'`$"
-
-# Make sure its installed
-apt-get -y install munin-node
-
-if grep -q "^allow $reip$" /etc/munin/munin-node.conf ; then
-  echo $ip already has access.
-else
-  echo "# added by $0 `date`" >> /etc/munin/munin-node.conf
-  echo allow $reip >> /etc/munin/munin-node.conf
-fi
-
-service munin-node reload
-
-# Ubuntu package already enables all plugins at install time.
-
-# now tell remote server about our IP
-relation-set ip=`ifconfig  | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f2 | awk '{ print $1}'|head -n 1`

=== target is u'memcached_hooks.py'
=== modified file 'hooks/nrpe-external-master-relation-changed'
--- hooks/nrpe-external-master-relation-changed	2013-04-17 15:19:48 +0000
+++ hooks/nrpe-external-master-relation-changed	1970-01-01 00:00:00 +0000
@@ -1,19 +0,0 @@
-#!/bin/bash
-set -eux
-
-if [ ! -d /usr/local/lib/nagios/plugins ]; then
-    mkdir -p /usr/local/lib/nagios/plugins
-fi
-if [ ! -f /usr/local/lib/nagios/plugins/check_memcache.py ]; then
-    cp files/nrpe-external-master/check_memcache.py /usr/local/lib/nagios/plugins/check_memcache.py
-    chmod 555 /usr/local/lib/nagios/plugins/check_memcache.py
-fi
-
-export NAGIOS_HOSTNAME="$(config-get nagios_context)-${JUJU_UNIT_NAME//\//-}"
-export NAGIOS_SERVICEGROUP="$(config-get nagios-context)"
-export TCP_PORT="$(config-get tcp-port)"
-
-cheetah fill --env -p templates/nrpe_check_memcached.cfg.tmpl > /etc/nagios/nrpe.d/check_memcached.cfg
-cheetah fill --env -p templates/nrpe_export.cfg.tmpl > /var/lib/nagios/export/service__${NAGIOS_HOSTNAME}_check_memcached.cfg
-
-/etc/init.d/nagios-nrpe-server reload

=== target is u'memcached_hooks.py'
=== modified file 'hooks/start'
--- hooks/start	2011-12-05 22:55:46 +0000
+++ hooks/start	1970-01-01 00:00:00 +0000
@@ -1,2 +0,0 @@
-#!/bin/bash
-service memcached start

=== target is u'memcached_hooks.py'
=== modified file 'hooks/stop'
--- hooks/stop	2011-12-05 22:55:46 +0000
+++ hooks/stop	1970-01-01 00:00:00 +0000
@@ -1,3 +0,0 @@
-#!/bin/bash
-
-service memcached stop

=== target is u'memcached_hooks.py'
=== modified file 'hooks/upgrade-charm'
--- hooks/upgrade-charm	2011-12-05 22:55:46 +0000
+++ hooks/upgrade-charm	1970-01-01 00:00:00 +0000
@@ -1,4 +0,0 @@
-#!/bin/sh
-base=`dirname $0`
-$base/install
-exec $base/config-changed

=== target is u'memcached_hooks.py'
=== modified file 'metadata.yaml'
--- metadata.yaml	2014-01-15 18:42:43 +0000
+++ metadata.yaml	2014-12-09 22:04:35 +0000
@@ -1,6 +1,6 @@
 name: memcached
 summary: "A high-performance memory object caching system"
-maintainer: Clint Byrum <[e-mail]>
+maintainer: Felipe Reyes <[e-mail]>
 description: 
  memcached optimizes specific high-load serving applications that are designed
  to take advantage of its versatile no-locking memory access system. Clients
@@ -8,7 +8,7 @@
  of the specific application. Traditionally this has been used in mod_perl
  apps to avoid storing large chunks of data in Apache memory, and to share
  this burden across several machines.
-categories: ["applications"]
+tags: ["system"]
 provides:
   cache:
     interface: memcache

=== added file 'templates/memcached.conf'
--- templates/memcached.conf	1970-01-01 00:00:00 +0000
+++ templates/memcached.conf	2014-12-09 22:04:35 +0000
@@ -0,0 +1,62 @@
+################################
+#
+# This config file generated by the juju memcached charm. Changes
+# may not be preserved.
+#
+################################
+#
+# memcached default config file
+# 2003 - Jay Bonci <[e-mail]>
+# This configuration file is read by the start-memcached script provided as
+# part of the Debian GNU/Linux distribution. 
+
+# Run memcached as a daemon. This command is implied, and is not needed for the
+# daemon to run. See the README.Debian that comes with this package for more
+# information.
+-d
+
+# Log memcached's output to /var/log/memcached
+logfile /var/log/memcached.log
+
+# Be verbose
+# -v
+
+# Be even more verbose (print client commands as well)
+# -vv
+
+# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
+# Note that the daemon will grow to this size, but does not start out holding this much
+# memory
+-m {{mem_size}}
+
+# Default connection port is 11211
+-p {{tcp_port}}
+
+# Run the daemon as root. The start-memcached will default to running as root if no
+# -u command is present in this config file
+-u memcache
+
+# Specify which IP address to listen on. The default is to listen on all IP addresses
+# This parameter is one of the only security measures that memcached has, so make sure
+# it's listening on a firewalled interface.
+-l 0.0.0.0
+
+# Limit the number of simultaneous incoming connections. The daemon default is 1024
+-c {{connection_limit}}
+
+# Lock down all paged memory. Consult with the README and homepage before you do this
+# -k
+
+# Return error when memory is exhausted (rather than removing items)
+# -M
+
+# Maximize core file limit
+# -r
+
+{% if request_limit != -1 %}-R {{request_limit}}{% endif %}
+{% if min_item_size != -1 %}-n {{min_item_size}}{% endif %}
+{% if slab_page_size != -1 %}-I {{slab_page_size}}{% endif %}
+{% if threads != -1 %}-t {{threads}}{% endif %}
+{% if disable_auto_cleanup %}-M{% endif %}
+{% if disable_cas %}-C{% endif %}
+{% if factor != -1.0 %}-f {{factor}}{% endif %}

=== added file 'templates/munin-node.conf'
--- templates/munin-node.conf	1970-01-01 00:00:00 +0000
+++ templates/munin-node.conf	2014-12-09 22:04:35 +0000
@@ -0,0 +1,66 @@
+################################
+#
+# This config file generated by the juju memcached charm. Changes
+# may not be preserved.
+#
+################################
+
+log_level 4
+log_file /var/log/munin/munin-node.log
+pid_file /var/run/munin/munin-node.pid
+
+background 1
+setsid 1
+
+user root
+group root
+
+# This is the timeout for the whole transaction.
+# Units are in sec. Default is 15 min
+#
+# global_timeout 900
+
+# This is the timeout for each plugin.
+# Units are in sec. Default is 1 min
+#
+# timeout 60
+
+# Regexps for files to ignore
+ignore_file [\#~]$
+ignore_file DEADJOE$
+ignore_file \.bak$
+ignore_file %$
+ignore_file \.dpkg-(tmp|new|old|dist)$
+ignore_file \.rpm(save|new)$
+ignore_file \.pod$
+
+# Set this if the client doesn't report the correct hostname when
+# telnetting to localhost, port 4949
+#
+#host_name localhost.localdomain
+
+# A list of addresses that are allowed to connect.  This must be a
+# regular expression, since Net::Server does not understand CIDR-style
+# network notation unless the perl module Net::CIDR is installed.  You
+# may repeat the allow line as many times as you'd like
+
+allow ^127\.0\.0\.1$
+allow ^::1$
+{% if munin_server %}allow {{munin_server}}{% endif %}
+# If you have installed the Net::CIDR perl module, you can use one or more
+# cidr_allow and cidr_deny address/mask patterns.  A connecting client must
+# match any cidr_allow, and not match any cidr_deny.  Note that a netmask
+# *must* be provided, even if it's /32
+#
+# Example:
+#
+# cidr_allow 127.0.0.1/32
+# cidr_allow 192.0.2.0/24
+# cidr_deny  192.0.2.42/32
+
+# Which address to bind to;
+host *
+# host 127.0.0.1
+
+# And which port
+port 4949

=== modified file 'templates/nrpe_check_memcached.cfg.tmpl'
--- templates/nrpe_check_memcached.cfg.tmpl	2013-04-17 15:19:48 +0000
+++ templates/nrpe_check_memcached.cfg.tmpl	2014-12-09 22:04:35 +0000
@@ -1,4 +1,4 @@
 #---------------------------------------------------
 # This file is Juju managed
 #---------------------------------------------------
-command[check_memcached]=/usr/local/lib/nagios/plugins/check_memcache.py -H 127.0.0.1 -p ${TCP_PORT} -c version
+command[check_memcached]=/usr/local/lib/nagios/plugins/check_memcache.py -H 127.0.0.1 -p {{tcp_port}} -c version

=== modified file 'templates/nrpe_export.cfg.tmpl'
--- templates/nrpe_export.cfg.tmpl	2013-04-15 11:11:09 +0000
+++ templates/nrpe_export.cfg.tmpl	2014-12-09 22:04:35 +0000
@@ -3,8 +3,8 @@
 #---------------------------------------------------
 define service {
     use                             active-service
-    host_name                       ${NAGIOS_HOSTNAME}
-    service_description             ${NAGIOS_HOSTNAME} memcached
+    host_name                       {{NAGIOS_HOSTNAME}}
+    service_description             {{NAGIOS_HOSTNAME}} memcached
     check_command                   check_nrpe!check_memcached
-    servicegroups                   ${NAGIOS_SERVICEGROUP}, 
+    servicegroups                   {{NAGIOS_SERVICEGROUP}},
 }

=== added file 'test_requirements.txt'
--- test_requirements.txt	1970-01-01 00:00:00 +0000
+++ test_requirements.txt	2014-12-09 22:04:35 +0000
@@ -0,0 +1,7 @@
+nose
+Jinja2
+mock
+PyYAML
+six
+coverage
+flake8

=== modified file 'tests/10_deploy_test.py'
--- tests/10_deploy_test.py	2014-02-06 18:10:07 +0000
+++ tests/10_deploy_test.py	2014-12-09 22:04:35 +0000
@@ -1,6 +1,6 @@
 #!/usr/bin/python3
 
-## This Amulet test deploys memcached.
+# This Amulet test deploys memcached.
 
 import amulet
 import telnetlib
@@ -9,7 +9,7 @@
 # The number of seconds to wait for the environment to setup.
 seconds = 1200
 
-d = amulet.Deployment()
+d = amulet.Deployment(series="trusty")
 # Add the memcached charm to the deployment.
 d.add('memcached')
 # Add the mediawiki charm to the deployment.
@@ -39,7 +39,7 @@
 # Get the sentry for memcached.
 memcached_unit = d.sentry.unit['memcached/0']
 
-## Test if the memcached service is running.
+# Test if the memcached service is running.
 
 # Run the command that checks if the memcached server instance is running.
 command = 'service memcached status'
@@ -55,12 +55,22 @@
     print(output)
     print(message)
 
-## Test memcached using telnet commands.
+# Test memcached using telnet commands.
 
 # Get the public address for memcached instance.
 memcached_address = memcached_unit.info['public-address']
 # Get the port for memcached instance.
-memcached_port = configuration['tcp-port'] 
+memcached_port = configuration['tcp-port']
+
+try:
+    telnetlib.Telnet(memcached_address, memcached_port)
+    raise Exception(('memcached tcp port {} '
+                     'is not closed').format(memcached_port))
+except TimeoutError:  # noqa this exception only available in py3
+    pass  # this is good
+
+# open memcache to be able to connect from this machine
+memcached_unit.run('ufw allow {}'.format(memcached_port))
 
 try:
     # Connect to memcached via telnet.
@@ -95,13 +105,13 @@
         amulet.raise_status(amulet.FAIL, msg=message)
     tn.write(b'quit\n')
 except Exception as e:
-    message = 'An error occurred communicating with memcached over telnet ' \
-        '{0}:{1} {3}'.format(memcached_address, memcached_port, str(e))
+    message = ('An error occurred communicating with memcached over telnet '
+               '{0}:{1} {2}').format(memcached_address, memcached_port, str(e))
     amulet.raise_status(amulet.FAIL, msg=message)
 finally:
     tn.close()
 
-## Test if the memcached service is configured properly.
+# Test if the memcached service is configured properly.
 
 # Get the contents of the memcached configuration file.
 config_string = memcached_unit.file_contents('/etc/memcached.conf')
@@ -125,7 +135,7 @@
     message = 'The memcached deployment was configured correctly.'
     print(message)
 
-## Test if the relation is complete and data was exchanged properly.
+# Test if the relation is complete and data was exchanged properly.
 
 memcached_unit = d.sentry.unit['memcached/0']
 # Get the relation from memcached to mediawiki.
@@ -134,8 +144,9 @@
 # Make sure the relation got the port information set by the configuration.
 if (configuration['tcp-port'] != int(relation['port']) or
    configuration['udp-port'] != int(relation['udp-port'])):
-    message = 'The memcached relation was not configured correctly, port: ' \
-              '{0} udp-port: {1}'.format(relation['port'], relation['udp-port'])
+    message = ('The memcached relation was not configured correctly, port: '
+               '{0} udp-port: {1}').format(relation['port'],
+                                           relation['udp-port'])
     amulet.raise_status(amulet.FAIL, msg=message)
 else:
     message = 'The memcached relation was configured correctly.'

=== added directory 'unit_tests'
=== added file 'unit_tests/__init__.py'
--- unit_tests/__init__.py	1970-01-01 00:00:00 +0000
+++ unit_tests/__init__.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,2 @@
+import sys
+sys.path.append('hooks/')

=== added file 'unit_tests/test_memcached_hooks.py'
--- unit_tests/test_memcached_hooks.py	1970-01-01 00:00:00 +0000
+++ unit_tests/test_memcached_hooks.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,268 @@
+__author__ = 'Felipe Reyes <[e-mail]>'
+
+import mock
+import os
+import shutil
+import tempfile
+from test_utils import CharmTestCase
+import memcached_hooks
+
+
+DOT = os.path.dirname(os.path.abspath(__file__))
+TO_PATCH = [
+    'apt_install',
+    'service_start',
+    'service_stop',
+    'relation_ids',
+    'relation_set',
+    'relation_get',
+    'unit_get',
+    'config',
+    'log',
+]
+FREE_MEM_SMALL = """             total       used       free     shared    \
+buffers     cached
+Mem:          2002        744       1257          0         82        433
+-/+ buffers/cache:        227       1774
+Swap:            0          0          0
+"""
+FREE_MEM_BIG = """             total       used       free     shared    \
+buffers     cached
+Mem:         24010      13371      10639        207        484       6669
+-/+ buffers/cache:       6216      17793
+Swap:         7811          0       7811
+"""
+
+
[e-mail]('charmhelpers.contrib.network.ufw.is_enabled', lambda: True)
[e-mail]('charmhelpers.core.hookenv.log', mock.MagicMock())
+class TestMemcachedHooks(CharmTestCase):
+    def setUp(self):
+        self.fake_repo = {}
+        super(TestMemcachedHooks, self).setUp(memcached_hooks, TO_PATCH)
+        self.relation_get.return_value = '127.0.0.1'
+        self.tmpdir = tempfile.mkdtemp()
+        memcached_hooks.ETC_DEFAULT_MEMCACHED = os.path.join(self.tmpdir,
+                                                             "memcached")
+        memcached_hooks.ETC_MEMCACHED_CONF = os.path.join(self.tmpdir,
+                                                          "memcached.conf")
+
+    def tearDown(self):
+        super(TestMemcachedHooks, self).tearDown()
+        shutil.rmtree(self.tmpdir, ignore_errors=True)
+
+    @mock.patch('subprocess.check_output')
+    def test_install(self, check_output):
+        memcached_hooks.install()
+
+        self.apt_install.assert_called_with(["memcached", "python-cheetah",
+                                             "python-memcache"], fatal=True)
+        check_output.assert_any_call(['ufw', 'allow', 'ssh'])
+
+    def test_start(self):
+        memcached_hooks.start()
+        self.service_start.assert_called_with('memcached')
+
+    def test_stop(self):
+        memcached_hooks.stop()
+        self.service_stop.assert_called_with('memcached')
+
+    @mock.patch('subprocess.check_output')
+    @mock.patch('memcached_hooks.relation_get')
+    @mock.patch('memcached_utils.grant_access')
+    @mock.patch('memcached_utils.config')
+    @mock.patch('memcached_utils.log')
+    def test_cache_relation_joined(self, log, config, grant_access,
+                                   relation_get, check_output):
+        configs = {'tcp-port': '1234', 'udp-port': '3456'}
+
+        def f(c):
+            return configs.get(c, None)
+
+        self.config.side_effect = f
+        config.side_effect = f
+        relation_get.return_value = '127.0.1.1'
+        self.unit_get.return_value = '127.0.0.1'
+        self.relation_ids.return_value = ['cache:1', 'cache:2']
+        memcached_hooks.cache_relation_joined()
+        self.relation_set.assert_any_call('cache:1', **{'host': '127.0.0.1',
+                                                        'port': '1234',
+                                                        'udp-port': '3456'})
+        self.relation_set.assert_any_call('cache:2', **{'host': '127.0.0.1',
+                                                        'port': '1234',
+                                                        'udp-port': '3456'})
+        grant_access.assert_called_with('127.0.1.1')
+
+    @mock.patch('subprocess.check_output')
+    @mock.patch('memcached_hooks.relation_get')
+    @mock.patch('memcached_utils.config')
+    @mock.patch('memcached_utils.log')
+    @mock.patch('charmhelpers.contrib.network.ufw.revoke_access')
+    def test_cache_relation_departed(self, revoke_access, log, config,
+                                     relation_get, check_output):
+        config.return_value = 1234
+        relation_get.return_value = '127.0.1.1'
+        memcached_hooks.cache_relation_departed()
+
+        revoke_access.assert_any_call('127.0.1.1', port='1234', proto='tcp')
+        revoke_access.assert_any_call('127.0.1.1', port='1234', proto='udp')
+
+    @mock.patch('subprocess.Popen')
+    @mock.patch('subprocess.check_output')
+    @mock.patch('charmhelpers.core.templating.render')
+    @mock.patch('memcached_utils.config')
+    @mock.patch('memcached_utils.log')
+    def test_config_changed_defaults(self, log, config, render, check_output,
+                                     popen):
+        p = mock.Mock()
+        p.configure_mock(**{'communicate.return_value': ('stdout', 'stderr'),
+                            'returncode': 0})
+        popen.return_value = p
+
+        configs = self.test_config.config
+
+        def f(c):
+            return configs.get(c, None)
+
+        self.config.side_effect = f
+        config.side_effect = f
+
+        memcached_hooks.config_changed()
+
+        passed_vars = render.call_args[0][2]
+
+        self.assertEqual(passed_vars['mem_size'],
+                         self.test_config.config['size'])
+
+    @mock.patch('subprocess.Popen')
+    @mock.patch('charmhelpers.core.templating.render')
+    @mock.patch('subprocess.check_output')
+    @mock.patch('memcached_utils.log')
+    def test_config_changed_size_set_0_small(self, log, check_output, render,
+                                             popen):
+        p = mock.Mock()
+        p.configure_mock(**{'communicate.return_value': ('stdout', 'stderr'),
+                            'returncode': 0})
+        popen.return_value = p
+
+        configs = {'size': 0}
+
+        def f(c):
+            return configs.get(c, None)
+
+        self.config.side_effect = f
+
+        def g(*args, **kwargs):
+            if args[0] == ['free', '-m']:
+                return FREE_MEM_SMALL
+            else:
+                return ""
+
+        check_output.side_effect = g
+
+        memcached_hooks.config_changed()
+        passed_vars = render.call_args[0][2]
+
+        self.assertEqual(passed_vars['mem_size'], 1596)
+        self.assertFalse(passed_vars['large_pages_enabled'])
+
+    @mock.patch('subprocess.Popen')
+    @mock.patch('charmhelpers.core.templating.render')
+    @mock.patch('subprocess.check_output')
+    @mock.patch('memcached_utils.log')
+    def test_config_changed_size_set_0_big(self, log, check_output, render,
+                                           popen):
+        p = mock.Mock()
+        p.configure_mock(**{'communicate.return_value': ('stdout', 'stderr'),
+                            'returncode': 0})
+        popen.return_value = p
+
+        configs = {'size': 0,
+                   'disable-large-pages': False}
+
+        def f(c):
+            return configs.get(c, None)
+
+        self.config.side_effect = f
+
+        def g(*args, **kwargs):
+            if args[0] == ['free', '-m']:
+                return FREE_MEM_BIG
+            if args[0] == ['sysctl', '-n', 'vm.nr_hugepages']:
+                return int(16013 / 2)
+            else:
+                return ""
+
+        check_output.side_effect = g
+
+        memcached_hooks.config_changed()
+        passed_vars = render.call_args[0][2]
+
+        self.assertEqual(passed_vars['mem_size'], 16013)
+        check_output.assert_any_call(['sysctl', '-w',
+                                      'vm.nr_hugepages={}'.format(16013/2)])
+        self.assertTrue(passed_vars['large_pages_enabled'])
+
+    @mock.patch('subprocess.Popen')
+    @mock.patch('charmhelpers.core.templating.render')
+    @mock.patch('subprocess.check_output')
+    @mock.patch('memcached_utils.log')
+    def test_config_changed_size_failed_set_hpages(self, log, check_output,
+                                                   render, popen):
+        p = mock.Mock()
+        p.configure_mock(**{'communicate.return_value': ('stdout', 'stderr'),
+                            'returncode': 0})
+        popen.return_value = p
+
+        configs = {'size': 0,
+                   'disable-large-pages': False}
+
+        def f(c):
+            return configs.get(c, None)
+
+        self.config.side_effect = f
+
+        def g(*args, **kwargs):
+            if args[0] == ['free', '-m']:
+                return FREE_MEM_BIG
+            if args[0] == ['sysctl', '-n', 'vm.nr_hugepages']:
+                return 16
+            else:
+                return ""
+
+        check_output.side_effect = g
+
+        memcached_hooks.config_changed()
+        passed_vars = render.call_args[0][2]
+
+        self.assertEqual(passed_vars['mem_size'], 16013)
+        self.assertFalse(passed_vars['large_pages_enabled'])
+        check_output.assert_any_call(['sysctl', '-w',
+                                      'vm.nr_hugepages={}'.format(16013/2)])
+        check_output.assert_any_call(['sysctl', '-w',
+                                      'vm.nr_hugepages=16'])
+
+    @mock.patch('os.fchown')
+    @mock.patch('os.chown')
+    @mock.patch('memcached_utils.log')
+    @mock.patch('subprocess.Popen')
+    @mock.patch('memcached_utils.config')
+    @mock.patch('subprocess.check_output')
+    @mock.patch('charmhelpers.contrib.network.ufw.enable')
+    @mock.patch('charmhelpers.contrib.network.ufw.service')
+    @mock.patch('charmhelpers.core.hookenv.charm_dir')
+    @mock.patch('charmhelpers.core.host.log')
+    def test_upgrade_charm(self, log, charm_dir, service, enable, check_output,
+                           config, popen, *args):
+        p = mock.Mock()
+        p.configure_mock(**{'communicate.return_value': ('stdout', 'stderr'),
+                            'returncode': 0})
+        popen.return_value = p
+
+        charm_dir.return_value = os.path.join(DOT, '..')
+        config.return_value = 12111
+        memcached_hooks.upgrade_charm()
+        self.apt_install.assert_any_call(['memcached', 'python-cheetah',
+                                          'python-memcache'], fatal=True)
+        enable.assert_called_with()
+        service.assert_called_with('ssh', 'open')

=== added file 'unit_tests/test_utils.py'
--- unit_tests/test_utils.py	1970-01-01 00:00:00 +0000
+++ unit_tests/test_utils.py	2014-12-09 22:04:35 +0000
@@ -0,0 +1,122 @@
+# grabbed from nova-cloud-controller/unit_tests/test_utils.py
+import logging
+import unittest
+import os
+import yaml
+
+from contextlib import contextmanager
+from mock import patch, MagicMock
+
+
+def load_config():
+    '''
+    Walk backwords from __file__ looking for config.yaml, load and return the
+    'options' section'
+    '''
+    config = None
+    f = __file__
+    while config is None:
+        d = os.path.dirname(f)
+        if os.path.isfile(os.path.join(d, 'config.yaml')):
+            config = os.path.join(d, 'config.yaml')
+            break
+        f = d
+
+    if not config:
+        logging.error('Could not find config.yaml in any parent directory '
+                      'of %s. ' % file)
+        raise Exception
+
+    return yaml.safe_load(open(config).read())['options']
+
+
+def get_default_config():
+    '''
+    Load default charm config from config.yaml return as a dict.
+    If no default is set in config.yaml, its value is None.
+    '''
+    default_config = {}
+    config = load_config()
+    for k, v in config.iteritems():
+        if 'default' in v:
+            default_config[k] = v['default']
+        else:
+            default_config[k] = None
+    return default_config
+
+
+class CharmTestCase(unittest.TestCase):
+
+    def setUp(self, obj, patches):
+        super(CharmTestCase, self).setUp()
+        self.patches = patches
+        self.obj = obj
+        self.test_config = TestConfig()
+        self.test_relation = TestRelation()
+        self.patch_all()
+
+    def patch(self, method):
+        _m = patch.object(self.obj, method)
+        mock = _m.start()
+        self.addCleanup(_m.stop)
+        return mock
+
+    def patch_all(self):
+        for method in self.patches:
+            setattr(self, method, self.patch(method))
+
+
+class TestConfig(object):
+
+    def __init__(self):
+        self.config = get_default_config()
+
+    def get(self, attr=None):
+        if not attr:
+            return self.get_all()
+        try:
+            return self.config[attr]
+        except KeyError:
+            return None
+
+    def get_all(self):
+        return self.config
+
+    def set(self, attr, value):
+        if attr not in self.config:
+            raise KeyError
+        self.config[attr] = value
+
+
+class TestRelation(object):
+
+    def __init__(self, relation_data={}):
+        self.relation_data = relation_data
+
+    def set(self, relation_data):
+        self.relation_data = relation_data
+
+    def get(self, attr=None, unit=None, rid=None):
+        if attr is None:
+            return self.relation_data
+        elif attr in self.relation_data:
+            return self.relation_data[attr]
+        return None
+
+
+@contextmanager
+def patch_open():
+    '''Patch open() to allow mocking both open() itself and the file that is
+    yielded.
+
+    Yields the mock for "open" and "file", respectively.'''
+    mock_open = MagicMock(spec=open)
+    mock_file = MagicMock(spec=file)
+
+    @contextmanager
+    def stub_open(*args, **kwargs):
+        mock_open(*args, **kwargs)
+        yield mock_file
+
+    with patch('__builtin__.open', stub_open):
+        yield mock_open, mock_file