Merge lp:~bloodearnest/charms/precise/gunicorn/cleanup into lp:~charmers/charms/precise/gunicorn/trunk

[Simon Davy]

Clean up of gunicorn charm, adding charmhelpers, tests and re-add env_extra param

Clean up of gunicorn charm.

Apologies for the large diff - it's mostly bundling charm helpers

Main highlights
 - use charmhelpers, and thus remove lots of code that was doing the same stuff
 - tests \o/
 - slight cleanups and refactors (e.g. not shelling out to multiprocess, when we can just import it)
 - add support back in for "env_extra", which we use in our IS version to set process envvars (was strangely removed in python rewrite), albeit with a slightly different (better) format.
 - add support for config-changed hook (only when wsgi-file relation is present)

[lp:~bloodearnest/charms/precise/gunicorn/cleanup]

31. By Simon Davy on 2014-02-27

added config changed hook

32. By Simon Davy on 2014-02-27

fix config-changed, and cleaner access log handling

33. By Simon Davy on 2014-02-27

minor formatting tweaks

34. By Simon Davy on 2014-02-27

changed to restart rather than reload

[Michael Nelson (michael.nelson)]

+1. I'm now using this version of the charm for deploying a new service (as I needed the env_extra option).

Most of the green diff below is just the result of syncing a recent charm-helpers.

Great to see some tests, and great to trim down the hooks.py by using charmhelpers instead.

Thanks!

review: Approve

[lp:~bloodearnest/charms/precise/gunicorn/cleanup]

35. By Simon Davy on 2014-03-04

add make unit_test target

[Marco Ceppi (marcoceppi)]

LGTM +1

review: Approve

=== added file '.bzrignore'
--- .bzrignore	1970-01-01 00:00:00 +0000
+++ .bzrignore	2014-03-04 16:16:33 +0000
@@ -0,0 +1,1 @@
+bin

=== added file 'Makefile'
--- Makefile	1970-01-01 00:00:00 +0000
+++ Makefile	2014-03-04 16:16:33 +0000
@@ -0,0 +1,12 @@
+#!/usr/bin/make
+PYTHON := /usr/bin/env python
+
+unit_test:
+	nosetests hooks
+
+sync-charm-helpers: bin/charm_helpers_sync.py
+	@mkdir -p bin
+	@$(PYTHON) bin/charm_helpers_sync.py -c charm-helpers.yaml
+
+bin/charm_helpers_sync.py:
+	@bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py > bin/charm_helpers_sync.py

=== added file 'charm-helpers.yaml'
--- charm-helpers.yaml	1970-01-01 00:00:00 +0000
+++ charm-helpers.yaml	2014-03-04 16:16:33 +0000
@@ -0,0 +1,5 @@
+branch: lp:charm-helpers
+destination: hooks/charmhelpers
+include:
+    - core
+    - fetch

=== modified file 'config.yaml'
--- config.yaml	2013-05-29 18:40:56 +0000
+++ config.yaml	2014-03-04 16:16:33 +0000
@@ -79,3 +79,9 @@
         type: int
         default: 8080
         description: "Port the application will be listenning."
+    env_extra:
+        type: string
+        default: ""
+        description: >
+            List of environment variables for the wsgi process. 
+            e.g. FOO="bar" BAZ="1 2 3"

=== added directory 'hooks/charmhelpers'
=== added file 'hooks/charmhelpers/__init__.py'
=== added directory 'hooks/charmhelpers/core'
=== added file 'hooks/charmhelpers/core/__init__.py'
=== added file 'hooks/charmhelpers/core/hookenv.py'
--- hooks/charmhelpers/core/hookenv.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/hookenv.py	2014-03-04 16:16:33 +0000
@@ -0,0 +1,401 @@
+"Interactions with the Juju environment"
+# Copyright 2013 Canonical Ltd.
+#
+# Authors:
+#  Charm Helpers Developers <[e-mail]>
+
+import os
+import json
+import yaml
+import subprocess
+import sys
+import UserDict
+from subprocess import CalledProcessError
+
+CRITICAL = "CRITICAL"
+ERROR = "ERROR"
+WARNING = "WARNING"
+INFO = "INFO"
+DEBUG = "DEBUG"
+MARKER = object()
+
+cache = {}
+
+
+def cached(func):
+    """Cache return values for multiple executions of func + args
+
+    For example:
+
+        @cached
+        def unit_get(attribute):
+            pass
+
+        unit_get('test')
+
+    will cache the result of unit_get + 'test' for future calls.
+    """
+    def wrapper(*args, **kwargs):
+        global cache
+        key = str((func, args, kwargs))
+        try:
+            return cache[key]
+        except KeyError:
+            res = func(*args, **kwargs)
+            cache[key] = res
+            return res
+    return wrapper
+
+
+def flush(key):
+    """Flushes any entries from function cache where the
+    key is found in the function+args """
+    flush_list = []
+    for item in cache:
+        if key in item:
+            flush_list.append(item)
+    for item in flush_list:
+        del cache[item]
+
+
+def log(message, level=None):
+    """Write a message to the juju log"""
+    command = ['juju-log']
+    if level:
+        command += ['-l', level]
+    command += [message]
+    subprocess.call(command)
+
+
+class Serializable(UserDict.IterableUserDict):
+    """Wrapper, an object that can be serialized to yaml or json"""
+
+    def __init__(self, obj):
+        # wrap the object
+        UserDict.IterableUserDict.__init__(self)
+        self.data = obj
+
+    def __getattr__(self, attr):
+        # See if this object has attribute.
+        if attr in ("json", "yaml", "data"):
+            return self.__dict__[attr]
+        # Check for attribute in wrapped object.
+        got = getattr(self.data, attr, MARKER)
+        if got is not MARKER:
+            return got
+        # Proxy to the wrapped object via dict interface.
+        try:
+            return self.data[attr]
+        except KeyError:
+            raise AttributeError(attr)
+
+    def __getstate__(self):
+        # Pickle as a standard dictionary.
+        return self.data
+
+    def __setstate__(self, state):
+        # Unpickle into our wrapper.
+        self.data = state
+
+    def json(self):
+        """Serialize the object to json"""
+        return json.dumps(self.data)
+
+    def yaml(self):
+        """Serialize the object to yaml"""
+        return yaml.dump(self.data)
+
+
+def execution_environment():
+    """A convenient bundling of the current execution context"""
+    context = {}
+    context['conf'] = config()
+    if relation_id():
+        context['reltype'] = relation_type()
+        context['relid'] = relation_id()
+        context['rel'] = relation_get()
+    context['unit'] = local_unit()
+    context['rels'] = relations()
+    context['env'] = os.environ
+    return context
+
+
+def in_relation_hook():
+    """Determine whether we're running in a relation hook"""
+    return 'JUJU_RELATION' in os.environ
+
+
+def relation_type():
+    """The scope for the current relation hook"""
+    return os.environ.get('JUJU_RELATION', None)
+
+
+def relation_id():
+    """The relation ID for the current relation hook"""
+    return os.environ.get('JUJU_RELATION_ID', None)
+
+
+def local_unit():
+    """Local unit ID"""
+    return os.environ['JUJU_UNIT_NAME']
+
+
+def remote_unit():
+    """The remote unit for the current relation hook"""
+    return os.environ['JUJU_REMOTE_UNIT']
+
+
+def service_name():
+    """The name service group this unit belongs to"""
+    return local_unit().split('/')[0]
+
+
+def hook_name():
+    """The name of the currently executing hook"""
+    return os.path.basename(sys.argv[0])
+
+
+@cached
+def config(scope=None):
+    """Juju charm configuration"""
+    config_cmd_line = ['config-get']
+    if scope is not None:
+        config_cmd_line.append(scope)
+    config_cmd_line.append('--format=json')
+    try:
+        return json.loads(subprocess.check_output(config_cmd_line))
+    except ValueError:
+        return None
+
+
+@cached
+def relation_get(attribute=None, unit=None, rid=None):
+    """Get relation information"""
+    _args = ['relation-get', '--format=json']
+    if rid:
+        _args.append('-r')
+        _args.append(rid)
+    _args.append(attribute or '-')
+    if unit:
+        _args.append(unit)
+    try:
+        return json.loads(subprocess.check_output(_args))
+    except ValueError:
+        return None
+    except CalledProcessError, e:
+        if e.returncode == 2:
+            return None
+        raise
+
+
+def relation_set(relation_id=None, relation_settings={}, **kwargs):
+    """Set relation information for the current unit"""
+    relation_cmd_line = ['relation-set']
+    if relation_id is not None:
+        relation_cmd_line.extend(('-r', relation_id))
+    for k, v in (relation_settings.items() + kwargs.items()):
+        if v is None:
+            relation_cmd_line.append('{}='.format(k))
+        else:
+            relation_cmd_line.append('{}={}'.format(k, v))
+    subprocess.check_call(relation_cmd_line)
+    # Flush cache of any relation-gets for local unit
+    flush(local_unit())
+
+
+@cached
+def relation_ids(reltype=None):
+    """A list of relation_ids"""
+    reltype = reltype or relation_type()
+    relid_cmd_line = ['relation-ids', '--format=json']
+    if reltype is not None:
+        relid_cmd_line.append(reltype)
+        return json.loads(subprocess.check_output(relid_cmd_line)) or []
+    return []
+
+
+@cached
+def related_units(relid=None):
+    """A list of related units"""
+    relid = relid or relation_id()
+    units_cmd_line = ['relation-list', '--format=json']
+    if relid is not None:
+        units_cmd_line.extend(('-r', relid))
+    return json.loads(subprocess.check_output(units_cmd_line)) or []
+
+
+@cached
+def relation_for_unit(unit=None, rid=None):
+    """Get the json represenation of a unit's relation"""
+    unit = unit or remote_unit()
+    relation = relation_get(unit=unit, rid=rid)
+    for key in relation:
+        if key.endswith('-list'):
+            relation[key] = relation[key].split()
+    relation['__unit__'] = unit
+    return relation
+
+
+@cached
+def relations_for_id(relid=None):
+    """Get relations of a specific relation ID"""
+    relation_data = []
+    relid = relid or relation_ids()
+    for unit in related_units(relid):
+        unit_data = relation_for_unit(unit, relid)
+        unit_data['__relid__'] = relid
+        relation_data.append(unit_data)
+    return relation_data
+
+
+@cached
+def relations_of_type(reltype=None):
+    """Get relations of a specific type"""
+    relation_data = []
+    reltype = reltype or relation_type()
+    for relid in relation_ids(reltype):
+        for relation in relations_for_id(relid):
+            relation['__relid__'] = relid
+            relation_data.append(relation)
+    return relation_data
+
+
+@cached
+def relation_types():
+    """Get a list of relation types supported by this charm"""
+    charmdir = os.environ.get('CHARM_DIR', '')
+    mdf = open(os.path.join(charmdir, 'metadata.yaml'))
+    md = yaml.safe_load(mdf)
+    rel_types = []
+    for key in ('provides', 'requires', 'peers'):
+        section = md.get(key)
+        if section:
+            rel_types.extend(section.keys())
+    mdf.close()
+    return rel_types
+
+
+@cached
+def relations():
+    """Get a nested dictionary of relation data for all related units"""
+    rels = {}
+    for reltype in relation_types():
+        relids = {}
+        for relid in relation_ids(reltype):
+            units = {local_unit(): relation_get(unit=local_unit(), rid=relid)}
+            for unit in related_units(relid):
+                reldata = relation_get(unit=unit, rid=relid)
+                units[unit] = reldata
+            relids[relid] = units
+        rels[reltype] = relids
+    return rels
+
+
+@cached
+def is_relation_made(relation, keys='private-address'):
+    '''
+    Determine whether a relation is established by checking for
+    presence of key(s).  If a list of keys is provided, they
+    must all be present for the relation to be identified as made
+    '''
+    if isinstance(keys, str):
+        keys = [keys]
+    for r_id in relation_ids(relation):
+        for unit in related_units(r_id):
+            context = {}
+            for k in keys:
+                context[k] = relation_get(k, rid=r_id,
+                                          unit=unit)
+            if None not in context.values():
+                return True
+    return False
+
+
+def open_port(port, protocol="TCP"):
+    """Open a service network port"""
+    _args = ['open-port']
+    _args.append('{}/{}'.format(port, protocol))
+    subprocess.check_call(_args)
+
+
+def close_port(port, protocol="TCP"):
+    """Close a service network port"""
+    _args = ['close-port']
+    _args.append('{}/{}'.format(port, protocol))
+    subprocess.check_call(_args)
+
+
+@cached
+def unit_get(attribute):
+    """Get the unit ID for the remote unit"""
+    _args = ['unit-get', '--format=json', attribute]
+    try:
+        return json.loads(subprocess.check_output(_args))
+    except ValueError:
+        return None
+
+
+def unit_private_ip():
+    """Get this unit's private IP address"""
+    return unit_get('private-address')
+
+
+class UnregisteredHookError(Exception):
+    """Raised when an undefined hook is called"""
+    pass
+
+
+class Hooks(object):
+    """A convenient handler for hook functions.
+
+    Example:
+        hooks = Hooks()
+
+        # register a hook, taking its name from the function name
+        @hooks.hook()
+        def install():
+            ...
+
+        # register a hook, providing a custom hook name
+        @hooks.hook("config-changed")
+        def config_changed():
+            ...
+
+        if __name__ == "__main__":
+            # execute a hook based on the name the program is called by
+            hooks.execute(sys.argv)
+    """
+
+    def __init__(self):
+        super(Hooks, self).__init__()
+        self._hooks = {}
+
+    def register(self, name, function):
+        """Register a hook"""
+        self._hooks[name] = function
+
+    def execute(self, args):
+        """Execute a registered hook based on args[0]"""
+        hook_name = os.path.basename(args[0])
+        if hook_name in self._hooks:
+            self._hooks[hook_name]()
+        else:
+            raise UnregisteredHookError(hook_name)
+
+    def hook(self, *hook_names):
+        """Decorator, registering them as hooks"""
+        def wrapper(decorated):
+            for hook_name in hook_names:
+                self.register(hook_name, decorated)
+            else:
+                self.register(decorated.__name__, decorated)
+                if '_' in decorated.__name__:
+                    self.register(
+                        decorated.__name__.replace('_', '-'), decorated)
+            return decorated
+        return wrapper
+
+
+def charm_dir():
+    """Return the root directory of the current charm"""
+    return os.environ.get('CHARM_DIR')

=== added file 'hooks/charmhelpers/core/host.py'
--- hooks/charmhelpers/core/host.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/host.py	2014-03-04 16:16:33 +0000
@@ -0,0 +1,291 @@
+"""Tools for working with the host system"""
+# Copyright 2012 Canonical Ltd.
+#
+# Authors:
+#  Nick Moffitt <[e-mail]>
+#  Matthew Wedgwood <[e-mail]>
+
+import os
+import pwd
+import grp
+import random
+import string
+import subprocess
+import hashlib
+
+from collections import OrderedDict
+
+from hookenv import log
+
+
+def service_start(service_name):
+    """Start a system service"""
+    return service('start', service_name)
+
+
+def service_stop(service_name):
+    """Stop a system service"""
+    return service('stop', service_name)
+
+
+def service_restart(service_name):
+    """Restart a system service"""
+    return service('restart', service_name)
+
+
+def service_reload(service_name, restart_on_failure=False):
+    """Reload a system service, optionally falling back to restart if reload fails"""
+    service_result = service('reload', service_name)
+    if not service_result and restart_on_failure:
+        service_result = service('restart', service_name)
+    return service_result
+
+
+def service(action, service_name):
+    """Control a system service"""
+    cmd = ['service', service_name, action]
+    return subprocess.call(cmd) == 0
+
+
+def service_running(service):
+    """Determine whether a system service is running"""
+    try:
+        output = subprocess.check_output(['service', service, 'status'])
+    except subprocess.CalledProcessError:
+        return False
+    else:
+        if ("start/running" in output or "is running" in output):
+            return True
+        else:
+            return False
+
+
+def adduser(username, password=None, shell='/bin/bash', system_user=False):
+    """Add a user to the system"""
+    try:
+        user_info = pwd.getpwnam(username)
+        log('user {0} already exists!'.format(username))
+    except KeyError:
+        log('creating user {0}'.format(username))
+        cmd = ['useradd']
+        if system_user or password is None:
+            cmd.append('--system')
+        else:
+            cmd.extend([
+                '--create-home',
+                '--shell', shell,
+                '--password', password,
+            ])
+        cmd.append(username)
+        subprocess.check_call(cmd)
+        user_info = pwd.getpwnam(username)
+    return user_info
+
+
+def add_user_to_group(username, group):
+    """Add a user to a group"""
+    cmd = [
+        'gpasswd', '-a',
+        username,
+        group
+    ]
+    log("Adding user {} to group {}".format(username, group))
+    subprocess.check_call(cmd)
+
+
+def rsync(from_path, to_path, flags='-r', options=None):
+    """Replicate the contents of a path"""
+    options = options or ['--delete', '--executability']
+    cmd = ['/usr/bin/rsync', flags]
+    cmd.extend(options)
+    cmd.append(from_path)
+    cmd.append(to_path)
+    log(" ".join(cmd))
+    return subprocess.check_output(cmd).strip()
+
+
+def symlink(source, destination):
+    """Create a symbolic link"""
+    log("Symlinking {} as {}".format(source, destination))
+    cmd = [
+        'ln',
+        '-sf',
+        source,
+        destination,
+    ]
+    subprocess.check_call(cmd)
+
+
+def mkdir(path, owner='root', group='root', perms=0555, force=False):
+    """Create a directory"""
+    log("Making dir {} {}:{} {:o}".format(path, owner, group,
+                                          perms))
+    uid = pwd.getpwnam(owner).pw_uid
+    gid = grp.getgrnam(group).gr_gid
+    realpath = os.path.abspath(path)
+    if os.path.exists(realpath):
+        if force and not os.path.isdir(realpath):
+            log("Removing non-directory file {} prior to mkdir()".format(path))
+            os.unlink(realpath)
+    else:
+        os.makedirs(realpath, perms)
+    os.chown(realpath, uid, gid)
+
+
+def write_file(path, content, owner='root', group='root', perms=0444):
+    """Create or overwrite a file with the contents of a string"""
+    log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
+    uid = pwd.getpwnam(owner).pw_uid
+    gid = grp.getgrnam(group).gr_gid
+    with open(path, 'w') as target:
+        os.fchown(target.fileno(), uid, gid)
+        os.fchmod(target.fileno(), perms)
+        target.write(content)
+
+
+def mount(device, mountpoint, options=None, persist=False):
+    """Mount a filesystem at a particular mountpoint"""
+    cmd_args = ['mount']
+    if options is not None:
+        cmd_args.extend(['-o', options])
+    cmd_args.extend([device, mountpoint])
+    try:
+        subprocess.check_output(cmd_args)
+    except subprocess.CalledProcessError, e:
+        log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output))
+        return False
+    if persist:
+        # TODO: update fstab
+        pass
+    return True
+
+
+def umount(mountpoint, persist=False):
+    """Unmount a filesystem"""
+    cmd_args = ['umount', mountpoint]
+    try:
+        subprocess.check_output(cmd_args)
+    except subprocess.CalledProcessError, e:
+        log('Error unmounting {}\n{}'.format(mountpoint, e.output))
+        return False
+    if persist:
+        # TODO: update fstab
+        pass
+    return True
+
+
+def mounts():
+    """Get a list of all mounted volumes as [[mountpoint,device],[...]]"""
+    with open('/proc/mounts') as f:
+        # [['/mount/point','/dev/path'],[...]]
+        system_mounts = [m[1::-1] for m in [l.strip().split()
+                                            for l in f.readlines()]]
+    return system_mounts
+
+
+def file_hash(path):
+    """Generate a md5 hash of the contents of 'path' or None if not found """
+    if os.path.exists(path):
+        h = hashlib.md5()
+        with open(path, 'r') as source:
+            h.update(source.read())  # IGNORE:E1101 - it does have update
+        return h.hexdigest()
+    else:
+        return None
+
+
+def restart_on_change(restart_map):
+    """Restart services based on configuration files changing
+
+    This function is used a decorator, for example
+
+        @restart_on_change({
+            '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]
+            })
+        def ceph_client_changed():
+            ...
+
+    In this example, the cinder-api and cinder-volume services
+    would be restarted if /etc/ceph/ceph.conf is changed by the
+    ceph_client_changed function.
+    """
+    def wrap(f):
+        def wrapped_f(*args):
+            checksums = {}
+            for path in restart_map:
+                checksums[path] = file_hash(path)
+            f(*args)
+            restarts = []
+            for path in restart_map:
+                if checksums[path] != file_hash(path):
+                    restarts += restart_map[path]
+            for service_name in list(OrderedDict.fromkeys(restarts)):
+                service('restart', service_name)
+        return wrapped_f
+    return wrap
+
+
+def lsb_release():
+    """Return /etc/lsb-release in a dict"""
+    d = {}
+    with open('/etc/lsb-release', 'r') as lsb:
+        for l in lsb:
+            k, v = l.split('=')
+            d[k.strip()] = v.strip()
+    return d
+
+
+def pwgen(length=None):
+    """Generate a random pasword."""
+    if length is None:
+        length = random.choice(range(35, 45))
+    alphanumeric_chars = [
+        l for l in (string.letters + string.digits)
+        if l not in 'l0QD1vAEIOUaeiou']
+    random_chars = [
+        random.choice(alphanumeric_chars) for _ in range(length)]
+    return(''.join(random_chars))
+
+
+def list_nics(nic_type):
+    '''Return a list of nics of given type(s)'''
+    if isinstance(nic_type, basestring):
+        int_types = [nic_type]
+    else:
+        int_types = nic_type
+    interfaces = []
+    for int_type in int_types:
+        cmd = ['ip', 'addr', 'show', 'label', int_type + '*']
+        ip_output = subprocess.check_output(cmd).split('\n')
+        ip_output = (line for line in ip_output if line)
+        for line in ip_output:
+            if line.split()[1].startswith(int_type):
+                interfaces.append(line.split()[1].replace(":", ""))
+    return interfaces
+
+
+def set_nic_mtu(nic, mtu):
+    '''Set MTU on a network interface'''
+    cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
+    subprocess.check_call(cmd)
+
+
+def get_nic_mtu(nic):
+    cmd = ['ip', 'addr', 'show', nic]
+    ip_output = subprocess.check_output(cmd).split('\n')
+    mtu = ""
+    for line in ip_output:
+        words = line.split()
+        if 'mtu' in words:
+            mtu = words[words.index("mtu") + 1]
+    return mtu
+
+
+def get_nic_hwaddr(nic):
+    cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
+    ip_output = subprocess.check_output(cmd)
+    hwaddr = ""
+    words = ip_output.split()
+    if 'link/ether' in words:
+        hwaddr = words[words.index('link/ether') + 1]
+    return hwaddr

=== added directory 'hooks/charmhelpers/fetch'
=== added file 'hooks/charmhelpers/fetch/__init__.py'
--- hooks/charmhelpers/fetch/__init__.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/__init__.py	2014-03-04 16:16:33 +0000
@@ -0,0 +1,279 @@
+import importlib
+from yaml import safe_load
+from charmhelpers.core.host import (
+    lsb_release
+)
+from urlparse import (
+    urlparse,
+    urlunparse,
+)
+import subprocess
+from charmhelpers.core.hookenv import (
+    config,
+    log,
+)
+import apt_pkg
+import os
+
+CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
+deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
+"""
+PROPOSED_POCKET = """# Proposed
+deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
+"""
+CLOUD_ARCHIVE_POCKETS = {
+    # Folsom
+    'folsom': 'precise-updates/folsom',
+    'precise-folsom': 'precise-updates/folsom',
+    'precise-folsom/updates': 'precise-updates/folsom',
+    'precise-updates/folsom': 'precise-updates/folsom',
+    'folsom/proposed': 'precise-proposed/folsom',
+    'precise-folsom/proposed': 'precise-proposed/folsom',
+    'precise-proposed/folsom': 'precise-proposed/folsom',
+    # Grizzly
+    'grizzly': 'precise-updates/grizzly',
+    'precise-grizzly': 'precise-updates/grizzly',
+    'precise-grizzly/updates': 'precise-updates/grizzly',
+    'precise-updates/grizzly': 'precise-updates/grizzly',
+    'grizzly/proposed': 'precise-proposed/grizzly',
+    'precise-grizzly/proposed': 'precise-proposed/grizzly',
+    'precise-proposed/grizzly': 'precise-proposed/grizzly',
+    # Havana
+    'havana': 'precise-updates/havana',
+    'precise-havana': 'precise-updates/havana',
+    'precise-havana/updates': 'precise-updates/havana',
+    'precise-updates/havana': 'precise-updates/havana',
+    'havana/proposed': 'precise-proposed/havana',
+    'precise-havana/proposed': 'precise-proposed/havana',
+    'precise-proposed/havana': 'precise-proposed/havana',
+    # Icehouse
+    'icehouse': 'precise-updates/icehouse',
+    'precise-icehouse': 'precise-updates/icehouse',
+    'precise-icehouse/updates': 'precise-updates/icehouse',
+    'precise-updates/icehouse': 'precise-updates/icehouse',
+    'icehouse/proposed': 'precise-proposed/icehouse',
+    'precise-icehouse/proposed': 'precise-proposed/icehouse',
+    'precise-proposed/icehouse': 'precise-proposed/icehouse',
+}
+
+
+def filter_installed_packages(packages):
+    """Returns a list of packages that require installation"""
+    apt_pkg.init()
+    cache = apt_pkg.Cache()
+    _pkgs = []
+    for package in packages:
+        try:
+            p = cache[package]
+            p.current_ver or _pkgs.append(package)
+        except KeyError:
+            log('Package {} has no installation candidate.'.format(package),
+                level='WARNING')
+            _pkgs.append(package)
+    return _pkgs
+
+
+def apt_install(packages, options=None, fatal=False):
+    """Install one or more packages"""
+    if options is None:
+        options = ['--option=Dpkg::Options::=--force-confold']
+
+    cmd = ['apt-get', '--assume-yes']
+    cmd.extend(options)
+    cmd.append('install')
+    if isinstance(packages, basestring):
+        cmd.append(packages)
+    else:
+        cmd.extend(packages)
+    log("Installing {} with options: {}".format(packages,
+                                                options))
+    env = os.environ.copy()
+    if 'DEBIAN_FRONTEND' not in env:
+        env['DEBIAN_FRONTEND'] = 'noninteractive'
+
+    if fatal:
+        subprocess.check_call(cmd, env=env)
+    else:
+        subprocess.call(cmd, env=env)
+
+
+def apt_update(fatal=False):
+    """Update local apt cache"""
+    cmd = ['apt-get', 'update']
+    if fatal:
+        subprocess.check_call(cmd)
+    else:
+        subprocess.call(cmd)
+
+
+def apt_purge(packages, fatal=False):
+    """Purge one or more packages"""
+    cmd = ['apt-get', '--assume-yes', 'purge']
+    if isinstance(packages, basestring):
+        cmd.append(packages)
+    else:
+        cmd.extend(packages)
+    log("Purging {}".format(packages))
+    if fatal:
+        subprocess.check_call(cmd)
+    else:
+        subprocess.call(cmd)
+
+
+def apt_hold(packages, fatal=False):
+    """Hold one or more packages"""
+    cmd = ['apt-mark', 'hold']
+    if isinstance(packages, basestring):
+        cmd.append(packages)
+    else:
+        cmd.extend(packages)
+    log("Holding {}".format(packages))
+    if fatal:
+        subprocess.check_call(cmd)
+    else:
+        subprocess.call(cmd)
+
+
+def add_source(source, key=None):
+    if (source.startswith('ppa:') or
+        source.startswith('http:') or
+        source.startswith('deb ') or
+            source.startswith('cloud-archive:')):
+        subprocess.check_call(['add-apt-repository', '--yes', source])
+    elif source.startswith('cloud:'):
+        apt_install(filter_installed_packages(['ubuntu-cloud-keyring']),
+                    fatal=True)
+        pocket = source.split(':')[-1]
+        if pocket not in CLOUD_ARCHIVE_POCKETS:
+            raise SourceConfigError(
+                'Unsupported cloud: source option %s' %
+                pocket)
+        actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket]
+        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
+            apt.write(CLOUD_ARCHIVE.format(actual_pocket))
+    elif source == 'proposed':
+        release = lsb_release()['DISTRIB_CODENAME']
+        with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
+            apt.write(PROPOSED_POCKET.format(release))
+    if key:
+        subprocess.check_call(['apt-key', 'import', key])
+
+
+class SourceConfigError(Exception):
+    pass
+
+
+def configure_sources(update=False,
+                      sources_var='install_sources',
+                      keys_var='install_keys'):
+    """
+    Configure multiple sources from charm configuration
+
+    Example config:
+        install_sources:
+          - "ppa:foo"
+          - "http://example.com/repo precise main"
+        install_keys:
+          - null
+          - "a1b2c3d4"
+
+    Note that 'null' (a.k.a. None) should not be quoted.
+    """
+    sources = safe_load(config(sources_var))
+    keys = config(keys_var)
+    if keys is not None:
+        keys = safe_load(keys)
+    if isinstance(sources, basestring) and (
+            keys is None or isinstance(keys, basestring)):
+        add_source(sources, keys)
+    else:
+        if not len(sources) == len(keys):
+            msg = 'Install sources and keys lists are different lengths'
+            raise SourceConfigError(msg)
+        for src_num in range(len(sources)):
+            add_source(sources[src_num], keys[src_num])
+    if update:
+        apt_update(fatal=True)
+
+# The order of this list is very important. Handlers should be listed in from
+# least- to most-specific URL matching.
+FETCH_HANDLERS = (
+    'charmhelpers.fetch.archiveurl.ArchiveUrlFetchHandler',
+    'charmhelpers.fetch.bzrurl.BzrUrlFetchHandler',
+)
+
+
+class UnhandledSource(Exception):
+    pass
+
+
+def install_remote(source):
+    """
+    Install a file tree from a remote source
+
+    The specified source should be a url of the form:
+        scheme://[host]/path[#[option=value][&...]]
+
+    Schemes supported are based on this modules submodules
+    Options supported are submodule-specific"""
+    # We ONLY check for True here because can_handle may return a string
+    # explaining why it can't handle a given source.
+    handlers = [h for h in plugins() if h.can_handle(source) is True]
+    installed_to = None
+    for handler in handlers:
+        try:
+            installed_to = handler.install(source)
+        except UnhandledSource:
+            pass
+    if not installed_to:
+        raise UnhandledSource("No handler found for source {}".format(source))
+    return installed_to
+
+
+def install_from_config(config_var_name):
+    charm_config = config()
+    source = charm_config[config_var_name]
+    return install_remote(source)
+
+
+class BaseFetchHandler(object):
+
+    """Base class for FetchHandler implementations in fetch plugins"""
+
+    def can_handle(self, source):
+        """Returns True if the source can be handled. Otherwise returns
+        a string explaining why it cannot"""
+        return "Wrong source type"
+
+    def install(self, source):
+        """Try to download and unpack the source. Return the path to the
+        unpacked files or raise UnhandledSource."""
+        raise UnhandledSource("Wrong source type {}".format(source))
+
+    def parse_url(self, url):
+        return urlparse(url)
+
+    def base_url(self, url):
+        """Return url without querystring or fragment"""
+        parts = list(self.parse_url(url))
+        parts[4:] = ['' for i in parts[4:]]
+        return urlunparse(parts)
+
+
+def plugins(fetch_handlers=None):
+    if not fetch_handlers:
+        fetch_handlers = FETCH_HANDLERS
+    plugin_list = []
+    for handler_name in fetch_handlers:
+        package, classname = handler_name.rsplit('.', 1)
+        try:
+            handler_class = getattr(
+                importlib.import_module(package),
+                classname)
+            plugin_list.append(handler_class())
+        except (ImportError, AttributeError):
+            # Skip missing plugins so that they can be ommitted from
+            # installation if desired
+            log("FetchHandler {} not found, skipping plugin".format(
+                handler_name))
+    return plugin_list

=== added file 'hooks/charmhelpers/fetch/archiveurl.py'
--- hooks/charmhelpers/fetch/archiveurl.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/archiveurl.py	2014-03-04 16:16:33 +0000
@@ -0,0 +1,48 @@
+import os
+import urllib2
+from charmhelpers.fetch import (
+    BaseFetchHandler,
+    UnhandledSource
+)
+from charmhelpers.payload.archive import (
+    get_archive_handler,
+    extract,
+)
+from charmhelpers.core.host import mkdir
+
+
+class ArchiveUrlFetchHandler(BaseFetchHandler):
+    """Handler for archives via generic URLs"""
+    def can_handle(self, source):
+        url_parts = self.parse_url(source)
+        if url_parts.scheme not in ('http', 'https', 'ftp', 'file'):
+            return "Wrong source type"
+        if get_archive_handler(self.base_url(source)):
+            return True
+        return False
+
+    def download(self, source, dest):
+        # propogate all exceptions
+        # URLError, OSError, etc
+        response = urllib2.urlopen(source)
+        try:
+            with open(dest, 'w') as dest_file:
+                dest_file.write(response.read())
+        except Exception as e:
+            if os.path.isfile(dest):
+                os.unlink(dest)
+            raise e
+
+    def install(self, source):
+        url_parts = self.parse_url(source)
+        dest_dir = os.path.join(os.environ.get('CHARM_DIR'), 'fetched')
+        if not os.path.exists(dest_dir):
+            mkdir(dest_dir, perms=0755)
+        dld_file = os.path.join(dest_dir, os.path.basename(url_parts.path))
+        try:
+            self.download(source, dld_file)
+        except urllib2.URLError as e:
+            raise UnhandledSource(e.reason)
+        except OSError as e:
+            raise UnhandledSource(e.strerror)
+        return extract(dld_file)

=== added file 'hooks/charmhelpers/fetch/bzrurl.py'
--- hooks/charmhelpers/fetch/bzrurl.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/bzrurl.py	2014-03-04 16:16:33 +0000
@@ -0,0 +1,49 @@
+import os
+from charmhelpers.fetch import (
+    BaseFetchHandler,
+    UnhandledSource
+)
+from charmhelpers.core.host import mkdir
+
+try:
+    from bzrlib.branch import Branch
+except ImportError:
+    from charmhelpers.fetch import apt_install
+    apt_install("python-bzrlib")
+    from bzrlib.branch import Branch
+
+
+class BzrUrlFetchHandler(BaseFetchHandler):
+    """Handler for bazaar branches via generic and lp URLs"""
+    def can_handle(self, source):
+        url_parts = self.parse_url(source)
+        if url_parts.scheme not in ('bzr+ssh', 'lp'):
+            return False
+        else:
+            return True
+
+    def branch(self, source, dest):
+        url_parts = self.parse_url(source)
+        # If we use lp:branchname scheme we need to load plugins
+        if not self.can_handle(source):
+            raise UnhandledSource("Cannot handle {}".format(source))
+        if url_parts.scheme == "lp":
+            from bzrlib.plugin import load_plugins
+            load_plugins()
+        try:
+            remote_branch = Branch.open(source)
+            remote_branch.bzrdir.sprout(dest).open_branch()
+        except Exception as e:
+            raise e
+
+    def install(self, source):
+        url_parts = self.parse_url(source)
+        branch_name = url_parts.path.strip("/").split("/")[-1]
+        dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", branch_name)
+        if not os.path.exists(dest_dir):
+            mkdir(dest_dir, perms=0755)
+        try:
+            self.branch(source, dest_dir)
+        except OSError as e:
+            raise UnhandledSource(e.strerror)
+        return dest_dir

=== added symlink 'hooks/config-changed'
=== target is u'hooks.py'
=== modified file 'hooks/hooks.py'
--- hooks/hooks.py	2013-06-05 12:55:47 +0000
+++ hooks/hooks.py	2014-03-04 16:16:33 +0000
@@ -1,313 +1,24 @@
 #!/usr/bin/env python
 # vim: et ai ts=4 sw=4:
 
-import json
 import os
-import re
-import subprocess
 import sys
-import time
-from pwd import getpwnam
-from grp import getgrnam
-
-CHARM_PACKAGES = ["gunicorn"]
-
-INJECTED_WARNING = """
-#------------------------------------------------------------------------------
-# The following is the import code for the settings directory injected by Juju
-#------------------------------------------------------------------------------
-"""
-
+from multiprocessing import cpu_count
+import shlex
+
+from charmhelpers.core import hookenv
+from charmhelpers.core import host
+from charmhelpers import fetch
+
+hooks = hookenv.Hooks()
+
+
+CHARM_PACKAGES = ["gunicorn", "python-jinja2"]
 
 ###############################################################################
 # Supporting functions
 ###############################################################################
-MSG_CRITICAL = "CRITICAL"
-MSG_DEBUG = "DEBUG"
-MSG_INFO = "INFO"
-MSG_ERROR = "ERROR"
-MSG_WARNING = "WARNING"
-
-
-def juju_log(level, msg):
-    subprocess.call(['juju-log', '-l', level, msg])
-
-#------------------------------------------------------------------------------
-# run: Run a command, return the output
-#------------------------------------------------------------------------------
-def run(command, exit_on_error=True, cwd=None):
-    try:
-        juju_log(MSG_DEBUG, command)
-        return subprocess.check_output(
-            command, stderr=subprocess.STDOUT, shell=True, cwd=cwd)
-    except subprocess.CalledProcessError, e:
-        juju_log(MSG_ERROR, "status=%d, output=%s" % (e.returncode, e.output))
-        if exit_on_error:
-            sys.exit(e.returncode)
-        else:
-            raise
-
-
-#------------------------------------------------------------------------------
-# install_file: install a file resource. overwites existing files.
-#------------------------------------------------------------------------------
-def install_file(contents, dest, owner="root", group="root", mode=0600):
-    uid = getpwnam(owner)[2]
-    gid = getgrnam(group)[2]
-    dest_fd = os.open(dest, os.O_WRONLY | os.O_TRUNC | os.O_CREAT, mode)
-    os.fchown(dest_fd, uid, gid)
-    with os.fdopen(dest_fd, 'w') as destfile:
-        destfile.write(str(contents))
-
-
-#------------------------------------------------------------------------------
-# install_dir: create a directory
-#------------------------------------------------------------------------------
-def install_dir(dirname, owner="root", group="root", mode=0700):
-    command = \
-    '/usr/bin/install -o {} -g {} -m {} -d {}'.format(owner, group, oct(mode),
-        dirname)
-    return run(command)
-
-#------------------------------------------------------------------------------
-# config_get:  Returns a dictionary containing all of the config information
-#              Optional parameter: scope
-#              scope: limits the scope of the returned configuration to the
-#                     desired config item.
-#------------------------------------------------------------------------------
-def config_get(scope=None):
-    try:
-        config_cmd_line = ['config-get']
-        if scope is not None:
-            config_cmd_line.append(scope)
-        config_cmd_line.append('--format=json')
-        config_data = json.loads(subprocess.check_output(config_cmd_line))
-    except:
-        config_data = None
-    finally:
-        return(config_data)
-
-
-#------------------------------------------------------------------------------
-# relation_json:  Returns json-formatted relation data
-#                Optional parameters: scope, relation_id
-#                scope:        limits the scope of the returned data to the
-#                              desired item.
-#                unit_name:    limits the data ( and optionally the scope )
-#                              to the specified unit
-#                relation_id:  specify relation id for out of context usage.
-#------------------------------------------------------------------------------
-def relation_json(scope=None, unit_name=None, relation_id=None):
-    command = ['relation-get', '--format=json']
-    if relation_id is not None:
-        command.extend(('-r', relation_id))
-    if scope is not None:
-        command.append(scope)
-    else:
-        command.append('-')
-    if unit_name is not None:
-        command.append(unit_name)
-    output = subprocess.check_output(command, stderr=subprocess.STDOUT)
-    return output or None
-
-
-#------------------------------------------------------------------------------
-# relation_get:  Returns a dictionary containing the relation information
-#                Optional parameters: scope, relation_id
-#                scope:        limits the scope of the returned data to the
-#                              desired item.
-#                unit_name:    limits the data ( and optionally the scope )
-#                              to the specified unit
-#------------------------------------------------------------------------------
-def relation_get(scope=None, unit_name=None, relation_id=None):
-    j = relation_json(scope, unit_name, relation_id)
-    if j:
-        return json.loads(j)
-    else:
-        return None
-
-
-def relation_set(keyvalues, relation_id=None):
-    args = []
-    if relation_id:
-        args.extend(['-r', relation_id])
-    args.extend(["{}='{}'".format(k, v or '') for k, v in keyvalues.items()])
-    run("relation-set {}".format(' '.join(args)))
-
-    ## Posting json to relation-set doesn't seem to work as documented?
-    ## Bug #1116179
-    ##
-    ## cmd = ['relation-set']
-    ## if relation_id:
-    ##     cmd.extend(['-r', relation_id])
-    ## p = Popen(
-    ##     cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
-    ##     stderr=subprocess.PIPE)
-    ## (out, err) = p.communicate(json.dumps(keyvalues))
-    ## if p.returncode:
-    ##     juju_log(MSG_ERROR, err)
-    ##     sys.exit(1)
-    ## juju_log(MSG_DEBUG, "relation-set {}".format(repr(keyvalues)))
-
-
-def relation_list(relation_id=None):
-    """Return the list of units participating in the relation."""
-    if relation_id is None:
-        relation_id = os.environ['JUJU_RELATION_ID']
-    cmd = ['relation-list', '--format=json', '-r', relation_id]
-    json_units = subprocess.check_output(cmd).strip()
-    if json_units:
-        return json.loads(subprocess.check_output(cmd))
-    return []
-
-
-#------------------------------------------------------------------------------
-# relation_ids:  Returns a list of relation ids
-#                optional parameters: relation_type
-#                relation_type: return relations only of this type
-#------------------------------------------------------------------------------
-def relation_ids(relation_types=('db',)):
-    # accept strings or iterators
-    if isinstance(relation_types, basestring):
-        reltypes = [relation_types, ]
-    else:
-        reltypes = relation_types
-    relids = []
-    for reltype in reltypes:
-        relid_cmd_line = ['relation-ids', '--format=json', reltype]
-        json_relids = subprocess.check_output(relid_cmd_line).strip()
-        if json_relids:
-            relids.extend(json.loads(json_relids))
-    return relids
-
-
-#------------------------------------------------------------------------------
-# relation_get_all:  Returns a dictionary containing the relation information
-#                optional parameters: relation_type
-#                relation_type: limits the scope of the returned data to the
-#                               desired item.
-#------------------------------------------------------------------------------
-def relation_get_all(*args, **kwargs):
-    relation_data = []
-    relids = relation_ids(*args, **kwargs)
-    for relid in relids:
-        units_cmd_line = ['relation-list', '--format=json', '-r', relid]
-        json_units = subprocess.check_output(units_cmd_line).strip()
-        if json_units:
-            for unit in json.loads(json_units):
-                unit_data = \
-                    json.loads(relation_json(relation_id=relid,
-                        unit_name=unit))
-                for key in unit_data:
-                    if key.endswith('-list'):
-                        unit_data[key] = unit_data[key].split()
-                unit_data['relation-id'] = relid
-                unit_data['unit'] = unit
-                relation_data.append(unit_data)
-    return relation_data
-
-def apt_get_update():
-    cmd_line = ['apt-get', 'update']
-    return(subprocess.call(cmd_line))
-
-
-#------------------------------------------------------------------------------
-# apt_get_install( packages ):  Installs package(s)
-#------------------------------------------------------------------------------
-def apt_get_install(packages=None):
-    if packages is None:
-        return(False)
-    cmd_line = ['apt-get', '-y', 'install', '-qq']
-    if isinstance(packages, list):
-        cmd_line.extend(packages)
-    else:
-        cmd_line.append(packages)
-    return(subprocess.call(cmd_line))
-
-
-#------------------------------------------------------------------------------
-# pip_install( package ):  Installs a python package
-#------------------------------------------------------------------------------
-def pip_install(packages=None, upgrade=False):
-    cmd_line = ['pip', 'install']
-    if packages is None:
-        return(False)
-    if upgrade:
-        cmd_line.append('-u')
-    if packages.startswith('svn+') or packages.startswith('git+') or \
-       packages.startswith('hg+') or packages.startswith('bzr+'):
-        cmd_line.append('-e')
-    cmd_line.append(packages)
-    return run(cmd_line)
-
-#------------------------------------------------------------------------------
-# pip_install_req( path ):  Installs a requirements file
-#------------------------------------------------------------------------------
-def pip_install_req(path=None, upgrade=False):
-    cmd_line = ['pip', 'install']
-    if path is None:
-        return(False)
-    if upgrade:
-        cmd_line.append('-u')
-    cmd_line.append('-r')
-    cmd_line.append(path)
-    cwd = os.path.dirname(path)
-    return run(cmd_line)
-
-#------------------------------------------------------------------------------
-# open_port:  Convenience function to open a port in juju to
-#             expose a service
-#------------------------------------------------------------------------------
-def open_port(port=None, protocol="TCP"):
-    if port is None:
-        return(None)
-    return(subprocess.call(['open-port', "%d/%s" %
-        (int(port), protocol)]))
-
-
-#------------------------------------------------------------------------------
-# close_port:  Convenience function to close a port in juju to
-#              unexpose a service
-#------------------------------------------------------------------------------
-def close_port(port=None, protocol="TCP"):
-    if port is None:
-        return(None)
-    return(subprocess.call(['close-port', "%d/%s" %
-        (int(port), protocol)]))
-
-
-#------------------------------------------------------------------------------
-# update_service_ports:  Convenience function that evaluate the old and new
-#                        service ports to decide which ports need to be
-#                        opened and which to close
-#------------------------------------------------------------------------------
-def update_service_port(old_service_port=None, new_service_port=None):
-    if old_service_port is None or new_service_port is None:
-        return(None)
-    if new_service_port != old_service_port:
-        close_port(old_service_port)
-        open_port(new_service_port)
-
-#
-# Utils
-#
-
-def install_or_append(contents, dest, owner="root", group="root", mode=0600):
-    if os.path.exists(dest):
-        uid = getpwnam(owner)[2]
-        gid = getgrnam(group)[2]
-        dest_fd = os.open(dest, os.O_APPEND, mode)
-        os.fchown(dest_fd, uid, gid)
-        with os.fdopen(dest_fd, 'a') as destfile:
-            destfile.write(str(contents))
-    else:
-        install_file(contents, dest, owner, group, mode)
-
-def token_sql_safe(value):
-    # Only allow alphanumeric + underscore in database identifiers
-    if re.search('[^A-Za-z0-9_]', value):
-        return False
-    return True
+
 
 def sanitize(s):
     s = s.replace(':', '_')
@@ -317,175 +28,104 @@
     s = s.replace("'", '_')
     return s
 
-def user_name(relid, remote_unit, admin=False, schema=False):
-    components = [sanitize(relid), sanitize(remote_unit)]
-    if admin:
-        components.append("admin")
-    elif schema:
-        components.append("schema")
-    return "_".join(components)
-
-def get_relation_host():
-    remote_host = run("relation-get ip")
-    if not remote_host:
-        # remote unit $JUJU_REMOTE_UNIT uses deprecated 'ip=' component of
-        # interface.
-        remote_host = run("relation-get private-address")
-    return remote_host
-
-
-def get_unit_host():
-    this_host = run("unit-get private-address")
-    return this_host.strip()
+
+def sanitized_service_name():
+    return sanitize(hookenv.remote_unit().split('/')[0])
+
+
+def upstart_conf_path(name):
+    return '/etc/init/%s.conf' % name
+
 
 def process_template(template_name, template_vars, destination):
-    # --- exported service configuration file
+    # deferred import so install hook can run to install jinja2
     from jinja2 import Environment, FileSystemLoader
-    template_env = Environment(
-        loader=FileSystemLoader(os.path.join(os.environ['CHARM_DIR'],
-        'templates')))
+    path = os.path.join(os.environ['CHARM_DIR'], 'templates')
+    template_env = Environment(loader=FileSystemLoader(path))
 
     template = \
         template_env.get_template(template_name).render(template_vars)
 
     with open(destination, 'w') as inject_file:
         inject_file.write(str(template))
-
-def append_template(template_name, template_vars, path, try_append=False):
-
-    # --- exported service configuration file
-    from jinja2 import Environment, FileSystemLoader
-    template_env = Environment(
-        loader=FileSystemLoader(os.path.join(os.environ['CHARM_DIR'],
-        'templates')))
-
-    template = \
-        template_env.get_template(template_name).render(template_vars)
-
-    append = False
-    if os.path.exists(path):
-        with open(path, 'r') as inject_file:
-            if not str(template) in inject_file:
-                append = True
-    else:       
-        append = True
-        
-    if append == True:
-        with open(path, 'a') as inject_file:
-            inject_file.write(INJECTED_WARNING)
-            inject_file.write(str(template))
-
+    hookenv.log('written gunicorn upstart config to %s' % destination)
 
 
 ###############################################################################
 # Hook functions
 ###############################################################################
+
+
[e-mail]('install', 'upgrade-charm')
 def install():
-
-    for retry in xrange(0,24):
-        if apt_get_install(CHARM_PACKAGES):
-            time.sleep(10)
-        else:
-            break
-
-def upgrade():
-
-    apt_get_update()
-    for retry in xrange(0,24):
-        if apt_get_install(CHARM_PACKAGES):
-            time.sleep(10)
-        else:
-            break
-
-def wsgi_file_relation_joined_changed():
-    wsgi_config = config_data
-    relation_id = os.environ['JUJU_RELATION_ID']
-    juju_log(MSG_INFO, "JUJU_RELATION_ID: %s".format(relation_id))
-
-    remote_unit_name = sanitize(os.environ['JUJU_REMOTE_UNIT'].split('/')[0])
-    juju_log(MSG_INFO, "JUJU_REMOTE_UNIT: %s".format(remote_unit_name))
-    wsgi_config['unit_name'] = remote_unit_name
-
-    project_conf = '/etc/init/%s.conf' % remote_unit_name
-
-    working_dir = relation_get('working_dir')
+    fetch.apt_update()
+    fetch.apt_install(CHARM_PACKAGES)
+
+
[e-mail](
+    "config-changed",
+    "wsgi-file-relation-joined",
+    "wsgi-file-relation-changed")
+def configure_gunicorn():
+    wsgi_config = hookenv.config()
+
+    relations = hookenv.relations_of_type('wsgi-file')
+    if not relations:
+        hookenv.log("No wsgi-file relation, nothing to do")
+        return
+
+    relation_data = relations[0]
+
+    service_name = sanitized_service_name()
+    wsgi_config['unit_name'] = service_name
+
+    project_conf = upstart_conf_path(service_name)
+
+    working_dir = relation_data.get('working_dir', None)
     if not working_dir:
         return
 
     wsgi_config['working_dir'] = working_dir
-    wsgi_config['project_name'] = remote_unit_name
+    wsgi_config['project_name'] = service_name
 
-    for v in wsgi_config.keys():
-        if v.startswith('wsgi_') or v in ['python_path', 'listen_ip', 'port']:
-            upstream_value = relation_get(v)
-            if upstream_value:
-                wsgi_config[v] = upstream_value
+    # any valid config item can be overidden by a relation item
+    for key, relation_value in relation_data.items():
+        if key in wsgi_config:
+            wsgi_config[key] = relation_value
 
     if wsgi_config['wsgi_worker_class'] == 'eventlet':
-        apt_get_install('python-eventlet')
-    elif  wsgi_config['wsgi_worker_class'] == 'gevent':
-        apt_get_install('python-gevent')
+        fetch.apt_install('python-eventlet')
+    elif wsgi_config['wsgi_worker_class'] == 'gevent':
+        fetch.apt_install('python-gevent')
     elif wsgi_config['wsgi_worker_class'] == 'tornado':
-        apt_get_install('python-tornado')
+        fetch.apt_install('python-tornado')
 
     if wsgi_config['wsgi_workers'] == 0:
-        res = run('python -c "import multiprocessing ; print(multiprocessing.cpu_count())"')
-        wsgi_config['wsgi_workers'] = int(res) + 1
-
-    if wsgi_config['wsgi_access_logfile']:
-        wsgi_config['wsgi_extra'] = " ".join([
-            wsgi_config['wsgi_extra'],
-            '--access-logformat=%s' % wsgi_config['wsgi_access_logfile'],
-            '--access-logformat="%s"' % wsgi_config['wsgi_access_logformat']
-            ])
-
-    wsgi_config['wsgi_wsgi_file'] = wsgi_config['wsgi_wsgi_file']
+        wsgi_config['wsgi_workers'] = cpu_count() + 1
+
+    env_extra = wsgi_config.get('env_extra', '')
+    wsgi_config['env_extra'] = [
+        v.split('=') for v in shlex.split(env_extra) if '=' in v
+    ]
 
     process_template('upstart.tmpl', wsgi_config, project_conf)
 
-
-    # We need this because when the contained charm configuration or code changed
-    # Gunicorn needs to restart to run the new code.
-    run("service %s restart || service %s start" % (remote_unit_name, remote_unit_name))
-
-
+    # We need this because when the contained charm configuration or code
+    # changed Gunicorn needs to restart to run the new code.
+    host.service_restart(service_name)
+
+
[e-mail]("wsgi_file_relation_broken")
 def wsgi_file_relation_broken():
-    remote_unit_name = sanitize(os.environ['JUJU_REMOTE_UNIT'].split('/')[0])
-
-    run('service %s stop' % remote_unit_name)
-    run('rm /etc/init/%s.conf' % remote_unit_name)
-
-
-###############################################################################
-# Global variables
-###############################################################################
-config_data = config_get()
-juju_log(MSG_DEBUG, "got config: %s" % str(config_data))
-
-unit_name = os.environ['JUJU_UNIT_NAME'].split('/')[0]
-
-hook_name = os.path.basename(sys.argv[0])
-
-###############################################################################
-# Main section
-###############################################################################
-def main():
-    juju_log(MSG_INFO, "Running {} hook".format(hook_name))
-    if hook_name == "install":
-        install()
-
-    elif hook_name == "upgrade-charm":
-        upgrade()
-
-    elif hook_name in ["wsgi-file-relation-joined", "wsgi-file-relation-changed"]:
-        wsgi_file_relation_joined_changed()
-
-    elif hook_name == "wsgi-file-relation-broken":
-        wsgi_file_relation_broken()
-
-    else:
-        print "Unknown hook {}".format(hook_name)
-        raise SystemExit(1)
-
-if __name__ == '__main__':
-    raise SystemExit(main())
+    service_name = sanitized_service_name()
+    host.service_stop(service_name)
+    try:
+        os.remove(upstart_conf_path(service_name))
+    except OSError as exc:
+        if exc.errno != 2:
+            raise
+    hookenv.log("removed gunicorn upstart config")
+
+
+if __name__ == "__main__":
+    hooks.execute(sys.argv)

=== added directory 'hooks/tests'
=== added file 'hooks/tests/test_hooks.py'
--- hooks/tests/test_hooks.py	1970-01-01 00:00:00 +0000
+++ hooks/tests/test_hooks.py	2014-03-04 16:16:33 +0000
@@ -0,0 +1,157 @@
+from unittest import TestCase
+from mock import patch
+
+import yaml
+
+import hooks
+
+
+def load_config_defaults():
+    with open("config.yaml") as conf:
+        config_schema = yaml.safe_load(conf)
+    defaults = {}
+    for key, value in config_schema['options'].items():
+        defaults[key] = value['default']
+    return defaults
+
+DEFAULTS = load_config_defaults()
+
+
+class HookTestCase(TestCase):
+    maxDiff = None
+
+    SERVICE_NAME = 'some_juju_service'
+    WORKING_DIR = '/some_path'
+
+    _object = object()
+    mocks = {}
+
+    def apply_patch(self, name, value=_object, return_value=_object):
+        if value is not self._object:
+            patcher = patch(name, value)
+        else:
+            patcher = patch(name)
+
+        mock_obj = patcher.start()
+        self.addCleanup(patcher.stop)
+
+        if value is self._object and return_value is not self._object:
+            mock_obj.return_value = return_value
+
+        self.mocks[name] = mock_obj
+        return mock_obj
+
+    def setUp(self):
+        super(HookTestCase, self).setUp()
+        # There's quite a bit of mocking here, due to the large amounts of
+        # environment context to juju hooks
+
+        self.config = DEFAULTS.copy()
+        self.relation_data = {'working_dir': self.WORKING_DIR}
+
+        # intercept all charmsupport usage
+        self.hookenv = self.apply_patch('hooks.hookenv')
+        self.fetch = self.apply_patch('hooks.fetch')
+        self.host = self.apply_patch('hooks.host')
+
+        self.hookenv.config.return_value = self.config
+        self.hookenv.relations_of_type.return_value = [self.relation_data]
+
+        # mocking utilities that touch the host/environment
+        self.process_template = self.apply_patch('hooks.process_template')
+        self.apply_patch(
+            'hooks.sanitized_service_name', return_value=self.SERVICE_NAME)
+        self.apply_patch('hooks.cpu_count', return_value=1)
+
+    def assert_wsgi_config_applied(self, expected):
+        tmpl, config, path = self.process_template.call_args[0]
+        self.assertEqual(tmpl, 'upstart.tmpl')
+        self.assertEqual(path, '/etc/init/%s.conf' % self.SERVICE_NAME)
+        self.assertEqual(config, expected)
+        self.host.service_restart.assert_called_once_with(self.SERVICE_NAME)
+
+    def get_default_context(self):
+        expected = DEFAULTS.copy()
+        expected['unit_name'] = self.SERVICE_NAME
+        expected['working_dir'] = self.WORKING_DIR
+        expected['project_name'] = self.SERVICE_NAME
+        expected['wsgi_workers'] = 2
+        expected['env_extra'] = []
+        fmt = expected['wsgi_access_logformat'].replace('"', '\\"')
+        expected['wsgi_access_logformat'] = fmt
+        return expected
+
+    def test_python_install_hook(self):
+        hooks.install()
+        self.assertTrue(self.fetch.apt_update.called)
+        self.fetch.apt_install.assert_called_once_with(
+            ['gunicorn', 'python-jinja2'])
+
+    def test_default_configure_gunicorn(self):
+        hooks.configure_gunicorn()
+        expected = self.get_default_context()
+        self.assert_wsgi_config_applied(expected)
+
+    def test_configure_gunicorn_no_working_dir(self):
+        del self.relation_data['working_dir']
+        hooks.configure_gunicorn()
+        self.assertFalse(self.process_template.called)
+        self.assertFalse(self.host.service_restart.called)
+
+    def test_configure_gunicorn_relation_data(self):
+        self.relation_data['port'] = 9999
+        self.relation_data['wsgi_workers'] = 1
+        self.relation_data['unknown'] = 'value'
+
+        hooks.configure_gunicorn()
+
+        self.assertFalse(self.fetch.apt_install.called)
+
+        expected = self.get_default_context()
+        expected['wsgi_workers'] = 1
+        expected['port'] = 9999
+
+        self.assert_wsgi_config_applied(expected)
+
+    def test_env_extra_parsing(self):
+        self.relation_data['env_extra'] = 'A=1 B="2" C="3 4" D= E'
+
+        hooks.configure_gunicorn()
+
+        expected = self.get_default_context()
+        expected['env_extra'] = [
+            ['A', '1'],
+            ['B', '2'],
+            ['C', '3 4'],
+            ['D', ''],
+            # no E
+        ]
+
+        self.assert_wsgi_config_applied(expected)
+
+    def do_worker_class(self, worker_class):
+        self.relation_data['wsgi_worker_class'] = worker_class
+        hooks.configure_gunicorn()
+        self.fetch.apt_install.assert_called_once_with(
+            'python-%s' % worker_class)
+        expected = self.get_default_context()
+        expected['wsgi_worker_class'] = worker_class
+        self.assert_wsgi_config_applied(expected)
+
+    def test_configure_worker_class_eventlet(self):
+        self.do_worker_class('eventlet')
+
+    def test_configure_worker_class_tornado(self):
+        self.do_worker_class('tornado')
+
+    def test_configure_worker_class_gevent(self):
+        self.do_worker_class('gevent')
+
+
+
+    @patch('hooks.os.remove')
+    def test_wsgi_file_relation_broken(self, remove):
+        hooks.wsgi_file_relation_broken()
+        self.host.service_stop.assert_called_once_with(self.SERVICE_NAME)
+        remove.assert_called_once_with(
+            '/etc/init/%s.conf' % self.SERVICE_NAME)

=== added file 'hooks/tests/test_template.py'
--- hooks/tests/test_template.py	1970-01-01 00:00:00 +0000
+++ hooks/tests/test_template.py	2014-03-04 16:16:33 +0000
@@ -0,0 +1,125 @@
+import os
+from unittest import TestCase
+from mock import patch, MagicMock
+
+import hooks
+
+EXPECTED = """
+#--------------------------------------------------------------
+# This file is managed by Juju; ANY CHANGES WILL BE OVERWRITTEN
+#--------------------------------------------------------------
+
+description "Gunicorn daemon for the PROJECT_NAME project"
+
+start on (local-filesystems and net-device-up IFACE=eth0)
+stop on runlevel [!12345]
+
+# If the process quits unexpectadly trigger a respawn
+respawn
+respawn limit 10 5
+
+setuid WSGI_USER
+setgid WSGI_GROUP
+chdir WORKING_DIR
+
+# This line can be removed and replace with the --pythonpath PYTHON_PATH \\
+# option with Gunicorn>1.17
+env PYTHONPATH=PYTHON_PATH
+env A="1"
+env B="1 2"
+
+
+exec gunicorn \\
+    --name=PROJECT_NAME \\
+    --workers=WSGI_WORKERS \\
+    --worker-class=WSGI_WORKER_CLASS \\
+    --worker-connections=WSGI_WORKER_CONNECTIONS \\
+    --max-requests=WSGI_MAX_REQUESTS \\
+    --backlog=WSGI_BACKLOG \\
+    --timeout=WSGI_TIMEOUT \\
+    --keep-alive=WSGI_KEEP_ALIVE \\
+    --umask=WSGI_UMASK \\
+    --bind=LISTEN_IP:PORT \\
+    --log-file=WSGI_LOG_FILE \\
+    --log-level=WSGI_LOG_LEVEL \\
+    --access-logfile=WSGI_ACCESS_LOGFILE \\
+    --access-logformat=WSGI_ACCESS_LOGFORMAT \\
+    WSGI_EXTRA \\
+    WSGI_WSGI_FILE
+""".strip()
+
+
+class TemplateTestCase(TestCase):
+    maxDiff = None
+
+    def setUp(self):
+        super(TemplateTestCase, self).setUp()
+        patch_open = patch('hooks.open', create=True)
+        self.open = patch_open.start()
+        self.addCleanup(patch_open.stop)
+
+        self.open.return_value = MagicMock(spec=file)
+        self.file = self.open.return_value.__enter__.return_value
+
+        patch_environ = patch.dict(os.environ, CHARM_DIR='.')
+        patch_environ.start()
+        self.addCleanup(patch_environ.stop)
+
+        patch_hookenv = patch('hooks.hookenv')
+        patch_hookenv.start()
+        self.addCleanup(patch_hookenv.stop)
+
+    def get_test_context(self):
+        keys = [
+            'project_name',
+            'wsgi_user',
+            'wsgi_group',
+            'working_dir',
+            'python_path',
+            'wsgi_workers',
+            'wsgi_worker_class',
+            'wsgi_worker_connections',
+            'wsgi_max_requests',
+            'wsgi_backlog',
+            'wsgi_timeout',
+            'wsgi_keep_alive',
+            'wsgi_umask',
+            'wsgi_log_file',
+            'wsgi_log_level',
+            'wsgi_access_logfile',
+            'wsgi_access_logformat',
+            'listen_ip',
+            'port',
+            'wsgi_extra',
+            'wsgi_wsgi_file',
+        ]
+        ctx = dict((k, k.upper()) for k in keys)
+        ctx['env_extra'] = [["A", "1"], ["B", "1 2"]]
+        return ctx
+
+    def test_template(self):
+
+        ctx = self.get_test_context()
+
+        hooks.process_template('upstart.tmpl', ctx, 'path')
+        output = self.file.write.call_args[0][0]
+
+        self.assertMultiLineEqual(EXPECTED, output)
+
+    def test_no_access_logfile(self):
+        ctx = self.get_test_context()
+        ctx['wsgi_access_logfile'] = ""
+
+        hooks.process_template('upstart.tmpl', ctx, 'path')
+        output = self.file.write.call_args[0][0]
+
+        self.assertNotIn('--access-logfile', output)
+
+    def test_no_access_logformat(self):
+        ctx = self.get_test_context()
+        ctx['wsgi_access_logformat'] = ""
+
+        hooks.process_template('upstart.tmpl', ctx, 'path')
+        output = self.file.write.call_args[0][0]
+
+        self.assertNotIn('--access-logformat', output)

=== modified file 'metadata.yaml'
--- metadata.yaml	2013-06-04 18:36:25 +0000
+++ metadata.yaml	2014-03-04 16:16:33 +0000
@@ -7,6 +7,10 @@
   pre-fork worker model ported from Ruby's Unicorn project. The Gunicorn server
   is broadly compatible with various web frameworks, simply implemented, light on
   server resources, and fairly speedy.
+
+  The providers of the wsgi relation must provide working_dir, plus optionally
+  any of the charm's config directives, which will override the current config
+  for the charm.
 subordinate: true
 requires:
     wsgi-file:

=== modified file 'templates/upstart.tmpl'
--- templates/upstart.tmpl	2013-05-31 15:41:27 +0000
+++ templates/upstart.tmpl	2014-03-04 16:16:33 +0000
@@ -2,13 +2,14 @@
 # This file is managed by Juju; ANY CHANGES WILL BE OVERWRITTEN
 #--------------------------------------------------------------
 
-description "Gunicorn daemon for the {{ project_name }} project"
+description "Gunicorn daemon for the {{ project_name }} project"
 
 start on (local-filesystems and net-device-up IFACE=eth0)
 stop on runlevel [!12345]
 
 # If the process quits unexpectadly trigger a respawn
 respawn
+respawn limit 10 5
 
 setuid {{ wsgi_user }}
 setgid {{ wsgi_group }}
@@ -17,6 +18,9 @@
 # This line can be removed and replace with the --pythonpath {{ python_path }} \
 # option with Gunicorn>1.17
 env PYTHONPATH={{ python_path }}
+{% for name, value in env_extra -%}
+env {{ name }}="{{ value }}"
+{% endfor %}
 
 exec gunicorn \
     --name={{ project_name }} \
@@ -31,4 +35,11 @@
     --bind={{ listen_ip }}:{{ port }} \
     --log-file={{ wsgi_log_file }} \
     --log-level={{ wsgi_log_level }} \
-    {{ wsgi_extra }} {{ wsgi_wsgi_file }}
+    {%- if wsgi_access_logfile %}
+    --access-logfile={{ wsgi_access_logfile }} \
+    {%- endif %}
+    {%- if wsgi_access_logformat %}
+    --access-logformat={{ wsgi_access_logformat }} \
+    {%- endif %}
+    {{ wsgi_extra }} \
+    {{ wsgi_wsgi_file }}

=== added file 'test_requirments.txt'
--- test_requirments.txt	1970-01-01 00:00:00 +0000
+++ test_requirments.txt	2014-03-04 16:16:33 +0000
@@ -0,0 +1,6 @@
+mock
+nose
+pyaml
+python-apt
+jinja2
+