Merge lp:~andreserl/charms/precise/glance/port into lp:~openstack-charmers/charms/precise/glance/python-redux

[Andres Rodriguez]

[Adam Gandelman (gandelman-a)]

Generally looks good. A couple of things:

- Can you sync the latest charm-helpers? There has been a lot added, and we've already rearranged the layout helpers in contrib.cluster and contrib.openstack. Also, with the version of the helpers synced currently, I cannot run the tests without python-dnspython installed. This should be resolved if you sync the latest, but it will require some modifications to your charms to properly import stuff as per the restructured I mention.

- Needs some lint cleanup

- Can you add a .coveragerc and adjust the Makefile to also report on code coverage? This will let you know what code you haven't hit with your tests. See the current nova pyredux branches for examples on that.

It would be good to get glance_utils.py covered by tests, too, but this is a good start and shouldn't block merging this.

review: Needs Fixing

[lp:~andreserl/charms/precise/glance/port]

197. By Andres Rodriguez on 2013-08-14

fix lint

198. By Andres Rodriguez on 2013-08-14

use latest charmhelpers and make modifications accordingly

199. By Andres Rodriguez on 2013-08-14

Add coveragerc

[Andres Rodriguez (andreserl)]

Adam,

Addressed:

1. sync charm-helpers and adapt to correctly import.
2. lint cleanup done
3. Add coveragerc
4. Is in my TODO.

review: Approve

[Andres Rodriguez (andreserl)]

review: Abstain

[lp:~andreserl/charms/precise/glance/port]

200. By Andres Rodriguez on 2013-08-14

really add coveragerc

=== added file '.coveragerc'
--- .coveragerc	1970-01-01 00:00:00 +0000
+++ .coveragerc	2013-08-14 22:57:15 +0000
@@ -0,0 +1,6 @@
+[report]
+# Regexes for lines to exclude from consideration
+exclude_lines =
+    if __name__ == .__main__.:
+include=
+    hooks/glance_*
 
=== modified file 'Makefile'
--- Makefile	2013-07-11 14:17:30 +0000
+++ Makefile	2013-08-14 22:57:15 +0000
@@ -6,3 +6,7 @@
 
 sync:
 	@charm-helper-sync -c charm-helpers-sync.yaml
+
+test:
+	#@nosetests -svd tests/
+	@$(PYTHON) /usr/bin/nosetests --nologcapture --with-coverage  tests
 
=== modified file 'charm-helpers.yaml'
--- charm-helpers.yaml	2013-07-05 19:12:08 +0000
+++ charm-helpers.yaml	2013-08-14 22:57:15 +0000
@@ -1,4 +1,4 @@
-branch: lp:~openstack-charmers/charm-tools/pyrewrite-helpers
+branch: lp:charm-helpers
 destination: hooks/charmhelpers
 include:
     - core
 
=== added file 'hooks/__init__.py'
=== added file 'hooks/charmhelpers/contrib/hahelpers/apache.py'
--- hooks/charmhelpers/contrib/hahelpers/apache.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/hahelpers/apache.py	2013-08-14 22:57:15 +0000
@@ -0,0 +1,58 @@
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This file is sourced from lp:openstack-charm-helpers
+#
+# Authors:
+#  James Page <[e-mail]>
+#  Adam Gandelman <[e-mail]>
+#
+
+import subprocess
+
+from charmhelpers.core.hookenv import (
+    config as config_get,
+    relation_get,
+    relation_ids,
+    related_units as relation_list,
+    log,
+    INFO,
+)
+
+
+def get_cert():
+    cert = config_get('ssl_cert')
+    key = config_get('ssl_key')
+    if not (cert and key):
+        log("Inspecting identity-service relations for SSL certificate.",
+            level=INFO)
+        cert = key = None
+        for r_id in relation_ids('identity-service'):
+            for unit in relation_list(r_id):
+                if not cert:
+                    cert = relation_get('ssl_cert',
+                                        rid=r_id, unit=unit)
+                if not key:
+                    key = relation_get('ssl_key',
+                                       rid=r_id, unit=unit)
+    return (cert, key)
+
+
+def get_ca_cert():
+    ca_cert = None
+    log("Inspecting identity-service relations for CA SSL certificate.",
+        level=INFO)
+    for r_id in relation_ids('identity-service'):
+        for unit in relation_list(r_id):
+            if not ca_cert:
+                ca_cert = relation_get('ca_cert',
+                                       rid=r_id, unit=unit)
+    return ca_cert
+
+
+def install_ca_cert(ca_cert):
+    if ca_cert:
+        with open('/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt',
+                  'w') as crt:
+            crt.write(ca_cert)
+        subprocess.check_call(['update-ca-certificates', '--fresh'])
 
=== removed file 'hooks/charmhelpers/contrib/hahelpers/apache_utils.py'
--- hooks/charmhelpers/contrib/hahelpers/apache_utils.py	2013-06-24 16:05:57 +0000
+++ hooks/charmhelpers/contrib/hahelpers/apache_utils.py	1970-01-01 00:00:00 +0000
@@ -1,196 +0,0 @@
-#
-# Copyright 2012 Canonical Ltd.
-#
-# This file is sourced from lp:openstack-charm-helpers
-#
-# Authors:
-#  James Page <[e-mail]>
-#  Adam Gandelman <[e-mail]>
-#
-
-from utils import (
-    relation_ids,
-    relation_list,
-    relation_get,
-    render_template,
-    juju_log,
-    config_get,
-    install,
-    get_host_ip,
-    restart
-    )
-from cluster_utils import https
-
-import os
-import subprocess
-from base64 import b64decode
-
-APACHE_SITE_DIR = "/etc/apache2/sites-available"
-SITE_TEMPLATE = "apache2_site.tmpl"
-RELOAD_CHECK = "To activate the new configuration"
-
-
-def get_cert():
-    cert = config_get('ssl_cert')
-    key = config_get('ssl_key')
-    if not (cert and key):
-        juju_log('INFO',
-                 "Inspecting identity-service relations for SSL certificate.")
-        cert = key = None
-        for r_id in relation_ids('identity-service'):
-            for unit in relation_list(r_id):
-                if not cert:
-                    cert = relation_get('ssl_cert',
-                                        rid=r_id, unit=unit)
-                if not key:
-                    key = relation_get('ssl_key',
-                                       rid=r_id, unit=unit)
-    return (cert, key)
-
-
-def get_ca_cert():
-    ca_cert = None
-    juju_log('INFO',
-             "Inspecting identity-service relations for CA SSL certificate.")
-    for r_id in relation_ids('identity-service'):
-        for unit in relation_list(r_id):
-            if not ca_cert:
-                ca_cert = relation_get('ca_cert',
-                                       rid=r_id, unit=unit)
-    return ca_cert
-
-
-def install_ca_cert(ca_cert):
-    if ca_cert:
-        with open('/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt',
-                  'w') as crt:
-            crt.write(ca_cert)
-        subprocess.check_call(['update-ca-certificates', '--fresh'])
-
-
-def enable_https(port_maps, namespace, cert, key, ca_cert=None):
-    '''
-    For a given number of port mappings, configures apache2
-    HTTPs local reverse proxying using certficates and keys provided in
-    either configuration data (preferred) or relation data.  Assumes ports
-    are not in use (calling charm should ensure that).
-
-    port_maps: dict: external to internal port mappings
-    namespace: str: name of charm
-    '''
-    def _write_if_changed(path, new_content):
-        content = None
-        if os.path.exists(path):
-            with open(path, 'r') as f:
-                content = f.read().strip()
-        if content != new_content:
-            with open(path, 'w') as f:
-                f.write(new_content)
-            return True
-        else:
-            return False
-
-    juju_log('INFO', "Enabling HTTPS for port mappings: {}".format(port_maps))
-    http_restart = False
-
-    if cert:
-        cert = b64decode(cert)
-    if key:
-        key = b64decode(key)
-    if ca_cert:
-        ca_cert = b64decode(ca_cert)
-
-    if not cert and not key:
-        juju_log('ERROR',
-                 "Expected but could not find SSL certificate data, not "
-                 "configuring HTTPS!")
-        return False
-
-    install('apache2')
-    if RELOAD_CHECK in subprocess.check_output(['a2enmod', 'ssl',
-                                                'proxy', 'proxy_http']):
-        http_restart = True
-
-    ssl_dir = os.path.join('/etc/apache2/ssl', namespace)
-    if not os.path.exists(ssl_dir):
-        os.makedirs(ssl_dir)
-
-    if (_write_if_changed(os.path.join(ssl_dir, 'cert'), cert)):
-        http_restart = True
-    if (_write_if_changed(os.path.join(ssl_dir, 'key'), key)):
-        http_restart = True
-    os.chmod(os.path.join(ssl_dir, 'key'), 0600)
-
-    install_ca_cert(ca_cert)
-
-    sites_dir = '/etc/apache2/sites-available'
-    for ext_port, int_port in port_maps.items():
-        juju_log('INFO',
-                 'Creating apache2 reverse proxy vhost'
-                 ' for {}:{}'.format(ext_port,
-                                     int_port))
-        site = "{}_{}".format(namespace, ext_port)
-        site_path = os.path.join(sites_dir, site)
-        with open(site_path, 'w') as fsite:
-            context = {
-                "ext": ext_port,
-                "int": int_port,
-                "namespace": namespace,
-                "private_address": get_host_ip()
-                }
-            fsite.write(render_template(SITE_TEMPLATE,
-                                        context))
-
-        if RELOAD_CHECK in subprocess.check_output(['a2ensite', site]):
-            http_restart = True
-
-    if http_restart:
-        restart('apache2')
-
-    return True
-
-
-def disable_https(port_maps, namespace):
-    '''
-    Ensure HTTPS reverse proxying is disables for given port mappings
-
-    port_maps: dict: of ext -> int port mappings
-    namespace: str: name of chamr
-    '''
-    juju_log('INFO', 'Ensuring HTTPS disabled for {}'.format(port_maps))
-
-    if (not os.path.exists('/etc/apache2') or
-        not os.path.exists(os.path.join('/etc/apache2/ssl', namespace))):
-        return
-
-    http_restart = False
-    for ext_port in port_maps.keys():
-        if os.path.exists(os.path.join(APACHE_SITE_DIR,
-                                       "{}_{}".format(namespace,
-                                                      ext_port))):
-            juju_log('INFO',
-                     "Disabling HTTPS reverse proxy"
-                     " for {} {}.".format(namespace,
-                                          ext_port))
-            if (RELOAD_CHECK in
-                subprocess.check_output(['a2dissite',
-                                         '{}_{}'.format(namespace,
-                                                        ext_port)])):
-                http_restart = True
-
-    if http_restart:
-        restart(['apache2'])
-
-
-def setup_https(port_maps, namespace, cert, key, ca_cert=None):
-    '''
-    Ensures HTTPS is either enabled or disabled for given port
-    mapping.
-
-    port_maps: dict: of ext -> int port mappings
-    namespace: str: name of charm
-    '''
-    if not https:
-        disable_https(port_maps, namespace)
-    else:
-        enable_https(port_maps, namespace, cert, key, ca_cert)
 
=== added file 'hooks/charmhelpers/contrib/hahelpers/ceph.py'
--- hooks/charmhelpers/contrib/hahelpers/ceph.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/hahelpers/ceph.py	2013-08-14 22:57:15 +0000
@@ -0,0 +1,291 @@
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This file is sourced from lp:openstack-charm-helpers
+#
+# Authors:
+#  James Page <[e-mail]>
+#  Adam Gandelman <[e-mail]>
+#
+
+import commands
+import os
+import shutil
+import time
+
+from subprocess import (
+    check_call,
+    check_output,
+    CalledProcessError
+)
+
+from charmhelpers.core.hookenv import (
+    relation_get,
+    relation_ids,
+    related_units,
+    log,
+    INFO,
+    ERROR
+)
+
+from charmhelpers.core.host import (
+    apt_install,
+    mount,
+    mounts,
+    service_start,
+    service_stop,
+    umount,
+)
+
+KEYRING = '/etc/ceph/ceph.client.%s.keyring'
+KEYFILE = '/etc/ceph/ceph.client.%s.key'
+
+CEPH_CONF = """[global]
+ auth supported = %(auth)s
+ keyring = %(keyring)s
+ mon host = %(mon_hosts)s
+"""
+
+
+def running(service):
+    # this local util can be dropped as soon the following branch lands
+    # in lp:charm-helpers
+    # https://code.launchpad.net/~gandelman-a/charm-helpers/service_running/
+    try:
+        output = check_output(['service', service, 'status'])
+    except CalledProcessError:
+        return False
+    else:
+        if ("start/running" in output or "is running" in output):
+            return True
+        else:
+            return False
+
+
+def install():
+    ceph_dir = "/etc/ceph"
+    if not os.path.isdir(ceph_dir):
+        os.mkdir(ceph_dir)
+    apt_install('ceph-common', fatal=True)
+
+
+def rbd_exists(service, pool, rbd_img):
+    (rc, out) = commands.getstatusoutput('rbd list --id %s --pool %s' %
+                                         (service, pool))
+    return rbd_img in out
+
+
+def create_rbd_image(service, pool, image, sizemb):
+    cmd = [
+        'rbd',
+        'create',
+        image,
+        '--size',
+        str(sizemb),
+        '--id',
+        service,
+        '--pool',
+        pool
+    ]
+    check_call(cmd)
+
+
+def pool_exists(service, name):
+    (rc, out) = commands.getstatusoutput("rados --id %s lspools" % service)
+    return name in out
+
+
+def create_pool(service, name):
+    cmd = [
+        'rados',
+        '--id',
+        service,
+        'mkpool',
+        name
+    ]
+    check_call(cmd)
+
+
+def keyfile_path(service):
+    return KEYFILE % service
+
+
+def keyring_path(service):
+    return KEYRING % service
+
+
+def create_keyring(service, key):
+    keyring = keyring_path(service)
+    if os.path.exists(keyring):
+        log('ceph: Keyring exists at %s.' % keyring, level=INFO)
+    cmd = [
+        'ceph-authtool',
+        keyring,
+        '--create-keyring',
+        '--name=client.%s' % service,
+        '--add-key=%s' % key
+    ]
+    check_call(cmd)
+    log('ceph: Created new ring at %s.' % keyring, level=INFO)
+
+
+def create_key_file(service, key):
+    # create a file containing the key
+    keyfile = keyfile_path(service)
+    if os.path.exists(keyfile):
+        log('ceph: Keyfile exists at %s.' % keyfile, level=INFO)
+    fd = open(keyfile, 'w')
+    fd.write(key)
+    fd.close()
+    log('ceph: Created new keyfile at %s.' % keyfile, level=INFO)
+
+
+def get_ceph_nodes():
+    hosts = []
+    for r_id in relation_ids('ceph'):
+        for unit in related_units(r_id):
+            hosts.append(relation_get('private-address', unit=unit, rid=r_id))
+    return hosts
+
+
+def configure(service, key, auth):
+    create_keyring(service, key)
+    create_key_file(service, key)
+    hosts = get_ceph_nodes()
+    mon_hosts = ",".join(map(str, hosts))
+    keyring = keyring_path(service)
+    with open('/etc/ceph/ceph.conf', 'w') as ceph_conf:
+        ceph_conf.write(CEPH_CONF % locals())
+    modprobe_kernel_module('rbd')
+
+
+def image_mapped(image_name):
+    (rc, out) = commands.getstatusoutput('rbd showmapped')
+    return image_name in out
+
+
+def map_block_storage(service, pool, image):
+    cmd = [
+        'rbd',
+        'map',
+        '%s/%s' % (pool, image),
+        '--user',
+        service,
+        '--secret',
+        keyfile_path(service),
+    ]
+    check_call(cmd)
+
+
+def filesystem_mounted(fs):
+    return fs in [f for m, f in mounts()]
+
+
+def make_filesystem(blk_device, fstype='ext4', timeout=10):
+    count = 0
+    e_noent = os.errno.ENOENT
+    while not os.path.exists(blk_device):
+        if count >= timeout:
+            log('ceph: gave up waiting on block device %s' % blk_device,
+                level=ERROR)
+            raise IOError(e_noent, os.strerror(e_noent), blk_device)
+        log('ceph: waiting for block device %s to appear' % blk_device,
+            level=INFO)
+        count += 1
+        time.sleep(1)
+    else:
+        log('ceph: Formatting block device %s as filesystem %s.' %
+            (blk_device, fstype), level=INFO)
+        check_call(['mkfs', '-t', fstype, blk_device])
+
+
+def place_data_on_ceph(service, blk_device, data_src_dst, fstype='ext4'):
+    # mount block device into /mnt
+    mount(blk_device, '/mnt')
+
+    # copy data to /mnt
+    try:
+        copy_files(data_src_dst, '/mnt')
+    except:
+        pass
+
+    # umount block device
+    umount('/mnt')
+
+    _dir = os.stat(data_src_dst)
+    uid = _dir.st_uid
+    gid = _dir.st_gid
+
+    # re-mount where the data should originally be
+    mount(blk_device, data_src_dst, persist=True)
+
+    # ensure original ownership of new mount.
+    cmd = ['chown', '-R', '%s:%s' % (uid, gid), data_src_dst]
+    check_call(cmd)
+
+
+# TODO: re-use
+def modprobe_kernel_module(module):
+    log('ceph: Loading kernel module', level=INFO)
+    cmd = ['modprobe', module]
+    check_call(cmd)
+    cmd = 'echo %s >> /etc/modules' % module
+    check_call(cmd, shell=True)
+
+
+def copy_files(src, dst, symlinks=False, ignore=None):
+    for item in os.listdir(src):
+        s = os.path.join(src, item)
+        d = os.path.join(dst, item)
+        if os.path.isdir(s):
+            shutil.copytree(s, d, symlinks, ignore)
+        else:
+            shutil.copy2(s, d)
+
+
+def ensure_ceph_storage(service, pool, rbd_img, sizemb, mount_point,
+                        blk_device, fstype, system_services=[]):
+    """
+    To be called from the current cluster leader.
+    Ensures given pool and RBD image exists, is mapped to a block device,
+    and the device is formatted and mounted at the given mount_point.
+
+    If formatting a device for the first time, data existing at mount_point
+    will be migrated to the RBD device before being remounted.
+
+    All services listed in system_services will be stopped prior to data
+    migration and restarted when complete.
+    """
+    # Ensure pool, RBD image, RBD mappings are in place.
+    if not pool_exists(service, pool):
+        log('ceph: Creating new pool %s.' % pool, level=INFO)
+        create_pool(service, pool)
+
+    if not rbd_exists(service, pool, rbd_img):
+        log('ceph: Creating RBD image (%s).' % rbd_img, level=INFO)
+        create_rbd_image(service, pool, rbd_img, sizemb)
+
+    if not image_mapped(rbd_img):
+        log('ceph: Mapping RBD Image as a Block Device.', level=INFO)
+        map_block_storage(service, pool, rbd_img)
+
+    # make file system
+    # TODO: What happens if for whatever reason this is run again and
+    # the data is already in the rbd device and/or is mounted??
+    # When it is mounted already, it will fail to make the fs
+    # XXX: This is really sketchy!  Need to at least add an fstab entry
+    #      otherwise this hook will blow away existing data if its executed
+    #      after a reboot.
+    if not filesystem_mounted(mount_point):
+        make_filesystem(blk_device, fstype)
+
+        for svc in system_services:
+            if running(svc):
+                log('Stopping services %s prior to migrating data.' % svc,
+                    level=INFO)
+                service_stop(svc)
+
+        place_data_on_ceph(service, blk_device, mount_point, fstype)
+
+        for svc in system_services:
+            service_start(svc)
 
=== removed file 'hooks/charmhelpers/contrib/hahelpers/ceph_utils.py'
--- hooks/charmhelpers/contrib/hahelpers/ceph_utils.py	2013-06-24 16:05:57 +0000
+++ hooks/charmhelpers/contrib/hahelpers/ceph_utils.py	1970-01-01 00:00:00 +0000
@@ -1,256 +0,0 @@
-#
-# Copyright 2012 Canonical Ltd.
-#
-# This file is sourced from lp:openstack-charm-helpers
-#
-# Authors:
-#  James Page <[e-mail]>
-#  Adam Gandelman <[e-mail]>
-#
-
-import commands
-import subprocess
-import os
-import shutil
-import utils
-
-KEYRING = '/etc/ceph/ceph.client.%s.keyring'
-KEYFILE = '/etc/ceph/ceph.client.%s.key'
-
-CEPH_CONF = """[global]
- auth supported = %(auth)s
- keyring = %(keyring)s
- mon host = %(mon_hosts)s
-"""
-
-
-def execute(cmd):
-    subprocess.check_call(cmd)
-
-
-def execute_shell(cmd):
-    subprocess.check_call(cmd, shell=True)
-
-
-def install():
-    ceph_dir = "/etc/ceph"
-    if not os.path.isdir(ceph_dir):
-        os.mkdir(ceph_dir)
-    utils.install('ceph-common')
-
-
-def rbd_exists(service, pool, rbd_img):
-    (rc, out) = commands.getstatusoutput('rbd list --id %s --pool %s' %\
-                                         (service, pool))
-    return rbd_img in out
-
-
-def create_rbd_image(service, pool, image, sizemb):
-    cmd = [
-        'rbd',
-        'create',
-        image,
-        '--size',
-        str(sizemb),
-        '--id',
-        service,
-        '--pool',
-        pool
-        ]
-    execute(cmd)
-
-
-def pool_exists(service, name):
-    (rc, out) = commands.getstatusoutput("rados --id %s lspools" % service)
-    return name in out
-
-
-def create_pool(service, name):
-    cmd = [
-        'rados',
-        '--id',
-        service,
-        'mkpool',
-        name
-        ]
-    execute(cmd)
-
-
-def keyfile_path(service):
-    return KEYFILE % service
-
-
-def keyring_path(service):
-    return KEYRING % service
-
-
-def create_keyring(service, key):
-    keyring = keyring_path(service)
-    if os.path.exists(keyring):
-        utils.juju_log('INFO', 'ceph: Keyring exists at %s.' % keyring)
-    cmd = [
-        'ceph-authtool',
-        keyring,
-        '--create-keyring',
-        '--name=client.%s' % service,
-        '--add-key=%s' % key
-        ]
-    execute(cmd)
-    utils.juju_log('INFO', 'ceph: Created new ring at %s.' % keyring)
-
-
-def create_key_file(service, key):
-    # create a file containing the key
-    keyfile = keyfile_path(service)
-    if os.path.exists(keyfile):
-        utils.juju_log('INFO', 'ceph: Keyfile exists at %s.' % keyfile)
-    fd = open(keyfile, 'w')
-    fd.write(key)
-    fd.close()
-    utils.juju_log('INFO', 'ceph: Created new keyfile at %s.' % keyfile)
-
-
-def get_ceph_nodes():
-    hosts = []
-    for r_id in utils.relation_ids('ceph'):
-        for unit in utils.relation_list(r_id):
-            hosts.append(utils.relation_get('private-address',
-                                            unit=unit, rid=r_id))
-    return hosts
-
-
-def configure(service, key, auth):
-    create_keyring(service, key)
-    create_key_file(service, key)
-    hosts = get_ceph_nodes()
-    mon_hosts = ",".join(map(str, hosts))
-    keyring = keyring_path(service)
-    with open('/etc/ceph/ceph.conf', 'w') as ceph_conf:
-        ceph_conf.write(CEPH_CONF % locals())
-    modprobe_kernel_module('rbd')
-
-
-def image_mapped(image_name):
-    (rc, out) = commands.getstatusoutput('rbd showmapped')
-    return image_name in out
-
-
-def map_block_storage(service, pool, image):
-    cmd = [
-        'rbd',
-        'map',
-        '%s/%s' % (pool, image),
-        '--user',
-        service,
-        '--secret',
-        keyfile_path(service),
-        ]
-    execute(cmd)
-
-
-def filesystem_mounted(fs):
-    return subprocess.call(['grep', '-wqs', fs, '/proc/mounts']) == 0
-
-
-def make_filesystem(blk_device, fstype='ext4'):
-    utils.juju_log('INFO',
-                   'ceph: Formatting block device %s as filesystem %s.' %\
-                   (blk_device, fstype))
-    cmd = ['mkfs', '-t', fstype, blk_device]
-    execute(cmd)
-
-
-def place_data_on_ceph(service, blk_device, data_src_dst, fstype='ext4'):
-    # mount block device into /mnt
-    cmd = ['mount', '-t', fstype, blk_device, '/mnt']
-    execute(cmd)
-
-    # copy data to /mnt
-    try:
-        copy_files(data_src_dst, '/mnt')
-    except:
-        pass
-
-    # umount block device
-    cmd = ['umount', '/mnt']
-    execute(cmd)
-
-    _dir = os.stat(data_src_dst)
-    uid = _dir.st_uid
-    gid = _dir.st_gid
-
-    # re-mount where the data should originally be
-    cmd = ['mount', '-t', fstype, blk_device, data_src_dst]
-    execute(cmd)
-
-    # ensure original ownership of new mount.
-    cmd = ['chown', '-R', '%s:%s' % (uid, gid), data_src_dst]
-    execute(cmd)
-
-
-# TODO: re-use
-def modprobe_kernel_module(module):
-    utils.juju_log('INFO', 'Loading kernel module')
-    cmd = ['modprobe', module]
-    execute(cmd)
-    cmd = 'echo %s >> /etc/modules' % module
-    execute_shell(cmd)
-
-
-def copy_files(src, dst, symlinks=False, ignore=None):
-    for item in os.listdir(src):
-        s = os.path.join(src, item)
-        d = os.path.join(dst, item)
-        if os.path.isdir(s):
-            shutil.copytree(s, d, symlinks, ignore)
-        else:
-            shutil.copy2(s, d)
-
-
-def ensure_ceph_storage(service, pool, rbd_img, sizemb, mount_point,
-                        blk_device, fstype, system_services=[]):
-    """
-    To be called from the current cluster leader.
-    Ensures given pool and RBD image exists, is mapped to a block device,
-    and the device is formatted and mounted at the given mount_point.
-
-    If formatting a device for the first time, data existing at mount_point
-    will be migrated to the RBD device before being remounted.
-
-    All services listed in system_services will be stopped prior to data
-    migration and restarted when complete.
-    """
-    # Ensure pool, RBD image, RBD mappings are in place.
-    if not pool_exists(service, pool):
-        utils.juju_log('INFO', 'ceph: Creating new pool %s.' % pool)
-        create_pool(service, pool)
-
-    if not rbd_exists(service, pool, rbd_img):
-        utils.juju_log('INFO', 'ceph: Creating RBD image (%s).' % rbd_img)
-        create_rbd_image(service, pool, rbd_img, sizemb)
-
-    if not image_mapped(rbd_img):
-        utils.juju_log('INFO', 'ceph: Mapping RBD Image as a Block Device.')
-        map_block_storage(service, pool, rbd_img)
-
-    # make file system
-    # TODO: What happens if for whatever reason this is run again and
-    # the data is already in the rbd device and/or is mounted??
-    # When it is mounted already, it will fail to make the fs
-    # XXX: This is really sketchy!  Need to at least add an fstab entry
-    #      otherwise this hook will blow away existing data if its executed
-    #      after a reboot.
-    if not filesystem_mounted(mount_point):
-        make_filesystem(blk_device, fstype)
-
-        for svc in system_services:
-            if utils.running(svc):
-                utils.juju_log('INFO',
-                               'Stopping services %s prior to migrating '\
-                               'data' % svc)
-                utils.stop(svc)
-
-        place_data_on_ceph(service, blk_device, mount_point, fstype)
-
-        for svc in system_services:
-            utils.start(svc)
 
=== added file 'hooks/charmhelpers/contrib/hahelpers/cluster.py'
--- hooks/charmhelpers/contrib/hahelpers/cluster.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/hahelpers/cluster.py	2013-08-14 22:57:15 +0000
@@ -0,0 +1,181 @@
+#
+# Copyright 2012 Canonical Ltd.
+#
+# Authors:
+#  James Page <[e-mail]>
+#  Adam Gandelman <[e-mail]>
+#
+
+import subprocess
+import os
+
+from socket import gethostname as get_unit_hostname
+
+from charmhelpers.core.hookenv import (
+    log,
+    relation_ids,
+    related_units as relation_list,
+    relation_get,
+    config as config_get,
+    INFO,
+    ERROR,
+    unit_get,
+)
+
+
+class HAIncompleteConfig(Exception):
+    pass
+
+
+def is_clustered():
+    for r_id in (relation_ids('ha') or []):
+        for unit in (relation_list(r_id) or []):
+            clustered = relation_get('clustered',
+                                     rid=r_id,
+                                     unit=unit)
+            if clustered:
+                return True
+    return False
+
+
+def is_leader(resource):
+    cmd = [
+        "crm", "resource",
+        "show", resource
+    ]
+    try:
+        status = subprocess.check_output(cmd)
+    except subprocess.CalledProcessError:
+        return False
+    else:
+        if get_unit_hostname() in status:
+            return True
+        else:
+            return False
+
+
+def peer_units():
+    peers = []
+    for r_id in (relation_ids('cluster') or []):
+        for unit in (relation_list(r_id) or []):
+            peers.append(unit)
+    return peers
+
+
+def oldest_peer(peers):
+    local_unit_no = int(os.getenv('JUJU_UNIT_NAME').split('/')[1])
+    for peer in peers:
+        remote_unit_no = int(peer.split('/')[1])
+        if remote_unit_no < local_unit_no:
+            return False
+    return True
+
+
+def eligible_leader(resource):
+    if is_clustered():
+        if not is_leader(resource):
+            log('Deferring action to CRM leader.', level=INFO)
+            return False
+    else:
+        peers = peer_units()
+        if peers and not oldest_peer(peers):
+            log('Deferring action to oldest service unit.', level=INFO)
+            return False
+    return True
+
+
+def https():
+    '''
+    Determines whether enough data has been provided in configuration
+    or relation data to configure HTTPS
+    .
+    returns: boolean
+    '''
+    if config_get('use-https') == "yes":
+        return True
+    if config_get('ssl_cert') and config_get('ssl_key'):
+        return True
+    for r_id in relation_ids('identity-service'):
+        for unit in relation_list(r_id):
+            if None not in [
+                relation_get('https_keystone', rid=r_id, unit=unit),
+                relation_get('ssl_cert', rid=r_id, unit=unit),
+                relation_get('ssl_key', rid=r_id, unit=unit),
+                relation_get('ca_cert', rid=r_id, unit=unit),
+            ]:
+                return True
+    return False
+
+
+def determine_api_port(public_port):
+    '''
+    Determine correct API server listening port based on
+    existence of HTTPS reverse proxy and/or haproxy.
+
+    public_port: int: standard public port for given service
+
+    returns: int: the correct listening port for the API service
+    '''
+    i = 0
+    if len(peer_units()) > 0 or is_clustered():
+        i += 1
+    if https():
+        i += 1
+    return public_port - (i * 10)
+
+
+def determine_haproxy_port(public_port):
+    '''
+    Description: Determine correct proxy listening port based on public IP +
+    existence of HTTPS reverse proxy.
+
+    public_port: int: standard public port for given service
+
+    returns: int: the correct listening port for the HAProxy service
+    '''
+    i = 0
+    if https():
+        i += 1
+    return public_port - (i * 10)
+
+
+def get_hacluster_config():
+    '''
+    Obtains all relevant configuration from charm configuration required
+    for initiating a relation to hacluster:
+
+        ha-bindiface, ha-mcastport, vip, vip_iface, vip_cidr
+
+    returns: dict: A dict containing settings keyed by setting name.
+    raises: HAIncompleteConfig if settings are missing.
+    '''
+    settings = ['ha-bindiface', 'ha-mcastport', 'vip', 'vip_iface', 'vip_cidr']
+    conf = {}
+    for setting in settings:
+        conf[setting] = config_get(setting)
+    missing = []
+    [missing.append(s) for s, v in conf.iteritems() if v is None]
+    if missing:
+        log('Insufficient config data to configure hacluster.', level=ERROR)
+        raise HAIncompleteConfig
+    return conf
+
+
+def canonical_url(configs, vip_setting='vip'):
+    '''
+    Returns the correct HTTP URL to this host given the state of HTTPS
+    configuration and hacluster.
+
+    :configs    : OSTemplateRenderer: A config tempating object to inspect for
+                                      a complete https context.
+    :vip_setting:                str: Setting in charm config that specifies
+                                      VIP address.
+    '''
+    scheme = 'http'
+    if 'https' in configs.complete_contexts():
+        scheme = 'https'
+    if is_clustered():
+        addr = config_get(vip_setting)
+    else:
+        addr = unit_get('private-address')
+    return '%s://%s' % (scheme, addr)
 
=== removed file 'hooks/charmhelpers/contrib/hahelpers/cluster_utils.py'
--- hooks/charmhelpers/contrib/hahelpers/cluster_utils.py	2013-07-09 17:24:46 +0000
+++ hooks/charmhelpers/contrib/hahelpers/cluster_utils.py	1970-01-01 00:00:00 +0000
@@ -1,176 +0,0 @@
-#
-# Copyright 2012 Canonical Ltd.
-#
-# This file is sourced from lp:openstack-charm-helpers
-#
-# Authors:
-#  James Page <[e-mail]>
-#  Adam Gandelman <[e-mail]>
-#
-
-from utils import (
-    juju_log,
-    relation_ids,
-    relation_list,
-    relation_get,
-    get_unit_hostname,
-    config_get
-)
-import subprocess
-import os
-
-
-class HAIncompleteConfig(Exception):
-    pass
-
-
-def is_clustered():
-    for r_id in (relation_ids('ha') or []):
-        for unit in (relation_list(r_id) or []):
-            clustered = relation_get('clustered',
-                                     rid=r_id,
-                                     unit=unit)
-            if clustered:
-                return True
-    return False
-
-
-def is_leader(resource):
-    cmd = [
-        "crm", "resource",
-        "show", resource
-        ]
-    try:
-        status = subprocess.check_output(cmd)
-    except subprocess.CalledProcessError:
-        return False
-    else:
-        if get_unit_hostname() in status:
-            return True
-        else:
-            return False
-
-
-def peer_units():
-    peers = []
-    for r_id in (relation_ids('cluster') or []):
-        for unit in (relation_list(r_id) or []):
-            peers.append(unit)
-    return peers
-
-
-def oldest_peer(peers):
-    local_unit_no = int(os.getenv('JUJU_UNIT_NAME').split('/')[1])
-    for peer in peers:
-        remote_unit_no = int(peer.split('/')[1])
-        if remote_unit_no < local_unit_no:
-            return False
-    return True
-
-
-def eligible_leader(resource):
-    if is_clustered():
-        if not is_leader(resource):
-            juju_log('INFO', 'Deferring action to CRM leader.')
-            return False
-    else:
-        peers = peer_units()
-        if peers and not oldest_peer(peers):
-            juju_log('INFO', 'Deferring action to oldest service unit.')
-            return False
-    return True
-
-
-def https():
-    '''
-    Determines whether enough data has been provided in configuration
-    or relation data to configure HTTPS
-    .
-    returns: boolean
-    '''
-    if config_get('use-https') == "yes":
-        return True
-    if config_get('ssl_cert') and config_get('ssl_key'):
-        return True
-    for r_id in relation_ids('identity-service'):
-        for unit in relation_list(r_id):
-            if (relation_get('https_keystone', rid=r_id, unit=unit) and
-                relation_get('ssl_cert', rid=r_id, unit=unit) and
-                relation_get('ssl_key', rid=r_id, unit=unit) and
-                relation_get('ca_cert', rid=r_id, unit=unit)):
-                return True
-    return False
-
-
-def determine_api_port(public_port):
-    '''
-    Determine correct API server listening port based on
-    existence of HTTPS reverse proxy and/or haproxy.
-
-    public_port: int: standard public port for given service
-
-    returns: int: the correct listening port for the API service
-    '''
-    i = 0
-    if len(peer_units()) > 0 or is_clustered():
-        i += 1
-    if https():
-        i += 1
-    return public_port - (i * 10)
-
-
-def determine_haproxy_port(public_port):
-    '''
-    Description: Determine correct proxy listening port based on public IP +
-    existence of HTTPS reverse proxy.
-
-    public_port: int: standard public port for given service
-
-    returns: int: the correct listening port for the HAProxy service
-    '''
-    i = 0
-    if https():
-        i += 1
-    return public_port - (i * 10)
-
-
-def get_hacluster_config():
-    '''
-    Obtains all relevant configuration from charm configuration required
-    for initiating a relation to hacluster:
-
-        ha-bindiface, ha-mcastport, vip, vip_iface, vip_cidr
-
-    returns: dict: A dict containing settings keyed by setting name.
-    raises: HAIncompleteConfig if settings are missing.
-    '''
-    settings = ['ha-bindiface', 'ha-mcastport', 'vip', 'vip_iface', 'vip_cidr']
-    conf = {}
-    for setting in settings:
-        conf[setting] = config_get(setting)
-    missing = []
-    [missing.append(s) for s, v in conf.iteritems() if v is None]
-    if missing:
-        juju_log('Insufficient config data to configure hacluster.')
-        raise HAIncompleteConfig
-    return conf
-
-
-def canonical_url(configs, vip_setting='vip'):
-    '''
-    Returns the correct HTTP URL to this host given the state of HTTPS
-    configuration and hacluster.
-
-    :configs    : OSTemplateRenderer: A config tempating object to inspect for
-                                      a complete https context.
-    :vip_setting:                str: Setting in charm config that specifies
-                                      VIP address.
-    '''
-    scheme = 'http'
-    if 'https' in configs.complete_contexts():
-        scheme = 'https'
-    if is_clustered():
-        addr = config_get(vip_setting)
-    else:
-        addr = get_unit_hostname()
-    return '%s://%s' % (scheme, addr)
 
=== removed file 'hooks/charmhelpers/contrib/hahelpers/haproxy_utils.py'
--- hooks/charmhelpers/contrib/hahelpers/haproxy_utils.py	2013-06-24 16:05:57 +0000
+++ hooks/charmhelpers/contrib/hahelpers/haproxy_utils.py	1970-01-01 00:00:00 +0000
@@ -1,55 +0,0 @@
-#
-# Copyright 2012 Canonical Ltd.
-#
-# This file is sourced from lp:openstack-charm-helpers
-#
-# Authors:
-#  James Page <[e-mail]>
-#  Adam Gandelman <[e-mail]>
-#
-
-from utils import (
-    relation_ids,
-    relation_list,
-    relation_get,
-    unit_get,
-    reload,
-    render_template
-    )
-import os
-
-HAPROXY_CONF = '/etc/haproxy/haproxy.cfg'
-HAPROXY_DEFAULT = '/etc/default/haproxy'
-
-
-def configure_haproxy(service_ports):
-    '''
-    Configure HAProxy based on the current peers in the service
-    cluster using the provided port map:
-
-        "swift": [ 8080, 8070 ]
-
-    HAproxy will also be reloaded/started if required
-
-    service_ports: dict: dict of lists of [ frontend, backend ]
-    '''
-    cluster_hosts = {}
-    cluster_hosts[os.getenv('JUJU_UNIT_NAME').replace('/', '-')] = \
-        unit_get('private-address')
-    for r_id in relation_ids('cluster'):
-        for unit in relation_list(r_id):
-            cluster_hosts[unit.replace('/', '-')] = \
-                relation_get(attribute='private-address',
-                             rid=r_id,
-                             unit=unit)
-    context = {
-        'units': cluster_hosts,
-        'service_ports': service_ports
-        }
-    with open(HAPROXY_CONF, 'w') as f:
-        f.write(render_template(os.path.basename(HAPROXY_CONF),
-                                context))
-    with open(HAPROXY_DEFAULT, 'w') as f:
-        f.write('ENABLED=1')
-
-    reload('haproxy')
 
=== removed file 'hooks/charmhelpers/contrib/hahelpers/utils.py'
--- hooks/charmhelpers/contrib/hahelpers/utils.py	2013-06-24 16:05:57 +0000
+++ hooks/charmhelpers/contrib/hahelpers/utils.py	1970-01-01 00:00:00 +0000
@@ -1,333 +0,0 @@
-#
-# Copyright 2012 Canonical Ltd.
-#
-# This file is sourced from lp:openstack-charm-helpers
-#
-# Authors:
-#  James Page <[e-mail]>
-#  Paul Collins <[e-mail]>
-#  Adam Gandelman <[e-mail]>
-#
-
-import json
-import os
-import subprocess
-import socket
-import sys
-
-
-def do_hooks(hooks):
-    hook = os.path.basename(sys.argv[0])
-
-    try:
-        hook_func = hooks[hook]
-    except KeyError:
-        juju_log('INFO',
-                 "This charm doesn't know how to handle '{}'.".format(hook))
-    else:
-        hook_func()
-
-
-def install(*pkgs):
-    cmd = [
-        'apt-get',
-        '-y',
-        'install'
-          ]
-    for pkg in pkgs:
-        cmd.append(pkg)
-    subprocess.check_call(cmd)
-
-TEMPLATES_DIR = 'templates'
-
-try:
-    import jinja2
-except ImportError:
-    install('python-jinja2')
-    import jinja2
-
-try:
-    import dns.resolver
-except ImportError:
-    install('python-dnspython')
-    import dns.resolver
-
-
-def render_template(template_name, context, template_dir=TEMPLATES_DIR):
-    templates = jinja2.Environment(
-                    loader=jinja2.FileSystemLoader(template_dir)
-                    )
-    template = templates.get_template(template_name)
-    return template.render(context)
-
-CLOUD_ARCHIVE = \
-""" # Ubuntu Cloud Archive
-deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
-"""
-
-CLOUD_ARCHIVE_POCKETS = {
-    'folsom': 'precise-updates/folsom',
-    'folsom/updates': 'precise-updates/folsom',
-    'folsom/proposed': 'precise-proposed/folsom',
-    'grizzly': 'precise-updates/grizzly',
-    'grizzly/updates': 'precise-updates/grizzly',
-    'grizzly/proposed': 'precise-proposed/grizzly'
-    }
-
-
-def configure_source():
-    source = str(config_get('openstack-origin'))
-    if not source:
-        return
-    if source.startswith('ppa:'):
-        cmd = [
-            'add-apt-repository',
-            source
-            ]
-        subprocess.check_call(cmd)
-    if source.startswith('cloud:'):
-        # CA values should be formatted as cloud:ubuntu-openstack/pocket, eg:
-        #   cloud:precise-folsom/updates or cloud:precise-folsom/proposed
-        install('ubuntu-cloud-keyring')
-        pocket = source.split(':')[1]
-        pocket = pocket.split('-')[1]
-        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
-            apt.write(CLOUD_ARCHIVE.format(CLOUD_ARCHIVE_POCKETS[pocket]))
-    if source.startswith('deb'):
-        l = len(source.split('|'))
-        if l == 2:
-            (apt_line, key) = source.split('|')
-            cmd = [
-                'apt-key',
-                'adv', '--keyserver keyserver.ubuntu.com',
-                '--recv-keys', key
-                ]
-            subprocess.check_call(cmd)
-        elif l == 1:
-            apt_line = source
-
-        with open('/etc/apt/sources.list.d/quantum.list', 'w') as apt:
-            apt.write(apt_line + "\n")
-    cmd = [
-        'apt-get',
-        'update'
-        ]
-    subprocess.check_call(cmd)
-
-# Protocols
-TCP = 'TCP'
-UDP = 'UDP'
-
-
-def expose(port, protocol='TCP'):
-    cmd = [
-        'open-port',
-        '{}/{}'.format(port, protocol)
-        ]
-    subprocess.check_call(cmd)
-
-
-def juju_log(severity, message):
-    cmd = [
-        'juju-log',
-        '--log-level', severity,
-        message
-        ]
-    subprocess.check_call(cmd)
-
-
-cache = {}
-
-
-def cached(func):
-    def wrapper(*args, **kwargs):
-        global cache
-        key = str((func, args, kwargs))
-        try:
-            return cache[key]
-        except KeyError:
-            res = func(*args, **kwargs)
-            cache[key] = res
-            return res
-    return wrapper
-
-
-@cached
-def relation_ids(relation):
-    cmd = [
-        'relation-ids',
-        relation
-        ]
-    result = str(subprocess.check_output(cmd)).split()
-    if result == "":
-        return None
-    else:
-        return result
-
-
-@cached
-def relation_list(rid):
-    cmd = [
-        'relation-list',
-        '-r', rid,
-        ]
-    result = str(subprocess.check_output(cmd)).split()
-    if result == "":
-        return None
-    else:
-        return result
-
-
-@cached
-def relation_get(attribute, unit=None, rid=None):
-    cmd = [
-        'relation-get',
-        ]
-    if rid:
-        cmd.append('-r')
-        cmd.append(rid)
-    cmd.append(attribute)
-    if unit:
-        cmd.append(unit)
-    value = subprocess.check_output(cmd).strip()  # IGNORE:E1103
-    if value == "":
-        return None
-    else:
-        return value
-
-
-@cached
-def relation_get_dict(relation_id=None, remote_unit=None):
-    """Obtain all relation data as dict by way of JSON"""
-    cmd = [
-        'relation-get', '--format=json'
-        ]
-    if relation_id:
-        cmd.append('-r')
-        cmd.append(relation_id)
-    if remote_unit:
-        remote_unit_orig = os.getenv('JUJU_REMOTE_UNIT', None)
-        os.environ['JUJU_REMOTE_UNIT'] = remote_unit
-    j = subprocess.check_output(cmd)
-    if remote_unit and remote_unit_orig:
-        os.environ['JUJU_REMOTE_UNIT'] = remote_unit_orig
-    d = json.loads(j)
-    settings = {}
-    # convert unicode to strings
-    for k, v in d.iteritems():
-        settings[str(k)] = str(v)
-    return settings
-
-
-def relation_set(**kwargs):
-    cmd = [
-        'relation-set'
-        ]
-    args = []
-    for k, v in kwargs.items():
-        if k == 'rid':
-            if v:
-                cmd.append('-r')
-                cmd.append(v)
-        else:
-            args.append('{}={}'.format(k, v))
-    cmd += args
-    subprocess.check_call(cmd)
-
-
-@cached
-def unit_get(attribute):
-    cmd = [
-        'unit-get',
-        attribute
-        ]
-    value = subprocess.check_output(cmd).strip()  # IGNORE:E1103
-    if value == "":
-        return None
-    else:
-        return value
-
-
-@cached
-def config_get(attribute):
-    cmd = [
-        'config-get',
-        '--format',
-        'json',
-        ]
-    out = subprocess.check_output(cmd).strip()  # IGNORE:E1103
-    cfg = json.loads(out)
-
-    try:
-        return cfg[attribute]
-    except KeyError:
-        return None
-
-
-@cached
-def get_unit_hostname():
-    return socket.gethostname()
-
-
-@cached
-def get_host_ip(hostname=None):
-    hostname = hostname or unit_get('private-address')
-    try:
-        # Test to see if already an IPv4 address
-        socket.inet_aton(hostname)
-        return hostname
-    except socket.error:
-        answers = dns.resolver.query(hostname, 'A')
-        if answers:
-            return answers[0].address
-    return None
-
-
-def _svc_control(service, action):
-    subprocess.check_call(['service', service, action])
-
-
-def restart(*services):
-    for service in services:
-        _svc_control(service, 'restart')
-
-
-def stop(*services):
-    for service in services:
-        _svc_control(service, 'stop')
-
-
-def start(*services):
-    for service in services:
-        _svc_control(service, 'start')
-
-
-def reload(*services):
-    for service in services:
-        try:
-            _svc_control(service, 'reload')
-        except subprocess.CalledProcessError:
-            # Reload failed - either service does not support reload
-            # or it was not running - restart will fixup most things
-            _svc_control(service, 'restart')
-
-
-def running(service):
-    try:
-        output = subprocess.check_output(['service', service, 'status'])
-    except subprocess.CalledProcessError:
-        return False
-    else:
-        if ("start/running" in output or
-            "is running" in output):
-            return True
-        else:
-            return False
-
-
-def is_relation_made(relation, key='private-address'):
-    for r_id in (relation_ids(relation) or []):
-        for unit in (relation_list(r_id) or []):
-            if relation_get(key, rid=r_id, unit=unit):
-                return True
-    return False
 
=== modified file 'hooks/charmhelpers/contrib/openstack/context.py'
--- hooks/charmhelpers/contrib/openstack/context.py	2013-07-05 19:12:08 +0000
+++ hooks/charmhelpers/contrib/openstack/context.py	2013-08-14 22:57:15 +0000
@@ -16,7 +16,7 @@
     unit_get,
 )
 
-from charmhelpers.contrib.hahelpers.cluster_utils import (
+from charmhelpers.contrib.hahelpers.cluster import (
     determine_api_port,
     determine_haproxy_port,
     https,
@@ -24,7 +24,7 @@
     peer_units,
 )
 
-from charmhelpers.contrib.hahelpers.apache_utils import (
+from charmhelpers.contrib.hahelpers.apache import (
     get_cert,
     get_ca_cert,
 )
@@ -42,7 +42,7 @@
         if v is None or v == '':
             _missing.append(k)
     if _missing:
-        print 'Missing required data: %s' % ' '.join(_missing)
+        log('Missing required data: %s' % ' '.join(_missing), level='INFO')
         return False
     return True
 
@@ -106,8 +106,8 @@
                     'admin_password': relation_get('service_password', rid=rid,
                                                    unit=unit),
                     # XXX: Hard-coded http.
-                    'service_protocol':  'http',
-                    'auth_protocol':  'http',
+                    'service_protocol': 'http',
+                    'auth_protocol': 'http',
                 }
         if not context_complete(ctxt):
             return {}
@@ -202,6 +202,30 @@
             with open('/etc/default/haproxy', 'w') as out:
                 out.write('ENABLED=1\n')
             return ctxt
+        log('HAProxy context is incomplete, this unit has no peers.')
+        return {}
+
+
+class ImageServiceContext(OSContextGenerator):
+    interfaces = ['image-servce']
+
+    def __call__(self):
+        '''
+        Obtains the glance API server from the image-service relation.  Useful
+        in nova and cinder (currently).
+        '''
+        log('Generating template context for image-service.')
+        rids = relation_ids('image-service')
+        if not rids:
+            return {}
+        for rid in rids:
+            for unit in related_units(rid):
+                api_server = relation_get('glance-api-server',
+                                          rid=rid, unit=unit)
+                if api_server:
+                    return {'glance_api_servers': api_server}
+        log('ImageService context is incomplete. '
+            'Missing required relation data.')
         return {}
 
 
 
=== removed file 'hooks/charmhelpers/contrib/openstack/openstack_utils.py'
--- hooks/charmhelpers/contrib/openstack/openstack_utils.py	2013-07-11 17:29:14 +0000
+++ hooks/charmhelpers/contrib/openstack/openstack_utils.py	1970-01-01 00:00:00 +0000
@@ -1,270 +0,0 @@
-#!/usr/bin/python
-
-# Common python helper functions used for OpenStack charms.
-
-import apt_pkg as apt
-import subprocess
-import os
-import sys
-
-from distutils.version import StrictVersion
-
-from charmhelpers.core.hookenv import (
-    config,
-    charm_dir,
-)
-
-CLOUD_ARCHIVE_URL = "http://ubuntu-cloud.archive.canonical.com/ubuntu"
-CLOUD_ARCHIVE_KEY_ID = '5EDB1B62EC4926EA'
-
-ubuntu_openstack_release = {
-    'oneiric': 'diablo',
-    'precise': 'essex',
-    'quantal': 'folsom',
-    'raring': 'grizzly',
-}
-
-
-openstack_codenames = {
-    '2011.2': 'diablo',
-    '2012.1': 'essex',
-    '2012.2': 'folsom',
-    '2013.1': 'grizzly',
-    '2013.2': 'havana',
-}
-
-# The ugly duckling
-swift_codenames = {
-    '1.4.3': 'diablo',
-    '1.4.8': 'essex',
-    '1.7.4': 'folsom',
-    '1.7.6': 'grizzly',
-    '1.7.7': 'grizzly',
-    '1.8.0': 'grizzly',
-}
-
-
-def juju_log(msg):
-    subprocess.check_call(['juju-log', msg])
-
-
-def error_out(msg):
-    juju_log("FATAL ERROR: %s" % msg)
-    sys.exit(1)
-
-
-def lsb_release():
-    '''Return /etc/lsb-release in a dict'''
-    lsb = open('/etc/lsb-release', 'r')
-    d = {}
-    for l in lsb:
-        k, v = l.split('=')
-        d[k.strip()] = v.strip()
-    return d
-
-
-def get_os_codename_install_source(src):
-    '''Derive OpenStack release codename from a given installation source.'''
-    ubuntu_rel = lsb_release()['DISTRIB_CODENAME']
-
-    rel = ''
-    if src == 'distro':
-        try:
-            rel = ubuntu_openstack_release[ubuntu_rel]
-        except KeyError:
-            e = 'Could not derive openstack release for '\
-                'this Ubuntu release: %s' % ubuntu_rel
-            error_out(e)
-        return rel
-
-    if src.startswith('cloud:'):
-        ca_rel = src.split(':')[1]
-        ca_rel = ca_rel.split('%s-' % ubuntu_rel)[1].split('/')[0]
-        return ca_rel
-
-    # Best guess match based on deb string provided
-    if src.startswith('deb') or src.startswith('ppa'):
-        for k, v in openstack_codenames.iteritems():
-            if v in src:
-                return v
-
-
-def get_os_version_install_source(src):
-    codename = get_os_codename_install_source(src)
-    return get_os_version_codename(codename)
-
-
-def get_os_codename_version(vers):
-    '''Determine OpenStack codename from version number.'''
-    try:
-        return openstack_codenames[vers]
-    except KeyError:
-        e = 'Could not determine OpenStack codename for version %s' % vers
-        error_out(e)
-
-
-def get_os_version_codename(codename):
-    '''Determine OpenStack version number from codename.'''
-    for k, v in openstack_codenames.iteritems():
-        if v == codename:
-            return k
-    e = 'Could not derive OpenStack version for '\
-        'codename: %s' % codename
-    error_out(e)
-
-
-def get_os_codename_package(pkg, fatal=True):
-    '''Derive OpenStack release codename from an installed package.'''
-    apt.init()
-    cache = apt.Cache()
-
-    try:
-        pkg = cache[pkg]
-    except:
-        if not fatal:
-            return None
-        e = 'Could not determine version of installed package: %s' % pkg
-        error_out(e)
-
-    if pkg.current_ver != None:
-        vers = apt.UpstreamVersion(pkg.current_ver.ver_str)
-    else:
-        return None
-
-    try:
-        if 'swift' in pkg.name:
-            vers = vers[:5]
-            return swift_codenames[vers]
-        else:
-            vers = vers[:6]
-            return openstack_codenames[vers]
-    except KeyError:
-        e = 'Could not determine OpenStack codename for version %s' % vers
-        error_out(e)
-
-
-def get_os_version_package(pkg, fatal=True):
-    '''Derive OpenStack version number from an installed package.'''
-    codename = get_os_codename_package(pkg, fatal=fatal)
-
-    if not codename:
-        return None
-
-    if 'swift' in pkg:
-        vers_map = swift_codenames
-    else:
-        vers_map = openstack_codenames
-
-    for version, cname in vers_map.iteritems():
-        if cname == codename:
-            return version
-    #e = "Could not determine OpenStack version for package: %s" % pkg
-    #error_out(e)
-
-
-def import_key(keyid):
-    cmd = "apt-key adv --keyserver keyserver.ubuntu.com " \
-          "--recv-keys %s" % keyid
-    try:
-        subprocess.check_call(cmd.split(' '))
-    except subprocess.CalledProcessError:
-        error_out("Error importing repo key %s" % keyid)
-
-
-def configure_installation_source(rel):
-    '''Configure apt installation source.'''
-    if rel == 'distro':
-        return
-    elif rel[:4] == "ppa:":
-        src = rel
-        subprocess.check_call(["add-apt-repository", "-y", src])
-    elif rel[:3] == "deb":
-        l = len(rel.split('|'))
-        if l == 2:
-            src, key = rel.split('|')
-            juju_log("Importing PPA key from keyserver for %s" % src)
-            import_key(key)
-        elif l == 1:
-            src = rel
-        with open('/etc/apt/sources.list.d/juju_deb.list', 'w') as f:
-            f.write(src)
-    elif rel[:6] == 'cloud:':
-        ubuntu_rel = lsb_release()['DISTRIB_CODENAME']
-        rel = rel.split(':')[1]
-        u_rel = rel.split('-')[0]
-        ca_rel = rel.split('-')[1]
-
-        if u_rel != ubuntu_rel:
-            e = 'Cannot install from Cloud Archive pocket %s on this Ubuntu '\
-                'version (%s)' % (ca_rel, ubuntu_rel)
-            error_out(e)
-
-        if 'staging' in ca_rel:
-            # staging is just a regular PPA.
-            os_rel = ca_rel.split('/')[0]
-            ppa = 'ppa:ubuntu-cloud-archive/%s-staging' % os_rel
-            cmd = 'add-apt-repository -y %s' % ppa
-            subprocess.check_call(cmd.split(' '))
-            return
-
-        # map charm config options to actual archive pockets.
-        pockets = {
-            'folsom': 'precise-updates/folsom',
-            'folsom/updates': 'precise-updates/folsom',
-            'folsom/proposed': 'precise-proposed/folsom',
-            'grizzly': 'precise-updates/grizzly',
-            'grizzly/updates': 'precise-updates/grizzly',
-            'grizzly/proposed': 'precise-proposed/grizzly'
-        }
-
-        try:
-            pocket = pockets[ca_rel]
-        except KeyError:
-            e = 'Invalid Cloud Archive release specified: %s' % rel
-            error_out(e)
-
-        src = "deb %s %s main" % (CLOUD_ARCHIVE_URL, pocket)
-        # TODO: Replace key import with cloud archive keyring pkg.
-        import_key(CLOUD_ARCHIVE_KEY_ID)
-
-        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as f:
-            f.write(src)
-    else:
-        error_out("Invalid openstack-release specified: %s" % rel)
-
-
-def save_script_rc(script_path="scripts/scriptrc", **env_vars):
-    """
-    Write an rc file in the charm-delivered directory containing
-    exported environment variables provided by env_vars. Any charm scripts run
-    outside the juju hook environment can source this scriptrc to obtain
-    updated config information necessary to perform health checks or
-    service changes.
-    """
-    unit_name = os.getenv('JUJU_UNIT_NAME').replace('/', '-')
-    juju_rc_path = "%s/%s" % (charm_dir(), script_path)
-    if not os.path.exists(os.path.dirname(juju_rc_path)):
-        os.mkdir(os.path.dirname(juju_rc_path))
-    with open(juju_rc_path, 'wb') as rc_script:
-        rc_script.write(
-            "#!/bin/bash\n")
-        [rc_script.write('export %s=%s\n' % (u, p))
-         for u, p in env_vars.iteritems() if u != "script_path"]
-
-
-def openstack_upgrade_available(package):
-    """
-    Determines if an OpenStack upgrade is available from installation
-    source, based on version of installed package.
-
-    :param package: str: Name of installed package.
-
-    :returns: bool:    : Returns True if configured installation source offers
-                         a newer version of package.
-
-    """
-
-    src = config('openstack-origin')
-    cur_vers = get_os_version_package(package)
-    available_vers = get_os_version_install_source(src)
-    return StrictVersion(available_vers) > StrictVersion(cur_vers)
 
=== modified file 'hooks/charmhelpers/contrib/openstack/templating.py'
--- hooks/charmhelpers/contrib/openstack/templating.py	2013-07-05 19:12:08 +0000
+++ hooks/charmhelpers/contrib/openstack/templating.py	2013-08-14 22:57:15 +0000
@@ -1,85 +1,20 @@
-import logging
 import os
 
+from charmhelpers.core.host import apt_install
+
+from charmhelpers.core.hookenv import (
+    log,
+    ERROR,
+    INFO
+)
+
+from charmhelpers.contrib.openstack.utils import OPENSTACK_CODENAMES
+
 try:
-    import jinja2
+    from jinja2 import FileSystemLoader, ChoiceLoader, Environment
 except ImportError:
-    pass
-
-
-logging.basicConfig(level=logging.INFO)
-
-"""
-
-WIP Abstract templating system for the OpenStack charms.
-
-The idea is that an openstack charm can register a number of config files
-associated with common context generators.  The context generators are
-responsible for inspecting charm config/relation data/deployment state
-and presenting correct context to the template. Generic context generators
-could live somewhere in charmhelpers.contrib.openstack, and each
-charm can implement their own specific ones as well.
-
-Ideally a charm would register all its config files somewhere in its namespace,
-eg cinder_utils.py:
-
-from charmhelpers.contrib.openstack import templating, context
-
-config_files = {
-    '/etc/cinder/cinder.conf': [context.shared_db,
-                                context.amqp,
-                                context.ceph],
-    '/etc/cinder/api-paste.ini': [context.identity_service]
-}
-
-configs = templating.OSConfigRenderer(template_dir='templates/')
-
-[configs.register(k, v) for k, v in config_files.iteritems()]
-
-Hooks can then render config files as need, eg:
-
-def config_changed():
-    configs.render_all()
-
-def db_changed():
-    configs.render('/etc/cinder/cinder.conf')
-    check_call(['cinder-manage', 'db', 'sync'])
-
-This would look very similar for nova/glance/etc.
-
-
-The OSTemplteLoader is responsible for creating a jinja2.ChoiceLoader that
-should help reduce fragmentation of a charms' templates across OpenStack
-releases, so we do not need to maintain many copies of templates or juggle
-symlinks. The constructed loader lets the template be loaded from the most
-recent OS release-specific template dir or a base template dir.
-
-For example, say cinder has no changes in config structure across any OS
-releases, all OS releases share the same templates from the base directory:
-
-
-templates/api-paste.ini
-templates/cinder.conf
-
-Then, since Grizzly and beyond, cinder.conf's format has changed:
-
-templates/api-paste.ini
-templates/cinder.conf
-templates/grizzly/cinder.conf
-
-
-Grizzly and beyond will load from templates/grizzly, but any release prior will
-load from templates/.   If some change in Icehouse breaks config format again:
-
-templates/api-paste.ini
-templates/cinder.conf
-templates/grizzly/cinder.conf
-templates/icehouse/cinder.conf
-
-Icehouse and beyond will load from icehouse/, Grizzly + Havan from grizzly/,
-previous releases from the base templates/
-
-"""
+    # python-jinja2 may not be installed yet, or we're running unittests.
+    FileSystemLoader = ChoiceLoader = Environment = None
 
 
 class OSConfigException(Exception):
@@ -107,36 +42,36 @@
                           jinja2.FilesystemLoaders, ordered in descending
                           order by OpenStack release.
     """
-    tmpl_dirs = (
-        ('essex', os.path.join(templates_dir, 'essex')),
-        ('folsom', os.path.join(templates_dir, 'folsom')),
-        ('grizzly', os.path.join(templates_dir, 'grizzly')),
-        ('havana', os.path.join(templates_dir, 'havana')),
-        ('icehouse', os.path.join(templates_dir, 'icehouse')),
-    )
+    tmpl_dirs = [(rel, os.path.join(templates_dir, rel))
+                 for rel in OPENSTACK_CODENAMES.itervalues()]
 
     if not os.path.isdir(templates_dir):
-        logging.error('Templates directory not found @ %s.' % templates_dir)
+        log('Templates directory not found @ %s.' % templates_dir,
+            level=ERROR)
         raise OSConfigException
 
     # the bottom contains tempaltes_dir and possibly a common templates dir
     # shipped with the helper.
-    loaders = [jinja2.FileSystemLoader(templates_dir)]
+    loaders = [FileSystemLoader(templates_dir)]
     helper_templates = os.path.join(os.path.dirname(__file__), 'templates')
     if os.path.isdir(helper_templates):
-        loaders.append(jinja2.FileSystemLoader(helper_templates))
+        loaders.append(FileSystemLoader(helper_templates))
 
     for rel, tmpl_dir in tmpl_dirs:
         if os.path.isdir(tmpl_dir):
-            loaders.insert(0, jinja2.FileSystemLoader(tmpl_dir))
+            loaders.insert(0, FileSystemLoader(tmpl_dir))
         if rel == os_release:
             break
-    logging.info('Creating choice loader with dirs: %s' %
-                 [l.searchpath for l in loaders])
-    return jinja2.ChoiceLoader(loaders)
+    log('Creating choice loader with dirs: %s' %
+        [l.searchpath for l in loaders], level=INFO)
+    return ChoiceLoader(loaders)
 
 
 class OSConfigTemplate(object):
+    """
+    Associates a config file template with a list of context generators.
+    Responsible for constructing a template context based on those generators.
+    """
     def __init__(self, config_file, contexts):
         self.config_file = config_file
 
@@ -170,46 +105,141 @@
 
 
 class OSConfigRenderer(object):
+    """
+    This class provides a common templating system to be used by OpenStack
+    charms.  It is intended to help charms share common code and templates,
+    and ease the burden of managing config templates across multiple OpenStack
+    releases.
+
+    Basic usage:
+        # import some common context generates from charmhelpers
+        from charmhelpers.contrib.openstack import context
+
+        # Create a renderer object for a specific OS release.
+        configs = OSConfigRenderer(templates_dir='/tmp/templates',
+                                   openstack_release='folsom')
+        # register some config files with context generators.
+        configs.register(config_file='/etc/nova/nova.conf',
+                         contexts=[context.SharedDBContext(),
+                                   context.AMQPContext()])
+        configs.register(config_file='/etc/nova/api-paste.ini',
+                         contexts=[context.IdentityServiceContext()])
+        configs.register(config_file='/etc/haproxy/haproxy.conf',
+                         contexts=[context.HAProxyContext()])
+        # write out a single config
+        configs.write('/etc/nova/nova.conf')
+        # write out all registered configs
+        configs.write_all()
+
+    Details:
+
+    OpenStack Releases and template loading
+    ---------------------------------------
+    When the object is instantiated, it is associated with a specific OS
+    release.  This dictates how the template loader will be constructed.
+
+    The constructed loader attempts to load the template from several places
+    in the following order:
+        - from the most recent OS release-specific template dir (if one exists)
+        - the base templates_dir
+        - a template directory shipped in the charm with this helper file.
+
+
+    For the example above, '/tmp/templates' contains the following structure:
+        /tmp/templates/nova.conf
+        /tmp/templates/api-paste.ini
+        /tmp/templates/grizzly/api-paste.ini
+        /tmp/templates/havana/api-paste.ini
+
+    Since it was registered with the grizzly release, it first seraches
+    the grizzly directory for nova.conf, then the templates dir.
+
+    When writing api-paste.ini, it will find the template in the grizzly
+    directory.
+
+    If the object were created with folsom, it would fall back to the
+    base templates dir for its api-paste.ini template.
+
+    This system should help manage changes in config files through
+    openstack releases, allowing charms to fall back to the most recently
+    updated config template for a given release
+
+    The haproxy.conf, since it is not shipped in the templates dir, will
+    be loaded from the module directory's template directory, eg
+    $CHARM/hooks/charmhelpers/contrib/openstack/templates.  This allows
+    us to ship common templates (haproxy, apache) with the helpers.
+
+    Context generators
+    ---------------------------------------
+    Context generators are used to generate template contexts during hook
+    execution.  Doing so may require inspecting service relations, charm
+    config, etc.  When registered, a config file is associated with a list
+    of generators.  When a template is rendered and written, all context
+    generates are called in a chain to generate the context dictionary
+    passed to the jinja2 template. See context.py for more info.
+    """
     def __init__(self, templates_dir, openstack_release):
         if not os.path.isdir(templates_dir):
-            logging.error('Could not locate templates dir %s' % templates_dir)
+            log('Could not locate templates dir %s' % templates_dir,
+                level=ERROR)
             raise OSConfigException
+
         self.templates_dir = templates_dir
         self.openstack_release = openstack_release
         self.templates = {}
         self._tmpl_env = None
 
+        if None in [Environment, ChoiceLoader, FileSystemLoader]:
+            # if this code is running, the object is created pre-install hook.
+            # jinja2 shouldn't get touched until the module is reloaded on next
+            # hook execution, with proper jinja2 bits successfully imported.
+            apt_install('python-jinja2')
+
     def register(self, config_file, contexts):
+        """
+        Register a config file with a list of context generators to be called
+        during rendering.
+        """
         self.templates[config_file] = OSConfigTemplate(config_file=config_file,
                                                        contexts=contexts)
-        logging.info('Registered config file: %s' % config_file)
+        log('Registered config file: %s' % config_file, level=INFO)
 
     def _get_tmpl_env(self):
         if not self._tmpl_env:
             loader = get_loader(self.templates_dir, self.openstack_release)
-            self._tmpl_env = jinja2.Environment(loader=loader)
+            self._tmpl_env = Environment(loader=loader)
+
+    def _get_template(self, template):
+        self._get_tmpl_env()
+        template = self._tmpl_env.get_template(template)
+        log('Loaded template from %s' % template.filename, level=INFO)
+        return template
 
     def render(self, config_file):
         if config_file not in self.templates:
-            logging.error('Config not registered: %s' % config_file)
+            log('Config not registered: %s' % config_file, level=ERROR)
             raise OSConfigException
         ctxt = self.templates[config_file].context()
         _tmpl = os.path.basename(config_file)
-        logging.info('Rendering from template: %s' % _tmpl)
-        self._get_tmpl_env()
-        _tmpl = self._tmpl_env.get_template(_tmpl)
-        logging.info('Loaded template from %s' % _tmpl.filename)
-        return _tmpl.render(ctxt)
+        log('Rendering from template: %s' % _tmpl, level=INFO)
+        template = self._get_template(_tmpl)
+        return template.render(ctxt)
 
     def write(self, config_file):
+        """
+        Write a single config file, raises if config file is not registered.
+        """
         if config_file not in self.templates:
-            logging.error('Config not registered: %s' % config_file)
+            log('Config not registered: %s' % config_file, level=ERROR)
             raise OSConfigException
         with open(config_file, 'wb') as out:
             out.write(self.render(config_file))
-        logging.info('Wrote template %s.' % config_file)
+        log('Wrote template %s.' % config_file, level=INFO)
 
     def write_all(self):
+        """
+        Write out all registered config files.
+        """
         [self.write(k) for k in self.templates.iterkeys()]
 
     def set_release(self, openstack_release):
 
=== added file 'hooks/charmhelpers/contrib/openstack/utils.py'
--- hooks/charmhelpers/contrib/openstack/utils.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/openstack/utils.py	2013-08-14 22:57:15 +0000
@@ -0,0 +1,273 @@
+#!/usr/bin/python
+
+# Common python helper functions used for OpenStack charms.
+
+from collections import OrderedDict
+
+import apt_pkg as apt
+import subprocess
+import os
+import sys
+
+from charmhelpers.core.hookenv import (
+    config,
+    log as juju_log,
+    charm_dir,
+)
+
+from charmhelpers.core.host import (
+    lsb_release,
+    apt_install,
+)
+
+CLOUD_ARCHIVE_URL = "http://ubuntu-cloud.archive.canonical.com/ubuntu"
+CLOUD_ARCHIVE_KEY_ID = '5EDB1B62EC4926EA'
+
+UBUNTU_OPENSTACK_RELEASE = OrderedDict([
+    ('oneiric', 'diablo'),
+    ('precise', 'essex'),
+    ('quantal', 'folsom'),
+    ('raring', 'grizzly'),
+    ('saucy', 'havana'),
+])
+
+
+OPENSTACK_CODENAMES = OrderedDict([
+    ('2011.2', 'diablo'),
+    ('2012.1', 'essex'),
+    ('2012.2', 'folsom'),
+    ('2013.1', 'grizzly'),
+    ('2013.2', 'havana'),
+    ('2014.1', 'icehouse'),
+])
+
+# The ugly duckling
+SWIFT_CODENAMES = {
+    '1.4.3': 'diablo',
+    '1.4.8': 'essex',
+    '1.7.4': 'folsom',
+    '1.7.6': 'grizzly',
+    '1.7.7': 'grizzly',
+    '1.8.0': 'grizzly',
+    '1.9.0': 'havana',
+    '1.9.1': 'havana',
+}
+
+
+def error_out(msg):
+    juju_log("FATAL ERROR: %s" % msg, level='ERROR')
+    sys.exit(1)
+
+
+def get_os_codename_install_source(src):
+    '''Derive OpenStack release codename from a given installation source.'''
+    ubuntu_rel = lsb_release()['DISTRIB_CODENAME']
+    rel = ''
+    if src == 'distro':
+        try:
+            rel = UBUNTU_OPENSTACK_RELEASE[ubuntu_rel]
+        except KeyError:
+            e = 'Could not derive openstack release for '\
+                'this Ubuntu release: %s' % ubuntu_rel
+            error_out(e)
+        return rel
+
+    if src.startswith('cloud:'):
+        ca_rel = src.split(':')[1]
+        ca_rel = ca_rel.split('%s-' % ubuntu_rel)[1].split('/')[0]
+        return ca_rel
+
+    # Best guess match based on deb string provided
+    if src.startswith('deb') or src.startswith('ppa'):
+        for k, v in OPENSTACK_CODENAMES.iteritems():
+            if v in src:
+                return v
+
+
+def get_os_version_install_source(src):
+    codename = get_os_codename_install_source(src)
+    return get_os_version_codename(codename)
+
+
+def get_os_codename_version(vers):
+    '''Determine OpenStack codename from version number.'''
+    try:
+        return OPENSTACK_CODENAMES[vers]
+    except KeyError:
+        e = 'Could not determine OpenStack codename for version %s' % vers
+        error_out(e)
+
+
+def get_os_version_codename(codename):
+    '''Determine OpenStack version number from codename.'''
+    for k, v in OPENSTACK_CODENAMES.iteritems():
+        if v == codename:
+            return k
+    e = 'Could not derive OpenStack version for '\
+        'codename: %s' % codename
+    error_out(e)
+
+
+def get_os_codename_package(package, fatal=True):
+    '''Derive OpenStack release codename from an installed package.'''
+    apt.init()
+    cache = apt.Cache()
+
+    try:
+        pkg = cache[package]
+    except:
+        if not fatal:
+            return None
+        # the package is unknown to the current apt cache.
+        e = 'Could not determine version of package with no installation '\
+            'candidate: %s' % package
+        error_out(e)
+
+    if not pkg.current_ver:
+        if not fatal:
+            return None
+        # package is known, but no version is currently installed.
+        e = 'Could not determine version of uninstalled package: %s' % package
+        error_out(e)
+
+    vers = apt.UpstreamVersion(pkg.current_ver.ver_str)
+
+    try:
+        if 'swift' in pkg.name:
+            vers = vers[:5]
+            return SWIFT_CODENAMES[vers]
+        else:
+            vers = vers[:6]
+            return OPENSTACK_CODENAMES[vers]
+    except KeyError:
+        e = 'Could not determine OpenStack codename for version %s' % vers
+        error_out(e)
+
+
+def get_os_version_package(pkg, fatal=True):
+    '''Derive OpenStack version number from an installed package.'''
+    codename = get_os_codename_package(pkg, fatal=fatal)
+
+    if not codename:
+        return None
+
+    if 'swift' in pkg:
+        vers_map = SWIFT_CODENAMES
+    else:
+        vers_map = OPENSTACK_CODENAMES
+
+    for version, cname in vers_map.iteritems():
+        if cname == codename:
+            return version
+    #e = "Could not determine OpenStack version for package: %s" % pkg
+    #error_out(e)
+
+
+def import_key(keyid):
+    cmd = "apt-key adv --keyserver keyserver.ubuntu.com " \
+          "--recv-keys %s" % keyid
+    try:
+        subprocess.check_call(cmd.split(' '))
+    except subprocess.CalledProcessError:
+        error_out("Error importing repo key %s" % keyid)
+
+
+def configure_installation_source(rel):
+    '''Configure apt installation source.'''
+    if rel == 'distro':
+        return
+    elif rel[:4] == "ppa:":
+        src = rel
+        subprocess.check_call(["add-apt-repository", "-y", src])
+    elif rel[:3] == "deb":
+        l = len(rel.split('|'))
+        if l == 2:
+            src, key = rel.split('|')
+            juju_log("Importing PPA key from keyserver for %s" % src)
+            import_key(key)
+        elif l == 1:
+            src = rel
+        with open('/etc/apt/sources.list.d/juju_deb.list', 'w') as f:
+            f.write(src)
+    elif rel[:6] == 'cloud:':
+        ubuntu_rel = lsb_release()['DISTRIB_CODENAME']
+        rel = rel.split(':')[1]
+        u_rel = rel.split('-')[0]
+        ca_rel = rel.split('-')[1]
+
+        if u_rel != ubuntu_rel:
+            e = 'Cannot install from Cloud Archive pocket %s on this Ubuntu '\
+                'version (%s)' % (ca_rel, ubuntu_rel)
+            error_out(e)
+
+        if 'staging' in ca_rel:
+            # staging is just a regular PPA.
+            os_rel = ca_rel.split('/')[0]
+            ppa = 'ppa:ubuntu-cloud-archive/%s-staging' % os_rel
+            cmd = 'add-apt-repository -y %s' % ppa
+            subprocess.check_call(cmd.split(' '))
+            return
+
+        # map charm config options to actual archive pockets.
+        pockets = {
+            'folsom': 'precise-updates/folsom',
+            'folsom/updates': 'precise-updates/folsom',
+            'folsom/proposed': 'precise-proposed/folsom',
+            'grizzly': 'precise-updates/grizzly',
+            'grizzly/updates': 'precise-updates/grizzly',
+            'grizzly/proposed': 'precise-proposed/grizzly',
+            'havana': 'precise-updates/havana',
+            'havana/updates': 'precise-updates/havana',
+            'havana/proposed': 'precise-proposed/havana',
+        }
+
+        try:
+            pocket = pockets[ca_rel]
+        except KeyError:
+            e = 'Invalid Cloud Archive release specified: %s' % rel
+            error_out(e)
+
+        src = "deb %s %s main" % (CLOUD_ARCHIVE_URL, pocket)
+        apt_install('ubuntu-cloud-keyring', fatal=True)
+
+        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as f:
+            f.write(src)
+    else:
+        error_out("Invalid openstack-release specified: %s" % rel)
+
+
+def save_script_rc(script_path="scripts/scriptrc", **env_vars):
+    """
+    Write an rc file in the charm-delivered directory containing
+    exported environment variables provided by env_vars. Any charm scripts run
+    outside the juju hook environment can source this scriptrc to obtain
+    updated config information necessary to perform health checks or
+    service changes.
+    """
+    juju_rc_path = "%s/%s" % (charm_dir(), script_path)
+    if not os.path.exists(os.path.dirname(juju_rc_path)):
+        os.mkdir(os.path.dirname(juju_rc_path))
+    with open(juju_rc_path, 'wb') as rc_script:
+        rc_script.write(
+            "#!/bin/bash\n")
+        [rc_script.write('export %s=%s\n' % (u, p))
+         for u, p in env_vars.iteritems() if u != "script_path"]
+
+
+def openstack_upgrade_available(package):
+    """
+    Determines if an OpenStack upgrade is available from installation
+    source, based on version of installed package.
+
+    :param package: str: Name of installed package.
+
+    :returns: bool:    : Returns True if configured installation source offers
+                         a newer version of package.
+
+    """
+
+    src = config('openstack-origin')
+    cur_vers = get_os_version_package(package)
+    available_vers = get_os_version_install_source(src)
+    apt.init()
+    return apt.version_compare(available_vers, cur_vers) == 1
 
=== modified file 'hooks/charmhelpers/core/hookenv.py'
--- hooks/charmhelpers/core/hookenv.py	2013-07-05 16:17:32 +0000
+++ hooks/charmhelpers/core/hookenv.py	2013-08-14 22:57:15 +0000
@@ -197,7 +197,7 @@
     relid_cmd_line = ['relation-ids', '--format=json']
     if reltype is not None:
         relid_cmd_line.append(reltype)
-        return json.loads(subprocess.check_output(relid_cmd_line))
+        return json.loads(subprocess.check_output(relid_cmd_line)) or []
     return []
 
 
@@ -208,7 +208,7 @@
     units_cmd_line = ['relation-list', '--format=json']
     if relid is not None:
         units_cmd_line.extend(('-r', relid))
-    return json.loads(subprocess.check_output(units_cmd_line))
+    return json.loads(subprocess.check_output(units_cmd_line)) or []
 
 
 @cached
@@ -335,5 +335,6 @@
             return decorated
         return wrapper
 
+
 def charm_dir():
     return os.environ.get('CHARM_DIR')
 
=== modified file 'hooks/charmhelpers/core/host.py'
--- hooks/charmhelpers/core/host.py	2013-07-05 16:17:32 +0000
+++ hooks/charmhelpers/core/host.py	2013-08-14 22:57:15 +0000
@@ -9,12 +9,14 @@
 import os
 import pwd
 import grp
+import random
+import string
 import subprocess
 import hashlib
 
 from collections import OrderedDict
 
-from hookenv import log, execution_environment
+from hookenv import log
 
 
 def service_start(service_name):
@@ -39,6 +41,18 @@
     return subprocess.call(cmd) == 0
 
 
+def service_running(service):
+    try:
+        output = subprocess.check_output(['service', service, 'status'])
+    except subprocess.CalledProcessError:
+        return False
+    else:
+        if ("start/running" in output or "is running" in output):
+            return True
+        else:
+            return False
+
+
 def adduser(username, password=None, shell='/bin/bash', system_user=False):
     """Add a user"""
     try:
@@ -74,36 +88,33 @@
 
 def rsync(from_path, to_path, flags='-r', options=None):
     """Replicate the contents of a path"""
-    context = execution_environment()
     options = options or ['--delete', '--executability']
     cmd = ['/usr/bin/rsync', flags]
     cmd.extend(options)
-    cmd.append(from_path.format(**context))
-    cmd.append(to_path.format(**context))
+    cmd.append(from_path)
+    cmd.append(to_path)
     log(" ".join(cmd))
     return subprocess.check_output(cmd).strip()
 
 
 def symlink(source, destination):
     """Create a symbolic link"""
-    context = execution_environment()
     log("Symlinking {} as {}".format(source, destination))
     cmd = [
         'ln',
         '-sf',
-        source.format(**context),
-        destination.format(**context)
+        source,
+        destination,
     ]
     subprocess.check_call(cmd)
 
 
 def mkdir(path, owner='root', group='root', perms=0555, force=False):
     """Create a directory"""
-    context = execution_environment()
     log("Making dir {} {}:{} {:o}".format(path, owner, group,
                                           perms))
-    uid = pwd.getpwnam(owner.format(**context)).pw_uid
-    gid = grp.getgrnam(group.format(**context)).gr_gid
+    uid = pwd.getpwnam(owner).pw_uid
+    gid = grp.getgrnam(group).gr_gid
     realpath = os.path.abspath(path)
     if os.path.exists(realpath):
         if force and not os.path.isdir(realpath):
@@ -114,28 +125,15 @@
     os.chown(realpath, uid, gid)
 
 
-def write_file(path, fmtstr, owner='root', group='root', perms=0444, **kwargs):
+def write_file(path, content, owner='root', group='root', perms=0444):
     """Create or overwrite a file with the contents of a string"""
-    context = execution_environment()
-    context.update(kwargs)
-    log("Writing file {} {}:{} {:o}".format(path, owner, group,
-        perms))
-    uid = pwd.getpwnam(owner.format(**context)).pw_uid
-    gid = grp.getgrnam(group.format(**context)).gr_gid
-    with open(path.format(**context), 'w') as target:
+    log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
+    uid = pwd.getpwnam(owner).pw_uid
+    gid = grp.getgrnam(group).gr_gid
+    with open(path, 'w') as target:
         os.fchown(target.fileno(), uid, gid)
         os.fchmod(target.fileno(), perms)
-        target.write(fmtstr.format(**context))
-
-
-def render_template_file(source, destination, **kwargs):
-    """Create or overwrite a file using a template"""
-    log("Rendering template {} for {}".format(source,
-        destination))
-    context = execution_environment()
-    with open(source.format(**context), 'r') as template:
-        write_file(destination.format(**context), template.read(),
-                   **kwargs)
+        target.write(content)
 
 
 def filter_installed_packages(packages):
@@ -271,3 +269,15 @@
             k, v = l.split('=')
             d[k.strip()] = v.strip()
     return d
+
+
+def pwgen(length=None):
+    '''Generate a random pasword.'''
+    if length is None:
+        length = random.choice(range(35, 45))
+    alphanumeric_chars = [
+        l for l in (string.letters + string.digits)
+        if l not in 'l0QD1vAEIOUaeiou']
+    random_chars = [
+        random.choice(alphanumeric_chars) for _ in range(length)]
+    return(''.join(random_chars))
 
=== modified file 'hooks/glance_contexts.py'
--- hooks/glance_contexts.py	2013-07-05 21:13:02 +0000
+++ hooks/glance_contexts.py	2013-08-14 22:57:15 +0000
@@ -8,7 +8,7 @@
     ApacheSSLContext as SSLContext,
 )
 
-from charmhelpers.contrib.hahelpers.cluster_utils import (
+from charmhelpers.contrib.hahelpers.cluster import (
     determine_api_port,
     determine_haproxy_port,
 )
 
=== modified file 'hooks/glance_relations.py'
--- hooks/glance_relations.py	2013-07-11 15:07:15 +0000
+++ hooks/glance_relations.py	2013-08-14 22:57:15 +0000
@@ -13,7 +13,6 @@
     PACKAGES,
     SERVICES,
     CHARM,
-    SERVICE_NAME,
     GLANCE_REGISTRY_CONF,
     GLANCE_REGISTRY_PASTE_INI,
     GLANCE_API_CONF,
@@ -22,12 +21,13 @@
     CEPH_CONF, )
 
 from charmhelpers.core.hookenv import (
-    config as charm_conf,
+    config,
     Hooks,
     log as juju_log,
     relation_get,
     relation_set,
     relation_ids,
+    service_name,
     unit_get,
     UnregisteredHookError, )
 
@@ -37,16 +37,14 @@
     apt_update,
     service_stop, )
 
-from charmhelpers.contrib.hahelpers.cluster_utils import (
+from charmhelpers.contrib.hahelpers.cluster import (
     eligible_leader,
     is_clustered, )
 
-from charmhelpers.contrib.openstack.openstack_utils import (
+from charmhelpers.contrib.openstack.utils import (
     configure_installation_source,
     get_os_codename_package,
-    get_os_codename_install_source,
-    get_os_version_codename,
-    save_script_rc,
+    openstack_upgrade_available,
     lsb_release, )
 
 from subprocess import (
@@ -58,13 +56,12 @@
 
 CONFIGS = register_configs()
 
-config = charm_conf()
 
 @hooks.hook('install')
 def install_hook():
     juju_log('Installing glance packages')
 
-    src = config['openstack-origin']
+    src = config('openstack-origin')
     if (lsb_release()['DISTRIB_CODENAME'] == 'precise' and
        src == 'distro'):
         src = 'cloud:precise-folsom'
@@ -82,7 +79,7 @@
 
 @hooks.hook('shared-db-relation-joined')
 def db_joined():
-    relation_set(database=config['database'], username=config['database-user'],
+    relation_set(database=config('database'), username=config('database-user'),
                  hostname=unit_get('private-address'))
 
 
@@ -123,13 +120,13 @@
 
     host = unit_get('private-address')
     if is_clustered():
-        host = config["vip"]
+        host = config("vip")
 
     relation_data = {
-        'glance-api-server': "%s://%s:9292" % (scheme, host), }
+        'glance_api_server': "%s://%s:9292" % (scheme, host), }
 
-    juju_log("%s: image-service_joined: To peer glance-api-server=%s" %
-             (CHARM, relation_data['glance-api-server']))
+    juju_log("%s: image-service_joined: To peer glance_api_server=%s" %
+             (CHARM, relation_data['glance_api_server']))
 
     relation_set(relation_id=relation_id, **relation_data)
 
@@ -164,7 +161,7 @@
         juju_log('ceph relation incomplete. Peer not ready?')
         return
 
-    if not ensure_ceph_keyring(service=SERVICE_NAME):
+    if not ensure_ceph_keyring(service=service_name()):
         juju_log('Could not create ceph keyring: peer not ready?')
         return
 
@@ -172,7 +169,7 @@
     CONFIGS.write(CEPH_CONF)
 
     if eligible_leader(CLUSTER_RES):
-        ensure_ceph_pool(service=SERVICE_NAME)
+        ensure_ceph_pool(service=service_name())
 
 
 @hooks.hook('identity-service-relation-joined')
@@ -187,13 +184,13 @@
 
     host = unit_get('private-address')
     if is_clustered():
-        host = config["vip"]
+        host = config("vip")
 
     url = "%s://%s:9292" % (scheme, host)
 
     relation_data = {
         'service': 'glance',
-        'region': config['region'],
+        'region': config('region'),
         'public_url': url,
         'admin_url': url,
         'internal_url': url, }
@@ -226,25 +223,16 @@
 @hooks.hook('config-changed')
 @restart_on_change(restart_map())
 def config_changed():
-    # Determine whether or not we should do an upgrade, based on whether or not
-    # the version offered in openstack-origin is greater than what is installed
-    install_src = config["openstack-origin"]
-    available = get_os_codename_install_source(install_src)
-    installed = get_os_codename_package("glance-common")
-
-    if (available and
-       get_os_version_codename(available) >
-       get_os_version_codename(installed)):
-        juju_log('%s: Upgrading OpenStack release: %s -> %s' %
-                 (CHARM, installed, available))
+    if openstack_upgrade_available('glance-common'):
+        juju_log('Upgrading OpenStack release')
         do_openstack_upgrade(CONFIGS)
 
     configure_https()
 
-    env_vars = {'OPENSTACK_PORT_MCASTPORT': config["ha-mcastport"],
-                'OPENSTACK_SERVICE_API': "glance-api",
-                'OPENSTACK_SERVICE_REGISTRY': "glance-registry"}
-    save_script_rc(**env_vars)
+    #env_vars = {'OPENSTACK_PORT_MCASTPORT': config("ha-mcastport"),
+    #            'OPENSTACK_SERVICE_API': "glance-api",
+    #            'OPENSTACK_SERVICE_REGISTRY': "glance-registry"}
+    #save_script_rc(**env_vars)
 
 
 @hooks.hook('cluster-relation-changed')
@@ -261,11 +249,11 @@
 
 @hooks.hook('ha-relation-joined')
 def ha_relation_joined():
-    corosync_bindiface = config["ha-bindiface"]
-    corosync_mcastport = config["ha-mcastport"]
-    vip = config["vip"]
-    vip_iface = config["vip_iface"]
-    vip_cidr = config["vip_cidr"]
+    corosync_bindiface = config("ha-bindiface")
+    corosync_mcastport = config("ha-mcastport")
+    vip = config("vip")
+    vip_iface = config("vip_iface")
+    vip_cidr = config("vip_cidr")
 
     #if vip and vip_iface and vip_cidr and \
     #    corosync_bindiface and corosync_mcastport:
@@ -299,7 +287,7 @@
         juju_log('glance subordinate is not fully clustered.')
         return
     if eligible_leader(CLUSTER_RES):
-        host = config["vip"]
+        host = config("vip")
         scheme = "http"
         if 'https' in CONFIGS.complete_contexts():
             scheme = "https"
@@ -309,14 +297,14 @@
         for r_id in relation_ids('identity-service'):
             relation_set(relation_id=r_id,
                          service="glance",
-                         region=config["region"],
+                         region=config("region"),
                          public_url=url,
                          admin_url=url,
                          internal_url=url)
 
         for r_id in relation_ids('image-service'):
             relation_data = {
-                'glance-api-server': url, }
+                'glance_api_server': url, }
             relation_set(relation_id=r_id, **relation_data)
 
 
@@ -333,7 +321,6 @@
     else:
         cmd = ['a2dissite', 'openstack_https_frontend']
         check_call(cmd)
-        return
 
     for r_id in relation_ids('identity-service'):
         keystone_joined(relation_id=r_id)
 
=== modified file 'hooks/glance_utils.py'
--- hooks/glance_utils.py	2013-07-11 14:51:24 +0000
+++ hooks/glance_utils.py	2013-08-14 22:57:15 +0000
@@ -16,24 +16,23 @@
     log as juju_log,
     relation_get,
     relation_ids,
-    related_units,
-    service_name, )
+    related_units, )
 
 from charmhelpers.contrib.openstack import (
     templating,
     context, )
 
-from charmhelpers.contrib.hahelpers.cluster_utils import (
+from charmhelpers.contrib.hahelpers.cluster import (
     eligible_leader,
 )
 
-from charmhelpers.contrib.hahelpers.ceph_utils import (
+from charmhelpers.contrib.hahelpers.ceph import (
     create_keyring as ceph_create_keyring,
     create_pool as ceph_create_pool,
     keyring_path as ceph_keyring_path,
     pool_exists as ceph_pool_exists, )
 
-from charmhelpers.contrib.openstack.openstack_utils import (
+from charmhelpers.contrib.openstack.utils import (
     get_os_codename_install_source,
     get_os_codename_package,
     configure_installation_source, )
@@ -48,7 +47,6 @@
     "glance-api", "glance-registry", ]
 
 CHARM = "glance"
-SERVICE_NAME = service_name()
 
 GLANCE_REGISTRY_CONF = "/etc/glance/glance-registry.conf"
 GLANCE_REGISTRY_PASTE_INI = "/etc/glance/glance-registry-paste.ini"
 
=== added directory 'tests'
=== added file 'tests/__init__.py'
--- tests/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/__init__.py	2013-08-14 22:57:15 +0000
@@ -0,0 +1,3 @@
+import sys
+
+sys.path.append('hooks/')
 
=== added file 'tests/test_glance_relations.py'
--- tests/test_glance_relations.py	1970-01-01 00:00:00 +0000
+++ tests/test_glance_relations.py	2013-08-14 22:57:15 +0000
@@ -0,0 +1,517 @@
+from mock import call, patch, MagicMock
+
+from tests.test_utils import CharmTestCase
+
+import hooks.glance_utils as utils
+
+_reg = utils.register_configs
+_map = utils.restart_map
+
+utils.register_configs = MagicMock()
+utils.restart_map = MagicMock()
+
+import hooks.glance_relations as relations
+
+utils.register_configs = _reg
+utils.restart_map = _map
+
+TO_PATCH = [
+    # charmhelpers.core.hookenv
+    'Hooks',
+    'config',
+    'juju_log',
+    'relation_ids',
+    'relation_set',
+    'relation_get',
+    'service_name',
+    'unit_get',
+    # charmhelpers.core.host
+    'apt_install',
+    'apt_update',
+    'restart_on_change',
+    'service_stop',
+    #charmhelpers.contrib.openstack.utils
+    'configure_installation_source',
+    'get_os_codename_package',
+    'openstack_upgrade_available',
+    # charmhelpers.contrib.hahelpers.cluster_utils
+    'eligible_leader',
+    'is_clustered',
+    # glance_utils
+    'restart_map',
+    'register_configs',
+    'do_openstack_upgrade',
+    'migrate_database',
+    'ensure_ceph_keyring',
+    'ensure_ceph_pool',
+    # other
+    'getstatusoutput',
+    'check_call',
+]
+
+
+class GlanceRelationTests(CharmTestCase):
+    def setUp(self):
+        super(GlanceRelationTests, self).setUp(relations, TO_PATCH)
+        self.config.side_effect = self.test_config.get
+
+    def test_install_hook(self):
+        repo = 'cloud:precise-grizzly'
+        self.test_config.set('openstack-origin', repo)
+        self.service_stop.return_value = True
+        relations.install_hook()
+        self.configure_installation_source.assert_called_with(repo)
+        self.assertTrue(self.apt_update.called)
+        self.apt_install.assert_called_with(['apache2', 'glance', 'python-mysqldb',
+                                             'python-swift', 'python-keystone',
+                                             'uuid', 'haproxy'])
+
+    def test_db_joined(self):
+        self.unit_get.return_value = 'glance.foohost.com'
+        relations.db_joined()
+        self.relation_set.assert_called_with(database='glance', username='glance',
+                                             hostname='glance.foohost.com')
+        self.unit_get.assert_called_with('private-address')
+
+    @patch.object(relations, 'CONFIGS')
+    def test_db_changed_missing_relation_data(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = []
+        relations.db_changed()
+        self.juju_log.assert_called_with(
+            'shared-db relation incomplete. Peer not ready?'
+        )
+
+    def _shared_db_test(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['shared-db']
+        configs.write = MagicMock()
+        relations.db_changed()
+
+    @patch.object(relations, 'CONFIGS')
+    def test_db_changed_no_essex(self, configs):
+        self._shared_db_test(configs)
+        self.assertEquals([call('/etc/glance/glance-registry.conf'),
+                           call('/etc/glance/glance-api.conf')],
+                           configs.write.call_args_list)
+        self.juju_log.assert_called_with(
+            'Cluster leader, performing db sync'
+        )
+        self.migrate_database.assert_called_with()
+
+    @patch.object(relations, 'CONFIGS')
+    def test_db_changed_with_essex_not_setting_version_control(self, configs):
+        self.get_os_codename_package.return_value = "essex"
+        self.getstatusoutput.return_value = (0, "version")
+        self._shared_db_test(configs)
+        self.assertEquals([call('/etc/glance/glance-registry.conf')],
+                           configs.write.call_args_list)
+        self.juju_log.assert_called_with(
+            'Cluster leader, performing db sync'
+        )
+        self.migrate_database.assert_called_with()
+
+    @patch.object(relations, 'CONFIGS')
+    def test_db_changed_with_essex_setting_version_control(self, configs):
+        self.get_os_codename_package.return_value = "essex"
+        self.getstatusoutput.return_value = (1, "version")
+        self._shared_db_test(configs)
+        self.assertEquals([call('/etc/glance/glance-registry.conf')],
+                           configs.write.call_args_list)
+        self.check_call.assert_called_with(
+            ["glance-manage", "version_control", "0"]
+        )
+        self.juju_log.assert_called_with(
+            'Cluster leader, performing db sync'
+        )
+        self.migrate_database.assert_called_with()
+
+    @patch.object(relations, 'CONFIGS')
+    def test_image_service_joined_clustered_with_https(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['https']
+        configs.write = MagicMock()
+        self.unit_get.return_value = 'glance.foohost.com'
+        self.is_clustered.return_value = True
+        self.test_config.set('vip', '10.10.10.10')
+        relations.image_service_joined()
+        self.assertTrue(self.eligible_leader.called)
+        self.unit_get.assert_called_with('private-address')
+        self.relation_set.assert_called_with(relation_id=None,
+                                             glance_api_server="https://10.10.10.10:9292")
+
+    @patch.object(relations, 'CONFIGS')
+    def test_image_service_joined_not_clustered_with_https(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['https']
+        configs.write = MagicMock()
+        self.unit_get.return_value = 'glance.foohost.com'
+        self.is_clustered.return_value = False
+        relations.image_service_joined()
+        self.assertTrue(self.eligible_leader.called)
+        self.unit_get.assert_called_with('private-address')
+        self.relation_set.assert_called_with(relation_id=None,
+                                             glance_api_server="https://glance.foohost.com:9292")
+
+    @patch.object(relations, 'CONFIGS')
+    def test_image_service_joined_clustered_with_http(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['']
+        configs.write = MagicMock()
+        self.unit_get.return_value = 'glance.foohost.com'
+        self.is_clustered.return_value = True
+        self.test_config.set('vip', '10.10.10.10')
+        relations.image_service_joined()
+        self.assertTrue(self.eligible_leader.called)
+        self.unit_get.assert_called_with('private-address')
+        self.relation_set.assert_called_with(relation_id=None,
+                                             glance_api_server="http://10.10.10.10:9292")
+
+    @patch.object(relations, 'CONFIGS')
+    def test_image_service_joined_not_clustered_with_http(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = []
+        configs.write = MagicMock()
+        self.unit_get.return_value = 'glance.foohost.com'
+        self.is_clustered.return_value = False
+        relations.image_service_joined()
+        self.assertTrue(self.eligible_leader.called)
+        self.unit_get.assert_called_with('private-address')
+        self.relation_set.assert_called_with(relation_id=None,
+                                             glance_api_server="http://glance.foohost.com:9292")
+
+    @patch.object(relations, 'CONFIGS')
+    def test_object_store_joined_without_identity_service(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['']
+        configs.write = MagicMock()
+        relations.object_store_joined()
+        self.juju_log.assert_called_with(
+            'Deferring swift stora configuration until '
+            'an identity-service relation exists'
+        )
+
+    @patch.object(relations, 'CONFIGS')
+    def test_object_store_joined_with_identity_service_without_object_store(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['identity-service']
+        configs.write = MagicMock()
+        relations.object_store_joined()
+        self.juju_log.assert_called_with(
+            'swift relation incomplete'
+        )
+
+    @patch.object(relations, 'CONFIGS')
+    def test_object_store_joined_with_identity_service_with_object_store(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['identity-service', 'object-store']
+        configs.write = MagicMock()
+        relations.object_store_joined()
+        self.assertEquals([call('/etc/glance/glance-api.conf')],
+                           configs.write.call_args_list)
+
+    @patch('os.mkdir')
+    @patch('os.path.isdir')
+    def test_ceph_joined(self, isdir, mkdir):
+        isdir.return_value = False
+        relations.ceph_joined()
+        mkdir.assert_called_with('/etc/ceph')
+        self.apt_install.assert_called_with(['ceph-common', 'python-ceph'])
+
+    @patch.object(relations, 'CONFIGS')
+    def test_ceph_changed_missing_relation_data(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = []
+        configs.write = MagicMock()
+        relations.ceph_changed()
+        self.juju_log.assert_called_with(
+            'ceph relation incomplete. Peer not ready?'
+        )
+
+    @patch.object(relations, 'CONFIGS')
+    def test_ceph_changed_no_keyring(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['ceph']
+        configs.write = MagicMock()
+        self.ensure_ceph_keyring.return_value = False
+        relations.ceph_changed()
+        self.juju_log.assert_called_with(
+            'Could not create ceph keyring: peer not ready?'
+        )
+
+    @patch.object(relations, 'CONFIGS')
+    def test_ceph_changed_with_key_and_relation_data(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['ceph']
+        configs.write = MagicMock()
+        self.ensure_ceph_keyring.return_value = True
+        relations.ceph_changed()
+        self.assertEquals([call('/etc/glance/glance-api.conf'),
+                           call('/etc/ceph/ceph.conf')],
+                           configs.write.call_args_list)
+        self.ensure_ceph_pool.assert_called_with(service=self.service_name())
+
+    @patch.object(relations, 'CONFIGS')
+    def test_keystone_joined_not_clustered(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['']
+        configs.write = MagicMock()
+        self.unit_get.return_value = 'glance.foohost.com'
+        self.test_config.set('region', 'FirstRegion')
+        self.is_clustered.return_value = False
+        relations.keystone_joined()
+        self.unit_get.assert_called_with('private-address')
+        self.relation_set.assert_called_with(
+            relation_id=None,
+            service='glance',
+            region='FirstRegion',
+            public_url='http://glance.foohost.com:9292',
+            admin_url='http://glance.foohost.com:9292',
+            internal_url='http://glance.foohost.com:9292',
+        )
+
+    @patch.object(relations, 'CONFIGS')
+    def test_keystone_joined_clustered(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['']
+        configs.write = MagicMock()
+        self.unit_get.return_value = 'glance.foohost.com'
+        self.test_config.set('region', 'FirstRegion')
+        self.test_config.set('vip', '10.10.10.10')
+        self.is_clustered.return_value = True
+        relations.keystone_joined()
+        self.unit_get.assert_called_with('private-address')
+        self.relation_set.assert_called_with(
+            relation_id=None,
+            service='glance',
+            region='FirstRegion',
+            public_url='http://10.10.10.10:9292',
+            admin_url='http://10.10.10.10:9292',
+            internal_url='http://10.10.10.10:9292',
+        )
+
+
+    @patch.object(relations, 'CONFIGS')
+    def test_keystone_joined_not_clustered_with_https(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['https']
+        configs.write = MagicMock()
+        self.unit_get.return_value = 'glance.foohost.com'
+        self.test_config.set('region', 'FirstRegion')
+        self.is_clustered.return_value = False
+        relations.keystone_joined()
+        self.unit_get.assert_called_with('private-address')
+        self.relation_set.assert_called_with(
+            relation_id=None,
+            service='glance',
+            region='FirstRegion',
+            public_url='https://glance.foohost.com:9292',
+            admin_url='https://glance.foohost.com:9292',
+            internal_url='https://glance.foohost.com:9292',
+        )
+
+    @patch.object(relations, 'CONFIGS')
+    def test_keystone_joined_clustered_with_https(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['https']
+        configs.write = MagicMock()
+        self.unit_get.return_value = 'glance.foohost.com'
+        self.test_config.set('region', 'FirstRegion')
+        self.test_config.set('vip', '10.10.10.10')
+        self.is_clustered.return_value = True
+        relations.keystone_joined()
+        self.unit_get.assert_called_with('private-address')
+        self.relation_set.assert_called_with(
+            relation_id=None,
+            service='glance',
+            region='FirstRegion',
+            public_url='https://10.10.10.10:9292',
+            admin_url='https://10.10.10.10:9292',
+            internal_url='https://10.10.10.10:9292',
+        )
+
+    @patch.object(relations, 'configure_https')
+    @patch.object(relations, 'CONFIGS')
+    def test_keystone_changed_no_object_store_relation(self, configs, configure_https):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['identity-service']
+        configs.write = MagicMock()
+        self.relation_ids.return_value = False
+        relations.keystone_changed()
+        self.assertEquals([call('/etc/glance/glance-api.conf'),
+                           call('/etc/glance/glance-registry.conf'),
+                           call('/etc/glance/glance-api-paste.ini'),
+                           call('/etc/glance/glance-registry-paste.ini')],
+                           configs.write.call_args_list)
+        self.assertTrue(configure_https.called)
+
+    @patch.object(relations, 'configure_https')
+    @patch.object(relations, 'object_store_joined')
+    @patch.object(relations, 'CONFIGS')
+    def test_keystone_changed_no_object_store_relation(self, configs, object_store_joined, configure_https):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['identity-service']
+        configs.write = MagicMock()
+        self.relation_ids.return_value = True
+        relations.keystone_changed()
+        self.assertEquals([call('/etc/glance/glance-api.conf'),
+                           call('/etc/glance/glance-registry.conf'),
+                           call('/etc/glance/glance-api-paste.ini'),
+                           call('/etc/glance/glance-registry-paste.ini')],
+                           configs.write.call_args_list)
+        object_store_joined.assert_called_with()
+        self.assertTrue(configure_https.called)
+
+    @patch.object(relations, 'configure_https')
+    def test_config_changed_no_openstack_upgrade(self, configure_https):
+        self.openstack_upgrade_available.return_value = False
+        relations.config_changed()
+        self.assertTrue(configure_https.called)
+
+    @patch.object(relations, 'configure_https')
+    def test_config_changed_with_openstack_upgrade(self, configure_https):
+        self.openstack_upgrade_available.return_value = True
+        relations.config_changed()
+        self.juju_log.assert_called_with(
+            'Upgrading OpenStack release'
+        )
+        self.assertTrue(self.do_openstack_upgrade.called)
+        self.assertTrue(configure_https.called)
+
+    @patch.object(relations, 'CONFIGS')
+    def test_cluster_changed(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['cluster']
+        configs.write = MagicMock()
+        relations.cluster_changed()
+        self.assertEquals([call('/etc/glance/glance-api.conf'),
+                           call('/etc/haproxy/haproxy.cfg')],
+                           configs.write.call_args_list)
+
+    @patch.object(relations, 'cluster_changed')
+    def test_upgrade_charm(self, cluster_changed):
+        relations.upgrade_charm()
+        cluster_changed.assert_called_with()
+
+    def test_ha_relation_joined(self):
+        self.test_config.set('ha-bindiface', 'em0')
+        self.test_config.set('ha-mcastport', '8080')
+        self.test_config.set('vip', '10.10.10.10')
+        self.test_config.set('vip_iface', 'em1')
+        self.test_config.set('vip_cidr', '24')
+        relations.ha_relation_joined()
+        args = {
+            'corosync_bindiface': 'em0',
+            'corosync_mcastport': '8080',
+            'init_services': {'res_glance_haproxy': 'haproxy'},
+            'resources': {'res_glance_vip': 'ocf:heartbeat:IPaddr2',
+                          'res_glance_haproxy': 'lsb:haproxy'},
+            'resource_params': {'res_glance_vip': 'params ip="10.10.10.10" cidr_netmask="24" nic="em1"',
+                                'res_glance_haproxy': 'op monitor interval="5s"'},
+            'clones': {'cl_glance_haproxy': 'res_glance_haproxy'}
+        }
+        self.relation_set.assert_called_with(**args)
+
+    def test_ha_relation_changed_not_clustered(self):
+        self.relation_get.return_value = False
+        relations.ha_relation_changed()
+        self.juju_log.assert_called_with('glance subordinate is not fully clustered.')
+
+    @patch.object(relations, 'CONFIGS')
+    def test_ha_relation_changed_with_https(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['https']
+        configs.write = MagicMock()
+        self.relation_get.return_value = True
+        self.test_config.set('vip', '10.10.10.10')
+        self.test_config.set('region', 'FirstRegion')
+        self.relation_ids.return_value = ['relation-made:0']
+        relations.ha_relation_changed()
+        self.juju_log.assert_called_with('glance: Cluster configured, notifying other services')
+        self.assertEquals([call('identity-service'), call('image-service')],
+                          self.relation_ids.call_args_list)
+        ex = [
+            call(service='glance',
+                 region='FirstRegion',
+                 public_url='https://10.10.10.10:9292',
+                 internal_url='https://10.10.10.10:9292',
+                 relation_id='relation-made:0',
+                 admin_url='https://10.10.10.10:9292'),
+            call(glance_api_server='https://10.10.10.10:9292',
+                 relation_id='relation-made:0')
+        ]
+        self.assertEquals(ex, self.relation_set.call_args_list)
+
+    @patch.object(relations, 'CONFIGS')
+    def test_ha_relation_changed_with_http(self, configs):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['']
+        configs.write = MagicMock()
+        self.relation_get.return_value = True
+        self.test_config.set('vip', '10.10.10.10')
+        self.test_config.set('region', 'FirstRegion')
+        self.relation_ids.return_value = ['relation-made:0']
+        relations.ha_relation_changed()
+        self.juju_log.assert_called_with('glance: Cluster configured, notifying other services')
+        self.assertEquals([call('identity-service'), call('image-service')],
+                          self.relation_ids.call_args_list)
+        ex = [
+            call(service='glance',
+                 region='FirstRegion',
+                 public_url='http://10.10.10.10:9292',
+                 internal_url='http://10.10.10.10:9292',
+                 relation_id='relation-made:0',
+                 admin_url='http://10.10.10.10:9292'),
+            call(glance_api_server='http://10.10.10.10:9292',
+                 relation_id='relation-made:0')
+        ]
+        self.assertEquals(ex, self.relation_set.call_args_list)
+
+    @patch.object(relations, 'keystone_joined')
+    @patch.object(relations, 'CONFIGS')
+    def test_configure_https_enable_with_identity_service(self, configs, keystone_joined):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['https']
+        configs.write = MagicMock()
+        self.relation_ids.return_value = ['identity-service:0']
+        relations.configure_https()
+        cmd = ['a2ensite', 'openstack_https_frontend']
+        self.check_call.assert_called_with(cmd)
+        keystone_joined.assert_called_with(relation_id='identity-service:0')
+
+    @patch.object(relations, 'keystone_joined')
+    @patch.object(relations, 'CONFIGS')
+    def test_configure_https_disable_with_keystone_joined(self, configs, keystone_joined):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['']
+        configs.write = MagicMock()
+        self.relation_ids.return_value = ['identity-service:0']
+        relations.configure_https()
+        cmd = ['a2dissite', 'openstack_https_frontend']
+        self.check_call.assert_called_with(cmd)
+        keystone_joined.assert_called_with(relation_id='identity-service:0')
+
+    @patch.object(relations, 'image_service_joined')
+    @patch.object(relations, 'CONFIGS')
+    def test_configure_https_enable_with_image_service(self, configs, image_service_joined):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['https']
+        configs.write = MagicMock()
+        self.relation_ids.return_value = ['image-service:0']
+        relations.configure_https()
+        cmd = ['a2ensite', 'openstack_https_frontend']
+        self.check_call.assert_called_with(cmd)
+        image_service_joined.assert_called_with(relation_id='image-service:0')
+
+    @patch.object(relations, 'image_service_joined')
+    @patch.object(relations, 'CONFIGS')
+    def test_configure_https_disable_with_image_service(self, configs, image_service_joined):
+        configs.complete_contexts = MagicMock()
+        configs.complete_contexts.return_value = ['']
+        configs.write = MagicMock()
+        self.relation_ids.return_value = ['image-service:0']
+        relations.configure_https()
+        cmd = ['a2dissite', 'openstack_https_frontend']
+        self.check_call.assert_called_with(cmd)
+        image_service_joined.assert_called_with(relation_id='image-service:0')
+
 
=== added file 'tests/test_utils.py'
--- tests/test_utils.py	1970-01-01 00:00:00 +0000
+++ tests/test_utils.py	2013-08-14 22:57:15 +0000
@@ -0,0 +1,118 @@
+import logging
+import unittest
+import os
+import yaml
+
+from contextlib import contextmanager
+from mock import patch, MagicMock
+
+
+def load_config():
+    '''
+    Walk backwords from __file__ looking for config.yaml, load and return the
+    'options' section'
+    '''
+    config = None
+    f = __file__
+    while config is None:
+        d = os.path.dirname(f)
+        if os.path.isfile(os.path.join(d, 'config.yaml')):
+            config = os.path.join(d, 'config.yaml')
+            break
+        f = d
+
+    if not config:
+        logging.error('Could not find config.yaml in any parent directory '
+                      'of %s. ' % file)
+        raise Exception
+
+    return yaml.safe_load(open(config).read())['options']
+
+
+def get_default_config():
+    '''
+    Load default charm config from config.yaml return as a dict.
+    If no default is set in config.yaml, its value is None.
+    '''
+    default_config = {}
+    config = load_config()
+    for k, v in config.iteritems():
+        if 'default' in v:
+            default_config[k] = v['default']
+        else:
+            default_config[k] = None
+    return default_config
+
+
+class CharmTestCase(unittest.TestCase):
+    def setUp(self, obj, patches):
+        super(CharmTestCase, self).setUp()
+        self.patches = patches
+        self.obj = obj
+        self.test_config = TestConfig()
+        self.test_relation = TestRelation()
+        self.patch_all()
+
+    def patch(self, method):
+        _m = patch.object(self.obj, method)
+        mock = _m.start()
+        self.addCleanup(_m.stop)
+        return mock
+
+    def patch_all(self):
+        for method in self.patches:
+            setattr(self, method, self.patch(method))
+
+
+class TestConfig(object):
+    def __init__(self):
+        self.config = get_default_config()
+
+    def get(self, attr=None):
+        if not attr:
+            return self.get_all()
+        try:
+            return self.config[attr]
+        except KeyError:
+            return None
+
+    def get_all(self):
+        return self.config
+
+    def set(self, attr, value):
+            if attr not in self.config:
+                raise KeyError
+            self.config[attr] = value
+
+
+class TestRelation(object):
+    def __init__(self, relation_data={}):
+        self.relation_data = relation_data
+
+    def set(self, relation_data):
+        self.relation_data = relation_data
+
+    def get(self, attr=None, unit=None, rid=None):
+        if attr == None:
+            return self.relation_data
+        elif attr in self.relation_data:
+            return self.relation_data[attr]
+        return None
+
+
+@contextmanager
+def patch_open():
+    '''Patch open() to allow mocking both open() itself and the file that is
+    yielded.
+
+    Yields the mock for "open" and "file", respectively.'''
+    mock_open = MagicMock(spec=open)
+    mock_file = MagicMock(spec=file)
+
+    @contextmanager
+    def stub_open(*args, **kwargs):
+        mock_open(*args, **kwargs)
+        yield mock_file
+
+    with patch('__builtin__.open', stub_open):
+        yield mock_open, mock_file