Merge lp:~gandelman-a/charm-helpers/ha_os_cleanup into lp:charm-helpers

[Adam Gandelman]

Removes use of local helpers from hahelpers (from code thats still in use).
Drops a bunch of unused shell helper code from contrib.openstack.
Avoids installing packages at module load time

This cleans up the hahelpers and openstack helpers a bit. Anything that still relies on localized helper code is likely unused and will be dropped from the helpers once the Openstack templating stuff lands.

[lp:~gandelman-a/charm-helpers/ha_os_cleanup]

52. By Adam Gandelman on 2013-07-10

hahelpers lint cleanup.

[James Page (james-page)]

Hi Adam

Nice work!

Feedback:

1) hookenv.log as juju_log

Nice idea; however some calls have the level/message ordered incorrectly:

            juju_log('INFO', 'Deferring action to CRM leader.')

I found quite a few instances of this.

2) units.install/utils.restart

I found quite a bit of use of these two deprecated helpers; can we switch to using core.host now? Not sure whether that was meant to be in this branch or not.

3) ceph.py/haproxy.py

Is this still in use? Its using utils exclusively.

review: Needs Fixing

[lp:~gandelman-a/charm-helpers/ha_os_cleanup]

53. By Adam Gandelman on 2013-07-11

Fix log vs juju_log usage.

54. By Adam Gandelman on 2013-07-11

Drop unused code from apache.py.

55. By Adam Gandelman on 2013-07-11

Drop haproxy.py, this will now be handled by template contexts.

56. By Adam Gandelman on 2013-07-11

ceph.py: Remove execute() for direct subprocess usage. Usage host.log instead of local juju_log.

57. By Adam Gandelman on 2013-07-11

drop hahelpers.utils. Add some ceph tests for code thats currently used.

58. By Adam Gandelman on 2013-07-11

apache.py: update log usage.

[Adam Gandelman (gandelman-a)]

Hey James- Thanks for looking at this.

Updated:

- Updated logging

- I've dropped utils.py entirely and updated the hahelpers to use common helpers. I've proposed one local helper that doesn't have an comparable core helper (utils.running()). I've proposed the following, and moved it from utils.py to ceph.py in the meantime:

   https://code.launchpad.net/~gandelman-a/charm-helpers/service_running/+merge/174261

- I dropped old code from apache.py and dropped haproxy.py entirely. Added some tests for code in ceph.py thats currently in use by charms that sync from lp:charm-helpers. Theres still some unused stuff in ceph.py that may be worth keeping if we want to move mysql + rabbitmq (and other RBD users) to use charm-helpers.

review: Needs Resubmitting

[James Page (james-page)]

I have some plans to rework the ceph helper into a generic storage helper.

review: Approve

=== removed file 'charmhelpers/contrib/hahelpers/IMPORT'
--- charmhelpers/contrib/hahelpers/IMPORT	2013-05-30 23:26:24 +0000
+++ charmhelpers/contrib/hahelpers/IMPORT	1970-01-01 00:00:00 +0000
@@ -1,7 +0,0 @@
-Source: lp:~openstack-charmers/openstack-charm-helpers/ha-helpers
-
-ha-helpers/lib/apache_utils.py -> charm-helpers/charmhelpers/contrib/openstackhelpers/apache_utils.py
-ha-helpers/lib/cluster_utils.py -> charm-helpers/charmhelpers/contrib/openstackhelpers/cluster_utils.py
-ha-helpers/lib/ceph_utils.py -> charm-helpers/charmhelpers/contrib/openstackhelpers/ceph_utils.py
-ha-helpers/lib/haproxy_utils.py -> charm-helpers/charmhelpers/contrib/openstackhelpers/haproxy_utils.py
-ha-helpers/lib/utils.py -> charm-helpers/charmhelpers/contrib/openstackhelpers/utils.py

=== renamed file 'charmhelpers/contrib/hahelpers/apache_utils.py' => 'charmhelpers/contrib/hahelpers/apache.py'
--- charmhelpers/contrib/hahelpers/apache_utils.py	2013-06-12 08:20:24 +0000
+++ charmhelpers/contrib/hahelpers/apache.py	2013-07-11 19:29:39 +0000
@@ -8,34 +8,24 @@
 #  Adam Gandelman <[e-mail]>
 #
 
-from utils import (
+import subprocess
+
+from charmhelpers.core.hookenv import (
+    config as config_get,
+    relation_get,
     relation_ids,
-    relation_list,
-    relation_get,
-    render_template,
-    juju_log,
-    config_get,
-    install,
-    get_host_ip,
-    restart
-    )
-from cluster_utils import https
-
-import os
-import subprocess
-from base64 import b64decode
-
-APACHE_SITE_DIR = "/etc/apache2/sites-available"
-SITE_TEMPLATE = "apache2_site.tmpl"
-RELOAD_CHECK = "To activate the new configuration"
+    related_units as relation_list,
+    log,
+    INFO,
+)
 
 
 def get_cert():
     cert = config_get('ssl_cert')
     key = config_get('ssl_key')
     if not (cert and key):
-        juju_log('INFO',
-                 "Inspecting identity-service relations for SSL certificate.")
+        log("Inspecting identity-service relations for SSL certificate.",
+            level=INFO)
         cert = key = None
         for r_id in relation_ids('identity-service'):
             for unit in relation_list(r_id):
@@ -50,8 +40,8 @@
 
 def get_ca_cert():
     ca_cert = None
-    juju_log('INFO',
-             "Inspecting identity-service relations for CA SSL certificate.")
+    log("Inspecting identity-service relations for CA SSL certificate.",
+        level=INFO)
     for r_id in relation_ids('identity-service'):
         for unit in relation_list(r_id):
             if not ca_cert:
@@ -66,131 +56,3 @@
                   'w') as crt:
             crt.write(ca_cert)
         subprocess.check_call(['update-ca-certificates', '--fresh'])
-
-
-def enable_https(port_maps, namespace, cert, key, ca_cert=None):
-    '''
-    For a given number of port mappings, configures apache2
-    HTTPs local reverse proxying using certficates and keys provided in
-    either configuration data (preferred) or relation data.  Assumes ports
-    are not in use (calling charm should ensure that).
-
-    port_maps: dict: external to internal port mappings
-    namespace: str: name of charm
-    '''
-    def _write_if_changed(path, new_content):
-        content = None
-        if os.path.exists(path):
-            with open(path, 'r') as f:
-                content = f.read().strip()
-        if content != new_content:
-            with open(path, 'w') as f:
-                f.write(new_content)
-            return True
-        else:
-            return False
-
-    juju_log('INFO', "Enabling HTTPS for port mappings: {}".format(port_maps))
-    http_restart = False
-
-    if cert:
-        cert = b64decode(cert)
-    if key:
-        key = b64decode(key)
-    if ca_cert:
-        ca_cert = b64decode(ca_cert)
-
-    if not cert and not key:
-        juju_log('ERROR',
-                 "Expected but could not find SSL certificate data, not "
-                 "configuring HTTPS!")
-        return False
-
-    install('apache2')
-    if RELOAD_CHECK in subprocess.check_output(['a2enmod', 'ssl',
-                                                'proxy', 'proxy_http']):
-        http_restart = True
-
-    ssl_dir = os.path.join('/etc/apache2/ssl', namespace)
-    if not os.path.exists(ssl_dir):
-        os.makedirs(ssl_dir)
-
-    if (_write_if_changed(os.path.join(ssl_dir, 'cert'), cert)):
-        http_restart = True
-    if (_write_if_changed(os.path.join(ssl_dir, 'key'), key)):
-        http_restart = True
-    os.chmod(os.path.join(ssl_dir, 'key'), 0600)
-
-    install_ca_cert(ca_cert)
-
-    sites_dir = '/etc/apache2/sites-available'
-    for ext_port, int_port in port_maps.items():
-        juju_log('INFO',
-                 'Creating apache2 reverse proxy vhost'
-                 ' for {}:{}'.format(ext_port,
-                                     int_port))
-        site = "{}_{}".format(namespace, ext_port)
-        site_path = os.path.join(sites_dir, site)
-        with open(site_path, 'w') as fsite:
-            context = {
-                "ext": ext_port,
-                "int": int_port,
-                "namespace": namespace,
-                "private_address": get_host_ip()
-                }
-            fsite.write(render_template(SITE_TEMPLATE,
-                                        context))
-
-        if RELOAD_CHECK in subprocess.check_output(['a2ensite', site]):
-            http_restart = True
-
-    if http_restart:
-        restart('apache2')
-
-    return True
-
-
-def disable_https(port_maps, namespace):
-    '''
-    Ensure HTTPS reverse proxying is disables for given port mappings
-
-    port_maps: dict: of ext -> int port mappings
-    namespace: str: name of chamr
-    '''
-    juju_log('INFO', 'Ensuring HTTPS disabled for {}'.format(port_maps))
-
-    if (not os.path.exists('/etc/apache2') or
-        not os.path.exists(os.path.join('/etc/apache2/ssl', namespace))):
-        return
-
-    http_restart = False
-    for ext_port in port_maps.keys():
-        if os.path.exists(os.path.join(APACHE_SITE_DIR,
-                                       "{}_{}".format(namespace,
-                                                      ext_port))):
-            juju_log('INFO',
-                     "Disabling HTTPS reverse proxy"
-                     " for {} {}.".format(namespace,
-                                          ext_port))
-            if (RELOAD_CHECK in
-                subprocess.check_output(['a2dissite',
-                                         '{}_{}'.format(namespace,
-                                                        ext_port)])):
-                http_restart = True
-
-    if http_restart:
-        restart(['apache2'])
-
-
-def setup_https(port_maps, namespace, cert, key, ca_cert=None):
-    '''
-    Ensures HTTPS is either enabled or disabled for given port
-    mapping.
-
-    port_maps: dict: of ext -> int port mappings
-    namespace: str: name of charm
-    '''
-    if not https:
-        disable_https(port_maps, namespace)
-    else:
-        enable_https(port_maps, namespace, cert, key, ca_cert)

=== renamed file 'charmhelpers/contrib/hahelpers/ceph_utils.py' => 'charmhelpers/contrib/hahelpers/ceph.py'
--- charmhelpers/contrib/hahelpers/ceph_utils.py	2013-06-12 08:20:24 +0000
+++ charmhelpers/contrib/hahelpers/ceph.py	2013-07-11 19:29:39 +0000
@@ -9,10 +9,31 @@
 #
 
 import commands
-import subprocess
 import os
 import shutil
-import utils
+
+from subprocess import (
+    check_call,
+    check_output,
+    CalledProcessError
+)
+
+from charmhelpers.core.hookenv import (
+    relation_get,
+    relation_ids,
+    related_units,
+    log,
+    INFO,
+)
+
+from charmhelpers.core.host import (
+    apt_install,
+    mount,
+    mounts,
+    service_start,
+    service_stop,
+    umount,
+)
 
 KEYRING = '/etc/ceph/ceph.client.%s.keyring'
 KEYFILE = '/etc/ceph/ceph.client.%s.key'
@@ -24,23 +45,30 @@
 """
 
 
-def execute(cmd):
-    subprocess.check_call(cmd)
-
-
-def execute_shell(cmd):
-    subprocess.check_call(cmd, shell=True)
+def running(service):
+    # this local util can be dropped as soon the following branch lands
+    # in lp:charm-helpers
+    # https://code.launchpad.net/~gandelman-a/charm-helpers/service_running/
+    try:
+        output = check_output(['service', service, 'status'])
+    except CalledProcessError:
+        return False
+    else:
+        if ("start/running" in output or "is running" in output):
+            return True
+        else:
+            return False
 
 
 def install():
     ceph_dir = "/etc/ceph"
     if not os.path.isdir(ceph_dir):
         os.mkdir(ceph_dir)
-    utils.install('ceph-common')
+    apt_install('ceph-common', fatal=True)
 
 
 def rbd_exists(service, pool, rbd_img):
-    (rc, out) = commands.getstatusoutput('rbd list --id %s --pool %s' %\
+    (rc, out) = commands.getstatusoutput('rbd list --id %s --pool %s' %
                                          (service, pool))
     return rbd_img in out
 
@@ -56,8 +84,8 @@
         service,
         '--pool',
         pool
-        ]
-    execute(cmd)
+    ]
+    check_call(cmd)
 
 
 def pool_exists(service, name):
@@ -72,8 +100,8 @@
         service,
         'mkpool',
         name
-        ]
-    execute(cmd)
+    ]
+    check_call(cmd)
 
 
 def keyfile_path(service):
@@ -87,35 +115,34 @@
 def create_keyring(service, key):
     keyring = keyring_path(service)
     if os.path.exists(keyring):
-        utils.juju_log('INFO', 'ceph: Keyring exists at %s.' % keyring)
+        log('ceph: Keyring exists at %s.' % keyring, level=INFO)
     cmd = [
         'ceph-authtool',
         keyring,
         '--create-keyring',
         '--name=client.%s' % service,
         '--add-key=%s' % key
-        ]
-    execute(cmd)
-    utils.juju_log('INFO', 'ceph: Created new ring at %s.' % keyring)
+    ]
+    check_call(cmd)
+    log('ceph: Created new ring at %s.' % keyring, level=INFO)
 
 
 def create_key_file(service, key):
     # create a file containing the key
     keyfile = keyfile_path(service)
     if os.path.exists(keyfile):
-        utils.juju_log('INFO', 'ceph: Keyfile exists at %s.' % keyfile)
+        log('ceph: Keyfile exists at %s.' % keyfile, level=INFO)
     fd = open(keyfile, 'w')
     fd.write(key)
     fd.close()
-    utils.juju_log('INFO', 'ceph: Created new keyfile at %s.' % keyfile)
+    log('ceph: Created new keyfile at %s.' % keyfile, level=INFO)
 
 
 def get_ceph_nodes():
     hosts = []
-    for r_id in utils.relation_ids('ceph'):
-        for unit in utils.relation_list(r_id):
-            hosts.append(utils.relation_get('private-address',
-                                            unit=unit, rid=r_id))
+    for r_id in relation_ids('ceph'):
+        for unit in related_units(r_id):
+            hosts.append(relation_get('private-address', unit=unit, rid=r_id))
     return hosts
 
 
@@ -144,26 +171,24 @@
         service,
         '--secret',
         keyfile_path(service),
-        ]
-    execute(cmd)
+    ]
+    check_call(cmd)
 
 
 def filesystem_mounted(fs):
-    return subprocess.call(['grep', '-wqs', fs, '/proc/mounts']) == 0
+    return fs in [f for m, f in mounts()]
 
 
 def make_filesystem(blk_device, fstype='ext4'):
-    utils.juju_log('INFO',
-                   'ceph: Formatting block device %s as filesystem %s.' %\
-                   (blk_device, fstype))
+    log('ceph: Formatting block device %s as filesystem %s.' %
+        (blk_device, fstype), level=INFO)
     cmd = ['mkfs', '-t', fstype, blk_device]
-    execute(cmd)
+    check_call(cmd)
 
 
 def place_data_on_ceph(service, blk_device, data_src_dst, fstype='ext4'):
     # mount block device into /mnt
-    cmd = ['mount', '-t', fstype, blk_device, '/mnt']
-    execute(cmd)
+    mount(blk_device, '/mnt')
 
     # copy data to /mnt
     try:
@@ -172,29 +197,27 @@
         pass
 
     # umount block device
-    cmd = ['umount', '/mnt']
-    execute(cmd)
+    umount('/mnt')
 
     _dir = os.stat(data_src_dst)
     uid = _dir.st_uid
     gid = _dir.st_gid
 
     # re-mount where the data should originally be
-    cmd = ['mount', '-t', fstype, blk_device, data_src_dst]
-    execute(cmd)
+    mount(blk_device, data_src_dst, persist=True)
 
     # ensure original ownership of new mount.
     cmd = ['chown', '-R', '%s:%s' % (uid, gid), data_src_dst]
-    execute(cmd)
+    check_call(cmd)
 
 
 # TODO: re-use
 def modprobe_kernel_module(module):
-    utils.juju_log('INFO', 'Loading kernel module')
+    log('ceph: Loading kernel module', level=INFO)
     cmd = ['modprobe', module]
-    execute(cmd)
+    check_call(cmd)
     cmd = 'echo %s >> /etc/modules' % module
-    execute_shell(cmd)
+    check_call(cmd, shell=True)
 
 
 def copy_files(src, dst, symlinks=False, ignore=None):
@@ -222,15 +245,15 @@
     """
     # Ensure pool, RBD image, RBD mappings are in place.
     if not pool_exists(service, pool):
-        utils.juju_log('INFO', 'ceph: Creating new pool %s.' % pool)
+        log('ceph: Creating new pool %s.' % pool, level=INFO)
         create_pool(service, pool)
 
     if not rbd_exists(service, pool, rbd_img):
-        utils.juju_log('INFO', 'ceph: Creating RBD image (%s).' % rbd_img)
+        log('ceph: Creating RBD image (%s).' % rbd_img, level=INFO)
         create_rbd_image(service, pool, rbd_img, sizemb)
 
     if not image_mapped(rbd_img):
-        utils.juju_log('INFO', 'ceph: Mapping RBD Image as a Block Device.')
+        log('ceph: Mapping RBD Image as a Block Device.', level=INFO)
         map_block_storage(service, pool, rbd_img)
 
     # make file system
@@ -244,13 +267,12 @@
         make_filesystem(blk_device, fstype)
 
         for svc in system_services:
-            if utils.running(svc):
-                utils.juju_log('INFO',
-                               'Stopping services %s prior to migrating '\
-                               'data' % svc)
-                utils.stop(svc)
+            if running(svc):
+                log('Stopping services %s prior to migrating data.' % svc,
+                    level=INFO)
+                service_stop(svc)
 
         place_data_on_ceph(service, blk_device, mount_point, fstype)
 
         for svc in system_services:
-            utils.start(svc)
+            service_start(svc)

=== renamed file 'charmhelpers/contrib/hahelpers/cluster_utils.py' => 'charmhelpers/contrib/hahelpers/cluster.py'
--- charmhelpers/contrib/hahelpers/cluster_utils.py	2013-07-09 18:05:11 +0000
+++ charmhelpers/contrib/hahelpers/cluster.py	2013-07-11 19:29:39 +0000
@@ -1,23 +1,25 @@
 #
 # Copyright 2012 Canonical Ltd.
 #
-# This file is sourced from lp:openstack-charm-helpers
-#
 # Authors:
 #  James Page <[e-mail]>
 #  Adam Gandelman <[e-mail]>
 #
 
-from utils import (
-    juju_log,
+import subprocess
+import os
+
+from socket import gethostname as get_unit_hostname
+
+from charmhelpers.core.hookenv import (
+    log,
     relation_ids,
-    relation_list,
+    related_units as relation_list,
     relation_get,
-    get_unit_hostname,
-    config_get
+    config as config_get,
+    INFO,
+    ERROR,
 )
-import subprocess
-import os
 
 
 class HAIncompleteConfig(Exception):
@@ -39,7 +41,7 @@
     cmd = [
         "crm", "resource",
         "show", resource
-        ]
+    ]
     try:
         status = subprocess.check_output(cmd)
     except subprocess.CalledProcessError:
@@ -71,12 +73,12 @@
 def eligible_leader(resource):
     if is_clustered():
         if not is_leader(resource):
-            juju_log('INFO', 'Deferring action to CRM leader.')
+            log('Deferring action to CRM leader.', level=INFO)
             return False
     else:
         peers = peer_units()
         if peers and not oldest_peer(peers):
-            juju_log('INFO', 'Deferring action to oldest service unit.')
+            log('Deferring action to oldest service unit.', level=INFO)
             return False
     return True
 
@@ -153,7 +155,7 @@
     missing = []
     [missing.append(s) for s, v in conf.iteritems() if v is None]
     if missing:
-        juju_log('Insufficient config data to configure hacluster.')
+        log('Insufficient config data to configure hacluster.', level=ERROR)
         raise HAIncompleteConfig
     return conf
 

=== removed file 'charmhelpers/contrib/hahelpers/haproxy_utils.py'
--- charmhelpers/contrib/hahelpers/haproxy_utils.py	2013-06-12 08:20:24 +0000
+++ charmhelpers/contrib/hahelpers/haproxy_utils.py	1970-01-01 00:00:00 +0000
@@ -1,55 +0,0 @@
-#
-# Copyright 2012 Canonical Ltd.
-#
-# This file is sourced from lp:openstack-charm-helpers
-#
-# Authors:
-#  James Page <[e-mail]>
-#  Adam Gandelman <[e-mail]>
-#
-
-from utils import (
-    relation_ids,
-    relation_list,
-    relation_get,
-    unit_get,
-    reload,
-    render_template
-    )
-import os
-
-HAPROXY_CONF = '/etc/haproxy/haproxy.cfg'
-HAPROXY_DEFAULT = '/etc/default/haproxy'
-
-
-def configure_haproxy(service_ports):
-    '''
-    Configure HAProxy based on the current peers in the service
-    cluster using the provided port map:
-
-        "swift": [ 8080, 8070 ]
-
-    HAproxy will also be reloaded/started if required
-
-    service_ports: dict: dict of lists of [ frontend, backend ]
-    '''
-    cluster_hosts = {}
-    cluster_hosts[os.getenv('JUJU_UNIT_NAME').replace('/', '-')] = \
-        unit_get('private-address')
-    for r_id in relation_ids('cluster'):
-        for unit in relation_list(r_id):
-            cluster_hosts[unit.replace('/', '-')] = \
-                relation_get(attribute='private-address',
-                             rid=r_id,
-                             unit=unit)
-    context = {
-        'units': cluster_hosts,
-        'service_ports': service_ports
-        }
-    with open(HAPROXY_CONF, 'w') as f:
-        f.write(render_template(os.path.basename(HAPROXY_CONF),
-                                context))
-    with open(HAPROXY_DEFAULT, 'w') as f:
-        f.write('ENABLED=1')
-
-    reload('haproxy')

=== removed file 'charmhelpers/contrib/hahelpers/utils.py'
--- charmhelpers/contrib/hahelpers/utils.py	2013-06-12 08:20:24 +0000
+++ charmhelpers/contrib/hahelpers/utils.py	1970-01-01 00:00:00 +0000
@@ -1,333 +0,0 @@
-#
-# Copyright 2012 Canonical Ltd.
-#
-# This file is sourced from lp:openstack-charm-helpers
-#
-# Authors:
-#  James Page <[e-mail]>
-#  Paul Collins <[e-mail]>
-#  Adam Gandelman <[e-mail]>
-#
-
-import json
-import os
-import subprocess
-import socket
-import sys
-
-
-def do_hooks(hooks):
-    hook = os.path.basename(sys.argv[0])
-
-    try:
-        hook_func = hooks[hook]
-    except KeyError:
-        juju_log('INFO',
-                 "This charm doesn't know how to handle '{}'.".format(hook))
-    else:
-        hook_func()
-
-
-def install(*pkgs):
-    cmd = [
-        'apt-get',
-        '-y',
-        'install'
-          ]
-    for pkg in pkgs:
-        cmd.append(pkg)
-    subprocess.check_call(cmd)
-
-TEMPLATES_DIR = 'templates'
-
-try:
-    import jinja2
-except ImportError:
-    install('python-jinja2')
-    import jinja2
-
-try:
-    import dns.resolver
-except ImportError:
-    install('python-dnspython')
-    import dns.resolver
-
-
-def render_template(template_name, context, template_dir=TEMPLATES_DIR):
-    templates = jinja2.Environment(
-                    loader=jinja2.FileSystemLoader(template_dir)
-                    )
-    template = templates.get_template(template_name)
-    return template.render(context)
-
-CLOUD_ARCHIVE = \
-""" # Ubuntu Cloud Archive
-deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
-"""
-
-CLOUD_ARCHIVE_POCKETS = {
-    'folsom': 'precise-updates/folsom',
-    'folsom/updates': 'precise-updates/folsom',
-    'folsom/proposed': 'precise-proposed/folsom',
-    'grizzly': 'precise-updates/grizzly',
-    'grizzly/updates': 'precise-updates/grizzly',
-    'grizzly/proposed': 'precise-proposed/grizzly'
-    }
-
-
-def configure_source():
-    source = str(config_get('openstack-origin'))
-    if not source:
-        return
-    if source.startswith('ppa:'):
-        cmd = [
-            'add-apt-repository',
-            source
-            ]
-        subprocess.check_call(cmd)
-    if source.startswith('cloud:'):
-        # CA values should be formatted as cloud:ubuntu-openstack/pocket, eg:
-        #   cloud:precise-folsom/updates or cloud:precise-folsom/proposed
-        install('ubuntu-cloud-keyring')
-        pocket = source.split(':')[1]
-        pocket = pocket.split('-')[1]
-        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
-            apt.write(CLOUD_ARCHIVE.format(CLOUD_ARCHIVE_POCKETS[pocket]))
-    if source.startswith('deb'):
-        l = len(source.split('|'))
-        if l == 2:
-            (apt_line, key) = source.split('|')
-            cmd = [
-                'apt-key',
-                'adv', '--keyserver keyserver.ubuntu.com',
-                '--recv-keys', key
-                ]
-            subprocess.check_call(cmd)
-        elif l == 1:
-            apt_line = source
-
-        with open('/etc/apt/sources.list.d/quantum.list', 'w') as apt:
-            apt.write(apt_line + "\n")
-    cmd = [
-        'apt-get',
-        'update'
-        ]
-    subprocess.check_call(cmd)
-
-# Protocols
-TCP = 'TCP'
-UDP = 'UDP'
-
-
-def expose(port, protocol='TCP'):
-    cmd = [
-        'open-port',
-        '{}/{}'.format(port, protocol)
-        ]
-    subprocess.check_call(cmd)
-
-
-def juju_log(severity, message):
-    cmd = [
-        'juju-log',
-        '--log-level', severity,
-        message
-        ]
-    subprocess.check_call(cmd)
-
-
-cache = {}
-
-
-def cached(func):
-    def wrapper(*args, **kwargs):
-        global cache
-        key = str((func, args, kwargs))
-        try:
-            return cache[key]
-        except KeyError:
-            res = func(*args, **kwargs)
-            cache[key] = res
-            return res
-    return wrapper
-
-
-@cached
-def relation_ids(relation):
-    cmd = [
-        'relation-ids',
-        relation
-        ]
-    result = str(subprocess.check_output(cmd)).split()
-    if result == "":
-        return None
-    else:
-        return result
-
-
-@cached
-def relation_list(rid):
-    cmd = [
-        'relation-list',
-        '-r', rid,
-        ]
-    result = str(subprocess.check_output(cmd)).split()
-    if result == "":
-        return None
-    else:
-        return result
-
-
-@cached
-def relation_get(attribute, unit=None, rid=None):
-    cmd = [
-        'relation-get',
-        ]
-    if rid:
-        cmd.append('-r')
-        cmd.append(rid)
-    cmd.append(attribute)
-    if unit:
-        cmd.append(unit)
-    value = subprocess.check_output(cmd).strip()  # IGNORE:E1103
-    if value == "":
-        return None
-    else:
-        return value
-
-
-@cached
-def relation_get_dict(relation_id=None, remote_unit=None):
-    """Obtain all relation data as dict by way of JSON"""
-    cmd = [
-        'relation-get', '--format=json'
-        ]
-    if relation_id:
-        cmd.append('-r')
-        cmd.append(relation_id)
-    if remote_unit:
-        remote_unit_orig = os.getenv('JUJU_REMOTE_UNIT', None)
-        os.environ['JUJU_REMOTE_UNIT'] = remote_unit
-    j = subprocess.check_output(cmd)
-    if remote_unit and remote_unit_orig:
-        os.environ['JUJU_REMOTE_UNIT'] = remote_unit_orig
-    d = json.loads(j)
-    settings = {}
-    # convert unicode to strings
-    for k, v in d.iteritems():
-        settings[str(k)] = str(v)
-    return settings
-
-
-def relation_set(**kwargs):
-    cmd = [
-        'relation-set'
-        ]
-    args = []
-    for k, v in kwargs.items():
-        if k == 'rid':
-            if v:
-                cmd.append('-r')
-                cmd.append(v)
-        else:
-            args.append('{}={}'.format(k, v))
-    cmd += args
-    subprocess.check_call(cmd)
-
-
-@cached
-def unit_get(attribute):
-    cmd = [
-        'unit-get',
-        attribute
-        ]
-    value = subprocess.check_output(cmd).strip()  # IGNORE:E1103
-    if value == "":
-        return None
-    else:
-        return value
-
-
-@cached
-def config_get(attribute):
-    cmd = [
-        'config-get',
-        '--format',
-        'json',
-        ]
-    out = subprocess.check_output(cmd).strip()  # IGNORE:E1103
-    cfg = json.loads(out)
-
-    try:
-        return cfg[attribute]
-    except KeyError:
-        return None
-
-
-@cached
-def get_unit_hostname():
-    return socket.gethostname()
-
-
-@cached
-def get_host_ip(hostname=None):
-    hostname = hostname or unit_get('private-address')
-    try:
-        # Test to see if already an IPv4 address
-        socket.inet_aton(hostname)
-        return hostname
-    except socket.error:
-        answers = dns.resolver.query(hostname, 'A')
-        if answers:
-            return answers[0].address
-    return None
-
-
-def _svc_control(service, action):
-    subprocess.check_call(['service', service, action])
-
-
-def restart(*services):
-    for service in services:
-        _svc_control(service, 'restart')
-
-
-def stop(*services):
-    for service in services:
-        _svc_control(service, 'stop')
-
-
-def start(*services):
-    for service in services:
-        _svc_control(service, 'start')
-
-
-def reload(*services):
-    for service in services:
-        try:
-            _svc_control(service, 'reload')
-        except subprocess.CalledProcessError:
-            # Reload failed - either service does not support reload
-            # or it was not running - restart will fixup most things
-            _svc_control(service, 'restart')
-
-
-def running(service):
-    try:
-        output = subprocess.check_output(['service', service, 'status'])
-    except subprocess.CalledProcessError:
-        return False
-    else:
-        if ("start/running" in output or
-            "is running" in output):
-            return True
-        else:
-            return False
-
-
-def is_relation_made(relation, key='private-address'):
-    for r_id in (relation_ids(relation) or []):
-        for unit in (relation_list(r_id) or []):
-            if relation_get(key, rid=r_id, unit=unit):
-                return True
-    return False

=== removed file 'charmhelpers/contrib/openstack/IMPORT'
--- charmhelpers/contrib/openstack/IMPORT	2013-05-30 23:26:24 +0000
+++ charmhelpers/contrib/openstack/IMPORT	1970-01-01 00:00:00 +0000
@@ -1,9 +0,0 @@
-Source: lp:~openstack-charmers/openstack-charm-helpers/ha-helpers
-
-ha-helpers/lib/openstack-common -> charm-helpers/charmhelpers/contrib/openstackhelpers/openstack-common
-ha-helpers/lib/openstack_common.py -> charm-helpers/charmhelpers/contrib/openstackhelpers/openstack_common.py
-ha-helpers/lib/nova -> charm-helpers/charmhelpers/contrib/openstackhelpers/nova
-ha-helpers/lib/nova/nova-common -> charm-helpers/charmhelpers/contrib/openstackhelpers/nova/nova-common
-ha-helpers/lib/nova/grizzly -> charm-helpers/charmhelpers/contrib/openstackhelpers/nova/grizzly
-ha-helpers/lib/nova/essex -> charm-helpers/charmhelpers/contrib/openstackhelpers/nova/essex
-ha-helpers/lib/nova/folsom -> charm-helpers/charmhelpers/contrib/openstackhelpers/nova/folsom

=== removed directory 'charmhelpers/contrib/openstack/nova'
=== removed file 'charmhelpers/contrib/openstack/nova/essex'
--- charmhelpers/contrib/openstack/nova/essex	2013-05-30 23:26:24 +0000
+++ charmhelpers/contrib/openstack/nova/essex	1970-01-01 00:00:00 +0000
@@ -1,43 +0,0 @@
-#!/bin/bash -e
-
-# Essex-specific functions
-
-nova_set_or_update() {
-  # Set a config option in nova.conf or api-paste.ini, depending
-  # Defaults to updating nova.conf
-  local key=$1
-  local value=$2
-  local conf_file=$3
-  local pattern=""
-
-  local nova_conf=${NOVA_CONF:-/etc/nova/nova.conf}
-  local api_conf=${API_CONF:-/etc/nova/api-paste.ini}
-  local libvirtd_conf=${LIBVIRTD_CONF:-/etc/libvirt/libvirtd.conf}
-  [[ -z $key ]] && juju-log "$CHARM set_or_update: value $value missing key" && exit 1
-  [[ -z $value ]] && juju-log "$CHARM set_or_update: key $key missing value" && exit 1
-  [[ -z "$conf_file" ]] && conf_file=$nova_conf
-
-  case "$conf_file" in
-    "$nova_conf") match="\-\-$key="
-                  pattern="--$key="
-                  out=$pattern
-                  ;;
-    "$api_conf"|"$libvirtd_conf") match="^$key = "
-                 pattern="$match"
-                 out="$key = "
-                 ;;
-    *) error_out "ERROR: set_or_update: Invalid conf_file ($conf_file)"
-  esac
-
-  cat $conf_file | grep "$match$value" >/dev/null &&
-    juju-log "$CHARM: $key=$value already in set in $conf_file" \
-      && return 0
-  if cat $conf_file | grep "$match" >/dev/null ; then
-    juju-log "$CHARM: Updating $conf_file, $key=$value"
-    sed -i "s|\($pattern\).*|\1$value|" $conf_file
-  else
-    juju-log "$CHARM: Setting new option $key=$value in $conf_file"
-    echo "$out$value" >>$conf_file
-  fi
-  CONFIG_CHANGED=True
-}

=== removed file 'charmhelpers/contrib/openstack/nova/folsom'
--- charmhelpers/contrib/openstack/nova/folsom	2013-05-30 23:26:24 +0000
+++ charmhelpers/contrib/openstack/nova/folsom	1970-01-01 00:00:00 +0000
@@ -1,81 +0,0 @@
-#!/bin/bash -e
-
-# Folsom-specific functions
-
-nova_set_or_update() {
-  # TODO: This needs to be shared among folsom, grizzly and beyond.
-  # Set a config option in nova.conf or api-paste.ini, depending
-  # Defaults to updating nova.conf
-  local key="$1"
-  local value="$2"
-  local conf_file="$3"
-  local section="${4:-DEFAULT}"
-
-  local nova_conf=${NOVA_CONF:-/etc/nova/nova.conf}
-  local api_conf=${API_CONF:-/etc/nova/api-paste.ini}
-  local quantum_conf=${QUANTUM_CONF:-/etc/quantum/quantum.conf}
-  local quantum_api_conf=${QUANTUM_API_CONF:-/etc/quantum/api-paste.ini}
-  local quantum_plugin_conf=${QUANTUM_PLUGIN_CONF:-/etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini}
-  local libvirtd_conf=${LIBVIRTD_CONF:-/etc/libvirt/libvirtd.conf}
-
-  [[ -z $key ]] && juju-log "$CHARM: set_or_update: value $value missing key" && exit 1
-  [[ -z $value ]] && juju-log "$CHARM: set_or_update: key $key missing value" && exit 1
-
-  [[ -z "$conf_file" ]] && conf_file=$nova_conf
-
-  local pattern=""
-  case "$conf_file" in
-    "$nova_conf") match="^$key="
-                  pattern="$key="
-                  out=$pattern
-                  ;;
-    "$api_conf"|"$quantum_conf"|"$quantum_api_conf"|"$quantum_plugin_conf"| \
-    "$libvirtd_conf")
-                 match="^$key = "
-                 pattern="$match"
-                 out="$key = "
-                 ;;
-    *) juju-log "$CHARM ERROR: set_or_update: Invalid conf_file ($conf_file)"
-  esac
-
-  cat $conf_file | grep "$match$value" >/dev/null &&
-    juju-log "$CHARM: $key=$value already in set in $conf_file" \
-      && return 0
-
-  case $conf_file in
-    "$quantum_conf"|"$quantum_api_conf"|"$quantum_plugin_conf")
-        python -c "
-import ConfigParser
-config = ConfigParser.RawConfigParser()
-config.read('$conf_file')
-config.set('$section','$key','$value')
-with open('$conf_file', 'wb') as configfile:
-    config.write(configfile)
-"
-      ;;
-    *)
-      if cat $conf_file | grep "$match" >/dev/null ; then
-        juju-log "$CHARM: Updating $conf_file, $key=$value"
-        sed -i "s|\($pattern\).*|\1$value|" $conf_file
-      else
-        juju-log "$CHARM: Setting new option $key=$value in $conf_file"
-        echo "$out$value" >>$conf_file
-      fi
-      ;;
-  esac
-  CONFIG_CHANGED="True"
-}
-
-# Upgrade Helpers
-nova_pre_upgrade() {
-  # Pre-upgrade helper.  Caller should pass the version of OpenStack we are
-  # upgrading from.
-  return 0 # Nothing to do here, yet.
-}
-
-nova_post_upgrade() {
-  # Post-upgrade helper.  Caller should pass the version of OpenStack we are
-  # upgrading from.
-  juju-log "$CHARM: Running post-upgrade hook: $upgrade_from -> folsom."
-  # nothing to do here yet.
-}

=== removed symlink 'charmhelpers/contrib/openstack/nova/grizzly'
=== target was u'folsom'
=== removed file 'charmhelpers/contrib/openstack/nova/nova-common'
--- charmhelpers/contrib/openstack/nova/nova-common	2013-05-30 23:26:24 +0000
+++ charmhelpers/contrib/openstack/nova/nova-common	1970-01-01 00:00:00 +0000
@@ -1,147 +0,0 @@
-#!/bin/bash -e
-
-# Common utility functions used across all nova charms.
-
-CONFIG_CHANGED=False
-
-# Load the common OpenStack helper library.
-if [[ -e $CHARM_DIR/lib/openstack-common ]] ; then
-  . $CHARM_DIR/lib/openstack-common
-else
-  juju-log "Couldn't load $CHARM_DIR/lib/opentack-common." && exit 1
-fi
-
-set_or_update() {
-  # Update config flags in nova.conf or api-paste.ini.
-  # Config layout changed in Folsom, so this is now OpenStack release specific.
-  local rel=$(get_os_codename_package "nova-common")
-  . $CHARM_DIR/lib/nova/$rel
-  nova_set_or_update $@
-}
-
-function set_config_flags() {
-  # Set user-defined nova.conf flags from deployment config
-  juju-log "$CHARM: Processing config-flags."
-  flags=$(config-get config-flags)
-  if [[ "$flags" != "None" && -n "$flags" ]] ; then
-    for f in $(echo $flags | sed -e 's/,/ /g') ; do
-      k=$(echo $f | cut -d= -f1)
-      v=$(echo $f | cut -d= -f2)
-      set_or_update "$k" "$v"
-    done
-  fi
-}
-
-configure_volume_service() {
-  local svc="$1"
-  local cur_vers="$(get_os_codename_package "nova-common")"
-  case "$svc" in
-    "cinder")
-      set_or_update "volume_api_class" "nova.volume.cinder.API" ;;
-    "nova-volume")
-      # nova-volume only supported before grizzly.
-      [[ "$cur_vers" == "essex" ]] || [[ "$cur_vers" == "folsom" ]] &&
-        set_or_update "volume_api_class" "nova.volume.api.API"
-      ;;
-    *) juju-log "$CHARM ERROR - configure_volume_service: Invalid service $svc"
-       return 1 ;;
-  esac
-}
-
-function configure_network_manager {
-  local manager="$1"
-  echo "$CHARM: configuring $manager network manager"
-  case $1 in
-    "FlatManager")
-      set_or_update "network_manager" "nova.network.manager.FlatManager"
-      ;;
-    "FlatDHCPManager")
-      set_or_update "network_manager" "nova.network.manager.FlatDHCPManager"
-
-      if [[ "$CHARM" == "nova-compute" ]] ; then
-        local flat_interface=$(config-get flat-interface)
-        local ec2_host=$(relation-get ec2_host)
-        set_or_update flat_inteface "$flat_interface"
-        set_or_update ec2_dmz_host "$ec2_host"
-
-        # Ensure flat_interface has link.
-        if ip link show $flat_interface >/dev/null 2>&1 ; then
-          ip link set $flat_interface up
-        fi
-
-        # work around (LP: #1035172)
-        if [[ -e /dev/vhost-net ]] ; then
-          iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM \
-            --checksum-fill
-        fi
-      fi
-
-      ;;
-    "Quantum")
-      local local_ip=$(get_ip `unit-get private-address`)
-      [[ -n $local_ip ]] || {
-        juju-log "Unable to resolve local IP address"
-        exit 1
-      }
-      set_or_update "network_api_class" "nova.network.quantumv2.api.API"
-      set_or_update "quantum_auth_strategy" "keystone"
-      set_or_update "core_plugin" "$QUANTUM_CORE_PLUGIN" "$QUANTUM_CONF"
-      set_or_update "bind_host" "0.0.0.0" "$QUANTUM_CONF"
-      if [ "$QUANTUM_PLUGIN" == "ovs" ]; then
-          set_or_update "tenant_network_type" "gre" $QUANTUM_PLUGIN_CONF "OVS"
-          set_or_update "enable_tunneling" "True" $QUANTUM_PLUGIN_CONF "OVS"
-          set_or_update "tunnel_id_ranges" "1:1000" $QUANTUM_PLUGIN_CONF "OVS"
-          set_or_update "local_ip" "$local_ip" $QUANTUM_PLUGIN_CONF "OVS"
-      fi
-      ;;
-    *) juju-log "ERROR: Invalid network manager $1" && exit 1 ;;
-  esac
-}
-
-function trigger_remote_service_restarts() {
-  # Trigger a service restart on all other nova nodes that have a relation
-  # via the cloud-controller interface.
-
-  # possible relations to other nova services.
-  local relations="cloud-compute nova-volume-service"
-
-  for rel in $relations; do
-    local r_ids=$(relation-ids $rel)
-    for r_id in $r_ids ; do
-      juju-log "$CHARM: Triggering a service restart on relation $r_id."
-      relation-set -r $r_id restart-trigger=$(uuid)
-    done
-  done
-}
-
-do_openstack_upgrade() {
-  # update openstack components to those provided by a new installation source
-  # it is assumed the calling hook has confirmed that the upgrade is sane.
-  local rel="$1"
-  shift
-  local packages=$@
-
-  orig_os_rel=$(get_os_codename_package "nova-common")
-  new_rel=$(get_os_codename_install_source "$rel")
-
-  # Backup the config directory.
-  local stamp=$(date +"%Y%m%d%M%S")
-  tar -pcf /var/lib/juju/$CHARM-backup-$stamp.tar $CONF_DIR
-
-  # load the release helper library for pre/post upgrade hooks specific to the
-  # release we are upgrading to.
-  . $CHARM_DIR/lib/nova/$new_rel
-
-  # new release specific pre-upgrade hook
-  nova_pre_upgrade "$orig_os_rel"
-
-  # Setup apt repository access and kick off the actual package upgrade.
-  configure_install_source "$rel"
-  apt-get update
-  DEBIAN_FRONTEND=noninteractive apt-get --option Dpkg::Options::=--force-confold -y \
-     install --no-install-recommends $packages
-
-  # new release sepcific post-upgrade hook
-  nova_post_upgrade "$orig_os_rel"
-
-}

=== removed file 'charmhelpers/contrib/openstack/openstack-common'
--- charmhelpers/contrib/openstack/openstack-common	2013-05-30 23:26:24 +0000
+++ charmhelpers/contrib/openstack/openstack-common	1970-01-01 00:00:00 +0000
@@ -1,781 +0,0 @@
-#!/bin/bash -e
-
-# Common utility functions used across all OpenStack charms.
-
-error_out() {
-  juju-log "$CHARM ERROR: $@"
-  exit 1
-}
-
-function service_ctl_status {
-  # Return 0 if a service is running, 1 otherwise.
-  local svc="$1"
-  local status=$(service $svc status | cut -d/ -f1 | awk '{ print $2 }')
-  case $status in
-    "start") return 0 ;;
-    "stop") return 1 ;;
-    *) error_out "Unexpected status of service $svc: $status" ;;
-  esac
-}
-
-function service_ctl {
-  # control a specific service, or all (as defined by $SERVICES)
-  # service restarts will only occur depending on global $CONFIG_CHANGED,
-  # which should be updated in charm's set_or_update().
-  local config_changed=${CONFIG_CHANGED:-True}
-  if [[ $1 == "all" ]] ; then
-    ctl="$SERVICES"
-  else
-    ctl="$1"
-  fi
-  action="$2"
-  if [[ -z "$ctl" ]] || [[ -z "$action" ]] ; then
-    error_out "ERROR service_ctl: Not enough arguments"
-  fi
-
-  for i in $ctl ; do
-    case $action in
-      "start")
-        service_ctl_status $i || service $i start ;;
-      "stop")
-        service_ctl_status $i && service $i stop || return 0 ;;
-      "restart")
-        if [[ "$config_changed" == "True" ]] ; then
-          service_ctl_status $i && service $i restart || service $i start
-        fi
-        ;;
-    esac
-    if [[ $? != 0 ]] ; then
-      juju-log "$CHARM: service_ctl ERROR - Service $i failed to $action"
-    fi
-  done
-  # all configs should have been reloaded on restart of all services, reset
-  # flag if its being used.
-  if [[ "$action" == "restart" ]] && [[ -n "$CONFIG_CHANGED" ]] &&
-     [[ "$ctl" == "all" ]]; then
-    CONFIG_CHANGED="False"
-  fi
-}
-
-function configure_install_source {
-  # Setup and configure installation source based on a config flag.
-  local src="$1"
-
-  # Default to installing from the main Ubuntu archive.
-  [[ $src == "distro" ]] || [[ -z "$src" ]] && return 0
-
-  . /etc/lsb-release
-
-  # standard 'ppa:someppa/name' format.
-   if [[ "${src:0:4}" == "ppa:" ]] ; then
-    juju-log "$CHARM: Configuring installation from custom src ($src)"
-    add-apt-repository -y "$src" || error_out "Could not configure PPA access."
-    return 0
-  fi
-
-  # standard 'deb http://url/ubuntu main' entries. gpg key ids must
-  # be appended to the end of url after a |, ie:
-  # 'deb http://url/ubuntu main|$GPGKEYID'
-  if [[ "${src:0:3}" == "deb" ]] ; then
-    juju-log "$CHARM: Configuring installation from custom src URL ($src)"
-    if echo "$src" | grep -q "|" ; then
-      # gpg key id tagged to end of url folloed by a |
-      url=$(echo $src | cut -d'|' -f1)
-      key=$(echo $src | cut -d'|' -f2)
-      juju-log "$CHARM: Importing repository key: $key"
-      apt-key adv --keyserver keyserver.ubuntu.com --recv-keys "$key" || \
-        juju-log "$CHARM WARN: Could not import key from keyserver: $key"
-    else
-      juju-log "$CHARM No repository key specified."
-      url="$src"
-    fi
-    echo "$url" > /etc/apt/sources.list.d/juju_deb.list
-    return 0
-  fi
-
-  # Cloud Archive
-  if [[ "${src:0:6}" == "cloud:" ]] ; then
-
-    # current os releases supported by the UCA.
-    local cloud_archive_versions="folsom grizzly"
-
-    local ca_rel=$(echo $src | cut -d: -f2)
-    local u_rel=$(echo $ca_rel | cut -d- -f1)
-    local os_rel=$(echo $ca_rel | cut -d- -f2 | cut -d/ -f1)
-
-    [[ "$u_rel" != "$DISTRIB_CODENAME" ]] &&
-      error_out "Cannot install from Cloud Archive pocket $src " \
-                "on this Ubuntu version ($DISTRIB_CODENAME)!"
-
-    valid_release=""
-    for rel in $cloud_archive_versions ; do
-      if [[ "$os_rel" == "$rel" ]] ; then
-        valid_release=1
-        juju-log "Installing OpenStack ($os_rel) from the Ubuntu Cloud Archive."
-      fi
-    done
-    if [[ -z "$valid_release" ]] ; then
-      error_out "OpenStack release ($os_rel) not supported by "\
-                "the Ubuntu Cloud Archive."
-    fi
-
-    # CA staging repos are standard PPAs.
-    if echo $ca_rel | grep -q "staging" ; then
-      add-apt-repository -y ppa:ubuntu-cloud-archive/${os_rel}-staging
-      return 0
-    fi
-
-    # the others are LP-external deb repos.
-    case "$ca_rel" in
-      "$u_rel-$os_rel"|"$u_rel-$os_rel/updates") pocket="$u_rel-updates/$os_rel" ;;
-      "$u_rel-$os_rel/proposed") pocket="$u_rel-proposed/$os_rel" ;;
-      "$u_rel-$os_rel"|"$os_rel/updates") pocket="$u_rel-updates/$os_rel" ;;
-      "$u_rel-$os_rel/proposed") pocket="$u_rel-proposed/$os_rel" ;;
-      *) error_out "Invalid Cloud Archive repo specified: $src"
-    esac
-
-    apt-get -y install ubuntu-cloud-keyring
-    entry="deb http://ubuntu-cloud.archive.canonical.com/ubuntu $pocket main"
-    echo "$entry" \
-      >/etc/apt/sources.list.d/ubuntu-cloud-archive-$DISTRIB_CODENAME.list
-    return 0
-  fi
-
-  error_out "Invalid installation source specified in config: $src"
-
-}
-
-get_os_codename_install_source() {
-  # derive the openstack release provided by a supported installation source.
-  local rel="$1"
-  local codename="unknown"
-  . /etc/lsb-release
-
-  # map ubuntu releases to the openstack version shipped with it.
-  if [[ "$rel" == "distro" ]] ; then
-    case "$DISTRIB_CODENAME" in
-      "oneiric") codename="diablo" ;;
-      "precise") codename="essex" ;;
-      "quantal") codename="folsom" ;;
-      "raring")  codename="grizzly" ;;
-    esac
-  fi
-
-  # derive version from cloud archive strings.
-  if [[ "${rel:0:6}" == "cloud:" ]] ; then
-    rel=$(echo $rel | cut -d: -f2)
-    local u_rel=$(echo $rel | cut -d- -f1)
-    local ca_rel=$(echo $rel | cut -d- -f2)
-    if [[ "$u_rel" == "$DISTRIB_CODENAME" ]] ; then
-      case "$ca_rel" in
-        "folsom"|"folsom/updates"|"folsom/proposed"|"folsom/staging")
-          codename="folsom" ;;
-        "grizzly"|"grizzly/updates"|"grizzly/proposed"|"grizzly/staging")
-          codename="grizzly" ;;
-      esac
-    fi
-  fi
-
-  # have a guess based on the deb string provided
-  if [[ "${rel:0:3}" == "deb" ]] || \
-     [[ "${rel:0:3}" == "ppa" ]] ; then
-    CODENAMES="diablo essex folsom grizzly havana"
-    for cname in $CODENAMES; do
-      if echo $rel | grep -q $cname; then
-        codename=$cname
-      fi
-    done
-  fi
-  echo $codename
-}
-
-get_os_codename_package() {
-  local pkg_vers=$(dpkg -l | grep "$1" | awk '{ print $3 }') || echo "none"
-  pkg_vers=$(echo $pkg_vers | cut -d: -f2) # epochs
-  case "${pkg_vers:0:6}" in
-    "2011.2") echo "diablo" ;;
-    "2012.1") echo "essex" ;;
-    "2012.2") echo "folsom" ;;
-    "2013.1") echo "grizzly" ;;
-    "2013.2") echo "havana" ;;
-  esac
-}
-
-get_os_version_codename() {
-  case "$1" in
-    "diablo") echo "2011.2" ;;
-    "essex") echo "2012.1" ;;
-    "folsom") echo "2012.2" ;;
-    "grizzly") echo "2013.1" ;;
-    "havana") echo "2013.2" ;;
-  esac
-}
-
-get_ip() {
-  dpkg -l | grep -q python-dnspython || {
-    apt-get -y install python-dnspython 2>&1 > /dev/null
-  }
-  hostname=$1
-  python -c "
-import dns.resolver
-import socket
-try:
-  # Test to see if already an IPv4 address
-  socket.inet_aton('$hostname')
-  print '$hostname'
-except socket.error:
-  try:
-    answers = dns.resolver.query('$hostname', 'A')
-    if answers:
-      print answers[0].address
-  except dns.resolver.NXDOMAIN:
-    pass
-"
-}
-
-# Common storage routines used by cinder, nova-volume and swift-storage.
-clean_storage() {
-  # if configured to overwrite existing storage, we unmount the block-dev
-  # if mounted and clear any previous pv signatures
-  local block_dev="$1"
-  juju-log "Cleaining storage '$block_dev'"
-  if grep -q "^$block_dev" /proc/mounts ; then
-    mp=$(grep "^$block_dev" /proc/mounts   | awk '{ print $2 }')
-    juju-log "Unmounting $block_dev from $mp"
-    umount "$mp" || error_out "ERROR: Could not unmount storage from $mp"
-  fi
-  if pvdisplay "$block_dev" >/dev/null 2>&1 ; then
-    juju-log "Removing existing LVM PV signatures from $block_dev"
-
-    # deactivate any volgroups that may be built on this dev
-    vg=$(pvdisplay $block_dev | grep "VG Name" | awk '{ print $3 }')
-    if [[ -n "$vg" ]] ; then
-      juju-log "Deactivating existing volume group: $vg"
-      vgchange -an "$vg" ||
-        error_out "ERROR: Could not deactivate volgroup $vg.  Is it in use?"
-    fi
-    echo "yes" | pvremove -ff "$block_dev" ||
-      error_out "Could not pvremove $block_dev"
-  else
-    juju-log "Zapping disk of all GPT and MBR structures"
-    sgdisk --zap-all $block_dev ||
-      error_out "Unable to zap $block_dev"
-  fi
-}
-
-function get_block_device() {
-  # given a string, return full path to the block device for that
-  # if input is not a block device, find a loopback device
-  local input="$1"
-
-  case "$input" in
-    /dev/*) [[ ! -b "$input" ]] && error_out "$input does not exist."
-            echo "$input"; return 0;;
-    /*) :;;
-    *)  [[ ! -b "/dev/$input" ]] && error_out "/dev/$input does not exist."
-        echo "/dev/$input"; return 0;;
-  esac
-
-  # this represents a file
-  # support "/path/to/file|5G"
-  local fpath size oifs="$IFS"
-  if [ "${input#*|}" != "${input}" ]; then
-    size=${input##*|}
-    fpath=${input%|*}
-  else
-    fpath=${input}
-    size=5G
-  fi
-
-  ## loop devices are not namespaced.  This is bad for containers.
-  ## it means that the output of 'losetup' may have the given $fpath
-  ## in it, but that may not represent this containers $fpath, but
-  ## another containers.  To address that, we really need to
-  ## allow some uniq container-id to be expanded within path.
-  ## TODO: find a unique container-id that will be consistent for
-  ##       this container throughout its lifetime and expand it
-  ##       in the fpath.
-  # fpath=${fpath//%{id}/$THAT_ID}
-
-  local found=""
-  # parse through 'losetup -a' output, looking for this file
-  # output is expected to look like:
-  #   /dev/loop0: [0807]:961814 (/tmp/my.img)
-  found=$(losetup -a |
-    awk 'BEGIN { found=0; }
-         $3 == f { sub(/:$/,"",$1); print $1; found=found+1; }
-         END { if( found == 0 || found == 1 ) { exit(0); }; exit(1); }' \
-         f="($fpath)")
-
-  if [ $? -ne 0 ]; then
-    echo "multiple devices found for $fpath: $found" 1>&2
-    return 1;
-  fi
-
-  [ -n "$found" -a -b "$found" ] && { echo "$found"; return 1; }
-
-  if [ -n "$found" ]; then
-    echo "confused, $found is not a block device for $fpath";
-    return 1;
-  fi
-
-  # no existing device was found, create one
-  mkdir -p "${fpath%/*}"
-  truncate --size "$size" "$fpath" ||
-    { echo "failed to create $fpath of size $size"; return 1; }
-
-  found=$(losetup --find --show "$fpath") ||
-    { echo "failed to setup loop device for $fpath" 1>&2; return 1; }
-
-  echo "$found"
-  return 0
-}
-
-HAPROXY_CFG=/etc/haproxy/haproxy.cfg
-HAPROXY_DEFAULT=/etc/default/haproxy
-##########################################################################
-# Description: Configures HAProxy services for Openstack API's
-# Parameters:
-#   Space delimited list of service:port:mode combinations for which
-#   haproxy service configuration should be generated for.  The function
-#   assumes the name of the peer relation is 'cluster' and that every
-#   service unit in the peer relation is running the same services.
-#
-#   Services that do not specify :mode in parameter will default to http.
-#
-# Example
-#   configure_haproxy cinder_api:8776:8756:tcp nova_api:8774:8764:http
-##########################################################################
-configure_haproxy() {
-  local address=`unit-get private-address`
-  local name=${JUJU_UNIT_NAME////-}
-  cat > $HAPROXY_CFG << EOF
-global
-  log 127.0.0.1 local0
-  log 127.0.0.1 local1 notice
-  maxconn 20000
-  user haproxy
-  group haproxy
-  spread-checks 0
-
-defaults
-  log global
-  mode http
-  option httplog
-  option dontlognull
-  retries 3
-  timeout queue 1000
-  timeout connect 1000
-  timeout client 30000
-  timeout server 30000
-
-listen stats :8888
-  mode http
-  stats enable
-  stats hide-version
-  stats realm Haproxy\ Statistics
-  stats uri /
-  stats auth admin:password
-
-EOF
-  for service in $@; do
-    local service_name=$(echo $service | cut -d : -f 1)
-    local haproxy_listen_port=$(echo $service | cut -d : -f 2)
-    local api_listen_port=$(echo $service | cut -d : -f 3)
-    local mode=$(echo $service | cut -d : -f 4)
-    [[ -z "$mode" ]] && mode="http"
-    juju-log "Adding haproxy configuration entry for $service "\
-             "($haproxy_listen_port -> $api_listen_port)"
-    cat >> $HAPROXY_CFG << EOF
-listen $service_name 0.0.0.0:$haproxy_listen_port
-  balance roundrobin
-  mode $mode
-  option ${mode}log
-  server $name $address:$api_listen_port check
-EOF
-    local r_id=""
-    local unit=""
-    for r_id in `relation-ids cluster`; do
-      for unit in `relation-list -r $r_id`; do
-        local unit_name=${unit////-}
-        local unit_address=`relation-get -r $r_id private-address $unit`
-        if [ -n "$unit_address" ]; then
-          echo "  server $unit_name $unit_address:$api_listen_port check" \
-            >> $HAPROXY_CFG
-        fi
-      done
-    done
-  done
-  echo "ENABLED=1" > $HAPROXY_DEFAULT
-  service haproxy restart
-}
-
-##########################################################################
-# Description: Query HA interface to determine is cluster is configured
-# Returns: 0 if configured, 1 if not configured
-##########################################################################
-is_clustered() {
-  local r_id=""
-  local unit=""
-  for r_id in $(relation-ids ha); do
-    if [ -n "$r_id" ]; then
-      for unit in $(relation-list -r $r_id); do
-         clustered=$(relation-get -r $r_id clustered $unit)
-         if [ -n "$clustered" ]; then
-           juju-log "Unit is haclustered"
-           return 0
-         fi
-      done
-    fi
-  done
-  juju-log "Unit is not haclustered"
-  return 1
-}
-
-##########################################################################
-# Description: Return a list of all peers in cluster relations
-##########################################################################
-peer_units() {
-  local peers=""
-  local r_id=""
-  for r_id in $(relation-ids cluster); do
-    peers="$peers $(relation-list -r $r_id)"
-  done
-  echo $peers
-}
-
-##########################################################################
-# Description: Determines whether the current unit is the oldest of all
-#              its peers - supports partial leader election
-# Returns: 0 if oldest, 1 if not
-##########################################################################
-oldest_peer() {
-  peers=$1
-  local l_unit_no=$(echo $JUJU_UNIT_NAME | cut -d / -f 2)
-  for peer in $peers; do
-    echo "Comparing $JUJU_UNIT_NAME with peers: $peers"
-    local r_unit_no=$(echo $peer | cut -d / -f 2)
-    if (($r_unit_no<$l_unit_no)); then
-        juju-log "Not oldest peer; deferring"
-        return 1
-    fi
-  done
-  juju-log "Oldest peer; might take charge?"
-  return 0
-}
-
-##########################################################################
-# Description: Determines whether the current service units is the
-#              leader within a) a cluster of its peers or b) across a
-#              set of unclustered peers.
-# Parameters: CRM resource to check ownership of if clustered
-# Returns: 0 if leader, 1 if not
-##########################################################################
-eligible_leader() {
-  if is_clustered; then
-    if ! is_leader $1; then
-      juju-log 'Deferring action to CRM leader'
-      return 1
-    fi
-  else
-    peers=$(peer_units)
-    if [ -n "$peers" ] && ! oldest_peer "$peers"; then
-      juju-log 'Deferring action to oldest service unit.'
-      return 1
-    fi
-  fi
-  return 0
-}
-
-##########################################################################
-# Description: Query Cluster peer interface to see if peered
-# Returns: 0 if peered, 1 if not peered
-##########################################################################
-is_peered() {
-  local r_id=$(relation-ids cluster)
-  if [ -n "$r_id" ]; then
-    if [ -n "$(relation-list -r $r_id)" ]; then
-      juju-log "Unit peered"
-      return 0
-    fi
-  fi
-  juju-log "Unit not peered"
-  return 1
-}
-
-##########################################################################
-# Description: Determines whether host is owner of clustered services
-# Parameters: Name of CRM resource to check ownership of
-# Returns: 0 if leader, 1 if not leader
-##########################################################################
-is_leader() {
-  hostname=`hostname`
-  if [ -x /usr/sbin/crm ]; then
-    if crm resource show $1 | grep -q $hostname; then
-      juju-log "$hostname is cluster leader."
-      return 0
-    fi
-  fi
-  juju-log "$hostname is not cluster leader."
-  return 1
-}
-
-##########################################################################
-# Description: Determines whether enough data has been provided in
-# configuration or relation data to configure HTTPS.
-# Parameters:  None
-# Returns: 0 if HTTPS can be configured, 1 if not.
-##########################################################################
-https() {
-  local r_id=""
-  if [[ -n "$(config-get ssl_cert)" ]] &&
-     [[ -n "$(config-get ssl_key)" ]] ; then
-    return 0
-  fi
-  for r_id in $(relation-ids identity-service) ; do
-    for unit in $(relation-list -r $r_id) ; do
-      if [[ "$(relation-get -r $r_id https_keystone $unit)" == "True" ]] &&
-         [[ -n "$(relation-get -r $r_id ssl_cert $unit)" ]] &&
-         [[ -n "$(relation-get -r $r_id ssl_key $unit)" ]] &&
-         [[ -n "$(relation-get -r $r_id ca_cert $unit)" ]] ; then
-          return 0
-      fi
-    done
-  done
-  return 1
-}
-
-##########################################################################
-# Description: For a given number of port mappings, configures apache2
-# HTTPs local reverse proxying using certficates and keys provided in
-# either configuration data (preferred) or relation data.  Assumes ports
-# are not in use (calling charm should ensure that).
-# Parameters:  Variable number of proxy port mappings as
-# $internal:$external.
-# Returns: 0 if reverse proxy(s) have been configured, 0 if not.
-##########################################################################
-enable_https() {
-  local port_maps="$@"
-  local http_restart=""
-  juju-log "Enabling HTTPS for port mappings: $port_maps."
-
-  # allow overriding of keystone provided certs with those set manually
-  # in config.
-  local cert=$(config-get ssl_cert)
-  local key=$(config-get ssl_key)
-  local ca_cert=""
-  if [[ -z "$cert" ]] || [[ -z "$key" ]] ; then
-    juju-log "Inspecting identity-service relations for SSL certificate."
-    local r_id=""
-    cert=""
-    key=""
-    ca_cert=""
-    for r_id in $(relation-ids identity-service) ; do
-      for unit in $(relation-list -r $r_id) ; do
-        [[ -z "$cert" ]] && cert="$(relation-get -r $r_id ssl_cert $unit)"
-        [[ -z "$key" ]] && key="$(relation-get -r $r_id ssl_key $unit)"
-        [[ -z "$ca_cert" ]] && ca_cert="$(relation-get -r $r_id ca_cert $unit)"
-      done
-    done
-    [[ -n "$cert" ]] && cert=$(echo $cert | base64 -di)
-    [[ -n "$key" ]] && key=$(echo $key | base64 -di)
-    [[ -n "$ca_cert" ]] && ca_cert=$(echo $ca_cert | base64 -di)
-  else
-    juju-log "Using SSL certificate provided in service config."
-  fi
-
-  [[ -z "$cert" ]] || [[ -z "$key" ]] &&
-    juju-log "Expected but could not find SSL certificate data, not "\
-             "configuring HTTPS!" && return 1
-
-  apt-get -y install apache2
-  a2enmod ssl proxy proxy_http | grep -v "To activate the new configuration" &&
-    http_restart=1
-
-  mkdir -p /etc/apache2/ssl/$CHARM
-  echo "$cert" >/etc/apache2/ssl/$CHARM/cert
-  echo "$key" >/etc/apache2/ssl/$CHARM/key
-  if [[ -n "$ca_cert" ]] ; then
-    juju-log "Installing Keystone supplied CA cert."
-    echo "$ca_cert" >/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt
-    update-ca-certificates --fresh
-
-    # XXX TODO: Find a better way of exporting this?
-    if [[ "$CHARM" == "nova-cloud-controller" ]] ; then
-      [[ -e /var/www/keystone_juju_ca_cert.crt ]] &&
-        rm -rf /var/www/keystone_juju_ca_cert.crt
-      ln -s  /usr/local/share/ca-certificates/keystone_juju_ca_cert.crt \
-             /var/www/keystone_juju_ca_cert.crt
-    fi
-
-  fi
-  for port_map in $port_maps ; do
-    local ext_port=$(echo $port_map | cut -d: -f1)
-    local int_port=$(echo $port_map | cut -d: -f2)
-    juju-log "Creating apache2 reverse proxy vhost for $port_map."
-    cat >/etc/apache2/sites-available/${CHARM}_${ext_port} <<END
-Listen $ext_port
-NameVirtualHost *:$ext_port
-<VirtualHost *:$ext_port>
-    ServerName $(unit-get private-address)
-    SSLEngine on
-    SSLCertificateFile /etc/apache2/ssl/$CHARM/cert
-    SSLCertificateKeyFile /etc/apache2/ssl/$CHARM/key
-    ProxyPass / http://localhost:$int_port/
-    ProxyPassReverse / http://localhost:$int_port/
-    ProxyPreserveHost on
-</VirtualHost>
-<Proxy *>
-    Order deny,allow
-    Allow from all
-</Proxy>
-<Location />
-    Order allow,deny
-    Allow from all
-</Location>
-END
-    a2ensite ${CHARM}_${ext_port} | grep -v "To activate the new configuration" &&
-      http_restart=1
-  done
-  if [[ -n "$http_restart" ]] ; then
-    service apache2 restart
-  fi
-}
-
-##########################################################################
-# Description: Ensure HTTPS reverse proxying is disabled for given port
-# mappings.
-# Parameters:  Variable number of proxy port mappings as
-# $internal:$external.
-# Returns: 0 if reverse proxy is not active for all portmaps, 1 on error.
-##########################################################################
-disable_https() {
-  local port_maps="$@"
-  local http_restart=""
-  juju-log "Ensuring HTTPS disabled for $port_maps."
-  ( [[ ! -d /etc/apache2 ]] || [[ ! -d /etc/apache2/ssl/$CHARM ]] ) && return 0
-  for port_map in $port_maps ; do
-    local ext_port=$(echo $port_map | cut -d: -f1)
-    local int_port=$(echo $port_map | cut -d: -f2)
-    if [[ -e /etc/apache2/sites-available/${CHARM}_${ext_port} ]] ; then
-      juju-log "Disabling HTTPS reverse proxy for $CHARM $port_map."
-      a2dissite ${CHARM}_${ext_port} | grep -v "To activate the new configuration" &&
-        http_restart=1
-    fi
-  done
-  if [[ -n "$http_restart" ]] ; then
-    service apache2 restart
-  fi
-}
-
-
-##########################################################################
-# Description: Ensures HTTPS is either enabled or disabled for given port
-# mapping.
-# Parameters:  Variable number of proxy port mappings as
-# $internal:$external.
-# Returns: 0 if HTTPS reverse proxy is in place, 1 if it is not.
-##########################################################################
-setup_https() {
-  # configure https via apache reverse proxying either
-  # using certs provided by config or keystone.
-  [[ -z "$CHARM" ]] &&
-    error_out "setup_https(): CHARM not set."
-  if ! https ; then
-    disable_https $@
-  else
-    enable_https $@
-  fi
-}
-
-##########################################################################
-# Description: Determine correct API server listening port based on
-# existence of HTTPS reverse proxy and/or haproxy.
-# Paremeters: The standard public port for given service.
-# Returns: The correct listening port for API service.
-##########################################################################
-determine_api_port() {
-  local public_port="$1"
-  local i=0
-  ( [[ -n "$(peer_units)" ]] || is_clustered >/dev/null 2>&1 ) && i=$[$i + 1]
-  https >/dev/null 2>&1 && i=$[$i + 1]
-  echo $[$public_port - $[$i * 10]]
-}
-
-##########################################################################
-# Description: Determine correct proxy listening port based on public IP +
-# existence of HTTPS reverse proxy.
-# Paremeters: The standard public port for given service.
-# Returns: The correct listening port for haproxy service public address.
-##########################################################################
-determine_haproxy_port() {
-  local public_port="$1"
-  local i=0
-  https >/dev/null 2>&1 && i=$[$i + 1]
-  echo $[$public_port - $[$i * 10]]
-}
-
-##########################################################################
-# Description: Print the value for a given config option in an OpenStack
-# .ini style configuration file.
-# Parameters: File path, option to retrieve, optional
-# section name (default=DEFAULT)
-# Returns: Prints value if set, prints nothing otherwise.
-##########################################################################
-local_config_get() {
-  # return config values set in openstack .ini config files.
-  # default placeholders starting (eg, %AUTH_HOST%) treated as
-  # unset values.
-  local file="$1"
-  local option="$2"
-  local section="$3"
-  [[ -z "$section" ]] && section="DEFAULT"
-  python -c "
-import ConfigParser
-config = ConfigParser.RawConfigParser()
-config.read('$file')
-try:
-  value = config.get('$section', '$option')
-except:
-  print ''
-  exit(0)
-if value.startswith('%'): exit(0)
-print value
-"
-}
-
-##########################################################################
-# Description: Creates an rc file exporting environment variables to a
-# script_path local to the charm's installed directory.
-# Any charm scripts run outside the juju hook environment can source this
-# scriptrc to obtain updated config information necessary to perform health
-# checks or service changes
-#
-# Parameters:
-#   An array of '=' delimited  ENV_VAR:value combinations to export.
-#   If optional script_path key is not provided in the array, script_path
-#     defaults to scripts/scriptrc
-##########################################################################
-function save_script_rc {
-  if [ ! -n "$JUJU_UNIT_NAME" ]; then
-     echo "Error: Missing JUJU_UNIT_NAME environment variable"
-     exit 1
-  fi
-  # our default unit_path
-  unit_path="/var/lib/juju/units/${JUJU_UNIT_NAME/\//-}/charm/scripts/scriptrc"
-  echo $unit_path
-  tmp_rc="/tmp/${JUJU_UNIT_NAME/\//-}rc"
-
-  echo "#!/bin/bash" > $tmp_rc
-  for env_var in "${@}"
-  do
-    if `echo $env_var | grep -q script_path`; then
-       # well then we need to reset the new unit-local script path
-       unit_path="/var/lib/juju/units/${JUJU_UNIT_NAME/\//-}/charm/${env_var/script_path=/}"
-    else
-       echo "export $env_var" >> $tmp_rc
-    fi
-  done
-  chmod 755 $tmp_rc
-  mv $tmp_rc $unit_path
-}

=== renamed file 'charmhelpers/contrib/openstack/openstack_utils.py' => 'charmhelpers/contrib/openstack/utils.py'
--- charmhelpers/contrib/openstack/openstack_utils.py	2013-07-09 18:48:53 +0000
+++ charmhelpers/contrib/openstack/utils.py	2013-07-11 19:29:39 +0000
@@ -9,6 +9,7 @@
 
 from charmhelpers.core.hookenv import (
     config,
+    log as juju_log,
 )
 
 from charmhelpers.core.host import (
@@ -45,12 +46,8 @@
 }
 
 
-def juju_log(msg):
-    subprocess.check_call(['juju-log', msg])
-
-
 def error_out(msg):
-    juju_log("FATAL ERROR: %s" % msg)
+    juju_log("FATAL ERROR: %s" % msg, level='ERROR')
     sys.exit(1)
 
 

=== added file 'tests/contrib/hahelpers/test_apache_utils.py'
--- tests/contrib/hahelpers/test_apache_utils.py	1970-01-01 00:00:00 +0000
+++ tests/contrib/hahelpers/test_apache_utils.py	2013-07-11 19:29:39 +0000
@@ -0,0 +1,44 @@
+from mock import patch
+
+from testtools import TestCase
+
+import charmhelpers.contrib.hahelpers.apache as apache_utils
+
+
+class ApacheUtilsTests(TestCase):
+    def setUp(self):
+        super(ApacheUtilsTests, self).setUp()
+        [self._patch(m) for m in [
+            'log',
+            'config_get',
+            'relation_get',
+            'relation_ids',
+            'relation_list',
+        ]]
+
+    def _patch(self, method):
+        _m = patch.object(apache_utils, method)
+        mock = _m.start()
+        self.addCleanup(_m.stop)
+        setattr(self, method, mock)
+
+    def test_get_cert_from_config(self):
+        '''Ensure cert and key from charm config override relation'''
+        self.config_get.side_effect = [
+            'some_ca_cert',  # config_get('ssl_cert')
+            'some_ca_key',  # config_Get('ssl_key')
+        ]
+        result = apache_utils.get_cert()
+        self.assertEquals(('some_ca_cert', 'some_ca_key'), result)
+
+    def test_get_cert_from_Relation(self):
+        self.config_get.return_value = None
+        self.relation_ids.return_value = 'identity-service:0'
+        self.relation_list.return_value = 'keystone/0'
+        self.relation_get.side_effect = [
+            'keystone_provided_cert',
+            'keystone_provided_key',
+        ]
+        result = apache_utils.get_cert()
+        self.assertEquals(('keystone_provided_cert', 'keystone_provided_key'),
+                          result)

=== added file 'tests/contrib/hahelpers/test_ceph_utils.py'
--- tests/contrib/hahelpers/test_ceph_utils.py	1970-01-01 00:00:00 +0000
+++ tests/contrib/hahelpers/test_ceph_utils.py	2013-07-11 19:29:39 +0000
@@ -0,0 +1,59 @@
+from mock import patch
+
+from testtools import TestCase
+
+import charmhelpers.contrib.hahelpers.ceph as ceph_utils
+
+
+LS_POOLS = """
+images
+volumes
+rbd
+"""
+
+
+class CephUtilsTests(TestCase):
+    def setUp(self):
+        super(CephUtilsTests, self).setUp()
+        [self._patch(m) for m in [
+            'check_call',
+            'log',
+        ]]
+
+    def _patch(self, method):
+        _m = patch.object(ceph_utils, method)
+        mock = _m.start()
+        self.addCleanup(_m.stop)
+        setattr(self, method, mock)
+
+    def test_create_keyring(self):
+        '''It creates a new ceph keyring'''
+        ceph_utils.create_keyring('cinder', 'cephkey')
+        _cmd = ['ceph-authtool', '/etc/ceph/ceph.client.cinder.keyring',
+                '--create-keyring', '--name=client.cinder',
+                '--add-key=cephkey']
+        self.check_call.assert_called_with(_cmd)
+
+    def test_create_pool(self):
+        '''It creates rados pool correctly'''
+        ceph_utils.create_pool(service='cinder', name='foo')
+        self.check_call.assert_called_with(
+            ['rados', '--id', 'cinder', 'mkpool', 'foo']
+        )
+
+    def test_keyring_path(self):
+        '''It correctly dervies keyring path from service name'''
+        result = ceph_utils.keyring_path('cinder')
+        self.assertEquals('/etc/ceph/ceph.client.cinder.keyring', result)
+
+    @patch('commands.getstatusoutput')
+    def test_pool_exists(self, get_output):
+        '''It detects an rbd pool exists'''
+        get_output.return_value = (0, LS_POOLS)
+        self.assertTrue(ceph_utils.pool_exists('cinder', 'volumes'))
+
+    @patch('commands.getstatusoutput')
+    def test_pool_does_not_exist(self, get_output):
+        '''It detects an rbd pool exists'''
+        get_output.return_value = (0, LS_POOLS)
+        self.assertFalse(ceph_utils.pool_exists('cinder', 'foo'))

=== modified file 'tests/contrib/hahelpers/test_cluster_utils.py'
--- tests/contrib/hahelpers/test_cluster_utils.py	2013-07-09 18:05:11 +0000
+++ tests/contrib/hahelpers/test_cluster_utils.py	2013-07-11 19:29:39 +0000
@@ -4,14 +4,14 @@
 from subprocess import CalledProcessError
 from testtools import TestCase
 
-import charmhelpers.contrib.hahelpers.cluster_utils as cluster_utils
+import charmhelpers.contrib.hahelpers.cluster as cluster_utils
 
 
 class ClusterUtilsTests(TestCase):
     def setUp(self):
         super(ClusterUtilsTests, self).setUp()
         [self._patch(m) for m in [
-            'juju_log',
+            'log',
             'relation_ids',
             'relation_list',
             'relation_get',

=== modified file 'tests/contrib/openstack/test_openstack_utils.py'
--- tests/contrib/openstack/test_openstack_utils.py	2013-07-09 18:48:53 +0000
+++ tests/contrib/openstack/test_openstack_utils.py	2013-07-11 19:29:39 +0000
@@ -1,12 +1,13 @@
-from copy import copy
 import os
+import subprocess
 import unittest
+
+from copy import copy
 from testtools import TestCase
-import charmhelpers.contrib.openstack.openstack_utils as openstack
-import subprocess
-
 from mock import MagicMock, patch, call
 
+import charmhelpers.contrib.openstack.utils as openstack
+
 
 # mocked return of openstack.lsb_release()
 FAKE_RELEASE = {
@@ -76,7 +77,7 @@
         cache.__getitem__.side_effect = cache_get
         return cache
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.lsb_release')
+    @patch('charmhelpers.contrib.openstack.utils.lsb_release')
     def test_os_codename_from_install_source(self, mocked_lsb):
         '''Test mapping install source to OpenStack release name'''
         mocked_lsb.return_value = FAKE_RELEASE
@@ -108,14 +109,14 @@
         openstack.get_os_version_install_source('cloud:precise-grizzly')
         version.assert_called_with('grizzly')
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.lsb_release')
+    @patch('charmhelpers.contrib.openstack.utils.lsb_release')
     def test_os_codename_from_bad_install_source(self, mocked_lsb):
         '''Test mapping install source to OpenStack release name'''
         _fake_release = copy(FAKE_RELEASE)
         _fake_release['DISTRIB_CODENAME'] = 'natty'
 
         mocked_lsb.return_value = _fake_release
-        _e = 'charmhelpers.contrib.openstack.openstack_utils.error_out'
+        _e = 'charmhelpers.contrib.openstack.utils.error_out'
         with patch(_e) as mocked_err:
             openstack.get_os_codename_install_source('distro')
             _er = ('Could not derive openstack release for this Ubuntu '
@@ -127,7 +128,7 @@
         self.assertEquals(openstack.get_os_codename_version('2013.1'),
                           'grizzly')
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.error_out')
+    @patch('charmhelpers.contrib.openstack.utils.error_out')
     def test_os_codename_from_bad_version(self, mocked_error):
         '''Test mapping a bad OpenStack numerical versions to code name'''
         openstack.get_os_codename_version('2014.5.5')
@@ -140,7 +141,7 @@
         self.assertEquals(openstack.get_os_version_codename('folsom'),
                           '2012.2')
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.error_out')
+    @patch('charmhelpers.contrib.openstack.utils.error_out')
     def test_os_version_from_bad_codename(self, mocked_error):
         '''Test mapping a bad OpenStack codename to numerical version'''
         openstack.get_os_version_codename('foo')
@@ -157,7 +158,7 @@
                 self.assertEquals(openstack.get_os_codename_package(pkg),
                                   vers['os_release'])
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.error_out')
+    @patch('charmhelpers.contrib.openstack.utils.error_out')
     def test_os_codename_from_bad_package_version(self, mocked_error):
         '''Test deriving OpenStack codename for a poorly versioned package'''
         with patch('apt_pkg.Cache') as cache:
@@ -166,7 +167,7 @@
             _e = ('Could not determine OpenStack codename for version 2016.1')
             mocked_error.assert_called_with(_e)
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.error_out')
+    @patch('charmhelpers.contrib.openstack.utils.error_out')
     def test_os_codename_from_bad_package(self, mocked_error):
         '''Test deriving OpenStack codename from an uninstalled package'''
         with patch('apt_pkg.Cache') as cache:
@@ -189,7 +190,7 @@
                 openstack.get_os_codename_package('foo', fatal=False)
             )
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.error_out')
+    @patch('charmhelpers.contrib.openstack.utils.error_out')
     def test_os_version_from_package(self, mocked_error):
         '''Test deriving OpenStack version from an installed package'''
         with patch('apt_pkg.Cache') as cache:
@@ -200,7 +201,7 @@
                 self.assertEquals(openstack.get_os_version_package(pkg),
                                   vers['os_version'])
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.error_out')
+    @patch('charmhelpers.contrib.openstack.utils.error_out')
     def test_os_version_from_bad_package(self, mocked_error):
         '''Test deriving OpenStack version from an uninstalled package'''
         with patch('apt_pkg.Cache') as cache:
@@ -223,20 +224,14 @@
                 openstack.get_os_version_package('foo', fatal=False)
             )
 
-    def test_juju_log(self):
-        '''Test shelling out to juju-log'''
-        with patch('subprocess.check_call') as mocked_subprocess:
-            openstack.juju_log('foo')
-            mocked_subprocess.assert_called_with(['juju-log', 'foo'])
-
+    @patch.object(openstack, 'juju_log')
     @patch('sys.exit')
-    def test_error_out(self, mocked_exit):
+    def test_error_out(self, mocked_exit, juju_log):
         '''Test erroring out'''
-        with patch('subprocess.check_call') as mocked_subprocess:
-            openstack.error_out('Everything broke.')
-            _log = ['juju-log', 'FATAL ERROR: Everything broke.']
-            mocked_subprocess.assert_called_with(_log)
-            mocked_exit.assert_called_with(1)
+        openstack.error_out('Everything broke.')
+        _log = 'FATAL ERROR: Everything broke.'
+        juju_log.assert_called_with(_log, level='ERROR')
+        mocked_exit.assert_called_with(1)
 
     def test_configure_install_source_distro(self):
         '''Test configuring installation from distro'''
@@ -251,8 +246,8 @@
             mock.assert_called_with(ex_cmd)
 
     @patch('__builtin__.open')
-    @patch('charmhelpers.contrib.openstack.openstack_utils.juju_log')
-    @patch('charmhelpers.contrib.openstack.openstack_utils.import_key')
+    @patch('charmhelpers.contrib.openstack.utils.juju_log')
+    @patch('charmhelpers.contrib.openstack.utils.import_key')
     def test_configure_install_source_deb_url(self, _import, _log, _open):
         '''Test configuring installation source from deb repo url'''
         _file = MagicMock(spec=file)
@@ -267,12 +262,12 @@
         openstack.configure_installation_source(src)
         _file.__enter__().write.assert_called_with(src)
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.error_out')
+    @patch('charmhelpers.contrib.openstack.utils.error_out')
     def test_configure_bad_install_source(self, _error):
         openstack.configure_installation_source('foo')
         _error.assert_called_with('Invalid openstack-release specified: foo')
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.lsb_release')
+    @patch('charmhelpers.contrib.openstack.utils.lsb_release')
     def test_configure_install_source_uca_staging(self, _lsb):
         '''Test configuring installation source from UCA staging sources'''
         _lsb.return_value = FAKE_RELEASE
@@ -285,8 +280,8 @@
             _subp.assert_called_with(cmd)
 
     @patch('__builtin__.open')
-    @patch('charmhelpers.contrib.openstack.openstack_utils.import_key')
-    @patch('charmhelpers.contrib.openstack.openstack_utils.lsb_release')
+    @patch('charmhelpers.contrib.openstack.utils.import_key')
+    @patch('charmhelpers.contrib.openstack.utils.lsb_release')
     def test_configure_install_source_uca_repos(self, _lsb, _import, _open):
         '''Test configuring installation source from UCA sources'''
         _lsb.return_value = FAKE_RELEASE
@@ -301,7 +296,7 @@
             )
             _file.__enter__().write.assert_called_with(url)
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.error_out')
+    @patch('charmhelpers.contrib.openstack.utils.error_out')
     def test_configure_install_source_bad_uca(self, mocked_error):
         '''Test configuring installation source from bad UCA source'''
         try:
@@ -321,7 +316,7 @@
                    '--recv-keys', 'foo']
             _subp.assert_called_with(cmd)
 
-    @patch('charmhelpers.contrib.openstack.openstack_utils.error_out')
+    @patch('charmhelpers.contrib.openstack.utils.error_out')
     def test_import_bad_apt_key(self, mocked_error):
         '''Ensure error when importing apt key fails'''
         with patch('subprocess.check_call') as _subp: