Merge lp:~james-page/charms/precise/mysql/refactoring-review into lp:~openstack-charmers/charms/precise/mysql/ha-support

[James Page]

Review and refactoring so we can actually propose this against lp:charms/mysql

1) I've reworked the ceph and utils package into a lib top-level package and synced with the openstack charm helpers as written for swift-proxy (and in a bit keystone). This makes it clear which parts are sourced elsewhere and which parts are core the the charm

2) renamed hook targets to .py

ha-relations -> ha_relations.py
shared-db-relations -> shared_db_relations.py

3) Made shared_db_relations use as much of lib/* as possible

4) Reworked ha_relations to use lib/utils.py versions of _get methods, dropped config dictionary.

5) PEP-8 tidied what I touched.

6) is_relation_made()

I left this in ha_relations.py for the time being but I did add an optional key; this is used specifically in the ceph check to ensure that the relation is made and is actually usable - otherwise HA will start before ceph has provided keys etc.etc.etc.

[Andres Rodriguez (andreserl)]

Looks sane to me. Awaiting Adam's review.

review: Approve

[lp:~james-page/charms/precise/mysql/refactoring-review]

111. By James Page on 2013-03-15

Remove redundant storage marker

112. By James Page on 2013-03-15

Rebased on trunk charm

[Adam Gandelman (gandelman-a)]

+1 This looks great, James!

review: Approve

[lp:~james-page/charms/precise/mysql/refactoring-review]

113. By James Page on 2013-03-16

Re-sync with ha-helper

=== added file 'README.md'
--- README.md	1970-01-01 00:00:00 +0000
+++ README.md	2013-03-16 07:49:20 +0000
@@ -0,0 +1,98 @@
+# Overview
+
+MySQL is a fast, stable and true multi-user, multi-threaded SQL database server.
+SQL (Structured Query Language) is the most popular database query language in
+the world. The main goals of MySQL are speed, robustness and ease of use.
+
+Percona is fork of MySQL by Percona Inc. which focuses on maximizing
+performance, particularly for heavy workloads. It is a drop-in replacement for
+MySQL and features XtraDB, a drop-in replacement for the InnoDB storage engine.
+
+[http://www.mysql.com](http://www.mysql.com)
+
+[http://www.percona.com/software/percona-server](http://www.percona.com/software/percona-server)
+
+# Usage
+
+## General
+
+To deploy a MySQL service:
+
+    juju deploy mysql
+
+Once deployed, you can ssh into the deployed service and access the
+MySQL console as the MySQL root user:
+
+    juju ssh <unit>
+    mysql -u root -p
+    # enter root password - /var/lib/juju/mysql.passwd
+
+To change deployment to a Percona server:
+
+    juju set mysql flavor=percona
+
+## Optimization
+
+You can tweak various options to optimize your MySQL deployment:
+
+* max-connections - Maximum connections allowed to server or '-1' for default.
+
+* preferred-storage-engine - A comma separated list of storage engines to
+  optimize for. First in the list is marked as default storage engine. 'InnoDB'
+  or 'MyISAM' are acceptable values.
+
+* tuning-level - Specify 'safest', 'fast' or 'unsafe' to choose required
+  transaction safety. This option determines the flush value for innodb commit
+  and binary logs. Specify 'safest' for full ACID compliance. 'fast' relaxes the
+  compliance for performance and 'unsafe' will remove most restrictions.
+
+* dataset-size - Memory allocation for all caches (InnoDB buffer pool, MyISAM
+  key, query). Suffix value with 'K', 'M', 'G' or 'T' to indicate unit of
+  kilobyte, megabyte, gigabyte or terabyte respectively. Suffix value with '%'
+  to use percentage of machine's total memory.
+
+* query-cache-type - Specify 'ON', 'DEMAND' or 'OFF' to turn query cache on,
+  selectively (dependent on queries) or off.
+
+* query-cache-size - Size of query cache (no. of bytes) or '-1' to use 20%
+  of memory allocation.
+
+Each of these can be applied by running:
+
+    juju set <service> <option>=<value>
+
+e.g.
+
+    juju set mysql preferred-storage-engine=InnoDB
+    juju set mysql dataset-size=50%
+    juju set mysql query-cache-type=ON
+    juju set mysql query-cache-size=-1
+
+## Replication
+
+MySQL supports the ability to replicate databases to slave instances. This
+allows you, for example, to load balance read queries across multiple slaves or
+use a slave to perform backups, all whilst not impeding the master's
+performance.
+
+To deploy a slave:
+
+    # deploy second service
+    juju deploy mysql mysql-slave
+
+    # add master to slave relation
+    juju add-relation mysql:master mysql-slave:slave
+
+Any changes to the master are reflected on the slave.
+
+Any queries that modify the database(s) should be applied to the master only.
+The slave should be treated strictly as read only.
+
+You can add further slaves with:
+
+    juju add-unit mysql-slave
+
+## Monitoring
+
+This charm provides relations that support monitoring via either Nagios or
+Munin. Refer to the appropriate charm for usage.

=== modified file 'config.yaml'
--- config.yaml	2013-02-19 21:11:35 +0000
+++ config.yaml	2013-03-16 07:49:20 +0000
@@ -33,7 +33,7 @@
         description: If binlogging is enabled, this is the format that will be used. Ignored when tuning-level == fast.
     vip:
         type: string
-        description: "Virtual IP to use to front keystone in ha configuration"
+        description: "Virtual IP to use to front mysql in ha configuration"
     vip_iface:
         type: string
         default: eth0
@@ -55,11 +55,11 @@
           Default multicast port number that will be used to communicate between
           HA Cluster nodes.
     block-size:
-        type: string
-        default: 5G
+        type: int
+        default: 5
         description: |
           Default block storage size to create when setting up MySQL block storage.
-          This value should be specified in GB (e.g. 100G).
+          This value should be specified in GB (e.g. 100 not 100GB).
     rbd-name:
         type: string
         default: mysql1

=== modified symlink 'hooks/ceph-relation-changed'
=== target changed u'ha-relations' => u'ha_relations.py'
=== modified symlink 'hooks/ceph-relation-joined'
=== target changed u'ha-relations' => u'ha_relations.py'
=== removed file 'hooks/ceph.py'
--- hooks/ceph.py	2013-02-28 00:01:30 +0000
+++ hooks/ceph.py	1970-01-01 00:00:00 +0000
@@ -1,240 +0,0 @@
-import utils
-import commands
-import re
-import subprocess
-import sys
-import os
-import shutil
-
-KEYRING = '/etc/ceph/ceph.client.%s.keyring'
-KEYFILE = '/etc/ceph/ceph.client.%s.key'
-
-CEPH_CONF = """[global]
- auth supported = %(auth)s
- keyring = %(keyring)s
- mon host = %(mon_hosts)s
-"""
-
-def execute(cmd):
-    subprocess.check_call(cmd)
-
-
-def execute_shell(cmd):
-    subprocess.check_call(cmd, shell=True)
-
-
-def install():
-    ceph_dir = "/etc/ceph"
-    if not os.path.isdir(ceph_dir):
-        os.mkdir(ceph_dir)
-    utils.install('ceph-common')
-
-
-def rbd_exists(service, pool, rbd_img):
-    (rc, out) = commands.getstatusoutput('rbd list --id %s --pool %s' %\
-                                         (service, pool))
-    return rbd_img in out
-
-
-def create_rbd_image(service, pool, image, sizemb):
-    cmd = [
-        'rbd',
-        'create',
-        image,
-        '--size',
-        str(sizemb),
-        '--id',
-        service,
-        '--pool',
-        pool
-        ]
-    execute(cmd)
-
-
-def pool_exists(service, name):
-    (rc, out) = commands.getstatusoutput("rados --id %s lspools" % service)
-    return name in out
-
-def create_pool(service, name):
-    cmd = [
-        'rados',
-        '--id',
-        service,
-        'mkpool',
-        name
-        ]
-    execute(cmd)
-
-
-def keyfile_path(service):
-    return KEYFILE % service
-
-def keyring_path(service):
-    return KEYRING % service
-
-def create_keyring(service, key):
-    keyring = keyring_path(service)
-    if os.path.exists(keyring):
-        utils.juju_log('INFO', 'ceph: Keyring exists at %s.' % keyring)
-    cmd = [
-        'ceph-authtool',
-        keyring,
-        '--create-keyring',
-        '--name=client.%s' % service,
-        '--add-key=%s' % key
-        ]
-    execute(cmd)
-    utils.juju_log('INFO', 'ceph: Created new ring at %s.' % keyring)
-
-
-def create_key_file(service, key):
-    # create a file containing the key
-    keyfile = keyfile_path(service)
-    if os.path.exists(keyfile):
-        utils.juju_log('INFO', 'ceph: Keyfile exists at %s.'  % keyfile)
-    fd = open(keyfile, 'w')
-    fd.write(key)
-    fd.close()
-    utils.juju_log('INFO', 'ceph: Created new keyfile at %s.'  % keyfile)
-
-
-def get_ceph_nodes():
-    hosts = []
-    for r_id in utils.relation_ids('ceph'):
-        for unit in utils.relation_list(r_id):
-            hosts.append(utils.relation_get('private-address',
-                                            unit=unit, rid=r_id))
-    return hosts
-
-
-def configure(service, key, auth):
-    create_keyring(service, key)
-    create_key_file(service, key)
-    hosts = get_ceph_nodes()
-    mon_hosts = ",".join(map(str, hosts))
-    keyring = keyring_path(service)
-    with open('/etc/ceph/ceph.conf', 'w') as ceph_conf:
-        ceph_conf.write(CEPH_CONF % locals())
-    modprobe_kernel_module('rbd')
-
-
-def image_mapped(image_name):
-    (rc, out) = commands.getstatusoutput('rbd showmapped')
-    return image_name in out
-
-def map_block_storage(service, pool, image):
-    cmd = [
-        'rbd',
-        'map',
-        '%s/%s' % (pool, image),
-        '--user',
-        service,
-        '--secret',
-        keyfile_path(service),
-        ]
-    execute(cmd)
-
-
-def filesystem_mounted(fs):
-    return subprocess.call(['grep', '-wqs', fs, '/proc/mounts']) == 0
-
-def make_filesystem(blk_device, fstype='ext4'):
-    utils.juju_log('INFO',
-                   'ceph: Formatting block device %s as filesystem %s.' %\
-                   (blk_device, fstype))
-    cmd = ['mkfs', '-t', fstype, blk_device]
-    execute(cmd)
-
-
-def place_data_on_ceph(service, blk_device, data_src_dst, fstype='ext4'):
-    # mount block device into /mnt
-    cmd = ['mount', '-t', fstype, blk_device, '/mnt']
-    execute(cmd)
-
-    # copy data to /mnt
-    try:
-        copy_files(data_src_dst, '/mnt')
-    except:
-        pass
-
-    # umount block device
-    cmd = ['umount', '/mnt']
-    execute(cmd)
-
-    _dir = os.stat(data_src_dst)
-    uid = _dir.st_uid
-    gid = _dir.st_gid
-
-    # re-mount where the data should originally be
-    cmd = ['mount', '-t', fstype, blk_device, data_src_dst]
-    execute(cmd)
-
-    # ensure original ownership of new mount.
-    cmd = ['chown', '-R', '%s:%s' % (uid, gid), data_src_dst]
-    execute(cmd)
-
-# TODO: re-use
-def modprobe_kernel_module(module):
-    utils.juju_log('INFO','Loading kernel module')
-    cmd = ['modprobe', module]
-    execute(cmd)
-    cmd = 'echo %s >> /etc/modules' % module
-    execute_shell(cmd)
-
-
-def copy_files(src, dst, symlinks=False, ignore=None):
-    for item in os.listdir(src):
-        s = os.path.join(src, item)
-        d = os.path.join(dst, item)
-        if os.path.isdir(s):
-            shutil.copytree(s, d, symlinks, ignore)
-        else:
-            shutil.copy2(s, d)
-
-def ensure_ceph_storage(service, pool, rbd_img, sizemb, mount_point,
-                        blk_device, fstype, system_services=[]):
-    """
-    To be called from the current cluster leader.
-    Ensures given pool and RBD image exists, is mapped to a block device,
-    and the device is formatted and mounted at the given mount_point.
-
-    If formatting a device for the first time, data existing at mount_point
-    will be migrated to the RBD device before being remounted.
-
-    All services listed in system_services will be stopped prior to data
-    migration and restarted when complete.
-    """
-    # Ensure pool, RBD image, RBD mappings are in place.
-    if not pool_exists(service, pool):
-        utils.juju_log('INFO', 'ceph: Creating new pool %s.' % pool)
-        create_pool(service, pool)
-
-    if not rbd_exists(service, pool, rbd_img):
-        utils.juju_log('INFO', 'ceph: Creating RBD image (%s).' % rbd_img)
-        create_rbd_image(service, pool, rbd_img, sizemb)
-
-    if not image_mapped(rbd_img):
-        utils.juju_log('INFO', 'ceph: Mapping RBD Image as a Block Device.')
-        map_block_storage(service, pool, rbd_img)
-
-    # make file system
-    # TODO: What happens if for whatever reason this is run again and
-    # the data is already in the rbd device and/or is mounted??
-    # When it is mounted already, it will fail to make the fs
-    # XXX: This is really sketchy!  Need to at least add an fstab entry
-    #      otherwise this hook will blow away existing data if its executed
-    #      after a reboot.
-    if not filesystem_mounted(mount_point):
-        make_filesystem(blk_device, fstype)
-
-        for svc in system_services:
-            if utils.running(svc):
-                utils.juju_log('INFO',
-                               'Stopping services %s prior to migrating '\
-                               'data' % svc)
-                utils.stop(svc)
-
-        place_data_on_ceph(service, blk_device, mount_point, fstype)
-
-        for svc in system_services:
-            utils.start(svc)

=== added symlink 'hooks/cluster-relation-changed'
=== target is u'ha_relations.py'
=== removed symlink 'hooks/cluster-relation-changed'
=== target was u'ha-relations'
=== modified file 'hooks/config-changed'
--- hooks/config-changed	2013-02-19 21:22:51 +0000
+++ hooks/config-changed	2013-03-16 07:49:20 +0000
@@ -7,7 +7,6 @@
 import hashlib
 import os
 import sys
-import utils
 from string import upper
 
 num_re = re.compile('^[0-9]+$')

=== modified symlink 'hooks/ha-relation-changed'
=== target changed u'ha-relations' => u'ha_relations.py'
=== modified symlink 'hooks/ha-relation-joined'
=== target changed u'ha-relations' => u'ha_relations.py'
=== renamed file 'hooks/ha-relations' => 'hooks/ha_relations.py'
--- hooks/ha-relations	2013-03-01 20:56:10 +0000
+++ hooks/ha_relations.py	2013-03-16 07:49:20 +0000
@@ -1,84 +1,72 @@
 #!/usr/bin/env python
 
-import json
 import sys
-import subprocess
 import os
-import time
-import commands
-
-import utils
-import ceph
-
-STORAGEMARKER = '/var/lib/juju/storageconfigured'
+
+import lib.utils as utils
+import lib.ceph_utils as ceph
+import lib.cluster_utils as cluster
 
 # CEPH
 DATA_SRC_DST = '/var/lib/mysql'
-
 SERVICE_NAME = os.getenv('JUJU_UNIT_NAME').split('/')[0]
 POOL_NAME = SERVICE_NAME
-
-config=json.loads(subprocess.check_output(['config-get','--format=json']))
+LEADER_RES = 'res_mysql_vip'
 
 
 def ha_relation_joined():
-
-    # Checking vip values
-    if not 'vip' in config:
-        utils.juju_log('WARNING', 'NO Virtual IP was defined, bailing')
-        sys.exit(1)
-
-    if config['vip_iface'] == "None" or not config['vip_iface']:
-        utils.juju_log('WARNING', 'NO Virtual IP interface was defined, bailing')
-        sys.exit(1)
-
-    if config['vip_cidr'] == "None" or not config['vip_cidr']:
-        utils.juju_log('WARNING', 'NO CIDR was defined for the Virtual IP, bailing')
-        sys.exit(1)
-
-    # Obtain the config values necessary for the cluster config. These
-    # include multicast port and interface to bind to.
-    corosync_bindiface = config['ha-bindiface']
-    corosync_mcastport = config['ha-mcastport']
+    vip = utils.config_get('vip')
+    vip_iface = utils.config_get('vip_iface')
+    vip_cidr = utils.config_get('vip_cidr')
+    corosync_bindiface = utils.config_get('ha-bindiface')
+    corosync_mcastport = utils.config_get('ha-mcastport')
+
+    if None in [vip, vip_cidr, vip_iface]:
+        utils.juju_log('WARNING',
+                       'Insufficient VIP information to configure cluster')
+        sys.exit(1)
 
     # Starting configuring resources.
     init_services = {
-            'res_mysqld':'mysql',
+            'res_mysqld': 'mysql',
         }
 
-
     # If the 'ha' relation has been made *before* the 'ceph' relation,
-    # it doesn't make sense to make it until after the 'ceph' relation
-    # is made
-    if not utils.is_relation_made('ceph'):
+    # it doesn't make sense to make it until after the 'ceph' relation is made
+    if not is_relation_made('ceph', 'auth'):
         utils.juju_log('INFO',
-                '*ceph* relation does not exist. Not sending *ha* relation data')
+                       '*ceph* relation does not exist. '
+                       'Not sending *ha* relation data yet')
         return
     else:
         utils.juju_log('INFO',
-                    '*ceph* relation exists. Sending *ha* relation data')
+                       '*ceph* relation exists. Sending *ha* relation data')
 
         block_storage = 'ceph'
 
         resources = {
-                'res_mysql_rbd':'ocf:ceph:rbd',
-                'res_mysql_fs':'ocf:heartbeat:Filesystem',
-                'res_mysql_vip':'ocf:heartbeat:IPaddr2',
-                'res_mysqld':'upstart:mysql',
+            'res_mysql_rbd': 'ocf:ceph:rbd',
+            'res_mysql_fs': 'ocf:heartbeat:Filesystem',
+            'res_mysql_vip': 'ocf:heartbeat:IPaddr2',
+            'res_mysqld': 'upstart:mysql',
             }
 
+        rbd_name = utils.config_get('rbd-name')
         resource_params = {
-                'res_mysql_rbd':'params name="%s" pool="%s" user="%s" secret="%s"' % (
-                                config['rbd-name'], POOL_NAME, SERVICE_NAME, ceph.keyfile_path(SERVICE_NAME)),
-                'res_mysql_fs':'params device="/dev/rbd/%s/%s" directory="%s" fstype="ext4" op start start-delay="10s"' % (
-                                POOL_NAME, config['rbd-name'], DATA_SRC_DST),
-                'res_mysql_vip':'params ip="%s" cidr_netmask="%s" nic="%s"' % (config['vip'],
-                                config['vip_cidr'], config['vip_iface']),
-                'res_mysqld':'op start start-delay="5s" op monitor interval="5s"',
+            'res_mysql_rbd': 'params name="%s" pool="%s" user="%s" '
+                             'secret="%s"' % \
+                             (rbd_name, POOL_NAME,
+                              SERVICE_NAME, ceph.keyfile_path(SERVICE_NAME)),
+            'res_mysql_fs': 'params device="/dev/rbd/%s/%s" directory="%s" '
+                            'fstype="ext4" op start start-delay="10s"' % \
+                            (POOL_NAME, rbd_name, DATA_SRC_DST),
+            'res_mysql_vip': 'params ip="%s" cidr_netmask="%s" nic="%s"' % \
+                             (vip, vip_cidr, vip_iface),
+            'res_mysqld': 'op start start-delay="5s" op monitor interval="5s"',
             }
 
         groups = {
-                'grp_mysql':'res_mysql_rbd res_mysql_fs res_mysql_vip res_mysqld',
+            'grp_mysql': 'res_mysql_rbd res_mysql_fs res_mysql_vip res_mysqld',
             }
 
         for rel_id in utils.relation_ids('ha'):
@@ -93,15 +81,13 @@
 
 
 def ha_relation_changed():
-    relation_data = utils.relation_get_dict()
-    if ('clustered' in relation_data and
-        utils.is_leader()):
+    clustered = utils.relation_get('clustered')
+    if (clustered and cluster.is_leader(LEADER_RES)):
         utils.juju_log('INFO', 'Cluster configured, notifying other services')
-        # Tell all related services to start using
-        # the VIP
+        # Tell all related services to start using the VIP
         for r_id in utils.relation_ids('shared-db'):
             utils.relation_set(rid=r_id,
-                           db_host=config['vip'])
+                               db_host=utils.config_get('vip'))
 
 
 def ceph_joined():
@@ -112,21 +98,17 @@
 
 def ceph_changed():
     utils.juju_log('INFO', 'Start Ceph Relation Changed')
-
-    # TODO: ask james: What happens if the relation data has changed?
-    # do we reconfigure ceph? What do we do with the data?
     auth = utils.relation_get('auth')
     key = utils.relation_get('key')
     if None in [auth, key]:
         utils.juju_log('INFO', 'Missing key or auth in relation')
-        sys.exit(0)
-
+        return
 
     ceph.configure(service=SERVICE_NAME, key=key, auth=auth)
 
-    if utils.eligible_leader():
-        sizemb = int(config['block-size'].split('G')[0]) * 1024
-        rbd_img = config['rbd-name']
+    if cluster.eligible_leader(LEADER_RES):
+        sizemb = int(utils.config_get('block-size')) * 1024
+        rbd_img = utils.config_get('rbd-name')
         blk_device = '/dev/rbd/%s/%s' % (POOL_NAME, rbd_img)
         ceph.ensure_ceph_storage(service=SERVICE_NAME, pool=POOL_NAME,
                                  rbd_img=rbd_img, sizemb=sizemb,
@@ -138,46 +120,39 @@
                        'This is not the peer leader. Not configuring RBD.')
         # Stopping MySQL
         if utils.running('mysql'):
-            utils.juju_log('INFO','Stopping MySQL...')
+            utils.juju_log('INFO', 'Stopping MySQL...')
             utils.stop('mysql')
 
-
     # If 'ha' relation has been made before the 'ceph' relation
     # it is important to make sure the ha-relation data is being
     # sent.
-    if utils.is_relation_made('ha'):
+    if is_relation_made('ha'):
         utils.juju_log('INFO',
-                       '*ha* relation exists. Making sure the ha relation data is sent.')
+                       '*ha* relation exists. Making sure the ha'
+                       ' relation data is sent.')
         ha_relation_joined()
         return
-    else:
-        utils.juju_log('INFO',
-                       '*ha* relation does not exist.')
 
     utils.juju_log('INFO', 'Finish Ceph Relation Changed')
 
 
-def cluster_changed():
-    utils.juju_log('INFO', 'Begin cluster changed hook.')
-
-    if config['block-size'] == "None":
-        utils.juju_log('WARNING', 'NO block storage size configured, bailing')
-        return
-
-    utils.juju_log('INFO', 'End install hook.')
+def is_relation_made(relation, key='private-address'):
+    relation_data = []
+    for r_id in (utils.relation_ids(relation) or []):
+        for unit in (utils.relation_list(r_id) or []):
+            relation_data.append(utils.relation_get(key,
+                                                    rid=r_id,
+                                                    unit=unit))
+    if not relation_data:
+        return False
+    return True
 
 
 hooks = {
-    "cluster-relation-changed": cluster_changed,
     "ha-relation-joined": ha_relation_joined,
     "ha-relation-changed": ha_relation_changed,
     "ceph-relation-joined": ceph_joined,
     "ceph-relation-changed": ceph_changed,
 }
 
-# keystone-hooks gets called by symlink corresponding to the requested relation
-# hook.
-arg0 = sys.argv[0].split("/").pop()
-if arg0 not in hooks.keys():
-    error_out("Unsupported hook: %s" % arg0)
-hooks[arg0]()
+utils.do_hooks(hooks)

=== added directory 'hooks/lib'
=== added file 'hooks/lib/__init__.py'
=== added file 'hooks/lib/ceph_utils.py'
--- hooks/lib/ceph_utils.py	1970-01-01 00:00:00 +0000
+++ hooks/lib/ceph_utils.py	2013-03-16 07:49:20 +0000
@@ -0,0 +1,256 @@
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This file is sourced from lp:openstack-charm-helpers
+#
+# Authors:
+#  James Page <[e-mail]>
+#  Adam Gandelman <[e-mail]>
+#
+
+import commands
+import subprocess
+import os
+import shutil
+import lib.utils as utils
+
+KEYRING = '/etc/ceph/ceph.client.%s.keyring'
+KEYFILE = '/etc/ceph/ceph.client.%s.key'
+
+CEPH_CONF = """[global]
+ auth supported = %(auth)s
+ keyring = %(keyring)s
+ mon host = %(mon_hosts)s
+"""
+
+
+def execute(cmd):
+    subprocess.check_call(cmd)
+
+
+def execute_shell(cmd):
+    subprocess.check_call(cmd, shell=True)
+
+
+def install():
+    ceph_dir = "/etc/ceph"
+    if not os.path.isdir(ceph_dir):
+        os.mkdir(ceph_dir)
+    utils.install('ceph-common')
+
+
+def rbd_exists(service, pool, rbd_img):
+    (rc, out) = commands.getstatusoutput('rbd list --id %s --pool %s' %\
+                                         (service, pool))
+    return rbd_img in out
+
+
+def create_rbd_image(service, pool, image, sizemb):
+    cmd = [
+        'rbd',
+        'create',
+        image,
+        '--size',
+        str(sizemb),
+        '--id',
+        service,
+        '--pool',
+        pool
+        ]
+    execute(cmd)
+
+
+def pool_exists(service, name):
+    (rc, out) = commands.getstatusoutput("rados --id %s lspools" % service)
+    return name in out
+
+
+def create_pool(service, name):
+    cmd = [
+        'rados',
+        '--id',
+        service,
+        'mkpool',
+        name
+        ]
+    execute(cmd)
+
+
+def keyfile_path(service):
+    return KEYFILE % service
+
+
+def keyring_path(service):
+    return KEYRING % service
+
+
+def create_keyring(service, key):
+    keyring = keyring_path(service)
+    if os.path.exists(keyring):
+        utils.juju_log('INFO', 'ceph: Keyring exists at %s.' % keyring)
+    cmd = [
+        'ceph-authtool',
+        keyring,
+        '--create-keyring',
+        '--name=client.%s' % service,
+        '--add-key=%s' % key
+        ]
+    execute(cmd)
+    utils.juju_log('INFO', 'ceph: Created new ring at %s.' % keyring)
+
+
+def create_key_file(service, key):
+    # create a file containing the key
+    keyfile = keyfile_path(service)
+    if os.path.exists(keyfile):
+        utils.juju_log('INFO', 'ceph: Keyfile exists at %s.' % keyfile)
+    fd = open(keyfile, 'w')
+    fd.write(key)
+    fd.close()
+    utils.juju_log('INFO', 'ceph: Created new keyfile at %s.' % keyfile)
+
+
+def get_ceph_nodes():
+    hosts = []
+    for r_id in utils.relation_ids('ceph'):
+        for unit in utils.relation_list(r_id):
+            hosts.append(utils.relation_get('private-address',
+                                            unit=unit, rid=r_id))
+    return hosts
+
+
+def configure(service, key, auth):
+    create_keyring(service, key)
+    create_key_file(service, key)
+    hosts = get_ceph_nodes()
+    mon_hosts = ",".join(map(str, hosts))
+    keyring = keyring_path(service)
+    with open('/etc/ceph/ceph.conf', 'w') as ceph_conf:
+        ceph_conf.write(CEPH_CONF % locals())
+    modprobe_kernel_module('rbd')
+
+
+def image_mapped(image_name):
+    (rc, out) = commands.getstatusoutput('rbd showmapped')
+    return image_name in out
+
+
+def map_block_storage(service, pool, image):
+    cmd = [
+        'rbd',
+        'map',
+        '%s/%s' % (pool, image),
+        '--user',
+        service,
+        '--secret',
+        keyfile_path(service),
+        ]
+    execute(cmd)
+
+
+def filesystem_mounted(fs):
+    return subprocess.call(['grep', '-wqs', fs, '/proc/mounts']) == 0
+
+
+def make_filesystem(blk_device, fstype='ext4'):
+    utils.juju_log('INFO',
+                   'ceph: Formatting block device %s as filesystem %s.' %\
+                   (blk_device, fstype))
+    cmd = ['mkfs', '-t', fstype, blk_device]
+    execute(cmd)
+
+
+def place_data_on_ceph(service, blk_device, data_src_dst, fstype='ext4'):
+    # mount block device into /mnt
+    cmd = ['mount', '-t', fstype, blk_device, '/mnt']
+    execute(cmd)
+
+    # copy data to /mnt
+    try:
+        copy_files(data_src_dst, '/mnt')
+    except:
+        pass
+
+    # umount block device
+    cmd = ['umount', '/mnt']
+    execute(cmd)
+
+    _dir = os.stat(data_src_dst)
+    uid = _dir.st_uid
+    gid = _dir.st_gid
+
+    # re-mount where the data should originally be
+    cmd = ['mount', '-t', fstype, blk_device, data_src_dst]
+    execute(cmd)
+
+    # ensure original ownership of new mount.
+    cmd = ['chown', '-R', '%s:%s' % (uid, gid), data_src_dst]
+    execute(cmd)
+
+
+# TODO: re-use
+def modprobe_kernel_module(module):
+    utils.juju_log('INFO', 'Loading kernel module')
+    cmd = ['modprobe', module]
+    execute(cmd)
+    cmd = 'echo %s >> /etc/modules' % module
+    execute_shell(cmd)
+
+
+def copy_files(src, dst, symlinks=False, ignore=None):
+    for item in os.listdir(src):
+        s = os.path.join(src, item)
+        d = os.path.join(dst, item)
+        if os.path.isdir(s):
+            shutil.copytree(s, d, symlinks, ignore)
+        else:
+            shutil.copy2(s, d)
+
+
+def ensure_ceph_storage(service, pool, rbd_img, sizemb, mount_point,
+                        blk_device, fstype, system_services=[]):
+    """
+    To be called from the current cluster leader.
+    Ensures given pool and RBD image exists, is mapped to a block device,
+    and the device is formatted and mounted at the given mount_point.
+
+    If formatting a device for the first time, data existing at mount_point
+    will be migrated to the RBD device before being remounted.
+
+    All services listed in system_services will be stopped prior to data
+    migration and restarted when complete.
+    """
+    # Ensure pool, RBD image, RBD mappings are in place.
+    if not pool_exists(service, pool):
+        utils.juju_log('INFO', 'ceph: Creating new pool %s.' % pool)
+        create_pool(service, pool)
+
+    if not rbd_exists(service, pool, rbd_img):
+        utils.juju_log('INFO', 'ceph: Creating RBD image (%s).' % rbd_img)
+        create_rbd_image(service, pool, rbd_img, sizemb)
+
+    if not image_mapped(rbd_img):
+        utils.juju_log('INFO', 'ceph: Mapping RBD Image as a Block Device.')
+        map_block_storage(service, pool, rbd_img)
+
+    # make file system
+    # TODO: What happens if for whatever reason this is run again and
+    # the data is already in the rbd device and/or is mounted??
+    # When it is mounted already, it will fail to make the fs
+    # XXX: This is really sketchy!  Need to at least add an fstab entry
+    #      otherwise this hook will blow away existing data if its executed
+    #      after a reboot.
+    if not filesystem_mounted(mount_point):
+        make_filesystem(blk_device, fstype)
+
+        for svc in system_services:
+            if utils.running(svc):
+                utils.juju_log('INFO',
+                               'Stopping services %s prior to migrating '\
+                               'data' % svc)
+                utils.stop(svc)
+
+        place_data_on_ceph(service, blk_device, mount_point, fstype)
+
+        for svc in system_services:
+            utils.start(svc)

=== added file 'hooks/lib/cluster_utils.py'
--- hooks/lib/cluster_utils.py	1970-01-01 00:00:00 +0000
+++ hooks/lib/cluster_utils.py	2013-03-16 07:49:20 +0000
@@ -0,0 +1,130 @@
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This file is sourced from lp:openstack-charm-helpers
+#
+# Authors:
+#  James Page <[e-mail]>
+#  Adam Gandelman <[e-mail]>
+#
+
+from lib.utils import (
+    juju_log,
+    relation_ids,
+    relation_list,
+    relation_get,
+    get_unit_hostname,
+    config_get
+    )
+import subprocess
+import os
+
+
+def is_clustered():
+    for r_id in (relation_ids('ha') or []):
+        for unit in (relation_list(r_id) or []):
+            clustered = relation_get('clustered',
+                                     rid=r_id,
+                                     unit=unit)
+            if clustered:
+                return True
+    return False
+
+
+def is_leader(resource):
+    cmd = [
+        "crm", "resource",
+        "show", resource
+        ]
+    try:
+        status = subprocess.check_output(cmd)
+    except subprocess.CalledProcessError:
+        return False
+    else:
+        if get_unit_hostname() in status:
+            return True
+        else:
+            return False
+
+
+def peer_units():
+    peers = []
+    for r_id in (relation_ids('cluster') or []):
+        for unit in (relation_list(r_id) or []):
+            peers.append(unit)
+    return peers
+
+
+def oldest_peer(peers):
+    local_unit_no = os.getenv('JUJU_UNIT_NAME').split('/')[1]
+    for peer in peers:
+        remote_unit_no = peer.split('/')[1]
+        if remote_unit_no < local_unit_no:
+            return False
+    return True
+
+
+def eligible_leader(resource):
+    if is_clustered():
+        if not is_leader(resource):
+            juju_log('INFO', 'Deferring action to CRM leader.')
+            return False
+    else:
+        peers = peer_units()
+        if peers and not oldest_peer(peers):
+            juju_log('INFO', 'Deferring action to oldest service unit.')
+            return False
+    return True
+
+
+def https():
+    '''
+    Determines whether enough data has been provided in configuration
+    or relation data to configure HTTPS
+    .
+    returns: boolean
+    '''
+    if config_get('use-https') == "yes":
+        return True
+    if config_get('ssl_cert') and config_get('ssl_key'):
+        return True
+    for r_id in relation_ids('identity-service'):
+        for unit in relation_list(r_id):
+            if (relation_get('https_keystone', rid=r_id, unit=unit) and
+                relation_get('ssl_cert', rid=r_id, unit=unit) and
+                relation_get('ssl_key', rid=r_id, unit=unit) and
+                relation_get('ca_cert', rid=r_id, unit=unit)):
+                return True
+    return False
+
+
+def determine_api_port(public_port):
+    '''
+    Determine correct API server listening port based on
+    existence of HTTPS reverse proxy and/or haproxy.
+
+    public_port: int: standard public port for given service
+
+    returns: int: the correct listening port for the API service
+    '''
+    i = 0
+    if len(peer_units()) > 0 or is_clustered():
+        i += 1
+    if https():
+        i += 1
+    return public_port - (i * 10)
+
+
+def determine_haproxy_port(public_port):
+    '''
+    Description: Determine correct proxy listening port based on public IP +
+    existence of HTTPS reverse proxy.
+
+    public_port: int: standard public port for given service
+
+    returns: int: the correct listening port for the HAProxy service
+    '''
+    i = 0
+    if https():
+        i += 1
+    return public_port - (i * 10)

=== added file 'hooks/lib/utils.py'
--- hooks/lib/utils.py	1970-01-01 00:00:00 +0000
+++ hooks/lib/utils.py	2013-03-16 07:49:20 +0000
@@ -0,0 +1,275 @@
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This file is sourced from lp:openstack-charm-helpers
+#
+# Authors:
+#  James Page <[e-mail]>
+#  Paul Collins <[e-mail]>
+#  Adam Gandelman <[e-mail]>
+#
+
+import json
+import os
+import subprocess
+import socket
+import sys
+
+
+def do_hooks(hooks):
+    hook = os.path.basename(sys.argv[0])
+
+    try:
+        hook_func = hooks[hook]
+    except KeyError:
+        juju_log('INFO',
+                 "This charm doesn't know how to handle '{}'.".format(hook))
+    else:
+        hook_func()
+
+
+def install(*pkgs):
+    cmd = [
+        'apt-get',
+        '-y',
+        'install'
+          ]
+    for pkg in pkgs:
+        cmd.append(pkg)
+    subprocess.check_call(cmd)
+
+TEMPLATES_DIR = 'templates'
+
+try:
+    import jinja2
+except ImportError:
+    install('python-jinja2')
+    import jinja2
+
+try:
+    import dns.resolver
+except ImportError:
+    install('python-dnspython')
+    import dns.resolver
+
+
+def render_template(template_name, context, template_dir=TEMPLATES_DIR):
+    templates = jinja2.Environment(
+                    loader=jinja2.FileSystemLoader(template_dir)
+                    )
+    template = templates.get_template(template_name)
+    return template.render(context)
+
+CLOUD_ARCHIVE = \
+""" # Ubuntu Cloud Archive
+deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
+"""
+
+CLOUD_ARCHIVE_POCKETS = {
+    'folsom': 'precise-updates/folsom',
+    'folsom/updates': 'precise-updates/folsom',
+    'folsom/proposed': 'precise-proposed/folsom',
+    'grizzly': 'precise-updates/grizzly',
+    'grizzly/updates': 'precise-updates/grizzly',
+    'grizzly/proposed': 'precise-proposed/grizzly'
+    }
+
+
+def configure_source():
+    source = str(config_get('openstack-origin'))
+    if not source:
+        return
+    if source.startswith('ppa:'):
+        cmd = [
+            'add-apt-repository',
+            source
+            ]
+        subprocess.check_call(cmd)
+    if source.startswith('cloud:'):
+        install('ubuntu-cloud-keyring')
+        pocket = source.split(':')[1]
+        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
+            apt.write(CLOUD_ARCHIVE.format(CLOUD_ARCHIVE_POCKETS[pocket]))
+    if source.startswith('deb'):
+        l = len(source.split('|'))
+        if l == 2:
+            (apt_line, key) = source.split('|')
+            cmd = [
+                'apt-key',
+                'adv', '--keyserver keyserver.ubuntu.com',
+                '--recv-keys', key
+                ]
+            subprocess.check_call(cmd)
+        elif l == 1:
+            apt_line = source
+
+        with open('/etc/apt/sources.list.d/quantum.list', 'w') as apt:
+            apt.write(apt_line + "\n")
+    cmd = [
+        'apt-get',
+        'update'
+        ]
+    subprocess.check_call(cmd)
+
+# Protocols
+TCP = 'TCP'
+UDP = 'UDP'
+
+
+def expose(port, protocol='TCP'):
+    cmd = [
+        'open-port',
+        '{}/{}'.format(port, protocol)
+        ]
+    subprocess.check_call(cmd)
+
+
+def juju_log(severity, message):
+    cmd = [
+        'juju-log',
+        '--log-level', severity,
+        message
+        ]
+    subprocess.check_call(cmd)
+
+
+def relation_ids(relation):
+    cmd = [
+        'relation-ids',
+        relation
+        ]
+    result = str(subprocess.check_output(cmd)).split()
+    if result == "":
+        return None
+    else:
+        return result
+
+
+def relation_list(rid):
+    cmd = [
+        'relation-list',
+        '-r', rid,
+        ]
+    result = str(subprocess.check_output(cmd)).split()
+    if result == "":
+        return None
+    else:
+        return result
+
+
+def relation_get(attribute, unit=None, rid=None):
+    cmd = [
+        'relation-get',
+        ]
+    if rid:
+        cmd.append('-r')
+        cmd.append(rid)
+    cmd.append(attribute)
+    if unit:
+        cmd.append(unit)
+    value = subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    if value == "":
+        return None
+    else:
+        return value
+
+
+def relation_set(**kwargs):
+    cmd = [
+        'relation-set'
+        ]
+    args = []
+    for k, v in kwargs.items():
+        if k == 'rid':
+            if v:
+                cmd.append('-r')
+                cmd.append(v)
+        else:
+            args.append('{}={}'.format(k, v))
+    cmd += args
+    subprocess.check_call(cmd)
+
+
+def unit_get(attribute):
+    cmd = [
+        'unit-get',
+        attribute
+        ]
+    value = subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    if value == "":
+        return None
+    else:
+        return value
+
+
+def config_get(attribute):
+    cmd = [
+        'config-get',
+        '--format',
+        'json',
+        ]
+    out = subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    cfg = json.loads(out)
+
+    try:
+        return cfg[attribute]
+    except KeyError:
+        return None
+
+
+def get_unit_hostname():
+    return socket.gethostname()
+
+
+def get_host_ip(hostname=unit_get('private-address')):
+    try:
+        # Test to see if already an IPv4 address
+        socket.inet_aton(hostname)
+        return hostname
+    except socket.error:
+        answers = dns.resolver.query(hostname, 'A')
+        if answers:
+            return answers[0].address
+    return None
+
+
+def _svc_control(service, action):
+    subprocess.check_call(['service', service, action])
+
+
+def restart(*services):
+    for service in services:
+        _svc_control(service, 'restart')
+
+
+def stop(*services):
+    for service in services:
+        _svc_control(service, 'stop')
+
+
+def start(*services):
+    for service in services:
+        _svc_control(service, 'start')
+
+
+def reload(*services):
+    for service in services:
+        try:
+            _svc_control(service, 'reload')
+        except subprocess.CalledProcessError:
+            # Reload failed - either service does not support reload
+            # or it was not running - restart will fixup most things
+            _svc_control(service, 'restart')
+
+
+def running(service):
+    try:
+        output = subprocess.check_output(['service', service, 'status'])
+    except subprocess.CalledProcessError:
+        return False
+    else:
+        if ("start/running" in output or
+            "is running" in output):
+            return True
+        else:
+            return False

=== modified symlink 'hooks/shared-db-relation-changed'
=== target changed u'shared-db-relations' => u'shared_db_relations.py'
=== modified symlink 'hooks/shared-db-relation-joined'
=== target changed u'shared-db-relations' => u'shared_db_relations.py'
=== removed file 'hooks/shared-db-relations'
--- hooks/shared-db-relations	2013-02-19 23:11:24 +0000
+++ hooks/shared-db-relations	1970-01-01 00:00:00 +0000
@@ -1,152 +0,0 @@
-#!/usr/bin/python
-#
-# Create relations between a shared database to many peers.
-# Join does nothing.   Peer requests access to $DATABASE from $REMOTE_HOST.
-# It's up to the hooks to ensure database exists, peer has access and
-# clean up grants after a broken/departed peer (TODO)
-#
-# Author: Adam Gandelman <[e-mail]>
-
-
-from common import *
-import sys
-import subprocess
-import json
-import socket
-import os
-import utils
-
-config=json.loads(subprocess.check_output(['config-get','--format=json']))
-
-def pwgen():
-    return subprocess.check_output(['pwgen', '-s', '16']).strip()
-
-
-def relation_get():
-    return json.loads(subprocess.check_output(
-                        ['relation-get',
-                         '--format',
-                         'json']
-                        )
-                      )
-
-
-def relation_set(**kwargs):
-    cmd = [ 'relation-set' ]
-    args = []
-    for k, v in kwargs.items():
-        if k == 'rid':
-            cmd.append('-r')
-            cmd.append(v)
-        else:
-            args.append('{}={}'.format(k, v))
-    cmd += args
-    subprocess.check_call(cmd)
-
-
-def shared_db_changed():
-
-    def configure_db(hostname,
-                     database,
-                     username):
-        passwd_file = "/var/lib/mysql/mysql-{}.passwd"\
-                        .format(username)
-        if hostname != local_hostname:
-            remote_ip = socket.gethostbyname(hostname)
-        else:
-            remote_ip = '127.0.0.1'
-
-        if not os.path.exists(passwd_file):
-            password = pwgen()
-            with open(passwd_file, 'w') as pfile:
-                pfile.write(password)
-        else:
-            with open(passwd_file) as pfile:
-                password = pfile.read().strip()
-
-        if not database_exists(database):
-            create_database(database)
-        if not grant_exists(database,
-                            username,
-                            remote_ip):
-            create_grant(database,
-                         username,
-                         remote_ip, password)
-        return password
-
-    if not utils.eligible_leader():
-        utils.juju_log('INFO',
-                       'MySQL service is peered, bailing shared-db relation')
-        return
-
-    settings = relation_get()
-    local_hostname = socket.getfqdn()
-    singleset = set([ 
-        'database',
-        'username',
-        'hostname'
-        ])
-
-    if singleset.issubset(settings):
-        # Process a single database configuration
-        password = configure_db(settings['hostname'],
-                                settings['database'],
-                                settings['username'])
-        if not utils.is_clustered():
-            relation_set(db_host=local_hostname,
-                         password=password)
-        else:
-            relation_set(db_host=config["vip"],
-                         password=password)
-
-    else:
-        # Process multiple database setup requests.
-        # from incoming relation data:
-        #  nova_database=xxx nova_username=xxx nova_hostname=xxx
-        #  quantum_database=xxx quantum_username=xxx quantum_hostname=xxx
-        # create
-        #{
-        #   "nova": {
-        #        "username": xxx,
-        #        "database": xxx,
-        #        "hostname": xxx
-        #    },
-        #    "quantum": {
-        #        "username": xxx,
-        #        "database": xxx,
-        #        "hostname": xxx
-        #    }
-        #}
-        #
-        databases = {}
-        for k, v in settings.iteritems():
-            db = k.split('_')[0]
-            x = '_'.join(k.split('_')[1:])
-            if db not in databases:
-                databases[db] = {}
-            databases[db][x] = v
-        return_data = {}
-        for db in databases:
-            if singleset.issubset(databases[db]):
-                return_data['_'.join([ db, 'password' ])] = \
-                    configure_db(databases[db]['hostname'],
-                                 databases[db]['database'],
-                                 databases[db]['username'])
-        relation_set(**return_data)
-        if not utils.is_clustered():
-           relation_set(db_host=local_hostname)
-        else:
-            relation_set(db_host=config["vip"])
-
-hook = os.path.basename(sys.argv[0])
-hooks = {
-    "shared-db-relation-changed": shared_db_changed
-    }
-try:
-    hook_func = hooks[hook]
-except KeyError:
-    pass
-else:
-    hook_func()
-
-sys.exit(0)

=== added file 'hooks/shared_db_relations.py'
--- hooks/shared_db_relations.py	1970-01-01 00:00:00 +0000
+++ hooks/shared_db_relations.py	2013-03-16 07:49:20 +0000
@@ -0,0 +1,139 @@
+#!/usr/bin/python
+#
+# Create relations between a shared database to many peers.
+# Join does nothing.   Peer requests access to $DATABASE from $REMOTE_HOST.
+# It's up to the hooks to ensure database exists, peer has access and
+# clean up grants after a broken/departed peer (TODO)
+#
+# Author: Adam Gandelman <[e-mail]>
+
+
+from common import (
+    database_exists,
+    create_database,
+    grant_exists,
+    create_grant
+    )
+import subprocess
+import json
+import socket
+import os
+import lib.utils as utils
+import lib.cluster_utils as cluster
+
+LEADER_RES = 'res_mysql_vip'
+
+
+def pwgen():
+    return str(subprocess.check_output(['pwgen', '-s', '16'])).strip()
+
+
+def relation_get():
+    return json.loads(subprocess.check_output(
+                        ['relation-get',
+                         '--format',
+                         'json']
+                        )
+                      )
+
+
+def shared_db_changed():
+
+    def configure_db(hostname,
+                     database,
+                     username):
+        passwd_file = "/var/lib/mysql/mysql-{}.passwd"\
+                        .format(username)
+        if hostname != local_hostname:
+            remote_ip = socket.gethostbyname(hostname)
+        else:
+            remote_ip = '127.0.0.1'
+
+        if not os.path.exists(passwd_file):
+            password = pwgen()
+            with open(passwd_file, 'w') as pfile:
+                pfile.write(password)
+        else:
+            with open(passwd_file) as pfile:
+                password = pfile.read().strip()
+
+        if not database_exists(database):
+            create_database(database)
+        if not grant_exists(database,
+                            username,
+                            remote_ip):
+            create_grant(database,
+                         username,
+                         remote_ip, password)
+        return password
+
+    if not cluster.eligible_leader(LEADER_RES):
+        utils.juju_log('INFO',
+                       'MySQL service is peered, bailing shared-db relation'
+                       ' as this service unit is not the leader')
+        return
+
+    settings = relation_get()
+    local_hostname = socket.getfqdn()
+    singleset = set([
+        'database',
+        'username',
+        'hostname'
+        ])
+
+    if singleset.issubset(settings):
+        # Process a single database configuration
+        password = configure_db(settings['hostname'],
+                                settings['database'],
+                                settings['username'])
+        if not cluster.is_clustered():
+            utils.relation_set(db_host=local_hostname,
+                               password=password)
+        else:
+            utils.relation_set(db_host=utils.config_get("vip"),
+                               password=password)
+
+    else:
+        # Process multiple database setup requests.
+        # from incoming relation data:
+        #  nova_database=xxx nova_username=xxx nova_hostname=xxx
+        #  quantum_database=xxx quantum_username=xxx quantum_hostname=xxx
+        # create
+        #{
+        #   "nova": {
+        #        "username": xxx,
+        #        "database": xxx,
+        #        "hostname": xxx
+        #    },
+        #    "quantum": {
+        #        "username": xxx,
+        #        "database": xxx,
+        #        "hostname": xxx
+        #    }
+        #}
+        #
+        databases = {}
+        for k, v in settings.iteritems():
+            db = k.split('_')[0]
+            x = '_'.join(k.split('_')[1:])
+            if db not in databases:
+                databases[db] = {}
+            databases[db][x] = v
+        return_data = {}
+        for db in databases:
+            if singleset.issubset(databases[db]):
+                return_data['_'.join([db, 'password'])] = \
+                    configure_db(databases[db]['hostname'],
+                                 databases[db]['database'],
+                                 databases[db]['username'])
+        utils.relation_set(**return_data)
+        if not cluster.is_clustered():
+            utils.relation_set(db_host=local_hostname)
+        else:
+            utils.relation_set(db_host=utils.config_get("vip"))
+
+hooks = {
+    "shared-db-relation-changed": shared_db_changed
+    }
+
+utils.do_hooks(hooks)

=== removed file 'hooks/utils.py'
--- hooks/utils.py	2013-02-15 21:12:40 +0000
+++ hooks/utils.py	1970-01-01 00:00:00 +0000
@@ -1,446 +0,0 @@
-
-#
-# Copyright 2012 Canonical Ltd.
-#
-# Authors:
-#  James Page <[e-mail]>
-#  Paul Collins <[e-mail]>
-#
-
-import os
-import subprocess
-import socket
-import sys
-import fcntl
-import struct
-import json
-import time
-
-
-def do_hooks(hooks):
-    hook = os.path.basename(sys.argv[0])
-
-    try:
-        hooks[hook]()
-    except KeyError:
-        juju_log('INFO',
-                 "This charm doesn't know how to handle '{}'.".format(hook))
-
-
-def can_install():
-    try:
-        fd = os.open("/var/lib/dpkg/lock", os.O_RDWR|os.O_CREAT|os.O_NOFOLLOW, 0640)
-        fcntl.lockf(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
-    except IOError, message:
-        os.close(fd)
-        return False
-    else:
-        fcntl.lockf(fd, fcntl.LOCK_UN)
-        os.close(fd)
-        return True
-
-
-def install(*pkgs):
-    cmd = [
-        'apt-get',
-        '-y',
-        'install'
-          ]
-    for pkg in pkgs:
-        cmd.append(pkg)
-    while not can_install():
-        juju_log('INFO',
-                 "dpkg is busy, can't install %s yet, waiting..." % pkgs)
-        time.sleep(1)
-    subprocess.check_call(cmd)
-
-TEMPLATES_DIR = 'templates'
-
-try:
-    import jinja2
-except ImportError:
-    install('python-jinja2')
-    import jinja2
-
-try:
-    from netaddr import IPNetwork
-except ImportError:
-    install('python-netaddr')
-    from netaddr import IPNetwork
-
-try:
-    import dns.resolver
-except ImportError:
-    install('python-dnspython')
-    import dns.resolver
-
-
-def render_template(template_name, context, template_dir=TEMPLATES_DIR):
-    templates = jinja2.Environment(
-                    loader=jinja2.FileSystemLoader(template_dir)
-                    )
-    template = templates.get_template(template_name)
-    return template.render(context)
-
-
-CLOUD_ARCHIVE = \
-""" # Ubuntu Cloud Archive
-deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
-"""
-
-CLOUD_ARCHIVE_POCKETS = {
-    'precise-folsom': 'precise-updates/folsom',
-    'precise-folsom/updates': 'precise-updates/folsom',
-    'precise-folsom/proposed': 'precise-proposed/folsom',
-    'precise-grizzly': 'precise-updates/grizzly',
-    'precise-grizzly/updates': 'precise-updates/grizzly',
-    'precise-grizzly/proposed': 'precise-proposed/grizzly'
-    }
-
-
-def execute(cmd, die=False, echo=False):
-    """ Executes a command 
-
-    if die=True, script will exit(1) if command does not return 0
-    if echo=True, output of command will be printed to stdout
-
-    returns a tuple: (stdout, stderr, return code)
-    """
-    p = subprocess.Popen(cmd.split(" "),
-                         stdout=subprocess.PIPE,
-                         stdin=subprocess.PIPE,
-                         stderr=subprocess.PIPE)
-    stdout=""
-    stderr=""
-
-    def print_line(l):
-        if echo:
-            print l.strip('\n')
-            sys.stdout.flush()
-
-    for l in iter(p.stdout.readline, ''):
-        print_line(l)
-        stdout += l
-    for l in iter(p.stderr.readline, ''):
-        print_line(l)
-        stderr += l
-
-    p.communicate()
-    rc = p.returncode
-
-    if die and rc != 0:
-        error_out("ERROR: command %s return non-zero.\n" % cmd)
-    return (stdout, stderr, rc)
-
-
-def configure_source():
-    source = str(config_get('openstack-origin'))
-    if not source:
-        return
-    if source.startswith('ppa:'):
-        cmd = [
-            'add-apt-repository',
-            source
-            ]
-        subprocess.check_call(cmd)
-    if source.startswith('cloud:'):
-        install('ubuntu-cloud-keyring')
-        pocket = source.split(':')[1]
-        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
-            apt.write(CLOUD_ARCHIVE.format(CLOUD_ARCHIVE_POCKETS[pocket]))
-    if source.startswith('deb'):
-        l = len(source.split('|'))
-        if l == 2:
-            (apt_line, key) = source.split('|')
-            cmd = [
-                'apt-key',
-                'adv', '--keyserver keyserver.ubuntu.com',
-                '--recv-keys', key
-                ]
-            subprocess.check_call(cmd)
-        elif l == 1:
-            apt_line = source
-
-        with open('/etc/apt/sources.list.d/quantum.list', 'w') as apt:
-            apt.write(apt_line + "\n")
-    cmd = [
-        'apt-get',
-        'update'
-        ]
-    subprocess.check_call(cmd)
-
-# Protocols
-TCP = 'TCP'
-UDP = 'UDP'
-
-
-def expose(port, protocol='TCP'):
-    cmd = [
-        'open-port',
-        '{}/{}'.format(port, protocol)
-        ]
-    subprocess.check_call(cmd)
-
-
-def juju_log(severity, message):
-    cmd = [
-        'juju-log',
-        '--log-level', severity,
-        message
-        ]
-    subprocess.check_call(cmd)
-
-
-def relation_ids(relation):
-    cmd = [
-        'relation-ids',
-        relation
-        ]
-    return subprocess.check_output(cmd).split()  # IGNORE:E1103
-
-
-def relation_list(rid):
-    cmd = [
-        'relation-list',
-        '-r', rid,
-        ]
-    return subprocess.check_output(cmd).split()  # IGNORE:E1103
-
-
-def relation_get(attribute, unit=None, rid=None):
-    cmd = [
-        'relation-get',
-        ]
-    if rid:
-        cmd.append('-r')
-        cmd.append(rid)
-    cmd.append(attribute)
-    if unit:
-        cmd.append(unit)
-    value = subprocess.check_output(cmd).strip()  # IGNORE:E1103
-    if value == "":
-        return None
-    else:
-        return value
-
-
-def relation_get_dict(relation_id=None, remote_unit=None):
-    """Obtain all relation data as dict by way of JSON"""
-    cmd = 'relation-get --format=json'
-    if relation_id:
-        cmd += ' -r %s' % relation_id
-    if remote_unit:
-        remote_unit_orig = os.getenv('JUJU_REMOTE_UNIT', None)
-        os.environ['JUJU_REMOTE_UNIT'] = remote_unit
-    j = execute(cmd, die=True)[0]
-    if remote_unit and remote_unit_orig:
-        os.environ['JUJU_REMOTE_UNIT'] = remote_unit_orig
-    d = json.loads(j)
-    settings = {}
-    # convert unicode to strings
-    for k, v in d.iteritems():
-        settings[str(k)] = str(v)
-    return settings
-
-
-def relation_set(**kwargs):
-    cmd = [
-        'relation-set'
-        ]
-    args = []
-    for k, v in kwargs.items():
-        if k == 'rid':
-            cmd.append('-r')
-            cmd.append(v)
-        else:
-            args.append('{}={}'.format(k, v))
-    cmd += args
-    subprocess.check_call(cmd)
-
-
-def unit_get(attribute):
-    cmd = [
-        'unit-get',
-        attribute
-        ]
-    return subprocess.check_output(cmd).strip()  # IGNORE:E1103
-
-
-def config_get(attribute):
-    cmd = [
-        'config-get',
-        attribute
-        ]
-    return subprocess.check_output(cmd).strip()  # IGNORE:E1103
-
-
-def get_unit_hostname():
-    return socket.gethostname()
-
-
-def get_unit_name():
-    return os.environ.get('JUJU_UNIT_NAME').replace('/','-')
-
-
-def get_host_ip(hostname=unit_get('private-address')):
-    try:
-        # Test to see if already an IPv4 address
-        socket.inet_aton(hostname)
-        return hostname
-    except socket.error:
-        pass
-    try:
-        answers = dns.resolver.query(hostname, 'A')
-        if answers:
-            return answers[0].address
-    except dns.resolver.NXDOMAIN:
-        pass
-    return None
-
-
-def restart(*services):
-    for service in services:
-        subprocess.check_call(['service', service, 'restart'])
-
-
-def stop(*services):
-    for service in services:
-        subprocess.check_call(['service', service, 'stop'])
-
-
-def start(*services):
-    for service in services:
-        subprocess.check_call(['service', service, 'start'])
-
-
-def running(service):
-    try:
-        output = subprocess.check_output(['service', service, 'status'])
-    except subprocess.CalledProcessError:
-        return False
-    else:
-        if ("start/running" in output or
-            "is running" in output):
-            return True
-        else:
-            return False
-
-
-def disable_upstart_services(*services):
-    for service in services:
-        with open("/etc/init/{}.override".format(service), "w") as override:
-            override.write("manual")
-
-
-def enable_upstart_services(*services):
-    for service in services:
-        path = '/etc/init/{}.override'.format(service)
-        if os.path.exists(path):
-            os.remove(path)
-
-
-def disable_lsb_services(*services):
-    for service in services:
-        subprocess.check_call(['update-rc.d', '-f', service, 'remove'])
-
-
-def enable_lsb_services(*services):
-    for service in services:
-        subprocess.check_call(['update-rc.d', '-f', service, 'defaults'])
-
-
-def get_iface_ipaddr(iface):
-    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
-    return socket.inet_ntoa(fcntl.ioctl(
-        s.fileno(),
-        0x8919,  # SIOCGIFADDR
-        struct.pack('256s', iface[:15])
-    )[20:24])
-
-
-def get_iface_netmask(iface):
-    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
-    return socket.inet_ntoa(fcntl.ioctl(
-        s.fileno(),
-        0x891b,  # SIOCGIFNETMASK
-        struct.pack('256s', iface[:15])
-    )[20:24])
-
-
-def get_netmask_cidr(netmask):
-    netmask = netmask.split('.')
-    binary_str = ''
-    for octet in netmask:
-        binary_str += bin(int(octet))[2:].zfill(8)
-    return str(len(binary_str.rstrip('0')))
-
-
-def get_network_address(iface):
-    if iface:
-        network = "{}/{}".format(get_iface_ipaddr(iface),
-                                 get_netmask_cidr(get_iface_netmask(iface)))
-        ip = IPNetwork(network)
-        return str(ip.network)
-    else:
-        return None
-
-
-def is_clustered():
-    for r_id in (relation_ids('ha') or []):
-        for unit in (relation_list(r_id) or []):
-            relation_data = \
-                relation_get_dict(relation_id=r_id,
-                                  remote_unit=unit)
-            if 'clustered' in relation_data:
-                return True
-    return False
-
-
-def is_leader():
-    status = execute('crm resource show res_mysql_vip', echo=True)[0].strip()
-    hostname = execute('hostname', echo=True)[0].strip()
-    if hostname in status:
-        return True
-    else:
-        return False
-
-
-def peer_units():
-    peers = []
-    for r_id in (relation_ids('cluster') or []):
-        for unit in (relation_list(r_id) or []):
-            peers.append(unit)
-    return peers
-
-
-def oldest_peer(peers):
-    local_unit_no = os.getenv('JUJU_UNIT_NAME').split('/')[1]
-    for peer in peers:
-        remote_unit_no = peer.split('/')[1]
-        if remote_unit_no < local_unit_no:
-            return False
-    return True
-
-
-def eligible_leader():
-    if is_clustered():
-        if not is_leader():
-            juju_log('INFO', 'Deferring action to CRM leader.')
-            return False
-    else:
-        peers = peer_units()
-        if peers and not oldest_peer(peers):
-            juju_log('INFO', 'Deferring action to oldest service unit.')
-            return False
-    return True
-
-
-def is_relation_made(relation=None):
-    relation_data = []
-    for r_id in (relation_ids(relation) or []):
-        for unit in (relation_list(r_id) or []):
-            relation_data.append(relation_get_dict(relation_id=r_id,
-                                 remote_unit=unit))
-    if not relation_data:
-        return False
-    return True

=== removed directory 'templates'
=== removed file 'templates/ceph.conf'
--- templates/ceph.conf	2013-01-28 17:59:02 +0000
+++ templates/ceph.conf	1970-01-01 00:00:00 +0000
@@ -1,5 +0,0 @@
-[global]
- auth supported = {{ auth }}
- keyring = {{ keyring }}
- mon host = {{ mon_hosts }}
-