Merge lp:~openstack-charmers/charms/precise/rabbitmq-server/ha-support into lp:charms/rabbitmq-server

[James Page]

This branch includes:

1) Redux from bash->python and use of helpers from the openstack charm helpers project

2) Support for Active/Passive HA using a shared ceph block device and the hacluster suboridinate charm.

[Andreas Hasenack (ahasenack)]

In the shell version, amqp_changed() sets a "hostname" in the end:
  juju-log "rabbitmq-server: Returning credentials for $RABBIT_USER@$HOSTNAME"
  relation-set hostname=`unit-get private-address`
  relation-set password=$PASSW

The python version doesn't, it only sets the password:
    relation_settings = {
        'password': password
    }

And the vip if it's clustered. If should set the hostname too.

That being said, "hostname" is a somewhat bad name for this. Just looking at it you never know which hostname it's talking about. Maybe something like "rabbit_hostname" would be better? Just a thought, as this would break compatibility with consumers of this charm (unless both are set).

[James Page (james-page)]

Good spot Andreas

We will get that fixed up so we don't regress the hook interface as it stands today.

[lp:~openstack-charmers/charms/precise/rabbitmq-server/ha-support]

47. By James Page on 2013-06-06

Fixup hooks to restore previous behaviour for non-clustered services

=== added file '.project'
--- .project	1970-01-01 00:00:00 +0000
+++ .project	2013-06-06 15:35:31 +0000
@@ -0,0 +1,17 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<projectDescription>
+	<name>rabbitmq-server</name>
+	<comment></comment>
+	<projects>
+	</projects>
+	<buildSpec>
+		<buildCommand>
+			<name>org.python.pydev.PyDevBuilder</name>
+			<arguments>
+			</arguments>
+		</buildCommand>
+	</buildSpec>
+	<natures>
+		<nature>org.python.pydev.pythonNature</nature>
+	</natures>
+</projectDescription>

=== added file '.pydevproject'
--- .pydevproject	1970-01-01 00:00:00 +0000
+++ .pydevproject	2013-06-06 15:35:31 +0000
@@ -0,0 +1,8 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?eclipse-pydev version="1.0"?><pydev_project>
+<pydev_property name="org.python.pydev.PYTHON_PROJECT_VERSION">python 2.7</pydev_property>
+<pydev_property name="org.python.pydev.PYTHON_PROJECT_INTERPRETER">Default</pydev_property>
+<pydev_pathproperty name="org.python.pydev.PROJECT_SOURCE_PATH">
+<path>/rabbitmq-server/hooks</path>
+</pydev_pathproperty>
+</pydev_project>

=== modified file 'config.yaml'
--- config.yaml	2013-03-01 17:36:31 +0000
+++ config.yaml	2013-06-06 15:35:31 +0000
@@ -27,3 +27,40 @@
           juju-myservice-0
       If you're running multiple environments with the same services in them
       this allows you to differentiate between them.
+  # HA configuration settings
+  vip:
+      type: string
+      description: "Virtual IP to use to front rabbitmq in ha configuration"
+  vip_iface:
+      type: string
+      default: eth0
+      description: "Network Interface where to place the Virtual IP"
+  vip_cidr:
+      type: int
+      default: 24
+      description: "Netmask that will be used for the Virtual IP"
+  ha-bindiface:
+      type: string
+      default: eth0
+      description: |
+        Default network interface on which HA cluster will bind to communication
+        with the other members of the HA Cluster.
+  ha-mcastport:
+      type: int
+      default: 5406
+      description: |
+        Default multicast port number that will be used to communicate between
+        HA Cluster nodes.
+  rbd-size:
+      type: string
+      default: 5G
+      description: |
+        Default rbd storage size to create when setting up block storage.
+        This value should be specified in GB (e.g. 100G).
+  rbd-name:
+      type: string
+      default: rabbitmq1
+      description: |
+        The name that will be used to create the Ceph's RBD image with. If the
+        image name exists in Ceph, it will be re-used and the data will be
+        overwritten.

=== modified symlink 'hooks/amqp-relation-changed'
=== target changed u'rabbitmq-relations' => u'rabbitmq_server_relations.py'
=== removed symlink 'hooks/amqp-relation-joined'
=== target was u'rabbitmq-relations'
=== added symlink 'hooks/ceph-relation-changed'
=== target is u'rabbitmq_server_relations.py'
=== added symlink 'hooks/ceph-relation-joined'
=== target is u'rabbitmq_server_relations.py'
=== modified symlink 'hooks/cluster-relation-changed'
=== target changed u'rabbitmq-relations' => u'rabbitmq_server_relations.py'
=== modified symlink 'hooks/cluster-relation-joined'
=== target changed u'rabbitmq-relations' => u'rabbitmq_server_relations.py'
=== added symlink 'hooks/config-changed'
=== target is u'rabbitmq_server_relations.py'
=== removed file 'hooks/config-changed'
--- hooks/config-changed	2012-11-21 15:30:39 +0000
+++ hooks/config-changed	1970-01-01 00:00:00 +0000
@@ -1,70 +0,0 @@
-#!/bin/bash
-set -eu
-
-juju-log "rabbitmq-server: Firing config hook"
-
-export HOME=/root		# (HOME is not set on first run)
-RABBIT_PLUGINS=/usr/lib/rabbitmq/lib/rabbitmq_server-*/sbin/rabbitmq-plugins
-if [ "`config-get management_plugin`" == "True" ]; then
-	$RABBIT_PLUGINS enable rabbitmq_management
-	open-port 55672/tcp
-else
-	$RABBIT_PLUGINS disable rabbitmq_management
-	close-port 55672/tcp
-fi
-
-ssl_enabled=`config-get ssl_enabled`
-
-cd /etc/rabbitmq
-
-new_config=`mktemp /etc/rabbitmq/.rabbitmq.config.XXXXXX`
-chgrp rabbitmq "$new_config"
-chmod g+r "$new_config"
-exec 3> "$new_config"
-
-cat >&3 <<EOF
-[
-    {rabbit, [
-EOF
-
-ssl_key_file=/etc/rabbitmq/rabbit-server-privkey.pem
-ssl_cert_file=/etc/rabbitmq/rabbit-server-cert.pem
-
-if [ "$ssl_enabled" == "True" ]; then
-	umask 027
-	config-get ssl_key > "$ssl_key_file"
-	config-get ssl_cert > "$ssl_cert_file"
-	chgrp rabbitmq "$ssl_key_file" "$ssl_cert_file"
-	if [ ! -s "$ssl_key_file" ]; then
-		juju-log "ssl_key not set - can't configure SSL"
-		exit 0
-	fi
-	if [ ! -s "$ssl_cert_file" ]; then
-		juju-log "ssl_cert not set - can't configure SSL"
-		exit 0
-	fi
-	cat >&3 <<EOF
-	{ssl_listeners, [`config-get ssl_port`]},
-	{ssl_options, [
-		{certfile,"$ssl_cert_file"},
-		{keyfile,"$ssl_key_file"}
-	]},
-	open-port `config-get ssl_port`/tcp
-EOF
-fi
-
-cat >&3 <<EOF
-        {tcp_listeners, [5672]}
-    ]}
-].
-EOF
-
-exec 3>&-
-
-if [ -f rabbitmq.config ]; then
-	mv rabbitmq.config{,.bak}
-fi
-
-mv "$new_config" rabbitmq.config
-
-/etc/init.d/rabbitmq-server restart

=== added symlink 'hooks/ha-relation-changed'
=== target is u'rabbitmq_server_relations.py'
=== added symlink 'hooks/ha-relation-joined'
=== target is u'rabbitmq_server_relations.py'
=== modified symlink 'hooks/install'
=== target changed u'rabbitmq-relations' => u'rabbitmq_server_relations.py'
=== added directory 'hooks/lib'
=== added file 'hooks/lib/__init__.py'
=== added file 'hooks/lib/ceph_utils.py'
--- hooks/lib/ceph_utils.py	1970-01-01 00:00:00 +0000
+++ hooks/lib/ceph_utils.py	2013-06-06 15:35:31 +0000
@@ -0,0 +1,256 @@
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This file is sourced from lp:openstack-charm-helpers
+#
+# Authors:
+#  James Page <[e-mail]>
+#  Adam Gandelman <[e-mail]>
+#
+
+import commands
+import subprocess
+import os
+import shutil
+import lib.utils as utils
+
+KEYRING = '/etc/ceph/ceph.client.%s.keyring'
+KEYFILE = '/etc/ceph/ceph.client.%s.key'
+
+CEPH_CONF = """[global]
+ auth supported = %(auth)s
+ keyring = %(keyring)s
+ mon host = %(mon_hosts)s
+"""
+
+
+def execute(cmd):
+    subprocess.check_call(cmd)
+
+
+def execute_shell(cmd):
+    subprocess.check_call(cmd, shell=True)
+
+
+def install():
+    ceph_dir = "/etc/ceph"
+    if not os.path.isdir(ceph_dir):
+        os.mkdir(ceph_dir)
+    utils.install('ceph-common')
+
+
+def rbd_exists(service, pool, rbd_img):
+    (rc, out) = commands.getstatusoutput('rbd list --id %s --pool %s' %\
+                                         (service, pool))
+    return rbd_img in out
+
+
+def create_rbd_image(service, pool, image, sizemb):
+    cmd = [
+        'rbd',
+        'create',
+        image,
+        '--size',
+        str(sizemb),
+        '--id',
+        service,
+        '--pool',
+        pool
+        ]
+    execute(cmd)
+
+
+def pool_exists(service, name):
+    (rc, out) = commands.getstatusoutput("rados --id %s lspools" % service)
+    return name in out
+
+
+def create_pool(service, name):
+    cmd = [
+        'rados',
+        '--id',
+        service,
+        'mkpool',
+        name
+        ]
+    execute(cmd)
+
+
+def keyfile_path(service):
+    return KEYFILE % service
+
+
+def keyring_path(service):
+    return KEYRING % service
+
+
+def create_keyring(service, key):
+    keyring = keyring_path(service)
+    if os.path.exists(keyring):
+        utils.juju_log('INFO', 'ceph: Keyring exists at %s.' % keyring)
+    cmd = [
+        'ceph-authtool',
+        keyring,
+        '--create-keyring',
+        '--name=client.%s' % service,
+        '--add-key=%s' % key
+        ]
+    execute(cmd)
+    utils.juju_log('INFO', 'ceph: Created new ring at %s.' % keyring)
+
+
+def create_key_file(service, key):
+    # create a file containing the key
+    keyfile = keyfile_path(service)
+    if os.path.exists(keyfile):
+        utils.juju_log('INFO', 'ceph: Keyfile exists at %s.' % keyfile)
+    fd = open(keyfile, 'w')
+    fd.write(key)
+    fd.close()
+    utils.juju_log('INFO', 'ceph: Created new keyfile at %s.' % keyfile)
+
+
+def get_ceph_nodes():
+    hosts = []
+    for r_id in utils.relation_ids('ceph'):
+        for unit in utils.relation_list(r_id):
+            hosts.append(utils.relation_get('private-address',
+                                            unit=unit, rid=r_id))
+    return hosts
+
+
+def configure(service, key, auth):
+    create_keyring(service, key)
+    create_key_file(service, key)
+    hosts = get_ceph_nodes()
+    mon_hosts = ",".join(map(str, hosts))
+    keyring = keyring_path(service)
+    with open('/etc/ceph/ceph.conf', 'w') as ceph_conf:
+        ceph_conf.write(CEPH_CONF % locals())
+    modprobe_kernel_module('rbd')
+
+
+def image_mapped(image_name):
+    (rc, out) = commands.getstatusoutput('rbd showmapped')
+    return image_name in out
+
+
+def map_block_storage(service, pool, image):
+    cmd = [
+        'rbd',
+        'map',
+        '%s/%s' % (pool, image),
+        '--user',
+        service,
+        '--secret',
+        keyfile_path(service),
+        ]
+    execute(cmd)
+
+
+def filesystem_mounted(fs):
+    return subprocess.call(['grep', '-wqs', fs, '/proc/mounts']) == 0
+
+
+def make_filesystem(blk_device, fstype='ext4'):
+    utils.juju_log('INFO',
+                   'ceph: Formatting block device %s as filesystem %s.' %\
+                   (blk_device, fstype))
+    cmd = ['mkfs', '-t', fstype, blk_device]
+    execute(cmd)
+
+
+def place_data_on_ceph(service, blk_device, data_src_dst, fstype='ext4'):
+    # mount block device into /mnt
+    cmd = ['mount', '-t', fstype, blk_device, '/mnt']
+    execute(cmd)
+
+    # copy data to /mnt
+    try:
+        copy_files(data_src_dst, '/mnt')
+    except:
+        pass
+
+    # umount block device
+    cmd = ['umount', '/mnt']
+    execute(cmd)
+
+    _dir = os.stat(data_src_dst)
+    uid = _dir.st_uid
+    gid = _dir.st_gid
+
+    # re-mount where the data should originally be
+    cmd = ['mount', '-t', fstype, blk_device, data_src_dst]
+    execute(cmd)
+
+    # ensure original ownership of new mount.
+    cmd = ['chown', '-R', '%s:%s' % (uid, gid), data_src_dst]
+    execute(cmd)
+
+
+# TODO: re-use
+def modprobe_kernel_module(module):
+    utils.juju_log('INFO', 'Loading kernel module')
+    cmd = ['modprobe', module]
+    execute(cmd)
+    cmd = 'echo %s >> /etc/modules' % module
+    execute_shell(cmd)
+
+
+def copy_files(src, dst, symlinks=False, ignore=None):
+    for item in os.listdir(src):
+        s = os.path.join(src, item)
+        d = os.path.join(dst, item)
+        if os.path.isdir(s):
+            shutil.copytree(s, d, symlinks, ignore)
+        else:
+            shutil.copy2(s, d)
+
+
+def ensure_ceph_storage(service, pool, rbd_img, sizemb, mount_point,
+                        blk_device, fstype, system_services=[]):
+    """
+    To be called from the current cluster leader.
+    Ensures given pool and RBD image exists, is mapped to a block device,
+    and the device is formatted and mounted at the given mount_point.
+
+    If formatting a device for the first time, data existing at mount_point
+    will be migrated to the RBD device before being remounted.
+
+    All services listed in system_services will be stopped prior to data
+    migration and restarted when complete.
+    """
+    # Ensure pool, RBD image, RBD mappings are in place.
+    if not pool_exists(service, pool):
+        utils.juju_log('INFO', 'ceph: Creating new pool %s.' % pool)
+        create_pool(service, pool)
+
+    if not rbd_exists(service, pool, rbd_img):
+        utils.juju_log('INFO', 'ceph: Creating RBD image (%s).' % rbd_img)
+        create_rbd_image(service, pool, rbd_img, sizemb)
+
+    if not image_mapped(rbd_img):
+        utils.juju_log('INFO', 'ceph: Mapping RBD Image as a Block Device.')
+        map_block_storage(service, pool, rbd_img)
+
+    # make file system
+    # TODO: What happens if for whatever reason this is run again and
+    # the data is already in the rbd device and/or is mounted??
+    # When it is mounted already, it will fail to make the fs
+    # XXX: This is really sketchy!  Need to at least add an fstab entry
+    #      otherwise this hook will blow away existing data if its executed
+    #      after a reboot.
+    if not filesystem_mounted(mount_point):
+        make_filesystem(blk_device, fstype)
+
+        for svc in system_services:
+            if utils.running(svc):
+                utils.juju_log('INFO',
+                               'Stopping services %s prior to migrating '\
+                               'data' % svc)
+                utils.stop(svc)
+
+        place_data_on_ceph(service, blk_device, mount_point, fstype)
+
+        for svc in system_services:
+            utils.start(svc)

=== added file 'hooks/lib/cluster_utils.py'
--- hooks/lib/cluster_utils.py	1970-01-01 00:00:00 +0000
+++ hooks/lib/cluster_utils.py	2013-06-06 15:35:31 +0000
@@ -0,0 +1,130 @@
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This file is sourced from lp:openstack-charm-helpers
+#
+# Authors:
+#  James Page <[e-mail]>
+#  Adam Gandelman <[e-mail]>
+#
+
+from lib.utils import (
+    juju_log,
+    relation_ids,
+    relation_list,
+    relation_get,
+    get_unit_hostname,
+    config_get
+    )
+import subprocess
+import os
+
+
+def is_clustered():
+    for r_id in (relation_ids('ha') or []):
+        for unit in (relation_list(r_id) or []):
+            clustered = relation_get('clustered',
+                                     rid=r_id,
+                                     unit=unit)
+            if clustered:
+                return True
+    return False
+
+
+def is_leader(resource):
+    cmd = [
+        "crm", "resource",
+        "show", resource
+        ]
+    try:
+        status = subprocess.check_output(cmd)
+    except subprocess.CalledProcessError:
+        return False
+    else:
+        if get_unit_hostname() in status:
+            return True
+        else:
+            return False
+
+
+def peer_units():
+    peers = []
+    for r_id in (relation_ids('cluster') or []):
+        for unit in (relation_list(r_id) or []):
+            peers.append(unit)
+    return peers
+
+
+def oldest_peer(peers):
+    local_unit_no = os.getenv('JUJU_UNIT_NAME').split('/')[1]
+    for peer in peers:
+        remote_unit_no = peer.split('/')[1]
+        if remote_unit_no < local_unit_no:
+            return False
+    return True
+
+
+def eligible_leader(resource):
+    if is_clustered():
+        if not is_leader(resource):
+            juju_log('INFO', 'Deferring action to CRM leader.')
+            return False
+    else:
+        peers = peer_units()
+        if peers and not oldest_peer(peers):
+            juju_log('INFO', 'Deferring action to oldest service unit.')
+            return False
+    return True
+
+
+def https():
+    '''
+    Determines whether enough data has been provided in configuration
+    or relation data to configure HTTPS
+    .
+    returns: boolean
+    '''
+    if config_get('use-https') == "yes":
+        return True
+    if config_get('ssl_cert') and config_get('ssl_key'):
+        return True
+    for r_id in relation_ids('identity-service'):
+        for unit in relation_list(r_id):
+            if (relation_get('https_keystone', rid=r_id, unit=unit) and
+                relation_get('ssl_cert', rid=r_id, unit=unit) and
+                relation_get('ssl_key', rid=r_id, unit=unit) and
+                relation_get('ca_cert', rid=r_id, unit=unit)):
+                return True
+    return False
+
+
+def determine_api_port(public_port):
+    '''
+    Determine correct API server listening port based on
+    existence of HTTPS reverse proxy and/or haproxy.
+
+    public_port: int: standard public port for given service
+
+    returns: int: the correct listening port for the API service
+    '''
+    i = 0
+    if len(peer_units()) > 0 or is_clustered():
+        i += 1
+    if https():
+        i += 1
+    return public_port - (i * 10)
+
+
+def determine_haproxy_port(public_port):
+    '''
+    Description: Determine correct proxy listening port based on public IP +
+    existence of HTTPS reverse proxy.
+
+    public_port: int: standard public port for given service
+
+    returns: int: the correct listening port for the HAProxy service
+    '''
+    i = 0
+    if https():
+        i += 1
+    return public_port - (i * 10)

=== added file 'hooks/lib/openstack_common.py'
--- hooks/lib/openstack_common.py	1970-01-01 00:00:00 +0000
+++ hooks/lib/openstack_common.py	2013-06-06 15:35:31 +0000
@@ -0,0 +1,230 @@
+#!/usr/bin/python
+
+# Common python helper functions used for OpenStack charms.
+
+import apt_pkg as apt
+import subprocess
+import os
+
+CLOUD_ARCHIVE_URL = "http://ubuntu-cloud.archive.canonical.com/ubuntu"
+CLOUD_ARCHIVE_KEY_ID = '5EDB1B62EC4926EA'
+
+ubuntu_openstack_release = {
+    'oneiric': 'diablo',
+    'precise': 'essex',
+    'quantal': 'folsom',
+    'raring': 'grizzly',
+}
+
+
+openstack_codenames = {
+    '2011.2': 'diablo',
+    '2012.1': 'essex',
+    '2012.2': 'folsom',
+    '2013.1': 'grizzly',
+    '2013.2': 'havana',
+}
+
+# The ugly duckling
+swift_codenames = {
+    '1.4.3': 'diablo',
+    '1.4.8': 'essex',
+    '1.7.4': 'folsom',
+    '1.7.6': 'grizzly',
+    '1.7.7': 'grizzly',
+    '1.8.0': 'grizzly',
+}
+
+
+def juju_log(msg):
+    subprocess.check_call(['juju-log', msg])
+
+
+def error_out(msg):
+    juju_log("FATAL ERROR: %s" % msg)
+    exit(1)
+
+
+def lsb_release():
+    '''Return /etc/lsb-release in a dict'''
+    lsb = open('/etc/lsb-release', 'r')
+    d = {}
+    for l in lsb:
+        k, v = l.split('=')
+        d[k.strip()] = v.strip()
+    return d
+
+
+def get_os_codename_install_source(src):
+    '''Derive OpenStack release codename from a given installation source.'''
+    ubuntu_rel = lsb_release()['DISTRIB_CODENAME']
+
+    rel = ''
+    if src == 'distro':
+        try:
+            rel = ubuntu_openstack_release[ubuntu_rel]
+        except KeyError:
+            e = 'Code not derive openstack release for '\
+                'this Ubuntu release: %s' % rel
+            error_out(e)
+        return rel
+
+    if src.startswith('cloud:'):
+        ca_rel = src.split(':')[1]
+        ca_rel = ca_rel.split('%s-' % ubuntu_rel)[1].split('/')[0]
+        return ca_rel
+
+    # Best guess match based on deb string provided
+    if src.startswith('deb') or src.startswith('ppa'):
+        for k, v in openstack_codenames.iteritems():
+            if v in src:
+                return v
+
+
+def get_os_codename_version(vers):
+    '''Determine OpenStack codename from version number.'''
+    try:
+        return openstack_codenames[vers]
+    except KeyError:
+        e = 'Could not determine OpenStack codename for version %s' % vers
+        error_out(e)
+
+
+def get_os_version_codename(codename):
+    '''Determine OpenStack version number from codename.'''
+    for k, v in openstack_codenames.iteritems():
+        if v == codename:
+            return k
+    e = 'Code not derive OpenStack version for '\
+        'codename: %s' % codename
+    error_out(e)
+
+
+def get_os_codename_package(pkg):
+    '''Derive OpenStack release codename from an installed package.'''
+    apt.init()
+    cache = apt.Cache()
+    try:
+        pkg = cache[pkg]
+    except:
+        e = 'Could not determine version of installed package: %s' % pkg
+        error_out(e)
+
+    vers = apt.UpstreamVersion(pkg.current_ver.ver_str)
+
+    try:
+        if 'swift' in pkg.name:
+            vers = vers[:5]
+            return swift_codenames[vers]
+        else:
+            vers = vers[:6]
+            return openstack_codenames[vers]
+    except KeyError:
+        e = 'Could not determine OpenStack codename for version %s' % vers
+        error_out(e)
+
+
+def get_os_version_package(pkg):
+    '''Derive OpenStack version number from an installed package.'''
+    codename = get_os_codename_package(pkg)
+
+    if 'swift' in pkg:
+        vers_map = swift_codenames
+    else:
+        vers_map = openstack_codenames
+
+    for version, cname in vers_map.iteritems():
+        if cname == codename:
+            return version
+    e = "Could not determine OpenStack version for package: %s" % pkg
+    error_out(e)
+
+
+def configure_installation_source(rel):
+    '''Configure apt installation source.'''
+
+    def _import_key(keyid):
+        cmd = "apt-key adv --keyserver keyserver.ubuntu.com " \
+              "--recv-keys %s" % keyid
+        try:
+            subprocess.check_call(cmd.split(' '))
+        except subprocess.CalledProcessError:
+            error_out("Error importing repo key %s" % keyid)
+
+    if rel == 'distro':
+        return
+    elif rel[:4] == "ppa:":
+        src = rel
+        subprocess.check_call(["add-apt-repository", "-y", src])
+    elif rel[:3] == "deb":
+        l = len(rel.split('|'))
+        if l == 2:
+            src, key = rel.split('|')
+            juju_log("Importing PPA key from keyserver for %s" % src)
+            _import_key(key)
+        elif l == 1:
+            src = rel
+        else:
+            error_out("Invalid openstack-release: %s" % rel)
+
+        with open('/etc/apt/sources.list.d/juju_deb.list', 'w') as f:
+            f.write(src)
+    elif rel[:6] == 'cloud:':
+        ubuntu_rel = lsb_release()['DISTRIB_CODENAME']
+        rel = rel.split(':')[1]
+        u_rel = rel.split('-')[0]
+        ca_rel = rel.split('-')[1]
+
+        if u_rel != ubuntu_rel:
+            e = 'Cannot install from Cloud Archive pocket %s on this Ubuntu '\
+                'version (%s)' % (ca_rel, ubuntu_rel)
+            error_out(e)
+
+        if 'staging' in ca_rel:
+            # staging is just a regular PPA.
+            os_rel = ca_rel.split('/')[0]
+            ppa = 'ppa:ubuntu-cloud-archive/%s-staging' % os_rel
+            cmd = 'add-apt-repository -y %s' % ppa
+            subprocess.check_call(cmd.split(' '))
+            return
+
+        # map charm config options to actual archive pockets.
+        pockets = {
+            'folsom': 'precise-updates/folsom',
+            'folsom/updates': 'precise-updates/folsom',
+            'folsom/proposed': 'precise-proposed/folsom',
+            'grizzly': 'precise-updates/grizzly',
+            'grizzly/updates': 'precise-updates/grizzly',
+            'grizzly/proposed': 'precise-proposed/grizzly'
+        }
+
+        try:
+            pocket = pockets[ca_rel]
+        except KeyError:
+            e = 'Invalid Cloud Archive release specified: %s' % rel
+            error_out(e)
+
+        src = "deb %s %s main" % (CLOUD_ARCHIVE_URL, pocket)
+        _import_key(CLOUD_ARCHIVE_KEY_ID)
+
+        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as f:
+            f.write(src)
+    else:
+        error_out("Invalid openstack-release specified: %s" % rel)
+
+
+def save_script_rc(script_path="scripts/scriptrc", **env_vars):
+    """
+    Write an rc file in the charm-delivered directory containing
+    exported environment variables provided by env_vars. Any charm scripts run
+    outside the juju hook environment can source this scriptrc to obtain
+    updated config information necessary to perform health checks or
+    service changes.
+    """
+    unit_name = os.getenv('JUJU_UNIT_NAME').replace('/', '-')
+    juju_rc_path = "/var/lib/juju/units/%s/charm/%s" % (unit_name, script_path)
+    with open(juju_rc_path, 'wb') as rc_script:
+        rc_script.write(
+            "#!/bin/bash\n")
+        [rc_script.write('export %s=%s\n' % (u, p))
+         for u, p in env_vars.iteritems() if u != "script_path"]

=== added file 'hooks/lib/utils.py'
--- hooks/lib/utils.py	1970-01-01 00:00:00 +0000
+++ hooks/lib/utils.py	2013-06-06 15:35:31 +0000
@@ -0,0 +1,295 @@
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This file is sourced from lp:openstack-charm-helpers
+#
+# Authors:
+#  James Page <[e-mail]>
+#  Paul Collins <[e-mail]>
+#  Adam Gandelman <[e-mail]>
+#
+
+import json
+import os
+import subprocess
+import socket
+import sys
+
+
+def do_hooks(hooks):
+    hook = os.path.basename(sys.argv[0])
+
+    try:
+        hook_func = hooks[hook]
+    except KeyError:
+        juju_log('INFO',
+                 "This charm doesn't know how to handle '{}'.".format(hook))
+    else:
+        hook_func()
+
+
+def install(*pkgs):
+    cmd = [
+        'apt-get',
+        '-y',
+        'install'
+          ]
+    for pkg in pkgs:
+        cmd.append(pkg)
+    subprocess.check_call(cmd)
+
+TEMPLATES_DIR = 'templates'
+
+try:
+    import jinja2
+except ImportError:
+    install('python-jinja2')
+    import jinja2
+
+try:
+    import dns.resolver
+except ImportError:
+    install('python-dnspython')
+    import dns.resolver
+
+
+def render_template(template_name, context, template_dir=TEMPLATES_DIR):
+    templates = jinja2.Environment(
+                    loader=jinja2.FileSystemLoader(template_dir)
+                    )
+    template = templates.get_template(template_name)
+    return template.render(context)
+
+CLOUD_ARCHIVE = \
+""" # Ubuntu Cloud Archive
+deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
+"""
+
+CLOUD_ARCHIVE_POCKETS = {
+    'folsom': 'precise-updates/folsom',
+    'folsom/updates': 'precise-updates/folsom',
+    'folsom/proposed': 'precise-proposed/folsom',
+    'grizzly': 'precise-updates/grizzly',
+    'grizzly/updates': 'precise-updates/grizzly',
+    'grizzly/proposed': 'precise-proposed/grizzly'
+    }
+
+
+def configure_source():
+    source = str(config_get('openstack-origin'))
+    if not source:
+        return
+    if source.startswith('ppa:'):
+        cmd = [
+            'add-apt-repository',
+            source
+            ]
+        subprocess.check_call(cmd)
+    if source.startswith('cloud:'):
+        install('ubuntu-cloud-keyring')
+        pocket = source.split(':')[1]
+        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
+            apt.write(CLOUD_ARCHIVE.format(CLOUD_ARCHIVE_POCKETS[pocket]))
+    if source.startswith('deb'):
+        l = len(source.split('|'))
+        if l == 2:
+            (apt_line, key) = source.split('|')
+            cmd = [
+                'apt-key',
+                'adv', '--keyserver keyserver.ubuntu.com',
+                '--recv-keys', key
+                ]
+            subprocess.check_call(cmd)
+        elif l == 1:
+            apt_line = source
+
+        with open('/etc/apt/sources.list.d/quantum.list', 'w') as apt:
+            apt.write(apt_line + "\n")
+    cmd = [
+        'apt-get',
+        'update'
+        ]
+    subprocess.check_call(cmd)
+
+# Protocols
+TCP = 'TCP'
+UDP = 'UDP'
+
+
+def expose(port, protocol='TCP'):
+    cmd = [
+        'open-port',
+        '{}/{}'.format(port, protocol)
+        ]
+    subprocess.check_call(cmd)
+
+
+def open_port(port, protocol='TCP'):
+    expose(port, protocol)
+
+
+def close_port(port, protocol='TCP'):
+    cmd = [
+        'close-port',
+        '{}/{}'.format(port, protocol)
+        ]
+    subprocess.check_call(cmd)
+
+
+def juju_log(severity, message):
+    cmd = [
+        'juju-log',
+        '--log-level', severity,
+        message
+        ]
+    subprocess.check_call(cmd)
+
+
+def relation_ids(relation):
+    cmd = [
+        'relation-ids',
+        relation
+        ]
+    result = str(subprocess.check_output(cmd)).split()
+    if result == "":
+        return None
+    else:
+        return result
+
+
+def relation_list(rid):
+    cmd = [
+        'relation-list',
+        '-r', rid,
+        ]
+    result = str(subprocess.check_output(cmd)).split()
+    if result == "":
+        return None
+    else:
+        return result
+
+
+def relation_get(attribute, unit=None, rid=None):
+    cmd = [
+        'relation-get',
+        ]
+    if rid:
+        cmd.append('-r')
+        cmd.append(rid)
+    cmd.append(attribute)
+    if unit:
+        cmd.append(unit)
+    value = subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    if value == "":
+        return None
+    else:
+        return value
+
+
+def relation_set(**kwargs):
+    cmd = [
+        'relation-set'
+        ]
+    args = []
+    for k, v in kwargs.items():
+        if k == 'rid':
+            if v:
+                cmd.append('-r')
+                cmd.append(v)
+        else:
+            args.append('{}={}'.format(k, v))
+    cmd += args
+    subprocess.check_call(cmd)
+
+
+def unit_get(attribute):
+    cmd = [
+        'unit-get',
+        attribute
+        ]
+    value = subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    if value == "":
+        return None
+    else:
+        return value
+
+
+def config_get(attribute):
+    cmd = [
+        'config-get',
+        '--format',
+        'json',
+        ]
+    out = subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    cfg = json.loads(out)
+
+    try:
+        return cfg[attribute]
+    except KeyError:
+        return None
+
+
+def get_unit_hostname():
+    return socket.gethostname()
+
+
+def get_host_ip(hostname=unit_get('private-address')):
+    try:
+        # Test to see if already an IPv4 address
+        socket.inet_aton(hostname)
+        return hostname
+    except socket.error:
+        answers = dns.resolver.query(hostname, 'A')
+        if answers:
+            return answers[0].address
+    return None
+
+
+def _svc_control(service, action):
+    subprocess.check_call(['service', service, action])
+
+
+def restart(*services):
+    for service in services:
+        _svc_control(service, 'restart')
+
+
+def stop(*services):
+    for service in services:
+        _svc_control(service, 'stop')
+
+
+def start(*services):
+    for service in services:
+        _svc_control(service, 'start')
+
+
+def reload(*services):
+    for service in services:
+        try:
+            _svc_control(service, 'reload')
+        except subprocess.CalledProcessError:
+            # Reload failed - either service does not support reload
+            # or it was not running - restart will fixup most things
+            _svc_control(service, 'restart')
+
+
+def running(service):
+    try:
+        output = subprocess.check_output(['service', service, 'status'])
+    except subprocess.CalledProcessError:
+        return False
+    else:
+        if ("start/running" in output or
+            "is running" in output):
+            return True
+        else:
+            return False
+
+
+def is_relation_made(relation, key='private-address'):
+    for r_id in (relation_ids(relation) or []):
+        for unit in (relation_list(r_id) or []):
+            if relation_get(key, rid=r_id, unit=unit):
+                return True
+    return False

=== added file 'hooks/rabbit_utils.py'
--- hooks/rabbit_utils.py	1970-01-01 00:00:00 +0000
+++ hooks/rabbit_utils.py	2013-06-06 15:35:31 +0000
@@ -0,0 +1,176 @@
+import os
+import pwd
+import grp
+import re
+import subprocess
+import glob
+import lib.utils as utils
+import apt_pkg as apt
+
+PACKAGES = ['pwgen', 'rabbitmq-server']
+
+RABBITMQ_CTL = '/usr/sbin/rabbitmqctl'
+COOKIE_PATH = '/var/lib/rabbitmq/.erlang.cookie'
+ENV_CONF = '/etc/rabbitmq/rabbitmq-env.conf'
+RABBITMQ_CONF = '/etc/rabbitmq/rabbitmq.config'
+
+def vhost_exists(vhost):
+    cmd = [RABBITMQ_CTL, 'list_vhosts']
+    out = subprocess.check_output(cmd)
+    for line in out.split('\n')[1:]:
+        if line == vhost:
+            utils.juju_log('INFO', 'vhost (%s) already exists.' % vhost)
+            return True
+    return False
+
+
+def create_vhost(vhost):
+    if vhost_exists(vhost):
+        return
+    cmd = [RABBITMQ_CTL, 'add_vhost', vhost]
+    subprocess.check_call(cmd)
+    utils.juju_log('INFO', 'Created new vhost (%s).' % vhost)
+
+
+def user_exists(user):
+    cmd = [RABBITMQ_CTL, 'list_users']
+    out = subprocess.check_output(cmd)
+    for line in out.split('\n')[1:]:
+        _user = line.split('\t')[0]
+        if _user == user:
+            admin = line.split('\t')[1]
+            return True, (admin == '[administrator]')
+    return False, False
+
+
+def create_user(user, password, admin=False):
+    exists, is_admin = user_exists(user)
+
+    if not exists:
+        cmd = [RABBITMQ_CTL, 'add_user', user, password]
+        subprocess.check_call(cmd)
+        utils.juju_log('INFO', 'Created new user (%s).' % user)
+
+    if admin == is_admin:
+        return
+
+    if admin:
+        cmd = [RABBITMQ_CTL, 'set_user_tags', user, 'administrator']
+        utils.juju_log('INFO', 'Granting user (%s) admin access.')
+    else:
+        cmd = [RABBITMQ_CTL, 'set_user_tags', user]
+        utils.juju_log('INFO', 'Revoking user (%s) admin access.')
+
+
+def grant_permissions(user, vhost):
+    cmd = [RABBITMQ_CTL, 'set_permissions', '-p',
+           vhost, user, '.*', '.*', '.*']
+    subprocess.check_call(cmd)
+
+
+def service(action):
+    cmd = ['service', 'rabbitmq-server', action]
+    subprocess.check_call(cmd)
+
+
+def rabbit_version():
+    apt.init()
+    cache = apt.Cache()
+    pkg = cache['rabbitmq-server']
+    if pkg.current_ver:
+        return apt.upstream_version(pkg.current_ver.ver_str)
+    else:
+        return None
+
+
+def cluster_with(host):
+    utils.juju_log('INFO', 'Clustering with remote rabbit host (%s).' % host)
+    vers = rabbit_version()
+    if vers >= '3.0.1-1':
+        cluster_cmd = 'join_cluster'
+    else:
+        cluster_cmd = 'cluster'
+    out = subprocess.check_output([RABBITMQ_CTL, 'cluster_status'])
+    for line in out.split('\n'):
+        if re.search(host, line):
+            utils.juju_log('INFO', 'Host already clustered with %s.' % host)
+            return
+    cmd = [RABBITMQ_CTL, 'stop_app']
+    subprocess.check_call(cmd)
+    cmd = [RABBITMQ_CTL, cluster_cmd, 'rabbit@%s' % host]
+    subprocess.check_call(cmd)
+    cmd = [RABBITMQ_CTL, 'start_app']
+    subprocess.check_call(cmd)
+
+
+def set_node_name(name):
+    # update or append RABBITMQ_NODENAME to environment config.
+    # rabbitmq.conf.d is not present on all releases, so use or create
+    # rabbitmq-env.conf instead.
+    if not os.path.isfile(ENV_CONF):
+        utils.juju_log('INFO', '%s does not exist, creating.' % ENV_CONF)
+        with open(ENV_CONF, 'wb') as out:
+            out.write('RABBITMQ_NODENAME=%s\n' % name)
+        return
+
+    out = []
+    f = False
+    for line in open(ENV_CONF).readlines():
+        if line.strip().startswith('RABBITMQ_NODENAME'):
+            f = True
+            line = 'RABBITMQ_NODENAME=%s\n' % name
+        out.append(line)
+    if not f:
+        out.append('RABBITMQ_NODENAME=%s\n' % name)
+    utils.juju_log('INFO', 'Updating %s, RABBITMQ_NODENAME=%s' %\
+                   (ENV_CONF, name))
+    with open(ENV_CONF, 'wb') as conf:
+        conf.write(''.join(out))
+
+
+def get_node_name():
+    if not os.path.exists(ENV_CONF):
+        return None
+    env_conf = open(ENV_CONF, 'r').readlines()
+    node_name = None
+    for l in env_conf:
+        if l.startswith('RABBITMQ_NODENAME'):
+            node_name = l.split('=')[1].strip()
+    return node_name
+
+
+def _manage_plugin(plugin, action):
+    os.environ['HOME'] = '/root'
+    _rabbitmq_plugins = \
+        glob.glob('/usr/lib/rabbitmq/lib/rabbitmq_server-*/sbin/rabbitmq-plugins')[0]
+    subprocess.check_call([ _rabbitmq_plugins, action, plugin])
+
+
+def enable_plugin(plugin):
+    _manage_plugin(plugin, 'enable')
+
+
+def disable_plugin(plugin):
+    _manage_plugin(plugin, 'disable')
+
+ssl_key_file = "/etc/rabbitmq/rabbit-server-privkey.pem"
+ssl_cert_file = "/etc/rabbitmq/rabbit-server-cert.pem"
+
+
+def enable_ssl(ssl_key, ssl_cert, ssl_port):
+    uid = pwd.getpwnam("root").pw_uid
+    gid = grp.getgrnam("rabbitmq").gr_gid
+    with open(ssl_key_file, 'w') as key_file:
+        key_file.write(ssl_key)
+    os.chmod(ssl_key_file, 0640)
+    os.chown(ssl_key_file, uid, gid)
+    with open(ssl_cert_file, 'w') as cert_file:
+        cert_file.write(ssl_cert)
+    os.chmod(ssl_cert_file, 0640)
+    os.chown(ssl_cert_file, uid, gid)
+    with open(RABBITMQ_CONF, 'w') as rmq_conf:
+        rmq_conf.write(utils.render_template(os.path.basename(RABBITMQ_CONF),
+                              { "ssl_port": ssl_port,
+                                "ssl_cert_file": ssl_cert_file,
+                                "ssl_key_file": ssl_key_file})
+                       )

=== removed file 'hooks/rabbitmq-common'
--- hooks/rabbitmq-common	2011-12-05 19:55:29 +0000
+++ hooks/rabbitmq-common	1970-01-01 00:00:00 +0000
@@ -1,56 +0,0 @@
-#!/bin/bash
-#
-#  rabbitmq-common - common formula shell functions and config variables
-#
-#  Copyright (C) 2011  Canonical Ltd.
-#  Author: Adam Gandelman <[e-mail]>
-#
-#  This program is free software: you can redistribute it and/or modify
-#  it under the terms of the GNU General Public License as published by
-#  the Free Software Foundation, either version 3 of the License, or
-#  (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-#  but WITHOUT ANY WARRANTY; without even the implied warranty of
-#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-#  GNU General Public License for more details.
-#
-#  You should have received a copy of the GNU General Public License
-#  along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-RABBIT_CTL='rabbitmqctl'
-HOSTNAME=`hostname -f`
-ERLANG_COOKIE="/var/lib/rabbitmq/.erlang.cookie"
-
-function user_exists {
-  $RABBIT_CTL list_users | grep -wq "^$1"
-}
-
-function user_is_admin {
-  $RABBIT_CTL list_users | grep -w "^$1" | grep -q "administrator"
-}
-
-function vhost_exists {
-  $RABBIT_CTL list_vhosts | grep "^$VHOST\$" >/dev/null
-}
-
-function create_vhost {
-  $RABBIT_CTL add_vhost $VHOST
-}
-
-function user_create {
-  juju-log "rabbitmq: Creating user $1."
-
-  $RABBIT_CTL add_user $1 $PASSWORD || return 1
-
-  # grant the user all permissions on the default vhost /
-  # TODO: investigate sane permissions
-  juju-log "rabbitmq: Granting permission to $1 on vhost /"
-  $RABBIT_CTL set_permissions -p $VHOST $1 ".*" ".*" ".*"
-
-  if [[ $2 == 'admin' ]]  ; then
-    user_is_admin $1 && return 0
-    juju-log "rabbitmq: Granting user $1 admin access"
-    $RABBIT_CTL set_user_tags "$1" administrator || return 1
-  fi
-}

=== removed file 'hooks/rabbitmq-relations'
--- hooks/rabbitmq-relations	2013-03-01 17:36:31 +0000
+++ hooks/rabbitmq-relations	1970-01-01 00:00:00 +0000
@@ -1,127 +0,0 @@
-#!/bin/bash
-#
-#    rabbitmq-relations - relations to be used by formula, referenced
-#                         via symlink
-#
-#    Copyright (C) 2011  Canonical Ltd.
-#    Author: Adam Gandelman <[e-mail]>
-#
-#    This program is free software: you can redistribute it and/or modify
-#    it under the terms of the GNU General Public License as published by
-#    the Free Software Foundation, either version 3 of the License, or
-#    (at your option) any later version.
-#
-#    This program is distributed in the hope that it will be useful,
-#    but WITHOUT ANY WARRANTY; without even the implied warranty of
-#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-#    GNU General Public License for more details.
-#
-#    You should have received a copy of the GNU General Public License
-#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-set -u
-FORMULA_DIR=$(dirname $0)
-ARG0=${0##*/}
-
-if [[ -e $FORMULA_DIR/rabbitmq-common ]] ; then
-  . $FORMULA_DIR/rabbitmq-common
-else
-  juju-log "rabbitmq-server: ERROR Could not load $FORMULA_DIR/rabbitmq-common"
-  exit 1
-fi
-
-juju-log "rabbitmq-server: Firing hook $ARG0."
-
-function install_hook() {
-  [ -d exec.d ] && ( for f in exec.d/*/charm-pre-install; do [ -x $f ] && /bin/sh -c "$f";done )
-
-  [[ ! `which pwgen` ]] && apt-get -y install pwgen
-  DEBIAN_FRONTEND=noninteractive apt-get -qqy \
-    install --no-install-recommends rabbitmq-server
-  rc=$?
-  service rabbitmq-server stop
-  open-port 5672/tcp
-}
-
-function upgrade_charm {
-  [ -d exec.d ] && ( for f in exec.d/*/charm-pre-install; do [ -x $f ] && /bin/sh -c "$f";done )
-}  
-
-function amqp_changed() {
-  # Connecting clients should request a username and vhost.
-  # In reponse, we generate a password for new users,
-  # grant the user access on the default vhost "/",
-  # and tell it where to reach us.
-  RABBIT_USER=`relation-get username`
-  VHOST=`relation-get vhost`
-  if [[ -z $RABBIT_USER ]] || [[ -z $VHOST ]] ; then
-    juju-log "rabbitmq-server: RABBIT_USER||VHOST not yet received from peer."
-    exit 0
-  fi
-  PASSWD_FILE="/var/lib/juju/$RABBIT_USER.passwd"
-  if [[ -e $PASSWD_FILE ]] ; then
-    PASSWORD=$(cat $PASSWD_FILE)
-  else
-    PASSWORD=$(pwgen 10 1)
-    echo $PASSWORD >$PASSWD_FILE
-    chmod 0400 $PASSWD_FILE
-  fi
-  if ! vhost_exists ; then
-    juju-log "rabbitmq-server: Creating vhost $VHOST"
-    create_vhost
-  fi
-  if ! user_exists $RABBIT_USER ; then
-    juju-log "rabbitmq-server: Creating user $RABBIT_USER"
-    user_create $RABBIT_USER admin || exit 1
-  else
-    juju-log "rabbitmq-server: user $RABBIT_USER already exists."
-  fi
-  juju-log "rabbitmq-server: Returning credentials for $RABBIT_USER@$HOSTNAME"
-  relation-set hostname=`unit-get private-address`
-  relation-set password=$PASSWORD
-}
-
-function cluster_joined {
-  REMOTE_UNIT_ID=$(echo $JUJU_REMOTE_UNIT | cut -d/ -f2)
-  LOCAL_UNIT_ID=$(echo $JUJU_UNIT_NAME | cut -d/ -f2)
-  [[ $LOCAL_UNIT_ID -gt $REMOTE_UNIT_ID ]] && echo "Relation greater" && exit 0
-  if [[ ! -e $ERLANG_COOKIE ]] ; then
-    juju-log "rabbitmq-server: ERROR Could not find cookie at $ERLANG_COOKIE"
-    exit 1
-  fi
-  relation-set cookie=$(cat $ERLANG_COOKIE) host=$HOSTNAME
-}
-
-function cluster_changed {
-  REMOTE_UNIT_ID=$(echo $JUJU_REMOTE_UNIT | cut -d/ -f2)
-  LOCAL_UNIT_ID=$(echo $JUJU_UNIT_NAME | cut -d/ -f2)
-  [[ $LOCAL_UNIT_ID -lt $REMOTE_UNIT_ID ]] && echo "Relation lesser" && exit 0
-
-  REMOTE_HOST=$(relation-get host)
-  COOKIE_VALUE=$(relation-get cookie)
-  [[ -z $REMOTE_HOST ]] || [[ -z $COOKIE_VALUE ]] && \
-    juju-log "rabbimtq-server: REMOTE_HOST||COOKIE_VALUE not yet set." \
-    exit 0
-
-  service rabbitmq-server stop
-  echo -n $COOKIE_VALUE > $ERLANG_COOKIE
-  service rabbitmq-server start
-  rabbitmqctl reset
-  rabbitmqctl cluster rabbit@$HOSTNAME rabbit@$REMOTE_HOST
-  rabbitmqctl start_app
-}
-
-case $ARG0 in
-  "install") install_hook ;;
-  "start") service rabbitmq-server status || service rabbitmq-server start ;;
-  "stop") service rabbitmq-server status && service rabbitmq-server stop ;;
-  "upgrade-charm") upgrade_charm ;;
-  "amqp-relation-joined") exit 0 ;;
-  "amqp-relation-changed") amqp_changed ;;
-  "cluster-relation-joined") cluster_joined ;;
-  "cluster-relation-changed") cluster_changed ;;
-esac
-
-rc=$?
-juju-log "rabbitmq-server: Hook $ARG0 complete. Exiting $rc"
-exit $rc

=== added file 'hooks/rabbitmq_server_relations.py'
--- hooks/rabbitmq_server_relations.py	1970-01-01 00:00:00 +0000
+++ hooks/rabbitmq_server_relations.py	2013-06-06 15:35:31 +0000
@@ -0,0 +1,302 @@
+#!/usr/bin/python
+
+import os
+import shutil
+import sys
+import subprocess
+import glob
+
+
+import rabbit_utils as rabbit
+import lib.utils as utils
+import lib.cluster_utils as cluster
+import lib.ceph_utils as ceph
+import lib.openstack_common as openstack
+
+
+SERVICE_NAME = os.getenv('JUJU_UNIT_NAME').split('/')[0]
+POOL_NAME = SERVICE_NAME
+RABBIT_DIR = '/var/lib/rabbitmq'
+
+
+def install():
+    pre_install_hooks()
+    utils.install(*rabbit.PACKAGES)
+    utils.expose(5672)
+
+
+def amqp_changed(relation_id=None, remote_unit=None):
+    if not cluster.eligible_leader('res_rabbitmq_vip'):
+        msg = 'amqp_changed(): Deferring amqp_changed to eligible_leader.'
+        utils.juju_log('INFO', msg)
+        return
+
+    rabbit_user = utils.relation_get('username', rid=relation_id,
+                                     unit=remote_unit)
+    vhost = utils.relation_get('vhost', rid=relation_id, unit=remote_unit)
+    if None in [rabbit_user, vhost]:
+        utils.juju_log('INFO', 'amqp_changed(): Relation not ready.')
+        return
+
+    password_file = os.path.join(RABBIT_DIR, '%s.passwd' % rabbit_user)
+    if os.path.exists(password_file):
+        password = open(password_file).read().strip()
+    else:
+        cmd = ['pwgen', '64', '1']
+        password = subprocess.check_output(cmd).strip()
+        with open(password_file, 'wb') as out:
+            out.write(password)
+
+    rabbit.create_vhost(vhost)
+    rabbit.create_user(rabbit_user, password)
+    rabbit.grant_permissions(rabbit_user, vhost)
+    rabbit_hostname = utils.unit_get('private-address')
+
+    relation_settings = {
+        'password': password,
+        'hostname': rabbit_hostname
+    }
+    if cluster.is_clustered():
+        relation_settings['clustered'] = 'true'
+        relation_settings['vip'] = utils.config_get('vip')
+    if relation_id:
+        relation_settings['rid'] = relation_id
+    utils.relation_set(**relation_settings)
+
+
+def cluster_joined():
+    if utils.is_relation_made('ha'):
+        utils.juju_log('INFO',
+                       'hacluster relation is present, skipping native '\
+                       'rabbitmq cluster config.')
+        return
+    l_unit_no = os.getenv('JUJU_UNIT_NAME').split('/')[1]
+    r_unit_no = os.getenv('JUJU_REMOTE_UNIT').split('/')[1]
+    if l_unit_no > r_unit_no:
+        utils.juju_log('INFO', 'cluster_joined: Relation greater.')
+        return
+    rabbit.COOKIE_PATH = '/var/lib/rabbitmq/.erlang.cookie'
+    if not os.path.isfile(rabbit.COOKIE_PATH):
+        utils.juju_log('ERROR', 'erlang cookie missing from %s' %\
+                       rabbit.COOKIE_PATH)
+    cookie = open(rabbit.COOKIE_PATH, 'r').read().strip()
+    local_hostname = subprocess.check_output(['hostname']).strip()
+    utils.relation_set(cookie=cookie, host=local_hostname)
+
+
+def cluster_changed():
+    if utils.is_relation_made('ha'):
+        utils.juju_log('INFO',
+                       'hacluster relation is present, skipping native '\
+                       'rabbitmq cluster config.')
+        return
+    l_unit_no = os.getenv('JUJU_UNIT_NAME').split('/')[1]
+    r_unit_no = os.getenv('JUJU_REMOTE_UNIT').split('/')[1]
+    if l_unit_no < r_unit_no:
+        utils.juju_log('INFO', 'cluster_joined: Relation lesser.')
+        return
+
+    remote_host = utils.relation_get('host')
+    cookie = utils.relation_get('cookie')
+    if None in [remote_host, cookie]:
+        utils.juju_log('INFO',
+                       'cluster_joined: remote_host|cookie not yet set.')
+        return
+
+    if open(rabbit.COOKIE_PATH, 'r').read().strip() == cookie:
+        utils.juju_log('INFO', 'Cookie already synchronized with peer.')
+        return
+
+    utils.juju_log('INFO', 'Synchronizing erlang cookie from peer.')
+    rabbit.service('stop')
+    with open(rabbit.COOKIE_PATH, 'wb') as out:
+        out.write(cookie)
+    rabbit.service('start')
+    rabbit.cluster_with(remote_host)
+
+
+def ha_joined():
+    corosync_bindiface = utils.config_get('ha-bindiface')
+    corosync_mcastport = utils.config_get('ha-mcastport')
+    vip = utils.config_get('vip')
+    vip_iface = utils.config_get('vip_iface')
+    vip_cidr = utils.config_get('vip_cidr')
+    rbd_name = utils.config_get('rbd-name')
+
+    if None in [corosync_bindiface, corosync_mcastport, vip, vip_iface,
+                vip_cidr, rbd_name]:
+        utils.juju_log('ERROR', 'Insufficient configuration data to '\
+                       'configure hacluster.')
+        sys.exit(1)
+
+    if not utils.is_relation_made('ceph', 'auth'):
+        utils.juju_log('INFO',
+                       'ha_joined: No ceph relation yet, deferring.')
+        return
+
+    name = '%s@localhost' % SERVICE_NAME
+    if rabbit.get_node_name() != name:
+        utils.juju_log('INFO', 'Stopping rabbitmq-server.')
+        utils.stop('rabbitmq-server')
+        rabbit.set_node_name('%s@localhost' % SERVICE_NAME)
+    else:
+        utils.juju_log('INFO', 'Node name already set to %s.' % name)
+
+    relation_settings = {}
+    relation_settings['corosync_bindiface'] = corosync_bindiface
+    relation_settings['corosync_mcastport'] = corosync_mcastport
+
+    relation_settings['resources'] = {
+        'res_rabbitmq_rbd': 'ocf:ceph:rbd',
+        'res_rabbitmq_fs': 'ocf:heartbeat:Filesystem',
+        'res_rabbitmq_vip': 'ocf:heartbeat:IPaddr2',
+        'res_rabbitmq-server': 'lsb:rabbitmq-server',
+    }
+
+    relation_settings['resource_params'] = {
+        'res_rabbitmq_rbd': 'params name="%s" pool="%s" user="%s" '
+                            'secret="%s"' % \
+                            (rbd_name, POOL_NAME,
+                             SERVICE_NAME, ceph.keyfile_path(SERVICE_NAME)),
+        'res_rabbitmq_fs': 'params device="/dev/rbd/%s/%s" directory="%s" '\
+                           'fstype="ext4" op start start-delay="10s"' %\
+                           (POOL_NAME, rbd_name, RABBIT_DIR),
+        'res_rabbitmq_vip': 'params ip="%s" cidr_netmask="%s" nic="%s"' %\
+                            (vip, vip_cidr, vip_iface),
+        'res_rabbitmq-server': 'op start start-delay="5s" op monitor interval="5s"',
+    }
+
+    relation_settings['groups'] = {
+        'grp_rabbitmq': 'res_rabbitmq_rbd res_rabbitmq_fs res_rabbitmq_vip '\
+                        'res_rabbitmq-server',
+    }
+
+    for rel_id in utils.relation_ids('ha'):
+        utils.relation_set(rid=rel_id, **relation_settings)
+
+    env_vars = {
+        'OPENSTACK_PORT_EPMD': 4369,
+        'OPENSTACK_PORT_MCASTPORT': utils.config_get('ha-mcastport'),
+    }
+    openstack.save_script_rc(**env_vars)
+
+
+def ha_changed():
+    if not cluster.is_clustered():
+        return
+    vip = utils.config_get('vip')
+    utils.juju_log('INFO', 'ha_changed(): We are now HA clustered. '
+                   'Advertising our VIP (%s) to all AMQP clients.' %\
+                   vip)
+    # need to re-authenticate all clients since node-name changed.
+    for rid in utils.relation_ids('amqp'):
+        for unit in utils.relation_list(rid):
+            amqp_changed(relation_id=rid, remote_unit=unit)
+
+
+def ceph_joined():
+    utils.juju_log('INFO', 'Start Ceph Relation Joined')
+    ceph.install()
+    utils.juju_log('INFO', 'Finish Ceph Relation Joined')
+
+
+def ceph_changed():
+    utils.juju_log('INFO', 'Start Ceph Relation Changed')
+    auth = utils.relation_get('auth')
+    key = utils.relation_get('key')
+    if None in [auth, key]:
+        utils.juju_log('INFO', 'Missing key or auth in relation')
+        sys.exit(0)
+
+    ceph.configure(service=SERVICE_NAME, key=key, auth=auth)
+
+    if cluster.eligible_leader('res_rabbitmq_vip'):
+        rbd_img = utils.config_get('rbd-name')
+        rbd_size = utils.config_get('rbd-size')
+        sizemb = int(rbd_size.split('G')[0]) * 1024
+        blk_device = '/dev/rbd/%s/%s' % (POOL_NAME, rbd_img)
+        ceph.ensure_ceph_storage(service=SERVICE_NAME, pool=POOL_NAME,
+                                 rbd_img=rbd_img, sizemb=sizemb,
+                                 fstype='ext4', mount_point=RABBIT_DIR,
+                                 blk_device=blk_device,
+                                 system_services=['rabbitmq-server'])
+    else:
+        utils.juju_log('INFO',
+                       'This is not the peer leader. Not configuring RBD.')
+        utils.juju_log('INFO', 'Stopping rabbitmq-server.')
+        utils.stop('rabbitmq-server')
+
+    # If 'ha' relation has been made before the 'ceph' relation
+    # it is important to make sure the ha-relation data is being
+    # sent.
+    if utils.is_relation_made('ha'):
+        utils.juju_log('INFO', '*ha* relation exists. Triggering ha_joined()')
+        ha_joined()
+    else:
+        utils.juju_log('INFO', '*ha* relation does not exist.')
+    utils.juju_log('INFO', 'Finish Ceph Relation Changed')
+
+
+def upgrade_charm():
+    pre_install_hooks()
+    # Ensure older passwd files in /var/lib/juju are moved to
+    # /var/lib/rabbitmq which will end up replicated if clustered.
+    for f in [f for f in os.listdir('/var/lib/juju')
+              if os.path.isfile(os.path.join('/var/lib/juju', f))]:
+        if f.endswith('.passwd'):
+            s = os.path.join('/var/lib/juju', f)
+            d = os.path.join('/var/lib/rabbitmq', f)
+            utils.juju_log('INFO',
+                           'upgrade_charm: Migrating stored passwd'
+                           ' from %s to %s.' % (s, d))
+            shutil.move(s, d)
+
+MAN_PLUGIN = 'rabbitmq_management'
+
+def config_changed():
+    if utils.config_get('management_plugin') == True:
+        rabbit.enable_plugin(MAN_PLUGIN)
+        utils.open_port(55672)
+    else:
+        rabbit.disable_plugin(MAN_PLUGIN)
+        utils.close_port(55672)
+    
+    if utils.config_get('ssl_enabled') == True:
+        ssl_key = utils.config_get('ssl_key')
+        ssl_cert = utils.config_get('ssl_cert')
+        ssl_port = utils.config_get('ssl_port')
+        if None in [ ssl_key, ssl_cert, ssl_port ]:
+            utils.juju_log('ERROR',
+                           'Please provide ssl_key, ssl_cert and ssl_port'
+                           ' config when enabling SSL support')
+            sys.exit(1)
+        else:
+            rabbit.enable_ssl(ssl_key, ssl_cert, ssl_port)
+            utils.open_port(ssl_port)
+    else:
+        if os.path.exists(rabbit.RABBITMQ_CONF):
+            os.remove(rabbit.RABBITMQ_CONF)
+        utils.close_port(utils.config_get('ssl_port'))
+    
+    utils.restart('rabbitmq-server')
+
+
+def pre_install_hooks():
+    for f in glob.glob('exec.d/*/charm-pre-install'):
+        if os.path.isfile(f) and os.access(f, os.X_OK):
+            subprocess.check_call(['sh', '-c', f])
+
+hooks = {
+    'install': install,
+    'amqp-relation-changed': amqp_changed,
+    'cluster-relation-joined': cluster_joined,
+    'cluster-relation-changed': cluster_changed,
+    'ha-relation-joined': ha_joined,
+    'ha-relation-changed': ha_changed,
+    'ceph-relation-joined': ceph_joined,
+    'ceph-relation-changed': ceph_changed,
+    'upgrade-charm': upgrade_charm,
+    'config-changed': config_changed
+}
+
+utils.do_hooks(hooks)

=== removed symlink 'hooks/start'
=== target was u'rabbitmq-relations'
=== removed symlink 'hooks/stop'
=== target was u'rabbitmq-relations'
=== modified symlink 'hooks/upgrade-charm'
=== target changed u'rabbitmq-relations' => u'rabbitmq_server_relations.py'
=== modified file 'metadata.yaml'
--- metadata.yaml	2013-04-22 19:38:52 +0000
+++ metadata.yaml	2013-06-06 15:35:31 +0000
@@ -9,9 +9,15 @@
 provides:
   amqp:
     interface: rabbitmq
+requires:
   nrpe-external-master:
     interface: nrpe-external-master
     scope: container
+  ha:
+    interface: hacluster
+    scope: container
+  ceph:
+    interface: ceph-client
 peers:
   cluster:
-    interface: rabbitmq
+    interface: rabbitmq-ha

=== modified file 'revision'
--- revision	2013-03-13 18:36:42 +0000
+++ revision	2013-06-06 15:35:31 +0000
@@ -1,1 +1,1 @@
-38
+95

=== added directory 'scripts'
=== added file 'scripts/add_to_cluster'
--- scripts/add_to_cluster	1970-01-01 00:00:00 +0000
+++ scripts/add_to_cluster	2013-06-06 15:35:31 +0000
@@ -0,0 +1,13 @@
+#!/bin/bash
+service corosync start || /bin/true
+sleep 2
+while ! service pacemaker start; do
+    echo "Attempting to start pacemaker"
+    sleep 1;
+done;
+crm node online
+sleep 2
+while crm status | egrep -q 'Stopped$'; do
+    echo "Waiting for nodes to come online"
+    sleep 1
+done

=== added file 'scripts/remove_from_cluster'
--- scripts/remove_from_cluster	1970-01-01 00:00:00 +0000
+++ scripts/remove_from_cluster	2013-06-06 15:35:31 +0000
@@ -0,0 +1,4 @@
+#!/bin/bash
+crm node standby
+service pacemaker stop
+service corosync stop

=== added directory 'templates'
=== added file 'templates/rabbitmq.config'
--- templates/rabbitmq.config	1970-01-01 00:00:00 +0000
+++ templates/rabbitmq.config	2013-06-06 15:35:31 +0000
@@ -0,0 +1,10 @@
+[
+    {rabbit, [    
+    	{ssl_listeners, [{{ ssl_port }}]},
+    	{ssl_options, [
+        	{certfile, "{{ ssl_cert_file }}"},
+        	{keyfile, "{{ ssl_key_file }}"}
+    	]},
+    	{tcp_listeners, [5672]}
+    ]}
+].    
\ No newline at end of file