Merge lp:~1chb1n/charms/trusty/rabbitmq-server/amulet-refactor-1509b into lp:~openstack-charmers-archive/charms/trusty/rabbitmq-server/next

[Ryan Beisner]

Refactor amulet tests, deprecate old tests

Sync tests/charmhelpers from https://code.launchpad.net/~1chb1n/charm-helpers/amulet-rmq-helpers

Pull in cluster fixes from lp:~thedac/charms/trusty/rabbitmq-server/native-cluster-race-fixes

Provides regression testing against recurrence of:
https://bugs.launchpad.net/charms/+source/rabbitmq-server/+bug/1486177

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8626 rabbitmq-server-next for 1chb1n mp270102
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8626/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9328 rabbitmq-server-next for 1chb1n mp270102
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9328/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6238 rabbitmq-server-next for 1chb1n mp270102
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 124
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12270304/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6238/

[Ryan Beisner (1chb1n)]

Ignore amulet fail #6238. Undercloud woes.

[lp:~1chb1n/charms/trusty/rabbitmq-server/amulet-refactor-1509b]

112. By Ryan Beisner on 2015-09-04

sync tests/charmhelpers from lp:~1chb1n/charm-helpers/amulet-rmq-helpers/

113. By Ryan Beisner on 2015-09-04

pull in proposed cluster race fix from lp:~thedac/charms/trusty/rabbitmq-server/native-cluster-race-fixes

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9390 rabbitmq-server-next for 1chb1n mp270102
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9390/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8685 rabbitmq-server-next for 1chb1n mp270102
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8685/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6256 rabbitmq-server-next for 1chb1n mp270102
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6256/

[Ryan Beisner (1chb1n)]

Re-running multiple iterations of the amulet test at this rev to re-confirm LE resolution...

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6259 rabbitmq-server-next for 1chb1n mp270102
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12279813/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6259/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6258 rabbitmq-server-next for 1chb1n mp270102
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12279853/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6258/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6260 rabbitmq-server-next for 1chb1n mp270102
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6260/

[lp:~1chb1n/charms/trusty/rabbitmq-server/amulet-refactor-1509b]

114. By Ryan Beisner on 2015-09-08

pull in changes from proposed cluster race fix lp:~thedac/charms/trusty/rabbitmq-server/native-cluster-race-fixes

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8832 rabbitmq-server-next for 1chb1n mp270102
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8832/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9594 rabbitmq-server-next for 1chb1n mp270102
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9594/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6317 rabbitmq-server-next for 1chb1n mp270102
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6317/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6322 rabbitmq-server-next for 1chb1n mp270102
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6322/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6321 rabbitmq-server-next for 1chb1n mp270102
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6321/

[Liam Young (gnuoy)]

Thanks for cleaning this up and getting it working, looks really good. Approved

review: Approve

=== modified file 'Makefile'
--- Makefile	2015-04-20 11:13:39 +0000
+++ Makefile	2015-09-08 14:01:41 +0000
@@ -29,14 +29,15 @@
 	@$(PYTHON) bin/charm_helpers_sync.py -c charm-helpers.yaml
 	@$(PYTHON) bin/charm_helpers_sync.py -c charm-helpers-tests.yaml
 
-publish: lint
+publish: lint test
 	bzr push lp:charms/rabbitmq-server
 	bzr push lp:charms/trusty/rabbitmq-server
 
-unit_test: .venv
+test: .venv
 	@echo Starting tests...
-	env CHARM_DIR=$(CHARM_DIR) $(TEST_PREFIX) .venv/bin/nosetests unit_tests/
+	env CHARM_DIR=$(CHARM_DIR) $(TEST_PREFIX) .venv/bin/nosetests \
+        --nologcapture --with-coverage unit_tests/
 
 functional_test:
 	@echo Starting amulet tests...
-	@juju test -v -p AMULET_HTTP_PROXY,OS_USERNAME,OS_TENANT_NAME,OS_REGION_NAME,OS_PASSWORD,OS_AUTH_URL --timeout 900
+	@juju test -v -p AMULET_HTTP_PROXY,AMULET_OS_VIP --timeout 2700

=== modified file 'charm-helpers-tests.yaml'
--- charm-helpers-tests.yaml	2015-07-31 22:00:49 +0000
+++ charm-helpers-tests.yaml	2015-09-08 14:01:41 +0000
@@ -1,6 +1,5 @@
 destination: tests/charmhelpers
 branch: lp:charm-helpers
 include:
-    - core
-    - cli
-    - contrib.ssl
+    - contrib.amulet
+    - contrib.openstack.amulet

=== modified file 'hooks/rabbit_utils.py'
--- hooks/rabbit_utils.py	2015-09-01 04:53:33 +0000
+++ hooks/rabbit_utils.py	2015-09-08 14:01:41 +0000
@@ -299,7 +299,12 @@
             cmd = [RABBITMQ_CTL, 'stop_app']
             subprocess.check_call(cmd)
             cmd = [RABBITMQ_CTL, cluster_cmd, 'rabbit@%s' % node]
-            subprocess.check_call(cmd)
+            try:
+                subprocess.check_output(cmd, stderr=subprocess.STDOUT)
+            except subprocess.CalledProcessError as e:
+                if not e.returncode == 2 or \
+                        "{ok,already_member}" not in e.output:
+                    raise e
             cmd = [RABBITMQ_CTL, 'start_app']
             subprocess.check_call(cmd)
             log('Host clustered with %s.' % node)

=== modified file 'hooks/rabbitmq_server_relations.py'
--- hooks/rabbitmq_server_relations.py	2015-09-01 04:53:33 +0000
+++ hooks/rabbitmq_server_relations.py	2015-09-08 14:01:41 +0000
@@ -81,6 +81,7 @@
     peer_store,
     peer_store_and_set,
     peer_retrieve_by_prefix,
+    leader_get,
 )
 
 from charmhelpers.contrib.network.ip import get_address_in_network
@@ -304,31 +305,35 @@
         return
 
     if is_elected_leader('res_rabbitmq_vip'):
+        log('Leader peer_storing cookie', level=INFO)
         cookie = open(rabbit.COOKIE_PATH, 'r').read().strip()
         peer_store('cookie', cookie)
 
 
 @hooks.hook('cluster-relation-changed')
 def cluster_changed():
+    # Future travelers beware ordering is significant
+    rdata = relation_get()
+    # sync passwords
+    blacklist = ['hostname', 'private-address', 'public-address']
+    whitelist = [a for a in rdata.keys() if a not in blacklist]
+    peer_echo(includes=whitelist)
+
     cookie = peer_retrieve('cookie')
     if not cookie:
         log('cluster_joined: cookie not yet set.', level=INFO)
         return
 
-    rdata = relation_get()
     if config('prefer-ipv6') and rdata.get('hostname'):
-        private_address = rdata['private-address']
-        hostname = rdata['hostname']
+        private_address = rdata.get('private-address')
+        hostname = rdata.get('hostname')
         if hostname:
             rabbit.update_hosts_file({private_address: hostname})
 
-    # sync passwords
-    blacklist = ['hostname', 'private-address', 'public-address']
-    whitelist = [a for a in rdata.keys() if a not in blacklist]
-    peer_echo(includes=whitelist)
-
     if not is_sufficient_peers():
         # Stop rabbit until leader has finished configuring
+        log('Not enough peers, stopping until leader is configured',
+            level=INFO)
         service_stop('rabbitmq-server')
         return
 
@@ -358,9 +363,12 @@
             amqp_changed(relation_id=rid, remote_unit=unit)
 
 
-def update_cookie():
+def update_cookie(leaders_cookie=None):
     # sync cookie
-    cookie = peer_retrieve('cookie')
+    if leaders_cookie:
+        cookie = leaders_cookie
+    else:
+        cookie = peer_retrieve('cookie')
     cookie_local = None
     with open(rabbit.COOKIE_PATH, 'r') as f:
         cookie_local = f.read().strip()
@@ -763,6 +771,18 @@
 
 @hooks.hook('leader-settings-changed')
 def leader_settings_changed():
+    # Get cookie from leader, update cookie locally and
+    # force cluster-relation-changed hooks to run on peers
+    cookie = leader_get(attribute='cookie')
+    if cookie:
+        update_cookie(leaders_cookie=cookie)
+        # Force cluster-relation-changed hooks to run on peers
+        # This will precipitate peer clustering
+        # Without this a chicken and egg scenario prevails when
+        # using LE and peerstorage
+        for rid in relation_ids('cluster'):
+            relation_set(relation_id=rid, relation_settings={'cookie': cookie})
+
     # If leader has changed and access credentials, ripple these
     # out from all units
     for rid in relation_ids('amqp'):

=== modified file 'metadata.yaml'
--- metadata.yaml	2013-11-15 19:15:16 +0000
+++ metadata.yaml	2015-09-08 14:01:41 +0000
@@ -5,7 +5,10 @@
  RabbitMQ is an implementation of AMQP, the emerging standard for high
  performance enterprise messaging. The RabbitMQ server is a robust and
  scalable implementation of an AMQP broker.
-categories: ["misc"]
+tags:
+  - openstack
+  - amqp
+  - misc
 provides:
   amqp:
     interface: rabbitmq

=== added file 'tests/00-setup'
--- tests/00-setup	1970-01-01 00:00:00 +0000
+++ tests/00-setup	2015-09-08 14:01:41 +0000
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+set -ex
+
+sudo add-apt-repository --yes ppa:juju/stable
+sudo apt-get update --yes
+sudo apt-get install --yes python-amulet \
+                           python-cinderclient \
+                           python-distro-info \
+                           python-glanceclient \
+                           python-heatclient \
+                           python-keystoneclient \
+                           python-neutronclient \
+                           python-novaclient \
+                           python-pika \
+                           python-swiftclient

=== added file 'tests/014-basic-precise-icehouse'
--- tests/014-basic-precise-icehouse	1970-01-01 00:00:00 +0000
+++ tests/014-basic-precise-icehouse	2015-09-08 14:01:41 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on precise-icehouse."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='precise',
+                                    openstack='cloud:precise-icehouse',
+                                    source='cloud:precise-updates/icehouse')
+    deployment.run_tests()

=== added file 'tests/015-basic-trusty-icehouse'
--- tests/015-basic-trusty-icehouse	1970-01-01 00:00:00 +0000
+++ tests/015-basic-trusty-icehouse	2015-09-08 14:01:41 +0000
@@ -0,0 +1,9 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on trusty-icehouse."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='trusty')
+    deployment.run_tests()

=== added file 'tests/016-basic-trusty-juno'
--- tests/016-basic-trusty-juno	1970-01-01 00:00:00 +0000
+++ tests/016-basic-trusty-juno	2015-09-08 14:01:41 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on trusty-juno."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='trusty',
+                                    openstack='cloud:trusty-juno',
+                                    source='cloud:trusty-updates/juno')
+    deployment.run_tests()

=== added file 'tests/017-basic-trusty-kilo'
--- tests/017-basic-trusty-kilo	1970-01-01 00:00:00 +0000
+++ tests/017-basic-trusty-kilo	2015-09-08 14:01:41 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on trusty-kilo."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='trusty',
+                                    openstack='cloud:trusty-kilo',
+                                    source='cloud:trusty-updates/kilo')
+    deployment.run_tests()

=== added file 'tests/019-basic-vivid-kilo'
--- tests/019-basic-vivid-kilo	1970-01-01 00:00:00 +0000
+++ tests/019-basic-vivid-kilo	2015-09-08 14:01:41 +0000
@@ -0,0 +1,9 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on vivid-kilo."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='vivid')
+    deployment.run_tests()

=== added file 'tests/020-basic-trusty-liberty'
--- tests/020-basic-trusty-liberty	1970-01-01 00:00:00 +0000
+++ tests/020-basic-trusty-liberty	2015-09-08 14:01:41 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on trusty-liberty."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='trusty',
+                                    openstack='cloud:trusty-liberty',
+                                    source='cloud:trusty-updates/liberty')
+    deployment.run_tests()

=== added file 'tests/021-basic-wily-liberty'
--- tests/021-basic-wily-liberty	1970-01-01 00:00:00 +0000
+++ tests/021-basic-wily-liberty	2015-09-08 14:01:41 +0000
@@ -0,0 +1,9 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on wily-liberty."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='wily')
+    deployment.run_tests()

=== added file 'tests/basic_deployment.py'
--- tests/basic_deployment.py	1970-01-01 00:00:00 +0000
+++ tests/basic_deployment.py	2015-09-08 14:01:41 +0000
@@ -0,0 +1,492 @@
+#!/usr/bin/python
+"""
+Basic 3-node rabbitmq-server native cluster + nrpe functional tests
+
+Cinder is present to exercise and inspect amqp relation functionality.
+
+Each individual test is idempotent, in that it creates/deletes
+a rmq test user, enables or disables ssl as needed.
+
+Test order is not required, however tests are numbered to keep
+relevant tests grouped together in run order.
+"""
+
+import amulet
+import time
+
+from charmhelpers.contrib.openstack.amulet.deployment import (
+    OpenStackAmuletDeployment
+)
+
+from charmhelpers.contrib.openstack.amulet.utils import (
+    OpenStackAmuletUtils,
+    DEBUG,
+    # ERROR
+)
+
+# Use DEBUG to turn on debug logging
+u = OpenStackAmuletUtils(DEBUG)
+
+
+class RmqBasicDeployment(OpenStackAmuletDeployment):
+    """Amulet tests on a basic rabbitmq cluster deployment. Verify
+       relations, service status, users and endpoint service catalog."""
+
+    def __init__(self, series=None, openstack=None, source=None, stable=False):
+        """Deploy the entire test environment."""
+        super(RmqBasicDeployment, self).__init__(series, openstack, source,
+                                                 stable)
+        self._add_services()
+        self._add_relations()
+        self._configure_services()
+        self._deploy()
+        self._initialize_tests()
+
+    def _add_services(self):
+        """Add services
+
+           Add the services that we're testing, where rmq is local,
+           and the rest of the service are from lp branches that are
+           compatible with the local charm (e.g. stable or next).
+           """
+        this_service = {
+            'name': 'rabbitmq-server',
+            'units': 3
+        }
+        other_services = [{'name': 'cinder'},
+                          {'name': 'nrpe'}]
+
+        super(RmqBasicDeployment, self)._add_services(this_service,
+                                                      other_services)
+
+    def _add_relations(self):
+        """Add relations for the services."""
+        relations = {'cinder:amqp': 'rabbitmq-server:amqp',
+                     'nrpe:nrpe-external-master':
+                     'rabbitmq-server:nrpe-external-master'}
+
+        super(RmqBasicDeployment, self)._add_relations(relations)
+
+    def _configure_services(self):
+        """Configure all of the services."""
+        rmq_config = {
+            'min-cluster-size': '3',
+            'max-cluster-tries': '6',
+            'ssl': 'off',
+            'management_plugin': 'False',
+            'stats_cron_schedule': '*/1 * * * *'
+        }
+        cinder_config = {}
+        configs = {'rabbitmq-server': rmq_config,
+                   'cinder': cinder_config}
+        super(RmqBasicDeployment, self)._configure_services(configs)
+
+    def _initialize_tests(self):
+        """Perform final initialization before tests get run."""
+        # Access the sentries for inspecting service units
+        self.rmq0_sentry = self.d.sentry.unit['rabbitmq-server/0']
+        self.rmq1_sentry = self.d.sentry.unit['rabbitmq-server/1']
+        self.rmq2_sentry = self.d.sentry.unit['rabbitmq-server/2']
+        self.cinder_sentry = self.d.sentry.unit['cinder/0']
+        self.nrpe_sentry = self.d.sentry.unit['nrpe/0']
+        u.log.debug('openstack release val: {}'.format(
+            self._get_openstack_release()))
+        u.log.debug('openstack release str: {}'.format(
+            self._get_openstack_release_string()))
+
+        # Let things settle a bit before moving forward
+        time.sleep(30)
+
+    def _get_rmq_sentry_units(self):
+        """Local helper specific to this 3-node rmq series of tests."""
+        return [self.rmq0_sentry,
+                self.rmq1_sentry,
+                self.rmq2_sentry]
+
+    def _test_rmq_amqp_messages_all_units(self, sentry_units,
+                                          ssl=False, port=None):
+        """Reusable test to send amqp messages to every listed rmq unit
+        and check every listed rmq unit for messages.
+
+        :param sentry_units: list of sentry units
+        :returns: None if successful.  Raise on error.
+        """
+
+        # Add test user if it does not already exist
+        u.add_rmq_test_user(sentry_units)
+
+        # Handle ssl
+        if ssl:
+            u.configure_rmq_ssl_on(sentry_units, self.d, port=port)
+        else:
+            u.configure_rmq_ssl_off(sentry_units, self.d)
+
+        # Publish and get amqp messages in all possible unit combinations.
+        # Qty of checks == (qty of units) ^ 2
+        amqp_msg_counter = 1
+        host_names = u.get_unit_hostnames(sentry_units)
+
+        for dest_unit in sentry_units:
+            dest_unit_name = dest_unit.info['unit_name']
+            dest_unit_host = dest_unit.info['public-address']
+            dest_unit_host_name = host_names[dest_unit_name]
+
+            for check_unit in sentry_units:
+                check_unit_name = check_unit.info['unit_name']
+                check_unit_host = check_unit.info['public-address']
+                check_unit_host_name = host_names[check_unit_name]
+
+                amqp_msg_stamp = u.get_uuid_epoch_stamp()
+                amqp_msg = ('Message {}@{} {}'.format(amqp_msg_counter,
+                                                      dest_unit_host,
+                                                      amqp_msg_stamp)).upper()
+                # Publish amqp message
+                u.log.debug('Publish message to: {} '
+                            '({} {})'.format(dest_unit_host,
+                                             dest_unit_name,
+                                             dest_unit_host_name))
+
+                u.publish_amqp_message_by_unit(dest_unit,
+                                               amqp_msg, ssl=ssl,
+                                               port=port)
+
+                # Wait a bit before checking for message
+                time.sleep(2)
+
+                # Get amqp message
+                u.log.debug('Get message from:   {} '
+                            '({} {})'.format(check_unit_host,
+                                             check_unit_name,
+                                             check_unit_host_name))
+
+                amqp_msg_rcvd = u.get_amqp_message_by_unit(check_unit,
+                                                           ssl=ssl,
+                                                           port=port)
+
+                # Validate amqp message content
+                if amqp_msg == amqp_msg_rcvd:
+                    u.log.debug('Message {} received '
+                                'OK.'.format(amqp_msg_counter))
+                else:
+                    u.log.error('Expected: {}'.format(amqp_msg))
+                    u.log.error('Actual:   {}'.format(amqp_msg_rcvd))
+                    msg = 'Message {} mismatch.'.format(amqp_msg_counter)
+                    amulet.raise_status(amulet.FAIL, msg)
+
+                amqp_msg_counter += 1
+
+        # Delete the test user
+        u.delete_rmq_test_user(sentry_units)
+
+    def test_100_rmq_processes(self):
+        """Verify that the expected service processes are running
+        on each rabbitmq-server unit."""
+
+        # Beam and epmd sometimes briefly have more than one PID,
+        # True checks for at least 1.
+        rmq_processes = {
+            'beam': True,
+            'epmd': True,
+        }
+
+        # Units with process names and PID quantities expected
+        expected_processes = {
+            self.rmq0_sentry: rmq_processes,
+            self.rmq1_sentry: rmq_processes,
+            self.rmq2_sentry: rmq_processes
+        }
+
+        actual_pids = u.get_unit_process_ids(expected_processes)
+        ret = u.validate_unit_process_ids(expected_processes, actual_pids)
+        if ret:
+            amulet.raise_status(amulet.FAIL, msg=ret)
+
+        u.log.info('OK\n')
+
+    def test_102_services(self):
+        """Verify that the expected services are running on the
+           corresponding service units."""
+        services = {
+            self.rmq0_sentry: ['rabbitmq-server'],
+            self.rmq1_sentry: ['rabbitmq-server'],
+            self.rmq2_sentry: ['rabbitmq-server'],
+            self.cinder_sentry: ['cinder-api',
+                                 'cinder-scheduler',
+                                 'cinder-volume'],
+        }
+        ret = u.validate_services_by_name(services)
+        if ret:
+            amulet.raise_status(amulet.FAIL, msg=ret)
+
+        u.log.info('OK\n')
+
+    def test_200_rmq_cinder_amqp_relation(self):
+        """Verify the rabbitmq-server:cinder amqp relation data"""
+        u.log.debug('Checking rmq:cinder amqp relation data...')
+        unit = self.rmq0_sentry
+        relation = ['amqp', 'cinder:amqp']
+        expected = {
+            'private-address': u.valid_ip,
+            'password': u.not_null,
+            'hostname': u.valid_ip
+        }
+        ret = u.validate_relation_data(unit, relation, expected)
+        if ret:
+            msg = u.relation_error('amqp cinder', ret)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        u.log.info('OK\n')
+
+    def test_201_cinder_rmq_amqp_relation(self):
+        """Verify the cinder:rabbitmq-server amqp relation data"""
+        u.log.debug('Checking cinder:rmq amqp relation data...')
+        unit = self.cinder_sentry
+        relation = ['amqp', 'rabbitmq-server:amqp']
+        expected = {
+            'private-address': u.valid_ip,
+            'vhost': 'openstack',
+            'username': u.not_null
+        }
+        ret = u.validate_relation_data(unit, relation, expected)
+        if ret:
+            msg = u.relation_error('cinder amqp', ret)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        u.log.info('OK\n')
+
+    def test_202_rmq_nrpe_ext_master_relation(self):
+        """Verify rabbitmq-server:nrpe nrpe-external-master relation data"""
+        u.log.debug('Checking rmq:nrpe external master relation data...')
+        unit = self.rmq0_sentry
+        relation = ['nrpe-external-master',
+                    'nrpe:nrpe-external-master']
+
+        mon_sub = ('monitors:\n  remote:\n    nrpe:\n      rabbitmq: '
+                   '{command: check_rabbitmq}\n      rabbitmq_queue: '
+                   '{command: check_rabbitmq_queue}\n')
+
+        expected = {
+            'private-address': u.valid_ip,
+            'monitors': mon_sub
+        }
+
+        ret = u.validate_relation_data(unit, relation, expected)
+        if ret:
+            msg = u.relation_error('amqp nrpe', ret)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        u.log.info('OK\n')
+
+    def test_203_nrpe_rmq_ext_master_relation(self):
+        """Verify nrpe:rabbitmq-server nrpe-external-master relation data"""
+        u.log.debug('Checking nrpe:rmq external master relation data...')
+        unit = self.nrpe_sentry
+        relation = ['nrpe-external-master',
+                    'rabbitmq-server:nrpe-external-master']
+
+        expected = {
+            'private-address': u.valid_ip
+        }
+
+        ret = u.validate_relation_data(unit, relation, expected)
+        if ret:
+            msg = u.relation_error('nrpe amqp', ret)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        u.log.info('OK\n')
+
+    def test_300_rmq_config(self):
+        """Verify the data in the rabbitmq conf file."""
+        conf = '/etc/rabbitmq/rabbitmq-env.conf'
+        sentry_units = self._get_rmq_sentry_units()
+        for unit in sentry_units:
+            host_name = unit.file_contents('/etc/hostname').strip()
+            u.log.debug('Checking rabbitmq config file data on '
+                        '{} ({})...'.format(unit.info['unit_name'],
+                                            host_name))
+            expected = {
+                'RABBITMQ_NODENAME': 'rabbit@{}'.format(host_name)
+            }
+
+            file_contents = unit.file_contents(conf)
+            u.validate_sectionless_conf(file_contents, expected)
+
+        u.log.info('OK\n')
+
+    def test_400_rmq_cluster_running_nodes(self):
+        """Verify that cluster status from each rmq juju unit shows
+        every cluster node as a running member in that cluster."""
+        u.log.debug('Checking that all units are in cluster_status '
+                    'running nodes...')
+
+        sentry_units = self._get_rmq_sentry_units()
+
+        ret = u.validate_rmq_cluster_running_nodes(sentry_units)
+        if ret:
+            amulet.raise_status(amulet.FAIL, msg=ret)
+
+        u.log.info('OK\n')
+
+    def test_402_rmq_connect_with_ssl_off(self):
+        """Verify successful non-ssl amqp connection to all units when
+        charm config option for ssl is set False."""
+        u.log.debug('Confirming that non-ssl connection succeeds when '
+                    'ssl config is off...')
+        sentry_units = self._get_rmq_sentry_units()
+        u.add_rmq_test_user(sentry_units)
+        u.configure_rmq_ssl_off(sentry_units, self.d)
+
+        # Check amqp connection for all units, expect connections to succeed
+        for unit in sentry_units:
+            connection = u.connect_amqp_by_unit(unit, ssl=False, fatal=False)
+            connection.close()
+
+        u.delete_rmq_test_user(sentry_units)
+        u.log.info('OK\n')
+
+    def test_404_rmq_ssl_connect_with_ssl_off(self):
+        """Verify unsuccessful ssl amqp connection to all units when
+        charm config option for ssl is set False."""
+        u.log.debug('Confirming that ssl connection fails when ssl '
+                    'config is off...')
+        sentry_units = self._get_rmq_sentry_units()
+        u.add_rmq_test_user(sentry_units)
+        u.configure_rmq_ssl_off(sentry_units, self.d)
+
+        # Check ssl amqp connection for all units, expect connections to fail
+        for unit in sentry_units:
+            connection = u.connect_amqp_by_unit(unit, ssl=True,
+                                                port=5971, fatal=False)
+            if connection:
+                connection.close()
+                msg = 'SSL connection unexpectedly succeeded with ssl=off'
+                amulet.raise_status(amulet.FAIL, msg)
+
+        u.delete_rmq_test_user(sentry_units)
+        u.log.info('OK - Confirmed that ssl connection attempt fails '
+                   'when ssl config is off.')
+
+    def test_406_rmq_amqp_messages_all_units_ssl_off(self):
+        """Send amqp messages to every rmq unit and check every rmq unit
+        for messages.  Standard amqp tcp port, no ssl."""
+        u.log.debug('Checking amqp message publish/get on all units '
+                    '(ssl off)...')
+
+        sentry_units = self._get_rmq_sentry_units()
+        self._test_rmq_amqp_messages_all_units(sentry_units, ssl=False)
+        u.log.info('OK\n')
+
+    def test_408_rmq_amqp_messages_all_units_ssl_on(self):
+        """Send amqp messages with ssl enabled, to every rmq unit and
+        check every rmq unit for messages.  Standard ssl tcp port."""
+        u.log.debug('Checking amqp message publish/get on all units '
+                    '(ssl on)...')
+
+        sentry_units = self._get_rmq_sentry_units()
+        self._test_rmq_amqp_messages_all_units(sentry_units,
+                                               ssl=True, port=5671)
+        u.log.info('OK\n')
+
+    def test_410_rmq_amqp_messages_all_units_ssl_alt_port(self):
+        """Send amqp messages with ssl on, to every rmq unit and check
+        every rmq unit for messages.  Custom ssl tcp port."""
+        u.log.debug('Checking amqp message publish/get on all units '
+                    '(ssl on)...')
+
+        sentry_units = self._get_rmq_sentry_units()
+        self._test_rmq_amqp_messages_all_units(sentry_units,
+                                               ssl=True, port=5999)
+        u.log.info('OK\n')
+
+    def test_412_rmq_management_plugin(self):
+        """Enable and check management plugin."""
+        u.log.debug('Checking tcp socket connect to management plugin '
+                    'port on all rmq units...')
+
+        sentry_units = self._get_rmq_sentry_units()
+        mgmt_port = 15672
+
+        # Enable management plugin
+        u.log.debug('Enabling management_plugin charm config option...')
+        config = {'management_plugin': 'True'}
+        self.d.configure('rabbitmq-server', config)
+
+        # Check tcp connect to management plugin port
+        max_wait = 120
+        tries = 0
+        ret = u.port_knock_units(sentry_units, mgmt_port)
+        while ret and tries < (max_wait / 12):
+            time.sleep(12)
+            u.log.debug('Attempt {}: {}'.format(tries, ret))
+            ret = u.port_knock_units(sentry_units, mgmt_port)
+            tries += 1
+
+        if ret:
+            amulet.raise_status(amulet.FAIL, ret)
+        else:
+            u.log.debug('Connect to all units (OK)\n')
+
+        # Disable management plugin
+        u.log.debug('Disabling management_plugin charm config option...')
+        config = {'management_plugin': 'False'}
+        self.d.configure('rabbitmq-server', config)
+
+        # Negative check - tcp connect to management plugin port
+        u.log.info('Expect tcp connect fail since charm config '
+                   'option is disabled.')
+        tries = 0
+        ret = u.port_knock_units(sentry_units, mgmt_port, expect_success=False)
+        while ret and tries < (max_wait / 12):
+            time.sleep(12)
+            u.log.debug('Attempt {}: {}'.format(tries, ret))
+            ret = u.port_knock_units(sentry_units, mgmt_port,
+                                     expect_success=False)
+            tries += 1
+
+        if ret:
+            amulet.raise_status(amulet.FAIL, ret)
+        else:
+            u.log.info('Confirm mgmt port closed on all units (OK)\n')
+
+    def test_414_rmq_nrpe_monitors(self):
+        """Check rabbimq-server nrpe monitor basic functionality."""
+        sentry_units = self._get_rmq_sentry_units()
+        host_names = u.get_unit_hostnames(sentry_units)
+
+        # check_rabbitmq monitor
+        u.log.debug('Checking nrpe check_rabbitmq on units...')
+        cmds = ['egrep -oh /usr/local.* /etc/nagios/nrpe.d/'
+                'check_rabbitmq.cfg']
+        ret = u.check_commands_on_units(cmds, sentry_units)
+        if ret:
+            amulet.raise_status(amulet.FAIL, msg=ret)
+
+        u.log.debug('Sleeping 70s for 1m cron job to run...')
+        time.sleep(70)
+
+        # check_rabbitmq_queue monitor
+        u.log.debug('Checking nrpe check_rabbitmq_queue on units...')
+        cmds = ['egrep -oh /usr/local.* /etc/nagios/nrpe.d/'
+                'check_rabbitmq_queue.cfg']
+        ret = u.check_commands_on_units(cmds, sentry_units)
+        if ret:
+            amulet.raise_status(amulet.FAIL, msg=ret)
+
+        # check dat file existence
+        u.log.debug('Checking nrpe dat file existence on units...')
+        for sentry_unit in sentry_units:
+            unit_name = sentry_unit.info['unit_name']
+            unit_host_name = host_names[unit_name]
+
+            cmds = [
+                'stat /var/lib/rabbitmq/data/{}_general_stats.dat'.format(
+                    unit_host_name),
+                'stat /var/lib/rabbitmq/data/{}_queue_stats.dat'.format(
+                    unit_host_name)
+            ]
+
+            ret = u.check_commands_on_units(cmds, [sentry_unit])
+            if ret:
+                amulet.raise_status(amulet.FAIL, msg=ret)
+
+        u.log.info('OK\n')

=== added file 'tests/charmhelpers/__init__.py'
--- tests/charmhelpers/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/__init__.py	2015-09-08 14:01:41 +0000
@@ -0,0 +1,38 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+# Bootstrap charm-helpers, installing its dependencies if necessary using
+# only standard libraries.
+import subprocess
+import sys
+
+try:
+    import six  # flake8: noqa
+except ImportError:
+    if sys.version_info.major == 2:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python-six'])
+    else:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python3-six'])
+    import six  # flake8: noqa
+
+try:
+    import yaml  # flake8: noqa
+except ImportError:
+    if sys.version_info.major == 2:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml'])
+    else:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml'])
+    import yaml  # flake8: noqa

=== removed file 'tests/charmhelpers/__init__.py'
--- tests/charmhelpers/__init__.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,38 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-# Bootstrap charm-helpers, installing its dependencies if necessary using
-# only standard libraries.
-import subprocess
-import sys
-
-try:
-    import six  # flake8: noqa
-except ImportError:
-    if sys.version_info.major == 2:
-        subprocess.check_call(['apt-get', 'install', '-y', 'python-six'])
-    else:
-        subprocess.check_call(['apt-get', 'install', '-y', 'python3-six'])
-    import six  # flake8: noqa
-
-try:
-    import yaml  # flake8: noqa
-except ImportError:
-    if sys.version_info.major == 2:
-        subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml'])
-    else:
-        subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml'])
-    import yaml  # flake8: noqa

=== removed directory 'tests/charmhelpers/cli'
=== removed file 'tests/charmhelpers/cli/__init__.py'
--- tests/charmhelpers/cli/__init__.py	2015-08-19 13:49:53 +0000
+++ tests/charmhelpers/cli/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,191 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import inspect
-import argparse
-import sys
-
-from six.moves import zip
-
-from charmhelpers.core import unitdata
-
-
-class OutputFormatter(object):
-    def __init__(self, outfile=sys.stdout):
-        self.formats = (
-            "raw",
-            "json",
-            "py",
-            "yaml",
-            "csv",
-            "tab",
-        )
-        self.outfile = outfile
-
-    def add_arguments(self, argument_parser):
-        formatgroup = argument_parser.add_mutually_exclusive_group()
-        choices = self.supported_formats
-        formatgroup.add_argument("--format", metavar='FMT',
-                                 help="Select output format for returned data, "
-                                      "where FMT is one of: {}".format(choices),
-                                 choices=choices, default='raw')
-        for fmt in self.formats:
-            fmtfunc = getattr(self, fmt)
-            formatgroup.add_argument("-{}".format(fmt[0]),
-                                     "--{}".format(fmt), action='store_const',
-                                     const=fmt, dest='format',
-                                     help=fmtfunc.__doc__)
-
-    @property
-    def supported_formats(self):
-        return self.formats
-
-    def raw(self, output):
-        """Output data as raw string (default)"""
-        if isinstance(output, (list, tuple)):
-            output = '\n'.join(map(str, output))
-        self.outfile.write(str(output))
-
-    def py(self, output):
-        """Output data as a nicely-formatted python data structure"""
-        import pprint
-        pprint.pprint(output, stream=self.outfile)
-
-    def json(self, output):
-        """Output data in JSON format"""
-        import json
-        json.dump(output, self.outfile)
-
-    def yaml(self, output):
-        """Output data in YAML format"""
-        import yaml
-        yaml.safe_dump(output, self.outfile)
-
-    def csv(self, output):
-        """Output data as excel-compatible CSV"""
-        import csv
-        csvwriter = csv.writer(self.outfile)
-        csvwriter.writerows(output)
-
-    def tab(self, output):
-        """Output data in excel-compatible tab-delimited format"""
-        import csv
-        csvwriter = csv.writer(self.outfile, dialect=csv.excel_tab)
-        csvwriter.writerows(output)
-
-    def format_output(self, output, fmt='raw'):
-        fmtfunc = getattr(self, fmt)
-        fmtfunc(output)
-
-
-class CommandLine(object):
-    argument_parser = None
-    subparsers = None
-    formatter = None
-    exit_code = 0
-
-    def __init__(self):
-        if not self.argument_parser:
-            self.argument_parser = argparse.ArgumentParser(description='Perform common charm tasks')
-        if not self.formatter:
-            self.formatter = OutputFormatter()
-            self.formatter.add_arguments(self.argument_parser)
-        if not self.subparsers:
-            self.subparsers = self.argument_parser.add_subparsers(help='Commands')
-
-    def subcommand(self, command_name=None):
-        """
-        Decorate a function as a subcommand. Use its arguments as the
-        command-line arguments"""
-        def wrapper(decorated):
-            cmd_name = command_name or decorated.__name__
-            subparser = self.subparsers.add_parser(cmd_name,
-                                                   description=decorated.__doc__)
-            for args, kwargs in describe_arguments(decorated):
-                subparser.add_argument(*args, **kwargs)
-            subparser.set_defaults(func=decorated)
-            return decorated
-        return wrapper
-
-    def test_command(self, decorated):
-        """
-        Subcommand is a boolean test function, so bool return values should be
-        converted to a 0/1 exit code.
-        """
-        decorated._cli_test_command = True
-        return decorated
-
-    def no_output(self, decorated):
-        """
-        Subcommand is not expected to return a value, so don't print a spurious None.
-        """
-        decorated._cli_no_output = True
-        return decorated
-
-    def subcommand_builder(self, command_name, description=None):
-        """
-        Decorate a function that builds a subcommand. Builders should accept a
-        single argument (the subparser instance) and return the function to be
-        run as the command."""
-        def wrapper(decorated):
-            subparser = self.subparsers.add_parser(command_name)
-            func = decorated(subparser)
-            subparser.set_defaults(func=func)
-            subparser.description = description or func.__doc__
-        return wrapper
-
-    def run(self):
-        "Run cli, processing arguments and executing subcommands."
-        arguments = self.argument_parser.parse_args()
-        argspec = inspect.getargspec(arguments.func)
-        vargs = []
-        for arg in argspec.args:
-            vargs.append(getattr(arguments, arg))
-        if argspec.varargs:
-            vargs.extend(getattr(arguments, argspec.varargs))
-        output = arguments.func(*vargs)
-        if getattr(arguments.func, '_cli_test_command', False):
-            self.exit_code = 0 if output else 1
-            output = ''
-        if getattr(arguments.func, '_cli_no_output', False):
-            output = ''
-        self.formatter.format_output(output, arguments.format)
-        if unitdata._KV:
-            unitdata._KV.flush()
-
-
-cmdline = CommandLine()
-
-
-def describe_arguments(func):
-    """
-    Analyze a function's signature and return a data structure suitable for
-    passing in as arguments to an argparse parser's add_argument() method."""
-
-    argspec = inspect.getargspec(func)
-    # we should probably raise an exception somewhere if func includes **kwargs
-    if argspec.defaults:
-        positional_args = argspec.args[:-len(argspec.defaults)]
-        keyword_names = argspec.args[-len(argspec.defaults):]
-        for arg, default in zip(keyword_names, argspec.defaults):
-            yield ('--{}'.format(arg),), {'default': default}
-    else:
-        positional_args = argspec.args
-
-    for arg in positional_args:
-        yield (arg,), {}
-    if argspec.varargs:
-        yield (argspec.varargs,), {'nargs': '*'}

=== removed file 'tests/charmhelpers/cli/benchmark.py'
--- tests/charmhelpers/cli/benchmark.py	2015-07-31 22:00:49 +0000
+++ tests/charmhelpers/cli/benchmark.py	1970-01-01 00:00:00 +0000
@@ -1,36 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from . import cmdline
-from charmhelpers.contrib.benchmark import Benchmark
-
-
[e-mail](command_name='benchmark-start')
-def start():
-    Benchmark.start()
-
-
[e-mail](command_name='benchmark-finish')
-def finish():
-    Benchmark.finish()
-
-
[e-mail]_builder('benchmark-composite', description="Set the benchmark composite score")
-def service(subparser):
-    subparser.add_argument("value", help="The composite score.")
-    subparser.add_argument("units", help="The units the composite score represents, i.e., 'reads/sec'.")
-    subparser.add_argument("direction", help="'asc' if a lower score is better, 'desc' if a higher score is better.")
-    return Benchmark.set_composite_score

=== removed file 'tests/charmhelpers/cli/commands.py'
--- tests/charmhelpers/cli/commands.py	2015-08-19 13:49:53 +0000
+++ tests/charmhelpers/cli/commands.py	1970-01-01 00:00:00 +0000
@@ -1,32 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-"""
-This module loads sub-modules into the python runtime so they can be
-discovered via the inspect module. In order to prevent flake8 from (rightfully)
-telling us these are unused modules, throw a ' # noqa' at the end of each import
-so that the warning is suppressed.
-"""
-
-from . import CommandLine  # noqa
-
-"""
-Import the sub-modules which have decorated subcommands to register with chlp.
-"""
-from . import host  # noqa
-from . import benchmark  # noqa
-from . import unitdata  # noqa
-from . import hookenv  # noqa

=== removed file 'tests/charmhelpers/cli/hookenv.py'
--- tests/charmhelpers/cli/hookenv.py	2015-08-19 13:49:53 +0000
+++ tests/charmhelpers/cli/hookenv.py	1970-01-01 00:00:00 +0000
@@ -1,23 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from . import cmdline
-from charmhelpers.core import hookenv
-
-
-cmdline.subcommand('relation-id')(hookenv.relation_id._wrapped)
-cmdline.subcommand('service-name')(hookenv.service_name)
-cmdline.subcommand('remote-service-name')(hookenv.remote_service_name._wrapped)

=== removed file 'tests/charmhelpers/cli/host.py'
--- tests/charmhelpers/cli/host.py	2015-07-31 22:00:49 +0000
+++ tests/charmhelpers/cli/host.py	1970-01-01 00:00:00 +0000
@@ -1,31 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from . import cmdline
-from charmhelpers.core import host
-
-
[e-mail]()
-def mounts():
-    "List mounts"
-    return host.mounts()
-
-
[e-mail]_builder('service', description="Control system services")
-def service(subparser):
-    subparser.add_argument("action", help="The action to perform (start, stop, etc...)")
-    subparser.add_argument("service_name", help="Name of the service to control")
-    return host.service

=== removed file 'tests/charmhelpers/cli/unitdata.py'
--- tests/charmhelpers/cli/unitdata.py	2015-07-31 22:00:49 +0000
+++ tests/charmhelpers/cli/unitdata.py	1970-01-01 00:00:00 +0000
@@ -1,39 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from . import cmdline
-from charmhelpers.core import unitdata
-
-
[e-mail]_builder('unitdata', description="Store and retrieve data")
-def unitdata_cmd(subparser):
-    nested = subparser.add_subparsers()
-    get_cmd = nested.add_parser('get', help='Retrieve data')
-    get_cmd.add_argument('key', help='Key to retrieve the value of')
-    get_cmd.set_defaults(action='get', value=None)
-    set_cmd = nested.add_parser('set', help='Store data')
-    set_cmd.add_argument('key', help='Key to set')
-    set_cmd.add_argument('value', help='Value to store')
-    set_cmd.set_defaults(action='set')
-
-    def _unitdata_cmd(action, key, value):
-        if action == 'get':
-            return unitdata.kv().get(key)
-        elif action == 'set':
-            unitdata.kv().set(key, value)
-            unitdata.kv().flush()
-            return ''
-    return _unitdata_cmd

=== added directory 'tests/charmhelpers/contrib'
=== removed directory 'tests/charmhelpers/contrib'
=== added file 'tests/charmhelpers/contrib/__init__.py'
--- tests/charmhelpers/contrib/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/__init__.py	2015-09-08 14:01:41 +0000
@@ -0,0 +1,15 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== removed file 'tests/charmhelpers/contrib/__init__.py'
--- tests/charmhelpers/contrib/__init__.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/contrib/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,15 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== added directory 'tests/charmhelpers/contrib/amulet'
=== added file 'tests/charmhelpers/contrib/amulet/__init__.py'
--- tests/charmhelpers/contrib/amulet/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/amulet/__init__.py	2015-09-08 14:01:41 +0000
@@ -0,0 +1,15 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== added file 'tests/charmhelpers/contrib/amulet/deployment.py'
--- tests/charmhelpers/contrib/amulet/deployment.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/amulet/deployment.py	2015-09-08 14:01:41 +0000
@@ -0,0 +1,93 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import amulet
+import os
+import six
+
+
+class AmuletDeployment(object):
+    """Amulet deployment.
+
+       This class provides generic Amulet deployment and test runner
+       methods.
+       """
+
+    def __init__(self, series=None):
+        """Initialize the deployment environment."""
+        self.series = None
+
+        if series:
+            self.series = series
+            self.d = amulet.Deployment(series=self.series)
+        else:
+            self.d = amulet.Deployment()
+
+    def _add_services(self, this_service, other_services):
+        """Add services.
+
+           Add services to the deployment where this_service is the local charm
+           that we're testing and other_services are the other services that
+           are being used in the local amulet tests.
+           """
+        if this_service['name'] != os.path.basename(os.getcwd()):
+            s = this_service['name']
+            msg = "The charm's root directory name needs to be {}".format(s)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        if 'units' not in this_service:
+            this_service['units'] = 1
+
+        self.d.add(this_service['name'], units=this_service['units'])
+
+        for svc in other_services:
+            if 'location' in svc:
+                branch_location = svc['location']
+            elif self.series:
+                branch_location = 'cs:{}/{}'.format(self.series, svc['name']),
+            else:
+                branch_location = None
+
+            if 'units' not in svc:
+                svc['units'] = 1
+
+            self.d.add(svc['name'], charm=branch_location, units=svc['units'])
+
+    def _add_relations(self, relations):
+        """Add all of the relations for the services."""
+        for k, v in six.iteritems(relations):
+            self.d.relate(k, v)
+
+    def _configure_services(self, configs):
+        """Configure all of the services."""
+        for service, config in six.iteritems(configs):
+            self.d.configure(service, config)
+
+    def _deploy(self):
+        """Deploy environment and wait for all hooks to finish executing."""
+        try:
+            self.d.setup(timeout=900)
+            self.d.sentry.wait(timeout=900)
+        except amulet.helpers.TimeoutError:
+            amulet.raise_status(amulet.FAIL, msg="Deployment timed out")
+        except Exception:
+            raise
+
+    def run_tests(self):
+        """Run all of the methods that are prefixed with 'test_'."""
+        for test in dir(self):
+            if test.startswith('test_'):
+                getattr(self, test)()

=== added file 'tests/charmhelpers/contrib/amulet/utils.py'
--- tests/charmhelpers/contrib/amulet/utils.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/amulet/utils.py	2015-09-08 14:01:41 +0000
@@ -0,0 +1,778 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import io
+import json
+import logging
+import os
+import re
+import socket
+import subprocess
+import sys
+import time
+import uuid
+
+import amulet
+import distro_info
+import six
+from six.moves import configparser
+if six.PY3:
+    from urllib import parse as urlparse
+else:
+    import urlparse
+
+
+class AmuletUtils(object):
+    """Amulet utilities.
+
+       This class provides common utility functions that are used by Amulet
+       tests.
+       """
+
+    def __init__(self, log_level=logging.ERROR):
+        self.log = self.get_logger(level=log_level)
+        self.ubuntu_releases = self.get_ubuntu_releases()
+
+    def get_logger(self, name="amulet-logger", level=logging.DEBUG):
+        """Get a logger object that will log to stdout."""
+        log = logging
+        logger = log.getLogger(name)
+        fmt = log.Formatter("%(asctime)s %(funcName)s "
+                            "%(levelname)s: %(message)s")
+
+        handler = log.StreamHandler(stream=sys.stdout)
+        handler.setLevel(level)
+        handler.setFormatter(fmt)
+
+        logger.addHandler(handler)
+        logger.setLevel(level)
+
+        return logger
+
+    def valid_ip(self, ip):
+        if re.match(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$", ip):
+            return True
+        else:
+            return False
+
+    def valid_url(self, url):
+        p = re.compile(
+            r'^(?:http|ftp)s?://'
+            r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}\.?)|'  # noqa
+            r'localhost|'
+            r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'
+            r'(?::\d+)?'
+            r'(?:/?|[/?]\S+)$',
+            re.IGNORECASE)
+        if p.match(url):
+            return True
+        else:
+            return False
+
+    def get_ubuntu_release_from_sentry(self, sentry_unit):
+        """Get Ubuntu release codename from sentry unit.
+
+        :param sentry_unit: amulet sentry/service unit pointer
+        :returns: list of strings - release codename, failure message
+        """
+        msg = None
+        cmd = 'lsb_release -cs'
+        release, code = sentry_unit.run(cmd)
+        if code == 0:
+            self.log.debug('{} lsb_release: {}'.format(
+                sentry_unit.info['unit_name'], release))
+        else:
+            msg = ('{} `{}` returned {} '
+                   '{}'.format(sentry_unit.info['unit_name'],
+                               cmd, release, code))
+        if release not in self.ubuntu_releases:
+            msg = ("Release ({}) not found in Ubuntu releases "
+                   "({})".format(release, self.ubuntu_releases))
+        return release, msg
+
+    def validate_services(self, commands):
+        """Validate that lists of commands succeed on service units.  Can be
+           used to verify system services are running on the corresponding
+           service units.
+
+        :param commands: dict with sentry keys and arbitrary command list vals
+        :returns: None if successful, Failure string message otherwise
+        """
+        self.log.debug('Checking status of system services...')
+
+        # /!\ DEPRECATION WARNING (beisner):
+        # New and existing tests should be rewritten to use
+        # validate_services_by_name() as it is aware of init systems.
+        self.log.warn('DEPRECATION WARNING:  use '
+                      'validate_services_by_name instead of validate_services '
+                      'due to init system differences.')
+
+        for k, v in six.iteritems(commands):
+            for cmd in v:
+                output, code = k.run(cmd)
+                self.log.debug('{} `{}` returned '
+                               '{}'.format(k.info['unit_name'],
+                                           cmd, code))
+                if code != 0:
+                    return "command `{}` returned {}".format(cmd, str(code))
+        return None
+
+    def validate_services_by_name(self, sentry_services):
+        """Validate system service status by service name, automatically
+           detecting init system based on Ubuntu release codename.
+
+        :param sentry_services: dict with sentry keys and svc list values
+        :returns: None if successful, Failure string message otherwise
+        """
+        self.log.debug('Checking status of system services...')
+
+        # Point at which systemd became a thing
+        systemd_switch = self.ubuntu_releases.index('vivid')
+
+        for sentry_unit, services_list in six.iteritems(sentry_services):
+            # Get lsb_release codename from unit
+            release, ret = self.get_ubuntu_release_from_sentry(sentry_unit)
+            if ret:
+                return ret
+
+            for service_name in services_list:
+                if (self.ubuntu_releases.index(release) >= systemd_switch or
+                        service_name in ['rabbitmq-server', 'apache2']):
+                    # init is systemd (or regular sysv)
+                    cmd = 'sudo service {} status'.format(service_name)
+                    output, code = sentry_unit.run(cmd)
+                    service_running = code == 0
+                elif self.ubuntu_releases.index(release) < systemd_switch:
+                    # init is upstart
+                    cmd = 'sudo status {}'.format(service_name)
+                    output, code = sentry_unit.run(cmd)
+                    service_running = code == 0 and "start/running" in output
+
+                self.log.debug('{} `{}` returned '
+                               '{}'.format(sentry_unit.info['unit_name'],
+                                           cmd, code))
+                if not service_running:
+                    return u"command `{}` returned {} {}".format(
+                        cmd, output, str(code))
+        return None
+
+    def _get_config(self, unit, filename):
+        """Get a ConfigParser object for parsing a unit's config file."""
+        file_contents = unit.file_contents(filename)
+
+        # NOTE(beisner):  by default, ConfigParser does not handle options
+        # with no value, such as the flags used in the mysql my.cnf file.
+        # https://bugs.python.org/issue7005
+        config = configparser.ConfigParser(allow_no_value=True)
+        config.readfp(io.StringIO(file_contents))
+        return config
+
+    def validate_config_data(self, sentry_unit, config_file, section,
+                             expected):
+        """Validate config file data.
+
+           Verify that the specified section of the config file contains
+           the expected option key:value pairs.
+
+           Compare expected dictionary data vs actual dictionary data.
+           The values in the 'expected' dictionary can be strings, bools, ints,
+           longs, or can be a function that evaluates a variable and returns a
+           bool.
+           """
+        self.log.debug('Validating config file data ({} in {} on {})'
+                       '...'.format(section, config_file,
+                                    sentry_unit.info['unit_name']))
+        config = self._get_config(sentry_unit, config_file)
+
+        if section != 'DEFAULT' and not config.has_section(section):
+            return "section [{}] does not exist".format(section)
+
+        for k in expected.keys():
+            if not config.has_option(section, k):
+                return "section [{}] is missing option {}".format(section, k)
+
+            actual = config.get(section, k)
+            v = expected[k]
+            if (isinstance(v, six.string_types) or
+                    isinstance(v, bool) or
+                    isinstance(v, six.integer_types)):
+                # handle explicit values
+                if actual != v:
+                    return "section [{}] {}:{} != expected {}:{}".format(
+                           section, k, actual, k, expected[k])
+            # handle function pointers, such as not_null or valid_ip
+            elif not v(actual):
+                return "section [{}] {}:{} != expected {}:{}".format(
+                       section, k, actual, k, expected[k])
+        return None
+
+    def _validate_dict_data(self, expected, actual):
+        """Validate dictionary data.
+
+           Compare expected dictionary data vs actual dictionary data.
+           The values in the 'expected' dictionary can be strings, bools, ints,
+           longs, or can be a function that evaluates a variable and returns a
+           bool.
+           """
+        self.log.debug('actual: {}'.format(repr(actual)))
+        self.log.debug('expected: {}'.format(repr(expected)))
+
+        for k, v in six.iteritems(expected):
+            if k in actual:
+                if (isinstance(v, six.string_types) or
+                        isinstance(v, bool) or
+                        isinstance(v, six.integer_types)):
+                    # handle explicit values
+                    if v != actual[k]:
+                        return "{}:{}".format(k, actual[k])
+                # handle function pointers, such as not_null or valid_ip
+                elif not v(actual[k]):
+                    return "{}:{}".format(k, actual[k])
+            else:
+                return "key '{}' does not exist".format(k)
+        return None
+
+    def validate_relation_data(self, sentry_unit, relation, expected):
+        """Validate actual relation data based on expected relation data."""
+        actual = sentry_unit.relation(relation[0], relation[1])
+        return self._validate_dict_data(expected, actual)
+
+    def _validate_list_data(self, expected, actual):
+        """Compare expected list vs actual list data."""
+        for e in expected:
+            if e not in actual:
+                return "expected item {} not found in actual list".format(e)
+        return None
+
+    def not_null(self, string):
+        if string is not None:
+            return True
+        else:
+            return False
+
+    def _get_file_mtime(self, sentry_unit, filename):
+        """Get last modification time of file."""
+        return sentry_unit.file_stat(filename)['mtime']
+
+    def _get_dir_mtime(self, sentry_unit, directory):
+        """Get last modification time of directory."""
+        return sentry_unit.directory_stat(directory)['mtime']
+
+    def _get_proc_start_time(self, sentry_unit, service, pgrep_full=None):
+        """Get start time of a process based on the last modification time
+           of the /proc/pid directory.
+
+        :sentry_unit:  The sentry unit to check for the service on
+        :service:  service name to look for in process table
+        :pgrep_full:  [Deprecated] Use full command line search mode with pgrep
+        :returns:  epoch time of service process start
+        :param commands:  list of bash commands
+        :param sentry_units:  list of sentry unit pointers
+        :returns:  None if successful; Failure message otherwise
+        """
+        if pgrep_full is not None:
+            # /!\ DEPRECATION WARNING (beisner):
+            # No longer implemented, as pidof is now used instead of pgrep.
+            # https://bugs.launchpad.net/charm-helpers/+bug/1474030
+            self.log.warn('DEPRECATION WARNING:  pgrep_full bool is no '
+                          'longer implemented re: lp 1474030.')
+
+        pid_list = self.get_process_id_list(sentry_unit, service)
+        pid = pid_list[0]
+        proc_dir = '/proc/{}'.format(pid)
+        self.log.debug('Pid for {} on {}: {}'.format(
+            service, sentry_unit.info['unit_name'], pid))
+
+        return self._get_dir_mtime(sentry_unit, proc_dir)
+
+    def service_restarted(self, sentry_unit, service, filename,
+                          pgrep_full=None, sleep_time=20):
+        """Check if service was restarted.
+
+           Compare a service's start time vs a file's last modification time
+           (such as a config file for that service) to determine if the service
+           has been restarted.
+           """
+        # /!\ DEPRECATION WARNING (beisner):
+        # This method is prone to races in that no before-time is known.
+        # Use validate_service_config_changed instead.
+
+        # NOTE(beisner) pgrep_full is no longer implemented, as pidof is now
+        # used instead of pgrep.  pgrep_full is still passed through to ensure
+        # deprecation WARNS.  lp1474030
+        self.log.warn('DEPRECATION WARNING:  use '
+                      'validate_service_config_changed instead of '
+                      'service_restarted due to known races.')
+
+        time.sleep(sleep_time)
+        if (self._get_proc_start_time(sentry_unit, service, pgrep_full) >=
+                self._get_file_mtime(sentry_unit, filename)):
+            return True
+        else:
+            return False
+
+    def service_restarted_since(self, sentry_unit, mtime, service,
+                                pgrep_full=None, sleep_time=20,
+                                retry_count=2, retry_sleep_time=30):
+        """Check if service was been started after a given time.
+
+        Args:
+          sentry_unit (sentry): The sentry unit to check for the service on
+          mtime (float): The epoch time to check against
+          service (string): service name to look for in process table
+          pgrep_full: [Deprecated] Use full command line search mode with pgrep
+          sleep_time (int): Seconds to sleep before looking for process
+          retry_count (int): If service is not found, how many times to retry
+
+        Returns:
+          bool: True if service found and its start time it newer than mtime,
+                False if service is older than mtime or if service was
+                not found.
+        """
+        # NOTE(beisner) pgrep_full is no longer implemented, as pidof is now
+        # used instead of pgrep.  pgrep_full is still passed through to ensure
+        # deprecation WARNS.  lp1474030
+
+        unit_name = sentry_unit.info['unit_name']
+        self.log.debug('Checking that %s service restarted since %s on '
+                       '%s' % (service, mtime, unit_name))
+        time.sleep(sleep_time)
+        proc_start_time = None
+        tries = 0
+        while tries <= retry_count and not proc_start_time:
+            try:
+                proc_start_time = self._get_proc_start_time(sentry_unit,
+                                                            service,
+                                                            pgrep_full)
+                self.log.debug('Attempt {} to get {} proc start time on {} '
+                               'OK'.format(tries, service, unit_name))
+            except IOError:
+                # NOTE(beisner) - race avoidance, proc may not exist yet.
+                # https://bugs.launchpad.net/charm-helpers/+bug/1474030
+                self.log.debug('Attempt {} to get {} proc start time on {} '
+                               'failed'.format(tries, service, unit_name))
+                time.sleep(retry_sleep_time)
+                tries += 1
+
+        if not proc_start_time:
+            self.log.warn('No proc start time found, assuming service did '
+                          'not start')
+            return False
+        if proc_start_time >= mtime:
+            self.log.debug('Proc start time is newer than provided mtime'
+                           '(%s >= %s) on %s (OK)' % (proc_start_time,
+                                                      mtime, unit_name))
+            return True
+        else:
+            self.log.warn('Proc start time (%s) is older than provided mtime '
+                          '(%s) on %s, service did not '
+                          'restart' % (proc_start_time, mtime, unit_name))
+            return False
+
+    def config_updated_since(self, sentry_unit, filename, mtime,
+                             sleep_time=20):
+        """Check if file was modified after a given time.
+
+        Args:
+          sentry_unit (sentry): The sentry unit to check the file mtime on
+          filename (string): The file to check mtime of
+          mtime (float): The epoch time to check against
+          sleep_time (int): Seconds to sleep before looking for process
+
+        Returns:
+          bool: True if file was modified more recently than mtime, False if
+                file was modified before mtime,
+        """
+        self.log.debug('Checking %s updated since %s' % (filename, mtime))
+        time.sleep(sleep_time)
+        file_mtime = self._get_file_mtime(sentry_unit, filename)
+        if file_mtime >= mtime:
+            self.log.debug('File mtime is newer than provided mtime '
+                           '(%s >= %s)' % (file_mtime, mtime))
+            return True
+        else:
+            self.log.warn('File mtime %s is older than provided mtime %s'
+                          % (file_mtime, mtime))
+            return False
+
+    def validate_service_config_changed(self, sentry_unit, mtime, service,
+                                        filename, pgrep_full=None,
+                                        sleep_time=20, retry_count=2,
+                                        retry_sleep_time=30):
+        """Check service and file were updated after mtime
+
+        Args:
+          sentry_unit (sentry): The sentry unit to check for the service on
+          mtime (float): The epoch time to check against
+          service (string): service name to look for in process table
+          filename (string): The file to check mtime of
+          pgrep_full: [Deprecated] Use full command line search mode with pgrep
+          sleep_time (int): Initial sleep in seconds to pass to test helpers
+          retry_count (int): If service is not found, how many times to retry
+          retry_sleep_time (int): Time in seconds to wait between retries
+
+        Typical Usage:
+            u = OpenStackAmuletUtils(ERROR)
+            ...
+            mtime = u.get_sentry_time(self.cinder_sentry)
+            self.d.configure('cinder', {'verbose': 'True', 'debug': 'True'})
+            if not u.validate_service_config_changed(self.cinder_sentry,
+                                                     mtime,
+                                                     'cinder-api',
+                                                     '/etc/cinder/cinder.conf')
+                amulet.raise_status(amulet.FAIL, msg='update failed')
+        Returns:
+          bool: True if both service and file where updated/restarted after
+                mtime, False if service is older than mtime or if service was
+                not found or if filename was modified before mtime.
+        """
+
+        # NOTE(beisner) pgrep_full is no longer implemented, as pidof is now
+        # used instead of pgrep.  pgrep_full is still passed through to ensure
+        # deprecation WARNS.  lp1474030
+
+        service_restart = self.service_restarted_since(
+            sentry_unit, mtime,
+            service,
+            pgrep_full=pgrep_full,
+            sleep_time=sleep_time,
+            retry_count=retry_count,
+            retry_sleep_time=retry_sleep_time)
+
+        config_update = self.config_updated_since(
+            sentry_unit,
+            filename,
+            mtime,
+            sleep_time=0)
+
+        return service_restart and config_update
+
+    def get_sentry_time(self, sentry_unit):
+        """Return current epoch time on a sentry"""
+        cmd = "date +'%s'"
+        return float(sentry_unit.run(cmd)[0])
+
+    def relation_error(self, name, data):
+        return 'unexpected relation data in {} - {}'.format(name, data)
+
+    def endpoint_error(self, name, data):
+        return 'unexpected endpoint data in {} - {}'.format(name, data)
+
+    def get_ubuntu_releases(self):
+        """Return a list of all Ubuntu releases in order of release."""
+        _d = distro_info.UbuntuDistroInfo()
+        _release_list = _d.all
+        return _release_list
+
+    def file_to_url(self, file_rel_path):
+        """Convert a relative file path to a file URL."""
+        _abs_path = os.path.abspath(file_rel_path)
+        return urlparse.urlparse(_abs_path, scheme='file').geturl()
+
+    def check_commands_on_units(self, commands, sentry_units):
+        """Check that all commands in a list exit zero on all
+        sentry units in a list.
+
+        :param commands:  list of bash commands
+        :param sentry_units:  list of sentry unit pointers
+        :returns: None if successful; Failure message otherwise
+        """
+        self.log.debug('Checking exit codes for {} commands on {} '
+                       'sentry units...'.format(len(commands),
+                                                len(sentry_units)))
+        for sentry_unit in sentry_units:
+            for cmd in commands:
+                output, code = sentry_unit.run(cmd)
+                if code == 0:
+                    self.log.debug('{} `{}` returned {} '
+                                   '(OK)'.format(sentry_unit.info['unit_name'],
+                                                 cmd, code))
+                else:
+                    return ('{} `{}` returned {} '
+                            '{}'.format(sentry_unit.info['unit_name'],
+                                        cmd, code, output))
+        return None
+
+    def get_process_id_list(self, sentry_unit, process_name,
+                            expect_success=True):
+        """Get a list of process ID(s) from a single sentry juju unit
+        for a single process name.
+
+        :param sentry_unit: Amulet sentry instance (juju unit)
+        :param process_name: Process name
+        :param expect_success: If False, expect the PID to be missing,
+            raise if it is present.
+        :returns: List of process IDs
+        """
+        cmd = 'pidof -x {}'.format(process_name)
+        if not expect_success:
+            cmd += " || exit 0 && exit 1"
+        output, code = sentry_unit.run(cmd)
+        if code != 0:
+            msg = ('{} `{}` returned {} '
+                   '{}'.format(sentry_unit.info['unit_name'],
+                               cmd, code, output))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+        return str(output).split()
+
+    def get_unit_process_ids(self, unit_processes, expect_success=True):
+        """Construct a dict containing unit sentries, process names, and
+        process IDs.
+
+        :param unit_processes: A dictionary of Amulet sentry instance
+            to list of process names.
+        :param expect_success: if False expect the processes to not be
+            running, raise if they are.
+        :returns: Dictionary of Amulet sentry instance to dictionary
+            of process names to PIDs.
+        """
+        pid_dict = {}
+        for sentry_unit, process_list in six.iteritems(unit_processes):
+            pid_dict[sentry_unit] = {}
+            for process in process_list:
+                pids = self.get_process_id_list(
+                    sentry_unit, process, expect_success=expect_success)
+                pid_dict[sentry_unit].update({process: pids})
+        return pid_dict
+
+    def validate_unit_process_ids(self, expected, actual):
+        """Validate process id quantities for services on units."""
+        self.log.debug('Checking units for running processes...')
+        self.log.debug('Expected PIDs: {}'.format(expected))
+        self.log.debug('Actual PIDs: {}'.format(actual))
+
+        if len(actual) != len(expected):
+            return ('Unit count mismatch.  expected, actual: {}, '
+                    '{} '.format(len(expected), len(actual)))
+
+        for (e_sentry, e_proc_names) in six.iteritems(expected):
+            e_sentry_name = e_sentry.info['unit_name']
+            if e_sentry in actual.keys():
+                a_proc_names = actual[e_sentry]
+            else:
+                return ('Expected sentry ({}) not found in actual dict data.'
+                        '{}'.format(e_sentry_name, e_sentry))
+
+            if len(e_proc_names.keys()) != len(a_proc_names.keys()):
+                return ('Process name count mismatch.  expected, actual: {}, '
+                        '{}'.format(len(expected), len(actual)))
+
+            for (e_proc_name, e_pids_length), (a_proc_name, a_pids) in \
+                    zip(e_proc_names.items(), a_proc_names.items()):
+                if e_proc_name != a_proc_name:
+                    return ('Process name mismatch.  expected, actual: {}, '
+                            '{}'.format(e_proc_name, a_proc_name))
+
+                a_pids_length = len(a_pids)
+                fail_msg = ('PID count mismatch. {} ({}) expected, actual: '
+                            '{}, {} ({})'.format(e_sentry_name, e_proc_name,
+                                                 e_pids_length, a_pids_length,
+                                                 a_pids))
+
+                # If expected is not bool, ensure PID quantities match
+                if not isinstance(e_pids_length, bool) and \
+                        a_pids_length != e_pids_length:
+                    return fail_msg
+                # If expected is bool True, ensure 1 or more PIDs exist
+                elif isinstance(e_pids_length, bool) and \
+                        e_pids_length is True and a_pids_length < 1:
+                    return fail_msg
+                # If expected is bool False, ensure 0 PIDs exist
+                elif isinstance(e_pids_length, bool) and \
+                        e_pids_length is False and a_pids_length != 0:
+                    return fail_msg
+                else:
+                    self.log.debug('PID check OK: {} {} {}: '
+                                   '{}'.format(e_sentry_name, e_proc_name,
+                                               e_pids_length, a_pids))
+        return None
+
+    def validate_list_of_identical_dicts(self, list_of_dicts):
+        """Check that all dicts within a list are identical."""
+        hashes = []
+        for _dict in list_of_dicts:
+            hashes.append(hash(frozenset(_dict.items())))
+
+        self.log.debug('Hashes: {}'.format(hashes))
+        if len(set(hashes)) == 1:
+            self.log.debug('Dicts within list are identical')
+        else:
+            return 'Dicts within list are not identical'
+
+        return None
+
+    def validate_sectionless_conf(self, file_contents, expected):
+        """A crude conf parser.  Useful to inspect configuration files which
+        do not have section headers (as would be necessary in order to use
+        the configparser).  Such as openstack-dashboard or rabbitmq confs."""
+        for line in file_contents.split('\n'):
+            if '=' in line:
+                args = line.split('=')
+                if len(args) <= 1:
+                    continue
+                key = args[0].strip()
+                value = args[1].strip()
+                if key in expected.keys():
+                    if expected[key] != value:
+                        msg = ('Config mismatch.  Expected, actual:  {}, '
+                               '{}'.format(expected[key], value))
+                        amulet.raise_status(amulet.FAIL, msg=msg)
+
+    def get_unit_hostnames(self, units):
+        """Return a dict of juju unit names to hostnames."""
+        host_names = {}
+        for unit in units:
+            host_names[unit.info['unit_name']] = \
+                str(unit.file_contents('/etc/hostname').strip())
+        self.log.debug('Unit host names: {}'.format(host_names))
+        return host_names
+
+    def run_cmd_unit(self, sentry_unit, cmd):
+        """Run a command on a unit, return the output and exit code."""
+        output, code = sentry_unit.run(cmd)
+        if code == 0:
+            self.log.debug('{} `{}` command returned {} '
+                           '(OK)'.format(sentry_unit.info['unit_name'],
+                                         cmd, code))
+        else:
+            msg = ('{} `{}` command returned {} '
+                   '{}'.format(sentry_unit.info['unit_name'],
+                               cmd, code, output))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+        return str(output), code
+
+    def file_exists_on_unit(self, sentry_unit, file_name):
+        """Check if a file exists on a unit."""
+        try:
+            sentry_unit.file_stat(file_name)
+            return True
+        except IOError:
+            return False
+        except Exception as e:
+            msg = 'Error checking file {}: {}'.format(file_name, e)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+    def file_contents_safe(self, sentry_unit, file_name,
+                           max_wait=60, fatal=False):
+        """Get file contents from a sentry unit.  Wrap amulet file_contents
+        with retry logic to address races where a file checks as existing,
+        but no longer exists by the time file_contents is called.
+        Return None if file not found. Optionally raise if fatal is True."""
+        unit_name = sentry_unit.info['unit_name']
+        file_contents = False
+        tries = 0
+        while not file_contents and tries < (max_wait / 4):
+            try:
+                file_contents = sentry_unit.file_contents(file_name)
+            except IOError:
+                self.log.debug('Attempt {} to open file {} from {} '
+                               'failed'.format(tries, file_name,
+                                               unit_name))
+                time.sleep(4)
+                tries += 1
+
+        if file_contents:
+            return file_contents
+        elif not fatal:
+            return None
+        elif fatal:
+            msg = 'Failed to get file contents from unit.'
+            amulet.raise_status(amulet.FAIL, msg)
+
+    def port_knock_tcp(self, host="localhost", port=22, timeout=15):
+        """Open a TCP socket to check for a listening sevice on a host.
+
+        :param host: host name or IP address, default to localhost
+        :param port: TCP port number, default to 22
+        :param timeout: Connect timeout, default to 15 seconds
+        :returns: True if successful, False if connect failed
+        """
+
+        # Resolve host name if possible
+        try:
+            connect_host = socket.gethostbyname(host)
+            host_human = "{} ({})".format(connect_host, host)
+        except socket.error as e:
+            self.log.warn('Unable to resolve address: '
+                          '{} ({}) Trying anyway!'.format(host, e))
+            connect_host = host
+            host_human = connect_host
+
+        # Attempt socket connection
+        try:
+            knock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            knock.settimeout(timeout)
+            knock.connect((connect_host, port))
+            knock.close()
+            self.log.debug('Socket connect OK for host '
+                           '{} on port {}.'.format(host_human, port))
+            return True
+        except socket.error as e:
+            self.log.debug('Socket connect FAIL for'
+                           ' {} port {} ({})'.format(host_human, port, e))
+            return False
+
+    def port_knock_units(self, sentry_units, port=22,
+                         timeout=15, expect_success=True):
+        """Open a TCP socket to check for a listening sevice on each
+        listed juju unit.
+
+        :param sentry_units: list of sentry unit pointers
+        :param port: TCP port number, default to 22
+        :param timeout: Connect timeout, default to 15 seconds
+        :expect_success: True by default, set False to invert logic
+        :returns: None if successful, Failure message otherwise
+        """
+        for unit in sentry_units:
+            host = unit.info['public-address']
+            connected = self.port_knock_tcp(host, port, timeout)
+            if not connected and expect_success:
+                return 'Socket connect failed.'
+            elif connected and not expect_success:
+                return 'Socket connected unexpectedly.'
+
+    def get_uuid_epoch_stamp(self):
+        """Returns a stamp string based on uuid4 and epoch time.  Useful in
+        generating test messages which need to be unique-ish."""
+        return '[{}-{}]'.format(uuid.uuid4(), time.time())
+
+# amulet juju action helpers:
+    def run_action(self, unit_sentry, action,
+                   _check_output=subprocess.check_output):
+        """Run the named action on a given unit sentry.
+
+        _check_output parameter is used for dependency injection.
+
+        @return action_id.
+        """
+        unit_id = unit_sentry.info["unit_name"]
+        command = ["juju", "action", "do", "--format=json", unit_id, action]
+        self.log.info("Running command: %s\n" % " ".join(command))
+        output = _check_output(command, universal_newlines=True)
+        data = json.loads(output)
+        action_id = data[u'Action queued with id']
+        return action_id
+
+    def wait_on_action(self, action_id, _check_output=subprocess.check_output):
+        """Wait for a given action, returning if it completed or not.
+
+        _check_output parameter is used for dependency injection.
+        """
+        command = ["juju", "action", "fetch", "--format=json", "--wait=0",
+                   action_id]
+        output = _check_output(command, universal_newlines=True)
+        data = json.loads(output)
+        return data.get(u"status") == "completed"

=== added directory 'tests/charmhelpers/contrib/openstack'
=== added file 'tests/charmhelpers/contrib/openstack/__init__.py'
--- tests/charmhelpers/contrib/openstack/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/openstack/__init__.py	2015-09-08 14:01:41 +0000
@@ -0,0 +1,15 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== added directory 'tests/charmhelpers/contrib/openstack/amulet'
=== added file 'tests/charmhelpers/contrib/openstack/amulet/__init__.py'
--- tests/charmhelpers/contrib/openstack/amulet/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/openstack/amulet/__init__.py	2015-09-08 14:01:41 +0000
@@ -0,0 +1,15 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== added file 'tests/charmhelpers/contrib/openstack/amulet/deployment.py'
--- tests/charmhelpers/contrib/openstack/amulet/deployment.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/openstack/amulet/deployment.py	2015-09-08 14:01:41 +0000
@@ -0,0 +1,198 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import six
+from collections import OrderedDict
+from charmhelpers.contrib.amulet.deployment import (
+    AmuletDeployment
+)
+
+
+class OpenStackAmuletDeployment(AmuletDeployment):
+    """OpenStack amulet deployment.
+
+       This class inherits from AmuletDeployment and has additional support
+       that is specifically for use by OpenStack charms.
+       """
+
+    def __init__(self, series=None, openstack=None, source=None, stable=True):
+        """Initialize the deployment environment."""
+        super(OpenStackAmuletDeployment, self).__init__(series)
+        self.openstack = openstack
+        self.source = source
+        self.stable = stable
+        # Note(coreycb): this needs to be changed when new next branches come
+        # out.
+        self.current_next = "trusty"
+
+    def _determine_branch_locations(self, other_services):
+        """Determine the branch locations for the other services.
+
+           Determine if the local branch being tested is derived from its
+           stable or next (dev) branch, and based on this, use the corresonding
+           stable or next branches for the other_services."""
+
+        # Charms outside the lp:~openstack-charmers namespace
+        base_charms = ['mysql', 'mongodb', 'nrpe']
+
+        # Force these charms to current series even when using an older series.
+        # ie. Use trusty/nrpe even when series is precise, as the P charm
+        # does not possess the necessary external master config and hooks.
+        force_series_current = ['nrpe']
+
+        if self.series in ['precise', 'trusty']:
+            base_series = self.series
+        else:
+            base_series = self.current_next
+
+        if self.stable:
+            for svc in other_services:
+                if svc['name'] in force_series_current:
+                    base_series = self.current_next
+
+                temp = 'lp:charms/{}/{}'
+                svc['location'] = temp.format(base_series,
+                                              svc['name'])
+        else:
+            for svc in other_services:
+                if svc['name'] in force_series_current:
+                    base_series = self.current_next
+
+                if svc['name'] in base_charms:
+                    temp = 'lp:charms/{}/{}'
+                    svc['location'] = temp.format(base_series,
+                                                  svc['name'])
+                else:
+                    temp = 'lp:~openstack-charmers/charms/{}/{}/next'
+                    svc['location'] = temp.format(self.current_next,
+                                                  svc['name'])
+        return other_services
+
+    def _add_services(self, this_service, other_services):
+        """Add services to the deployment and set openstack-origin/source."""
+        other_services = self._determine_branch_locations(other_services)
+
+        super(OpenStackAmuletDeployment, self)._add_services(this_service,
+                                                             other_services)
+
+        services = other_services
+        services.append(this_service)
+
+        # Charms which should use the source config option
+        use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',
+                      'ceph-osd', 'ceph-radosgw']
+
+        # Charms which can not use openstack-origin, ie. many subordinates
+        no_origin = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe']
+
+        if self.openstack:
+            for svc in services:
+                if svc['name'] not in use_source + no_origin:
+                    config = {'openstack-origin': self.openstack}
+                    self.d.configure(svc['name'], config)
+
+        if self.source:
+            for svc in services:
+                if svc['name'] in use_source and svc['name'] not in no_origin:
+                    config = {'source': self.source}
+                    self.d.configure(svc['name'], config)
+
+    def _configure_services(self, configs):
+        """Configure all of the services."""
+        for service, config in six.iteritems(configs):
+            self.d.configure(service, config)
+
+    def _get_openstack_release(self):
+        """Get openstack release.
+
+           Return an integer representing the enum value of the openstack
+           release.
+           """
+        # Must be ordered by OpenStack release (not by Ubuntu release):
+        (self.precise_essex, self.precise_folsom, self.precise_grizzly,
+         self.precise_havana, self.precise_icehouse,
+         self.trusty_icehouse, self.trusty_juno, self.utopic_juno,
+         self.trusty_kilo, self.vivid_kilo, self.trusty_liberty,
+         self.wily_liberty) = range(12)
+
+        releases = {
+            ('precise', None): self.precise_essex,
+            ('precise', 'cloud:precise-folsom'): self.precise_folsom,
+            ('precise', 'cloud:precise-grizzly'): self.precise_grizzly,
+            ('precise', 'cloud:precise-havana'): self.precise_havana,
+            ('precise', 'cloud:precise-icehouse'): self.precise_icehouse,
+            ('trusty', None): self.trusty_icehouse,
+            ('trusty', 'cloud:trusty-juno'): self.trusty_juno,
+            ('trusty', 'cloud:trusty-kilo'): self.trusty_kilo,
+            ('trusty', 'cloud:trusty-liberty'): self.trusty_liberty,
+            ('utopic', None): self.utopic_juno,
+            ('vivid', None): self.vivid_kilo,
+            ('wily', None): self.wily_liberty}
+        return releases[(self.series, self.openstack)]
+
+    def _get_openstack_release_string(self):
+        """Get openstack release string.
+
+           Return a string representing the openstack release.
+           """
+        releases = OrderedDict([
+            ('precise', 'essex'),
+            ('quantal', 'folsom'),
+            ('raring', 'grizzly'),
+            ('saucy', 'havana'),
+            ('trusty', 'icehouse'),
+            ('utopic', 'juno'),
+            ('vivid', 'kilo'),
+            ('wily', 'liberty'),
+        ])
+        if self.openstack:
+            os_origin = self.openstack.split(':')[1]
+            return os_origin.split('%s-' % self.series)[1].split('/')[0]
+        else:
+            return releases[self.series]
+
+    def get_ceph_expected_pools(self, radosgw=False):
+        """Return a list of expected ceph pools in a ceph + cinder + glance
+        test scenario, based on OpenStack release and whether ceph radosgw
+        is flagged as present or not."""
+
+        if self._get_openstack_release() >= self.trusty_kilo:
+            # Kilo or later
+            pools = [
+                'rbd',
+                'cinder',
+                'glance'
+            ]
+        else:
+            # Juno or earlier
+            pools = [
+                'data',
+                'metadata',
+                'rbd',
+                'cinder',
+                'glance'
+            ]
+
+        if radosgw:
+            pools.extend([
+                '.rgw.root',
+                '.rgw.control',
+                '.rgw',
+                '.rgw.gc',
+                '.users.uid'
+            ])
+
+        return pools

=== added file 'tests/charmhelpers/contrib/openstack/amulet/utils.py'
--- tests/charmhelpers/contrib/openstack/amulet/utils.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/openstack/amulet/utils.py	2015-09-08 14:01:41 +0000
@@ -0,0 +1,963 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import amulet
+import json
+import logging
+import os
+import six
+import time
+import urllib
+
+import cinderclient.v1.client as cinder_client
+import glanceclient.v1.client as glance_client
+import heatclient.v1.client as heat_client
+import keystoneclient.v2_0 as keystone_client
+import novaclient.v1_1.client as nova_client
+import pika
+import swiftclient
+
+from charmhelpers.contrib.amulet.utils import (
+    AmuletUtils
+)
+
+DEBUG = logging.DEBUG
+ERROR = logging.ERROR
+
+
+class OpenStackAmuletUtils(AmuletUtils):
+    """OpenStack amulet utilities.
+
+       This class inherits from AmuletUtils and has additional support
+       that is specifically for use by OpenStack charm tests.
+       """
+
+    def __init__(self, log_level=ERROR):
+        """Initialize the deployment environment."""
+        super(OpenStackAmuletUtils, self).__init__(log_level)
+
+    def validate_endpoint_data(self, endpoints, admin_port, internal_port,
+                               public_port, expected):
+        """Validate endpoint data.
+
+           Validate actual endpoint data vs expected endpoint data. The ports
+           are used to find the matching endpoint.
+           """
+        self.log.debug('Validating endpoint data...')
+        self.log.debug('actual: {}'.format(repr(endpoints)))
+        found = False
+        for ep in endpoints:
+            self.log.debug('endpoint: {}'.format(repr(ep)))
+            if (admin_port in ep.adminurl and
+                    internal_port in ep.internalurl and
+                    public_port in ep.publicurl):
+                found = True
+                actual = {'id': ep.id,
+                          'region': ep.region,
+                          'adminurl': ep.adminurl,
+                          'internalurl': ep.internalurl,
+                          'publicurl': ep.publicurl,
+                          'service_id': ep.service_id}
+                ret = self._validate_dict_data(expected, actual)
+                if ret:
+                    return 'unexpected endpoint data - {}'.format(ret)
+
+        if not found:
+            return 'endpoint not found'
+
+    def validate_svc_catalog_endpoint_data(self, expected, actual):
+        """Validate service catalog endpoint data.
+
+           Validate a list of actual service catalog endpoints vs a list of
+           expected service catalog endpoints.
+           """
+        self.log.debug('Validating service catalog endpoint data...')
+        self.log.debug('actual: {}'.format(repr(actual)))
+        for k, v in six.iteritems(expected):
+            if k in actual:
+                ret = self._validate_dict_data(expected[k][0], actual[k][0])
+                if ret:
+                    return self.endpoint_error(k, ret)
+            else:
+                return "endpoint {} does not exist".format(k)
+        return ret
+
+    def validate_tenant_data(self, expected, actual):
+        """Validate tenant data.
+
+           Validate a list of actual tenant data vs list of expected tenant
+           data.
+           """
+        self.log.debug('Validating tenant data...')
+        self.log.debug('actual: {}'.format(repr(actual)))
+        for e in expected:
+            found = False
+            for act in actual:
+                a = {'enabled': act.enabled, 'description': act.description,
+                     'name': act.name, 'id': act.id}
+                if e['name'] == a['name']:
+                    found = True
+                    ret = self._validate_dict_data(e, a)
+                    if ret:
+                        return "unexpected tenant data - {}".format(ret)
+            if not found:
+                return "tenant {} does not exist".format(e['name'])
+        return ret
+
+    def validate_role_data(self, expected, actual):
+        """Validate role data.
+
+           Validate a list of actual role data vs a list of expected role
+           data.
+           """
+        self.log.debug('Validating role data...')
+        self.log.debug('actual: {}'.format(repr(actual)))
+        for e in expected:
+            found = False
+            for act in actual:
+                a = {'name': act.name, 'id': act.id}
+                if e['name'] == a['name']:
+                    found = True
+                    ret = self._validate_dict_data(e, a)
+                    if ret:
+                        return "unexpected role data - {}".format(ret)
+            if not found:
+                return "role {} does not exist".format(e['name'])
+        return ret
+
+    def validate_user_data(self, expected, actual):
+        """Validate user data.
+
+           Validate a list of actual user data vs a list of expected user
+           data.
+           """
+        self.log.debug('Validating user data...')
+        self.log.debug('actual: {}'.format(repr(actual)))
+        for e in expected:
+            found = False
+            for act in actual:
+                a = {'enabled': act.enabled, 'name': act.name,
+                     'email': act.email, 'tenantId': act.tenantId,
+                     'id': act.id}
+                if e['name'] == a['name']:
+                    found = True
+                    ret = self._validate_dict_data(e, a)
+                    if ret:
+                        return "unexpected user data - {}".format(ret)
+            if not found:
+                return "user {} does not exist".format(e['name'])
+        return ret
+
+    def validate_flavor_data(self, expected, actual):
+        """Validate flavor data.
+
+           Validate a list of actual flavors vs a list of expected flavors.
+           """
+        self.log.debug('Validating flavor data...')
+        self.log.debug('actual: {}'.format(repr(actual)))
+        act = [a.name for a in actual]
+        return self._validate_list_data(expected, act)
+
+    def tenant_exists(self, keystone, tenant):
+        """Return True if tenant exists."""
+        self.log.debug('Checking if tenant exists ({})...'.format(tenant))
+        return tenant in [t.name for t in keystone.tenants.list()]
+
+    def authenticate_cinder_admin(self, keystone_sentry, username,
+                                  password, tenant):
+        """Authenticates admin user with cinder."""
+        # NOTE(beisner): cinder python client doesn't accept tokens.
+        service_ip = \
+            keystone_sentry.relation('shared-db',
+                                     'mysql:shared-db')['private-address']
+        ept = "http://{}:5000/v2.0".format(service_ip.strip().decode('utf-8'))
+        return cinder_client.Client(username, password, tenant, ept)
+
+    def authenticate_keystone_admin(self, keystone_sentry, user, password,
+                                    tenant):
+        """Authenticates admin user with the keystone admin endpoint."""
+        self.log.debug('Authenticating keystone admin...')
+        unit = keystone_sentry
+        service_ip = unit.relation('shared-db',
+                                   'mysql:shared-db')['private-address']
+        ep = "http://{}:35357/v2.0".format(service_ip.strip().decode('utf-8'))
+        return keystone_client.Client(username=user, password=password,
+                                      tenant_name=tenant, auth_url=ep)
+
+    def authenticate_keystone_user(self, keystone, user, password, tenant):
+        """Authenticates a regular user with the keystone public endpoint."""
+        self.log.debug('Authenticating keystone user ({})...'.format(user))
+        ep = keystone.service_catalog.url_for(service_type='identity',
+                                              endpoint_type='publicURL')
+        return keystone_client.Client(username=user, password=password,
+                                      tenant_name=tenant, auth_url=ep)
+
+    def authenticate_glance_admin(self, keystone):
+        """Authenticates admin user with glance."""
+        self.log.debug('Authenticating glance admin...')
+        ep = keystone.service_catalog.url_for(service_type='image',
+                                              endpoint_type='adminURL')
+        return glance_client.Client(ep, token=keystone.auth_token)
+
+    def authenticate_heat_admin(self, keystone):
+        """Authenticates the admin user with heat."""
+        self.log.debug('Authenticating heat admin...')
+        ep = keystone.service_catalog.url_for(service_type='orchestration',
+                                              endpoint_type='publicURL')
+        return heat_client.Client(endpoint=ep, token=keystone.auth_token)
+
+    def authenticate_nova_user(self, keystone, user, password, tenant):
+        """Authenticates a regular user with nova-api."""
+        self.log.debug('Authenticating nova user ({})...'.format(user))
+        ep = keystone.service_catalog.url_for(service_type='identity',
+                                              endpoint_type='publicURL')
+        return nova_client.Client(username=user, api_key=password,
+                                  project_id=tenant, auth_url=ep)
+
+    def authenticate_swift_user(self, keystone, user, password, tenant):
+        """Authenticates a regular user with swift api."""
+        self.log.debug('Authenticating swift user ({})...'.format(user))
+        ep = keystone.service_catalog.url_for(service_type='identity',
+                                              endpoint_type='publicURL')
+        return swiftclient.Connection(authurl=ep,
+                                      user=user,
+                                      key=password,
+                                      tenant_name=tenant,
+                                      auth_version='2.0')
+
+    def create_cirros_image(self, glance, image_name):
+        """Download the latest cirros image and upload it to glance,
+        validate and return a resource pointer.
+
+        :param glance: pointer to authenticated glance connection
+        :param image_name: display name for new image
+        :returns: glance image pointer
+        """
+        self.log.debug('Creating glance cirros image '
+                       '({})...'.format(image_name))
+
+        # Download cirros image
+        http_proxy = os.getenv('AMULET_HTTP_PROXY')
+        self.log.debug('AMULET_HTTP_PROXY: {}'.format(http_proxy))
+        if http_proxy:
+            proxies = {'http': http_proxy}
+            opener = urllib.FancyURLopener(proxies)
+        else:
+            opener = urllib.FancyURLopener()
+
+        f = opener.open('http://download.cirros-cloud.net/version/released')
+        version = f.read().strip()
+        cirros_img = 'cirros-{}-x86_64-disk.img'.format(version)
+        local_path = os.path.join('tests', cirros_img)
+
+        if not os.path.exists(local_path):
+            cirros_url = 'http://{}/{}/{}'.format('download.cirros-cloud.net',
+                                                  version, cirros_img)
+            opener.retrieve(cirros_url, local_path)
+        f.close()
+
+        # Create glance image
+        with open(local_path) as f:
+            image = glance.images.create(name=image_name, is_public=True,
+                                         disk_format='qcow2',
+                                         container_format='bare', data=f)
+
+        # Wait for image to reach active status
+        img_id = image.id
+        ret = self.resource_reaches_status(glance.images, img_id,
+                                           expected_stat='active',
+                                           msg='Image status wait')
+        if not ret:
+            msg = 'Glance image failed to reach expected state.'
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        # Re-validate new image
+        self.log.debug('Validating image attributes...')
+        val_img_name = glance.images.get(img_id).name
+        val_img_stat = glance.images.get(img_id).status
+        val_img_pub = glance.images.get(img_id).is_public
+        val_img_cfmt = glance.images.get(img_id).container_format
+        val_img_dfmt = glance.images.get(img_id).disk_format
+        msg_attr = ('Image attributes - name:{} public:{} id:{} stat:{} '
+                    'container fmt:{} disk fmt:{}'.format(
+                        val_img_name, val_img_pub, img_id,
+                        val_img_stat, val_img_cfmt, val_img_dfmt))
+
+        if val_img_name == image_name and val_img_stat == 'active' \
+                and val_img_pub is True and val_img_cfmt == 'bare' \
+                and val_img_dfmt == 'qcow2':
+            self.log.debug(msg_attr)
+        else:
+            msg = ('Volume validation failed, {}'.format(msg_attr))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        return image
+
+    def delete_image(self, glance, image):
+        """Delete the specified image."""
+
+        # /!\ DEPRECATION WARNING
+        self.log.warn('/!\\ DEPRECATION WARNING:  use '
+                      'delete_resource instead of delete_image.')
+        self.log.debug('Deleting glance image ({})...'.format(image))
+        return self.delete_resource(glance.images, image, msg='glance image')
+
+    def create_instance(self, nova, image_name, instance_name, flavor):
+        """Create the specified instance."""
+        self.log.debug('Creating instance '
+                       '({}|{}|{})'.format(instance_name, image_name, flavor))
+        image = nova.images.find(name=image_name)
+        flavor = nova.flavors.find(name=flavor)
+        instance = nova.servers.create(name=instance_name, image=image,
+                                       flavor=flavor)
+
+        count = 1
+        status = instance.status
+        while status != 'ACTIVE' and count < 60:
+            time.sleep(3)
+            instance = nova.servers.get(instance.id)
+            status = instance.status
+            self.log.debug('instance status: {}'.format(status))
+            count += 1
+
+        if status != 'ACTIVE':
+            self.log.error('instance creation timed out')
+            return None
+
+        return instance
+
+    def delete_instance(self, nova, instance):
+        """Delete the specified instance."""
+
+        # /!\ DEPRECATION WARNING
+        self.log.warn('/!\\ DEPRECATION WARNING:  use '
+                      'delete_resource instead of delete_instance.')
+        self.log.debug('Deleting instance ({})...'.format(instance))
+        return self.delete_resource(nova.servers, instance,
+                                    msg='nova instance')
+
+    def create_or_get_keypair(self, nova, keypair_name="testkey"):
+        """Create a new keypair, or return pointer if it already exists."""
+        try:
+            _keypair = nova.keypairs.get(keypair_name)
+            self.log.debug('Keypair ({}) already exists, '
+                           'using it.'.format(keypair_name))
+            return _keypair
+        except:
+            self.log.debug('Keypair ({}) does not exist, '
+                           'creating it.'.format(keypair_name))
+
+        _keypair = nova.keypairs.create(name=keypair_name)
+        return _keypair
+
+    def create_cinder_volume(self, cinder, vol_name="demo-vol", vol_size=1,
+                             img_id=None, src_vol_id=None, snap_id=None):
+        """Create cinder volume, optionally from a glance image, OR
+        optionally as a clone of an existing volume, OR optionally
+        from a snapshot.  Wait for the new volume status to reach
+        the expected status, validate and return a resource pointer.
+
+        :param vol_name: cinder volume display name
+        :param vol_size: size in gigabytes
+        :param img_id: optional glance image id
+        :param src_vol_id: optional source volume id to clone
+        :param snap_id: optional snapshot id to use
+        :returns: cinder volume pointer
+        """
+        # Handle parameter input and avoid impossible combinations
+        if img_id and not src_vol_id and not snap_id:
+            # Create volume from image
+            self.log.debug('Creating cinder volume from glance image...')
+            bootable = 'true'
+        elif src_vol_id and not img_id and not snap_id:
+            # Clone an existing volume
+            self.log.debug('Cloning cinder volume...')
+            bootable = cinder.volumes.get(src_vol_id).bootable
+        elif snap_id and not src_vol_id and not img_id:
+            # Create volume from snapshot
+            self.log.debug('Creating cinder volume from snapshot...')
+            snap = cinder.volume_snapshots.find(id=snap_id)
+            vol_size = snap.size
+            snap_vol_id = cinder.volume_snapshots.get(snap_id).volume_id
+            bootable = cinder.volumes.get(snap_vol_id).bootable
+        elif not img_id and not src_vol_id and not snap_id:
+            # Create volume
+            self.log.debug('Creating cinder volume...')
+            bootable = 'false'
+        else:
+            # Impossible combination of parameters
+            msg = ('Invalid method use - name:{} size:{} img_id:{} '
+                   'src_vol_id:{} snap_id:{}'.format(vol_name, vol_size,
+                                                     img_id, src_vol_id,
+                                                     snap_id))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        # Create new volume
+        try:
+            vol_new = cinder.volumes.create(display_name=vol_name,
+                                            imageRef=img_id,
+                                            size=vol_size,
+                                            source_volid=src_vol_id,
+                                            snapshot_id=snap_id)
+            vol_id = vol_new.id
+        except Exception as e:
+            msg = 'Failed to create volume: {}'.format(e)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        # Wait for volume to reach available status
+        ret = self.resource_reaches_status(cinder.volumes, vol_id,
+                                           expected_stat="available",
+                                           msg="Volume status wait")
+        if not ret:
+            msg = 'Cinder volume failed to reach expected state.'
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        # Re-validate new volume
+        self.log.debug('Validating volume attributes...')
+        val_vol_name = cinder.volumes.get(vol_id).display_name
+        val_vol_boot = cinder.volumes.get(vol_id).bootable
+        val_vol_stat = cinder.volumes.get(vol_id).status
+        val_vol_size = cinder.volumes.get(vol_id).size
+        msg_attr = ('Volume attributes - name:{} id:{} stat:{} boot:'
+                    '{} size:{}'.format(val_vol_name, vol_id,
+                                        val_vol_stat, val_vol_boot,
+                                        val_vol_size))
+
+        if val_vol_boot == bootable and val_vol_stat == 'available' \
+                and val_vol_name == vol_name and val_vol_size == vol_size:
+            self.log.debug(msg_attr)
+        else:
+            msg = ('Volume validation failed, {}'.format(msg_attr))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        return vol_new
+
+    def delete_resource(self, resource, resource_id,
+                        msg="resource", max_wait=120):
+        """Delete one openstack resource, such as one instance, keypair,
+        image, volume, stack, etc., and confirm deletion within max wait time.
+
+        :param resource: pointer to os resource type, ex:glance_client.images
+        :param resource_id: unique name or id for the openstack resource
+        :param msg: text to identify purpose in logging
+        :param max_wait: maximum wait time in seconds
+        :returns: True if successful, otherwise False
+        """
+        self.log.debug('Deleting OpenStack resource '
+                       '{} ({})'.format(resource_id, msg))
+        num_before = len(list(resource.list()))
+        resource.delete(resource_id)
+
+        tries = 0
+        num_after = len(list(resource.list()))
+        while num_after != (num_before - 1) and tries < (max_wait / 4):
+            self.log.debug('{} delete check: '
+                           '{} [{}:{}] {}'.format(msg, tries,
+                                                  num_before,
+                                                  num_after,
+                                                  resource_id))
+            time.sleep(4)
+            num_after = len(list(resource.list()))
+            tries += 1
+
+        self.log.debug('{}:  expected, actual count = {}, '
+                       '{}'.format(msg, num_before - 1, num_after))
+
+        if num_after == (num_before - 1):
+            return True
+        else:
+            self.log.error('{} delete timed out'.format(msg))
+            return False
+
+    def resource_reaches_status(self, resource, resource_id,
+                                expected_stat='available',
+                                msg='resource', max_wait=120):
+        """Wait for an openstack resources status to reach an
+           expected status within a specified time.  Useful to confirm that
+           nova instances, cinder vols, snapshots, glance images, heat stacks
+           and other resources eventually reach the expected status.
+
+        :param resource: pointer to os resource type, ex: heat_client.stacks
+        :param resource_id: unique id for the openstack resource
+        :param expected_stat: status to expect resource to reach
+        :param msg: text to identify purpose in logging
+        :param max_wait: maximum wait time in seconds
+        :returns: True if successful, False if status is not reached
+        """
+
+        tries = 0
+        resource_stat = resource.get(resource_id).status
+        while resource_stat != expected_stat and tries < (max_wait / 4):
+            self.log.debug('{} status check: '
+                           '{} [{}:{}] {}'.format(msg, tries,
+                                                  resource_stat,
+                                                  expected_stat,
+                                                  resource_id))
+            time.sleep(4)
+            resource_stat = resource.get(resource_id).status
+            tries += 1
+
+        self.log.debug('{}:  expected, actual status = {}, '
+                       '{}'.format(msg, resource_stat, expected_stat))
+
+        if resource_stat == expected_stat:
+            return True
+        else:
+            self.log.debug('{} never reached expected status: '
+                           '{}'.format(resource_id, expected_stat))
+            return False
+
+    def get_ceph_osd_id_cmd(self, index):
+        """Produce a shell command that will return a ceph-osd id."""
+        return ("`initctl list | grep 'ceph-osd ' | "
+                "awk 'NR=={} {{ print $2 }}' | "
+                "grep -o '[0-9]*'`".format(index + 1))
+
+    def get_ceph_pools(self, sentry_unit):
+        """Return a dict of ceph pools from a single ceph unit, with
+        pool name as keys, pool id as vals."""
+        pools = {}
+        cmd = 'sudo ceph osd lspools'
+        output, code = sentry_unit.run(cmd)
+        if code != 0:
+            msg = ('{} `{}` returned {} '
+                   '{}'.format(sentry_unit.info['unit_name'],
+                               cmd, code, output))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        # Example output: 0 data,1 metadata,2 rbd,3 cinder,4 glance,
+        for pool in str(output).split(','):
+            pool_id_name = pool.split(' ')
+            if len(pool_id_name) == 2:
+                pool_id = pool_id_name[0]
+                pool_name = pool_id_name[1]
+                pools[pool_name] = int(pool_id)
+
+        self.log.debug('Pools on {}: {}'.format(sentry_unit.info['unit_name'],
+                                                pools))
+        return pools
+
+    def get_ceph_df(self, sentry_unit):
+        """Return dict of ceph df json output, including ceph pool state.
+
+        :param sentry_unit: Pointer to amulet sentry instance (juju unit)
+        :returns: Dict of ceph df output
+        """
+        cmd = 'sudo ceph df --format=json'
+        output, code = sentry_unit.run(cmd)
+        if code != 0:
+            msg = ('{} `{}` returned {} '
+                   '{}'.format(sentry_unit.info['unit_name'],
+                               cmd, code, output))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+        return json.loads(output)
+
+    def get_ceph_pool_sample(self, sentry_unit, pool_id=0):
+        """Take a sample of attributes of a ceph pool, returning ceph
+        pool name, object count and disk space used for the specified
+        pool ID number.
+
+        :param sentry_unit: Pointer to amulet sentry instance (juju unit)
+        :param pool_id: Ceph pool ID
+        :returns: List of pool name, object count, kb disk space used
+        """
+        df = self.get_ceph_df(sentry_unit)
+        pool_name = df['pools'][pool_id]['name']
+        obj_count = df['pools'][pool_id]['stats']['objects']
+        kb_used = df['pools'][pool_id]['stats']['kb_used']
+        self.log.debug('Ceph {} pool (ID {}): {} objects, '
+                       '{} kb used'.format(pool_name, pool_id,
+                                           obj_count, kb_used))
+        return pool_name, obj_count, kb_used
+
+    def validate_ceph_pool_samples(self, samples, sample_type="resource pool"):
+        """Validate ceph pool samples taken over time, such as pool
+        object counts or pool kb used, before adding, after adding, and
+        after deleting items which affect those pool attributes.  The
+        2nd element is expected to be greater than the 1st; 3rd is expected
+        to be less than the 2nd.
+
+        :param samples: List containing 3 data samples
+        :param sample_type: String for logging and usage context
+        :returns: None if successful, Failure message otherwise
+        """
+        original, created, deleted = range(3)
+        if samples[created] <= samples[original] or \
+                samples[deleted] >= samples[created]:
+            return ('Ceph {} samples ({}) '
+                    'unexpected.'.format(sample_type, samples))
+        else:
+            self.log.debug('Ceph {} samples (OK): '
+                           '{}'.format(sample_type, samples))
+            return None
+
+# rabbitmq/amqp specific helpers:
+    def add_rmq_test_user(self, sentry_units,
+                          username="testuser1", password="changeme"):
+        """Add a test user via the first rmq juju unit, check connection as
+        the new user against all sentry units.
+
+        :param sentry_units: list of sentry unit pointers
+        :param username: amqp user name, default to testuser1
+        :param password: amqp user password
+        :returns: None if successful.  Raise on error.
+        """
+        self.log.debug('Adding rmq user ({})...'.format(username))
+
+        # Check that user does not already exist
+        cmd_user_list = 'rabbitmqctl list_users'
+        output, _ = self.run_cmd_unit(sentry_units[0], cmd_user_list)
+        if username in output:
+            self.log.warning('User ({}) already exists, returning '
+                             'gracefully.'.format(username))
+            return
+
+        perms = '".*" ".*" ".*"'
+        cmds = ['rabbitmqctl add_user {} {}'.format(username, password),
+                'rabbitmqctl set_permissions {} {}'.format(username, perms)]
+
+        # Add user via first unit
+        for cmd in cmds:
+            output, _ = self.run_cmd_unit(sentry_units[0], cmd)
+
+        # Check connection against the other sentry_units
+        self.log.debug('Checking user connect against units...')
+        for sentry_unit in sentry_units:
+            connection = self.connect_amqp_by_unit(sentry_unit, ssl=False,
+                                                   username=username,
+                                                   password=password)
+            connection.close()
+
+    def delete_rmq_test_user(self, sentry_units, username="testuser1"):
+        """Delete a rabbitmq user via the first rmq juju unit.
+
+        :param sentry_units: list of sentry unit pointers
+        :param username: amqp user name, default to testuser1
+        :param password: amqp user password
+        :returns: None if successful or no such user.
+        """
+        self.log.debug('Deleting rmq user ({})...'.format(username))
+
+        # Check that the user exists
+        cmd_user_list = 'rabbitmqctl list_users'
+        output, _ = self.run_cmd_unit(sentry_units[0], cmd_user_list)
+
+        if username not in output:
+            self.log.warning('User ({}) does not exist, returning '
+                             'gracefully.'.format(username))
+            return
+
+        # Delete the user
+        cmd_user_del = 'rabbitmqctl delete_user {}'.format(username)
+        output, _ = self.run_cmd_unit(sentry_units[0], cmd_user_del)
+
+    def get_rmq_cluster_status(self, sentry_unit):
+        """Execute rabbitmq cluster status command on a unit and return
+        the full output.
+
+        :param unit: sentry unit
+        :returns: String containing console output of cluster status command
+        """
+        cmd = 'rabbitmqctl cluster_status'
+        output, _ = self.run_cmd_unit(sentry_unit, cmd)
+        self.log.debug('{} cluster_status:\n{}'.format(
+            sentry_unit.info['unit_name'], output))
+        return str(output)
+
+    def get_rmq_cluster_running_nodes(self, sentry_unit):
+        """Parse rabbitmqctl cluster_status output string, return list of
+        running rabbitmq cluster nodes.
+
+        :param unit: sentry unit
+        :returns: List containing node names of running nodes
+        """
+        # NOTE(beisner): rabbitmqctl cluster_status output is not
+        # json-parsable, do string chop foo, then json.loads that.
+        str_stat = self.get_rmq_cluster_status(sentry_unit)
+        if 'running_nodes' in str_stat:
+            pos_start = str_stat.find("{running_nodes,") + 15
+            pos_end = str_stat.find("]},", pos_start) + 1
+            str_run_nodes = str_stat[pos_start:pos_end].replace("'", '"')
+            run_nodes = json.loads(str_run_nodes)
+            return run_nodes
+        else:
+            return []
+
+    def validate_rmq_cluster_running_nodes(self, sentry_units):
+        """Check that all rmq unit hostnames are represented in the
+        cluster_status output of all units.
+
+        :param host_names: dict of juju unit names to host names
+        :param units: list of sentry unit pointers (all rmq units)
+        :returns: None if successful, otherwise return error message
+        """
+        host_names = self.get_unit_hostnames(sentry_units)
+        errors = []
+
+        # Query every unit for cluster_status running nodes
+        for query_unit in sentry_units:
+            query_unit_name = query_unit.info['unit_name']
+            running_nodes = self.get_rmq_cluster_running_nodes(query_unit)
+
+            # Confirm that every unit is represented in the queried unit's
+            # cluster_status running nodes output.
+            for validate_unit in sentry_units:
+                val_host_name = host_names[validate_unit.info['unit_name']]
+                val_node_name = 'rabbit@{}'.format(val_host_name)
+
+                if val_node_name not in running_nodes:
+                    errors.append('Cluster member check failed on {}: {} not '
+                                  'in {}\n'.format(query_unit_name,
+                                                   val_node_name,
+                                                   running_nodes))
+        if errors:
+            return ''.join(errors)
+
+    def rmq_ssl_is_enabled_on_unit(self, sentry_unit, port=None):
+        """Check a single juju rmq unit for ssl and port in the config file."""
+        host = sentry_unit.info['public-address']
+        unit_name = sentry_unit.info['unit_name']
+
+        conf_file = '/etc/rabbitmq/rabbitmq.config'
+        conf_contents = str(self.file_contents_safe(sentry_unit,
+                                                    conf_file, max_wait=16))
+        # Checks
+        conf_ssl = 'ssl' in conf_contents
+        conf_port = str(port) in conf_contents
+
+        # Port explicitly checked in config
+        if port and conf_port and conf_ssl:
+            self.log.debug('SSL is enabled  @{}:{} '
+                           '({})'.format(host, port, unit_name))
+            return True
+        elif port and not conf_port and conf_ssl:
+            self.log.debug('SSL is enabled @{} but not on port {} '
+                           '({})'.format(host, port, unit_name))
+            return False
+        # Port not checked (useful when checking that ssl is disabled)
+        elif not port and conf_ssl:
+            self.log.debug('SSL is enabled  @{}:{} '
+                           '({})'.format(host, port, unit_name))
+            return True
+        elif not port and not conf_ssl:
+            self.log.debug('SSL not enabled @{}:{} '
+                           '({})'.format(host, port, unit_name))
+            return False
+        else:
+            msg = ('Unknown condition when checking SSL status @{}:{} '
+                   '({})'.format(host, port, unit_name))
+            amulet.raise_status(amulet.FAIL, msg)
+
+    def validate_rmq_ssl_enabled_units(self, sentry_units, port=None):
+        """Check that ssl is enabled on rmq juju sentry units.
+
+        :param sentry_units: list of all rmq sentry units
+        :param port: optional ssl port override to validate
+        :returns: None if successful, otherwise return error message
+        """
+        for sentry_unit in sentry_units:
+            if not self.rmq_ssl_is_enabled_on_unit(sentry_unit, port=port):
+                return ('Unexpected condition:  ssl is disabled on unit '
+                        '({})'.format(sentry_unit.info['unit_name']))
+        return None
+
+    def validate_rmq_ssl_disabled_units(self, sentry_units):
+        """Check that ssl is enabled on listed rmq juju sentry units.
+
+        :param sentry_units: list of all rmq sentry units
+        :returns: True if successful.  Raise on error.
+        """
+        for sentry_unit in sentry_units:
+            if self.rmq_ssl_is_enabled_on_unit(sentry_unit):
+                return ('Unexpected condition:  ssl is enabled on unit '
+                        '({})'.format(sentry_unit.info['unit_name']))
+        return None
+
+    def configure_rmq_ssl_on(self, sentry_units, deployment,
+                             port=None, max_wait=60):
+        """Turn ssl charm config option on, with optional non-default
+        ssl port specification.  Confirm that it is enabled on every
+        unit.
+
+        :param sentry_units: list of sentry units
+        :param deployment: amulet deployment object pointer
+        :param port: amqp port, use defaults if None
+        :param max_wait: maximum time to wait in seconds to confirm
+        :returns: None if successful.  Raise on error.
+        """
+        self.log.debug('Setting ssl charm config option:  on')
+
+        # Enable RMQ SSL
+        config = {'ssl': 'on'}
+        if port:
+            config['ssl_port'] = port
+
+        deployment.configure('rabbitmq-server', config)
+
+        # Confirm
+        tries = 0
+        ret = self.validate_rmq_ssl_enabled_units(sentry_units, port=port)
+        while ret and tries < (max_wait / 4):
+            time.sleep(4)
+            self.log.debug('Attempt {}: {}'.format(tries, ret))
+            ret = self.validate_rmq_ssl_enabled_units(sentry_units, port=port)
+            tries += 1
+
+        if ret:
+            amulet.raise_status(amulet.FAIL, ret)
+
+    def configure_rmq_ssl_off(self, sentry_units, deployment, max_wait=60):
+        """Turn ssl charm config option off, confirm that it is disabled
+        on every unit.
+
+        :param sentry_units: list of sentry units
+        :param deployment: amulet deployment object pointer
+        :param max_wait: maximum time to wait in seconds to confirm
+        :returns: None if successful.  Raise on error.
+        """
+        self.log.debug('Setting ssl charm config option:  off')
+
+        # Disable RMQ SSL
+        config = {'ssl': 'off'}
+        deployment.configure('rabbitmq-server', config)
+
+        # Confirm
+        tries = 0
+        ret = self.validate_rmq_ssl_disabled_units(sentry_units)
+        while ret and tries < (max_wait / 4):
+            time.sleep(4)
+            self.log.debug('Attempt {}: {}'.format(tries, ret))
+            ret = self.validate_rmq_ssl_disabled_units(sentry_units)
+            tries += 1
+
+        if ret:
+            amulet.raise_status(amulet.FAIL, ret)
+
+    def connect_amqp_by_unit(self, sentry_unit, ssl=False,
+                             port=None, fatal=True,
+                             username="testuser1", password="changeme"):
+        """Establish and return a pika amqp connection to the rabbitmq service
+        running on a rmq juju unit.
+
+        :param sentry_unit: sentry unit pointer
+        :param ssl: boolean, default to False
+        :param port: amqp port, use defaults if None
+        :param fatal: boolean, default to True (raises on connect error)
+        :param username: amqp user name, default to testuser1
+        :param password: amqp user password
+        :returns: pika amqp connection pointer or None if failed and non-fatal
+        """
+        host = sentry_unit.info['public-address']
+        unit_name = sentry_unit.info['unit_name']
+
+        # Default port logic if port is not specified
+        if ssl and not port:
+            port = 5671
+        elif not ssl and not port:
+            port = 5672
+
+        self.log.debug('Connecting to amqp on {}:{} ({}) as '
+                       '{}...'.format(host, port, unit_name, username))
+
+        try:
+            credentials = pika.PlainCredentials(username, password)
+            parameters = pika.ConnectionParameters(host=host, port=port,
+                                                   credentials=credentials,
+                                                   ssl=ssl,
+                                                   connection_attempts=3,
+                                                   retry_delay=5,
+                                                   socket_timeout=1)
+            connection = pika.BlockingConnection(parameters)
+            assert connection.server_properties['product'] == 'RabbitMQ'
+            self.log.debug('Connect OK')
+            return connection
+        except Exception as e:
+            msg = ('amqp connection failed to {}:{} as '
+                   '{} ({})'.format(host, port, username, str(e)))
+            if fatal:
+                amulet.raise_status(amulet.FAIL, msg)
+            else:
+                self.log.warn(msg)
+                return None
+
+    def publish_amqp_message_by_unit(self, sentry_unit, message,
+                                     queue="test", ssl=False,
+                                     username="testuser1",
+                                     password="changeme",
+                                     port=None):
+        """Publish an amqp message to a rmq juju unit.
+
+        :param sentry_unit: sentry unit pointer
+        :param message: amqp message string
+        :param queue: message queue, default to test
+        :param username: amqp user name, default to testuser1
+        :param password: amqp user password
+        :param ssl: boolean, default to False
+        :param port: amqp port, use defaults if None
+        :returns: None.  Raises exception if publish failed.
+        """
+        self.log.debug('Publishing message to {} queue:\n{}'.format(queue,
+                                                                    message))
+        connection = self.connect_amqp_by_unit(sentry_unit, ssl=ssl,
+                                               port=port,
+                                               username=username,
+                                               password=password)
+
+        # NOTE(beisner): extra debug here re: pika hang potential:
+        #   https://github.com/pika/pika/issues/297
+        #   https://groups.google.com/forum/#!topic/rabbitmq-users/Ja0iyfF0Szw
+        self.log.debug('Defining channel...')
+        channel = connection.channel()
+        self.log.debug('Declaring queue...')
+        channel.queue_declare(queue=queue, auto_delete=False, durable=True)
+        self.log.debug('Publishing message...')
+        channel.basic_publish(exchange='', routing_key=queue, body=message)
+        self.log.debug('Closing channel...')
+        channel.close()
+        self.log.debug('Closing connection...')
+        connection.close()
+
+    def get_amqp_message_by_unit(self, sentry_unit, queue="test",
+                                 username="testuser1",
+                                 password="changeme",
+                                 ssl=False, port=None):
+        """Get an amqp message from a rmq juju unit.
+
+        :param sentry_unit: sentry unit pointer
+        :param queue: message queue, default to test
+        :param username: amqp user name, default to testuser1
+        :param password: amqp user password
+        :param ssl: boolean, default to False
+        :param port: amqp port, use defaults if None
+        :returns: amqp message body as string.  Raise if get fails.
+        """
+        connection = self.connect_amqp_by_unit(sentry_unit, ssl=ssl,
+                                               port=port,
+                                               username=username,
+                                               password=password)
+        channel = connection.channel()
+        method_frame, _, body = channel.basic_get(queue)
+
+        if method_frame:
+            self.log.debug('Retreived message from {} queue:\n{}'.format(queue,
+                                                                         body))
+            channel.basic_ack(method_frame.delivery_tag)
+            channel.close()
+            connection.close()
+            return body
+        else:
+            msg = 'No message retrieved.'
+            amulet.raise_status(amulet.FAIL, msg)

=== removed directory 'tests/charmhelpers/contrib/ssl'
=== removed file 'tests/charmhelpers/contrib/ssl/__init__.py'
--- tests/charmhelpers/contrib/ssl/__init__.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/contrib/ssl/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,94 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import subprocess
-from charmhelpers.core import hookenv
-
-
-def generate_selfsigned(keyfile, certfile, keysize="1024", config=None, subject=None, cn=None):
-    """Generate selfsigned SSL keypair
-
-    You must provide one of the 3 optional arguments:
-    config, subject or cn
-    If more than one is provided the leftmost will be used
-
-    Arguments:
-    keyfile -- (required) full path to the keyfile to be created
-    certfile -- (required) full path to the certfile to be created
-    keysize -- (optional) SSL key length
-    config -- (optional) openssl configuration file
-    subject -- (optional) dictionary with SSL subject variables
-    cn -- (optional) cerfificate common name
-
-    Required keys in subject dict:
-    cn -- Common name (eq. FQDN)
-
-    Optional keys in subject dict
-    country -- Country Name (2 letter code)
-    state -- State or Province Name (full name)
-    locality -- Locality Name (eg, city)
-    organization -- Organization Name (eg, company)
-    organizational_unit -- Organizational Unit Name (eg, section)
-    email -- Email Address
-    """
-
-    cmd = []
-    if config:
-        cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
-               "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
-               "-keyout", keyfile,
-               "-out", certfile, "-config", config]
-    elif subject:
-        ssl_subject = ""
-        if "country" in subject:
-            ssl_subject = ssl_subject + "/C={}".format(subject["country"])
-        if "state" in subject:
-            ssl_subject = ssl_subject + "/ST={}".format(subject["state"])
-        if "locality" in subject:
-            ssl_subject = ssl_subject + "/L={}".format(subject["locality"])
-        if "organization" in subject:
-            ssl_subject = ssl_subject + "/O={}".format(subject["organization"])
-        if "organizational_unit" in subject:
-            ssl_subject = ssl_subject + "/OU={}".format(subject["organizational_unit"])
-        if "cn" in subject:
-            ssl_subject = ssl_subject + "/CN={}".format(subject["cn"])
-        else:
-            hookenv.log("When using \"subject\" argument you must "
-                        "provide \"cn\" field at very least")
-            return False
-        if "email" in subject:
-            ssl_subject = ssl_subject + "/emailAddress={}".format(subject["email"])
-
-        cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
-               "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
-               "-keyout", keyfile,
-               "-out", certfile, "-subj", ssl_subject]
-    elif cn:
-        cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
-               "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
-               "-keyout", keyfile,
-               "-out", certfile, "-subj", "/CN={}".format(cn)]
-
-    if not cmd:
-        hookenv.log("No config, subject or cn provided,"
-                    "unable to generate self signed SSL certificates")
-        return False
-    try:
-        subprocess.check_call(cmd)
-        return True
-    except Exception as e:
-        print("Execution of openssl command failed:\n{}".format(e))
-        return False

=== removed file 'tests/charmhelpers/contrib/ssl/service.py'
--- tests/charmhelpers/contrib/ssl/service.py	2015-04-16 21:35:24 +0000
+++ tests/charmhelpers/contrib/ssl/service.py	1970-01-01 00:00:00 +0000
@@ -1,279 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-from os.path import join as path_join
-from os.path import exists
-import subprocess
-
-from charmhelpers.core.hookenv import log, DEBUG
-
-STD_CERT = "standard"
-
-# Mysql server is fairly picky about cert creation
-# and types, spec its creation separately for now.
-MYSQL_CERT = "mysql"
-
-
-class ServiceCA(object):
-
-    default_expiry = str(365 * 2)
-    default_ca_expiry = str(365 * 6)
-
-    def __init__(self, name, ca_dir, cert_type=STD_CERT):
-        self.name = name
-        self.ca_dir = ca_dir
-        self.cert_type = cert_type
-
-    ###############
-    # Hook Helper API
-    @staticmethod
-    def get_ca(type=STD_CERT):
-        service_name = os.environ['JUJU_UNIT_NAME'].split('/')[0]
-        ca_path = os.path.join(os.environ['CHARM_DIR'], 'ca')
-        ca = ServiceCA(service_name, ca_path, type)
-        ca.init()
-        return ca
-
-    @classmethod
-    def get_service_cert(cls, type=STD_CERT):
-        service_name = os.environ['JUJU_UNIT_NAME'].split('/')[0]
-        ca = cls.get_ca()
-        crt, key = ca.get_or_create_cert(service_name)
-        return crt, key, ca.get_ca_bundle()
-
-    ###############
-
-    def init(self):
-        log("initializing service ca", level=DEBUG)
-        if not exists(self.ca_dir):
-            self._init_ca_dir(self.ca_dir)
-            self._init_ca()
-
-    @property
-    def ca_key(self):
-        return path_join(self.ca_dir, 'private', 'cacert.key')
-
-    @property
-    def ca_cert(self):
-        return path_join(self.ca_dir, 'cacert.pem')
-
-    @property
-    def ca_conf(self):
-        return path_join(self.ca_dir, 'ca.cnf')
-
-    @property
-    def signing_conf(self):
-        return path_join(self.ca_dir, 'signing.cnf')
-
-    def _init_ca_dir(self, ca_dir):
-        os.mkdir(ca_dir)
-        for i in ['certs', 'crl', 'newcerts', 'private']:
-            sd = path_join(ca_dir, i)
-            if not exists(sd):
-                os.mkdir(sd)
-
-        if not exists(path_join(ca_dir, 'serial')):
-            with open(path_join(ca_dir, 'serial'), 'w') as fh:
-                fh.write('02\n')
-
-        if not exists(path_join(ca_dir, 'index.txt')):
-            with open(path_join(ca_dir, 'index.txt'), 'w') as fh:
-                fh.write('')
-
-    def _init_ca(self):
-        """Generate the root ca's cert and key.
-        """
-        if not exists(path_join(self.ca_dir, 'ca.cnf')):
-            with open(path_join(self.ca_dir, 'ca.cnf'), 'w') as fh:
-                fh.write(
-                    CA_CONF_TEMPLATE % (self.get_conf_variables()))
-
-        if not exists(path_join(self.ca_dir, 'signing.cnf')):
-            with open(path_join(self.ca_dir, 'signing.cnf'), 'w') as fh:
-                fh.write(
-                    SIGNING_CONF_TEMPLATE % (self.get_conf_variables()))
-
-        if exists(self.ca_cert) or exists(self.ca_key):
-            raise RuntimeError("Initialized called when CA already exists")
-        cmd = ['openssl', 'req', '-config', self.ca_conf,
-               '-x509', '-nodes', '-newkey', 'rsa',
-               '-days', self.default_ca_expiry,
-               '-keyout', self.ca_key, '-out', self.ca_cert,
-               '-outform', 'PEM']
-        output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
-        log("CA Init:\n %s" % output, level=DEBUG)
-
-    def get_conf_variables(self):
-        return dict(
-            org_name="juju",
-            org_unit_name="%s service" % self.name,
-            common_name=self.name,
-            ca_dir=self.ca_dir)
-
-    def get_or_create_cert(self, common_name):
-        if common_name in self:
-            return self.get_certificate(common_name)
-        return self.create_certificate(common_name)
-
-    def create_certificate(self, common_name):
-        if common_name in self:
-            return self.get_certificate(common_name)
-        key_p = path_join(self.ca_dir, "certs", "%s.key" % common_name)
-        crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
-        csr_p = path_join(self.ca_dir, "certs", "%s.csr" % common_name)
-        self._create_certificate(common_name, key_p, csr_p, crt_p)
-        return self.get_certificate(common_name)
-
-    def get_certificate(self, common_name):
-        if common_name not in self:
-            raise ValueError("No certificate for %s" % common_name)
-        key_p = path_join(self.ca_dir, "certs", "%s.key" % common_name)
-        crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
-        with open(crt_p) as fh:
-            crt = fh.read()
-        with open(key_p) as fh:
-            key = fh.read()
-        return crt, key
-
-    def __contains__(self, common_name):
-        crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
-        return exists(crt_p)
-
-    def _create_certificate(self, common_name, key_p, csr_p, crt_p):
-        template_vars = self.get_conf_variables()
-        template_vars['common_name'] = common_name
-        subj = '/O=%(org_name)s/OU=%(org_unit_name)s/CN=%(common_name)s' % (
-            template_vars)
-
-        log("CA Create Cert %s" % common_name, level=DEBUG)
-        cmd = ['openssl', 'req', '-sha1', '-newkey', 'rsa:2048',
-               '-nodes', '-days', self.default_expiry,
-               '-keyout', key_p, '-out', csr_p, '-subj', subj]
-        subprocess.check_call(cmd, stderr=subprocess.PIPE)
-        cmd = ['openssl', 'rsa', '-in', key_p, '-out', key_p]
-        subprocess.check_call(cmd, stderr=subprocess.PIPE)
-
-        log("CA Sign Cert %s" % common_name, level=DEBUG)
-        if self.cert_type == MYSQL_CERT:
-            cmd = ['openssl', 'x509', '-req',
-                   '-in', csr_p, '-days', self.default_expiry,
-                   '-CA', self.ca_cert, '-CAkey', self.ca_key,
-                   '-set_serial', '01', '-out', crt_p]
-        else:
-            cmd = ['openssl', 'ca', '-config', self.signing_conf,
-                   '-extensions', 'req_extensions',
-                   '-days', self.default_expiry, '-notext',
-                   '-in', csr_p, '-out', crt_p, '-subj', subj, '-batch']
-        log("running %s" % " ".join(cmd), level=DEBUG)
-        subprocess.check_call(cmd, stderr=subprocess.PIPE)
-
-    def get_ca_bundle(self):
-        with open(self.ca_cert) as fh:
-            return fh.read()
-
-
-CA_CONF_TEMPLATE = """
-[ ca ]
-default_ca = CA_default
-
-[ CA_default ]
-dir                     = %(ca_dir)s
-policy                  = policy_match
-database                = $dir/index.txt
-serial                  = $dir/serial
-certs                   = $dir/certs
-crl_dir                 = $dir/crl
-new_certs_dir           = $dir/newcerts
-certificate             = $dir/cacert.pem
-private_key             = $dir/private/cacert.key
-RANDFILE                = $dir/private/.rand
-default_md              = default
-
-[ req ]
-default_bits            = 1024
-default_md              = sha1
-
-prompt                  = no
-distinguished_name      = ca_distinguished_name
-
-x509_extensions         = ca_extensions
-
-[ ca_distinguished_name ]
-organizationName        = %(org_name)s
-organizationalUnitName  = %(org_unit_name)s Certificate Authority
-
-
-[ policy_match ]
-countryName             = optional
-stateOrProvinceName     = optional
-organizationName        = match
-organizationalUnitName  = optional
-commonName              = supplied
-
-[ ca_extensions ]
-basicConstraints        = critical,CA:true
-subjectKeyIdentifier    = hash
-authorityKeyIdentifier  = keyid:always, issuer
-keyUsage                = cRLSign, keyCertSign
-"""
-
-
-SIGNING_CONF_TEMPLATE = """
-[ ca ]
-default_ca = CA_default
-
-[ CA_default ]
-dir                     = %(ca_dir)s
-policy                  = policy_match
-database                = $dir/index.txt
-serial                  = $dir/serial
-certs                   = $dir/certs
-crl_dir                 = $dir/crl
-new_certs_dir           = $dir/newcerts
-certificate             = $dir/cacert.pem
-private_key             = $dir/private/cacert.key
-RANDFILE                = $dir/private/.rand
-default_md              = default
-
-[ req ]
-default_bits            = 1024
-default_md              = sha1
-
-prompt                  = no
-distinguished_name      = req_distinguished_name
-
-x509_extensions         = req_extensions
-
-[ req_distinguished_name ]
-organizationName        = %(org_name)s
-organizationalUnitName  = %(org_unit_name)s machine resources
-commonName              = %(common_name)s
-
-[ policy_match ]
-countryName             = optional
-stateOrProvinceName     = optional
-organizationName        = match
-organizationalUnitName  = optional
-commonName              = supplied
-
-[ req_extensions ]
-basicConstraints        = CA:false
-subjectKeyIdentifier    = hash
-authorityKeyIdentifier  = keyid:always, issuer
-keyUsage                = digitalSignature, keyEncipherment, keyAgreement
-extendedKeyUsage        = serverAuth, clientAuth
-"""

=== removed directory 'tests/charmhelpers/core'
=== removed file 'tests/charmhelpers/core/__init__.py'
--- tests/charmhelpers/core/__init__.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/core/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,15 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== removed file 'tests/charmhelpers/core/decorators.py'
--- tests/charmhelpers/core/decorators.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/core/decorators.py	1970-01-01 00:00:00 +0000
@@ -1,57 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-#
-# Copyright 2014 Canonical Ltd.
-#
-# Authors:
-#  Edward Hope-Morley <[e-mail]>
-#
-
-import time
-
-from charmhelpers.core.hookenv import (
-    log,
-    INFO,
-)
-
-
-def retry_on_exception(num_retries, base_delay=0, exc_type=Exception):
-    """If the decorated function raises exception exc_type, allow num_retries
-    retry attempts before raise the exception.
-    """
-    def _retry_on_exception_inner_1(f):
-        def _retry_on_exception_inner_2(*args, **kwargs):
-            retries = num_retries
-            multiplier = 1
-            while True:
-                try:
-                    return f(*args, **kwargs)
-                except exc_type:
-                    if not retries:
-                        raise
-
-                delay = base_delay * multiplier
-                multiplier += 1
-                log("Retrying '%s' %d more times (delay=%s)" %
-                    (f.__name__, retries, delay), level=INFO)
-                retries -= 1
-                if delay:
-                    time.sleep(delay)
-
-        return _retry_on_exception_inner_2
-
-    return _retry_on_exception_inner_1

=== removed file 'tests/charmhelpers/core/files.py'
--- tests/charmhelpers/core/files.py	2015-07-29 10:48:05 +0000
+++ tests/charmhelpers/core/files.py	1970-01-01 00:00:00 +0000
@@ -1,45 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-__author__ = 'Jorge Niedbalski <[e-mail]>'
-
-import os
-import subprocess
-
-
-def sed(filename, before, after, flags='g'):
-    """
-    Search and replaces the given pattern on filename.
-
-    :param filename: relative or absolute file path.
-    :param before: expression to be replaced (see 'man sed')
-    :param after: expression to replace with (see 'man sed')
-    :param flags: sed-compatible regex flags in example, to make
-    the  search and replace case insensitive, specify ``flags="i"``.
-    The ``g`` flag is always specified regardless, so you do not
-    need to remember to include it when overriding this parameter.
-    :returns: If the sed command exit code was zero then return,
-    otherwise raise CalledProcessError.
-    """
-    expression = r's/{0}/{1}/{2}'.format(before,
-                                         after, flags)
-
-    return subprocess.check_call(["sed", "-i", "-r", "-e",
-                                  expression,
-                                  os.path.expanduser(filename)])

=== removed file 'tests/charmhelpers/core/fstab.py'
--- tests/charmhelpers/core/fstab.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/core/fstab.py	1970-01-01 00:00:00 +0000
@@ -1,134 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import io
-import os
-
-__author__ = 'Jorge Niedbalski R. <[e-mail]>'
-
-
-class Fstab(io.FileIO):
-    """This class extends file in order to implement a file reader/writer
-    for file `/etc/fstab`
-    """
-
-    class Entry(object):
-        """Entry class represents a non-comment line on the `/etc/fstab` file
-        """
-        def __init__(self, device, mountpoint, filesystem,
-                     options, d=0, p=0):
-            self.device = device
-            self.mountpoint = mountpoint
-            self.filesystem = filesystem
-
-            if not options:
-                options = "defaults"
-
-            self.options = options
-            self.d = int(d)
-            self.p = int(p)
-
-        def __eq__(self, o):
-            return str(self) == str(o)
-
-        def __str__(self):
-            return "{} {} {} {} {} {}".format(self.device,
-                                              self.mountpoint,
-                                              self.filesystem,
-                                              self.options,
-                                              self.d,
-                                              self.p)
-
-    DEFAULT_PATH = os.path.join(os.path.sep, 'etc', 'fstab')
-
-    def __init__(self, path=None):
-        if path:
-            self._path = path
-        else:
-            self._path = self.DEFAULT_PATH
-        super(Fstab, self).__init__(self._path, 'rb+')
-
-    def _hydrate_entry(self, line):
-        # NOTE: use split with no arguments to split on any
-        #       whitespace including tabs
-        return Fstab.Entry(*filter(
-            lambda x: x not in ('', None),
-            line.strip("\n").split()))
-
-    @property
-    def entries(self):
-        self.seek(0)
-        for line in self.readlines():
-            line = line.decode('us-ascii')
-            try:
-                if line.strip() and not line.strip().startswith("#"):
-                    yield self._hydrate_entry(line)
-            except ValueError:
-                pass
-
-    def get_entry_by_attr(self, attr, value):
-        for entry in self.entries:
-            e_attr = getattr(entry, attr)
-            if e_attr == value:
-                return entry
-        return None
-
-    def add_entry(self, entry):
-        if self.get_entry_by_attr('device', entry.device):
-            return False
-
-        self.write((str(entry) + '\n').encode('us-ascii'))
-        self.truncate()
-        return entry
-
-    def remove_entry(self, entry):
-        self.seek(0)
-
-        lines = [l.decode('us-ascii') for l in self.readlines()]
-
-        found = False
-        for index, line in enumerate(lines):
-            if line.strip() and not line.strip().startswith("#"):
-                if self._hydrate_entry(line) == entry:
-                    found = True
-                    break
-
-        if not found:
-            return False
-
-        lines.remove(line)
-
-        self.seek(0)
-        self.write(''.join(lines).encode('us-ascii'))
-        self.truncate()
-        return True
-
-    @classmethod
-    def remove_by_mountpoint(cls, mountpoint, path=None):
-        fstab = cls(path=path)
-        entry = fstab.get_entry_by_attr('mountpoint', mountpoint)
-        if entry:
-            return fstab.remove_entry(entry)
-        return False
-
-    @classmethod
-    def add(cls, device, mountpoint, filesystem, options=None, path=None):
-        return cls(path=path).add_entry(Fstab.Entry(device,
-                                                    mountpoint, filesystem,
-                                                    options=options))

=== removed file 'tests/charmhelpers/core/hookenv.py'
--- tests/charmhelpers/core/hookenv.py	2015-09-03 09:41:42 +0000
+++ tests/charmhelpers/core/hookenv.py	1970-01-01 00:00:00 +0000
@@ -1,898 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-"Interactions with the Juju environment"
-# Copyright 2013 Canonical Ltd.
-#
-# Authors:
-#  Charm Helpers Developers <[e-mail]>
-
-from __future__ import print_function
-import copy
-from distutils.version import LooseVersion
-from functools import wraps
-import glob
-import os
-import json
-import yaml
-import subprocess
-import sys
-import errno
-import tempfile
-from subprocess import CalledProcessError
-
-import six
-if not six.PY3:
-    from UserDict import UserDict
-else:
-    from collections import UserDict
-
-CRITICAL = "CRITICAL"
-ERROR = "ERROR"
-WARNING = "WARNING"
-INFO = "INFO"
-DEBUG = "DEBUG"
-MARKER = object()
-
-cache = {}
-
-
-def cached(func):
-    """Cache return values for multiple executions of func + args
-
-    For example::
-
-        @cached
-        def unit_get(attribute):
-            pass
-
-        unit_get('test')
-
-    will cache the result of unit_get + 'test' for future calls.
-    """
-    @wraps(func)
-    def wrapper(*args, **kwargs):
-        global cache
-        key = str((func, args, kwargs))
-        try:
-            return cache[key]
-        except KeyError:
-            pass  # Drop out of the exception handler scope.
-        res = func(*args, **kwargs)
-        cache[key] = res
-        return res
-    wrapper._wrapped = func
-    return wrapper
-
-
-def flush(key):
-    """Flushes any entries from function cache where the
-    key is found in the function+args """
-    flush_list = []
-    for item in cache:
-        if key in item:
-            flush_list.append(item)
-    for item in flush_list:
-        del cache[item]
-
-
-def log(message, level=None):
-    """Write a message to the juju log"""
-    command = ['juju-log']
-    if level:
-        command += ['-l', level]
-    if not isinstance(message, six.string_types):
-        message = repr(message)
-    command += [message]
-    # Missing juju-log should not cause failures in unit tests
-    # Send log output to stderr
-    try:
-        subprocess.call(command)
-    except OSError as e:
-        if e.errno == errno.ENOENT:
-            if level:
-                message = "{}: {}".format(level, message)
-            message = "juju-log: {}".format(message)
-            print(message, file=sys.stderr)
-        else:
-            raise
-
-
-class Serializable(UserDict):
-    """Wrapper, an object that can be serialized to yaml or json"""
-
-    def __init__(self, obj):
-        # wrap the object
-        UserDict.__init__(self)
-        self.data = obj
-
-    def __getattr__(self, attr):
-        # See if this object has attribute.
-        if attr in ("json", "yaml", "data"):
-            return self.__dict__[attr]
-        # Check for attribute in wrapped object.
-        got = getattr(self.data, attr, MARKER)
-        if got is not MARKER:
-            return got
-        # Proxy to the wrapped object via dict interface.
-        try:
-            return self.data[attr]
-        except KeyError:
-            raise AttributeError(attr)
-
-    def __getstate__(self):
-        # Pickle as a standard dictionary.
-        return self.data
-
-    def __setstate__(self, state):
-        # Unpickle into our wrapper.
-        self.data = state
-
-    def json(self):
-        """Serialize the object to json"""
-        return json.dumps(self.data)
-
-    def yaml(self):
-        """Serialize the object to yaml"""
-        return yaml.dump(self.data)
-
-
-def execution_environment():
-    """A convenient bundling of the current execution context"""
-    context = {}
-    context['conf'] = config()
-    if relation_id():
-        context['reltype'] = relation_type()
-        context['relid'] = relation_id()
-        context['rel'] = relation_get()
-    context['unit'] = local_unit()
-    context['rels'] = relations()
-    context['env'] = os.environ
-    return context
-
-
-def in_relation_hook():
-    """Determine whether we're running in a relation hook"""
-    return 'JUJU_RELATION' in os.environ
-
-
-def relation_type():
-    """The scope for the current relation hook"""
-    return os.environ.get('JUJU_RELATION', None)
-
-
-@cached
-def relation_id(relation_name=None, service_or_unit=None):
-    """The relation ID for the current or a specified relation"""
-    if not relation_name and not service_or_unit:
-        return os.environ.get('JUJU_RELATION_ID', None)
-    elif relation_name and service_or_unit:
-        service_name = service_or_unit.split('/')[0]
-        for relid in relation_ids(relation_name):
-            remote_service = remote_service_name(relid)
-            if remote_service == service_name:
-                return relid
-    else:
-        raise ValueError('Must specify neither or both of relation_name and service_or_unit')
-
-
-def local_unit():
-    """Local unit ID"""
-    return os.environ['JUJU_UNIT_NAME']
-
-
-def remote_unit():
-    """The remote unit for the current relation hook"""
-    return os.environ.get('JUJU_REMOTE_UNIT', None)
-
-
-def service_name():
-    """The name service group this unit belongs to"""
-    return local_unit().split('/')[0]
-
-
-@cached
-def remote_service_name(relid=None):
-    """The remote service name for a given relation-id (or the current relation)"""
-    if relid is None:
-        unit = remote_unit()
-    else:
-        units = related_units(relid)
-        unit = units[0] if units else None
-    return unit.split('/')[0] if unit else None
-
-
-def hook_name():
-    """The name of the currently executing hook"""
-    return os.environ.get('JUJU_HOOK_NAME', os.path.basename(sys.argv[0]))
-
-
-class Config(dict):
-    """A dictionary representation of the charm's config.yaml, with some
-    extra features:
-
-    - See which values in the dictionary have changed since the previous hook.
-    - For values that have changed, see what the previous value was.
-    - Store arbitrary data for use in a later hook.
-
-    NOTE: Do not instantiate this object directly - instead call
-    ``hookenv.config()``, which will return an instance of :class:`Config`.
-
-    Example usage::
-
-        >>> # inside a hook
-        >>> from charmhelpers.core import hookenv
-        >>> config = hookenv.config()
-        >>> config['foo']
-        'bar'
-        >>> # store a new key/value for later use
-        >>> config['mykey'] = 'myval'
-
-
-        >>> # user runs `juju set mycharm foo=baz`
-        >>> # now we're inside subsequent config-changed hook
-        >>> config = hookenv.config()
-        >>> config['foo']
-        'baz'
-        >>> # test to see if this val has changed since last hook
-        >>> config.changed('foo')
-        True
-        >>> # what was the previous value?
-        >>> config.previous('foo')
-        'bar'
-        >>> # keys/values that we add are preserved across hooks
-        >>> config['mykey']
-        'myval'
-
-    """
-    CONFIG_FILE_NAME = '.juju-persistent-config'
-
-    def __init__(self, *args, **kw):
-        super(Config, self).__init__(*args, **kw)
-        self.implicit_save = True
-        self._prev_dict = None
-        self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME)
-        if os.path.exists(self.path):
-            self.load_previous()
-        atexit(self._implicit_save)
-
-    def load_previous(self, path=None):
-        """Load previous copy of config from disk.
-
-        In normal usage you don't need to call this method directly - it
-        is called automatically at object initialization.
-
-        :param path:
-
-            File path from which to load the previous config. If `None`,
-            config is loaded from the default location. If `path` is
-            specified, subsequent `save()` calls will write to the same
-            path.
-
-        """
-        self.path = path or self.path
-        with open(self.path) as f:
-            self._prev_dict = json.load(f)
-        for k, v in copy.deepcopy(self._prev_dict).items():
-            if k not in self:
-                self[k] = v
-
-    def changed(self, key):
-        """Return True if the current value for this key is different from
-        the previous value.
-
-        """
-        if self._prev_dict is None:
-            return True
-        return self.previous(key) != self.get(key)
-
-    def previous(self, key):
-        """Return previous value for this key, or None if there
-        is no previous value.
-
-        """
-        if self._prev_dict:
-            return self._prev_dict.get(key)
-        return None
-
-    def save(self):
-        """Save this config to disk.
-
-        If the charm is using the :mod:`Services Framework <services.base>`
-        or :meth:'@hook <Hooks.hook>' decorator, this
-        is called automatically at the end of successful hook execution.
-        Otherwise, it should be called directly by user code.
-
-        To disable automatic saves, set ``implicit_save=False`` on this
-        instance.
-
-        """
-        with open(self.path, 'w') as f:
-            json.dump(self, f)
-
-    def _implicit_save(self):
-        if self.implicit_save:
-            self.save()
-
-
-@cached
-def config(scope=None):
-    """Juju charm configuration"""
-    config_cmd_line = ['config-get']
-    if scope is not None:
-        config_cmd_line.append(scope)
-    config_cmd_line.append('--format=json')
-    try:
-        config_data = json.loads(
-            subprocess.check_output(config_cmd_line).decode('UTF-8'))
-        if scope is not None:
-            return config_data
-        return Config(config_data)
-    except ValueError:
-        return None
-
-
-@cached
-def relation_get(attribute=None, unit=None, rid=None):
-    """Get relation information"""
-    _args = ['relation-get', '--format=json']
-    if rid:
-        _args.append('-r')
-        _args.append(rid)
-    _args.append(attribute or '-')
-    if unit:
-        _args.append(unit)
-    try:
-        return json.loads(subprocess.check_output(_args).decode('UTF-8'))
-    except ValueError:
-        return None
-    except CalledProcessError as e:
-        if e.returncode == 2:
-            return None
-        raise
-
-
-def relation_set(relation_id=None, relation_settings=None, **kwargs):
-    """Set relation information for the current unit"""
-    relation_settings = relation_settings if relation_settings else {}
-    relation_cmd_line = ['relation-set']
-    accepts_file = "--file" in subprocess.check_output(
-        relation_cmd_line + ["--help"], universal_newlines=True)
-    if relation_id is not None:
-        relation_cmd_line.extend(('-r', relation_id))
-    settings = relation_settings.copy()
-    settings.update(kwargs)
-    for key, value in settings.items():
-        # Force value to be a string: it always should, but some call
-        # sites pass in things like dicts or numbers.
-        if value is not None:
-            settings[key] = "{}".format(value)
-    if accepts_file:
-        # --file was introduced in Juju 1.23.2. Use it by default if
-        # available, since otherwise we'll break if the relation data is
-        # too big. Ideally we should tell relation-set to read the data from
-        # stdin, but that feature is broken in 1.23.2: Bug #1454678.
-        with tempfile.NamedTemporaryFile(delete=False) as settings_file:
-            settings_file.write(yaml.safe_dump(settings).encode("utf-8"))
-        subprocess.check_call(
-            relation_cmd_line + ["--file", settings_file.name])
-        os.remove(settings_file.name)
-    else:
-        for key, value in settings.items():
-            if value is None:
-                relation_cmd_line.append('{}='.format(key))
-            else:
-                relation_cmd_line.append('{}={}'.format(key, value))
-        subprocess.check_call(relation_cmd_line)
-    # Flush cache of any relation-gets for local unit
-    flush(local_unit())
-
-
-def relation_clear(r_id=None):
-    ''' Clears any relation data already set on relation r_id '''
-    settings = relation_get(rid=r_id,
-                            unit=local_unit())
-    for setting in settings:
-        if setting not in ['public-address', 'private-address']:
-            settings[setting] = None
-    relation_set(relation_id=r_id,
-                 **settings)
-
-
-@cached
-def relation_ids(reltype=None):
-    """A list of relation_ids"""
-    reltype = reltype or relation_type()
-    relid_cmd_line = ['relation-ids', '--format=json']
-    if reltype is not None:
-        relid_cmd_line.append(reltype)
-        return json.loads(
-            subprocess.check_output(relid_cmd_line).decode('UTF-8')) or []
-    return []
-
-
-@cached
-def related_units(relid=None):
-    """A list of related units"""
-    relid = relid or relation_id()
-    units_cmd_line = ['relation-list', '--format=json']
-    if relid is not None:
-        units_cmd_line.extend(('-r', relid))
-    return json.loads(
-        subprocess.check_output(units_cmd_line).decode('UTF-8')) or []
-
-
-@cached
-def relation_for_unit(unit=None, rid=None):
-    """Get the json represenation of a unit's relation"""
-    unit = unit or remote_unit()
-    relation = relation_get(unit=unit, rid=rid)
-    for key in relation:
-        if key.endswith('-list'):
-            relation[key] = relation[key].split()
-    relation['__unit__'] = unit
-    return relation
-
-
-@cached
-def relations_for_id(relid=None):
-    """Get relations of a specific relation ID"""
-    relation_data = []
-    relid = relid or relation_ids()
-    for unit in related_units(relid):
-        unit_data = relation_for_unit(unit, relid)
-        unit_data['__relid__'] = relid
-        relation_data.append(unit_data)
-    return relation_data
-
-
-@cached
-def relations_of_type(reltype=None):
-    """Get relations of a specific type"""
-    relation_data = []
-    reltype = reltype or relation_type()
-    for relid in relation_ids(reltype):
-        for relation in relations_for_id(relid):
-            relation['__relid__'] = relid
-            relation_data.append(relation)
-    return relation_data
-
-
-@cached
-def metadata():
-    """Get the current charm metadata.yaml contents as a python object"""
-    with open(os.path.join(charm_dir(), 'metadata.yaml')) as md:
-        return yaml.safe_load(md)
-
-
-@cached
-def relation_types():
-    """Get a list of relation types supported by this charm"""
-    rel_types = []
-    md = metadata()
-    for key in ('provides', 'requires', 'peers'):
-        section = md.get(key)
-        if section:
-            rel_types.extend(section.keys())
-    return rel_types
-
-
-@cached
-def relation_to_interface(relation_name):
-    """
-    Given the name of a relation, return the interface that relation uses.
-
-    :returns: The interface name, or ``None``.
-    """
-    return relation_to_role_and_interface(relation_name)[1]
-
-
-@cached
-def relation_to_role_and_interface(relation_name):
-    """
-    Given the name of a relation, return the role and the name of the interface
-    that relation uses (where role is one of ``provides``, ``requires``, or ``peer``).
-
-    :returns: A tuple containing ``(role, interface)``, or ``(None, None)``.
-    """
-    _metadata = metadata()
-    for role in ('provides', 'requires', 'peer'):
-        interface = _metadata.get(role, {}).get(relation_name, {}).get('interface')
-        if interface:
-            return role, interface
-    return None, None
-
-
-@cached
-def role_and_interface_to_relations(role, interface_name):
-    """
-    Given a role and interface name, return a list of relation names for the
-    current charm that use that interface under that role (where role is one
-    of ``provides``, ``requires``, or ``peer``).
-
-    :returns: A list of relation names.
-    """
-    _metadata = metadata()
-    results = []
-    for relation_name, relation in _metadata.get(role, {}).items():
-        if relation['interface'] == interface_name:
-            results.append(relation_name)
-    return results
-
-
-@cached
-def interface_to_relations(interface_name):
-    """
-    Given an interface, return a list of relation names for the current
-    charm that use that interface.
-
-    :returns: A list of relation names.
-    """
-    results = []
-    for role in ('provides', 'requires', 'peer'):
-        results.extend(role_and_interface_to_relations(role, interface_name))
-    return results
-
-
-@cached
-def charm_name():
-    """Get the name of the current charm as is specified on metadata.yaml"""
-    return metadata().get('name')
-
-
-@cached
-def relations():
-    """Get a nested dictionary of relation data for all related units"""
-    rels = {}
-    for reltype in relation_types():
-        relids = {}
-        for relid in relation_ids(reltype):
-            units = {local_unit(): relation_get(unit=local_unit(), rid=relid)}
-            for unit in related_units(relid):
-                reldata = relation_get(unit=unit, rid=relid)
-                units[unit] = reldata
-            relids[relid] = units
-        rels[reltype] = relids
-    return rels
-
-
-@cached
-def is_relation_made(relation, keys='private-address'):
-    '''
-    Determine whether a relation is established by checking for
-    presence of key(s).  If a list of keys is provided, they
-    must all be present for the relation to be identified as made
-    '''
-    if isinstance(keys, str):
-        keys = [keys]
-    for r_id in relation_ids(relation):
-        for unit in related_units(r_id):
-            context = {}
-            for k in keys:
-                context[k] = relation_get(k, rid=r_id,
-                                          unit=unit)
-            if None not in context.values():
-                return True
-    return False
-
-
-def open_port(port, protocol="TCP"):
-    """Open a service network port"""
-    _args = ['open-port']
-    _args.append('{}/{}'.format(port, protocol))
-    subprocess.check_call(_args)
-
-
-def close_port(port, protocol="TCP"):
-    """Close a service network port"""
-    _args = ['close-port']
-    _args.append('{}/{}'.format(port, protocol))
-    subprocess.check_call(_args)
-
-
-@cached
-def unit_get(attribute):
-    """Get the unit ID for the remote unit"""
-    _args = ['unit-get', '--format=json', attribute]
-    try:
-        return json.loads(subprocess.check_output(_args).decode('UTF-8'))
-    except ValueError:
-        return None
-
-
-def unit_public_ip():
-    """Get this unit's public IP address"""
-    return unit_get('public-address')
-
-
-def unit_private_ip():
-    """Get this unit's private IP address"""
-    return unit_get('private-address')
-
-
-class UnregisteredHookError(Exception):
-    """Raised when an undefined hook is called"""
-    pass
-
-
-class Hooks(object):
-    """A convenient handler for hook functions.
-
-    Example::
-
-        hooks = Hooks()
-
-        # register a hook, taking its name from the function name
-        @hooks.hook()
-        def install():
-            pass  # your code here
-
-        # register a hook, providing a custom hook name
-        @hooks.hook("config-changed")
-        def config_changed():
-            pass  # your code here
-
-        if __name__ == "__main__":
-            # execute a hook based on the name the program is called by
-            hooks.execute(sys.argv)
-    """
-
-    def __init__(self, config_save=None):
-        super(Hooks, self).__init__()
-        self._hooks = {}
-
-        # For unknown reasons, we allow the Hooks constructor to override
-        # config().implicit_save.
-        if config_save is not None:
-            config().implicit_save = config_save
-
-    def register(self, name, function):
-        """Register a hook"""
-        self._hooks[name] = function
-
-    def execute(self, args):
-        """Execute a registered hook based on args[0]"""
-        _run_atstart()
-        hook_name = os.path.basename(args[0])
-        if hook_name in self._hooks:
-            try:
-                self._hooks[hook_name]()
-            except SystemExit as x:
-                if x.code is None or x.code == 0:
-                    _run_atexit()
-                raise
-            _run_atexit()
-        else:
-            raise UnregisteredHookError(hook_name)
-
-    def hook(self, *hook_names):
-        """Decorator, registering them as hooks"""
-        def wrapper(decorated):
-            for hook_name in hook_names:
-                self.register(hook_name, decorated)
-            else:
-                self.register(decorated.__name__, decorated)
-                if '_' in decorated.__name__:
-                    self.register(
-                        decorated.__name__.replace('_', '-'), decorated)
-            return decorated
-        return wrapper
-
-
-def charm_dir():
-    """Return the root directory of the current charm"""
-    return os.environ.get('CHARM_DIR')
-
-
-@cached
-def action_get(key=None):
-    """Gets the value of an action parameter, or all key/value param pairs"""
-    cmd = ['action-get']
-    if key is not None:
-        cmd.append(key)
-    cmd.append('--format=json')
-    action_data = json.loads(subprocess.check_output(cmd).decode('UTF-8'))
-    return action_data
-
-
-def action_set(values):
-    """Sets the values to be returned after the action finishes"""
-    cmd = ['action-set']
-    for k, v in list(values.items()):
-        cmd.append('{}={}'.format(k, v))
-    subprocess.check_call(cmd)
-
-
-def action_fail(message):
-    """Sets the action status to failed and sets the error message.
-
-    The results set by action_set are preserved."""
-    subprocess.check_call(['action-fail', message])
-
-
-def action_name():
-    """Get the name of the currently executing action."""
-    return os.environ.get('JUJU_ACTION_NAME')
-
-
-def action_uuid():
-    """Get the UUID of the currently executing action."""
-    return os.environ.get('JUJU_ACTION_UUID')
-
-
-def action_tag():
-    """Get the tag for the currently executing action."""
-    return os.environ.get('JUJU_ACTION_TAG')
-
-
-def status_set(workload_state, message):
-    """Set the workload state with a message
-
-    Use status-set to set the workload state with a message which is visible
-    to the user via juju status. If the status-set command is not found then
-    assume this is juju < 1.23 and juju-log the message unstead.
-
-    workload_state -- valid juju workload state.
-    message        -- status update message
-    """
-    valid_states = ['maintenance', 'blocked', 'waiting', 'active']
-    if workload_state not in valid_states:
-        raise ValueError(
-            '{!r} is not a valid workload state'.format(workload_state)
-        )
-    cmd = ['status-set', workload_state, message]
-    try:
-        ret = subprocess.call(cmd)
-        if ret == 0:
-            return
-    except OSError as e:
-        if e.errno != errno.ENOENT:
-            raise
-    log_message = 'status-set failed: {} {}'.format(workload_state,
-                                                    message)
-    log(log_message, level='INFO')
-
-
-def status_get():
-    """Retrieve the previously set juju workload state and message
-
-    If the status-get command is not found then assume this is juju < 1.23 and
-    return 'unknown', ""
-
-    """
-    cmd = ['status-get', "--format=json", "--include-data"]
-    try:
-        raw_status = subprocess.check_output(cmd)
-    except OSError as e:
-        if e.errno == errno.ENOENT:
-            return ('unknown', "")
-        else:
-            raise
-    else:
-        status = json.loads(raw_status.decode("UTF-8"))
-        return (status["status"], status["message"])
-
-
-def translate_exc(from_exc, to_exc):
-    def inner_translate_exc1(f):
-        def inner_translate_exc2(*args, **kwargs):
-            try:
-                return f(*args, **kwargs)
-            except from_exc:
-                raise to_exc
-
-        return inner_translate_exc2
-
-    return inner_translate_exc1
-
-
-@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
-def is_leader():
-    """Does the current unit hold the juju leadership
-
-    Uses juju to determine whether the current unit is the leader of its peers
-    """
-    cmd = ['is-leader', '--format=json']
-    return json.loads(subprocess.check_output(cmd).decode('UTF-8'))
-
-
-@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
-def leader_get(attribute=None):
-    """Juju leader get value(s)"""
-    cmd = ['leader-get', '--format=json'] + [attribute or '-']
-    return json.loads(subprocess.check_output(cmd).decode('UTF-8'))
-
-
-@translate_exc(from_exc=OSError, to_exc=NotImplementedError)
-def leader_set(settings=None, **kwargs):
-    """Juju leader set value(s)"""
-    # Don't log secrets.
-    # log("Juju leader-set '%s'" % (settings), level=DEBUG)
-    cmd = ['leader-set']
-    settings = settings or {}
-    settings.update(kwargs)
-    for k, v in settings.items():
-        if v is None:
-            cmd.append('{}='.format(k))
-        else:
-            cmd.append('{}={}'.format(k, v))
-    subprocess.check_call(cmd)
-
-
-@cached
-def juju_version():
-    """Full version string (eg. '1.23.3.1-trusty-amd64')"""
-    # Per https://bugs.launchpad.net/juju-core/+bug/1455368/comments/1
-    jujud = glob.glob('/var/lib/juju/tools/machine-*/jujud')[0]
-    return subprocess.check_output([jujud, 'version'],
-                                   universal_newlines=True).strip()
-
-
-@cached
-def has_juju_version(minimum_version):
-    """Return True if the Juju version is at least the provided version"""
-    return LooseVersion(juju_version()) >= LooseVersion(minimum_version)
-
-
-_atexit = []
-_atstart = []
-
-
-def atstart(callback, *args, **kwargs):
-    '''Schedule a callback to run before the main hook.
-
-    Callbacks are run in the order they were added.
-
-    This is useful for modules and classes to perform initialization
-    and inject behavior. In particular:
-
-        - Run common code before all of your hooks, such as logging
-          the hook name or interesting relation data.
-        - Defer object or module initialization that requires a hook
-          context until we know there actually is a hook context,
-          making testing easier.
-        - Rather than requiring charm authors to include boilerplate to
-          invoke your helper's behavior, have it run automatically if
-          your object is instantiated or module imported.
-
-    This is not at all useful after your hook framework as been launched.
-    '''
-    global _atstart
-    _atstart.append((callback, args, kwargs))
-
-
-def atexit(callback, *args, **kwargs):
-    '''Schedule a callback to run on successful hook completion.
-
-    Callbacks are run in the reverse order that they were added.'''
-    _atexit.append((callback, args, kwargs))
-
-
-def _run_atstart():
-    '''Hook frameworks must invoke this before running the main hook body.'''
-    global _atstart
-    for callback, args, kwargs in _atstart:
-        callback(*args, **kwargs)
-    del _atstart[:]
-
-
-def _run_atexit():
-    '''Hook frameworks must invoke this after the main hook body has
-    successfully completed. Do not invoke it if the hook fails.'''
-    global _atexit
-    for callback, args, kwargs in reversed(_atexit):
-        callback(*args, **kwargs)
-    del _atexit[:]

=== removed file 'tests/charmhelpers/core/host.py'
--- tests/charmhelpers/core/host.py	2015-08-19 13:49:53 +0000
+++ tests/charmhelpers/core/host.py	1970-01-01 00:00:00 +0000
@@ -1,570 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-"""Tools for working with the host system"""
-# Copyright 2012 Canonical Ltd.
-#
-# Authors:
-#  Nick Moffitt <[e-mail]>
-#  Matthew Wedgwood <[e-mail]>
-
-import os
-import re
-import pwd