Merge lp:~1chb1n/charms/trusty/rabbitmq-server/amulet-refactor-1508 into lp:~openstack-charmers-archive/charms/trusty/rabbitmq-server/next

[Ryan Beisner]

** Do Not Merge, replaced by MP:
https://code.launchpad.net/~1chb1n/charms/trusty/rabbitmq-server/amulet-refactor-1509b/+merge/270102

For a clean break from growing conflict due to multiple other landings.
**

Original Description:
Refactor amulet tests, deprecate old tests, pull in thedac's cluster race fix and collect_rabbitmq_stats script fix, sync charm-helpers.

Cluster race bug (fix proposed by thedac, merged into this):
https://bugs.launchpad.net/charms/+source/rabbitmq-server/+bug/1486177

PID file bug for Vivid and later (fix landed in rmq/next, rebased into this):
https://bugs.launchpad.net/charms/+source/rabbitmq-server/+bug/1485722

This charm merge proposal is dependent on charm-helpers mp @:
https://code.launchpad.net/~1chb1n/charm-helpers/amulet-rmq-helpers

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8491 rabbitmq-server-next for 1chb1n mp268811
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8491/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #7882 rabbitmq-server-next for 1chb1n mp268811
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7882/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #5952 rabbitmq-server-next for 1chb1n mp268811
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12146373/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5952/

[Ryan Beisner (1chb1n)]

In amulet test 5952, all combos except Vivid-Kilo pass all tests. All combos, including V-K cluster ok. The V-K fail is a packaging issue (bug 1485722).

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8541 rabbitmq-server-next for 1chb1n mp268811
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8541/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #5953 rabbitmq-server-next for 1chb1n mp268811
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
ERROR subprocess encountered error code 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12148380/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5953/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #7930 rabbitmq-server-next for 1chb1n mp268811
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7930/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #5958 rabbitmq-server-next for 1chb1n mp268811
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
2015-08-22 06:21:18,640 run_cmd_unit DEBUG: rabbitmq-server/2 `bash -c "$(egrep -oh /usr/local.* /etc/nagios/nrpe.d/check_rabbitmq.cfg)"` command returned 0 (OK)
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12148852/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5958/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8039 rabbitmq-server-next for 1chb1n mp268811
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8039/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8706 rabbitmq-server-next for 1chb1n mp268811
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8706/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6029 rabbitmq-server-next for 1chb1n mp268811
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
2015-08-25 20:07:58,933 publish_amqp_message_by_unit DEBUG: Publishing message to test queue:
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12194886/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6029/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8089 rabbitmq-server-next for 1chb1n mp268811
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8089/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8758 rabbitmq-server-next for 1chb1n mp268811
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8758/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8763 rabbitmq-server-next for 1chb1n mp268811
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8763/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8094 rabbitmq-server-next for 1chb1n mp268811
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8094/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6041 rabbitmq-server-next for 1chb1n mp268811
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6041/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8814 rabbitmq-server-next for 1chb1n mp268811
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8814/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8142 rabbitmq-server-next for 1chb1n mp268811
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8142/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6043 rabbitmq-server-next for 1chb1n mp268811
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6043/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6044 rabbitmq-server-next for 1chb1n mp268811
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12203258/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6044/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6047 rabbitmq-server-next for 1chb1n mp268811
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6047/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8817 rabbitmq-server-next for 1chb1n mp268811
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8817/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8145 rabbitmq-server-next for 1chb1n mp268811
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8145/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6049 rabbitmq-server-next for 1chb1n mp268811
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
2015-08-27 04:27:53,775 _test_rmq_amqp_messages_all_units DEBUG: Publish message to: 172.17.107.154 (rabbitmq-server/1 juju-osci-sv07-machine-3)
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12203625/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6049/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6052 rabbitmq-server-next for 1chb1n mp268811
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
MESSAGE 7@172.17.117.126 [62A5E9AD-3A96-4865-B303-DC431ECF9D43-1440653092.55]
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12203737/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6052/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8828 rabbitmq-server-next for 1chb1n mp268811
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8828/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8156 rabbitmq-server-next for 1chb1n mp268811
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8156/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6062 rabbitmq-server-next for 1chb1n mp268811
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
2015-08-27 16:27:34,911 connect_amqp_by_unit DEBUG: Connecting to amqp on 172.17.105.60:5999 (rabbitmq-server/0) as testuser1...
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12207067/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6062/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8251 rabbitmq-server-next for 1chb1n mp268811
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8251/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8932 rabbitmq-server-next for 1chb1n mp268811
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8932/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8409 rabbitmq-server-next for 1chb1n mp268811
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8409/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9100 rabbitmq-server-next for 1chb1n mp268811
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9100/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9152 rabbitmq-server-next for 1chb1n mp268811
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9152/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8458 rabbitmq-server-next for 1chb1n mp268811
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8458/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6163 rabbitmq-server-next for 1chb1n mp268811
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6163/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6164 rabbitmq-server-next for 1chb1n mp268811
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6164/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8464 rabbitmq-server-next for 1chb1n mp269749
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8464/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9158 rabbitmq-server-next for 1chb1n mp269749
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9158/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6170 rabbitmq-server-next for 1chb1n mp269749
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6170/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9160 rabbitmq-server-next for 1chb1n mp269749
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9160/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8466 rabbitmq-server-next for 1chb1n mp269749
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8466/

[lp:~1chb1n/charms/trusty/rabbitmq-server/amulet-refactor-1508]

122. By Ryan Beisner on 2015-09-01

resync tests/charmhelpers for lint fixes

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8468 rabbitmq-server-next for 1chb1n mp269749
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8468/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9166 rabbitmq-server-next for 1chb1n mp269749
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9166/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9167 rabbitmq-server-next for 1chb1n mp269749
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [.venv] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/12246735/
Build: http://10.245.162.77:8080/job/charm_lint_check/9167/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9168 rabbitmq-server-next for 1chb1n mp269749
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [.venv] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/12246906/
Build: http://10.245.162.77:8080/job/charm_lint_check/9168/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8472 rabbitmq-server-next for 1chb1n mp269749
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
make: *** [.venv] Error 1
ERROR:root:Make target returned non-zero.

Full unit test output: http://paste.ubuntu.com/12246907/
Build: http://10.245.162.77:8080/job/charm_unit_test/8472/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9169 rabbitmq-server-next for 1chb1n mp269749
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9169/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6175 rabbitmq-server-next for 1chb1n mp269749
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6175/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9170 rabbitmq-server-next for 1chb1n mp269749
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9170/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8473 rabbitmq-server-next for 1chb1n mp269749
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8473/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8474 rabbitmq-server-next for 1chb1n mp269749
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8474/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6176 rabbitmq-server-next for 1chb1n mp269749
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12247170/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6176/

[lp:~1chb1n/charms/trusty/rabbitmq-server/amulet-refactor-1508]

123. By Ryan Beisner on 2015-09-01

resync tests/charmhelpers for review changes

[Ryan Beisner (1chb1n)]

Ignore amulet fail #6176, it was an undercloud issue:
01:12:36.848 juju-test.conductor.015-basic-trusty-icehouse DEBUG : Killed by timeout after 2700 seconds

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8475 rabbitmq-server-next for 1chb1n mp269749
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8475/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9171 rabbitmq-server-next for 1chb1n mp269749
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9171/

[David Ames (thedac)]

Great work adding such thorough test suite.

I have one statement and one question.

I would argue that the new tests encompass all that the old tests covered and more and therefore, keeping them in deprecated is unneeded.

My only question is if we are being too openstack-centric for a general purpose charm like rabbitmq. The only difference that icehouse, juno, kilo, liberty makes is really on the cinder deploy. In this sense we are testing cinder more than rabbitmq. Now I agree we need this test coverage for openstack but is the rabbitmq-server amulet tests the place to do it? This is truly a question as I don't know.

[Ryan Beisner (1chb1n)]

Thank you for the review.

# re: Old Tests
There is still one old test which is not represented in the refactored tests, because the deployment topology is completely different than what we're exercising here. It is the non-native rmq cluster with ceph and hacluster relations. I'd like to keep the deprecated tests as reference while I /or someone/ refactors that into a mojo spec. Undertaker beware, as all of the original tests' bugs remain there (such as cluster member checks not checking cluster members, races in ordering, etc.).

I wouldn't be against removing that dir, but also didn't want to lose track of it before we re-captured the intent of that old test. Hence the stash. ;-)

# re: OpenStack vs Generic
I think that the tests are largely rmq-centric in that cluster verification and amqp messaging represent the bulk of the test duration, and 3 of the 5 units deployed are rmq. Cinder is present only to have "something" to exercise the amqp relation of the rabbitmq-server charm, and nrpe for the external-master interface.

The rabbitmq-server charm will receive a lot of benefit from being dragged along the OpenStack charm testing path, in that it will now be exercised against Precise, Trusty, Vivid (and soon Wily), whereas it was only formerly exercised against Trusty in its amulet tests.

For example, while writing these new tests, we identified and resolved a blocker for the rabbitmq-server charm (bug 1485722), where deploys failed on Vivid when nrpe was in the mix. Exercising the supported releases in this way helps us to identify and address issues like this before the next release lands.

...

Thanks again, good questions!

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6180 rabbitmq-server-next for 1chb1n mp269749
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6180/

[lp:~1chb1n/charms/trusty/rabbitmq-server/amulet-refactor-1508]

124. By Ryan Beisner on 2015-09-02

resync tests/charmhelpers

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #8524 rabbitmq-server-next for 1chb1n mp269749
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/8524/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #9224 rabbitmq-server-next for 1chb1n mp269749
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/9224/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6185 rabbitmq-server-next for 1chb1n mp269749
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6185/

[David Ames (thedac)]

Approved on my end

review: Approve

[lp:~1chb1n/charms/trusty/rabbitmq-server/amulet-refactor-1508]

125. By Ryan Beisner on 2015-09-03

revert hooks, pull from rmq/next

Unmerged revisions

125. By Ryan Beisner on 2015-09-03

revert hooks, pull from rmq/next

124. By Ryan Beisner on 2015-09-02

resync tests/charmhelpers

123. By Ryan Beisner on 2015-09-01

resync tests/charmhelpers for review changes

122. By Ryan Beisner on 2015-09-01

resync tests/charmhelpers for lint fixes

121. By Ryan Beisner on 2015-09-01

re-merge lp:~thedac/charms/trusty/rabbitmq-server/native-cluster-race-fixes

120. By Ryan Beisner on 2015-09-01

update comments

119. By Ryan Beisner on 2015-09-01

re-merge lp:~thedac/charms/trusty/rabbitmq-server/native-cluster-race-fixes

118. By Ryan Beisner on 2015-08-31

Re-sync tests/charmhelpers re: updated fix for service_restarted_since bug 1474030

117. By Ryan Beisner on 2015-08-31

 - close connections and channels when done;
 - update debug feedback;
 - fix math on mgmt plugin check.

116. By Ryan Beisner on 2015-08-27

re-merge lp:~thedac/charms/trusty/rabbitmq-server/native-cluster-race-fixes

=== modified file 'Makefile'
--- Makefile	2015-04-20 11:13:39 +0000
+++ Makefile	2015-09-03 18:38:40 +0000
@@ -29,14 +29,15 @@
 	@$(PYTHON) bin/charm_helpers_sync.py -c charm-helpers.yaml
 	@$(PYTHON) bin/charm_helpers_sync.py -c charm-helpers-tests.yaml
 
-publish: lint
+publish: lint test
 	bzr push lp:charms/rabbitmq-server
 	bzr push lp:charms/trusty/rabbitmq-server
 
-unit_test: .venv
+test: .venv
 	@echo Starting tests...
-	env CHARM_DIR=$(CHARM_DIR) $(TEST_PREFIX) .venv/bin/nosetests unit_tests/
+	env CHARM_DIR=$(CHARM_DIR) $(TEST_PREFIX) .venv/bin/nosetests \
+        --nologcapture --with-coverage unit_tests/
 
 functional_test:
 	@echo Starting amulet tests...
-	@juju test -v -p AMULET_HTTP_PROXY,OS_USERNAME,OS_TENANT_NAME,OS_REGION_NAME,OS_PASSWORD,OS_AUTH_URL --timeout 900
+	@juju test -v -p AMULET_HTTP_PROXY,AMULET_OS_VIP --timeout 2700

=== modified file 'charm-helpers-tests.yaml'
--- charm-helpers-tests.yaml	2015-07-31 22:00:49 +0000
+++ charm-helpers-tests.yaml	2015-09-03 18:38:40 +0000
@@ -1,6 +1,5 @@
 destination: tests/charmhelpers
 branch: lp:charm-helpers
 include:
-    - core
-    - cli
-    - contrib.ssl
+    - contrib.amulet
+    - contrib.openstack.amulet

=== modified file 'metadata.yaml'
--- metadata.yaml	2013-11-15 19:15:16 +0000
+++ metadata.yaml	2015-09-03 18:38:40 +0000
@@ -5,7 +5,10 @@
  RabbitMQ is an implementation of AMQP, the emerging standard for high
  performance enterprise messaging. The RabbitMQ server is a robust and
  scalable implementation of an AMQP broker.
-categories: ["misc"]
+tags:
+  - openstack
+  - amqp
+  - misc
 provides:
   amqp:
     interface: rabbitmq

=== added file 'tests/00-setup'
--- tests/00-setup	1970-01-01 00:00:00 +0000
+++ tests/00-setup	2015-09-03 18:38:40 +0000
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+set -ex
+
+sudo add-apt-repository --yes ppa:juju/stable
+sudo apt-get update --yes
+sudo apt-get install --yes python-amulet \
+                           python-cinderclient \
+                           python-distro-info \
+                           python-glanceclient \
+                           python-heatclient \
+                           python-keystoneclient \
+                           python-neutronclient \
+                           python-novaclient \
+                           python-pika \
+                           python-swiftclient

=== added file 'tests/014-basic-precise-icehouse'
--- tests/014-basic-precise-icehouse	1970-01-01 00:00:00 +0000
+++ tests/014-basic-precise-icehouse	2015-09-03 18:38:40 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on precise-icehouse."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='precise',
+                                    openstack='cloud:precise-icehouse',
+                                    source='cloud:precise-updates/icehouse')
+    deployment.run_tests()

=== added file 'tests/015-basic-trusty-icehouse'
--- tests/015-basic-trusty-icehouse	1970-01-01 00:00:00 +0000
+++ tests/015-basic-trusty-icehouse	2015-09-03 18:38:40 +0000
@@ -0,0 +1,9 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on trusty-icehouse."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='trusty')
+    deployment.run_tests()

=== added file 'tests/016-basic-trusty-juno'
--- tests/016-basic-trusty-juno	1970-01-01 00:00:00 +0000
+++ tests/016-basic-trusty-juno	2015-09-03 18:38:40 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on trusty-juno."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='trusty',
+                                    openstack='cloud:trusty-juno',
+                                    source='cloud:trusty-updates/juno')
+    deployment.run_tests()

=== added file 'tests/017-basic-trusty-kilo'
--- tests/017-basic-trusty-kilo	1970-01-01 00:00:00 +0000
+++ tests/017-basic-trusty-kilo	2015-09-03 18:38:40 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on trusty-kilo."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='trusty',
+                                    openstack='cloud:trusty-kilo',
+                                    source='cloud:trusty-updates/kilo')
+    deployment.run_tests()

=== added file 'tests/019-basic-vivid-kilo'
--- tests/019-basic-vivid-kilo	1970-01-01 00:00:00 +0000
+++ tests/019-basic-vivid-kilo	2015-09-03 18:38:40 +0000
@@ -0,0 +1,9 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on vivid-kilo."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='vivid')
+    deployment.run_tests()

=== added file 'tests/020-basic-trusty-liberty'
--- tests/020-basic-trusty-liberty	1970-01-01 00:00:00 +0000
+++ tests/020-basic-trusty-liberty	2015-09-03 18:38:40 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on trusty-liberty."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='trusty',
+                                    openstack='cloud:trusty-liberty',
+                                    source='cloud:trusty-updates/liberty')
+    deployment.run_tests()

=== added file 'tests/021-basic-wily-liberty'
--- tests/021-basic-wily-liberty	1970-01-01 00:00:00 +0000
+++ tests/021-basic-wily-liberty	2015-09-03 18:38:40 +0000
@@ -0,0 +1,9 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic rabbitmq-server deployment on wily-liberty."""
+
+from basic_deployment import RmqBasicDeployment
+
+if __name__ == '__main__':
+    deployment = RmqBasicDeployment(series='wily')
+    deployment.run_tests()

=== added file 'tests/basic_deployment.py'
--- tests/basic_deployment.py	1970-01-01 00:00:00 +0000
+++ tests/basic_deployment.py	2015-09-03 18:38:40 +0000
@@ -0,0 +1,492 @@
+#!/usr/bin/python
+"""
+Basic 3-node rabbitmq-server native cluster + nrpe functional tests
+
+Cinder is present to exercise and inspect amqp relation functionality.
+
+Each individual test is idempotent, in that it creates/deletes
+a rmq test user, enables or disables ssl as needed.
+
+Test order is not required, however tests are numbered to keep
+relevant tests grouped together in run order.
+"""
+
+import amulet
+import time
+
+from charmhelpers.contrib.openstack.amulet.deployment import (
+    OpenStackAmuletDeployment
+)
+
+from charmhelpers.contrib.openstack.amulet.utils import (
+    OpenStackAmuletUtils,
+    DEBUG,
+    # ERROR
+)
+
+# Use DEBUG to turn on debug logging
+u = OpenStackAmuletUtils(DEBUG)
+
+
+class RmqBasicDeployment(OpenStackAmuletDeployment):
+    """Amulet tests on a basic rabbitmq cluster deployment. Verify
+       relations, service status, users and endpoint service catalog."""
+
+    def __init__(self, series=None, openstack=None, source=None, stable=False):
+        """Deploy the entire test environment."""
+        super(RmqBasicDeployment, self).__init__(series, openstack, source,
+                                                 stable)
+        self._add_services()
+        self._add_relations()
+        self._configure_services()
+        self._deploy()
+        self._initialize_tests()
+
+    def _add_services(self):
+        """Add services
+
+           Add the services that we're testing, where rmq is local,
+           and the rest of the service are from lp branches that are
+           compatible with the local charm (e.g. stable or next).
+           """
+        this_service = {
+            'name': 'rabbitmq-server',
+            'units': 3
+        }
+        other_services = [{'name': 'cinder'},
+                          {'name': 'nrpe'}]
+
+        super(RmqBasicDeployment, self)._add_services(this_service,
+                                                      other_services)
+
+    def _add_relations(self):
+        """Add relations for the services."""
+        relations = {'cinder:amqp': 'rabbitmq-server:amqp',
+                     'nrpe:nrpe-external-master':
+                     'rabbitmq-server:nrpe-external-master'}
+
+        super(RmqBasicDeployment, self)._add_relations(relations)
+
+    def _configure_services(self):
+        """Configure all of the services."""
+        rmq_config = {
+            'min-cluster-size': '3',
+            'max-cluster-tries': '6',
+            'ssl': 'off',
+            'management_plugin': 'False',
+            'stats_cron_schedule': '*/1 * * * *'
+        }
+        cinder_config = {}
+        configs = {'rabbitmq-server': rmq_config,
+                   'cinder': cinder_config}
+        super(RmqBasicDeployment, self)._configure_services(configs)
+
+    def _initialize_tests(self):
+        """Perform final initialization before tests get run."""
+        # Access the sentries for inspecting service units
+        self.rmq0_sentry = self.d.sentry.unit['rabbitmq-server/0']
+        self.rmq1_sentry = self.d.sentry.unit['rabbitmq-server/1']
+        self.rmq2_sentry = self.d.sentry.unit['rabbitmq-server/2']
+        self.cinder_sentry = self.d.sentry.unit['cinder/0']
+        self.nrpe_sentry = self.d.sentry.unit['nrpe/0']
+        u.log.debug('openstack release val: {}'.format(
+            self._get_openstack_release()))
+        u.log.debug('openstack release str: {}'.format(
+            self._get_openstack_release_string()))
+
+        # Let things settle a bit before moving forward
+        time.sleep(30)
+
+    def _get_rmq_sentry_units(self):
+        """Local helper specific to this 3-node rmq series of tests."""
+        return [self.rmq0_sentry,
+                self.rmq1_sentry,
+                self.rmq2_sentry]
+
+    def _test_rmq_amqp_messages_all_units(self, sentry_units,
+                                          ssl=False, port=None):
+        """Reusable test to send amqp messages to every listed rmq unit
+        and check every listed rmq unit for messages.
+
+        :param sentry_units: list of sentry units
+        :returns: None if successful.  Raise on error.
+        """
+
+        # Add test user if it does not already exist
+        u.add_rmq_test_user(sentry_units)
+
+        # Handle ssl
+        if ssl:
+            u.configure_rmq_ssl_on(sentry_units, self.d, port=port)
+        else:
+            u.configure_rmq_ssl_off(sentry_units, self.d)
+
+        # Publish and get amqp messages in all possible unit combinations.
+        # Qty of checks == (qty of units) ^ 2
+        amqp_msg_counter = 1
+        host_names = u.get_unit_hostnames(sentry_units)
+
+        for dest_unit in sentry_units:
+            dest_unit_name = dest_unit.info['unit_name']
+            dest_unit_host = dest_unit.info['public-address']
+            dest_unit_host_name = host_names[dest_unit_name]
+
+            for check_unit in sentry_units:
+                check_unit_name = check_unit.info['unit_name']
+                check_unit_host = check_unit.info['public-address']
+                check_unit_host_name = host_names[check_unit_name]
+
+                amqp_msg_stamp = u.get_uuid_epoch_stamp()
+                amqp_msg = ('Message {}@{} {}'.format(amqp_msg_counter,
+                                                      dest_unit_host,
+                                                      amqp_msg_stamp)).upper()
+                # Publish amqp message
+                u.log.debug('Publish message to: {} '
+                            '({} {})'.format(dest_unit_host,
+                                             dest_unit_name,
+                                             dest_unit_host_name))
+
+                u.publish_amqp_message_by_unit(dest_unit,
+                                               amqp_msg, ssl=ssl,
+                                               port=port)
+
+                # Wait a bit before checking for message
+                time.sleep(2)
+
+                # Get amqp message
+                u.log.debug('Get message from:   {} '
+                            '({} {})'.format(check_unit_host,
+                                             check_unit_name,
+                                             check_unit_host_name))
+
+                amqp_msg_rcvd = u.get_amqp_message_by_unit(check_unit,
+                                                           ssl=ssl,
+                                                           port=port)
+
+                # Validate amqp message content
+                if amqp_msg == amqp_msg_rcvd:
+                    u.log.debug('Message {} received '
+                                'OK.'.format(amqp_msg_counter))
+                else:
+                    u.log.error('Expected: {}'.format(amqp_msg))
+                    u.log.error('Actual:   {}'.format(amqp_msg_rcvd))
+                    msg = 'Message {} mismatch.'.format(amqp_msg_counter)
+                    amulet.raise_status(amulet.FAIL, msg)
+
+                amqp_msg_counter += 1
+
+        # Delete the test user
+        u.delete_rmq_test_user(sentry_units)
+
+    def test_100_rmq_processes(self):
+        """Verify that the expected service processes are running
+        on each rabbitmq-server unit."""
+
+        # Beam and epmd sometimes briefly have more than one PID,
+        # True checks for at least 1.
+        rmq_processes = {
+            'beam': True,
+            'epmd': True,
+        }
+
+        # Units with process names and PID quantities expected
+        expected_processes = {
+            self.rmq0_sentry: rmq_processes,
+            self.rmq1_sentry: rmq_processes,
+            self.rmq2_sentry: rmq_processes
+        }
+
+        actual_pids = u.get_unit_process_ids(expected_processes)
+        ret = u.validate_unit_process_ids(expected_processes, actual_pids)
+        if ret:
+            amulet.raise_status(amulet.FAIL, msg=ret)
+
+        u.log.info('OK\n')
+
+    def test_102_services(self):
+        """Verify that the expected services are running on the
+           corresponding service units."""
+        services = {
+            self.rmq0_sentry: ['rabbitmq-server'],
+            self.rmq1_sentry: ['rabbitmq-server'],
+            self.rmq2_sentry: ['rabbitmq-server'],
+            self.cinder_sentry: ['cinder-api',
+                                 'cinder-scheduler',
+                                 'cinder-volume'],
+        }
+        ret = u.validate_services_by_name(services)
+        if ret:
+            amulet.raise_status(amulet.FAIL, msg=ret)
+
+        u.log.info('OK\n')
+
+    def test_200_rmq_cinder_amqp_relation(self):
+        """Verify the rabbitmq-server:cinder amqp relation data"""
+        u.log.debug('Checking rmq:cinder amqp relation data...')
+        unit = self.rmq0_sentry
+        relation = ['amqp', 'cinder:amqp']
+        expected = {
+            'private-address': u.valid_ip,
+            'password': u.not_null,
+            'hostname': u.valid_ip
+        }
+        ret = u.validate_relation_data(unit, relation, expected)
+        if ret:
+            msg = u.relation_error('amqp cinder', ret)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        u.log.info('OK\n')
+
+    def test_201_cinder_rmq_amqp_relation(self):
+        """Verify the cinder:rabbitmq-server amqp relation data"""
+        u.log.debug('Checking cinder:rmq amqp relation data...')
+        unit = self.cinder_sentry
+        relation = ['amqp', 'rabbitmq-server:amqp']
+        expected = {
+            'private-address': u.valid_ip,
+            'vhost': 'openstack',
+            'username': u.not_null
+        }
+        ret = u.validate_relation_data(unit, relation, expected)
+        if ret:
+            msg = u.relation_error('cinder amqp', ret)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        u.log.info('OK\n')
+
+    def test_202_rmq_nrpe_ext_master_relation(self):
+        """Verify rabbitmq-server:nrpe nrpe-external-master relation data"""
+        u.log.debug('Checking rmq:nrpe external master relation data...')
+        unit = self.rmq0_sentry
+        relation = ['nrpe-external-master',
+                    'nrpe:nrpe-external-master']
+
+        mon_sub = ('monitors:\n  remote:\n    nrpe:\n      rabbitmq: '
+                   '{command: check_rabbitmq}\n      rabbitmq_queue: '
+                   '{command: check_rabbitmq_queue}\n')
+
+        expected = {
+            'private-address': u.valid_ip,
+            'monitors': mon_sub
+        }
+
+        ret = u.validate_relation_data(unit, relation, expected)
+        if ret:
+            msg = u.relation_error('amqp nrpe', ret)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        u.log.info('OK\n')
+
+    def test_203_nrpe_rmq_ext_master_relation(self):
+        """Verify nrpe:rabbitmq-server nrpe-external-master relation data"""
+        u.log.debug('Checking nrpe:rmq external master relation data...')
+        unit = self.nrpe_sentry
+        relation = ['nrpe-external-master',
+                    'rabbitmq-server:nrpe-external-master']
+
+        expected = {
+            'private-address': u.valid_ip
+        }
+
+        ret = u.validate_relation_data(unit, relation, expected)
+        if ret:
+            msg = u.relation_error('nrpe amqp', ret)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        u.log.info('OK\n')
+
+    def test_300_rmq_config(self):
+        """Verify the data in the rabbitmq conf file."""
+        conf = '/etc/rabbitmq/rabbitmq-env.conf'
+        sentry_units = self._get_rmq_sentry_units()
+        for unit in sentry_units:
+            host_name = unit.file_contents('/etc/hostname').strip()
+            u.log.debug('Checking rabbitmq config file data on '
+                        '{} ({})...'.format(unit.info['unit_name'],
+                                            host_name))
+            expected = {
+                'RABBITMQ_NODENAME': 'rabbit@{}'.format(host_name)
+            }
+
+            file_contents = unit.file_contents(conf)
+            u.validate_sectionless_conf(file_contents, expected)
+
+        u.log.info('OK\n')
+
+    def test_400_rmq_cluster_running_nodes(self):
+        """Verify that cluster status from each rmq juju unit shows
+        every cluster node as a running member in that cluster."""
+        u.log.debug('Checking that all units are in cluster_status '
+                    'running nodes...')
+
+        sentry_units = self._get_rmq_sentry_units()
+
+        ret = u.validate_rmq_cluster_running_nodes(sentry_units)
+        if ret:
+            amulet.raise_status(amulet.FAIL, msg=ret)
+
+        u.log.info('OK\n')
+
+    def test_402_rmq_connect_with_ssl_off(self):
+        """Verify successful non-ssl amqp connection to all units when
+        charm config option for ssl is set False."""
+        u.log.debug('Confirming that non-ssl connection succeeds when '
+                    'ssl config is off...')
+        sentry_units = self._get_rmq_sentry_units()
+        u.add_rmq_test_user(sentry_units)
+        u.configure_rmq_ssl_off(sentry_units, self.d)
+
+        # Check amqp connection for all units, expect connections to succeed
+        for unit in sentry_units:
+            connection = u.connect_amqp_by_unit(unit, ssl=False, fatal=False)
+            connection.close()
+
+        u.delete_rmq_test_user(sentry_units)
+        u.log.info('OK\n')
+
+    def test_404_rmq_ssl_connect_with_ssl_off(self):
+        """Verify unsuccessful ssl amqp connection to all units when
+        charm config option for ssl is set False."""
+        u.log.debug('Confirming that ssl connection fails when ssl '
+                    'config is off...')
+        sentry_units = self._get_rmq_sentry_units()
+        u.add_rmq_test_user(sentry_units)
+        u.configure_rmq_ssl_off(sentry_units, self.d)
+
+        # Check ssl amqp connection for all units, expect connections to fail
+        for unit in sentry_units:
+            connection = u.connect_amqp_by_unit(unit, ssl=True,
+                                                port=5971, fatal=False)
+            if connection:
+                connection.close()
+                msg = 'SSL connection unexpectedly succeeded with ssl=off'
+                amulet.raise_status(amulet.FAIL, msg)
+
+        u.delete_rmq_test_user(sentry_units)
+        u.log.info('OK - Confirmed that ssl connection attempt fails '
+                   'when ssl config is off.')
+
+    def test_406_rmq_amqp_messages_all_units_ssl_off(self):
+        """Send amqp messages to every rmq unit and check every rmq unit
+        for messages.  Standard amqp tcp port, no ssl."""
+        u.log.debug('Checking amqp message publish/get on all units '
+                    '(ssl off)...')
+
+        sentry_units = self._get_rmq_sentry_units()
+        self._test_rmq_amqp_messages_all_units(sentry_units, ssl=False)
+        u.log.info('OK\n')
+
+    def test_408_rmq_amqp_messages_all_units_ssl_on(self):
+        """Send amqp messages with ssl enabled, to every rmq unit and
+        check every rmq unit for messages.  Standard ssl tcp port."""
+        u.log.debug('Checking amqp message publish/get on all units '
+                    '(ssl on)...')
+
+        sentry_units = self._get_rmq_sentry_units()
+        self._test_rmq_amqp_messages_all_units(sentry_units,
+                                               ssl=True, port=5671)
+        u.log.info('OK\n')
+
+    def test_410_rmq_amqp_messages_all_units_ssl_alt_port(self):
+        """Send amqp messages with ssl on, to every rmq unit and check
+        every rmq unit for messages.  Custom ssl tcp port."""
+        u.log.debug('Checking amqp message publish/get on all units '
+                    '(ssl on)...')
+
+        sentry_units = self._get_rmq_sentry_units()
+        self._test_rmq_amqp_messages_all_units(sentry_units,
+                                               ssl=True, port=5999)
+        u.log.info('OK\n')
+
+    def test_412_rmq_management_plugin(self):
+        """Enable and check management plugin."""
+        u.log.debug('Checking tcp socket connect to management plugin '
+                    'port on all rmq units...')
+
+        sentry_units = self._get_rmq_sentry_units()
+        mgmt_port = 15672
+
+        # Enable management plugin
+        u.log.debug('Enabling management_plugin charm config option...')
+        config = {'management_plugin': 'True'}
+        self.d.configure('rabbitmq-server', config)
+
+        # Check tcp connect to management plugin port
+        max_wait = 120
+        tries = 0
+        ret = u.port_knock_units(sentry_units, mgmt_port)
+        while ret and tries < (max_wait / 12):
+            time.sleep(12)
+            u.log.debug('Attempt {}: {}'.format(tries, ret))
+            ret = u.port_knock_units(sentry_units, mgmt_port)
+            tries += 1
+
+        if ret:
+            amulet.raise_status(amulet.FAIL, ret)
+        else:
+            u.log.debug('Connect to all units (OK)\n')
+
+        # Disable management plugin
+        u.log.debug('Disabling management_plugin charm config option...')
+        config = {'management_plugin': 'False'}
+        self.d.configure('rabbitmq-server', config)
+
+        # Negative check - tcp connect to management plugin port
+        u.log.info('Expect tcp connect fail since charm config '
+                   'option is disabled.')
+        tries = 0
+        ret = u.port_knock_units(sentry_units, mgmt_port, expect_success=False)
+        while ret and tries < (max_wait / 12):
+            time.sleep(12)
+            u.log.debug('Attempt {}: {}'.format(tries, ret))
+            ret = u.port_knock_units(sentry_units, mgmt_port,
+                                     expect_success=False)
+            tries += 1
+
+        if ret:
+            amulet.raise_status(amulet.FAIL, ret)
+        else:
+            u.log.info('Confirm mgmt port closed on all units (OK)\n')
+
+    def test_414_rmq_nrpe_monitors(self):
+        """Check rabbimq-server nrpe monitor basic functionality."""
+        sentry_units = self._get_rmq_sentry_units()
+        host_names = u.get_unit_hostnames(sentry_units)
+
+        # check_rabbitmq monitor
+        u.log.debug('Checking nrpe check_rabbitmq on units...')
+        cmds = ['egrep -oh /usr/local.* /etc/nagios/nrpe.d/'
+                'check_rabbitmq.cfg']
+        ret = u.check_commands_on_units(cmds, sentry_units)
+        if ret:
+            amulet.raise_status(amulet.FAIL, msg=ret)
+
+        u.log.debug('Sleeping 70s for 1m cron job to run...')
+        time.sleep(70)
+
+        # check_rabbitmq_queue monitor
+        u.log.debug('Checking nrpe check_rabbitmq_queue on units...')
+        cmds = ['egrep -oh /usr/local.* /etc/nagios/nrpe.d/'
+                'check_rabbitmq_queue.cfg']
+        ret = u.check_commands_on_units(cmds, sentry_units)
+        if ret:
+            amulet.raise_status(amulet.FAIL, msg=ret)
+
+        # check dat file existence
+        u.log.debug('Checking nrpe dat file existence on units...')
+        for sentry_unit in sentry_units:
+            unit_name = sentry_unit.info['unit_name']
+            unit_host_name = host_names[unit_name]
+
+            cmds = [
+                'stat /var/lib/rabbitmq/data/{}_general_stats.dat'.format(
+                    unit_host_name),
+                'stat /var/lib/rabbitmq/data/{}_queue_stats.dat'.format(
+                    unit_host_name)
+            ]
+
+            ret = u.check_commands_on_units(cmds, [sentry_unit])
+            if ret:
+                amulet.raise_status(amulet.FAIL, msg=ret)
+
+        u.log.info('OK\n')

=== added directory 'tests/charmhelpers'
=== renamed directory 'tests/charmhelpers' => 'tests/charmhelpers.moved'
=== renamed file 'tests/charmhelpers/core/hookenv.py' => 'tests/charmhelpers.moved/core/hookenv.py.THIS'
=== added file 'tests/charmhelpers/__init__.py'
--- tests/charmhelpers/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/__init__.py	2015-09-03 18:38:40 +0000
@@ -0,0 +1,38 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+# Bootstrap charm-helpers, installing its dependencies if necessary using
+# only standard libraries.
+import subprocess
+import sys
+
+try:
+    import six  # flake8: noqa
+except ImportError:
+    if sys.version_info.major == 2:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python-six'])
+    else:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python3-six'])
+    import six  # flake8: noqa
+
+try:
+    import yaml  # flake8: noqa
+except ImportError:
+    if sys.version_info.major == 2:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml'])
+    else:
+        subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml'])
+    import yaml  # flake8: noqa

=== removed file 'tests/charmhelpers/__init__.py'
--- tests/charmhelpers/__init__.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,38 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-# Bootstrap charm-helpers, installing its dependencies if necessary using
-# only standard libraries.
-import subprocess
-import sys
-
-try:
-    import six  # flake8: noqa
-except ImportError:
-    if sys.version_info.major == 2:
-        subprocess.check_call(['apt-get', 'install', '-y', 'python-six'])
-    else:
-        subprocess.check_call(['apt-get', 'install', '-y', 'python3-six'])
-    import six  # flake8: noqa
-
-try:
-    import yaml  # flake8: noqa
-except ImportError:
-    if sys.version_info.major == 2:
-        subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml'])
-    else:
-        subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml'])
-    import yaml  # flake8: noqa

=== removed directory 'tests/charmhelpers/cli'
=== removed file 'tests/charmhelpers/cli/__init__.py'
--- tests/charmhelpers/cli/__init__.py	2015-08-19 13:49:53 +0000
+++ tests/charmhelpers/cli/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,191 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import inspect
-import argparse
-import sys
-
-from six.moves import zip
-
-from charmhelpers.core import unitdata
-
-
-class OutputFormatter(object):
-    def __init__(self, outfile=sys.stdout):
-        self.formats = (
-            "raw",
-            "json",
-            "py",
-            "yaml",
-            "csv",
-            "tab",
-        )
-        self.outfile = outfile
-
-    def add_arguments(self, argument_parser):
-        formatgroup = argument_parser.add_mutually_exclusive_group()
-        choices = self.supported_formats
-        formatgroup.add_argument("--format", metavar='FMT',
-                                 help="Select output format for returned data, "
-                                      "where FMT is one of: {}".format(choices),
-                                 choices=choices, default='raw')
-        for fmt in self.formats:
-            fmtfunc = getattr(self, fmt)
-            formatgroup.add_argument("-{}".format(fmt[0]),
-                                     "--{}".format(fmt), action='store_const',
-                                     const=fmt, dest='format',
-                                     help=fmtfunc.__doc__)
-
-    @property
-    def supported_formats(self):
-        return self.formats
-
-    def raw(self, output):
-        """Output data as raw string (default)"""
-        if isinstance(output, (list, tuple)):
-            output = '\n'.join(map(str, output))
-        self.outfile.write(str(output))
-
-    def py(self, output):
-        """Output data as a nicely-formatted python data structure"""
-        import pprint
-        pprint.pprint(output, stream=self.outfile)
-
-    def json(self, output):
-        """Output data in JSON format"""
-        import json
-        json.dump(output, self.outfile)
-
-    def yaml(self, output):
-        """Output data in YAML format"""
-        import yaml
-        yaml.safe_dump(output, self.outfile)
-
-    def csv(self, output):
-        """Output data as excel-compatible CSV"""
-        import csv
-        csvwriter = csv.writer(self.outfile)
-        csvwriter.writerows(output)
-
-    def tab(self, output):
-        """Output data in excel-compatible tab-delimited format"""
-        import csv
-        csvwriter = csv.writer(self.outfile, dialect=csv.excel_tab)
-        csvwriter.writerows(output)
-
-    def format_output(self, output, fmt='raw'):
-        fmtfunc = getattr(self, fmt)
-        fmtfunc(output)
-
-
-class CommandLine(object):
-    argument_parser = None
-    subparsers = None
-    formatter = None
-    exit_code = 0
-
-    def __init__(self):
-        if not self.argument_parser:
-            self.argument_parser = argparse.ArgumentParser(description='Perform common charm tasks')
-        if not self.formatter:
-            self.formatter = OutputFormatter()
-            self.formatter.add_arguments(self.argument_parser)
-        if not self.subparsers:
-            self.subparsers = self.argument_parser.add_subparsers(help='Commands')
-
-    def subcommand(self, command_name=None):
-        """
-        Decorate a function as a subcommand. Use its arguments as the
-        command-line arguments"""
-        def wrapper(decorated):
-            cmd_name = command_name or decorated.__name__
-            subparser = self.subparsers.add_parser(cmd_name,
-                                                   description=decorated.__doc__)
-            for args, kwargs in describe_arguments(decorated):
-                subparser.add_argument(*args, **kwargs)
-            subparser.set_defaults(func=decorated)
-            return decorated
-        return wrapper
-
-    def test_command(self, decorated):
-        """
-        Subcommand is a boolean test function, so bool return values should be
-        converted to a 0/1 exit code.
-        """
-        decorated._cli_test_command = True
-        return decorated
-
-    def no_output(self, decorated):
-        """
-        Subcommand is not expected to return a value, so don't print a spurious None.
-        """
-        decorated._cli_no_output = True
-        return decorated
-
-    def subcommand_builder(self, command_name, description=None):
-        """
-        Decorate a function that builds a subcommand. Builders should accept a
-        single argument (the subparser instance) and return the function to be
-        run as the command."""
-        def wrapper(decorated):
-            subparser = self.subparsers.add_parser(command_name)
-            func = decorated(subparser)
-            subparser.set_defaults(func=func)
-            subparser.description = description or func.__doc__
-        return wrapper
-
-    def run(self):
-        "Run cli, processing arguments and executing subcommands."
-        arguments = self.argument_parser.parse_args()
-        argspec = inspect.getargspec(arguments.func)
-        vargs = []
-        for arg in argspec.args:
-            vargs.append(getattr(arguments, arg))
-        if argspec.varargs:
-            vargs.extend(getattr(arguments, argspec.varargs))
-        output = arguments.func(*vargs)
-        if getattr(arguments.func, '_cli_test_command', False):
-            self.exit_code = 0 if output else 1
-            output = ''
-        if getattr(arguments.func, '_cli_no_output', False):
-            output = ''
-        self.formatter.format_output(output, arguments.format)
-        if unitdata._KV:
-            unitdata._KV.flush()
-
-
-cmdline = CommandLine()
-
-
-def describe_arguments(func):
-    """
-    Analyze a function's signature and return a data structure suitable for
-    passing in as arguments to an argparse parser's add_argument() method."""
-
-    argspec = inspect.getargspec(func)
-    # we should probably raise an exception somewhere if func includes **kwargs
-    if argspec.defaults:
-        positional_args = argspec.args[:-len(argspec.defaults)]
-        keyword_names = argspec.args[-len(argspec.defaults):]
-        for arg, default in zip(keyword_names, argspec.defaults):
-            yield ('--{}'.format(arg),), {'default': default}
-    else:
-        positional_args = argspec.args
-
-    for arg in positional_args:
-        yield (arg,), {}
-    if argspec.varargs:
-        yield (argspec.varargs,), {'nargs': '*'}

=== removed file 'tests/charmhelpers/cli/benchmark.py'
--- tests/charmhelpers/cli/benchmark.py	2015-07-31 22:00:49 +0000
+++ tests/charmhelpers/cli/benchmark.py	1970-01-01 00:00:00 +0000
@@ -1,36 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from . import cmdline
-from charmhelpers.contrib.benchmark import Benchmark
-
-
[e-mail](command_name='benchmark-start')
-def start():
-    Benchmark.start()
-
-
[e-mail](command_name='benchmark-finish')
-def finish():
-    Benchmark.finish()
-
-
[e-mail]_builder('benchmark-composite', description="Set the benchmark composite score")
-def service(subparser):
-    subparser.add_argument("value", help="The composite score.")
-    subparser.add_argument("units", help="The units the composite score represents, i.e., 'reads/sec'.")
-    subparser.add_argument("direction", help="'asc' if a lower score is better, 'desc' if a higher score is better.")
-    return Benchmark.set_composite_score

=== removed file 'tests/charmhelpers/cli/commands.py'
--- tests/charmhelpers/cli/commands.py	2015-08-19 13:49:53 +0000
+++ tests/charmhelpers/cli/commands.py	1970-01-01 00:00:00 +0000
@@ -1,32 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-"""
-This module loads sub-modules into the python runtime so they can be
-discovered via the inspect module. In order to prevent flake8 from (rightfully)
-telling us these are unused modules, throw a ' # noqa' at the end of each import
-so that the warning is suppressed.
-"""
-
-from . import CommandLine  # noqa
-
-"""
-Import the sub-modules which have decorated subcommands to register with chlp.
-"""
-from . import host  # noqa
-from . import benchmark  # noqa
-from . import unitdata  # noqa
-from . import hookenv  # noqa

=== removed file 'tests/charmhelpers/cli/hookenv.py'
--- tests/charmhelpers/cli/hookenv.py	2015-08-19 13:49:53 +0000
+++ tests/charmhelpers/cli/hookenv.py	1970-01-01 00:00:00 +0000
@@ -1,23 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from . import cmdline
-from charmhelpers.core import hookenv
-
-
-cmdline.subcommand('relation-id')(hookenv.relation_id._wrapped)
-cmdline.subcommand('service-name')(hookenv.service_name)
-cmdline.subcommand('remote-service-name')(hookenv.remote_service_name._wrapped)

=== removed file 'tests/charmhelpers/cli/host.py'
--- tests/charmhelpers/cli/host.py	2015-07-31 22:00:49 +0000
+++ tests/charmhelpers/cli/host.py	1970-01-01 00:00:00 +0000
@@ -1,31 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from . import cmdline
-from charmhelpers.core import host
-
-
[e-mail]()
-def mounts():
-    "List mounts"
-    return host.mounts()
-
-
[e-mail]_builder('service', description="Control system services")
-def service(subparser):
-    subparser.add_argument("action", help="The action to perform (start, stop, etc...)")
-    subparser.add_argument("service_name", help="Name of the service to control")
-    return host.service

=== removed file 'tests/charmhelpers/cli/unitdata.py'
--- tests/charmhelpers/cli/unitdata.py	2015-07-31 22:00:49 +0000
+++ tests/charmhelpers/cli/unitdata.py	1970-01-01 00:00:00 +0000
@@ -1,39 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from . import cmdline
-from charmhelpers.core import unitdata
-
-
[e-mail]_builder('unitdata', description="Store and retrieve data")
-def unitdata_cmd(subparser):
-    nested = subparser.add_subparsers()
-    get_cmd = nested.add_parser('get', help='Retrieve data')
-    get_cmd.add_argument('key', help='Key to retrieve the value of')
-    get_cmd.set_defaults(action='get', value=None)
-    set_cmd = nested.add_parser('set', help='Store data')
-    set_cmd.add_argument('key', help='Key to set')
-    set_cmd.add_argument('value', help='Value to store')
-    set_cmd.set_defaults(action='set')
-
-    def _unitdata_cmd(action, key, value):
-        if action == 'get':
-            return unitdata.kv().get(key)
-        elif action == 'set':
-            unitdata.kv().set(key, value)
-            unitdata.kv().flush()
-            return ''
-    return _unitdata_cmd

=== added directory 'tests/charmhelpers/contrib'
=== removed directory 'tests/charmhelpers/contrib'
=== added file 'tests/charmhelpers/contrib/__init__.py'
--- tests/charmhelpers/contrib/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/__init__.py	2015-09-03 18:38:40 +0000
@@ -0,0 +1,15 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== removed file 'tests/charmhelpers/contrib/__init__.py'
--- tests/charmhelpers/contrib/__init__.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/contrib/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,15 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== added directory 'tests/charmhelpers/contrib/amulet'
=== added file 'tests/charmhelpers/contrib/amulet/__init__.py'
--- tests/charmhelpers/contrib/amulet/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/amulet/__init__.py	2015-09-03 18:38:40 +0000
@@ -0,0 +1,15 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== added file 'tests/charmhelpers/contrib/amulet/deployment.py'
--- tests/charmhelpers/contrib/amulet/deployment.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/amulet/deployment.py	2015-09-03 18:38:40 +0000
@@ -0,0 +1,93 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import amulet
+import os
+import six
+
+
+class AmuletDeployment(object):
+    """Amulet deployment.
+
+       This class provides generic Amulet deployment and test runner
+       methods.
+       """
+
+    def __init__(self, series=None):
+        """Initialize the deployment environment."""
+        self.series = None
+
+        if series:
+            self.series = series
+            self.d = amulet.Deployment(series=self.series)
+        else:
+            self.d = amulet.Deployment()
+
+    def _add_services(self, this_service, other_services):
+        """Add services.
+
+           Add services to the deployment where this_service is the local charm
+           that we're testing and other_services are the other services that
+           are being used in the local amulet tests.
+           """
+        if this_service['name'] != os.path.basename(os.getcwd()):
+            s = this_service['name']
+            msg = "The charm's root directory name needs to be {}".format(s)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        if 'units' not in this_service:
+            this_service['units'] = 1
+
+        self.d.add(this_service['name'], units=this_service['units'])
+
+        for svc in other_services:
+            if 'location' in svc:
+                branch_location = svc['location']
+            elif self.series:
+                branch_location = 'cs:{}/{}'.format(self.series, svc['name']),
+            else:
+                branch_location = None
+
+            if 'units' not in svc:
+                svc['units'] = 1
+
+            self.d.add(svc['name'], charm=branch_location, units=svc['units'])
+
+    def _add_relations(self, relations):
+        """Add all of the relations for the services."""
+        for k, v in six.iteritems(relations):
+            self.d.relate(k, v)
+
+    def _configure_services(self, configs):
+        """Configure all of the services."""
+        for service, config in six.iteritems(configs):
+            self.d.configure(service, config)
+
+    def _deploy(self):
+        """Deploy environment and wait for all hooks to finish executing."""
+        try:
+            self.d.setup(timeout=900)
+            self.d.sentry.wait(timeout=900)
+        except amulet.helpers.TimeoutError:
+            amulet.raise_status(amulet.FAIL, msg="Deployment timed out")
+        except Exception:
+            raise
+
+    def run_tests(self):
+        """Run all of the methods that are prefixed with 'test_'."""
+        for test in dir(self):
+            if test.startswith('test_'):
+                getattr(self, test)()

=== added file 'tests/charmhelpers/contrib/amulet/utils.py'
--- tests/charmhelpers/contrib/amulet/utils.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/amulet/utils.py	2015-09-03 18:38:40 +0000
@@ -0,0 +1,778 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import io
+import json
+import logging
+import os
+import re
+import socket
+import subprocess
+import sys
+import time
+import uuid
+
+import amulet
+import distro_info
+import six
+from six.moves import configparser
+if six.PY3:
+    from urllib import parse as urlparse
+else:
+    import urlparse
+
+
+class AmuletUtils(object):
+    """Amulet utilities.
+
+       This class provides common utility functions that are used by Amulet
+       tests.
+       """
+
+    def __init__(self, log_level=logging.ERROR):
+        self.log = self.get_logger(level=log_level)
+        self.ubuntu_releases = self.get_ubuntu_releases()
+
+    def get_logger(self, name="amulet-logger", level=logging.DEBUG):
+        """Get a logger object that will log to stdout."""
+        log = logging
+        logger = log.getLogger(name)
+        fmt = log.Formatter("%(asctime)s %(funcName)s "
+                            "%(levelname)s: %(message)s")
+
+        handler = log.StreamHandler(stream=sys.stdout)
+        handler.setLevel(level)
+        handler.setFormatter(fmt)
+
+        logger.addHandler(handler)
+        logger.setLevel(level)
+
+        return logger
+
+    def valid_ip(self, ip):
+        if re.match(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$", ip):
+            return True
+        else:
+            return False
+
+    def valid_url(self, url):
+        p = re.compile(
+            r'^(?:http|ftp)s?://'
+            r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}\.?)|'  # noqa
+            r'localhost|'
+            r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'
+            r'(?::\d+)?'
+            r'(?:/?|[/?]\S+)$',
+            re.IGNORECASE)
+        if p.match(url):
+            return True
+        else:
+            return False
+
+    def get_ubuntu_release_from_sentry(self, sentry_unit):
+        """Get Ubuntu release codename from sentry unit.
+
+        :param sentry_unit: amulet sentry/service unit pointer
+        :returns: list of strings - release codename, failure message
+        """
+        msg = None
+        cmd = 'lsb_release -cs'
+        release, code = sentry_unit.run(cmd)
+        if code == 0:
+            self.log.debug('{} lsb_release: {}'.format(
+                sentry_unit.info['unit_name'], release))
+        else:
+            msg = ('{} `{}` returned {} '
+                   '{}'.format(sentry_unit.info['unit_name'],
+                               cmd, release, code))
+        if release not in self.ubuntu_releases:
+            msg = ("Release ({}) not found in Ubuntu releases "
+                   "({})".format(release, self.ubuntu_releases))
+        return release, msg
+
+    def validate_services(self, commands):
+        """Validate that lists of commands succeed on service units.  Can be
+           used to verify system services are running on the corresponding
+           service units.
+
+        :param commands: dict with sentry keys and arbitrary command list vals
+        :returns: None if successful, Failure string message otherwise
+        """
+        self.log.debug('Checking status of system services...')
+
+        # /!\ DEPRECATION WARNING (beisner):
+        # New and existing tests should be rewritten to use
+        # validate_services_by_name() as it is aware of init systems.
+        self.log.warn('DEPRECATION WARNING:  use '
+                      'validate_services_by_name instead of validate_services '
+                      'due to init system differences.')
+
+        for k, v in six.iteritems(commands):
+            for cmd in v:
+                output, code = k.run(cmd)
+                self.log.debug('{} `{}` returned '
+                               '{}'.format(k.info['unit_name'],
+                                           cmd, code))
+                if code != 0:
+                    return "command `{}` returned {}".format(cmd, str(code))
+        return None
+
+    def validate_services_by_name(self, sentry_services):
+        """Validate system service status by service name, automatically
+           detecting init system based on Ubuntu release codename.
+
+        :param sentry_services: dict with sentry keys and svc list values
+        :returns: None if successful, Failure string message otherwise
+        """
+        self.log.debug('Checking status of system services...')
+
+        # Point at which systemd became a thing
+        systemd_switch = self.ubuntu_releases.index('vivid')
+
+        for sentry_unit, services_list in six.iteritems(sentry_services):
+            # Get lsb_release codename from unit
+            release, ret = self.get_ubuntu_release_from_sentry(sentry_unit)
+            if ret:
+                return ret
+
+            for service_name in services_list:
+                if (self.ubuntu_releases.index(release) >= systemd_switch or
+                        service_name in ['rabbitmq-server', 'apache2']):
+                    # init is systemd (or regular sysv)
+                    cmd = 'sudo service {} status'.format(service_name)
+                    output, code = sentry_unit.run(cmd)
+                    service_running = code == 0
+                elif self.ubuntu_releases.index(release) < systemd_switch:
+                    # init is upstart
+                    cmd = 'sudo status {}'.format(service_name)
+                    output, code = sentry_unit.run(cmd)
+                    service_running = code == 0 and "start/running" in output
+
+                self.log.debug('{} `{}` returned '
+                               '{}'.format(sentry_unit.info['unit_name'],
+                                           cmd, code))
+                if not service_running:
+                    return u"command `{}` returned {} {}".format(
+                        cmd, output, str(code))
+        return None
+
+    def _get_config(self, unit, filename):
+        """Get a ConfigParser object for parsing a unit's config file."""
+        file_contents = unit.file_contents(filename)
+
+        # NOTE(beisner):  by default, ConfigParser does not handle options
+        # with no value, such as the flags used in the mysql my.cnf file.
+        # https://bugs.python.org/issue7005
+        config = configparser.ConfigParser(allow_no_value=True)
+        config.readfp(io.StringIO(file_contents))
+        return config
+
+    def validate_config_data(self, sentry_unit, config_file, section,
+                             expected):
+        """Validate config file data.
+
+           Verify that the specified section of the config file contains
+           the expected option key:value pairs.
+
+           Compare expected dictionary data vs actual dictionary data.
+           The values in the 'expected' dictionary can be strings, bools, ints,
+           longs, or can be a function that evaluates a variable and returns a
+           bool.
+           """
+        self.log.debug('Validating config file data ({} in {} on {})'
+                       '...'.format(section, config_file,
+                                    sentry_unit.info['unit_name']))
+        config = self._get_config(sentry_unit, config_file)
+
+        if section != 'DEFAULT' and not config.has_section(section):
+            return "section [{}] does not exist".format(section)
+
+        for k in expected.keys():
+            if not config.has_option(section, k):
+                return "section [{}] is missing option {}".format(section, k)
+
+            actual = config.get(section, k)
+            v = expected[k]
+            if (isinstance(v, six.string_types) or
+                    isinstance(v, bool) or
+                    isinstance(v, six.integer_types)):
+                # handle explicit values
+                if actual != v:
+                    return "section [{}] {}:{} != expected {}:{}".format(
+                           section, k, actual, k, expected[k])
+            # handle function pointers, such as not_null or valid_ip
+            elif not v(actual):
+                return "section [{}] {}:{} != expected {}:{}".format(
+                       section, k, actual, k, expected[k])
+        return None
+
+    def _validate_dict_data(self, expected, actual):
+        """Validate dictionary data.
+
+           Compare expected dictionary data vs actual dictionary data.
+           The values in the 'expected' dictionary can be strings, bools, ints,
+           longs, or can be a function that evaluates a variable and returns a
+           bool.
+           """
+        self.log.debug('actual: {}'.format(repr(actual)))
+        self.log.debug('expected: {}'.format(repr(expected)))
+
+        for k, v in six.iteritems(expected):
+            if k in actual:
+                if (isinstance(v, six.string_types) or
+                        isinstance(v, bool) or
+                        isinstance(v, six.integer_types)):
+                    # handle explicit values
+                    if v != actual[k]:
+                        return "{}:{}".format(k, actual[k])
+                # handle function pointers, such as not_null or valid_ip
+                elif not v(actual[k]):
+                    return "{}:{}".format(k, actual[k])
+            else:
+                return "key '{}' does not exist".format(k)
+        return None
+
+    def validate_relation_data(self, sentry_unit, relation, expected):
+        """Validate actual relation data based on expected relation data."""
+        actual = sentry_unit.relation(relation[0], relation[1])
+        return self._validate_dict_data(expected, actual)
+
+    def _validate_list_data(self, expected, actual):
+        """Compare expected list vs actual list data."""
+        for e in expected:
+            if e not in actual:
+                return "expected item {} not found in actual list".format(e)
+        return None
+
+    def not_null(self, string):
+        if string is not None:
+            return True
+        else:
+            return False
+
+    def _get_file_mtime(self, sentry_unit, filename):
+        """Get last modification time of file."""
+        return sentry_unit.file_stat(filename)['mtime']
+
+    def _get_dir_mtime(self, sentry_unit, directory):
+        """Get last modification time of directory."""
+        return sentry_unit.directory_stat(directory)['mtime']
+
+    def _get_proc_start_time(self, sentry_unit, service, pgrep_full=None):
+        """Get start time of a process based on the last modification time
+           of the /proc/pid directory.
+
+        :sentry_unit:  The sentry unit to check for the service on
+        :service:  service name to look for in process table
+        :pgrep_full:  [Deprecated] Use full command line search mode with pgrep
+        :returns:  epoch time of service process start
+        :param commands:  list of bash commands
+        :param sentry_units:  list of sentry unit pointers
+        :returns:  None if successful; Failure message otherwise
+        """
+        if pgrep_full is not None:
+            # /!\ DEPRECATION WARNING (beisner):
+            # No longer implemented, as pidof is now used instead of pgrep.
+            # https://bugs.launchpad.net/charm-helpers/+bug/1474030
+            self.log.warn('DEPRECATION WARNING:  pgrep_full bool is no '
+                          'longer implemented re: lp 1474030.')
+
+        pid_list = self.get_process_id_list(sentry_unit, service)
+        pid = pid_list[0]
+        proc_dir = '/proc/{}'.format(pid)
+        self.log.debug('Pid for {} on {}: {}'.format(
+            service, sentry_unit.info['unit_name'], pid))
+
+        return self._get_dir_mtime(sentry_unit, proc_dir)
+
+    def service_restarted(self, sentry_unit, service, filename,
+                          pgrep_full=None, sleep_time=20):
+        """Check if service was restarted.
+
+           Compare a service's start time vs a file's last modification time
+           (such as a config file for that service) to determine if the service
+           has been restarted.
+           """
+        # /!\ DEPRECATION WARNING (beisner):
+        # This method is prone to races in that no before-time is known.
+        # Use validate_service_config_changed instead.
+
+        # NOTE(beisner) pgrep_full is no longer implemented, as pidof is now
+        # used instead of pgrep.  pgrep_full is still passed through to ensure
+        # deprecation WARNS.  lp1474030
+        self.log.warn('DEPRECATION WARNING:  use '
+                      'validate_service_config_changed instead of '
+                      'service_restarted due to known races.')
+
+        time.sleep(sleep_time)
+        if (self._get_proc_start_time(sentry_unit, service, pgrep_full) >=
+                self._get_file_mtime(sentry_unit, filename)):
+            return True
+        else:
+            return False
+
+    def service_restarted_since(self, sentry_unit, mtime, service,
+                                pgrep_full=None, sleep_time=20,
+                                retry_count=2, retry_sleep_time=30):
+        """Check if service was been started after a given time.
+
+        Args:
+          sentry_unit (sentry): The sentry unit to check for the service on
+          mtime (float): The epoch time to check against
+          service (string): service name to look for in process table
+          pgrep_full: [Deprecated] Use full command line search mode with pgrep
+          sleep_time (int): Seconds to sleep before looking for process
+          retry_count (int): If service is not found, how many times to retry
+
+        Returns:
+          bool: True if service found and its start time it newer than mtime,
+                False if service is older than mtime or if service was
+                not found.
+        """
+        # NOTE(beisner) pgrep_full is no longer implemented, as pidof is now
+        # used instead of pgrep.  pgrep_full is still passed through to ensure
+        # deprecation WARNS.  lp1474030
+
+        unit_name = sentry_unit.info['unit_name']
+        self.log.debug('Checking that %s service restarted since %s on '
+                       '%s' % (service, mtime, unit_name))
+        time.sleep(sleep_time)
+        proc_start_time = None
+        tries = 0
+        while tries <= retry_count and not proc_start_time:
+            try:
+                proc_start_time = self._get_proc_start_time(sentry_unit,
+                                                            service,
+                                                            pgrep_full)
+                self.log.debug('Attempt {} to get {} proc start time on {} '
+                               'OK'.format(tries, service, unit_name))
+            except IOError:
+                # NOTE(beisner) - race avoidance, proc may not exist yet.
+                # https://bugs.launchpad.net/charm-helpers/+bug/1474030
+                self.log.debug('Attempt {} to get {} proc start time on {} '
+                               'failed'.format(tries, service, unit_name))
+                time.sleep(retry_sleep_time)
+                tries += 1
+
+        if not proc_start_time:
+            self.log.warn('No proc start time found, assuming service did '
+                          'not start')
+            return False
+        if proc_start_time >= mtime:
+            self.log.debug('Proc start time is newer than provided mtime'
+                           '(%s >= %s) on %s (OK)' % (proc_start_time,
+                                                      mtime, unit_name))
+            return True
+        else:
+            self.log.warn('Proc start time (%s) is older than provided mtime '
+                          '(%s) on %s, service did not '
+                          'restart' % (proc_start_time, mtime, unit_name))
+            return False
+
+    def config_updated_since(self, sentry_unit, filename, mtime,
+                             sleep_time=20):
+        """Check if file was modified after a given time.
+
+        Args:
+          sentry_unit (sentry): The sentry unit to check the file mtime on
+          filename (string): The file to check mtime of
+          mtime (float): The epoch time to check against
+          sleep_time (int): Seconds to sleep before looking for process
+
+        Returns:
+          bool: True if file was modified more recently than mtime, False if
+                file was modified before mtime,
+        """
+        self.log.debug('Checking %s updated since %s' % (filename, mtime))
+        time.sleep(sleep_time)
+        file_mtime = self._get_file_mtime(sentry_unit, filename)
+        if file_mtime >= mtime:
+            self.log.debug('File mtime is newer than provided mtime '
+                           '(%s >= %s)' % (file_mtime, mtime))
+            return True
+        else:
+            self.log.warn('File mtime %s is older than provided mtime %s'
+                          % (file_mtime, mtime))
+            return False
+
+    def validate_service_config_changed(self, sentry_unit, mtime, service,
+                                        filename, pgrep_full=None,
+                                        sleep_time=20, retry_count=2,
+                                        retry_sleep_time=30):
+        """Check service and file were updated after mtime
+
+        Args:
+          sentry_unit (sentry): The sentry unit to check for the service on
+          mtime (float): The epoch time to check against
+          service (string): service name to look for in process table
+          filename (string): The file to check mtime of
+          pgrep_full: [Deprecated] Use full command line search mode with pgrep
+          sleep_time (int): Initial sleep in seconds to pass to test helpers
+          retry_count (int): If service is not found, how many times to retry
+          retry_sleep_time (int): Time in seconds to wait between retries
+
+        Typical Usage:
+            u = OpenStackAmuletUtils(ERROR)
+            ...
+            mtime = u.get_sentry_time(self.cinder_sentry)
+            self.d.configure('cinder', {'verbose': 'True', 'debug': 'True'})
+            if not u.validate_service_config_changed(self.cinder_sentry,
+                                                     mtime,
+                                                     'cinder-api',
+                                                     '/etc/cinder/cinder.conf')
+                amulet.raise_status(amulet.FAIL, msg='update failed')
+        Returns:
+          bool: True if both service and file where updated/restarted after
+                mtime, False if service is older than mtime or if service was
+                not found or if filename was modified before mtime.
+        """
+
+        # NOTE(beisner) pgrep_full is no longer implemented, as pidof is now
+        # used instead of pgrep.  pgrep_full is still passed through to ensure
+        # deprecation WARNS.  lp1474030
+
+        service_restart = self.service_restarted_since(
+            sentry_unit, mtime,
+            service,
+            pgrep_full=pgrep_full,
+            sleep_time=sleep_time,
+            retry_count=retry_count,
+            retry_sleep_time=retry_sleep_time)
+
+        config_update = self.config_updated_since(
+            sentry_unit,
+            filename,
+            mtime,
+            sleep_time=0)
+
+        return service_restart and config_update
+
+    def get_sentry_time(self, sentry_unit):
+        """Return current epoch time on a sentry"""
+        cmd = "date +'%s'"
+        return float(sentry_unit.run(cmd)[0])
+
+    def relation_error(self, name, data):
+        return 'unexpected relation data in {} - {}'.format(name, data)
+
+    def endpoint_error(self, name, data):
+        return 'unexpected endpoint data in {} - {}'.format(name, data)
+
+    def get_ubuntu_releases(self):
+        """Return a list of all Ubuntu releases in order of release."""
+        _d = distro_info.UbuntuDistroInfo()
+        _release_list = _d.all
+        return _release_list
+
+    def file_to_url(self, file_rel_path):
+        """Convert a relative file path to a file URL."""
+        _abs_path = os.path.abspath(file_rel_path)
+        return urlparse.urlparse(_abs_path, scheme='file').geturl()
+
+    def check_commands_on_units(self, commands, sentry_units):
+        """Check that all commands in a list exit zero on all
+        sentry units in a list.
+
+        :param commands:  list of bash commands
+        :param sentry_units:  list of sentry unit pointers
+        :returns: None if successful; Failure message otherwise
+        """
+        self.log.debug('Checking exit codes for {} commands on {} '
+                       'sentry units...'.format(len(commands),
+                                                len(sentry_units)))
+        for sentry_unit in sentry_units:
+            for cmd in commands:
+                output, code = sentry_unit.run(cmd)
+                if code == 0:
+                    self.log.debug('{} `{}` returned {} '
+                                   '(OK)'.format(sentry_unit.info['unit_name'],
+                                                 cmd, code))
+                else:
+                    return ('{} `{}` returned {} '
+                            '{}'.format(sentry_unit.info['unit_name'],
+                                        cmd, code, output))
+        return None
+
+    def get_process_id_list(self, sentry_unit, process_name,
+                            expect_success=True):
+        """Get a list of process ID(s) from a single sentry juju unit
+        for a single process name.
+
+        :param sentry_unit: Amulet sentry instance (juju unit)
+        :param process_name: Process name
+        :param expect_success: If False, expect the PID to be missing,
+            raise if it is present.
+        :returns: List of process IDs
+        """
+        cmd = 'pidof -x {}'.format(process_name)
+        if not expect_success:
+            cmd += " || exit 0 && exit 1"
+        output, code = sentry_unit.run(cmd)
+        if code != 0:
+            msg = ('{} `{}` returned {} '
+                   '{}'.format(sentry_unit.info['unit_name'],
+                               cmd, code, output))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+        return str(output).split()
+
+    def get_unit_process_ids(self, unit_processes, expect_success=True):
+        """Construct a dict containing unit sentries, process names, and
+        process IDs.
+
+        :param unit_processes: A dictionary of Amulet sentry instance
+            to list of process names.
+        :param expect_success: if False expect the processes to not be
+            running, raise if they are.
+        :returns: Dictionary of Amulet sentry instance to dictionary
+            of process names to PIDs.
+        """
+        pid_dict = {}
+        for sentry_unit, process_list in six.iteritems(unit_processes):
+            pid_dict[sentry_unit] = {}
+            for process in process_list:
+                pids = self.get_process_id_list(
+                    sentry_unit, process, expect_success=expect_success)
+                pid_dict[sentry_unit].update({process: pids})
+        return pid_dict
+
+    def validate_unit_process_ids(self, expected, actual):
+        """Validate process id quantities for services on units."""
+        self.log.debug('Checking units for running processes...')
+        self.log.debug('Expected PIDs: {}'.format(expected))
+        self.log.debug('Actual PIDs: {}'.format(actual))
+
+        if len(actual) != len(expected):
+            return ('Unit count mismatch.  expected, actual: {}, '
+                    '{} '.format(len(expected), len(actual)))
+
+        for (e_sentry, e_proc_names) in six.iteritems(expected):
+            e_sentry_name = e_sentry.info['unit_name']
+            if e_sentry in actual.keys():
+                a_proc_names = actual[e_sentry]
+            else:
+                return ('Expected sentry ({}) not found in actual dict data.'
+                        '{}'.format(e_sentry_name, e_sentry))
+
+            if len(e_proc_names.keys()) != len(a_proc_names.keys()):
+                return ('Process name count mismatch.  expected, actual: {}, '
+                        '{}'.format(len(expected), len(actual)))
+
+            for (e_proc_name, e_pids_length), (a_proc_name, a_pids) in \
+                    zip(e_proc_names.items(), a_proc_names.items()):
+                if e_proc_name != a_proc_name:
+                    return ('Process name mismatch.  expected, actual: {}, '
+                            '{}'.format(e_proc_name, a_proc_name))
+
+                a_pids_length = len(a_pids)
+                fail_msg = ('PID count mismatch. {} ({}) expected, actual: '
+                            '{}, {} ({})'.format(e_sentry_name, e_proc_name,
+                                                 e_pids_length, a_pids_length,
+                                                 a_pids))
+
+                # If expected is not bool, ensure PID quantities match
+                if not isinstance(e_pids_length, bool) and \
+                        a_pids_length != e_pids_length:
+                    return fail_msg
+                # If expected is bool True, ensure 1 or more PIDs exist
+                elif isinstance(e_pids_length, bool) and \
+                        e_pids_length is True and a_pids_length < 1:
+                    return fail_msg
+                # If expected is bool False, ensure 0 PIDs exist
+                elif isinstance(e_pids_length, bool) and \
+                        e_pids_length is False and a_pids_length != 0:
+                    return fail_msg
+                else:
+                    self.log.debug('PID check OK: {} {} {}: '
+                                   '{}'.format(e_sentry_name, e_proc_name,
+                                               e_pids_length, a_pids))
+        return None
+
+    def validate_list_of_identical_dicts(self, list_of_dicts):
+        """Check that all dicts within a list are identical."""
+        hashes = []
+        for _dict in list_of_dicts:
+            hashes.append(hash(frozenset(_dict.items())))
+
+        self.log.debug('Hashes: {}'.format(hashes))
+        if len(set(hashes)) == 1:
+            self.log.debug('Dicts within list are identical')
+        else:
+            return 'Dicts within list are not identical'
+
+        return None
+
+    def validate_sectionless_conf(self, file_contents, expected):
+        """A crude conf parser.  Useful to inspect configuration files which
+        do not have section headers (as would be necessary in order to use
+        the configparser).  Such as openstack-dashboard or rabbitmq confs."""
+        for line in file_contents.split('\n'):
+            if '=' in line:
+                args = line.split('=')
+                if len(args) <= 1:
+                    continue
+                key = args[0].strip()
+                value = args[1].strip()
+                if key in expected.keys():
+                    if expected[key] != value:
+                        msg = ('Config mismatch.  Expected, actual:  {}, '
+                               '{}'.format(expected[key], value))
+                        amulet.raise_status(amulet.FAIL, msg=msg)
+
+    def get_unit_hostnames(self, units):
+        """Return a dict of juju unit names to hostnames."""
+        host_names = {}
+        for unit in units:
+            host_names[unit.info['unit_name']] = \
+                str(unit.file_contents('/etc/hostname').strip())
+        self.log.debug('Unit host names: {}'.format(host_names))
+        return host_names
+
+    def run_cmd_unit(self, sentry_unit, cmd):
+        """Run a command on a unit, return the output and exit code."""
+        output, code = sentry_unit.run(cmd)
+        if code == 0:
+            self.log.debug('{} `{}` command returned {} '
+                           '(OK)'.format(sentry_unit.info['unit_name'],
+                                         cmd, code))
+        else:
+            msg = ('{} `{}` command returned {} '
+                   '{}'.format(sentry_unit.info['unit_name'],
+                               cmd, code, output))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+        return str(output), code
+
+    def file_exists_on_unit(self, sentry_unit, file_name):
+        """Check if a file exists on a unit."""
+        try:
+            sentry_unit.file_stat(file_name)
+            return True
+        except IOError:
+            return False
+        except Exception as e:
+            msg = 'Error checking file {}: {}'.format(file_name, e)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+    def file_contents_safe(self, sentry_unit, file_name,
+                           max_wait=60, fatal=False):
+        """Get file contents from a sentry unit.  Wrap amulet file_contents
+        with retry logic to address races where a file checks as existing,
+        but no longer exists by the time file_contents is called.
+        Return None if file not found. Optionally raise if fatal is True."""
+        unit_name = sentry_unit.info['unit_name']
+        file_contents = False
+        tries = 0
+        while not file_contents and tries < (max_wait / 4):
+            try:
+                file_contents = sentry_unit.file_contents(file_name)
+            except IOError:
+                self.log.debug('Attempt {} to open file {} from {} '
+                               'failed'.format(tries, file_name,
+                                               unit_name))
+                time.sleep(4)
+                tries += 1
+
+        if file_contents:
+            return file_contents
+        elif not fatal:
+            return None
+        elif fatal:
+            msg = 'Failed to get file contents from unit.'
+            amulet.raise_status(amulet.FAIL, msg)
+
+    def port_knock_tcp(self, host="localhost", port=22, timeout=15):
+        """Open a TCP socket to check for a listening sevice on a host.
+
+        :param host: host name or IP address, default to localhost
+        :param port: TCP port number, default to 22
+        :param timeout: Connect timeout, default to 15 seconds
+        :returns: True if successful, False if connect failed
+        """
+
+        # Resolve host name if possible
+        try:
+            connect_host = socket.gethostbyname(host)
+            host_human = "{} ({})".format(connect_host, host)
+        except socket.error as e:
+            self.log.warn('Unable to resolve address: '
+                          '{} ({}) Trying anyway!'.format(host, e))
+            connect_host = host
+            host_human = connect_host
+
+        # Attempt socket connection
+        try:
+            knock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            knock.settimeout(timeout)
+            knock.connect((connect_host, port))
+            knock.close()
+            self.log.debug('Socket connect OK for host '
+                           '{} on port {}.'.format(host_human, port))
+            return True
+        except socket.error as e:
+            self.log.debug('Socket connect FAIL for'
+                           ' {} port {} ({})'.format(host_human, port, e))
+            return False
+
+    def port_knock_units(self, sentry_units, port=22,
+                         timeout=15, expect_success=True):
+        """Open a TCP socket to check for a listening sevice on each
+        listed juju unit.
+
+        :param sentry_units: list of sentry unit pointers
+        :param port: TCP port number, default to 22
+        :param timeout: Connect timeout, default to 15 seconds
+        :expect_success: True by default, set False to invert logic
+        :returns: None if successful, Failure message otherwise
+        """
+        for unit in sentry_units:
+            host = unit.info['public-address']
+            connected = self.port_knock_tcp(host, port, timeout)
+            if not connected and expect_success:
+                return 'Socket connect failed.'
+            elif connected and not expect_success:
+                return 'Socket connected unexpectedly.'
+
+    def get_uuid_epoch_stamp(self):
+        """Returns a stamp string based on uuid4 and epoch time.  Useful in
+        generating test messages which need to be unique-ish."""
+        return '[{}-{}]'.format(uuid.uuid4(), time.time())
+
+# amulet juju action helpers:
+    def run_action(self, unit_sentry, action,
+                   _check_output=subprocess.check_output):
+        """Run the named action on a given unit sentry.
+
+        _check_output parameter is used for dependency injection.
+
+        @return action_id.
+        """
+        unit_id = unit_sentry.info["unit_name"]
+        command = ["juju", "action", "do", "--format=json", unit_id, action]
+        self.log.info("Running command: %s\n" % " ".join(command))
+        output = _check_output(command, universal_newlines=True)
+        data = json.loads(output)
+        action_id = data[u'Action queued with id']
+        return action_id
+
+    def wait_on_action(self, action_id, _check_output=subprocess.check_output):
+        """Wait for a given action, returning if it completed or not.
+
+        _check_output parameter is used for dependency injection.
+        """
+        command = ["juju", "action", "fetch", "--format=json", "--wait=0",
+                   action_id]
+        output = _check_output(command, universal_newlines=True)
+        data = json.loads(output)
+        return data.get(u"status") == "completed"

=== added directory 'tests/charmhelpers/contrib/openstack'
=== added file 'tests/charmhelpers/contrib/openstack/__init__.py'
--- tests/charmhelpers/contrib/openstack/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/openstack/__init__.py	2015-09-03 18:38:40 +0000
@@ -0,0 +1,15 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== added directory 'tests/charmhelpers/contrib/openstack/amulet'
=== added file 'tests/charmhelpers/contrib/openstack/amulet/__init__.py'
--- tests/charmhelpers/contrib/openstack/amulet/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/openstack/amulet/__init__.py	2015-09-03 18:38:40 +0000
@@ -0,0 +1,15 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== added file 'tests/charmhelpers/contrib/openstack/amulet/deployment.py'
--- tests/charmhelpers/contrib/openstack/amulet/deployment.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/openstack/amulet/deployment.py	2015-09-03 18:38:40 +0000
@@ -0,0 +1,198 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import six
+from collections import OrderedDict
+from charmhelpers.contrib.amulet.deployment import (
+    AmuletDeployment
+)
+
+
+class OpenStackAmuletDeployment(AmuletDeployment):
+    """OpenStack amulet deployment.
+
+       This class inherits from AmuletDeployment and has additional support
+       that is specifically for use by OpenStack charms.
+       """
+
+    def __init__(self, series=None, openstack=None, source=None, stable=True):
+        """Initialize the deployment environment."""
+        super(OpenStackAmuletDeployment, self).__init__(series)
+        self.openstack = openstack
+        self.source = source
+        self.stable = stable
+        # Note(coreycb): this needs to be changed when new next branches come
+        # out.
+        self.current_next = "trusty"
+
+    def _determine_branch_locations(self, other_services):
+        """Determine the branch locations for the other services.
+
+           Determine if the local branch being tested is derived from its
+           stable or next (dev) branch, and based on this, use the corresonding
+           stable or next branches for the other_services."""
+
+        # Charms outside the lp:~openstack-charmers namespace
+        base_charms = ['mysql', 'mongodb', 'nrpe']
+
+        # Force these charms to current series even when using an older series.
+        # ie. Use trusty/nrpe even when series is precise, as the P charm
+        # does not possess the necessary external master config and hooks.
+        force_series_current = ['nrpe']
+
+        if self.series in ['precise', 'trusty']:
+            base_series = self.series
+        else:
+            base_series = self.current_next
+
+        if self.stable:
+            for svc in other_services:
+                if svc['name'] in force_series_current:
+                    base_series = self.current_next
+
+                temp = 'lp:charms/{}/{}'
+                svc['location'] = temp.format(base_series,
+                                              svc['name'])
+        else:
+            for svc in other_services:
+                if svc['name'] in force_series_current:
+                    base_series = self.current_next
+
+                if svc['name'] in base_charms:
+                    temp = 'lp:charms/{}/{}'
+                    svc['location'] = temp.format(base_series,
+                                                  svc['name'])
+                else:
+                    temp = 'lp:~openstack-charmers/charms/{}/{}/next'
+                    svc['location'] = temp.format(self.current_next,
+                                                  svc['name'])
+        return other_services
+
+    def _add_services(self, this_service, other_services):
+        """Add services to the deployment and set openstack-origin/source."""
+        other_services = self._determine_branch_locations(other_services)
+
+        super(OpenStackAmuletDeployment, self)._add_services(this_service,
+                                                             other_services)
+
+        services = other_services
+        services.append(this_service)
+
+        # Charms which should use the source config option
+        use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',
+                      'ceph-osd', 'ceph-radosgw']
+
+        # Charms which can not use openstack-origin, ie. many subordinates
+        no_origin = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe']
+
+        if self.openstack:
+            for svc in services:
+                if svc['name'] not in use_source + no_origin:
+                    config = {'openstack-origin': self.openstack}
+                    self.d.configure(svc['name'], config)
+
+        if self.source:
+            for svc in services:
+                if svc['name'] in use_source and svc['name'] not in no_origin:
+                    config = {'source': self.source}
+                    self.d.configure(svc['name'], config)
+
+    def _configure_services(self, configs):
+        """Configure all of the services."""
+        for service, config in six.iteritems(configs):
+            self.d.configure(service, config)
+
+    def _get_openstack_release(self):
+        """Get openstack release.
+
+           Return an integer representing the enum value of the openstack
+           release.
+           """
+        # Must be ordered by OpenStack release (not by Ubuntu release):
+        (self.precise_essex, self.precise_folsom, self.precise_grizzly,
+         self.precise_havana, self.precise_icehouse,
+         self.trusty_icehouse, self.trusty_juno, self.utopic_juno,
+         self.trusty_kilo, self.vivid_kilo, self.trusty_liberty,
+         self.wily_liberty) = range(12)
+
+        releases = {
+            ('precise', None): self.precise_essex,
+            ('precise', 'cloud:precise-folsom'): self.precise_folsom,
+            ('precise', 'cloud:precise-grizzly'): self.precise_grizzly,
+            ('precise', 'cloud:precise-havana'): self.precise_havana,
+            ('precise', 'cloud:precise-icehouse'): self.precise_icehouse,
+            ('trusty', None): self.trusty_icehouse,
+            ('trusty', 'cloud:trusty-juno'): self.trusty_juno,
+            ('trusty', 'cloud:trusty-kilo'): self.trusty_kilo,
+            ('trusty', 'cloud:trusty-liberty'): self.trusty_liberty,
+            ('utopic', None): self.utopic_juno,
+            ('vivid', None): self.vivid_kilo,
+            ('wily', None): self.wily_liberty}
+        return releases[(self.series, self.openstack)]
+
+    def _get_openstack_release_string(self):
+        """Get openstack release string.
+
+           Return a string representing the openstack release.
+           """
+        releases = OrderedDict([
+            ('precise', 'essex'),
+            ('quantal', 'folsom'),
+            ('raring', 'grizzly'),
+            ('saucy', 'havana'),
+            ('trusty', 'icehouse'),
+            ('utopic', 'juno'),
+            ('vivid', 'kilo'),
+            ('wily', 'liberty'),
+        ])
+        if self.openstack:
+            os_origin = self.openstack.split(':')[1]
+            return os_origin.split('%s-' % self.series)[1].split('/')[0]
+        else:
+            return releases[self.series]
+
+    def get_ceph_expected_pools(self, radosgw=False):
+        """Return a list of expected ceph pools in a ceph + cinder + glance
+        test scenario, based on OpenStack release and whether ceph radosgw
+        is flagged as present or not."""
+
+        if self._get_openstack_release() >= self.trusty_kilo:
+            # Kilo or later
+            pools = [
+                'rbd',
+                'cinder',
+                'glance'
+            ]
+        else:
+            # Juno or earlier
+            pools = [
+                'data',
+                'metadata',
+                'rbd',
+                'cinder',
+                'glance'
+            ]
+
+        if radosgw:
+            pools.extend([
+                '.rgw.root',
+                '.rgw.control',
+                '.rgw',
+                '.rgw.gc',
+                '.users.uid'
+            ])
+
+        return pools

=== added file 'tests/charmhelpers/contrib/openstack/amulet/utils.py'
--- tests/charmhelpers/contrib/openstack/amulet/utils.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/contrib/openstack/amulet/utils.py	2015-09-03 18:38:40 +0000
@@ -0,0 +1,963 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import amulet
+import json
+import logging
+import os
+import six
+import time
+import urllib
+
+import cinderclient.v1.client as cinder_client
+import glanceclient.v1.client as glance_client
+import heatclient.v1.client as heat_client
+import keystoneclient.v2_0 as keystone_client
+import novaclient.v1_1.client as nova_client
+import pika
+import swiftclient
+
+from charmhelpers.contrib.amulet.utils import (
+    AmuletUtils
+)
+
+DEBUG = logging.DEBUG
+ERROR = logging.ERROR
+
+
+class OpenStackAmuletUtils(AmuletUtils):
+    """OpenStack amulet utilities.
+
+       This class inherits from AmuletUtils and has additional support
+       that is specifically for use by OpenStack charm tests.
+       """
+
+    def __init__(self, log_level=ERROR):
+        """Initialize the deployment environment."""
+        super(OpenStackAmuletUtils, self).__init__(log_level)
+
+    def validate_endpoint_data(self, endpoints, admin_port, internal_port,
+                               public_port, expected):
+        """Validate endpoint data.
+
+           Validate actual endpoint data vs expected endpoint data. The ports
+           are used to find the matching endpoint.
+           """
+        self.log.debug('Validating endpoint data...')
+        self.log.debug('actual: {}'.format(repr(endpoints)))
+        found = False
+        for ep in endpoints:
+            self.log.debug('endpoint: {}'.format(repr(ep)))
+            if (admin_port in ep.adminurl and
+                    internal_port in ep.internalurl and
+                    public_port in ep.publicurl):
+                found = True
+                actual = {'id': ep.id,
+                          'region': ep.region,
+                          'adminurl': ep.adminurl,
+                          'internalurl': ep.internalurl,
+                          'publicurl': ep.publicurl,
+                          'service_id': ep.service_id}
+                ret = self._validate_dict_data(expected, actual)
+                if ret:
+                    return 'unexpected endpoint data - {}'.format(ret)
+
+        if not found:
+            return 'endpoint not found'
+
+    def validate_svc_catalog_endpoint_data(self, expected, actual):
+        """Validate service catalog endpoint data.
+
+           Validate a list of actual service catalog endpoints vs a list of
+           expected service catalog endpoints.
+           """
+        self.log.debug('Validating service catalog endpoint data...')
+        self.log.debug('actual: {}'.format(repr(actual)))
+        for k, v in six.iteritems(expected):
+            if k in actual:
+                ret = self._validate_dict_data(expected[k][0], actual[k][0])
+                if ret:
+                    return self.endpoint_error(k, ret)
+            else:
+                return "endpoint {} does not exist".format(k)
+        return ret
+
+    def validate_tenant_data(self, expected, actual):
+        """Validate tenant data.
+
+           Validate a list of actual tenant data vs list of expected tenant
+           data.
+           """
+        self.log.debug('Validating tenant data...')
+        self.log.debug('actual: {}'.format(repr(actual)))
+        for e in expected:
+            found = False
+            for act in actual:
+                a = {'enabled': act.enabled, 'description': act.description,
+                     'name': act.name, 'id': act.id}
+                if e['name'] == a['name']:
+                    found = True
+                    ret = self._validate_dict_data(e, a)
+                    if ret:
+                        return "unexpected tenant data - {}".format(ret)
+            if not found:
+                return "tenant {} does not exist".format(e['name'])
+        return ret
+
+    def validate_role_data(self, expected, actual):
+        """Validate role data.
+
+           Validate a list of actual role data vs a list of expected role
+           data.
+           """
+        self.log.debug('Validating role data...')
+        self.log.debug('actual: {}'.format(repr(actual)))
+        for e in expected:
+            found = False
+            for act in actual:
+                a = {'name': act.name, 'id': act.id}
+                if e['name'] == a['name']:
+                    found = True
+                    ret = self._validate_dict_data(e, a)
+                    if ret:
+                        return "unexpected role data - {}".format(ret)
+            if not found:
+                return "role {} does not exist".format(e['name'])
+        return ret
+
+    def validate_user_data(self, expected, actual):
+        """Validate user data.
+
+           Validate a list of actual user data vs a list of expected user
+           data.
+           """
+        self.log.debug('Validating user data...')
+        self.log.debug('actual: {}'.format(repr(actual)))
+        for e in expected:
+            found = False
+            for act in actual:
+                a = {'enabled': act.enabled, 'name': act.name,
+                     'email': act.email, 'tenantId': act.tenantId,
+                     'id': act.id}
+                if e['name'] == a['name']:
+                    found = True
+                    ret = self._validate_dict_data(e, a)
+                    if ret:
+                        return "unexpected user data - {}".format(ret)
+            if not found:
+                return "user {} does not exist".format(e['name'])
+        return ret
+
+    def validate_flavor_data(self, expected, actual):
+        """Validate flavor data.
+
+           Validate a list of actual flavors vs a list of expected flavors.
+           """
+        self.log.debug('Validating flavor data...')
+        self.log.debug('actual: {}'.format(repr(actual)))
+        act = [a.name for a in actual]
+        return self._validate_list_data(expected, act)
+
+    def tenant_exists(self, keystone, tenant):
+        """Return True if tenant exists."""
+        self.log.debug('Checking if tenant exists ({})...'.format(tenant))
+        return tenant in [t.name for t in keystone.tenants.list()]
+
+    def authenticate_cinder_admin(self, keystone_sentry, username,
+                                  password, tenant):
+        """Authenticates admin user with cinder."""
+        # NOTE(beisner): cinder python client doesn't accept tokens.
+        service_ip = \
+            keystone_sentry.relation('shared-db',
+                                     'mysql:shared-db')['private-address']
+        ept = "http://{}:5000/v2.0".format(service_ip.strip().decode('utf-8'))
+        return cinder_client.Client(username, password, tenant, ept)
+
+    def authenticate_keystone_admin(self, keystone_sentry, user, password,
+                                    tenant):
+        """Authenticates admin user with the keystone admin endpoint."""
+        self.log.debug('Authenticating keystone admin...')
+        unit = keystone_sentry
+        service_ip = unit.relation('shared-db',
+                                   'mysql:shared-db')['private-address']
+        ep = "http://{}:35357/v2.0".format(service_ip.strip().decode('utf-8'))
+        return keystone_client.Client(username=user, password=password,
+                                      tenant_name=tenant, auth_url=ep)
+
+    def authenticate_keystone_user(self, keystone, user, password, tenant):
+        """Authenticates a regular user with the keystone public endpoint."""
+        self.log.debug('Authenticating keystone user ({})...'.format(user))
+        ep = keystone.service_catalog.url_for(service_type='identity',
+                                              endpoint_type='publicURL')
+        return keystone_client.Client(username=user, password=password,
+                                      tenant_name=tenant, auth_url=ep)
+
+    def authenticate_glance_admin(self, keystone):
+        """Authenticates admin user with glance."""
+        self.log.debug('Authenticating glance admin...')
+        ep = keystone.service_catalog.url_for(service_type='image',
+                                              endpoint_type='adminURL')
+        return glance_client.Client(ep, token=keystone.auth_token)
+
+    def authenticate_heat_admin(self, keystone):
+        """Authenticates the admin user with heat."""
+        self.log.debug('Authenticating heat admin...')
+        ep = keystone.service_catalog.url_for(service_type='orchestration',
+                                              endpoint_type='publicURL')
+        return heat_client.Client(endpoint=ep, token=keystone.auth_token)
+
+    def authenticate_nova_user(self, keystone, user, password, tenant):
+        """Authenticates a regular user with nova-api."""
+        self.log.debug('Authenticating nova user ({})...'.format(user))
+        ep = keystone.service_catalog.url_for(service_type='identity',
+                                              endpoint_type='publicURL')
+        return nova_client.Client(username=user, api_key=password,
+                                  project_id=tenant, auth_url=ep)
+
+    def authenticate_swift_user(self, keystone, user, password, tenant):
+        """Authenticates a regular user with swift api."""
+        self.log.debug('Authenticating swift user ({})...'.format(user))
+        ep = keystone.service_catalog.url_for(service_type='identity',
+                                              endpoint_type='publicURL')
+        return swiftclient.Connection(authurl=ep,
+                                      user=user,
+                                      key=password,
+                                      tenant_name=tenant,
+                                      auth_version='2.0')
+
+    def create_cirros_image(self, glance, image_name):
+        """Download the latest cirros image and upload it to glance,
+        validate and return a resource pointer.
+
+        :param glance: pointer to authenticated glance connection
+        :param image_name: display name for new image
+        :returns: glance image pointer
+        """
+        self.log.debug('Creating glance cirros image '
+                       '({})...'.format(image_name))
+
+        # Download cirros image
+        http_proxy = os.getenv('AMULET_HTTP_PROXY')
+        self.log.debug('AMULET_HTTP_PROXY: {}'.format(http_proxy))
+        if http_proxy:
+            proxies = {'http': http_proxy}
+            opener = urllib.FancyURLopener(proxies)
+        else:
+            opener = urllib.FancyURLopener()
+
+        f = opener.open('http://download.cirros-cloud.net/version/released')
+        version = f.read().strip()
+        cirros_img = 'cirros-{}-x86_64-disk.img'.format(version)
+        local_path = os.path.join('tests', cirros_img)
+
+        if not os.path.exists(local_path):
+            cirros_url = 'http://{}/{}/{}'.format('download.cirros-cloud.net',
+                                                  version, cirros_img)
+            opener.retrieve(cirros_url, local_path)
+        f.close()
+
+        # Create glance image
+        with open(local_path) as f:
+            image = glance.images.create(name=image_name, is_public=True,
+                                         disk_format='qcow2',
+                                         container_format='bare', data=f)
+
+        # Wait for image to reach active status
+        img_id = image.id
+        ret = self.resource_reaches_status(glance.images, img_id,
+                                           expected_stat='active',
+                                           msg='Image status wait')
+        if not ret:
+            msg = 'Glance image failed to reach expected state.'
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        # Re-validate new image
+        self.log.debug('Validating image attributes...')
+        val_img_name = glance.images.get(img_id).name
+        val_img_stat = glance.images.get(img_id).status
+        val_img_pub = glance.images.get(img_id).is_public
+        val_img_cfmt = glance.images.get(img_id).container_format
+        val_img_dfmt = glance.images.get(img_id).disk_format
+        msg_attr = ('Image attributes - name:{} public:{} id:{} stat:{} '
+                    'container fmt:{} disk fmt:{}'.format(
+                        val_img_name, val_img_pub, img_id,
+                        val_img_stat, val_img_cfmt, val_img_dfmt))
+
+        if val_img_name == image_name and val_img_stat == 'active' \
+                and val_img_pub is True and val_img_cfmt == 'bare' \
+                and val_img_dfmt == 'qcow2':
+            self.log.debug(msg_attr)
+        else:
+            msg = ('Volume validation failed, {}'.format(msg_attr))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        return image
+
+    def delete_image(self, glance, image):
+        """Delete the specified image."""
+
+        # /!\ DEPRECATION WARNING
+        self.log.warn('/!\\ DEPRECATION WARNING:  use '
+                      'delete_resource instead of delete_image.')
+        self.log.debug('Deleting glance image ({})...'.format(image))
+        return self.delete_resource(glance.images, image, msg='glance image')
+
+    def create_instance(self, nova, image_name, instance_name, flavor):
+        """Create the specified instance."""
+        self.log.debug('Creating instance '
+                       '({}|{}|{})'.format(instance_name, image_name, flavor))
+        image = nova.images.find(name=image_name)
+        flavor = nova.flavors.find(name=flavor)
+        instance = nova.servers.create(name=instance_name, image=image,
+                                       flavor=flavor)
+
+        count = 1
+        status = instance.status
+        while status != 'ACTIVE' and count < 60:
+            time.sleep(3)
+            instance = nova.servers.get(instance.id)
+            status = instance.status
+            self.log.debug('instance status: {}'.format(status))
+            count += 1
+
+        if status != 'ACTIVE':
+            self.log.error('instance creation timed out')
+            return None
+
+        return instance
+
+    def delete_instance(self, nova, instance):
+        """Delete the specified instance."""
+
+        # /!\ DEPRECATION WARNING
+        self.log.warn('/!\\ DEPRECATION WARNING:  use '
+                      'delete_resource instead of delete_instance.')
+        self.log.debug('Deleting instance ({})...'.format(instance))
+        return self.delete_resource(nova.servers, instance,
+                                    msg='nova instance')
+
+    def create_or_get_keypair(self, nova, keypair_name="testkey"):
+        """Create a new keypair, or return pointer if it already exists."""
+        try:
+            _keypair = nova.keypairs.get(keypair_name)
+            self.log.debug('Keypair ({}) already exists, '
+                           'using it.'.format(keypair_name))
+            return _keypair
+        except:
+            self.log.debug('Keypair ({}) does not exist, '
+                           'creating it.'.format(keypair_name))
+
+        _keypair = nova.keypairs.create(name=keypair_name)
+        return _keypair
+
+    def create_cinder_volume(self, cinder, vol_name="demo-vol", vol_size=1,
+                             img_id=None, src_vol_id=None, snap_id=None):
+        """Create cinder volume, optionally from a glance image, OR
+        optionally as a clone of an existing volume, OR optionally
+        from a snapshot.  Wait for the new volume status to reach
+        the expected status, validate and return a resource pointer.
+
+        :param vol_name: cinder volume display name
+        :param vol_size: size in gigabytes
+        :param img_id: optional glance image id
+        :param src_vol_id: optional source volume id to clone
+        :param snap_id: optional snapshot id to use
+        :returns: cinder volume pointer
+        """
+        # Handle parameter input and avoid impossible combinations
+        if img_id and not src_vol_id and not snap_id:
+            # Create volume from image
+            self.log.debug('Creating cinder volume from glance image...')
+            bootable = 'true'
+        elif src_vol_id and not img_id and not snap_id:
+            # Clone an existing volume
+            self.log.debug('Cloning cinder volume...')
+            bootable = cinder.volumes.get(src_vol_id).bootable
+        elif snap_id and not src_vol_id and not img_id:
+            # Create volume from snapshot
+            self.log.debug('Creating cinder volume from snapshot...')
+            snap = cinder.volume_snapshots.find(id=snap_id)
+            vol_size = snap.size
+            snap_vol_id = cinder.volume_snapshots.get(snap_id).volume_id
+            bootable = cinder.volumes.get(snap_vol_id).bootable
+        elif not img_id and not src_vol_id and not snap_id:
+            # Create volume
+            self.log.debug('Creating cinder volume...')
+            bootable = 'false'
+        else:
+            # Impossible combination of parameters
+            msg = ('Invalid method use - name:{} size:{} img_id:{} '
+                   'src_vol_id:{} snap_id:{}'.format(vol_name, vol_size,
+                                                     img_id, src_vol_id,
+                                                     snap_id))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        # Create new volume
+        try:
+            vol_new = cinder.volumes.create(display_name=vol_name,
+                                            imageRef=img_id,
+                                            size=vol_size,
+                                            source_volid=src_vol_id,
+                                            snapshot_id=snap_id)
+            vol_id = vol_new.id
+        except Exception as e:
+            msg = 'Failed to create volume: {}'.format(e)
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        # Wait for volume to reach available status
+        ret = self.resource_reaches_status(cinder.volumes, vol_id,
+                                           expected_stat="available",
+                                           msg="Volume status wait")
+        if not ret:
+            msg = 'Cinder volume failed to reach expected state.'
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        # Re-validate new volume
+        self.log.debug('Validating volume attributes...')
+        val_vol_name = cinder.volumes.get(vol_id).display_name
+        val_vol_boot = cinder.volumes.get(vol_id).bootable
+        val_vol_stat = cinder.volumes.get(vol_id).status
+        val_vol_size = cinder.volumes.get(vol_id).size
+        msg_attr = ('Volume attributes - name:{} id:{} stat:{} boot:'
+                    '{} size:{}'.format(val_vol_name, vol_id,
+                                        val_vol_stat, val_vol_boot,
+                                        val_vol_size))
+
+        if val_vol_boot == bootable and val_vol_stat == 'available' \
+                and val_vol_name == vol_name and val_vol_size == vol_size:
+            self.log.debug(msg_attr)
+        else:
+            msg = ('Volume validation failed, {}'.format(msg_attr))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        return vol_new
+
+    def delete_resource(self, resource, resource_id,
+                        msg="resource", max_wait=120):
+        """Delete one openstack resource, such as one instance, keypair,
+        image, volume, stack, etc., and confirm deletion within max wait time.
+
+        :param resource: pointer to os resource type, ex:glance_client.images
+        :param resource_id: unique name or id for the openstack resource
+        :param msg: text to identify purpose in logging
+        :param max_wait: maximum wait time in seconds
+        :returns: True if successful, otherwise False
+        """
+        self.log.debug('Deleting OpenStack resource '
+                       '{} ({})'.format(resource_id, msg))
+        num_before = len(list(resource.list()))
+        resource.delete(resource_id)
+
+        tries = 0
+        num_after = len(list(resource.list()))
+        while num_after != (num_before - 1) and tries < (max_wait / 4):
+            self.log.debug('{} delete check: '
+                           '{} [{}:{}] {}'.format(msg, tries,
+                                                  num_before,
+                                                  num_after,
+                                                  resource_id))
+            time.sleep(4)
+            num_after = len(list(resource.list()))
+            tries += 1
+
+        self.log.debug('{}:  expected, actual count = {}, '
+                       '{}'.format(msg, num_before - 1, num_after))
+
+        if num_after == (num_before - 1):
+            return True
+        else:
+            self.log.error('{} delete timed out'.format(msg))
+            return False
+
+    def resource_reaches_status(self, resource, resource_id,
+                                expected_stat='available',
+                                msg='resource', max_wait=120):
+        """Wait for an openstack resources status to reach an
+           expected status within a specified time.  Useful to confirm that
+           nova instances, cinder vols, snapshots, glance images, heat stacks
+           and other resources eventually reach the expected status.
+
+        :param resource: pointer to os resource type, ex: heat_client.stacks
+        :param resource_id: unique id for the openstack resource
+        :param expected_stat: status to expect resource to reach
+        :param msg: text to identify purpose in logging
+        :param max_wait: maximum wait time in seconds
+        :returns: True if successful, False if status is not reached
+        """
+
+        tries = 0
+        resource_stat = resource.get(resource_id).status
+        while resource_stat != expected_stat and tries < (max_wait / 4):
+            self.log.debug('{} status check: '
+                           '{} [{}:{}] {}'.format(msg, tries,
+                                                  resource_stat,
+                                                  expected_stat,
+                                                  resource_id))
+            time.sleep(4)
+            resource_stat = resource.get(resource_id).status
+            tries += 1
+
+        self.log.debug('{}:  expected, actual status = {}, '
+                       '{}'.format(msg, resource_stat, expected_stat))
+
+        if resource_stat == expected_stat:
+            return True
+        else:
+            self.log.debug('{} never reached expected status: '
+                           '{}'.format(resource_id, expected_stat))
+            return False
+
+    def get_ceph_osd_id_cmd(self, index):
+        """Produce a shell command that will return a ceph-osd id."""
+        return ("`initctl list | grep 'ceph-osd ' | "
+                "awk 'NR=={} {{ print $2 }}' | "
+                "grep -o '[0-9]*'`".format(index + 1))
+
+    def get_ceph_pools(self, sentry_unit):
+        """Return a dict of ceph pools from a single ceph unit, with
+        pool name as keys, pool id as vals."""
+        pools = {}
+        cmd = 'sudo ceph osd lspools'
+        output, code = sentry_unit.run(cmd)
+        if code != 0:
+            msg = ('{} `{}` returned {} '
+                   '{}'.format(sentry_unit.info['unit_name'],
+                               cmd, code, output))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+
+        # Example output: 0 data,1 metadata,2 rbd,3 cinder,4 glance,
+        for pool in str(output).split(','):
+            pool_id_name = pool.split(' ')
+            if len(pool_id_name) == 2:
+                pool_id = pool_id_name[0]
+                pool_name = pool_id_name[1]
+                pools[pool_name] = int(pool_id)
+
+        self.log.debug('Pools on {}: {}'.format(sentry_unit.info['unit_name'],
+                                                pools))
+        return pools
+
+    def get_ceph_df(self, sentry_unit):
+        """Return dict of ceph df json output, including ceph pool state.
+
+        :param sentry_unit: Pointer to amulet sentry instance (juju unit)
+        :returns: Dict of ceph df output
+        """
+        cmd = 'sudo ceph df --format=json'
+        output, code = sentry_unit.run(cmd)
+        if code != 0:
+            msg = ('{} `{}` returned {} '
+                   '{}'.format(sentry_unit.info['unit_name'],
+                               cmd, code, output))
+            amulet.raise_status(amulet.FAIL, msg=msg)
+        return json.loads(output)
+
+    def get_ceph_pool_sample(self, sentry_unit, pool_id=0):
+        """Take a sample of attributes of a ceph pool, returning ceph
+        pool name, object count and disk space used for the specified
+        pool ID number.
+
+        :param sentry_unit: Pointer to amulet sentry instance (juju unit)
+        :param pool_id: Ceph pool ID
+        :returns: List of pool name, object count, kb disk space used
+        """
+        df = self.get_ceph_df(sentry_unit)
+        pool_name = df['pools'][pool_id]['name']
+        obj_count = df['pools'][pool_id]['stats']['objects']
+        kb_used = df['pools'][pool_id]['stats']['kb_used']
+        self.log.debug('Ceph {} pool (ID {}): {} objects, '
+                       '{} kb used'.format(pool_name, pool_id,
+                                           obj_count, kb_used))
+        return pool_name, obj_count, kb_used
+
+    def validate_ceph_pool_samples(self, samples, sample_type="resource pool"):
+        """Validate ceph pool samples taken over time, such as pool
+        object counts or pool kb used, before adding, after adding, and
+        after deleting items which affect those pool attributes.  The
+        2nd element is expected to be greater than the 1st; 3rd is expected
+        to be less than the 2nd.
+
+        :param samples: List containing 3 data samples
+        :param sample_type: String for logging and usage context
+        :returns: None if successful, Failure message otherwise
+        """
+        original, created, deleted = range(3)
+        if samples[created] <= samples[original] or \
+                samples[deleted] >= samples[created]:
+            return ('Ceph {} samples ({}) '
+                    'unexpected.'.format(sample_type, samples))
+        else:
+            self.log.debug('Ceph {} samples (OK): '
+                           '{}'.format(sample_type, samples))
+            return None
+
+# rabbitmq/amqp specific helpers:
+    def add_rmq_test_user(self, sentry_units,
+                          username="testuser1", password="changeme"):
+        """Add a test user via the first rmq juju unit, check connection as
+        the new user against all sentry units.
+
+        :param sentry_units: list of sentry unit pointers
+        :param username: amqp user name, default to testuser1
+        :param password: amqp user password
+        :returns: None if successful.  Raise on error.
+        """
+        self.log.debug('Adding rmq user ({})...'.format(username))
+
+        # Check that user does not already exist
+        cmd_user_list = 'rabbitmqctl list_users'
+        output, _ = self.run_cmd_unit(sentry_units[0], cmd_user_list)
+        if username in output:
+            self.log.warning('User ({}) already exists, returning '
+                             'gracefully.'.format(username))
+            return
+
+        perms = '".*" ".*" ".*"'
+        cmds = ['rabbitmqctl add_user {} {}'.format(username, password),
+                'rabbitmqctl set_permissions {} {}'.format(username, perms)]
+
+        # Add user via first unit
+        for cmd in cmds:
+            output, _ = self.run_cmd_unit(sentry_units[0], cmd)
+
+        # Check connection against the other sentry_units
+        self.log.debug('Checking user connect against units...')
+        for sentry_unit in sentry_units:
+            connection = self.connect_amqp_by_unit(sentry_unit, ssl=False,
+                                                   username=username,
+                                                   password=password)
+            connection.close()
+
+    def delete_rmq_test_user(self, sentry_units, username="testuser1"):
+        """Delete a rabbitmq user via the first rmq juju unit.
+
+        :param sentry_units: list of sentry unit pointers
+        :param username: amqp user name, default to testuser1
+        :param password: amqp user password
+        :returns: None if successful or no such user.
+        """
+        self.log.debug('Deleting rmq user ({})...'.format(username))
+
+        # Check that the user exists
+        cmd_user_list = 'rabbitmqctl list_users'
+        output, _ = self.run_cmd_unit(sentry_units[0], cmd_user_list)
+
+        if username not in output:
+            self.log.warning('User ({}) does not exist, returning '
+                             'gracefully.'.format(username))
+            return
+
+        # Delete the user
+        cmd_user_del = 'rabbitmqctl delete_user {}'.format(username)
+        output, _ = self.run_cmd_unit(sentry_units[0], cmd_user_del)
+
+    def get_rmq_cluster_status(self, sentry_unit):
+        """Execute rabbitmq cluster status command on a unit and return
+        the full output.
+
+        :param unit: sentry unit
+        :returns: String containing console output of cluster status command
+        """
+        cmd = 'rabbitmqctl cluster_status'
+        output, _ = self.run_cmd_unit(sentry_unit, cmd)
+        self.log.debug('{} cluster_status:\n{}'.format(
+            sentry_unit.info['unit_name'], output))
+        return str(output)
+
+    def get_rmq_cluster_running_nodes(self, sentry_unit):
+        """Parse rabbitmqctl cluster_status output string, return list of
+        running rabbitmq cluster nodes.
+
+        :param unit: sentry unit
+        :returns: List containing node names of running nodes
+        """
+        # NOTE(beisner): rabbitmqctl cluster_status output is not
+        # json-parsable, do string chop foo, then json.loads that.
+        str_stat = self.get_rmq_cluster_status(sentry_unit)
+        if 'running_nodes' in str_stat:
+            pos_start = str_stat.find("{running_nodes,") + 15
+            pos_end = str_stat.find("]},", pos_start) + 1
+            str_run_nodes = str_stat[pos_start:pos_end].replace("'", '"')
+            run_nodes = json.loads(str_run_nodes)
+            return run_nodes
+        else:
+            return []
+
+    def validate_rmq_cluster_running_nodes(self, sentry_units):
+        """Check that all rmq unit hostnames are represented in the
+        cluster_status output of all units.
+
+        :param host_names: dict of juju unit names to host names
+        :param units: list of sentry unit pointers (all rmq units)
+        :returns: None if successful, otherwise return error message
+        """
+        host_names = self.get_unit_hostnames(sentry_units)
+        errors = []
+
+        # Query every unit for cluster_status running nodes
+        for query_unit in sentry_units:
+            query_unit_name = query_unit.info['unit_name']
+            running_nodes = self.get_rmq_cluster_running_nodes(query_unit)
+
+            # Confirm that every unit is represented in the queried unit's
+            # cluster_status running nodes output.
+            for validate_unit in sentry_units:
+                val_host_name = host_names[validate_unit.info['unit_name']]
+                val_node_name = 'rabbit@{}'.format(val_host_name)
+
+                if val_node_name not in running_nodes:
+                    errors.append('Cluster member check failed on {}: {} not '
+                                  'in {}\n'.format(query_unit_name,
+                                                   val_node_name,
+                                                   running_nodes))
+        if errors:
+            return ''.join(errors)
+
+    def rmq_ssl_is_enabled_on_unit(self, sentry_unit, port=None):
+        """Check a single juju rmq unit for ssl and port in the config file."""
+        host = sentry_unit.info['public-address']
+        unit_name = sentry_unit.info['unit_name']
+
+        conf_file = '/etc/rabbitmq/rabbitmq.config'
+        conf_contents = str(self.file_contents_safe(sentry_unit,
+                                                    conf_file, max_wait=16))
+        # Checks
+        conf_ssl = 'ssl' in conf_contents
+        conf_port = str(port) in conf_contents
+
+        # Port explicitly checked in config
+        if port and conf_port and conf_ssl:
+            self.log.debug('SSL is enabled  @{}:{} '
+                           '({})'.format(host, port, unit_name))
+            return True
+        elif port and not conf_port and conf_ssl:
+            self.log.debug('SSL is enabled @{} but not on port {} '
+                           '({})'.format(host, port, unit_name))
+            return False
+        # Port not checked (useful when checking that ssl is disabled)
+        elif not port and conf_ssl:
+            self.log.debug('SSL is enabled  @{}:{} '
+                           '({})'.format(host, port, unit_name))
+            return True
+        elif not port and not conf_ssl:
+            self.log.debug('SSL not enabled @{}:{} '
+                           '({})'.format(host, port, unit_name))
+            return False
+        else:
+            msg = ('Unknown condition when checking SSL status @{}:{} '
+                   '({})'.format(host, port, unit_name))
+            amulet.raise_status(amulet.FAIL, msg)
+
+    def validate_rmq_ssl_enabled_units(self, sentry_units, port=None):
+        """Check that ssl is enabled on rmq juju sentry units.
+
+        :param sentry_units: list of all rmq sentry units
+        :param port: optional ssl port override to validate
+        :returns: None if successful, otherwise return error message
+        """
+        for sentry_unit in sentry_units:
+            if not self.rmq_ssl_is_enabled_on_unit(sentry_unit, port=port):
+                return ('Unexpected condition:  ssl is disabled on unit '
+                        '({})'.format(sentry_unit.info['unit_name']))
+        return None
+
+    def validate_rmq_ssl_disabled_units(self, sentry_units):
+        """Check that ssl is enabled on listed rmq juju sentry units.
+
+        :param sentry_units: list of all rmq sentry units
+        :returns: True if successful.  Raise on error.
+        """
+        for sentry_unit in sentry_units:
+            if self.rmq_ssl_is_enabled_on_unit(sentry_unit):
+                return ('Unexpected condition:  ssl is enabled on unit '
+                        '({})'.format(sentry_unit.info['unit_name']))
+        return None
+
+    def configure_rmq_ssl_on(self, sentry_units, deployment,
+                             port=None, max_wait=60):
+        """Turn ssl charm config option on, with optional non-default
+        ssl port specification.  Confirm that it is enabled on every
+        unit.
+
+        :param sentry_units: list of sentry units
+        :param deployment: amulet deployment object pointer
+        :param port: amqp port, use defaults if None
+        :param max_wait: maximum time to wait in seconds to confirm
+        :returns: None if successful.  Raise on error.
+        """
+        self.log.debug('Setting ssl charm config option:  on')
+
+        # Enable RMQ SSL
+        config = {'ssl': 'on'}
+        if port:
+            config['ssl_port'] = port
+
+        deployment.configure('rabbitmq-server', config)
+
+        # Confirm
+        tries = 0
+        ret = self.validate_rmq_ssl_enabled_units(sentry_units, port=port)
+        while ret and tries < (max_wait / 4):
+            time.sleep(4)
+            self.log.debug('Attempt {}: {}'.format(tries, ret))
+            ret = self.validate_rmq_ssl_enabled_units(sentry_units, port=port)
+            tries += 1
+
+        if ret:
+            amulet.raise_status(amulet.FAIL, ret)
+
+    def configure_rmq_ssl_off(self, sentry_units, deployment, max_wait=60):
+        """Turn ssl charm config option off, confirm that it is disabled
+        on every unit.
+
+        :param sentry_units: list of sentry units
+        :param deployment: amulet deployment object pointer
+        :param max_wait: maximum time to wait in seconds to confirm
+        :returns: None if successful.  Raise on error.
+        """
+        self.log.debug('Setting ssl charm config option:  off')
+
+        # Disable RMQ SSL
+        config = {'ssl': 'off'}
+        deployment.configure('rabbitmq-server', config)
+
+        # Confirm
+        tries = 0
+        ret = self.validate_rmq_ssl_disabled_units(sentry_units)
+        while ret and tries < (max_wait / 4):
+            time.sleep(4)
+            self.log.debug('Attempt {}: {}'.format(tries, ret))
+            ret = self.validate_rmq_ssl_disabled_units(sentry_units)
+            tries += 1
+
+        if ret:
+            amulet.raise_status(amulet.FAIL, ret)
+
+    def connect_amqp_by_unit(self, sentry_unit, ssl=False,
+                             port=None, fatal=True,
+                             username="testuser1", password="changeme"):
+        """Establish and return a pika amqp connection to the rabbitmq service
+        running on a rmq juju unit.
+
+        :param sentry_unit: sentry unit pointer
+        :param ssl: boolean, default to False
+        :param port: amqp port, use defaults if None
+        :param fatal: boolean, default to True (raises on connect error)
+        :param username: amqp user name, default to testuser1
+        :param password: amqp user password
+        :returns: pika amqp connection pointer or None if failed and non-fatal
+        """
+        host = sentry_unit.info['public-address']
+        unit_name = sentry_unit.info['unit_name']
+
+        # Default port logic if port is not specified
+        if ssl and not port:
+            port = 5671
+        elif not ssl and not port:
+            port = 5672
+
+        self.log.debug('Connecting to amqp on {}:{} ({}) as '
+                       '{}...'.format(host, port, unit_name, username))
+
+        try:
+            credentials = pika.PlainCredentials(username, password)
+            parameters = pika.ConnectionParameters(host=host, port=port,
+                                                   credentials=credentials,
+                                                   ssl=ssl,
+                                                   connection_attempts=3,
+                                                   retry_delay=5,
+                                                   socket_timeout=1)
+            connection = pika.BlockingConnection(parameters)
+            assert connection.server_properties['product'] == 'RabbitMQ'
+            self.log.debug('Connect OK')
+            return connection
+        except Exception as e:
+            msg = ('amqp connection failed to {}:{} as '
+                   '{} ({})'.format(host, port, username, str(e)))
+            if fatal:
+                amulet.raise_status(amulet.FAIL, msg)
+            else:
+                self.log.warn(msg)
+                return None
+
+    def publish_amqp_message_by_unit(self, sentry_unit, message,
+                                     queue="test", ssl=False,
+                                     username="testuser1",
+                                     password="changeme",
+                                     port=None):
+        """Publish an amqp message to a rmq juju unit.
+
+        :param sentry_unit: sentry unit pointer
+        :param message: amqp message string
+        :param queue: message queue, default to test
+        :param username: amqp user name, default to testuser1
+        :param password: amqp user password
+        :param ssl: boolean, default to False
+        :param port: amqp port, use defaults if None
+        :returns: None.  Raises exception if publish failed.
+        """
+        self.log.debug('Publishing message to {} queue:\n{}'.format(queue,
+                                                                    message))
+        connection = self.connect_amqp_by_unit(sentry_unit, ssl=ssl,
+                                               port=port,
+                                               username=username,
+                                               password=password)
+
+        # NOTE(beisner): extra debug here re: pika hang potential:
+        #   https://github.com/pika/pika/issues/297
+        #   https://groups.google.com/forum/#!topic/rabbitmq-users/Ja0iyfF0Szw
+        self.log.debug('Defining channel...')
+        channel = connection.channel()
+        self.log.debug('Declaring queue...')
+        channel.queue_declare(queue=queue, auto_delete=False, durable=True)
+        self.log.debug('Publishing message...')
+        channel.basic_publish(exchange='', routing_key=queue, body=message)
+        self.log.debug('Closing channel...')
+        channel.close()
+        self.log.debug('Closing connection...')
+        connection.close()
+
+    def get_amqp_message_by_unit(self, sentry_unit, queue="test",
+                                 username="testuser1",
+                                 password="changeme",
+                                 ssl=False, port=None):
+        """Get an amqp message from a rmq juju unit.
+
+        :param sentry_unit: sentry unit pointer
+        :param queue: message queue, default to test
+        :param username: amqp user name, default to testuser1
+        :param password: amqp user password
+        :param ssl: boolean, default to False
+        :param port: amqp port, use defaults if None
+        :returns: amqp message body as string.  Raise if get fails.
+        """
+        connection = self.connect_amqp_by_unit(sentry_unit, ssl=ssl,
+                                               port=port,
+                                               username=username,
+                                               password=password)
+        channel = connection.channel()
+        method_frame, _, body = channel.basic_get(queue)
+
+        if method_frame:
+            self.log.debug('Retreived message from {} queue:\n{}'.format(queue,
+                                                                         body))
+            channel.basic_ack(method_frame.delivery_tag)
+            channel.close()
+            connection.close()
+            return body
+        else:
+            msg = 'No message retrieved.'
+            amulet.raise_status(amulet.FAIL, msg)

=== removed directory 'tests/charmhelpers/contrib/ssl'
=== removed file 'tests/charmhelpers/contrib/ssl/__init__.py'
--- tests/charmhelpers/contrib/ssl/__init__.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/contrib/ssl/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,94 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import subprocess
-from charmhelpers.core import hookenv
-
-
-def generate_selfsigned(keyfile, certfile, keysize="1024", config=None, subject=None, cn=None):
-    """Generate selfsigned SSL keypair
-
-    You must provide one of the 3 optional arguments:
-    config, subject or cn
-    If more than one is provided the leftmost will be used
-
-    Arguments:
-    keyfile -- (required) full path to the keyfile to be created
-    certfile -- (required) full path to the certfile to be created
-    keysize -- (optional) SSL key length
-    config -- (optional) openssl configuration file
-    subject -- (optional) dictionary with SSL subject variables
-    cn -- (optional) cerfificate common name
-
-    Required keys in subject dict:
-    cn -- Common name (eq. FQDN)
-
-    Optional keys in subject dict
-    country -- Country Name (2 letter code)
-    state -- State or Province Name (full name)
-    locality -- Locality Name (eg, city)
-    organization -- Organization Name (eg, company)
-    organizational_unit -- Organizational Unit Name (eg, section)
-    email -- Email Address
-    """
-
-    cmd = []
-    if config:
-        cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
-               "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
-               "-keyout", keyfile,
-               "-out", certfile, "-config", config]
-    elif subject:
-        ssl_subject = ""
-        if "country" in subject:
-            ssl_subject = ssl_subject + "/C={}".format(subject["country"])
-        if "state" in subject:
-            ssl_subject = ssl_subject + "/ST={}".format(subject["state"])
-        if "locality" in subject:
-            ssl_subject = ssl_subject + "/L={}".format(subject["locality"])
-        if "organization" in subject:
-            ssl_subject = ssl_subject + "/O={}".format(subject["organization"])
-        if "organizational_unit" in subject:
-            ssl_subject = ssl_subject + "/OU={}".format(subject["organizational_unit"])
-        if "cn" in subject:
-            ssl_subject = ssl_subject + "/CN={}".format(subject["cn"])
-        else:
-            hookenv.log("When using \"subject\" argument you must "
-                        "provide \"cn\" field at very least")
-            return False
-        if "email" in subject:
-            ssl_subject = ssl_subject + "/emailAddress={}".format(subject["email"])
-
-        cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
-               "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
-               "-keyout", keyfile,
-               "-out", certfile, "-subj", ssl_subject]
-    elif cn:
-        cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
-               "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
-               "-keyout", keyfile,
-               "-out", certfile, "-subj", "/CN={}".format(cn)]
-
-    if not cmd:
-        hookenv.log("No config, subject or cn provided,"
-                    "unable to generate self signed SSL certificates")
-        return False
-    try:
-        subprocess.check_call(cmd)
-        return True
-    except Exception as e:
-        print("Execution of openssl command failed:\n{}".format(e))
-        return False

=== removed file 'tests/charmhelpers/contrib/ssl/service.py'
--- tests/charmhelpers/contrib/ssl/service.py	2015-04-16 21:35:24 +0000
+++ tests/charmhelpers/contrib/ssl/service.py	1970-01-01 00:00:00 +0000
@@ -1,279 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-from os.path import join as path_join
-from os.path import exists
-import subprocess
-
-from charmhelpers.core.hookenv import log, DEBUG
-
-STD_CERT = "standard"
-
-# Mysql server is fairly picky about cert creation
-# and types, spec its creation separately for now.
-MYSQL_CERT = "mysql"
-
-
-class ServiceCA(object):
-
-    default_expiry = str(365 * 2)
-    default_ca_expiry = str(365 * 6)
-
-    def __init__(self, name, ca_dir, cert_type=STD_CERT):
-        self.name = name
-        self.ca_dir = ca_dir
-        self.cert_type = cert_type
-
-    ###############
-    # Hook Helper API
-    @staticmethod
-    def get_ca(type=STD_CERT):
-        service_name = os.environ['JUJU_UNIT_NAME'].split('/')[0]
-        ca_path = os.path.join(os.environ['CHARM_DIR'], 'ca')
-        ca = ServiceCA(service_name, ca_path, type)
-        ca.init()
-        return ca
-
-    @classmethod
-    def get_service_cert(cls, type=STD_CERT):
-        service_name = os.environ['JUJU_UNIT_NAME'].split('/')[0]
-        ca = cls.get_ca()
-        crt, key = ca.get_or_create_cert(service_name)
-        return crt, key, ca.get_ca_bundle()
-
-    ###############
-
-    def init(self):
-        log("initializing service ca", level=DEBUG)
-        if not exists(self.ca_dir):
-            self._init_ca_dir(self.ca_dir)
-            self._init_ca()
-
-    @property
-    def ca_key(self):
-        return path_join(self.ca_dir, 'private', 'cacert.key')
-
-    @property
-    def ca_cert(self):
-        return path_join(self.ca_dir, 'cacert.pem')
-
-    @property
-    def ca_conf(self):
-        return path_join(self.ca_dir, 'ca.cnf')
-
-    @property
-    def signing_conf(self):
-        return path_join(self.ca_dir, 'signing.cnf')
-
-    def _init_ca_dir(self, ca_dir):
-        os.mkdir(ca_dir)
-        for i in ['certs', 'crl', 'newcerts', 'private']:
-            sd = path_join(ca_dir, i)
-            if not exists(sd):
-                os.mkdir(sd)
-
-        if not exists(path_join(ca_dir, 'serial')):
-            with open(path_join(ca_dir, 'serial'), 'w') as fh:
-                fh.write('02\n')
-
-        if not exists(path_join(ca_dir, 'index.txt')):
-            with open(path_join(ca_dir, 'index.txt'), 'w') as fh:
-                fh.write('')
-
-    def _init_ca(self):
-        """Generate the root ca's cert and key.
-        """
-        if not exists(path_join(self.ca_dir, 'ca.cnf')):
-            with open(path_join(self.ca_dir, 'ca.cnf'), 'w') as fh:
-                fh.write(
-                    CA_CONF_TEMPLATE % (self.get_conf_variables()))
-
-        if not exists(path_join(self.ca_dir, 'signing.cnf')):
-            with open(path_join(self.ca_dir, 'signing.cnf'), 'w') as fh:
-                fh.write(
-                    SIGNING_CONF_TEMPLATE % (self.get_conf_variables()))
-
-        if exists(self.ca_cert) or exists(self.ca_key):
-            raise RuntimeError("Initialized called when CA already exists")
-        cmd = ['openssl', 'req', '-config', self.ca_conf,
-               '-x509', '-nodes', '-newkey', 'rsa',
-               '-days', self.default_ca_expiry,
-               '-keyout', self.ca_key, '-out', self.ca_cert,
-               '-outform', 'PEM']
-        output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
-        log("CA Init:\n %s" % output, level=DEBUG)
-
-    def get_conf_variables(self):
-        return dict(
-            org_name="juju",
-            org_unit_name="%s service" % self.name,
-            common_name=self.name,
-            ca_dir=self.ca_dir)
-
-    def get_or_create_cert(self, common_name):
-        if common_name in self:
-            return self.get_certificate(common_name)
-        return self.create_certificate(common_name)
-
-    def create_certificate(self, common_name):
-        if common_name in self:
-            return self.get_certificate(common_name)
-        key_p = path_join(self.ca_dir, "certs", "%s.key" % common_name)
-        crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
-        csr_p = path_join(self.ca_dir, "certs", "%s.csr" % common_name)
-        self._create_certificate(common_name, key_p, csr_p, crt_p)
-        return self.get_certificate(common_name)
-
-    def get_certificate(self, common_name):
-        if common_name not in self:
-            raise ValueError("No certificate for %s" % common_name)
-        key_p = path_join(self.ca_dir, "certs", "%s.key" % common_name)
-        crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
-        with open(crt_p) as fh:
-            crt = fh.read()
-        with open(key_p) as fh:
-            key = fh.read()
-        return crt, key
-
-    def __contains__(self, common_name):
-        crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
-        return exists(crt_p)
-
-    def _create_certificate(self, common_name, key_p, csr_p, crt_p):
-        template_vars = self.get_conf_variables()
-        template_vars['common_name'] = common_name
-        subj = '/O=%(org_name)s/OU=%(org_unit_name)s/CN=%(common_name)s' % (
-            template_vars)
-
-        log("CA Create Cert %s" % common_name, level=DEBUG)
-        cmd = ['openssl', 'req', '-sha1', '-newkey', 'rsa:2048',
-               '-nodes', '-days', self.default_expiry,
-               '-keyout', key_p, '-out', csr_p, '-subj', subj]
-        subprocess.check_call(cmd, stderr=subprocess.PIPE)
-        cmd = ['openssl', 'rsa', '-in', key_p, '-out', key_p]
-        subprocess.check_call(cmd, stderr=subprocess.PIPE)
-
-        log("CA Sign Cert %s" % common_name, level=DEBUG)
-        if self.cert_type == MYSQL_CERT:
-            cmd = ['openssl', 'x509', '-req',
-                   '-in', csr_p, '-days', self.default_expiry,
-                   '-CA', self.ca_cert, '-CAkey', self.ca_key,
-                   '-set_serial', '01', '-out', crt_p]
-        else:
-            cmd = ['openssl', 'ca', '-config', self.signing_conf,
-                   '-extensions', 'req_extensions',
-                   '-days', self.default_expiry, '-notext',
-                   '-in', csr_p, '-out', crt_p, '-subj', subj, '-batch']
-        log("running %s" % " ".join(cmd), level=DEBUG)
-        subprocess.check_call(cmd, stderr=subprocess.PIPE)
-
-    def get_ca_bundle(self):
-        with open(self.ca_cert) as fh:
-            return fh.read()
-
-
-CA_CONF_TEMPLATE = """
-[ ca ]
-default_ca = CA_default
-
-[ CA_default ]
-dir                     = %(ca_dir)s
-policy                  = policy_match
-database                = $dir/index.txt
-serial                  = $dir/serial
-certs                   = $dir/certs
-crl_dir                 = $dir/crl
-new_certs_dir           = $dir/newcerts
-certificate             = $dir/cacert.pem
-private_key             = $dir/private/cacert.key
-RANDFILE                = $dir/private/.rand
-default_md              = default
-
-[ req ]
-default_bits            = 1024
-default_md              = sha1
-
-prompt                  = no
-distinguished_name      = ca_distinguished_name
-
-x509_extensions         = ca_extensions
-
-[ ca_distinguished_name ]
-organizationName        = %(org_name)s
-organizationalUnitName  = %(org_unit_name)s Certificate Authority
-
-
-[ policy_match ]
-countryName             = optional
-stateOrProvinceName     = optional
-organizationName        = match
-organizationalUnitName  = optional
-commonName              = supplied
-
-[ ca_extensions ]
-basicConstraints        = critical,CA:true
-subjectKeyIdentifier    = hash
-authorityKeyIdentifier  = keyid:always, issuer
-keyUsage                = cRLSign, keyCertSign
-"""
-
-
-SIGNING_CONF_TEMPLATE = """
-[ ca ]
-default_ca = CA_default
-
-[ CA_default ]
-dir                     = %(ca_dir)s
-policy                  = policy_match
-database                = $dir/index.txt
-serial                  = $dir/serial
-certs                   = $dir/certs
-crl_dir                 = $dir/crl
-new_certs_dir           = $dir/newcerts
-certificate             = $dir/cacert.pem
-private_key             = $dir/private/cacert.key
-RANDFILE                = $dir/private/.rand
-default_md              = default
-
-[ req ]
-default_bits            = 1024
-default_md              = sha1
-
-prompt                  = no
-distinguished_name      = req_distinguished_name
-
-x509_extensions         = req_extensions
-
-[ req_distinguished_name ]
-organizationName        = %(org_name)s
-organizationalUnitName  = %(org_unit_name)s machine resources
-commonName              = %(common_name)s
-
-[ policy_match ]
-countryName             = optional
-stateOrProvinceName     = optional
-organizationName        = match
-organizationalUnitName  = optional
-commonName              = supplied
-
-[ req_extensions ]
-basicConstraints        = CA:false
-subjectKeyIdentifier    = hash
-authorityKeyIdentifier  = keyid:always, issuer
-keyUsage                = digitalSignature, keyEncipherment, keyAgreement
-extendedKeyUsage        = serverAuth, clientAuth
-"""

=== removed file 'tests/charmhelpers/core/__init__.py'
--- tests/charmhelpers/core/__init__.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/core/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,15 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.

=== removed file 'tests/charmhelpers/core/decorators.py'
--- tests/charmhelpers/core/decorators.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/core/decorators.py	1970-01-01 00:00:00 +0000
@@ -1,57 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-#
-# Copyright 2014 Canonical Ltd.
-#
-# Authors:
-#  Edward Hope-Morley <[e-mail]>
-#
-
-import time
-
-from charmhelpers.core.hookenv import (
-    log,
-    INFO,
-)
-
-
-def retry_on_exception(num_retries, base_delay=0, exc_type=Exception):
-    """If the decorated function raises exception exc_type, allow num_retries
-    retry attempts before raise the exception.
-    """
-    def _retry_on_exception_inner_1(f):
-        def _retry_on_exception_inner_2(*args, **kwargs):
-            retries = num_retries
-            multiplier = 1
-            while True:
-                try:
-                    return f(*args, **kwargs)
-                except exc_type:
-                    if not retries:
-                        raise
-
-                delay = base_delay * multiplier
-                multiplier += 1
-                log("Retrying '%s' %d more times (delay=%s)" %
-                    (f.__name__, retries, delay), level=INFO)
-                retries -= 1
-                if delay:
-                    time.sleep(delay)
-
-        return _retry_on_exception_inner_2
-
-    return _retry_on_exception_inner_1

=== removed file 'tests/charmhelpers/core/files.py'
--- tests/charmhelpers/core/files.py	2015-07-29 10:48:05 +0000
+++ tests/charmhelpers/core/files.py	1970-01-01 00:00:00 +0000
@@ -1,45 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-__author__ = 'Jorge Niedbalski <[e-mail]>'
-
-import os
-import subprocess
-
-
-def sed(filename, before, after, flags='g'):
-    """
-    Search and replaces the given pattern on filename.
-
-    :param filename: relative or absolute file path.
-    :param before: expression to be replaced (see 'man sed')
-    :param after: expression to replace with (see 'man sed')
-    :param flags: sed-compatible regex flags in example, to make
-    the  search and replace case insensitive, specify ``flags="i"``.
-    The ``g`` flag is always specified regardless, so you do not
-    need to remember to include it when overriding this parameter.
-    :returns: If the sed command exit code was zero then return,
-    otherwise raise CalledProcessError.
-    """
-    expression = r's/{0}/{1}/{2}'.format(before,
-                                         after, flags)
-
-    return subprocess.check_call(["sed", "-i", "-r", "-e",
-                                  expression,
-                                  os.path.expanduser(filename)])

=== removed file 'tests/charmhelpers/core/fstab.py'
--- tests/charmhelpers/core/fstab.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/core/fstab.py	1970-01-01 00:00:00 +0000
@@ -1,134 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import io
-import os
-
-__author__ = 'Jorge Niedbalski R. <[e-mail]>'
-
-
-class Fstab(io.FileIO):
-    """This class extends file in order to implement a file reader/writer
-    for file `/etc/fstab`
-    """
-
-    class Entry(object):
-        """Entry class represents a non-comment line on the `/etc/fstab` file
-        """
-        def __init__(self, device, mountpoint, filesystem,
-                     options, d=0, p=0):
-            self.device = device
-            self.mountpoint = mountpoint
-            self.filesystem = filesystem
-
-            if not options:
-                options = "defaults"
-
-            self.options = options
-            self.d = int(d)
-            self.p = int(p)
-
-        def __eq__(self, o):
-            return str(self) == str(o)
-
-        def __str__(self):
-            return "{} {} {} {} {} {}".format(self.device,
-                                              self.mountpoint,
-                                              self.filesystem,
-                                              self.options,
-                                              self.d,
-                                              self.p)
-
-    DEFAULT_PATH = os.path.join(os.path.sep, 'etc', 'fstab')
-
-    def __init__(self, path=None):
-        if path:
-            self._path = path
-        else:
-            self._path = self.DEFAULT_PATH
-        super(Fstab, self).__init__(self._path, 'rb+')
-
-    def _hydrate_entry(self, line):
-        # NOTE: use split with no arguments to split on any
-        #       whitespace including tabs
-        return Fstab.Entry(*filter(
-            lambda x: x not in ('', None),
-            line.strip("\n").split()))
-
-    @property
-    def entries(self):
-        self.seek(0)
-        for line in self.readlines():
-            line = line.decode('us-ascii')
-            try:
-                if line.strip() and not line.strip().startswith("#"):
-                    yield self._hydrate_entry(line)
-            except ValueError:
-                pass
-
-    def get_entry_by_attr(self, attr, value):
-        for entry in self.entries:
-            e_attr = getattr(entry, attr)
-            if e_attr == value:
-                return entry
-        return None
-
-    def add_entry(self, entry):
-        if self.get_entry_by_attr('device', entry.device):
-            return False
-
-        self.write((str(entry) + '\n').encode('us-ascii'))
-        self.truncate()
-        return entry
-
-    def remove_entry(self, entry):
-        self.seek(0)
-
-        lines = [l.decode('us-ascii') for l in self.readlines()]
-
-        found = False
-        for index, line in enumerate(lines):
-            if line.strip() and not line.strip().startswith("#"):
-                if self._hydrate_entry(line) == entry:
-                    found = True
-                    break
-
-        if not found:
-            return False
-
-        lines.remove(line)
-
-        self.seek(0)
-        self.write(''.join(lines).encode('us-ascii'))
-        self.truncate()
-        return True
-
-    @classmethod
-    def remove_by_mountpoint(cls, mountpoint, path=None):
-        fstab = cls(path=path)
-        entry = fstab.get_entry_by_attr('mountpoint', mountpoint)
-        if entry:
-            return fstab.remove_entry(entry)
-        return False
-
-    @classmethod
-    def add(cls, device, mountpoint, filesystem, options=None, path=None):
-        return cls(path=path).add_entry(Fstab.Entry(device,
-                                                    mountpoint, filesystem,
-                                                    options=options))

=== removed file 'tests/charmhelpers/core/host.py'
--- tests/charmhelpers/core/host.py	2015-08-19 13:49:53 +0000
+++ tests/charmhelpers/core/host.py	1970-01-01 00:00:00 +0000
@@ -1,570 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-"""Tools for working with the host system"""
-# Copyright 2012 Canonical Ltd.
-#
-# Authors:
-#  Nick Moffitt <[e-mail]>
-#  Matthew Wedgwood <[e-mail]>
-
-import os
-import re
-import pwd
-import glob
-import grp
-import random
-import string
-import subprocess
-import hashlib
-from contextlib import contextmanager
-from collections import OrderedDict
-
-import six
-
-from .hookenv import log
-from .fstab import Fstab
-
-
-def service_start(service_name):
-    """Start a system service"""
-    return service('start', service_name)
-
-
-def service_stop(service_name):
-    """Stop a system service"""
-    return service('stop', service_name)
-
-
-def service_restart(service_name):
-    """Restart a system service"""
-    return service('restart', service_name)
-
-
-def service_reload(service_name, restart_on_failure=False):
-    """Reload a system service, optionally falling back to restart if
-    reload fails"""
-    service_result = service('reload', service_name)
-    if not service_result and restart_on_failure:
-        service_result = service('restart', service_name)
-    return service_result
-
-
-def service_pause(service_name, init_dir=None):
-    """Pause a system service.
-
-    Stop it, and prevent it from starting again at boot."""
-    if init_dir is None:
-        init_dir = "/etc/init"
-    stopped = service_stop(service_name)
-    # XXX: Support systemd too
-    override_path = os.path.join(
-        init_dir, '{}.override'.format(service_name))
-    with open(override_path, 'w') as fh:
-        fh.write("manual\n")
-    return stopped
-
-
-def service_resume(service_name, init_dir=None):
-    """Resume a system service.
-
-    Reenable starting again at boot. Start the service"""
-    # XXX: Support systemd too
-    if init_dir is None:
-        init_dir = "/etc/init"
-    override_path = os.path.join(
-        init_dir, '{}.override'.format(service_name))
-    if os.path.exists(override_path):
-        os.unlink(override_path)
-    started = service_start(service_name)
-    return started
-
-
-def service(action, service_name):
-    """Control a system service"""
-    cmd = ['service', service_name, action]
-    return subprocess.call(cmd) == 0
-
-
-def service_running(service):
-    """Determine whether a system service is running"""
-    try:
-        output = subprocess.check_output(
-            ['service', service, 'status'],
-            stderr=subprocess.STDOUT).decode('UTF-8')
-    except subprocess.CalledProcessError:
-        return False
-    else:
-        if ("start/running" in output or "is running" in output):
-            return True
-        else:
-            return False
-
-
-def service_available(service_name):
-    """Determine whether a system service is available"""
-    try:
-        subprocess.check_output(
-            ['service', service_name, 'status'],
-            stderr=subprocess.STDOUT).decode('UTF-8')
-    except subprocess.CalledProcessError as e:
-        return b'unrecognized service' not in e.output
-    else:
-        return True
-
-
-def adduser(username, password=None, shell='/bin/bash', system_user=False):
-    """Add a user to the system"""
-    try:
-        user_info = pwd.getpwnam(username)
-        log('user {0} already exists!'.format(username))
-    except KeyError:
-        log('creating user {0}'.format(username))
-        cmd = ['useradd']
-        if system_user or password is None:
-            cmd.append('--system')
-        else:
-            cmd.extend([
-                '--create-home',
-                '--shell', shell,
-                '--password', password,
-            ])
-        cmd.append(username)
-        subprocess.check_call(cmd)
-        user_info = pwd.getpwnam(username)
-    return user_info
-
-
-def user_exists(username):
-    """Check if a user exists"""
-    try:
-        pwd.getpwnam(username)
-        user_exists = True
-    except KeyError:
-        user_exists = False
-    return user_exists
-
-
-def add_group(group_name, system_group=False):
-    """Add a group to the system"""
-    try:
-        group_info = grp.getgrnam(group_name)
-        log('group {0} already exists!'.format(group_name))
-    except KeyError:
-        log('creating group {0}'.format(group_name))
-        cmd = ['addgroup']
-        if system_group:
-            cmd.append('--system')
-        else:
-            cmd.extend([
-                '--group',
-            ])
-        cmd.append(group_name)
-        subprocess.check_call(cmd)
-        group_info = grp.getgrnam(group_name)
-    return group_info
-
-
-def add_user_to_group(username, group):
-    """Add a user to a group"""
-    cmd = ['gpasswd', '-a', username, group]
-    log("Adding user {} to group {}".format(username, group))
-    subprocess.check_call(cmd)
-
-
-def rsync(from_path, to_path, flags='-r', options=None):
-    """Replicate the contents of a path"""
-    options = options or ['--delete', '--executability']
-    cmd = ['/usr/bin/rsync', flags]
-    cmd.extend(options)
-    cmd.append(from_path)
-    cmd.append(to_path)
-    log(" ".join(cmd))
-    return subprocess.check_output(cmd).decode('UTF-8').strip()
-
-
-def symlink(source, destination):
-    """Create a symbolic link"""
-    log("Symlinking {} as {}".format(source, destination))
-    cmd = [
-        'ln',
-        '-sf',
-        source,
-        destination,
-    ]
-    subprocess.check_call(cmd)
-
-
-def mkdir(path, owner='root', group='root', perms=0o555, force=False):
-    """Create a directory"""
-    log("Making dir {} {}:{} {:o}".format(path, owner, group,
-                                          perms))
-    uid = pwd.getpwnam(owner).pw_uid
-    gid = grp.getgrnam(group).gr_gid
-    realpath = os.path.abspath(path)
-    path_exists = os.path.exists(realpath)
-    if path_exists and force:
-        if not os.path.isdir(realpath):
-            log("Removing non-directory file {} prior to mkdir()".format(path))
-            os.unlink(realpath)
-            os.makedirs(realpath, perms)
-    elif not path_exists:
-        os.makedirs(realpath, perms)
-    os.chown(realpath, uid, gid)
-    os.chmod(realpath, perms)
-
-
-def write_file(path, content, owner='root', group='root', perms=0o444):
-    """Create or overwrite a file with the contents of a byte string."""
-    log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
-    uid = pwd.getpwnam(owner).pw_uid
-    gid = grp.getgrnam(group).gr_gid
-    with open(path, 'wb') as target:
-        os.fchown(target.fileno(), uid, gid)
-        os.fchmod(target.fileno(), perms)
-        target.write(content)
-
-
-def fstab_remove(mp):
-    """Remove the given mountpoint entry from /etc/fstab
-    """
-    return Fstab.remove_by_mountpoint(mp)
-
-
-def fstab_add(dev, mp, fs, options=None):
-    """Adds the given device entry to the /etc/fstab file
-    """
-    return Fstab.add(dev, mp, fs, options=options)
-
-
-def mount(device, mountpoint, options=None, persist=False, filesystem="ext3"):
-    """Mount a filesystem at a particular mountpoint"""
-    cmd_args = ['mount']
-    if options is not None:
-        cmd_args.extend(['-o', options])
-    cmd_args.extend([device, mountpoint])
-    try:
-        subprocess.check_output(cmd_args)
-    except subprocess.CalledProcessError as e:
-        log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output))
-        return False
-
-    if persist:
-        return fstab_add(device, mountpoint, filesystem, options=options)
-    return True
-
-
-def umount(mountpoint, persist=False):
-    """Unmount a filesystem"""
-    cmd_args = ['umount', mountpoint]
-    try:
-        subprocess.check_output(cmd_args)
-    except subprocess.CalledProcessError as e:
-        log('Error unmounting {}\n{}'.format(mountpoint, e.output))
-        return False
-
-    if persist:
-        return fstab_remove(mountpoint)
-    return True
-
-
-def mounts():
-    """Get a list of all mounted volumes as [[mountpoint,device],[...]]"""
-    with open('/proc/mounts') as f:
-        # [['/mount/point','/dev/path'],[...]]
-        system_mounts = [m[1::-1] for m in [l.strip().split()
-                                            for l in f.readlines()]]
-    return system_mounts
-
-
-def fstab_mount(mountpoint):
-    """Mount filesystem using fstab"""
-    cmd_args = ['mount', mountpoint]
-    try:
-        subprocess.check_output(cmd_args)
-    except subprocess.CalledProcessError as e:
-        log('Error unmounting {}\n{}'.format(mountpoint, e.output))
-        return False
-    return True
-
-
-def file_hash(path, hash_type='md5'):
-    """
-    Generate a hash checksum of the contents of 'path' or None if not found.
-
-    :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,
-                          such as md5, sha1, sha256, sha512, etc.
-    """
-    if os.path.exists(path):
-        h = getattr(hashlib, hash_type)()
-        with open(path, 'rb') as source:
-            h.update(source.read())
-        return h.hexdigest()
-    else:
-        return None
-
-
-def path_hash(path):
-    """
-    Generate a hash checksum of all files matching 'path'. Standard wildcards
-    like '*' and '?' are supported, see documentation for the 'glob' module for
-    more information.
-
-    :return: dict: A { filename: hash } dictionary for all matched files.
-                   Empty if none found.
-    """
-    return {
-        filename: file_hash(filename)
-        for filename in glob.iglob(path)
-    }
-
-
-def check_hash(path, checksum, hash_type='md5'):
-    """
-    Validate a file using a cryptographic checksum.
-
-    :param str checksum: Value of the checksum used to validate the file.
-    :param str hash_type: Hash algorithm used to generate `checksum`.
-        Can be any hash alrgorithm supported by :mod:`hashlib`,
-        such as md5, sha1, sha256, sha512, etc.
-    :raises ChecksumError: If the file fails the checksum
-
-    """
-    actual_checksum = file_hash(path, hash_type)
-    if checksum != actual_checksum:
-        raise ChecksumError("'%s' != '%s'" % (checksum, actual_checksum))
-
-
-class ChecksumError(ValueError):
-    pass
-
-
-def restart_on_change(restart_map, stopstart=False):
-    """Restart services based on configuration files changing
-
-    This function is used a decorator, for example::
-
-        @restart_on_change({
-            '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]
-            '/etc/apache/sites-enabled/*': [ 'apache2' ]
-            })
-        def config_changed():
-            pass  # your code here
-
-    In this example, the cinder-api and cinder-volume services
-    would be restarted if /etc/ceph/ceph.conf is changed by the
-    ceph_client_changed function. The apache2 service would be
-    restarted if any file matching the pattern got changed, created
-    or removed. Standard wildcards are supported, see documentation
-    for the 'glob' module for more information.
-    """
-    def wrap(f):
-        def wrapped_f(*args, **kwargs):
-            checksums = {path: path_hash(path) for path in restart_map}
-            f(*args, **kwargs)
-            restarts = []
-            for path in restart_map:
-                if path_hash(path) != checksums[path]:
-                    restarts += restart_map[path]
-            services_list = list(OrderedDict.fromkeys(restarts))
-            if not stopstart:
-                for service_name in services_list:
-                    service('restart', service_name)
-            else:
-                for action in ['stop', 'start']:
-                    for service_name in services_list:
-                        service(action, service_name)
-        return wrapped_f
-    return wrap
-
-
-def lsb_release():
-    """Return /etc/lsb-release in a dict"""
-    d = {}
-    with open('/etc/lsb-release', 'r') as lsb:
-        for l in lsb:
-            k, v = l.split('=')
-            d[k.strip()] = v.strip()
-    return d
-
-
-def pwgen(length=None):
-    """Generate a random pasword."""
-    if length is None:
-        # A random length is ok to use a weak PRNG
-        length = random.choice(range(35, 45))
-    alphanumeric_chars = [
-        l for l in (string.ascii_letters + string.digits)
-        if l not in 'l0QD1vAEIOUaeiou']
-    # Use a crypto-friendly PRNG (e.g. /dev/urandom) for making the
-    # actual password
-    random_generator = random.SystemRandom()
-    random_chars = [
-        random_generator.choice(alphanumeric_chars) for _ in range(length)]
-    return(''.join(random_chars))
-
-
-def is_phy_iface(interface):
-    """Returns True if interface is not virtual, otherwise False."""
-    if interface:
-        sys_net = '/sys/class/net'
-        if os.path.isdir(sys_net):
-            for iface in glob.glob(os.path.join(sys_net, '*')):
-                if '/virtual/' in os.path.realpath(iface):
-                    continue
-
-                if interface == os.path.basename(iface):
-                    return True
-
-    return False
-
-
-def get_bond_master(interface):
-    """Returns bond master if interface is bond slave otherwise None.
-
-    NOTE: the provided interface is expected to be physical
-    """
-    if interface:
-        iface_path = '/sys/class/net/%s' % (interface)
-        if os.path.exists(iface_path):
-            if '/virtual/' in os.path.realpath(iface_path):
-                return None
-
-            master = os.path.join(iface_path, 'master')
-            if os.path.exists(master):
-                master = os.path.realpath(master)
-                # make sure it is a bond master
-                if os.path.exists(os.path.join(master, 'bonding')):
-                    return os.path.basename(master)
-
-    return None
-
-
-def list_nics(nic_type=None):
-    '''Return a list of nics of given type(s)'''
-    if isinstance(nic_type, six.string_types):
-        int_types = [nic_type]
-    else:
-        int_types = nic_type
-
-    interfaces = []
-    if nic_type:
-        for int_type in int_types:
-            cmd = ['ip', 'addr', 'show', 'label', int_type + '*']
-            ip_output = subprocess.check_output(cmd).decode('UTF-8')
-            ip_output = ip_output.split('\n')
-            ip_output = (line for line in ip_output if line)
-            for line in ip_output:
-                if line.split()[1].startswith(int_type):
-                    matched = re.search('.*: (' + int_type +
-                                        r'[0-9]+\.[0-9]+)@.*', line)
-                    if matched:
-                        iface = matched.groups()[0]
-                    else:
-                        iface = line.split()[1].replace(":", "")
-
-                    if iface not in interfaces:
-                        interfaces.append(iface)
-    else:
-        cmd = ['ip', 'a']
-        ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
-        ip_output = (line.strip() for line in ip_output if line)
-
-        key = re.compile('^[0-9]+:\s+(.+):')
-        for line in ip_output:
-            matched = re.search(key, line)
-            if matched:
-                iface = matched.group(1)
-                iface = iface.partition("@")[0]
-                if iface not in interfaces:
-                    interfaces.append(iface)
-
-    return interfaces
-
-
-def set_nic_mtu(nic, mtu):
-    '''Set MTU on a network interface'''
-    cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
-    subprocess.check_call(cmd)
-
-
-def get_nic_mtu(nic):
-    cmd = ['ip', 'addr', 'show', nic]
-    ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
-    mtu = ""
-    for line in ip_output:
-        words = line.split()
-        if 'mtu' in words:
-            mtu = words[words.index("mtu") + 1]
-    return mtu
-
-
-def get_nic_hwaddr(nic):
-    cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
-    ip_output = subprocess.check_output(cmd).decode('UTF-8')
-    hwaddr = ""
-    words = ip_output.split()
-    if 'link/ether' in words:
-        hwaddr = words[words.index('link/ether') + 1]
-    return hwaddr
-
-
-def cmp_pkgrevno(package, revno, pkgcache=None):
-    '''Compare supplied revno with the revno of the installed package
-
-    *  1 => Installed revno is greater than supplied arg
-    *  0 => Installed revno is the same as supplied arg
-    * -1 => Installed revno is less than supplied arg
-
-    This function imports apt_cache function from charmhelpers.fetch if
-    the pkgcache argument is None. Be sure to add charmhelpers.fetch if
-    you call this function, or pass an apt_pkg.Cache() instance.
-    '''
-    import apt_pkg
-    if not pkgcache:
-        from charmhelpers.fetch import apt_cache
-        pkgcache = apt_cache()
-    pkg = pkgcache[package]
-    return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
-
-
-@contextmanager
-def chdir(d):
-    cur = os.getcwd()
-    try:
-        yield os.chdir(d)
-    finally:
-        os.chdir(cur)
-
-
-def chownr(path, owner, group, follow_links=True):
-    uid = pwd.getpwnam(owner).pw_uid
-    gid = grp.getgrnam(group).gr_gid
-    if follow_links:
-        chown = os.chown
-    else:
-        chown = os.lchown
-
-    for root, dirs, files in os.walk(path):
-        for name in dirs + files:
-            full = os.path.join(root, name)
-            broken_symlink = os.path.lexists(full) and not os.path.exists(full)
-            if not broken_symlink:
-                chown(full, uid, gid)
-
-
-def lchownr(path, owner, group):
-    chownr(path, owner, group, follow_links=False)

=== removed file 'tests/charmhelpers/core/hugepage.py'
--- tests/charmhelpers/core/hugepage.py	2015-08-19 13:49:53 +0000
+++ tests/charmhelpers/core/hugepage.py	1970-01-01 00:00:00 +0000
@@ -1,62 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import yaml
-from charmhelpers.core import fstab
-from charmhelpers.core import sysctl
-from charmhelpers.core.host import (
-    add_group,
-    add_user_to_group,
-    fstab_mount,
-    mkdir,
-)
-
-
-def hugepage_support(user, group='hugetlb', nr_hugepages=256,
-                     max_map_count=65536, mnt_point='/run/hugepages/kvm',
-                     pagesize='2MB', mount=True):
-    """Enable hugepages on system.
-
-    Args:
-    user (str)  -- Username to allow access to hugepages to
-    group (str) -- Group name to own hugepages
-    nr_hugepages (int) -- Number of pages to reserve
-    max_map_count (int) -- Number of Virtual Memory Areas a process can own
-    mnt_point (str) -- Directory to mount hugepages on
-    pagesize (str) -- Size of hugepages
-    mount (bool) -- Whether to Mount hugepages
-    """
-    group_info = add_group(group)
-    gid = group_info.gr_gid
-    add_user_to_group(user, group)
-    sysctl_settings = {
-        'vm.nr_hugepages': nr_hugepages,
-        'vm.max_map_count': max_map_count,
-        'vm.hugetlb_shm_group': gid,
-    }
-    sysctl.create(yaml.dump(sysctl_settings), '/etc/sysctl.d/10-hugepage.conf')
-    mkdir(mnt_point, owner='root', group='root', perms=0o755, force=False)
-    lfstab = fstab.Fstab()
-    fstab_entry = lfstab.get_entry_by_attr('mountpoint', mnt_point)
-    if fstab_entry:
-        lfstab.remove_entry(fstab_entry)
-    entry = lfstab.Entry('nodev', mnt_point, 'hugetlbfs',
-                         'mode=1770,gid={},pagesize={}'.format(gid, pagesize), 0, 0)
-    lfstab.add_entry(entry)
-    if mount:
-        fstab_mount(mnt_point)

=== removed directory 'tests/charmhelpers/core/services'
=== removed file 'tests/charmhelpers/core/services/__init__.py'
--- tests/charmhelpers/core/services/__init__.py	2015-04-13 22:11:34 +0000
+++ tests/charmhelpers/core/services/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,18 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-from .base import *  # NOQA
-from .helpers import *  # NOQA

=== removed file 'tests/charmhelpers/core/services/base.py'
--- tests/charmhelpers/core/services/base.py	2015-07-29 10:48:05 +0000
+++ tests/charmhelpers/core/services/base.py	1970-01-01 00:00:00 +0000
@@ -1,353 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-import json
-from inspect import getargspec
-from collections import Iterable, OrderedDict
-
-from charmhelpers.core import host
-from charmhelpers.core import hookenv
-
-
-__all__ = ['ServiceManager', 'ManagerCallback',
-           'PortManagerCallback', 'open_ports', 'close_ports', 'manage_ports',
-           'service_restart', 'service_stop']
-
-
-class ServiceManager(object):
-    def __init__(self, services=None):
-        """
-        Register a list of services, given their definitions.
-
-        Service definitions are dicts in the following formats (all keys except
-        'service' are optional)::
-
-            {
-                "service": <service name>,
-                "required_data": <list of required data contexts>,
-                "provided_data": <list of provided data contexts>,
-                "data_ready": <one or more callbacks>,
-                "data_lost": <one or more callbacks>,
-                "start": <one or more callbacks>,
-                "stop": <one or more callbacks>,
-                "ports": <list of ports to manage>,
-            }
-
-        The 'required_data' list should contain dicts of required data (or
-        dependency managers that act like dicts and know how to collect the data).
-        Only when all items in the 'required_data' list are populated are the list
-        of 'data_ready' and 'start' callbacks executed.  See `is_ready()` for more
-        information.
-
-        The 'provided_data' list should contain relation data providers, most likely
-        a subclass of :class:`charmhelpers.core.services.helpers.RelationContext`,
-        that will indicate a set of data to set on a given relation.
-
-        The 'data_ready' value should be either a single callback, or a list of
-        callbacks, to be called when all items in 'required_data' pass `is_ready()`.
-        Each callback will be called with the service name as the only parameter.
-        After all of the 'data_ready' callbacks are called, the 'start' callbacks
-        are fired.
-
-        The 'data_lost' value should be either a single callback, or a list of
-        callbacks, to be called when a 'required_data' item no longer passes
-        `is_ready()`.  Each callback will be called with the service name as the
-        only parameter.  After all of the 'data_lost' callbacks are called,
-        the 'stop' callbacks are fired.
-
-        The 'start' value should be either a single callback, or a list of
-        callbacks, to be called when starting the service, after the 'data_ready'
-        callbacks are complete.  Each callback will be called with the service
-        name as the only parameter.  This defaults to
-        `[host.service_start, services.open_ports]`.
-
-        The 'stop' value should be either a single callback, or a list of
-        callbacks, to be called when stopping the service.  If the service is
-        being stopped because it no longer has all of its 'required_data', this
-        will be called after all of the 'data_lost' callbacks are complete.
-        Each callback will be called with the service name as the only parameter.
-        This defaults to `[services.close_ports, host.service_stop]`.
-
-        The 'ports' value should be a list of ports to manage.  The default
-        'start' handler will open the ports after the service is started,
-        and the default 'stop' handler will close the ports prior to stopping
-        the service.
-
-
-        Examples:
-
-        The following registers an Upstart service called bingod that depends on
-        a mongodb relation and which runs a custom `db_migrate` function prior to
-        restarting the service, and a Runit service called spadesd::
-
-            manager = services.ServiceManager([
-                {
-                    'service': 'bingod',
-                    'ports': [80, 443],
-                    'required_data': [MongoRelation(), config(), {'my': 'data'}],
-                    'data_ready': [
-                        services.template(source='bingod.conf'),
-                        services.template(source='bingod.ini',
-                                          target='/etc/bingod.ini',
-                                          owner='bingo', perms=0400),
-                    ],
-                },
-                {
-                    'service': 'spadesd',
-                    'data_ready': services.template(source='spadesd_run.j2',
-                                                    target='/etc/sv/spadesd/run',
-                                                    perms=0555),
-                    'start': runit_start,
-                    'stop': runit_stop,
-                },
-            ])
-            manager.manage()
-        """
-        self._ready_file = os.path.join(hookenv.charm_dir(), 'READY-SERVICES.json')
-        self._ready = None
-        self.services = OrderedDict()
-        for service in services or []:
-            service_name = service['service']
-            self.services[service_name] = service
-
-    def manage(self):
-        """
-        Handle the current hook by doing The Right Thing with the registered services.
-        """
-        hookenv._run_atstart()
-        try:
-            hook_name = hookenv.hook_name()
-            if hook_name == 'stop':
-                self.stop_services()
-            else:
-                self.reconfigure_services()
-                self.provide_data()
-        except SystemExit as x:
-            if x.code is None or x.code == 0:
-                hookenv._run_atexit()
-        hookenv._run_atexit()
-
-    def provide_data(self):
-        """
-        Set the relation data for each provider in the ``provided_data`` list.
-
-        A provider must have a `name` attribute, which indicates which relation
-        to set data on, and a `provide_data()` method, which returns a dict of
-        data to set.
-
-        The `provide_data()` method can optionally accept two parameters:
-
-          * ``remote_service`` The name of the remote service that the data will
-            be provided to.  The `provide_data()` method will be called once
-            for each connected service (not unit).  This allows the method to
-            tailor its data to the given service.
-          * ``service_ready`` Whether or not the service definition had all of
-            its requirements met, and thus the ``data_ready`` callbacks run.
-
-        Note that the ``provided_data`` methods are now called **after** the
-        ``data_ready`` callbacks are run.  This gives the ``data_ready`` callbacks
-        a chance to generate any data necessary for the providing to the remote
-        services.
-        """
-        for service_name, service in self.services.items():
-            service_ready = self.is_ready(service_name)
-            for provider in service.get('provided_data', []):
-                for relid in hookenv.relation_ids(provider.name):
-                    units = hookenv.related_units(relid)
-                    if not units:
-                        continue
-                    remote_service = units[0].split('/')[0]
-                    argspec = getargspec(provider.provide_data)
-                    if len(argspec.args) > 1:
-                        data = provider.provide_data(remote_service, service_ready)
-                    else:
-                        data = provider.provide_data()
-                    if data:
-                        hookenv.relation_set(relid, data)
-
-    def reconfigure_services(self, *service_names):
-        """
-        Update all files for one or more registered services, and,
-        if ready, optionally restart them.
-
-        If no service names are given, reconfigures all registered services.
-        """
-        for service_name in service_names or self.services.keys():
-            if self.is_ready(service_name):
-                self.fire_event('data_ready', service_name)
-                self.fire_event('start', service_name, default=[
-                    service_restart,
-                    manage_ports])
-                self.save_ready(service_name)
-            else:
-                if self.was_ready(service_name):
-                    self.fire_event('data_lost', service_name)
-                self.fire_event('stop', service_name, default=[
-                    manage_ports,
-                    service_stop])
-                self.save_lost(service_name)
-
-    def stop_services(self, *service_names):
-        """
-        Stop one or more registered services, by name.
-
-        If no service names are given, stops all registered services.
-        """
-        for service_name in service_names or self.services.keys():
-            self.fire_event('stop', service_name, default=[
-                manage_ports,
-                service_stop])
-
-    def get_service(self, service_name):
-        """
-        Given the name of a registered service, return its service definition.
-        """
-        service = self.services.get(service_name)
-        if not service:
-            raise KeyError('Service not registered: %s' % service_name)
-        return service
-
-    def fire_event(self, event_name, service_name, default=None):
-        """
-        Fire a data_ready, data_lost, start, or stop event on a given service.
-        """
-        service = self.get_service(service_name)
-        callbacks = service.get(event_name, default)
-        if not callbacks:
-            return
-        if not isinstance(callbacks, Iterable):
-            callbacks = [callbacks]
-        for callback in callbacks:
-            if isinstance(callback, ManagerCallback):
-                callback(self, service_name, event_name)
-            else:
-                callback(service_name)
-
-    def is_ready(self, service_name):
-        """
-        Determine if a registered service is ready, by checking its 'required_data'.
-
-        A 'required_data' item can be any mapping type, and is considered ready
-        if `bool(item)` evaluates as True.
-        """
-        service = self.get_service(service_name)
-        reqs = service.get('required_data', [])
-        return all(bool(req) for req in reqs)
-
-    def _load_ready_file(self):
-        if self._ready is not None:
-            return
-        if os.path.exists(self._ready_file):
-            with open(self._ready_file) as fp:
-                self._ready = set(json.load(fp))
-        else:
-            self._ready = set()
-
-    def _save_ready_file(self):
-        if self._ready is None:
-            return
-        with open(self._ready_file, 'w') as fp:
-            json.dump(list(self._ready), fp)
-
-    def save_ready(self, service_name):
-        """
-        Save an indicator that the given service is now data_ready.
-        """
-        self._load_ready_file()
-        self._ready.add(service_name)
-        self._save_ready_file()
-
-    def save_lost(self, service_name):
-        """
-        Save an indicator that the given service is no longer data_ready.
-        """
-        self._load_ready_file()
-        self._ready.discard(service_name)
-        self._save_ready_file()
-
-    def was_ready(self, service_name):
-        """
-        Determine if the given service was previously data_ready.
-        """
-        self._load_ready_file()
-        return service_name in self._ready
-
-
-class ManagerCallback(object):
-    """
-    Special case of a callback that takes the `ServiceManager` instance
-    in addition to the service name.
-
-    Subclasses should implement `__call__` which should accept three parameters:
-
-        * `manager`       The `ServiceManager` instance
-        * `service_name`  The name of the service it's being triggered for
-        * `event_name`    The name of the event that this callback is handling
-    """
-    def __call__(self, manager, service_name, event_name):
-        raise NotImplementedError()
-
-
-class PortManagerCallback(ManagerCallback):
-    """
-    Callback class that will open or close ports, for use as either
-    a start or stop action.
-    """
-    def __call__(self, manager, service_name, event_name):
-        service = manager.get_service(service_name)
-        new_ports = service.get('ports', [])
-        port_file = os.path.join(hookenv.charm_dir(), '.{}.ports'.format(service_name))
-        if os.path.exists(port_file):
-            with open(port_file) as fp:
-                old_ports = fp.read().split(',')
-            for old_port in old_ports:
-                if bool(old_port):
-                    old_port = int(old_port)
-                    if old_port not in new_ports:
-                        hookenv.close_port(old_port)
-        with open(port_file, 'w') as fp:
-            fp.write(','.join(str(port) for port in new_ports))
-        for port in new_ports:
-            if event_name == 'start':
-                hookenv.open_port(port)
-            elif event_name == 'stop':
-                hookenv.close_port(port)
-
-
-def service_stop(service_name):
-    """
-    Wrapper around host.service_stop to prevent spurious "unknown service"
-    messages in the logs.
-    """
-    if host.service_running(service_name):
-        host.service_stop(service_name)
-
-
-def service_restart(service_name):
-    """
-    Wrapper around host.service_restart to prevent spurious "unknown service"
-    messages in the logs.
-    """
-    if host.service_available(service_name):
-        if host.service_running(service_name):
-            host.service_restart(service_name)
-        else:
-            host.service_start(service_name)
-
-
-# Convenience aliases
-open_ports = close_ports = manage_ports = PortManagerCallback()

=== removed file 'tests/charmhelpers/core/services/helpers.py'
--- tests/charmhelpers/core/services/helpers.py	2015-08-19 13:49:53 +0000
+++ tests/charmhelpers/core/services/helpers.py	1970-01-01 00:00:00 +0000
@@ -1,283 +0,0 @@
-# Copyright 2014-2015 Canonical Limited.
-#
-# This file is part of charm-helpers.
-#
-# charm-helpers is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# charm-helpers is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the