Juju Charms Collection
swift-storage package

Merge lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions into lp:~openstack-charmers-archive/charms/trusty/swift-storage/next

  1. Trusty Tahr (14.04)
  2. add-pause-resume-actions
  3. Merge into next
Proposed by Adam Collard
Status: Superseded
Proposed branch: lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions
Merge into: lp:~openstack-charmers-archive/charms/trusty/swift-storage/next
Diff against target: 1145 lines (+603/-135)
19 files modified
.coveragerc (+1/-0)
Makefile (+2/-0)
actions.yaml (+5/-0)
actions/actions.py (+93/-0)
charmhelpers/cli/__init__.py (+1/-5)
charmhelpers/cli/commands.py (+4/-4)
charmhelpers/contrib/openstack/amulet/deployment.py (+2/-2)
charmhelpers/contrib/openstack/utils.py (+51/-14)
charmhelpers/contrib/storage/linux/utils.py (+3/-2)
charmhelpers/core/hookenv.py (+1/-20)
charmhelpers/core/host.py (+2/-2)
charmhelpers/fetch/__init__.py (+8/-0)
lib/swift_storage_utils.py (+1/-15)
setup.cfg (+1/-1)
tests/00-setup (+1/-0)
tests/basic_deployment.py (+102/-47)
tests/charmhelpers/contrib/amulet/utils.py (+84/-21)
tests/charmhelpers/contrib/openstack/amulet/deployment.py (+2/-2)
unit_tests/test_actions.py (+239/-0)
To merge this branch: bzr merge lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions
Reviewer Review Type Date Requested Status
Landscape Pending
OpenStack Charmers Pending
Review via email: mp+267681@code.launchpad.net

This proposal has been superseded by a proposal from 2015-08-17.

Description of the change

This branch adds actions for pausing and resuming services on swift-storage units.

In addition to just stopping the services, the actions also set the status of the unit to be "maintenance" with a message on how to resume - I'm not attached to the message, bikesheds welcome :).

Further refinements to the charm to support pause mode will be coming in follow-up branches, notably guarding calls to service_start and service_restart to prevent config-changed, *-relation-changed and other hooks from (re)-starting a unit which should be paused.

To post a comment you must log in.
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7869 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7869/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7290 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7290/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7871 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7871/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7292 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7292/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5759 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12057851/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5759/

lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions updated
88. By Adam Collard

Fix flake8 findings

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7922 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7922/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7340 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7340/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5763 swift-storage-next for adam-collard mp267681
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5763/

Revision history for this message
Ryan Beisner (1chb1n) wrote :

RE: Amulet tests

Thank for your contribution and clean-up here. Backgrounder: I've not gotten down the list to swift-* charm amulet test updates, but intend to complete the vivid enablement in 15.08 and wily in 15.09. There are also a number of refactors and helper optimizations, which we're making across all of the os-charm amulet tests, so I'll be shifting these around a bit when I get down the list to the swift-*s.

1 easy change that I'd request: Please chmod -x tests/basic_deployment.py ... set not-executable, as it's just a module with class definitions called by the actual test executables (NNN-<ubuntu>-<openstack> files which are set +x for known-working combos).

Take note that juju test (and bundletester) will juju bootstrap then execute anything in the tests/ dir which is +x, and that any yamls in the tests/ dir will be assumed by bundletester to be a test definition. While we are prepping to switch completely to bundletester, our first priority with the amulet tests is in updating the horizon of u:os test coverage to catch up with charm development efforts. Then we will resume the bundletester effort.

Thanks again, holler with any ?s.

lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions updated
89. By Adam Collard

Remove #! from basic_deployment.py (beisner's review)

90. By Adam Collard

chmod -x

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7926 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7926/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7344 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7344/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5767 swift-storage-next for adam-collard mp267681
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5767/

lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions updated
91. By Adam Collard

Sync charm-helpers

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7927 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7927/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7345 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7345/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5768 swift-storage-next for adam-collard mp267681
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5768/

lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions updated
92. By Adam Collard

Switch tests to using validate_service_by_name()

93. By Adam Collard

WiP amulet test

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7929 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7929/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7347 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7347/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5770 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12063967/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5770/

lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions updated
94. By Adam Collard

validate_serviceS_by_name (fix typo)

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7983 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7983/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7397 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7397/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5775 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 124
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12070002/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5775/

lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions updated
95. By Adam Collard

Move tests into basic_deployment so they get ran for each combination

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7985 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7985/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7399 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7399/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5777 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12070240/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5777/

lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions updated
96. By Adam Collard

Move actions test to the end so basic service tests are done upfront

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7987 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7987/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7401 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7401/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5779 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12070492/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5779/

Revision history for this message
Ryan Beisner (1chb1n) wrote :

Hi. Kicking in with some hopefully helpful hints. I see the proposed test is failing when trying to confirm a system service that doesn't appear to be present at a particular release combo.

Since the basic_deployment.py set of tests is expected to pass for all supported release combos, each test_ method may need to use release detection logic as needed. In some cases, openstack components have shifted options, sections, etc., in conf files. In other cases, some system services may or may not be present across that entire timeline. There was particularly a lot of upstream shifting-around introduced at >= Kilo ... and < Icehouse.

As a usage example, the heat amulet test detects and checks things slightly differently here:
http://bazaar.launchpad.net/~openstack-charmers/charms/trusty/heat/next/view/head:/tests/basic_deployment.py#L313

Which ultimately uses this for comparison:
http://bazaar.launchpad.net/~charm-helpers/charm-helpers/devel/view/head:/charmhelpers/contrib/openstack/amulet/deployment.py#L109

lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions updated
97. By Adam Collard

Fix tests

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #8042 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8042/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7453 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7453/

Revision history for this message
Adam Collard (adam-collard) wrote :

> Hi. Kicking in with some hopefully helpful hints. I see the proposed test is
> failing when trying to confirm a system service that doesn't appear to be
> present at a particular release combo.

That was my first guess too, fortunately (or unfortunately?) it wasn't that issue. The problem was the utils code for looking at what processes are running uses "pidof $name" but when $name is the name of a script it fails (using the -x option works as expected). Instead of yet another round of fixing charm-helpers, I opted to just roll my own for this test - there are comments in the test to further explain why.

In the absence of access to a testing environment that could satisfy the requirements, I was using UOSCI bot to run my tests for me. Now that's been resolved I have (hopefully!) got some green tests :)

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5786 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12078138/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5786/

lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions updated
98. By Adam Collard

Only care about swift-container-sync for >= Icehouse

99. By Adam Collard

s/pop/remove (d'oh!)

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #8044 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8044/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7455 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7455/

lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions updated
100. By Adam Collard

More z's

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5788 swift-storage-next for adam-collard mp267681
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5788/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #8045 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8045/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7456 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7456/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5789 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 124
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12079074/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5789/

Revision history for this message
Ryan Beisner (1chb1n) wrote :

RE: amulet tests

Thank you for this work!

Just a couple of things that will make future life easier:

_run_action and _wait_on_action:
Please consider landing these in charmhelpers/contrib/amulet/utils.py, as they will be increasingly-common needs.

_assert_services:
Please consider enhancing the existing validate_unit_process_ids amulet helper to fit the needs here. I think it would be a fairly easy mod. We could do -x on pidof in all cases (or add a bool with default False to "check scripts too" if you want to be safer about it). Then add a boolean postive/negative switch to be able to invert its expectations (like you've done with should_run). fwiw, expect_success (default True) is such a bool name that I've used elsewhere for the same purpose in a different context (rmq wip).

lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions updated
101. By Adam Collard

Depend on amulet-actions-helpers branch of charm-helpers, use those actin helpers (beisner's review)

102. By Adam Collard

Point charm-helpers to latest branch, update tests to use helpers. Add comment about why validate_unit_process_ids is not helpful.

103. By Adam Collard

run_action takes the sentry object, not the unit name

104. By Adam Collard

Use wait_on_action helper for resume test too

105. By Adam Collard

Sync charm-helpers

106. By Adam Collard

Revert to pointing at regular charm-helpers

107. By Adam Collard

Merge in from fix-service-status

Revision history for this message
Adam Collard (adam-collard) wrote :

Hi Ryan,

Thanks for taking a look at this.

On Fri, 14 Aug 2015 at 17:10 Ryan Beisner <email address hidden>
wrote:

> RE: amulet tests
>
> Thank you for this work!
>
> Just a couple of things that will make future life easier:
>
> _run_action and _wait_on_action:
> Please consider landing these in charmhelpers/contrib/amulet/utils.py, as
> they will be increasingly-common needs.
>

Done. See charm-helpers r428

>
> _assert_services:
> Please consider enhancing the existing validate_unit_process_ids amulet
> helper to fit the needs here. I think it would be a fairly easy mod. We
> could do -x on pidof in all cases (or add a bool with default False to
> "check scripts too" if you want to be safer about it). Then add a boolean
> postive/negative switch to be able to invert its expectations (like you've
> done with should_run). fwiw, expect_success (default True) is such a bool
> name that I've used elsewhere for the same purpose in a different context
> (rmq wip).
>

I modified get_process_id_list and get_unit_process_ids but didn't end up
using validate_unit_process_ids because the validation is already done for
me in the previous two methods. See comment in the test itself.

I've split the branch up so that the new code is cleanly separated from the
fixes I made to existing code (see pre-req branch).

--
>
> https://code.launchpad.net/~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions/+merge/267681
> You are the owner of
> lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions.
>
> Launchpad-Message-Rationale: Owner
> Launchpad-Notification-Type: code-review
> Launchpad-Branch:
> ~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions
>

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.coveragerc'
2--- .coveragerc 2015-08-11 08:09:29 +0000
3+++ .coveragerc 2015-08-17 13:44:46 +0000
4@@ -3,5 +3,6 @@
5 exclude_lines =
6 if __name__ == .__main__.:
7 include=
8+ actions/actions.py
9 hooks/swift_storage_*
10 lib/swift_storage_*
11
12=== modified file 'Makefile'
13--- Makefile 2015-08-04 11:48:35 +0000
14+++ Makefile 2015-08-17 13:44:46 +0000
15@@ -30,3 +30,5 @@
16 publish: lint test
17 bzr push lp:charms/swift-storage
18 bzr push lp:charms/trusty/swift-storage
19+
20+.PHONY: lint unit_test test sync publish
21
22=== added directory 'actions'
23=== added file 'actions.yaml'
24--- actions.yaml 1970-01-01 00:00:00 +0000
25+++ actions.yaml 2015-08-17 13:44:46 +0000
26@@ -0,0 +1,5 @@
27+pause:
28+ description: Pause the swift-storage unit. This action will stop Swift services.
29+resume:
30+ description: Resume the swift-storage unit. This action will start Swift services.
31+
32
33=== added file 'actions/__init__.py'
34=== added file 'actions/actions.py'
35--- actions/actions.py 1970-01-01 00:00:00 +0000
36+++ actions/actions.py 2015-08-17 13:44:46 +0000
37@@ -0,0 +1,93 @@
38+#!/usr/bin/python
39+
40+import argparse
41+import os
42+import sys
43+import yaml
44+
45+from charmhelpers.core.host import service_pause, service_resume
46+from charmhelpers.core.hookenv import action_fail, status_set
47+from charmhelpers.contrib.openstack.utils import get_os_codename_package
48+
49+from lib.swift_storage_utils import SWIFT_SVCS
50+
51+
52+def _get_services():
53+ """Return a list of services that need to be (un)paused."""
54+ services = SWIFT_SVCS[:]
55+ # Before Icehouse there was no swift-container-sync
56+ if get_os_codename_package("swift-container") < "icehouse":
57+ services.remove("swift-container-sync")
58+ return services
59+
60+
61+def get_action_parser(actions_yaml_path, action_name,
62+ get_services=_get_services):
63+ """Make an argparse.ArgumentParser seeded from actions.yaml definitions."""
64+ with open(actions_yaml_path) as fh:
65+ doc = yaml.load(fh)[action_name]["description"]
66+ parser = argparse.ArgumentParser(description=doc)
67+ parser.add_argument("--services", default=get_services())
68+ # TODO: Add arguments for params defined in the actions.yaml
69+ return parser
70+
71+
72+def pause(args):
73+ """Pause all the swift services.
74+
75+ @raises Exception if any services fail to stop
76+ """
77+ for service in args.services:
78+ stopped = service_pause(service)
79+ if not stopped:
80+ raise Exception("{} didn't stop cleanly.".format(service))
81+ status_set(
82+ "maintenance", "Paused. Use 'resume' action to resume normal service.")
83+
84+
85+def resume(args):
86+ """Resume all the swift services.
87+
88+ @raises Exception if any services fail to start
89+ """
90+ for service in args.services:
91+ started = service_resume(service)
92+ if not started:
93+ raise Exception("{} didn't start cleanly.".format(service))
94+ status_set("active", "")
95+
96+
97+# A dictionary of all the defined actions to callables (which take
98+# parsed arguments).
99+ACTIONS = {"pause": pause, "resume": resume}
100+
101+
102+def main(argv):
103+ action_name = _get_action_name()
104+ actions_yaml_path = _get_actions_yaml_path()
105+ parser = get_action_parser(actions_yaml_path, action_name)
106+ args = parser.parse_args(argv)
107+ try:
108+ action = ACTIONS[action_name]
109+ except KeyError:
110+ return "Action %s undefined" % action_name
111+ else:
112+ try:
113+ action(args)
114+ except Exception as e:
115+ action_fail(str(e))
116+
117+
118+def _get_action_name():
119+ """Return the name of the action."""
120+ return os.path.basename(__file__)
121+
122+
123+def _get_actions_yaml_path():
124+ """Return the path to actions.yaml"""
125+ cwd = os.path.dirname(__file__)
126+ return os.path.join(cwd, "..", "actions.yaml")
127+
128+
129+if __name__ == "__main__":
130+ sys.exit(main(sys.argv[1:]))
131
132=== added symlink 'actions/charmhelpers'
133=== target is u'../charmhelpers/'
134=== added symlink 'actions/lib'
135=== target is u'../lib'
136=== added symlink 'actions/pause'
137=== target is u'actions.py'
138=== added symlink 'actions/resume'
139=== target is u'actions.py'
140=== modified file 'charmhelpers/cli/__init__.py'
141--- charmhelpers/cli/__init__.py 2015-07-31 13:11:32 +0000
142+++ charmhelpers/cli/__init__.py 2015-08-17 13:44:46 +0000
143@@ -152,15 +152,11 @@
144 arguments = self.argument_parser.parse_args()
145 argspec = inspect.getargspec(arguments.func)
146 vargs = []
147- kwargs = {}
148 for arg in argspec.args:
149 vargs.append(getattr(arguments, arg))
150 if argspec.varargs:
151 vargs.extend(getattr(arguments, argspec.varargs))
152- if argspec.keywords:
153- for kwarg in argspec.keywords.items():
154- kwargs[kwarg] = getattr(arguments, kwarg)
155- output = arguments.func(*vargs, **kwargs)
156+ output = arguments.func(*vargs)
157 if getattr(arguments.func, '_cli_test_command', False):
158 self.exit_code = 0 if output else 1
159 output = ''
160
161=== modified file 'charmhelpers/cli/commands.py'
162--- charmhelpers/cli/commands.py 2015-07-31 13:11:32 +0000
163+++ charmhelpers/cli/commands.py 2015-08-17 13:44:46 +0000
164@@ -26,7 +26,7 @@
165 """
166 Import the sub-modules which have decorated subcommands to register with chlp.
167 """
168-import host # noqa
169-import benchmark # noqa
170-import unitdata # noqa
171-from charmhelpers.core import hookenv # noqa
172+from . import host # noqa
173+from . import benchmark # noqa
174+from . import unitdata # noqa
175+from . import hookenv # noqa
176
177=== modified file 'charmhelpers/contrib/openstack/amulet/deployment.py'
178--- charmhelpers/contrib/openstack/amulet/deployment.py 2015-07-29 10:49:43 +0000
179+++ charmhelpers/contrib/openstack/amulet/deployment.py 2015-08-17 13:44:46 +0000
180@@ -44,7 +44,7 @@
181 Determine if the local branch being tested is derived from its
182 stable or next (dev) branch, and based on this, use the corresonding
183 stable or next branches for the other_services."""
184- base_charms = ['mysql', 'mongodb']
185+ base_charms = ['mysql', 'mongodb', 'nrpe']
186
187 if self.series in ['precise', 'trusty']:
188 base_series = self.series
189@@ -81,7 +81,7 @@
190 'ceph-osd', 'ceph-radosgw']
191 # Most OpenStack subordinate charms do not expose an origin option
192 # as that is controlled by the principle.
193- ignore = ['cinder-ceph', 'hacluster', 'neutron-openvswitch']
194+ ignore = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe']
195
196 if self.openstack:
197 for svc in services:
198
199=== modified file 'charmhelpers/contrib/openstack/utils.py'
200--- charmhelpers/contrib/openstack/utils.py 2015-07-29 10:49:43 +0000
201+++ charmhelpers/contrib/openstack/utils.py 2015-08-17 13:44:46 +0000
202@@ -24,6 +24,7 @@
203 import json
204 import os
205 import sys
206+import re
207
208 import six
209 import yaml
210@@ -69,7 +70,6 @@
211 DISTRO_PROPOSED = ('deb http://archive.ubuntu.com/ubuntu/ %s-proposed '
212 'restricted main multiverse universe')
213
214-
215 UBUNTU_OPENSTACK_RELEASE = OrderedDict([
216 ('oneiric', 'diablo'),
217 ('precise', 'essex'),
218@@ -118,6 +118,34 @@
219 ('2.3.0', 'liberty'),
220 ])
221
222+# >= Liberty version->codename mapping
223+PACKAGE_CODENAMES = {
224+ 'nova-common': OrderedDict([
225+ ('12.0.0', 'liberty'),
226+ ]),
227+ 'neutron-common': OrderedDict([
228+ ('7.0.0', 'liberty'),
229+ ]),
230+ 'cinder-common': OrderedDict([
231+ ('7.0.0', 'liberty'),
232+ ]),
233+ 'keystone': OrderedDict([
234+ ('8.0.0', 'liberty'),
235+ ]),
236+ 'horizon-common': OrderedDict([
237+ ('8.0.0', 'liberty'),
238+ ]),
239+ 'ceilometer-common': OrderedDict([
240+ ('5.0.0', 'liberty'),
241+ ]),
242+ 'heat-common': OrderedDict([
243+ ('5.0.0', 'liberty'),
244+ ]),
245+ 'glance-common': OrderedDict([
246+ ('11.0.0', 'liberty'),
247+ ]),
248+}
249+
250 DEFAULT_LOOPBACK_SIZE = '5G'
251
252
253@@ -201,20 +229,29 @@
254 error_out(e)
255
256 vers = apt.upstream_version(pkg.current_ver.ver_str)
257+ match = re.match('^(\d)\.(\d)\.(\d)', vers)
258+ if match:
259+ vers = match.group(0)
260
261- try:
262- if 'swift' in pkg.name:
263- swift_vers = vers[:5]
264- if swift_vers not in SWIFT_CODENAMES:
265- # Deal with 1.10.0 upward
266- swift_vers = vers[:6]
267- return SWIFT_CODENAMES[swift_vers]
268- else:
269- vers = vers[:6]
270- return OPENSTACK_CODENAMES[vers]
271- except KeyError:
272- e = 'Could not determine OpenStack codename for version %s' % vers
273- error_out(e)
274+ # >= Liberty independent project versions
275+ if (package in PACKAGE_CODENAMES and
276+ vers in PACKAGE_CODENAMES[package]):
277+ return PACKAGE_CODENAMES[package][vers]
278+ else:
279+ # < Liberty co-ordinated project versions
280+ try:
281+ if 'swift' in pkg.name:
282+ swift_vers = vers[:5]
283+ if swift_vers not in SWIFT_CODENAMES:
284+ # Deal with 1.10.0 upward
285+ swift_vers = vers[:6]
286+ return SWIFT_CODENAMES[swift_vers]
287+ else:
288+ vers = vers[:6]
289+ return OPENSTACK_CODENAMES[vers]
290+ except KeyError:
291+ e = 'Could not determine OpenStack codename for version %s' % vers
292+ error_out(e)
293
294
295 def get_os_version_package(pkg, fatal=True):
296
297=== modified file 'charmhelpers/contrib/storage/linux/utils.py'
298--- charmhelpers/contrib/storage/linux/utils.py 2015-07-29 10:49:43 +0000
299+++ charmhelpers/contrib/storage/linux/utils.py 2015-08-17 13:44:46 +0000
300@@ -43,9 +43,10 @@
301
302 :param block_device: str: Full path of block device to clean.
303 '''
304+ # https://github.com/ceph/ceph/commit/fdd7f8d83afa25c4e09aaedd90ab93f3b64a677b
305 # sometimes sgdisk exits non-zero; this is OK, dd will clean up
306- call(['sgdisk', '--zap-all', '--mbrtogpt',
307- '--clear', block_device])
308+ call(['sgdisk', '--zap-all', '--', block_device])
309+ call(['sgdisk', '--clear', '--mbrtogpt', '--', block_device])
310 dev_end = check_output(['blockdev', '--getsz',
311 block_device]).decode('UTF-8')
312 gpt_end = int(dev_end.split()[0]) - 100
313
314=== modified file 'charmhelpers/core/hookenv.py'
315--- charmhelpers/core/hookenv.py 2015-08-03 14:00:44 +0000
316+++ charmhelpers/core/hookenv.py 2015-08-17 13:44:46 +0000
317@@ -34,23 +34,6 @@
318 import tempfile
319 from subprocess import CalledProcessError
320
321-try:
322- from charmhelpers.cli import cmdline
323-except ImportError as e:
324- # due to the anti-pattern of partially synching charmhelpers directly
325- # into charms, it's possible that charmhelpers.cli is not available;
326- # if that's the case, they don't really care about using the cli anyway,
327- # so mock it out
328- if str(e) == 'No module named cli':
329- class cmdline(object):
330- @classmethod
331- def subcommand(cls, *args, **kwargs):
332- def _wrap(func):
333- return func
334- return _wrap
335- else:
336- raise
337-
338 import six
339 if not six.PY3:
340 from UserDict import UserDict
341@@ -91,6 +74,7 @@
342 res = func(*args, **kwargs)
343 cache[key] = res
344 return res
345+ wrapper._wrapped = func
346 return wrapper
347
348
349@@ -190,7 +174,6 @@
350 return os.environ.get('JUJU_RELATION', None)
351
352
353-@cmdline.subcommand()
354 @cached
355 def relation_id(relation_name=None, service_or_unit=None):
356 """The relation ID for the current or a specified relation"""
357@@ -216,13 +199,11 @@
358 return os.environ.get('JUJU_REMOTE_UNIT', None)
359
360
361-@cmdline.subcommand()
362 def service_name():
363 """The name service group this unit belongs to"""
364 return local_unit().split('/')[0]
365
366
367-@cmdline.subcommand()
368 @cached
369 def remote_service_name(relid=None):
370 """The remote service name for a given relation-id (or the current relation)"""
371
372=== modified file 'charmhelpers/core/host.py'
373--- charmhelpers/core/host.py 2015-07-29 10:49:43 +0000
374+++ charmhelpers/core/host.py 2015-08-17 13:44:46 +0000
375@@ -72,7 +72,7 @@
376 stopped = service_stop(service_name)
377 # XXX: Support systemd too
378 override_path = os.path.join(
379- init_dir, '{}.conf.override'.format(service_name))
380+ init_dir, '{}.override'.format(service_name))
381 with open(override_path, 'w') as fh:
382 fh.write("manual\n")
383 return stopped
384@@ -86,7 +86,7 @@
385 if init_dir is None:
386 init_dir = "/etc/init"
387 override_path = os.path.join(
388- init_dir, '{}.conf.override'.format(service_name))
389+ init_dir, '{}.override'.format(service_name))
390 if os.path.exists(override_path):
391 os.unlink(override_path)
392 started = service_start(service_name)
393
394=== modified file 'charmhelpers/fetch/__init__.py'
395--- charmhelpers/fetch/__init__.py 2015-07-29 10:49:43 +0000
396+++ charmhelpers/fetch/__init__.py 2015-08-17 13:44:46 +0000
397@@ -90,6 +90,14 @@
398 'kilo/proposed': 'trusty-proposed/kilo',
399 'trusty-kilo/proposed': 'trusty-proposed/kilo',
400 'trusty-proposed/kilo': 'trusty-proposed/kilo',
401+ # Liberty
402+ 'liberty': 'trusty-updates/liberty',
403+ 'trusty-liberty': 'trusty-updates/liberty',
404+ 'trusty-liberty/updates': 'trusty-updates/liberty',
405+ 'trusty-updates/liberty': 'trusty-updates/liberty',
406+ 'liberty/proposed': 'trusty-proposed/liberty',
407+ 'trusty-liberty/proposed': 'trusty-proposed/liberty',
408+ 'trusty-proposed/liberty': 'trusty-proposed/liberty',
409 }
410
411 # The order of this list is very important. Handlers should be listed in from
412
413=== modified file 'lib/swift_storage_utils.py'
414--- lib/swift_storage_utils.py 2015-07-17 09:57:19 +0000
415+++ lib/swift_storage_utils.py 2015-08-17 13:44:46 +0000
416@@ -83,21 +83,7 @@
417 'swift-object-updater', 'swift-object-replicator'
418 ]
419
420-SWIFT_SVCS = [
421- 'swift-account-auditor',
422- 'swift-account-reaper',
423- 'swift-account-replicator',
424- 'swift-account-server',
425- 'swift-container-auditor',
426- 'swift-container-replicator',
427- 'swift-container-server',
428- 'swift-container-sync',
429- 'swift-container-updater',
430- 'swift-object-auditor',
431- 'swift-object-replicator',
432- 'swift-object-server',
433- 'swift-object-updater',
434- ]
435+SWIFT_SVCS = ACCOUNT_SVCS + CONTAINER_SVCS + OBJECT_SVCS
436
437 RESTART_MAP = {
438 '/etc/rsync-juju.d/050-swift-storage.conf': ['rsync'],
439
440=== modified file 'setup.cfg'
441--- setup.cfg 2015-08-11 08:09:29 +0000
442+++ setup.cfg 2015-08-17 13:44:46 +0000
443@@ -2,5 +2,5 @@
444 verbosity=2
445 with-coverage=1
446 cover-erase=1
447-cover-package=lib,hooks.swift_storage_hooks
448+cover-package=actions.actions,lib,hooks.swift_storage_hooks
449
450
451=== modified file 'tests/00-setup'
452--- tests/00-setup 2014-09-29 21:14:05 +0000
453+++ tests/00-setup 2015-08-17 13:44:46 +0000
454@@ -7,5 +7,6 @@
455 sudo apt-get install --yes python-amulet \
456 python-swiftclient \
457 python-glanceclient \
458+ python-heatclient \
459 python-keystoneclient \
460 python-novaclient
461
462=== modified file 'tests/basic_deployment.py'
463--- tests/basic_deployment.py 2015-04-16 21:31:35 +0000
464+++ tests/basic_deployment.py 2015-08-17 13:44:46 +0000
465@@ -1,5 +1,3 @@
466-#!/usr/bin/python
467-
468 import amulet
469 import swiftclient
470
471@@ -9,8 +7,7 @@
472
473 from charmhelpers.contrib.openstack.amulet.utils import (
474 OpenStackAmuletUtils,
475- DEBUG, # flake8: noqa
476- ERROR
477+ DEBUG,
478 )
479
480 # Use DEBUG to turn on debug logging
481@@ -46,12 +43,12 @@
482 def _add_relations(self):
483 """Add all of the relations for the services."""
484 relations = {
485- 'keystone:shared-db': 'mysql:shared-db',
486- 'swift-proxy:identity-service': 'keystone:identity-service',
487- 'swift-storage:swift-storage': 'swift-proxy:swift-storage',
488- 'glance:identity-service': 'keystone:identity-service',
489- 'glance:shared-db': 'mysql:shared-db',
490- 'glance:object-store': 'swift-proxy:object-store'
491+ 'keystone:shared-db': 'mysql:shared-db',
492+ 'swift-proxy:identity-service': 'keystone:identity-service',
493+ 'swift-storage:swift-storage': 'swift-proxy:swift-storage',
494+ 'glance:identity-service': 'keystone:identity-service',
495+ 'glance:shared-db': 'mysql:shared-db',
496+ 'glance:object-store': 'swift-proxy:object-store'
497 }
498 super(SwiftStorageBasicDeployment, self)._add_relations(relations)
499
500@@ -59,9 +56,11 @@
501 """Configure all of the services."""
502 keystone_config = {'admin-password': 'openstack',
503 'admin-token': 'ubuntutesting'}
504- swift_proxy_config = {'zone-assignment': 'manual',
505- 'replicas': '1',
506- 'swift-hash': 'fdfef9d4-8b06-11e2-8ac0-531c923c8fae'}
507+ swift_proxy_config = {
508+ 'zone-assignment': 'manual',
509+ 'replicas': '1',
510+ 'swift-hash': 'fdfef9d4-8b06-11e2-8ac0-531c923c8fae'
511+ }
512 swift_storage_config = {'zone': '1',
513 'block-device': 'vdb',
514 'overwrite': 'true'}
515@@ -89,15 +88,16 @@
516 self.glance = u.authenticate_glance_admin(self.keystone)
517
518 # Authenticate swift user
519- keystone_relation = self.keystone_sentry.relation('identity-service',
520- 'swift-proxy:identity-service')
521+ keystone_relation = self.keystone_sentry.relation(
522+ 'identity-service', 'swift-proxy:identity-service')
523 ep = self.keystone.service_catalog.url_for(service_type='identity',
524 endpoint_type='publicURL')
525- self.swift = swiftclient.Connection(authurl=ep,
526- user=keystone_relation['service_username'],
527- key=keystone_relation['service_password'],
528- tenant_name=keystone_relation['service_tenant'],
529- auth_version='2.0')
530+ self.swift = swiftclient.Connection(
531+ authurl=ep,
532+ user=keystone_relation['service_username'],
533+ key=keystone_relation['service_password'],
534+ tenant_name=keystone_relation['service_tenant'],
535+ auth_version='2.0')
536
537 # Create a demo tenant/role/user
538 self.demo_tenant = 'demoTenant'
539@@ -122,29 +122,30 @@
540 def test_services(self):
541 """Verify the expected services are running on the corresponding
542 service units."""
543- swift_storage_services = ['status swift-account',
544- 'status swift-account-auditor',
545- 'status swift-account-reaper',
546- 'status swift-account-replicator',
547- 'status swift-container',
548- 'status swift-container-auditor',
549- 'status swift-container-replicator',
550- 'status swift-container-updater',
551- 'status swift-object',
552- 'status swift-object-auditor',
553- 'status swift-object-replicator',
554- 'status swift-object-updater']
555+ swift_storage_services = ['swift-account',
556+ 'swift-account-auditor',
557+ 'swift-account-reaper',
558+ 'swift-account-replicator',
559+ 'swift-container',
560+ 'swift-container-auditor',
561+ 'swift-container-replicator',
562+ 'swift-container-updater',
563+ 'swift-object',
564+ 'swift-object-auditor',
565+ 'swift-object-replicator',
566+ 'swift-object-updater']
567 if self._get_openstack_release() >= self.precise_icehouse:
568- swift_storage_services.append('status swift-container-sync')
569- commands = {
570- self.mysql_sentry: ['status mysql'],
571- self.keystone_sentry: ['status keystone'],
572- self.glance_sentry: ['status glance-registry', 'status glance-api'],
573- self.swift_proxy_sentry: ['status swift-proxy'],
574+ swift_storage_services.append('swift-container-sync')
575+ service_names = {
576+ self.mysql_sentry: ['mysql'],
577+ self.keystone_sentry: ['keystone'],
578+ self.glance_sentry: [
579+ 'glance-registry', 'glance-api'],
580+ self.swift_proxy_sentry: ['swift-proxy'],
581 self.swift_storage_sentry: swift_storage_services
582 }
583
584- ret = u.validate_services(commands)
585+ ret = u.validate_services_by_name(service_names)
586 if ret:
587 amulet.raise_status(amulet.FAIL, msg=ret)
588
589@@ -303,8 +304,8 @@
590 file."""
591 unit = self.swift_storage_sentry
592 conf = '/etc/swift/swift.conf'
593- swift_proxy_relation = self.swift_proxy_sentry.relation('swift-storage',
594- 'swift-storage:swift-storage')
595+ swift_proxy_relation = self.swift_proxy_sentry.relation(
596+ 'swift-storage', 'swift-storage:swift-storage')
597 expected = {
598 'swift_hash_path_suffix': swift_proxy_relation['swift_hash']
599 }
600@@ -405,7 +406,8 @@
601
602 def test_image_create(self):
603 """Create an instance in glance, which is backed by swift, and validate
604- that some of the metadata for the image match in glance and swift."""
605+ that some of the metadata for the image match in glance and swift.
606+ """
607 # NOTE(coreycb): Skipping failing test on folsom until resolved. On
608 # folsom only, uploading an image to glance gets 400 Bad
609 # Request - Error uploading image: (error): [Errno 111]
610@@ -435,7 +437,8 @@
611 # Validate that swift object's checksum/size match that from glance
612 headers, containers = self.swift.get_account()
613 if len(containers) != 1:
614- msg = "Expected 1 swift container, found {}".format(len(containers))
615+ msg = "Expected 1 swift container, found {}".format(
616+ len(containers))
617 amulet.raise_status(amulet.FAIL, msg=msg)
618
619 container_name = containers[0].get('name')
620@@ -449,14 +452,66 @@
621 swift_object_md5 = objects[0].get('hash')
622
623 if glance_image_size != swift_object_size:
624- msg = "Glance image size {} != swift object size {}".format( \
625- glance_image_size, swift_object_size)
626+ msg = "Glance image size {} != swift object size {}".format(
627+ glance_image_size, swift_object_size)
628 amulet.raise_status(amulet.FAIL, msg=msg)
629
630 if glance_image_md5 != swift_object_md5:
631- msg = "Glance image hash {} != swift object hash {}".format( \
632- glance_image_md5, swift_object_md5)
633+ msg = "Glance image hash {} != swift object hash {}".format(
634+ glance_image_md5, swift_object_md5)
635 amulet.raise_status(amulet.FAIL, msg=msg)
636
637 # Cleanup
638 u.delete_image(self.glance, image)
639+
640+ def _assert_services(self, should_run):
641+ swift_storage_services = ['swift-account-auditor',
642+ 'swift-account-reaper',
643+ 'swift-account-replicator',
644+ 'swift-account-server',
645+ 'swift-container-auditor',
646+ 'swift-container-replicator',
647+ 'swift-container-server',
648+ 'swift-container-sync',
649+ 'swift-container-updater',
650+ 'swift-object-auditor',
651+ 'swift-object-replicator',
652+ 'swift-object-server',
653+ 'swift-object-updater']
654+ if self._get_openstack_release() < self.precise_icehouse:
655+ swift_storage_services.remove('swift-container-sync')
656+
657+ u.get_unit_process_ids(
658+ {self.swift_storage_sentry: swift_storage_services},
659+ expect_success=should_run)
660+ # No point using validate_unit_process_ids, since we don't
661+ # care about how many PIDs, merely that they're running, so
662+ # would populate expected with either True or False. This
663+ # validation is already performed in get_process_id_list
664+
665+ def _test_pause(self):
666+ u.log.info("Testing pause action")
667+ self._assert_services(should_run=True)
668+ pause_action_id = u.run_action(self.swift_storage_sentry, "pause")
669+ assert u.wait_on_action(pause_action_id), "Pause action failed."
670+
671+ self._assert_services(should_run=False)
672+
673+ def _test_resume(self):
674+ u.log.info("Testing resume action")
675+ # service is left paused by _test_pause
676+ self._assert_services(should_run=False)
677+ resume_action_id = u.run_action(self.swift_storage_sentry, "resume")
678+ assert u.wait_on_action(resume_action_id), "Resume action failed."
679+
680+ self._assert_services(should_run=True)
681+
682+ def test_z_actions(self):
683+ """Pause and then resume swift-storage.
684+
685+ Note(sparkiegeek): The method name with the _z_ is a little odd
686+ but it forces the test to run last. It just makes things
687+ easier because restarting services requires re-authorization.
688+ """
689+ self._test_pause()
690+ self._test_resume()
691
692=== modified file 'tests/charmhelpers/contrib/amulet/utils.py'
693--- tests/charmhelpers/contrib/amulet/utils.py 2015-07-29 10:49:43 +0000
694+++ tests/charmhelpers/contrib/amulet/utils.py 2015-08-17 13:44:46 +0000
695@@ -14,17 +14,23 @@
696 # You should have received a copy of the GNU Lesser General Public License
697 # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
698
699-import amulet
700-import ConfigParser
701-import distro_info
702 import io
703+import json
704 import logging
705 import os
706 import re
707-import six
708+import subprocess
709 import sys
710 import time
711-import urlparse
712+
713+import amulet
714+import distro_info
715+import six
716+from six.moves import configparser
717+if six.PY3:
718+ from urllib import parse as urlparse
719+else:
720+ import urlparse
721
722
723 class AmuletUtils(object):
724@@ -142,19 +148,23 @@
725
726 for service_name in services_list:
727 if (self.ubuntu_releases.index(release) >= systemd_switch or
728- service_name == "rabbitmq-server"):
729- # init is systemd
730+ service_name in ['rabbitmq-server', 'apache2']):
731+ # init is systemd (or regular sysv)
732 cmd = 'sudo service {} status'.format(service_name)
733+ output, code = sentry_unit.run(cmd)
734+ service_running = code == 0
735 elif self.ubuntu_releases.index(release) < systemd_switch:
736 # init is upstart
737 cmd = 'sudo status {}'.format(service_name)
738+ output, code = sentry_unit.run(cmd)
739+ service_running = code == 0 and "start/running" in output
740
741- output, code = sentry_unit.run(cmd)
742 self.log.debug('{} `{}` returned '
743 '{}'.format(sentry_unit.info['unit_name'],
744 cmd, code))
745- if code != 0:
746- return "command `{}` returned {}".format(cmd, str(code))
747+ if not service_running:
748+ return u"command `{}` returned {} {}".format(
749+ cmd, output, str(code))
750 return None
751
752 def _get_config(self, unit, filename):
753@@ -164,7 +174,7 @@
754 # NOTE(beisner): by default, ConfigParser does not handle options
755 # with no value, such as the flags used in the mysql my.cnf file.
756 # https://bugs.python.org/issue7005
757- config = ConfigParser.ConfigParser(allow_no_value=True)
758+ config = configparser.ConfigParser(allow_no_value=True)
759 config.readfp(io.StringIO(file_contents))
760 return config
761
762@@ -450,15 +460,20 @@
763 cmd, code, output))
764 return None
765
766- def get_process_id_list(self, sentry_unit, process_name):
767+ def get_process_id_list(self, sentry_unit, process_name,
768+ expect_success=True):
769 """Get a list of process ID(s) from a single sentry juju unit
770 for a single process name.
771
772- :param sentry_unit: Pointer to amulet sentry instance (juju unit)
773+ :param sentry_unit: Amulet sentry instance (juju unit)
774 :param process_name: Process name
775+ :param expect_success: If False, expect the PID to be missing,
776+ raise if it is present.
777 :returns: List of process IDs
778 """
779- cmd = 'pidof {}'.format(process_name)
780+ cmd = 'pidof -x {}'.format(process_name)
781+ if not expect_success:
782+ cmd += " || exit 0 && exit 1"
783 output, code = sentry_unit.run(cmd)
784 if code != 0:
785 msg = ('{} `{}` returned {} '
786@@ -467,14 +482,23 @@
787 amulet.raise_status(amulet.FAIL, msg=msg)
788 return str(output).split()
789
790- def get_unit_process_ids(self, unit_processes):
791+ def get_unit_process_ids(self, unit_processes, expect_success=True):
792 """Construct a dict containing unit sentries, process names, and
793- process IDs."""
794+ process IDs.
795+
796+ :param unit_processes: A dictionary of Amulet sentry instance
797+ to list of process names.
798+ :param expect_success: if False expect the processes to not be
799+ running, raise if they are.
800+ :returns: Dictionary of Amulet sentry instance to dictionary
801+ of process names to PIDs.
802+ """
803 pid_dict = {}
804- for sentry_unit, process_list in unit_processes.iteritems():
805+ for sentry_unit, process_list in six.iteritems(unit_processes):
806 pid_dict[sentry_unit] = {}
807 for process in process_list:
808- pids = self.get_process_id_list(sentry_unit, process)
809+ pids = self.get_process_id_list(
810+ sentry_unit, process, expect_success=expect_success)
811 pid_dict[sentry_unit].update({process: pids})
812 return pid_dict
813
814@@ -488,7 +512,7 @@
815 return ('Unit count mismatch. expected, actual: {}, '
816 '{} '.format(len(expected), len(actual)))
817
818- for (e_sentry, e_proc_names) in expected.iteritems():
819+ for (e_sentry, e_proc_names) in six.iteritems(expected):
820 e_sentry_name = e_sentry.info['unit_name']
821 if e_sentry in actual.keys():
822 a_proc_names = actual[e_sentry]
823@@ -507,11 +531,23 @@
824 '{}'.format(e_proc_name, a_proc_name))
825
826 a_pids_length = len(a_pids)
827- if e_pids_length != a_pids_length:
828- return ('PID count mismatch. {} ({}) expected, actual: '
829+ fail_msg = ('PID count mismatch. {} ({}) expected, actual: '
830 '{}, {} ({})'.format(e_sentry_name, e_proc_name,
831 e_pids_length, a_pids_length,
832 a_pids))
833+
834+ # If expected is not bool, ensure PID quantities match
835+ if not isinstance(e_pids_length, bool) and \
836+ a_pids_length != e_pids_length:
837+ return fail_msg
838+ # If expected is bool True, ensure 1 or more PIDs exist
839+ elif isinstance(e_pids_length, bool) and \
840+ e_pids_length is True and a_pids_length < 1:
841+ return fail_msg
842+ # If expected is bool False, ensure 0 PIDs exist
843+ elif isinstance(e_pids_length, bool) and \
844+ e_pids_length is False and a_pids_length != 0:
845+ return fail_msg
846 else:
847 self.log.debug('PID check OK: {} {} {}: '
848 '{}'.format(e_sentry_name, e_proc_name,
849@@ -531,3 +567,30 @@
850 return 'Dicts within list are not identical'
851
852 return None
853+
854+ def run_action(self, unit_sentry, action,
855+ _check_output=subprocess.check_output):
856+ """Run the named action on a given unit sentry.
857+
858+ _check_output parameter is used for dependency injection.
859+
860+ @return action_id.
861+ """
862+ unit_id = unit_sentry.info["unit_name"]
863+ command = ["juju", "action", "do", "--format=json", unit_id, action]
864+ self.log.info("Running command: %s\n" % " ".join(command))
865+ output = _check_output(command, universal_newlines=True)
866+ data = json.loads(output)
867+ action_id = data[u'Action queued with id']
868+ return action_id
869+
870+ def wait_on_action(self, action_id, _check_output=subprocess.check_output):
871+ """Wait for a given action, returning if it completed or not.
872+
873+ _check_output parameter is used for dependency injection.
874+ """
875+ command = ["juju", "action", "fetch", "--format=json", "--wait=0",
876+ action_id]
877+ output = _check_output(command, universal_newlines=True)
878+ data = json.loads(output)
879+ return data.get(u"status") == "completed"
880
881=== modified file 'tests/charmhelpers/contrib/openstack/amulet/deployment.py'
882--- tests/charmhelpers/contrib/openstack/amulet/deployment.py 2015-07-29 10:49:43 +0000
883+++ tests/charmhelpers/contrib/openstack/amulet/deployment.py 2015-08-17 13:44:46 +0000
884@@ -44,7 +44,7 @@
885 Determine if the local branch being tested is derived from its
886 stable or next (dev) branch, and based on this, use the corresonding
887 stable or next branches for the other_services."""
888- base_charms = ['mysql', 'mongodb']
889+ base_charms = ['mysql', 'mongodb', 'nrpe']
890
891 if self.series in ['precise', 'trusty']:
892 base_series = self.series
893@@ -81,7 +81,7 @@
894 'ceph-osd', 'ceph-radosgw']
895 # Most OpenStack subordinate charms do not expose an origin option
896 # as that is controlled by the principle.
897- ignore = ['cinder-ceph', 'hacluster', 'neutron-openvswitch']
898+ ignore = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe']
899
900 if self.openstack:
901 for svc in services:
902
903=== added file 'unit_tests/test_actions.py'
904--- unit_tests/test_actions.py 1970-01-01 00:00:00 +0000
905+++ unit_tests/test_actions.py 2015-08-17 13:44:46 +0000
906@@ -0,0 +1,239 @@
907+import argparse
908+import tempfile
909+import unittest
910+
911+import mock
912+import yaml
913+
914+from test_utils import CharmTestCase
915+
916+import actions.actions
917+
918+
919+class PauseTestCase(CharmTestCase):
920+
921+ def setUp(self):
922+ super(PauseTestCase, self).setUp(
923+ actions.actions, ["service_pause", "status_set"])
924+
925+ class FakeArgs(object):
926+ services = ['swift-account',
927+ 'swift-account-auditor',
928+ 'swift-account-reaper',
929+ 'swift-account-replicator',
930+ 'swift-container',
931+ 'swift-container-auditor',
932+ 'swift-container-updater',
933+ 'swift-container-replicator',
934+ 'swift-container-sync',
935+ 'swift-object',
936+ 'swift-object-auditor',
937+ 'swift-object-updater',
938+ 'swift-object-replicator']
939+ self.args = FakeArgs()
940+
941+ def test_pauses_services(self):
942+ """Pause action pauses all of the Swift services."""
943+ pause_calls = []
944+
945+ def fake_service_pause(svc):
946+ pause_calls.append(svc)
947+ return True
948+
949+ self.service_pause.side_effect = fake_service_pause
950+
951+ actions.actions.pause(self.args)
952+ self.assertEqual(pause_calls, ['swift-account',
953+ 'swift-account-auditor',
954+ 'swift-account-reaper',
955+ 'swift-account-replicator',
956+ 'swift-container',
957+ 'swift-container-auditor',
958+ 'swift-container-updater',
959+ 'swift-container-replicator',
960+ 'swift-container-sync',
961+ 'swift-object',
962+ 'swift-object-auditor',
963+ 'swift-object-updater',
964+ 'swift-object-replicator'])
965+
966+ def test_bails_out_early_on_error(self):
967+ """Pause action fails early if there are errors stopping a service."""
968+ pause_calls = []
969+
970+ def maybe_kill(svc):
971+ if svc == "swift-container":
972+ return False
973+ else:
974+ pause_calls.append(svc)
975+ return True
976+
977+ self.service_pause.side_effect = maybe_kill
978+ self.assertRaisesRegexp(
979+ Exception, "swift-container didn't stop cleanly.",
980+ actions.actions.pause, self.args)
981+ self.assertEqual(pause_calls, ['swift-account',
982+ 'swift-account-auditor',
983+ 'swift-account-reaper',
984+ 'swift-account-replicator'])
985+
986+ def test_status_mode(self):
987+ """Pause action sets the status to maintenance."""
988+ status_calls = []
989+ self.status_set.side_effect = lambda state, msg: status_calls.append(
990+ state)
991+
992+ actions.actions.pause(self.args)
993+ self.assertEqual(status_calls, ["maintenance"])
994+
995+ def test_status_message(self):
996+ """Pause action sets a status message reflecting that it's paused."""
997+ status_calls = []
998+ self.status_set.side_effect = lambda state, msg: status_calls.append(
999+ msg)
1000+
1001+ actions.actions.pause(self.args)
1002+ self.assertEqual(
1003+ status_calls, ["Paused. "
1004+ "Use 'resume' action to resume normal service."])
1005+
1006+
1007+class ResumeTestCase(CharmTestCase):
1008+
1009+ def setUp(self):
1010+ super(ResumeTestCase, self).setUp(
1011+ actions.actions, ["service_resume", "status_set"])
1012+
1013+ class FakeArgs(object):
1014+ services = ['swift-account',
1015+ 'swift-account-auditor',
1016+ 'swift-account-reaper',
1017+ 'swift-account-replicator',
1018+ 'swift-container',
1019+ 'swift-container-auditor',
1020+ 'swift-container-updater',
1021+ 'swift-container-replicator',
1022+ 'swift-container-sync',
1023+ 'swift-object',
1024+ 'swift-object-auditor',
1025+ 'swift-object-updater',
1026+ 'swift-object-replicator']
1027+ self.args = FakeArgs()
1028+
1029+ def test_resumes_services(self):
1030+ """Resume action resumes all of the Swift services."""
1031+ resume_calls = []
1032+
1033+ def fake_service_resume(svc):
1034+ resume_calls.append(svc)
1035+ return True
1036+
1037+ self.service_resume.side_effect = fake_service_resume
1038+ actions.actions.resume(self.args)
1039+ self.assertEqual(resume_calls, ['swift-account',
1040+ 'swift-account-auditor',
1041+ 'swift-account-reaper',
1042+ 'swift-account-replicator',
1043+ 'swift-container',
1044+ 'swift-container-auditor',
1045+ 'swift-container-updater',
1046+ 'swift-container-replicator',
1047+ 'swift-container-sync',
1048+ 'swift-object',
1049+ 'swift-object-auditor',
1050+ 'swift-object-updater',
1051+ 'swift-object-replicator'])
1052+
1053+ def test_bails_out_early_on_error(self):
1054+ """Resume action fails early if there are errors starting a service."""
1055+ resume_calls = []
1056+
1057+ def maybe_kill(svc):
1058+ if svc == "swift-container":
1059+ return False
1060+ else:
1061+ resume_calls.append(svc)
1062+ return True
1063+
1064+ self.service_resume.side_effect = maybe_kill
1065+ self.assertRaisesRegexp(
1066+ Exception, "swift-container didn't start cleanly.",
1067+ actions.actions.resume, self.args)
1068+ self.assertEqual(resume_calls, ['swift-account',
1069+ 'swift-account-auditor',
1070+ 'swift-account-reaper',
1071+ 'swift-account-replicator'])
1072+
1073+ def test_status_mode(self):
1074+ """Resume action sets the status to maintenance."""
1075+ status_calls = []
1076+ self.status_set.side_effect = lambda state, msg: status_calls.append(
1077+ state)
1078+
1079+ actions.actions.resume(self.args)
1080+ self.assertEqual(status_calls, ["active"])
1081+
1082+ def test_status_message(self):
1083+ """Resume action sets an empty status message."""
1084+ status_calls = []
1085+ self.status_set.side_effect = lambda state, msg: status_calls.append(
1086+ msg)
1087+
1088+ actions.actions.resume(self.args)
1089+ self.assertEqual(status_calls, [""])
1090+
1091+
1092+class GetActionParserTestCase(unittest.TestCase):
1093+
1094+ def test_definition_from_yaml(self):
1095+ """ArgumentParser is seeded from actions.yaml."""
1096+ actions_yaml = tempfile.NamedTemporaryFile(
1097+ prefix="GetActionParserTestCase", suffix="yaml")
1098+ actions_yaml.write(yaml.dump({"foo": {"description": "Foo is bar"}}))
1099+ actions_yaml.seek(0)
1100+ parser = actions.actions.get_action_parser(actions_yaml.name, "foo",
1101+ get_services=lambda: [])
1102+ self.assertEqual(parser.description, 'Foo is bar')
1103+
1104+
1105+class MainTestCase(CharmTestCase):
1106+
1107+ def setUp(self):
1108+ super(MainTestCase, self).setUp(
1109+ actions.actions, ["_get_action_name",
1110+ "get_action_parser",
1111+ "action_fail"])
1112+
1113+ def test_invokes_pause(self):
1114+ dummy_calls = []
1115+
1116+ def dummy_action(args):
1117+ dummy_calls.append(True)
1118+
1119+ self._get_action_name.side_effect = lambda: "foo"
1120+ self.get_action_parser = lambda: argparse.ArgumentParser()
1121+ with mock.patch.dict(actions.actions.ACTIONS, {"foo": dummy_action}):
1122+ actions.actions.main([])
1123+ self.assertEqual(dummy_calls, [True])
1124+
1125+ def test_unknown_action(self):
1126+ """Unknown actions aren't a traceback."""
1127+ self._get_action_name.side_effect = lambda: "foo"
1128+ self.get_action_parser = lambda: argparse.ArgumentParser()
1129+ exit_string = actions.actions.main([])
1130+ self.assertEqual("Action foo undefined", exit_string)
1131+
1132+ def test_failing_action(self):
1133+ """Actions which traceback trigger action_fail() calls."""
1134+ dummy_calls = []
1135+
1136+ self.action_fail.side_effect = dummy_calls.append
1137+ self._get_action_name.side_effect = lambda: "foo"
1138+
1139+ def dummy_action(args):
1140+ raise ValueError("uh oh")
1141+
1142+ self.get_action_parser = lambda: argparse.ArgumentParser()
1143+ with mock.patch.dict(actions.actions.ACTIONS, {"foo": dummy_action}):
1144+ actions.actions.main([])
1145+ self.assertEqual(dummy_calls, ["uh oh"])

Subscribers

People subscribed via source and target branches