Merge lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions into lp:~openstack-charmers-archive/charms/trusty/swift-storage/next

Proposed by Adam Collard
Status: Superseded
Proposed branch: lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions
Merge into: lp:~openstack-charmers-archive/charms/trusty/swift-storage/next
Diff against target: 1145 lines (+603/-135)
19 files modified
.coveragerc (+1/-0)
Makefile (+2/-0)
actions.yaml (+5/-0)
actions/actions.py (+93/-0)
charmhelpers/cli/__init__.py (+1/-5)
charmhelpers/cli/commands.py (+4/-4)
charmhelpers/contrib/openstack/amulet/deployment.py (+2/-2)
charmhelpers/contrib/openstack/utils.py (+51/-14)
charmhelpers/contrib/storage/linux/utils.py (+3/-2)
charmhelpers/core/hookenv.py (+1/-20)
charmhelpers/core/host.py (+2/-2)
charmhelpers/fetch/__init__.py (+8/-0)
lib/swift_storage_utils.py (+1/-15)
setup.cfg (+1/-1)
tests/00-setup (+1/-0)
tests/basic_deployment.py (+102/-47)
tests/charmhelpers/contrib/amulet/utils.py (+84/-21)
tests/charmhelpers/contrib/openstack/amulet/deployment.py (+2/-2)
unit_tests/test_actions.py (+239/-0)
To merge this branch: bzr merge lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions
Reviewer Review Type Date Requested Status
Landscape Pending
OpenStack Charmers Pending
Review via email: mp+267681@code.launchpad.net

This proposal has been superseded by a proposal from 2015-08-17.

Description of the change

This branch adds actions for pausing and resuming services on swift-storage units.

In addition to just stopping the services, the actions also set the status of the unit to be "maintenance" with a message on how to resume - I'm not attached to the message, bikesheds welcome :).

Further refinements to the charm to support pause mode will be coming in follow-up branches, notably guarding calls to service_start and service_restart to prevent config-changed, *-relation-changed and other hooks from (re)-starting a unit which should be paused.

To post a comment you must log in.
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7869 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7869/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7290 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7290/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7871 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7871/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7292 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7292/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5759 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12057851/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5759/

88. By Adam Collard

Fix flake8 findings

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7922 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7922/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7340 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7340/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5763 swift-storage-next for adam-collard mp267681
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5763/

Revision history for this message
Ryan Beisner (1chb1n) wrote :

RE: Amulet tests

Thank for your contribution and clean-up here. Backgrounder: I've not gotten down the list to swift-* charm amulet test updates, but intend to complete the vivid enablement in 15.08 and wily in 15.09. There are also a number of refactors and helper optimizations, which we're making across all of the os-charm amulet tests, so I'll be shifting these around a bit when I get down the list to the swift-*s.

1 easy change that I'd request: Please chmod -x tests/basic_deployment.py ... set not-executable, as it's just a module with class definitions called by the actual test executables (NNN-<ubuntu>-<openstack> files which are set +x for known-working combos).

Take note that juju test (and bundletester) will juju bootstrap then execute anything in the tests/ dir which is +x, and that any yamls in the tests/ dir will be assumed by bundletester to be a test definition. While we are prepping to switch completely to bundletester, our first priority with the amulet tests is in updating the horizon of u:os test coverage to catch up with charm development efforts. Then we will resume the bundletester effort.

Thanks again, holler with any ?s.

89. By Adam Collard

Remove #! from basic_deployment.py (beisner's review)

90. By Adam Collard

chmod -x

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7926 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7926/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7344 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7344/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5767 swift-storage-next for adam-collard mp267681
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5767/

91. By Adam Collard

Sync charm-helpers

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7927 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7927/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7345 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7345/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5768 swift-storage-next for adam-collard mp267681
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5768/

92. By Adam Collard

Switch tests to using validate_service_by_name()

93. By Adam Collard

WiP amulet test

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7929 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7929/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7347 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7347/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5770 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12063967/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5770/

94. By Adam Collard

validate_serviceS_by_name (fix typo)

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7983 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7983/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7397 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7397/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5775 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 124
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12070002/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5775/

95. By Adam Collard

Move tests into basic_deployment so they get ran for each combination

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7985 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7985/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7399 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7399/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5777 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12070240/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5777/

96. By Adam Collard

Move actions test to the end so basic service tests are done upfront

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #7987 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7987/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7401 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7401/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5779 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12070492/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5779/

Revision history for this message
Ryan Beisner (1chb1n) wrote :

Hi. Kicking in with some hopefully helpful hints. I see the proposed test is failing when trying to confirm a system service that doesn't appear to be present at a particular release combo.

Since the basic_deployment.py set of tests is expected to pass for all supported release combos, each test_ method may need to use release detection logic as needed. In some cases, openstack components have shifted options, sections, etc., in conf files. In other cases, some system services may or may not be present across that entire timeline. There was particularly a lot of upstream shifting-around introduced at >= Kilo ... and < Icehouse.

As a usage example, the heat amulet test detects and checks things slightly differently here:
http://bazaar.launchpad.net/~openstack-charmers/charms/trusty/heat/next/view/head:/tests/basic_deployment.py#L313

Which ultimately uses this for comparison:
http://bazaar.launchpad.net/~charm-helpers/charm-helpers/devel/view/head:/charmhelpers/contrib/openstack/amulet/deployment.py#L109

97. By Adam Collard

Fix tests

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #8042 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8042/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7453 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7453/

Revision history for this message
Adam Collard (adam-collard) wrote :

> Hi. Kicking in with some hopefully helpful hints. I see the proposed test is
> failing when trying to confirm a system service that doesn't appear to be
> present at a particular release combo.

That was my first guess too, fortunately (or unfortunately?) it wasn't that issue. The problem was the utils code for looking at what processes are running uses "pidof $name" but when $name is the name of a script it fails (using the -x option works as expected). Instead of yet another round of fixing charm-helpers, I opted to just roll my own for this test - there are comments in the test to further explain why.

In the absence of access to a testing environment that could satisfy the requirements, I was using UOSCI bot to run my tests for me. Now that's been resolved I have (hopefully!) got some green tests :)

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5786 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12078138/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5786/

98. By Adam Collard

Only care about swift-container-sync for >= Icehouse

99. By Adam Collard

s/pop/remove (d'oh!)

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #8044 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8044/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7455 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7455/

100. By Adam Collard

More z's

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5788 swift-storage-next for adam-collard mp267681
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5788/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #8045 swift-storage-next for adam-collard mp267681
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8045/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #7456 swift-storage-next for adam-collard mp267681
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7456/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #5789 swift-storage-next for adam-collard mp267681
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 124
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12079074/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5789/

Revision history for this message
Ryan Beisner (1chb1n) wrote :

RE: amulet tests

Thank you for this work!

Just a couple of things that will make future life easier:

_run_action and _wait_on_action:
Please consider landing these in charmhelpers/contrib/amulet/utils.py, as they will be increasingly-common needs.

_assert_services:
Please consider enhancing the existing validate_unit_process_ids amulet helper to fit the needs here. I think it would be a fairly easy mod. We could do -x on pidof in all cases (or add a bool with default False to "check scripts too" if you want to be safer about it). Then add a boolean postive/negative switch to be able to invert its expectations (like you've done with should_run). fwiw, expect_success (default True) is such a bool name that I've used elsewhere for the same purpose in a different context (rmq wip).

101. By Adam Collard

Depend on amulet-actions-helpers branch of charm-helpers, use those actin helpers (beisner's review)

102. By Adam Collard

Point charm-helpers to latest branch, update tests to use helpers. Add comment about why validate_unit_process_ids is not helpful.

103. By Adam Collard

run_action takes the sentry object, not the unit name

104. By Adam Collard

Use wait_on_action helper for resume test too

105. By Adam Collard

Sync charm-helpers

106. By Adam Collard

Revert to pointing at regular charm-helpers

107. By Adam Collard

Merge in from fix-service-status

Revision history for this message
Adam Collard (adam-collard) wrote :

Hi Ryan,

Thanks for taking a look at this.

On Fri, 14 Aug 2015 at 17:10 Ryan Beisner <email address hidden>
wrote:

> RE: amulet tests
>
> Thank you for this work!
>
> Just a couple of things that will make future life easier:
>
> _run_action and _wait_on_action:
> Please consider landing these in charmhelpers/contrib/amulet/utils.py, as
> they will be increasingly-common needs.
>

Done. See charm-helpers r428

>
> _assert_services:
> Please consider enhancing the existing validate_unit_process_ids amulet
> helper to fit the needs here. I think it would be a fairly easy mod. We
> could do -x on pidof in all cases (or add a bool with default False to
> "check scripts too" if you want to be safer about it). Then add a boolean
> postive/negative switch to be able to invert its expectations (like you've
> done with should_run). fwiw, expect_success (default True) is such a bool
> name that I've used elsewhere for the same purpose in a different context
> (rmq wip).
>

I modified get_process_id_list and get_unit_process_ids but didn't end up
using validate_unit_process_ids because the validation is already done for
me in the previous two methods. See comment in the test itself.

I've split the branch up so that the new code is cleanly separated from the
fixes I made to existing code (see pre-req branch).

--
>
> https://code.launchpad.net/~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions/+merge/267681
> You are the owner of
> lp:~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions.
>
> Launchpad-Message-Rationale: Owner
> Launchpad-Notification-Type: code-review
> Launchpad-Branch:
> ~adam-collard/charms/trusty/swift-storage/add-pause-resume-actions
>

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file '.coveragerc'
--- .coveragerc 2015-08-11 08:09:29 +0000
+++ .coveragerc 2015-08-17 13:44:46 +0000
@@ -3,5 +3,6 @@
3exclude_lines =3exclude_lines =
4 if __name__ == .__main__.:4 if __name__ == .__main__.:
5include=5include=
6 actions/actions.py
6 hooks/swift_storage_*7 hooks/swift_storage_*
7 lib/swift_storage_*8 lib/swift_storage_*
89
=== modified file 'Makefile'
--- Makefile 2015-08-04 11:48:35 +0000
+++ Makefile 2015-08-17 13:44:46 +0000
@@ -30,3 +30,5 @@
30publish: lint test30publish: lint test
31 bzr push lp:charms/swift-storage31 bzr push lp:charms/swift-storage
32 bzr push lp:charms/trusty/swift-storage32 bzr push lp:charms/trusty/swift-storage
33
34.PHONY: lint unit_test test sync publish
3335
=== added directory 'actions'
=== added file 'actions.yaml'
--- actions.yaml 1970-01-01 00:00:00 +0000
+++ actions.yaml 2015-08-17 13:44:46 +0000
@@ -0,0 +1,5 @@
1pause:
2 description: Pause the swift-storage unit. This action will stop Swift services.
3resume:
4 description: Resume the swift-storage unit. This action will start Swift services.
5
06
=== added file 'actions/__init__.py'
=== added file 'actions/actions.py'
--- actions/actions.py 1970-01-01 00:00:00 +0000
+++ actions/actions.py 2015-08-17 13:44:46 +0000
@@ -0,0 +1,93 @@
1#!/usr/bin/python
2
3import argparse
4import os
5import sys
6import yaml
7
8from charmhelpers.core.host import service_pause, service_resume
9from charmhelpers.core.hookenv import action_fail, status_set
10from charmhelpers.contrib.openstack.utils import get_os_codename_package
11
12from lib.swift_storage_utils import SWIFT_SVCS
13
14
15def _get_services():
16 """Return a list of services that need to be (un)paused."""
17 services = SWIFT_SVCS[:]
18 # Before Icehouse there was no swift-container-sync
19 if get_os_codename_package("swift-container") < "icehouse":
20 services.remove("swift-container-sync")
21 return services
22
23
24def get_action_parser(actions_yaml_path, action_name,
25 get_services=_get_services):
26 """Make an argparse.ArgumentParser seeded from actions.yaml definitions."""
27 with open(actions_yaml_path) as fh:
28 doc = yaml.load(fh)[action_name]["description"]
29 parser = argparse.ArgumentParser(description=doc)
30 parser.add_argument("--services", default=get_services())
31 # TODO: Add arguments for params defined in the actions.yaml
32 return parser
33
34
35def pause(args):
36 """Pause all the swift services.
37
38 @raises Exception if any services fail to stop
39 """
40 for service in args.services:
41 stopped = service_pause(service)
42 if not stopped:
43 raise Exception("{} didn't stop cleanly.".format(service))
44 status_set(
45 "maintenance", "Paused. Use 'resume' action to resume normal service.")
46
47
48def resume(args):
49 """Resume all the swift services.
50
51 @raises Exception if any services fail to start
52 """
53 for service in args.services:
54 started = service_resume(service)
55 if not started:
56 raise Exception("{} didn't start cleanly.".format(service))
57 status_set("active", "")
58
59
60# A dictionary of all the defined actions to callables (which take
61# parsed arguments).
62ACTIONS = {"pause": pause, "resume": resume}
63
64
65def main(argv):
66 action_name = _get_action_name()
67 actions_yaml_path = _get_actions_yaml_path()
68 parser = get_action_parser(actions_yaml_path, action_name)
69 args = parser.parse_args(argv)
70 try:
71 action = ACTIONS[action_name]
72 except KeyError:
73 return "Action %s undefined" % action_name
74 else:
75 try:
76 action(args)
77 except Exception as e:
78 action_fail(str(e))
79
80
81def _get_action_name():
82 """Return the name of the action."""
83 return os.path.basename(__file__)
84
85
86def _get_actions_yaml_path():
87 """Return the path to actions.yaml"""
88 cwd = os.path.dirname(__file__)
89 return os.path.join(cwd, "..", "actions.yaml")
90
91
92if __name__ == "__main__":
93 sys.exit(main(sys.argv[1:]))
094
=== added symlink 'actions/charmhelpers'
=== target is u'../charmhelpers/'
=== added symlink 'actions/lib'
=== target is u'../lib'
=== added symlink 'actions/pause'
=== target is u'actions.py'
=== added symlink 'actions/resume'
=== target is u'actions.py'
=== modified file 'charmhelpers/cli/__init__.py'
--- charmhelpers/cli/__init__.py 2015-07-31 13:11:32 +0000
+++ charmhelpers/cli/__init__.py 2015-08-17 13:44:46 +0000
@@ -152,15 +152,11 @@
152 arguments = self.argument_parser.parse_args()152 arguments = self.argument_parser.parse_args()
153 argspec = inspect.getargspec(arguments.func)153 argspec = inspect.getargspec(arguments.func)
154 vargs = []154 vargs = []
155 kwargs = {}
156 for arg in argspec.args:155 for arg in argspec.args:
157 vargs.append(getattr(arguments, arg))156 vargs.append(getattr(arguments, arg))
158 if argspec.varargs:157 if argspec.varargs:
159 vargs.extend(getattr(arguments, argspec.varargs))158 vargs.extend(getattr(arguments, argspec.varargs))
160 if argspec.keywords:159 output = arguments.func(*vargs)
161 for kwarg in argspec.keywords.items():
162 kwargs[kwarg] = getattr(arguments, kwarg)
163 output = arguments.func(*vargs, **kwargs)
164 if getattr(arguments.func, '_cli_test_command', False):160 if getattr(arguments.func, '_cli_test_command', False):
165 self.exit_code = 0 if output else 1161 self.exit_code = 0 if output else 1
166 output = ''162 output = ''
167163
=== modified file 'charmhelpers/cli/commands.py'
--- charmhelpers/cli/commands.py 2015-07-31 13:11:32 +0000
+++ charmhelpers/cli/commands.py 2015-08-17 13:44:46 +0000
@@ -26,7 +26,7 @@
26"""26"""
27Import the sub-modules which have decorated subcommands to register with chlp.27Import the sub-modules which have decorated subcommands to register with chlp.
28"""28"""
29import host # noqa29from . import host # noqa
30import benchmark # noqa30from . import benchmark # noqa
31import unitdata # noqa31from . import unitdata # noqa
32from charmhelpers.core import hookenv # noqa32from . import hookenv # noqa
3333
=== modified file 'charmhelpers/contrib/openstack/amulet/deployment.py'
--- charmhelpers/contrib/openstack/amulet/deployment.py 2015-07-29 10:49:43 +0000
+++ charmhelpers/contrib/openstack/amulet/deployment.py 2015-08-17 13:44:46 +0000
@@ -44,7 +44,7 @@
44 Determine if the local branch being tested is derived from its44 Determine if the local branch being tested is derived from its
45 stable or next (dev) branch, and based on this, use the corresonding45 stable or next (dev) branch, and based on this, use the corresonding
46 stable or next branches for the other_services."""46 stable or next branches for the other_services."""
47 base_charms = ['mysql', 'mongodb']47 base_charms = ['mysql', 'mongodb', 'nrpe']
4848
49 if self.series in ['precise', 'trusty']:49 if self.series in ['precise', 'trusty']:
50 base_series = self.series50 base_series = self.series
@@ -81,7 +81,7 @@
81 'ceph-osd', 'ceph-radosgw']81 'ceph-osd', 'ceph-radosgw']
82 # Most OpenStack subordinate charms do not expose an origin option82 # Most OpenStack subordinate charms do not expose an origin option
83 # as that is controlled by the principle.83 # as that is controlled by the principle.
84 ignore = ['cinder-ceph', 'hacluster', 'neutron-openvswitch']84 ignore = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe']
8585
86 if self.openstack:86 if self.openstack:
87 for svc in services:87 for svc in services:
8888
=== modified file 'charmhelpers/contrib/openstack/utils.py'
--- charmhelpers/contrib/openstack/utils.py 2015-07-29 10:49:43 +0000
+++ charmhelpers/contrib/openstack/utils.py 2015-08-17 13:44:46 +0000
@@ -24,6 +24,7 @@
24import json24import json
25import os25import os
26import sys26import sys
27import re
2728
28import six29import six
29import yaml30import yaml
@@ -69,7 +70,6 @@
69DISTRO_PROPOSED = ('deb http://archive.ubuntu.com/ubuntu/ %s-proposed '70DISTRO_PROPOSED = ('deb http://archive.ubuntu.com/ubuntu/ %s-proposed '
70 'restricted main multiverse universe')71 'restricted main multiverse universe')
7172
72
73UBUNTU_OPENSTACK_RELEASE = OrderedDict([73UBUNTU_OPENSTACK_RELEASE = OrderedDict([
74 ('oneiric', 'diablo'),74 ('oneiric', 'diablo'),
75 ('precise', 'essex'),75 ('precise', 'essex'),
@@ -118,6 +118,34 @@
118 ('2.3.0', 'liberty'),118 ('2.3.0', 'liberty'),
119])119])
120120
121# >= Liberty version->codename mapping
122PACKAGE_CODENAMES = {
123 'nova-common': OrderedDict([
124 ('12.0.0', 'liberty'),
125 ]),
126 'neutron-common': OrderedDict([
127 ('7.0.0', 'liberty'),
128 ]),
129 'cinder-common': OrderedDict([
130 ('7.0.0', 'liberty'),
131 ]),
132 'keystone': OrderedDict([
133 ('8.0.0', 'liberty'),
134 ]),
135 'horizon-common': OrderedDict([
136 ('8.0.0', 'liberty'),
137 ]),
138 'ceilometer-common': OrderedDict([
139 ('5.0.0', 'liberty'),
140 ]),
141 'heat-common': OrderedDict([
142 ('5.0.0', 'liberty'),
143 ]),
144 'glance-common': OrderedDict([
145 ('11.0.0', 'liberty'),
146 ]),
147}
148
121DEFAULT_LOOPBACK_SIZE = '5G'149DEFAULT_LOOPBACK_SIZE = '5G'
122150
123151
@@ -201,20 +229,29 @@
201 error_out(e)229 error_out(e)
202230
203 vers = apt.upstream_version(pkg.current_ver.ver_str)231 vers = apt.upstream_version(pkg.current_ver.ver_str)
232 match = re.match('^(\d)\.(\d)\.(\d)', vers)
233 if match:
234 vers = match.group(0)
204235
205 try:236 # >= Liberty independent project versions
206 if 'swift' in pkg.name:237 if (package in PACKAGE_CODENAMES and
207 swift_vers = vers[:5]238 vers in PACKAGE_CODENAMES[package]):
208 if swift_vers not in SWIFT_CODENAMES:239 return PACKAGE_CODENAMES[package][vers]
209 # Deal with 1.10.0 upward240 else:
210 swift_vers = vers[:6]241 # < Liberty co-ordinated project versions
211 return SWIFT_CODENAMES[swift_vers]242 try:
212 else:243 if 'swift' in pkg.name:
213 vers = vers[:6]244 swift_vers = vers[:5]
214 return OPENSTACK_CODENAMES[vers]245 if swift_vers not in SWIFT_CODENAMES:
215 except KeyError:246 # Deal with 1.10.0 upward
216 e = 'Could not determine OpenStack codename for version %s' % vers247 swift_vers = vers[:6]
217 error_out(e)248 return SWIFT_CODENAMES[swift_vers]
249 else:
250 vers = vers[:6]
251 return OPENSTACK_CODENAMES[vers]
252 except KeyError:
253 e = 'Could not determine OpenStack codename for version %s' % vers
254 error_out(e)
218255
219256
220def get_os_version_package(pkg, fatal=True):257def get_os_version_package(pkg, fatal=True):
221258
=== modified file 'charmhelpers/contrib/storage/linux/utils.py'
--- charmhelpers/contrib/storage/linux/utils.py 2015-07-29 10:49:43 +0000
+++ charmhelpers/contrib/storage/linux/utils.py 2015-08-17 13:44:46 +0000
@@ -43,9 +43,10 @@
4343
44 :param block_device: str: Full path of block device to clean.44 :param block_device: str: Full path of block device to clean.
45 '''45 '''
46 # https://github.com/ceph/ceph/commit/fdd7f8d83afa25c4e09aaedd90ab93f3b64a677b
46 # sometimes sgdisk exits non-zero; this is OK, dd will clean up47 # sometimes sgdisk exits non-zero; this is OK, dd will clean up
47 call(['sgdisk', '--zap-all', '--mbrtogpt',48 call(['sgdisk', '--zap-all', '--', block_device])
48 '--clear', block_device])49 call(['sgdisk', '--clear', '--mbrtogpt', '--', block_device])
49 dev_end = check_output(['blockdev', '--getsz',50 dev_end = check_output(['blockdev', '--getsz',
50 block_device]).decode('UTF-8')51 block_device]).decode('UTF-8')
51 gpt_end = int(dev_end.split()[0]) - 10052 gpt_end = int(dev_end.split()[0]) - 100
5253
=== modified file 'charmhelpers/core/hookenv.py'
--- charmhelpers/core/hookenv.py 2015-08-03 14:00:44 +0000
+++ charmhelpers/core/hookenv.py 2015-08-17 13:44:46 +0000
@@ -34,23 +34,6 @@
34import tempfile34import tempfile
35from subprocess import CalledProcessError35from subprocess import CalledProcessError
3636
37try:
38 from charmhelpers.cli import cmdline
39except ImportError as e:
40 # due to the anti-pattern of partially synching charmhelpers directly
41 # into charms, it's possible that charmhelpers.cli is not available;
42 # if that's the case, they don't really care about using the cli anyway,
43 # so mock it out
44 if str(e) == 'No module named cli':
45 class cmdline(object):
46 @classmethod
47 def subcommand(cls, *args, **kwargs):
48 def _wrap(func):
49 return func
50 return _wrap
51 else:
52 raise
53
54import six37import six
55if not six.PY3:38if not six.PY3:
56 from UserDict import UserDict39 from UserDict import UserDict
@@ -91,6 +74,7 @@
91 res = func(*args, **kwargs)74 res = func(*args, **kwargs)
92 cache[key] = res75 cache[key] = res
93 return res76 return res
77 wrapper._wrapped = func
94 return wrapper78 return wrapper
9579
9680
@@ -190,7 +174,6 @@
190 return os.environ.get('JUJU_RELATION', None)174 return os.environ.get('JUJU_RELATION', None)
191175
192176
193@cmdline.subcommand()
194@cached177@cached
195def relation_id(relation_name=None, service_or_unit=None):178def relation_id(relation_name=None, service_or_unit=None):
196 """The relation ID for the current or a specified relation"""179 """The relation ID for the current or a specified relation"""
@@ -216,13 +199,11 @@
216 return os.environ.get('JUJU_REMOTE_UNIT', None)199 return os.environ.get('JUJU_REMOTE_UNIT', None)
217200
218201
219@cmdline.subcommand()
220def service_name():202def service_name():
221 """The name service group this unit belongs to"""203 """The name service group this unit belongs to"""
222 return local_unit().split('/')[0]204 return local_unit().split('/')[0]
223205
224206
225@cmdline.subcommand()
226@cached207@cached
227def remote_service_name(relid=None):208def remote_service_name(relid=None):
228 """The remote service name for a given relation-id (or the current relation)"""209 """The remote service name for a given relation-id (or the current relation)"""
229210
=== modified file 'charmhelpers/core/host.py'
--- charmhelpers/core/host.py 2015-07-29 10:49:43 +0000
+++ charmhelpers/core/host.py 2015-08-17 13:44:46 +0000
@@ -72,7 +72,7 @@
72 stopped = service_stop(service_name)72 stopped = service_stop(service_name)
73 # XXX: Support systemd too73 # XXX: Support systemd too
74 override_path = os.path.join(74 override_path = os.path.join(
75 init_dir, '{}.conf.override'.format(service_name))75 init_dir, '{}.override'.format(service_name))
76 with open(override_path, 'w') as fh:76 with open(override_path, 'w') as fh:
77 fh.write("manual\n")77 fh.write("manual\n")
78 return stopped78 return stopped
@@ -86,7 +86,7 @@
86 if init_dir is None:86 if init_dir is None:
87 init_dir = "/etc/init"87 init_dir = "/etc/init"
88 override_path = os.path.join(88 override_path = os.path.join(
89 init_dir, '{}.conf.override'.format(service_name))89 init_dir, '{}.override'.format(service_name))
90 if os.path.exists(override_path):90 if os.path.exists(override_path):
91 os.unlink(override_path)91 os.unlink(override_path)
92 started = service_start(service_name)92 started = service_start(service_name)
9393
=== modified file 'charmhelpers/fetch/__init__.py'
--- charmhelpers/fetch/__init__.py 2015-07-29 10:49:43 +0000
+++ charmhelpers/fetch/__init__.py 2015-08-17 13:44:46 +0000
@@ -90,6 +90,14 @@
90 'kilo/proposed': 'trusty-proposed/kilo',90 'kilo/proposed': 'trusty-proposed/kilo',
91 'trusty-kilo/proposed': 'trusty-proposed/kilo',91 'trusty-kilo/proposed': 'trusty-proposed/kilo',
92 'trusty-proposed/kilo': 'trusty-proposed/kilo',92 'trusty-proposed/kilo': 'trusty-proposed/kilo',
93 # Liberty
94 'liberty': 'trusty-updates/liberty',
95 'trusty-liberty': 'trusty-updates/liberty',
96 'trusty-liberty/updates': 'trusty-updates/liberty',
97 'trusty-updates/liberty': 'trusty-updates/liberty',
98 'liberty/proposed': 'trusty-proposed/liberty',
99 'trusty-liberty/proposed': 'trusty-proposed/liberty',
100 'trusty-proposed/liberty': 'trusty-proposed/liberty',
93}101}
94102
95# The order of this list is very important. Handlers should be listed in from103# The order of this list is very important. Handlers should be listed in from
96104
=== modified file 'lib/swift_storage_utils.py'
--- lib/swift_storage_utils.py 2015-07-17 09:57:19 +0000
+++ lib/swift_storage_utils.py 2015-08-17 13:44:46 +0000
@@ -83,21 +83,7 @@
83 'swift-object-updater', 'swift-object-replicator'83 'swift-object-updater', 'swift-object-replicator'
84]84]
8585
86SWIFT_SVCS = [86SWIFT_SVCS = ACCOUNT_SVCS + CONTAINER_SVCS + OBJECT_SVCS
87 'swift-account-auditor',
88 'swift-account-reaper',
89 'swift-account-replicator',
90 'swift-account-server',
91 'swift-container-auditor',
92 'swift-container-replicator',
93 'swift-container-server',
94 'swift-container-sync',
95 'swift-container-updater',
96 'swift-object-auditor',
97 'swift-object-replicator',
98 'swift-object-server',
99 'swift-object-updater',
100 ]
10187
102RESTART_MAP = {88RESTART_MAP = {
103 '/etc/rsync-juju.d/050-swift-storage.conf': ['rsync'],89 '/etc/rsync-juju.d/050-swift-storage.conf': ['rsync'],
10490
=== modified file 'setup.cfg'
--- setup.cfg 2015-08-11 08:09:29 +0000
+++ setup.cfg 2015-08-17 13:44:46 +0000
@@ -2,5 +2,5 @@
2verbosity=22verbosity=2
3with-coverage=13with-coverage=1
4cover-erase=14cover-erase=1
5cover-package=lib,hooks.swift_storage_hooks5cover-package=actions.actions,lib,hooks.swift_storage_hooks
66
77
=== modified file 'tests/00-setup'
--- tests/00-setup 2014-09-29 21:14:05 +0000
+++ tests/00-setup 2015-08-17 13:44:46 +0000
@@ -7,5 +7,6 @@
7sudo apt-get install --yes python-amulet \7sudo apt-get install --yes python-amulet \
8 python-swiftclient \8 python-swiftclient \
9 python-glanceclient \9 python-glanceclient \
10 python-heatclient \
10 python-keystoneclient \11 python-keystoneclient \
11 python-novaclient12 python-novaclient
1213
=== modified file 'tests/basic_deployment.py'
--- tests/basic_deployment.py 2015-04-16 21:31:35 +0000
+++ tests/basic_deployment.py 2015-08-17 13:44:46 +0000
@@ -1,5 +1,3 @@
1#!/usr/bin/python
2
3import amulet1import amulet
4import swiftclient2import swiftclient
53
@@ -9,8 +7,7 @@
97
10from charmhelpers.contrib.openstack.amulet.utils import (8from charmhelpers.contrib.openstack.amulet.utils import (
11 OpenStackAmuletUtils,9 OpenStackAmuletUtils,
12 DEBUG, # flake8: noqa10 DEBUG,
13 ERROR
14)11)
1512
16# Use DEBUG to turn on debug logging13# Use DEBUG to turn on debug logging
@@ -46,12 +43,12 @@
46 def _add_relations(self):43 def _add_relations(self):
47 """Add all of the relations for the services."""44 """Add all of the relations for the services."""
48 relations = {45 relations = {
49 'keystone:shared-db': 'mysql:shared-db',46 'keystone:shared-db': 'mysql:shared-db',
50 'swift-proxy:identity-service': 'keystone:identity-service',47 'swift-proxy:identity-service': 'keystone:identity-service',
51 'swift-storage:swift-storage': 'swift-proxy:swift-storage',48 'swift-storage:swift-storage': 'swift-proxy:swift-storage',
52 'glance:identity-service': 'keystone:identity-service',49 'glance:identity-service': 'keystone:identity-service',
53 'glance:shared-db': 'mysql:shared-db',50 'glance:shared-db': 'mysql:shared-db',
54 'glance:object-store': 'swift-proxy:object-store'51 'glance:object-store': 'swift-proxy:object-store'
55 }52 }
56 super(SwiftStorageBasicDeployment, self)._add_relations(relations)53 super(SwiftStorageBasicDeployment, self)._add_relations(relations)
5754
@@ -59,9 +56,11 @@
59 """Configure all of the services."""56 """Configure all of the services."""
60 keystone_config = {'admin-password': 'openstack',57 keystone_config = {'admin-password': 'openstack',
61 'admin-token': 'ubuntutesting'}58 'admin-token': 'ubuntutesting'}
62 swift_proxy_config = {'zone-assignment': 'manual',59 swift_proxy_config = {
63 'replicas': '1',60 'zone-assignment': 'manual',
64 'swift-hash': 'fdfef9d4-8b06-11e2-8ac0-531c923c8fae'}61 'replicas': '1',
62 'swift-hash': 'fdfef9d4-8b06-11e2-8ac0-531c923c8fae'
63 }
65 swift_storage_config = {'zone': '1',64 swift_storage_config = {'zone': '1',
66 'block-device': 'vdb',65 'block-device': 'vdb',
67 'overwrite': 'true'}66 'overwrite': 'true'}
@@ -89,15 +88,16 @@
89 self.glance = u.authenticate_glance_admin(self.keystone)88 self.glance = u.authenticate_glance_admin(self.keystone)
9089
91 # Authenticate swift user90 # Authenticate swift user
92 keystone_relation = self.keystone_sentry.relation('identity-service',91 keystone_relation = self.keystone_sentry.relation(
93 'swift-proxy:identity-service')92 'identity-service', 'swift-proxy:identity-service')
94 ep = self.keystone.service_catalog.url_for(service_type='identity',93 ep = self.keystone.service_catalog.url_for(service_type='identity',
95 endpoint_type='publicURL')94 endpoint_type='publicURL')
96 self.swift = swiftclient.Connection(authurl=ep,95 self.swift = swiftclient.Connection(
97 user=keystone_relation['service_username'],96 authurl=ep,
98 key=keystone_relation['service_password'],97 user=keystone_relation['service_username'],
99 tenant_name=keystone_relation['service_tenant'],98 key=keystone_relation['service_password'],
100 auth_version='2.0')99 tenant_name=keystone_relation['service_tenant'],
100 auth_version='2.0')
101101
102 # Create a demo tenant/role/user102 # Create a demo tenant/role/user
103 self.demo_tenant = 'demoTenant'103 self.demo_tenant = 'demoTenant'
@@ -122,29 +122,30 @@
122 def test_services(self):122 def test_services(self):
123 """Verify the expected services are running on the corresponding123 """Verify the expected services are running on the corresponding
124 service units."""124 service units."""
125 swift_storage_services = ['status swift-account',125 swift_storage_services = ['swift-account',
126 'status swift-account-auditor',126 'swift-account-auditor',
127 'status swift-account-reaper',127 'swift-account-reaper',
128 'status swift-account-replicator',128 'swift-account-replicator',
129 'status swift-container',129 'swift-container',
130 'status swift-container-auditor',130 'swift-container-auditor',
131 'status swift-container-replicator',131 'swift-container-replicator',
132 'status swift-container-updater',132 'swift-container-updater',
133 'status swift-object',133 'swift-object',
134 'status swift-object-auditor',134 'swift-object-auditor',
135 'status swift-object-replicator',135 'swift-object-replicator',
136 'status swift-object-updater']136 'swift-object-updater']
137 if self._get_openstack_release() >= self.precise_icehouse:137 if self._get_openstack_release() >= self.precise_icehouse:
138 swift_storage_services.append('status swift-container-sync')138 swift_storage_services.append('swift-container-sync')
139 commands = {139 service_names = {
140 self.mysql_sentry: ['status mysql'],140 self.mysql_sentry: ['mysql'],
141 self.keystone_sentry: ['status keystone'],141 self.keystone_sentry: ['keystone'],
142 self.glance_sentry: ['status glance-registry', 'status glance-api'],142 self.glance_sentry: [
143 self.swift_proxy_sentry: ['status swift-proxy'],143 'glance-registry', 'glance-api'],
144 self.swift_proxy_sentry: ['swift-proxy'],
144 self.swift_storage_sentry: swift_storage_services145 self.swift_storage_sentry: swift_storage_services
145 }146 }
146147
147 ret = u.validate_services(commands)148 ret = u.validate_services_by_name(service_names)
148 if ret:149 if ret:
149 amulet.raise_status(amulet.FAIL, msg=ret)150 amulet.raise_status(amulet.FAIL, msg=ret)
150151
@@ -303,8 +304,8 @@
303 file."""304 file."""
304 unit = self.swift_storage_sentry305 unit = self.swift_storage_sentry
305 conf = '/etc/swift/swift.conf'306 conf = '/etc/swift/swift.conf'
306 swift_proxy_relation = self.swift_proxy_sentry.relation('swift-storage',307 swift_proxy_relation = self.swift_proxy_sentry.relation(
307 'swift-storage:swift-storage')308 'swift-storage', 'swift-storage:swift-storage')
308 expected = {309 expected = {
309 'swift_hash_path_suffix': swift_proxy_relation['swift_hash']310 'swift_hash_path_suffix': swift_proxy_relation['swift_hash']
310 }311 }
@@ -405,7 +406,8 @@
405406
406 def test_image_create(self):407 def test_image_create(self):
407 """Create an instance in glance, which is backed by swift, and validate408 """Create an instance in glance, which is backed by swift, and validate
408 that some of the metadata for the image match in glance and swift."""409 that some of the metadata for the image match in glance and swift.
410 """
409 # NOTE(coreycb): Skipping failing test on folsom until resolved. On411 # NOTE(coreycb): Skipping failing test on folsom until resolved. On
410 # folsom only, uploading an image to glance gets 400 Bad412 # folsom only, uploading an image to glance gets 400 Bad
411 # Request - Error uploading image: (error): [Errno 111]413 # Request - Error uploading image: (error): [Errno 111]
@@ -435,7 +437,8 @@
435 # Validate that swift object's checksum/size match that from glance437 # Validate that swift object's checksum/size match that from glance
436 headers, containers = self.swift.get_account()438 headers, containers = self.swift.get_account()
437 if len(containers) != 1:439 if len(containers) != 1:
438 msg = "Expected 1 swift container, found {}".format(len(containers))440 msg = "Expected 1 swift container, found {}".format(
441 len(containers))
439 amulet.raise_status(amulet.FAIL, msg=msg)442 amulet.raise_status(amulet.FAIL, msg=msg)
440443
441 container_name = containers[0].get('name')444 container_name = containers[0].get('name')
@@ -449,14 +452,66 @@
449 swift_object_md5 = objects[0].get('hash')452 swift_object_md5 = objects[0].get('hash')
450453
451 if glance_image_size != swift_object_size:454 if glance_image_size != swift_object_size:
452 msg = "Glance image size {} != swift object size {}".format( \455 msg = "Glance image size {} != swift object size {}".format(
453 glance_image_size, swift_object_size)456 glance_image_size, swift_object_size)
454 amulet.raise_status(amulet.FAIL, msg=msg)457 amulet.raise_status(amulet.FAIL, msg=msg)
455458
456 if glance_image_md5 != swift_object_md5:459 if glance_image_md5 != swift_object_md5:
457 msg = "Glance image hash {} != swift object hash {}".format( \460 msg = "Glance image hash {} != swift object hash {}".format(
458 glance_image_md5, swift_object_md5)461 glance_image_md5, swift_object_md5)
459 amulet.raise_status(amulet.FAIL, msg=msg)462 amulet.raise_status(amulet.FAIL, msg=msg)
460463
461 # Cleanup464 # Cleanup
462 u.delete_image(self.glance, image)465 u.delete_image(self.glance, image)
466
467 def _assert_services(self, should_run):
468 swift_storage_services = ['swift-account-auditor',
469 'swift-account-reaper',
470 'swift-account-replicator',
471 'swift-account-server',
472 'swift-container-auditor',
473 'swift-container-replicator',
474 'swift-container-server',
475 'swift-container-sync',
476 'swift-container-updater',
477 'swift-object-auditor',
478 'swift-object-replicator',
479 'swift-object-server',
480 'swift-object-updater']
481 if self._get_openstack_release() < self.precise_icehouse:
482 swift_storage_services.remove('swift-container-sync')
483
484 u.get_unit_process_ids(
485 {self.swift_storage_sentry: swift_storage_services},
486 expect_success=should_run)
487 # No point using validate_unit_process_ids, since we don't
488 # care about how many PIDs, merely that they're running, so
489 # would populate expected with either True or False. This
490 # validation is already performed in get_process_id_list
491
492 def _test_pause(self):
493 u.log.info("Testing pause action")
494 self._assert_services(should_run=True)
495 pause_action_id = u.run_action(self.swift_storage_sentry, "pause")
496 assert u.wait_on_action(pause_action_id), "Pause action failed."
497
498 self._assert_services(should_run=False)
499
500 def _test_resume(self):
501 u.log.info("Testing resume action")
502 # service is left paused by _test_pause
503 self._assert_services(should_run=False)
504 resume_action_id = u.run_action(self.swift_storage_sentry, "resume")
505 assert u.wait_on_action(resume_action_id), "Resume action failed."
506
507 self._assert_services(should_run=True)
508
509 def test_z_actions(self):
510 """Pause and then resume swift-storage.
511
512 Note(sparkiegeek): The method name with the _z_ is a little odd
513 but it forces the test to run last. It just makes things
514 easier because restarting services requires re-authorization.
515 """
516 self._test_pause()
517 self._test_resume()
463518
=== modified file 'tests/charmhelpers/contrib/amulet/utils.py'
--- tests/charmhelpers/contrib/amulet/utils.py 2015-07-29 10:49:43 +0000
+++ tests/charmhelpers/contrib/amulet/utils.py 2015-08-17 13:44:46 +0000
@@ -14,17 +14,23 @@
14# You should have received a copy of the GNU Lesser General Public License14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1616
17import amulet
18import ConfigParser
19import distro_info
20import io17import io
18import json
21import logging19import logging
22import os20import os
23import re21import re
24import six22import subprocess
25import sys23import sys
26import time24import time
27import urlparse25
26import amulet
27import distro_info
28import six
29from six.moves import configparser
30if six.PY3:
31 from urllib import parse as urlparse
32else:
33 import urlparse
2834
2935
30class AmuletUtils(object):36class AmuletUtils(object):
@@ -142,19 +148,23 @@
142148
143 for service_name in services_list:149 for service_name in services_list:
144 if (self.ubuntu_releases.index(release) >= systemd_switch or150 if (self.ubuntu_releases.index(release) >= systemd_switch or
145 service_name == "rabbitmq-server"):151 service_name in ['rabbitmq-server', 'apache2']):
146 # init is systemd152 # init is systemd (or regular sysv)
147 cmd = 'sudo service {} status'.format(service_name)153 cmd = 'sudo service {} status'.format(service_name)
154 output, code = sentry_unit.run(cmd)
155 service_running = code == 0
148 elif self.ubuntu_releases.index(release) < systemd_switch:156 elif self.ubuntu_releases.index(release) < systemd_switch:
149 # init is upstart157 # init is upstart
150 cmd = 'sudo status {}'.format(service_name)158 cmd = 'sudo status {}'.format(service_name)
159 output, code = sentry_unit.run(cmd)
160 service_running = code == 0 and "start/running" in output
151161
152 output, code = sentry_unit.run(cmd)
153 self.log.debug('{} `{}` returned '162 self.log.debug('{} `{}` returned '
154 '{}'.format(sentry_unit.info['unit_name'],163 '{}'.format(sentry_unit.info['unit_name'],
155 cmd, code))164 cmd, code))
156 if code != 0:165 if not service_running:
157 return "command `{}` returned {}".format(cmd, str(code))166 return u"command `{}` returned {} {}".format(
167 cmd, output, str(code))
158 return None168 return None
159169
160 def _get_config(self, unit, filename):170 def _get_config(self, unit, filename):
@@ -164,7 +174,7 @@
164 # NOTE(beisner): by default, ConfigParser does not handle options174 # NOTE(beisner): by default, ConfigParser does not handle options
165 # with no value, such as the flags used in the mysql my.cnf file.175 # with no value, such as the flags used in the mysql my.cnf file.
166 # https://bugs.python.org/issue7005176 # https://bugs.python.org/issue7005
167 config = ConfigParser.ConfigParser(allow_no_value=True)177 config = configparser.ConfigParser(allow_no_value=True)
168 config.readfp(io.StringIO(file_contents))178 config.readfp(io.StringIO(file_contents))
169 return config179 return config
170180
@@ -450,15 +460,20 @@
450 cmd, code, output))460 cmd, code, output))
451 return None461 return None
452462
453 def get_process_id_list(self, sentry_unit, process_name):463 def get_process_id_list(self, sentry_unit, process_name,
464 expect_success=True):
454 """Get a list of process ID(s) from a single sentry juju unit465 """Get a list of process ID(s) from a single sentry juju unit
455 for a single process name.466 for a single process name.
456467
457 :param sentry_unit: Pointer to amulet sentry instance (juju unit)468 :param sentry_unit: Amulet sentry instance (juju unit)
458 :param process_name: Process name469 :param process_name: Process name
470 :param expect_success: If False, expect the PID to be missing,
471 raise if it is present.
459 :returns: List of process IDs472 :returns: List of process IDs
460 """473 """
461 cmd = 'pidof {}'.format(process_name)474 cmd = 'pidof -x {}'.format(process_name)
475 if not expect_success:
476 cmd += " || exit 0 && exit 1"
462 output, code = sentry_unit.run(cmd)477 output, code = sentry_unit.run(cmd)
463 if code != 0:478 if code != 0:
464 msg = ('{} `{}` returned {} '479 msg = ('{} `{}` returned {} '
@@ -467,14 +482,23 @@
467 amulet.raise_status(amulet.FAIL, msg=msg)482 amulet.raise_status(amulet.FAIL, msg=msg)
468 return str(output).split()483 return str(output).split()
469484
470 def get_unit_process_ids(self, unit_processes):485 def get_unit_process_ids(self, unit_processes, expect_success=True):
471 """Construct a dict containing unit sentries, process names, and486 """Construct a dict containing unit sentries, process names, and
472 process IDs."""487 process IDs.
488
489 :param unit_processes: A dictionary of Amulet sentry instance
490 to list of process names.
491 :param expect_success: if False expect the processes to not be
492 running, raise if they are.
493 :returns: Dictionary of Amulet sentry instance to dictionary
494 of process names to PIDs.
495 """
473 pid_dict = {}496 pid_dict = {}
474 for sentry_unit, process_list in unit_processes.iteritems():497 for sentry_unit, process_list in six.iteritems(unit_processes):
475 pid_dict[sentry_unit] = {}498 pid_dict[sentry_unit] = {}
476 for process in process_list:499 for process in process_list:
477 pids = self.get_process_id_list(sentry_unit, process)500 pids = self.get_process_id_list(
501 sentry_unit, process, expect_success=expect_success)
478 pid_dict[sentry_unit].update({process: pids})502 pid_dict[sentry_unit].update({process: pids})
479 return pid_dict503 return pid_dict
480504
@@ -488,7 +512,7 @@
488 return ('Unit count mismatch. expected, actual: {}, '512 return ('Unit count mismatch. expected, actual: {}, '
489 '{} '.format(len(expected), len(actual)))513 '{} '.format(len(expected), len(actual)))
490514
491 for (e_sentry, e_proc_names) in expected.iteritems():515 for (e_sentry, e_proc_names) in six.iteritems(expected):
492 e_sentry_name = e_sentry.info['unit_name']516 e_sentry_name = e_sentry.info['unit_name']
493 if e_sentry in actual.keys():517 if e_sentry in actual.keys():
494 a_proc_names = actual[e_sentry]518 a_proc_names = actual[e_sentry]
@@ -507,11 +531,23 @@
507 '{}'.format(e_proc_name, a_proc_name))531 '{}'.format(e_proc_name, a_proc_name))
508532
509 a_pids_length = len(a_pids)533 a_pids_length = len(a_pids)
510 if e_pids_length != a_pids_length:534 fail_msg = ('PID count mismatch. {} ({}) expected, actual: '
511 return ('PID count mismatch. {} ({}) expected, actual: '
512 '{}, {} ({})'.format(e_sentry_name, e_proc_name,535 '{}, {} ({})'.format(e_sentry_name, e_proc_name,
513 e_pids_length, a_pids_length,536 e_pids_length, a_pids_length,
514 a_pids))537 a_pids))
538
539 # If expected is not bool, ensure PID quantities match
540 if not isinstance(e_pids_length, bool) and \
541 a_pids_length != e_pids_length:
542 return fail_msg
543 # If expected is bool True, ensure 1 or more PIDs exist
544 elif isinstance(e_pids_length, bool) and \
545 e_pids_length is True and a_pids_length < 1:
546 return fail_msg
547 # If expected is bool False, ensure 0 PIDs exist
548 elif isinstance(e_pids_length, bool) and \
549 e_pids_length is False and a_pids_length != 0:
550 return fail_msg
515 else:551 else:
516 self.log.debug('PID check OK: {} {} {}: '552 self.log.debug('PID check OK: {} {} {}: '
517 '{}'.format(e_sentry_name, e_proc_name,553 '{}'.format(e_sentry_name, e_proc_name,
@@ -531,3 +567,30 @@
531 return 'Dicts within list are not identical'567 return 'Dicts within list are not identical'
532568
533 return None569 return None
570
571 def run_action(self, unit_sentry, action,
572 _check_output=subprocess.check_output):
573 """Run the named action on a given unit sentry.
574
575 _check_output parameter is used for dependency injection.
576
577 @return action_id.
578 """
579 unit_id = unit_sentry.info["unit_name"]
580 command = ["juju", "action", "do", "--format=json", unit_id, action]
581 self.log.info("Running command: %s\n" % " ".join(command))
582 output = _check_output(command, universal_newlines=True)
583 data = json.loads(output)
584 action_id = data[u'Action queued with id']
585 return action_id
586
587 def wait_on_action(self, action_id, _check_output=subprocess.check_output):
588 """Wait for a given action, returning if it completed or not.
589
590 _check_output parameter is used for dependency injection.
591 """
592 command = ["juju", "action", "fetch", "--format=json", "--wait=0",
593 action_id]
594 output = _check_output(command, universal_newlines=True)
595 data = json.loads(output)
596 return data.get(u"status") == "completed"
534597
=== modified file 'tests/charmhelpers/contrib/openstack/amulet/deployment.py'
--- tests/charmhelpers/contrib/openstack/amulet/deployment.py 2015-07-29 10:49:43 +0000
+++ tests/charmhelpers/contrib/openstack/amulet/deployment.py 2015-08-17 13:44:46 +0000
@@ -44,7 +44,7 @@
44 Determine if the local branch being tested is derived from its44 Determine if the local branch being tested is derived from its
45 stable or next (dev) branch, and based on this, use the corresonding45 stable or next (dev) branch, and based on this, use the corresonding
46 stable or next branches for the other_services."""46 stable or next branches for the other_services."""
47 base_charms = ['mysql', 'mongodb']47 base_charms = ['mysql', 'mongodb', 'nrpe']
4848
49 if self.series in ['precise', 'trusty']:49 if self.series in ['precise', 'trusty']:
50 base_series = self.series50 base_series = self.series
@@ -81,7 +81,7 @@
81 'ceph-osd', 'ceph-radosgw']81 'ceph-osd', 'ceph-radosgw']
82 # Most OpenStack subordinate charms do not expose an origin option82 # Most OpenStack subordinate charms do not expose an origin option
83 # as that is controlled by the principle.83 # as that is controlled by the principle.
84 ignore = ['cinder-ceph', 'hacluster', 'neutron-openvswitch']84 ignore = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe']
8585
86 if self.openstack:86 if self.openstack:
87 for svc in services:87 for svc in services:
8888
=== added file 'unit_tests/test_actions.py'
--- unit_tests/test_actions.py 1970-01-01 00:00:00 +0000
+++ unit_tests/test_actions.py 2015-08-17 13:44:46 +0000
@@ -0,0 +1,239 @@
1import argparse
2import tempfile
3import unittest
4
5import mock
6import yaml
7
8from test_utils import CharmTestCase
9
10import actions.actions
11
12
13class PauseTestCase(CharmTestCase):
14
15 def setUp(self):
16 super(PauseTestCase, self).setUp(
17 actions.actions, ["service_pause", "status_set"])
18
19 class FakeArgs(object):
20 services = ['swift-account',
21 'swift-account-auditor',
22 'swift-account-reaper',
23 'swift-account-replicator',
24 'swift-container',
25 'swift-container-auditor',
26 'swift-container-updater',
27 'swift-container-replicator',
28 'swift-container-sync',
29 'swift-object',
30 'swift-object-auditor',
31 'swift-object-updater',
32 'swift-object-replicator']
33 self.args = FakeArgs()
34
35 def test_pauses_services(self):
36 """Pause action pauses all of the Swift services."""
37 pause_calls = []
38
39 def fake_service_pause(svc):
40 pause_calls.append(svc)
41 return True
42
43 self.service_pause.side_effect = fake_service_pause
44
45 actions.actions.pause(self.args)
46 self.assertEqual(pause_calls, ['swift-account',
47 'swift-account-auditor',
48 'swift-account-reaper',
49 'swift-account-replicator',
50 'swift-container',
51 'swift-container-auditor',
52 'swift-container-updater',
53 'swift-container-replicator',
54 'swift-container-sync',
55 'swift-object',
56 'swift-object-auditor',
57 'swift-object-updater',
58 'swift-object-replicator'])
59
60 def test_bails_out_early_on_error(self):
61 """Pause action fails early if there are errors stopping a service."""
62 pause_calls = []
63
64 def maybe_kill(svc):
65 if svc == "swift-container":
66 return False
67 else:
68 pause_calls.append(svc)
69 return True
70
71 self.service_pause.side_effect = maybe_kill
72 self.assertRaisesRegexp(
73 Exception, "swift-container didn't stop cleanly.",
74 actions.actions.pause, self.args)
75 self.assertEqual(pause_calls, ['swift-account',
76 'swift-account-auditor',
77 'swift-account-reaper',
78 'swift-account-replicator'])
79
80 def test_status_mode(self):
81 """Pause action sets the status to maintenance."""
82 status_calls = []
83 self.status_set.side_effect = lambda state, msg: status_calls.append(
84 state)
85
86 actions.actions.pause(self.args)
87 self.assertEqual(status_calls, ["maintenance"])
88
89 def test_status_message(self):
90 """Pause action sets a status message reflecting that it's paused."""
91 status_calls = []
92 self.status_set.side_effect = lambda state, msg: status_calls.append(
93 msg)
94
95 actions.actions.pause(self.args)
96 self.assertEqual(
97 status_calls, ["Paused. "
98 "Use 'resume' action to resume normal service."])
99
100
101class ResumeTestCase(CharmTestCase):
102
103 def setUp(self):
104 super(ResumeTestCase, self).setUp(
105 actions.actions, ["service_resume", "status_set"])
106
107 class FakeArgs(object):
108 services = ['swift-account',
109 'swift-account-auditor',
110 'swift-account-reaper',
111 'swift-account-replicator',
112 'swift-container',
113 'swift-container-auditor',
114 'swift-container-updater',
115 'swift-container-replicator',
116 'swift-container-sync',
117 'swift-object',
118 'swift-object-auditor',
119 'swift-object-updater',
120 'swift-object-replicator']
121 self.args = FakeArgs()
122
123 def test_resumes_services(self):
124 """Resume action resumes all of the Swift services."""
125 resume_calls = []
126
127 def fake_service_resume(svc):
128 resume_calls.append(svc)
129 return True
130
131 self.service_resume.side_effect = fake_service_resume
132 actions.actions.resume(self.args)
133 self.assertEqual(resume_calls, ['swift-account',
134 'swift-account-auditor',
135 'swift-account-reaper',
136 'swift-account-replicator',
137 'swift-container',
138 'swift-container-auditor',
139 'swift-container-updater',
140 'swift-container-replicator',
141 'swift-container-sync',
142 'swift-object',
143 'swift-object-auditor',
144 'swift-object-updater',
145 'swift-object-replicator'])
146
147 def test_bails_out_early_on_error(self):
148 """Resume action fails early if there are errors starting a service."""
149 resume_calls = []
150
151 def maybe_kill(svc):
152 if svc == "swift-container":
153 return False
154 else:
155 resume_calls.append(svc)
156 return True
157
158 self.service_resume.side_effect = maybe_kill
159 self.assertRaisesRegexp(
160 Exception, "swift-container didn't start cleanly.",
161 actions.actions.resume, self.args)
162 self.assertEqual(resume_calls, ['swift-account',
163 'swift-account-auditor',
164 'swift-account-reaper',
165 'swift-account-replicator'])
166
167 def test_status_mode(self):
168 """Resume action sets the status to maintenance."""
169 status_calls = []
170 self.status_set.side_effect = lambda state, msg: status_calls.append(
171 state)
172
173 actions.actions.resume(self.args)
174 self.assertEqual(status_calls, ["active"])
175
176 def test_status_message(self):
177 """Resume action sets an empty status message."""
178 status_calls = []
179 self.status_set.side_effect = lambda state, msg: status_calls.append(
180 msg)
181
182 actions.actions.resume(self.args)
183 self.assertEqual(status_calls, [""])
184
185
186class GetActionParserTestCase(unittest.TestCase):
187
188 def test_definition_from_yaml(self):
189 """ArgumentParser is seeded from actions.yaml."""
190 actions_yaml = tempfile.NamedTemporaryFile(
191 prefix="GetActionParserTestCase", suffix="yaml")
192 actions_yaml.write(yaml.dump({"foo": {"description": "Foo is bar"}}))
193 actions_yaml.seek(0)
194 parser = actions.actions.get_action_parser(actions_yaml.name, "foo",
195 get_services=lambda: [])
196 self.assertEqual(parser.description, 'Foo is bar')
197
198
199class MainTestCase(CharmTestCase):
200
201 def setUp(self):
202 super(MainTestCase, self).setUp(
203 actions.actions, ["_get_action_name",
204 "get_action_parser",
205 "action_fail"])
206
207 def test_invokes_pause(self):
208 dummy_calls = []
209
210 def dummy_action(args):
211 dummy_calls.append(True)
212
213 self._get_action_name.side_effect = lambda: "foo"
214 self.get_action_parser = lambda: argparse.ArgumentParser()
215 with mock.patch.dict(actions.actions.ACTIONS, {"foo": dummy_action}):
216 actions.actions.main([])
217 self.assertEqual(dummy_calls, [True])
218
219 def test_unknown_action(self):
220 """Unknown actions aren't a traceback."""
221 self._get_action_name.side_effect = lambda: "foo"
222 self.get_action_parser = lambda: argparse.ArgumentParser()
223 exit_string = actions.actions.main([])
224 self.assertEqual("Action foo undefined", exit_string)
225
226 def test_failing_action(self):
227 """Actions which traceback trigger action_fail() calls."""
228 dummy_calls = []
229
230 self.action_fail.side_effect = dummy_calls.append
231 self._get_action_name.side_effect = lambda: "foo"
232
233 def dummy_action(args):
234 raise ValueError("uh oh")
235
236 self.get_action_parser = lambda: argparse.ArgumentParser()
237 with mock.patch.dict(actions.actions.ACTIONS, {"foo": dummy_action}):
238 actions.actions.main([])
239 self.assertEqual(dummy_calls, ["uh oh"])

Subscribers

People subscribed via source and target branches