Merge lp:~gnuoy/charms/trusty/cinder-ceph/1453940-stable into lp:~openstack-charmers-archive/charms/precise/cinder-ceph/trunk
- Trusty Tahr (14.04)
- 1453940-stable
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 36 |
Proposed branch: | lp:~gnuoy/charms/trusty/cinder-ceph/1453940-stable |
Merge into: | lp:~openstack-charmers-archive/charms/precise/cinder-ceph/trunk |
Diff against target: |
751 lines (+357/-82) 12 files modified
hooks/charmhelpers/cli/__init__.py (+1/-5) hooks/charmhelpers/cli/commands.py (+4/-4) hooks/charmhelpers/cli/hookenv.py (+23/-0) hooks/charmhelpers/contrib/openstack/context.py (+8/-9) hooks/charmhelpers/contrib/openstack/utils.py (+7/-5) hooks/charmhelpers/contrib/storage/linux/ceph.py (+224/-2) hooks/charmhelpers/contrib/storage/linux/utils.py (+3/-2) hooks/charmhelpers/core/hookenv.py (+1/-20) hooks/charmhelpers/core/host.py (+2/-2) hooks/cinder_hooks.py (+13/-22) tests/basic_deployment.py (+52/-8) unit_tests/test_cinder_hooks.py (+19/-3) |
To merge this branch: | bzr merge lp:~gnuoy/charms/trusty/cinder-ceph/1453940-stable |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Chris Glass (community) | Approve | ||
Ryan Beisner (community) | Needs Fixing | ||
Review via email:
|
Commit message
Description of the change
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #10411 cinder-ceph for gnuoy mp271261
LINT OK: passed
Build: http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #6551 cinder-ceph for gnuoy mp271261
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #10436 cinder-ceph for gnuoy mp271261
LINT OK: passed
Build: http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #9629 cinder-ceph for gnuoy mp271261
UNIT OK: passed
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #6578 cinder-ceph for gnuoy mp271261
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Ryan Beisner (1chb1n) wrote : | # |
To save you the time of grepping logs, that amulet fail is real.
00:17:33.288 2015-09-22 13:59:57,026 test_201_
...
00:17:33.289 Broker request invalid or failed: None
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Liam Young (gnuoy) wrote : | # |
OSCI failure caused by ceph having not landed yet.
- 38. By Liam Young
-
Empty commit to trigger osci
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #9652 cinder-ceph for gnuoy mp271261
UNIT OK: passed
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #6585 cinder-ceph for gnuoy mp271261
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
- 39. By Liam Young
-
Empty commit to trigger osci
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #9654 cinder-ceph for gnuoy mp271261
UNIT OK: passed
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #10463 cinder-ceph for gnuoy mp271261
LINT OK: passed
Build: http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #6606 cinder-ceph for gnuoy mp271261
AMULET FAIL: amulet-test failed
AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.
Full amulet test output: http://
Build: http://
- 40. By Liam Young
-
Empty commit to kick osci
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_unit_test #9712 cinder-ceph for gnuoy mp271261
UNIT OK: passed
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_lint_check #10524 cinder-ceph for gnuoy mp271261
LINT OK: passed
Build: http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Liam Young (gnuoy) wrote : | # |
Since the latest revision of ceph is still not in the charm store I tests an amulet run manually with the latest ceph from stable. It LGTM http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Chris Glass (tribaal) wrote : | # |
After further testing on a bastion, this should pass. In order to cut down on wait time I'm going to go ahead and merge this already.
+1
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
uosci-testing-bot (uosci-testing-bot) wrote : | # |
charm_amulet_test #6666 cinder-ceph for gnuoy mp271261
AMULET OK: passed
Build: http://
Preview Diff
1 | === modified file 'hooks/charmhelpers/cli/__init__.py' |
2 | --- hooks/charmhelpers/cli/__init__.py 2015-08-10 16:34:26 +0000 |
3 | +++ hooks/charmhelpers/cli/__init__.py 2015-09-23 07:31:24 +0000 |
4 | @@ -152,15 +152,11 @@ |
5 | arguments = self.argument_parser.parse_args() |
6 | argspec = inspect.getargspec(arguments.func) |
7 | vargs = [] |
8 | - kwargs = {} |
9 | for arg in argspec.args: |
10 | vargs.append(getattr(arguments, arg)) |
11 | if argspec.varargs: |
12 | vargs.extend(getattr(arguments, argspec.varargs)) |
13 | - if argspec.keywords: |
14 | - for kwarg in argspec.keywords.items(): |
15 | - kwargs[kwarg] = getattr(arguments, kwarg) |
16 | - output = arguments.func(*vargs, **kwargs) |
17 | + output = arguments.func(*vargs) |
18 | if getattr(arguments.func, '_cli_test_command', False): |
19 | self.exit_code = 0 if output else 1 |
20 | output = '' |
21 | |
22 | === modified file 'hooks/charmhelpers/cli/commands.py' |
23 | --- hooks/charmhelpers/cli/commands.py 2015-08-10 16:34:26 +0000 |
24 | +++ hooks/charmhelpers/cli/commands.py 2015-09-23 07:31:24 +0000 |
25 | @@ -26,7 +26,7 @@ |
26 | """ |
27 | Import the sub-modules which have decorated subcommands to register with chlp. |
28 | """ |
29 | -import host # noqa |
30 | -import benchmark # noqa |
31 | -import unitdata # noqa |
32 | -from charmhelpers.core import hookenv # noqa |
33 | +from . import host # noqa |
34 | +from . import benchmark # noqa |
35 | +from . import unitdata # noqa |
36 | +from . import hookenv # noqa |
37 | |
38 | === added file 'hooks/charmhelpers/cli/hookenv.py' |
39 | --- hooks/charmhelpers/cli/hookenv.py 1970-01-01 00:00:00 +0000 |
40 | +++ hooks/charmhelpers/cli/hookenv.py 2015-09-23 07:31:24 +0000 |
41 | @@ -0,0 +1,23 @@ |
42 | +# Copyright 2014-2015 Canonical Limited. |
43 | +# |
44 | +# This file is part of charm-helpers. |
45 | +# |
46 | +# charm-helpers is free software: you can redistribute it and/or modify |
47 | +# it under the terms of the GNU Lesser General Public License version 3 as |
48 | +# published by the Free Software Foundation. |
49 | +# |
50 | +# charm-helpers is distributed in the hope that it will be useful, |
51 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
52 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
53 | +# GNU Lesser General Public License for more details. |
54 | +# |
55 | +# You should have received a copy of the GNU Lesser General Public License |
56 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
57 | + |
58 | +from . import cmdline |
59 | +from charmhelpers.core import hookenv |
60 | + |
61 | + |
62 | +cmdline.subcommand('relation-id')(hookenv.relation_id._wrapped) |
63 | +cmdline.subcommand('service-name')(hookenv.service_name) |
64 | +cmdline.subcommand('remote-service-name')(hookenv.remote_service_name._wrapped) |
65 | |
66 | === modified file 'hooks/charmhelpers/contrib/openstack/context.py' |
67 | --- hooks/charmhelpers/contrib/openstack/context.py 2015-08-10 16:34:26 +0000 |
68 | +++ hooks/charmhelpers/contrib/openstack/context.py 2015-09-23 07:31:24 +0000 |
69 | @@ -483,13 +483,15 @@ |
70 | |
71 | log('Generating template context for ceph', level=DEBUG) |
72 | mon_hosts = [] |
73 | - auth = None |
74 | - key = None |
75 | - use_syslog = str(config('use-syslog')).lower() |
76 | + ctxt = { |
77 | + 'use_syslog': str(config('use-syslog')).lower() |
78 | + } |
79 | for rid in relation_ids('ceph'): |
80 | for unit in related_units(rid): |
81 | - auth = relation_get('auth', rid=rid, unit=unit) |
82 | - key = relation_get('key', rid=rid, unit=unit) |
83 | + if not ctxt.get('auth'): |
84 | + ctxt['auth'] = relation_get('auth', rid=rid, unit=unit) |
85 | + if not ctxt.get('key'): |
86 | + ctxt['key'] = relation_get('key', rid=rid, unit=unit) |
87 | ceph_pub_addr = relation_get('ceph-public-address', rid=rid, |
88 | unit=unit) |
89 | unit_priv_addr = relation_get('private-address', rid=rid, |
90 | @@ -498,10 +500,7 @@ |
91 | ceph_addr = format_ipv6_addr(ceph_addr) or ceph_addr |
92 | mon_hosts.append(ceph_addr) |
93 | |
94 | - ctxt = {'mon_hosts': ' '.join(sorted(mon_hosts)), |
95 | - 'auth': auth, |
96 | - 'key': key, |
97 | - 'use_syslog': use_syslog} |
98 | + ctxt['mon_hosts'] = ' '.join(sorted(mon_hosts)) |
99 | |
100 | if not os.path.isdir('/etc/ceph'): |
101 | os.mkdir('/etc/ceph') |
102 | |
103 | === modified file 'hooks/charmhelpers/contrib/openstack/utils.py' |
104 | --- hooks/charmhelpers/contrib/openstack/utils.py 2015-08-10 16:34:26 +0000 |
105 | +++ hooks/charmhelpers/contrib/openstack/utils.py 2015-09-23 07:31:24 +0000 |
106 | @@ -1,5 +1,3 @@ |
107 | -#!/usr/bin/python |
108 | - |
109 | # Copyright 2014-2015 Canonical Limited. |
110 | # |
111 | # This file is part of charm-helpers. |
112 | @@ -167,9 +165,9 @@ |
113 | error_out(e) |
114 | |
115 | |
116 | -def get_os_version_codename(codename): |
117 | +def get_os_version_codename(codename, version_map=OPENSTACK_CODENAMES): |
118 | '''Determine OpenStack version number from codename.''' |
119 | - for k, v in six.iteritems(OPENSTACK_CODENAMES): |
120 | + for k, v in six.iteritems(version_map): |
121 | if v == codename: |
122 | return k |
123 | e = 'Could not derive OpenStack version for '\ |
124 | @@ -392,7 +390,11 @@ |
125 | import apt_pkg as apt |
126 | src = config('openstack-origin') |
127 | cur_vers = get_os_version_package(package) |
128 | - available_vers = get_os_version_install_source(src) |
129 | + if "swift" in package: |
130 | + codename = get_os_codename_install_source(src) |
131 | + available_vers = get_os_version_codename(codename, SWIFT_CODENAMES) |
132 | + else: |
133 | + available_vers = get_os_version_install_source(src) |
134 | apt.init() |
135 | return apt.version_compare(available_vers, cur_vers) == 1 |
136 | |
137 | |
138 | === modified file 'hooks/charmhelpers/contrib/storage/linux/ceph.py' |
139 | --- hooks/charmhelpers/contrib/storage/linux/ceph.py 2015-08-10 16:34:26 +0000 |
140 | +++ hooks/charmhelpers/contrib/storage/linux/ceph.py 2015-09-23 07:31:24 +0000 |
141 | @@ -28,6 +28,7 @@ |
142 | import shutil |
143 | import json |
144 | import time |
145 | +import uuid |
146 | |
147 | from subprocess import ( |
148 | check_call, |
149 | @@ -35,8 +36,10 @@ |
150 | CalledProcessError, |
151 | ) |
152 | from charmhelpers.core.hookenv import ( |
153 | + local_unit, |
154 | relation_get, |
155 | relation_ids, |
156 | + relation_set, |
157 | related_units, |
158 | log, |
159 | DEBUG, |
160 | @@ -411,17 +414,52 @@ |
161 | |
162 | The API is versioned and defaults to version 1. |
163 | """ |
164 | - def __init__(self, api_version=1): |
165 | + def __init__(self, api_version=1, request_id=None): |
166 | self.api_version = api_version |
167 | + if request_id: |
168 | + self.request_id = request_id |
169 | + else: |
170 | + self.request_id = str(uuid.uuid1()) |
171 | self.ops = [] |
172 | |
173 | def add_op_create_pool(self, name, replica_count=3): |
174 | self.ops.append({'op': 'create-pool', 'name': name, |
175 | 'replicas': replica_count}) |
176 | |
177 | + def set_ops(self, ops): |
178 | + """Set request ops to provided value. |
179 | + |
180 | + Useful for injecting ops that come from a previous request |
181 | + to allow comparisons to ensure validity. |
182 | + """ |
183 | + self.ops = ops |
184 | + |
185 | @property |
186 | def request(self): |
187 | - return json.dumps({'api-version': self.api_version, 'ops': self.ops}) |
188 | + return json.dumps({'api-version': self.api_version, 'ops': self.ops, |
189 | + 'request-id': self.request_id}) |
190 | + |
191 | + def _ops_equal(self, other): |
192 | + if len(self.ops) == len(other.ops): |
193 | + for req_no in range(0, len(self.ops)): |
194 | + for key in ['replicas', 'name', 'op']: |
195 | + if self.ops[req_no][key] != other.ops[req_no][key]: |
196 | + return False |
197 | + else: |
198 | + return False |
199 | + return True |
200 | + |
201 | + def __eq__(self, other): |
202 | + if not isinstance(other, self.__class__): |
203 | + return False |
204 | + if self.api_version == other.api_version and \ |
205 | + self._ops_equal(other): |
206 | + return True |
207 | + else: |
208 | + return False |
209 | + |
210 | + def __ne__(self, other): |
211 | + return not self.__eq__(other) |
212 | |
213 | |
214 | class CephBrokerRsp(object): |
215 | @@ -431,14 +469,198 @@ |
216 | |
217 | The API is versioned and defaults to version 1. |
218 | """ |
219 | + |
220 | def __init__(self, encoded_rsp): |
221 | self.api_version = None |
222 | self.rsp = json.loads(encoded_rsp) |
223 | |
224 | @property |
225 | + def request_id(self): |
226 | + return self.rsp.get('request-id') |
227 | + |
228 | + @property |
229 | def exit_code(self): |
230 | return self.rsp.get('exit-code') |
231 | |
232 | @property |
233 | def exit_msg(self): |
234 | return self.rsp.get('stderr') |
235 | + |
236 | + |
237 | +# Ceph Broker Conversation: |
238 | +# If a charm needs an action to be taken by ceph it can create a CephBrokerRq |
239 | +# and send that request to ceph via the ceph relation. The CephBrokerRq has a |
240 | +# unique id so that the client can identity which CephBrokerRsp is associated |
241 | +# with the request. Ceph will also respond to each client unit individually |
242 | +# creating a response key per client unit eg glance/0 will get a CephBrokerRsp |
243 | +# via key broker-rsp-glance-0 |
244 | +# |
245 | +# To use this the charm can just do something like: |
246 | +# |
247 | +# from charmhelpers.contrib.storage.linux.ceph import ( |
248 | +# send_request_if_needed, |
249 | +# is_request_complete, |
250 | +# CephBrokerRq, |
251 | +# ) |
252 | +# |
253 | +# @hooks.hook('ceph-relation-changed') |
254 | +# def ceph_changed(): |
255 | +# rq = CephBrokerRq() |
256 | +# rq.add_op_create_pool(name='poolname', replica_count=3) |
257 | +# |
258 | +# if is_request_complete(rq): |
259 | +# <Request complete actions> |
260 | +# else: |
261 | +# send_request_if_needed(get_ceph_request()) |
262 | +# |
263 | +# CephBrokerRq and CephBrokerRsp are serialized into JSON. Below is an example |
264 | +# of glance having sent a request to ceph which ceph has successfully processed |
265 | +# 'ceph:8': { |
266 | +# 'ceph/0': { |
267 | +# 'auth': 'cephx', |
268 | +# 'broker-rsp-glance-0': '{"request-id": "0bc7dc54", "exit-code": 0}', |
269 | +# 'broker_rsp': '{"request-id": "0da543b8", "exit-code": 0}', |
270 | +# 'ceph-public-address': '10.5.44.103', |
271 | +# 'key': 'AQCLDttVuHXINhAAvI144CB09dYchhHyTUY9BQ==', |
272 | +# 'private-address': '10.5.44.103', |
273 | +# }, |
274 | +# 'glance/0': { |
275 | +# 'broker_req': ('{"api-version": 1, "request-id": "0bc7dc54", ' |
276 | +# '"ops": [{"replicas": 3, "name": "glance", ' |
277 | +# '"op": "create-pool"}]}'), |
278 | +# 'private-address': '10.5.44.109', |
279 | +# }, |
280 | +# } |
281 | + |
282 | +def get_previous_request(rid): |
283 | + """Return the last ceph broker request sent on a given relation |
284 | + |
285 | + @param rid: Relation id to query for request |
286 | + """ |
287 | + request = None |
288 | + broker_req = relation_get(attribute='broker_req', rid=rid, |
289 | + unit=local_unit()) |
290 | + if broker_req: |
291 | + request_data = json.loads(broker_req) |
292 | + request = CephBrokerRq(api_version=request_data['api-version'], |
293 | + request_id=request_data['request-id']) |
294 | + request.set_ops(request_data['ops']) |
295 | + |
296 | + return request |
297 | + |
298 | + |
299 | +def get_request_states(request): |
300 | + """Return a dict of requests per relation id with their corresponding |
301 | + completion state. |
302 | + |
303 | + This allows a charm, which has a request for ceph, to see whether there is |
304 | + an equivalent request already being processed and if so what state that |
305 | + request is in. |
306 | + |
307 | + @param request: A CephBrokerRq object |
308 | + """ |
309 | + complete = [] |
310 | + requests = {} |
311 | + for rid in relation_ids('ceph'): |
312 | + complete = False |
313 | + previous_request = get_previous_request(rid) |
314 | + if request == previous_request: |
315 | + sent = True |
316 | + complete = is_request_complete_for_rid(previous_request, rid) |
317 | + else: |
318 | + sent = False |
319 | + complete = False |
320 | + |
321 | + requests[rid] = { |
322 | + 'sent': sent, |
323 | + 'complete': complete, |
324 | + } |
325 | + |
326 | + return requests |
327 | + |
328 | + |
329 | +def is_request_sent(request): |
330 | + """Check to see if a functionally equivalent request has already been sent |
331 | + |
332 | + Returns True if a similair request has been sent |
333 | + |
334 | + @param request: A CephBrokerRq object |
335 | + """ |
336 | + states = get_request_states(request) |
337 | + for rid in states.keys(): |
338 | + if not states[rid]['sent']: |
339 | + return False |
340 | + |
341 | + return True |
342 | + |
343 | + |
344 | +def is_request_complete(request): |
345 | + """Check to see if a functionally equivalent request has already been |
346 | + completed |
347 | + |
348 | + Returns True if a similair request has been completed |
349 | + |
350 | + @param request: A CephBrokerRq object |
351 | + """ |
352 | + states = get_request_states(request) |
353 | + for rid in states.keys(): |
354 | + if not states[rid]['complete']: |
355 | + return False |
356 | + |
357 | + return True |
358 | + |
359 | + |
360 | +def is_request_complete_for_rid(request, rid): |
361 | + """Check if a given request has been completed on the given relation |
362 | + |
363 | + @param request: A CephBrokerRq object |
364 | + @param rid: Relation ID |
365 | + """ |
366 | + broker_key = get_broker_rsp_key() |
367 | + for unit in related_units(rid): |
368 | + rdata = relation_get(rid=rid, unit=unit) |
369 | + if rdata.get(broker_key): |
370 | + rsp = CephBrokerRsp(rdata.get(broker_key)) |
371 | + if rsp.request_id == request.request_id: |
372 | + if not rsp.exit_code: |
373 | + return True |
374 | + else: |
375 | + # The remote unit sent no reply targeted at this unit so either the |
376 | + # remote ceph cluster does not support unit targeted replies or it |
377 | + # has not processed our request yet. |
378 | + if rdata.get('broker_rsp'): |
379 | + request_data = json.loads(rdata['broker_rsp']) |
380 | + if request_data.get('request-id'): |
381 | + log('Ignoring legacy broker_rsp without unit key as remote ' |
382 | + 'service supports unit specific replies', level=DEBUG) |
383 | + else: |
384 | + log('Using legacy broker_rsp as remote service does not ' |
385 | + 'supports unit specific replies', level=DEBUG) |
386 | + rsp = CephBrokerRsp(rdata['broker_rsp']) |
387 | + if not rsp.exit_code: |
388 | + return True |
389 | + |
390 | + return False |
391 | + |
392 | + |
393 | +def get_broker_rsp_key(): |
394 | + """Return broker response key for this unit |
395 | + |
396 | + This is the key that ceph is going to use to pass request status |
397 | + information back to this unit |
398 | + """ |
399 | + return 'broker-rsp-' + local_unit().replace('/', '-') |
400 | + |
401 | + |
402 | +def send_request_if_needed(request): |
403 | + """Send broker request if an equivalent request has not already been sent |
404 | + |
405 | + @param request: A CephBrokerRq object |
406 | + """ |
407 | + if is_request_sent(request): |
408 | + log('Request already sent but not complete, not sending new request', |
409 | + level=DEBUG) |
410 | + else: |
411 | + for rid in relation_ids('ceph'): |
412 | + log('Sending request {}'.format(request.request_id), level=DEBUG) |
413 | + relation_set(relation_id=rid, broker_req=request.request) |
414 | |
415 | === modified file 'hooks/charmhelpers/contrib/storage/linux/utils.py' |
416 | --- hooks/charmhelpers/contrib/storage/linux/utils.py 2015-08-10 16:34:26 +0000 |
417 | +++ hooks/charmhelpers/contrib/storage/linux/utils.py 2015-09-23 07:31:24 +0000 |
418 | @@ -43,9 +43,10 @@ |
419 | |
420 | :param block_device: str: Full path of block device to clean. |
421 | ''' |
422 | + # https://github.com/ceph/ceph/commit/fdd7f8d83afa25c4e09aaedd90ab93f3b64a677b |
423 | # sometimes sgdisk exits non-zero; this is OK, dd will clean up |
424 | - call(['sgdisk', '--zap-all', '--mbrtogpt', |
425 | - '--clear', block_device]) |
426 | + call(['sgdisk', '--zap-all', '--', block_device]) |
427 | + call(['sgdisk', '--clear', '--mbrtogpt', '--', block_device]) |
428 | dev_end = check_output(['blockdev', '--getsz', |
429 | block_device]).decode('UTF-8') |
430 | gpt_end = int(dev_end.split()[0]) - 100 |
431 | |
432 | === modified file 'hooks/charmhelpers/core/hookenv.py' |
433 | --- hooks/charmhelpers/core/hookenv.py 2015-08-10 16:34:26 +0000 |
434 | +++ hooks/charmhelpers/core/hookenv.py 2015-09-23 07:31:24 +0000 |
435 | @@ -34,23 +34,6 @@ |
436 | import tempfile |
437 | from subprocess import CalledProcessError |
438 | |
439 | -try: |
440 | - from charmhelpers.cli import cmdline |
441 | -except ImportError as e: |
442 | - # due to the anti-pattern of partially synching charmhelpers directly |
443 | - # into charms, it's possible that charmhelpers.cli is not available; |
444 | - # if that's the case, they don't really care about using the cli anyway, |
445 | - # so mock it out |
446 | - if str(e) == 'No module named cli': |
447 | - class cmdline(object): |
448 | - @classmethod |
449 | - def subcommand(cls, *args, **kwargs): |
450 | - def _wrap(func): |
451 | - return func |
452 | - return _wrap |
453 | - else: |
454 | - raise |
455 | - |
456 | import six |
457 | if not six.PY3: |
458 | from UserDict import UserDict |
459 | @@ -91,6 +74,7 @@ |
460 | res = func(*args, **kwargs) |
461 | cache[key] = res |
462 | return res |
463 | + wrapper._wrapped = func |
464 | return wrapper |
465 | |
466 | |
467 | @@ -190,7 +174,6 @@ |
468 | return os.environ.get('JUJU_RELATION', None) |
469 | |
470 | |
471 | -@cmdline.subcommand() |
472 | @cached |
473 | def relation_id(relation_name=None, service_or_unit=None): |
474 | """The relation ID for the current or a specified relation""" |
475 | @@ -216,13 +199,11 @@ |
476 | return os.environ.get('JUJU_REMOTE_UNIT', None) |
477 | |
478 | |
479 | -@cmdline.subcommand() |
480 | def service_name(): |
481 | """The name service group this unit belongs to""" |
482 | return local_unit().split('/')[0] |
483 | |
484 | |
485 | -@cmdline.subcommand() |
486 | @cached |
487 | def remote_service_name(relid=None): |
488 | """The remote service name for a given relation-id (or the current relation)""" |
489 | |
490 | === modified file 'hooks/charmhelpers/core/host.py' |
491 | --- hooks/charmhelpers/core/host.py 2015-08-10 16:34:26 +0000 |
492 | +++ hooks/charmhelpers/core/host.py 2015-09-23 07:31:24 +0000 |
493 | @@ -72,7 +72,7 @@ |
494 | stopped = service_stop(service_name) |
495 | # XXX: Support systemd too |
496 | override_path = os.path.join( |
497 | - init_dir, '{}.conf.override'.format(service_name)) |
498 | + init_dir, '{}.override'.format(service_name)) |
499 | with open(override_path, 'w') as fh: |
500 | fh.write("manual\n") |
501 | return stopped |
502 | @@ -86,7 +86,7 @@ |
503 | if init_dir is None: |
504 | init_dir = "/etc/init" |
505 | override_path = os.path.join( |
506 | - init_dir, '{}.conf.override'.format(service_name)) |
507 | + init_dir, '{}.override'.format(service_name)) |
508 | if os.path.exists(override_path): |
509 | os.unlink(override_path) |
510 | started = service_start(service_name) |
511 | |
512 | === modified file 'hooks/cinder_hooks.py' |
513 | --- hooks/cinder_hooks.py 2015-09-16 12:55:55 +0000 |
514 | +++ hooks/cinder_hooks.py 2015-09-23 07:31:24 +0000 |
515 | @@ -17,12 +17,9 @@ |
516 | UnregisteredHookError, |
517 | config, |
518 | service_name, |
519 | - relation_get, |
520 | relation_set, |
521 | relation_ids, |
522 | log, |
523 | - INFO, |
524 | - ERROR, |
525 | ) |
526 | from charmhelpers.fetch import apt_install, apt_update |
527 | from charmhelpers.core.host import ( |
528 | @@ -30,9 +27,10 @@ |
529 | service_restart, |
530 | ) |
531 | from charmhelpers.contrib.storage.linux.ceph import ( |
532 | + send_request_if_needed, |
533 | + is_request_complete, |
534 | ensure_ceph_keyring, |
535 | CephBrokerRq, |
536 | - CephBrokerRsp, |
537 | delete_keyring, |
538 | ) |
539 | from charmhelpers.payload.execd import execd_preinstall |
540 | @@ -55,6 +53,14 @@ |
541 | os.mkdir('/etc/ceph') |
542 | |
543 | |
544 | +def get_ceph_request(): |
545 | + service = service_name() |
546 | + rq = CephBrokerRq() |
547 | + replicas = config('ceph-osd-replication-count') |
548 | + rq.add_op_create_pool(name=service, replica_count=replicas) |
549 | + return rq |
550 | + |
551 | + |
552 | @hooks.hook('ceph-relation-changed') |
553 | @restart_on_change(restart_map()) |
554 | def ceph_changed(): |
555 | @@ -68,32 +74,17 @@ |
556 | log('Could not create ceph keyring: peer not ready?') |
557 | return |
558 | |
559 | - settings = relation_get() |
560 | - if settings and 'broker_rsp' in settings: |
561 | - rsp = CephBrokerRsp(settings['broker_rsp']) |
562 | - # Non-zero return code implies failure |
563 | - if rsp.exit_code: |
564 | - log("Ceph broker request failed (rc=%s, msg=%s)" % |
565 | - (rsp.exit_code, rsp.exit_msg), level=ERROR) |
566 | - return |
567 | - |
568 | - log("Ceph broker request succeeded (rc=%s, msg=%s)" % |
569 | - (rsp.exit_code, rsp.exit_msg), level=INFO) |
570 | + if is_request_complete(get_ceph_request()): |
571 | + log('Request complete') |
572 | CONFIGS.write_all() |
573 | set_ceph_env_variables(service=service) |
574 | for rid in relation_ids('storage-backend'): |
575 | storage_backend(rid) |
576 | - |
577 | # Ensure that cinder-volume is restarted since only now can we |
578 | # guarantee that ceph resources are ready. |
579 | service_restart('cinder-volume') |
580 | else: |
581 | - rq = CephBrokerRq() |
582 | - replicas = config('ceph-osd-replication-count') |
583 | - rq.add_op_create_pool(name=service, replica_count=replicas) |
584 | - for rid in relation_ids('ceph'): |
585 | - relation_set(relation_id=rid, broker_req=rq.request) |
586 | - log("Request(s) sent to Ceph broker (rid=%s)" % (rid)) |
587 | + send_request_if_needed(get_ceph_request()) |
588 | |
589 | |
590 | @hooks.hook('ceph-relation-broken') |
591 | |
592 | === modified file 'tests/basic_deployment.py' |
593 | --- tests/basic_deployment.py 2015-08-10 17:28:35 +0000 |
594 | +++ tests/basic_deployment.py 2015-09-23 07:31:24 +0000 |
595 | @@ -4,6 +4,7 @@ |
596 | Basic cinder-ceph functional test. |
597 | """ |
598 | import amulet |
599 | +import json |
600 | import time |
601 | |
602 | from charmhelpers.contrib.openstack.amulet.deployment import ( |
603 | @@ -279,40 +280,83 @@ |
604 | amulet.raise_status(amulet.FAIL, |
605 | msg='cinder endpoint: {}'.format(ret)) |
606 | |
607 | + def validate_broker_req(self, unit, relation, expected): |
608 | + rel_data = json.loads(unit.relation( |
609 | + relation[0], |
610 | + relation[1])['broker_req']) |
611 | + if rel_data['api-version'] != expected['api-version']: |
612 | + return "Broker request api mismatch" |
613 | + for index in range(0, len(rel_data['ops'])): |
614 | + actual_op = rel_data['ops'][index] |
615 | + expected_op = expected['ops'][index] |
616 | + for key in ['op', 'name', 'replicas']: |
617 | + if actual_op[key] == expected_op[key]: |
618 | + u.log.debug("OK op {} key {}".format(index, key)) |
619 | + else: |
620 | + return "Mismatch, op: {} key: {}".format(index, key) |
621 | + return None |
622 | + |
623 | + def get_broker_request(self): |
624 | + client_unit = self.cinder_ceph_sentry |
625 | + broker_req = json.loads(client_unit.relation( |
626 | + 'ceph', |
627 | + 'ceph:client')['broker_req']) |
628 | + return broker_req |
629 | + |
630 | + def get_broker_response(self): |
631 | + broker_request = self.get_broker_request() |
632 | + response_key = "broker-rsp-cinder-ceph-0" |
633 | + ceph_sentrys = [self.ceph0_sentry, self.ceph1_sentry, self.ceph2_sentry] |
634 | + for sentry in ceph_sentrys: |
635 | + relation_data = sentry.relation('client', 'cinder-ceph:ceph') |
636 | + if relation_data.get(response_key): |
637 | + broker_response = json.loads(relation_data[response_key]) |
638 | + if broker_request['request-id'] == broker_response['request-id']: |
639 | + return broker_response |
640 | + |
641 | def test_200_cinderceph_ceph_ceph_relation(self): |
642 | u.log.debug('Checking cinder-ceph:ceph to ceph:client ' |
643 | 'relation data...') |
644 | unit = self.cinder_ceph_sentry |
645 | relation = ['ceph', 'ceph:client'] |
646 | |
647 | - req = ('{"api-version": 1, "ops": [{"replicas": 3, "name": ' |
648 | - '"cinder-ceph", "op": "create-pool"}]}') |
649 | - |
650 | + req = { |
651 | + "api-version": 1, |
652 | + "ops": [{"replicas": 3, "name": "cinder-ceph", "op": "create-pool"}] |
653 | + } |
654 | expected = { |
655 | 'private-address': u.valid_ip, |
656 | - 'broker_req': req |
657 | + 'broker_req': u.not_null, |
658 | } |
659 | ret = u.validate_relation_data(unit, relation, expected) |
660 | if ret: |
661 | msg = u.relation_error('cinder-ceph ceph', ret) |
662 | amulet.raise_status(amulet.FAIL, msg=msg) |
663 | + ret = self.validate_broker_req(unit, relation, req) |
664 | + if ret: |
665 | + msg = u.relation_error('cinder-ceph ceph', ret) |
666 | + amulet.raise_status(amulet.FAIL, msg=msg) |
667 | |
668 | def test_201_ceph_cinderceph_ceph_relation(self): |
669 | u.log.debug('Checking ceph:client to cinder-ceph:ceph ' |
670 | 'relation data...') |
671 | - unit = self.ceph0_sentry |
672 | + response_key = "broker-rsp-cinder-ceph-0" |
673 | + ceph_unit = self.ceph0_sentry |
674 | relation = ['client', 'cinder-ceph:ceph'] |
675 | expected = { |
676 | 'key': u.not_null, |
677 | 'private-address': u.valid_ip, |
678 | - 'broker_rsp': '{"exit-code": 0}', |
679 | 'ceph-public-address': u.valid_ip, |
680 | - 'auth': 'none' |
681 | + 'auth': 'none', |
682 | } |
683 | - ret = u.validate_relation_data(unit, relation, expected) |
684 | + ret = u.validate_relation_data(ceph_unit, relation, expected) |
685 | if ret: |
686 | msg = u.relation_error('cinder cinder-ceph storage-backend', ret) |
687 | amulet.raise_status(amulet.FAIL, msg=msg) |
688 | + broker_response = self.get_broker_response() |
689 | + if not broker_response or broker_response['exit-code'] != 0: |
690 | + msg='Broker request invalid or failed: {}'.format(broker_response) |
691 | + amulet.raise_status(amulet.FAIL, msg=msg) |
692 | |
693 | def test_202_cinderceph_cinder_backend_relation(self): |
694 | u.log.debug('Checking cinder-ceph:storage-backend to ' |
695 | |
696 | === modified file 'unit_tests/test_cinder_hooks.py' |
697 | --- unit_tests/test_cinder_hooks.py 2015-09-16 12:55:55 +0000 |
698 | +++ unit_tests/test_cinder_hooks.py 2015-09-23 07:31:24 +0000 |
699 | @@ -18,11 +18,12 @@ |
700 | 'register_configs', |
701 | 'restart_map', |
702 | 'set_ceph_env_variables', |
703 | + 'is_request_complete', |
704 | + 'send_request_if_needed', |
705 | 'CONFIGS', |
706 | # charmhelpers.core.hookenv |
707 | 'config', |
708 | 'relation_ids', |
709 | - 'relation_get', |
710 | 'relation_set', |
711 | 'service_name', |
712 | 'service_restart', |
713 | @@ -69,9 +70,8 @@ |
714 | @patch('charmhelpers.core.hookenv.config') |
715 | def test_ceph_changed(self, mock_config): |
716 | '''It ensures ceph assets created on ceph changed''' |
717 | + self.is_request_complete.return_value = True |
718 | self.CONFIGS.complete_contexts.return_value = ['ceph'] |
719 | - rsp = json.dumps({'exit_code': 0}) |
720 | - self.relation_get.return_value = {'broker_rsp': rsp} |
721 | self.service_name.return_value = 'cinder' |
722 | self.ensure_ceph_keyring.return_value = True |
723 | hooks.hooks.execute(['hooks/ceph-relation-changed']) |
724 | @@ -81,11 +81,27 @@ |
725 | self.assertTrue(self.CONFIGS.write_all.called) |
726 | self.set_ceph_env_variables.assert_called_with(service='cinder') |
727 | |
728 | + @patch.object(hooks, 'get_ceph_request') |
729 | + @patch('charmhelpers.core.hookenv.config') |
730 | + def test_ceph_changed_newrq(self, mock_config, mock_get_ceph_request): |
731 | + '''It ensures ceph assets created on ceph changed''' |
732 | + mock_get_ceph_request.return_value = 'cephreq' |
733 | + self.is_request_complete.return_value = False |
734 | + self.CONFIGS.complete_contexts.return_value = ['ceph'] |
735 | + self.service_name.return_value = 'cinder' |
736 | + self.ensure_ceph_keyring.return_value = True |
737 | + hooks.hooks.execute(['hooks/ceph-relation-changed']) |
738 | + self.ensure_ceph_keyring.assert_called_with(service='cinder', |
739 | + user='cinder', |
740 | + group='cinder') |
741 | + self.send_request_if_needed.assert_called_with('cephreq') |
742 | + |
743 | @patch('charmhelpers.core.hookenv.config') |
744 | def test_ceph_changed_no_keys(self, mock_config): |
745 | '''It ensures ceph assets created on ceph changed''' |
746 | self.CONFIGS.complete_contexts.return_value = ['ceph'] |
747 | self.service_name.return_value = 'cinder' |
748 | + self.is_request_complete.return_value = True |
749 | self.ensure_ceph_keyring.return_value = False |
750 | hooks.hooks.execute(['hooks/ceph-relation-changed']) |
751 | # NOTE(jamespage): If ensure_ceph keyring fails, then |
charm_unit_test #9552 cinder-ceph for gnuoy mp271261
UNIT OK: passed
Build: http:// 10.245. 162.77: 8080/job/ charm_unit_ test/9552/