Merge lp:~gnuoy/charms/trusty/cinder-ceph/1453940-stable into lp:~openstack-charmers-archive/charms/precise/cinder-ceph/trunk

Proposed by Liam Young on 2015-09-16
Status: Merged
Merged at revision: 36
Proposed branch: lp:~gnuoy/charms/trusty/cinder-ceph/1453940-stable
Merge into: lp:~openstack-charmers-archive/charms/precise/cinder-ceph/trunk
Diff against target: 751 lines (+357/-82)
12 files modified
hooks/charmhelpers/cli/__init__.py (+1/-5)
hooks/charmhelpers/cli/commands.py (+4/-4)
hooks/charmhelpers/cli/hookenv.py (+23/-0)
hooks/charmhelpers/contrib/openstack/context.py (+8/-9)
hooks/charmhelpers/contrib/openstack/utils.py (+7/-5)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+224/-2)
hooks/charmhelpers/contrib/storage/linux/utils.py (+3/-2)
hooks/charmhelpers/core/hookenv.py (+1/-20)
hooks/charmhelpers/core/host.py (+2/-2)
hooks/cinder_hooks.py (+13/-22)
tests/basic_deployment.py (+52/-8)
unit_tests/test_cinder_hooks.py (+19/-3)
To merge this branch: bzr merge lp:~gnuoy/charms/trusty/cinder-ceph/1453940-stable
Reviewer Review Type Date Requested Status
Chris Glass (community) 2015-09-16 Approve on 2015-09-23
Ryan Beisner Needs Fixing on 2015-09-22
Review via email: mp+271261@code.launchpad.net
To post a comment you must log in.

charm_unit_test #9552 cinder-ceph for gnuoy mp271261
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9552/

charm_lint_check #10411 cinder-ceph for gnuoy mp271261
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10411/

charm_amulet_test #6551 cinder-ceph for gnuoy mp271261
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12514387/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6551/

charm_lint_check #10436 cinder-ceph for gnuoy mp271261
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10436/

charm_unit_test #9629 cinder-ceph for gnuoy mp271261
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9629/

charm_amulet_test #6578 cinder-ceph for gnuoy mp271261
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12521251/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6578/

Ryan Beisner (1chb1n) wrote :

To save you the time of grepping logs, that amulet fail is real.

00:17:33.288 2015-09-22 13:59:57,026 test_201_ceph_cinderceph_ceph_relation DEBUG: Checking ceph:client to cinder-ceph:ceph relation data...

...

00:17:33.289 Broker request invalid or failed: None

review: Needs Fixing
Liam Young (gnuoy) wrote :

OSCI failure caused by ceph having not landed yet.

38. By Liam Young on 2015-09-22

Empty commit to trigger osci

charm_unit_test #9652 cinder-ceph for gnuoy mp271261
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9652/

charm_amulet_test #6585 cinder-ceph for gnuoy mp271261
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12521979/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6585/

39. By Liam Young on 2015-09-22

Empty commit to trigger osci

charm_unit_test #9654 cinder-ceph for gnuoy mp271261
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9654/

charm_lint_check #10463 cinder-ceph for gnuoy mp271261
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10463/

charm_amulet_test #6606 cinder-ceph for gnuoy mp271261
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12526301/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6606/

40. By Liam Young on 2015-09-23

Empty commit to kick osci

charm_unit_test #9712 cinder-ceph for gnuoy mp271261
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9712/

charm_lint_check #10524 cinder-ceph for gnuoy mp271261
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10524/

Liam Young (gnuoy) wrote :

Since the latest revision of ceph is still not in the charm store I tests an amulet run manually with the latest ceph from stable. It LGTM http://paste.ubuntu.com/12529898/

Chris Glass (tribaal) wrote :

After further testing on a bastion, this should pass. In order to cut down on wait time I'm going to go ahead and merge this already.

+1

review: Approve

charm_amulet_test #6666 cinder-ceph for gnuoy mp271261
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6666/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'hooks/charmhelpers/cli/__init__.py'
2--- hooks/charmhelpers/cli/__init__.py 2015-08-10 16:34:26 +0000
3+++ hooks/charmhelpers/cli/__init__.py 2015-09-23 07:31:24 +0000
4@@ -152,15 +152,11 @@
5 arguments = self.argument_parser.parse_args()
6 argspec = inspect.getargspec(arguments.func)
7 vargs = []
8- kwargs = {}
9 for arg in argspec.args:
10 vargs.append(getattr(arguments, arg))
11 if argspec.varargs:
12 vargs.extend(getattr(arguments, argspec.varargs))
13- if argspec.keywords:
14- for kwarg in argspec.keywords.items():
15- kwargs[kwarg] = getattr(arguments, kwarg)
16- output = arguments.func(*vargs, **kwargs)
17+ output = arguments.func(*vargs)
18 if getattr(arguments.func, '_cli_test_command', False):
19 self.exit_code = 0 if output else 1
20 output = ''
21
22=== modified file 'hooks/charmhelpers/cli/commands.py'
23--- hooks/charmhelpers/cli/commands.py 2015-08-10 16:34:26 +0000
24+++ hooks/charmhelpers/cli/commands.py 2015-09-23 07:31:24 +0000
25@@ -26,7 +26,7 @@
26 """
27 Import the sub-modules which have decorated subcommands to register with chlp.
28 """
29-import host # noqa
30-import benchmark # noqa
31-import unitdata # noqa
32-from charmhelpers.core import hookenv # noqa
33+from . import host # noqa
34+from . import benchmark # noqa
35+from . import unitdata # noqa
36+from . import hookenv # noqa
37
38=== added file 'hooks/charmhelpers/cli/hookenv.py'
39--- hooks/charmhelpers/cli/hookenv.py 1970-01-01 00:00:00 +0000
40+++ hooks/charmhelpers/cli/hookenv.py 2015-09-23 07:31:24 +0000
41@@ -0,0 +1,23 @@
42+# Copyright 2014-2015 Canonical Limited.
43+#
44+# This file is part of charm-helpers.
45+#
46+# charm-helpers is free software: you can redistribute it and/or modify
47+# it under the terms of the GNU Lesser General Public License version 3 as
48+# published by the Free Software Foundation.
49+#
50+# charm-helpers is distributed in the hope that it will be useful,
51+# but WITHOUT ANY WARRANTY; without even the implied warranty of
52+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
53+# GNU Lesser General Public License for more details.
54+#
55+# You should have received a copy of the GNU Lesser General Public License
56+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
57+
58+from . import cmdline
59+from charmhelpers.core import hookenv
60+
61+
62+cmdline.subcommand('relation-id')(hookenv.relation_id._wrapped)
63+cmdline.subcommand('service-name')(hookenv.service_name)
64+cmdline.subcommand('remote-service-name')(hookenv.remote_service_name._wrapped)
65
66=== modified file 'hooks/charmhelpers/contrib/openstack/context.py'
67--- hooks/charmhelpers/contrib/openstack/context.py 2015-08-10 16:34:26 +0000
68+++ hooks/charmhelpers/contrib/openstack/context.py 2015-09-23 07:31:24 +0000
69@@ -483,13 +483,15 @@
70
71 log('Generating template context for ceph', level=DEBUG)
72 mon_hosts = []
73- auth = None
74- key = None
75- use_syslog = str(config('use-syslog')).lower()
76+ ctxt = {
77+ 'use_syslog': str(config('use-syslog')).lower()
78+ }
79 for rid in relation_ids('ceph'):
80 for unit in related_units(rid):
81- auth = relation_get('auth', rid=rid, unit=unit)
82- key = relation_get('key', rid=rid, unit=unit)
83+ if not ctxt.get('auth'):
84+ ctxt['auth'] = relation_get('auth', rid=rid, unit=unit)
85+ if not ctxt.get('key'):
86+ ctxt['key'] = relation_get('key', rid=rid, unit=unit)
87 ceph_pub_addr = relation_get('ceph-public-address', rid=rid,
88 unit=unit)
89 unit_priv_addr = relation_get('private-address', rid=rid,
90@@ -498,10 +500,7 @@
91 ceph_addr = format_ipv6_addr(ceph_addr) or ceph_addr
92 mon_hosts.append(ceph_addr)
93
94- ctxt = {'mon_hosts': ' '.join(sorted(mon_hosts)),
95- 'auth': auth,
96- 'key': key,
97- 'use_syslog': use_syslog}
98+ ctxt['mon_hosts'] = ' '.join(sorted(mon_hosts))
99
100 if not os.path.isdir('/etc/ceph'):
101 os.mkdir('/etc/ceph')
102
103=== modified file 'hooks/charmhelpers/contrib/openstack/utils.py'
104--- hooks/charmhelpers/contrib/openstack/utils.py 2015-08-10 16:34:26 +0000
105+++ hooks/charmhelpers/contrib/openstack/utils.py 2015-09-23 07:31:24 +0000
106@@ -1,5 +1,3 @@
107-#!/usr/bin/python
108-
109 # Copyright 2014-2015 Canonical Limited.
110 #
111 # This file is part of charm-helpers.
112@@ -167,9 +165,9 @@
113 error_out(e)
114
115
116-def get_os_version_codename(codename):
117+def get_os_version_codename(codename, version_map=OPENSTACK_CODENAMES):
118 '''Determine OpenStack version number from codename.'''
119- for k, v in six.iteritems(OPENSTACK_CODENAMES):
120+ for k, v in six.iteritems(version_map):
121 if v == codename:
122 return k
123 e = 'Could not derive OpenStack version for '\
124@@ -392,7 +390,11 @@
125 import apt_pkg as apt
126 src = config('openstack-origin')
127 cur_vers = get_os_version_package(package)
128- available_vers = get_os_version_install_source(src)
129+ if "swift" in package:
130+ codename = get_os_codename_install_source(src)
131+ available_vers = get_os_version_codename(codename, SWIFT_CODENAMES)
132+ else:
133+ available_vers = get_os_version_install_source(src)
134 apt.init()
135 return apt.version_compare(available_vers, cur_vers) == 1
136
137
138=== modified file 'hooks/charmhelpers/contrib/storage/linux/ceph.py'
139--- hooks/charmhelpers/contrib/storage/linux/ceph.py 2015-08-10 16:34:26 +0000
140+++ hooks/charmhelpers/contrib/storage/linux/ceph.py 2015-09-23 07:31:24 +0000
141@@ -28,6 +28,7 @@
142 import shutil
143 import json
144 import time
145+import uuid
146
147 from subprocess import (
148 check_call,
149@@ -35,8 +36,10 @@
150 CalledProcessError,
151 )
152 from charmhelpers.core.hookenv import (
153+ local_unit,
154 relation_get,
155 relation_ids,
156+ relation_set,
157 related_units,
158 log,
159 DEBUG,
160@@ -411,17 +414,52 @@
161
162 The API is versioned and defaults to version 1.
163 """
164- def __init__(self, api_version=1):
165+ def __init__(self, api_version=1, request_id=None):
166 self.api_version = api_version
167+ if request_id:
168+ self.request_id = request_id
169+ else:
170+ self.request_id = str(uuid.uuid1())
171 self.ops = []
172
173 def add_op_create_pool(self, name, replica_count=3):
174 self.ops.append({'op': 'create-pool', 'name': name,
175 'replicas': replica_count})
176
177+ def set_ops(self, ops):
178+ """Set request ops to provided value.
179+
180+ Useful for injecting ops that come from a previous request
181+ to allow comparisons to ensure validity.
182+ """
183+ self.ops = ops
184+
185 @property
186 def request(self):
187- return json.dumps({'api-version': self.api_version, 'ops': self.ops})
188+ return json.dumps({'api-version': self.api_version, 'ops': self.ops,
189+ 'request-id': self.request_id})
190+
191+ def _ops_equal(self, other):
192+ if len(self.ops) == len(other.ops):
193+ for req_no in range(0, len(self.ops)):
194+ for key in ['replicas', 'name', 'op']:
195+ if self.ops[req_no][key] != other.ops[req_no][key]:
196+ return False
197+ else:
198+ return False
199+ return True
200+
201+ def __eq__(self, other):
202+ if not isinstance(other, self.__class__):
203+ return False
204+ if self.api_version == other.api_version and \
205+ self._ops_equal(other):
206+ return True
207+ else:
208+ return False
209+
210+ def __ne__(self, other):
211+ return not self.__eq__(other)
212
213
214 class CephBrokerRsp(object):
215@@ -431,14 +469,198 @@
216
217 The API is versioned and defaults to version 1.
218 """
219+
220 def __init__(self, encoded_rsp):
221 self.api_version = None
222 self.rsp = json.loads(encoded_rsp)
223
224 @property
225+ def request_id(self):
226+ return self.rsp.get('request-id')
227+
228+ @property
229 def exit_code(self):
230 return self.rsp.get('exit-code')
231
232 @property
233 def exit_msg(self):
234 return self.rsp.get('stderr')
235+
236+
237+# Ceph Broker Conversation:
238+# If a charm needs an action to be taken by ceph it can create a CephBrokerRq
239+# and send that request to ceph via the ceph relation. The CephBrokerRq has a
240+# unique id so that the client can identity which CephBrokerRsp is associated
241+# with the request. Ceph will also respond to each client unit individually
242+# creating a response key per client unit eg glance/0 will get a CephBrokerRsp
243+# via key broker-rsp-glance-0
244+#
245+# To use this the charm can just do something like:
246+#
247+# from charmhelpers.contrib.storage.linux.ceph import (
248+# send_request_if_needed,
249+# is_request_complete,
250+# CephBrokerRq,
251+# )
252+#
253+# @hooks.hook('ceph-relation-changed')
254+# def ceph_changed():
255+# rq = CephBrokerRq()
256+# rq.add_op_create_pool(name='poolname', replica_count=3)
257+#
258+# if is_request_complete(rq):
259+# <Request complete actions>
260+# else:
261+# send_request_if_needed(get_ceph_request())
262+#
263+# CephBrokerRq and CephBrokerRsp are serialized into JSON. Below is an example
264+# of glance having sent a request to ceph which ceph has successfully processed
265+# 'ceph:8': {
266+# 'ceph/0': {
267+# 'auth': 'cephx',
268+# 'broker-rsp-glance-0': '{"request-id": "0bc7dc54", "exit-code": 0}',
269+# 'broker_rsp': '{"request-id": "0da543b8", "exit-code": 0}',
270+# 'ceph-public-address': '10.5.44.103',
271+# 'key': 'AQCLDttVuHXINhAAvI144CB09dYchhHyTUY9BQ==',
272+# 'private-address': '10.5.44.103',
273+# },
274+# 'glance/0': {
275+# 'broker_req': ('{"api-version": 1, "request-id": "0bc7dc54", '
276+# '"ops": [{"replicas": 3, "name": "glance", '
277+# '"op": "create-pool"}]}'),
278+# 'private-address': '10.5.44.109',
279+# },
280+# }
281+
282+def get_previous_request(rid):
283+ """Return the last ceph broker request sent on a given relation
284+
285+ @param rid: Relation id to query for request
286+ """
287+ request = None
288+ broker_req = relation_get(attribute='broker_req', rid=rid,
289+ unit=local_unit())
290+ if broker_req:
291+ request_data = json.loads(broker_req)
292+ request = CephBrokerRq(api_version=request_data['api-version'],
293+ request_id=request_data['request-id'])
294+ request.set_ops(request_data['ops'])
295+
296+ return request
297+
298+
299+def get_request_states(request):
300+ """Return a dict of requests per relation id with their corresponding
301+ completion state.
302+
303+ This allows a charm, which has a request for ceph, to see whether there is
304+ an equivalent request already being processed and if so what state that
305+ request is in.
306+
307+ @param request: A CephBrokerRq object
308+ """
309+ complete = []
310+ requests = {}
311+ for rid in relation_ids('ceph'):
312+ complete = False
313+ previous_request = get_previous_request(rid)
314+ if request == previous_request:
315+ sent = True
316+ complete = is_request_complete_for_rid(previous_request, rid)
317+ else:
318+ sent = False
319+ complete = False
320+
321+ requests[rid] = {
322+ 'sent': sent,
323+ 'complete': complete,
324+ }
325+
326+ return requests
327+
328+
329+def is_request_sent(request):
330+ """Check to see if a functionally equivalent request has already been sent
331+
332+ Returns True if a similair request has been sent
333+
334+ @param request: A CephBrokerRq object
335+ """
336+ states = get_request_states(request)
337+ for rid in states.keys():
338+ if not states[rid]['sent']:
339+ return False
340+
341+ return True
342+
343+
344+def is_request_complete(request):
345+ """Check to see if a functionally equivalent request has already been
346+ completed
347+
348+ Returns True if a similair request has been completed
349+
350+ @param request: A CephBrokerRq object
351+ """
352+ states = get_request_states(request)
353+ for rid in states.keys():
354+ if not states[rid]['complete']:
355+ return False
356+
357+ return True
358+
359+
360+def is_request_complete_for_rid(request, rid):
361+ """Check if a given request has been completed on the given relation
362+
363+ @param request: A CephBrokerRq object
364+ @param rid: Relation ID
365+ """
366+ broker_key = get_broker_rsp_key()
367+ for unit in related_units(rid):
368+ rdata = relation_get(rid=rid, unit=unit)
369+ if rdata.get(broker_key):
370+ rsp = CephBrokerRsp(rdata.get(broker_key))
371+ if rsp.request_id == request.request_id:
372+ if not rsp.exit_code:
373+ return True
374+ else:
375+ # The remote unit sent no reply targeted at this unit so either the
376+ # remote ceph cluster does not support unit targeted replies or it
377+ # has not processed our request yet.
378+ if rdata.get('broker_rsp'):
379+ request_data = json.loads(rdata['broker_rsp'])
380+ if request_data.get('request-id'):
381+ log('Ignoring legacy broker_rsp without unit key as remote '
382+ 'service supports unit specific replies', level=DEBUG)
383+ else:
384+ log('Using legacy broker_rsp as remote service does not '
385+ 'supports unit specific replies', level=DEBUG)
386+ rsp = CephBrokerRsp(rdata['broker_rsp'])
387+ if not rsp.exit_code:
388+ return True
389+
390+ return False
391+
392+
393+def get_broker_rsp_key():
394+ """Return broker response key for this unit
395+
396+ This is the key that ceph is going to use to pass request status
397+ information back to this unit
398+ """
399+ return 'broker-rsp-' + local_unit().replace('/', '-')
400+
401+
402+def send_request_if_needed(request):
403+ """Send broker request if an equivalent request has not already been sent
404+
405+ @param request: A CephBrokerRq object
406+ """
407+ if is_request_sent(request):
408+ log('Request already sent but not complete, not sending new request',
409+ level=DEBUG)
410+ else:
411+ for rid in relation_ids('ceph'):
412+ log('Sending request {}'.format(request.request_id), level=DEBUG)
413+ relation_set(relation_id=rid, broker_req=request.request)
414
415=== modified file 'hooks/charmhelpers/contrib/storage/linux/utils.py'
416--- hooks/charmhelpers/contrib/storage/linux/utils.py 2015-08-10 16:34:26 +0000
417+++ hooks/charmhelpers/contrib/storage/linux/utils.py 2015-09-23 07:31:24 +0000
418@@ -43,9 +43,10 @@
419
420 :param block_device: str: Full path of block device to clean.
421 '''
422+ # https://github.com/ceph/ceph/commit/fdd7f8d83afa25c4e09aaedd90ab93f3b64a677b
423 # sometimes sgdisk exits non-zero; this is OK, dd will clean up
424- call(['sgdisk', '--zap-all', '--mbrtogpt',
425- '--clear', block_device])
426+ call(['sgdisk', '--zap-all', '--', block_device])
427+ call(['sgdisk', '--clear', '--mbrtogpt', '--', block_device])
428 dev_end = check_output(['blockdev', '--getsz',
429 block_device]).decode('UTF-8')
430 gpt_end = int(dev_end.split()[0]) - 100
431
432=== modified file 'hooks/charmhelpers/core/hookenv.py'
433--- hooks/charmhelpers/core/hookenv.py 2015-08-10 16:34:26 +0000
434+++ hooks/charmhelpers/core/hookenv.py 2015-09-23 07:31:24 +0000
435@@ -34,23 +34,6 @@
436 import tempfile
437 from subprocess import CalledProcessError
438
439-try:
440- from charmhelpers.cli import cmdline
441-except ImportError as e:
442- # due to the anti-pattern of partially synching charmhelpers directly
443- # into charms, it's possible that charmhelpers.cli is not available;
444- # if that's the case, they don't really care about using the cli anyway,
445- # so mock it out
446- if str(e) == 'No module named cli':
447- class cmdline(object):
448- @classmethod
449- def subcommand(cls, *args, **kwargs):
450- def _wrap(func):
451- return func
452- return _wrap
453- else:
454- raise
455-
456 import six
457 if not six.PY3:
458 from UserDict import UserDict
459@@ -91,6 +74,7 @@
460 res = func(*args, **kwargs)
461 cache[key] = res
462 return res
463+ wrapper._wrapped = func
464 return wrapper
465
466
467@@ -190,7 +174,6 @@
468 return os.environ.get('JUJU_RELATION', None)
469
470
471-@cmdline.subcommand()
472 @cached
473 def relation_id(relation_name=None, service_or_unit=None):
474 """The relation ID for the current or a specified relation"""
475@@ -216,13 +199,11 @@
476 return os.environ.get('JUJU_REMOTE_UNIT', None)
477
478
479-@cmdline.subcommand()
480 def service_name():
481 """The name service group this unit belongs to"""
482 return local_unit().split('/')[0]
483
484
485-@cmdline.subcommand()
486 @cached
487 def remote_service_name(relid=None):
488 """The remote service name for a given relation-id (or the current relation)"""
489
490=== modified file 'hooks/charmhelpers/core/host.py'
491--- hooks/charmhelpers/core/host.py 2015-08-10 16:34:26 +0000
492+++ hooks/charmhelpers/core/host.py 2015-09-23 07:31:24 +0000
493@@ -72,7 +72,7 @@
494 stopped = service_stop(service_name)
495 # XXX: Support systemd too
496 override_path = os.path.join(
497- init_dir, '{}.conf.override'.format(service_name))
498+ init_dir, '{}.override'.format(service_name))
499 with open(override_path, 'w') as fh:
500 fh.write("manual\n")
501 return stopped
502@@ -86,7 +86,7 @@
503 if init_dir is None:
504 init_dir = "/etc/init"
505 override_path = os.path.join(
506- init_dir, '{}.conf.override'.format(service_name))
507+ init_dir, '{}.override'.format(service_name))
508 if os.path.exists(override_path):
509 os.unlink(override_path)
510 started = service_start(service_name)
511
512=== modified file 'hooks/cinder_hooks.py'
513--- hooks/cinder_hooks.py 2015-09-16 12:55:55 +0000
514+++ hooks/cinder_hooks.py 2015-09-23 07:31:24 +0000
515@@ -17,12 +17,9 @@
516 UnregisteredHookError,
517 config,
518 service_name,
519- relation_get,
520 relation_set,
521 relation_ids,
522 log,
523- INFO,
524- ERROR,
525 )
526 from charmhelpers.fetch import apt_install, apt_update
527 from charmhelpers.core.host import (
528@@ -30,9 +27,10 @@
529 service_restart,
530 )
531 from charmhelpers.contrib.storage.linux.ceph import (
532+ send_request_if_needed,
533+ is_request_complete,
534 ensure_ceph_keyring,
535 CephBrokerRq,
536- CephBrokerRsp,
537 delete_keyring,
538 )
539 from charmhelpers.payload.execd import execd_preinstall
540@@ -55,6 +53,14 @@
541 os.mkdir('/etc/ceph')
542
543
544+def get_ceph_request():
545+ service = service_name()
546+ rq = CephBrokerRq()
547+ replicas = config('ceph-osd-replication-count')
548+ rq.add_op_create_pool(name=service, replica_count=replicas)
549+ return rq
550+
551+
552 @hooks.hook('ceph-relation-changed')
553 @restart_on_change(restart_map())
554 def ceph_changed():
555@@ -68,32 +74,17 @@
556 log('Could not create ceph keyring: peer not ready?')
557 return
558
559- settings = relation_get()
560- if settings and 'broker_rsp' in settings:
561- rsp = CephBrokerRsp(settings['broker_rsp'])
562- # Non-zero return code implies failure
563- if rsp.exit_code:
564- log("Ceph broker request failed (rc=%s, msg=%s)" %
565- (rsp.exit_code, rsp.exit_msg), level=ERROR)
566- return
567-
568- log("Ceph broker request succeeded (rc=%s, msg=%s)" %
569- (rsp.exit_code, rsp.exit_msg), level=INFO)
570+ if is_request_complete(get_ceph_request()):
571+ log('Request complete')
572 CONFIGS.write_all()
573 set_ceph_env_variables(service=service)
574 for rid in relation_ids('storage-backend'):
575 storage_backend(rid)
576-
577 # Ensure that cinder-volume is restarted since only now can we
578 # guarantee that ceph resources are ready.
579 service_restart('cinder-volume')
580 else:
581- rq = CephBrokerRq()
582- replicas = config('ceph-osd-replication-count')
583- rq.add_op_create_pool(name=service, replica_count=replicas)
584- for rid in relation_ids('ceph'):
585- relation_set(relation_id=rid, broker_req=rq.request)
586- log("Request(s) sent to Ceph broker (rid=%s)" % (rid))
587+ send_request_if_needed(get_ceph_request())
588
589
590 @hooks.hook('ceph-relation-broken')
591
592=== modified file 'tests/basic_deployment.py'
593--- tests/basic_deployment.py 2015-08-10 17:28:35 +0000
594+++ tests/basic_deployment.py 2015-09-23 07:31:24 +0000
595@@ -4,6 +4,7 @@
596 Basic cinder-ceph functional test.
597 """
598 import amulet
599+import json
600 import time
601
602 from charmhelpers.contrib.openstack.amulet.deployment import (
603@@ -279,40 +280,83 @@
604 amulet.raise_status(amulet.FAIL,
605 msg='cinder endpoint: {}'.format(ret))
606
607+ def validate_broker_req(self, unit, relation, expected):
608+ rel_data = json.loads(unit.relation(
609+ relation[0],
610+ relation[1])['broker_req'])
611+ if rel_data['api-version'] != expected['api-version']:
612+ return "Broker request api mismatch"
613+ for index in range(0, len(rel_data['ops'])):
614+ actual_op = rel_data['ops'][index]
615+ expected_op = expected['ops'][index]
616+ for key in ['op', 'name', 'replicas']:
617+ if actual_op[key] == expected_op[key]:
618+ u.log.debug("OK op {} key {}".format(index, key))
619+ else:
620+ return "Mismatch, op: {} key: {}".format(index, key)
621+ return None
622+
623+ def get_broker_request(self):
624+ client_unit = self.cinder_ceph_sentry
625+ broker_req = json.loads(client_unit.relation(
626+ 'ceph',
627+ 'ceph:client')['broker_req'])
628+ return broker_req
629+
630+ def get_broker_response(self):
631+ broker_request = self.get_broker_request()
632+ response_key = "broker-rsp-cinder-ceph-0"
633+ ceph_sentrys = [self.ceph0_sentry, self.ceph1_sentry, self.ceph2_sentry]
634+ for sentry in ceph_sentrys:
635+ relation_data = sentry.relation('client', 'cinder-ceph:ceph')
636+ if relation_data.get(response_key):
637+ broker_response = json.loads(relation_data[response_key])
638+ if broker_request['request-id'] == broker_response['request-id']:
639+ return broker_response
640+
641 def test_200_cinderceph_ceph_ceph_relation(self):
642 u.log.debug('Checking cinder-ceph:ceph to ceph:client '
643 'relation data...')
644 unit = self.cinder_ceph_sentry
645 relation = ['ceph', 'ceph:client']
646
647- req = ('{"api-version": 1, "ops": [{"replicas": 3, "name": '
648- '"cinder-ceph", "op": "create-pool"}]}')
649-
650+ req = {
651+ "api-version": 1,
652+ "ops": [{"replicas": 3, "name": "cinder-ceph", "op": "create-pool"}]
653+ }
654 expected = {
655 'private-address': u.valid_ip,
656- 'broker_req': req
657+ 'broker_req': u.not_null,
658 }
659 ret = u.validate_relation_data(unit, relation, expected)
660 if ret:
661 msg = u.relation_error('cinder-ceph ceph', ret)
662 amulet.raise_status(amulet.FAIL, msg=msg)
663+ ret = self.validate_broker_req(unit, relation, req)
664+ if ret:
665+ msg = u.relation_error('cinder-ceph ceph', ret)
666+ amulet.raise_status(amulet.FAIL, msg=msg)
667
668 def test_201_ceph_cinderceph_ceph_relation(self):
669 u.log.debug('Checking ceph:client to cinder-ceph:ceph '
670 'relation data...')
671- unit = self.ceph0_sentry
672+ response_key = "broker-rsp-cinder-ceph-0"
673+ ceph_unit = self.ceph0_sentry
674 relation = ['client', 'cinder-ceph:ceph']
675 expected = {
676 'key': u.not_null,
677 'private-address': u.valid_ip,
678- 'broker_rsp': '{"exit-code": 0}',
679 'ceph-public-address': u.valid_ip,
680- 'auth': 'none'
681+ 'auth': 'none',
682 }
683- ret = u.validate_relation_data(unit, relation, expected)
684+ ret = u.validate_relation_data(ceph_unit, relation, expected)
685 if ret:
686 msg = u.relation_error('cinder cinder-ceph storage-backend', ret)
687 amulet.raise_status(amulet.FAIL, msg=msg)
688+ broker_response = self.get_broker_response()
689+ if not broker_response or broker_response['exit-code'] != 0:
690+ msg='Broker request invalid or failed: {}'.format(broker_response)
691+ amulet.raise_status(amulet.FAIL, msg=msg)
692
693 def test_202_cinderceph_cinder_backend_relation(self):
694 u.log.debug('Checking cinder-ceph:storage-backend to '
695
696=== modified file 'unit_tests/test_cinder_hooks.py'
697--- unit_tests/test_cinder_hooks.py 2015-09-16 12:55:55 +0000
698+++ unit_tests/test_cinder_hooks.py 2015-09-23 07:31:24 +0000
699@@ -18,11 +18,12 @@
700 'register_configs',
701 'restart_map',
702 'set_ceph_env_variables',
703+ 'is_request_complete',
704+ 'send_request_if_needed',
705 'CONFIGS',
706 # charmhelpers.core.hookenv
707 'config',
708 'relation_ids',
709- 'relation_get',
710 'relation_set',
711 'service_name',
712 'service_restart',
713@@ -69,9 +70,8 @@
714 @patch('charmhelpers.core.hookenv.config')
715 def test_ceph_changed(self, mock_config):
716 '''It ensures ceph assets created on ceph changed'''
717+ self.is_request_complete.return_value = True
718 self.CONFIGS.complete_contexts.return_value = ['ceph']
719- rsp = json.dumps({'exit_code': 0})
720- self.relation_get.return_value = {'broker_rsp': rsp}
721 self.service_name.return_value = 'cinder'
722 self.ensure_ceph_keyring.return_value = True
723 hooks.hooks.execute(['hooks/ceph-relation-changed'])
724@@ -81,11 +81,27 @@
725 self.assertTrue(self.CONFIGS.write_all.called)
726 self.set_ceph_env_variables.assert_called_with(service='cinder')
727
728+ @patch.object(hooks, 'get_ceph_request')
729+ @patch('charmhelpers.core.hookenv.config')
730+ def test_ceph_changed_newrq(self, mock_config, mock_get_ceph_request):
731+ '''It ensures ceph assets created on ceph changed'''
732+ mock_get_ceph_request.return_value = 'cephreq'
733+ self.is_request_complete.return_value = False
734+ self.CONFIGS.complete_contexts.return_value = ['ceph']
735+ self.service_name.return_value = 'cinder'
736+ self.ensure_ceph_keyring.return_value = True
737+ hooks.hooks.execute(['hooks/ceph-relation-changed'])
738+ self.ensure_ceph_keyring.assert_called_with(service='cinder',
739+ user='cinder',
740+ group='cinder')
741+ self.send_request_if_needed.assert_called_with('cephreq')
742+
743 @patch('charmhelpers.core.hookenv.config')
744 def test_ceph_changed_no_keys(self, mock_config):
745 '''It ensures ceph assets created on ceph changed'''
746 self.CONFIGS.complete_contexts.return_value = ['ceph']
747 self.service_name.return_value = 'cinder'
748+ self.is_request_complete.return_value = True
749 self.ensure_ceph_keyring.return_value = False
750 hooks.hooks.execute(['hooks/ceph-relation-changed'])
751 # NOTE(jamespage): If ensure_ceph keyring fails, then

Subscribers

People subscribed via source and target branches