Merge lp:~gnuoy/charms/trusty/cinder-ceph/1453940-stable into lp:~openstack-charmers-archive/charms/precise/cinder-ceph/trunk

Proposed by Liam Young on 2015-09-16
Status: Merged
Merged at revision: 36
Proposed branch: lp:~gnuoy/charms/trusty/cinder-ceph/1453940-stable
Merge into: lp:~openstack-charmers-archive/charms/precise/cinder-ceph/trunk
Diff against target: 751 lines (+357/-82)
12 files modified
hooks/charmhelpers/cli/__init__.py (+1/-5)
hooks/charmhelpers/cli/commands.py (+4/-4)
hooks/charmhelpers/cli/hookenv.py (+23/-0)
hooks/charmhelpers/contrib/openstack/context.py (+8/-9)
hooks/charmhelpers/contrib/openstack/utils.py (+7/-5)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+224/-2)
hooks/charmhelpers/contrib/storage/linux/utils.py (+3/-2)
hooks/charmhelpers/core/hookenv.py (+1/-20)
hooks/charmhelpers/core/host.py (+2/-2)
hooks/cinder_hooks.py (+13/-22)
tests/basic_deployment.py (+52/-8)
unit_tests/test_cinder_hooks.py (+19/-3)
To merge this branch: bzr merge lp:~gnuoy/charms/trusty/cinder-ceph/1453940-stable
Reviewer Review Type Date Requested Status
Chris Glass (community) 2015-09-16 Approve on 2015-09-23
Ryan Beisner Needs Fixing on 2015-09-22
Review via email: mp+271261@code.launchpad.net
To post a comment you must log in.

charm_unit_test #9552 cinder-ceph for gnuoy mp271261
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9552/

charm_lint_check #10411 cinder-ceph for gnuoy mp271261
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10411/

charm_amulet_test #6551 cinder-ceph for gnuoy mp271261
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12514387/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6551/

charm_lint_check #10436 cinder-ceph for gnuoy mp271261
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10436/

charm_unit_test #9629 cinder-ceph for gnuoy mp271261
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9629/

charm_amulet_test #6578 cinder-ceph for gnuoy mp271261
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12521251/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6578/

Ryan Beisner (1chb1n) wrote :

To save you the time of grepping logs, that amulet fail is real.

00:17:33.288 2015-09-22 13:59:57,026 test_201_ceph_cinderceph_ceph_relation DEBUG: Checking ceph:client to cinder-ceph:ceph relation data...

...

00:17:33.289 Broker request invalid or failed: None

review: Needs Fixing
Liam Young (gnuoy) wrote :

OSCI failure caused by ceph having not landed yet.

38. By Liam Young on 2015-09-22

Empty commit to trigger osci

charm_unit_test #9652 cinder-ceph for gnuoy mp271261
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9652/

charm_amulet_test #6585 cinder-ceph for gnuoy mp271261
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12521979/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6585/

39. By Liam Young on 2015-09-22

Empty commit to trigger osci

charm_unit_test #9654 cinder-ceph for gnuoy mp271261
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9654/

charm_lint_check #10463 cinder-ceph for gnuoy mp271261
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10463/

charm_amulet_test #6606 cinder-ceph for gnuoy mp271261
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12526301/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6606/

40. By Liam Young on 2015-09-23

Empty commit to kick osci

charm_unit_test #9712 cinder-ceph for gnuoy mp271261
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/9712/

charm_lint_check #10524 cinder-ceph for gnuoy mp271261
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/10524/

Liam Young (gnuoy) wrote :

Since the latest revision of ceph is still not in the charm store I tests an amulet run manually with the latest ceph from stable. It LGTM http://paste.ubuntu.com/12529898/

Chris Glass (tribaal) wrote :

After further testing on a bastion, this should pass. In order to cut down on wait time I'm going to go ahead and merge this already.

+1

review: Approve

charm_amulet_test #6666 cinder-ceph for gnuoy mp271261
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6666/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'hooks/charmhelpers/cli/__init__.py'
--- hooks/charmhelpers/cli/__init__.py 2015-08-10 16:34:26 +0000
+++ hooks/charmhelpers/cli/__init__.py 2015-09-23 07:31:24 +0000
@@ -152,15 +152,11 @@
152 arguments = self.argument_parser.parse_args()152 arguments = self.argument_parser.parse_args()
153 argspec = inspect.getargspec(arguments.func)153 argspec = inspect.getargspec(arguments.func)
154 vargs = []154 vargs = []
155 kwargs = {}
156 for arg in argspec.args:155 for arg in argspec.args:
157 vargs.append(getattr(arguments, arg))156 vargs.append(getattr(arguments, arg))
158 if argspec.varargs:157 if argspec.varargs:
159 vargs.extend(getattr(arguments, argspec.varargs))158 vargs.extend(getattr(arguments, argspec.varargs))
160 if argspec.keywords:159 output = arguments.func(*vargs)
161 for kwarg in argspec.keywords.items():
162 kwargs[kwarg] = getattr(arguments, kwarg)
163 output = arguments.func(*vargs, **kwargs)
164 if getattr(arguments.func, '_cli_test_command', False):160 if getattr(arguments.func, '_cli_test_command', False):
165 self.exit_code = 0 if output else 1161 self.exit_code = 0 if output else 1
166 output = ''162 output = ''
167163
=== modified file 'hooks/charmhelpers/cli/commands.py'
--- hooks/charmhelpers/cli/commands.py 2015-08-10 16:34:26 +0000
+++ hooks/charmhelpers/cli/commands.py 2015-09-23 07:31:24 +0000
@@ -26,7 +26,7 @@
26"""26"""
27Import the sub-modules which have decorated subcommands to register with chlp.27Import the sub-modules which have decorated subcommands to register with chlp.
28"""28"""
29import host # noqa29from . import host # noqa
30import benchmark # noqa30from . import benchmark # noqa
31import unitdata # noqa31from . import unitdata # noqa
32from charmhelpers.core import hookenv # noqa32from . import hookenv # noqa
3333
=== added file 'hooks/charmhelpers/cli/hookenv.py'
--- hooks/charmhelpers/cli/hookenv.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/cli/hookenv.py 2015-09-23 07:31:24 +0000
@@ -0,0 +1,23 @@
1# Copyright 2014-2015 Canonical Limited.
2#
3# This file is part of charm-helpers.
4#
5# charm-helpers is free software: you can redistribute it and/or modify
6# it under the terms of the GNU Lesser General Public License version 3 as
7# published by the Free Software Foundation.
8#
9# charm-helpers is distributed in the hope that it will be useful,
10# but WITHOUT ANY WARRANTY; without even the implied warranty of
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12# GNU Lesser General Public License for more details.
13#
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
16
17from . import cmdline
18from charmhelpers.core import hookenv
19
20
21cmdline.subcommand('relation-id')(hookenv.relation_id._wrapped)
22cmdline.subcommand('service-name')(hookenv.service_name)
23cmdline.subcommand('remote-service-name')(hookenv.remote_service_name._wrapped)
024
=== modified file 'hooks/charmhelpers/contrib/openstack/context.py'
--- hooks/charmhelpers/contrib/openstack/context.py 2015-08-10 16:34:26 +0000
+++ hooks/charmhelpers/contrib/openstack/context.py 2015-09-23 07:31:24 +0000
@@ -483,13 +483,15 @@
483483
484 log('Generating template context for ceph', level=DEBUG)484 log('Generating template context for ceph', level=DEBUG)
485 mon_hosts = []485 mon_hosts = []
486 auth = None486 ctxt = {
487 key = None487 'use_syslog': str(config('use-syslog')).lower()
488 use_syslog = str(config('use-syslog')).lower()488 }
489 for rid in relation_ids('ceph'):489 for rid in relation_ids('ceph'):
490 for unit in related_units(rid):490 for unit in related_units(rid):
491 auth = relation_get('auth', rid=rid, unit=unit)491 if not ctxt.get('auth'):
492 key = relation_get('key', rid=rid, unit=unit)492 ctxt['auth'] = relation_get('auth', rid=rid, unit=unit)
493 if not ctxt.get('key'):
494 ctxt['key'] = relation_get('key', rid=rid, unit=unit)
493 ceph_pub_addr = relation_get('ceph-public-address', rid=rid,495 ceph_pub_addr = relation_get('ceph-public-address', rid=rid,
494 unit=unit)496 unit=unit)
495 unit_priv_addr = relation_get('private-address', rid=rid,497 unit_priv_addr = relation_get('private-address', rid=rid,
@@ -498,10 +500,7 @@
498 ceph_addr = format_ipv6_addr(ceph_addr) or ceph_addr500 ceph_addr = format_ipv6_addr(ceph_addr) or ceph_addr
499 mon_hosts.append(ceph_addr)501 mon_hosts.append(ceph_addr)
500502
501 ctxt = {'mon_hosts': ' '.join(sorted(mon_hosts)),503 ctxt['mon_hosts'] = ' '.join(sorted(mon_hosts))
502 'auth': auth,
503 'key': key,
504 'use_syslog': use_syslog}
505504
506 if not os.path.isdir('/etc/ceph'):505 if not os.path.isdir('/etc/ceph'):
507 os.mkdir('/etc/ceph')506 os.mkdir('/etc/ceph')
508507
=== modified file 'hooks/charmhelpers/contrib/openstack/utils.py'
--- hooks/charmhelpers/contrib/openstack/utils.py 2015-08-10 16:34:26 +0000
+++ hooks/charmhelpers/contrib/openstack/utils.py 2015-09-23 07:31:24 +0000
@@ -1,5 +1,3 @@
1#!/usr/bin/python
2
3# Copyright 2014-2015 Canonical Limited.1# Copyright 2014-2015 Canonical Limited.
4#2#
5# This file is part of charm-helpers.3# This file is part of charm-helpers.
@@ -167,9 +165,9 @@
167 error_out(e)165 error_out(e)
168166
169167
170def get_os_version_codename(codename):168def get_os_version_codename(codename, version_map=OPENSTACK_CODENAMES):
171 '''Determine OpenStack version number from codename.'''169 '''Determine OpenStack version number from codename.'''
172 for k, v in six.iteritems(OPENSTACK_CODENAMES):170 for k, v in six.iteritems(version_map):
173 if v == codename:171 if v == codename:
174 return k172 return k
175 e = 'Could not derive OpenStack version for '\173 e = 'Could not derive OpenStack version for '\
@@ -392,7 +390,11 @@
392 import apt_pkg as apt390 import apt_pkg as apt
393 src = config('openstack-origin')391 src = config('openstack-origin')
394 cur_vers = get_os_version_package(package)392 cur_vers = get_os_version_package(package)
395 available_vers = get_os_version_install_source(src)393 if "swift" in package:
394 codename = get_os_codename_install_source(src)
395 available_vers = get_os_version_codename(codename, SWIFT_CODENAMES)
396 else:
397 available_vers = get_os_version_install_source(src)
396 apt.init()398 apt.init()
397 return apt.version_compare(available_vers, cur_vers) == 1399 return apt.version_compare(available_vers, cur_vers) == 1
398400
399401
=== modified file 'hooks/charmhelpers/contrib/storage/linux/ceph.py'
--- hooks/charmhelpers/contrib/storage/linux/ceph.py 2015-08-10 16:34:26 +0000
+++ hooks/charmhelpers/contrib/storage/linux/ceph.py 2015-09-23 07:31:24 +0000
@@ -28,6 +28,7 @@
28import shutil28import shutil
29import json29import json
30import time30import time
31import uuid
3132
32from subprocess import (33from subprocess import (
33 check_call,34 check_call,
@@ -35,8 +36,10 @@
35 CalledProcessError,36 CalledProcessError,
36)37)
37from charmhelpers.core.hookenv import (38from charmhelpers.core.hookenv import (
39 local_unit,
38 relation_get,40 relation_get,
39 relation_ids,41 relation_ids,
42 relation_set,
40 related_units,43 related_units,
41 log,44 log,
42 DEBUG,45 DEBUG,
@@ -411,17 +414,52 @@
411414
412 The API is versioned and defaults to version 1.415 The API is versioned and defaults to version 1.
413 """416 """
414 def __init__(self, api_version=1):417 def __init__(self, api_version=1, request_id=None):
415 self.api_version = api_version418 self.api_version = api_version
419 if request_id:
420 self.request_id = request_id
421 else:
422 self.request_id = str(uuid.uuid1())
416 self.ops = []423 self.ops = []
417424
418 def add_op_create_pool(self, name, replica_count=3):425 def add_op_create_pool(self, name, replica_count=3):
419 self.ops.append({'op': 'create-pool', 'name': name,426 self.ops.append({'op': 'create-pool', 'name': name,
420 'replicas': replica_count})427 'replicas': replica_count})
421428
429 def set_ops(self, ops):
430 """Set request ops to provided value.
431
432 Useful for injecting ops that come from a previous request
433 to allow comparisons to ensure validity.
434 """
435 self.ops = ops
436
422 @property437 @property
423 def request(self):438 def request(self):
424 return json.dumps({'api-version': self.api_version, 'ops': self.ops})439 return json.dumps({'api-version': self.api_version, 'ops': self.ops,
440 'request-id': self.request_id})
441
442 def _ops_equal(self, other):
443 if len(self.ops) == len(other.ops):
444 for req_no in range(0, len(self.ops)):
445 for key in ['replicas', 'name', 'op']:
446 if self.ops[req_no][key] != other.ops[req_no][key]:
447 return False
448 else:
449 return False
450 return True
451
452 def __eq__(self, other):
453 if not isinstance(other, self.__class__):
454 return False
455 if self.api_version == other.api_version and \
456 self._ops_equal(other):
457 return True
458 else:
459 return False
460
461 def __ne__(self, other):
462 return not self.__eq__(other)
425463
426464
427class CephBrokerRsp(object):465class CephBrokerRsp(object):
@@ -431,14 +469,198 @@
431469
432 The API is versioned and defaults to version 1.470 The API is versioned and defaults to version 1.
433 """471 """
472
434 def __init__(self, encoded_rsp):473 def __init__(self, encoded_rsp):
435 self.api_version = None474 self.api_version = None
436 self.rsp = json.loads(encoded_rsp)475 self.rsp = json.loads(encoded_rsp)
437476
438 @property477 @property
478 def request_id(self):
479 return self.rsp.get('request-id')
480
481 @property
439 def exit_code(self):482 def exit_code(self):
440 return self.rsp.get('exit-code')483 return self.rsp.get('exit-code')
441484
442 @property485 @property
443 def exit_msg(self):486 def exit_msg(self):
444 return self.rsp.get('stderr')487 return self.rsp.get('stderr')
488
489
490# Ceph Broker Conversation:
491# If a charm needs an action to be taken by ceph it can create a CephBrokerRq
492# and send that request to ceph via the ceph relation. The CephBrokerRq has a
493# unique id so that the client can identity which CephBrokerRsp is associated
494# with the request. Ceph will also respond to each client unit individually
495# creating a response key per client unit eg glance/0 will get a CephBrokerRsp
496# via key broker-rsp-glance-0
497#
498# To use this the charm can just do something like:
499#
500# from charmhelpers.contrib.storage.linux.ceph import (
501# send_request_if_needed,
502# is_request_complete,
503# CephBrokerRq,
504# )
505#
506# @hooks.hook('ceph-relation-changed')
507# def ceph_changed():
508# rq = CephBrokerRq()
509# rq.add_op_create_pool(name='poolname', replica_count=3)
510#
511# if is_request_complete(rq):
512# <Request complete actions>
513# else:
514# send_request_if_needed(get_ceph_request())
515#
516# CephBrokerRq and CephBrokerRsp are serialized into JSON. Below is an example
517# of glance having sent a request to ceph which ceph has successfully processed
518# 'ceph:8': {
519# 'ceph/0': {
520# 'auth': 'cephx',
521# 'broker-rsp-glance-0': '{"request-id": "0bc7dc54", "exit-code": 0}',
522# 'broker_rsp': '{"request-id": "0da543b8", "exit-code": 0}',
523# 'ceph-public-address': '10.5.44.103',
524# 'key': 'AQCLDttVuHXINhAAvI144CB09dYchhHyTUY9BQ==',
525# 'private-address': '10.5.44.103',
526# },
527# 'glance/0': {
528# 'broker_req': ('{"api-version": 1, "request-id": "0bc7dc54", '
529# '"ops": [{"replicas": 3, "name": "glance", '
530# '"op": "create-pool"}]}'),
531# 'private-address': '10.5.44.109',
532# },
533# }
534
535def get_previous_request(rid):
536 """Return the last ceph broker request sent on a given relation
537
538 @param rid: Relation id to query for request
539 """
540 request = None
541 broker_req = relation_get(attribute='broker_req', rid=rid,
542 unit=local_unit())
543 if broker_req:
544 request_data = json.loads(broker_req)
545 request = CephBrokerRq(api_version=request_data['api-version'],
546 request_id=request_data['request-id'])
547 request.set_ops(request_data['ops'])
548
549 return request
550
551
552def get_request_states(request):
553 """Return a dict of requests per relation id with their corresponding
554 completion state.
555
556 This allows a charm, which has a request for ceph, to see whether there is
557 an equivalent request already being processed and if so what state that
558 request is in.
559
560 @param request: A CephBrokerRq object
561 """
562 complete = []
563 requests = {}
564 for rid in relation_ids('ceph'):
565 complete = False
566 previous_request = get_previous_request(rid)
567 if request == previous_request:
568 sent = True
569 complete = is_request_complete_for_rid(previous_request, rid)
570 else:
571 sent = False
572 complete = False
573
574 requests[rid] = {
575 'sent': sent,
576 'complete': complete,
577 }
578
579 return requests
580
581
582def is_request_sent(request):
583 """Check to see if a functionally equivalent request has already been sent
584
585 Returns True if a similair request has been sent
586
587 @param request: A CephBrokerRq object
588 """
589 states = get_request_states(request)
590 for rid in states.keys():
591 if not states[rid]['sent']:
592 return False
593
594 return True
595
596
597def is_request_complete(request):
598 """Check to see if a functionally equivalent request has already been
599 completed
600
601 Returns True if a similair request has been completed
602
603 @param request: A CephBrokerRq object
604 """
605 states = get_request_states(request)
606 for rid in states.keys():
607 if not states[rid]['complete']:
608 return False
609
610 return True
611
612
613def is_request_complete_for_rid(request, rid):
614 """Check if a given request has been completed on the given relation
615
616 @param request: A CephBrokerRq object
617 @param rid: Relation ID
618 """
619 broker_key = get_broker_rsp_key()
620 for unit in related_units(rid):
621 rdata = relation_get(rid=rid, unit=unit)
622 if rdata.get(broker_key):
623 rsp = CephBrokerRsp(rdata.get(broker_key))
624 if rsp.request_id == request.request_id:
625 if not rsp.exit_code:
626 return True
627 else:
628 # The remote unit sent no reply targeted at this unit so either the
629 # remote ceph cluster does not support unit targeted replies or it
630 # has not processed our request yet.
631 if rdata.get('broker_rsp'):
632 request_data = json.loads(rdata['broker_rsp'])
633 if request_data.get('request-id'):
634 log('Ignoring legacy broker_rsp without unit key as remote '
635 'service supports unit specific replies', level=DEBUG)
636 else:
637 log('Using legacy broker_rsp as remote service does not '
638 'supports unit specific replies', level=DEBUG)
639 rsp = CephBrokerRsp(rdata['broker_rsp'])
640 if not rsp.exit_code:
641 return True
642
643 return False
644
645
646def get_broker_rsp_key():
647 """Return broker response key for this unit
648
649 This is the key that ceph is going to use to pass request status
650 information back to this unit
651 """
652 return 'broker-rsp-' + local_unit().replace('/', '-')
653
654
655def send_request_if_needed(request):
656 """Send broker request if an equivalent request has not already been sent
657
658 @param request: A CephBrokerRq object
659 """
660 if is_request_sent(request):
661 log('Request already sent but not complete, not sending new request',
662 level=DEBUG)
663 else:
664 for rid in relation_ids('ceph'):
665 log('Sending request {}'.format(request.request_id), level=DEBUG)
666 relation_set(relation_id=rid, broker_req=request.request)
445667
=== modified file 'hooks/charmhelpers/contrib/storage/linux/utils.py'
--- hooks/charmhelpers/contrib/storage/linux/utils.py 2015-08-10 16:34:26 +0000
+++ hooks/charmhelpers/contrib/storage/linux/utils.py 2015-09-23 07:31:24 +0000
@@ -43,9 +43,10 @@
4343
44 :param block_device: str: Full path of block device to clean.44 :param block_device: str: Full path of block device to clean.
45 '''45 '''
46 # https://github.com/ceph/ceph/commit/fdd7f8d83afa25c4e09aaedd90ab93f3b64a677b
46 # sometimes sgdisk exits non-zero; this is OK, dd will clean up47 # sometimes sgdisk exits non-zero; this is OK, dd will clean up
47 call(['sgdisk', '--zap-all', '--mbrtogpt',48 call(['sgdisk', '--zap-all', '--', block_device])
48 '--clear', block_device])49 call(['sgdisk', '--clear', '--mbrtogpt', '--', block_device])
49 dev_end = check_output(['blockdev', '--getsz',50 dev_end = check_output(['blockdev', '--getsz',
50 block_device]).decode('UTF-8')51 block_device]).decode('UTF-8')
51 gpt_end = int(dev_end.split()[0]) - 10052 gpt_end = int(dev_end.split()[0]) - 100
5253
=== modified file 'hooks/charmhelpers/core/hookenv.py'
--- hooks/charmhelpers/core/hookenv.py 2015-08-10 16:34:26 +0000
+++ hooks/charmhelpers/core/hookenv.py 2015-09-23 07:31:24 +0000
@@ -34,23 +34,6 @@
34import tempfile34import tempfile
35from subprocess import CalledProcessError35from subprocess import CalledProcessError
3636
37try:
38 from charmhelpers.cli import cmdline
39except ImportError as e:
40 # due to the anti-pattern of partially synching charmhelpers directly
41 # into charms, it's possible that charmhelpers.cli is not available;
42 # if that's the case, they don't really care about using the cli anyway,
43 # so mock it out
44 if str(e) == 'No module named cli':
45 class cmdline(object):
46 @classmethod
47 def subcommand(cls, *args, **kwargs):
48 def _wrap(func):
49 return func
50 return _wrap
51 else:
52 raise
53
54import six37import six
55if not six.PY3:38if not six.PY3:
56 from UserDict import UserDict39 from UserDict import UserDict
@@ -91,6 +74,7 @@
91 res = func(*args, **kwargs)74 res = func(*args, **kwargs)
92 cache[key] = res75 cache[key] = res
93 return res76 return res
77 wrapper._wrapped = func
94 return wrapper78 return wrapper
9579
9680
@@ -190,7 +174,6 @@
190 return os.environ.get('JUJU_RELATION', None)174 return os.environ.get('JUJU_RELATION', None)
191175
192176
193@cmdline.subcommand()
194@cached177@cached
195def relation_id(relation_name=None, service_or_unit=None):178def relation_id(relation_name=None, service_or_unit=None):
196 """The relation ID for the current or a specified relation"""179 """The relation ID for the current or a specified relation"""
@@ -216,13 +199,11 @@
216 return os.environ.get('JUJU_REMOTE_UNIT', None)199 return os.environ.get('JUJU_REMOTE_UNIT', None)
217200
218201
219@cmdline.subcommand()
220def service_name():202def service_name():
221 """The name service group this unit belongs to"""203 """The name service group this unit belongs to"""
222 return local_unit().split('/')[0]204 return local_unit().split('/')[0]
223205
224206
225@cmdline.subcommand()
226@cached207@cached
227def remote_service_name(relid=None):208def remote_service_name(relid=None):
228 """The remote service name for a given relation-id (or the current relation)"""209 """The remote service name for a given relation-id (or the current relation)"""
229210
=== modified file 'hooks/charmhelpers/core/host.py'
--- hooks/charmhelpers/core/host.py 2015-08-10 16:34:26 +0000
+++ hooks/charmhelpers/core/host.py 2015-09-23 07:31:24 +0000
@@ -72,7 +72,7 @@
72 stopped = service_stop(service_name)72 stopped = service_stop(service_name)
73 # XXX: Support systemd too73 # XXX: Support systemd too
74 override_path = os.path.join(74 override_path = os.path.join(
75 init_dir, '{}.conf.override'.format(service_name))75 init_dir, '{}.override'.format(service_name))
76 with open(override_path, 'w') as fh:76 with open(override_path, 'w') as fh:
77 fh.write("manual\n")77 fh.write("manual\n")
78 return stopped78 return stopped
@@ -86,7 +86,7 @@
86 if init_dir is None:86 if init_dir is None:
87 init_dir = "/etc/init"87 init_dir = "/etc/init"
88 override_path = os.path.join(88 override_path = os.path.join(
89 init_dir, '{}.conf.override'.format(service_name))89 init_dir, '{}.override'.format(service_name))
90 if os.path.exists(override_path):90 if os.path.exists(override_path):
91 os.unlink(override_path)91 os.unlink(override_path)
92 started = service_start(service_name)92 started = service_start(service_name)
9393
=== modified file 'hooks/cinder_hooks.py'
--- hooks/cinder_hooks.py 2015-09-16 12:55:55 +0000
+++ hooks/cinder_hooks.py 2015-09-23 07:31:24 +0000
@@ -17,12 +17,9 @@
17 UnregisteredHookError,17 UnregisteredHookError,
18 config,18 config,
19 service_name,19 service_name,
20 relation_get,
21 relation_set,20 relation_set,
22 relation_ids,21 relation_ids,
23 log,22 log,
24 INFO,
25 ERROR,
26)23)
27from charmhelpers.fetch import apt_install, apt_update24from charmhelpers.fetch import apt_install, apt_update
28from charmhelpers.core.host import (25from charmhelpers.core.host import (
@@ -30,9 +27,10 @@
30 service_restart,27 service_restart,
31)28)
32from charmhelpers.contrib.storage.linux.ceph import (29from charmhelpers.contrib.storage.linux.ceph import (
30 send_request_if_needed,
31 is_request_complete,
33 ensure_ceph_keyring,32 ensure_ceph_keyring,
34 CephBrokerRq,33 CephBrokerRq,
35 CephBrokerRsp,
36 delete_keyring,34 delete_keyring,
37)35)
38from charmhelpers.payload.execd import execd_preinstall36from charmhelpers.payload.execd import execd_preinstall
@@ -55,6 +53,14 @@
55 os.mkdir('/etc/ceph')53 os.mkdir('/etc/ceph')
5654
5755
56def get_ceph_request():
57 service = service_name()
58 rq = CephBrokerRq()
59 replicas = config('ceph-osd-replication-count')
60 rq.add_op_create_pool(name=service, replica_count=replicas)
61 return rq
62
63
58@hooks.hook('ceph-relation-changed')64@hooks.hook('ceph-relation-changed')
59@restart_on_change(restart_map())65@restart_on_change(restart_map())
60def ceph_changed():66def ceph_changed():
@@ -68,32 +74,17 @@
68 log('Could not create ceph keyring: peer not ready?')74 log('Could not create ceph keyring: peer not ready?')
69 return75 return
7076
71 settings = relation_get()77 if is_request_complete(get_ceph_request()):
72 if settings and 'broker_rsp' in settings:78 log('Request complete')
73 rsp = CephBrokerRsp(settings['broker_rsp'])
74 # Non-zero return code implies failure
75 if rsp.exit_code:
76 log("Ceph broker request failed (rc=%s, msg=%s)" %
77 (rsp.exit_code, rsp.exit_msg), level=ERROR)
78 return
79
80 log("Ceph broker request succeeded (rc=%s, msg=%s)" %
81 (rsp.exit_code, rsp.exit_msg), level=INFO)
82 CONFIGS.write_all()79 CONFIGS.write_all()
83 set_ceph_env_variables(service=service)80 set_ceph_env_variables(service=service)
84 for rid in relation_ids('storage-backend'):81 for rid in relation_ids('storage-backend'):
85 storage_backend(rid)82 storage_backend(rid)
86
87 # Ensure that cinder-volume is restarted since only now can we83 # Ensure that cinder-volume is restarted since only now can we
88 # guarantee that ceph resources are ready.84 # guarantee that ceph resources are ready.
89 service_restart('cinder-volume')85 service_restart('cinder-volume')
90 else:86 else:
91 rq = CephBrokerRq()87 send_request_if_needed(get_ceph_request())
92 replicas = config('ceph-osd-replication-count')
93 rq.add_op_create_pool(name=service, replica_count=replicas)
94 for rid in relation_ids('ceph'):
95 relation_set(relation_id=rid, broker_req=rq.request)
96 log("Request(s) sent to Ceph broker (rid=%s)" % (rid))
9788
9889
99@hooks.hook('ceph-relation-broken')90@hooks.hook('ceph-relation-broken')
10091
=== modified file 'tests/basic_deployment.py'
--- tests/basic_deployment.py 2015-08-10 17:28:35 +0000
+++ tests/basic_deployment.py 2015-09-23 07:31:24 +0000
@@ -4,6 +4,7 @@
4Basic cinder-ceph functional test.4Basic cinder-ceph functional test.
5"""5"""
6import amulet6import amulet
7import json
7import time8import time
89
9from charmhelpers.contrib.openstack.amulet.deployment import (10from charmhelpers.contrib.openstack.amulet.deployment import (
@@ -279,40 +280,83 @@
279 amulet.raise_status(amulet.FAIL,280 amulet.raise_status(amulet.FAIL,
280 msg='cinder endpoint: {}'.format(ret))281 msg='cinder endpoint: {}'.format(ret))
281282
283 def validate_broker_req(self, unit, relation, expected):
284 rel_data = json.loads(unit.relation(
285 relation[0],
286 relation[1])['broker_req'])
287 if rel_data['api-version'] != expected['api-version']:
288 return "Broker request api mismatch"
289 for index in range(0, len(rel_data['ops'])):
290 actual_op = rel_data['ops'][index]
291 expected_op = expected['ops'][index]
292 for key in ['op', 'name', 'replicas']:
293 if actual_op[key] == expected_op[key]:
294 u.log.debug("OK op {} key {}".format(index, key))
295 else:
296 return "Mismatch, op: {} key: {}".format(index, key)
297 return None
298
299 def get_broker_request(self):
300 client_unit = self.cinder_ceph_sentry
301 broker_req = json.loads(client_unit.relation(
302 'ceph',
303 'ceph:client')['broker_req'])
304 return broker_req
305
306 def get_broker_response(self):
307 broker_request = self.get_broker_request()
308 response_key = "broker-rsp-cinder-ceph-0"
309 ceph_sentrys = [self.ceph0_sentry, self.ceph1_sentry, self.ceph2_sentry]
310 for sentry in ceph_sentrys:
311 relation_data = sentry.relation('client', 'cinder-ceph:ceph')
312 if relation_data.get(response_key):
313 broker_response = json.loads(relation_data[response_key])
314 if broker_request['request-id'] == broker_response['request-id']:
315 return broker_response
316
282 def test_200_cinderceph_ceph_ceph_relation(self):317 def test_200_cinderceph_ceph_ceph_relation(self):
283 u.log.debug('Checking cinder-ceph:ceph to ceph:client '318 u.log.debug('Checking cinder-ceph:ceph to ceph:client '
284 'relation data...')319 'relation data...')
285 unit = self.cinder_ceph_sentry320 unit = self.cinder_ceph_sentry
286 relation = ['ceph', 'ceph:client']321 relation = ['ceph', 'ceph:client']
287322
288 req = ('{"api-version": 1, "ops": [{"replicas": 3, "name": '323 req = {
289 '"cinder-ceph", "op": "create-pool"}]}')324 "api-version": 1,
290325 "ops": [{"replicas": 3, "name": "cinder-ceph", "op": "create-pool"}]
326 }
291 expected = {327 expected = {
292 'private-address': u.valid_ip,328 'private-address': u.valid_ip,
293 'broker_req': req329 'broker_req': u.not_null,
294 }330 }
295 ret = u.validate_relation_data(unit, relation, expected)331 ret = u.validate_relation_data(unit, relation, expected)
296 if ret:332 if ret:
297 msg = u.relation_error('cinder-ceph ceph', ret)333 msg = u.relation_error('cinder-ceph ceph', ret)
298 amulet.raise_status(amulet.FAIL, msg=msg)334 amulet.raise_status(amulet.FAIL, msg=msg)
335 ret = self.validate_broker_req(unit, relation, req)
336 if ret:
337 msg = u.relation_error('cinder-ceph ceph', ret)
338 amulet.raise_status(amulet.FAIL, msg=msg)
299339
300 def test_201_ceph_cinderceph_ceph_relation(self):340 def test_201_ceph_cinderceph_ceph_relation(self):
301 u.log.debug('Checking ceph:client to cinder-ceph:ceph '341 u.log.debug('Checking ceph:client to cinder-ceph:ceph '
302 'relation data...')342 'relation data...')
303 unit = self.ceph0_sentry343 response_key = "broker-rsp-cinder-ceph-0"
344 ceph_unit = self.ceph0_sentry
304 relation = ['client', 'cinder-ceph:ceph']345 relation = ['client', 'cinder-ceph:ceph']
305 expected = {346 expected = {
306 'key': u.not_null,347 'key': u.not_null,
307 'private-address': u.valid_ip,348 'private-address': u.valid_ip,
308 'broker_rsp': '{"exit-code": 0}',
309 'ceph-public-address': u.valid_ip,349 'ceph-public-address': u.valid_ip,
310 'auth': 'none'350 'auth': 'none',
311 }351 }
312 ret = u.validate_relation_data(unit, relation, expected)352 ret = u.validate_relation_data(ceph_unit, relation, expected)
313 if ret:353 if ret:
314 msg = u.relation_error('cinder cinder-ceph storage-backend', ret)354 msg = u.relation_error('cinder cinder-ceph storage-backend', ret)
315 amulet.raise_status(amulet.FAIL, msg=msg)355 amulet.raise_status(amulet.FAIL, msg=msg)
356 broker_response = self.get_broker_response()
357 if not broker_response or broker_response['exit-code'] != 0:
358 msg='Broker request invalid or failed: {}'.format(broker_response)
359 amulet.raise_status(amulet.FAIL, msg=msg)
316360
317 def test_202_cinderceph_cinder_backend_relation(self):361 def test_202_cinderceph_cinder_backend_relation(self):
318 u.log.debug('Checking cinder-ceph:storage-backend to '362 u.log.debug('Checking cinder-ceph:storage-backend to '
319363
=== modified file 'unit_tests/test_cinder_hooks.py'
--- unit_tests/test_cinder_hooks.py 2015-09-16 12:55:55 +0000
+++ unit_tests/test_cinder_hooks.py 2015-09-23 07:31:24 +0000
@@ -18,11 +18,12 @@
18 'register_configs',18 'register_configs',
19 'restart_map',19 'restart_map',
20 'set_ceph_env_variables',20 'set_ceph_env_variables',
21 'is_request_complete',
22 'send_request_if_needed',
21 'CONFIGS',23 'CONFIGS',
22 # charmhelpers.core.hookenv24 # charmhelpers.core.hookenv
23 'config',25 'config',
24 'relation_ids',26 'relation_ids',
25 'relation_get',
26 'relation_set',27 'relation_set',
27 'service_name',28 'service_name',
28 'service_restart',29 'service_restart',
@@ -69,9 +70,8 @@
69 @patch('charmhelpers.core.hookenv.config')70 @patch('charmhelpers.core.hookenv.config')
70 def test_ceph_changed(self, mock_config):71 def test_ceph_changed(self, mock_config):
71 '''It ensures ceph assets created on ceph changed'''72 '''It ensures ceph assets created on ceph changed'''
73 self.is_request_complete.return_value = True
72 self.CONFIGS.complete_contexts.return_value = ['ceph']74 self.CONFIGS.complete_contexts.return_value = ['ceph']
73 rsp = json.dumps({'exit_code': 0})
74 self.relation_get.return_value = {'broker_rsp': rsp}
75 self.service_name.return_value = 'cinder'75 self.service_name.return_value = 'cinder'
76 self.ensure_ceph_keyring.return_value = True76 self.ensure_ceph_keyring.return_value = True
77 hooks.hooks.execute(['hooks/ceph-relation-changed'])77 hooks.hooks.execute(['hooks/ceph-relation-changed'])
@@ -81,11 +81,27 @@
81 self.assertTrue(self.CONFIGS.write_all.called)81 self.assertTrue(self.CONFIGS.write_all.called)
82 self.set_ceph_env_variables.assert_called_with(service='cinder')82 self.set_ceph_env_variables.assert_called_with(service='cinder')
8383
84 @patch.object(hooks, 'get_ceph_request')
85 @patch('charmhelpers.core.hookenv.config')
86 def test_ceph_changed_newrq(self, mock_config, mock_get_ceph_request):
87 '''It ensures ceph assets created on ceph changed'''
88 mock_get_ceph_request.return_value = 'cephreq'
89 self.is_request_complete.return_value = False
90 self.CONFIGS.complete_contexts.return_value = ['ceph']
91 self.service_name.return_value = 'cinder'
92 self.ensure_ceph_keyring.return_value = True
93 hooks.hooks.execute(['hooks/ceph-relation-changed'])
94 self.ensure_ceph_keyring.assert_called_with(service='cinder',
95 user='cinder',
96 group='cinder')
97 self.send_request_if_needed.assert_called_with('cephreq')
98
84 @patch('charmhelpers.core.hookenv.config')99 @patch('charmhelpers.core.hookenv.config')
85 def test_ceph_changed_no_keys(self, mock_config):100 def test_ceph_changed_no_keys(self, mock_config):
86 '''It ensures ceph assets created on ceph changed'''101 '''It ensures ceph assets created on ceph changed'''
87 self.CONFIGS.complete_contexts.return_value = ['ceph']102 self.CONFIGS.complete_contexts.return_value = ['ceph']
88 self.service_name.return_value = 'cinder'103 self.service_name.return_value = 'cinder'
104 self.is_request_complete.return_value = True
89 self.ensure_ceph_keyring.return_value = False105 self.ensure_ceph_keyring.return_value = False
90 hooks.hooks.execute(['hooks/ceph-relation-changed'])106 hooks.hooks.execute(['hooks/ceph-relation-changed'])
91 # NOTE(jamespage): If ensure_ceph keyring fails, then107 # NOTE(jamespage): If ensure_ceph keyring fails, then

Subscribers

People subscribed via source and target branches