Merge lp:~brad-marshall/charms/trusty/rabbitmq-server/nagios-fixes-sync-charmhelpers into lp:charms/trusty/rabbitmq-server

Proposed by Brad Marshall
Status: Merged
Merged at revision: 85
Proposed branch: lp:~brad-marshall/charms/trusty/rabbitmq-server/nagios-fixes-sync-charmhelpers
Merge into: lp:charms/trusty/rabbitmq-server
Diff against target: 1626 lines (+1012/-144)
29 files modified
charm-helpers.yaml (+1/-1)
config.yaml (+6/-0)
hooks/charmhelpers/contrib/charmsupport/nrpe.py (+41/-7)
hooks/charmhelpers/contrib/hahelpers/cluster.py (+5/-1)
hooks/charmhelpers/contrib/network/ip.py (+84/-1)
hooks/charmhelpers/contrib/openstack/amulet/deployment.py (+5/-2)
hooks/charmhelpers/contrib/openstack/context.py (+29/-10)
hooks/charmhelpers/contrib/openstack/files/__init__.py (+18/-0)
hooks/charmhelpers/contrib/openstack/files/check_haproxy.sh (+32/-0)
hooks/charmhelpers/contrib/openstack/files/check_haproxy_queue_depth.sh (+30/-0)
hooks/charmhelpers/contrib/openstack/ip.py (+37/-0)
hooks/charmhelpers/contrib/openstack/templates/ceph.conf (+15/-0)
hooks/charmhelpers/contrib/openstack/templates/haproxy.cfg (+58/-0)
hooks/charmhelpers/contrib/openstack/templates/openstack_https_frontend (+24/-0)
hooks/charmhelpers/contrib/openstack/templates/openstack_https_frontend.conf (+24/-0)
hooks/charmhelpers/contrib/openstack/templates/zeromq (+14/-0)
hooks/charmhelpers/contrib/openstack/utils.py (+7/-72)
hooks/charmhelpers/contrib/python/packages.py (+2/-2)
hooks/charmhelpers/core/fstab.py (+4/-4)
hooks/charmhelpers/core/host.py (+5/-5)
hooks/charmhelpers/core/services/helpers.py (+12/-4)
hooks/charmhelpers/core/strutils.py (+42/-0)
hooks/charmhelpers/core/sysctl.py (+13/-7)
hooks/charmhelpers/core/templating.py (+3/-3)
hooks/charmhelpers/core/unitdata.py (+477/-0)
hooks/charmhelpers/fetch/archiveurl.py (+10/-10)
hooks/charmhelpers/fetch/giturl.py (+1/-1)
hooks/rabbit_utils.py (+8/-0)
hooks/rabbitmq_server_relations.py (+5/-14)
To merge this branch: bzr merge lp:~brad-marshall/charms/trusty/rabbitmq-server/nagios-fixes-sync-charmhelpers
Reviewer Review Type Date Requested Status
Marco Ceppi (community) Approve
Adam Israel (community) Approve
Review Queue (community) automated testing Needs Fixing
OpenStack Charmers Pending
Review via email: mp+251551@code.launchpad.net

Description of the change

Add nagios-servicegroup config option, add nrpe check for service

To post a comment you must log in.
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #2183 rabbitmq-server for brad-marshall mp251551
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/2183/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #2393 rabbitmq-server for brad-marshall mp251551
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
  unit_tests/test_rabbitmq_server_relations.py:6:1: E402 module level import not at top of file
  make: *** [lint] Error 1

Full lint test output: http://paste.ubuntu.com/10520218/
Build: http://10.245.162.77:8080/job/charm_lint_check/2393/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #2313 rabbitmq-server for brad-marshall mp251551
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
  ERROR subprocess encountered error code 1
  make: *** [functional_test] Error 1

Full amulet test output: http://paste.ubuntu.com/10520280/
Build: http://10.245.162.77:8080/job/charm_amulet_test/2313/

86. By Brad Marshall

[bradm] Remove init nrpe checks, something isn't quite right with them for the checks.

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #2491 rabbitmq-server for brad-marshall mp251551
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
  unit_tests/test_rabbitmq_server_relations.py:6:1: E402 module level import not at top of file
  make: *** [lint] Error 1

Full lint test output: http://paste.ubuntu.com/10553544/
Build: http://10.245.162.77:8080/job/charm_lint_check/2491/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #2363 rabbitmq-server for brad-marshall mp251551
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
  ERROR subprocess encountered error code 1
  make: *** [functional_test] Error 1

Full amulet test output: http://paste.ubuntu.com/10553563/
Build: http://10.245.162.77:8080/job/charm_amulet_test/2363/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #2281 rabbitmq-server for brad-marshall mp251551
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/2281/

Revision history for this message
Review Queue (review-queue) wrote :

This items has failed automated testing! Results available here http://reports.vapour.ws/charm-tests/charm-bundle-test-11088-results

review: Needs Fixing (automated testing)
Revision history for this message
Antonio Rosales (arosales) wrote :

Running /tmp/bundletester-Ts1jVN/rabbitmq-server/tests/10_basic_deploy_test.py on AWS it looks like SSL cert verification is failing:

Testing ssl connection to rabbitmq-server.
Failed to create an ssl connection to 54.173.110.197:5671
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:598)

This also appears to be occurring across the other substrates.

@Brad,
Are you seeing this in your testing also?

-thanks,
Antonio

Revision history for this message
Adam Israel (aisrael) wrote :

Hi Brad,

Thanks for your work on improving the rabbitmq-server charm. I reviewed the merge proposal and your changes look solid. There are issues with the unit tests upstream, and bugs have been opened against those. I don't see anything in this MP that would caused the test failures, so you have a +1 from me.

review: Approve
Revision history for this message
Marco Ceppi (marcoceppi) wrote :

LGTM +1

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'charm-helpers.yaml'
--- charm-helpers.yaml 2015-01-23 08:23:05 +0000
+++ charm-helpers.yaml 2015-03-06 06:41:14 +0000
@@ -4,7 +4,7 @@
4 - fetch4 - fetch
5 - core5 - core
6 - contrib.charmsupport6 - contrib.charmsupport
7 - contrib.openstack7 - contrib.openstack|inc=*
8 - contrib.storage8 - contrib.storage
9 - contrib.peerstorage9 - contrib.peerstorage
10 - contrib.python.packages10 - contrib.python.packages
1111
=== modified file 'config.yaml'
--- config.yaml 2015-01-23 08:28:27 +0000
+++ config.yaml 2015-03-06 06:41:14 +0000
@@ -45,6 +45,12 @@
45 juju-myservice-045 juju-myservice-0
46 If you're running multiple environments with the same services in them46 If you're running multiple environments with the same services in them
47 this allows you to differentiate between them.47 this allows you to differentiate between them.
48 nagios_servicegroups:
49 default: ""
50 type: string
51 description: |
52 A comma-separated list of nagios servicegroups.
53 If left empty, the nagios_context will be used as the servicegroup
48 # HA configuration settings54 # HA configuration settings
49 vip:55 vip:
50 type: string56 type: string
5157
=== modified file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py'
--- hooks/charmhelpers/contrib/charmsupport/nrpe.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/contrib/charmsupport/nrpe.py 2015-03-06 06:41:14 +0000
@@ -24,6 +24,8 @@
24import pwd24import pwd
25import grp25import grp
26import os26import os
27import glob
28import shutil
27import re29import re
28import shlex30import shlex
29import yaml31import yaml
@@ -161,7 +163,7 @@
161 log('Check command not found: {}'.format(parts[0]))163 log('Check command not found: {}'.format(parts[0]))
162 return ''164 return ''
163165
164 def write(self, nagios_context, hostname, nagios_servicegroups=None):166 def write(self, nagios_context, hostname, nagios_servicegroups):
165 nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format(167 nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format(
166 self.command)168 self.command)
167 with open(nrpe_check_file, 'w') as nrpe_check_config:169 with open(nrpe_check_file, 'w') as nrpe_check_config:
@@ -177,14 +179,11 @@
177 nagios_servicegroups)179 nagios_servicegroups)
178180
179 def write_service_config(self, nagios_context, hostname,181 def write_service_config(self, nagios_context, hostname,
180 nagios_servicegroups=None):182 nagios_servicegroups):
181 for f in os.listdir(NRPE.nagios_exportdir):183 for f in os.listdir(NRPE.nagios_exportdir):
182 if re.search('.*{}.cfg'.format(self.command), f):184 if re.search('.*{}.cfg'.format(self.command), f):
183 os.remove(os.path.join(NRPE.nagios_exportdir, f))185 os.remove(os.path.join(NRPE.nagios_exportdir, f))
184186
185 if not nagios_servicegroups:
186 nagios_servicegroups = nagios_context
187
188 templ_vars = {187 templ_vars = {
189 'nagios_hostname': hostname,188 'nagios_hostname': hostname,
190 'nagios_servicegroup': nagios_servicegroups,189 'nagios_servicegroup': nagios_servicegroups,
@@ -211,10 +210,10 @@
211 super(NRPE, self).__init__()210 super(NRPE, self).__init__()
212 self.config = config()211 self.config = config()
213 self.nagios_context = self.config['nagios_context']212 self.nagios_context = self.config['nagios_context']
214 if 'nagios_servicegroups' in self.config:213 if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']:
215 self.nagios_servicegroups = self.config['nagios_servicegroups']214 self.nagios_servicegroups = self.config['nagios_servicegroups']
216 else:215 else:
217 self.nagios_servicegroups = 'juju'216 self.nagios_servicegroups = self.nagios_context
218 self.unit_name = local_unit().replace('/', '-')217 self.unit_name = local_unit().replace('/', '-')
219 if hostname:218 if hostname:
220 self.hostname = hostname219 self.hostname = hostname
@@ -322,3 +321,38 @@
322 check_cmd='check_status_file.py -f '321 check_cmd='check_status_file.py -f '
323 '/var/lib/nagios/service-check-%s.txt' % svc,322 '/var/lib/nagios/service-check-%s.txt' % svc,
324 )323 )
324
325
326def copy_nrpe_checks():
327 """
328 Copy the nrpe checks into place
329
330 """
331 NAGIOS_PLUGINS = '/usr/local/lib/nagios/plugins'
332 nrpe_files_dir = os.path.join(os.getenv('CHARM_DIR'), 'hooks',
333 'charmhelpers', 'contrib', 'openstack',
334 'files')
335
336 if not os.path.exists(NAGIOS_PLUGINS):
337 os.makedirs(NAGIOS_PLUGINS)
338 for fname in glob.glob(os.path.join(nrpe_files_dir, "check_*")):
339 if os.path.isfile(fname):
340 shutil.copy2(fname,
341 os.path.join(NAGIOS_PLUGINS, os.path.basename(fname)))
342
343
344def add_haproxy_checks(nrpe, unit_name):
345 """
346 Add checks for each service in list
347
348 :param NRPE nrpe: NRPE object to add check to
349 :param str unit_name: Unit name to use in check description
350 """
351 nrpe.add_check(
352 shortname='haproxy_servers',
353 description='Check HAProxy {%s}' % unit_name,
354 check_cmd='check_haproxy.sh')
355 nrpe.add_check(
356 shortname='haproxy_queue',
357 description='Check HAProxy queue depth {%s}' % unit_name,
358 check_cmd='check_haproxy_queue_depth.sh')
325359
=== modified file 'hooks/charmhelpers/contrib/hahelpers/cluster.py'
--- hooks/charmhelpers/contrib/hahelpers/cluster.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/contrib/hahelpers/cluster.py 2015-03-06 06:41:14 +0000
@@ -48,6 +48,9 @@
48from charmhelpers.core.decorators import (48from charmhelpers.core.decorators import (
49 retry_on_exception,49 retry_on_exception,
50)50)
51from charmhelpers.core.strutils import (
52 bool_from_string,
53)
5154
5255
53class HAIncompleteConfig(Exception):56class HAIncompleteConfig(Exception):
@@ -164,7 +167,8 @@
164 .167 .
165 returns: boolean168 returns: boolean
166 '''169 '''
167 if config_get('use-https') == "yes":170 use_https = config_get('use-https')
171 if use_https and bool_from_string(use_https):
168 return True172 return True
169 if config_get('ssl_cert') and config_get('ssl_key'):173 if config_get('ssl_cert') and config_get('ssl_key'):
170 return True174 return True
171175
=== modified file 'hooks/charmhelpers/contrib/network/ip.py'
--- hooks/charmhelpers/contrib/network/ip.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/contrib/network/ip.py 2015-03-06 06:41:14 +0000
@@ -17,13 +17,16 @@
17import glob17import glob
18import re18import re
19import subprocess19import subprocess
20import six
21import socket
2022
21from functools import partial23from functools import partial
2224
23from charmhelpers.core.hookenv import unit_get25from charmhelpers.core.hookenv import unit_get
24from charmhelpers.fetch import apt_install26from charmhelpers.fetch import apt_install
25from charmhelpers.core.hookenv import (27from charmhelpers.core.hookenv import (
26 log28 log,
29 WARNING,
27)30)
2831
29try:32try:
@@ -365,3 +368,83 @@
365 return True368 return True
366369
367 return False370 return False
371
372
373def is_ip(address):
374 """
375 Returns True if address is a valid IP address.
376 """
377 try:
378 # Test to see if already an IPv4 address
379 socket.inet_aton(address)
380 return True
381 except socket.error:
382 return False
383
384
385def ns_query(address):
386 try:
387 import dns.resolver
388 except ImportError:
389 apt_install('python-dnspython')
390 import dns.resolver
391
392 if isinstance(address, dns.name.Name):
393 rtype = 'PTR'
394 elif isinstance(address, six.string_types):
395 rtype = 'A'
396 else:
397 return None
398
399 answers = dns.resolver.query(address, rtype)
400 if answers:
401 return str(answers[0])
402 return None
403
404
405def get_host_ip(hostname, fallback=None):
406 """
407 Resolves the IP for a given hostname, or returns
408 the input if it is already an IP.
409 """
410 if is_ip(hostname):
411 return hostname
412
413 ip_addr = ns_query(hostname)
414 if not ip_addr:
415 try:
416 ip_addr = socket.gethostbyname(hostname)
417 except:
418 log("Failed to resolve hostname '%s'" % (hostname),
419 level=WARNING)
420 return fallback
421 return ip_addr
422
423
424def get_hostname(address, fqdn=True):
425 """
426 Resolves hostname for given IP, or returns the input
427 if it is already a hostname.
428 """
429 if is_ip(address):
430 try:
431 import dns.reversename
432 except ImportError:
433 apt_install("python-dnspython")
434 import dns.reversename
435
436 rev = dns.reversename.from_address(address)
437 result = ns_query(rev)
438 if not result:
439 return None
440 else:
441 result = address
442
443 if fqdn:
444 # strip trailing .
445 if result.endswith('.'):
446 return result[:-1]
447 else:
448 return result
449 else:
450 return result.split('.')[0]
368451
=== modified file 'hooks/charmhelpers/contrib/openstack/amulet/deployment.py'
--- hooks/charmhelpers/contrib/openstack/amulet/deployment.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/contrib/openstack/amulet/deployment.py 2015-03-06 06:41:14 +0000
@@ -71,16 +71,19 @@
71 services.append(this_service)71 services.append(this_service)
72 use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',72 use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',
73 'ceph-osd', 'ceph-radosgw']73 'ceph-osd', 'ceph-radosgw']
74 # Openstack subordinate charms do not expose an origin option as that
75 # is controlled by the principle
76 ignore = ['neutron-openvswitch']
7477
75 if self.openstack:78 if self.openstack:
76 for svc in services:79 for svc in services:
77 if svc['name'] not in use_source:80 if svc['name'] not in use_source + ignore:
78 config = {'openstack-origin': self.openstack}81 config = {'openstack-origin': self.openstack}
79 self.d.configure(svc['name'], config)82 self.d.configure(svc['name'], config)
8083
81 if self.source:84 if self.source:
82 for svc in services:85 for svc in services:
83 if svc['name'] in use_source:86 if svc['name'] in use_source and svc['name'] not in ignore:
84 config = {'source': self.source}87 config = {'source': self.source}
85 self.d.configure(svc['name'], config)88 self.d.configure(svc['name'], config)
8689
8790
=== modified file 'hooks/charmhelpers/contrib/openstack/context.py'
--- hooks/charmhelpers/contrib/openstack/context.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/contrib/openstack/context.py 2015-03-06 06:41:14 +0000
@@ -191,7 +191,7 @@
191 unit=local_unit())191 unit=local_unit())
192 if set_hostname != access_hostname:192 if set_hostname != access_hostname:
193 relation_set(relation_settings={hostname_key: access_hostname})193 relation_set(relation_settings={hostname_key: access_hostname})
194 return ctxt # Defer any further hook execution for now....194 return None # Defer any further hook execution for now....
195195
196 password_setting = 'password'196 password_setting = 'password'
197 if self.relation_prefix:197 if self.relation_prefix:
@@ -279,9 +279,25 @@
279class IdentityServiceContext(OSContextGenerator):279class IdentityServiceContext(OSContextGenerator):
280 interfaces = ['identity-service']280 interfaces = ['identity-service']
281281
282 def __init__(self, service=None, service_user=None):
283 self.service = service
284 self.service_user = service_user
285
282 def __call__(self):286 def __call__(self):
283 log('Generating template context for identity-service', level=DEBUG)287 log('Generating template context for identity-service', level=DEBUG)
284 ctxt = {}288 ctxt = {}
289
290 if self.service and self.service_user:
291 # This is required for pki token signing if we don't want /tmp to
292 # be used.
293 cachedir = '/var/cache/%s' % (self.service)
294 if not os.path.isdir(cachedir):
295 log("Creating service cache dir %s" % (cachedir), level=DEBUG)
296 mkdir(path=cachedir, owner=self.service_user,
297 group=self.service_user, perms=0o700)
298
299 ctxt['signing_dir'] = cachedir
300
285 for rid in relation_ids('identity-service'):301 for rid in relation_ids('identity-service'):
286 for unit in related_units(rid):302 for unit in related_units(rid):
287 rdata = relation_get(rid=rid, unit=unit)303 rdata = relation_get(rid=rid, unit=unit)
@@ -291,15 +307,16 @@
291 auth_host = format_ipv6_addr(auth_host) or auth_host307 auth_host = format_ipv6_addr(auth_host) or auth_host
292 svc_protocol = rdata.get('service_protocol') or 'http'308 svc_protocol = rdata.get('service_protocol') or 'http'
293 auth_protocol = rdata.get('auth_protocol') or 'http'309 auth_protocol = rdata.get('auth_protocol') or 'http'
294 ctxt = {'service_port': rdata.get('service_port'),310 ctxt.update({'service_port': rdata.get('service_port'),
295 'service_host': serv_host,311 'service_host': serv_host,
296 'auth_host': auth_host,312 'auth_host': auth_host,
297 'auth_port': rdata.get('auth_port'),313 'auth_port': rdata.get('auth_port'),
298 'admin_tenant_name': rdata.get('service_tenant'),314 'admin_tenant_name': rdata.get('service_tenant'),
299 'admin_user': rdata.get('service_username'),315 'admin_user': rdata.get('service_username'),
300 'admin_password': rdata.get('service_password'),316 'admin_password': rdata.get('service_password'),
301 'service_protocol': svc_protocol,317 'service_protocol': svc_protocol,
302 'auth_protocol': auth_protocol}318 'auth_protocol': auth_protocol})
319
303 if context_complete(ctxt):320 if context_complete(ctxt):
304 # NOTE(jamespage) this is required for >= icehouse321 # NOTE(jamespage) this is required for >= icehouse
305 # so a missing value just indicates keystone needs322 # so a missing value just indicates keystone needs
@@ -1021,6 +1038,8 @@
1021 for unit in related_units(rid):1038 for unit in related_units(rid):
1022 ctxt['zmq_nonce'] = relation_get('nonce', unit, rid)1039 ctxt['zmq_nonce'] = relation_get('nonce', unit, rid)
1023 ctxt['zmq_host'] = relation_get('host', unit, rid)1040 ctxt['zmq_host'] = relation_get('host', unit, rid)
1041 ctxt['zmq_redis_address'] = relation_get(
1042 'zmq_redis_address', unit, rid)
10241043
1025 return ctxt1044 return ctxt
10261045
10271046
=== added directory 'hooks/charmhelpers/contrib/openstack/files'
=== added file 'hooks/charmhelpers/contrib/openstack/files/__init__.py'
--- hooks/charmhelpers/contrib/openstack/files/__init__.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/openstack/files/__init__.py 2015-03-06 06:41:14 +0000
@@ -0,0 +1,18 @@
1# Copyright 2014-2015 Canonical Limited.
2#
3# This file is part of charm-helpers.
4#
5# charm-helpers is free software: you can redistribute it and/or modify
6# it under the terms of the GNU Lesser General Public License version 3 as
7# published by the Free Software Foundation.
8#
9# charm-helpers is distributed in the hope that it will be useful,
10# but WITHOUT ANY WARRANTY; without even the implied warranty of
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12# GNU Lesser General Public License for more details.
13#
14# You should have received a copy of the GNU Lesser General Public License
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
16
17# dummy __init__.py to fool syncer into thinking this is a syncable python
18# module
019
=== added file 'hooks/charmhelpers/contrib/openstack/files/check_haproxy.sh'
--- hooks/charmhelpers/contrib/openstack/files/check_haproxy.sh 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/openstack/files/check_haproxy.sh 2015-03-06 06:41:14 +0000
@@ -0,0 +1,32 @@
1#!/bin/bash
2#--------------------------------------------
3# This file is managed by Juju
4#--------------------------------------------
5#
6# Copyright 2009,2012 Canonical Ltd.
7# Author: Tom Haddon
8
9CRITICAL=0
10NOTACTIVE=''
11LOGFILE=/var/log/nagios/check_haproxy.log
12AUTH=$(grep -r "stats auth" /etc/haproxy | head -1 | awk '{print $4}')
13
14for appserver in $(grep ' server' /etc/haproxy/haproxy.cfg | awk '{print $2'});
15do
16 output=$(/usr/lib/nagios/plugins/check_http -a ${AUTH} -I 127.0.0.1 -p 8888 --regex="class=\"(active|backup)(2|3).*${appserver}" -e ' 200 OK')
17 if [ $? != 0 ]; then
18 date >> $LOGFILE
19 echo $output >> $LOGFILE
20 /usr/lib/nagios/plugins/check_http -a ${AUTH} -I 127.0.0.1 -p 8888 -v | grep $appserver >> $LOGFILE 2>&1
21 CRITICAL=1
22 NOTACTIVE="${NOTACTIVE} $appserver"
23 fi
24done
25
26if [ $CRITICAL = 1 ]; then
27 echo "CRITICAL:${NOTACTIVE}"
28 exit 2
29fi
30
31echo "OK: All haproxy instances looking good"
32exit 0
033
=== added file 'hooks/charmhelpers/contrib/openstack/files/check_haproxy_queue_depth.sh'
--- hooks/charmhelpers/contrib/openstack/files/check_haproxy_queue_depth.sh 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/openstack/files/check_haproxy_queue_depth.sh 2015-03-06 06:41:14 +0000
@@ -0,0 +1,30 @@
1#!/bin/bash
2#--------------------------------------------
3# This file is managed by Juju
4#--------------------------------------------
5#
6# Copyright 2009,2012 Canonical Ltd.
7# Author: Tom Haddon
8
9# These should be config options at some stage
10CURRQthrsh=0
11MAXQthrsh=100
12
13AUTH=$(grep -r "stats auth" /etc/haproxy | head -1 | awk '{print $4}')
14
15HAPROXYSTATS=$(/usr/lib/nagios/plugins/check_http -a ${AUTH} -I 127.0.0.1 -p 8888 -u '/;csv' -v)
16
17for BACKEND in $(echo $HAPROXYSTATS| xargs -n1 | grep BACKEND | awk -F , '{print $1}')
18do
19 CURRQ=$(echo "$HAPROXYSTATS" | grep $BACKEND | grep BACKEND | cut -d , -f 3)
20 MAXQ=$(echo "$HAPROXYSTATS" | grep $BACKEND | grep BACKEND | cut -d , -f 4)
21
22 if [[ $CURRQ -gt $CURRQthrsh || $MAXQ -gt $MAXQthrsh ]] ; then
23 echo "CRITICAL: queue depth for $BACKEND - CURRENT:$CURRQ MAX:$MAXQ"
24 exit 2
25 fi
26done
27
28echo "OK: All haproxy queue depths looking good"
29exit 0
30
031
=== modified file 'hooks/charmhelpers/contrib/openstack/ip.py'
--- hooks/charmhelpers/contrib/openstack/ip.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/contrib/openstack/ip.py 2015-03-06 06:41:14 +0000
@@ -26,6 +26,8 @@
26)26)
27from charmhelpers.contrib.hahelpers.cluster import is_clustered27from charmhelpers.contrib.hahelpers.cluster import is_clustered
2828
29from functools import partial
30
29PUBLIC = 'public'31PUBLIC = 'public'
30INTERNAL = 'int'32INTERNAL = 'int'
31ADMIN = 'admin'33ADMIN = 'admin'
@@ -107,3 +109,38 @@
107 "clustered=%s)" % (net_type, clustered))109 "clustered=%s)" % (net_type, clustered))
108110
109 return resolved_address111 return resolved_address
112
113
114def endpoint_url(configs, url_template, port, endpoint_type=PUBLIC,
115 override=None):
116 """Returns the correct endpoint URL to advertise to Keystone.
117
118 This method provides the correct endpoint URL which should be advertised to
119 the keystone charm for endpoint creation. This method allows for the url to
120 be overridden to force a keystone endpoint to have specific URL for any of
121 the defined scopes (admin, internal, public).
122
123 :param configs: OSTemplateRenderer config templating object to inspect
124 for a complete https context.
125 :param url_template: str format string for creating the url template. Only
126 two values will be passed - the scheme+hostname
127 returned by the canonical_url and the port.
128 :param endpoint_type: str endpoint type to resolve.
129 :param override: str the name of the config option which overrides the
130 endpoint URL defined by the charm itself. None will
131 disable any overrides (default).
132 """
133 if override:
134 # Return any user-defined overrides for the keystone endpoint URL.
135 user_value = config(override)
136 if user_value:
137 return user_value.strip()
138
139 return url_template % (canonical_url(configs, endpoint_type), port)
140
141
142public_endpoint = partial(endpoint_url, endpoint_type=PUBLIC)
143
144internal_endpoint = partial(endpoint_url, endpoint_type=INTERNAL)
145
146admin_endpoint = partial(endpoint_url, endpoint_type=ADMIN)
110147
=== added file 'hooks/charmhelpers/contrib/openstack/templates/ceph.conf'
--- hooks/charmhelpers/contrib/openstack/templates/ceph.conf 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/openstack/templates/ceph.conf 2015-03-06 06:41:14 +0000
@@ -0,0 +1,15 @@
1###############################################################################
2# [ WARNING ]
3# cinder configuration file maintained by Juju
4# local changes may be overwritten.
5###############################################################################
6[global]
7{% if auth -%}
8 auth_supported = {{ auth }}
9 keyring = /etc/ceph/$cluster.$name.keyring
10 mon host = {{ mon_hosts }}
11{% endif -%}
12 log to syslog = {{ use_syslog }}
13 err to syslog = {{ use_syslog }}
14 clog to syslog = {{ use_syslog }}
15
016
=== added file 'hooks/charmhelpers/contrib/openstack/templates/haproxy.cfg'
--- hooks/charmhelpers/contrib/openstack/templates/haproxy.cfg 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/openstack/templates/haproxy.cfg 2015-03-06 06:41:14 +0000
@@ -0,0 +1,58 @@
1global
2 log {{ local_host }} local0
3 log {{ local_host }} local1 notice
4 maxconn 20000
5 user haproxy
6 group haproxy
7 spread-checks 0
8
9defaults
10 log global
11 mode tcp
12 option tcplog
13 option dontlognull
14 retries 3
15 timeout queue 1000
16 timeout connect 1000
17{% if haproxy_client_timeout -%}
18 timeout client {{ haproxy_client_timeout }}
19{% else -%}
20 timeout client 30000
21{% endif -%}
22
23{% if haproxy_server_timeout -%}
24 timeout server {{ haproxy_server_timeout }}
25{% else -%}
26 timeout server 30000
27{% endif -%}
28
29listen stats {{ stat_port }}
30 mode http
31 stats enable
32 stats hide-version
33 stats realm Haproxy\ Statistics
34 stats uri /
35 stats auth admin:password
36
37{% if frontends -%}
38{% for service, ports in service_ports.items() -%}
39frontend tcp-in_{{ service }}
40 bind *:{{ ports[0] }}
41 {% if ipv6 -%}
42 bind :::{{ ports[0] }}
43 {% endif -%}
44 {% for frontend in frontends -%}
45 acl net_{{ frontend }} dst {{ frontends[frontend]['network'] }}
46 use_backend {{ service }}_{{ frontend }} if net_{{ frontend }}
47 {% endfor -%}
48 default_backend {{ service }}_{{ default_backend }}
49
50{% for frontend in frontends -%}
51backend {{ service }}_{{ frontend }}
52 balance leastconn
53 {% for unit, address in frontends[frontend]['backends'].items() -%}
54 server {{ unit }} {{ address }}:{{ ports[1] }} check
55 {% endfor %}
56{% endfor -%}
57{% endfor -%}
58{% endif -%}
059
=== added file 'hooks/charmhelpers/contrib/openstack/templates/openstack_https_frontend'
--- hooks/charmhelpers/contrib/openstack/templates/openstack_https_frontend 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/openstack/templates/openstack_https_frontend 2015-03-06 06:41:14 +0000
@@ -0,0 +1,24 @@
1{% if endpoints -%}
2{% for ext_port in ext_ports -%}
3Listen {{ ext_port }}
4{% endfor -%}
5{% for address, endpoint, ext, int in endpoints -%}
6<VirtualHost {{ address }}:{{ ext }}>
7 ServerName {{ endpoint }}
8 SSLEngine on
9 SSLCertificateFile /etc/apache2/ssl/{{ namespace }}/cert_{{ endpoint }}
10 SSLCertificateKeyFile /etc/apache2/ssl/{{ namespace }}/key_{{ endpoint }}
11 ProxyPass / http://localhost:{{ int }}/
12 ProxyPassReverse / http://localhost:{{ int }}/
13 ProxyPreserveHost on
14</VirtualHost>
15{% endfor -%}
16<Proxy *>
17 Order deny,allow
18 Allow from all
19</Proxy>
20<Location />
21 Order allow,deny
22 Allow from all
23</Location>
24{% endif -%}
025
=== added file 'hooks/charmhelpers/contrib/openstack/templates/openstack_https_frontend.conf'
--- hooks/charmhelpers/contrib/openstack/templates/openstack_https_frontend.conf 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/openstack/templates/openstack_https_frontend.conf 2015-03-06 06:41:14 +0000
@@ -0,0 +1,24 @@
1{% if endpoints -%}
2{% for ext_port in ext_ports -%}
3Listen {{ ext_port }}
4{% endfor -%}
5{% for address, endpoint, ext, int in endpoints -%}
6<VirtualHost {{ address }}:{{ ext }}>
7 ServerName {{ endpoint }}
8 SSLEngine on
9 SSLCertificateFile /etc/apache2/ssl/{{ namespace }}/cert_{{ endpoint }}
10 SSLCertificateKeyFile /etc/apache2/ssl/{{ namespace }}/key_{{ endpoint }}
11 ProxyPass / http://localhost:{{ int }}/
12 ProxyPassReverse / http://localhost:{{ int }}/
13 ProxyPreserveHost on
14</VirtualHost>
15{% endfor -%}
16<Proxy *>
17 Order deny,allow
18 Allow from all
19</Proxy>
20<Location />
21 Order allow,deny
22 Allow from all
23</Location>
24{% endif -%}
025
=== added file 'hooks/charmhelpers/contrib/openstack/templates/zeromq'
--- hooks/charmhelpers/contrib/openstack/templates/zeromq 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/openstack/templates/zeromq 2015-03-06 06:41:14 +0000
@@ -0,0 +1,14 @@
1{% if zmq_host -%}
2# ZeroMQ configuration (restart-nonce: {{ zmq_nonce }})
3rpc_backend = zmq
4rpc_zmq_host = {{ zmq_host }}
5{% if zmq_redis_address -%}
6rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_redis.MatchMakerRedis
7matchmaker_heartbeat_freq = 15
8matchmaker_heartbeat_ttl = 30
9[matchmaker_redis]
10host = {{ zmq_redis_address }}
11{% else -%}
12rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing
13{% endif -%}
14{% endif -%}
015
=== modified file 'hooks/charmhelpers/contrib/openstack/utils.py'
--- hooks/charmhelpers/contrib/openstack/utils.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/contrib/openstack/utils.py 2015-03-06 06:41:14 +0000
@@ -23,12 +23,13 @@
23import subprocess23import subprocess
24import json24import json
25import os25import os
26import socket
27import sys26import sys
2827
29import six28import six
30import yaml29import yaml
3130
31from charmhelpers.contrib.network import ip
32
32from charmhelpers.core.hookenv import (33from charmhelpers.core.hookenv import (
33 config,34 config,
34 log as juju_log,35 log as juju_log,
@@ -103,6 +104,7 @@
103 ('2.1.0', 'juno'),104 ('2.1.0', 'juno'),
104 ('2.2.0', 'juno'),105 ('2.2.0', 'juno'),
105 ('2.2.1', 'kilo'),106 ('2.2.1', 'kilo'),
107 ('2.2.2', 'kilo'),
106])108])
107109
108DEFAULT_LOOPBACK_SIZE = '5G'110DEFAULT_LOOPBACK_SIZE = '5G'
@@ -420,77 +422,10 @@
420 else:422 else:
421 zap_disk(block_device)423 zap_disk(block_device)
422424
423425is_ip = ip.is_ip
424def is_ip(address):426ns_query = ip.ns_query
425 """427get_host_ip = ip.get_host_ip
426 Returns True if address is a valid IP address.428get_hostname = ip.get_hostname
427 """
428 try:
429 # Test to see if already an IPv4 address
430 socket.inet_aton(address)
431 return True
432 except socket.error:
433 return False
434
435
436def ns_query(address):
437 try:
438 import dns.resolver
439 except ImportError:
440 apt_install('python-dnspython')
441 import dns.resolver
442
443 if isinstance(address, dns.name.Name):
444 rtype = 'PTR'
445 elif isinstance(address, six.string_types):
446 rtype = 'A'
447 else:
448 return None
449
450 answers = dns.resolver.query(address, rtype)
451 if answers:
452 return str(answers[0])
453 return None
454
455
456def get_host_ip(hostname):
457 """
458 Resolves the IP for a given hostname, or returns
459 the input if it is already an IP.
460 """
461 if is_ip(hostname):
462 return hostname
463
464 return ns_query(hostname)
465
466
467def get_hostname(address, fqdn=True):
468 """
469 Resolves hostname for given IP, or returns the input
470 if it is already a hostname.
471 """
472 if is_ip(address):
473 try:
474 import dns.reversename
475 except ImportError:
476 apt_install('python-dnspython')
477 import dns.reversename
478
479 rev = dns.reversename.from_address(address)
480 result = ns_query(rev)
481 if not result:
482 return None
483 else:
484 result = address
485
486 if fqdn:
487 # strip trailing .
488 if result.endswith('.'):
489 return result[:-1]
490 else:
491 return result
492 else:
493 return result.split('.')[0]
494429
495430
496def get_matchmaker_map(mm_file='/etc/oslo/matchmaker_ring.json'):431def get_matchmaker_map(mm_file='/etc/oslo/matchmaker_ring.json'):
497432
=== modified file 'hooks/charmhelpers/contrib/python/packages.py'
--- hooks/charmhelpers/contrib/python/packages.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/contrib/python/packages.py 2015-03-06 06:41:14 +0000
@@ -17,8 +17,6 @@
17# You should have received a copy of the GNU Lesser General Public License17# You should have received a copy of the GNU Lesser General Public License
18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1919
20__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
21
22from charmhelpers.fetch import apt_install, apt_update20from charmhelpers.fetch import apt_install, apt_update
23from charmhelpers.core.hookenv import log21from charmhelpers.core.hookenv import log
2422
@@ -29,6 +27,8 @@
29 apt_install('python-pip')27 apt_install('python-pip')
30 from pip import main as pip_execute28 from pip import main as pip_execute
3129
30__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
31
3232
33def parse_options(given, available):33def parse_options(given, available):
34 """Given a set of options, check if available"""34 """Given a set of options, check if available"""
3535
=== modified file 'hooks/charmhelpers/core/fstab.py'
--- hooks/charmhelpers/core/fstab.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/core/fstab.py 2015-03-06 06:41:14 +0000
@@ -17,11 +17,11 @@
17# You should have received a copy of the GNU Lesser General Public License17# You should have received a copy of the GNU Lesser General Public License
18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1919
20__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
21
22import io20import io
23import os21import os
2422
23__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
24
2525
26class Fstab(io.FileIO):26class Fstab(io.FileIO):
27 """This class extends file in order to implement a file reader/writer27 """This class extends file in order to implement a file reader/writer
@@ -77,7 +77,7 @@
77 for line in self.readlines():77 for line in self.readlines():
78 line = line.decode('us-ascii')78 line = line.decode('us-ascii')
79 try:79 try:
80 if line.strip() and not line.startswith("#"):80 if line.strip() and not line.strip().startswith("#"):
81 yield self._hydrate_entry(line)81 yield self._hydrate_entry(line)
82 except ValueError:82 except ValueError:
83 pass83 pass
@@ -104,7 +104,7 @@
104104
105 found = False105 found = False
106 for index, line in enumerate(lines):106 for index, line in enumerate(lines):
107 if not line.startswith("#"):107 if line.strip() and not line.strip().startswith("#"):
108 if self._hydrate_entry(line) == entry:108 if self._hydrate_entry(line) == entry:
109 found = True109 found = True
110 break110 break
111111
=== modified file 'hooks/charmhelpers/core/host.py'
--- hooks/charmhelpers/core/host.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/core/host.py 2015-03-06 06:41:14 +0000
@@ -191,11 +191,11 @@
191191
192192
193def write_file(path, content, owner='root', group='root', perms=0o444):193def write_file(path, content, owner='root', group='root', perms=0o444):
194 """Create or overwrite a file with the contents of a string"""194 """Create or overwrite a file with the contents of a byte string."""
195 log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))195 log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
196 uid = pwd.getpwnam(owner).pw_uid196 uid = pwd.getpwnam(owner).pw_uid
197 gid = grp.getgrnam(group).gr_gid197 gid = grp.getgrnam(group).gr_gid
198 with open(path, 'w') as target:198 with open(path, 'wb') as target:
199 os.fchown(target.fileno(), uid, gid)199 os.fchown(target.fileno(), uid, gid)
200 os.fchmod(target.fileno(), perms)200 os.fchmod(target.fileno(), perms)
201 target.write(content)201 target.write(content)
@@ -305,11 +305,11 @@
305 ceph_client_changed function.305 ceph_client_changed function.
306 """306 """
307 def wrap(f):307 def wrap(f):
308 def wrapped_f(*args):308 def wrapped_f(*args, **kwargs):
309 checksums = {}309 checksums = {}
310 for path in restart_map:310 for path in restart_map:
311 checksums[path] = file_hash(path)311 checksums[path] = file_hash(path)
312 f(*args)312 f(*args, **kwargs)
313 restarts = []313 restarts = []
314 for path in restart_map:314 for path in restart_map:
315 if checksums[path] != file_hash(path):315 if checksums[path] != file_hash(path):
@@ -361,7 +361,7 @@
361 ip_output = (line for line in ip_output if line)361 ip_output = (line for line in ip_output if line)
362 for line in ip_output:362 for line in ip_output:
363 if line.split()[1].startswith(int_type):363 if line.split()[1].startswith(int_type):
364 matched = re.search('.*: (bond[0-9]+\.[0-9]+)@.*', line)364 matched = re.search('.*: (' + int_type + r'[0-9]+\.[0-9]+)@.*', line)
365 if matched:365 if matched:
366 interface = matched.groups()[0]366 interface = matched.groups()[0]
367 else:367 else:
368368
=== modified file 'hooks/charmhelpers/core/services/helpers.py'
--- hooks/charmhelpers/core/services/helpers.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/core/services/helpers.py 2015-03-06 06:41:14 +0000
@@ -45,12 +45,14 @@
45 """45 """
46 name = None46 name = None
47 interface = None47 interface = None
48 required_keys = []
4948
50 def __init__(self, name=None, additional_required_keys=None):49 def __init__(self, name=None, additional_required_keys=None):
50 if not hasattr(self, 'required_keys'):
51 self.required_keys = []
52
51 if name is not None:53 if name is not None:
52 self.name = name54 self.name = name
53 if additional_required_keys is not None:55 if additional_required_keys:
54 self.required_keys.extend(additional_required_keys)56 self.required_keys.extend(additional_required_keys)
55 self.get_data()57 self.get_data()
5658
@@ -134,7 +136,10 @@
134 """136 """
135 name = 'db'137 name = 'db'
136 interface = 'mysql'138 interface = 'mysql'
137 required_keys = ['host', 'user', 'password', 'database']139
140 def __init__(self, *args, **kwargs):
141 self.required_keys = ['host', 'user', 'password', 'database']
142 super(HttpRelation).__init__(self, *args, **kwargs)
138143
139144
140class HttpRelation(RelationContext):145class HttpRelation(RelationContext):
@@ -146,7 +151,10 @@
146 """151 """
147 name = 'website'152 name = 'website'
148 interface = 'http'153 interface = 'http'
149 required_keys = ['host', 'port']154
155 def __init__(self, *args, **kwargs):
156 self.required_keys = ['host', 'port']
157 super(HttpRelation).__init__(self, *args, **kwargs)
150158
151 def provide_data(self):159 def provide_data(self):
152 return {160 return {
153161
=== added file 'hooks/charmhelpers/core/strutils.py'
--- hooks/charmhelpers/core/strutils.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/strutils.py 2015-03-06 06:41:14 +0000
@@ -0,0 +1,42 @@
1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3
4# Copyright 2014-2015 Canonical Limited.
5#
6# This file is part of charm-helpers.
7#
8# charm-helpers is free software: you can redistribute it and/or modify
9# it under the terms of the GNU Lesser General Public License version 3 as
10# published by the Free Software Foundation.
11#
12# charm-helpers is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15# GNU Lesser General Public License for more details.
16#
17# You should have received a copy of the GNU Lesser General Public License
18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
19
20import six
21
22
23def bool_from_string(value):
24 """Interpret string value as boolean.
25
26 Returns True if value translates to True otherwise False.
27 """
28 if isinstance(value, six.string_types):
29 value = six.text_type(value)
30 else:
31 msg = "Unable to interpret non-string value '%s' as boolean" % (value)
32 raise ValueError(msg)
33
34 value = value.strip().lower()
35
36 if value in ['y', 'yes', 'true', 't']:
37 return True
38 elif value in ['n', 'no', 'false', 'f']:
39 return False
40
41 msg = "Unable to interpret string value '%s' as boolean" % (value)
42 raise ValueError(msg)
043
=== modified file 'hooks/charmhelpers/core/sysctl.py'
--- hooks/charmhelpers/core/sysctl.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/core/sysctl.py 2015-03-06 06:41:14 +0000
@@ -17,8 +17,6 @@
17# You should have received a copy of the GNU Lesser General Public License17# You should have received a copy of the GNU Lesser General Public License
18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1919
20__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
21
22import yaml20import yaml
2321
24from subprocess import check_call22from subprocess import check_call
@@ -26,25 +24,33 @@
26from charmhelpers.core.hookenv import (24from charmhelpers.core.hookenv import (
27 log,25 log,
28 DEBUG,26 DEBUG,
27 ERROR,
29)28)
3029
30__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
31
3132
32def create(sysctl_dict, sysctl_file):33def create(sysctl_dict, sysctl_file):
33 """Creates a sysctl.conf file from a YAML associative array34 """Creates a sysctl.conf file from a YAML associative array
3435
35 :param sysctl_dict: a dict of sysctl options eg { 'kernel.max_pid': 1337 }36 :param sysctl_dict: a YAML-formatted string of sysctl options eg "{ 'kernel.max_pid': 1337 }"
36 :type sysctl_dict: dict37 :type sysctl_dict: str
37 :param sysctl_file: path to the sysctl file to be saved38 :param sysctl_file: path to the sysctl file to be saved
38 :type sysctl_file: str or unicode39 :type sysctl_file: str or unicode
39 :returns: None40 :returns: None
40 """41 """
41 sysctl_dict = yaml.load(sysctl_dict)42 try:
43 sysctl_dict_parsed = yaml.safe_load(sysctl_dict)
44 except yaml.YAMLError:
45 log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict),
46 level=ERROR)
47 return
4248
43 with open(sysctl_file, "w") as fd:49 with open(sysctl_file, "w") as fd:
44 for key, value in sysctl_dict.items():50 for key, value in sysctl_dict_parsed.items():
45 fd.write("{}={}\n".format(key, value))51 fd.write("{}={}\n".format(key, value))
4652
47 log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict),53 log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict_parsed),
48 level=DEBUG)54 level=DEBUG)
4955
50 check_call(["sysctl", "-p", sysctl_file])56 check_call(["sysctl", "-p", sysctl_file])
5157
=== modified file 'hooks/charmhelpers/core/templating.py'
--- hooks/charmhelpers/core/templating.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/core/templating.py 2015-03-06 06:41:14 +0000
@@ -21,7 +21,7 @@
2121
2222
23def render(source, target, context, owner='root', group='root',23def render(source, target, context, owner='root', group='root',
24 perms=0o444, templates_dir=None):24 perms=0o444, templates_dir=None, encoding='UTF-8'):
25 """25 """
26 Render a template.26 Render a template.
2727
@@ -64,5 +64,5 @@
64 level=hookenv.ERROR)64 level=hookenv.ERROR)
65 raise e65 raise e
66 content = template.render(context)66 content = template.render(context)
67 host.mkdir(os.path.dirname(target), owner, group)67 host.mkdir(os.path.dirname(target), owner, group, perms=0o755)
68 host.write_file(target, content, owner, group, perms)68 host.write_file(target, content.encode(encoding), owner, group, perms)
6969
=== added file 'hooks/charmhelpers/core/unitdata.py'
--- hooks/charmhelpers/core/unitdata.py 1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/unitdata.py 2015-03-06 06:41:14 +0000
@@ -0,0 +1,477 @@
1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3#
4# Copyright 2014-2015 Canonical Limited.
5#
6# This file is part of charm-helpers.
7#
8# charm-helpers is free software: you can redistribute it and/or modify
9# it under the terms of the GNU Lesser General Public License version 3 as
10# published by the Free Software Foundation.
11#
12# charm-helpers is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15# GNU Lesser General Public License for more details.
16#
17# You should have received a copy of the GNU Lesser General Public License
18# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
19#
20#
21# Authors:
22# Kapil Thangavelu <kapil.foss@gmail.com>
23#
24"""
25Intro
26-----
27
28A simple way to store state in units. This provides a key value
29storage with support for versioned, transactional operation,
30and can calculate deltas from previous values to simplify unit logic
31when processing changes.
32
33
34Hook Integration
35----------------
36
37There are several extant frameworks for hook execution, including
38
39 - charmhelpers.core.hookenv.Hooks
40 - charmhelpers.core.services.ServiceManager
41
42The storage classes are framework agnostic, one simple integration is
43via the HookData contextmanager. It will record the current hook
44execution environment (including relation data, config data, etc.),
45setup a transaction and allow easy access to the changes from
46previously seen values. One consequence of the integration is the
47reservation of particular keys ('rels', 'unit', 'env', 'config',
48'charm_revisions') for their respective values.
49
50Here's a fully worked integration example using hookenv.Hooks::
51
52 from charmhelper.core import hookenv, unitdata
53
54 hook_data = unitdata.HookData()
55 db = unitdata.kv()
56 hooks = hookenv.Hooks()
57
58 @hooks.hook
59 def config_changed():
60 # Print all changes to configuration from previously seen
61 # values.
62 for changed, (prev, cur) in hook_data.conf.items():
63 print('config changed', changed,
64 'previous value', prev,
65 'current value', cur)
66
67 # Get some unit specific bookeeping
68 if not db.get('pkg_key'):
69 key = urllib.urlopen('https://example.com/pkg_key').read()
70 db.set('pkg_key', key)
71
72 # Directly access all charm config as a mapping.
73 conf = db.getrange('config', True)
74
75 # Directly access all relation data as a mapping
76 rels = db.getrange('rels', True)
77
78 if __name__ == '__main__':
79 with hook_data():
80 hook.execute()
81
82
83A more basic integration is via the hook_scope context manager which simply
84manages transaction scope (and records hook name, and timestamp)::
85
86 >>> from unitdata import kv
87 >>> db = kv()
88 >>> with db.hook_scope('install'):
89 ... # do work, in transactional scope.
90 ... db.set('x', 1)
91 >>> db.get('x')
92 1
93
94
95Usage
96-----
97
98Values are automatically json de/serialized to preserve basic typing
99and complex data struct capabilities (dicts, lists, ints, booleans, etc).
100
101Individual values can be manipulated via get/set::
102
103 >>> kv.set('y', True)
104 >>> kv.get('y')
105 True
106
107 # We can set complex values (dicts, lists) as a single key.
108 >>> kv.set('config', {'a': 1, 'b': True'})
109
110 # Also supports returning dictionaries as a record which
111 # provides attribute access.
112 >>> config = kv.get('config', record=True)
113 >>> config.b
114 True
115
116
117Groups of keys can be manipulated with update/getrange::
118
119 >>> kv.update({'z': 1, 'y': 2}, prefix="gui.")
120 >>> kv.getrange('gui.', strip=True)
121 {'z': 1, 'y': 2}
122
123When updating values, its very helpful to understand which values
124have actually changed and how have they changed. The storage
125provides a delta method to provide for this::
126
127 >>> data = {'debug': True, 'option': 2}
128 >>> delta = kv.delta(data, 'config.')
129 >>> delta.debug.previous
130 None
131 >>> delta.debug.current
132 True
133 >>> delta
134 {'debug': (None, True), 'option': (None, 2)}
135
136Note the delta method does not persist the actual change, it needs to
137be explicitly saved via 'update' method::
138
139 >>> kv.update(data, 'config.')
140
141Values modified in the context of a hook scope retain historical values
142associated to the hookname.
143
144 >>> with db.hook_scope('config-changed'):
145 ... db.set('x', 42)
146 >>> db.gethistory('x')
147 [(1, u'x', 1, u'install', u'2015-01-21T16:49:30.038372'),
148 (2, u'x', 42, u'config-changed', u'2015-01-21T16:49:30.038786')]
149
150"""
151
152import collections
153import contextlib
154import datetime
155import json
156import os
157import pprint
158import sqlite3
159import sys
160
161__author__ = 'Kapil Thangavelu <kapil.foss@gmail.com>'
162
163
164class Storage(object):
165 """Simple key value database for local unit state within charms.
166
167 Modifications are automatically committed at hook exit. That's
168 currently regardless of exit code.
169
170 To support dicts, lists, integer, floats, and booleans values
171 are automatically json encoded/decoded.
172 """
173 def __init__(self, path=None):
174 self.db_path = path
175 if path is None:
176 self.db_path = os.path.join(
177 os.environ.get('CHARM_DIR', ''), '.unit-state.db')
178 self.conn = sqlite3.connect('%s' % self.db_path)
179 self.cursor = self.conn.cursor()
180 self.revision = None
181 self._closed = False
182 self._init()
183
184 def close(self):
185 if self._closed:
186 return
187 self.flush(False)
188 self.cursor.close()
189 self.conn.close()
190 self._closed = True
191
192 def _scoped_query(self, stmt, params=None):
193 if params is None:
194 params = []
195 return stmt, params
196
197 def get(self, key, default=None, record=False):
198 self.cursor.execute(
199 *self._scoped_query(
200 'select data from kv where key=?', [key]))
201 result = self.cursor.fetchone()
202 if not result:
203 return default
204 if record:
205 return Record(json.loads(result[0]))
206 return json.loads(result[0])
207
208 def getrange(self, key_prefix, strip=False):
209 stmt = "select key, data from kv where key like '%s%%'" % key_prefix
210 self.cursor.execute(*self._scoped_query(stmt))
211 result = self.cursor.fetchall()
212
213 if not result:
214 return None
215 if not strip:
216 key_prefix = ''
217 return dict([
218 (k[len(key_prefix):], json.loads(v)) for k, v in result])
219
220 def update(self, mapping, prefix=""):
221 for k, v in mapping.items():
222 self.set("%s%s" % (prefix, k), v)
223
224 def unset(self, key):
225 self.cursor.execute('delete from kv where key=?', [key])
226 if self.revision and self.cursor.rowcount:
227 self.cursor.execute(
228 'insert into kv_revisions values (?, ?, ?)',
229 [key, self.revision, json.dumps('DELETED')])
230
231 def set(self, key, value):
232 serialized = json.dumps(value)
233
234 self.cursor.execute(
235 'select data from kv where key=?', [key])
236 exists = self.cursor.fetchone()
237
238 # Skip mutations to the same value
239 if exists:
240 if exists[0] == serialized:
241 return value
242
243 if not exists:
244 self.cursor.execute(
245 'insert into kv (key, data) values (?, ?)',
246 (key, serialized))
247 else:
248 self.cursor.execute('''
249 update kv
250 set data = ?
251 where key = ?''', [serialized, key])
252
253 # Save
254 if not self.revision:
255 return value
256
257 self.cursor.execute(
258 'select 1 from kv_revisions where key=? and revision=?',
259 [key, self.revision])
260 exists = self.cursor.fetchone()
261
262 if not exists:
263 self.cursor.execute(
264 '''insert into kv_revisions (
265 revision, key, data) values (?, ?, ?)''',
266 (self.revision, key, serialized))
267 else:
268 self.cursor.execute(
269 '''
270 update kv_revisions
271 set data = ?
272 where key = ?
273 and revision = ?''',
274 [serialized, key, self.revision])
275
276 return value
277
278 def delta(self, mapping, prefix):
279 """
280 return a delta containing values that have changed.
281 """
282 previous = self.getrange(prefix, strip=True)
283 if not previous:
284 pk = set()
285 else:
286 pk = set(previous.keys())
287 ck = set(mapping.keys())
288 delta = DeltaSet()
289
290 # added
291 for k in ck.difference(pk):
292 delta[k] = Delta(None, mapping[k])
293
294 # removed
295 for k in pk.difference(ck):
296 delta[k] = Delta(previous[k], None)
297
298 # changed
299 for k in pk.intersection(ck):
300 c = mapping[k]
301 p = previous[k]
302 if c != p:
303 delta[k] = Delta(p, c)
304
305 return delta
306
307 @contextlib.contextmanager
308 def hook_scope(self, name=""):
309 """Scope all future interactions to the current hook execution
310 revision."""
311 assert not self.revision
312 self.cursor.execute(
313 'insert into hooks (hook, date) values (?, ?)',
314 (name or sys.argv[0],
315 datetime.datetime.utcnow().isoformat()))
316 self.revision = self.cursor.lastrowid
317 try:
318 yield self.revision
319 self.revision = None
320 except:
321 self.flush(False)
322 self.revision = None
323 raise
324 else:
325 self.flush()
326
327 def flush(self, save=True):
328 if save:
329 self.conn.commit()
330 elif self._closed:
331 return
332 else:
333 self.conn.rollback()
334
335 def _init(self):
336 self.cursor.execute('''
337 create table if not exists kv (
338 key text,
339 data text,
340 primary key (key)
341 )''')
342 self.cursor.execute('''
343 create table if not exists kv_revisions (
344 key text,
345 revision integer,
346 data text,
347 primary key (key, revision)
348 )''')
349 self.cursor.execute('''
350 create table if not exists hooks (
351 version integer primary key autoincrement,
352 hook text,
353 date text
354 )''')
355 self.conn.commit()
356
357 def gethistory(self, key, deserialize=False):
358 self.cursor.execute(
359 '''
360 select kv.revision, kv.key, kv.data, h.hook, h.date
361 from kv_revisions kv,
362 hooks h
363 where kv.key=?
364 and kv.revision = h.version
365 ''', [key])
366 if deserialize is False:
367 return self.cursor.fetchall()
368 return map(_parse_history, self.cursor.fetchall())
369
370 def debug(self, fh=sys.stderr):
371 self.cursor.execute('select * from kv')
372 pprint.pprint(self.cursor.fetchall(), stream=fh)
373 self.cursor.execute('select * from kv_revisions')
374 pprint.pprint(self.cursor.fetchall(), stream=fh)
375
376
377def _parse_history(d):
378 return (d[0], d[1], json.loads(d[2]), d[3],
379 datetime.datetime.strptime(d[-1], "%Y-%m-%dT%H:%M:%S.%f"))
380
381
382class HookData(object):
383 """Simple integration for existing hook exec frameworks.
384
385 Records all unit information, and stores deltas for processing
386 by the hook.
387
388 Sample::
389
390 from charmhelper.core import hookenv, unitdata
391
392 changes = unitdata.HookData()
393 db = unitdata.kv()
394 hooks = hookenv.Hooks()
395
396 @hooks.hook
397 def config_changed():
398 # View all changes to configuration
399 for changed, (prev, cur) in changes.conf.items():
400 print('config changed', changed,
401 'previous value', prev,
402 'current value', cur)
403
404 # Get some unit specific bookeeping
405 if not db.get('pkg_key'):
406 key = urllib.urlopen('https://example.com/pkg_key').read()
407 db.set('pkg_key', key)
408
409 if __name__ == '__main__':
410 with changes():
411 hook.execute()
412
413 """
414 def __init__(self):
415 self.kv = kv()
416 self.conf = None
417 self.rels = None
418
419 @contextlib.contextmanager
420 def __call__(self):
421 from charmhelpers.core import hookenv
422 hook_name = hookenv.hook_name()
423
424 with self.kv.hook_scope(hook_name):
425 self._record_charm_version(hookenv.charm_dir())
426 delta_config, delta_relation = self._record_hook(hookenv)
427 yield self.kv, delta_config, delta_relation
428
429 def _record_charm_version(self, charm_dir):
430 # Record revisions.. charm revisions are meaningless
431 # to charm authors as they don't control the revision.
432 # so logic dependnent on revision is not particularly
433 # useful, however it is useful for debugging analysis.
434 charm_rev = open(
435 os.path.join(charm_dir, 'revision')).read().strip()
436 charm_rev = charm_rev or '0'
437 revs = self.kv.get('charm_revisions', [])
438 if charm_rev not in revs:
439 revs.append(charm_rev.strip() or '0')
440 self.kv.set('charm_revisions', revs)
441
442 def _record_hook(self, hookenv):
443 data = hookenv.execution_environment()
444 self.conf = conf_delta = self.kv.delta(data['conf'], 'config')
445 self.rels = rels_delta = self.kv.delta(data['rels'], 'rels')
446 self.kv.set('env', data['env'])
447 self.kv.set('unit', data['unit'])
448 self.kv.set('relid', data.get('relid'))
449 return conf_delta, rels_delta
450
451
452class Record(dict):
453
454 __slots__ = ()
455
456 def __getattr__(self, k):
457 if k in self:
458 return self[k]
459 raise AttributeError(k)
460
461
462class DeltaSet(Record):
463
464 __slots__ = ()
465
466
467Delta = collections.namedtuple('Delta', ['previous', 'current'])
468
469
470_KV = None
471
472
473def kv():
474 global _KV
475 if _KV is None:
476 _KV = Storage()
477 return _KV
0478
=== modified file 'hooks/charmhelpers/fetch/archiveurl.py'
--- hooks/charmhelpers/fetch/archiveurl.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/fetch/archiveurl.py 2015-03-06 06:41:14 +0000
@@ -18,6 +18,16 @@
18import hashlib18import hashlib
19import re19import re
2020
21from charmhelpers.fetch import (
22 BaseFetchHandler,
23 UnhandledSource
24)
25from charmhelpers.payload.archive import (
26 get_archive_handler,
27 extract,
28)
29from charmhelpers.core.host import mkdir, check_hash
30
21import six31import six
22if six.PY3:32if six.PY3:
23 from urllib.request import (33 from urllib.request import (
@@ -35,16 +45,6 @@
35 )45 )
36 from urlparse import urlparse, urlunparse, parse_qs46 from urlparse import urlparse, urlunparse, parse_qs
3747
38from charmhelpers.fetch import (
39 BaseFetchHandler,
40 UnhandledSource
41)
42from charmhelpers.payload.archive import (
43 get_archive_handler,
44 extract,
45)
46from charmhelpers.core.host import mkdir, check_hash
47
4848
49def splituser(host):49def splituser(host):
50 '''urllib.splituser(), but six's support of this seems broken'''50 '''urllib.splituser(), but six's support of this seems broken'''
5151
=== modified file 'hooks/charmhelpers/fetch/giturl.py'
--- hooks/charmhelpers/fetch/giturl.py 2015-01-26 09:45:59 +0000
+++ hooks/charmhelpers/fetch/giturl.py 2015-03-06 06:41:14 +0000
@@ -32,7 +32,7 @@
32 apt_install("python-git")32 apt_install("python-git")
33 from git import Repo33 from git import Repo
3434
35from git.exc import GitCommandError35from git.exc import GitCommandError # noqa E402
3636
3737
38class GitUrlFetchHandler(BaseFetchHandler):38class GitUrlFetchHandler(BaseFetchHandler):
3939
=== modified file 'hooks/rabbit_utils.py'
--- hooks/rabbit_utils.py 2015-01-22 15:37:28 +0000
+++ hooks/rabbit_utils.py 2015-03-06 06:41:14 +0000
@@ -570,3 +570,11 @@
570 if svcs:570 if svcs:
571 _map.append((f, svcs))571 _map.append((f, svcs))
572 return OrderedDict(_map)572 return OrderedDict(_map)
573
574
575def services():
576 ''' Returns a list of services associate with this charm '''
577 _services = []
578 for v in restart_map().values():
579 _services = _services + v
580 return list(set(_services))
573581
=== modified file 'hooks/rabbitmq_server_relations.py'
--- hooks/rabbitmq_server_relations.py 2015-01-27 08:43:50 +0000
+++ hooks/rabbitmq_server_relations.py 2015-03-06 06:41:14 +0000
@@ -60,7 +60,7 @@
60 service_stop,60 service_stop,
61 service_restart,61 service_restart,
62)62)
63from charmhelpers.contrib.charmsupport.nrpe import NRPE63from charmhelpers.contrib.charmsupport import nrpe
64from charmhelpers.contrib.ssl.service import ServiceCA64from charmhelpers.contrib.ssl.service import ServiceCA
6565
66from charmhelpers.contrib.peerstorage import (66from charmhelpers.contrib.peerstorage import (
@@ -474,29 +474,20 @@
474 os.path.join(NAGIOS_PLUGINS, 'check_rabbitmq.py'))474 os.path.join(NAGIOS_PLUGINS, 'check_rabbitmq.py'))
475475
476 # Find out if nrpe set nagios_hostname476 # Find out if nrpe set nagios_hostname
477 hostname = None477 hostname = nrpe.get_nagios_hostname()
478 host_context = None478 myunit = nrpe.get_nagios_unit_name()
479 for rel in relations_of_type('nrpe-external-master'):479
480 if 'nagios_hostname' in rel:
481 hostname = rel['nagios_hostname']
482 host_context = rel['nagios_host_context']
483 break
484 # create unique user and vhost for each unit480 # create unique user and vhost for each unit
485 current_unit = local_unit().replace('/', '-')481 current_unit = local_unit().replace('/', '-')
486 user = 'nagios-%s' % current_unit482 user = 'nagios-%s' % current_unit
487 vhost = 'nagios-%s' % current_unit483 vhost = 'nagios-%s' % current_unit
488 password = rabbit.get_rabbit_password(user)484 password = rabbit.get_rabbit_password(user)
489485
490 if host_context:
491 myunit = "%s:%s" % (host_context, local_unit())
492 else:
493 myunit = local_unit()
494
495 rabbit.create_vhost(vhost)486 rabbit.create_vhost(vhost)
496 rabbit.create_user(user, password)487 rabbit.create_user(user, password)
497 rabbit.grant_permissions(user, vhost)488 rabbit.grant_permissions(user, vhost)
498489
499 nrpe_compat = NRPE(hostname=hostname)490 nrpe_compat = nrpe.NRPE(hostname=hostname)
500 nrpe_compat.add_check(491 nrpe_compat.add_check(
501 shortname=rabbit.RABBIT_USER,492 shortname=rabbit.RABBIT_USER,
502 description='Check RabbitMQ {%s}' % myunit,493 description='Check RabbitMQ {%s}' % myunit,

Subscribers

People subscribed via source and target branches