Merge lp:~brad-marshall/charms/trusty/mongodb/add-nrpe-checks into lp:charms/trusty/mongodb

Proposed by Brad Marshall
Status: Superseded
Proposed branch: lp:~brad-marshall/charms/trusty/mongodb/add-nrpe-checks
Merge into: lp:charms/trusty/mongodb
Diff against target: 2694 lines (+1553/-357)
33 files modified
charm-helpers-sync.yaml (+1/-0)
config.yaml (+11/-0)
files/nrpe-external-master/check_upstart_job (+72/-0)
hooks/charmhelpers/__init__.py (+16/-0)
hooks/charmhelpers/contrib/__init__.py (+15/-0)
hooks/charmhelpers/contrib/charmsupport/__init__.py (+15/-0)
hooks/charmhelpers/contrib/charmsupport/nrpe.py (+358/-0)
hooks/charmhelpers/contrib/charmsupport/volumes.py (+175/-0)
hooks/charmhelpers/contrib/hahelpers/__init__.py (+15/-0)
hooks/charmhelpers/contrib/hahelpers/cluster.py (+0/-248)
hooks/charmhelpers/contrib/python/__init__.py (+15/-0)
hooks/charmhelpers/contrib/python/packages.py (+0/-80)
hooks/charmhelpers/core/__init__.py (+15/-0)
hooks/charmhelpers/core/decorators.py (+16/-0)
hooks/charmhelpers/core/fstab.py (+19/-3)
hooks/charmhelpers/core/hookenv.py (+16/-0)
hooks/charmhelpers/core/host.py (+24/-5)
hooks/charmhelpers/core/services/__init__.py (+16/-0)
hooks/charmhelpers/core/services/base.py (+16/-0)
hooks/charmhelpers/core/services/helpers.py (+16/-0)
hooks/charmhelpers/core/strutils.py (+42/-0)
hooks/charmhelpers/core/sysctl.py (+28/-6)
hooks/charmhelpers/core/templating.py (+19/-3)
hooks/charmhelpers/core/unitdata.py (+477/-0)
hooks/charmhelpers/fetch/__init__.py (+16/-0)
hooks/charmhelpers/fetch/archiveurl.py (+26/-10)
hooks/charmhelpers/fetch/bzrurl.py (+25/-1)
hooks/charmhelpers/fetch/giturl.py (+20/-0)
hooks/charmhelpers/payload/__init__.py (+16/-0)
hooks/charmhelpers/payload/execd.py (+16/-0)
hooks/hooks.py (+33/-0)
metadata.yaml (+3/-0)
tests/03_deploy_replicaset.py (+1/-1)
To merge this branch: bzr merge lp:~brad-marshall/charms/trusty/mongodb/add-nrpe-checks
Reviewer Review Type Date Requested Status
Adam Israel (community) Needs Fixing
Review Queue (community) automated testing Needs Fixing
Liam Young Pending
Review via email: mp+241491@code.launchpad.net

This proposal has been superseded by a proposal from 2015-03-16.

Description of the change

Adds nrpe-external-master interface and adds basic nrpe checks.

To post a comment you must log in.
Revision history for this message
Ryan Beisner (1chb1n) wrote :

UOSCI bot says:
charm_lint_check #999 trusty-mongodb for brad-marshall mp241491
    LINT FAIL: lint-test missing

LINT Results (max last 5 lines):
INFO:root:Workspace dir: /var/lib/jenkins/workspace/charm_lint_check
INFO:root:Reading file: Makefile
INFO:root:Searching for: ['@flake8']
INFO:root:Search string not found in makefile target commands.
ERROR:root:No make target was executed.

Full lint test output: http://paste.ubuntu.com/8955813/
Build: http://10.98.191.181:8080/job/charm_lint_check/999/

Revision history for this message
Ryan Beisner (1chb1n) wrote :

UOSCI bot says:
charm_unit_test #834 trusty-mongodb for brad-marshall mp241491
    UNIT OK: passed

UNIT Results (max last 5 lines):
INFO:root:command: make -f Makefile unittest
  tests/10-unit.test
  test_success (test_write_log_rotate_config.TestWriteLogrotateConfigFile) ... ok
  Ran 1 test in 0.003s
  OK

Full unit test output: http://paste.ubuntu.com/8955816/
Build: http://10.98.191.181:8080/job/charm_unit_test/834/

Revision history for this message
Ryan Beisner (1chb1n) wrote :

UOSCI bot says:
charm_amulet_test #379 trusty-mongodb for brad-marshall mp241491
    AMULET FAIL: amulet-test missing

AMULET Results (max last 5 lines):
INFO:root:Workspace dir: /var/lib/jenkins/workspace/charm_amulet_test
INFO:root:Reading file: Makefile
INFO:root:Searching for: ['@juju test']
INFO:root:Search string not found in makefile target commands.
ERROR:root:No make target was executed.

Full amulet test output: http://paste.ubuntu.com/8955982/
Build: http://10.98.191.181:8080/job/charm_amulet_test/379/

57. By José Antonio Rey

Matt Bruzek 2014-09-23 Making the 00-setup file executable so it is run by juju test

58. By Charles Butler

[r=lazypower] <email address hidden> 2014-11-18 prevent unneeded package install

59. By Tim Van Steenburgh

Refactor tests

Revision history for this message
Review Queue (review-queue) wrote :

This items has failed automated testing! Results available here http://reports.vapour.ws/charm-tests/charm-bundle-test-10339-results

review: Needs Fixing (automated testing)
60. By Tim Van Steenburgh

[niedbalski] Clean up Makefile and test dirs

61. By Tim Van Steenburgh

[tvansteenburgh, r=niedbalski] Fixes regression/bug LP: #1400908

62. By Mario Splivalo

[mariosplivalo, r=niedbalski] Upgrades charmhelpers, and adds contrib.hahelpers.cluster, make lint fixes.

63. By Jorge Niedbalski

- Make sync after fixing LP: #1397134
- Fixes LP: #1401211

Revision history for this message
Adam Israel (aisrael) wrote :

Hi Brad,

I had the opportunity to review your merge proposal today. Thanks for all of your work on this so far.

I initially ran into a merge conflict with your branch against trunk, related to charm-helpers-sync.yaml. I resolved those to continue testing.

Of seven unit tests, make lint is failing:

hooks/hooks.py:1123:80: E501 line too long (92 > 79 characters)
hooks/hooks.py:1144:18: E251 unexpected spaces around keyword / parameter equals
hooks/hooks.py:1144:20: E251 unexpected spaces around keyword / parameter equals

Thanks again for your work on this! With those minor fixes, I don't see any other blockers to merging this proposal.

review: Needs Fixing
64. By Charles Butler

  [r=lazypower] dann frazier 2014-11-20 Use ppa:mongodb-arm64/ppa by default when running on arm64/trusty

65. By Matt Bruzek

[wes] Merge exec.d and volume relation_id patches from precise charm
[mbruzek] Fixed lint errors in hooks.py file so tests would pass.

66. By Mario Splivalo

[mariosplivalo, r=niedbalski,freyes] Fixes various replicaset issues bug LP: #1403698, LP: #1370542, LP: #1379604

unit tests Ok, amulet tests OK.

67. By Brad Marshall

[bradm] Sync charmhelpers, fix merge conflicts

68. By Brad Marshall

[bradm] Run pep8 and pyflakes over hooks.py

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #2065 mongodb for brad-marshall mp241491
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/2065/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #2276 mongodb for brad-marshall mp241491
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
  unit_tests/test_write_log_rotate_config.py:7:1: E402 module level import not at top of file
  make: *** [lint] Error 1

Full lint test output: http://paste.ubuntu.com/10398703/
Build: http://10.245.162.77:8080/job/charm_lint_check/2276/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #2181 mongodb for brad-marshall mp241491
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
  ERROR subprocess encountered error code 1
  make: *** [functional_test] Error 1

Full amulet test output: http://paste.ubuntu.com/10398847/
Build: http://10.245.162.77:8080/job/charm_amulet_test/2181/

69. By Brad Marshall

[bradm] Bump timeout to 30 minutes - about how long a test deploy took

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #2066 mongodb for brad-marshall mp241491
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
  E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
  make: *** [.venv] Error 100

Full unit test output: http://paste.ubuntu.com/10399876/
Build: http://10.245.162.77:8080/job/charm_unit_test/2066/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #2277 mongodb for brad-marshall mp241491
    LINT FAIL: lint-test failed
    LINT FAIL: charm-proof failed

LINT Results (max last 2 lines):
  E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
  make: *** [.venv] Error 100

Full lint test output: http://paste.ubuntu.com/10399877/
Build: http://10.245.162.77:8080/job/charm_lint_check/2277/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #2182 mongodb for brad-marshall mp241491
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/2182/

Revision history for this message
Brad Marshall (brad-marshall) wrote :

This should now be ready for a re-review

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #2403 mongodb for brad-marshall mp241491
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
  unit_tests/test_write_log_rotate_config.py:7:1: E402 module level import not at top of file
  make: *** [lint] Error 1

Full lint test output: http://paste.ubuntu.com/10520959/
Build: http://10.245.162.77:8080/job/charm_lint_check/2403/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #2193 mongodb for brad-marshall mp241491
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/2193/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #2322 mongodb for brad-marshall mp241491
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/2322/

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'charm-helpers-sync.yaml'
2--- charm-helpers-sync.yaml 2015-01-19 21:16:06 +0000
3+++ charm-helpers-sync.yaml 2015-02-25 01:49:51 +0000
4@@ -6,3 +6,4 @@
5 - contrib.hahelpers.cluster
6 - contrib.python.packages
7 - payload.execd
8+ - contrib.charmsupport
9
10=== modified file 'config.yaml'
11--- config.yaml 2015-01-21 21:47:18 +0000
12+++ config.yaml 2015-02-25 01:49:51 +0000
13@@ -215,3 +215,14 @@
14 description: >
15 Key ID to import to the apt keyring to support use with arbitary source
16 configuration from outside of Launchpad archives or PPA's.
17+ nagios_context:
18+ default: "juju"
19+ type: string
20+ description: |
21+ Used by the nrpe-external-master subordinate charm.
22+ A string that will be prepended to instance name to set the host name
23+ in nagios. So for instance the hostname would be something like:
24+ juju-myservice-0
25+ If you're running multiple environments with the same services in them
26+ this allows you to differentiate between them.
27+
28
29=== added directory 'files'
30=== added directory 'files/nrpe-external-master'
31=== added file 'files/nrpe-external-master/check_upstart_job'
32--- files/nrpe-external-master/check_upstart_job 1970-01-01 00:00:00 +0000
33+++ files/nrpe-external-master/check_upstart_job 2015-02-25 01:49:51 +0000
34@@ -0,0 +1,72 @@
35+#!/usr/bin/python
36+
37+#
38+# Copyright 2012, 2013 Canonical Ltd.
39+#
40+# Author: Paul Collins <paul.collins@canonical.com>
41+#
42+# Based on http://www.eurion.net/python-snippets/snippet/Upstart%20service%20status.html
43+#
44+
45+import sys
46+
47+import dbus
48+
49+
50+class Upstart(object):
51+ def __init__(self):
52+ self._bus = dbus.SystemBus()
53+ self._upstart = self._bus.get_object('com.ubuntu.Upstart',
54+ '/com/ubuntu/Upstart')
55+ def get_job(self, job_name):
56+ path = self._upstart.GetJobByName(job_name,
57+ dbus_interface='com.ubuntu.Upstart0_6')
58+ return self._bus.get_object('com.ubuntu.Upstart', path)
59+
60+ def get_properties(self, job):
61+ path = job.GetInstance([], dbus_interface='com.ubuntu.Upstart0_6.Job')
62+ instance = self._bus.get_object('com.ubuntu.Upstart', path)
63+ return instance.GetAll('com.ubuntu.Upstart0_6.Instance',
64+ dbus_interface=dbus.PROPERTIES_IFACE)
65+
66+ def get_job_instances(self, job_name):
67+ job = self.get_job(job_name)
68+ paths = job.GetAllInstances([], dbus_interface='com.ubuntu.Upstart0_6.Job')
69+ return [self._bus.get_object('com.ubuntu.Upstart', path) for path in paths]
70+
71+ def get_job_instance_properties(self, job):
72+ return job.GetAll('com.ubuntu.Upstart0_6.Instance',
73+ dbus_interface=dbus.PROPERTIES_IFACE)
74+
75+try:
76+ upstart = Upstart()
77+ try:
78+ job = upstart.get_job(sys.argv[1])
79+ props = upstart.get_properties(job)
80+
81+ if props['state'] == 'running':
82+ print 'OK: %s is running' % sys.argv[1]
83+ sys.exit(0)
84+ else:
85+ print 'CRITICAL: %s is not running' % sys.argv[1]
86+ sys.exit(2)
87+
88+ except dbus.DBusException as e:
89+ instances = upstart.get_job_instances(sys.argv[1])
90+ propses = [upstart.get_job_instance_properties(instance) for instance in instances]
91+ states = dict([(props['name'], props['state']) for props in propses])
92+ if len(states) != states.values().count('running'):
93+ not_running = []
94+ for name in states.keys():
95+ if states[name] != 'running':
96+ not_running.append(name)
97+ print 'CRITICAL: %d instances of %s not running: %s' % \
98+ (len(not_running), sys.argv[1], not_running.join(', '))
99+ sys.exit(2)
100+ else:
101+ print 'OK: %d instances of %s running' % (len(states), sys.argv[1])
102+
103+except dbus.DBusException as e:
104+ print 'CRITICAL: failed to get properties of \'%s\' from upstart' % sys.argv[1]
105+ sys.exit(2)
106+
107
108=== modified file 'hooks/charmhelpers/__init__.py'
109--- hooks/charmhelpers/__init__.py 2014-12-11 14:56:12 +0000
110+++ hooks/charmhelpers/__init__.py 2015-02-25 01:49:51 +0000
111@@ -1,3 +1,19 @@
112+# Copyright 2014-2015 Canonical Limited.
113+#
114+# This file is part of charm-helpers.
115+#
116+# charm-helpers is free software: you can redistribute it and/or modify
117+# it under the terms of the GNU Lesser General Public License version 3 as
118+# published by the Free Software Foundation.
119+#
120+# charm-helpers is distributed in the hope that it will be useful,
121+# but WITHOUT ANY WARRANTY; without even the implied warranty of
122+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
123+# GNU Lesser General Public License for more details.
124+#
125+# You should have received a copy of the GNU Lesser General Public License
126+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
127+
128 # Bootstrap charm-helpers, installing its dependencies if necessary using
129 # only standard libraries.
130 import subprocess
131
132=== added directory 'hooks/charmhelpers/contrib'
133=== removed directory 'hooks/charmhelpers/contrib'
134=== added file 'hooks/charmhelpers/contrib/__init__.py'
135--- hooks/charmhelpers/contrib/__init__.py 1970-01-01 00:00:00 +0000
136+++ hooks/charmhelpers/contrib/__init__.py 2015-02-25 01:49:51 +0000
137@@ -0,0 +1,15 @@
138+# Copyright 2014-2015 Canonical Limited.
139+#
140+# This file is part of charm-helpers.
141+#
142+# charm-helpers is free software: you can redistribute it and/or modify
143+# it under the terms of the GNU Lesser General Public License version 3 as
144+# published by the Free Software Foundation.
145+#
146+# charm-helpers is distributed in the hope that it will be useful,
147+# but WITHOUT ANY WARRANTY; without even the implied warranty of
148+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
149+# GNU Lesser General Public License for more details.
150+#
151+# You should have received a copy of the GNU Lesser General Public License
152+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
153
154=== removed file 'hooks/charmhelpers/contrib/__init__.py'
155=== added directory 'hooks/charmhelpers/contrib/charmsupport'
156=== added file 'hooks/charmhelpers/contrib/charmsupport/__init__.py'
157--- hooks/charmhelpers/contrib/charmsupport/__init__.py 1970-01-01 00:00:00 +0000
158+++ hooks/charmhelpers/contrib/charmsupport/__init__.py 2015-02-25 01:49:51 +0000
159@@ -0,0 +1,15 @@
160+# Copyright 2014-2015 Canonical Limited.
161+#
162+# This file is part of charm-helpers.
163+#
164+# charm-helpers is free software: you can redistribute it and/or modify
165+# it under the terms of the GNU Lesser General Public License version 3 as
166+# published by the Free Software Foundation.
167+#
168+# charm-helpers is distributed in the hope that it will be useful,
169+# but WITHOUT ANY WARRANTY; without even the implied warranty of
170+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
171+# GNU Lesser General Public License for more details.
172+#
173+# You should have received a copy of the GNU Lesser General Public License
174+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
175
176=== added file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py'
177--- hooks/charmhelpers/contrib/charmsupport/nrpe.py 1970-01-01 00:00:00 +0000
178+++ hooks/charmhelpers/contrib/charmsupport/nrpe.py 2015-02-25 01:49:51 +0000
179@@ -0,0 +1,358 @@
180+# Copyright 2014-2015 Canonical Limited.
181+#
182+# This file is part of charm-helpers.
183+#
184+# charm-helpers is free software: you can redistribute it and/or modify
185+# it under the terms of the GNU Lesser General Public License version 3 as
186+# published by the Free Software Foundation.
187+#
188+# charm-helpers is distributed in the hope that it will be useful,
189+# but WITHOUT ANY WARRANTY; without even the implied warranty of
190+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
191+# GNU Lesser General Public License for more details.
192+#
193+# You should have received a copy of the GNU Lesser General Public License
194+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
195+
196+"""Compatibility with the nrpe-external-master charm"""
197+# Copyright 2012 Canonical Ltd.
198+#
199+# Authors:
200+# Matthew Wedgwood <matthew.wedgwood@canonical.com>
201+
202+import subprocess
203+import pwd
204+import grp
205+import os
206+import glob
207+import shutil
208+import re
209+import shlex
210+import yaml
211+
212+from charmhelpers.core.hookenv import (
213+ config,
214+ local_unit,
215+ log,
216+ relation_ids,
217+ relation_set,
218+ relations_of_type,
219+)
220+
221+from charmhelpers.core.host import service
222+
223+# This module adds compatibility with the nrpe-external-master and plain nrpe
224+# subordinate charms. To use it in your charm:
225+#
226+# 1. Update metadata.yaml
227+#
228+# provides:
229+# (...)
230+# nrpe-external-master:
231+# interface: nrpe-external-master
232+# scope: container
233+#
234+# and/or
235+#
236+# provides:
237+# (...)
238+# local-monitors:
239+# interface: local-monitors
240+# scope: container
241+
242+#
243+# 2. Add the following to config.yaml
244+#
245+# nagios_context:
246+# default: "juju"
247+# type: string
248+# description: |
249+# Used by the nrpe subordinate charms.
250+# A string that will be prepended to instance name to set the host name
251+# in nagios. So for instance the hostname would be something like:
252+# juju-myservice-0
253+# If you're running multiple environments with the same services in them
254+# this allows you to differentiate between them.
255+# nagios_servicegroups:
256+# default: ""
257+# type: string
258+# description: |
259+# A comma-separated list of nagios servicegroups.
260+# If left empty, the nagios_context will be used as the servicegroup
261+#
262+# 3. Add custom checks (Nagios plugins) to files/nrpe-external-master
263+#
264+# 4. Update your hooks.py with something like this:
265+#
266+# from charmsupport.nrpe import NRPE
267+# (...)
268+# def update_nrpe_config():
269+# nrpe_compat = NRPE()
270+# nrpe_compat.add_check(
271+# shortname = "myservice",
272+# description = "Check MyService",
273+# check_cmd = "check_http -w 2 -c 10 http://localhost"
274+# )
275+# nrpe_compat.add_check(
276+# "myservice_other",
277+# "Check for widget failures",
278+# check_cmd = "/srv/myapp/scripts/widget_check"
279+# )
280+# nrpe_compat.write()
281+#
282+# def config_changed():
283+# (...)
284+# update_nrpe_config()
285+#
286+# def nrpe_external_master_relation_changed():
287+# update_nrpe_config()
288+#
289+# def local_monitors_relation_changed():
290+# update_nrpe_config()
291+#
292+# 5. ln -s hooks.py nrpe-external-master-relation-changed
293+# ln -s hooks.py local-monitors-relation-changed
294+
295+
296+class CheckException(Exception):
297+ pass
298+
299+
300+class Check(object):
301+ shortname_re = '[A-Za-z0-9-_]+$'
302+ service_template = ("""
303+#---------------------------------------------------
304+# This file is Juju managed
305+#---------------------------------------------------
306+define service {{
307+ use active-service
308+ host_name {nagios_hostname}
309+ service_description {nagios_hostname}[{shortname}] """
310+ """{description}
311+ check_command check_nrpe!{command}
312+ servicegroups {nagios_servicegroup}
313+}}
314+""")
315+
316+ def __init__(self, shortname, description, check_cmd):
317+ super(Check, self).__init__()
318+ # XXX: could be better to calculate this from the service name
319+ if not re.match(self.shortname_re, shortname):
320+ raise CheckException("shortname must match {}".format(
321+ Check.shortname_re))
322+ self.shortname = shortname
323+ self.command = "check_{}".format(shortname)
324+ # Note: a set of invalid characters is defined by the
325+ # Nagios server config
326+ # The default is: illegal_object_name_chars=`~!$%^&*"|'<>?,()=
327+ self.description = description
328+ self.check_cmd = self._locate_cmd(check_cmd)
329+
330+ def _locate_cmd(self, check_cmd):
331+ search_path = (
332+ '/usr/lib/nagios/plugins',
333+ '/usr/local/lib/nagios/plugins',
334+ )
335+ parts = shlex.split(check_cmd)
336+ for path in search_path:
337+ if os.path.exists(os.path.join(path, parts[0])):
338+ command = os.path.join(path, parts[0])
339+ if len(parts) > 1:
340+ command += " " + " ".join(parts[1:])
341+ return command
342+ log('Check command not found: {}'.format(parts[0]))
343+ return ''
344+
345+ def write(self, nagios_context, hostname, nagios_servicegroups):
346+ nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format(
347+ self.command)
348+ with open(nrpe_check_file, 'w') as nrpe_check_config:
349+ nrpe_check_config.write("# check {}\n".format(self.shortname))
350+ nrpe_check_config.write("command[{}]={}\n".format(
351+ self.command, self.check_cmd))
352+
353+ if not os.path.exists(NRPE.nagios_exportdir):
354+ log('Not writing service config as {} is not accessible'.format(
355+ NRPE.nagios_exportdir))
356+ else:
357+ self.write_service_config(nagios_context, hostname,
358+ nagios_servicegroups)
359+
360+ def write_service_config(self, nagios_context, hostname,
361+ nagios_servicegroups):
362+ for f in os.listdir(NRPE.nagios_exportdir):
363+ if re.search('.*{}.cfg'.format(self.command), f):
364+ os.remove(os.path.join(NRPE.nagios_exportdir, f))
365+
366+ templ_vars = {
367+ 'nagios_hostname': hostname,
368+ 'nagios_servicegroup': nagios_servicegroups,
369+ 'description': self.description,
370+ 'shortname': self.shortname,
371+ 'command': self.command,
372+ }
373+ nrpe_service_text = Check.service_template.format(**templ_vars)
374+ nrpe_service_file = '{}/service__{}_{}.cfg'.format(
375+ NRPE.nagios_exportdir, hostname, self.command)
376+ with open(nrpe_service_file, 'w') as nrpe_service_config:
377+ nrpe_service_config.write(str(nrpe_service_text))
378+
379+ def run(self):
380+ subprocess.call(self.check_cmd)
381+
382+
383+class NRPE(object):
384+ nagios_logdir = '/var/log/nagios'
385+ nagios_exportdir = '/var/lib/nagios/export'
386+ nrpe_confdir = '/etc/nagios/nrpe.d'
387+
388+ def __init__(self, hostname=None):
389+ super(NRPE, self).__init__()
390+ self.config = config()
391+ self.nagios_context = self.config['nagios_context']
392+ if 'nagios_servicegroups' in self.config and self.config['nagios_servicegroups']:
393+ self.nagios_servicegroups = self.config['nagios_servicegroups']
394+ else:
395+ self.nagios_servicegroups = self.nagios_context
396+ self.unit_name = local_unit().replace('/', '-')
397+ if hostname:
398+ self.hostname = hostname
399+ else:
400+ self.hostname = "{}-{}".format(self.nagios_context, self.unit_name)
401+ self.checks = []
402+
403+ def add_check(self, *args, **kwargs):
404+ self.checks.append(Check(*args, **kwargs))
405+
406+ def write(self):
407+ try:
408+ nagios_uid = pwd.getpwnam('nagios').pw_uid
409+ nagios_gid = grp.getgrnam('nagios').gr_gid
410+ except:
411+ log("Nagios user not set up, nrpe checks not updated")
412+ return
413+
414+ if not os.path.exists(NRPE.nagios_logdir):
415+ os.mkdir(NRPE.nagios_logdir)
416+ os.chown(NRPE.nagios_logdir, nagios_uid, nagios_gid)
417+
418+ nrpe_monitors = {}
419+ monitors = {"monitors": {"remote": {"nrpe": nrpe_monitors}}}
420+ for nrpecheck in self.checks:
421+ nrpecheck.write(self.nagios_context, self.hostname,
422+ self.nagios_servicegroups)
423+ nrpe_monitors[nrpecheck.shortname] = {
424+ "command": nrpecheck.command,
425+ }
426+
427+ service('restart', 'nagios-nrpe-server')
428+
429+ for rid in relation_ids("local-monitors"):
430+ relation_set(relation_id=rid, monitors=yaml.dump(monitors))
431+
432+
433+def get_nagios_hostcontext(relation_name='nrpe-external-master'):
434+ """
435+ Query relation with nrpe subordinate, return the nagios_host_context
436+
437+ :param str relation_name: Name of relation nrpe sub joined to
438+ """
439+ for rel in relations_of_type(relation_name):
440+ if 'nagios_hostname' in rel:
441+ return rel['nagios_host_context']
442+
443+
444+def get_nagios_hostname(relation_name='nrpe-external-master'):
445+ """
446+ Query relation with nrpe subordinate, return the nagios_hostname
447+
448+ :param str relation_name: Name of relation nrpe sub joined to
449+ """
450+ for rel in relations_of_type(relation_name):
451+ if 'nagios_hostname' in rel:
452+ return rel['nagios_hostname']
453+
454+
455+def get_nagios_unit_name(relation_name='nrpe-external-master'):
456+ """
457+ Return the nagios unit name prepended with host_context if needed
458+
459+ :param str relation_name: Name of relation nrpe sub joined to
460+ """
461+ host_context = get_nagios_hostcontext(relation_name)
462+ if host_context:
463+ unit = "%s:%s" % (host_context, local_unit())
464+ else:
465+ unit = local_unit()
466+ return unit
467+
468+
469+def add_init_service_checks(nrpe, services, unit_name):
470+ """
471+ Add checks for each service in list
472+
473+ :param NRPE nrpe: NRPE object to add check to
474+ :param list services: List of services to check
475+ :param str unit_name: Unit name to use in check description
476+ """
477+ for svc in services:
478+ upstart_init = '/etc/init/%s.conf' % svc
479+ sysv_init = '/etc/init.d/%s' % svc
480+ if os.path.exists(upstart_init):
481+ nrpe.add_check(
482+ shortname=svc,
483+ description='process check {%s}' % unit_name,
484+ check_cmd='check_upstart_job %s' % svc
485+ )
486+ elif os.path.exists(sysv_init):
487+ cronpath = '/etc/cron.d/nagios-service-check-%s' % svc
488+ cron_file = ('*/5 * * * * root '
489+ '/usr/local/lib/nagios/plugins/check_exit_status.pl '
490+ '-s /etc/init.d/%s status > '
491+ '/var/lib/nagios/service-check-%s.txt\n' % (svc,
492+ svc)
493+ )
494+ f = open(cronpath, 'w')
495+ f.write(cron_file)
496+ f.close()
497+ nrpe.add_check(
498+ shortname=svc,
499+ description='process check {%s}' % unit_name,
500+ check_cmd='check_status_file.py -f '
501+ '/var/lib/nagios/service-check-%s.txt' % svc,
502+ )
503+
504+
505+def copy_nrpe_checks():
506+ """
507+ Copy the nrpe checks into place
508+
509+ """
510+ NAGIOS_PLUGINS = '/usr/local/lib/nagios/plugins'
511+ nrpe_files_dir = os.path.join(os.getenv('CHARM_DIR'), 'hooks',
512+ 'charmhelpers', 'contrib', 'openstack',
513+ 'files')
514+
515+ if not os.path.exists(NAGIOS_PLUGINS):
516+ os.makedirs(NAGIOS_PLUGINS)
517+ for fname in glob.glob(os.path.join(nrpe_files_dir, "check_*")):
518+ if os.path.isfile(fname):
519+ shutil.copy2(fname,
520+ os.path.join(NAGIOS_PLUGINS, os.path.basename(fname)))
521+
522+
523+def add_haproxy_checks(nrpe, unit_name):
524+ """
525+ Add checks for each service in list
526+
527+ :param NRPE nrpe: NRPE object to add check to
528+ :param str unit_name: Unit name to use in check description
529+ """
530+ nrpe.add_check(
531+ shortname='haproxy_servers',
532+ description='Check HAProxy {%s}' % unit_name,
533+ check_cmd='check_haproxy.sh')
534+ nrpe.add_check(
535+ shortname='haproxy_queue',
536+ description='Check HAProxy queue depth {%s}' % unit_name,
537+ check_cmd='check_haproxy_queue_depth.sh')
538
539=== added file 'hooks/charmhelpers/contrib/charmsupport/volumes.py'
540--- hooks/charmhelpers/contrib/charmsupport/volumes.py 1970-01-01 00:00:00 +0000
541+++ hooks/charmhelpers/contrib/charmsupport/volumes.py 2015-02-25 01:49:51 +0000
542@@ -0,0 +1,175 @@
543+# Copyright 2014-2015 Canonical Limited.
544+#
545+# This file is part of charm-helpers.
546+#
547+# charm-helpers is free software: you can redistribute it and/or modify
548+# it under the terms of the GNU Lesser General Public License version 3 as
549+# published by the Free Software Foundation.
550+#
551+# charm-helpers is distributed in the hope that it will be useful,
552+# but WITHOUT ANY WARRANTY; without even the implied warranty of
553+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
554+# GNU Lesser General Public License for more details.
555+#
556+# You should have received a copy of the GNU Lesser General Public License
557+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
558+
559+'''
560+Functions for managing volumes in juju units. One volume is supported per unit.
561+Subordinates may have their own storage, provided it is on its own partition.
562+
563+Configuration stanzas::
564+
565+ volume-ephemeral:
566+ type: boolean
567+ default: true
568+ description: >
569+ If false, a volume is mounted as sepecified in "volume-map"
570+ If true, ephemeral storage will be used, meaning that log data
571+ will only exist as long as the machine. YOU HAVE BEEN WARNED.
572+ volume-map:
573+ type: string
574+ default: {}
575+ description: >
576+ YAML map of units to device names, e.g:
577+ "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }"
578+ Service units will raise a configure-error if volume-ephemeral
579+ is 'true' and no volume-map value is set. Use 'juju set' to set a
580+ value and 'juju resolved' to complete configuration.
581+
582+Usage::
583+
584+ from charmsupport.volumes import configure_volume, VolumeConfigurationError
585+ from charmsupport.hookenv import log, ERROR
586+ def post_mount_hook():
587+ stop_service('myservice')
588+ def post_mount_hook():
589+ start_service('myservice')
590+
591+ if __name__ == '__main__':
592+ try:
593+ configure_volume(before_change=pre_mount_hook,
594+ after_change=post_mount_hook)
595+ except VolumeConfigurationError:
596+ log('Storage could not be configured', ERROR)
597+
598+'''
599+
600+# XXX: Known limitations
601+# - fstab is neither consulted nor updated
602+
603+import os
604+from charmhelpers.core import hookenv
605+from charmhelpers.core import host
606+import yaml
607+
608+
609+MOUNT_BASE = '/srv/juju/volumes'
610+
611+
612+class VolumeConfigurationError(Exception):
613+ '''Volume configuration data is missing or invalid'''
614+ pass
615+
616+
617+def get_config():
618+ '''Gather and sanity-check volume configuration data'''
619+ volume_config = {}
620+ config = hookenv.config()
621+
622+ errors = False
623+
624+ if config.get('volume-ephemeral') in (True, 'True', 'true', 'Yes', 'yes'):
625+ volume_config['ephemeral'] = True
626+ else:
627+ volume_config['ephemeral'] = False
628+
629+ try:
630+ volume_map = yaml.safe_load(config.get('volume-map', '{}'))
631+ except yaml.YAMLError as e:
632+ hookenv.log("Error parsing YAML volume-map: {}".format(e),
633+ hookenv.ERROR)
634+ errors = True
635+ if volume_map is None:
636+ # probably an empty string
637+ volume_map = {}
638+ elif not isinstance(volume_map, dict):
639+ hookenv.log("Volume-map should be a dictionary, not {}".format(
640+ type(volume_map)))
641+ errors = True
642+
643+ volume_config['device'] = volume_map.get(os.environ['JUJU_UNIT_NAME'])
644+ if volume_config['device'] and volume_config['ephemeral']:
645+ # asked for ephemeral storage but also defined a volume ID
646+ hookenv.log('A volume is defined for this unit, but ephemeral '
647+ 'storage was requested', hookenv.ERROR)
648+ errors = True
649+ elif not volume_config['device'] and not volume_config['ephemeral']:
650+ # asked for permanent storage but did not define volume ID
651+ hookenv.log('Ephemeral storage was requested, but there is no volume '
652+ 'defined for this unit.', hookenv.ERROR)
653+ errors = True
654+
655+ unit_mount_name = hookenv.local_unit().replace('/', '-')
656+ volume_config['mountpoint'] = os.path.join(MOUNT_BASE, unit_mount_name)
657+
658+ if errors:
659+ return None
660+ return volume_config
661+
662+
663+def mount_volume(config):
664+ if os.path.exists(config['mountpoint']):
665+ if not os.path.isdir(config['mountpoint']):
666+ hookenv.log('Not a directory: {}'.format(config['mountpoint']))
667+ raise VolumeConfigurationError()
668+ else:
669+ host.mkdir(config['mountpoint'])
670+ if os.path.ismount(config['mountpoint']):
671+ unmount_volume(config)
672+ if not host.mount(config['device'], config['mountpoint'], persist=True):
673+ raise VolumeConfigurationError()
674+
675+
676+def unmount_volume(config):
677+ if os.path.ismount(config['mountpoint']):
678+ if not host.umount(config['mountpoint'], persist=True):
679+ raise VolumeConfigurationError()
680+
681+
682+def managed_mounts():
683+ '''List of all mounted managed volumes'''
684+ return filter(lambda mount: mount[0].startswith(MOUNT_BASE), host.mounts())
685+
686+
687+def configure_volume(before_change=lambda: None, after_change=lambda: None):
688+ '''Set up storage (or don't) according to the charm's volume configuration.
689+ Returns the mount point or "ephemeral". before_change and after_change
690+ are optional functions to be called if the volume configuration changes.
691+ '''
692+
693+ config = get_config()
694+ if not config:
695+ hookenv.log('Failed to read volume configuration', hookenv.CRITICAL)
696+ raise VolumeConfigurationError()
697+
698+ if config['ephemeral']:
699+ if os.path.ismount(config['mountpoint']):
700+ before_change()
701+ unmount_volume(config)
702+ after_change()
703+ return 'ephemeral'
704+ else:
705+ # persistent storage
706+ if os.path.ismount(config['mountpoint']):
707+ mounts = dict(managed_mounts())
708+ if mounts.get(config['mountpoint']) != config['device']:
709+ before_change()
710+ unmount_volume(config)
711+ mount_volume(config)
712+ after_change()
713+ else:
714+ before_change()
715+ mount_volume(config)
716+ after_change()
717+ return config['mountpoint']
718
719=== added directory 'hooks/charmhelpers/contrib/hahelpers'
720=== removed directory 'hooks/charmhelpers/contrib/hahelpers'
721=== added file 'hooks/charmhelpers/contrib/hahelpers/__init__.py'
722--- hooks/charmhelpers/contrib/hahelpers/__init__.py 1970-01-01 00:00:00 +0000
723+++ hooks/charmhelpers/contrib/hahelpers/__init__.py 2015-02-25 01:49:51 +0000
724@@ -0,0 +1,15 @@
725+# Copyright 2014-2015 Canonical Limited.
726+#
727+# This file is part of charm-helpers.
728+#
729+# charm-helpers is free software: you can redistribute it and/or modify
730+# it under the terms of the GNU Lesser General Public License version 3 as
731+# published by the Free Software Foundation.
732+#
733+# charm-helpers is distributed in the hope that it will be useful,
734+# but WITHOUT ANY WARRANTY; without even the implied warranty of
735+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
736+# GNU Lesser General Public License for more details.
737+#
738+# You should have received a copy of the GNU Lesser General Public License
739+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
740
741=== removed file 'hooks/charmhelpers/contrib/hahelpers/__init__.py'
742=== added file 'hooks/charmhelpers/contrib/hahelpers/cluster.py'
743--- hooks/charmhelpers/contrib/hahelpers/cluster.py 1970-01-01 00:00:00 +0000
744+++ hooks/charmhelpers/contrib/hahelpers/cluster.py 2015-02-25 01:49:51 +0000
745@@ -0,0 +1,272 @@
746+# Copyright 2014-2015 Canonical Limited.
747+#
748+# This file is part of charm-helpers.
749+#
750+# charm-helpers is free software: you can redistribute it and/or modify
751+# it under the terms of the GNU Lesser General Public License version 3 as
752+# published by the Free Software Foundation.
753+#
754+# charm-helpers is distributed in the hope that it will be useful,
755+# but WITHOUT ANY WARRANTY; without even the implied warranty of
756+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
757+# GNU Lesser General Public License for more details.
758+#
759+# You should have received a copy of the GNU Lesser General Public License
760+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
761+
762+#
763+# Copyright 2012 Canonical Ltd.
764+#
765+# Authors:
766+# James Page <james.page@ubuntu.com>
767+# Adam Gandelman <adamg@ubuntu.com>
768+#
769+
770+"""
771+Helpers for clustering and determining "cluster leadership" and other
772+clustering-related helpers.
773+"""
774+
775+import subprocess
776+import os
777+
778+from socket import gethostname as get_unit_hostname
779+
780+import six
781+
782+from charmhelpers.core.hookenv import (
783+ log,
784+ relation_ids,
785+ related_units as relation_list,
786+ relation_get,
787+ config as config_get,
788+ INFO,
789+ ERROR,
790+ WARNING,
791+ unit_get,
792+)
793+from charmhelpers.core.decorators import (
794+ retry_on_exception,
795+)
796+from charmhelpers.core.strutils import (
797+ bool_from_string,
798+)
799+
800+
801+class HAIncompleteConfig(Exception):
802+ pass
803+
804+
805+class CRMResourceNotFound(Exception):
806+ pass
807+
808+
809+def is_elected_leader(resource):
810+ """
811+ Returns True if the charm executing this is the elected cluster leader.
812+
813+ It relies on two mechanisms to determine leadership:
814+ 1. If the charm is part of a corosync cluster, call corosync to
815+ determine leadership.
816+ 2. If the charm is not part of a corosync cluster, the leader is
817+ determined as being "the alive unit with the lowest unit numer". In
818+ other words, the oldest surviving unit.
819+ """
820+ if is_clustered():
821+ if not is_crm_leader(resource):
822+ log('Deferring action to CRM leader.', level=INFO)
823+ return False
824+ else:
825+ peers = peer_units()
826+ if peers and not oldest_peer(peers):
827+ log('Deferring action to oldest service unit.', level=INFO)
828+ return False
829+ return True
830+
831+
832+def is_clustered():
833+ for r_id in (relation_ids('ha') or []):
834+ for unit in (relation_list(r_id) or []):
835+ clustered = relation_get('clustered',
836+ rid=r_id,
837+ unit=unit)
838+ if clustered:
839+ return True
840+ return False
841+
842+
843+@retry_on_exception(5, base_delay=2, exc_type=CRMResourceNotFound)
844+def is_crm_leader(resource, retry=False):
845+ """
846+ Returns True if the charm calling this is the elected corosync leader,
847+ as returned by calling the external "crm" command.
848+
849+ We allow this operation to be retried to avoid the possibility of getting a
850+ false negative. See LP #1396246 for more info.
851+ """
852+ cmd = ['crm', 'resource', 'show', resource]
853+ try:
854+ status = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
855+ if not isinstance(status, six.text_type):
856+ status = six.text_type(status, "utf-8")
857+ except subprocess.CalledProcessError:
858+ status = None
859+
860+ if status and get_unit_hostname() in status:
861+ return True
862+
863+ if status and "resource %s is NOT running" % (resource) in status:
864+ raise CRMResourceNotFound("CRM resource %s not found" % (resource))
865+
866+ return False
867+
868+
869+def is_leader(resource):
870+ log("is_leader is deprecated. Please consider using is_crm_leader "
871+ "instead.", level=WARNING)
872+ return is_crm_leader(resource)
873+
874+
875+def peer_units(peer_relation="cluster"):
876+ peers = []
877+ for r_id in (relation_ids(peer_relation) or []):
878+ for unit in (relation_list(r_id) or []):
879+ peers.append(unit)
880+ return peers
881+
882+
883+def peer_ips(peer_relation='cluster', addr_key='private-address'):
884+ '''Return a dict of peers and their private-address'''
885+ peers = {}
886+ for r_id in relation_ids(peer_relation):
887+ for unit in relation_list(r_id):
888+ peers[unit] = relation_get(addr_key, rid=r_id, unit=unit)
889+ return peers
890+
891+
892+def oldest_peer(peers):
893+ """Determines who the oldest peer is by comparing unit numbers."""
894+ local_unit_no = int(os.getenv('JUJU_UNIT_NAME').split('/')[1])
895+ for peer in peers:
896+ remote_unit_no = int(peer.split('/')[1])
897+ if remote_unit_no < local_unit_no:
898+ return False
899+ return True
900+
901+
902+def eligible_leader(resource):
903+ log("eligible_leader is deprecated. Please consider using "
904+ "is_elected_leader instead.", level=WARNING)
905+ return is_elected_leader(resource)
906+
907+
908+def https():
909+ '''
910+ Determines whether enough data has been provided in configuration
911+ or relation data to configure HTTPS
912+ .
913+ returns: boolean
914+ '''
915+ use_https = config_get('use-https')
916+ if use_https and bool_from_string(use_https):
917+ return True
918+ if config_get('ssl_cert') and config_get('ssl_key'):
919+ return True
920+ for r_id in relation_ids('identity-service'):
921+ for unit in relation_list(r_id):
922+ # TODO - needs fixing for new helper as ssl_cert/key suffixes with CN
923+ rel_state = [
924+ relation_get('https_keystone', rid=r_id, unit=unit),
925+ relation_get('ca_cert', rid=r_id, unit=unit),
926+ ]
927+ # NOTE: works around (LP: #1203241)
928+ if (None not in rel_state) and ('' not in rel_state):
929+ return True
930+ return False
931+
932+
933+def determine_api_port(public_port, singlenode_mode=False):
934+ '''
935+ Determine correct API server listening port based on
936+ existence of HTTPS reverse proxy and/or haproxy.
937+
938+ public_port: int: standard public port for given service
939+
940+ singlenode_mode: boolean: Shuffle ports when only a single unit is present
941+
942+ returns: int: the correct listening port for the API service
943+ '''
944+ i = 0
945+ if singlenode_mode:
946+ i += 1
947+ elif len(peer_units()) > 0 or is_clustered():
948+ i += 1
949+ if https():
950+ i += 1
951+ return public_port - (i * 10)
952+
953+
954+def determine_apache_port(public_port, singlenode_mode=False):
955+ '''
956+ Description: Determine correct apache listening port based on public IP +
957+ state of the cluster.
958+
959+ public_port: int: standard public port for given service
960+
961+ singlenode_mode: boolean: Shuffle ports when only a single unit is present
962+
963+ returns: int: the correct listening port for the HAProxy service
964+ '''
965+ i = 0
966+ if singlenode_mode:
967+ i += 1
968+ elif len(peer_units()) > 0 or is_clustered():
969+ i += 1
970+ return public_port - (i * 10)
971+
972+
973+def get_hacluster_config(exclude_keys=None):
974+ '''
975+ Obtains all relevant configuration from charm configuration required
976+ for initiating a relation to hacluster:
977+
978+ ha-bindiface, ha-mcastport, vip
979+
980+ param: exclude_keys: list of setting key(s) to be excluded.
981+ returns: dict: A dict containing settings keyed by setting name.
982+ raises: HAIncompleteConfig if settings are missing.
983+ '''
984+ settings = ['ha-bindiface', 'ha-mcastport', 'vip']
985+ conf = {}
986+ for setting in settings:
987+ if exclude_keys and setting in exclude_keys:
988+ continue
989+
990+ conf[setting] = config_get(setting)
991+ missing = []
992+ [missing.append(s) for s, v in six.iteritems(conf) if v is None]
993+ if missing:
994+ log('Insufficient config data to configure hacluster.', level=ERROR)
995+ raise HAIncompleteConfig
996+ return conf
997+
998+
999+def canonical_url(configs, vip_setting='vip'):
1000+ '''
1001+ Returns the correct HTTP URL to this host given the state of HTTPS
1002+ configuration and hacluster.
1003+
1004+ :configs : OSTemplateRenderer: A config tempating object to inspect for
1005+ a complete https context.
1006+
1007+ :vip_setting: str: Setting in charm config that specifies
1008+ VIP address.
1009+ '''
1010+ scheme = 'http'
1011+ if 'https' in configs.complete_contexts():
1012+ scheme = 'https'
1013+ if is_clustered():
1014+ addr = config_get(vip_setting)
1015+ else:
1016+ addr = unit_get('private-address')
1017+ return '%s://%s' % (scheme, addr)
1018
1019=== removed file 'hooks/charmhelpers/contrib/hahelpers/cluster.py'
1020--- hooks/charmhelpers/contrib/hahelpers/cluster.py 2015-01-19 21:16:06 +0000
1021+++ hooks/charmhelpers/contrib/hahelpers/cluster.py 1970-01-01 00:00:00 +0000
1022@@ -1,248 +0,0 @@
1023-#
1024-# Copyright 2012 Canonical Ltd.
1025-#
1026-# Authors:
1027-# James Page <james.page@ubuntu.com>
1028-# Adam Gandelman <adamg@ubuntu.com>
1029-#
1030-
1031-"""
1032-Helpers for clustering and determining "cluster leadership" and other
1033-clustering-related helpers.
1034-"""
1035-
1036-import subprocess
1037-import os
1038-
1039-from socket import gethostname as get_unit_hostname
1040-
1041-import six
1042-
1043-from charmhelpers.core.hookenv import (
1044- log,
1045- relation_ids,
1046- related_units as relation_list,
1047- relation_get,
1048- config as config_get,
1049- INFO,
1050- ERROR,
1051- WARNING,
1052- unit_get,
1053-)
1054-from charmhelpers.core.decorators import (
1055- retry_on_exception,
1056-)
1057-
1058-
1059-class HAIncompleteConfig(Exception):
1060- pass
1061-
1062-
1063-class CRMResourceNotFound(Exception):
1064- pass
1065-
1066-
1067-def is_elected_leader(resource):
1068- """
1069- Returns True if the charm executing this is the elected cluster leader.
1070-
1071- It relies on two mechanisms to determine leadership:
1072- 1. If the charm is part of a corosync cluster, call corosync to
1073- determine leadership.
1074- 2. If the charm is not part of a corosync cluster, the leader is
1075- determined as being "the alive unit with the lowest unit numer". In
1076- other words, the oldest surviving unit.
1077- """
1078- if is_clustered():
1079- if not is_crm_leader(resource):
1080- log('Deferring action to CRM leader.', level=INFO)
1081- return False
1082- else:
1083- peers = peer_units()
1084- if peers and not oldest_peer(peers):
1085- log('Deferring action to oldest service unit.', level=INFO)
1086- return False
1087- return True
1088-
1089-
1090-def is_clustered():
1091- for r_id in (relation_ids('ha') or []):
1092- for unit in (relation_list(r_id) or []):
1093- clustered = relation_get('clustered',
1094- rid=r_id,
1095- unit=unit)
1096- if clustered:
1097- return True
1098- return False
1099-
1100-
1101-@retry_on_exception(5, base_delay=2, exc_type=CRMResourceNotFound)
1102-def is_crm_leader(resource, retry=False):
1103- """
1104- Returns True if the charm calling this is the elected corosync leader,
1105- as returned by calling the external "crm" command.
1106-
1107- We allow this operation to be retried to avoid the possibility of getting a
1108- false negative. See LP #1396246 for more info.
1109- """
1110- cmd = ['crm', 'resource', 'show', resource]
1111- try:
1112- status = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
1113- if not isinstance(status, six.text_type):
1114- status = six.text_type(status, "utf-8")
1115- except subprocess.CalledProcessError:
1116- status = None
1117-
1118- if status and get_unit_hostname() in status:
1119- return True
1120-
1121- if status and "resource %s is NOT running" % (resource) in status:
1122- raise CRMResourceNotFound("CRM resource %s not found" % (resource))
1123-
1124- return False
1125-
1126-
1127-def is_leader(resource):
1128- log("is_leader is deprecated. Please consider using is_crm_leader "
1129- "instead.", level=WARNING)
1130- return is_crm_leader(resource)
1131-
1132-
1133-def peer_units(peer_relation="cluster"):
1134- peers = []
1135- for r_id in (relation_ids(peer_relation) or []):
1136- for unit in (relation_list(r_id) or []):
1137- peers.append(unit)
1138- return peers
1139-
1140-
1141-def peer_ips(peer_relation='cluster', addr_key='private-address'):
1142- '''Return a dict of peers and their private-address'''
1143- peers = {}
1144- for r_id in relation_ids(peer_relation):
1145- for unit in relation_list(r_id):
1146- peers[unit] = relation_get(addr_key, rid=r_id, unit=unit)
1147- return peers
1148-
1149-
1150-def oldest_peer(peers):
1151- """Determines who the oldest peer is by comparing unit numbers."""
1152- local_unit_no = int(os.getenv('JUJU_UNIT_NAME').split('/')[1])
1153- for peer in peers:
1154- remote_unit_no = int(peer.split('/')[1])
1155- if remote_unit_no < local_unit_no:
1156- return False
1157- return True
1158-
1159-
1160-def eligible_leader(resource):
1161- log("eligible_leader is deprecated. Please consider using "
1162- "is_elected_leader instead.", level=WARNING)
1163- return is_elected_leader(resource)
1164-
1165-
1166-def https():
1167- '''
1168- Determines whether enough data has been provided in configuration
1169- or relation data to configure HTTPS
1170- .
1171- returns: boolean
1172- '''
1173- if config_get('use-https') == "yes":
1174- return True
1175- if config_get('ssl_cert') and config_get('ssl_key'):
1176- return True
1177- for r_id in relation_ids('identity-service'):
1178- for unit in relation_list(r_id):
1179- # TODO - needs fixing for new helper as ssl_cert/key suffixes with CN
1180- rel_state = [
1181- relation_get('https_keystone', rid=r_id, unit=unit),
1182- relation_get('ca_cert', rid=r_id, unit=unit),
1183- ]
1184- # NOTE: works around (LP: #1203241)
1185- if (None not in rel_state) and ('' not in rel_state):
1186- return True
1187- return False
1188-
1189-
1190-def determine_api_port(public_port, singlenode_mode=False):
1191- '''
1192- Determine correct API server listening port based on
1193- existence of HTTPS reverse proxy and/or haproxy.
1194-
1195- public_port: int: standard public port for given service
1196-
1197- singlenode_mode: boolean: Shuffle ports when only a single unit is present
1198-
1199- returns: int: the correct listening port for the API service
1200- '''
1201- i = 0
1202- if singlenode_mode:
1203- i += 1
1204- elif len(peer_units()) > 0 or is_clustered():
1205- i += 1
1206- if https():
1207- i += 1
1208- return public_port - (i * 10)
1209-
1210-
1211-def determine_apache_port(public_port, singlenode_mode=False):
1212- '''
1213- Description: Determine correct apache listening port based on public IP +
1214- state of the cluster.
1215-
1216- public_port: int: standard public port for given service
1217-
1218- singlenode_mode: boolean: Shuffle ports when only a single unit is present
1219-
1220- returns: int: the correct listening port for the HAProxy service
1221- '''
1222- i = 0
1223- if singlenode_mode:
1224- i += 1
1225- elif len(peer_units()) > 0 or is_clustered():
1226- i += 1
1227- return public_port - (i * 10)
1228-
1229-
1230-def get_hacluster_config():
1231- '''
1232- Obtains all relevant configuration from charm configuration required
1233- for initiating a relation to hacluster:
1234-
1235- ha-bindiface, ha-mcastport, vip
1236-
1237- returns: dict: A dict containing settings keyed by setting name.
1238- raises: HAIncompleteConfig if settings are missing.
1239- '''
1240- settings = ['ha-bindiface', 'ha-mcastport', 'vip']
1241- conf = {}
1242- for setting in settings:
1243- conf[setting] = config_get(setting)
1244- missing = []
1245- [missing.append(s) for s, v in six.iteritems(conf) if v is None]
1246- if missing:
1247- log('Insufficient config data to configure hacluster.', level=ERROR)
1248- raise HAIncompleteConfig
1249- return conf
1250-
1251-
1252-def canonical_url(configs, vip_setting='vip'):
1253- '''
1254- Returns the correct HTTP URL to this host given the state of HTTPS
1255- configuration and hacluster.
1256-
1257- :configs : OSTemplateRenderer: A config tempating object to inspect for
1258- a complete https context.
1259-
1260- :vip_setting: str: Setting in charm config that specifies
1261- VIP address.
1262- '''
1263- scheme = 'http'
1264- if 'https' in configs.complete_contexts():
1265- scheme = 'https'
1266- if is_clustered():
1267- addr = config_get(vip_setting)
1268- else:
1269- addr = unit_get('private-address')
1270- return '%s://%s' % (scheme, addr)
1271
1272=== added directory 'hooks/charmhelpers/contrib/python'
1273=== removed directory 'hooks/charmhelpers/contrib/python'
1274=== added file 'hooks/charmhelpers/contrib/python/__init__.py'
1275--- hooks/charmhelpers/contrib/python/__init__.py 1970-01-01 00:00:00 +0000
1276+++ hooks/charmhelpers/contrib/python/__init__.py 2015-02-25 01:49:51 +0000
1277@@ -0,0 +1,15 @@
1278+# Copyright 2014-2015 Canonical Limited.
1279+#
1280+# This file is part of charm-helpers.
1281+#
1282+# charm-helpers is free software: you can redistribute it and/or modify
1283+# it under the terms of the GNU Lesser General Public License version 3 as
1284+# published by the Free Software Foundation.
1285+#
1286+# charm-helpers is distributed in the hope that it will be useful,
1287+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1288+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1289+# GNU Lesser General Public License for more details.
1290+#
1291+# You should have received a copy of the GNU Lesser General Public License
1292+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1293
1294=== removed file 'hooks/charmhelpers/contrib/python/__init__.py'
1295=== added file 'hooks/charmhelpers/contrib/python/packages.py'
1296--- hooks/charmhelpers/contrib/python/packages.py 1970-01-01 00:00:00 +0000
1297+++ hooks/charmhelpers/contrib/python/packages.py 2015-02-25 01:49:51 +0000
1298@@ -0,0 +1,96 @@
1299+#!/usr/bin/env python
1300+# coding: utf-8
1301+
1302+# Copyright 2014-2015 Canonical Limited.
1303+#
1304+# This file is part of charm-helpers.
1305+#
1306+# charm-helpers is free software: you can redistribute it and/or modify
1307+# it under the terms of the GNU Lesser General Public License version 3 as
1308+# published by the Free Software Foundation.
1309+#
1310+# charm-helpers is distributed in the hope that it will be useful,
1311+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1312+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1313+# GNU Lesser General Public License for more details.
1314+#
1315+# You should have received a copy of the GNU Lesser General Public License
1316+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1317+
1318+from charmhelpers.fetch import apt_install, apt_update
1319+from charmhelpers.core.hookenv import log
1320+
1321+try:
1322+ from pip import main as pip_execute
1323+except ImportError:
1324+ apt_update()
1325+ apt_install('python-pip')
1326+ from pip import main as pip_execute
1327+
1328+__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
1329+
1330+
1331+def parse_options(given, available):
1332+ """Given a set of options, check if available"""
1333+ for key, value in sorted(given.items()):
1334+ if key in available:
1335+ yield "--{0}={1}".format(key, value)
1336+
1337+
1338+def pip_install_requirements(requirements, **options):
1339+ """Install a requirements file """
1340+ command = ["install"]
1341+
1342+ available_options = ('proxy', 'src', 'log', )
1343+ for option in parse_options(options, available_options):
1344+ command.append(option)
1345+
1346+ command.append("-r {0}".format(requirements))
1347+ log("Installing from file: {} with options: {}".format(requirements,
1348+ command))
1349+ pip_execute(command)
1350+
1351+
1352+def pip_install(package, fatal=False, upgrade=False, **options):
1353+ """Install a python package"""
1354+ command = ["install"]
1355+
1356+ available_options = ('proxy', 'src', 'log', "index-url", )
1357+ for option in parse_options(options, available_options):
1358+ command.append(option)
1359+
1360+ if upgrade:
1361+ command.append('--upgrade')
1362+
1363+ if isinstance(package, list):
1364+ command.extend(package)
1365+ else:
1366+ command.append(package)
1367+
1368+ log("Installing {} package with options: {}".format(package,
1369+ command))
1370+ pip_execute(command)
1371+
1372+
1373+def pip_uninstall(package, **options):
1374+ """Uninstall a python package"""
1375+ command = ["uninstall", "-q", "-y"]
1376+
1377+ available_options = ('proxy', 'log', )
1378+ for option in parse_options(options, available_options):
1379+ command.append(option)
1380+
1381+ if isinstance(package, list):
1382+ command.extend(package)
1383+ else:
1384+ command.append(package)
1385+
1386+ log("Uninstalling {} package with options: {}".format(package,
1387+ command))
1388+ pip_execute(command)
1389+
1390+
1391+def pip_list():
1392+ """Returns the list of current python installed packages
1393+ """
1394+ return pip_execute(["list"])
1395
1396=== removed file 'hooks/charmhelpers/contrib/python/packages.py'
1397--- hooks/charmhelpers/contrib/python/packages.py 2015-01-21 12:31:56 +0000
1398+++ hooks/charmhelpers/contrib/python/packages.py 1970-01-01 00:00:00 +0000
1399@@ -1,80 +0,0 @@
1400-#!/usr/bin/env python
1401-# coding: utf-8
1402-
1403-__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
1404-
1405-from charmhelpers.fetch import apt_install, apt_update
1406-from charmhelpers.core.hookenv import log
1407-
1408-try:
1409- from pip import main as pip_execute
1410-except ImportError:
1411- apt_update()
1412- apt_install('python-pip')
1413- from pip import main as pip_execute
1414-
1415-
1416-def parse_options(given, available):
1417- """Given a set of options, check if available"""
1418- for key, value in sorted(given.items()):
1419- if key in available:
1420- yield "--{0}={1}".format(key, value)
1421-
1422-
1423-def pip_install_requirements(requirements, **options):
1424- """Install a requirements file """
1425- command = ["install"]
1426-
1427- available_options = ('proxy', 'src', 'log', )
1428- for option in parse_options(options, available_options):
1429- command.append(option)
1430-
1431- command.append("-r {0}".format(requirements))
1432- log("Installing from file: {} with options: {}".format(requirements,
1433- command))
1434- pip_execute(command)
1435-
1436-
1437-def pip_install(package, fatal=False, upgrade=False, **options):
1438- """Install a python package"""
1439- command = ["install"]
1440-
1441- available_options = ('proxy', 'src', 'log', "index-url", )
1442- for option in parse_options(options, available_options):
1443- command.append(option)
1444-
1445- if upgrade:
1446- command.append('--upgrade')
1447-
1448- if isinstance(package, list):
1449- command.extend(package)
1450- else:
1451- command.append(package)
1452-
1453- log("Installing {} package with options: {}".format(package,
1454- command))
1455- pip_execute(command)
1456-
1457-
1458-def pip_uninstall(package, **options):
1459- """Uninstall a python package"""
1460- command = ["uninstall", "-q", "-y"]
1461-
1462- available_options = ('proxy', 'log', )
1463- for option in parse_options(options, available_options):
1464- command.append(option)
1465-
1466- if isinstance(package, list):
1467- command.extend(package)
1468- else:
1469- command.append(package)
1470-
1471- log("Uninstalling {} package with options: {}".format(package,
1472- command))
1473- pip_execute(command)
1474-
1475-
1476-def pip_list():
1477- """Returns the list of current python installed packages
1478- """
1479- return pip_execute(["list"])
1480
1481=== modified file 'hooks/charmhelpers/core/__init__.py'
1482--- hooks/charmhelpers/core/__init__.py 2014-04-11 20:55:42 +0000
1483+++ hooks/charmhelpers/core/__init__.py 2015-02-25 01:49:51 +0000
1484@@ -0,0 +1,15 @@
1485+# Copyright 2014-2015 Canonical Limited.
1486+#
1487+# This file is part of charm-helpers.
1488+#
1489+# charm-helpers is free software: you can redistribute it and/or modify
1490+# it under the terms of the GNU Lesser General Public License version 3 as
1491+# published by the Free Software Foundation.
1492+#
1493+# charm-helpers is distributed in the hope that it will be useful,
1494+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1495+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1496+# GNU Lesser General Public License for more details.
1497+#
1498+# You should have received a copy of the GNU Lesser General Public License
1499+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1500
1501=== modified file 'hooks/charmhelpers/core/decorators.py'
1502--- hooks/charmhelpers/core/decorators.py 2015-01-19 16:22:18 +0000
1503+++ hooks/charmhelpers/core/decorators.py 2015-02-25 01:49:51 +0000
1504@@ -1,3 +1,19 @@
1505+# Copyright 2014-2015 Canonical Limited.
1506+#
1507+# This file is part of charm-helpers.
1508+#
1509+# charm-helpers is free software: you can redistribute it and/or modify
1510+# it under the terms of the GNU Lesser General Public License version 3 as
1511+# published by the Free Software Foundation.
1512+#
1513+# charm-helpers is distributed in the hope that it will be useful,
1514+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1515+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1516+# GNU Lesser General Public License for more details.
1517+#
1518+# You should have received a copy of the GNU Lesser General Public License
1519+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1520+
1521 #
1522 # Copyright 2014 Canonical Ltd.
1523 #
1524
1525=== modified file 'hooks/charmhelpers/core/fstab.py'
1526--- hooks/charmhelpers/core/fstab.py 2014-12-11 06:29:02 +0000
1527+++ hooks/charmhelpers/core/fstab.py 2015-02-25 01:49:51 +0000
1528@@ -1,11 +1,27 @@
1529 #!/usr/bin/env python
1530 # -*- coding: utf-8 -*-
1531
1532-__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
1533+# Copyright 2014-2015 Canonical Limited.
1534+#
1535+# This file is part of charm-helpers.
1536+#
1537+# charm-helpers is free software: you can redistribute it and/or modify
1538+# it under the terms of the GNU Lesser General Public License version 3 as
1539+# published by the Free Software Foundation.
1540+#
1541+# charm-helpers is distributed in the hope that it will be useful,
1542+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1543+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1544+# GNU Lesser General Public License for more details.
1545+#
1546+# You should have received a copy of the GNU Lesser General Public License
1547+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1548
1549 import io
1550 import os
1551
1552+__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
1553+
1554
1555 class Fstab(io.FileIO):
1556 """This class extends file in order to implement a file reader/writer
1557@@ -61,7 +77,7 @@
1558 for line in self.readlines():
1559 line = line.decode('us-ascii')
1560 try:
1561- if line.strip() and not line.startswith("#"):
1562+ if line.strip() and not line.strip().startswith("#"):
1563 yield self._hydrate_entry(line)
1564 except ValueError:
1565 pass
1566@@ -88,7 +104,7 @@
1567
1568 found = False
1569 for index, line in enumerate(lines):
1570- if not line.startswith("#"):
1571+ if line.strip() and not line.strip().startswith("#"):
1572 if self._hydrate_entry(line) == entry:
1573 found = True
1574 break
1575
1576=== modified file 'hooks/charmhelpers/core/hookenv.py'
1577--- hooks/charmhelpers/core/hookenv.py 2014-12-18 00:57:03 +0000
1578+++ hooks/charmhelpers/core/hookenv.py 2015-02-25 01:49:51 +0000
1579@@ -1,3 +1,19 @@
1580+# Copyright 2014-2015 Canonical Limited.
1581+#
1582+# This file is part of charm-helpers.
1583+#
1584+# charm-helpers is free software: you can redistribute it and/or modify
1585+# it under the terms of the GNU Lesser General Public License version 3 as
1586+# published by the Free Software Foundation.
1587+#
1588+# charm-helpers is distributed in the hope that it will be useful,
1589+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1590+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1591+# GNU Lesser General Public License for more details.
1592+#
1593+# You should have received a copy of the GNU Lesser General Public License
1594+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1595+
1596 "Interactions with the Juju environment"
1597 # Copyright 2013 Canonical Ltd.
1598 #
1599
1600=== modified file 'hooks/charmhelpers/core/host.py'
1601--- hooks/charmhelpers/core/host.py 2015-01-21 12:31:56 +0000
1602+++ hooks/charmhelpers/core/host.py 2015-02-25 01:49:51 +0000
1603@@ -1,3 +1,19 @@
1604+# Copyright 2014-2015 Canonical Limited.
1605+#
1606+# This file is part of charm-helpers.
1607+#
1608+# charm-helpers is free software: you can redistribute it and/or modify
1609+# it under the terms of the GNU Lesser General Public License version 3 as
1610+# published by the Free Software Foundation.
1611+#
1612+# charm-helpers is distributed in the hope that it will be useful,
1613+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1614+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1615+# GNU Lesser General Public License for more details.
1616+#
1617+# You should have received a copy of the GNU Lesser General Public License
1618+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1619+
1620 """Tools for working with the host system"""
1621 # Copyright 2012 Canonical Ltd.
1622 #
1623@@ -175,11 +191,11 @@
1624
1625
1626 def write_file(path, content, owner='root', group='root', perms=0o444):
1627- """Create or overwrite a file with the contents of a string"""
1628+ """Create or overwrite a file with the contents of a byte string."""
1629 log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
1630 uid = pwd.getpwnam(owner).pw_uid
1631 gid = grp.getgrnam(group).gr_gid
1632- with open(path, 'w') as target:
1633+ with open(path, 'wb') as target:
1634 os.fchown(target.fileno(), uid, gid)
1635 os.fchmod(target.fileno(), perms)
1636 target.write(content)
1637@@ -289,11 +305,11 @@
1638 ceph_client_changed function.
1639 """
1640 def wrap(f):
1641- def wrapped_f(*args):
1642+ def wrapped_f(*args, **kwargs):
1643 checksums = {}
1644 for path in restart_map:
1645 checksums[path] = file_hash(path)
1646- f(*args)
1647+ f(*args, **kwargs)
1648 restarts = []
1649 for path in restart_map:
1650 if checksums[path] != file_hash(path):
1651@@ -345,7 +361,7 @@
1652 ip_output = (line for line in ip_output if line)
1653 for line in ip_output:
1654 if line.split()[1].startswith(int_type):
1655- matched = re.search('.*: (bond[0-9]+\.[0-9]+)@.*', line)
1656+ matched = re.search('.*: (' + int_type + r'[0-9]+\.[0-9]+)@.*', line)
1657 if matched:
1658 interface = matched.groups()[0]
1659 else:
1660@@ -389,6 +405,9 @@
1661 * 0 => Installed revno is the same as supplied arg
1662 * -1 => Installed revno is less than supplied arg
1663
1664+ This function imports apt_cache function from charmhelpers.fetch if
1665+ the pkgcache argument is None. Be sure to add charmhelpers.fetch if
1666+ you call this function, or pass an apt_pkg.Cache() instance.
1667 '''
1668 import apt_pkg
1669 if not pkgcache:
1670
1671=== modified file 'hooks/charmhelpers/core/services/__init__.py'
1672--- hooks/charmhelpers/core/services/__init__.py 2014-12-09 23:58:57 +0000
1673+++ hooks/charmhelpers/core/services/__init__.py 2015-02-25 01:49:51 +0000
1674@@ -1,2 +1,18 @@
1675+# Copyright 2014-2015 Canonical Limited.
1676+#
1677+# This file is part of charm-helpers.
1678+#
1679+# charm-helpers is free software: you can redistribute it and/or modify
1680+# it under the terms of the GNU Lesser General Public License version 3 as
1681+# published by the Free Software Foundation.
1682+#
1683+# charm-helpers is distributed in the hope that it will be useful,
1684+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1685+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1686+# GNU Lesser General Public License for more details.
1687+#
1688+# You should have received a copy of the GNU Lesser General Public License
1689+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1690+
1691 from .base import * # NOQA
1692 from .helpers import * # NOQA
1693
1694=== modified file 'hooks/charmhelpers/core/services/base.py'
1695--- hooks/charmhelpers/core/services/base.py 2014-12-09 23:58:57 +0000
1696+++ hooks/charmhelpers/core/services/base.py 2015-02-25 01:49:51 +0000
1697@@ -1,3 +1,19 @@
1698+# Copyright 2014-2015 Canonical Limited.
1699+#
1700+# This file is part of charm-helpers.
1701+#
1702+# charm-helpers is free software: you can redistribute it and/or modify
1703+# it under the terms of the GNU Lesser General Public License version 3 as
1704+# published by the Free Software Foundation.
1705+#
1706+# charm-helpers is distributed in the hope that it will be useful,
1707+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1708+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1709+# GNU Lesser General Public License for more details.
1710+#
1711+# You should have received a copy of the GNU Lesser General Public License
1712+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1713+
1714 import os
1715 import re
1716 import json
1717
1718=== modified file 'hooks/charmhelpers/core/services/helpers.py'
1719--- hooks/charmhelpers/core/services/helpers.py 2014-12-09 23:58:57 +0000
1720+++ hooks/charmhelpers/core/services/helpers.py 2015-02-25 01:49:51 +0000
1721@@ -1,3 +1,19 @@
1722+# Copyright 2014-2015 Canonical Limited.
1723+#
1724+# This file is part of charm-helpers.
1725+#
1726+# charm-helpers is free software: you can redistribute it and/or modify
1727+# it under the terms of the GNU Lesser General Public License version 3 as
1728+# published by the Free Software Foundation.
1729+#
1730+# charm-helpers is distributed in the hope that it will be useful,
1731+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1732+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1733+# GNU Lesser General Public License for more details.
1734+#
1735+# You should have received a copy of the GNU Lesser General Public License
1736+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1737+
1738 import os
1739 import yaml
1740 from charmhelpers.core import hookenv
1741
1742=== added file 'hooks/charmhelpers/core/strutils.py'
1743--- hooks/charmhelpers/core/strutils.py 1970-01-01 00:00:00 +0000
1744+++ hooks/charmhelpers/core/strutils.py 2015-02-25 01:49:51 +0000
1745@@ -0,0 +1,42 @@
1746+#!/usr/bin/env python
1747+# -*- coding: utf-8 -*-
1748+
1749+# Copyright 2014-2015 Canonical Limited.
1750+#
1751+# This file is part of charm-helpers.
1752+#
1753+# charm-helpers is free software: you can redistribute it and/or modify
1754+# it under the terms of the GNU Lesser General Public License version 3 as
1755+# published by the Free Software Foundation.
1756+#
1757+# charm-helpers is distributed in the hope that it will be useful,
1758+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1759+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1760+# GNU Lesser General Public License for more details.
1761+#
1762+# You should have received a copy of the GNU Lesser General Public License
1763+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1764+
1765+import six
1766+
1767+
1768+def bool_from_string(value):
1769+ """Interpret string value as boolean.
1770+
1771+ Returns True if value translates to True otherwise False.
1772+ """
1773+ if isinstance(value, six.string_types):
1774+ value = six.text_type(value)
1775+ else:
1776+ msg = "Unable to interpret non-string value '%s' as boolean" % (value)
1777+ raise ValueError(msg)
1778+
1779+ value = value.strip().lower()
1780+
1781+ if value in ['y', 'yes', 'true', 't']:
1782+ return True
1783+ elif value in ['n', 'no', 'false', 'f']:
1784+ return False
1785+
1786+ msg = "Unable to interpret string value '%s' as boolean" % (value)
1787+ raise ValueError(msg)
1788
1789=== modified file 'hooks/charmhelpers/core/sysctl.py'
1790--- hooks/charmhelpers/core/sysctl.py 2014-12-09 23:58:57 +0000
1791+++ hooks/charmhelpers/core/sysctl.py 2015-02-25 01:49:51 +0000
1792@@ -1,7 +1,21 @@
1793 #!/usr/bin/env python
1794 # -*- coding: utf-8 -*-
1795
1796-__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
1797+# Copyright 2014-2015 Canonical Limited.
1798+#
1799+# This file is part of charm-helpers.
1800+#
1801+# charm-helpers is free software: you can redistribute it and/or modify
1802+# it under the terms of the GNU Lesser General Public License version 3 as
1803+# published by the Free Software Foundation.
1804+#
1805+# charm-helpers is distributed in the hope that it will be useful,
1806+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1807+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1808+# GNU Lesser General Public License for more details.
1809+#
1810+# You should have received a copy of the GNU Lesser General Public License
1811+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1812
1813 import yaml
1814
1815@@ -10,25 +24,33 @@
1816 from charmhelpers.core.hookenv import (
1817 log,
1818 DEBUG,
1819+ ERROR,
1820 )
1821
1822+__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
1823+
1824
1825 def create(sysctl_dict, sysctl_file):
1826 """Creates a sysctl.conf file from a YAML associative array
1827
1828- :param sysctl_dict: a dict of sysctl options eg { 'kernel.max_pid': 1337 }
1829- :type sysctl_dict: dict
1830+ :param sysctl_dict: a YAML-formatted string of sysctl options eg "{ 'kernel.max_pid': 1337 }"
1831+ :type sysctl_dict: str
1832 :param sysctl_file: path to the sysctl file to be saved
1833 :type sysctl_file: str or unicode
1834 :returns: None
1835 """
1836- sysctl_dict = yaml.load(sysctl_dict)
1837+ try:
1838+ sysctl_dict_parsed = yaml.safe_load(sysctl_dict)
1839+ except yaml.YAMLError:
1840+ log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict),
1841+ level=ERROR)
1842+ return
1843
1844 with open(sysctl_file, "w") as fd:
1845- for key, value in sysctl_dict.items():
1846+ for key, value in sysctl_dict_parsed.items():
1847 fd.write("{}={}\n".format(key, value))
1848
1849- log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict),
1850+ log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict_parsed),
1851 level=DEBUG)
1852
1853 check_call(["sysctl", "-p", sysctl_file])
1854
1855=== modified file 'hooks/charmhelpers/core/templating.py'
1856--- hooks/charmhelpers/core/templating.py 2015-01-19 21:16:06 +0000
1857+++ hooks/charmhelpers/core/templating.py 2015-02-25 01:49:51 +0000
1858@@ -1,3 +1,19 @@
1859+# Copyright 2014-2015 Canonical Limited.
1860+#
1861+# This file is part of charm-helpers.
1862+#
1863+# charm-helpers is free software: you can redistribute it and/or modify
1864+# it under the terms of the GNU Lesser General Public License version 3 as
1865+# published by the Free Software Foundation.
1866+#
1867+# charm-helpers is distributed in the hope that it will be useful,
1868+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1869+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1870+# GNU Lesser General Public License for more details.
1871+#
1872+# You should have received a copy of the GNU Lesser General Public License
1873+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1874+
1875 import os
1876
1877 from charmhelpers.core import host
1878@@ -5,7 +21,7 @@
1879
1880
1881 def render(source, target, context, owner='root', group='root',
1882- perms=0o444, templates_dir=None):
1883+ perms=0o444, templates_dir=None, encoding='UTF-8'):
1884 """
1885 Render a template.
1886
1887@@ -48,5 +64,5 @@
1888 level=hookenv.ERROR)
1889 raise e
1890 content = template.render(context)
1891- host.mkdir(os.path.dirname(target), owner, group)
1892- host.write_file(target, content, owner, group, perms)
1893+ host.mkdir(os.path.dirname(target), owner, group, perms=0o755)
1894+ host.write_file(target, content.encode(encoding), owner, group, perms)
1895
1896=== added file 'hooks/charmhelpers/core/unitdata.py'
1897--- hooks/charmhelpers/core/unitdata.py 1970-01-01 00:00:00 +0000
1898+++ hooks/charmhelpers/core/unitdata.py 2015-02-25 01:49:51 +0000
1899@@ -0,0 +1,477 @@
1900+#!/usr/bin/env python
1901+# -*- coding: utf-8 -*-
1902+#
1903+# Copyright 2014-2015 Canonical Limited.
1904+#
1905+# This file is part of charm-helpers.
1906+#
1907+# charm-helpers is free software: you can redistribute it and/or modify
1908+# it under the terms of the GNU Lesser General Public License version 3 as
1909+# published by the Free Software Foundation.
1910+#
1911+# charm-helpers is distributed in the hope that it will be useful,
1912+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1913+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1914+# GNU Lesser General Public License for more details.
1915+#
1916+# You should have received a copy of the GNU Lesser General Public License
1917+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1918+#
1919+#
1920+# Authors:
1921+# Kapil Thangavelu <kapil.foss@gmail.com>
1922+#
1923+"""
1924+Intro
1925+-----
1926+
1927+A simple way to store state in units. This provides a key value
1928+storage with support for versioned, transactional operation,
1929+and can calculate deltas from previous values to simplify unit logic
1930+when processing changes.
1931+
1932+
1933+Hook Integration
1934+----------------
1935+
1936+There are several extant frameworks for hook execution, including
1937+
1938+ - charmhelpers.core.hookenv.Hooks
1939+ - charmhelpers.core.services.ServiceManager
1940+
1941+The storage classes are framework agnostic, one simple integration is
1942+via the HookData contextmanager. It will record the current hook
1943+execution environment (including relation data, config data, etc.),
1944+setup a transaction and allow easy access to the changes from
1945+previously seen values. One consequence of the integration is the
1946+reservation of particular keys ('rels', 'unit', 'env', 'config',
1947+'charm_revisions') for their respective values.
1948+
1949+Here's a fully worked integration example using hookenv.Hooks::
1950+
1951+ from charmhelper.core import hookenv, unitdata
1952+
1953+ hook_data = unitdata.HookData()
1954+ db = unitdata.kv()
1955+ hooks = hookenv.Hooks()
1956+
1957+ @hooks.hook
1958+ def config_changed():
1959+ # Print all changes to configuration from previously seen
1960+ # values.
1961+ for changed, (prev, cur) in hook_data.conf.items():
1962+ print('config changed', changed,
1963+ 'previous value', prev,
1964+ 'current value', cur)
1965+
1966+ # Get some unit specific bookeeping
1967+ if not db.get('pkg_key'):
1968+ key = urllib.urlopen('https://example.com/pkg_key').read()
1969+ db.set('pkg_key', key)
1970+
1971+ # Directly access all charm config as a mapping.
1972+ conf = db.getrange('config', True)
1973+
1974+ # Directly access all relation data as a mapping
1975+ rels = db.getrange('rels', True)
1976+
1977+ if __name__ == '__main__':
1978+ with hook_data():
1979+ hook.execute()
1980+
1981+
1982+A more basic integration is via the hook_scope context manager which simply
1983+manages transaction scope (and records hook name, and timestamp)::
1984+
1985+ >>> from unitdata import kv
1986+ >>> db = kv()
1987+ >>> with db.hook_scope('install'):
1988+ ... # do work, in transactional scope.
1989+ ... db.set('x', 1)
1990+ >>> db.get('x')
1991+ 1
1992+
1993+
1994+Usage
1995+-----
1996+
1997+Values are automatically json de/serialized to preserve basic typing
1998+and complex data struct capabilities (dicts, lists, ints, booleans, etc).
1999+
2000+Individual values can be manipulated via get/set::
2001+
2002+ >>> kv.set('y', True)
2003+ >>> kv.get('y')
2004+ True
2005+
2006+ # We can set complex values (dicts, lists) as a single key.
2007+ >>> kv.set('config', {'a': 1, 'b': True'})
2008+
2009+ # Also supports returning dictionaries as a record which
2010+ # provides attribute access.
2011+ >>> config = kv.get('config', record=True)
2012+ >>> config.b
2013+ True
2014+
2015+
2016+Groups of keys can be manipulated with update/getrange::
2017+
2018+ >>> kv.update({'z': 1, 'y': 2}, prefix="gui.")
2019+ >>> kv.getrange('gui.', strip=True)
2020+ {'z': 1, 'y': 2}
2021+
2022+When updating values, its very helpful to understand which values
2023+have actually changed and how have they changed. The storage
2024+provides a delta method to provide for this::
2025+
2026+ >>> data = {'debug': True, 'option': 2}
2027+ >>> delta = kv.delta(data, 'config.')
2028+ >>> delta.debug.previous
2029+ None
2030+ >>> delta.debug.current
2031+ True
2032+ >>> delta
2033+ {'debug': (None, True), 'option': (None, 2)}
2034+
2035+Note the delta method does not persist the actual change, it needs to
2036+be explicitly saved via 'update' method::
2037+
2038+ >>> kv.update(data, 'config.')
2039+
2040+Values modified in the context of a hook scope retain historical values
2041+associated to the hookname.
2042+
2043+ >>> with db.hook_scope('config-changed'):
2044+ ... db.set('x', 42)
2045+ >>> db.gethistory('x')
2046+ [(1, u'x', 1, u'install', u'2015-01-21T16:49:30.038372'),
2047+ (2, u'x', 42, u'config-changed', u'2015-01-21T16:49:30.038786')]
2048+
2049+"""
2050+
2051+import collections
2052+import contextlib
2053+import datetime
2054+import json
2055+import os
2056+import pprint
2057+import sqlite3
2058+import sys
2059+
2060+__author__ = 'Kapil Thangavelu <kapil.foss@gmail.com>'
2061+
2062+
2063+class Storage(object):
2064+ """Simple key value database for local unit state within charms.
2065+
2066+ Modifications are automatically committed at hook exit. That's
2067+ currently regardless of exit code.
2068+
2069+ To support dicts, lists, integer, floats, and booleans values
2070+ are automatically json encoded/decoded.
2071+ """
2072+ def __init__(self, path=None):
2073+ self.db_path = path
2074+ if path is None:
2075+ self.db_path = os.path.join(
2076+ os.environ.get('CHARM_DIR', ''), '.unit-state.db')
2077+ self.conn = sqlite3.connect('%s' % self.db_path)
2078+ self.cursor = self.conn.cursor()
2079+ self.revision = None
2080+ self._closed = False
2081+ self._init()
2082+
2083+ def close(self):
2084+ if self._closed:
2085+ return
2086+ self.flush(False)
2087+ self.cursor.close()
2088+ self.conn.close()
2089+ self._closed = True
2090+
2091+ def _scoped_query(self, stmt, params=None):
2092+ if params is None:
2093+ params = []
2094+ return stmt, params
2095+
2096+ def get(self, key, default=None, record=False):
2097+ self.cursor.execute(
2098+ *self._scoped_query(
2099+ 'select data from kv where key=?', [key]))
2100+ result = self.cursor.fetchone()
2101+ if not result:
2102+ return default
2103+ if record:
2104+ return Record(json.loads(result[0]))
2105+ return json.loads(result[0])
2106+
2107+ def getrange(self, key_prefix, strip=False):
2108+ stmt = "select key, data from kv where key like '%s%%'" % key_prefix
2109+ self.cursor.execute(*self._scoped_query(stmt))
2110+ result = self.cursor.fetchall()
2111+
2112+ if not result:
2113+ return None
2114+ if not strip:
2115+ key_prefix = ''
2116+ return dict([
2117+ (k[len(key_prefix):], json.loads(v)) for k, v in result])
2118+
2119+ def update(self, mapping, prefix=""):
2120+ for k, v in mapping.items():
2121+ self.set("%s%s" % (prefix, k), v)
2122+
2123+ def unset(self, key):
2124+ self.cursor.execute('delete from kv where key=?', [key])
2125+ if self.revision and self.cursor.rowcount:
2126+ self.cursor.execute(
2127+ 'insert into kv_revisions values (?, ?, ?)',
2128+ [key, self.revision, json.dumps('DELETED')])
2129+
2130+ def set(self, key, value):
2131+ serialized = json.dumps(value)
2132+
2133+ self.cursor.execute(
2134+ 'select data from kv where key=?', [key])
2135+ exists = self.cursor.fetchone()
2136+
2137+ # Skip mutations to the same value
2138+ if exists:
2139+ if exists[0] == serialized:
2140+ return value
2141+
2142+ if not exists:
2143+ self.cursor.execute(
2144+ 'insert into kv (key, data) values (?, ?)',
2145+ (key, serialized))
2146+ else:
2147+ self.cursor.execute('''
2148+ update kv
2149+ set data = ?
2150+ where key = ?''', [serialized, key])
2151+
2152+ # Save
2153+ if not self.revision:
2154+ return value
2155+
2156+ self.cursor.execute(
2157+ 'select 1 from kv_revisions where key=? and revision=?',
2158+ [key, self.revision])
2159+ exists = self.cursor.fetchone()
2160+
2161+ if not exists:
2162+ self.cursor.execute(
2163+ '''insert into kv_revisions (
2164+ revision, key, data) values (?, ?, ?)''',
2165+ (self.revision, key, serialized))
2166+ else:
2167+ self.cursor.execute(
2168+ '''
2169+ update kv_revisions
2170+ set data = ?
2171+ where key = ?
2172+ and revision = ?''',
2173+ [serialized, key, self.revision])
2174+
2175+ return value
2176+
2177+ def delta(self, mapping, prefix):
2178+ """
2179+ return a delta containing values that have changed.
2180+ """
2181+ previous = self.getrange(prefix, strip=True)
2182+ if not previous:
2183+ pk = set()
2184+ else:
2185+ pk = set(previous.keys())
2186+ ck = set(mapping.keys())
2187+ delta = DeltaSet()
2188+
2189+ # added
2190+ for k in ck.difference(pk):
2191+ delta[k] = Delta(None, mapping[k])
2192+
2193+ # removed
2194+ for k in pk.difference(ck):
2195+ delta[k] = Delta(previous[k], None)
2196+
2197+ # changed
2198+ for k in pk.intersection(ck):
2199+ c = mapping[k]
2200+ p = previous[k]
2201+ if c != p:
2202+ delta[k] = Delta(p, c)
2203+
2204+ return delta
2205+
2206+ @contextlib.contextmanager
2207+ def hook_scope(self, name=""):
2208+ """Scope all future interactions to the current hook execution
2209+ revision."""
2210+ assert not self.revision
2211+ self.cursor.execute(
2212+ 'insert into hooks (hook, date) values (?, ?)',
2213+ (name or sys.argv[0],
2214+ datetime.datetime.utcnow().isoformat()))
2215+ self.revision = self.cursor.lastrowid
2216+ try:
2217+ yield self.revision
2218+ self.revision = None
2219+ except:
2220+ self.flush(False)
2221+ self.revision = None
2222+ raise
2223+ else:
2224+ self.flush()
2225+
2226+ def flush(self, save=True):
2227+ if save:
2228+ self.conn.commit()
2229+ elif self._closed:
2230+ return
2231+ else:
2232+ self.conn.rollback()
2233+
2234+ def _init(self):
2235+ self.cursor.execute('''
2236+ create table if not exists kv (
2237+ key text,
2238+ data text,
2239+ primary key (key)
2240+ )''')
2241+ self.cursor.execute('''
2242+ create table if not exists kv_revisions (
2243+ key text,
2244+ revision integer,
2245+ data text,
2246+ primary key (key, revision)
2247+ )''')
2248+ self.cursor.execute('''
2249+ create table if not exists hooks (
2250+ version integer primary key autoincrement,
2251+ hook text,
2252+ date text
2253+ )''')
2254+ self.conn.commit()
2255+
2256+ def gethistory(self, key, deserialize=False):
2257+ self.cursor.execute(
2258+ '''
2259+ select kv.revision, kv.key, kv.data, h.hook, h.date
2260+ from kv_revisions kv,
2261+ hooks h
2262+ where kv.key=?
2263+ and kv.revision = h.version
2264+ ''', [key])
2265+ if deserialize is False:
2266+ return self.cursor.fetchall()
2267+ return map(_parse_history, self.cursor.fetchall())
2268+
2269+ def debug(self, fh=sys.stderr):
2270+ self.cursor.execute('select * from kv')
2271+ pprint.pprint(self.cursor.fetchall(), stream=fh)
2272+ self.cursor.execute('select * from kv_revisions')
2273+ pprint.pprint(self.cursor.fetchall(), stream=fh)
2274+
2275+
2276+def _parse_history(d):
2277+ return (d[0], d[1], json.loads(d[2]), d[3],
2278+ datetime.datetime.strptime(d[-1], "%Y-%m-%dT%H:%M:%S.%f"))
2279+
2280+
2281+class HookData(object):
2282+ """Simple integration for existing hook exec frameworks.
2283+
2284+ Records all unit information, and stores deltas for processing
2285+ by the hook.
2286+
2287+ Sample::
2288+
2289+ from charmhelper.core import hookenv, unitdata
2290+
2291+ changes = unitdata.HookData()
2292+ db = unitdata.kv()
2293+ hooks = hookenv.Hooks()
2294+
2295+ @hooks.hook
2296+ def config_changed():
2297+ # View all changes to configuration
2298+ for changed, (prev, cur) in changes.conf.items():
2299+ print('config changed', changed,
2300+ 'previous value', prev,
2301+ 'current value', cur)
2302+
2303+ # Get some unit specific bookeeping
2304+ if not db.get('pkg_key'):
2305+ key = urllib.urlopen('https://example.com/pkg_key').read()
2306+ db.set('pkg_key', key)
2307+
2308+ if __name__ == '__main__':
2309+ with changes():
2310+ hook.execute()
2311+
2312+ """
2313+ def __init__(self):
2314+ self.kv = kv()
2315+ self.conf = None
2316+ self.rels = None
2317+
2318+ @contextlib.contextmanager
2319+ def __call__(self):
2320+ from charmhelpers.core import hookenv
2321+ hook_name = hookenv.hook_name()
2322+
2323+ with self.kv.hook_scope(hook_name):
2324+ self._record_charm_version(hookenv.charm_dir())
2325+ delta_config, delta_relation = self._record_hook(hookenv)
2326+ yield self.kv, delta_config, delta_relation
2327+
2328+ def _record_charm_version(self, charm_dir):
2329+ # Record revisions.. charm revisions are meaningless
2330+ # to charm authors as they don't control the revision.
2331+ # so logic dependnent on revision is not particularly
2332+ # useful, however it is useful for debugging analysis.
2333+ charm_rev = open(
2334+ os.path.join(charm_dir, 'revision')).read().strip()
2335+ charm_rev = charm_rev or '0'
2336+ revs = self.kv.get('charm_revisions', [])
2337+ if charm_rev not in revs:
2338+ revs.append(charm_rev.strip() or '0')
2339+ self.kv.set('charm_revisions', revs)
2340+
2341+ def _record_hook(self, hookenv):
2342+ data = hookenv.execution_environment()
2343+ self.conf = conf_delta = self.kv.delta(data['conf'], 'config')
2344+ self.rels = rels_delta = self.kv.delta(data['rels'], 'rels')
2345+ self.kv.set('env', data['env'])
2346+ self.kv.set('unit', data['unit'])
2347+ self.kv.set('relid', data.get('relid'))
2348+ return conf_delta, rels_delta
2349+
2350+
2351+class Record(dict):
2352+
2353+ __slots__ = ()
2354+
2355+ def __getattr__(self, k):
2356+ if k in self:
2357+ return self[k]
2358+ raise AttributeError(k)
2359+
2360+
2361+class DeltaSet(Record):
2362+
2363+ __slots__ = ()
2364+
2365+
2366+Delta = collections.namedtuple('Delta', ['previous', 'current'])
2367+
2368+
2369+_KV = None
2370+
2371+
2372+def kv():
2373+ global _KV
2374+ if _KV is None:
2375+ _KV = Storage()
2376+ return _KV
2377
2378=== modified file 'hooks/charmhelpers/fetch/__init__.py'
2379--- hooks/charmhelpers/fetch/__init__.py 2015-01-19 21:16:06 +0000
2380+++ hooks/charmhelpers/fetch/__init__.py 2015-02-25 01:49:51 +0000
2381@@ -1,3 +1,19 @@
2382+# Copyright 2014-2015 Canonical Limited.
2383+#
2384+# This file is part of charm-helpers.
2385+#
2386+# charm-helpers is free software: you can redistribute it and/or modify
2387+# it under the terms of the GNU Lesser General Public License version 3 as
2388+# published by the Free Software Foundation.
2389+#
2390+# charm-helpers is distributed in the hope that it will be useful,
2391+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2392+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2393+# GNU Lesser General Public License for more details.
2394+#
2395+# You should have received a copy of the GNU Lesser General Public License
2396+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2397+
2398 import importlib
2399 from tempfile import NamedTemporaryFile
2400 import time
2401
2402=== modified file 'hooks/charmhelpers/fetch/archiveurl.py'
2403--- hooks/charmhelpers/fetch/archiveurl.py 2014-12-11 06:29:02 +0000
2404+++ hooks/charmhelpers/fetch/archiveurl.py 2015-02-25 01:49:51 +0000
2405@@ -1,7 +1,33 @@
2406+# Copyright 2014-2015 Canonical Limited.
2407+#
2408+# This file is part of charm-helpers.
2409+#
2410+# charm-helpers is free software: you can redistribute it and/or modify
2411+# it under the terms of the GNU Lesser General Public License version 3 as
2412+# published by the Free Software Foundation.
2413+#
2414+# charm-helpers is distributed in the hope that it will be useful,
2415+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2416+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2417+# GNU Lesser General Public License for more details.
2418+#
2419+# You should have received a copy of the GNU Lesser General Public License
2420+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2421+
2422 import os
2423 import hashlib
2424 import re
2425
2426+from charmhelpers.fetch import (
2427+ BaseFetchHandler,
2428+ UnhandledSource
2429+)
2430+from charmhelpers.payload.archive import (
2431+ get_archive_handler,
2432+ extract,
2433+)
2434+from charmhelpers.core.host import mkdir, check_hash
2435+
2436 import six
2437 if six.PY3:
2438 from urllib.request import (
2439@@ -19,16 +45,6 @@
2440 )
2441 from urlparse import urlparse, urlunparse, parse_qs
2442
2443-from charmhelpers.fetch import (
2444- BaseFetchHandler,
2445- UnhandledSource
2446-)
2447-from charmhelpers.payload.archive import (
2448- get_archive_handler,
2449- extract,
2450-)
2451-from charmhelpers.core.host import mkdir, check_hash
2452-
2453
2454 def splituser(host):
2455 '''urllib.splituser(), but six's support of this seems broken'''
2456
2457=== modified file 'hooks/charmhelpers/fetch/bzrurl.py'
2458--- hooks/charmhelpers/fetch/bzrurl.py 2014-12-11 06:29:02 +0000
2459+++ hooks/charmhelpers/fetch/bzrurl.py 2015-02-25 01:49:51 +0000
2460@@ -1,3 +1,19 @@
2461+# Copyright 2014-2015 Canonical Limited.
2462+#
2463+# This file is part of charm-helpers.
2464+#
2465+# charm-helpers is free software: you can redistribute it and/or modify
2466+# it under the terms of the GNU Lesser General Public License version 3 as
2467+# published by the Free Software Foundation.
2468+#
2469+# charm-helpers is distributed in the hope that it will be useful,
2470+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2471+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2472+# GNU Lesser General Public License for more details.
2473+#
2474+# You should have received a copy of the GNU Lesser General Public License
2475+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2476+
2477 import os
2478 from charmhelpers.fetch import (
2479 BaseFetchHandler,
2480@@ -11,10 +27,12 @@
2481
2482 try:
2483 from bzrlib.branch import Branch
2484+ from bzrlib import bzrdir, workingtree, errors
2485 except ImportError:
2486 from charmhelpers.fetch import apt_install
2487 apt_install("python-bzrlib")
2488 from bzrlib.branch import Branch
2489+ from bzrlib import bzrdir, workingtree, errors
2490
2491
2492 class BzrUrlFetchHandler(BaseFetchHandler):
2493@@ -35,8 +53,14 @@
2494 from bzrlib.plugin import load_plugins
2495 load_plugins()
2496 try:
2497+ local_branch = bzrdir.BzrDir.create_branch_convenience(dest)
2498+ except errors.AlreadyControlDirError:
2499+ local_branch = Branch.open(dest)
2500+ try:
2501 remote_branch = Branch.open(source)
2502- remote_branch.bzrdir.sprout(dest).open_branch()
2503+ remote_branch.push(local_branch)
2504+ tree = workingtree.WorkingTree.open(dest)
2505+ tree.update()
2506 except Exception as e:
2507 raise e
2508
2509
2510=== modified file 'hooks/charmhelpers/fetch/giturl.py'
2511--- hooks/charmhelpers/fetch/giturl.py 2014-12-09 23:58:57 +0000
2512+++ hooks/charmhelpers/fetch/giturl.py 2015-02-25 01:49:51 +0000
2513@@ -1,3 +1,19 @@
2514+# Copyright 2014-2015 Canonical Limited.
2515+#
2516+# This file is part of charm-helpers.
2517+#
2518+# charm-helpers is free software: you can redistribute it and/or modify
2519+# it under the terms of the GNU Lesser General Public License version 3 as
2520+# published by the Free Software Foundation.
2521+#
2522+# charm-helpers is distributed in the hope that it will be useful,
2523+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2524+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2525+# GNU Lesser General Public License for more details.
2526+#
2527+# You should have received a copy of the GNU Lesser General Public License
2528+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2529+
2530 import os
2531 from charmhelpers.fetch import (
2532 BaseFetchHandler,
2533@@ -16,6 +32,8 @@
2534 apt_install("python-git")
2535 from git import Repo
2536
2537+from git.exc import GitCommandError # noqa E402
2538+
2539
2540 class GitUrlFetchHandler(BaseFetchHandler):
2541 """Handler for git branches via generic and github URLs"""
2542@@ -46,6 +64,8 @@
2543 mkdir(dest_dir, perms=0o755)
2544 try:
2545 self.clone(source, dest_dir, branch)
2546+ except GitCommandError as e:
2547+ raise UnhandledSource(e.message)
2548 except OSError as e:
2549 raise UnhandledSource(e.strerror)
2550 return dest_dir
2551
2552=== modified file 'hooks/charmhelpers/payload/__init__.py'
2553--- hooks/charmhelpers/payload/__init__.py 2014-08-21 14:11:00 +0000
2554+++ hooks/charmhelpers/payload/__init__.py 2015-02-25 01:49:51 +0000
2555@@ -1,1 +1,17 @@
2556+# Copyright 2014-2015 Canonical Limited.
2557+#
2558+# This file is part of charm-helpers.
2559+#
2560+# charm-helpers is free software: you can redistribute it and/or modify
2561+# it under the terms of the GNU Lesser General Public License version 3 as
2562+# published by the Free Software Foundation.
2563+#
2564+# charm-helpers is distributed in the hope that it will be useful,
2565+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2566+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2567+# GNU Lesser General Public License for more details.
2568+#
2569+# You should have received a copy of the GNU Lesser General Public License
2570+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2571+
2572 "Tools for working with files injected into a charm just before deployment."
2573
2574=== modified file 'hooks/charmhelpers/payload/execd.py'
2575--- hooks/charmhelpers/payload/execd.py 2014-08-21 14:11:00 +0000
2576+++ hooks/charmhelpers/payload/execd.py 2015-02-25 01:49:51 +0000
2577@@ -1,5 +1,21 @@
2578 #!/usr/bin/env python
2579
2580+# Copyright 2014-2015 Canonical Limited.
2581+#
2582+# This file is part of charm-helpers.
2583+#
2584+# charm-helpers is free software: you can redistribute it and/or modify
2585+# it under the terms of the GNU Lesser General Public License version 3 as
2586+# published by the Free Software Foundation.
2587+#
2588+# charm-helpers is distributed in the hope that it will be useful,
2589+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2590+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2591+# GNU Lesser General Public License for more details.
2592+#
2593+# You should have received a copy of the GNU Lesser General Public License
2594+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2595+
2596 import os
2597 import sys
2598 import subprocess
2599
2600=== modified file 'hooks/hooks.py'
2601--- hooks/hooks.py 2015-01-22 10:47:27 +0000
2602+++ hooks/hooks.py 2015-02-25 01:49:51 +0000
2603@@ -45,6 +45,7 @@
2604 relations_of_type,
2605 relation_id,
2606 relation_ids,
2607+ local_unit,
2608 Hooks,
2609 DEBUG,
2610 WARNING,
2611@@ -73,6 +74,8 @@
2612 apt_install("python-lockfile", fatal=True)
2613 from lockfile import FileLock
2614
2615+from charmhelpers.contrib.charmsupport.nrpe import NRPE
2616+
2617 hooks = Hooks()
2618
2619 ###############################################################################
2620@@ -1059,6 +1062,8 @@
2621 else:
2622 open_port(config_data['mongos_port'])
2623
2624+ update_nrpe_config()
2625+
2626 print "About to leave config_changed"
2627 return(True)
2628
2629@@ -1366,6 +1371,34 @@
2630 update_file(default_mongos_list, '\n'.join(config_servers))
2631
2632
2633+@hooks.hook('nrpe-external-master-relation-joined',
2634+ 'nrpe-external-master-relation-changed')
2635+def update_nrpe_config():
2636+ # Find out if nrpe set nagios_hostname
2637+ hostname = None
2638+ host_context = None
2639+ for rel in relations_of_type('nrpe-external-master'):
2640+ if 'nagios_hostname' in rel:
2641+ hostname = rel['nagios_hostname']
2642+ host_context = rel['nagios_host_context']
2643+ break
2644+ nrpe = NRPE(hostname=hostname)
2645+ apt_install('python-dbus')
2646+
2647+ if host_context:
2648+ current_unit = "%s:%s" % (host_context, local_unit())
2649+ else:
2650+ current_unit = local_unit()
2651+
2652+ nrpe.add_check(
2653+ shortname='mongodb',
2654+ description='process check {%s}' % current_unit,
2655+ check_cmd='check_upstart_job mongodb',
2656+ )
2657+
2658+ nrpe.write()
2659+
2660+
2661 def run(command, exit_on_error=True):
2662 '''Run a command and return the output.'''
2663 try:
2664
2665=== added symlink 'hooks/nrpe-external-master-relation-changed'
2666=== target is u'hooks.py'
2667=== added symlink 'hooks/nrpe-external-master-relation-joined'
2668=== target is u'hooks.py'
2669=== modified file 'metadata.yaml'
2670--- metadata.yaml 2014-07-29 19:57:17 +0000
2671+++ metadata.yaml 2015-02-25 01:49:51 +0000
2672@@ -21,6 +21,9 @@
2673 categories:
2674 - databases
2675 provides:
2676+ nrpe-external-master:
2677+ interface: nrpe-external-master
2678+ scope: container
2679 database:
2680 interface: mongodb
2681 configsvr:
2682
2683=== modified file 'tests/03_deploy_replicaset.py'
2684--- tests/03_deploy_replicaset.py 2015-01-12 23:28:29 +0000
2685+++ tests/03_deploy_replicaset.py 2015-02-25 01:49:51 +0000
2686@@ -11,7 +11,7 @@
2687 # Test Quick Config
2688 #########################################################
2689 scale = 3
2690-seconds = 900
2691+seconds = 1800
2692
2693 # amount of time to wait before testing for replicaset
2694 # status

Subscribers

People subscribed via source and target branches