Merge lp:~facundo/charms/trusty/tanuki-spec-manager/first-draft into lp:~tanuki/charms/trusty/tanuki-spec-manager/trunk

Proposed by Facundo Batista
Status: Merged
Merged at revision: 3
Proposed branch: lp:~facundo/charms/trusty/tanuki-spec-manager/first-draft
Merge into: lp:~tanuki/charms/trusty/tanuki-spec-manager/trunk
Diff against target: 11661 lines (+11245/-0)
74 files modified
README (+15/-0)
config.yaml (+10/-0)
hooks/actions.py (+94/-0)
hooks/charmhelpers/__init__.py (+38/-0)
hooks/charmhelpers/contrib/__init__.py (+15/-0)
hooks/charmhelpers/contrib/amulet/__init__.py (+15/-0)
hooks/charmhelpers/contrib/amulet/deployment.py (+93/-0)
hooks/charmhelpers/contrib/amulet/utils.py (+194/-0)
hooks/charmhelpers/contrib/ansible/__init__.py (+190/-0)
hooks/charmhelpers/contrib/charmhelpers/__init__.py (+208/-0)
hooks/charmhelpers/contrib/charmsupport/__init__.py (+15/-0)
hooks/charmhelpers/contrib/charmsupport/nrpe.py (+324/-0)
hooks/charmhelpers/contrib/charmsupport/volumes.py (+175/-0)
hooks/charmhelpers/contrib/database/mysql.py (+372/-0)
hooks/charmhelpers/contrib/hahelpers/__init__.py (+15/-0)
hooks/charmhelpers/contrib/hahelpers/apache.py (+82/-0)
hooks/charmhelpers/contrib/hahelpers/cluster.py (+268/-0)
hooks/charmhelpers/contrib/network/__init__.py (+15/-0)
hooks/charmhelpers/contrib/network/ip.py (+367/-0)
hooks/charmhelpers/contrib/network/ovs/__init__.py (+96/-0)
hooks/charmhelpers/contrib/network/ufw.py (+276/-0)
hooks/charmhelpers/contrib/openstack/__init__.py (+15/-0)
hooks/charmhelpers/contrib/openstack/alternatives.py (+33/-0)
hooks/charmhelpers/contrib/openstack/amulet/__init__.py (+15/-0)
hooks/charmhelpers/contrib/openstack/amulet/deployment.py (+108/-0)
hooks/charmhelpers/contrib/openstack/amulet/utils.py (+294/-0)
hooks/charmhelpers/contrib/openstack/context.py (+1054/-0)
hooks/charmhelpers/contrib/openstack/ip.py (+109/-0)
hooks/charmhelpers/contrib/openstack/neutron.py (+239/-0)
hooks/charmhelpers/contrib/openstack/templates/__init__.py (+18/-0)
hooks/charmhelpers/contrib/openstack/templating.py (+295/-0)
hooks/charmhelpers/contrib/openstack/utils.py (+641/-0)
hooks/charmhelpers/contrib/peerstorage/__init__.py (+148/-0)
hooks/charmhelpers/contrib/python/__init__.py (+15/-0)
hooks/charmhelpers/contrib/python/debug.py (+56/-0)
hooks/charmhelpers/contrib/python/packages.py (+96/-0)
hooks/charmhelpers/contrib/python/rpdb.py (+58/-0)
hooks/charmhelpers/contrib/python/version.py (+34/-0)
hooks/charmhelpers/contrib/saltstack/__init__.py (+118/-0)
hooks/charmhelpers/contrib/ssl/__init__.py (+94/-0)
hooks/charmhelpers/contrib/ssl/service.py (+283/-0)
hooks/charmhelpers/contrib/storage/__init__.py (+15/-0)
hooks/charmhelpers/contrib/storage/linux/__init__.py (+15/-0)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+444/-0)
hooks/charmhelpers/contrib/storage/linux/loopback.py (+78/-0)
hooks/charmhelpers/contrib/storage/linux/lvm.py (+105/-0)
hooks/charmhelpers/contrib/storage/linux/utils.py (+70/-0)
hooks/charmhelpers/contrib/templating/__init__.py (+15/-0)
hooks/charmhelpers/contrib/templating/contexts.py (+134/-0)
hooks/charmhelpers/contrib/templating/jinja.py (+39/-0)
hooks/charmhelpers/contrib/templating/pyformat.py (+29/-0)
hooks/charmhelpers/contrib/unison/__init__.py (+297/-0)
hooks/charmhelpers/core/__init__.py (+15/-0)
hooks/charmhelpers/core/decorators.py (+57/-0)
hooks/charmhelpers/core/fstab.py (+134/-0)
hooks/charmhelpers/core/hookenv.py (+568/-0)
hooks/charmhelpers/core/host.py (+446/-0)
hooks/charmhelpers/core/services/__init__.py (+18/-0)
hooks/charmhelpers/core/services/base.py (+329/-0)
hooks/charmhelpers/core/services/helpers.py (+259/-0)
hooks/charmhelpers/core/sysctl.py (+56/-0)
hooks/charmhelpers/core/templating.py (+68/-0)
hooks/charmhelpers/core/unitdata.py (+477/-0)
hooks/charmhelpers/fetch/__init__.py (+439/-0)
hooks/charmhelpers/fetch/archiveurl.py (+161/-0)
hooks/charmhelpers/fetch/bzrurl.py (+78/-0)
hooks/charmhelpers/fetch/giturl.py (+71/-0)
hooks/charmhelpers/payload/__init__.py (+17/-0)
hooks/charmhelpers/payload/archive.py (+73/-0)
hooks/charmhelpers/payload/execd.py (+66/-0)
hooks/hooks.py (+3/-0)
hooks/services.py (+31/-0)
metadata.yaml (+9/-0)
templates/upstart.conf (+29/-0)
To merge this branch: bzr merge lp:~facundo/charms/trusty/tanuki-spec-manager/first-draft
Reviewer Review Type Date Requested Status
Celso Providelo (community) Approve
Facundo Batista (community) Approve
Review via email: mp+262345@code.launchpad.net

Commit message

First draft of the charm for Spec Manager.

Description of the change

First draft of the charm for Spec Manager.

To post a comment you must log in.
Revision history for this message
Celso Providelo (cprov) wrote :

Facundo,

The diff is truncated, but you can land as long as you stop trying to install the spec_manager module (it will fail, because it's not installed atm) and remove the service action for filling website relation forcibly (remove the wsgi-relation-joined symlink and create the correct website-relation* ones).

[]

review: Needs Fixing
2. By Facundo Batista

Better website relation management.

Revision history for this message
Facundo Batista (facundo) wrote :

Done! will land

review: Approve
3. By Facundo Batista

Proper cleaning and website relation handling.

Revision history for this message
Celso Providelo (cprov) wrote :

Nice, thanks.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'README'
2--- README 1970-01-01 00:00:00 +0000
3+++ README 2015-06-18 17:19:58 +0000
4@@ -0,0 +1,15 @@
5+Overview
6+--------
7+
8+This charm provides the Spec Manager from lp:tanuki-spec-manager
9+
10+Configuration
11+-------------
12+
13+There are only two config options for this charm:
14+ * config-file: A base64 encoded string with the config file to use for
15+ adt-cloud-service.
16+ * environment: a string, one of "development" or "production"
17+
18+NOTE: This expects the payload code in the form of spec-manager.tgz
19+to exist with the snapshot of lp:tanuki-spec-manager that needs deploying
20
21=== added file 'config.yaml'
22--- config.yaml 1970-01-01 00:00:00 +0000
23+++ config.yaml 2015-06-18 17:19:58 +0000
24@@ -0,0 +1,10 @@
25+options:
26+ environment:
27+ type: string
28+ default: "production"
29+ description: |
30+ Environment (devel, staging, production, etc.) that we're running.
31+ config-file:
32+ type: string
33+ description: |
34+ base64 encoded string with the config file for adt-cloud-worker
35
36=== added directory 'files'
37=== added directory 'hooks'
38=== added file 'hooks/actions.py'
39--- hooks/actions.py 1970-01-01 00:00:00 +0000
40+++ hooks/actions.py 2015-06-18 17:19:58 +0000
41@@ -0,0 +1,94 @@
42+import base64
43+import os
44+import subprocess
45+
46+from charmhelpers import fetch
47+from charmhelpers.core import (
48+ hookenv,
49+ host,
50+)
51+from charmhelpers.core.services import helpers
52+from charmhelpers.payload import (archive, execd)
53+
54+
55+REQUIRED_PACKAGES = [
56+ 'python-virtualenv', 'python3-dev',
57+]
58+
59+WSGI_USER = 'www-data'
60+WSGI_GROUP = 'www-data'
61+
62+config = hookenv.config()
63+
64+SERVICE_DIR = '/srv/{}/spec-manager'.format(config['environment'])
65+LOG_DIR = os.path.join(SERVICE_DIR, 'logs')
66+
67+
68+def log_start(service_name):
69+ hookenv.log('spec-manager starting')
70+
71+
72+def basenode(service_name):
73+ hookenv.log("Executing basenode")
74+ execd.execd_preinstall()
75+
76+
77+def install_packages(service_name):
78+ hookenv.log('Installing dependencies...')
79+ fetch.configure_sources(update=True)
80+ fetch.apt_install(REQUIRED_PACKAGES, fatal=True)
81+
82+
83+def get_cloud_service_from_tarball(service_name):
84+ files_dir = os.path.join(hookenv.charm_dir(), 'files')
85+ tarball = os.path.join(files_dir, 'spec-manager.tgz')
86+ if not os.path.exists(SERVICE_DIR):
87+ hookenv.log('Installing the code for the first time from tarball')
88+ archive.extract_tarfile(tarball, os.path.dirname(SERVICE_DIR))
89+
90+
91+def get_config_file(service_name):
92+ config_file = config.get('config-file')
93+ if config_file is not None:
94+ with open(os.path.join(SERVICE_DIR, 'spec-manager.conf'), 'w') as f:
95+ f.write(base64.b64decode(config_file))
96+
97+
98+def install_python_packages(service_name):
99+ env_dir = os.path.join(SERVICE_DIR, 'env')
100+ if os.path.exists(env_dir):
101+ hookenv.log('Service venv already exists, nothing to do ...')
102+ return
103+
104+ hookenv.log('Installing python packages into the venv ...')
105+ subprocess.check_call(['virtualenv', '-p', 'python3', env_dir])
106+
107+ pip_cache = os.path.join(hookenv.charm_dir(), 'files', 'pip-cache')
108+ requirements = os.path.join(SERVICE_DIR, 'requirements_run.txt')
109+
110+ hookenv.log('Installing from download cache.')
111+ subprocess.check_call(['%s/bin/pip' % env_dir,
112+ 'install',
113+ '--no-index',
114+ '--find-links={}'.format(pip_cache),
115+ '-r', requirements])
116+
117+ subprocess.check_call(['%s/bin/pip' % env_dir,
118+ 'install', '--no-deps', '-e', SERVICE_DIR])
119+
120+
121+def prepare_for_wsgi(service_name):
122+ host.adduser(WSGI_USER)
123+ host.add_group(WSGI_GROUP)
124+ host.add_user_to_group(WSGI_USER, WSGI_GROUP)
125+ if not os.path.exists(LOG_DIR):
126+ host.mkdir(LOG_DIR, WSGI_USER, WSGI_GROUP, perms=0o755)
127+
128+
129+class WebsiteRelation(helpers.HttpRelation):
130+
131+ def provide_data(self):
132+ hookenv.log('Providing Website relation data.')
133+ data = super(WebsiteRelation, self).provide_data()
134+ data['port'] = 8000
135+ return data
136
137=== added directory 'hooks/charmhelpers'
138=== added file 'hooks/charmhelpers/__init__.py'
139--- hooks/charmhelpers/__init__.py 1970-01-01 00:00:00 +0000
140+++ hooks/charmhelpers/__init__.py 2015-06-18 17:19:58 +0000
141@@ -0,0 +1,38 @@
142+# Copyright 2014-2015 Canonical Limited.
143+#
144+# This file is part of charm-helpers.
145+#
146+# charm-helpers is free software: you can redistribute it and/or modify
147+# it under the terms of the GNU Lesser General Public License version 3 as
148+# published by the Free Software Foundation.
149+#
150+# charm-helpers is distributed in the hope that it will be useful,
151+# but WITHOUT ANY WARRANTY; without even the implied warranty of
152+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
153+# GNU Lesser General Public License for more details.
154+#
155+# You should have received a copy of the GNU Lesser General Public License
156+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
157+
158+# Bootstrap charm-helpers, installing its dependencies if necessary using
159+# only standard libraries.
160+import subprocess
161+import sys
162+
163+try:
164+ import six # flake8: noqa
165+except ImportError:
166+ if sys.version_info.major == 2:
167+ subprocess.check_call(['apt-get', 'install', '-y', 'python-six'])
168+ else:
169+ subprocess.check_call(['apt-get', 'install', '-y', 'python3-six'])
170+ import six # flake8: noqa
171+
172+try:
173+ import yaml # flake8: noqa
174+except ImportError:
175+ if sys.version_info.major == 2:
176+ subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml'])
177+ else:
178+ subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml'])
179+ import yaml # flake8: noqa
180
181=== added directory 'hooks/charmhelpers/contrib'
182=== added file 'hooks/charmhelpers/contrib/__init__.py'
183--- hooks/charmhelpers/contrib/__init__.py 1970-01-01 00:00:00 +0000
184+++ hooks/charmhelpers/contrib/__init__.py 2015-06-18 17:19:58 +0000
185@@ -0,0 +1,15 @@
186+# Copyright 2014-2015 Canonical Limited.
187+#
188+# This file is part of charm-helpers.
189+#
190+# charm-helpers is free software: you can redistribute it and/or modify
191+# it under the terms of the GNU Lesser General Public License version 3 as
192+# published by the Free Software Foundation.
193+#
194+# charm-helpers is distributed in the hope that it will be useful,
195+# but WITHOUT ANY WARRANTY; without even the implied warranty of
196+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
197+# GNU Lesser General Public License for more details.
198+#
199+# You should have received a copy of the GNU Lesser General Public License
200+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
201
202=== added directory 'hooks/charmhelpers/contrib/amulet'
203=== added file 'hooks/charmhelpers/contrib/amulet/__init__.py'
204--- hooks/charmhelpers/contrib/amulet/__init__.py 1970-01-01 00:00:00 +0000
205+++ hooks/charmhelpers/contrib/amulet/__init__.py 2015-06-18 17:19:58 +0000
206@@ -0,0 +1,15 @@
207+# Copyright 2014-2015 Canonical Limited.
208+#
209+# This file is part of charm-helpers.
210+#
211+# charm-helpers is free software: you can redistribute it and/or modify
212+# it under the terms of the GNU Lesser General Public License version 3 as
213+# published by the Free Software Foundation.
214+#
215+# charm-helpers is distributed in the hope that it will be useful,
216+# but WITHOUT ANY WARRANTY; without even the implied warranty of
217+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
218+# GNU Lesser General Public License for more details.
219+#
220+# You should have received a copy of the GNU Lesser General Public License
221+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
222
223=== added file 'hooks/charmhelpers/contrib/amulet/deployment.py'
224--- hooks/charmhelpers/contrib/amulet/deployment.py 1970-01-01 00:00:00 +0000
225+++ hooks/charmhelpers/contrib/amulet/deployment.py 2015-06-18 17:19:58 +0000
226@@ -0,0 +1,93 @@
227+# Copyright 2014-2015 Canonical Limited.
228+#
229+# This file is part of charm-helpers.
230+#
231+# charm-helpers is free software: you can redistribute it and/or modify
232+# it under the terms of the GNU Lesser General Public License version 3 as
233+# published by the Free Software Foundation.
234+#
235+# charm-helpers is distributed in the hope that it will be useful,
236+# but WITHOUT ANY WARRANTY; without even the implied warranty of
237+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
238+# GNU Lesser General Public License for more details.
239+#
240+# You should have received a copy of the GNU Lesser General Public License
241+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
242+
243+import amulet
244+import os
245+import six
246+
247+
248+class AmuletDeployment(object):
249+ """Amulet deployment.
250+
251+ This class provides generic Amulet deployment and test runner
252+ methods.
253+ """
254+
255+ def __init__(self, series=None):
256+ """Initialize the deployment environment."""
257+ self.series = None
258+
259+ if series:
260+ self.series = series
261+ self.d = amulet.Deployment(series=self.series)
262+ else:
263+ self.d = amulet.Deployment()
264+
265+ def _add_services(self, this_service, other_services):
266+ """Add services.
267+
268+ Add services to the deployment where this_service is the local charm
269+ that we're testing and other_services are the other services that
270+ are being used in the local amulet tests.
271+ """
272+ if this_service['name'] != os.path.basename(os.getcwd()):
273+ s = this_service['name']
274+ msg = "The charm's root directory name needs to be {}".format(s)
275+ amulet.raise_status(amulet.FAIL, msg=msg)
276+
277+ if 'units' not in this_service:
278+ this_service['units'] = 1
279+
280+ self.d.add(this_service['name'], units=this_service['units'])
281+
282+ for svc in other_services:
283+ if 'location' in svc:
284+ branch_location = svc['location']
285+ elif self.series:
286+ branch_location = 'cs:{}/{}'.format(self.series, svc['name']),
287+ else:
288+ branch_location = None
289+
290+ if 'units' not in svc:
291+ svc['units'] = 1
292+
293+ self.d.add(svc['name'], charm=branch_location, units=svc['units'])
294+
295+ def _add_relations(self, relations):
296+ """Add all of the relations for the services."""
297+ for k, v in six.iteritems(relations):
298+ self.d.relate(k, v)
299+
300+ def _configure_services(self, configs):
301+ """Configure all of the services."""
302+ for service, config in six.iteritems(configs):
303+ self.d.configure(service, config)
304+
305+ def _deploy(self):
306+ """Deploy environment and wait for all hooks to finish executing."""
307+ try:
308+ self.d.setup(timeout=900)
309+ self.d.sentry.wait(timeout=900)
310+ except amulet.helpers.TimeoutError:
311+ amulet.raise_status(amulet.FAIL, msg="Deployment timed out")
312+ except Exception:
313+ raise
314+
315+ def run_tests(self):
316+ """Run all of the methods that are prefixed with 'test_'."""
317+ for test in dir(self):
318+ if test.startswith('test_'):
319+ getattr(self, test)()
320
321=== added file 'hooks/charmhelpers/contrib/amulet/utils.py'
322--- hooks/charmhelpers/contrib/amulet/utils.py 1970-01-01 00:00:00 +0000
323+++ hooks/charmhelpers/contrib/amulet/utils.py 2015-06-18 17:19:58 +0000
324@@ -0,0 +1,194 @@
325+# Copyright 2014-2015 Canonical Limited.
326+#
327+# This file is part of charm-helpers.
328+#
329+# charm-helpers is free software: you can redistribute it and/or modify
330+# it under the terms of the GNU Lesser General Public License version 3 as
331+# published by the Free Software Foundation.
332+#
333+# charm-helpers is distributed in the hope that it will be useful,
334+# but WITHOUT ANY WARRANTY; without even the implied warranty of
335+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
336+# GNU Lesser General Public License for more details.
337+#
338+# You should have received a copy of the GNU Lesser General Public License
339+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
340+
341+import ConfigParser
342+import io
343+import logging
344+import re
345+import sys
346+import time
347+
348+import six
349+
350+
351+class AmuletUtils(object):
352+ """Amulet utilities.
353+
354+ This class provides common utility functions that are used by Amulet
355+ tests.
356+ """
357+
358+ def __init__(self, log_level=logging.ERROR):
359+ self.log = self.get_logger(level=log_level)
360+
361+ def get_logger(self, name="amulet-logger", level=logging.DEBUG):
362+ """Get a logger object that will log to stdout."""
363+ log = logging
364+ logger = log.getLogger(name)
365+ fmt = log.Formatter("%(asctime)s %(funcName)s "
366+ "%(levelname)s: %(message)s")
367+
368+ handler = log.StreamHandler(stream=sys.stdout)
369+ handler.setLevel(level)
370+ handler.setFormatter(fmt)
371+
372+ logger.addHandler(handler)
373+ logger.setLevel(level)
374+
375+ return logger
376+
377+ def valid_ip(self, ip):
378+ if re.match(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$", ip):
379+ return True
380+ else:
381+ return False
382+
383+ def valid_url(self, url):
384+ p = re.compile(
385+ r'^(?:http|ftp)s?://'
386+ r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}\.?)|' # noqa
387+ r'localhost|'
388+ r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'
389+ r'(?::\d+)?'
390+ r'(?:/?|[/?]\S+)$',
391+ re.IGNORECASE)
392+ if p.match(url):
393+ return True
394+ else:
395+ return False
396+
397+ def validate_services(self, commands):
398+ """Validate services.
399+
400+ Verify the specified services are running on the corresponding
401+ service units.
402+ """
403+ for k, v in six.iteritems(commands):
404+ for cmd in v:
405+ output, code = k.run(cmd)
406+ if code != 0:
407+ return "command `{}` returned {}".format(cmd, str(code))
408+ return None
409+
410+ def _get_config(self, unit, filename):
411+ """Get a ConfigParser object for parsing a unit's config file."""
412+ file_contents = unit.file_contents(filename)
413+ config = ConfigParser.ConfigParser()
414+ config.readfp(io.StringIO(file_contents))
415+ return config
416+
417+ def validate_config_data(self, sentry_unit, config_file, section,
418+ expected):
419+ """Validate config file data.
420+
421+ Verify that the specified section of the config file contains
422+ the expected option key:value pairs.
423+ """
424+ config = self._get_config(sentry_unit, config_file)
425+
426+ if section != 'DEFAULT' and not config.has_section(section):
427+ return "section [{}] does not exist".format(section)
428+
429+ for k in expected.keys():
430+ if not config.has_option(section, k):
431+ return "section [{}] is missing option {}".format(section, k)
432+ if config.get(section, k) != expected[k]:
433+ return "section [{}] {}:{} != expected {}:{}".format(
434+ section, k, config.get(section, k), k, expected[k])
435+ return None
436+
437+ def _validate_dict_data(self, expected, actual):
438+ """Validate dictionary data.
439+
440+ Compare expected dictionary data vs actual dictionary data.
441+ The values in the 'expected' dictionary can be strings, bools, ints,
442+ longs, or can be a function that evaluate a variable and returns a
443+ bool.
444+ """
445+ for k, v in six.iteritems(expected):
446+ if k in actual:
447+ if (isinstance(v, six.string_types) or
448+ isinstance(v, bool) or
449+ isinstance(v, six.integer_types)):
450+ if v != actual[k]:
451+ return "{}:{}".format(k, actual[k])
452+ elif not v(actual[k]):
453+ return "{}:{}".format(k, actual[k])
454+ else:
455+ return "key '{}' does not exist".format(k)
456+ return None
457+
458+ def validate_relation_data(self, sentry_unit, relation, expected):
459+ """Validate actual relation data based on expected relation data."""
460+ actual = sentry_unit.relation(relation[0], relation[1])
461+ self.log.debug('actual: {}'.format(repr(actual)))
462+ return self._validate_dict_data(expected, actual)
463+
464+ def _validate_list_data(self, expected, actual):
465+ """Compare expected list vs actual list data."""
466+ for e in expected:
467+ if e not in actual:
468+ return "expected item {} not found in actual list".format(e)
469+ return None
470+
471+ def not_null(self, string):
472+ if string is not None:
473+ return True
474+ else:
475+ return False
476+
477+ def _get_file_mtime(self, sentry_unit, filename):
478+ """Get last modification time of file."""
479+ return sentry_unit.file_stat(filename)['mtime']
480+
481+ def _get_dir_mtime(self, sentry_unit, directory):
482+ """Get last modification time of directory."""
483+ return sentry_unit.directory_stat(directory)['mtime']
484+
485+ def _get_proc_start_time(self, sentry_unit, service, pgrep_full=False):
486+ """Get process' start time.
487+
488+ Determine start time of the process based on the last modification
489+ time of the /proc/pid directory. If pgrep_full is True, the process
490+ name is matched against the full command line.
491+ """
492+ if pgrep_full:
493+ cmd = 'pgrep -o -f {}'.format(service)
494+ else:
495+ cmd = 'pgrep -o {}'.format(service)
496+ proc_dir = '/proc/{}'.format(sentry_unit.run(cmd)[0].strip())
497+ return self._get_dir_mtime(sentry_unit, proc_dir)
498+
499+ def service_restarted(self, sentry_unit, service, filename,
500+ pgrep_full=False, sleep_time=20):
501+ """Check if service was restarted.
502+
503+ Compare a service's start time vs a file's last modification time
504+ (such as a config file for that service) to determine if the service
505+ has been restarted.
506+ """
507+ time.sleep(sleep_time)
508+ if (self._get_proc_start_time(sentry_unit, service, pgrep_full) >=
509+ self._get_file_mtime(sentry_unit, filename)):
510+ return True
511+ else:
512+ return False
513+
514+ def relation_error(self, name, data):
515+ return 'unexpected relation data in {} - {}'.format(name, data)
516+
517+ def endpoint_error(self, name, data):
518+ return 'unexpected endpoint data in {} - {}'.format(name, data)
519
520=== added directory 'hooks/charmhelpers/contrib/ansible'
521=== added file 'hooks/charmhelpers/contrib/ansible/__init__.py'
522--- hooks/charmhelpers/contrib/ansible/__init__.py 1970-01-01 00:00:00 +0000
523+++ hooks/charmhelpers/contrib/ansible/__init__.py 2015-06-18 17:19:58 +0000
524@@ -0,0 +1,190 @@
525+# Copyright 2014-2015 Canonical Limited.
526+#
527+# This file is part of charm-helpers.
528+#
529+# charm-helpers is free software: you can redistribute it and/or modify
530+# it under the terms of the GNU Lesser General Public License version 3 as
531+# published by the Free Software Foundation.
532+#
533+# charm-helpers is distributed in the hope that it will be useful,
534+# but WITHOUT ANY WARRANTY; without even the implied warranty of
535+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
536+# GNU Lesser General Public License for more details.
537+#
538+# You should have received a copy of the GNU Lesser General Public License
539+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
540+
541+# Copyright 2013 Canonical Ltd.
542+#
543+# Authors:
544+# Charm Helpers Developers <juju@lists.ubuntu.com>
545+"""Charm Helpers ansible - declare the state of your machines.
546+
547+This helper enables you to declare your machine state, rather than
548+program it procedurally (and have to test each change to your procedures).
549+Your install hook can be as simple as::
550+
551+ {{{
552+ import charmhelpers.contrib.ansible
553+
554+
555+ def install():
556+ charmhelpers.contrib.ansible.install_ansible_support()
557+ charmhelpers.contrib.ansible.apply_playbook('playbooks/install.yaml')
558+ }}}
559+
560+and won't need to change (nor will its tests) when you change the machine
561+state.
562+
563+All of your juju config and relation-data are available as template
564+variables within your playbooks and templates. An install playbook looks
565+something like::
566+
567+ {{{
568+ ---
569+ - hosts: localhost
570+ user: root
571+
572+ tasks:
573+ - name: Add private repositories.
574+ template:
575+ src: ../templates/private-repositories.list.jinja2
576+ dest: /etc/apt/sources.list.d/private.list
577+
578+ - name: Update the cache.
579+ apt: update_cache=yes
580+
581+ - name: Install dependencies.
582+ apt: pkg={{ item }}
583+ with_items:
584+ - python-mimeparse
585+ - python-webob
586+ - sunburnt
587+
588+ - name: Setup groups.
589+ group: name={{ item.name }} gid={{ item.gid }}
590+ with_items:
591+ - { name: 'deploy_user', gid: 1800 }
592+ - { name: 'service_user', gid: 1500 }
593+
594+ ...
595+ }}}
596+
597+Read more online about `playbooks`_ and standard ansible `modules`_.
598+
599+.. _playbooks: http://www.ansibleworks.com/docs/playbooks.html
600+.. _modules: http://www.ansibleworks.com/docs/modules.html
601+
602+"""
603+import os
604+import subprocess
605+
606+import charmhelpers.contrib.templating.contexts
607+import charmhelpers.core.host
608+import charmhelpers.core.hookenv
609+import charmhelpers.fetch
610+
611+
612+charm_dir = os.environ.get('CHARM_DIR', '')
613+ansible_hosts_path = '/etc/ansible/hosts'
614+# Ansible will automatically include any vars in the following
615+# file in its inventory when run locally.
616+ansible_vars_path = '/etc/ansible/host_vars/localhost'
617+
618+
619+def install_ansible_support(from_ppa=True, ppa_location='ppa:rquillo/ansible'):
620+ """Installs the ansible package.
621+
622+ By default it is installed from the `PPA`_ linked from
623+ the ansible `website`_ or from a ppa specified by a charm config..
624+
625+ .. _PPA: https://launchpad.net/~rquillo/+archive/ansible
626+ .. _website: http://docs.ansible.com/intro_installation.html#latest-releases-via-apt-ubuntu
627+
628+ If from_ppa is empty, you must ensure that the package is available
629+ from a configured repository.
630+ """
631+ if from_ppa:
632+ charmhelpers.fetch.add_source(ppa_location)
633+ charmhelpers.fetch.apt_update(fatal=True)
634+ charmhelpers.fetch.apt_install('ansible')
635+ with open(ansible_hosts_path, 'w+') as hosts_file:
636+ hosts_file.write('localhost ansible_connection=local')
637+
638+
639+def apply_playbook(playbook, tags=None):
640+ tags = tags or []
641+ tags = ",".join(tags)
642+ charmhelpers.contrib.templating.contexts.juju_state_to_yaml(
643+ ansible_vars_path, namespace_separator='__',
644+ allow_hyphens_in_keys=False)
645+ # we want ansible's log output to be unbuffered
646+ env = os.environ.copy()
647+ env['PYTHONUNBUFFERED'] = "1"
648+ call = [
649+ 'ansible-playbook',
650+ '-c',
651+ 'local',
652+ playbook,
653+ ]
654+ if tags:
655+ call.extend(['--tags', '{}'.format(tags)])
656+ subprocess.check_call(call, env=env)
657+
658+
659+class AnsibleHooks(charmhelpers.core.hookenv.Hooks):
660+ """Run a playbook with the hook-name as the tag.
661+
662+ This helper builds on the standard hookenv.Hooks helper,
663+ but additionally runs the playbook with the hook-name specified
664+ using --tags (ie. running all the tasks tagged with the hook-name).
665+
666+ Example::
667+
668+ hooks = AnsibleHooks(playbook_path='playbooks/my_machine_state.yaml')
669+
670+ # All the tasks within my_machine_state.yaml tagged with 'install'
671+ # will be run automatically after do_custom_work()
672+ @hooks.hook()
673+ def install():
674+ do_custom_work()
675+
676+ # For most of your hooks, you won't need to do anything other
677+ # than run the tagged tasks for the hook:
678+ @hooks.hook('config-changed', 'start', 'stop')
679+ def just_use_playbook():
680+ pass
681+
682+ # As a convenience, you can avoid the above noop function by specifying
683+ # the hooks which are handled by ansible-only and they'll be registered
684+ # for you:
685+ # hooks = AnsibleHooks(
686+ # 'playbooks/my_machine_state.yaml',
687+ # default_hooks=['config-changed', 'start', 'stop'])
688+
689+ if __name__ == "__main__":
690+ # execute a hook based on the name the program is called by
691+ hooks.execute(sys.argv)
692+
693+ """
694+
695+ def __init__(self, playbook_path, default_hooks=None):
696+ """Register any hooks handled by ansible."""
697+ super(AnsibleHooks, self).__init__()
698+
699+ self.playbook_path = playbook_path
700+
701+ default_hooks = default_hooks or []
702+
703+ def noop(*args, **kwargs):
704+ pass
705+
706+ for hook in default_hooks:
707+ self.register(hook, noop)
708+
709+ def execute(self, args):
710+ """Execute the hook followed by the playbook using the hook as tag."""
711+ super(AnsibleHooks, self).execute(args)
712+ hook_name = os.path.basename(args[0])
713+ charmhelpers.contrib.ansible.apply_playbook(
714+ self.playbook_path, tags=[hook_name])
715
716=== added directory 'hooks/charmhelpers/contrib/charmhelpers'
717=== added file 'hooks/charmhelpers/contrib/charmhelpers/__init__.py'
718--- hooks/charmhelpers/contrib/charmhelpers/__init__.py 1970-01-01 00:00:00 +0000
719+++ hooks/charmhelpers/contrib/charmhelpers/__init__.py 2015-06-18 17:19:58 +0000
720@@ -0,0 +1,208 @@
721+# Copyright 2014-2015 Canonical Limited.
722+#
723+# This file is part of charm-helpers.
724+#
725+# charm-helpers is free software: you can redistribute it and/or modify
726+# it under the terms of the GNU Lesser General Public License version 3 as
727+# published by the Free Software Foundation.
728+#
729+# charm-helpers is distributed in the hope that it will be useful,
730+# but WITHOUT ANY WARRANTY; without even the implied warranty of
731+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
732+# GNU Lesser General Public License for more details.
733+#
734+# You should have received a copy of the GNU Lesser General Public License
735+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
736+
737+# Copyright 2012 Canonical Ltd. This software is licensed under the
738+# GNU Affero General Public License version 3 (see the file LICENSE).
739+
740+import warnings
741+warnings.warn("contrib.charmhelpers is deprecated", DeprecationWarning) # noqa
742+
743+import operator
744+import tempfile
745+import time
746+import yaml
747+import subprocess
748+
749+import six
750+if six.PY3:
751+ from urllib.request import urlopen
752+ from urllib.error import (HTTPError, URLError)
753+else:
754+ from urllib2 import (urlopen, HTTPError, URLError)
755+
756+"""Helper functions for writing Juju charms in Python."""
757+
758+__metaclass__ = type
759+__all__ = [
760+ # 'get_config', # core.hookenv.config()
761+ # 'log', # core.hookenv.log()
762+ # 'log_entry', # core.hookenv.log()
763+ # 'log_exit', # core.hookenv.log()
764+ # 'relation_get', # core.hookenv.relation_get()
765+ # 'relation_set', # core.hookenv.relation_set()
766+ # 'relation_ids', # core.hookenv.relation_ids()
767+ # 'relation_list', # core.hookenv.relation_units()
768+ # 'config_get', # core.hookenv.config()
769+ # 'unit_get', # core.hookenv.unit_get()
770+ # 'open_port', # core.hookenv.open_port()
771+ # 'close_port', # core.hookenv.close_port()
772+ # 'service_control', # core.host.service()
773+ 'unit_info', # client-side, NOT IMPLEMENTED
774+ 'wait_for_machine', # client-side, NOT IMPLEMENTED
775+ 'wait_for_page_contents', # client-side, NOT IMPLEMENTED
776+ 'wait_for_relation', # client-side, NOT IMPLEMENTED
777+ 'wait_for_unit', # client-side, NOT IMPLEMENTED
778+]
779+
780+
781+SLEEP_AMOUNT = 0.1
782+
783+
784+# We create a juju_status Command here because it makes testing much,
785+# much easier.
786+def juju_status():
787+ subprocess.check_call(['juju', 'status'])
788+
789+# re-implemented as charmhelpers.fetch.configure_sources()
790+# def configure_source(update=False):
791+# source = config_get('source')
792+# if ((source.startswith('ppa:') or
793+# source.startswith('cloud:') or
794+# source.startswith('http:'))):
795+# run('add-apt-repository', source)
796+# if source.startswith("http:"):
797+# run('apt-key', 'import', config_get('key'))
798+# if update:
799+# run('apt-get', 'update')
800+
801+
802+# DEPRECATED: client-side only
803+def make_charm_config_file(charm_config):
804+ charm_config_file = tempfile.NamedTemporaryFile(mode='w+')
805+ charm_config_file.write(yaml.dump(charm_config))
806+ charm_config_file.flush()
807+ # The NamedTemporaryFile instance is returned instead of just the name
808+ # because we want to take advantage of garbage collection-triggered
809+ # deletion of the temp file when it goes out of scope in the caller.
810+ return charm_config_file
811+
812+
813+# DEPRECATED: client-side only
814+def unit_info(service_name, item_name, data=None, unit=None):
815+ if data is None:
816+ data = yaml.safe_load(juju_status())
817+ service = data['services'].get(service_name)
818+ if service is None:
819+ # XXX 2012-02-08 gmb:
820+ # This allows us to cope with the race condition that we
821+ # have between deploying a service and having it come up in
822+ # `juju status`. We could probably do with cleaning it up so
823+ # that it fails a bit more noisily after a while.
824+ return ''
825+ units = service['units']
826+ if unit is not None:
827+ item = units[unit][item_name]
828+ else:
829+ # It might seem odd to sort the units here, but we do it to
830+ # ensure that when no unit is specified, the first unit for the
831+ # service (or at least the one with the lowest number) is the
832+ # one whose data gets returned.
833+ sorted_unit_names = sorted(units.keys())
834+ item = units[sorted_unit_names[0]][item_name]
835+ return item
836+
837+
838+# DEPRECATED: client-side only
839+def get_machine_data():
840+ return yaml.safe_load(juju_status())['machines']
841+
842+
843+# DEPRECATED: client-side only
844+def wait_for_machine(num_machines=1, timeout=300):
845+ """Wait `timeout` seconds for `num_machines` machines to come up.
846+
847+ This wait_for... function can be called by other wait_for functions
848+ whose timeouts might be too short in situations where only a bare
849+ Juju setup has been bootstrapped.
850+
851+ :return: A tuple of (num_machines, time_taken). This is used for
852+ testing.
853+ """
854+ # You may think this is a hack, and you'd be right. The easiest way
855+ # to tell what environment we're working in (LXC vs EC2) is to check
856+ # the dns-name of the first machine. If it's localhost we're in LXC
857+ # and we can just return here.
858+ if get_machine_data()[0]['dns-name'] == 'localhost':
859+ return 1, 0
860+ start_time = time.time()
861+ while True:
862+ # Drop the first machine, since it's the Zookeeper and that's
863+ # not a machine that we need to wait for. This will only work
864+ # for EC2 environments, which is why we return early above if
865+ # we're in LXC.
866+ machine_data = get_machine_data()
867+ non_zookeeper_machines = [
868+ machine_data[key] for key in list(machine_data.keys())[1:]]
869+ if len(non_zookeeper_machines) >= num_machines:
870+ all_machines_running = True
871+ for machine in non_zookeeper_machines:
872+ if machine.get('instance-state') != 'running':
873+ all_machines_running = False
874+ break
875+ if all_machines_running:
876+ break
877+ if time.time() - start_time >= timeout:
878+ raise RuntimeError('timeout waiting for service to start')
879+ time.sleep(SLEEP_AMOUNT)
880+ return num_machines, time.time() - start_time
881+
882+
883+# DEPRECATED: client-side only
884+def wait_for_unit(service_name, timeout=480):
885+ """Wait `timeout` seconds for a given service name to come up."""
886+ wait_for_machine(num_machines=1)
887+ start_time = time.time()
888+ while True:
889+ state = unit_info(service_name, 'agent-state')
890+ if 'error' in state or state == 'started':
891+ break
892+ if time.time() - start_time >= timeout:
893+ raise RuntimeError('timeout waiting for service to start')
894+ time.sleep(SLEEP_AMOUNT)
895+ if state != 'started':
896+ raise RuntimeError('unit did not start, agent-state: ' + state)
897+
898+
899+# DEPRECATED: client-side only
900+def wait_for_relation(service_name, relation_name, timeout=120):
901+ """Wait `timeout` seconds for a given relation to come up."""
902+ start_time = time.time()
903+ while True:
904+ relation = unit_info(service_name, 'relations').get(relation_name)
905+ if relation is not None and relation['state'] == 'up':
906+ break
907+ if time.time() - start_time >= timeout:
908+ raise RuntimeError('timeout waiting for relation to be up')
909+ time.sleep(SLEEP_AMOUNT)
910+
911+
912+# DEPRECATED: client-side only
913+def wait_for_page_contents(url, contents, timeout=120, validate=None):
914+ if validate is None:
915+ validate = operator.contains
916+ start_time = time.time()
917+ while True:
918+ try:
919+ stream = urlopen(url)
920+ except (HTTPError, URLError):
921+ pass
922+ else:
923+ page = stream.read()
924+ if validate(page, contents):
925+ return page
926+ if time.time() - start_time >= timeout:
927+ raise RuntimeError('timeout waiting for contents of ' + url)
928+ time.sleep(SLEEP_AMOUNT)
929
930=== added directory 'hooks/charmhelpers/contrib/charmsupport'
931=== added file 'hooks/charmhelpers/contrib/charmsupport/__init__.py'
932--- hooks/charmhelpers/contrib/charmsupport/__init__.py 1970-01-01 00:00:00 +0000
933+++ hooks/charmhelpers/contrib/charmsupport/__init__.py 2015-06-18 17:19:58 +0000
934@@ -0,0 +1,15 @@
935+# Copyright 2014-2015 Canonical Limited.
936+#
937+# This file is part of charm-helpers.
938+#
939+# charm-helpers is free software: you can redistribute it and/or modify
940+# it under the terms of the GNU Lesser General Public License version 3 as
941+# published by the Free Software Foundation.
942+#
943+# charm-helpers is distributed in the hope that it will be useful,
944+# but WITHOUT ANY WARRANTY; without even the implied warranty of
945+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
946+# GNU Lesser General Public License for more details.
947+#
948+# You should have received a copy of the GNU Lesser General Public License
949+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
950
951=== added file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py'
952--- hooks/charmhelpers/contrib/charmsupport/nrpe.py 1970-01-01 00:00:00 +0000
953+++ hooks/charmhelpers/contrib/charmsupport/nrpe.py 2015-06-18 17:19:58 +0000
954@@ -0,0 +1,324 @@
955+# Copyright 2014-2015 Canonical Limited.
956+#
957+# This file is part of charm-helpers.
958+#
959+# charm-helpers is free software: you can redistribute it and/or modify
960+# it under the terms of the GNU Lesser General Public License version 3 as
961+# published by the Free Software Foundation.
962+#
963+# charm-helpers is distributed in the hope that it will be useful,
964+# but WITHOUT ANY WARRANTY; without even the implied warranty of
965+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
966+# GNU Lesser General Public License for more details.
967+#
968+# You should have received a copy of the GNU Lesser General Public License
969+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
970+
971+"""Compatibility with the nrpe-external-master charm"""
972+# Copyright 2012 Canonical Ltd.
973+#
974+# Authors:
975+# Matthew Wedgwood <matthew.wedgwood@canonical.com>
976+
977+import subprocess
978+import pwd
979+import grp
980+import os
981+import re
982+import shlex
983+import yaml
984+
985+from charmhelpers.core.hookenv import (
986+ config,
987+ local_unit,
988+ log,
989+ relation_ids,
990+ relation_set,
991+ relations_of_type,
992+)
993+
994+from charmhelpers.core.host import service
995+
996+# This module adds compatibility with the nrpe-external-master and plain nrpe
997+# subordinate charms. To use it in your charm:
998+#
999+# 1. Update metadata.yaml
1000+#
1001+# provides:
1002+# (...)
1003+# nrpe-external-master:
1004+# interface: nrpe-external-master
1005+# scope: container
1006+#
1007+# and/or
1008+#
1009+# provides:
1010+# (...)
1011+# local-monitors:
1012+# interface: local-monitors
1013+# scope: container
1014+
1015+#
1016+# 2. Add the following to config.yaml
1017+#
1018+# nagios_context:
1019+# default: "juju"
1020+# type: string
1021+# description: |
1022+# Used by the nrpe subordinate charms.
1023+# A string that will be prepended to instance name to set the host name
1024+# in nagios. So for instance the hostname would be something like:
1025+# juju-myservice-0
1026+# If you're running multiple environments with the same services in them
1027+# this allows you to differentiate between them.
1028+# nagios_servicegroups:
1029+# default: ""
1030+# type: string
1031+# description: |
1032+# A comma-separated list of nagios servicegroups.
1033+# If left empty, the nagios_context will be used as the servicegroup
1034+#
1035+# 3. Add custom checks (Nagios plugins) to files/nrpe-external-master
1036+#
1037+# 4. Update your hooks.py with something like this:
1038+#
1039+# from charmsupport.nrpe import NRPE
1040+# (...)
1041+# def update_nrpe_config():
1042+# nrpe_compat = NRPE()
1043+# nrpe_compat.add_check(
1044+# shortname = "myservice",
1045+# description = "Check MyService",
1046+# check_cmd = "check_http -w 2 -c 10 http://localhost"
1047+# )
1048+# nrpe_compat.add_check(
1049+# "myservice_other",
1050+# "Check for widget failures",
1051+# check_cmd = "/srv/myapp/scripts/widget_check"
1052+# )
1053+# nrpe_compat.write()
1054+#
1055+# def config_changed():
1056+# (...)
1057+# update_nrpe_config()
1058+#
1059+# def nrpe_external_master_relation_changed():
1060+# update_nrpe_config()
1061+#
1062+# def local_monitors_relation_changed():
1063+# update_nrpe_config()
1064+#
1065+# 5. ln -s hooks.py nrpe-external-master-relation-changed
1066+# ln -s hooks.py local-monitors-relation-changed
1067+
1068+
1069+class CheckException(Exception):
1070+ pass
1071+
1072+
1073+class Check(object):
1074+ shortname_re = '[A-Za-z0-9-_]+$'
1075+ service_template = ("""
1076+#---------------------------------------------------
1077+# This file is Juju managed
1078+#---------------------------------------------------
1079+define service {{
1080+ use active-service
1081+ host_name {nagios_hostname}
1082+ service_description {nagios_hostname}[{shortname}] """
1083+ """{description}
1084+ check_command check_nrpe!{command}
1085+ servicegroups {nagios_servicegroup}
1086+}}
1087+""")
1088+
1089+ def __init__(self, shortname, description, check_cmd):
1090+ super(Check, self).__init__()
1091+ # XXX: could be better to calculate this from the service name
1092+ if not re.match(self.shortname_re, shortname):
1093+ raise CheckException("shortname must match {}".format(
1094+ Check.shortname_re))
1095+ self.shortname = shortname
1096+ self.command = "check_{}".format(shortname)
1097+ # Note: a set of invalid characters is defined by the
1098+ # Nagios server config
1099+ # The default is: illegal_object_name_chars=`~!$%^&*"|'<>?,()=
1100+ self.description = description
1101+ self.check_cmd = self._locate_cmd(check_cmd)
1102+
1103+ def _locate_cmd(self, check_cmd):
1104+ search_path = (
1105+ '/usr/lib/nagios/plugins',
1106+ '/usr/local/lib/nagios/plugins',
1107+ )
1108+ parts = shlex.split(check_cmd)
1109+ for path in search_path:
1110+ if os.path.exists(os.path.join(path, parts[0])):
1111+ command = os.path.join(path, parts[0])
1112+ if len(parts) > 1:
1113+ command += " " + " ".join(parts[1:])
1114+ return command
1115+ log('Check command not found: {}'.format(parts[0]))
1116+ return ''
1117+
1118+ def write(self, nagios_context, hostname, nagios_servicegroups=None):
1119+ nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format(
1120+ self.command)
1121+ with open(nrpe_check_file, 'w') as nrpe_check_config:
1122+ nrpe_check_config.write("# check {}\n".format(self.shortname))
1123+ nrpe_check_config.write("command[{}]={}\n".format(
1124+ self.command, self.check_cmd))
1125+
1126+ if not os.path.exists(NRPE.nagios_exportdir):
1127+ log('Not writing service config as {} is not accessible'.format(
1128+ NRPE.nagios_exportdir))
1129+ else:
1130+ self.write_service_config(nagios_context, hostname,
1131+ nagios_servicegroups)
1132+
1133+ def write_service_config(self, nagios_context, hostname,
1134+ nagios_servicegroups=None):
1135+ for f in os.listdir(NRPE.nagios_exportdir):
1136+ if re.search('.*{}.cfg'.format(self.command), f):
1137+ os.remove(os.path.join(NRPE.nagios_exportdir, f))
1138+
1139+ if not nagios_servicegroups:
1140+ nagios_servicegroups = nagios_context
1141+
1142+ templ_vars = {
1143+ 'nagios_hostname': hostname,
1144+ 'nagios_servicegroup': nagios_servicegroups,
1145+ 'description': self.description,
1146+ 'shortname': self.shortname,
1147+ 'command': self.command,
1148+ }
1149+ nrpe_service_text = Check.service_template.format(**templ_vars)
1150+ nrpe_service_file = '{}/service__{}_{}.cfg'.format(
1151+ NRPE.nagios_exportdir, hostname, self.command)
1152+ with open(nrpe_service_file, 'w') as nrpe_service_config:
1153+ nrpe_service_config.write(str(nrpe_service_text))
1154+
1155+ def run(self):
1156+ subprocess.call(self.check_cmd)
1157+
1158+
1159+class NRPE(object):
1160+ nagios_logdir = '/var/log/nagios'
1161+ nagios_exportdir = '/var/lib/nagios/export'
1162+ nrpe_confdir = '/etc/nagios/nrpe.d'
1163+
1164+ def __init__(self, hostname=None):
1165+ super(NRPE, self).__init__()
1166+ self.config = config()
1167+ self.nagios_context = self.config['nagios_context']
1168+ if 'nagios_servicegroups' in self.config:
1169+ self.nagios_servicegroups = self.config['nagios_servicegroups']
1170+ else:
1171+ self.nagios_servicegroups = 'juju'
1172+ self.unit_name = local_unit().replace('/', '-')
1173+ if hostname:
1174+ self.hostname = hostname
1175+ else:
1176+ self.hostname = "{}-{}".format(self.nagios_context, self.unit_name)
1177+ self.checks = []
1178+
1179+ def add_check(self, *args, **kwargs):
1180+ self.checks.append(Check(*args, **kwargs))
1181+
1182+ def write(self):
1183+ try:
1184+ nagios_uid = pwd.getpwnam('nagios').pw_uid
1185+ nagios_gid = grp.getgrnam('nagios').gr_gid
1186+ except:
1187+ log("Nagios user not set up, nrpe checks not updated")
1188+ return
1189+
1190+ if not os.path.exists(NRPE.nagios_logdir):
1191+ os.mkdir(NRPE.nagios_logdir)
1192+ os.chown(NRPE.nagios_logdir, nagios_uid, nagios_gid)
1193+
1194+ nrpe_monitors = {}
1195+ monitors = {"monitors": {"remote": {"nrpe": nrpe_monitors}}}
1196+ for nrpecheck in self.checks:
1197+ nrpecheck.write(self.nagios_context, self.hostname,
1198+ self.nagios_servicegroups)
1199+ nrpe_monitors[nrpecheck.shortname] = {
1200+ "command": nrpecheck.command,
1201+ }
1202+
1203+ service('restart', 'nagios-nrpe-server')
1204+
1205+ for rid in relation_ids("local-monitors"):
1206+ relation_set(relation_id=rid, monitors=yaml.dump(monitors))
1207+
1208+
1209+def get_nagios_hostcontext(relation_name='nrpe-external-master'):
1210+ """
1211+ Query relation with nrpe subordinate, return the nagios_host_context
1212+
1213+ :param str relation_name: Name of relation nrpe sub joined to
1214+ """
1215+ for rel in relations_of_type(relation_name):
1216+ if 'nagios_hostname' in rel:
1217+ return rel['nagios_host_context']
1218+
1219+
1220+def get_nagios_hostname(relation_name='nrpe-external-master'):
1221+ """
1222+ Query relation with nrpe subordinate, return the nagios_hostname
1223+
1224+ :param str relation_name: Name of relation nrpe sub joined to
1225+ """
1226+ for rel in relations_of_type(relation_name):
1227+ if 'nagios_hostname' in rel:
1228+ return rel['nagios_hostname']
1229+
1230+
1231+def get_nagios_unit_name(relation_name='nrpe-external-master'):
1232+ """
1233+ Return the nagios unit name prepended with host_context if needed
1234+
1235+ :param str relation_name: Name of relation nrpe sub joined to
1236+ """
1237+ host_context = get_nagios_hostcontext(relation_name)
1238+ if host_context:
1239+ unit = "%s:%s" % (host_context, local_unit())
1240+ else:
1241+ unit = local_unit()
1242+ return unit
1243+
1244+
1245+def add_init_service_checks(nrpe, services, unit_name):
1246+ """
1247+ Add checks for each service in list
1248+
1249+ :param NRPE nrpe: NRPE object to add check to
1250+ :param list services: List of services to check
1251+ :param str unit_name: Unit name to use in check description
1252+ """
1253+ for svc in services:
1254+ upstart_init = '/etc/init/%s.conf' % svc
1255+ sysv_init = '/etc/init.d/%s' % svc
1256+ if os.path.exists(upstart_init):
1257+ nrpe.add_check(
1258+ shortname=svc,
1259+ description='process check {%s}' % unit_name,
1260+ check_cmd='check_upstart_job %s' % svc
1261+ )
1262+ elif os.path.exists(sysv_init):
1263+ cronpath = '/etc/cron.d/nagios-service-check-%s' % svc
1264+ cron_file = ('*/5 * * * * root '
1265+ '/usr/local/lib/nagios/plugins/check_exit_status.pl '
1266+ '-s /etc/init.d/%s status > '
1267+ '/var/lib/nagios/service-check-%s.txt\n' % (svc,
1268+ svc)
1269+ )
1270+ f = open(cronpath, 'w')
1271+ f.write(cron_file)
1272+ f.close()
1273+ nrpe.add_check(
1274+ shortname=svc,
1275+ description='process check {%s}' % unit_name,
1276+ check_cmd='check_status_file.py -f '
1277+ '/var/lib/nagios/service-check-%s.txt' % svc,
1278+ )
1279
1280=== added file 'hooks/charmhelpers/contrib/charmsupport/volumes.py'
1281--- hooks/charmhelpers/contrib/charmsupport/volumes.py 1970-01-01 00:00:00 +0000
1282+++ hooks/charmhelpers/contrib/charmsupport/volumes.py 2015-06-18 17:19:58 +0000
1283@@ -0,0 +1,175 @@
1284+# Copyright 2014-2015 Canonical Limited.
1285+#
1286+# This file is part of charm-helpers.
1287+#
1288+# charm-helpers is free software: you can redistribute it and/or modify
1289+# it under the terms of the GNU Lesser General Public License version 3 as
1290+# published by the Free Software Foundation.
1291+#
1292+# charm-helpers is distributed in the hope that it will be useful,
1293+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1294+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1295+# GNU Lesser General Public License for more details.
1296+#
1297+# You should have received a copy of the GNU Lesser General Public License
1298+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1299+
1300+'''
1301+Functions for managing volumes in juju units. One volume is supported per unit.
1302+Subordinates may have their own storage, provided it is on its own partition.
1303+
1304+Configuration stanzas::
1305+
1306+ volume-ephemeral:
1307+ type: boolean
1308+ default: true
1309+ description: >
1310+ If false, a volume is mounted as sepecified in "volume-map"
1311+ If true, ephemeral storage will be used, meaning that log data
1312+ will only exist as long as the machine. YOU HAVE BEEN WARNED.
1313+ volume-map:
1314+ type: string
1315+ default: {}
1316+ description: >
1317+ YAML map of units to device names, e.g:
1318+ "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }"
1319+ Service units will raise a configure-error if volume-ephemeral
1320+ is 'true' and no volume-map value is set. Use 'juju set' to set a
1321+ value and 'juju resolved' to complete configuration.
1322+
1323+Usage::
1324+
1325+ from charmsupport.volumes import configure_volume, VolumeConfigurationError
1326+ from charmsupport.hookenv import log, ERROR
1327+ def post_mount_hook():
1328+ stop_service('myservice')
1329+ def post_mount_hook():
1330+ start_service('myservice')
1331+
1332+ if __name__ == '__main__':
1333+ try:
1334+ configure_volume(before_change=pre_mount_hook,
1335+ after_change=post_mount_hook)
1336+ except VolumeConfigurationError:
1337+ log('Storage could not be configured', ERROR)
1338+
1339+'''
1340+
1341+# XXX: Known limitations
1342+# - fstab is neither consulted nor updated
1343+
1344+import os
1345+from charmhelpers.core import hookenv
1346+from charmhelpers.core import host
1347+import yaml
1348+
1349+
1350+MOUNT_BASE = '/srv/juju/volumes'
1351+
1352+
1353+class VolumeConfigurationError(Exception):
1354+ '''Volume configuration data is missing or invalid'''
1355+ pass
1356+
1357+
1358+def get_config():
1359+ '''Gather and sanity-check volume configuration data'''
1360+ volume_config = {}
1361+ config = hookenv.config()
1362+
1363+ errors = False
1364+
1365+ if config.get('volume-ephemeral') in (True, 'True', 'true', 'Yes', 'yes'):
1366+ volume_config['ephemeral'] = True
1367+ else:
1368+ volume_config['ephemeral'] = False
1369+
1370+ try:
1371+ volume_map = yaml.safe_load(config.get('volume-map', '{}'))
1372+ except yaml.YAMLError as e:
1373+ hookenv.log("Error parsing YAML volume-map: {}".format(e),
1374+ hookenv.ERROR)
1375+ errors = True
1376+ if volume_map is None:
1377+ # probably an empty string
1378+ volume_map = {}
1379+ elif not isinstance(volume_map, dict):
1380+ hookenv.log("Volume-map should be a dictionary, not {}".format(
1381+ type(volume_map)))
1382+ errors = True
1383+
1384+ volume_config['device'] = volume_map.get(os.environ['JUJU_UNIT_NAME'])
1385+ if volume_config['device'] and volume_config['ephemeral']:
1386+ # asked for ephemeral storage but also defined a volume ID
1387+ hookenv.log('A volume is defined for this unit, but ephemeral '
1388+ 'storage was requested', hookenv.ERROR)
1389+ errors = True
1390+ elif not volume_config['device'] and not volume_config['ephemeral']:
1391+ # asked for permanent storage but did not define volume ID
1392+ hookenv.log('Ephemeral storage was requested, but there is no volume '
1393+ 'defined for this unit.', hookenv.ERROR)
1394+ errors = True
1395+
1396+ unit_mount_name = hookenv.local_unit().replace('/', '-')
1397+ volume_config['mountpoint'] = os.path.join(MOUNT_BASE, unit_mount_name)
1398+
1399+ if errors:
1400+ return None
1401+ return volume_config
1402+
1403+
1404+def mount_volume(config):
1405+ if os.path.exists(config['mountpoint']):
1406+ if not os.path.isdir(config['mountpoint']):
1407+ hookenv.log('Not a directory: {}'.format(config['mountpoint']))
1408+ raise VolumeConfigurationError()
1409+ else:
1410+ host.mkdir(config['mountpoint'])
1411+ if os.path.ismount(config['mountpoint']):
1412+ unmount_volume(config)
1413+ if not host.mount(config['device'], config['mountpoint'], persist=True):
1414+ raise VolumeConfigurationError()
1415+
1416+
1417+def unmount_volume(config):
1418+ if os.path.ismount(config['mountpoint']):
1419+ if not host.umount(config['mountpoint'], persist=True):
1420+ raise VolumeConfigurationError()
1421+
1422+
1423+def managed_mounts():
1424+ '''List of all mounted managed volumes'''
1425+ return filter(lambda mount: mount[0].startswith(MOUNT_BASE), host.mounts())
1426+
1427+
1428+def configure_volume(before_change=lambda: None, after_change=lambda: None):
1429+ '''Set up storage (or don't) according to the charm's volume configuration.
1430+ Returns the mount point or "ephemeral". before_change and after_change
1431+ are optional functions to be called if the volume configuration changes.
1432+ '''
1433+
1434+ config = get_config()
1435+ if not config:
1436+ hookenv.log('Failed to read volume configuration', hookenv.CRITICAL)
1437+ raise VolumeConfigurationError()
1438+
1439+ if config['ephemeral']:
1440+ if os.path.ismount(config['mountpoint']):
1441+ before_change()
1442+ unmount_volume(config)
1443+ after_change()
1444+ return 'ephemeral'
1445+ else:
1446+ # persistent storage
1447+ if os.path.ismount(config['mountpoint']):
1448+ mounts = dict(managed_mounts())
1449+ if mounts.get(config['mountpoint']) != config['device']:
1450+ before_change()
1451+ unmount_volume(config)
1452+ mount_volume(config)
1453+ after_change()
1454+ else:
1455+ before_change()
1456+ mount_volume(config)
1457+ after_change()
1458+ return config['mountpoint']
1459
1460=== added directory 'hooks/charmhelpers/contrib/database'
1461=== added file 'hooks/charmhelpers/contrib/database/__init__.py'
1462=== added file 'hooks/charmhelpers/contrib/database/mysql.py'
1463--- hooks/charmhelpers/contrib/database/mysql.py 1970-01-01 00:00:00 +0000
1464+++ hooks/charmhelpers/contrib/database/mysql.py 2015-06-18 17:19:58 +0000
1465@@ -0,0 +1,372 @@
1466+"""Helper for working with a MySQL database"""
1467+import json
1468+import socket
1469+import re
1470+import sys
1471+import platform
1472+import os
1473+import glob
1474+
1475+from string import upper
1476+
1477+from charmhelpers.core.host import (
1478+ mkdir,
1479+ pwgen,
1480+ write_file
1481+)
1482+from charmhelpers.core.hookenv import (
1483+ relation_get,
1484+ related_units,
1485+ unit_get,
1486+ log,
1487+ DEBUG,
1488+ INFO,
1489+)
1490+from charmhelpers.core.hookenv import config as config_get
1491+from charmhelpers.fetch import (
1492+ apt_install,
1493+ apt_update,
1494+ filter_installed_packages,
1495+)
1496+from charmhelpers.contrib.peerstorage import (
1497+ peer_store,
1498+ peer_retrieve,
1499+)
1500+
1501+try:
1502+ import MySQLdb
1503+except ImportError:
1504+ apt_update(fatal=True)
1505+ apt_install(filter_installed_packages(['python-mysqldb']), fatal=True)
1506+ import MySQLdb
1507+
1508+
1509+class MySQLHelper(object):
1510+
1511+ def __init__(self, rpasswdf_template, upasswdf_template, host='localhost'):
1512+ self.host = host
1513+ # Password file path templates
1514+ self.root_passwd_file_template = rpasswdf_template
1515+ self.user_passwd_file_template = upasswdf_template
1516+
1517+ def connect(self, user='root', password=None):
1518+ self.connection = MySQLdb.connect(user=user, host=self.host,
1519+ passwd=password)
1520+
1521+ def database_exists(self, db_name):
1522+ cursor = self.connection.cursor()
1523+ try:
1524+ cursor.execute("SHOW DATABASES")
1525+ databases = [i[0] for i in cursor.fetchall()]
1526+ finally:
1527+ cursor.close()
1528+
1529+ return db_name in databases
1530+
1531+ def create_database(self, db_name):
1532+ cursor = self.connection.cursor()
1533+ try:
1534+ cursor.execute("CREATE DATABASE {} CHARACTER SET UTF8"
1535+ .format(db_name))
1536+ finally:
1537+ cursor.close()
1538+
1539+ def grant_exists(self, db_name, db_user, remote_ip):
1540+ cursor = self.connection.cursor()
1541+ priv_string = "GRANT ALL PRIVILEGES ON `{}`.* " \
1542+ "TO '{}'@'{}'".format(db_name, db_user, remote_ip)
1543+ try:
1544+ cursor.execute("SHOW GRANTS for '{}'@'{}'".format(db_user,
1545+ remote_ip))
1546+ grants = [i[0] for i in cursor.fetchall()]
1547+ except MySQLdb.OperationalError:
1548+ return False
1549+ finally:
1550+ cursor.close()
1551+
1552+ # TODO: review for different grants
1553+ return priv_string in grants
1554+
1555+ def create_grant(self, db_name, db_user, remote_ip, password):
1556+ cursor = self.connection.cursor()
1557+ try:
1558+ # TODO: review for different grants
1559+ cursor.execute("GRANT ALL PRIVILEGES ON {}.* TO '{}'@'{}' "
1560+ "IDENTIFIED BY '{}'".format(db_name,
1561+ db_user,
1562+ remote_ip,
1563+ password))
1564+ finally:
1565+ cursor.close()
1566+
1567+ def create_admin_grant(self, db_user, remote_ip, password):
1568+ cursor = self.connection.cursor()
1569+ try:
1570+ cursor.execute("GRANT ALL PRIVILEGES ON *.* TO '{}'@'{}' "
1571+ "IDENTIFIED BY '{}'".format(db_user,
1572+ remote_ip,
1573+ password))
1574+ finally:
1575+ cursor.close()
1576+
1577+ def cleanup_grant(self, db_user, remote_ip):
1578+ cursor = self.connection.cursor()
1579+ try:
1580+ cursor.execute("DROP FROM mysql.user WHERE user='{}' "
1581+ "AND HOST='{}'".format(db_user,
1582+ remote_ip))
1583+ finally:
1584+ cursor.close()
1585+
1586+ def execute(self, sql):
1587+ """Execute arbitary SQL against the database."""
1588+ cursor = self.connection.cursor()
1589+ try:
1590+ cursor.execute(sql)
1591+ finally:
1592+ cursor.close()
1593+
1594+ def migrate_passwords_to_peer_relation(self):
1595+ """Migrate any passwords storage on disk to cluster peer relation."""
1596+ dirname = os.path.dirname(self.root_passwd_file_template)
1597+ path = os.path.join(dirname, '*.passwd')
1598+ for f in glob.glob(path):
1599+ _key = os.path.basename(f)
1600+ with open(f, 'r') as passwd:
1601+ _value = passwd.read().strip()
1602+
1603+ try:
1604+ peer_store(_key, _value)
1605+ os.unlink(f)
1606+ except ValueError:
1607+ # NOTE cluster relation not yet ready - skip for now
1608+ pass
1609+
1610+ def get_mysql_password_on_disk(self, username=None, password=None):
1611+ """Retrieve, generate or store a mysql password for the provided
1612+ username on disk."""
1613+ if username:
1614+ template = self.user_passwd_file_template
1615+ passwd_file = template.format(username)
1616+ else:
1617+ passwd_file = self.root_passwd_file_template
1618+
1619+ _password = None
1620+ if os.path.exists(passwd_file):
1621+ with open(passwd_file, 'r') as passwd:
1622+ _password = passwd.read().strip()
1623+ else:
1624+ mkdir(os.path.dirname(passwd_file), owner='root', group='root',
1625+ perms=0o770)
1626+ # Force permissions - for some reason the chmod in makedirs fails
1627+ os.chmod(os.path.dirname(passwd_file), 0o770)
1628+ _password = password or pwgen(length=32)
1629+ write_file(passwd_file, _password, owner='root', group='root',
1630+ perms=0o660)
1631+
1632+ return _password
1633+
1634+ def get_mysql_password(self, username=None, password=None):
1635+ """Retrieve, generate or store a mysql password for the provided
1636+ username using peer relation cluster."""
1637+ self.migrate_passwords_to_peer_relation()
1638+ if username:
1639+ _key = 'mysql-{}.passwd'.format(username)
1640+ else:
1641+ _key = 'mysql.passwd'
1642+
1643+ try:
1644+ _password = peer_retrieve(_key)
1645+ if _password is None:
1646+ _password = password or pwgen(length=32)
1647+ peer_store(_key, _password)
1648+ except ValueError:
1649+ # cluster relation is not yet started; use on-disk
1650+ _password = self.get_mysql_password_on_disk(username, password)
1651+
1652+ return _password
1653+
1654+ def get_mysql_root_password(self, password=None):
1655+ """Retrieve or generate mysql root password for service units."""
1656+ return self.get_mysql_password(username=None, password=password)
1657+
1658+ def get_allowed_units(self, database, username, relation_id=None):
1659+ """Get list of units with access grants for database with username.
1660+
1661+ This is typically used to provide shared-db relations with a list of
1662+ which units have been granted access to the given database.
1663+ """
1664+ self.connect(password=self.get_mysql_root_password())
1665+ allowed_units = set()
1666+ for unit in related_units(relation_id):
1667+ settings = relation_get(rid=relation_id, unit=unit)
1668+ # First check for setting with prefix, then without
1669+ for attr in ["%s_hostname" % (database), 'hostname']:
1670+ hosts = settings.get(attr, None)
1671+ if hosts:
1672+ break
1673+
1674+ if hosts:
1675+ # hostname can be json-encoded list of hostnames
1676+ try:
1677+ hosts = json.loads(hosts)
1678+ except ValueError:
1679+ hosts = [hosts]
1680+ else:
1681+ hosts = [settings['private-address']]
1682+
1683+ if hosts:
1684+ for host in hosts:
1685+ if self.grant_exists(database, username, host):
1686+ log("Grant exists for host '%s' on db '%s'" %
1687+ (host, database), level=DEBUG)
1688+ if unit not in allowed_units:
1689+ allowed_units.add(unit)
1690+ else:
1691+ log("Grant does NOT exist for host '%s' on db '%s'" %
1692+ (host, database), level=DEBUG)
1693+ else:
1694+ log("No hosts found for grant check", level=INFO)
1695+
1696+ return allowed_units
1697+
1698+ def configure_db(self, hostname, database, username, admin=False):
1699+ """Configure access to database for username from hostname."""
1700+ if config_get('prefer-ipv6'):
1701+ remote_ip = hostname
1702+ elif hostname != unit_get('private-address'):
1703+ try:
1704+ remote_ip = socket.gethostbyname(hostname)
1705+ except Exception:
1706+ # socket.gethostbyname doesn't support ipv6
1707+ remote_ip = hostname
1708+ else:
1709+ remote_ip = '127.0.0.1'
1710+
1711+ self.connect(password=self.get_mysql_root_password())
1712+ if not self.database_exists(database):
1713+ self.create_database(database)
1714+
1715+ password = self.get_mysql_password(username)
1716+ if not self.grant_exists(database, username, remote_ip):
1717+ if not admin:
1718+ self.create_grant(database, username, remote_ip, password)
1719+ else:
1720+ self.create_admin_grant(username, remote_ip, password)
1721+
1722+ return password
1723+
1724+
1725+class PerconaClusterHelper(object):
1726+
1727+ # Going for the biggest page size to avoid wasted bytes. InnoDB page size is
1728+ # 16MB
1729+ DEFAULT_PAGE_SIZE = 16 * 1024 * 1024
1730+
1731+ def human_to_bytes(self, human):
1732+ """Convert human readable configuration options to bytes."""
1733+ num_re = re.compile('^[0-9]+$')
1734+ if num_re.match(human):
1735+ return human
1736+
1737+ factors = {
1738+ 'K': 1024,
1739+ 'M': 1048576,
1740+ 'G': 1073741824,
1741+ 'T': 1099511627776
1742+ }
1743+ modifier = human[-1]
1744+ if modifier in factors:
1745+ return int(human[:-1]) * factors[modifier]
1746+
1747+ if modifier == '%':
1748+ total_ram = self.human_to_bytes(self.get_mem_total())
1749+ if self.is_32bit_system() and total_ram > self.sys_mem_limit():
1750+ total_ram = self.sys_mem_limit()
1751+ factor = int(human[:-1]) * 0.01
1752+ pctram = total_ram * factor
1753+ return int(pctram - (pctram % self.DEFAULT_PAGE_SIZE))
1754+
1755+ raise ValueError("Can only convert K,M,G, or T")
1756+
1757+ def is_32bit_system(self):
1758+ """Determine whether system is 32 or 64 bit."""
1759+ try:
1760+ return sys.maxsize < 2 ** 32
1761+ except OverflowError:
1762+ return False
1763+
1764+ def sys_mem_limit(self):
1765+ """Determine the default memory limit for the current service unit."""
1766+ if platform.machine() in ['armv7l']:
1767+ _mem_limit = self.human_to_bytes('2700M') # experimentally determined
1768+ else:
1769+ # Limit for x86 based 32bit systems
1770+ _mem_limit = self.human_to_bytes('4G')
1771+
1772+ return _mem_limit
1773+
1774+ def get_mem_total(self):
1775+ """Calculate the total memory in the current service unit."""
1776+ with open('/proc/meminfo') as meminfo_file:
1777+ for line in meminfo_file:
1778+ key, mem = line.split(':', 2)
1779+ if key == 'MemTotal':
1780+ mtot, modifier = mem.strip().split(' ')
1781+ return '%s%s' % (mtot, upper(modifier[0]))
1782+
1783+ def parse_config(self):
1784+ """Parse charm configuration and calculate values for config files."""
1785+ config = config_get()
1786+ mysql_config = {}
1787+ if 'max-connections' in config:
1788+ mysql_config['max_connections'] = config['max-connections']
1789+
1790+ # Total memory available for dataset
1791+ dataset_bytes = self.human_to_bytes(config['dataset-size'])
1792+ mysql_config['dataset_bytes'] = dataset_bytes
1793+
1794+ if 'query-cache-type' in config:
1795+ # Query Cache Configuration
1796+ mysql_config['query_cache_size'] = config['query-cache-size']
1797+ if (config['query-cache-size'] == -1 and
1798+ config['query-cache-type'] in ['ON', 'DEMAND']):
1799+ # Calculate the query cache size automatically
1800+ qcache_bytes = (dataset_bytes * 0.20)
1801+ qcache_bytes = int(qcache_bytes -
1802+ (qcache_bytes % self.DEFAULT_PAGE_SIZE))
1803+ mysql_config['query_cache_size'] = qcache_bytes
1804+ dataset_bytes -= qcache_bytes
1805+
1806+ # 5.5 allows the words, but not 5.1
1807+ if config['query-cache-type'] == 'ON':
1808+ mysql_config['query_cache_type'] = 1
1809+ elif config['query-cache-type'] == 'DEMAND':
1810+ mysql_config['query_cache_type'] = 2
1811+ else:
1812+ mysql_config['query_cache_type'] = 0
1813+
1814+ # Set a sane default key_buffer size
1815+ mysql_config['key_buffer'] = self.human_to_bytes('32M')
1816+
1817+ if 'preferred-storage-engine' in config:
1818+ # Storage engine configuration
1819+ preferred_engines = config['preferred-storage-engine'].split(',')
1820+ chunk_size = int(dataset_bytes / len(preferred_engines))
1821+ mysql_config['innodb_flush_log_at_trx_commit'] = 1
1822+ mysql_config['sync_binlog'] = 1
1823+ if 'InnoDB' in preferred_engines:
1824+ mysql_config['innodb_buffer_pool_size'] = chunk_size
1825+ if config['tuning-level'] == 'fast':
1826+ mysql_config['innodb_flush_log_at_trx_commit'] = 2
1827+ else:
1828+ mysql_config['innodb_buffer_pool_size'] = 0
1829+
1830+ mysql_config['default_storage_engine'] = preferred_engines[0]
1831+ if 'MyISAM' in preferred_engines:
1832+ mysql_config['key_buffer'] = chunk_size
1833+
1834+ if config['tuning-level'] == 'fast':
1835+ mysql_config['sync_binlog'] = 0
1836+
1837+ return mysql_config
1838
1839=== added directory 'hooks/charmhelpers/contrib/hahelpers'
1840=== added file 'hooks/charmhelpers/contrib/hahelpers/__init__.py'
1841--- hooks/charmhelpers/contrib/hahelpers/__init__.py 1970-01-01 00:00:00 +0000
1842+++ hooks/charmhelpers/contrib/hahelpers/__init__.py 2015-06-18 17:19:58 +0000
1843@@ -0,0 +1,15 @@
1844+# Copyright 2014-2015 Canonical Limited.
1845+#
1846+# This file is part of charm-helpers.
1847+#
1848+# charm-helpers is free software: you can redistribute it and/or modify
1849+# it under the terms of the GNU Lesser General Public License version 3 as
1850+# published by the Free Software Foundation.
1851+#
1852+# charm-helpers is distributed in the hope that it will be useful,
1853+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1854+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1855+# GNU Lesser General Public License for more details.
1856+#
1857+# You should have received a copy of the GNU Lesser General Public License
1858+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1859
1860=== added file 'hooks/charmhelpers/contrib/hahelpers/apache.py'
1861--- hooks/charmhelpers/contrib/hahelpers/apache.py 1970-01-01 00:00:00 +0000
1862+++ hooks/charmhelpers/contrib/hahelpers/apache.py 2015-06-18 17:19:58 +0000
1863@@ -0,0 +1,82 @@
1864+# Copyright 2014-2015 Canonical Limited.
1865+#
1866+# This file is part of charm-helpers.
1867+#
1868+# charm-helpers is free software: you can redistribute it and/or modify
1869+# it under the terms of the GNU Lesser General Public License version 3 as
1870+# published by the Free Software Foundation.
1871+#
1872+# charm-helpers is distributed in the hope that it will be useful,
1873+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1874+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1875+# GNU Lesser General Public License for more details.
1876+#
1877+# You should have received a copy of the GNU Lesser General Public License
1878+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1879+
1880+#
1881+# Copyright 2012 Canonical Ltd.
1882+#
1883+# This file is sourced from lp:openstack-charm-helpers
1884+#
1885+# Authors:
1886+# James Page <james.page@ubuntu.com>
1887+# Adam Gandelman <adamg@ubuntu.com>
1888+#
1889+
1890+import subprocess
1891+
1892+from charmhelpers.core.hookenv import (
1893+ config as config_get,
1894+ relation_get,
1895+ relation_ids,
1896+ related_units as relation_list,
1897+ log,
1898+ INFO,
1899+)
1900+
1901+
1902+def get_cert(cn=None):
1903+ # TODO: deal with multiple https endpoints via charm config
1904+ cert = config_get('ssl_cert')
1905+ key = config_get('ssl_key')
1906+ if not (cert and key):
1907+ log("Inspecting identity-service relations for SSL certificate.",
1908+ level=INFO)
1909+ cert = key = None
1910+ if cn:
1911+ ssl_cert_attr = 'ssl_cert_{}'.format(cn)
1912+ ssl_key_attr = 'ssl_key_{}'.format(cn)
1913+ else:
1914+ ssl_cert_attr = 'ssl_cert'
1915+ ssl_key_attr = 'ssl_key'
1916+ for r_id in relation_ids('identity-service'):
1917+ for unit in relation_list(r_id):
1918+ if not cert:
1919+ cert = relation_get(ssl_cert_attr,
1920+ rid=r_id, unit=unit)
1921+ if not key:
1922+ key = relation_get(ssl_key_attr,
1923+ rid=r_id, unit=unit)
1924+ return (cert, key)
1925+
1926+
1927+def get_ca_cert():
1928+ ca_cert = config_get('ssl_ca')
1929+ if ca_cert is None:
1930+ log("Inspecting identity-service relations for CA SSL certificate.",
1931+ level=INFO)
1932+ for r_id in relation_ids('identity-service'):
1933+ for unit in relation_list(r_id):
1934+ if ca_cert is None:
1935+ ca_cert = relation_get('ca_cert',
1936+ rid=r_id, unit=unit)
1937+ return ca_cert
1938+
1939+
1940+def install_ca_cert(ca_cert):
1941+ if ca_cert:
1942+ with open('/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt',
1943+ 'w') as crt:
1944+ crt.write(ca_cert)
1945+ subprocess.check_call(['update-ca-certificates', '--fresh'])
1946
1947=== added file 'hooks/charmhelpers/contrib/hahelpers/cluster.py'
1948--- hooks/charmhelpers/contrib/hahelpers/cluster.py 1970-01-01 00:00:00 +0000
1949+++ hooks/charmhelpers/contrib/hahelpers/cluster.py 2015-06-18 17:19:58 +0000
1950@@ -0,0 +1,268 @@
1951+# Copyright 2014-2015 Canonical Limited.
1952+#
1953+# This file is part of charm-helpers.
1954+#
1955+# charm-helpers is free software: you can redistribute it and/or modify
1956+# it under the terms of the GNU Lesser General Public License version 3 as
1957+# published by the Free Software Foundation.
1958+#
1959+# charm-helpers is distributed in the hope that it will be useful,
1960+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1961+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1962+# GNU Lesser General Public License for more details.
1963+#
1964+# You should have received a copy of the GNU Lesser General Public License
1965+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1966+
1967+#
1968+# Copyright 2012 Canonical Ltd.
1969+#
1970+# Authors:
1971+# James Page <james.page@ubuntu.com>
1972+# Adam Gandelman <adamg@ubuntu.com>
1973+#
1974+
1975+"""
1976+Helpers for clustering and determining "cluster leadership" and other
1977+clustering-related helpers.
1978+"""
1979+
1980+import subprocess
1981+import os
1982+
1983+from socket import gethostname as get_unit_hostname
1984+
1985+import six
1986+
1987+from charmhelpers.core.hookenv import (
1988+ log,
1989+ relation_ids,
1990+ related_units as relation_list,
1991+ relation_get,
1992+ config as config_get,
1993+ INFO,
1994+ ERROR,
1995+ WARNING,
1996+ unit_get,
1997+)
1998+from charmhelpers.core.decorators import (
1999+ retry_on_exception,
2000+)
2001+
2002+
2003+class HAIncompleteConfig(Exception):
2004+ pass
2005+
2006+
2007+class CRMResourceNotFound(Exception):
2008+ pass
2009+
2010+
2011+def is_elected_leader(resource):
2012+ """
2013+ Returns True if the charm executing this is the elected cluster leader.
2014+
2015+ It relies on two mechanisms to determine leadership:
2016+ 1. If the charm is part of a corosync cluster, call corosync to
2017+ determine leadership.
2018+ 2. If the charm is not part of a corosync cluster, the leader is
2019+ determined as being "the alive unit with the lowest unit numer". In
2020+ other words, the oldest surviving unit.
2021+ """
2022+ if is_clustered():
2023+ if not is_crm_leader(resource):
2024+ log('Deferring action to CRM leader.', level=INFO)
2025+ return False
2026+ else:
2027+ peers = peer_units()
2028+ if peers and not oldest_peer(peers):
2029+ log('Deferring action to oldest service unit.', level=INFO)
2030+ return False
2031+ return True
2032+
2033+
2034+def is_clustered():
2035+ for r_id in (relation_ids('ha') or []):
2036+ for unit in (relation_list(r_id) or []):
2037+ clustered = relation_get('clustered',
2038+ rid=r_id,
2039+ unit=unit)
2040+ if clustered:
2041+ return True
2042+ return False
2043+
2044+
2045+@retry_on_exception(5, base_delay=2, exc_type=CRMResourceNotFound)
2046+def is_crm_leader(resource, retry=False):
2047+ """
2048+ Returns True if the charm calling this is the elected corosync leader,
2049+ as returned by calling the external "crm" command.
2050+
2051+ We allow this operation to be retried to avoid the possibility of getting a
2052+ false negative. See LP #1396246 for more info.
2053+ """
2054+ cmd = ['crm', 'resource', 'show', resource]
2055+ try:
2056+ status = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
2057+ if not isinstance(status, six.text_type):
2058+ status = six.text_type(status, "utf-8")
2059+ except subprocess.CalledProcessError:
2060+ status = None
2061+
2062+ if status and get_unit_hostname() in status:
2063+ return True
2064+
2065+ if status and "resource %s is NOT running" % (resource) in status:
2066+ raise CRMResourceNotFound("CRM resource %s not found" % (resource))
2067+
2068+ return False
2069+
2070+
2071+def is_leader(resource):
2072+ log("is_leader is deprecated. Please consider using is_crm_leader "
2073+ "instead.", level=WARNING)
2074+ return is_crm_leader(resource)
2075+
2076+
2077+def peer_units(peer_relation="cluster"):
2078+ peers = []
2079+ for r_id in (relation_ids(peer_relation) or []):
2080+ for unit in (relation_list(r_id) or []):
2081+ peers.append(unit)
2082+ return peers
2083+
2084+
2085+def peer_ips(peer_relation='cluster', addr_key='private-address'):
2086+ '''Return a dict of peers and their private-address'''
2087+ peers = {}
2088+ for r_id in relation_ids(peer_relation):
2089+ for unit in relation_list(r_id):
2090+ peers[unit] = relation_get(addr_key, rid=r_id, unit=unit)
2091+ return peers
2092+
2093+
2094+def oldest_peer(peers):
2095+ """Determines who the oldest peer is by comparing unit numbers."""
2096+ local_unit_no = int(os.getenv('JUJU_UNIT_NAME').split('/')[1])
2097+ for peer in peers:
2098+ remote_unit_no = int(peer.split('/')[1])
2099+ if remote_unit_no < local_unit_no:
2100+ return False
2101+ return True
2102+
2103+
2104+def eligible_leader(resource):
2105+ log("eligible_leader is deprecated. Please consider using "
2106+ "is_elected_leader instead.", level=WARNING)
2107+ return is_elected_leader(resource)
2108+
2109+
2110+def https():
2111+ '''
2112+ Determines whether enough data has been provided in configuration
2113+ or relation data to configure HTTPS
2114+ .
2115+ returns: boolean
2116+ '''
2117+ if config_get('use-https') == "yes":
2118+ return True
2119+ if config_get('ssl_cert') and config_get('ssl_key'):
2120+ return True
2121+ for r_id in relation_ids('identity-service'):
2122+ for unit in relation_list(r_id):
2123+ # TODO - needs fixing for new helper as ssl_cert/key suffixes with CN
2124+ rel_state = [
2125+ relation_get('https_keystone', rid=r_id, unit=unit),
2126+ relation_get('ca_cert', rid=r_id, unit=unit),
2127+ ]
2128+ # NOTE: works around (LP: #1203241)
2129+ if (None not in rel_state) and ('' not in rel_state):
2130+ return True
2131+ return False
2132+
2133+
2134+def determine_api_port(public_port, singlenode_mode=False):
2135+ '''
2136+ Determine correct API server listening port based on
2137+ existence of HTTPS reverse proxy and/or haproxy.
2138+
2139+ public_port: int: standard public port for given service
2140+
2141+ singlenode_mode: boolean: Shuffle ports when only a single unit is present
2142+
2143+ returns: int: the correct listening port for the API service
2144+ '''
2145+ i = 0
2146+ if singlenode_mode:
2147+ i += 1
2148+ elif len(peer_units()) > 0 or is_clustered():
2149+ i += 1
2150+ if https():
2151+ i += 1
2152+ return public_port - (i * 10)
2153+
2154+
2155+def determine_apache_port(public_port, singlenode_mode=False):
2156+ '''
2157+ Description: Determine correct apache listening port based on public IP +
2158+ state of the cluster.
2159+
2160+ public_port: int: standard public port for given service
2161+
2162+ singlenode_mode: boolean: Shuffle ports when only a single unit is present
2163+
2164+ returns: int: the correct listening port for the HAProxy service
2165+ '''
2166+ i = 0
2167+ if singlenode_mode:
2168+ i += 1
2169+ elif len(peer_units()) > 0 or is_clustered():
2170+ i += 1
2171+ return public_port - (i * 10)
2172+
2173+
2174+def get_hacluster_config(exclude_keys=None):
2175+ '''
2176+ Obtains all relevant configuration from charm configuration required
2177+ for initiating a relation to hacluster:
2178+
2179+ ha-bindiface, ha-mcastport, vip
2180+
2181+ param: exclude_keys: list of setting key(s) to be excluded.
2182+ returns: dict: A dict containing settings keyed by setting name.
2183+ raises: HAIncompleteConfig if settings are missing.
2184+ '''
2185+ settings = ['ha-bindiface', 'ha-mcastport', 'vip']
2186+ conf = {}
2187+ for setting in settings:
2188+ if exclude_keys and setting in exclude_keys:
2189+ continue
2190+
2191+ conf[setting] = config_get(setting)
2192+ missing = []
2193+ [missing.append(s) for s, v in six.iteritems(conf) if v is None]
2194+ if missing:
2195+ log('Insufficient config data to configure hacluster.', level=ERROR)
2196+ raise HAIncompleteConfig
2197+ return conf
2198+
2199+
2200+def canonical_url(configs, vip_setting='vip'):
2201+ '''
2202+ Returns the correct HTTP URL to this host given the state of HTTPS
2203+ configuration and hacluster.
2204+
2205+ :configs : OSTemplateRenderer: A config tempating object to inspect for
2206+ a complete https context.
2207+
2208+ :vip_setting: str: Setting in charm config that specifies
2209+ VIP address.
2210+ '''
2211+ scheme = 'http'
2212+ if 'https' in configs.complete_contexts():
2213+ scheme = 'https'
2214+ if is_clustered():
2215+ addr = config_get(vip_setting)
2216+ else:
2217+ addr = unit_get('private-address')
2218+ return '%s://%s' % (scheme, addr)
2219
2220=== added directory 'hooks/charmhelpers/contrib/network'
2221=== added file 'hooks/charmhelpers/contrib/network/__init__.py'
2222--- hooks/charmhelpers/contrib/network/__init__.py 1970-01-01 00:00:00 +0000
2223+++ hooks/charmhelpers/contrib/network/__init__.py 2015-06-18 17:19:58 +0000
2224@@ -0,0 +1,15 @@
2225+# Copyright 2014-2015 Canonical Limited.
2226+#
2227+# This file is part of charm-helpers.
2228+#
2229+# charm-helpers is free software: you can redistribute it and/or modify
2230+# it under the terms of the GNU Lesser General Public License version 3 as
2231+# published by the Free Software Foundation.
2232+#
2233+# charm-helpers is distributed in the hope that it will be useful,
2234+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2235+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2236+# GNU Lesser General Public License for more details.
2237+#
2238+# You should have received a copy of the GNU Lesser General Public License
2239+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2240
2241=== added file 'hooks/charmhelpers/contrib/network/ip.py'
2242--- hooks/charmhelpers/contrib/network/ip.py 1970-01-01 00:00:00 +0000
2243+++ hooks/charmhelpers/contrib/network/ip.py 2015-06-18 17:19:58 +0000
2244@@ -0,0 +1,367 @@
2245+# Copyright 2014-2015 Canonical Limited.
2246+#
2247+# This file is part of charm-helpers.
2248+#
2249+# charm-helpers is free software: you can redistribute it and/or modify
2250+# it under the terms of the GNU Lesser General Public License version 3 as
2251+# published by the Free Software Foundation.
2252+#
2253+# charm-helpers is distributed in the hope that it will be useful,
2254+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2255+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2256+# GNU Lesser General Public License for more details.
2257+#
2258+# You should have received a copy of the GNU Lesser General Public License
2259+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2260+
2261+import glob
2262+import re
2263+import subprocess
2264+
2265+from functools import partial
2266+
2267+from charmhelpers.core.hookenv import unit_get
2268+from charmhelpers.fetch import apt_install
2269+from charmhelpers.core.hookenv import (
2270+ log
2271+)
2272+
2273+try:
2274+ import netifaces
2275+except ImportError:
2276+ apt_install('python-netifaces')
2277+ import netifaces
2278+
2279+try:
2280+ import netaddr
2281+except ImportError:
2282+ apt_install('python-netaddr')
2283+ import netaddr
2284+
2285+
2286+def _validate_cidr(network):
2287+ try:
2288+ netaddr.IPNetwork(network)
2289+ except (netaddr.core.AddrFormatError, ValueError):
2290+ raise ValueError("Network (%s) is not in CIDR presentation format" %
2291+ network)
2292+
2293+
2294+def no_ip_found_error_out(network):
2295+ errmsg = ("No IP address found in network: %s" % network)
2296+ raise ValueError(errmsg)
2297+
2298+
2299+def get_address_in_network(network, fallback=None, fatal=False):
2300+ """Get an IPv4 or IPv6 address within the network from the host.
2301+
2302+ :param network (str): CIDR presentation format. For example,
2303+ '192.168.1.0/24'.
2304+ :param fallback (str): If no address is found, return fallback.
2305+ :param fatal (boolean): If no address is found, fallback is not
2306+ set and fatal is True then exit(1).
2307+ """
2308+ if network is None:
2309+ if fallback is not None:
2310+ return fallback
2311+
2312+ if fatal:
2313+ no_ip_found_error_out(network)
2314+ else:
2315+ return None
2316+
2317+ _validate_cidr(network)
2318+ network = netaddr.IPNetwork(network)
2319+ for iface in netifaces.interfaces():
2320+ addresses = netifaces.ifaddresses(iface)
2321+ if network.version == 4 and netifaces.AF_INET in addresses:
2322+ addr = addresses[netifaces.AF_INET][0]['addr']
2323+ netmask = addresses[netifaces.AF_INET][0]['netmask']
2324+ cidr = netaddr.IPNetwork("%s/%s" % (addr, netmask))
2325+ if cidr in network:
2326+ return str(cidr.ip)
2327+
2328+ if network.version == 6 and netifaces.AF_INET6 in addresses:
2329+ for addr in addresses[netifaces.AF_INET6]:
2330+ if not addr['addr'].startswith('fe80'):
2331+ cidr = netaddr.IPNetwork("%s/%s" % (addr['addr'],
2332+ addr['netmask']))
2333+ if cidr in network:
2334+ return str(cidr.ip)
2335+
2336+ if fallback is not None:
2337+ return fallback
2338+
2339+ if fatal:
2340+ no_ip_found_error_out(network)
2341+
2342+ return None
2343+
2344+
2345+def is_ipv6(address):
2346+ """Determine whether provided address is IPv6 or not."""
2347+ try:
2348+ address = netaddr.IPAddress(address)
2349+ except netaddr.AddrFormatError:
2350+ # probably a hostname - so not an address at all!
2351+ return False
2352+
2353+ return address.version == 6
2354+
2355+
2356+def is_address_in_network(network, address):
2357+ """
2358+ Determine whether the provided address is within a network range.
2359+
2360+ :param network (str): CIDR presentation format. For example,
2361+ '192.168.1.0/24'.
2362+ :param address: An individual IPv4 or IPv6 address without a net
2363+ mask or subnet prefix. For example, '192.168.1.1'.
2364+ :returns boolean: Flag indicating whether address is in network.
2365+ """
2366+ try:
2367+ network = netaddr.IPNetwork(network)
2368+ except (netaddr.core.AddrFormatError, ValueError):
2369+ raise ValueError("Network (%s) is not in CIDR presentation format" %
2370+ network)
2371+
2372+ try:
2373+ address = netaddr.IPAddress(address)
2374+ except (netaddr.core.AddrFormatError, ValueError):
2375+ raise ValueError("Address (%s) is not in correct presentation format" %
2376+ address)
2377+
2378+ if address in network:
2379+ return True
2380+ else:
2381+ return False
2382+
2383+
2384+def _get_for_address(address, key):
2385+ """Retrieve an attribute of or the physical interface that
2386+ the IP address provided could be bound to.
2387+
2388+ :param address (str): An individual IPv4 or IPv6 address without a net
2389+ mask or subnet prefix. For example, '192.168.1.1'.
2390+ :param key: 'iface' for the physical interface name or an attribute
2391+ of the configured interface, for example 'netmask'.
2392+ :returns str: Requested attribute or None if address is not bindable.
2393+ """
2394+ address = netaddr.IPAddress(address)
2395+ for iface in netifaces.interfaces():
2396+ addresses = netifaces.ifaddresses(iface)
2397+ if address.version == 4 and netifaces.AF_INET in addresses:
2398+ addr = addresses[netifaces.AF_INET][0]['addr']
2399+ netmask = addresses[netifaces.AF_INET][0]['netmask']
2400+ network = netaddr.IPNetwork("%s/%s" % (addr, netmask))
2401+ cidr = network.cidr
2402+ if address in cidr:
2403+ if key == 'iface':
2404+ return iface
2405+ else:
2406+ return addresses[netifaces.AF_INET][0][key]
2407+
2408+ if address.version == 6 and netifaces.AF_INET6 in addresses:
2409+ for addr in addresses[netifaces.AF_INET6]:
2410+ if not addr['addr'].startswith('fe80'):
2411+ network = netaddr.IPNetwork("%s/%s" % (addr['addr'],
2412+ addr['netmask']))
2413+ cidr = network.cidr
2414+ if address in cidr:
2415+ if key == 'iface':
2416+ return iface
2417+ elif key == 'netmask' and cidr:
2418+ return str(cidr).split('/')[1]
2419+ else:
2420+ return addr[key]
2421+
2422+ return None
2423+
2424+
2425+get_iface_for_address = partial(_get_for_address, key='iface')
2426+
2427+
2428+get_netmask_for_address = partial(_get_for_address, key='netmask')
2429+
2430+
2431+def format_ipv6_addr(address):
2432+ """If address is IPv6, wrap it in '[]' otherwise return None.
2433+
2434+ This is required by most configuration files when specifying IPv6
2435+ addresses.
2436+ """
2437+ if is_ipv6(address):
2438+ return "[%s]" % address
2439+
2440+ return None
2441+
2442+
2443+def get_iface_addr(iface='eth0', inet_type='AF_INET', inc_aliases=False,
2444+ fatal=True, exc_list=None):
2445+ """Return the assigned IP address for a given interface, if any."""
2446+ # Extract nic if passed /dev/ethX
2447+ if '/' in iface:
2448+ iface = iface.split('/')[-1]
2449+
2450+ if not exc_list:
2451+ exc_list = []
2452+
2453+ try:
2454+ inet_num = getattr(netifaces, inet_type)
2455+ except AttributeError:
2456+ raise Exception("Unknown inet type '%s'" % str(inet_type))
2457+
2458+ interfaces = netifaces.interfaces()
2459+ if inc_aliases:
2460+ ifaces = []
2461+ for _iface in interfaces:
2462+ if iface == _iface or _iface.split(':')[0] == iface:
2463+ ifaces.append(_iface)
2464+
2465+ if fatal and not ifaces:
2466+ raise Exception("Invalid interface '%s'" % iface)
2467+
2468+ ifaces.sort()
2469+ else:
2470+ if iface not in interfaces:
2471+ if fatal:
2472+ raise Exception("Interface '%s' not found " % (iface))
2473+ else:
2474+ return []
2475+
2476+ else:
2477+ ifaces = [iface]
2478+
2479+ addresses = []
2480+ for netiface in ifaces:
2481+ net_info = netifaces.ifaddresses(netiface)
2482+ if inet_num in net_info:
2483+ for entry in net_info[inet_num]:
2484+ if 'addr' in entry and entry['addr'] not in exc_list:
2485+ addresses.append(entry['addr'])
2486+
2487+ if fatal and not addresses:
2488+ raise Exception("Interface '%s' doesn't have any %s addresses." %
2489+ (iface, inet_type))
2490+
2491+ return sorted(addresses)
2492+
2493+
2494+get_ipv4_addr = partial(get_iface_addr, inet_type='AF_INET')
2495+
2496+
2497+def get_iface_from_addr(addr):
2498+ """Work out on which interface the provided address is configured."""
2499+ for iface in netifaces.interfaces():
2500+ addresses = netifaces.ifaddresses(iface)
2501+ for inet_type in addresses:
2502+ for _addr in addresses[inet_type]:
2503+ _addr = _addr['addr']
2504+ # link local
2505+ ll_key = re.compile("(.+)%.*")
2506+ raw = re.match(ll_key, _addr)
2507+ if raw:
2508+ _addr = raw.group(1)
2509+
2510+ if _addr == addr:
2511+ log("Address '%s' is configured on iface '%s'" %
2512+ (addr, iface))
2513+ return iface
2514+
2515+ msg = "Unable to infer net iface on which '%s' is configured" % (addr)
2516+ raise Exception(msg)
2517+
2518+
2519+def sniff_iface(f):
2520+ """Ensure decorated function is called with a value for iface.
2521+
2522+ If no iface provided, inject net iface inferred from unit private address.
2523+ """
2524+ def iface_sniffer(*args, **kwargs):
2525+ if not kwargs.get('iface', None):
2526+ kwargs['iface'] = get_iface_from_addr(unit_get('private-address'))
2527+
2528+ return f(*args, **kwargs)
2529+
2530+ return iface_sniffer
2531+
2532+
2533+@sniff_iface
2534+def get_ipv6_addr(iface=None, inc_aliases=False, fatal=True, exc_list=None,
2535+ dynamic_only=True):
2536+ """Get assigned IPv6 address for a given interface.
2537+
2538+ Returns list of addresses found. If no address found, returns empty list.
2539+
2540+ If iface is None, we infer the current primary interface by doing a reverse
2541+ lookup on the unit private-address.
2542+
2543+ We currently only support scope global IPv6 addresses i.e. non-temporary
2544+ addresses. If no global IPv6 address is found, return the first one found
2545+ in the ipv6 address list.
2546+ """
2547+ addresses = get_iface_addr(iface=iface, inet_type='AF_INET6',
2548+ inc_aliases=inc_aliases, fatal=fatal,
2549+ exc_list=exc_list)
2550+
2551+ if addresses:
2552+ global_addrs = []
2553+ for addr in addresses:
2554+ key_scope_link_local = re.compile("^fe80::..(.+)%(.+)")
2555+ m = re.match(key_scope_link_local, addr)
2556+ if m:
2557+ eui_64_mac = m.group(1)
2558+ iface = m.group(2)
2559+ else:
2560+ global_addrs.append(addr)
2561+
2562+ if global_addrs:
2563+ # Make sure any found global addresses are not temporary
2564+ cmd = ['ip', 'addr', 'show', iface]
2565+ out = subprocess.check_output(cmd).decode('UTF-8')
2566+ if dynamic_only:
2567+ key = re.compile("inet6 (.+)/[0-9]+ scope global dynamic.*")
2568+ else:
2569+ key = re.compile("inet6 (.+)/[0-9]+ scope global.*")
2570+
2571+ addrs = []
2572+ for line in out.split('\n'):
2573+ line = line.strip()
2574+ m = re.match(key, line)
2575+ if m and 'temporary' not in line:
2576+ # Return the first valid address we find
2577+ for addr in global_addrs:
2578+ if m.group(1) == addr:
2579+ if not dynamic_only or \
2580+ m.group(1).endswith(eui_64_mac):
2581+ addrs.append(addr)
2582+
2583+ if addrs:
2584+ return addrs
2585+
2586+ if fatal:
2587+ raise Exception("Interface '%s' does not have a scope global "
2588+ "non-temporary ipv6 address." % iface)
2589+
2590+ return []
2591+
2592+
2593+def get_bridges(vnic_dir='/sys/devices/virtual/net'):
2594+ """Return a list of bridges on the system."""
2595+ b_regex = "%s/*/bridge" % vnic_dir
2596+ return [x.replace(vnic_dir, '').split('/')[1] for x in glob.glob(b_regex)]
2597+
2598+
2599+def get_bridge_nics(bridge, vnic_dir='/sys/devices/virtual/net'):
2600+ """Return a list of nics comprising a given bridge on the system."""
2601+ brif_regex = "%s/%s/brif/*" % (vnic_dir, bridge)
2602+ return [x.split('/')[-1] for x in glob.glob(brif_regex)]
2603+
2604+
2605+def is_bridge_member(nic):
2606+ """Check if a given nic is a member of a bridge."""
2607+ for bridge in get_bridges():
2608+ if nic in get_bridge_nics(bridge):
2609+ return True
2610+
2611+ return False
2612
2613=== added directory 'hooks/charmhelpers/contrib/network/ovs'
2614=== added file 'hooks/charmhelpers/contrib/network/ovs/__init__.py'
2615--- hooks/charmhelpers/contrib/network/ovs/__init__.py 1970-01-01 00:00:00 +0000
2616+++ hooks/charmhelpers/contrib/network/ovs/__init__.py 2015-06-18 17:19:58 +0000
2617@@ -0,0 +1,96 @@
2618+# Copyright 2014-2015 Canonical Limited.
2619+#
2620+# This file is part of charm-helpers.
2621+#
2622+# charm-helpers is free software: you can redistribute it and/or modify
2623+# it under the terms of the GNU Lesser General Public License version 3 as
2624+# published by the Free Software Foundation.
2625+#
2626+# charm-helpers is distributed in the hope that it will be useful,
2627+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2628+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2629+# GNU Lesser General Public License for more details.
2630+#
2631+# You should have received a copy of the GNU Lesser General Public License
2632+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2633+
2634+''' Helpers for interacting with OpenvSwitch '''
2635+import subprocess
2636+import os
2637+from charmhelpers.core.hookenv import (
2638+ log, WARNING
2639+)
2640+from charmhelpers.core.host import (
2641+ service
2642+)
2643+
2644+
2645+def add_bridge(name):
2646+ ''' Add the named bridge to openvswitch '''
2647+ log('Creating bridge {}'.format(name))
2648+ subprocess.check_call(["ovs-vsctl", "--", "--may-exist", "add-br", name])
2649+
2650+
2651+def del_bridge(name):
2652+ ''' Delete the named bridge from openvswitch '''
2653+ log('Deleting bridge {}'.format(name))
2654+ subprocess.check_call(["ovs-vsctl", "--", "--if-exists", "del-br", name])
2655+
2656+
2657+def add_bridge_port(name, port, promisc=False):
2658+ ''' Add a port to the named openvswitch bridge '''
2659+ log('Adding port {} to bridge {}'.format(port, name))
2660+ subprocess.check_call(["ovs-vsctl", "--", "--may-exist", "add-port",
2661+ name, port])
2662+ subprocess.check_call(["ip", "link", "set", port, "up"])
2663+ if promisc:
2664+ subprocess.check_call(["ip", "link", "set", port, "promisc", "on"])
2665+ else:
2666+ subprocess.check_call(["ip", "link", "set", port, "promisc", "off"])
2667+
2668+
2669+def del_bridge_port(name, port):
2670+ ''' Delete a port from the named openvswitch bridge '''
2671+ log('Deleting port {} from bridge {}'.format(port, name))
2672+ subprocess.check_call(["ovs-vsctl", "--", "--if-exists", "del-port",
2673+ name, port])
2674+ subprocess.check_call(["ip", "link", "set", port, "down"])
2675+ subprocess.check_call(["ip", "link", "set", port, "promisc", "off"])
2676+
2677+
2678+def set_manager(manager):
2679+ ''' Set the controller for the local openvswitch '''
2680+ log('Setting manager for local ovs to {}'.format(manager))
2681+ subprocess.check_call(['ovs-vsctl', 'set-manager',
2682+ 'ssl:{}'.format(manager)])
2683+
2684+
2685+CERT_PATH = '/etc/openvswitch/ovsclient-cert.pem'
2686+
2687+
2688+def get_certificate():
2689+ ''' Read openvswitch certificate from disk '''
2690+ if os.path.exists(CERT_PATH):
2691+ log('Reading ovs certificate from {}'.format(CERT_PATH))
2692+ with open(CERT_PATH, 'r') as cert:
2693+ full_cert = cert.read()
2694+ begin_marker = "-----BEGIN CERTIFICATE-----"
2695+ end_marker = "-----END CERTIFICATE-----"
2696+ begin_index = full_cert.find(begin_marker)
2697+ end_index = full_cert.rfind(end_marker)
2698+ if end_index == -1 or begin_index == -1:
2699+ raise RuntimeError("Certificate does not contain valid begin"
2700+ " and end markers.")
2701+ full_cert = full_cert[begin_index:(end_index + len(end_marker))]
2702+ return full_cert
2703+ else:
2704+ log('Certificate not found', level=WARNING)
2705+ return None
2706+
2707+
2708+def full_restart():
2709+ ''' Full restart and reload of openvswitch '''
2710+ if os.path.exists('/etc/init/openvswitch-force-reload-kmod.conf'):
2711+ service('start', 'openvswitch-force-reload-kmod')
2712+ else:
2713+ service('force-reload-kmod', 'openvswitch-switch')
2714
2715=== added file 'hooks/charmhelpers/contrib/network/ufw.py'
2716--- hooks/charmhelpers/contrib/network/ufw.py 1970-01-01 00:00:00 +0000
2717+++ hooks/charmhelpers/contrib/network/ufw.py 2015-06-18 17:19:58 +0000
2718@@ -0,0 +1,276 @@
2719+# Copyright 2014-2015 Canonical Limited.
2720+#
2721+# This file is part of charm-helpers.
2722+#
2723+# charm-helpers is free software: you can redistribute it and/or modify
2724+# it under the terms of the GNU Lesser General Public License version 3 as
2725+# published by the Free Software Foundation.
2726+#
2727+# charm-helpers is distributed in the hope that it will be useful,
2728+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2729+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2730+# GNU Lesser General Public License for more details.
2731+#
2732+# You should have received a copy of the GNU Lesser General Public License
2733+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2734+
2735+"""
2736+This module contains helpers to add and remove ufw rules.
2737+
2738+Examples:
2739+
2740+- open SSH port for subnet 10.0.3.0/24:
2741+
2742+ >>> from charmhelpers.contrib.network import ufw
2743+ >>> ufw.enable()
2744+ >>> ufw.grant_access(src='10.0.3.0/24', dst='any', port='22', proto='tcp')
2745+
2746+- open service by name as defined in /etc/services:
2747+
2748+ >>> from charmhelpers.contrib.network import ufw
2749+ >>> ufw.enable()
2750+ >>> ufw.service('ssh', 'open')
2751+
2752+- close service by port number:
2753+
2754+ >>> from charmhelpers.contrib.network import ufw
2755+ >>> ufw.enable()
2756+ >>> ufw.service('4949', 'close') # munin
2757+"""
2758+import re
2759+import os
2760+import subprocess
2761+from charmhelpers.core import hookenv
2762+
2763+__author__ = "Felipe Reyes <felipe.reyes@canonical.com>"
2764+
2765+
2766+class UFWError(Exception):
2767+ pass
2768+
2769+
2770+class UFWIPv6Error(UFWError):
2771+ pass
2772+
2773+
2774+def is_enabled():
2775+ """
2776+ Check if `ufw` is enabled
2777+
2778+ :returns: True if ufw is enabled
2779+ """
2780+ output = subprocess.check_output(['ufw', 'status'],
2781+ universal_newlines=True,
2782+ env={'LANG': 'en_US',
2783+ 'PATH': os.environ['PATH']})
2784+
2785+ m = re.findall(r'^Status: active\n', output, re.M)
2786+
2787+ return len(m) >= 1
2788+
2789+
2790+def is_ipv6_ok(soft_fail=False):
2791+ """
2792+ Check if IPv6 support is present and ip6tables functional
2793+
2794+ :param soft_fail: If set to True and IPv6 support is broken, then reports
2795+ that the host doesn't have IPv6 support, otherwise a
2796+ UFWIPv6Error exception is raised.
2797+ :returns: True if IPv6 is working, False otherwise
2798+ """
2799+
2800+ # do we have IPv6 in the machine?
2801+ if os.path.isdir('/proc/sys/net/ipv6'):
2802+ # is ip6tables kernel module loaded?
2803+ lsmod = subprocess.check_output(['lsmod'], universal_newlines=True)
2804+ matches = re.findall('^ip6_tables[ ]+', lsmod, re.M)
2805+ if len(matches) == 0:
2806+ # ip6tables support isn't complete, let's try to load it
2807+ try:
2808+ subprocess.check_output(['modprobe', 'ip6_tables'],
2809+ universal_newlines=True)
2810+ # great, we could load the module
2811+ return True
2812+ except subprocess.CalledProcessError as ex:
2813+ hookenv.log("Couldn't load ip6_tables module: %s" % ex.output,
2814+ level="WARN")
2815+ # we are in a world where ip6tables isn't working
2816+ if soft_fail:
2817+ # so we inform that the machine doesn't have IPv6
2818+ return False
2819+ else:
2820+ raise UFWIPv6Error("IPv6 firewall support broken")
2821+ else:
2822+ # the module is present :)
2823+ return True
2824+
2825+ else:
2826+ # the system doesn't have IPv6
2827+ return False
2828+
2829+
2830+def disable_ipv6():
2831+ """
2832+ Disable ufw IPv6 support in /etc/default/ufw
2833+ """
2834+ exit_code = subprocess.call(['sed', '-i', 's/IPV6=.*/IPV6=no/g',
2835+ '/etc/default/ufw'])
2836+ if exit_code == 0:
2837+ hookenv.log('IPv6 support in ufw disabled', level='INFO')
2838+ else:
2839+ hookenv.log("Couldn't disable IPv6 support in ufw", level="ERROR")
2840+ raise UFWError("Couldn't disable IPv6 support in ufw")
2841+
2842+
2843+def enable(soft_fail=False):
2844+ """
2845+ Enable ufw
2846+
2847+ :param soft_fail: If set to True silently disables IPv6 support in ufw,
2848+ otherwise a UFWIPv6Error exception is raised when IP6
2849+ support is broken.
2850+ :returns: True if ufw is successfully enabled
2851+ """
2852+ if is_enabled():
2853+ return True
2854+
2855+ if not is_ipv6_ok(soft_fail):
2856+ disable_ipv6()
2857+
2858+ output = subprocess.check_output(['ufw', 'enable'],
2859+ universal_newlines=True,
2860+ env={'LANG': 'en_US',
2861+ 'PATH': os.environ['PATH']})
2862+
2863+ m = re.findall('^Firewall is active and enabled on system startup\n',
2864+ output, re.M)
2865+ hookenv.log(output, level='DEBUG')
2866+
2867+ if len(m) == 0:
2868+ hookenv.log("ufw couldn't be enabled", level='WARN')
2869+ return False
2870+ else:
2871+ hookenv.log("ufw enabled", level='INFO')
2872+ return True
2873+
2874+
2875+def disable():
2876+ """
2877+ Disable ufw
2878+
2879+ :returns: True if ufw is successfully disabled
2880+ """
2881+ if not is_enabled():
2882+ return True
2883+
2884+ output = subprocess.check_output(['ufw', 'disable'],
2885+ universal_newlines=True,
2886+ env={'LANG': 'en_US',
2887+ 'PATH': os.environ['PATH']})
2888+
2889+ m = re.findall(r'^Firewall stopped and disabled on system startup\n',
2890+ output, re.M)
2891+ hookenv.log(output, level='DEBUG')
2892+
2893+ if len(m) == 0:
2894+ hookenv.log("ufw couldn't be disabled", level='WARN')
2895+ return False
2896+ else:
2897+ hookenv.log("ufw disabled", level='INFO')
2898+ return True
2899+
2900+
2901+def modify_access(src, dst='any', port=None, proto=None, action='allow'):
2902+ """
2903+ Grant access to an address or subnet
2904+
2905+ :param src: address (e.g. 192.168.1.234) or subnet
2906+ (e.g. 192.168.1.0/24).
2907+ :param dst: destiny of the connection, if the machine has multiple IPs and
2908+ connections to only one of those have to accepted this is the
2909+ field has to be set.
2910+ :param port: destiny port
2911+ :param proto: protocol (tcp or udp)
2912+ :param action: `allow` or `delete`
2913+ """
2914+ if not is_enabled():
2915+ hookenv.log('ufw is disabled, skipping modify_access()', level='WARN')
2916+ return
2917+
2918+ if action == 'delete':
2919+ cmd = ['ufw', 'delete', 'allow']
2920+ else:
2921+ cmd = ['ufw', action]
2922+
2923+ if src is not None:
2924+ cmd += ['from', src]
2925+
2926+ if dst is not None:
2927+ cmd += ['to', dst]
2928+
2929+ if port is not None:
2930+ cmd += ['port', str(port)]
2931+
2932+ if proto is not None:
2933+ cmd += ['proto', proto]
2934+
2935+ hookenv.log('ufw {}: {}'.format(action, ' '.join(cmd)), level='DEBUG')
2936+ p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
2937+ (stdout, stderr) = p.communicate()
2938+
2939+ hookenv.log(stdout, level='INFO')
2940+
2941+ if p.returncode != 0:
2942+ hookenv.log(stderr, level='ERROR')
2943+ hookenv.log('Error running: {}, exit code: {}'.format(' '.join(cmd),
2944+ p.returncode),
2945+ level='ERROR')
2946+
2947+
2948+def grant_access(src, dst='any', port=None, proto=None):
2949+ """
2950+ Grant access to an address or subnet
2951+
2952+ :param src: address (e.g. 192.168.1.234) or subnet
2953+ (e.g. 192.168.1.0/24).
2954+ :param dst: destiny of the connection, if the machine has multiple IPs and
2955+ connections to only one of those have to accepted this is the
2956+ field has to be set.
2957+ :param port: destiny port
2958+ :param proto: protocol (tcp or udp)
2959+ """
2960+ return modify_access(src, dst=dst, port=port, proto=proto, action='allow')
2961+
2962+
2963+def revoke_access(src, dst='any', port=None, proto=None):
2964+ """
2965+ Revoke access to an address or subnet
2966+
2967+ :param src: address (e.g. 192.168.1.234) or subnet
2968+ (e.g. 192.168.1.0/24).
2969+ :param dst: destiny of the connection, if the machine has multiple IPs and
2970+ connections to only one of those have to accepted this is the
2971+ field has to be set.
2972+ :param port: destiny port
2973+ :param proto: protocol (tcp or udp)
2974+ """
2975+ return modify_access(src, dst=dst, port=port, proto=proto, action='delete')
2976+
2977+
2978+def service(name, action):
2979+ """
2980+ Open/close access to a service
2981+
2982+ :param name: could be a service name defined in `/etc/services` or a port
2983+ number.
2984+ :param action: `open` or `close`
2985+ """
2986+ if action == 'open':
2987+ subprocess.check_output(['ufw', 'allow', str(name)],
2988+ universal_newlines=True)
2989+ elif action == 'close':
2990+ subprocess.check_output(['ufw', 'delete', 'allow', str(name)],
2991+ universal_newlines=True)
2992+ else:
2993+ raise UFWError(("'{}' not supported, use 'allow' "
2994+ "or 'delete'").format(action))
2995
2996=== added directory 'hooks/charmhelpers/contrib/openstack'
2997=== added file 'hooks/charmhelpers/contrib/openstack/__init__.py'
2998--- hooks/charmhelpers/contrib/openstack/__init__.py 1970-01-01 00:00:00 +0000
2999+++ hooks/charmhelpers/contrib/openstack/__init__.py 2015-06-18 17:19:58 +0000
3000@@ -0,0 +1,15 @@
3001+# Copyright 2014-2015 Canonical Limited.
3002+#
3003+# This file is part of charm-helpers.
3004+#
3005+# charm-helpers is free software: you can redistribute it and/or modify
3006+# it under the terms of the GNU Lesser General Public License version 3 as
3007+# published by the Free Software Foundation.
3008+#
3009+# charm-helpers is distributed in the hope that it will be useful,
3010+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3011+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3012+# GNU Lesser General Public License for more details.
3013+#
3014+# You should have received a copy of the GNU Lesser General Public License
3015+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3016
3017=== added file 'hooks/charmhelpers/contrib/openstack/alternatives.py'
3018--- hooks/charmhelpers/contrib/openstack/alternatives.py 1970-01-01 00:00:00 +0000
3019+++ hooks/charmhelpers/contrib/openstack/alternatives.py 2015-06-18 17:19:58 +0000
3020@@ -0,0 +1,33 @@
3021+# Copyright 2014-2015 Canonical Limited.
3022+#
3023+# This file is part of charm-helpers.
3024+#
3025+# charm-helpers is free software: you can redistribute it and/or modify
3026+# it under the terms of the GNU Lesser General Public License version 3 as
3027+# published by the Free Software Foundation.
3028+#
3029+# charm-helpers is distributed in the hope that it will be useful,
3030+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3031+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3032+# GNU Lesser General Public License for more details.
3033+#
3034+# You should have received a copy of the GNU Lesser General Public License
3035+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3036+
3037+''' Helper for managing alternatives for file conflict resolution '''
3038+
3039+import subprocess
3040+import shutil
3041+import os
3042+
3043+
3044+def install_alternative(name, target, source, priority=50):
3045+ ''' Install alternative configuration '''
3046+ if (os.path.exists(target) and not os.path.islink(target)):
3047+ # Move existing file/directory away before installing
3048+ shutil.move(target, '{}.bak'.format(target))
3049+ cmd = [
3050+ 'update-alternatives', '--force', '--install',
3051+ target, name, source, str(priority)
3052+ ]
3053+ subprocess.check_call(cmd)
3054
3055=== added directory 'hooks/charmhelpers/contrib/openstack/amulet'
3056=== added file 'hooks/charmhelpers/contrib/openstack/amulet/__init__.py'
3057--- hooks/charmhelpers/contrib/openstack/amulet/__init__.py 1970-01-01 00:00:00 +0000
3058+++ hooks/charmhelpers/contrib/openstack/amulet/__init__.py 2015-06-18 17:19:58 +0000
3059@@ -0,0 +1,15 @@
3060+# Copyright 2014-2015 Canonical Limited.
3061+#
3062+# This file is part of charm-helpers.
3063+#
3064+# charm-helpers is free software: you can redistribute it and/or modify
3065+# it under the terms of the GNU Lesser General Public License version 3 as
3066+# published by the Free Software Foundation.
3067+#
3068+# charm-helpers is distributed in the hope that it will be useful,
3069+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3070+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3071+# GNU Lesser General Public License for more details.
3072+#
3073+# You should have received a copy of the GNU Lesser General Public License
3074+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3075
3076=== added file 'hooks/charmhelpers/contrib/openstack/amulet/deployment.py'
3077--- hooks/charmhelpers/contrib/openstack/amulet/deployment.py 1970-01-01 00:00:00 +0000
3078+++ hooks/charmhelpers/contrib/openstack/amulet/deployment.py 2015-06-18 17:19:58 +0000
3079@@ -0,0 +1,108 @@
3080+# Copyright 2014-2015 Canonical Limited.
3081+#
3082+# This file is part of charm-helpers.
3083+#
3084+# charm-helpers is free software: you can redistribute it and/or modify
3085+# it under the terms of the GNU Lesser General Public License version 3 as
3086+# published by the Free Software Foundation.
3087+#
3088+# charm-helpers is distributed in the hope that it will be useful,
3089+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3090+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3091+# GNU Lesser General Public License for more details.
3092+#
3093+# You should have received a copy of the GNU Lesser General Public License
3094+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3095+
3096+import six
3097+from charmhelpers.contrib.amulet.deployment import (
3098+ AmuletDeployment
3099+)
3100+
3101+
3102+class OpenStackAmuletDeployment(AmuletDeployment):
3103+ """OpenStack amulet deployment.
3104+
3105+ This class inherits from AmuletDeployment and has additional support
3106+ that is specifically for use by OpenStack charms.
3107+ """
3108+
3109+ def __init__(self, series=None, openstack=None, source=None, stable=True):
3110+ """Initialize the deployment environment."""
3111+ super(OpenStackAmuletDeployment, self).__init__(series)
3112+ self.openstack = openstack
3113+ self.source = source
3114+ self.stable = stable
3115+ # Note(coreycb): this needs to be changed when new next branches come
3116+ # out.
3117+ self.current_next = "trusty"
3118+
3119+ def _determine_branch_locations(self, other_services):
3120+ """Determine the branch locations for the other services.
3121+
3122+ Determine if the local branch being tested is derived from its
3123+ stable or next (dev) branch, and based on this, use the corresonding
3124+ stable or next branches for the other_services."""
3125+ base_charms = ['mysql', 'mongodb', 'rabbitmq-server']
3126+
3127+ if self.stable:
3128+ for svc in other_services:
3129+ temp = 'lp:charms/{}'
3130+ svc['location'] = temp.format(svc['name'])
3131+ else:
3132+ for svc in other_services:
3133+ if svc['name'] in base_charms:
3134+ temp = 'lp:charms/{}'
3135+ svc['location'] = temp.format(svc['name'])
3136+ else:
3137+ temp = 'lp:~openstack-charmers/charms/{}/{}/next'
3138+ svc['location'] = temp.format(self.current_next,
3139+ svc['name'])
3140+ return other_services
3141+
3142+ def _add_services(self, this_service, other_services):
3143+ """Add services to the deployment and set openstack-origin/source."""
3144+ other_services = self._determine_branch_locations(other_services)
3145+
3146+ super(OpenStackAmuletDeployment, self)._add_services(this_service,
3147+ other_services)
3148+
3149+ services = other_services
3150+ services.append(this_service)
3151+ use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',
3152+ 'ceph-osd', 'ceph-radosgw']
3153+
3154+ if self.openstack:
3155+ for svc in services:
3156+ if svc['name'] not in use_source:
3157+ config = {'openstack-origin': self.openstack}
3158+ self.d.configure(svc['name'], config)
3159+
3160+ if self.source:
3161+ for svc in services:
3162+ if svc['name'] in use_source:
3163+ config = {'source': self.source}
3164+ self.d.configure(svc['name'], config)
3165+
3166+ def _configure_services(self, configs):
3167+ """Configure all of the services."""
3168+ for service, config in six.iteritems(configs):
3169+ self.d.configure(service, config)
3170+
3171+ def _get_openstack_release(self):
3172+ """Get openstack release.
3173+
3174+ Return an integer representing the enum value of the openstack
3175+ release.
3176+ """
3177+ (self.precise_essex, self.precise_folsom, self.precise_grizzly,
3178+ self.precise_havana, self.precise_icehouse,
3179+ self.trusty_icehouse) = range(6)
3180+ releases = {
3181+ ('precise', None): self.precise_essex,
3182+ ('precise', 'cloud:precise-folsom'): self.precise_folsom,
3183+ ('precise', 'cloud:precise-grizzly'): self.precise_grizzly,
3184+ ('precise', 'cloud:precise-havana'): self.precise_havana,
3185+ ('precise', 'cloud:precise-icehouse'): self.precise_icehouse,
3186+ ('trusty', None): self.trusty_icehouse}
3187+ return releases[(self.series, self.openstack)]
3188
3189=== added file 'hooks/charmhelpers/contrib/openstack/amulet/utils.py'
3190--- hooks/charmhelpers/contrib/openstack/amulet/utils.py 1970-01-01 00:00:00 +0000
3191+++ hooks/charmhelpers/contrib/openstack/amulet/utils.py 2015-06-18 17:19:58 +0000
3192@@ -0,0 +1,294 @@
3193+# Copyright 2014-2015 Canonical Limited.
3194+#
3195+# This file is part of charm-helpers.
3196+#
3197+# charm-helpers is free software: you can redistribute it and/or modify
3198+# it under the terms of the GNU Lesser General Public License version 3 as
3199+# published by the Free Software Foundation.
3200+#
3201+# charm-helpers is distributed in the hope that it will be useful,
3202+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3203+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3204+# GNU Lesser General Public License for more details.
3205+#
3206+# You should have received a copy of the GNU Lesser General Public License
3207+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3208+
3209+import logging
3210+import os
3211+import time
3212+import urllib
3213+
3214+import glanceclient.v1.client as glance_client
3215+import keystoneclient.v2_0 as keystone_client
3216+import novaclient.v1_1.client as nova_client
3217+
3218+import six
3219+
3220+from charmhelpers.contrib.amulet.utils import (
3221+ AmuletUtils
3222+)
3223+
3224+DEBUG = logging.DEBUG
3225+ERROR = logging.ERROR
3226+
3227+
3228+class OpenStackAmuletUtils(AmuletUtils):
3229+ """OpenStack amulet utilities.
3230+
3231+ This class inherits from AmuletUtils and has additional support
3232+ that is specifically for use by OpenStack charms.
3233+ """
3234+
3235+ def __init__(self, log_level=ERROR):
3236+ """Initialize the deployment environment."""
3237+ super(OpenStackAmuletUtils, self).__init__(log_level)
3238+
3239+ def validate_endpoint_data(self, endpoints, admin_port, internal_port,
3240+ public_port, expected):
3241+ """Validate endpoint data.
3242+
3243+ Validate actual endpoint data vs expected endpoint data. The ports
3244+ are used to find the matching endpoint.
3245+ """
3246+ found = False
3247+ for ep in endpoints:
3248+ self.log.debug('endpoint: {}'.format(repr(ep)))
3249+ if (admin_port in ep.adminurl and
3250+ internal_port in ep.internalurl and
3251+ public_port in ep.publicurl):
3252+ found = True
3253+ actual = {'id': ep.id,
3254+ 'region': ep.region,
3255+ 'adminurl': ep.adminurl,
3256+ 'internalurl': ep.internalurl,
3257+ 'publicurl': ep.publicurl,
3258+ 'service_id': ep.service_id}
3259+ ret = self._validate_dict_data(expected, actual)
3260+ if ret:
3261+ return 'unexpected endpoint data - {}'.format(ret)
3262+
3263+ if not found:
3264+ return 'endpoint not found'
3265+
3266+ def validate_svc_catalog_endpoint_data(self, expected, actual):
3267+ """Validate service catalog endpoint data.
3268+
3269+ Validate a list of actual service catalog endpoints vs a list of
3270+ expected service catalog endpoints.
3271+ """
3272+ self.log.debug('actual: {}'.format(repr(actual)))
3273+ for k, v in six.iteritems(expected):
3274+ if k in actual:
3275+ ret = self._validate_dict_data(expected[k][0], actual[k][0])
3276+ if ret:
3277+ return self.endpoint_error(k, ret)
3278+ else:
3279+ return "endpoint {} does not exist".format(k)
3280+ return ret
3281+
3282+ def validate_tenant_data(self, expected, actual):
3283+ """Validate tenant data.
3284+
3285+ Validate a list of actual tenant data vs list of expected tenant
3286+ data.
3287+ """
3288+ self.log.debug('actual: {}'.format(repr(actual)))
3289+ for e in expected:
3290+ found = False
3291+ for act in actual:
3292+ a = {'enabled': act.enabled, 'description': act.description,
3293+ 'name': act.name, 'id': act.id}
3294+ if e['name'] == a['name']:
3295+ found = True
3296+ ret = self._validate_dict_data(e, a)
3297+ if ret:
3298+ return "unexpected tenant data - {}".format(ret)
3299+ if not found:
3300+ return "tenant {} does not exist".format(e['name'])
3301+ return ret
3302+
3303+ def validate_role_data(self, expected, actual):
3304+ """Validate role data.
3305+
3306+ Validate a list of actual role data vs a list of expected role
3307+ data.
3308+ """
3309+ self.log.debug('actual: {}'.format(repr(actual)))
3310+ for e in expected:
3311+ found = False
3312+ for act in actual:
3313+ a = {'name': act.name, 'id': act.id}
3314+ if e['name'] == a['name']:
3315+ found = True
3316+ ret = self._validate_dict_data(e, a)
3317+ if ret:
3318+ return "unexpected role data - {}".format(ret)
3319+ if not found:
3320+ return "role {} does not exist".format(e['name'])
3321+ return ret
3322+
3323+ def validate_user_data(self, expected, actual):
3324+ """Validate user data.
3325+
3326+ Validate a list of actual user data vs a list of expected user
3327+ data.
3328+ """
3329+ self.log.debug('actual: {}'.format(repr(actual)))
3330+ for e in expected:
3331+ found = False
3332+ for act in actual:
3333+ a = {'enabled': act.enabled, 'name': act.name,
3334+ 'email': act.email, 'tenantId': act.tenantId,
3335+ 'id': act.id}
3336+ if e['name'] == a['name']:
3337+ found = True
3338+ ret = self._validate_dict_data(e, a)
3339+ if ret:
3340+ return "unexpected user data - {}".format(ret)
3341+ if not found:
3342+ return "user {} does not exist".format(e['name'])
3343+ return ret
3344+
3345+ def validate_flavor_data(self, expected, actual):
3346+ """Validate flavor data.
3347+
3348+ Validate a list of actual flavors vs a list of expected flavors.
3349+ """
3350+ self.log.debug('actual: {}'.format(repr(actual)))
3351+ act = [a.name for a in actual]
3352+ return self._validate_list_data(expected, act)
3353+
3354+ def tenant_exists(self, keystone, tenant):
3355+ """Return True if tenant exists."""
3356+ return tenant in [t.name for t in keystone.tenants.list()]
3357+
3358+ def authenticate_keystone_admin(self, keystone_sentry, user, password,
3359+ tenant):
3360+ """Authenticates admin user with the keystone admin endpoint."""
3361+ unit = keystone_sentry
3362+ service_ip = unit.relation('shared-db',
3363+ 'mysql:shared-db')['private-address']
3364+ ep = "http://{}:35357/v2.0".format(service_ip.strip().decode('utf-8'))
3365+ return keystone_client.Client(username=user, password=password,
3366+ tenant_name=tenant, auth_url=ep)
3367+
3368+ def authenticate_keystone_user(self, keystone, user, password, tenant):
3369+ """Authenticates a regular user with the keystone public endpoint."""
3370+ ep = keystone.service_catalog.url_for(service_type='identity',
3371+ endpoint_type='publicURL')
3372+ return keystone_client.Client(username=user, password=password,
3373+ tenant_name=tenant, auth_url=ep)
3374+
3375+ def authenticate_glance_admin(self, keystone):
3376+ """Authenticates admin user with glance."""
3377+ ep = keystone.service_catalog.url_for(service_type='image',
3378+ endpoint_type='adminURL')
3379+ return glance_client.Client(ep, token=keystone.auth_token)
3380+
3381+ def authenticate_nova_user(self, keystone, user, password, tenant):
3382+ """Authenticates a regular user with nova-api."""
3383+ ep = keystone.service_catalog.url_for(service_type='identity',
3384+ endpoint_type='publicURL')
3385+ return nova_client.Client(username=user, api_key=password,
3386+ project_id=tenant, auth_url=ep)
3387+
3388+ def create_cirros_image(self, glance, image_name):
3389+ """Download the latest cirros image and upload it to glance."""
3390+ http_proxy = os.getenv('AMULET_HTTP_PROXY')
3391+ self.log.debug('AMULET_HTTP_PROXY: {}'.format(http_proxy))
3392+ if http_proxy:
3393+ proxies = {'http': http_proxy}
3394+ opener = urllib.FancyURLopener(proxies)
3395+ else:
3396+ opener = urllib.FancyURLopener()
3397+
3398+ f = opener.open("http://download.cirros-cloud.net/version/released")
3399+ version = f.read().strip()
3400+ cirros_img = "cirros-{}-x86_64-disk.img".format(version)
3401+ local_path = os.path.join('tests', cirros_img)
3402+
3403+ if not os.path.exists(local_path):
3404+ cirros_url = "http://{}/{}/{}".format("download.cirros-cloud.net",
3405+ version, cirros_img)
3406+ opener.retrieve(cirros_url, local_path)
3407+ f.close()
3408+
3409+ with open(local_path) as f:
3410+ image = glance.images.create(name=image_name, is_public=True,
3411+ disk_format='qcow2',
3412+ container_format='bare', data=f)
3413+ count = 1
3414+ status = image.status
3415+ while status != 'active' and count < 10:
3416+ time.sleep(3)
3417+ image = glance.images.get(image.id)
3418+ status = image.status
3419+ self.log.debug('image status: {}'.format(status))
3420+ count += 1
3421+
3422+ if status != 'active':
3423+ self.log.error('image creation timed out')
3424+ return None
3425+
3426+ return image
3427+
3428+ def delete_image(self, glance, image):
3429+ """Delete the specified image."""
3430+ num_before = len(list(glance.images.list()))
3431+ glance.images.delete(image)
3432+
3433+ count = 1
3434+ num_after = len(list(glance.images.list()))
3435+ while num_after != (num_before - 1) and count < 10:
3436+ time.sleep(3)
3437+ num_after = len(list(glance.images.list()))
3438+ self.log.debug('number of images: {}'.format(num_after))
3439+ count += 1
3440+
3441+ if num_after != (num_before - 1):
3442+ self.log.error('image deletion timed out')
3443+ return False
3444+
3445+ return True
3446+
3447+ def create_instance(self, nova, image_name, instance_name, flavor):
3448+ """Create the specified instance."""
3449+ image = nova.images.find(name=image_name)
3450+ flavor = nova.flavors.find(name=flavor)
3451+ instance = nova.servers.create(name=instance_name, image=image,
3452+ flavor=flavor)
3453+
3454+ count = 1
3455+ status = instance.status
3456+ while status != 'ACTIVE' and count < 60:
3457+ time.sleep(3)
3458+ instance = nova.servers.get(instance.id)
3459+ status = instance.status
3460+ self.log.debug('instance status: {}'.format(status))
3461+ count += 1
3462+
3463+ if status != 'ACTIVE':
3464+ self.log.error('instance creation timed out')
3465+ return None
3466+
3467+ return instance
3468+
3469+ def delete_instance(self, nova, instance):
3470+ """Delete the specified instance."""
3471+ num_before = len(list(nova.servers.list()))
3472+ nova.servers.delete(instance)
3473+
3474+ count = 1
3475+ num_after = len(list(nova.servers.list()))
3476+ while num_after != (num_before - 1) and count < 10:
3477+ time.sleep(3)
3478+ num_after = len(list(nova.servers.list()))
3479+ self.log.debug('number of instances: {}'.format(num_after))
3480+ count += 1
3481+
3482+ if num_after != (num_before - 1):
3483+ self.log.error('instance deletion timed out')
3484+ return False
3485+
3486+ return True
3487
3488=== added file 'hooks/charmhelpers/contrib/openstack/context.py'
3489--- hooks/charmhelpers/contrib/openstack/context.py 1970-01-01 00:00:00 +0000
3490+++ hooks/charmhelpers/contrib/openstack/context.py 2015-06-18 17:19:58 +0000
3491@@ -0,0 +1,1054 @@
3492+# Copyright 2014-2015 Canonical Limited.
3493+#
3494+# This file is part of charm-helpers.
3495+#
3496+# charm-helpers is free software: you can redistribute it and/or modify
3497+# it under the terms of the GNU Lesser General Public License version 3 as
3498+# published by the Free Software Foundation.
3499+#
3500+# charm-helpers is distributed in the hope that it will be useful,
3501+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3502+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3503+# GNU Lesser General Public License for more details.
3504+#
3505+# You should have received a copy of the GNU Lesser General Public License
3506+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3507+
3508+import json
3509+import os
3510+import time
3511+from base64 import b64decode
3512+from subprocess import check_call
3513+
3514+import six
3515+
3516+from charmhelpers.fetch import (
3517+ apt_install,
3518+ filter_installed_packages,
3519+)
3520+from charmhelpers.core.hookenv import (
3521+ config,
3522+ is_relation_made,
3523+ local_unit,
3524+ log,
3525+ relation_get,
3526+ relation_ids,
3527+ related_units,
3528+ relation_set,
3529+ unit_get,
3530+ unit_private_ip,
3531+ charm_name,
3532+ DEBUG,
3533+ INFO,
3534+ WARNING,
3535+ ERROR,
3536+)
3537+
3538+from charmhelpers.core.sysctl import create as sysctl_create
3539+
3540+from charmhelpers.core.host import (
3541+ mkdir,
3542+ write_file,
3543+)
3544+from charmhelpers.contrib.hahelpers.cluster import (
3545+ determine_apache_port,
3546+ determine_api_port,
3547+ https,
3548+ is_clustered,
3549+)
3550+from charmhelpers.contrib.hahelpers.apache import (
3551+ get_cert,
3552+ get_ca_cert,
3553+ install_ca_cert,
3554+)
3555+from charmhelpers.contrib.openstack.neutron import (
3556+ neutron_plugin_attribute,
3557+)
3558+from charmhelpers.contrib.network.ip import (
3559+ get_address_in_network,
3560+ get_ipv6_addr,
3561+ get_netmask_for_address,
3562+ format_ipv6_addr,
3563+ is_address_in_network,
3564+)
3565+from charmhelpers.contrib.openstack.utils import get_host_ip
3566+
3567+CA_CERT_PATH = '/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt'
3568+ADDRESS_TYPES = ['admin', 'internal', 'public']
3569+
3570+
3571+class OSContextError(Exception):
3572+ pass
3573+
3574+
3575+def ensure_packages(packages):
3576+ """Install but do not upgrade required plugin packages."""
3577+ required = filter_installed_packages(packages)
3578+ if required:
3579+ apt_install(required, fatal=True)
3580+
3581+
3582+def context_complete(ctxt):
3583+ _missing = []
3584+ for k, v in six.iteritems(ctxt):
3585+ if v is None or v == '':
3586+ _missing.append(k)
3587+
3588+ if _missing:
3589+ log('Missing required data: %s' % ' '.join(_missing), level=INFO)
3590+ return False
3591+
3592+ return True
3593+
3594+
3595+def config_flags_parser(config_flags):
3596+ """Parses config flags string into dict.
3597+
3598+ The provided config_flags string may be a list of comma-separated values
3599+ which themselves may be comma-separated list of values.
3600+ """
3601+ if config_flags.find('==') >= 0:
3602+ log("config_flags is not in expected format (key=value)", level=ERROR)
3603+ raise OSContextError
3604+
3605+ # strip the following from each value.
3606+ post_strippers = ' ,'
3607+ # we strip any leading/trailing '=' or ' ' from the string then
3608+ # split on '='.
3609+ split = config_flags.strip(' =').split('=')
3610+ limit = len(split)
3611+ flags = {}
3612+ for i in range(0, limit - 1):
3613+ current = split[i]
3614+ next = split[i + 1]
3615+ vindex = next.rfind(',')
3616+ if (i == limit - 2) or (vindex < 0):
3617+ value = next
3618+ else:
3619+ value = next[:vindex]
3620+
3621+ if i == 0:
3622+ key = current
3623+ else:
3624+ # if this not the first entry, expect an embedded key.
3625+ index = current.rfind(',')
3626+ if index < 0:
3627+ log("Invalid config value(s) at index %s" % (i), level=ERROR)
3628+ raise OSContextError
3629+ key = current[index + 1:]
3630+
3631+ # Add to collection.
3632+ flags[key.strip(post_strippers)] = value.rstrip(post_strippers)
3633+
3634+ return flags
3635+
3636+
3637+class OSContextGenerator(object):
3638+ """Base class for all context generators."""
3639+ interfaces = []
3640+
3641+ def __call__(self):
3642+ raise NotImplementedError
3643+
3644+
3645+class SharedDBContext(OSContextGenerator):
3646+ interfaces = ['shared-db']
3647+
3648+ def __init__(self,
3649+ database=None, user=None, relation_prefix=None, ssl_dir=None):
3650+ """Allows inspecting relation for settings prefixed with
3651+ relation_prefix. This is useful for parsing access for multiple
3652+ databases returned via the shared-db interface (eg, nova_password,
3653+ quantum_password)
3654+ """
3655+ self.relation_prefix = relation_prefix
3656+ self.database = database
3657+ self.user = user
3658+ self.ssl_dir = ssl_dir
3659+
3660+ def __call__(self):
3661+ self.database = self.database or config('database')
3662+ self.user = self.user or config('database-user')
3663+ if None in [self.database, self.user]:
3664+ log("Could not generate shared_db context. Missing required charm "
3665+ "config options. (database name and user)", level=ERROR)
3666+ raise OSContextError
3667+
3668+ ctxt = {}
3669+
3670+ # NOTE(jamespage) if mysql charm provides a network upon which
3671+ # access to the database should be made, reconfigure relation
3672+ # with the service units local address and defer execution
3673+ access_network = relation_get('access-network')
3674+ if access_network is not None:
3675+ if self.relation_prefix is not None:
3676+ hostname_key = "{}_hostname".format(self.relation_prefix)
3677+ else:
3678+ hostname_key = "hostname"
3679+ access_hostname = get_address_in_network(access_network,
3680+ unit_get('private-address'))
3681+ set_hostname = relation_get(attribute=hostname_key,
3682+ unit=local_unit())
3683+ if set_hostname != access_hostname:
3684+ relation_set(relation_settings={hostname_key: access_hostname})
3685+ return ctxt # Defer any further hook execution for now....
3686+
3687+ password_setting = 'password'
3688+ if self.relation_prefix:
3689+ password_setting = self.relation_prefix + '_password'
3690+
3691+ for rid in relation_ids('shared-db'):
3692+ for unit in related_units(rid):
3693+ rdata = relation_get(rid=rid, unit=unit)
3694+ host = rdata.get('db_host')
3695+ host = format_ipv6_addr(host) or host
3696+ ctxt = {
3697+ 'database_host': host,
3698+ 'database': self.database,
3699+ 'database_user': self.user,
3700+ 'database_password': rdata.get(password_setting),
3701+ 'database_type': 'mysql'
3702+ }
3703+ if context_complete(ctxt):
3704+ db_ssl(rdata, ctxt, self.ssl_dir)
3705+ return ctxt
3706+ return {}
3707+
3708+
3709+class PostgresqlDBContext(OSContextGenerator):
3710+ interfaces = ['pgsql-db']
3711+
3712+ def __init__(self, database=None):
3713+ self.database = database
3714+
3715+ def __call__(self):
3716+ self.database = self.database or config('database')
3717+ if self.database is None:
3718+ log('Could not generate postgresql_db context. Missing required '
3719+ 'charm config options. (database name)', level=ERROR)
3720+ raise OSContextError
3721+
3722+ ctxt = {}
3723+ for rid in relation_ids(self.interfaces[0]):
3724+ for unit in related_units(rid):
3725+ rel_host = relation_get('host', rid=rid, unit=unit)
3726+ rel_user = relation_get('user', rid=rid, unit=unit)
3727+ rel_passwd = relation_get('password', rid=rid, unit=unit)
3728+ ctxt = {'database_host': rel_host,
3729+ 'database': self.database,
3730+ 'database_user': rel_user,
3731+ 'database_password': rel_passwd,
3732+ 'database_type': 'postgresql'}
3733+ if context_complete(ctxt):
3734+ return ctxt
3735+
3736+ return {}
3737+
3738+
3739+def db_ssl(rdata, ctxt, ssl_dir):
3740+ if 'ssl_ca' in rdata and ssl_dir:
3741+ ca_path = os.path.join(ssl_dir, 'db-client.ca')
3742+ with open(ca_path, 'w') as fh:
3743+ fh.write(b64decode(rdata['ssl_ca']))
3744+
3745+ ctxt['database_ssl_ca'] = ca_path
3746+ elif 'ssl_ca' in rdata:
3747+ log("Charm not setup for ssl support but ssl ca found", level=INFO)
3748+ return ctxt
3749+
3750+ if 'ssl_cert' in rdata:
3751+ cert_path = os.path.join(
3752+ ssl_dir, 'db-client.cert')
3753+ if not os.path.exists(cert_path):
3754+ log("Waiting 1m for ssl client cert validity", level=INFO)
3755+ time.sleep(60)
3756+
3757+ with open(cert_path, 'w') as fh:
3758+ fh.write(b64decode(rdata['ssl_cert']))
3759+
3760+ ctxt['database_ssl_cert'] = cert_path
3761+ key_path = os.path.join(ssl_dir, 'db-client.key')
3762+ with open(key_path, 'w') as fh:
3763+ fh.write(b64decode(rdata['ssl_key']))
3764+
3765+ ctxt['database_ssl_key'] = key_path
3766+
3767+ return ctxt
3768+
3769+
3770+class IdentityServiceContext(OSContextGenerator):
3771+ interfaces = ['identity-service']
3772+
3773+ def __call__(self):
3774+ log('Generating template context for identity-service', level=DEBUG)
3775+ ctxt = {}
3776+ for rid in relation_ids('identity-service'):
3777+ for unit in related_units(rid):
3778+ rdata = relation_get(rid=rid, unit=unit)
3779+ serv_host = rdata.get('service_host')
3780+ serv_host = format_ipv6_addr(serv_host) or serv_host
3781+ auth_host = rdata.get('auth_host')
3782+ auth_host = format_ipv6_addr(auth_host) or auth_host
3783+ svc_protocol = rdata.get('service_protocol') or 'http'
3784+ auth_protocol = rdata.get('auth_protocol') or 'http'
3785+ ctxt = {'service_port': rdata.get('service_port'),
3786+ 'service_host': serv_host,
3787+ 'auth_host': auth_host,
3788+ 'auth_port': rdata.get('auth_port'),
3789+ 'admin_tenant_name': rdata.get('service_tenant'),
3790+ 'admin_user': rdata.get('service_username'),
3791+ 'admin_password': rdata.get('service_password'),
3792+ 'service_protocol': svc_protocol,
3793+ 'auth_protocol': auth_protocol}
3794+ if context_complete(ctxt):
3795+ # NOTE(jamespage) this is required for >= icehouse
3796+ # so a missing value just indicates keystone needs
3797+ # upgrading
3798+ ctxt['admin_tenant_id'] = rdata.get('service_tenant_id')
3799+ return ctxt
3800+
3801+ return {}
3802+
3803+
3804+class AMQPContext(OSContextGenerator):
3805+
3806+ def __init__(self, ssl_dir=None, rel_name='amqp', relation_prefix=None):
3807+ self.ssl_dir = ssl_dir
3808+ self.rel_name = rel_name
3809+ self.relation_prefix = relation_prefix
3810+ self.interfaces = [rel_name]
3811+
3812+ def __call__(self):
3813+ log('Generating template context for amqp', level=DEBUG)
3814+ conf = config()
3815+ if self.relation_prefix:
3816+ user_setting = '%s-rabbit-user' % (self.relation_prefix)
3817+ vhost_setting = '%s-rabbit-vhost' % (self.relation_prefix)
3818+ else:
3819+ user_setting = 'rabbit-user'
3820+ vhost_setting = 'rabbit-vhost'
3821+
3822+ try:
3823+ username = conf[user_setting]
3824+ vhost = conf[vhost_setting]
3825+ except KeyError as e:
3826+ log('Could not generate shared_db context. Missing required charm '
3827+ 'config options: %s.' % e, level=ERROR)
3828+ raise OSContextError
3829+
3830+ ctxt = {}
3831+ for rid in relation_ids(self.rel_name):
3832+ ha_vip_only = False
3833+ for unit in related_units(rid):
3834+ if relation_get('clustered', rid=rid, unit=unit):
3835+ ctxt['clustered'] = True
3836+ vip = relation_get('vip', rid=rid, unit=unit)
3837+ vip = format_ipv6_addr(vip) or vip
3838+ ctxt['rabbitmq_host'] = vip
3839+ else:
3840+ host = relation_get('private-address', rid=rid, unit=unit)
3841+ host = format_ipv6_addr(host) or host
3842+ ctxt['rabbitmq_host'] = host
3843+
3844+ ctxt.update({
3845+ 'rabbitmq_user': username,
3846+ 'rabbitmq_password': relation_get('password', rid=rid,
3847+ unit=unit),
3848+ 'rabbitmq_virtual_host': vhost,
3849+ })
3850+
3851+ ssl_port = relation_get('ssl_port', rid=rid, unit=unit)
3852+ if ssl_port:
3853+ ctxt['rabbit_ssl_port'] = ssl_port
3854+
3855+ ssl_ca = relation_get('ssl_ca', rid=rid, unit=unit)
3856+ if ssl_ca:
3857+ ctxt['rabbit_ssl_ca'] = ssl_ca
3858+
3859+ if relation_get('ha_queues', rid=rid, unit=unit) is not None:
3860+ ctxt['rabbitmq_ha_queues'] = True
3861+
3862+ ha_vip_only = relation_get('ha-vip-only',
3863+ rid=rid, unit=unit) is not None
3864+
3865+ if context_complete(ctxt):
3866+ if 'rabbit_ssl_ca' in ctxt:
3867+ if not self.ssl_dir:
3868+ log("Charm not setup for ssl support but ssl ca "
3869+ "found", level=INFO)
3870+ break
3871+
3872+ ca_path = os.path.join(
3873+ self.ssl_dir, 'rabbit-client-ca.pem')
3874+ with open(ca_path, 'w') as fh:
3875+ fh.write(b64decode(ctxt['rabbit_ssl_ca']))
3876+ ctxt['rabbit_ssl_ca'] = ca_path
3877+
3878+ # Sufficient information found = break out!
3879+ break
3880+
3881+ # Used for active/active rabbitmq >= grizzly
3882+ if (('clustered' not in ctxt or ha_vip_only) and
3883+ len(related_units(rid)) > 1):
3884+ rabbitmq_hosts = []
3885+ for unit in related_units(rid):
3886+ host = relation_get('private-address', rid=rid, unit=unit)
3887+ host = format_ipv6_addr(host) or host
3888+ rabbitmq_hosts.append(host)
3889+
3890+ ctxt['rabbitmq_hosts'] = ','.join(sorted(rabbitmq_hosts))
3891+
3892+ if not context_complete(ctxt):
3893+ return {}
3894+
3895+ return ctxt
3896+
3897+
3898+class CephContext(OSContextGenerator):
3899+ """Generates context for /etc/ceph/ceph.conf templates."""
3900+ interfaces = ['ceph']
3901+
3902+ def __call__(self):
3903+ if not relation_ids('ceph'):
3904+ return {}
3905+
3906+ log('Generating template context for ceph', level=DEBUG)
3907+ mon_hosts = []
3908+ auth = None
3909+ key = None
3910+ use_syslog = str(config('use-syslog')).lower()
3911+ for rid in relation_ids('ceph'):
3912+ for unit in related_units(rid):
3913+ auth = relation_get('auth', rid=rid, unit=unit)
3914+ key = relation_get('key', rid=rid, unit=unit)
3915+ ceph_pub_addr = relation_get('ceph-public-address', rid=rid,
3916+ unit=unit)
3917+ unit_priv_addr = relation_get('private-address', rid=rid,
3918+ unit=unit)
3919+ ceph_addr = ceph_pub_addr or unit_priv_addr
3920+ ceph_addr = format_ipv6_addr(ceph_addr) or ceph_addr
3921+ mon_hosts.append(ceph_addr)
3922+
3923+ ctxt = {'mon_hosts': ' '.join(sorted(mon_hosts)),
3924+ 'auth': auth,
3925+ 'key': key,
3926+ 'use_syslog': use_syslog}
3927+
3928+ if not os.path.isdir('/etc/ceph'):
3929+ os.mkdir('/etc/ceph')
3930+
3931+ if not context_complete(ctxt):
3932+ return {}
3933+
3934+ ensure_packages(['ceph-common'])
3935+ return ctxt
3936+
3937+
3938+class HAProxyContext(OSContextGenerator):
3939+ """Provides half a context for the haproxy template, which describes
3940+ all peers to be included in the cluster. Each charm needs to include
3941+ its own context generator that describes the port mapping.
3942+ """
3943+ interfaces = ['cluster']
3944+
3945+ def __init__(self, singlenode_mode=False):
3946+ self.singlenode_mode = singlenode_mode
3947+
3948+ def __call__(self):
3949+ if not relation_ids('cluster') and not self.singlenode_mode:
3950+ return {}
3951+
3952+ if config('prefer-ipv6'):
3953+ addr = get_ipv6_addr(exc_list=[config('vip')])[0]
3954+ else:
3955+ addr = get_host_ip(unit_get('private-address'))
3956+
3957+ l_unit = local_unit().replace('/', '-')
3958+ cluster_hosts = {}
3959+
3960+ # NOTE(jamespage): build out map of configured network endpoints
3961+ # and associated backends
3962+ for addr_type in ADDRESS_TYPES:
3963+ cfg_opt = 'os-{}-network'.format(addr_type)
3964+ laddr = get_address_in_network(config(cfg_opt))
3965+ if laddr:
3966+ netmask = get_netmask_for_address(laddr)
3967+ cluster_hosts[laddr] = {'network': "{}/{}".format(laddr,
3968+ netmask),
3969+ 'backends': {l_unit: laddr}}
3970+ for rid in relation_ids('cluster'):
3971+ for unit in related_units(rid):
3972+ _laddr = relation_get('{}-address'.format(addr_type),
3973+ rid=rid, unit=unit)
3974+ if _laddr:
3975+ _unit = unit.replace('/', '-')
3976+ cluster_hosts[laddr]['backends'][_unit] = _laddr
3977+
3978+ # NOTE(jamespage) add backend based on private address - this
3979+ # with either be the only backend or the fallback if no acls
3980+ # match in the frontend
3981+ cluster_hosts[addr] = {}
3982+ netmask = get_netmask_for_address(addr)
3983+ cluster_hosts[addr] = {'network': "{}/{}".format(addr, netmask),
3984+ 'backends': {l_unit: addr}}
3985+ for rid in relation_ids('cluster'):
3986+ for unit in related_units(rid):
3987+ _laddr = relation_get('private-address',
3988+ rid=rid, unit=unit)
3989+ if _laddr:
3990+ _unit = unit.replace('/', '-')
3991+ cluster_hosts[addr]['backends'][_unit] = _laddr
3992+
3993+ ctxt = {
3994+ 'frontends': cluster_hosts,
3995+ 'default_backend': addr
3996+ }
3997+
3998+ if config('haproxy-server-timeout'):
3999+ ctxt['haproxy_server_timeout'] = config('haproxy-server-timeout')
4000+
4001+ if config('haproxy-client-timeout'):
4002+ ctxt['haproxy_client_timeout'] = config('haproxy-client-timeout')
4003+
4004+ if config('prefer-ipv6'):
4005+ ctxt['ipv6'] = True
4006+ ctxt['local_host'] = 'ip6-localhost'
4007+ ctxt['haproxy_host'] = '::'
4008+ ctxt['stat_port'] = ':::8888'
4009+ else:
4010+ ctxt['local_host'] = '127.0.0.1'
4011+ ctxt['haproxy_host'] = '0.0.0.0'
4012+ ctxt['stat_port'] = ':8888'
4013+
4014+ for frontend in cluster_hosts:
4015+ if (len(cluster_hosts[frontend]['backends']) > 1 or
4016+ self.singlenode_mode):
4017+ # Enable haproxy when we have enough peers.
4018+ log('Ensuring haproxy enabled in /etc/default/haproxy.',
4019+ level=DEBUG)
4020+ with open('/etc/default/haproxy', 'w') as out:
4021+ out.write('ENABLED=1\n')
4022+
4023+ return ctxt
4024+
4025+ log('HAProxy context is incomplete, this unit has no peers.',
4026+ level=INFO)
4027+ return {}
4028+
4029+
4030+class ImageServiceContext(OSContextGenerator):
4031+ interfaces = ['image-service']
4032+
4033+ def __call__(self):
4034+ """Obtains the glance API server from the image-service relation.
4035+ Useful in nova and cinder (currently).
4036+ """
4037+ log('Generating template context for image-service.', level=DEBUG)
4038+ rids = relation_ids('image-service')
4039+ if not rids:
4040+ return {}
4041+
4042+ for rid in rids:
4043+ for unit in related_units(rid):
4044+ api_server = relation_get('glance-api-server',
4045+ rid=rid, unit=unit)
4046+ if api_server:
4047+ return {'glance_api_servers': api_server}
4048+
4049+ log("ImageService context is incomplete. Missing required relation "
4050+ "data.", level=INFO)
4051+ return {}
4052+
4053+
4054+class ApacheSSLContext(OSContextGenerator):
4055+ """Generates a context for an apache vhost configuration that configures
4056+ HTTPS reverse proxying for one or many endpoints. Generated context
4057+ looks something like::
4058+
4059+ {
4060+ 'namespace': 'cinder',
4061+ 'private_address': 'iscsi.mycinderhost.com',
4062+ 'endpoints': [(8776, 8766), (8777, 8767)]
4063+ }
4064+
4065+ The endpoints list consists of a tuples mapping external ports
4066+ to internal ports.
4067+ """
4068+ interfaces = ['https']
4069+
4070+ # charms should inherit this context and set external ports
4071+ # and service namespace accordingly.
4072+ external_ports = []
4073+ service_namespace = None
4074+
4075+ def enable_modules(self):
4076+ cmd = ['a2enmod', 'ssl', 'proxy', 'proxy_http']
4077+ check_call(cmd)
4078+
4079+ def configure_cert(self, cn=None):
4080+ ssl_dir = os.path.join('/etc/apache2/ssl/', self.service_namespace)
4081+ mkdir(path=ssl_dir)
4082+ cert, key = get_cert(cn)
4083+ if cn:
4084+ cert_filename = 'cert_{}'.format(cn)
4085+ key_filename = 'key_{}'.format(cn)
4086+ else:
4087+ cert_filename = 'cert'
4088+ key_filename = 'key'
4089+
4090+ write_file(path=os.path.join(ssl_dir, cert_filename),
4091+ content=b64decode(cert))
4092+ write_file(path=os.path.join(ssl_dir, key_filename),
4093+ content=b64decode(key))
4094+
4095+ def configure_ca(self):
4096+ ca_cert = get_ca_cert()
4097+ if ca_cert:
4098+ install_ca_cert(b64decode(ca_cert))
4099+
4100+ def canonical_names(self):
4101+ """Figure out which canonical names clients will access this service.
4102+ """
4103+ cns = []
4104+ for r_id in relation_ids('identity-service'):
4105+ for unit in related_units(r_id):
4106+ rdata = relation_get(rid=r_id, unit=unit)
4107+ for k in rdata:
4108+ if k.startswith('ssl_key_'):
4109+ cns.append(k.lstrip('ssl_key_'))
4110+
4111+ return sorted(list(set(cns)))
4112+
4113+ def get_network_addresses(self):
4114+ """For each network configured, return corresponding address and vip
4115+ (if available).
4116+
4117+ Returns a list of tuples of the form:
4118+
4119+ [(address_in_net_a, vip_in_net_a),
4120+ (address_in_net_b, vip_in_net_b),
4121+ ...]
4122+
4123+ or, if no vip(s) available:
4124+
4125+ [(address_in_net_a, address_in_net_a),
4126+ (address_in_net_b, address_in_net_b),
4127+ ...]
4128+ """
4129+ addresses = []
4130+ if config('vip'):
4131+ vips = config('vip').split()
4132+ else:
4133+ vips = []
4134+
4135+ for net_type in ['os-internal-network', 'os-admin-network',
4136+ 'os-public-network']:
4137+ addr = get_address_in_network(config(net_type),
4138+ unit_get('private-address'))
4139+ if len(vips) > 1 and is_clustered():
4140+ if not config(net_type):
4141+ log("Multiple networks configured but net_type "
4142+ "is None (%s)." % net_type, level=WARNING)
4143+ continue
4144+
4145+ for vip in vips:
4146+ if is_address_in_network(config(net_type), vip):
4147+ addresses.append((addr, vip))
4148+ break
4149+
4150+ elif is_clustered() and config('vip'):
4151+ addresses.append((addr, config('vip')))
4152+ else:
4153+ addresses.append((addr, addr))
4154+
4155+ return sorted(addresses)
4156+
4157+ def __call__(self):
4158+ if isinstance(self.external_ports, six.string_types):
4159+ self.external_ports = [self.external_ports]
4160+
4161+ if not self.external_ports or not https():
4162+ return {}
4163+
4164+ self.configure_ca()
4165+ self.enable_modules()
4166+
4167+ ctxt = {'namespace': self.service_namespace,
4168+ 'endpoints': [],
4169+ 'ext_ports': []}
4170+
4171+ for cn in self.canonical_names():
4172+ self.configure_cert(cn)
4173+
4174+ addresses = self.get_network_addresses()
4175+ for address, endpoint in sorted(set(addresses)):
4176+ for api_port in self.external_ports:
4177+ ext_port = determine_apache_port(api_port,
4178+ singlenode_mode=True)
4179+ int_port = determine_api_port(api_port, singlenode_mode=True)
4180+ portmap = (address, endpoint, int(ext_port), int(int_port))
4181+ ctxt['endpoints'].append(portmap)
4182+ ctxt['ext_ports'].append(int(ext_port))
4183+
4184+ ctxt['ext_ports'] = sorted(list(set(ctxt['ext_ports'])))
4185+ return ctxt
4186+
4187+
4188+class NeutronContext(OSContextGenerator):
4189+ interfaces = []
4190+
4191+ @property
4192+ def plugin(self):
4193+ return None
4194+
4195+ @property
4196+ def network_manager(self):
4197+ return None
4198+
4199+ @property
4200+ def packages(self):
4201+ return neutron_plugin_attribute(self.plugin, 'packages',
4202+ self.network_manager)
4203+
4204+ @property
4205+ def neutron_security_groups(self):
4206+ return None
4207+
4208+ def _ensure_packages(self):
4209+ for pkgs in self.packages:
4210+ ensure_packages(pkgs)
4211+
4212+ def _save_flag_file(self):
4213+ if self.network_manager == 'quantum':
4214+ _file = '/etc/nova/quantum_plugin.conf'
4215+ else:
4216+ _file = '/etc/nova/neutron_plugin.conf'
4217+
4218+ with open(_file, 'wb') as out:
4219+ out.write(self.plugin + '\n')
4220+
4221+ def ovs_ctxt(self):
4222+ driver = neutron_plugin_attribute(self.plugin, 'driver',
4223+ self.network_manager)
4224+ config = neutron_plugin_attribute(self.plugin, 'config',
4225+ self.network_manager)
4226+ ovs_ctxt = {'core_plugin': driver,
4227+ 'neutron_plugin': 'ovs',
4228+ 'neutron_security_groups': self.neutron_security_groups,
4229+ 'local_ip': unit_private_ip(),
4230+ 'config': config}
4231+
4232+ return ovs_ctxt
4233+
4234+ def nvp_ctxt(self):
4235+ driver = neutron_plugin_attribute(self.plugin, 'driver',
4236+ self.network_manager)
4237+ config = neutron_plugin_attribute(self.plugin, 'config',
4238+ self.network_manager)
4239+ nvp_ctxt = {'core_plugin': driver,
4240+ 'neutron_plugin': 'nvp',
4241+ 'neutron_security_groups': self.neutron_security_groups,
4242+ 'local_ip': unit_private_ip(),
4243+ 'config': config}
4244+
4245+ return nvp_ctxt
4246+
4247+ def n1kv_ctxt(self):
4248+ driver = neutron_plugin_attribute(self.plugin, 'driver',
4249+ self.network_manager)
4250+ n1kv_config = neutron_plugin_attribute(self.plugin, 'config',
4251+ self.network_manager)
4252+ n1kv_user_config_flags = config('n1kv-config-flags')
4253+ restrict_policy_profiles = config('n1kv-restrict-policy-profiles')
4254+ n1kv_ctxt = {'core_plugin': driver,
4255+ 'neutron_plugin': 'n1kv',
4256+ 'neutron_security_groups': self.neutron_security_groups,
4257+ 'local_ip': unit_private_ip(),
4258+ 'config': n1kv_config,
4259+ 'vsm_ip': config('n1kv-vsm-ip'),
4260+ 'vsm_username': config('n1kv-vsm-username'),
4261+ 'vsm_password': config('n1kv-vsm-password'),
4262+ 'restrict_policy_profiles': restrict_policy_profiles}
4263+
4264+ if n1kv_user_config_flags:
4265+ flags = config_flags_parser(n1kv_user_config_flags)
4266+ n1kv_ctxt['user_config_flags'] = flags
4267+
4268+ return n1kv_ctxt
4269+
4270+ def calico_ctxt(self):
4271+ driver = neutron_plugin_attribute(self.plugin, 'driver',
4272+ self.network_manager)
4273+ config = neutron_plugin_attribute(self.plugin, 'config',
4274+ self.network_manager)
4275+ calico_ctxt = {'core_plugin': driver,
4276+ 'neutron_plugin': 'Calico',
4277+ 'neutron_security_groups': self.neutron_security_groups,
4278+ 'local_ip': unit_private_ip(),
4279+ 'config': config}
4280+
4281+ return calico_ctxt
4282+
4283+ def neutron_ctxt(self):
4284+ if https():
4285+ proto = 'https'
4286+ else:
4287+ proto = 'http'
4288+
4289+ if is_clustered():
4290+ host = config('vip')
4291+ else:
4292+ host = unit_get('private-address')
4293+
4294+ ctxt = {'network_manager': self.network_manager,
4295+ 'neutron_url': '%s://%s:%s' % (proto, host, '9696')}
4296+ return ctxt
4297+
4298+ def __call__(self):
4299+ self._ensure_packages()
4300+
4301+ if self.network_manager not in ['quantum', 'neutron']:
4302+ return {}
4303+
4304+ if not self.plugin:
4305+ return {}
4306+
4307+ ctxt = self.neutron_ctxt()
4308+
4309+ if self.plugin == 'ovs':
4310+ ctxt.update(self.ovs_ctxt())
4311+ elif self.plugin in ['nvp', 'nsx']:
4312+ ctxt.update(self.nvp_ctxt())
4313+ elif self.plugin == 'n1kv':
4314+ ctxt.update(self.n1kv_ctxt())
4315+ elif self.plugin == 'Calico':
4316+ ctxt.update(self.calico_ctxt())
4317+
4318+ alchemy_flags = config('neutron-alchemy-flags')
4319+ if alchemy_flags:
4320+ flags = config_flags_parser(alchemy_flags)
4321+ ctxt['neutron_alchemy_flags'] = flags
4322+
4323+ self._save_flag_file()
4324+ return ctxt
4325+
4326+
4327+class OSConfigFlagContext(OSContextGenerator):
4328+ """Provides support for user-defined config flags.
4329+
4330+ Users can define a comma-seperated list of key=value pairs
4331+ in the charm configuration and apply them at any point in
4332+ any file by using a template flag.
4333+
4334+ Sometimes users might want config flags inserted within a
4335+ specific section so this class allows users to specify the
4336+ template flag name, allowing for multiple template flags
4337+ (sections) within the same context.
4338+
4339+ NOTE: the value of config-flags may be a comma-separated list of
4340+ key=value pairs and some Openstack config files support
4341+ comma-separated lists as values.
4342+ """
4343+
4344+ def __init__(self, charm_flag='config-flags',
4345+ template_flag='user_config_flags'):
4346+ """
4347+ :param charm_flag: config flags in charm configuration.
4348+ :param template_flag: insert point for user-defined flags in template
4349+ file.
4350+ """
4351+ super(OSConfigFlagContext, self).__init__()
4352+ self._charm_flag = charm_flag
4353+ self._template_flag = template_flag
4354+
4355+ def __call__(self):
4356+ config_flags = config(self._charm_flag)
4357+ if not config_flags:
4358+ return {}
4359+
4360+ return {self._template_flag:
4361+ config_flags_parser(config_flags)}
4362+
4363+
4364+class SubordinateConfigContext(OSContextGenerator):
4365+
4366+ """
4367+ Responsible for inspecting relations to subordinates that
4368+ may be exporting required config via a json blob.
4369+
4370+ The subordinate interface allows subordinates to export their
4371+ configuration requirements to the principle for multiple config
4372+ files and multiple serivces. Ie, a subordinate that has interfaces
4373+ to both glance and nova may export to following yaml blob as json::
4374+
4375+ glance:
4376+ /etc/glance/glance-api.conf:
4377+ sections:
4378+ DEFAULT:
4379+ - [key1, value1]
4380+ /etc/glance/glance-registry.conf:
4381+ MYSECTION:
4382+ - [key2, value2]
4383+ nova:
4384+ /etc/nova/nova.conf:
4385+ sections:
4386+ DEFAULT:
4387+ - [key3, value3]
4388+
4389+
4390+ It is then up to the principle charms to subscribe this context to
4391+ the service+config file it is interestd in. Configuration data will
4392+ be available in the template context, in glance's case, as::
4393+
4394+ ctxt = {
4395+ ... other context ...
4396+ 'subordinate_config': {
4397+ 'DEFAULT': {
4398+ 'key1': 'value1',
4399+ },
4400+ 'MYSECTION': {
4401+ 'key2': 'value2',
4402+ },
4403+ }
4404+ }
4405+ """
4406+
4407+ def __init__(self, service, config_file, interface):
4408+ """
4409+ :param service : Service name key to query in any subordinate
4410+ data found
4411+ :param config_file : Service's config file to query sections
4412+ :param interface : Subordinate interface to inspect
4413+ """
4414+ self.service = service
4415+ self.config_file = config_file
4416+ self.interface = interface
4417+
4418+ def __call__(self):
4419+ ctxt = {'sections': {}}
4420+ for rid in relation_ids(self.interface):
4421+ for unit in related_units(rid):
4422+ sub_config = relation_get('subordinate_configuration',
4423+ rid=rid, unit=unit)
4424+ if sub_config and sub_config != '':
4425+ try:
4426+ sub_config = json.loads(sub_config)
4427+ except:
4428+ log('Could not parse JSON from subordinate_config '
4429+ 'setting from %s' % rid, level=ERROR)
4430+ continue
4431+
4432+ if self.service not in sub_config:
4433+ log('Found subordinate_config on %s but it contained'
4434+ 'nothing for %s service' % (rid, self.service),
4435+ level=INFO)
4436+ continue
4437+
4438+ sub_config = sub_config[self.service]
4439+ if self.config_file not in sub_config:
4440+ log('Found subordinate_config on %s but it contained'
4441+ 'nothing for %s' % (rid, self.config_file),
4442+ level=INFO)
4443+ continue
4444+
4445+ sub_config = sub_config[self.config_file]
4446+ for k, v in six.iteritems(sub_config):
4447+ if k == 'sections':
4448+ for section, config_dict in six.iteritems(v):
4449+ log("adding section '%s'" % (section),
4450+ level=DEBUG)
4451+ ctxt[k][section] = config_dict
4452+ else:
4453+ ctxt[k] = v
4454+
4455+ log("%d section(s) found" % (len(ctxt['sections'])), level=DEBUG)
4456+ return ctxt
4457+
4458+
4459+class LogLevelContext(OSContextGenerator):
4460+
4461+ def __call__(self):
4462+ ctxt = {}
4463+ ctxt['debug'] = \
4464+ False if config('debug') is None else config('debug')
4465+ ctxt['verbose'] = \
4466+ False if config('verbose') is None else config('verbose')
4467+
4468+ return ctxt
4469+
4470+
4471+class SyslogContext(OSContextGenerator):
4472+
4473+ def __call__(self):
4474+ ctxt = {'use_syslog': config('use-syslog')}
4475+ return ctxt
4476+
4477+
4478+class BindHostContext(OSContextGenerator):
4479+
4480+ def __call__(self):
4481+ if config('prefer-ipv6'):
4482+ return {'bind_host': '::'}
4483+ else:
4484+ return {'bind_host': '0.0.0.0'}
4485+
4486+
4487+class WorkerConfigContext(OSContextGenerator):
4488+
4489+ @property
4490+ def num_cpus(self):
4491+ try:
4492+ from psutil import NUM_CPUS
4493+ except ImportError:
4494+ apt_install('python-psutil', fatal=True)
4495+ from psutil import NUM_CPUS
4496+
4497+ return NUM_CPUS
4498+
4499+ def __call__(self):
4500+ multiplier = config('worker-multiplier') or 0
4501+ ctxt = {"workers": self.num_cpus * multiplier}
4502+ return ctxt
4503+
4504+
4505+class ZeroMQContext(OSContextGenerator):
4506+ interfaces = ['zeromq-configuration']
4507+
4508+ def __call__(self):
4509+ ctxt = {}
4510+ if is_relation_made('zeromq-configuration', 'host'):
4511+ for rid in relation_ids('zeromq-configuration'):
4512+ for unit in related_units(rid):
4513+ ctxt['zmq_nonce'] = relation_get('nonce', unit, rid)
4514+ ctxt['zmq_host'] = relation_get('host', unit, rid)
4515+
4516+ return ctxt
4517+
4518+
4519+class NotificationDriverContext(OSContextGenerator):
4520+
4521+ def __init__(self, zmq_relation='zeromq-configuration',
4522+ amqp_relation='amqp'):
4523+ """
4524+ :param zmq_relation: Name of Zeromq relation to check
4525+ """
4526+ self.zmq_relation = zmq_relation
4527+ self.amqp_relation = amqp_relation
4528+
4529+ def __call__(self):
4530+ ctxt = {'notifications': 'False'}
4531+ if is_relation_made(self.amqp_relation):
4532+ ctxt['notifications'] = "True"
4533+
4534+ return ctxt
4535+
4536+
4537+class SysctlContext(OSContextGenerator):
4538+ """This context check if the 'sysctl' option exists on configuration
4539+ then creates a file with the loaded contents"""
4540+ def __call__(self):
4541+ sysctl_dict = config('sysctl')
4542+ if sysctl_dict:
4543+ sysctl_create(sysctl_dict,
4544+ '/etc/sysctl.d/50-{0}.conf'.format(charm_name()))
4545+ return {'sysctl': sysctl_dict}
4546
4547=== added file 'hooks/charmhelpers/contrib/openstack/ip.py'
4548--- hooks/charmhelpers/contrib/openstack/ip.py 1970-01-01 00:00:00 +0000
4549+++ hooks/charmhelpers/contrib/openstack/ip.py 2015-06-18 17:19:58 +0000
4550@@ -0,0 +1,109 @@
4551+# Copyright 2014-2015 Canonical Limited.
4552+#
4553+# This file is part of charm-helpers.
4554+#
4555+# charm-helpers is free software: you can redistribute it and/or modify
4556+# it under the terms of the GNU Lesser General Public License version 3 as
4557+# published by the Free Software Foundation.
4558+#
4559+# charm-helpers is distributed in the hope that it will be useful,
4560+# but WITHOUT ANY WARRANTY; without even the implied warranty of
4561+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4562+# GNU Lesser General Public License for more details.
4563+#
4564+# You should have received a copy of the GNU Lesser General Public License
4565+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4566+
4567+from charmhelpers.core.hookenv import (
4568+ config,
4569+ unit_get,
4570+)
4571+from charmhelpers.contrib.network.ip import (
4572+ get_address_in_network,
4573+ is_address_in_network,
4574+ is_ipv6,
4575+ get_ipv6_addr,
4576+)
4577+from charmhelpers.contrib.hahelpers.cluster import is_clustered
4578+
4579+PUBLIC = 'public'
4580+INTERNAL = 'int'
4581+ADMIN = 'admin'
4582+
4583+ADDRESS_MAP = {
4584+ PUBLIC: {
4585+ 'config': 'os-public-network',
4586+ 'fallback': 'public-address'
4587+ },
4588+ INTERNAL: {
4589+ 'config': 'os-internal-network',
4590+ 'fallback': 'private-address'
4591+ },
4592+ ADMIN: {
4593+ 'config': 'os-admin-network',
4594+ 'fallback': 'private-address'
4595+ }
4596+}
4597+
4598+
4599+def canonical_url(configs, endpoint_type=PUBLIC):
4600+ """Returns the correct HTTP URL to this host given the state of HTTPS
4601+ configuration, hacluster and charm configuration.
4602+
4603+ :param configs: OSTemplateRenderer config templating object to inspect
4604+ for a complete https context.
4605+ :param endpoint_type: str endpoint type to resolve.
4606+ :param returns: str base URL for services on the current service unit.
4607+ """
4608+ scheme = 'http'
4609+ if 'https' in configs.complete_contexts():
4610+ scheme = 'https'
4611+ address = resolve_address(endpoint_type)
4612+ if is_ipv6(address):
4613+ address = "[{}]".format(address)
4614+ return '%s://%s' % (scheme, address)
4615+
4616+
4617+def resolve_address(endpoint_type=PUBLIC):
4618+ """Return unit address depending on net config.
4619+
4620+ If unit is clustered with vip(s) and has net splits defined, return vip on
4621+ correct network. If clustered with no nets defined, return primary vip.
4622+
4623+ If not clustered, return unit address ensuring address is on configured net
4624+ split if one is configured.
4625+
4626+ :param endpoint_type: Network endpoing type
4627+ """
4628+ resolved_address = None
4629+ vips = config('vip')
4630+ if vips:
4631+ vips = vips.split()
4632+
4633+ net_type = ADDRESS_MAP[endpoint_type]['config']
4634+ net_addr = config(net_type)
4635+ net_fallback = ADDRESS_MAP[endpoint_type]['fallback']
4636+ clustered = is_clustered()
4637+ if clustered:
4638+ if not net_addr:
4639+ # If no net-splits defined, we expect a single vip
4640+ resolved_address = vips[0]
4641+ else:
4642+ for vip in vips:
4643+ if is_address_in_network(net_addr, vip):
4644+ resolved_address = vip
4645+ break
4646+ else:
4647+ if config('prefer-ipv6'):
4648+ fallback_addr = get_ipv6_addr(exc_list=vips)[0]
4649+ else:
4650+ fallback_addr = unit_get(net_fallback)
4651+
4652+ resolved_address = get_address_in_network(net_addr, fallback_addr)
4653+
4654+ if resolved_address is None:
4655+ raise ValueError("Unable to resolve a suitable IP address based on "
4656+ "charm state and configuration. (net_type=%s, "
4657+ "clustered=%s)" % (net_type, clustered))
4658+
4659+ return resolved_address
4660
4661=== added file 'hooks/charmhelpers/contrib/openstack/neutron.py'
4662--- hooks/charmhelpers/contrib/openstack/neutron.py 1970-01-01 00:00:00 +0000
4663+++ hooks/charmhelpers/contrib/openstack/neutron.py 2015-06-18 17:19:58 +0000
4664@@ -0,0 +1,239 @@
4665+# Copyright 2014-2015 Canonical Limited.
4666+#
4667+# This file is part of charm-helpers.
4668+#
4669+# charm-helpers is free software: you can redistribute it and/or modify
4670+# it under the terms of the GNU Lesser General Public License version 3 as
4671+# published by the Free Software Foundation.
4672+#
4673+# charm-helpers is distributed in the hope that it will be useful,
4674+# but WITHOUT ANY WARRANTY; without even the implied warranty of
4675+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4676+# GNU Lesser General Public License for more details.
4677+#
4678+# You should have received a copy of the GNU Lesser General Public License
4679+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4680+
4681+# Various utilies for dealing with Neutron and the renaming from Quantum.
4682+
4683+from subprocess import check_output
4684+
4685+from charmhelpers.core.hookenv import (
4686+ config,
4687+ log,
4688+ ERROR,
4689+)
4690+
4691+from charmhelpers.contrib.openstack.utils import os_release
4692+
4693+
4694+def headers_package():
4695+ """Ensures correct linux-headers for running kernel are installed,
4696+ for building DKMS package"""
4697+ kver = check_output(['uname', '-r']).decode('UTF-8').strip()
4698+ return 'linux-headers-%s' % kver
4699+
4700+QUANTUM_CONF_DIR = '/etc/quantum'
4701+
4702+
4703+def kernel_version():
4704+ """ Retrieve the current major kernel version as a tuple e.g. (3, 13) """
4705+ kver = check_output(['uname', '-r']).decode('UTF-8').strip()
4706+ kver = kver.split('.')
4707+ return (int(kver[0]), int(kver[1]))
4708+
4709+
4710+def determine_dkms_package():
4711+ """ Determine which DKMS package should be used based on kernel version """
4712+ # NOTE: 3.13 kernels have support for GRE and VXLAN native
4713+ if kernel_version() >= (3, 13):
4714+ return []
4715+ else:
4716+ return ['openvswitch-datapath-dkms']
4717+
4718+
4719+# legacy
4720+
4721+
4722+def quantum_plugins():
4723+ from charmhelpers.contrib.openstack import context
4724+ return {
4725+ 'ovs': {
4726+ 'config': '/etc/quantum/plugins/openvswitch/'
4727+ 'ovs_quantum_plugin.ini',
4728+ 'driver': 'quantum.plugins.openvswitch.ovs_quantum_plugin.'
4729+ 'OVSQuantumPluginV2',
4730+ 'contexts': [
4731+ context.SharedDBContext(user=config('neutron-database-user'),
4732+ database=config('neutron-database'),
4733+ relation_prefix='neutron',
4734+ ssl_dir=QUANTUM_CONF_DIR)],
4735+ 'services': ['quantum-plugin-openvswitch-agent'],
4736+ 'packages': [[headers_package()] + determine_dkms_package(),
4737+ ['quantum-plugin-openvswitch-agent']],
4738+ 'server_packages': ['quantum-server',
4739+ 'quantum-plugin-openvswitch'],
4740+ 'server_services': ['quantum-server']
4741+ },
4742+ 'nvp': {
4743+ 'config': '/etc/quantum/plugins/nicira/nvp.ini',
4744+ 'driver': 'quantum.plugins.nicira.nicira_nvp_plugin.'
4745+ 'QuantumPlugin.NvpPluginV2',
4746+ 'contexts': [
4747+ context.SharedDBContext(user=config('neutron-database-user'),
4748+ database=config('neutron-database'),
4749+ relation_prefix='neutron',
4750+ ssl_dir=QUANTUM_CONF_DIR)],
4751+ 'services': [],
4752+ 'packages': [],
4753+ 'server_packages': ['quantum-server',
4754+ 'quantum-plugin-nicira'],
4755+ 'server_services': ['quantum-server']
4756+ }
4757+ }
4758+
4759+NEUTRON_CONF_DIR = '/etc/neutron'
4760+
4761+
4762+def neutron_plugins():
4763+ from charmhelpers.contrib.openstack import context
4764+ release = os_release('nova-common')
4765+ plugins = {
4766+ 'ovs': {
4767+ 'config': '/etc/neutron/plugins/openvswitch/'
4768+ 'ovs_neutron_plugin.ini',
4769+ 'driver': 'neutron.plugins.openvswitch.ovs_neutron_plugin.'
4770+ 'OVSNeutronPluginV2',
4771+ 'contexts': [
4772+ context.SharedDBContext(user=config('neutron-database-user'),
4773+ database=config('neutron-database'),
4774+ relation_prefix='neutron',
4775+ ssl_dir=NEUTRON_CONF_DIR)],
4776+ 'services': ['neutron-plugin-openvswitch-agent'],
4777+ 'packages': [[headers_package()] + determine_dkms_package(),
4778+ ['neutron-plugin-openvswitch-agent']],
4779+ 'server_packages': ['neutron-server',
4780+ 'neutron-plugin-openvswitch'],
4781+ 'server_services': ['neutron-server']
4782+ },
4783+ 'nvp': {
4784+ 'config': '/etc/neutron/plugins/nicira/nvp.ini',
4785+ 'driver': 'neutron.plugins.nicira.nicira_nvp_plugin.'
4786+ 'NeutronPlugin.NvpPluginV2',
4787+ 'contexts': [
4788+ context.SharedDBContext(user=config('neutron-database-user'),
4789+ database=config('neutron-database'),
4790+ relation_prefix='neutron',
4791+ ssl_dir=NEUTRON_CONF_DIR)],
4792+ 'services': [],
4793+ 'packages': [],
4794+ 'server_packages': ['neutron-server',
4795+ 'neutron-plugin-nicira'],
4796+ 'server_services': ['neutron-server']
4797+ },
4798+ 'nsx': {
4799+ 'config': '/etc/neutron/plugins/vmware/nsx.ini',
4800+ 'driver': 'vmware',
4801+ 'contexts': [
4802+ context.SharedDBContext(user=config('neutron-database-user'),
4803+ database=config('neutron-database'),
4804+ relation_prefix='neutron',
4805+ ssl_dir=NEUTRON_CONF_DIR)],
4806+ 'services': [],
4807+ 'packages': [],
4808+ 'server_packages': ['neutron-server',
4809+ 'neutron-plugin-vmware'],
4810+ 'server_services': ['neutron-server']
4811+ },
4812+ 'n1kv': {
4813+ 'config': '/etc/neutron/plugins/cisco/cisco_plugins.ini',
4814+ 'driver': 'neutron.plugins.cisco.network_plugin.PluginV2',
4815+ 'contexts': [
4816+ context.SharedDBContext(user=config('neutron-database-user'),
4817+ database=config('neutron-database'),
4818+ relation_prefix='neutron',
4819+ ssl_dir=NEUTRON_CONF_DIR)],
4820+ 'services': [],
4821+ 'packages': [[headers_package()] + determine_dkms_package(),
4822+ ['neutron-plugin-cisco']],
4823+ 'server_packages': ['neutron-server',
4824+ 'neutron-plugin-cisco'],
4825+ 'server_services': ['neutron-server']
4826+ },
4827+ 'Calico': {
4828+ 'config': '/etc/neutron/plugins/ml2/ml2_conf.ini',
4829+ 'driver': 'neutron.plugins.ml2.plugin.Ml2Plugin',
4830+ 'contexts': [
4831+ context.SharedDBContext(user=config('neutron-database-user'),
4832+ database=config('neutron-database'),
4833+ relation_prefix='neutron',
4834+ ssl_dir=NEUTRON_CONF_DIR)],
4835+ 'services': ['calico-felix',
4836+ 'bird',
4837+ 'neutron-dhcp-agent',
4838+ 'nova-api-metadata'],
4839+ 'packages': [[headers_package()] + determine_dkms_package(),
4840+ ['calico-compute',
4841+ 'bird',
4842+ 'neutron-dhcp-agent',
4843+ 'nova-api-metadata']],
4844+ 'server_packages': ['neutron-server', 'calico-control'],
4845+ 'server_services': ['neutron-server']
4846+ }
4847+ }
4848+ if release >= 'icehouse':
4849+ # NOTE: patch in ml2 plugin for icehouse onwards
4850+ plugins['ovs']['config'] = '/etc/neutron/plugins/ml2/ml2_conf.ini'
4851+ plugins['ovs']['driver'] = 'neutron.plugins.ml2.plugin.Ml2Plugin'
4852+ plugins['ovs']['server_packages'] = ['neutron-server',
4853+ 'neutron-plugin-ml2']
4854+ # NOTE: patch in vmware renames nvp->nsx for icehouse onwards
4855+ plugins['nvp'] = plugins['nsx']
4856+ return plugins
4857+
4858+
4859+def neutron_plugin_attribute(plugin, attr, net_manager=None):
4860+ manager = net_manager or network_manager()
4861+ if manager == 'quantum':
4862+ plugins = quantum_plugins()
4863+ elif manager == 'neutron':
4864+ plugins = neutron_plugins()
4865+ else:
4866+ log("Network manager '%s' does not support plugins." % (manager),
4867+ level=ERROR)
4868+ raise Exception
4869+
4870+ try:
4871+ _plugin = plugins[plugin]
4872+ except KeyError:
4873+ log('Unrecognised plugin for %s: %s' % (manager, plugin), level=ERROR)
4874+ raise Exception
4875+
4876+ try:
4877+ return _plugin[attr]
4878+ except KeyError:
4879+ return None
4880+
4881+
4882+def network_manager():
4883+ '''
4884+ Deals with the renaming of Quantum to Neutron in H and any situations
4885+ that require compatability (eg, deploying H with network-manager=quantum,
4886+ upgrading from G).
4887+ '''
4888+ release = os_release('nova-common')
4889+ manager = config('network-manager').lower()
4890+
4891+ if manager not in ['quantum', 'neutron']:
4892+ return manager
4893+
4894+ if release in ['essex']:
4895+ # E does not support neutron
4896+ log('Neutron networking not supported in Essex.', level=ERROR)
4897+ raise Exception
4898+ elif release in ['folsom', 'grizzly']:
4899+ # neutron is named quantum in F and G
4900+ return 'quantum'
4901+ else:
4902+ # ensure accurate naming for all releases post-H
4903+ return 'neutron'
4904
4905=== added directory 'hooks/charmhelpers/contrib/openstack/templates'
4906=== added file 'hooks/charmhelpers/contrib/openstack/templates/__init__.py'
4907--- hooks/charmhelpers/contrib/openstack/templates/__init__.py 1970-01-01 00:00:00 +0000
4908+++ hooks/charmhelpers/contrib/openstack/templates/__init__.py 2015-06-18 17:19:58 +0000
4909@@ -0,0 +1,18 @@
4910+# Copyright 2014-2015 Canonical Limited.
4911+#
4912+# This file is part of charm-helpers.
4913+#
4914+# charm-helpers is free software: you can redistribute it and/or modify
4915+# it under the terms of the GNU Lesser General Public License version 3 as
4916+# published by the Free Software Foundation.
4917+#
4918+# charm-helpers is distributed in the hope that it will be useful,
4919+# but WITHOUT ANY WARRANTY; without even the implied warranty of
4920+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4921+# GNU Lesser General Public License for more details.
4922+#
4923+# You should have received a copy of the GNU Lesser General Public License
4924+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4925+
4926+# dummy __init__.py to fool syncer into thinking this is a syncable python
4927+# module
4928
4929=== added file 'hooks/charmhelpers/contrib/openstack/templating.py'
4930--- hooks/charmhelpers/contrib/openstack/templating.py 1970-01-01 00:00:00 +0000
4931+++ hooks/charmhelpers/contrib/openstack/templating.py 2015-06-18 17:19:58 +0000
4932@@ -0,0 +1,295 @@
4933+# Copyright 2014-2015 Canonical Limited.
4934+#
4935+# This file is part of charm-helpers.
4936+#
4937+# charm-helpers is free software: you can redistribute it and/or modify
4938+# it under the terms of the GNU Lesser General Public License version 3 as
4939+# published by the Free Software Foundation.
4940+#
4941+# charm-helpers is distributed in the hope that it will be useful,
4942+# but WITHOUT ANY WARRANTY; without even the implied warranty of
4943+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4944+# GNU Lesser General Public License for more details.
4945+#
4946+# You should have received a copy of the GNU Lesser General Public License
4947+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4948+
4949+import os
4950+
4951+import six
4952+
4953+from charmhelpers.fetch import apt_install
4954+from charmhelpers.core.hookenv import (
4955+ log,
4956+ ERROR,
4957+ INFO
4958+)
4959+from charmhelpers.contrib.openstack.utils import OPENSTACK_CODENAMES
4960+
4961+try:
4962+ from jinja2 import FileSystemLoader, ChoiceLoader, Environment, exceptions
4963+except ImportError:
4964+ # python-jinja2 may not be installed yet, or we're running unittests.
4965+ FileSystemLoader = ChoiceLoader = Environment = exceptions = None
4966+
4967+
4968+class OSConfigException(Exception):
4969+ pass
4970+
4971+
4972+def get_loader(templates_dir, os_release):
4973+ """
4974+ Create a jinja2.ChoiceLoader containing template dirs up to
4975+ and including os_release. If directory template directory
4976+ is missing at templates_dir, it will be omitted from the loader.
4977+ templates_dir is added to the bottom of the search list as a base
4978+ loading dir.
4979+
4980+ A charm may also ship a templates dir with this module
4981+ and it will be appended to the bottom of the search list, eg::
4982+
4983+ hooks/charmhelpers/contrib/openstack/templates
4984+
4985+ :param templates_dir (str): Base template directory containing release
4986+ sub-directories.
4987+ :param os_release (str): OpenStack release codename to construct template
4988+ loader.
4989+ :returns: jinja2.ChoiceLoader constructed with a list of
4990+ jinja2.FilesystemLoaders, ordered in descending
4991+ order by OpenStack release.
4992+ """
4993+ tmpl_dirs = [(rel, os.path.join(templates_dir, rel))
4994+ for rel in six.itervalues(OPENSTACK_CODENAMES)]
4995+
4996+ if not os.path.isdir(templates_dir):
4997+ log('Templates directory not found @ %s.' % templates_dir,
4998+ level=ERROR)
4999+ raise OSConfigException
5000+
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches

to all changes: