Merge lp:~cbjchen/charms/trusty/ubuntu/lxc-network-config into lp:charms/trusty/ubuntu

Proposed by Liang Chen on 2015-02-23
Status: Merged
Merged at revision: 9
Proposed branch: lp:~cbjchen/charms/trusty/ubuntu/lxc-network-config
Merge into: lp:charms/trusty/ubuntu
Diff against target: 3472 lines (+3356/-0)
22 files modified
charm-helpers-hooks.yaml (+5/-0)
config.yaml (+8/-0)
hooks/charmhelpers/core/__init__.py (+15/-0)
hooks/charmhelpers/core/decorators.py (+57/-0)
hooks/charmhelpers/core/fstab.py (+134/-0)
hooks/charmhelpers/core/hookenv.py (+568/-0)
hooks/charmhelpers/core/host.py (+446/-0)
hooks/charmhelpers/core/services/__init__.py (+18/-0)
hooks/charmhelpers/core/services/base.py (+329/-0)
hooks/charmhelpers/core/services/helpers.py (+259/-0)
hooks/charmhelpers/core/strutils.py (+42/-0)
hooks/charmhelpers/core/sysctl.py (+56/-0)
hooks/charmhelpers/core/templating.py (+68/-0)
hooks/charmhelpers/core/unitdata.py (+477/-0)
hooks/charmhelpers/fetch/__init__.py (+439/-0)
hooks/charmhelpers/fetch/archiveurl.py (+161/-0)
hooks/charmhelpers/fetch/bzrurl.py (+78/-0)
hooks/charmhelpers/fetch/giturl.py (+71/-0)
hooks/config-changed (+43/-0)
hooks/hooks.py (+45/-0)
hooks/utils.py (+27/-0)
templates/lxc-bridge.conf (+10/-0)
To merge this branch: bzr merge lp:~cbjchen/charms/trusty/ubuntu/lxc-network-config
Reviewer Review Type Date Requested Status
Charles Butler (community) 2015-02-23 Approve on 2015-03-27
Antonio Rosales (community) community Approve on 2015-03-24
Adam Israel Needs Fixing on 2015-03-24
Review Queue (community) automated testing Needs Fixing on 2015-03-03
Review via email: mp+250666@code.launchpad.net

Description of the change

This patch provides a config option for the ubuntu
charm, so that users can choose whether to create
a different subnet or just bridge to the exist
network that the ubuntu instance is in for lxc
containers running in the ubuntu instance.

To post a comment you must log in.
Review Queue (review-queue) wrote :

This items has failed automated testing! Results available here http://reports.vapour.ws/charm-tests/charm-bundle-test-11062-results

review: Needs Fixing (automated testing)
Adam Israel (aisrael) wrote :

Hi Liang,

Thanks for your work on improving this charm's functionality. It's passing tests for me, but it would be really useful to update the README to reflect the new config option and how to use it. Once that is done, I'm happy to retest.

review: Needs Fixing
Antonio Rosales (arosales) wrote :

The failed automated testing was due to HP Cloud not standing up in time[0] all other tests on the other substrates passed:

[0] http://reports.vapour.ws/all-bundle-and-charm-results/charm-bundle-test-11062-results/charm/charm-testing-hp/2

I'll work on a branch to update the README.md per Adam's comments.

-thanks,
Antonio

review: Approve (community)
Antonio Rosales (arosales) wrote :

I have an MP against this branch to:
  -Add documenetation to the readme about the new new-lxc-network network option for LXC.
  -Add Amulet tests to ensure the new-lxc-network config option is operating correctly.
  -Update the default to be false for new-lxc-network. Reason is the goal of the Ubuntu charm is to be a very simple charm, and is a good indicator juju functions are working correctly.

-thanks,
Antonio

Liang Chen (cbjchen) wrote :

Hi Adam and Antonio,

Thanks for the review, and thanks for the help to supply the the documentation and tests. It makes sense to me to set the default value to false. I was just trying to make sure the behavior is the same as the previous version by default. I took a look at the patch, and it looks good to me.

Charles Butler (lazypower) wrote :

Liang,

Thank you for the contribution to the Ubuntu charm. I've taken some time to review your changes, and the follow up documentation + test from Antonio.

I've taken the liberty of merging his additions to your branch, and have pushed them to the store.

Thank you for the high quality submission, tests, and documentation. This is exactly the type of merge that makes me happy to perform a final review on. Keep up the great work!

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'charm-helpers-hooks.yaml'
2--- charm-helpers-hooks.yaml 1970-01-01 00:00:00 +0000
3+++ charm-helpers-hooks.yaml 2015-02-23 19:08:44 +0000
4@@ -0,0 +1,5 @@
5+branch: lp:charm-helpers
6+destination: hooks/charmhelpers
7+include:
8+ - core
9+ - fetch
10
11=== added file 'config.yaml'
12--- config.yaml 1970-01-01 00:00:00 +0000
13+++ config.yaml 2015-02-23 19:08:44 +0000
14@@ -0,0 +1,8 @@
15+options:
16+ new-lxc-network:
17+ type: boolean
18+ default: True
19+ description: |
20+ If True creates new network for lxc containers, otherwise using the
21+ same network of the node for lxc containers.
22+
23
24=== added file 'hooks/__init__.py'
25=== added directory 'hooks/charmhelpers'
26=== added file 'hooks/charmhelpers/__init__.py'
27=== added directory 'hooks/charmhelpers/core'
28=== added file 'hooks/charmhelpers/core/__init__.py'
29--- hooks/charmhelpers/core/__init__.py 1970-01-01 00:00:00 +0000
30+++ hooks/charmhelpers/core/__init__.py 2015-02-23 19:08:44 +0000
31@@ -0,0 +1,15 @@
32+# Copyright 2014-2015 Canonical Limited.
33+#
34+# This file is part of charm-helpers.
35+#
36+# charm-helpers is free software: you can redistribute it and/or modify
37+# it under the terms of the GNU Lesser General Public License version 3 as
38+# published by the Free Software Foundation.
39+#
40+# charm-helpers is distributed in the hope that it will be useful,
41+# but WITHOUT ANY WARRANTY; without even the implied warranty of
42+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
43+# GNU Lesser General Public License for more details.
44+#
45+# You should have received a copy of the GNU Lesser General Public License
46+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
47
48=== added file 'hooks/charmhelpers/core/decorators.py'
49--- hooks/charmhelpers/core/decorators.py 1970-01-01 00:00:00 +0000
50+++ hooks/charmhelpers/core/decorators.py 2015-02-23 19:08:44 +0000
51@@ -0,0 +1,57 @@
52+# Copyright 2014-2015 Canonical Limited.
53+#
54+# This file is part of charm-helpers.
55+#
56+# charm-helpers is free software: you can redistribute it and/or modify
57+# it under the terms of the GNU Lesser General Public License version 3 as
58+# published by the Free Software Foundation.
59+#
60+# charm-helpers is distributed in the hope that it will be useful,
61+# but WITHOUT ANY WARRANTY; without even the implied warranty of
62+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
63+# GNU Lesser General Public License for more details.
64+#
65+# You should have received a copy of the GNU Lesser General Public License
66+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
67+
68+#
69+# Copyright 2014 Canonical Ltd.
70+#
71+# Authors:
72+# Edward Hope-Morley <opentastic@gmail.com>
73+#
74+
75+import time
76+
77+from charmhelpers.core.hookenv import (
78+ log,
79+ INFO,
80+)
81+
82+
83+def retry_on_exception(num_retries, base_delay=0, exc_type=Exception):
84+ """If the decorated function raises exception exc_type, allow num_retries
85+ retry attempts before raise the exception.
86+ """
87+ def _retry_on_exception_inner_1(f):
88+ def _retry_on_exception_inner_2(*args, **kwargs):
89+ retries = num_retries
90+ multiplier = 1
91+ while True:
92+ try:
93+ return f(*args, **kwargs)
94+ except exc_type:
95+ if not retries:
96+ raise
97+
98+ delay = base_delay * multiplier
99+ multiplier += 1
100+ log("Retrying '%s' %d more times (delay=%s)" %
101+ (f.__name__, retries, delay), level=INFO)
102+ retries -= 1
103+ if delay:
104+ time.sleep(delay)
105+
106+ return _retry_on_exception_inner_2
107+
108+ return _retry_on_exception_inner_1
109
110=== added file 'hooks/charmhelpers/core/fstab.py'
111--- hooks/charmhelpers/core/fstab.py 1970-01-01 00:00:00 +0000
112+++ hooks/charmhelpers/core/fstab.py 2015-02-23 19:08:44 +0000
113@@ -0,0 +1,134 @@
114+#!/usr/bin/env python
115+# -*- coding: utf-8 -*-
116+
117+# Copyright 2014-2015 Canonical Limited.
118+#
119+# This file is part of charm-helpers.
120+#
121+# charm-helpers is free software: you can redistribute it and/or modify
122+# it under the terms of the GNU Lesser General Public License version 3 as
123+# published by the Free Software Foundation.
124+#
125+# charm-helpers is distributed in the hope that it will be useful,
126+# but WITHOUT ANY WARRANTY; without even the implied warranty of
127+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
128+# GNU Lesser General Public License for more details.
129+#
130+# You should have received a copy of the GNU Lesser General Public License
131+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
132+
133+import io
134+import os
135+
136+__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
137+
138+
139+class Fstab(io.FileIO):
140+ """This class extends file in order to implement a file reader/writer
141+ for file `/etc/fstab`
142+ """
143+
144+ class Entry(object):
145+ """Entry class represents a non-comment line on the `/etc/fstab` file
146+ """
147+ def __init__(self, device, mountpoint, filesystem,
148+ options, d=0, p=0):
149+ self.device = device
150+ self.mountpoint = mountpoint
151+ self.filesystem = filesystem
152+
153+ if not options:
154+ options = "defaults"
155+
156+ self.options = options
157+ self.d = int(d)
158+ self.p = int(p)
159+
160+ def __eq__(self, o):
161+ return str(self) == str(o)
162+
163+ def __str__(self):
164+ return "{} {} {} {} {} {}".format(self.device,
165+ self.mountpoint,
166+ self.filesystem,
167+ self.options,
168+ self.d,
169+ self.p)
170+
171+ DEFAULT_PATH = os.path.join(os.path.sep, 'etc', 'fstab')
172+
173+ def __init__(self, path=None):
174+ if path:
175+ self._path = path
176+ else:
177+ self._path = self.DEFAULT_PATH
178+ super(Fstab, self).__init__(self._path, 'rb+')
179+
180+ def _hydrate_entry(self, line):
181+ # NOTE: use split with no arguments to split on any
182+ # whitespace including tabs
183+ return Fstab.Entry(*filter(
184+ lambda x: x not in ('', None),
185+ line.strip("\n").split()))
186+
187+ @property
188+ def entries(self):
189+ self.seek(0)
190+ for line in self.readlines():
191+ line = line.decode('us-ascii')
192+ try:
193+ if line.strip() and not line.strip().startswith("#"):
194+ yield self._hydrate_entry(line)
195+ except ValueError:
196+ pass
197+
198+ def get_entry_by_attr(self, attr, value):
199+ for entry in self.entries:
200+ e_attr = getattr(entry, attr)
201+ if e_attr == value:
202+ return entry
203+ return None
204+
205+ def add_entry(self, entry):
206+ if self.get_entry_by_attr('device', entry.device):
207+ return False
208+
209+ self.write((str(entry) + '\n').encode('us-ascii'))
210+ self.truncate()
211+ return entry
212+
213+ def remove_entry(self, entry):
214+ self.seek(0)
215+
216+ lines = [l.decode('us-ascii') for l in self.readlines()]
217+
218+ found = False
219+ for index, line in enumerate(lines):
220+ if line.strip() and not line.strip().startswith("#"):
221+ if self._hydrate_entry(line) == entry:
222+ found = True
223+ break
224+
225+ if not found:
226+ return False
227+
228+ lines.remove(line)
229+
230+ self.seek(0)
231+ self.write(''.join(lines).encode('us-ascii'))
232+ self.truncate()
233+ return True
234+
235+ @classmethod
236+ def remove_by_mountpoint(cls, mountpoint, path=None):
237+ fstab = cls(path=path)
238+ entry = fstab.get_entry_by_attr('mountpoint', mountpoint)
239+ if entry:
240+ return fstab.remove_entry(entry)
241+ return False
242+
243+ @classmethod
244+ def add(cls, device, mountpoint, filesystem, options=None, path=None):
245+ return cls(path=path).add_entry(Fstab.Entry(device,
246+ mountpoint, filesystem,
247+ options=options))
248
249=== added file 'hooks/charmhelpers/core/hookenv.py'
250--- hooks/charmhelpers/core/hookenv.py 1970-01-01 00:00:00 +0000
251+++ hooks/charmhelpers/core/hookenv.py 2015-02-23 19:08:44 +0000
252@@ -0,0 +1,568 @@
253+# Copyright 2014-2015 Canonical Limited.
254+#
255+# This file is part of charm-helpers.
256+#
257+# charm-helpers is free software: you can redistribute it and/or modify
258+# it under the terms of the GNU Lesser General Public License version 3 as
259+# published by the Free Software Foundation.
260+#
261+# charm-helpers is distributed in the hope that it will be useful,
262+# but WITHOUT ANY WARRANTY; without even the implied warranty of
263+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
264+# GNU Lesser General Public License for more details.
265+#
266+# You should have received a copy of the GNU Lesser General Public License
267+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
268+
269+"Interactions with the Juju environment"
270+# Copyright 2013 Canonical Ltd.
271+#
272+# Authors:
273+# Charm Helpers Developers <juju@lists.ubuntu.com>
274+
275+import os
276+import json
277+import yaml
278+import subprocess
279+import sys
280+from subprocess import CalledProcessError
281+
282+import six
283+if not six.PY3:
284+ from UserDict import UserDict
285+else:
286+ from collections import UserDict
287+
288+CRITICAL = "CRITICAL"
289+ERROR = "ERROR"
290+WARNING = "WARNING"
291+INFO = "INFO"
292+DEBUG = "DEBUG"
293+MARKER = object()
294+
295+cache = {}
296+
297+
298+def cached(func):
299+ """Cache return values for multiple executions of func + args
300+
301+ For example::
302+
303+ @cached
304+ def unit_get(attribute):
305+ pass
306+
307+ unit_get('test')
308+
309+ will cache the result of unit_get + 'test' for future calls.
310+ """
311+ def wrapper(*args, **kwargs):
312+ global cache
313+ key = str((func, args, kwargs))
314+ try:
315+ return cache[key]
316+ except KeyError:
317+ res = func(*args, **kwargs)
318+ cache[key] = res
319+ return res
320+ return wrapper
321+
322+
323+def flush(key):
324+ """Flushes any entries from function cache where the
325+ key is found in the function+args """
326+ flush_list = []
327+ for item in cache:
328+ if key in item:
329+ flush_list.append(item)
330+ for item in flush_list:
331+ del cache[item]
332+
333+
334+def log(message, level=None):
335+ """Write a message to the juju log"""
336+ command = ['juju-log']
337+ if level:
338+ command += ['-l', level]
339+ if not isinstance(message, six.string_types):
340+ message = repr(message)
341+ command += [message]
342+ subprocess.call(command)
343+
344+
345+class Serializable(UserDict):
346+ """Wrapper, an object that can be serialized to yaml or json"""
347+
348+ def __init__(self, obj):
349+ # wrap the object
350+ UserDict.__init__(self)
351+ self.data = obj
352+
353+ def __getattr__(self, attr):
354+ # See if this object has attribute.
355+ if attr in ("json", "yaml", "data"):
356+ return self.__dict__[attr]
357+ # Check for attribute in wrapped object.
358+ got = getattr(self.data, attr, MARKER)
359+ if got is not MARKER:
360+ return got
361+ # Proxy to the wrapped object via dict interface.
362+ try:
363+ return self.data[attr]
364+ except KeyError:
365+ raise AttributeError(attr)
366+
367+ def __getstate__(self):
368+ # Pickle as a standard dictionary.
369+ return self.data
370+
371+ def __setstate__(self, state):
372+ # Unpickle into our wrapper.
373+ self.data = state
374+
375+ def json(self):
376+ """Serialize the object to json"""
377+ return json.dumps(self.data)
378+
379+ def yaml(self):
380+ """Serialize the object to yaml"""
381+ return yaml.dump(self.data)
382+
383+
384+def execution_environment():
385+ """A convenient bundling of the current execution context"""
386+ context = {}
387+ context['conf'] = config()
388+ if relation_id():
389+ context['reltype'] = relation_type()
390+ context['relid'] = relation_id()
391+ context['rel'] = relation_get()
392+ context['unit'] = local_unit()
393+ context['rels'] = relations()
394+ context['env'] = os.environ
395+ return context
396+
397+
398+def in_relation_hook():
399+ """Determine whether we're running in a relation hook"""
400+ return 'JUJU_RELATION' in os.environ
401+
402+
403+def relation_type():
404+ """The scope for the current relation hook"""
405+ return os.environ.get('JUJU_RELATION', None)
406+
407+
408+def relation_id():
409+ """The relation ID for the current relation hook"""
410+ return os.environ.get('JUJU_RELATION_ID', None)
411+
412+
413+def local_unit():
414+ """Local unit ID"""
415+ return os.environ['JUJU_UNIT_NAME']
416+
417+
418+def remote_unit():
419+ """The remote unit for the current relation hook"""
420+ return os.environ['JUJU_REMOTE_UNIT']
421+
422+
423+def service_name():
424+ """The name service group this unit belongs to"""
425+ return local_unit().split('/')[0]
426+
427+
428+def hook_name():
429+ """The name of the currently executing hook"""
430+ return os.path.basename(sys.argv[0])
431+
432+
433+class Config(dict):
434+ """A dictionary representation of the charm's config.yaml, with some
435+ extra features:
436+
437+ - See which values in the dictionary have changed since the previous hook.
438+ - For values that have changed, see what the previous value was.
439+ - Store arbitrary data for use in a later hook.
440+
441+ NOTE: Do not instantiate this object directly - instead call
442+ ``hookenv.config()``, which will return an instance of :class:`Config`.
443+
444+ Example usage::
445+
446+ >>> # inside a hook
447+ >>> from charmhelpers.core import hookenv
448+ >>> config = hookenv.config()
449+ >>> config['foo']
450+ 'bar'
451+ >>> # store a new key/value for later use
452+ >>> config['mykey'] = 'myval'
453+
454+
455+ >>> # user runs `juju set mycharm foo=baz`
456+ >>> # now we're inside subsequent config-changed hook
457+ >>> config = hookenv.config()
458+ >>> config['foo']
459+ 'baz'
460+ >>> # test to see if this val has changed since last hook
461+ >>> config.changed('foo')
462+ True
463+ >>> # what was the previous value?
464+ >>> config.previous('foo')
465+ 'bar'
466+ >>> # keys/values that we add are preserved across hooks
467+ >>> config['mykey']
468+ 'myval'
469+
470+ """
471+ CONFIG_FILE_NAME = '.juju-persistent-config'
472+
473+ def __init__(self, *args, **kw):
474+ super(Config, self).__init__(*args, **kw)
475+ self.implicit_save = True
476+ self._prev_dict = None
477+ self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME)
478+ if os.path.exists(self.path):
479+ self.load_previous()
480+
481+ def __getitem__(self, key):
482+ """For regular dict lookups, check the current juju config first,
483+ then the previous (saved) copy. This ensures that user-saved values
484+ will be returned by a dict lookup.
485+
486+ """
487+ try:
488+ return dict.__getitem__(self, key)
489+ except KeyError:
490+ return (self._prev_dict or {})[key]
491+
492+ def keys(self):
493+ prev_keys = []
494+ if self._prev_dict is not None:
495+ prev_keys = self._prev_dict.keys()
496+ return list(set(prev_keys + list(dict.keys(self))))
497+
498+ def load_previous(self, path=None):
499+ """Load previous copy of config from disk.
500+
501+ In normal usage you don't need to call this method directly - it
502+ is called automatically at object initialization.
503+
504+ :param path:
505+
506+ File path from which to load the previous config. If `None`,
507+ config is loaded from the default location. If `path` is
508+ specified, subsequent `save()` calls will write to the same
509+ path.
510+
511+ """
512+ self.path = path or self.path
513+ with open(self.path) as f:
514+ self._prev_dict = json.load(f)
515+
516+ def changed(self, key):
517+ """Return True if the current value for this key is different from
518+ the previous value.
519+
520+ """
521+ if self._prev_dict is None:
522+ return True
523+ return self.previous(key) != self.get(key)
524+
525+ def previous(self, key):
526+ """Return previous value for this key, or None if there
527+ is no previous value.
528+
529+ """
530+ if self._prev_dict:
531+ return self._prev_dict.get(key)
532+ return None
533+
534+ def save(self):
535+ """Save this config to disk.
536+
537+ If the charm is using the :mod:`Services Framework <services.base>`
538+ or :meth:'@hook <Hooks.hook>' decorator, this
539+ is called automatically at the end of successful hook execution.
540+ Otherwise, it should be called directly by user code.
541+
542+ To disable automatic saves, set ``implicit_save=False`` on this
543+ instance.
544+
545+ """
546+ if self._prev_dict:
547+ for k, v in six.iteritems(self._prev_dict):
548+ if k not in self:
549+ self[k] = v
550+ with open(self.path, 'w') as f:
551+ json.dump(self, f)
552+
553+
554+@cached
555+def config(scope=None):
556+ """Juju charm configuration"""
557+ config_cmd_line = ['config-get']
558+ if scope is not None:
559+ config_cmd_line.append(scope)
560+ config_cmd_line.append('--format=json')
561+ try:
562+ config_data = json.loads(
563+ subprocess.check_output(config_cmd_line).decode('UTF-8'))
564+ if scope is not None:
565+ return config_data
566+ return Config(config_data)
567+ except ValueError:
568+ return None
569+
570+
571+@cached
572+def relation_get(attribute=None, unit=None, rid=None):
573+ """Get relation information"""
574+ _args = ['relation-get', '--format=json']
575+ if rid:
576+ _args.append('-r')
577+ _args.append(rid)
578+ _args.append(attribute or '-')
579+ if unit:
580+ _args.append(unit)
581+ try:
582+ return json.loads(subprocess.check_output(_args).decode('UTF-8'))
583+ except ValueError:
584+ return None
585+ except CalledProcessError as e:
586+ if e.returncode == 2:
587+ return None
588+ raise
589+
590+
591+def relation_set(relation_id=None, relation_settings=None, **kwargs):
592+ """Set relation information for the current unit"""
593+ relation_settings = relation_settings if relation_settings else {}
594+ relation_cmd_line = ['relation-set']
595+ if relation_id is not None:
596+ relation_cmd_line.extend(('-r', relation_id))
597+ for k, v in (list(relation_settings.items()) + list(kwargs.items())):
598+ if v is None:
599+ relation_cmd_line.append('{}='.format(k))
600+ else:
601+ relation_cmd_line.append('{}={}'.format(k, v))
602+ subprocess.check_call(relation_cmd_line)
603+ # Flush cache of any relation-gets for local unit
604+ flush(local_unit())
605+
606+
607+@cached
608+def relation_ids(reltype=None):
609+ """A list of relation_ids"""
610+ reltype = reltype or relation_type()
611+ relid_cmd_line = ['relation-ids', '--format=json']
612+ if reltype is not None:
613+ relid_cmd_line.append(reltype)
614+ return json.loads(
615+ subprocess.check_output(relid_cmd_line).decode('UTF-8')) or []
616+ return []
617+
618+
619+@cached
620+def related_units(relid=None):
621+ """A list of related units"""
622+ relid = relid or relation_id()
623+ units_cmd_line = ['relation-list', '--format=json']
624+ if relid is not None:
625+ units_cmd_line.extend(('-r', relid))
626+ return json.loads(
627+ subprocess.check_output(units_cmd_line).decode('UTF-8')) or []
628+
629+
630+@cached
631+def relation_for_unit(unit=None, rid=None):
632+ """Get the json represenation of a unit's relation"""
633+ unit = unit or remote_unit()
634+ relation = relation_get(unit=unit, rid=rid)
635+ for key in relation:
636+ if key.endswith('-list'):
637+ relation[key] = relation[key].split()
638+ relation['__unit__'] = unit
639+ return relation
640+
641+
642+@cached
643+def relations_for_id(relid=None):
644+ """Get relations of a specific relation ID"""
645+ relation_data = []
646+ relid = relid or relation_ids()
647+ for unit in related_units(relid):
648+ unit_data = relation_for_unit(unit, relid)
649+ unit_data['__relid__'] = relid
650+ relation_data.append(unit_data)
651+ return relation_data
652+
653+
654+@cached
655+def relations_of_type(reltype=None):
656+ """Get relations of a specific type"""
657+ relation_data = []
658+ reltype = reltype or relation_type()
659+ for relid in relation_ids(reltype):
660+ for relation in relations_for_id(relid):
661+ relation['__relid__'] = relid
662+ relation_data.append(relation)
663+ return relation_data
664+
665+
666+@cached
667+def metadata():
668+ """Get the current charm metadata.yaml contents as a python object"""
669+ with open(os.path.join(charm_dir(), 'metadata.yaml')) as md:
670+ return yaml.safe_load(md)
671+
672+
673+@cached
674+def relation_types():
675+ """Get a list of relation types supported by this charm"""
676+ rel_types = []
677+ md = metadata()
678+ for key in ('provides', 'requires', 'peers'):
679+ section = md.get(key)
680+ if section:
681+ rel_types.extend(section.keys())
682+ return rel_types
683+
684+
685+@cached
686+def charm_name():
687+ """Get the name of the current charm as is specified on metadata.yaml"""
688+ return metadata().get('name')
689+
690+
691+@cached
692+def relations():
693+ """Get a nested dictionary of relation data for all related units"""
694+ rels = {}
695+ for reltype in relation_types():
696+ relids = {}
697+ for relid in relation_ids(reltype):
698+ units = {local_unit(): relation_get(unit=local_unit(), rid=relid)}
699+ for unit in related_units(relid):
700+ reldata = relation_get(unit=unit, rid=relid)
701+ units[unit] = reldata
702+ relids[relid] = units
703+ rels[reltype] = relids
704+ return rels
705+
706+
707+@cached
708+def is_relation_made(relation, keys='private-address'):
709+ '''
710+ Determine whether a relation is established by checking for
711+ presence of key(s). If a list of keys is provided, they
712+ must all be present for the relation to be identified as made
713+ '''
714+ if isinstance(keys, str):
715+ keys = [keys]
716+ for r_id in relation_ids(relation):
717+ for unit in related_units(r_id):
718+ context = {}
719+ for k in keys:
720+ context[k] = relation_get(k, rid=r_id,
721+ unit=unit)
722+ if None not in context.values():
723+ return True
724+ return False
725+
726+
727+def open_port(port, protocol="TCP"):
728+ """Open a service network port"""
729+ _args = ['open-port']
730+ _args.append('{}/{}'.format(port, protocol))
731+ subprocess.check_call(_args)
732+
733+
734+def close_port(port, protocol="TCP"):
735+ """Close a service network port"""
736+ _args = ['close-port']
737+ _args.append('{}/{}'.format(port, protocol))
738+ subprocess.check_call(_args)
739+
740+
741+@cached
742+def unit_get(attribute):
743+ """Get the unit ID for the remote unit"""
744+ _args = ['unit-get', '--format=json', attribute]
745+ try:
746+ return json.loads(subprocess.check_output(_args).decode('UTF-8'))
747+ except ValueError:
748+ return None
749+
750+
751+def unit_private_ip():
752+ """Get this unit's private IP address"""
753+ return unit_get('private-address')
754+
755+
756+class UnregisteredHookError(Exception):
757+ """Raised when an undefined hook is called"""
758+ pass
759+
760+
761+class Hooks(object):
762+ """A convenient handler for hook functions.
763+
764+ Example::
765+
766+ hooks = Hooks()
767+
768+ # register a hook, taking its name from the function name
769+ @hooks.hook()
770+ def install():
771+ pass # your code here
772+
773+ # register a hook, providing a custom hook name
774+ @hooks.hook("config-changed")
775+ def config_changed():
776+ pass # your code here
777+
778+ if __name__ == "__main__":
779+ # execute a hook based on the name the program is called by
780+ hooks.execute(sys.argv)
781+ """
782+
783+ def __init__(self, config_save=True):
784+ super(Hooks, self).__init__()
785+ self._hooks = {}
786+ self._config_save = config_save
787+
788+ def register(self, name, function):
789+ """Register a hook"""
790+ self._hooks[name] = function
791+
792+ def execute(self, args):
793+ """Execute a registered hook based on args[0]"""
794+ hook_name = os.path.basename(args[0])
795+ if hook_name in self._hooks:
796+ self._hooks[hook_name]()
797+ if self._config_save:
798+ cfg = config()
799+ if cfg.implicit_save:
800+ cfg.save()
801+ else:
802+ raise UnregisteredHookError(hook_name)
803+
804+ def hook(self, *hook_names):
805+ """Decorator, registering them as hooks"""
806+ def wrapper(decorated):
807+ for hook_name in hook_names:
808+ self.register(hook_name, decorated)
809+ else:
810+ self.register(decorated.__name__, decorated)
811+ if '_' in decorated.__name__:
812+ self.register(
813+ decorated.__name__.replace('_', '-'), decorated)
814+ return decorated
815+ return wrapper
816+
817+
818+def charm_dir():
819+ """Return the root directory of the current charm"""
820+ return os.environ.get('CHARM_DIR')
821
822=== added file 'hooks/charmhelpers/core/host.py'
823--- hooks/charmhelpers/core/host.py 1970-01-01 00:00:00 +0000
824+++ hooks/charmhelpers/core/host.py 2015-02-23 19:08:44 +0000
825@@ -0,0 +1,446 @@
826+# Copyright 2014-2015 Canonical Limited.
827+#
828+# This file is part of charm-helpers.
829+#
830+# charm-helpers is free software: you can redistribute it and/or modify
831+# it under the terms of the GNU Lesser General Public License version 3 as
832+# published by the Free Software Foundation.
833+#
834+# charm-helpers is distributed in the hope that it will be useful,
835+# but WITHOUT ANY WARRANTY; without even the implied warranty of
836+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
837+# GNU Lesser General Public License for more details.
838+#
839+# You should have received a copy of the GNU Lesser General Public License
840+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
841+
842+"""Tools for working with the host system"""
843+# Copyright 2012 Canonical Ltd.
844+#
845+# Authors:
846+# Nick Moffitt <nick.moffitt@canonical.com>
847+# Matthew Wedgwood <matthew.wedgwood@canonical.com>
848+
849+import os
850+import re
851+import pwd
852+import grp
853+import random
854+import string
855+import subprocess
856+import hashlib
857+from contextlib import contextmanager
858+from collections import OrderedDict
859+
860+import six
861+
862+from .hookenv import log
863+from .fstab import Fstab
864+
865+
866+def service_start(service_name):
867+ """Start a system service"""
868+ return service('start', service_name)
869+
870+
871+def service_stop(service_name):
872+ """Stop a system service"""
873+ return service('stop', service_name)
874+
875+
876+def service_restart(service_name):
877+ """Restart a system service"""
878+ return service('restart', service_name)
879+
880+
881+def service_reload(service_name, restart_on_failure=False):
882+ """Reload a system service, optionally falling back to restart if
883+ reload fails"""
884+ service_result = service('reload', service_name)
885+ if not service_result and restart_on_failure:
886+ service_result = service('restart', service_name)
887+ return service_result
888+
889+
890+def service(action, service_name):
891+ """Control a system service"""
892+ cmd = ['service', service_name, action]
893+ return subprocess.call(cmd) == 0
894+
895+
896+def service_running(service):
897+ """Determine whether a system service is running"""
898+ try:
899+ output = subprocess.check_output(
900+ ['service', service, 'status'],
901+ stderr=subprocess.STDOUT).decode('UTF-8')
902+ except subprocess.CalledProcessError:
903+ return False
904+ else:
905+ if ("start/running" in output or "is running" in output):
906+ return True
907+ else:
908+ return False
909+
910+
911+def service_available(service_name):
912+ """Determine whether a system service is available"""
913+ try:
914+ subprocess.check_output(
915+ ['service', service_name, 'status'],
916+ stderr=subprocess.STDOUT).decode('UTF-8')
917+ except subprocess.CalledProcessError as e:
918+ return 'unrecognized service' not in e.output
919+ else:
920+ return True
921+
922+
923+def adduser(username, password=None, shell='/bin/bash', system_user=False):
924+ """Add a user to the system"""
925+ try:
926+ user_info = pwd.getpwnam(username)
927+ log('user {0} already exists!'.format(username))
928+ except KeyError:
929+ log('creating user {0}'.format(username))
930+ cmd = ['useradd']
931+ if system_user or password is None:
932+ cmd.append('--system')
933+ else:
934+ cmd.extend([
935+ '--create-home',
936+ '--shell', shell,
937+ '--password', password,
938+ ])
939+ cmd.append(username)
940+ subprocess.check_call(cmd)
941+ user_info = pwd.getpwnam(username)
942+ return user_info
943+
944+
945+def add_group(group_name, system_group=False):
946+ """Add a group to the system"""
947+ try:
948+ group_info = grp.getgrnam(group_name)
949+ log('group {0} already exists!'.format(group_name))
950+ except KeyError:
951+ log('creating group {0}'.format(group_name))
952+ cmd = ['addgroup']
953+ if system_group:
954+ cmd.append('--system')
955+ else:
956+ cmd.extend([
957+ '--group',
958+ ])
959+ cmd.append(group_name)
960+ subprocess.check_call(cmd)
961+ group_info = grp.getgrnam(group_name)
962+ return group_info
963+
964+
965+def add_user_to_group(username, group):
966+ """Add a user to a group"""
967+ cmd = [
968+ 'gpasswd', '-a',
969+ username,
970+ group
971+ ]
972+ log("Adding user {} to group {}".format(username, group))
973+ subprocess.check_call(cmd)
974+
975+
976+def rsync(from_path, to_path, flags='-r', options=None):
977+ """Replicate the contents of a path"""
978+ options = options or ['--delete', '--executability']
979+ cmd = ['/usr/bin/rsync', flags]
980+ cmd.extend(options)
981+ cmd.append(from_path)
982+ cmd.append(to_path)
983+ log(" ".join(cmd))
984+ return subprocess.check_output(cmd).decode('UTF-8').strip()
985+
986+
987+def symlink(source, destination):
988+ """Create a symbolic link"""
989+ log("Symlinking {} as {}".format(source, destination))
990+ cmd = [
991+ 'ln',
992+ '-sf',
993+ source,
994+ destination,
995+ ]
996+ subprocess.check_call(cmd)
997+
998+
999+def mkdir(path, owner='root', group='root', perms=0o555, force=False):
1000+ """Create a directory"""
1001+ log("Making dir {} {}:{} {:o}".format(path, owner, group,
1002+ perms))
1003+ uid = pwd.getpwnam(owner).pw_uid
1004+ gid = grp.getgrnam(group).gr_gid
1005+ realpath = os.path.abspath(path)
1006+ path_exists = os.path.exists(realpath)
1007+ if path_exists and force:
1008+ if not os.path.isdir(realpath):
1009+ log("Removing non-directory file {} prior to mkdir()".format(path))
1010+ os.unlink(realpath)
1011+ os.makedirs(realpath, perms)
1012+ elif not path_exists:
1013+ os.makedirs(realpath, perms)
1014+ os.chown(realpath, uid, gid)
1015+ os.chmod(realpath, perms)
1016+
1017+
1018+def write_file(path, content, owner='root', group='root', perms=0o444):
1019+ """Create or overwrite a file with the contents of a byte string."""
1020+ log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
1021+ uid = pwd.getpwnam(owner).pw_uid
1022+ gid = grp.getgrnam(group).gr_gid
1023+ with open(path, 'wb') as target:
1024+ os.fchown(target.fileno(), uid, gid)
1025+ os.fchmod(target.fileno(), perms)
1026+ target.write(content)
1027+
1028+
1029+def fstab_remove(mp):
1030+ """Remove the given mountpoint entry from /etc/fstab
1031+ """
1032+ return Fstab.remove_by_mountpoint(mp)
1033+
1034+
1035+def fstab_add(dev, mp, fs, options=None):
1036+ """Adds the given device entry to the /etc/fstab file
1037+ """
1038+ return Fstab.add(dev, mp, fs, options=options)
1039+
1040+
1041+def mount(device, mountpoint, options=None, persist=False, filesystem="ext3"):
1042+ """Mount a filesystem at a particular mountpoint"""
1043+ cmd_args = ['mount']
1044+ if options is not None:
1045+ cmd_args.extend(['-o', options])
1046+ cmd_args.extend([device, mountpoint])
1047+ try:
1048+ subprocess.check_output(cmd_args)
1049+ except subprocess.CalledProcessError as e:
1050+ log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output))
1051+ return False
1052+
1053+ if persist:
1054+ return fstab_add(device, mountpoint, filesystem, options=options)
1055+ return True
1056+
1057+
1058+def umount(mountpoint, persist=False):
1059+ """Unmount a filesystem"""
1060+ cmd_args = ['umount', mountpoint]
1061+ try:
1062+ subprocess.check_output(cmd_args)
1063+ except subprocess.CalledProcessError as e:
1064+ log('Error unmounting {}\n{}'.format(mountpoint, e.output))
1065+ return False
1066+
1067+ if persist:
1068+ return fstab_remove(mountpoint)
1069+ return True
1070+
1071+
1072+def mounts():
1073+ """Get a list of all mounted volumes as [[mountpoint,device],[...]]"""
1074+ with open('/proc/mounts') as f:
1075+ # [['/mount/point','/dev/path'],[...]]
1076+ system_mounts = [m[1::-1] for m in [l.strip().split()
1077+ for l in f.readlines()]]
1078+ return system_mounts
1079+
1080+
1081+def file_hash(path, hash_type='md5'):
1082+ """
1083+ Generate a hash checksum of the contents of 'path' or None if not found.
1084+
1085+ :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,
1086+ such as md5, sha1, sha256, sha512, etc.
1087+ """
1088+ if os.path.exists(path):
1089+ h = getattr(hashlib, hash_type)()
1090+ with open(path, 'rb') as source:
1091+ h.update(source.read())
1092+ return h.hexdigest()
1093+ else:
1094+ return None
1095+
1096+
1097+def check_hash(path, checksum, hash_type='md5'):
1098+ """
1099+ Validate a file using a cryptographic checksum.
1100+
1101+ :param str checksum: Value of the checksum used to validate the file.
1102+ :param str hash_type: Hash algorithm used to generate `checksum`.
1103+ Can be any hash alrgorithm supported by :mod:`hashlib`,
1104+ such as md5, sha1, sha256, sha512, etc.
1105+ :raises ChecksumError: If the file fails the checksum
1106+
1107+ """
1108+ actual_checksum = file_hash(path, hash_type)
1109+ if checksum != actual_checksum:
1110+ raise ChecksumError("'%s' != '%s'" % (checksum, actual_checksum))
1111+
1112+
1113+class ChecksumError(ValueError):
1114+ pass
1115+
1116+
1117+def restart_on_change(restart_map, stopstart=False):
1118+ """Restart services based on configuration files changing
1119+
1120+ This function is used a decorator, for example::
1121+
1122+ @restart_on_change({
1123+ '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]
1124+ })
1125+ def ceph_client_changed():
1126+ pass # your code here
1127+
1128+ In this example, the cinder-api and cinder-volume services
1129+ would be restarted if /etc/ceph/ceph.conf is changed by the
1130+ ceph_client_changed function.
1131+ """
1132+ def wrap(f):
1133+ def wrapped_f(*args, **kwargs):
1134+ checksums = {}
1135+ for path in restart_map:
1136+ checksums[path] = file_hash(path)
1137+ f(*args, **kwargs)
1138+ restarts = []
1139+ for path in restart_map:
1140+ if checksums[path] != file_hash(path):
1141+ restarts += restart_map[path]
1142+ services_list = list(OrderedDict.fromkeys(restarts))
1143+ if not stopstart:
1144+ for service_name in services_list:
1145+ service('restart', service_name)
1146+ else:
1147+ for action in ['stop', 'start']:
1148+ for service_name in services_list:
1149+ service(action, service_name)
1150+ return wrapped_f
1151+ return wrap
1152+
1153+
1154+def lsb_release():
1155+ """Return /etc/lsb-release in a dict"""
1156+ d = {}
1157+ with open('/etc/lsb-release', 'r') as lsb:
1158+ for l in lsb:
1159+ k, v = l.split('=')
1160+ d[k.strip()] = v.strip()
1161+ return d
1162+
1163+
1164+def pwgen(length=None):
1165+ """Generate a random pasword."""
1166+ if length is None:
1167+ length = random.choice(range(35, 45))
1168+ alphanumeric_chars = [
1169+ l for l in (string.ascii_letters + string.digits)
1170+ if l not in 'l0QD1vAEIOUaeiou']
1171+ random_chars = [
1172+ random.choice(alphanumeric_chars) for _ in range(length)]
1173+ return(''.join(random_chars))
1174+
1175+
1176+def list_nics(nic_type):
1177+ '''Return a list of nics of given type(s)'''
1178+ if isinstance(nic_type, six.string_types):
1179+ int_types = [nic_type]
1180+ else:
1181+ int_types = nic_type
1182+ interfaces = []
1183+ for int_type in int_types:
1184+ cmd = ['ip', 'addr', 'show', 'label', int_type + '*']
1185+ ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
1186+ ip_output = (line for line in ip_output if line)
1187+ for line in ip_output:
1188+ if line.split()[1].startswith(int_type):
1189+ matched = re.search('.*: (' + int_type + r'[0-9]+\.[0-9]+)@.*', line)
1190+ if matched:
1191+ interface = matched.groups()[0]
1192+ else:
1193+ interface = line.split()[1].replace(":", "")
1194+ interfaces.append(interface)
1195+
1196+ return interfaces
1197+
1198+
1199+def set_nic_mtu(nic, mtu):
1200+ '''Set MTU on a network interface'''
1201+ cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
1202+ subprocess.check_call(cmd)
1203+
1204+
1205+def get_nic_mtu(nic):
1206+ cmd = ['ip', 'addr', 'show', nic]
1207+ ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
1208+ mtu = ""
1209+ for line in ip_output:
1210+ words = line.split()
1211+ if 'mtu' in words:
1212+ mtu = words[words.index("mtu") + 1]
1213+ return mtu
1214+
1215+
1216+def get_nic_hwaddr(nic):
1217+ cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
1218+ ip_output = subprocess.check_output(cmd).decode('UTF-8')
1219+ hwaddr = ""
1220+ words = ip_output.split()
1221+ if 'link/ether' in words:
1222+ hwaddr = words[words.index('link/ether') + 1]
1223+ return hwaddr
1224+
1225+
1226+def cmp_pkgrevno(package, revno, pkgcache=None):
1227+ '''Compare supplied revno with the revno of the installed package
1228+
1229+ * 1 => Installed revno is greater than supplied arg
1230+ * 0 => Installed revno is the same as supplied arg
1231+ * -1 => Installed revno is less than supplied arg
1232+
1233+ This function imports apt_cache function from charmhelpers.fetch if
1234+ the pkgcache argument is None. Be sure to add charmhelpers.fetch if
1235+ you call this function, or pass an apt_pkg.Cache() instance.
1236+ '''
1237+ import apt_pkg
1238+ if not pkgcache:
1239+ from charmhelpers.fetch import apt_cache
1240+ pkgcache = apt_cache()
1241+ pkg = pkgcache[package]
1242+ return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
1243+
1244+
1245+@contextmanager
1246+def chdir(d):
1247+ cur = os.getcwd()
1248+ try:
1249+ yield os.chdir(d)
1250+ finally:
1251+ os.chdir(cur)
1252+
1253+
1254+def chownr(path, owner, group, follow_links=True):
1255+ uid = pwd.getpwnam(owner).pw_uid
1256+ gid = grp.getgrnam(group).gr_gid
1257+ if follow_links:
1258+ chown = os.chown
1259+ else:
1260+ chown = os.lchown
1261+
1262+ for root, dirs, files in os.walk(path):
1263+ for name in dirs + files:
1264+ full = os.path.join(root, name)
1265+ broken_symlink = os.path.lexists(full) and not os.path.exists(full)
1266+ if not broken_symlink:
1267+ chown(full, uid, gid)
1268+
1269+
1270+def lchownr(path, owner, group):
1271+ chownr(path, owner, group, follow_links=False)
1272
1273=== added directory 'hooks/charmhelpers/core/services'
1274=== added file 'hooks/charmhelpers/core/services/__init__.py'
1275--- hooks/charmhelpers/core/services/__init__.py 1970-01-01 00:00:00 +0000
1276+++ hooks/charmhelpers/core/services/__init__.py 2015-02-23 19:08:44 +0000
1277@@ -0,0 +1,18 @@
1278+# Copyright 2014-2015 Canonical Limited.
1279+#
1280+# This file is part of charm-helpers.
1281+#
1282+# charm-helpers is free software: you can redistribute it and/or modify
1283+# it under the terms of the GNU Lesser General Public License version 3 as
1284+# published by the Free Software Foundation.
1285+#
1286+# charm-helpers is distributed in the hope that it will be useful,
1287+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1288+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1289+# GNU Lesser General Public License for more details.
1290+#
1291+# You should have received a copy of the GNU Lesser General Public License
1292+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1293+
1294+from .base import * # NOQA
1295+from .helpers import * # NOQA
1296
1297=== added file 'hooks/charmhelpers/core/services/base.py'
1298--- hooks/charmhelpers/core/services/base.py 1970-01-01 00:00:00 +0000
1299+++ hooks/charmhelpers/core/services/base.py 2015-02-23 19:08:44 +0000
1300@@ -0,0 +1,329 @@
1301+# Copyright 2014-2015 Canonical Limited.
1302+#
1303+# This file is part of charm-helpers.
1304+#
1305+# charm-helpers is free software: you can redistribute it and/or modify
1306+# it under the terms of the GNU Lesser General Public License version 3 as
1307+# published by the Free Software Foundation.
1308+#
1309+# charm-helpers is distributed in the hope that it will be useful,
1310+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1311+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1312+# GNU Lesser General Public License for more details.
1313+#
1314+# You should have received a copy of the GNU Lesser General Public License
1315+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1316+
1317+import os
1318+import re
1319+import json
1320+from collections import Iterable
1321+
1322+from charmhelpers.core import host
1323+from charmhelpers.core import hookenv
1324+
1325+
1326+__all__ = ['ServiceManager', 'ManagerCallback',
1327+ 'PortManagerCallback', 'open_ports', 'close_ports', 'manage_ports',
1328+ 'service_restart', 'service_stop']
1329+
1330+
1331+class ServiceManager(object):
1332+ def __init__(self, services=None):
1333+ """
1334+ Register a list of services, given their definitions.
1335+
1336+ Service definitions are dicts in the following formats (all keys except
1337+ 'service' are optional)::
1338+
1339+ {
1340+ "service": <service name>,
1341+ "required_data": <list of required data contexts>,
1342+ "provided_data": <list of provided data contexts>,
1343+ "data_ready": <one or more callbacks>,
1344+ "data_lost": <one or more callbacks>,
1345+ "start": <one or more callbacks>,
1346+ "stop": <one or more callbacks>,
1347+ "ports": <list of ports to manage>,
1348+ }
1349+
1350+ The 'required_data' list should contain dicts of required data (or
1351+ dependency managers that act like dicts and know how to collect the data).
1352+ Only when all items in the 'required_data' list are populated are the list
1353+ of 'data_ready' and 'start' callbacks executed. See `is_ready()` for more
1354+ information.
1355+
1356+ The 'provided_data' list should contain relation data providers, most likely
1357+ a subclass of :class:`charmhelpers.core.services.helpers.RelationContext`,
1358+ that will indicate a set of data to set on a given relation.
1359+
1360+ The 'data_ready' value should be either a single callback, or a list of
1361+ callbacks, to be called when all items in 'required_data' pass `is_ready()`.
1362+ Each callback will be called with the service name as the only parameter.
1363+ After all of the 'data_ready' callbacks are called, the 'start' callbacks
1364+ are fired.
1365+
1366+ The 'data_lost' value should be either a single callback, or a list of
1367+ callbacks, to be called when a 'required_data' item no longer passes
1368+ `is_ready()`. Each callback will be called with the service name as the
1369+ only parameter. After all of the 'data_lost' callbacks are called,
1370+ the 'stop' callbacks are fired.
1371+
1372+ The 'start' value should be either a single callback, or a list of
1373+ callbacks, to be called when starting the service, after the 'data_ready'
1374+ callbacks are complete. Each callback will be called with the service
1375+ name as the only parameter. This defaults to
1376+ `[host.service_start, services.open_ports]`.
1377+
1378+ The 'stop' value should be either a single callback, or a list of
1379+ callbacks, to be called when stopping the service. If the service is
1380+ being stopped because it no longer has all of its 'required_data', this
1381+ will be called after all of the 'data_lost' callbacks are complete.
1382+ Each callback will be called with the service name as the only parameter.
1383+ This defaults to `[services.close_ports, host.service_stop]`.
1384+
1385+ The 'ports' value should be a list of ports to manage. The default
1386+ 'start' handler will open the ports after the service is started,
1387+ and the default 'stop' handler will close the ports prior to stopping
1388+ the service.
1389+
1390+
1391+ Examples:
1392+
1393+ The following registers an Upstart service called bingod that depends on
1394+ a mongodb relation and which runs a custom `db_migrate` function prior to
1395+ restarting the service, and a Runit service called spadesd::
1396+
1397+ manager = services.ServiceManager([
1398+ {
1399+ 'service': 'bingod',
1400+ 'ports': [80, 443],
1401+ 'required_data': [MongoRelation(), config(), {'my': 'data'}],
1402+ 'data_ready': [
1403+ services.template(source='bingod.conf'),
1404+ services.template(source='bingod.ini',
1405+ target='/etc/bingod.ini',
1406+ owner='bingo', perms=0400),
1407+ ],
1408+ },
1409+ {
1410+ 'service': 'spadesd',
1411+ 'data_ready': services.template(source='spadesd_run.j2',
1412+ target='/etc/sv/spadesd/run',
1413+ perms=0555),
1414+ 'start': runit_start,
1415+ 'stop': runit_stop,
1416+ },
1417+ ])
1418+ manager.manage()
1419+ """
1420+ self._ready_file = os.path.join(hookenv.charm_dir(), 'READY-SERVICES.json')
1421+ self._ready = None
1422+ self.services = {}
1423+ for service in services or []:
1424+ service_name = service['service']
1425+ self.services[service_name] = service
1426+
1427+ def manage(self):
1428+ """
1429+ Handle the current hook by doing The Right Thing with the registered services.
1430+ """
1431+ hook_name = hookenv.hook_name()
1432+ if hook_name == 'stop':
1433+ self.stop_services()
1434+ else:
1435+ self.provide_data()
1436+ self.reconfigure_services()
1437+ cfg = hookenv.config()
1438+ if cfg.implicit_save:
1439+ cfg.save()
1440+
1441+ def provide_data(self):
1442+ """
1443+ Set the relation data for each provider in the ``provided_data`` list.
1444+
1445+ A provider must have a `name` attribute, which indicates which relation
1446+ to set data on, and a `provide_data()` method, which returns a dict of
1447+ data to set.
1448+ """
1449+ hook_name = hookenv.hook_name()
1450+ for service in self.services.values():
1451+ for provider in service.get('provided_data', []):
1452+ if re.match(r'{}-relation-(joined|changed)'.format(provider.name), hook_name):
1453+ data = provider.provide_data()
1454+ _ready = provider._is_ready(data) if hasattr(provider, '_is_ready') else data
1455+ if _ready:
1456+ hookenv.relation_set(None, data)
1457+
1458+ def reconfigure_services(self, *service_names):
1459+ """
1460+ Update all files for one or more registered services, and,
1461+ if ready, optionally restart them.
1462+
1463+ If no service names are given, reconfigures all registered services.
1464+ """
1465+ for service_name in service_names or self.services.keys():
1466+ if self.is_ready(service_name):
1467+ self.fire_event('data_ready', service_name)
1468+ self.fire_event('start', service_name, default=[
1469+ service_restart,
1470+ manage_ports])
1471+ self.save_ready(service_name)
1472+ else:
1473+ if self.was_ready(service_name):
1474+ self.fire_event('data_lost', service_name)
1475+ self.fire_event('stop', service_name, default=[
1476+ manage_ports,
1477+ service_stop])
1478+ self.save_lost(service_name)
1479+
1480+ def stop_services(self, *service_names):
1481+ """
1482+ Stop one or more registered services, by name.
1483+
1484+ If no service names are given, stops all registered services.
1485+ """
1486+ for service_name in service_names or self.services.keys():
1487+ self.fire_event('stop', service_name, default=[
1488+ manage_ports,
1489+ service_stop])
1490+
1491+ def get_service(self, service_name):
1492+ """
1493+ Given the name of a registered service, return its service definition.
1494+ """
1495+ service = self.services.get(service_name)
1496+ if not service:
1497+ raise KeyError('Service not registered: %s' % service_name)
1498+ return service
1499+
1500+ def fire_event(self, event_name, service_name, default=None):
1501+ """
1502+ Fire a data_ready, data_lost, start, or stop event on a given service.
1503+ """
1504+ service = self.get_service(service_name)
1505+ callbacks = service.get(event_name, default)
1506+ if not callbacks:
1507+ return
1508+ if not isinstance(callbacks, Iterable):
1509+ callbacks = [callbacks]
1510+ for callback in callbacks:
1511+ if isinstance(callback, ManagerCallback):
1512+ callback(self, service_name, event_name)
1513+ else:
1514+ callback(service_name)
1515+
1516+ def is_ready(self, service_name):
1517+ """
1518+ Determine if a registered service is ready, by checking its 'required_data'.
1519+
1520+ A 'required_data' item can be any mapping type, and is considered ready
1521+ if `bool(item)` evaluates as True.
1522+ """
1523+ service = self.get_service(service_name)
1524+ reqs = service.get('required_data', [])
1525+ return all(bool(req) for req in reqs)
1526+
1527+ def _load_ready_file(self):
1528+ if self._ready is not None:
1529+ return
1530+ if os.path.exists(self._ready_file):
1531+ with open(self._ready_file) as fp:
1532+ self._ready = set(json.load(fp))
1533+ else:
1534+ self._ready = set()
1535+
1536+ def _save_ready_file(self):
1537+ if self._ready is None:
1538+ return
1539+ with open(self._ready_file, 'w') as fp:
1540+ json.dump(list(self._ready), fp)
1541+
1542+ def save_ready(self, service_name):
1543+ """
1544+ Save an indicator that the given service is now data_ready.
1545+ """
1546+ self._load_ready_file()
1547+ self._ready.add(service_name)
1548+ self._save_ready_file()
1549+
1550+ def save_lost(self, service_name):
1551+ """
1552+ Save an indicator that the given service is no longer data_ready.
1553+ """
1554+ self._load_ready_file()
1555+ self._ready.discard(service_name)
1556+ self._save_ready_file()
1557+
1558+ def was_ready(self, service_name):
1559+ """
1560+ Determine if the given service was previously data_ready.
1561+ """
1562+ self._load_ready_file()
1563+ return service_name in self._ready
1564+
1565+
1566+class ManagerCallback(object):
1567+ """
1568+ Special case of a callback that takes the `ServiceManager` instance
1569+ in addition to the service name.
1570+
1571+ Subclasses should implement `__call__` which should accept three parameters:
1572+
1573+ * `manager` The `ServiceManager` instance
1574+ * `service_name` The name of the service it's being triggered for
1575+ * `event_name` The name of the event that this callback is handling
1576+ """
1577+ def __call__(self, manager, service_name, event_name):
1578+ raise NotImplementedError()
1579+
1580+
1581+class PortManagerCallback(ManagerCallback):
1582+ """
1583+ Callback class that will open or close ports, for use as either
1584+ a start or stop action.
1585+ """
1586+ def __call__(self, manager, service_name, event_name):
1587+ service = manager.get_service(service_name)
1588+ new_ports = service.get('ports', [])
1589+ port_file = os.path.join(hookenv.charm_dir(), '.{}.ports'.format(service_name))
1590+ if os.path.exists(port_file):
1591+ with open(port_file) as fp:
1592+ old_ports = fp.read().split(',')
1593+ for old_port in old_ports:
1594+ if bool(old_port):
1595+ old_port = int(old_port)
1596+ if old_port not in new_ports:
1597+ hookenv.close_port(old_port)
1598+ with open(port_file, 'w') as fp:
1599+ fp.write(','.join(str(port) for port in new_ports))
1600+ for port in new_ports:
1601+ if event_name == 'start':
1602+ hookenv.open_port(port)
1603+ elif event_name == 'stop':
1604+ hookenv.close_port(port)
1605+
1606+
1607+def service_stop(service_name):
1608+ """
1609+ Wrapper around host.service_stop to prevent spurious "unknown service"
1610+ messages in the logs.
1611+ """
1612+ if host.service_running(service_name):
1613+ host.service_stop(service_name)
1614+
1615+
1616+def service_restart(service_name):
1617+ """
1618+ Wrapper around host.service_restart to prevent spurious "unknown service"
1619+ messages in the logs.
1620+ """
1621+ if host.service_available(service_name):
1622+ if host.service_running(service_name):
1623+ host.service_restart(service_name)
1624+ else:
1625+ host.service_start(service_name)
1626+
1627+
1628+# Convenience aliases
1629+open_ports = close_ports = manage_ports = PortManagerCallback()
1630
1631=== added file 'hooks/charmhelpers/core/services/helpers.py'
1632--- hooks/charmhelpers/core/services/helpers.py 1970-01-01 00:00:00 +0000
1633+++ hooks/charmhelpers/core/services/helpers.py 2015-02-23 19:08:44 +0000
1634@@ -0,0 +1,259 @@
1635+# Copyright 2014-2015 Canonical Limited.
1636+#
1637+# This file is part of charm-helpers.
1638+#
1639+# charm-helpers is free software: you can redistribute it and/or modify
1640+# it under the terms of the GNU Lesser General Public License version 3 as
1641+# published by the Free Software Foundation.
1642+#
1643+# charm-helpers is distributed in the hope that it will be useful,
1644+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1645+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1646+# GNU Lesser General Public License for more details.
1647+#
1648+# You should have received a copy of the GNU Lesser General Public License
1649+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1650+
1651+import os
1652+import yaml
1653+from charmhelpers.core import hookenv
1654+from charmhelpers.core import templating
1655+
1656+from charmhelpers.core.services.base import ManagerCallback
1657+
1658+
1659+__all__ = ['RelationContext', 'TemplateCallback',
1660+ 'render_template', 'template']
1661+
1662+
1663+class RelationContext(dict):
1664+ """
1665+ Base class for a context generator that gets relation data from juju.
1666+
1667+ Subclasses must provide the attributes `name`, which is the name of the
1668+ interface of interest, `interface`, which is the type of the interface of
1669+ interest, and `required_keys`, which is the set of keys required for the
1670+ relation to be considered complete. The data for all interfaces matching
1671+ the `name` attribute that are complete will used to populate the dictionary
1672+ values (see `get_data`, below).
1673+
1674+ The generated context will be namespaced under the relation :attr:`name`,
1675+ to prevent potential naming conflicts.
1676+
1677+ :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
1678+ :param list additional_required_keys: Extend the list of :attr:`required_keys`
1679+ """
1680+ name = None
1681+ interface = None
1682+ required_keys = []
1683+
1684+ def __init__(self, name=None, additional_required_keys=None):
1685+ if name is not None:
1686+ self.name = name
1687+ if additional_required_keys is not None:
1688+ self.required_keys.extend(additional_required_keys)
1689+ self.get_data()
1690+
1691+ def __bool__(self):
1692+ """
1693+ Returns True if all of the required_keys are available.
1694+ """
1695+ return self.is_ready()
1696+
1697+ __nonzero__ = __bool__
1698+
1699+ def __repr__(self):
1700+ return super(RelationContext, self).__repr__()
1701+
1702+ def is_ready(self):
1703+ """
1704+ Returns True if all of the `required_keys` are available from any units.
1705+ """
1706+ ready = len(self.get(self.name, [])) > 0
1707+ if not ready:
1708+ hookenv.log('Incomplete relation: {}'.format(self.__class__.__name__), hookenv.DEBUG)
1709+ return ready
1710+
1711+ def _is_ready(self, unit_data):
1712+ """
1713+ Helper method that tests a set of relation data and returns True if
1714+ all of the `required_keys` are present.
1715+ """
1716+ return set(unit_data.keys()).issuperset(set(self.required_keys))
1717+
1718+ def get_data(self):
1719+ """
1720+ Retrieve the relation data for each unit involved in a relation and,
1721+ if complete, store it in a list under `self[self.name]`. This
1722+ is automatically called when the RelationContext is instantiated.
1723+
1724+ The units are sorted lexographically first by the service ID, then by
1725+ the unit ID. Thus, if an interface has two other services, 'db:1'
1726+ and 'db:2', with 'db:1' having two units, 'wordpress/0' and 'wordpress/1',
1727+ and 'db:2' having one unit, 'mediawiki/0', all of which have a complete
1728+ set of data, the relation data for the units will be stored in the
1729+ order: 'wordpress/0', 'wordpress/1', 'mediawiki/0'.
1730+
1731+ If you only care about a single unit on the relation, you can just
1732+ access it as `{{ interface[0]['key'] }}`. However, if you can at all
1733+ support multiple units on a relation, you should iterate over the list,
1734+ like::
1735+
1736+ {% for unit in interface -%}
1737+ {{ unit['key'] }}{% if not loop.last %},{% endif %}
1738+ {%- endfor %}
1739+
1740+ Note that since all sets of relation data from all related services and
1741+ units are in a single list, if you need to know which service or unit a
1742+ set of data came from, you'll need to extend this class to preserve
1743+ that information.
1744+ """
1745+ if not hookenv.relation_ids(self.name):
1746+ return
1747+
1748+ ns = self.setdefault(self.name, [])
1749+ for rid in sorted(hookenv.relation_ids(self.name)):
1750+ for unit in sorted(hookenv.related_units(rid)):
1751+ reldata = hookenv.relation_get(rid=rid, unit=unit)
1752+ if self._is_ready(reldata):
1753+ ns.append(reldata)
1754+
1755+ def provide_data(self):
1756+ """
1757+ Return data to be relation_set for this interface.
1758+ """
1759+ return {}
1760+
1761+
1762+class MysqlRelation(RelationContext):
1763+ """
1764+ Relation context for the `mysql` interface.
1765+
1766+ :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
1767+ :param list additional_required_keys: Extend the list of :attr:`required_keys`
1768+ """
1769+ name = 'db'
1770+ interface = 'mysql'
1771+ required_keys = ['host', 'user', 'password', 'database']
1772+
1773+
1774+class HttpRelation(RelationContext):
1775+ """
1776+ Relation context for the `http` interface.
1777+
1778+ :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
1779+ :param list additional_required_keys: Extend the list of :attr:`required_keys`
1780+ """
1781+ name = 'website'
1782+ interface = 'http'
1783+ required_keys = ['host', 'port']
1784+
1785+ def provide_data(self):
1786+ return {
1787+ 'host': hookenv.unit_get('private-address'),
1788+ 'port': 80,
1789+ }
1790+
1791+
1792+class RequiredConfig(dict):
1793+ """
1794+ Data context that loads config options with one or more mandatory options.
1795+
1796+ Once the required options have been changed from their default values, all
1797+ config options will be available, namespaced under `config` to prevent
1798+ potential naming conflicts (for example, between a config option and a
1799+ relation property).
1800+
1801+ :param list *args: List of options that must be changed from their default values.
1802+ """
1803+
1804+ def __init__(self, *args):
1805+ self.required_options = args
1806+ self['config'] = hookenv.config()
1807+ with open(os.path.join(hookenv.charm_dir(), 'config.yaml')) as fp:
1808+ self.config = yaml.load(fp).get('options', {})
1809+
1810+ def __bool__(self):
1811+ for option in self.required_options:
1812+ if option not in self['config']:
1813+ return False
1814+ current_value = self['config'][option]
1815+ default_value = self.config[option].get('default')
1816+ if current_value == default_value:
1817+ return False
1818+ if current_value in (None, '') and default_value in (None, ''):
1819+ return False
1820+ return True
1821+
1822+ def __nonzero__(self):
1823+ return self.__bool__()
1824+
1825+
1826+class StoredContext(dict):
1827+ """
1828+ A data context that always returns the data that it was first created with.
1829+
1830+ This is useful to do a one-time generation of things like passwords, that
1831+ will thereafter use the same value that was originally generated, instead
1832+ of generating a new value each time it is run.
1833+ """
1834+ def __init__(self, file_name, config_data):
1835+ """
1836+ If the file exists, populate `self` with the data from the file.
1837+ Otherwise, populate with the given data and persist it to the file.
1838+ """
1839+ if os.path.exists(file_name):
1840+ self.update(self.read_context(file_name))
1841+ else:
1842+ self.store_context(file_name, config_data)
1843+ self.update(config_data)
1844+
1845+ def store_context(self, file_name, config_data):
1846+ if not os.path.isabs(file_name):
1847+ file_name = os.path.join(hookenv.charm_dir(), file_name)
1848+ with open(file_name, 'w') as file_stream:
1849+ os.fchmod(file_stream.fileno(), 0o600)
1850+ yaml.dump(config_data, file_stream)
1851+
1852+ def read_context(self, file_name):
1853+ if not os.path.isabs(file_name):
1854+ file_name = os.path.join(hookenv.charm_dir(), file_name)
1855+ with open(file_name, 'r') as file_stream:
1856+ data = yaml.load(file_stream)
1857+ if not data:
1858+ raise OSError("%s is empty" % file_name)
1859+ return data
1860+
1861+
1862+class TemplateCallback(ManagerCallback):
1863+ """
1864+ Callback class that will render a Jinja2 template, for use as a ready
1865+ action.
1866+
1867+ :param str source: The template source file, relative to
1868+ `$CHARM_DIR/templates`
1869+
1870+ :param str target: The target to write the rendered template to
1871+ :param str owner: The owner of the rendered file
1872+ :param str group: The group of the rendered file
1873+ :param int perms: The permissions of the rendered file
1874+ """
1875+ def __init__(self, source, target,
1876+ owner='root', group='root', perms=0o444):
1877+ self.source = source
1878+ self.target = target
1879+ self.owner = owner
1880+ self.group = group
1881+ self.perms = perms
1882+
1883+ def __call__(self, manager, service_name, event_name):
1884+ service = manager.get_service(service_name)
1885+ context = {}
1886+ for ctx in service.get('required_data', []):
1887+ context.update(ctx)
1888+ templating.render(self.source, self.target, context,
1889+ self.owner, self.group, self.perms)
1890+
1891+
1892+# Convenience aliases for templates
1893+render_template = template = TemplateCallback
1894
1895=== added file 'hooks/charmhelpers/core/strutils.py'
1896--- hooks/charmhelpers/core/strutils.py 1970-01-01 00:00:00 +0000
1897+++ hooks/charmhelpers/core/strutils.py 2015-02-23 19:08:44 +0000
1898@@ -0,0 +1,42 @@
1899+#!/usr/bin/env python
1900+# -*- coding: utf-8 -*-
1901+
1902+# Copyright 2014-2015 Canonical Limited.
1903+#
1904+# This file is part of charm-helpers.
1905+#
1906+# charm-helpers is free software: you can redistribute it and/or modify
1907+# it under the terms of the GNU Lesser General Public License version 3 as
1908+# published by the Free Software Foundation.
1909+#
1910+# charm-helpers is distributed in the hope that it will be useful,
1911+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1912+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1913+# GNU Lesser General Public License for more details.
1914+#
1915+# You should have received a copy of the GNU Lesser General Public License
1916+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1917+
1918+import six
1919+
1920+
1921+def bool_from_string(value):
1922+ """Interpret string value as boolean.
1923+
1924+ Returns True if value translates to True otherwise False.
1925+ """
1926+ if isinstance(value, six.string_types):
1927+ value = six.text_type(value)
1928+ else:
1929+ msg = "Unable to interpret non-string value '%s' as boolean" % (value)
1930+ raise ValueError(msg)
1931+
1932+ value = value.strip().lower()
1933+
1934+ if value in ['y', 'yes', 'true', 't']:
1935+ return True
1936+ elif value in ['n', 'no', 'false', 'f']:
1937+ return False
1938+
1939+ msg = "Unable to interpret string value '%s' as boolean" % (value)
1940+ raise ValueError(msg)
1941
1942=== added file 'hooks/charmhelpers/core/sysctl.py'
1943--- hooks/charmhelpers/core/sysctl.py 1970-01-01 00:00:00 +0000
1944+++ hooks/charmhelpers/core/sysctl.py 2015-02-23 19:08:44 +0000
1945@@ -0,0 +1,56 @@
1946+#!/usr/bin/env python
1947+# -*- coding: utf-8 -*-
1948+
1949+# Copyright 2014-2015 Canonical Limited.
1950+#
1951+# This file is part of charm-helpers.
1952+#
1953+# charm-helpers is free software: you can redistribute it and/or modify
1954+# it under the terms of the GNU Lesser General Public License version 3 as
1955+# published by the Free Software Foundation.
1956+#
1957+# charm-helpers is distributed in the hope that it will be useful,
1958+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1959+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1960+# GNU Lesser General Public License for more details.
1961+#
1962+# You should have received a copy of the GNU Lesser General Public License
1963+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1964+
1965+import yaml
1966+
1967+from subprocess import check_call
1968+
1969+from charmhelpers.core.hookenv import (
1970+ log,
1971+ DEBUG,
1972+ ERROR,
1973+)
1974+
1975+__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
1976+
1977+
1978+def create(sysctl_dict, sysctl_file):
1979+ """Creates a sysctl.conf file from a YAML associative array
1980+
1981+ :param sysctl_dict: a YAML-formatted string of sysctl options eg "{ 'kernel.max_pid': 1337 }"
1982+ :type sysctl_dict: str
1983+ :param sysctl_file: path to the sysctl file to be saved
1984+ :type sysctl_file: str or unicode
1985+ :returns: None
1986+ """
1987+ try:
1988+ sysctl_dict_parsed = yaml.safe_load(sysctl_dict)
1989+ except yaml.YAMLError:
1990+ log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict),
1991+ level=ERROR)
1992+ return
1993+
1994+ with open(sysctl_file, "w") as fd:
1995+ for key, value in sysctl_dict_parsed.items():
1996+ fd.write("{}={}\n".format(key, value))
1997+
1998+ log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict_parsed),
1999+ level=DEBUG)
2000+
2001+ check_call(["sysctl", "-p", sysctl_file])
2002
2003=== added file 'hooks/charmhelpers/core/templating.py'
2004--- hooks/charmhelpers/core/templating.py 1970-01-01 00:00:00 +0000
2005+++ hooks/charmhelpers/core/templating.py 2015-02-23 19:08:44 +0000
2006@@ -0,0 +1,68 @@
2007+# Copyright 2014-2015 Canonical Limited.
2008+#
2009+# This file is part of charm-helpers.
2010+#
2011+# charm-helpers is free software: you can redistribute it and/or modify
2012+# it under the terms of the GNU Lesser General Public License version 3 as
2013+# published by the Free Software Foundation.
2014+#
2015+# charm-helpers is distributed in the hope that it will be useful,
2016+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2017+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2018+# GNU Lesser General Public License for more details.
2019+#
2020+# You should have received a copy of the GNU Lesser General Public License
2021+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2022+
2023+import os
2024+
2025+from charmhelpers.core import host
2026+from charmhelpers.core import hookenv
2027+
2028+
2029+def render(source, target, context, owner='root', group='root',
2030+ perms=0o444, templates_dir=None, encoding='UTF-8'):
2031+ """
2032+ Render a template.
2033+
2034+ The `source` path, if not absolute, is relative to the `templates_dir`.
2035+
2036+ The `target` path should be absolute.
2037+
2038+ The context should be a dict containing the values to be replaced in the
2039+ template.
2040+
2041+ The `owner`, `group`, and `perms` options will be passed to `write_file`.
2042+
2043+ If omitted, `templates_dir` defaults to the `templates` folder in the charm.
2044+
2045+ Note: Using this requires python-jinja2; if it is not installed, calling
2046+ this will attempt to use charmhelpers.fetch.apt_install to install it.
2047+ """
2048+ try:
2049+ from jinja2 import FileSystemLoader, Environment, exceptions
2050+ except ImportError:
2051+ try:
2052+ from charmhelpers.fetch import apt_install
2053+ except ImportError:
2054+ hookenv.log('Could not import jinja2, and could not import '
2055+ 'charmhelpers.fetch to install it',
2056+ level=hookenv.ERROR)
2057+ raise
2058+ apt_install('python-jinja2', fatal=True)
2059+ from jinja2 import FileSystemLoader, Environment, exceptions
2060+
2061+ if templates_dir is None:
2062+ templates_dir = os.path.join(hookenv.charm_dir(), 'templates')
2063+ loader = Environment(loader=FileSystemLoader(templates_dir))
2064+ try:
2065+ source = source
2066+ template = loader.get_template(source)
2067+ except exceptions.TemplateNotFound as e:
2068+ hookenv.log('Could not load template %s from %s.' %
2069+ (source, templates_dir),
2070+ level=hookenv.ERROR)
2071+ raise e
2072+ content = template.render(context)
2073+ host.mkdir(os.path.dirname(target), owner, group, perms=0o755)
2074+ host.write_file(target, content.encode(encoding), owner, group, perms)
2075
2076=== added file 'hooks/charmhelpers/core/unitdata.py'
2077--- hooks/charmhelpers/core/unitdata.py 1970-01-01 00:00:00 +0000
2078+++ hooks/charmhelpers/core/unitdata.py 2015-02-23 19:08:44 +0000
2079@@ -0,0 +1,477 @@
2080+#!/usr/bin/env python
2081+# -*- coding: utf-8 -*-
2082+#
2083+# Copyright 2014-2015 Canonical Limited.
2084+#
2085+# This file is part of charm-helpers.
2086+#
2087+# charm-helpers is free software: you can redistribute it and/or modify
2088+# it under the terms of the GNU Lesser General Public License version 3 as
2089+# published by the Free Software Foundation.
2090+#
2091+# charm-helpers is distributed in the hope that it will be useful,
2092+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2093+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2094+# GNU Lesser General Public License for more details.
2095+#
2096+# You should have received a copy of the GNU Lesser General Public License
2097+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2098+#
2099+#
2100+# Authors:
2101+# Kapil Thangavelu <kapil.foss@gmail.com>
2102+#
2103+"""
2104+Intro
2105+-----
2106+
2107+A simple way to store state in units. This provides a key value
2108+storage with support for versioned, transactional operation,
2109+and can calculate deltas from previous values to simplify unit logic
2110+when processing changes.
2111+
2112+
2113+Hook Integration
2114+----------------
2115+
2116+There are several extant frameworks for hook execution, including
2117+
2118+ - charmhelpers.core.hookenv.Hooks
2119+ - charmhelpers.core.services.ServiceManager
2120+
2121+The storage classes are framework agnostic, one simple integration is
2122+via the HookData contextmanager. It will record the current hook
2123+execution environment (including relation data, config data, etc.),
2124+setup a transaction and allow easy access to the changes from
2125+previously seen values. One consequence of the integration is the
2126+reservation of particular keys ('rels', 'unit', 'env', 'config',
2127+'charm_revisions') for their respective values.
2128+
2129+Here's a fully worked integration example using hookenv.Hooks::
2130+
2131+ from charmhelper.core import hookenv, unitdata
2132+
2133+ hook_data = unitdata.HookData()
2134+ db = unitdata.kv()
2135+ hooks = hookenv.Hooks()
2136+
2137+ @hooks.hook
2138+ def config_changed():
2139+ # Print all changes to configuration from previously seen
2140+ # values.
2141+ for changed, (prev, cur) in hook_data.conf.items():
2142+ print('config changed', changed,
2143+ 'previous value', prev,
2144+ 'current value', cur)
2145+
2146+ # Get some unit specific bookeeping
2147+ if not db.get('pkg_key'):
2148+ key = urllib.urlopen('https://example.com/pkg_key').read()
2149+ db.set('pkg_key', key)
2150+
2151+ # Directly access all charm config as a mapping.
2152+ conf = db.getrange('config', True)
2153+
2154+ # Directly access all relation data as a mapping
2155+ rels = db.getrange('rels', True)
2156+
2157+ if __name__ == '__main__':
2158+ with hook_data():
2159+ hook.execute()
2160+
2161+
2162+A more basic integration is via the hook_scope context manager which simply
2163+manages transaction scope (and records hook name, and timestamp)::
2164+
2165+ >>> from unitdata import kv
2166+ >>> db = kv()
2167+ >>> with db.hook_scope('install'):
2168+ ... # do work, in transactional scope.
2169+ ... db.set('x', 1)
2170+ >>> db.get('x')
2171+ 1
2172+
2173+
2174+Usage
2175+-----
2176+
2177+Values are automatically json de/serialized to preserve basic typing
2178+and complex data struct capabilities (dicts, lists, ints, booleans, etc).
2179+
2180+Individual values can be manipulated via get/set::
2181+
2182+ >>> kv.set('y', True)
2183+ >>> kv.get('y')
2184+ True
2185+
2186+ # We can set complex values (dicts, lists) as a single key.
2187+ >>> kv.set('config', {'a': 1, 'b': True'})
2188+
2189+ # Also supports returning dictionaries as a record which
2190+ # provides attribute access.
2191+ >>> config = kv.get('config', record=True)
2192+ >>> config.b
2193+ True
2194+
2195+
2196+Groups of keys can be manipulated with update/getrange::
2197+
2198+ >>> kv.update({'z': 1, 'y': 2}, prefix="gui.")
2199+ >>> kv.getrange('gui.', strip=True)
2200+ {'z': 1, 'y': 2}
2201+
2202+When updating values, its very helpful to understand which values
2203+have actually changed and how have they changed. The storage
2204+provides a delta method to provide for this::
2205+
2206+ >>> data = {'debug': True, 'option': 2}
2207+ >>> delta = kv.delta(data, 'config.')
2208+ >>> delta.debug.previous
2209+ None
2210+ >>> delta.debug.current
2211+ True
2212+ >>> delta
2213+ {'debug': (None, True), 'option': (None, 2)}
2214+
2215+Note the delta method does not persist the actual change, it needs to
2216+be explicitly saved via 'update' method::
2217+
2218+ >>> kv.update(data, 'config.')
2219+
2220+Values modified in the context of a hook scope retain historical values
2221+associated to the hookname.
2222+
2223+ >>> with db.hook_scope('config-changed'):
2224+ ... db.set('x', 42)
2225+ >>> db.gethistory('x')
2226+ [(1, u'x', 1, u'install', u'2015-01-21T16:49:30.038372'),
2227+ (2, u'x', 42, u'config-changed', u'2015-01-21T16:49:30.038786')]
2228+
2229+"""
2230+
2231+import collections
2232+import contextlib
2233+import datetime
2234+import json
2235+import os
2236+import pprint
2237+import sqlite3
2238+import sys
2239+
2240+__author__ = 'Kapil Thangavelu <kapil.foss@gmail.com>'
2241+
2242+
2243+class Storage(object):
2244+ """Simple key value database for local unit state within charms.
2245+
2246+ Modifications are automatically committed at hook exit. That's
2247+ currently regardless of exit code.
2248+
2249+ To support dicts, lists, integer, floats, and booleans values
2250+ are automatically json encoded/decoded.
2251+ """
2252+ def __init__(self, path=None):
2253+ self.db_path = path
2254+ if path is None:
2255+ self.db_path = os.path.join(
2256+ os.environ.get('CHARM_DIR', ''), '.unit-state.db')
2257+ self.conn = sqlite3.connect('%s' % self.db_path)
2258+ self.cursor = self.conn.cursor()
2259+ self.revision = None
2260+ self._closed = False
2261+ self._init()
2262+
2263+ def close(self):
2264+ if self._closed:
2265+ return
2266+ self.flush(False)
2267+ self.cursor.close()
2268+ self.conn.close()
2269+ self._closed = True
2270+
2271+ def _scoped_query(self, stmt, params=None):
2272+ if params is None:
2273+ params = []
2274+ return stmt, params
2275+
2276+ def get(self, key, default=None, record=False):
2277+ self.cursor.execute(
2278+ *self._scoped_query(
2279+ 'select data from kv where key=?', [key]))
2280+ result = self.cursor.fetchone()
2281+ if not result:
2282+ return default
2283+ if record:
2284+ return Record(json.loads(result[0]))
2285+ return json.loads(result[0])
2286+
2287+ def getrange(self, key_prefix, strip=False):
2288+ stmt = "select key, data from kv where key like '%s%%'" % key_prefix
2289+ self.cursor.execute(*self._scoped_query(stmt))
2290+ result = self.cursor.fetchall()
2291+
2292+ if not result:
2293+ return None
2294+ if not strip:
2295+ key_prefix = ''
2296+ return dict([
2297+ (k[len(key_prefix):], json.loads(v)) for k, v in result])
2298+
2299+ def update(self, mapping, prefix=""):
2300+ for k, v in mapping.items():
2301+ self.set("%s%s" % (prefix, k), v)
2302+
2303+ def unset(self, key):
2304+ self.cursor.execute('delete from kv where key=?', [key])
2305+ if self.revision and self.cursor.rowcount:
2306+ self.cursor.execute(
2307+ 'insert into kv_revisions values (?, ?, ?)',
2308+ [key, self.revision, json.dumps('DELETED')])
2309+
2310+ def set(self, key, value):
2311+ serialized = json.dumps(value)
2312+
2313+ self.cursor.execute(
2314+ 'select data from kv where key=?', [key])
2315+ exists = self.cursor.fetchone()
2316+
2317+ # Skip mutations to the same value
2318+ if exists:
2319+ if exists[0] == serialized:
2320+ return value
2321+
2322+ if not exists:
2323+ self.cursor.execute(
2324+ 'insert into kv (key, data) values (?, ?)',
2325+ (key, serialized))
2326+ else:
2327+ self.cursor.execute('''
2328+ update kv
2329+ set data = ?
2330+ where key = ?''', [serialized, key])
2331+
2332+ # Save
2333+ if not self.revision:
2334+ return value
2335+
2336+ self.cursor.execute(
2337+ 'select 1 from kv_revisions where key=? and revision=?',
2338+ [key, self.revision])
2339+ exists = self.cursor.fetchone()
2340+
2341+ if not exists:
2342+ self.cursor.execute(
2343+ '''insert into kv_revisions (
2344+ revision, key, data) values (?, ?, ?)''',
2345+ (self.revision, key, serialized))
2346+ else:
2347+ self.cursor.execute(
2348+ '''
2349+ update kv_revisions
2350+ set data = ?
2351+ where key = ?
2352+ and revision = ?''',
2353+ [serialized, key, self.revision])
2354+
2355+ return value
2356+
2357+ def delta(self, mapping, prefix):
2358+ """
2359+ return a delta containing values that have changed.
2360+ """
2361+ previous = self.getrange(prefix, strip=True)
2362+ if not previous:
2363+ pk = set()
2364+ else:
2365+ pk = set(previous.keys())
2366+ ck = set(mapping.keys())
2367+ delta = DeltaSet()
2368+
2369+ # added
2370+ for k in ck.difference(pk):
2371+ delta[k] = Delta(None, mapping[k])
2372+
2373+ # removed
2374+ for k in pk.difference(ck):
2375+ delta[k] = Delta(previous[k], None)
2376+
2377+ # changed
2378+ for k in pk.intersection(ck):
2379+ c = mapping[k]
2380+ p = previous[k]
2381+ if c != p:
2382+ delta[k] = Delta(p, c)
2383+
2384+ return delta
2385+
2386+ @contextlib.contextmanager
2387+ def hook_scope(self, name=""):
2388+ """Scope all future interactions to the current hook execution
2389+ revision."""
2390+ assert not self.revision
2391+ self.cursor.execute(
2392+ 'insert into hooks (hook, date) values (?, ?)',
2393+ (name or sys.argv[0],
2394+ datetime.datetime.utcnow().isoformat()))
2395+ self.revision = self.cursor.lastrowid
2396+ try:
2397+ yield self.revision
2398+ self.revision = None
2399+ except:
2400+ self.flush(False)
2401+ self.revision = None
2402+ raise
2403+ else:
2404+ self.flush()
2405+
2406+ def flush(self, save=True):
2407+ if save:
2408+ self.conn.commit()
2409+ elif self._closed:
2410+ return
2411+ else:
2412+ self.conn.rollback()
2413+
2414+ def _init(self):
2415+ self.cursor.execute('''
2416+ create table if not exists kv (
2417+ key text,
2418+ data text,
2419+ primary key (key)
2420+ )''')
2421+ self.cursor.execute('''
2422+ create table if not exists kv_revisions (
2423+ key text,
2424+ revision integer,
2425+ data text,
2426+ primary key (key, revision)
2427+ )''')
2428+ self.cursor.execute('''
2429+ create table if not exists hooks (
2430+ version integer primary key autoincrement,
2431+ hook text,
2432+ date text
2433+ )''')
2434+ self.conn.commit()
2435+
2436+ def gethistory(self, key, deserialize=False):
2437+ self.cursor.execute(
2438+ '''
2439+ select kv.revision, kv.key, kv.data, h.hook, h.date
2440+ from kv_revisions kv,
2441+ hooks h
2442+ where kv.key=?
2443+ and kv.revision = h.version
2444+ ''', [key])
2445+ if deserialize is False:
2446+ return self.cursor.fetchall()
2447+ return map(_parse_history, self.cursor.fetchall())
2448+
2449+ def debug(self, fh=sys.stderr):
2450+ self.cursor.execute('select * from kv')
2451+ pprint.pprint(self.cursor.fetchall(), stream=fh)
2452+ self.cursor.execute('select * from kv_revisions')
2453+ pprint.pprint(self.cursor.fetchall(), stream=fh)
2454+
2455+
2456+def _parse_history(d):
2457+ return (d[0], d[1], json.loads(d[2]), d[3],
2458+ datetime.datetime.strptime(d[-1], "%Y-%m-%dT%H:%M:%S.%f"))
2459+
2460+
2461+class HookData(object):
2462+ """Simple integration for existing hook exec frameworks.
2463+
2464+ Records all unit information, and stores deltas for processing
2465+ by the hook.
2466+
2467+ Sample::
2468+
2469+ from charmhelper.core import hookenv, unitdata
2470+
2471+ changes = unitdata.HookData()
2472+ db = unitdata.kv()
2473+ hooks = hookenv.Hooks()
2474+
2475+ @hooks.hook
2476+ def config_changed():
2477+ # View all changes to configuration
2478+ for changed, (prev, cur) in changes.conf.items():
2479+ print('config changed', changed,
2480+ 'previous value', prev,
2481+ 'current value', cur)
2482+
2483+ # Get some unit specific bookeeping
2484+ if not db.get('pkg_key'):
2485+ key = urllib.urlopen('https://example.com/pkg_key').read()
2486+ db.set('pkg_key', key)
2487+
2488+ if __name__ == '__main__':
2489+ with changes():
2490+ hook.execute()
2491+
2492+ """
2493+ def __init__(self):
2494+ self.kv = kv()
2495+ self.conf = None
2496+ self.rels = None
2497+
2498+ @contextlib.contextmanager
2499+ def __call__(self):
2500+ from charmhelpers.core import hookenv
2501+ hook_name = hookenv.hook_name()
2502+
2503+ with self.kv.hook_scope(hook_name):
2504+ self._record_charm_version(hookenv.charm_dir())
2505+ delta_config, delta_relation = self._record_hook(hookenv)
2506+ yield self.kv, delta_config, delta_relation
2507+
2508+ def _record_charm_version(self, charm_dir):
2509+ # Record revisions.. charm revisions are meaningless
2510+ # to charm authors as they don't control the revision.
2511+ # so logic dependnent on revision is not particularly
2512+ # useful, however it is useful for debugging analysis.
2513+ charm_rev = open(
2514+ os.path.join(charm_dir, 'revision')).read().strip()
2515+ charm_rev = charm_rev or '0'
2516+ revs = self.kv.get('charm_revisions', [])
2517+ if charm_rev not in revs:
2518+ revs.append(charm_rev.strip() or '0')
2519+ self.kv.set('charm_revisions', revs)
2520+
2521+ def _record_hook(self, hookenv):
2522+ data = hookenv.execution_environment()
2523+ self.conf = conf_delta = self.kv.delta(data['conf'], 'config')
2524+ self.rels = rels_delta = self.kv.delta(data['rels'], 'rels')
2525+ self.kv.set('env', data['env'])
2526+ self.kv.set('unit', data['unit'])
2527+ self.kv.set('relid', data.get('relid'))
2528+ return conf_delta, rels_delta
2529+
2530+
2531+class Record(dict):
2532+
2533+ __slots__ = ()
2534+
2535+ def __getattr__(self, k):
2536+ if k in self:
2537+ return self[k]
2538+ raise AttributeError(k)
2539+
2540+
2541+class DeltaSet(Record):
2542+
2543+ __slots__ = ()
2544+
2545+
2546+Delta = collections.namedtuple('Delta', ['previous', 'current'])
2547+
2548+
2549+_KV = None
2550+
2551+
2552+def kv():
2553+ global _KV
2554+ if _KV is None:
2555+ _KV = Storage()
2556+ return _KV
2557
2558=== added directory 'hooks/charmhelpers/fetch'
2559=== added file 'hooks/charmhelpers/fetch/__init__.py'
2560--- hooks/charmhelpers/fetch/__init__.py 1970-01-01 00:00:00 +0000
2561+++ hooks/charmhelpers/fetch/__init__.py 2015-02-23 19:08:44 +0000
2562@@ -0,0 +1,439 @@
2563+# Copyright 2014-2015 Canonical Limited.
2564+#
2565+# This file is part of charm-helpers.
2566+#
2567+# charm-helpers is free software: you can redistribute it and/or modify
2568+# it under the terms of the GNU Lesser General Public License version 3 as
2569+# published by the Free Software Foundation.
2570+#
2571+# charm-helpers is distributed in the hope that it will be useful,
2572+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2573+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2574+# GNU Lesser General Public License for more details.
2575+#
2576+# You should have received a copy of the GNU Lesser General Public License
2577+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2578+
2579+import importlib
2580+from tempfile import NamedTemporaryFile
2581+import time
2582+from yaml import safe_load
2583+from charmhelpers.core.host import (
2584+ lsb_release
2585+)
2586+import subprocess
2587+from charmhelpers.core.hookenv import (
2588+ config,
2589+ log,
2590+)
2591+import os
2592+
2593+import six
2594+if six.PY3:
2595+ from urllib.parse import urlparse, urlunparse
2596+else:
2597+ from urlparse import urlparse, urlunparse
2598+
2599+
2600+CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
2601+deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
2602+"""
2603+PROPOSED_POCKET = """# Proposed
2604+deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
2605+"""
2606+CLOUD_ARCHIVE_POCKETS = {
2607+ # Folsom
2608+ 'folsom': 'precise-updates/folsom',
2609+ 'precise-folsom': 'precise-updates/folsom',
2610+ 'precise-folsom/updates': 'precise-updates/folsom',
2611+ 'precise-updates/folsom': 'precise-updates/folsom',
2612+ 'folsom/proposed': 'precise-proposed/folsom',
2613+ 'precise-folsom/proposed': 'precise-proposed/folsom',
2614+ 'precise-proposed/folsom': 'precise-proposed/folsom',
2615+ # Grizzly
2616+ 'grizzly': 'precise-updates/grizzly',
2617+ 'precise-grizzly': 'precise-updates/grizzly',
2618+ 'precise-grizzly/updates': 'precise-updates/grizzly',
2619+ 'precise-updates/grizzly': 'precise-updates/grizzly',
2620+ 'grizzly/proposed': 'precise-proposed/grizzly',
2621+ 'precise-grizzly/proposed': 'precise-proposed/grizzly',
2622+ 'precise-proposed/grizzly': 'precise-proposed/grizzly',
2623+ # Havana
2624+ 'havana': 'precise-updates/havana',
2625+ 'precise-havana': 'precise-updates/havana',
2626+ 'precise-havana/updates': 'precise-updates/havana',
2627+ 'precise-updates/havana': 'precise-updates/havana',
2628+ 'havana/proposed': 'precise-proposed/havana',
2629+ 'precise-havana/proposed': 'precise-proposed/havana',
2630+ 'precise-proposed/havana': 'precise-proposed/havana',
2631+ # Icehouse
2632+ 'icehouse': 'precise-updates/icehouse',
2633+ 'precise-icehouse': 'precise-updates/icehouse',
2634+ 'precise-icehouse/updates': 'precise-updates/icehouse',
2635+ 'precise-updates/icehouse': 'precise-updates/icehouse',
2636+ 'icehouse/proposed': 'precise-proposed/icehouse',
2637+ 'precise-icehouse/proposed': 'precise-proposed/icehouse',
2638+ 'precise-proposed/icehouse': 'precise-proposed/icehouse',
2639+ # Juno
2640+ 'juno': 'trusty-updates/juno',
2641+ 'trusty-juno': 'trusty-updates/juno',
2642+ 'trusty-juno/updates': 'trusty-updates/juno',
2643+ 'trusty-updates/juno': 'trusty-updates/juno',
2644+ 'juno/proposed': 'trusty-proposed/juno',
2645+ 'trusty-juno/proposed': 'trusty-proposed/juno',
2646+ 'trusty-proposed/juno': 'trusty-proposed/juno',
2647+ # Kilo
2648+ 'kilo': 'trusty-updates/kilo',
2649+ 'trusty-kilo': 'trusty-updates/kilo',
2650+ 'trusty-kilo/updates': 'trusty-updates/kilo',
2651+ 'trusty-updates/kilo': 'trusty-updates/kilo',
2652+ 'kilo/proposed': 'trusty-proposed/kilo',
2653+ 'trusty-kilo/proposed': 'trusty-proposed/kilo',
2654+ 'trusty-proposed/kilo': 'trusty-proposed/kilo',
2655+}
2656+
2657+# The order of this list is very important. Handlers should be listed in from
2658+# least- to most-specific URL matching.
2659+FETCH_HANDLERS = (
2660+ 'charmhelpers.fetch.archiveurl.ArchiveUrlFetchHandler',
2661+ 'charmhelpers.fetch.bzrurl.BzrUrlFetchHandler',
2662+ 'charmhelpers.fetch.giturl.GitUrlFetchHandler',
2663+)
2664+
2665+APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT.
2666+APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks.
2667+APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
2668+
2669+
2670+class SourceConfigError(Exception):
2671+ pass
2672+
2673+
2674+class UnhandledSource(Exception):
2675+ pass
2676+
2677+
2678+class AptLockError(Exception):
2679+ pass
2680+
2681+
2682+class BaseFetchHandler(object):
2683+
2684+ """Base class for FetchHandler implementations in fetch plugins"""
2685+
2686+ def can_handle(self, source):
2687+ """Returns True if the source can be handled. Otherwise returns
2688+ a string explaining why it cannot"""
2689+ return "Wrong source type"
2690+
2691+ def install(self, source):
2692+ """Try to download and unpack the source. Return the path to the
2693+ unpacked files or raise UnhandledSource."""
2694+ raise UnhandledSource("Wrong source type {}".format(source))
2695+
2696+ def parse_url(self, url):
2697+ return urlparse(url)
2698+
2699+ def base_url(self, url):
2700+ """Return url without querystring or fragment"""
2701+ parts = list(self.parse_url(url))
2702+ parts[4:] = ['' for i in parts[4:]]
2703+ return urlunparse(parts)
2704+
2705+
2706+def filter_installed_packages(packages):
2707+ """Returns a list of packages that require installation"""
2708+ cache = apt_cache()
2709+ _pkgs = []
2710+ for package in packages:
2711+ try:
2712+ p = cache[package]
2713+ p.current_ver or _pkgs.append(package)
2714+ except KeyError:
2715+ log('Package {} has no installation candidate.'.format(package),
2716+ level='WARNING')
2717+ _pkgs.append(package)
2718+ return _pkgs
2719+
2720+
2721+def apt_cache(in_memory=True):
2722+ """Build and return an apt cache"""
2723+ import apt_pkg
2724+ apt_pkg.init()
2725+ if in_memory:
2726+ apt_pkg.config.set("Dir::Cache::pkgcache", "")
2727+ apt_pkg.config.set("Dir::Cache::srcpkgcache", "")
2728+ return apt_pkg.Cache()
2729+
2730+
2731+def apt_install(packages, options=None, fatal=False):
2732+ """Install one or more packages"""
2733+ if options is None:
2734+ options = ['--option=Dpkg::Options::=--force-confold']
2735+
2736+ cmd = ['apt-get', '--assume-yes']
2737+ cmd.extend(options)
2738+ cmd.append('install')
2739+ if isinstance(packages, six.string_types):
2740+ cmd.append(packages)
2741+ else:
2742+ cmd.extend(packages)
2743+ log("Installing {} with options: {}".format(packages,
2744+ options))
2745+ _run_apt_command(cmd, fatal)
2746+
2747+
2748+def apt_upgrade(options=None, fatal=False, dist=False):
2749+ """Upgrade all packages"""
2750+ if options is None:
2751+ options = ['--option=Dpkg::Options::=--force-confold']
2752+
2753+ cmd = ['apt-get', '--assume-yes']
2754+ cmd.extend(options)
2755+ if dist:
2756+ cmd.append('dist-upgrade')
2757+ else:
2758+ cmd.append('upgrade')
2759+ log("Upgrading with options: {}".format(options))
2760+ _run_apt_command(cmd, fatal)
2761+
2762+
2763+def apt_update(fatal=False):
2764+ """Update local apt cache"""
2765+ cmd = ['apt-get', 'update']
2766+ _run_apt_command(cmd, fatal)
2767+
2768+
2769+def apt_purge(packages, fatal=False):
2770+ """Purge one or more packages"""
2771+ cmd = ['apt-get', '--assume-yes', 'purge']
2772+ if isinstance(packages, six.string_types):
2773+ cmd.append(packages)
2774+ else:
2775+ cmd.extend(packages)
2776+ log("Purging {}".format(packages))
2777+ _run_apt_command(cmd, fatal)
2778+
2779+
2780+def apt_hold(packages, fatal=False):
2781+ """Hold one or more packages"""
2782+ cmd = ['apt-mark', 'hold']
2783+ if isinstance(packages, six.string_types):
2784+ cmd.append(packages)
2785+ else:
2786+ cmd.extend(packages)
2787+ log("Holding {}".format(packages))
2788+
2789+ if fatal:
2790+ subprocess.check_call(cmd)
2791+ else:
2792+ subprocess.call(cmd)
2793+
2794+
2795+def add_source(source, key=None):
2796+ """Add a package source to this system.
2797+
2798+ @param source: a URL or sources.list entry, as supported by
2799+ add-apt-repository(1). Examples::
2800+
2801+ ppa:charmers/example
2802+ deb https://stub:key@private.example.com/ubuntu trusty main
2803+
2804+ In addition:
2805+ 'proposed:' may be used to enable the standard 'proposed'
2806+ pocket for the release.
2807+ 'cloud:' may be used to activate official cloud archive pockets,
2808+ such as 'cloud:icehouse'
2809+ 'distro' may be used as a noop
2810+
2811+ @param key: A key to be added to the system's APT keyring and used
2812+ to verify the signatures on packages. Ideally, this should be an
2813+ ASCII format GPG public key including the block headers. A GPG key
2814+ id may also be used, but be aware that only insecure protocols are
2815+ available to retrieve the actual public key from a public keyserver
2816+ placing your Juju environment at risk. ppa and cloud archive keys
2817+ are securely added automtically, so sould not be provided.
2818+ """
2819+ if source is None:
2820+ log('Source is not present. Skipping')
2821+ return
2822+
2823+ if (source.startswith('ppa:') or
2824+ source.startswith('http') or
2825+ source.startswith('deb ') or
2826+ source.startswith('cloud-archive:')):
2827+ subprocess.check_call(['add-apt-repository', '--yes', source])
2828+ elif source.startswith('cloud:'):
2829+ apt_install(filter_installed_packages(['ubuntu-cloud-keyring']),
2830+ fatal=True)
2831+ pocket = source.split(':')[-1]
2832+ if pocket not in CLOUD_ARCHIVE_POCKETS:
2833+ raise SourceConfigError(
2834+ 'Unsupported cloud: source option %s' %
2835+ pocket)
2836+ actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket]
2837+ with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
2838+ apt.write(CLOUD_ARCHIVE.format(actual_pocket))
2839+ elif source == 'proposed':
2840+ release = lsb_release()['DISTRIB_CODENAME']
2841+ with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
2842+ apt.write(PROPOSED_POCKET.format(release))
2843+ elif source == 'distro':
2844+ pass
2845+ else:
2846+ log("Unknown source: {!r}".format(source))
2847+
2848+ if key:
2849+ if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
2850+ with NamedTemporaryFile('w+') as key_file:
2851+ key_file.write(key)
2852+ key_file.flush()
2853+ key_file.seek(0)
2854+ subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file)
2855+ else:
2856+ # Note that hkp: is in no way a secure protocol. Using a
2857+ # GPG key id is pointless from a security POV unless you
2858+ # absolutely trust your network and DNS.
2859+ subprocess.check_call(['apt-key', 'adv', '--keyserver',
2860+ 'hkp://keyserver.ubuntu.com:80', '--recv',
2861+ key])
2862+
2863+
2864+def configure_sources(update=False,
2865+ sources_var='install_sources',
2866+ keys_var='install_keys'):
2867+ """
2868+ Configure multiple sources from charm configuration.
2869+
2870+ The lists are encoded as yaml fragments in the configuration.
2871+ The frament needs to be included as a string. Sources and their
2872+ corresponding keys are of the types supported by add_source().
2873+
2874+ Example config:
2875+ install_sources: |
2876+ - "ppa:foo"
2877+ - "http://example.com/repo precise main"
2878+ install_keys: |
2879+ - null
2880+ - "a1b2c3d4"
2881+
2882+ Note that 'null' (a.k.a. None) should not be quoted.
2883+ """
2884+ sources = safe_load((config(sources_var) or '').strip()) or []
2885+ keys = safe_load((config(keys_var) or '').strip()) or None
2886+
2887+ if isinstance(sources, six.string_types):
2888+ sources = [sources]
2889+
2890+ if keys is None:
2891+ for source in sources:
2892+ add_source(source, None)
2893+ else:
2894+ if isinstance(keys, six.string_types):
2895+ keys = [keys]
2896+
2897+ if len(sources) != len(keys):
2898+ raise SourceConfigError(
2899+ 'Install sources and keys lists are different lengths')
2900+ for source, key in zip(sources, keys):
2901+ add_source(source, key)
2902+ if update:
2903+ apt_update(fatal=True)
2904+
2905+
2906+def install_remote(source, *args, **kwargs):
2907+ """
2908+ Install a file tree from a remote source
2909+
2910+ The specified source should be a url of the form:
2911+ scheme://[host]/path[#[option=value][&...]]
2912+
2913+ Schemes supported are based on this modules submodules.
2914+ Options supported are submodule-specific.
2915+ Additional arguments are passed through to the submodule.
2916+
2917+ For example::
2918+
2919+ dest = install_remote('http://example.com/archive.tgz',
2920+ checksum='deadbeef',
2921+ hash_type='sha1')
2922+
2923+ This will download `archive.tgz`, validate it using SHA1 and, if
2924+ the file is ok, extract it and return the directory in which it
2925+ was extracted. If the checksum fails, it will raise
2926+ :class:`charmhelpers.core.host.ChecksumError`.
2927+ """
2928+ # We ONLY check for True here because can_handle may return a string
2929+ # explaining why it can't handle a given source.
2930+ handlers = [h for h in plugins() if h.can_handle(source) is True]
2931+ installed_to = None
2932+ for handler in handlers:
2933+ try:
2934+ installed_to = handler.install(source, *args, **kwargs)
2935+ except UnhandledSource:
2936+ pass
2937+ if not installed_to:
2938+ raise UnhandledSource("No handler found for source {}".format(source))
2939+ return installed_to
2940+
2941+
2942+def install_from_config(config_var_name):
2943+ charm_config = config()
2944+ source = charm_config[config_var_name]
2945+ return install_remote(source)
2946+
2947+
2948+def plugins(fetch_handlers=None):
2949+ if not fetch_handlers:
2950+ fetch_handlers = FETCH_HANDLERS
2951+ plugin_list = []
2952+ for handler_name in fetch_handlers:
2953+ package, classname = handler_name.rsplit('.', 1)
2954+ try:
2955+ handler_class = getattr(
2956+ importlib.import_module(package),
2957+ classname)
2958+ plugin_list.append(handler_class())
2959+ except (ImportError, AttributeError):
2960+ # Skip missing plugins so that they can be ommitted from
2961+ # installation if desired
2962+ log("FetchHandler {} not found, skipping plugin".format(
2963+ handler_name))
2964+ return plugin_list
2965+
2966+
2967+def _run_apt_command(cmd, fatal=False):
2968+ """
2969+ Run an APT command, checking output and retrying if the fatal flag is set
2970+ to True.
2971+
2972+ :param: cmd: str: The apt command to run.
2973+ :param: fatal: bool: Whether the command's output should be checked and
2974+ retried.
2975+ """
2976+ env = os.environ.copy()
2977+
2978+ if 'DEBIAN_FRONTEND' not in env:
2979+ env['DEBIAN_FRONTEND'] = 'noninteractive'
2980+
2981+ if fatal:
2982+ retry_count = 0
2983+ result = None
2984+
2985+ # If the command is considered "fatal", we need to retry if the apt
2986+ # lock was not acquired.
2987+
2988+ while result is None or result == APT_NO_LOCK:
2989+ try:
2990+ result = subprocess.check_call(cmd, env=env)
2991+ except subprocess.CalledProcessError as e:
2992+ retry_count = retry_count + 1
2993+ if retry_count > APT_NO_LOCK_RETRY_COUNT:
2994+ raise
2995+ result = e.returncode
2996+ log("Couldn't acquire DPKG lock. Will retry in {} seconds."
2997+ "".format(APT_NO_LOCK_RETRY_DELAY))
2998+ time.sleep(APT_NO_LOCK_RETRY_DELAY)
2999+
3000+ else:
3001+ subprocess.call(cmd, env=env)
3002
3003=== added file 'hooks/charmhelpers/fetch/archiveurl.py'
3004--- hooks/charmhelpers/fetch/archiveurl.py 1970-01-01 00:00:00 +0000
3005+++ hooks/charmhelpers/fetch/archiveurl.py 2015-02-23 19:08:44 +0000
3006@@ -0,0 +1,161 @@
3007+# Copyright 2014-2015 Canonical Limited.
3008+#
3009+# This file is part of charm-helpers.
3010+#
3011+# charm-helpers is free software: you can redistribute it and/or modify
3012+# it under the terms of the GNU Lesser General Public License version 3 as
3013+# published by the Free Software Foundation.
3014+#
3015+# charm-helpers is distributed in the hope that it will be useful,
3016+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3017+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3018+# GNU Lesser General Public License for more details.
3019+#
3020+# You should have received a copy of the GNU Lesser General Public License
3021+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3022+
3023+import os
3024+import hashlib
3025+import re
3026+
3027+from charmhelpers.fetch import (
3028+ BaseFetchHandler,
3029+ UnhandledSource
3030+)
3031+from charmhelpers.payload.archive import (
3032+ get_archive_handler,
3033+ extract,
3034+)
3035+from charmhelpers.core.host import mkdir, check_hash
3036+
3037+import six
3038+if six.PY3:
3039+ from urllib.request import (
3040+ build_opener, install_opener, urlopen, urlretrieve,
3041+ HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
3042+ )
3043+ from urllib.parse import urlparse, urlunparse, parse_qs
3044+ from urllib.error import URLError
3045+else:
3046+ from urllib import urlretrieve
3047+ from urllib2 import (
3048+ build_opener, install_opener, urlopen,
3049+ HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
3050+ URLError
3051+ )
3052+ from urlparse import urlparse, urlunparse, parse_qs
3053+
3054+
3055+def splituser(host):
3056+ '''urllib.splituser(), but six's support of this seems broken'''
3057+ _userprog = re.compile('^(.*)@(.*)$')
3058+ match = _userprog.match(host)
3059+ if match:
3060+ return match.group(1, 2)
3061+ return None, host
3062+
3063+
3064+def splitpasswd(user):
3065+ '''urllib.splitpasswd(), but six's support of this is missing'''
3066+ _passwdprog = re.compile('^([^:]*):(.*)$', re.S)
3067+ match = _passwdprog.match(user)
3068+ if match:
3069+ return match.group(1, 2)
3070+ return user, None
3071+
3072+
3073+class ArchiveUrlFetchHandler(BaseFetchHandler):
3074+ """
3075+ Handler to download archive files from arbitrary URLs.
3076+
3077+ Can fetch from http, https, ftp, and file URLs.
3078+
3079+ Can install either tarballs (.tar, .tgz, .tbz2, etc) or zip files.
3080+
3081+ Installs the contents of the archive in $CHARM_DIR/fetched/.
3082+ """
3083+ def can_handle(self, source):
3084+ url_parts = self.parse_url(source)
3085+ if url_parts.scheme not in ('http', 'https', 'ftp', 'file'):
3086+ return "Wrong source type"
3087+ if get_archive_handler(self.base_url(source)):
3088+ return True
3089+ return False
3090+
3091+ def download(self, source, dest):
3092+ """
3093+ Download an archive file.
3094+
3095+ :param str source: URL pointing to an archive file.
3096+ :param str dest: Local path location to download archive file to.
3097+ """
3098+ # propogate all exceptions
3099+ # URLError, OSError, etc
3100+ proto, netloc, path, params, query, fragment = urlparse(source)
3101+ if proto in ('http', 'https'):
3102+ auth, barehost = splituser(netloc)
3103+ if auth is not None:
3104+ source = urlunparse((proto, barehost, path, params, query, fragment))
3105+ username, password = splitpasswd(auth)
3106+ passman = HTTPPasswordMgrWithDefaultRealm()
3107+ # Realm is set to None in add_password to force the username and password
3108+ # to be used whatever the realm
3109+ passman.add_password(None, source, username, password)
3110+ authhandler = HTTPBasicAuthHandler(passman)
3111+ opener = build_opener(authhandler)
3112+ install_opener(opener)
3113+ response = urlopen(source)
3114+ try:
3115+ with open(dest, 'w') as dest_file:
3116+ dest_file.write(response.read())
3117+ except Exception as e:
3118+ if os.path.isfile(dest):
3119+ os.unlink(dest)
3120+ raise e
3121+
3122+ # Mandatory file validation via Sha1 or MD5 hashing.
3123+ def download_and_validate(self, url, hashsum, validate="sha1"):
3124+ tempfile, headers = urlretrieve(url)
3125+ check_hash(tempfile, hashsum, validate)
3126+ return tempfile
3127+
3128+ def install(self, source, dest=None, checksum=None, hash_type='sha1'):
3129+ """
3130+ Download and install an archive file, with optional checksum validation.
3131+
3132+ The checksum can also be given on the `source` URL's fragment.
3133+ For example::
3134+
3135+ handler.install('http://example.com/file.tgz#sha1=deadbeef')
3136+
3137+ :param str source: URL pointing to an archive file.
3138+ :param str dest: Local destination path to install to. If not given,
3139+ installs to `$CHARM_DIR/archives/archive_file_name`.
3140+ :param str checksum: If given, validate the archive file after download.
3141+ :param str hash_type: Algorithm used to generate `checksum`.
3142+ Can be any hash alrgorithm supported by :mod:`hashlib`,
3143+ such as md5, sha1, sha256, sha512, etc.
3144+
3145+ """
3146+ url_parts = self.parse_url(source)
3147+ dest_dir = os.path.join(os.environ.get('CHARM_DIR'), 'fetched')
3148+ if not os.path.exists(dest_dir):
3149+ mkdir(dest_dir, perms=0o755)
3150+ dld_file = os.path.join(dest_dir, os.path.basename(url_parts.path))
3151+ try:
3152+ self.download(source, dld_file)
3153+ except URLError as e:
3154+ raise UnhandledSource(e.reason)
3155+ except OSError as e:
3156+ raise UnhandledSource(e.strerror)
3157+ options = parse_qs(url_parts.fragment)
3158+ for key, value in options.items():
3159+ if not six.PY3:
3160+ algorithms = hashlib.algorithms
3161+ else:
3162+ algorithms = hashlib.algorithms_available
3163+ if key in algorithms:
3164+ check_hash(dld_file, value, key)
3165+ if checksum:
3166+ check_hash(dld_file, checksum, hash_type)
3167+ return extract(dld_file, dest)
3168
3169=== added file 'hooks/charmhelpers/fetch/bzrurl.py'
3170--- hooks/charmhelpers/fetch/bzrurl.py 1970-01-01 00:00:00 +0000
3171+++ hooks/charmhelpers/fetch/bzrurl.py 2015-02-23 19:08:44 +0000
3172@@ -0,0 +1,78 @@
3173+# Copyright 2014-2015 Canonical Limited.
3174+#
3175+# This file is part of charm-helpers.
3176+#
3177+# charm-helpers is free software: you can redistribute it and/or modify
3178+# it under the terms of the GNU Lesser General Public License version 3 as
3179+# published by the Free Software Foundation.
3180+#
3181+# charm-helpers is distributed in the hope that it will be useful,
3182+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3183+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3184+# GNU Lesser General Public License for more details.
3185+#
3186+# You should have received a copy of the GNU Lesser General Public License
3187+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3188+
3189+import os
3190+from charmhelpers.fetch import (
3191+ BaseFetchHandler,
3192+ UnhandledSource
3193+)
3194+from charmhelpers.core.host import mkdir
3195+
3196+import six
3197+if six.PY3:
3198+ raise ImportError('bzrlib does not support Python3')
3199+
3200+try:
3201+ from bzrlib.branch import Branch
3202+ from bzrlib import bzrdir, workingtree, errors
3203+except ImportError:
3204+ from charmhelpers.fetch import apt_install
3205+ apt_install("python-bzrlib")
3206+ from bzrlib.branch import Branch
3207+ from bzrlib import bzrdir, workingtree, errors
3208+
3209+
3210+class BzrUrlFetchHandler(BaseFetchHandler):
3211+ """Handler for bazaar branches via generic and lp URLs"""
3212+ def can_handle(self, source):
3213+ url_parts = self.parse_url(source)
3214+ if url_parts.scheme not in ('bzr+ssh', 'lp'):
3215+ return False
3216+ else:
3217+ return True
3218+
3219+ def branch(self, source, dest):
3220+ url_parts = self.parse_url(source)
3221+ # If we use lp:branchname scheme we need to load plugins
3222+ if not self.can_handle(source):
3223+ raise UnhandledSource("Cannot handle {}".format(source))
3224+ if url_parts.scheme == "lp":
3225+ from bzrlib.plugin import load_plugins
3226+ load_plugins()
3227+ try:
3228+ local_branch = bzrdir.BzrDir.create_branch_convenience(dest)
3229+ except errors.AlreadyControlDirError:
3230+ local_branch = Branch.open(dest)
3231+ try:
3232+ remote_branch = Branch.open(source)
3233+ remote_branch.push(local_branch)
3234+ tree = workingtree.WorkingTree.open(dest)
3235+ tree.update()
3236+ except Exception as e:
3237+ raise e
3238+
3239+ def install(self, source):
3240+ url_parts = self.parse_url(source)
3241+ branch_name = url_parts.path.strip("/").split("/")[-1]
3242+ dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
3243+ branch_name)
3244+ if not os.path.exists(dest_dir):
3245+ mkdir(dest_dir, perms=0o755)
3246+ try:
3247+ self.branch(source, dest_dir)
3248+ except OSError as e:
3249+ raise UnhandledSource(e.strerror)
3250+ return dest_dir
3251
3252=== added file 'hooks/charmhelpers/fetch/giturl.py'
3253--- hooks/charmhelpers/fetch/giturl.py 1970-01-01 00:00:00 +0000
3254+++ hooks/charmhelpers/fetch/giturl.py 2015-02-23 19:08:44 +0000
3255@@ -0,0 +1,71 @@
3256+# Copyright 2014-2015 Canonical Limited.
3257+#
3258+# This file is part of charm-helpers.
3259+#
3260+# charm-helpers is free software: you can redistribute it and/or modify
3261+# it under the terms of the GNU Lesser General Public License version 3 as
3262+# published by the Free Software Foundation.
3263+#
3264+# charm-helpers is distributed in the hope that it will be useful,
3265+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3266+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3267+# GNU Lesser General Public License for more details.
3268+#
3269+# You should have received a copy of the GNU Lesser General Public License
3270+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3271+
3272+import os
3273+from charmhelpers.fetch import (
3274+ BaseFetchHandler,
3275+ UnhandledSource
3276+)
3277+from charmhelpers.core.host import mkdir
3278+
3279+import six
3280+if six.PY3:
3281+ raise ImportError('GitPython does not support Python 3')
3282+
3283+try:
3284+ from git import Repo
3285+except ImportError:
3286+ from charmhelpers.fetch import apt_install
3287+ apt_install("python-git")
3288+ from git import Repo
3289+
3290+from git.exc import GitCommandError # noqa E402
3291+
3292+
3293+class GitUrlFetchHandler(BaseFetchHandler):
3294+ """Handler for git branches via generic and github URLs"""
3295+ def can_handle(self, source):
3296+ url_parts = self.parse_url(source)
3297+ # TODO (mattyw) no support for ssh git@ yet
3298+ if url_parts.scheme not in ('http', 'https', 'git'):
3299+ return False
3300+ else:
3301+ return True
3302+
3303+ def clone(self, source, dest, branch):
3304+ if not self.can_handle(source):
3305+ raise UnhandledSource("Cannot handle {}".format(source))
3306+
3307+ repo = Repo.clone_from(source, dest)
3308+ repo.git.checkout(branch)
3309+
3310+ def install(self, source, branch="master", dest=None):
3311+ url_parts = self.parse_url(source)
3312+ branch_name = url_parts.path.strip("/").split("/")[-1]
3313+ if dest:
3314+ dest_dir = os.path.join(dest, branch_name)
3315+ else:
3316+ dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
3317+ branch_name)
3318+ if not os.path.exists(dest_dir):
3319+ mkdir(dest_dir, perms=0o755)
3320+ try:
3321+ self.clone(source, dest_dir, branch)
3322+ except GitCommandError as e:
3323+ raise UnhandledSource(e.message)
3324+ except OSError as e:
3325+ raise UnhandledSource(e.strerror)
3326+ return dest_dir
3327
3328=== added file 'hooks/config-changed'
3329--- hooks/config-changed 1970-01-01 00:00:00 +0000
3330+++ hooks/config-changed 2015-02-23 19:08:44 +0000
3331@@ -0,0 +1,43 @@
3332+#!/usr/bin/python
3333+
3334+#
3335+# Copyright 2015 Canonical Ltd.
3336+#
3337+# Authors:
3338+# Liang Chen <liang.chen@ubuntu.com>
3339+#
3340+import subprocess
3341+import sys
3342+
3343+from charmhelpers.core.hookenv import (
3344+ config,
3345+ Hooks, UnregisteredHookError,
3346+ log, ERROR,
3347+)
3348+
3349+from utils import render_template
3350+
3351+hooks = Hooks()
3352+
3353+
3354+def emit_lxc_br_conf():
3355+ lxc_context = {}
3356+ if config('new-lxc-network'):
3357+ lxc_context['new_network'] = True
3358+
3359+ lxc_bridge_conf = "/etc/network/interfaces.d/lxcbr0.cfg"
3360+ with open(lxc_bridge_conf, 'w') as lxc_br_conf:
3361+ lxc_br_conf.write(render_template('lxc-bridge.conf', lxc_context))
3362+
3363+
3364+@hooks.hook('config-changed')
3365+def config_changed():
3366+ emit_lxc_br_conf()
3367+ cmd = ['ifup', 'lxcbr0']
3368+ subprocess.check_call(cmd)
3369+
3370+if __name__ == '__main__':
3371+ try:
3372+ hooks.execute(sys.argv)
3373+ except UnregisteredHookError as e:
3374+ log('Unknown hook {} - skipping.'.format(e))
3375
3376=== added file 'hooks/hooks.py'
3377--- hooks/hooks.py 1970-01-01 00:00:00 +0000
3378+++ hooks/hooks.py 2015-02-23 19:08:44 +0000
3379@@ -0,0 +1,45 @@
3380+#!/usr/bin/python
3381+
3382+#
3383+# Copyright 2015 Canonical Ltd.
3384+#
3385+# Authors:
3386+# Liang Chen <liang.chen@ubuntu.com>
3387+#
3388+
3389+import subprocess
3390+import sys
3391+
3392+from charmhelpers.core.hookenv import (
3393+ config,
3394+ Hooks, UnregisteredHookError,
3395+ log,
3396+)
3397+
3398+from utils import render_template
3399+
3400+hooks = Hooks()
3401+
3402+
3403+def emit_lxc_br_conf():
3404+ lxc_context = {}
3405+ if config('new-lxc-network'):
3406+ lxc_context['new_network'] = True
3407+
3408+ lxc_bridge_conf = "/etc/network/interfaces.d/lxcbr0.cfg"
3409+ with open(lxc_bridge_conf, 'w') as lxc_br_conf:
3410+ lxc_br_conf.write(render_template('lxc-bridge.conf', lxc_context))
3411+
3412+
3413+@hooks.hook('config-changed')
3414+def config_changed():
3415+ emit_lxc_br_conf()
3416+ cmd = ['ifup', 'lxcbr0']
3417+ subprocess.check_call(cmd)
3418+
3419+
3420+if __name__ == '__main__':
3421+ try:
3422+ hooks.execute(sys.argv)
3423+ except UnregisteredHookError as e:
3424+ log('Unknown hook {} - skipping.'.format(e))
3425
3426=== added file 'hooks/utils.py'
3427--- hooks/utils.py 1970-01-01 00:00:00 +0000
3428+++ hooks/utils.py 2015-02-23 19:08:44 +0000
3429@@ -0,0 +1,27 @@
3430+#
3431+# Copyright 2015 Canonical Ltd.
3432+#
3433+# Authors:
3434+# Liang Chen <liang.chen@ubuntu.com>
3435+#
3436+
3437+TEMPLATES_DIR = 'templates'
3438+
3439+from charmhelpers.fetch import (
3440+ apt_install,
3441+ filter_installed_packages
3442+)
3443+
3444+try:
3445+ import jinja2
3446+except ImportError:
3447+ apt_install(filter_installed_packages(['python-jinja2']),
3448+ fatal=True)
3449+ import jinja2
3450+
3451+
3452+def render_template(template_name, context, template_dir=TEMPLATES_DIR):
3453+ templates = jinja2.Environment(
3454+ loader=jinja2.FileSystemLoader(template_dir))
3455+ template = templates.get_template(template_name)
3456+ return template.render(context)
3457
3458=== added directory 'templates'
3459=== added file 'templates/lxc-bridge.conf'
3460--- templates/lxc-bridge.conf 1970-01-01 00:00:00 +0000
3461+++ templates/lxc-bridge.conf 2015-02-23 19:08:44 +0000
3462@@ -0,0 +1,10 @@
3463+
3464+{% if new_network -%}
3465+auto eth0
3466+iface eth0 inet manual
3467+
3468+auto lxcbr0
3469+iface lxcbr0 inet dhcp
3470+ bridge_ports eth0
3471+
3472+{% endif -%}

Subscribers

People subscribed via source and target branches

to all changes: