Merge lp:~1chb1n/charms/trusty/ubuntu/simplify-and-test into lp:charms/trusty/ubuntu

Proposed by Ryan Beisner on 2015-04-29
Status: Merged
Merged at revision: 11
Proposed branch: lp:~1chb1n/charms/trusty/ubuntu/simplify-and-test
Merge into: lp:charms/trusty/ubuntu
Diff against target: 4371 lines (+655/-3480)
41 files modified
Makefile (+57/-0)
README.md (+5/-21)
charm-helpers-hooks.yaml (+0/-5)
charm-helpers-tests.yaml (+4/-0)
config.yaml (+0/-8)
files/bundle-example.yaml (+14/-0)
hooks/charmhelpers/core/__init__.py (+0/-15)
hooks/charmhelpers/core/decorators.py (+0/-57)
hooks/charmhelpers/core/fstab.py (+0/-134)
hooks/charmhelpers/core/hookenv.py (+0/-568)
hooks/charmhelpers/core/host.py (+0/-446)
hooks/charmhelpers/core/services/__init__.py (+0/-18)
hooks/charmhelpers/core/services/base.py (+0/-329)
hooks/charmhelpers/core/services/helpers.py (+0/-259)
hooks/charmhelpers/core/strutils.py (+0/-42)
hooks/charmhelpers/core/sysctl.py (+0/-56)
hooks/charmhelpers/core/templating.py (+0/-68)
hooks/charmhelpers/core/unitdata.py (+0/-477)
hooks/charmhelpers/fetch/__init__.py (+0/-439)
hooks/charmhelpers/fetch/archiveurl.py (+0/-161)
hooks/charmhelpers/fetch/bzrurl.py (+0/-78)
hooks/charmhelpers/fetch/giturl.py (+0/-71)
hooks/config-changed (+0/-43)
hooks/hooks.py (+0/-45)
hooks/install (+0/-2)
hooks/utils.py (+0/-27)
metadata.yaml (+1/-1)
templates/lxc-bridge.conf (+0/-10)
tests/00-setup.sh (+4/-10)
tests/010_basic_precise (+7/-0)
tests/015_basic_trusty (+7/-0)
tests/020_basic_utopic (+7/-0)
tests/025_basic_vivid (+7/-0)
tests/10-deploy-test.py (+0/-90)
tests/basic_deployment.py (+49/-0)
tests/charmhelpers/__init__.py (+38/-0)
tests/charmhelpers/contrib/__init__.py (+15/-0)
tests/charmhelpers/contrib/amulet/__init__.py (+15/-0)
tests/charmhelpers/contrib/amulet/deployment.py (+93/-0)
tests/charmhelpers/contrib/amulet/utils.py (+323/-0)
tests/tests.yaml (+9/-0)
To merge this branch: bzr merge lp:~1chb1n/charms/trusty/ubuntu/simplify-and-test
Reviewer Review Type Date Requested Status
Tim Van Steenburgh 2015-04-29 Approve on 2015-05-01
Review via email: mp+257773@code.launchpad.net

Commit message

Revert charm to have no hooks and no config options; add functional tests for all currently-supported Ubuntu releases; add bundletester usage examples.

Charm is now compatible with both 'juju test' and bundletester.

Description of the change

Revert charm to have no hooks and no config options; add functional tests for all currently-supported Ubuntu releases; add bundletester usage examples.

Charm is now compatible with both 'juju test' and bundletester.

To post a comment you must log in.
Ryan Beisner (1chb1n) wrote :

Tested via bundletester / amulet on Precise, Trusty, Utopic, Vivid. Ready for review.

Results: http://paste.ubuntu.com/10952530/

Marco Ceppi (marcoceppi) wrote :

See below

Ryan Beisner (1chb1n) wrote :

Thank you for the review. The bundle file is an example which won't be exercised in automation unless explicitly called. It accompanies the bundletester usage examples in the Makefile, which a user or charmer can optionally fire.

The juju-gui is present as a general use example, and as a handy way to fire up the most basic juju canvas. I think the Ubuntu charm is a good place for that, as well as these other usage examples which encourage easily adding functional test coverage. ;-)

To clarify, this MP aims to address 3 things:

1) Return the charm to a functional state on all supported releases.

2) Add test coverage for all supported releases, detect future breakage re: bug 1447765.

3) Add bundletester usage examples since we don't have that in one place anywhere else.

Tim Van Steenburgh (tvansteenburgh) wrote :

+1 LGTM, thanks Ryan.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'Makefile'
2--- Makefile 1970-01-01 00:00:00 +0000
3+++ Makefile 2015-04-30 13:14:23 +0000
4@@ -0,0 +1,57 @@
5+#!/usr/bin/make
6+PYTHON := /usr/bin/env python
7+
8+virtualenv:
9+ @echo Setting up python virtual env...
10+ virtualenv .venv
11+ .venv/bin/pip install flake8 charm-tools bundletester
12+
13+lint: virtualenv
14+ @echo Lint inspections and charm proof...
15+ .venv/bin/flake8 --exclude hooks/charmhelpers tests
16+ .venv/bin/charm proof
17+
18+test: virtualenv
19+ @echo No unit tests defined. This is an example placeholder.
20+
21+functional_test: virtualenv
22+ # Consume ./tests/tests.yaml to determine the tests to run,
23+ # in addition to functional tests in the ./tests dir.
24+ @echo Starting functional, lint and unit tests...
25+ .venv/bin/bundletester -v -F -l DEBUG -r dot -o results-all.dot
26+
27+test_example: virtualenv
28+ # A bundletester usage example to run only the matching tests.
29+ @echo Starting a subset of tests...
30+ .venv/bin/bundletester -v -F -l DEBUG -r json -o results-ex.json \
31+ --test-pattern 02*
32+
33+test_example2: virtualenv
34+ # A bundletester usage example to run only the specified tests,
35+ # with a different output format.
36+ @echo Starting a subset of tests...
37+ .venv/bin/bundletester -v -F -l DEBUG -r spec -o results-ex2.spec \
38+ 010_basic_precise 015_basic_trusty
39+
40+test_example3: virtualenv
41+ # A bundletester bundle usage example.
42+ @echo Starting a subset of tests...
43+ .venv/bin/bundletester -v -F -l DEBUG -r dot -o results-ex3.dot \
44+ -b files/bundle-example.yaml
45+
46+bin/charm_helpers_sync.py:
47+ @mkdir -p bin
48+ @bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \
49+ > bin/charm_helpers_sync.py
50+
51+sync: bin/charm_helpers_sync.py
52+ @echo Syncing charm helpers for functional tests...
53+ @$(PYTHON) bin/charm_helpers_sync.py -c charm-helpers-tests.yaml
54+
55+publish: clean lint
56+ bzr push lp:charms/trusty/ubuntu
57+
58+clean:
59+ @ echo Cleaning up venvs and pyc files...
60+ rm -rf .venv
61+ find -name *.pyc -delete
62
63=== modified file 'README.md'
64--- README.md 2015-03-25 19:59:09 +0000
65+++ README.md 2015-04-30 13:14:23 +0000
66@@ -25,32 +25,16 @@
67 juju deploy ubuntu mytestmachine1
68 juju deploy ubuntu mytestmachine2
69
70-and so on.
71-
72-# Configuration
73-
74-A configuration option to create a new network for LXC containers is provided via a boolean config option, 'new-lxc-network.'
75-The default is set to false indicating the same network of the node will also be used for LXC containers.
76-
77-To use a new network for LXC containers post juju deploy, issue:
78-
79- juju set ubuntu new-lxc-network=true
80-
81-If you want to have a new network for LXC containers on deploy set up a lxc-config.yaml:
82-
83- ubuntu:
84- new-lxc-network: true
85-
86-and pass that as an arugment to deploy:
87-
88- juju deploy --config lxc-config.yaml ubuntu
89-
90-For further information on setting configuration with Juju see https://jujucharms.com/docs/latest/charms-config
91+and so on.
92
93 ## Known Limitations and Issues
94
95 This charm does not provide anything other than a blank server, so it does not relate to other charms.
96
97+# Configuration
98+
99+This charm has no configuration options.
100+
101 # Contact Information
102
103
104
105=== removed file 'charm-helpers-hooks.yaml'
106--- charm-helpers-hooks.yaml 2015-02-19 16:21:31 +0000
107+++ charm-helpers-hooks.yaml 1970-01-01 00:00:00 +0000
108@@ -1,5 +0,0 @@
109-branch: lp:charm-helpers
110-destination: hooks/charmhelpers
111-include:
112- - core
113- - fetch
114
115=== added file 'charm-helpers-tests.yaml'
116--- charm-helpers-tests.yaml 1970-01-01 00:00:00 +0000
117+++ charm-helpers-tests.yaml 2015-04-30 13:14:23 +0000
118@@ -0,0 +1,4 @@
119+branch: lp:charm-helpers
120+destination: tests/charmhelpers
121+include:
122+ - contrib.amulet
123
124=== removed file 'config.yaml'
125--- config.yaml 2015-03-25 19:59:09 +0000
126+++ config.yaml 1970-01-01 00:00:00 +0000
127@@ -1,8 +0,0 @@
128-options:
129- new-lxc-network:
130- type: boolean
131- default: False
132- description: |
133- If True creates new network for lxc containers, otherwise using the
134- same network of the node for lxc containers.
135-
136
137=== added directory 'files'
138=== added file 'files/bundle-example.yaml'
139--- files/bundle-example.yaml 1970-01-01 00:00:00 +0000
140+++ files/bundle-example.yaml 2015-04-30 13:14:23 +0000
141@@ -0,0 +1,14 @@
142+"ubuntu-example-bundle":
143+ services:
144+ "ubuntu":
145+ charm: cs:trusty/ubuntu
146+ num_units: 1
147+ annotations:
148+ "gui-x": "0"
149+ "gui-y": "0"
150+ "juju-gui":
151+ charm: "cs:trusty/juju-gui"
152+ num_units: 1
153+ annotations:
154+ "gui-x": "500"
155+ "gui-y": "0"
156
157=== removed directory 'hooks'
158=== removed file 'hooks/__init__.py'
159=== removed directory 'hooks/charmhelpers'
160=== removed file 'hooks/charmhelpers/__init__.py'
161=== removed directory 'hooks/charmhelpers/core'
162=== removed file 'hooks/charmhelpers/core/__init__.py'
163--- hooks/charmhelpers/core/__init__.py 2015-02-19 16:21:31 +0000
164+++ hooks/charmhelpers/core/__init__.py 1970-01-01 00:00:00 +0000
165@@ -1,15 +0,0 @@
166-# Copyright 2014-2015 Canonical Limited.
167-#
168-# This file is part of charm-helpers.
169-#
170-# charm-helpers is free software: you can redistribute it and/or modify
171-# it under the terms of the GNU Lesser General Public License version 3 as
172-# published by the Free Software Foundation.
173-#
174-# charm-helpers is distributed in the hope that it will be useful,
175-# but WITHOUT ANY WARRANTY; without even the implied warranty of
176-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
177-# GNU Lesser General Public License for more details.
178-#
179-# You should have received a copy of the GNU Lesser General Public License
180-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
181
182=== removed file 'hooks/charmhelpers/core/decorators.py'
183--- hooks/charmhelpers/core/decorators.py 2015-02-19 16:21:31 +0000
184+++ hooks/charmhelpers/core/decorators.py 1970-01-01 00:00:00 +0000
185@@ -1,57 +0,0 @@
186-# Copyright 2014-2015 Canonical Limited.
187-#
188-# This file is part of charm-helpers.
189-#
190-# charm-helpers is free software: you can redistribute it and/or modify
191-# it under the terms of the GNU Lesser General Public License version 3 as
192-# published by the Free Software Foundation.
193-#
194-# charm-helpers is distributed in the hope that it will be useful,
195-# but WITHOUT ANY WARRANTY; without even the implied warranty of
196-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
197-# GNU Lesser General Public License for more details.
198-#
199-# You should have received a copy of the GNU Lesser General Public License
200-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
201-
202-#
203-# Copyright 2014 Canonical Ltd.
204-#
205-# Authors:
206-# Edward Hope-Morley <opentastic@gmail.com>
207-#
208-
209-import time
210-
211-from charmhelpers.core.hookenv import (
212- log,
213- INFO,
214-)
215-
216-
217-def retry_on_exception(num_retries, base_delay=0, exc_type=Exception):
218- """If the decorated function raises exception exc_type, allow num_retries
219- retry attempts before raise the exception.
220- """
221- def _retry_on_exception_inner_1(f):
222- def _retry_on_exception_inner_2(*args, **kwargs):
223- retries = num_retries
224- multiplier = 1
225- while True:
226- try:
227- return f(*args, **kwargs)
228- except exc_type:
229- if not retries:
230- raise
231-
232- delay = base_delay * multiplier
233- multiplier += 1
234- log("Retrying '%s' %d more times (delay=%s)" %
235- (f.__name__, retries, delay), level=INFO)
236- retries -= 1
237- if delay:
238- time.sleep(delay)
239-
240- return _retry_on_exception_inner_2
241-
242- return _retry_on_exception_inner_1
243
244=== removed file 'hooks/charmhelpers/core/fstab.py'
245--- hooks/charmhelpers/core/fstab.py 2015-02-19 16:21:31 +0000
246+++ hooks/charmhelpers/core/fstab.py 1970-01-01 00:00:00 +0000
247@@ -1,134 +0,0 @@
248-#!/usr/bin/env python
249-# -*- coding: utf-8 -*-
250-
251-# Copyright 2014-2015 Canonical Limited.
252-#
253-# This file is part of charm-helpers.
254-#
255-# charm-helpers is free software: you can redistribute it and/or modify
256-# it under the terms of the GNU Lesser General Public License version 3 as
257-# published by the Free Software Foundation.
258-#
259-# charm-helpers is distributed in the hope that it will be useful,
260-# but WITHOUT ANY WARRANTY; without even the implied warranty of
261-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
262-# GNU Lesser General Public License for more details.
263-#
264-# You should have received a copy of the GNU Lesser General Public License
265-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
266-
267-import io
268-import os
269-
270-__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
271-
272-
273-class Fstab(io.FileIO):
274- """This class extends file in order to implement a file reader/writer
275- for file `/etc/fstab`
276- """
277-
278- class Entry(object):
279- """Entry class represents a non-comment line on the `/etc/fstab` file
280- """
281- def __init__(self, device, mountpoint, filesystem,
282- options, d=0, p=0):
283- self.device = device
284- self.mountpoint = mountpoint
285- self.filesystem = filesystem
286-
287- if not options:
288- options = "defaults"
289-
290- self.options = options
291- self.d = int(d)
292- self.p = int(p)
293-
294- def __eq__(self, o):
295- return str(self) == str(o)
296-
297- def __str__(self):
298- return "{} {} {} {} {} {}".format(self.device,
299- self.mountpoint,
300- self.filesystem,
301- self.options,
302- self.d,
303- self.p)
304-
305- DEFAULT_PATH = os.path.join(os.path.sep, 'etc', 'fstab')
306-
307- def __init__(self, path=None):
308- if path:
309- self._path = path
310- else:
311- self._path = self.DEFAULT_PATH
312- super(Fstab, self).__init__(self._path, 'rb+')
313-
314- def _hydrate_entry(self, line):
315- # NOTE: use split with no arguments to split on any
316- # whitespace including tabs
317- return Fstab.Entry(*filter(
318- lambda x: x not in ('', None),
319- line.strip("\n").split()))
320-
321- @property
322- def entries(self):
323- self.seek(0)
324- for line in self.readlines():
325- line = line.decode('us-ascii')
326- try:
327- if line.strip() and not line.strip().startswith("#"):
328- yield self._hydrate_entry(line)
329- except ValueError:
330- pass
331-
332- def get_entry_by_attr(self, attr, value):
333- for entry in self.entries:
334- e_attr = getattr(entry, attr)
335- if e_attr == value:
336- return entry
337- return None
338-
339- def add_entry(self, entry):
340- if self.get_entry_by_attr('device', entry.device):
341- return False
342-
343- self.write((str(entry) + '\n').encode('us-ascii'))
344- self.truncate()
345- return entry
346-
347- def remove_entry(self, entry):
348- self.seek(0)
349-
350- lines = [l.decode('us-ascii') for l in self.readlines()]
351-
352- found = False
353- for index, line in enumerate(lines):
354- if line.strip() and not line.strip().startswith("#"):
355- if self._hydrate_entry(line) == entry:
356- found = True
357- break
358-
359- if not found:
360- return False
361-
362- lines.remove(line)
363-
364- self.seek(0)
365- self.write(''.join(lines).encode('us-ascii'))
366- self.truncate()
367- return True
368-
369- @classmethod
370- def remove_by_mountpoint(cls, mountpoint, path=None):
371- fstab = cls(path=path)
372- entry = fstab.get_entry_by_attr('mountpoint', mountpoint)
373- if entry:
374- return fstab.remove_entry(entry)
375- return False
376-
377- @classmethod
378- def add(cls, device, mountpoint, filesystem, options=None, path=None):
379- return cls(path=path).add_entry(Fstab.Entry(device,
380- mountpoint, filesystem,
381- options=options))
382
383=== removed file 'hooks/charmhelpers/core/hookenv.py'
384--- hooks/charmhelpers/core/hookenv.py 2015-02-19 16:21:31 +0000
385+++ hooks/charmhelpers/core/hookenv.py 1970-01-01 00:00:00 +0000
386@@ -1,568 +0,0 @@
387-# Copyright 2014-2015 Canonical Limited.
388-#
389-# This file is part of charm-helpers.
390-#
391-# charm-helpers is free software: you can redistribute it and/or modify
392-# it under the terms of the GNU Lesser General Public License version 3 as
393-# published by the Free Software Foundation.
394-#
395-# charm-helpers is distributed in the hope that it will be useful,
396-# but WITHOUT ANY WARRANTY; without even the implied warranty of
397-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
398-# GNU Lesser General Public License for more details.
399-#
400-# You should have received a copy of the GNU Lesser General Public License
401-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
402-
403-"Interactions with the Juju environment"
404-# Copyright 2013 Canonical Ltd.
405-#
406-# Authors:
407-# Charm Helpers Developers <juju@lists.ubuntu.com>
408-
409-import os
410-import json
411-import yaml
412-import subprocess
413-import sys
414-from subprocess import CalledProcessError
415-
416-import six
417-if not six.PY3:
418- from UserDict import UserDict
419-else:
420- from collections import UserDict
421-
422-CRITICAL = "CRITICAL"
423-ERROR = "ERROR"
424-WARNING = "WARNING"
425-INFO = "INFO"
426-DEBUG = "DEBUG"
427-MARKER = object()
428-
429-cache = {}
430-
431-
432-def cached(func):
433- """Cache return values for multiple executions of func + args
434-
435- For example::
436-
437- @cached
438- def unit_get(attribute):
439- pass
440-
441- unit_get('test')
442-
443- will cache the result of unit_get + 'test' for future calls.
444- """
445- def wrapper(*args, **kwargs):
446- global cache
447- key = str((func, args, kwargs))
448- try:
449- return cache[key]
450- except KeyError:
451- res = func(*args, **kwargs)
452- cache[key] = res
453- return res
454- return wrapper
455-
456-
457-def flush(key):
458- """Flushes any entries from function cache where the
459- key is found in the function+args """
460- flush_list = []
461- for item in cache:
462- if key in item:
463- flush_list.append(item)
464- for item in flush_list:
465- del cache[item]
466-
467-
468-def log(message, level=None):
469- """Write a message to the juju log"""
470- command = ['juju-log']
471- if level:
472- command += ['-l', level]
473- if not isinstance(message, six.string_types):
474- message = repr(message)
475- command += [message]
476- subprocess.call(command)
477-
478-
479-class Serializable(UserDict):
480- """Wrapper, an object that can be serialized to yaml or json"""
481-
482- def __init__(self, obj):
483- # wrap the object
484- UserDict.__init__(self)
485- self.data = obj
486-
487- def __getattr__(self, attr):
488- # See if this object has attribute.
489- if attr in ("json", "yaml", "data"):
490- return self.__dict__[attr]
491- # Check for attribute in wrapped object.
492- got = getattr(self.data, attr, MARKER)
493- if got is not MARKER:
494- return got
495- # Proxy to the wrapped object via dict interface.
496- try:
497- return self.data[attr]
498- except KeyError:
499- raise AttributeError(attr)
500-
501- def __getstate__(self):
502- # Pickle as a standard dictionary.
503- return self.data
504-
505- def __setstate__(self, state):
506- # Unpickle into our wrapper.
507- self.data = state
508-
509- def json(self):
510- """Serialize the object to json"""
511- return json.dumps(self.data)
512-
513- def yaml(self):
514- """Serialize the object to yaml"""
515- return yaml.dump(self.data)
516-
517-
518-def execution_environment():
519- """A convenient bundling of the current execution context"""
520- context = {}
521- context['conf'] = config()
522- if relation_id():
523- context['reltype'] = relation_type()
524- context['relid'] = relation_id()
525- context['rel'] = relation_get()
526- context['unit'] = local_unit()
527- context['rels'] = relations()
528- context['env'] = os.environ
529- return context
530-
531-
532-def in_relation_hook():
533- """Determine whether we're running in a relation hook"""
534- return 'JUJU_RELATION' in os.environ
535-
536-
537-def relation_type():
538- """The scope for the current relation hook"""
539- return os.environ.get('JUJU_RELATION', None)
540-
541-
542-def relation_id():
543- """The relation ID for the current relation hook"""
544- return os.environ.get('JUJU_RELATION_ID', None)
545-
546-
547-def local_unit():
548- """Local unit ID"""
549- return os.environ['JUJU_UNIT_NAME']
550-
551-
552-def remote_unit():
553- """The remote unit for the current relation hook"""
554- return os.environ['JUJU_REMOTE_UNIT']
555-
556-
557-def service_name():
558- """The name service group this unit belongs to"""
559- return local_unit().split('/')[0]
560-
561-
562-def hook_name():
563- """The name of the currently executing hook"""
564- return os.path.basename(sys.argv[0])
565-
566-
567-class Config(dict):
568- """A dictionary representation of the charm's config.yaml, with some
569- extra features:
570-
571- - See which values in the dictionary have changed since the previous hook.
572- - For values that have changed, see what the previous value was.
573- - Store arbitrary data for use in a later hook.
574-
575- NOTE: Do not instantiate this object directly - instead call
576- ``hookenv.config()``, which will return an instance of :class:`Config`.
577-
578- Example usage::
579-
580- >>> # inside a hook
581- >>> from charmhelpers.core import hookenv
582- >>> config = hookenv.config()
583- >>> config['foo']
584- 'bar'
585- >>> # store a new key/value for later use
586- >>> config['mykey'] = 'myval'
587-
588-
589- >>> # user runs `juju set mycharm foo=baz`
590- >>> # now we're inside subsequent config-changed hook
591- >>> config = hookenv.config()
592- >>> config['foo']
593- 'baz'
594- >>> # test to see if this val has changed since last hook
595- >>> config.changed('foo')
596- True
597- >>> # what was the previous value?
598- >>> config.previous('foo')
599- 'bar'
600- >>> # keys/values that we add are preserved across hooks
601- >>> config['mykey']
602- 'myval'
603-
604- """
605- CONFIG_FILE_NAME = '.juju-persistent-config'
606-
607- def __init__(self, *args, **kw):
608- super(Config, self).__init__(*args, **kw)
609- self.implicit_save = True
610- self._prev_dict = None
611- self.path = os.path.join(charm_dir(), Config.CONFIG_FILE_NAME)
612- if os.path.exists(self.path):
613- self.load_previous()
614-
615- def __getitem__(self, key):
616- """For regular dict lookups, check the current juju config first,
617- then the previous (saved) copy. This ensures that user-saved values
618- will be returned by a dict lookup.
619-
620- """
621- try:
622- return dict.__getitem__(self, key)
623- except KeyError:
624- return (self._prev_dict or {})[key]
625-
626- def keys(self):
627- prev_keys = []
628- if self._prev_dict is not None:
629- prev_keys = self._prev_dict.keys()
630- return list(set(prev_keys + list(dict.keys(self))))
631-
632- def load_previous(self, path=None):
633- """Load previous copy of config from disk.
634-
635- In normal usage you don't need to call this method directly - it
636- is called automatically at object initialization.
637-
638- :param path:
639-
640- File path from which to load the previous config. If `None`,
641- config is loaded from the default location. If `path` is
642- specified, subsequent `save()` calls will write to the same
643- path.
644-
645- """
646- self.path = path or self.path
647- with open(self.path) as f:
648- self._prev_dict = json.load(f)
649-
650- def changed(self, key):
651- """Return True if the current value for this key is different from
652- the previous value.
653-
654- """
655- if self._prev_dict is None:
656- return True
657- return self.previous(key) != self.get(key)
658-
659- def previous(self, key):
660- """Return previous value for this key, or None if there
661- is no previous value.
662-
663- """
664- if self._prev_dict:
665- return self._prev_dict.get(key)
666- return None
667-
668- def save(self):
669- """Save this config to disk.
670-
671- If the charm is using the :mod:`Services Framework <services.base>`
672- or :meth:'@hook <Hooks.hook>' decorator, this
673- is called automatically at the end of successful hook execution.
674- Otherwise, it should be called directly by user code.
675-
676- To disable automatic saves, set ``implicit_save=False`` on this
677- instance.
678-
679- """
680- if self._prev_dict:
681- for k, v in six.iteritems(self._prev_dict):
682- if k not in self:
683- self[k] = v
684- with open(self.path, 'w') as f:
685- json.dump(self, f)
686-
687-
688-@cached
689-def config(scope=None):
690- """Juju charm configuration"""
691- config_cmd_line = ['config-get']
692- if scope is not None:
693- config_cmd_line.append(scope)
694- config_cmd_line.append('--format=json')
695- try:
696- config_data = json.loads(
697- subprocess.check_output(config_cmd_line).decode('UTF-8'))
698- if scope is not None:
699- return config_data
700- return Config(config_data)
701- except ValueError:
702- return None
703-
704-
705-@cached
706-def relation_get(attribute=None, unit=None, rid=None):
707- """Get relation information"""
708- _args = ['relation-get', '--format=json']
709- if rid:
710- _args.append('-r')
711- _args.append(rid)
712- _args.append(attribute or '-')
713- if unit:
714- _args.append(unit)
715- try:
716- return json.loads(subprocess.check_output(_args).decode('UTF-8'))
717- except ValueError:
718- return None
719- except CalledProcessError as e:
720- if e.returncode == 2:
721- return None
722- raise
723-
724-
725-def relation_set(relation_id=None, relation_settings=None, **kwargs):
726- """Set relation information for the current unit"""
727- relation_settings = relation_settings if relation_settings else {}
728- relation_cmd_line = ['relation-set']
729- if relation_id is not None:
730- relation_cmd_line.extend(('-r', relation_id))
731- for k, v in (list(relation_settings.items()) + list(kwargs.items())):
732- if v is None:
733- relation_cmd_line.append('{}='.format(k))
734- else:
735- relation_cmd_line.append('{}={}'.format(k, v))
736- subprocess.check_call(relation_cmd_line)
737- # Flush cache of any relation-gets for local unit
738- flush(local_unit())
739-
740-
741-@cached
742-def relation_ids(reltype=None):
743- """A list of relation_ids"""
744- reltype = reltype or relation_type()
745- relid_cmd_line = ['relation-ids', '--format=json']
746- if reltype is not None:
747- relid_cmd_line.append(reltype)
748- return json.loads(
749- subprocess.check_output(relid_cmd_line).decode('UTF-8')) or []
750- return []
751-
752-
753-@cached
754-def related_units(relid=None):
755- """A list of related units"""
756- relid = relid or relation_id()
757- units_cmd_line = ['relation-list', '--format=json']
758- if relid is not None:
759- units_cmd_line.extend(('-r', relid))
760- return json.loads(
761- subprocess.check_output(units_cmd_line).decode('UTF-8')) or []
762-
763-
764-@cached
765-def relation_for_unit(unit=None, rid=None):
766- """Get the json represenation of a unit's relation"""
767- unit = unit or remote_unit()
768- relation = relation_get(unit=unit, rid=rid)
769- for key in relation:
770- if key.endswith('-list'):
771- relation[key] = relation[key].split()
772- relation['__unit__'] = unit
773- return relation
774-
775-
776-@cached
777-def relations_for_id(relid=None):
778- """Get relations of a specific relation ID"""
779- relation_data = []
780- relid = relid or relation_ids()
781- for unit in related_units(relid):
782- unit_data = relation_for_unit(unit, relid)
783- unit_data['__relid__'] = relid
784- relation_data.append(unit_data)
785- return relation_data
786-
787-
788-@cached
789-def relations_of_type(reltype=None):
790- """Get relations of a specific type"""
791- relation_data = []
792- reltype = reltype or relation_type()
793- for relid in relation_ids(reltype):
794- for relation in relations_for_id(relid):
795- relation['__relid__'] = relid
796- relation_data.append(relation)
797- return relation_data
798-
799-
800-@cached
801-def metadata():
802- """Get the current charm metadata.yaml contents as a python object"""
803- with open(os.path.join(charm_dir(), 'metadata.yaml')) as md:
804- return yaml.safe_load(md)
805-
806-
807-@cached
808-def relation_types():
809- """Get a list of relation types supported by this charm"""
810- rel_types = []
811- md = metadata()
812- for key in ('provides', 'requires', 'peers'):
813- section = md.get(key)
814- if section:
815- rel_types.extend(section.keys())
816- return rel_types
817-
818-
819-@cached
820-def charm_name():
821- """Get the name of the current charm as is specified on metadata.yaml"""
822- return metadata().get('name')
823-
824-
825-@cached
826-def relations():
827- """Get a nested dictionary of relation data for all related units"""
828- rels = {}
829- for reltype in relation_types():
830- relids = {}
831- for relid in relation_ids(reltype):
832- units = {local_unit(): relation_get(unit=local_unit(), rid=relid)}
833- for unit in related_units(relid):
834- reldata = relation_get(unit=unit, rid=relid)
835- units[unit] = reldata
836- relids[relid] = units
837- rels[reltype] = relids
838- return rels
839-
840-
841-@cached
842-def is_relation_made(relation, keys='private-address'):
843- '''
844- Determine whether a relation is established by checking for
845- presence of key(s). If a list of keys is provided, they
846- must all be present for the relation to be identified as made
847- '''
848- if isinstance(keys, str):
849- keys = [keys]
850- for r_id in relation_ids(relation):
851- for unit in related_units(r_id):
852- context = {}
853- for k in keys:
854- context[k] = relation_get(k, rid=r_id,
855- unit=unit)
856- if None not in context.values():
857- return True
858- return False
859-
860-
861-def open_port(port, protocol="TCP"):
862- """Open a service network port"""
863- _args = ['open-port']
864- _args.append('{}/{}'.format(port, protocol))
865- subprocess.check_call(_args)
866-
867-
868-def close_port(port, protocol="TCP"):
869- """Close a service network port"""
870- _args = ['close-port']
871- _args.append('{}/{}'.format(port, protocol))
872- subprocess.check_call(_args)
873-
874-
875-@cached
876-def unit_get(attribute):
877- """Get the unit ID for the remote unit"""
878- _args = ['unit-get', '--format=json', attribute]
879- try:
880- return json.loads(subprocess.check_output(_args).decode('UTF-8'))
881- except ValueError:
882- return None
883-
884-
885-def unit_private_ip():
886- """Get this unit's private IP address"""
887- return unit_get('private-address')
888-
889-
890-class UnregisteredHookError(Exception):
891- """Raised when an undefined hook is called"""
892- pass
893-
894-
895-class Hooks(object):
896- """A convenient handler for hook functions.
897-
898- Example::
899-
900- hooks = Hooks()
901-
902- # register a hook, taking its name from the function name
903- @hooks.hook()
904- def install():
905- pass # your code here
906-
907- # register a hook, providing a custom hook name
908- @hooks.hook("config-changed")
909- def config_changed():
910- pass # your code here
911-
912- if __name__ == "__main__":
913- # execute a hook based on the name the program is called by
914- hooks.execute(sys.argv)
915- """
916-
917- def __init__(self, config_save=True):
918- super(Hooks, self).__init__()
919- self._hooks = {}
920- self._config_save = config_save
921-
922- def register(self, name, function):
923- """Register a hook"""
924- self._hooks[name] = function
925-
926- def execute(self, args):
927- """Execute a registered hook based on args[0]"""
928- hook_name = os.path.basename(args[0])
929- if hook_name in self._hooks:
930- self._hooks[hook_name]()
931- if self._config_save:
932- cfg = config()
933- if cfg.implicit_save:
934- cfg.save()
935- else:
936- raise UnregisteredHookError(hook_name)
937-
938- def hook(self, *hook_names):
939- """Decorator, registering them as hooks"""
940- def wrapper(decorated):
941- for hook_name in hook_names:
942- self.register(hook_name, decorated)
943- else:
944- self.register(decorated.__name__, decorated)
945- if '_' in decorated.__name__:
946- self.register(
947- decorated.__name__.replace('_', '-'), decorated)
948- return decorated
949- return wrapper
950-
951-
952-def charm_dir():
953- """Return the root directory of the current charm"""
954- return os.environ.get('CHARM_DIR')
955
956=== removed file 'hooks/charmhelpers/core/host.py'
957--- hooks/charmhelpers/core/host.py 2015-02-19 16:21:31 +0000
958+++ hooks/charmhelpers/core/host.py 1970-01-01 00:00:00 +0000
959@@ -1,446 +0,0 @@
960-# Copyright 2014-2015 Canonical Limited.
961-#
962-# This file is part of charm-helpers.
963-#
964-# charm-helpers is free software: you can redistribute it and/or modify
965-# it under the terms of the GNU Lesser General Public License version 3 as
966-# published by the Free Software Foundation.
967-#
968-# charm-helpers is distributed in the hope that it will be useful,
969-# but WITHOUT ANY WARRANTY; without even the implied warranty of
970-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
971-# GNU Lesser General Public License for more details.
972-#
973-# You should have received a copy of the GNU Lesser General Public License
974-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
975-
976-"""Tools for working with the host system"""
977-# Copyright 2012 Canonical Ltd.
978-#
979-# Authors:
980-# Nick Moffitt <nick.moffitt@canonical.com>
981-# Matthew Wedgwood <matthew.wedgwood@canonical.com>
982-
983-import os
984-import re
985-import pwd
986-import grp
987-import random
988-import string
989-import subprocess
990-import hashlib
991-from contextlib import contextmanager
992-from collections import OrderedDict
993-
994-import six
995-
996-from .hookenv import log
997-from .fstab import Fstab
998-
999-
1000-def service_start(service_name):
1001- """Start a system service"""
1002- return service('start', service_name)
1003-
1004-
1005-def service_stop(service_name):
1006- """Stop a system service"""
1007- return service('stop', service_name)
1008-
1009-
1010-def service_restart(service_name):
1011- """Restart a system service"""
1012- return service('restart', service_name)
1013-
1014-
1015-def service_reload(service_name, restart_on_failure=False):
1016- """Reload a system service, optionally falling back to restart if
1017- reload fails"""
1018- service_result = service('reload', service_name)
1019- if not service_result and restart_on_failure:
1020- service_result = service('restart', service_name)
1021- return service_result
1022-
1023-
1024-def service(action, service_name):
1025- """Control a system service"""
1026- cmd = ['service', service_name, action]
1027- return subprocess.call(cmd) == 0
1028-
1029-
1030-def service_running(service):
1031- """Determine whether a system service is running"""
1032- try:
1033- output = subprocess.check_output(
1034- ['service', service, 'status'],
1035- stderr=subprocess.STDOUT).decode('UTF-8')
1036- except subprocess.CalledProcessError:
1037- return False
1038- else:
1039- if ("start/running" in output or "is running" in output):
1040- return True
1041- else:
1042- return False
1043-
1044-
1045-def service_available(service_name):
1046- """Determine whether a system service is available"""
1047- try:
1048- subprocess.check_output(
1049- ['service', service_name, 'status'],
1050- stderr=subprocess.STDOUT).decode('UTF-8')
1051- except subprocess.CalledProcessError as e:
1052- return 'unrecognized service' not in e.output
1053- else:
1054- return True
1055-
1056-
1057-def adduser(username, password=None, shell='/bin/bash', system_user=False):
1058- """Add a user to the system"""
1059- try:
1060- user_info = pwd.getpwnam(username)
1061- log('user {0} already exists!'.format(username))
1062- except KeyError:
1063- log('creating user {0}'.format(username))
1064- cmd = ['useradd']
1065- if system_user or password is None:
1066- cmd.append('--system')
1067- else:
1068- cmd.extend([
1069- '--create-home',
1070- '--shell', shell,
1071- '--password', password,
1072- ])
1073- cmd.append(username)
1074- subprocess.check_call(cmd)
1075- user_info = pwd.getpwnam(username)
1076- return user_info
1077-
1078-
1079-def add_group(group_name, system_group=False):
1080- """Add a group to the system"""
1081- try:
1082- group_info = grp.getgrnam(group_name)
1083- log('group {0} already exists!'.format(group_name))
1084- except KeyError:
1085- log('creating group {0}'.format(group_name))
1086- cmd = ['addgroup']
1087- if system_group:
1088- cmd.append('--system')
1089- else:
1090- cmd.extend([
1091- '--group',
1092- ])
1093- cmd.append(group_name)
1094- subprocess.check_call(cmd)
1095- group_info = grp.getgrnam(group_name)
1096- return group_info
1097-
1098-
1099-def add_user_to_group(username, group):
1100- """Add a user to a group"""
1101- cmd = [
1102- 'gpasswd', '-a',
1103- username,
1104- group
1105- ]
1106- log("Adding user {} to group {}".format(username, group))
1107- subprocess.check_call(cmd)
1108-
1109-
1110-def rsync(from_path, to_path, flags='-r', options=None):
1111- """Replicate the contents of a path"""
1112- options = options or ['--delete', '--executability']
1113- cmd = ['/usr/bin/rsync', flags]
1114- cmd.extend(options)
1115- cmd.append(from_path)
1116- cmd.append(to_path)
1117- log(" ".join(cmd))
1118- return subprocess.check_output(cmd).decode('UTF-8').strip()
1119-
1120-
1121-def symlink(source, destination):
1122- """Create a symbolic link"""
1123- log("Symlinking {} as {}".format(source, destination))
1124- cmd = [
1125- 'ln',
1126- '-sf',
1127- source,
1128- destination,
1129- ]
1130- subprocess.check_call(cmd)
1131-
1132-
1133-def mkdir(path, owner='root', group='root', perms=0o555, force=False):
1134- """Create a directory"""
1135- log("Making dir {} {}:{} {:o}".format(path, owner, group,
1136- perms))
1137- uid = pwd.getpwnam(owner).pw_uid
1138- gid = grp.getgrnam(group).gr_gid
1139- realpath = os.path.abspath(path)
1140- path_exists = os.path.exists(realpath)
1141- if path_exists and force:
1142- if not os.path.isdir(realpath):
1143- log("Removing non-directory file {} prior to mkdir()".format(path))
1144- os.unlink(realpath)
1145- os.makedirs(realpath, perms)
1146- elif not path_exists:
1147- os.makedirs(realpath, perms)
1148- os.chown(realpath, uid, gid)
1149- os.chmod(realpath, perms)
1150-
1151-
1152-def write_file(path, content, owner='root', group='root', perms=0o444):
1153- """Create or overwrite a file with the contents of a byte string."""
1154- log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
1155- uid = pwd.getpwnam(owner).pw_uid
1156- gid = grp.getgrnam(group).gr_gid
1157- with open(path, 'wb') as target:
1158- os.fchown(target.fileno(), uid, gid)
1159- os.fchmod(target.fileno(), perms)
1160- target.write(content)
1161-
1162-
1163-def fstab_remove(mp):
1164- """Remove the given mountpoint entry from /etc/fstab
1165- """
1166- return Fstab.remove_by_mountpoint(mp)
1167-
1168-
1169-def fstab_add(dev, mp, fs, options=None):
1170- """Adds the given device entry to the /etc/fstab file
1171- """
1172- return Fstab.add(dev, mp, fs, options=options)
1173-
1174-
1175-def mount(device, mountpoint, options=None, persist=False, filesystem="ext3"):
1176- """Mount a filesystem at a particular mountpoint"""
1177- cmd_args = ['mount']
1178- if options is not None:
1179- cmd_args.extend(['-o', options])
1180- cmd_args.extend([device, mountpoint])
1181- try:
1182- subprocess.check_output(cmd_args)
1183- except subprocess.CalledProcessError as e:
1184- log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output))
1185- return False
1186-
1187- if persist:
1188- return fstab_add(device, mountpoint, filesystem, options=options)
1189- return True
1190-
1191-
1192-def umount(mountpoint, persist=False):
1193- """Unmount a filesystem"""
1194- cmd_args = ['umount', mountpoint]
1195- try:
1196- subprocess.check_output(cmd_args)
1197- except subprocess.CalledProcessError as e:
1198- log('Error unmounting {}\n{}'.format(mountpoint, e.output))
1199- return False
1200-
1201- if persist:
1202- return fstab_remove(mountpoint)
1203- return True
1204-
1205-
1206-def mounts():
1207- """Get a list of all mounted volumes as [[mountpoint,device],[...]]"""
1208- with open('/proc/mounts') as f:
1209- # [['/mount/point','/dev/path'],[...]]
1210- system_mounts = [m[1::-1] for m in [l.strip().split()
1211- for l in f.readlines()]]
1212- return system_mounts
1213-
1214-
1215-def file_hash(path, hash_type='md5'):
1216- """
1217- Generate a hash checksum of the contents of 'path' or None if not found.
1218-
1219- :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,
1220- such as md5, sha1, sha256, sha512, etc.
1221- """
1222- if os.path.exists(path):
1223- h = getattr(hashlib, hash_type)()
1224- with open(path, 'rb') as source:
1225- h.update(source.read())
1226- return h.hexdigest()
1227- else:
1228- return None
1229-
1230-
1231-def check_hash(path, checksum, hash_type='md5'):
1232- """
1233- Validate a file using a cryptographic checksum.
1234-
1235- :param str checksum: Value of the checksum used to validate the file.
1236- :param str hash_type: Hash algorithm used to generate `checksum`.
1237- Can be any hash alrgorithm supported by :mod:`hashlib`,
1238- such as md5, sha1, sha256, sha512, etc.
1239- :raises ChecksumError: If the file fails the checksum
1240-
1241- """
1242- actual_checksum = file_hash(path, hash_type)
1243- if checksum != actual_checksum:
1244- raise ChecksumError("'%s' != '%s'" % (checksum, actual_checksum))
1245-
1246-
1247-class ChecksumError(ValueError):
1248- pass
1249-
1250-
1251-def restart_on_change(restart_map, stopstart=False):
1252- """Restart services based on configuration files changing
1253-
1254- This function is used a decorator, for example::
1255-
1256- @restart_on_change({
1257- '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]
1258- })
1259- def ceph_client_changed():
1260- pass # your code here
1261-
1262- In this example, the cinder-api and cinder-volume services
1263- would be restarted if /etc/ceph/ceph.conf is changed by the
1264- ceph_client_changed function.
1265- """
1266- def wrap(f):
1267- def wrapped_f(*args, **kwargs):
1268- checksums = {}
1269- for path in restart_map:
1270- checksums[path] = file_hash(path)
1271- f(*args, **kwargs)
1272- restarts = []
1273- for path in restart_map:
1274- if checksums[path] != file_hash(path):
1275- restarts += restart_map[path]
1276- services_list = list(OrderedDict.fromkeys(restarts))
1277- if not stopstart:
1278- for service_name in services_list:
1279- service('restart', service_name)
1280- else:
1281- for action in ['stop', 'start']:
1282- for service_name in services_list:
1283- service(action, service_name)
1284- return wrapped_f
1285- return wrap
1286-
1287-
1288-def lsb_release():
1289- """Return /etc/lsb-release in a dict"""
1290- d = {}
1291- with open('/etc/lsb-release', 'r') as lsb:
1292- for l in lsb:
1293- k, v = l.split('=')
1294- d[k.strip()] = v.strip()
1295- return d
1296-
1297-
1298-def pwgen(length=None):
1299- """Generate a random pasword."""
1300- if length is None:
1301- length = random.choice(range(35, 45))
1302- alphanumeric_chars = [
1303- l for l in (string.ascii_letters + string.digits)
1304- if l not in 'l0QD1vAEIOUaeiou']
1305- random_chars = [
1306- random.choice(alphanumeric_chars) for _ in range(length)]
1307- return(''.join(random_chars))
1308-
1309-
1310-def list_nics(nic_type):
1311- '''Return a list of nics of given type(s)'''
1312- if isinstance(nic_type, six.string_types):
1313- int_types = [nic_type]
1314- else:
1315- int_types = nic_type
1316- interfaces = []
1317- for int_type in int_types:
1318- cmd = ['ip', 'addr', 'show', 'label', int_type + '*']
1319- ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
1320- ip_output = (line for line in ip_output if line)
1321- for line in ip_output:
1322- if line.split()[1].startswith(int_type):
1323- matched = re.search('.*: (' + int_type + r'[0-9]+\.[0-9]+)@.*', line)
1324- if matched:
1325- interface = matched.groups()[0]
1326- else:
1327- interface = line.split()[1].replace(":", "")
1328- interfaces.append(interface)
1329-
1330- return interfaces
1331-
1332-
1333-def set_nic_mtu(nic, mtu):
1334- '''Set MTU on a network interface'''
1335- cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
1336- subprocess.check_call(cmd)
1337-
1338-
1339-def get_nic_mtu(nic):
1340- cmd = ['ip', 'addr', 'show', nic]
1341- ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
1342- mtu = ""
1343- for line in ip_output:
1344- words = line.split()
1345- if 'mtu' in words:
1346- mtu = words[words.index("mtu") + 1]
1347- return mtu
1348-
1349-
1350-def get_nic_hwaddr(nic):
1351- cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
1352- ip_output = subprocess.check_output(cmd).decode('UTF-8')
1353- hwaddr = ""
1354- words = ip_output.split()
1355- if 'link/ether' in words:
1356- hwaddr = words[words.index('link/ether') + 1]
1357- return hwaddr
1358-
1359-
1360-def cmp_pkgrevno(package, revno, pkgcache=None):
1361- '''Compare supplied revno with the revno of the installed package
1362-
1363- * 1 => Installed revno is greater than supplied arg
1364- * 0 => Installed revno is the same as supplied arg
1365- * -1 => Installed revno is less than supplied arg
1366-
1367- This function imports apt_cache function from charmhelpers.fetch if
1368- the pkgcache argument is None. Be sure to add charmhelpers.fetch if
1369- you call this function, or pass an apt_pkg.Cache() instance.
1370- '''
1371- import apt_pkg
1372- if not pkgcache:
1373- from charmhelpers.fetch import apt_cache
1374- pkgcache = apt_cache()
1375- pkg = pkgcache[package]
1376- return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
1377-
1378-
1379-@contextmanager
1380-def chdir(d):
1381- cur = os.getcwd()
1382- try:
1383- yield os.chdir(d)
1384- finally:
1385- os.chdir(cur)
1386-
1387-
1388-def chownr(path, owner, group, follow_links=True):
1389- uid = pwd.getpwnam(owner).pw_uid
1390- gid = grp.getgrnam(group).gr_gid
1391- if follow_links:
1392- chown = os.chown
1393- else:
1394- chown = os.lchown
1395-
1396- for root, dirs, files in os.walk(path):
1397- for name in dirs + files:
1398- full = os.path.join(root, name)
1399- broken_symlink = os.path.lexists(full) and not os.path.exists(full)
1400- if not broken_symlink:
1401- chown(full, uid, gid)
1402-
1403-
1404-def lchownr(path, owner, group):
1405- chownr(path, owner, group, follow_links=False)
1406
1407=== removed directory 'hooks/charmhelpers/core/services'
1408=== removed file 'hooks/charmhelpers/core/services/__init__.py'
1409--- hooks/charmhelpers/core/services/__init__.py 2015-02-19 16:21:31 +0000
1410+++ hooks/charmhelpers/core/services/__init__.py 1970-01-01 00:00:00 +0000
1411@@ -1,18 +0,0 @@
1412-# Copyright 2014-2015 Canonical Limited.
1413-#
1414-# This file is part of charm-helpers.
1415-#
1416-# charm-helpers is free software: you can redistribute it and/or modify
1417-# it under the terms of the GNU Lesser General Public License version 3 as
1418-# published by the Free Software Foundation.
1419-#
1420-# charm-helpers is distributed in the hope that it will be useful,
1421-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1422-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1423-# GNU Lesser General Public License for more details.
1424-#
1425-# You should have received a copy of the GNU Lesser General Public License
1426-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1427-
1428-from .base import * # NOQA
1429-from .helpers import * # NOQA
1430
1431=== removed file 'hooks/charmhelpers/core/services/base.py'
1432--- hooks/charmhelpers/core/services/base.py 2015-02-19 16:21:31 +0000
1433+++ hooks/charmhelpers/core/services/base.py 1970-01-01 00:00:00 +0000
1434@@ -1,329 +0,0 @@
1435-# Copyright 2014-2015 Canonical Limited.
1436-#
1437-# This file is part of charm-helpers.
1438-#
1439-# charm-helpers is free software: you can redistribute it and/or modify
1440-# it under the terms of the GNU Lesser General Public License version 3 as
1441-# published by the Free Software Foundation.
1442-#
1443-# charm-helpers is distributed in the hope that it will be useful,
1444-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1445-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1446-# GNU Lesser General Public License for more details.
1447-#
1448-# You should have received a copy of the GNU Lesser General Public License
1449-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1450-
1451-import os
1452-import re
1453-import json
1454-from collections import Iterable
1455-
1456-from charmhelpers.core import host
1457-from charmhelpers.core import hookenv
1458-
1459-
1460-__all__ = ['ServiceManager', 'ManagerCallback',
1461- 'PortManagerCallback', 'open_ports', 'close_ports', 'manage_ports',
1462- 'service_restart', 'service_stop']
1463-
1464-
1465-class ServiceManager(object):
1466- def __init__(self, services=None):
1467- """
1468- Register a list of services, given their definitions.
1469-
1470- Service definitions are dicts in the following formats (all keys except
1471- 'service' are optional)::
1472-
1473- {
1474- "service": <service name>,
1475- "required_data": <list of required data contexts>,
1476- "provided_data": <list of provided data contexts>,
1477- "data_ready": <one or more callbacks>,
1478- "data_lost": <one or more callbacks>,
1479- "start": <one or more callbacks>,
1480- "stop": <one or more callbacks>,
1481- "ports": <list of ports to manage>,
1482- }
1483-
1484- The 'required_data' list should contain dicts of required data (or
1485- dependency managers that act like dicts and know how to collect the data).
1486- Only when all items in the 'required_data' list are populated are the list
1487- of 'data_ready' and 'start' callbacks executed. See `is_ready()` for more
1488- information.
1489-
1490- The 'provided_data' list should contain relation data providers, most likely
1491- a subclass of :class:`charmhelpers.core.services.helpers.RelationContext`,
1492- that will indicate a set of data to set on a given relation.
1493-
1494- The 'data_ready' value should be either a single callback, or a list of
1495- callbacks, to be called when all items in 'required_data' pass `is_ready()`.
1496- Each callback will be called with the service name as the only parameter.
1497- After all of the 'data_ready' callbacks are called, the 'start' callbacks
1498- are fired.
1499-
1500- The 'data_lost' value should be either a single callback, or a list of
1501- callbacks, to be called when a 'required_data' item no longer passes
1502- `is_ready()`. Each callback will be called with the service name as the
1503- only parameter. After all of the 'data_lost' callbacks are called,
1504- the 'stop' callbacks are fired.
1505-
1506- The 'start' value should be either a single callback, or a list of
1507- callbacks, to be called when starting the service, after the 'data_ready'
1508- callbacks are complete. Each callback will be called with the service
1509- name as the only parameter. This defaults to
1510- `[host.service_start, services.open_ports]`.
1511-
1512- The 'stop' value should be either a single callback, or a list of
1513- callbacks, to be called when stopping the service. If the service is
1514- being stopped because it no longer has all of its 'required_data', this
1515- will be called after all of the 'data_lost' callbacks are complete.
1516- Each callback will be called with the service name as the only parameter.
1517- This defaults to `[services.close_ports, host.service_stop]`.
1518-
1519- The 'ports' value should be a list of ports to manage. The default
1520- 'start' handler will open the ports after the service is started,
1521- and the default 'stop' handler will close the ports prior to stopping
1522- the service.
1523-
1524-
1525- Examples:
1526-
1527- The following registers an Upstart service called bingod that depends on
1528- a mongodb relation and which runs a custom `db_migrate` function prior to
1529- restarting the service, and a Runit service called spadesd::
1530-
1531- manager = services.ServiceManager([
1532- {
1533- 'service': 'bingod',
1534- 'ports': [80, 443],
1535- 'required_data': [MongoRelation(), config(), {'my': 'data'}],
1536- 'data_ready': [
1537- services.template(source='bingod.conf'),
1538- services.template(source='bingod.ini',
1539- target='/etc/bingod.ini',
1540- owner='bingo', perms=0400),
1541- ],
1542- },
1543- {
1544- 'service': 'spadesd',
1545- 'data_ready': services.template(source='spadesd_run.j2',
1546- target='/etc/sv/spadesd/run',
1547- perms=0555),
1548- 'start': runit_start,
1549- 'stop': runit_stop,
1550- },
1551- ])
1552- manager.manage()
1553- """
1554- self._ready_file = os.path.join(hookenv.charm_dir(), 'READY-SERVICES.json')
1555- self._ready = None
1556- self.services = {}
1557- for service in services or []:
1558- service_name = service['service']
1559- self.services[service_name] = service
1560-
1561- def manage(self):
1562- """
1563- Handle the current hook by doing The Right Thing with the registered services.
1564- """
1565- hook_name = hookenv.hook_name()
1566- if hook_name == 'stop':
1567- self.stop_services()
1568- else:
1569- self.provide_data()
1570- self.reconfigure_services()
1571- cfg = hookenv.config()
1572- if cfg.implicit_save:
1573- cfg.save()
1574-
1575- def provide_data(self):
1576- """
1577- Set the relation data for each provider in the ``provided_data`` list.
1578-
1579- A provider must have a `name` attribute, which indicates which relation
1580- to set data on, and a `provide_data()` method, which returns a dict of
1581- data to set.
1582- """
1583- hook_name = hookenv.hook_name()
1584- for service in self.services.values():
1585- for provider in service.get('provided_data', []):
1586- if re.match(r'{}-relation-(joined|changed)'.format(provider.name), hook_name):
1587- data = provider.provide_data()
1588- _ready = provider._is_ready(data) if hasattr(provider, '_is_ready') else data
1589- if _ready:
1590- hookenv.relation_set(None, data)
1591-
1592- def reconfigure_services(self, *service_names):
1593- """
1594- Update all files for one or more registered services, and,
1595- if ready, optionally restart them.
1596-
1597- If no service names are given, reconfigures all registered services.
1598- """
1599- for service_name in service_names or self.services.keys():
1600- if self.is_ready(service_name):
1601- self.fire_event('data_ready', service_name)
1602- self.fire_event('start', service_name, default=[
1603- service_restart,
1604- manage_ports])
1605- self.save_ready(service_name)
1606- else:
1607- if self.was_ready(service_name):
1608- self.fire_event('data_lost', service_name)
1609- self.fire_event('stop', service_name, default=[
1610- manage_ports,
1611- service_stop])
1612- self.save_lost(service_name)
1613-
1614- def stop_services(self, *service_names):
1615- """
1616- Stop one or more registered services, by name.
1617-
1618- If no service names are given, stops all registered services.
1619- """
1620- for service_name in service_names or self.services.keys():
1621- self.fire_event('stop', service_name, default=[
1622- manage_ports,
1623- service_stop])
1624-
1625- def get_service(self, service_name):
1626- """
1627- Given the name of a registered service, return its service definition.
1628- """
1629- service = self.services.get(service_name)
1630- if not service:
1631- raise KeyError('Service not registered: %s' % service_name)
1632- return service
1633-
1634- def fire_event(self, event_name, service_name, default=None):
1635- """
1636- Fire a data_ready, data_lost, start, or stop event on a given service.
1637- """
1638- service = self.get_service(service_name)
1639- callbacks = service.get(event_name, default)
1640- if not callbacks:
1641- return
1642- if not isinstance(callbacks, Iterable):
1643- callbacks = [callbacks]
1644- for callback in callbacks:
1645- if isinstance(callback, ManagerCallback):
1646- callback(self, service_name, event_name)
1647- else:
1648- callback(service_name)
1649-
1650- def is_ready(self, service_name):
1651- """
1652- Determine if a registered service is ready, by checking its 'required_data'.
1653-
1654- A 'required_data' item can be any mapping type, and is considered ready
1655- if `bool(item)` evaluates as True.
1656- """
1657- service = self.get_service(service_name)
1658- reqs = service.get('required_data', [])
1659- return all(bool(req) for req in reqs)
1660-
1661- def _load_ready_file(self):
1662- if self._ready is not None:
1663- return
1664- if os.path.exists(self._ready_file):
1665- with open(self._ready_file) as fp:
1666- self._ready = set(json.load(fp))
1667- else:
1668- self._ready = set()
1669-
1670- def _save_ready_file(self):
1671- if self._ready is None:
1672- return
1673- with open(self._ready_file, 'w') as fp:
1674- json.dump(list(self._ready), fp)
1675-
1676- def save_ready(self, service_name):
1677- """
1678- Save an indicator that the given service is now data_ready.
1679- """
1680- self._load_ready_file()
1681- self._ready.add(service_name)
1682- self._save_ready_file()
1683-
1684- def save_lost(self, service_name):
1685- """
1686- Save an indicator that the given service is no longer data_ready.
1687- """
1688- self._load_ready_file()
1689- self._ready.discard(service_name)
1690- self._save_ready_file()
1691-
1692- def was_ready(self, service_name):
1693- """
1694- Determine if the given service was previously data_ready.
1695- """
1696- self._load_ready_file()
1697- return service_name in self._ready
1698-
1699-
1700-class ManagerCallback(object):
1701- """
1702- Special case of a callback that takes the `ServiceManager` instance
1703- in addition to the service name.
1704-
1705- Subclasses should implement `__call__` which should accept three parameters:
1706-
1707- * `manager` The `ServiceManager` instance
1708- * `service_name` The name of the service it's being triggered for
1709- * `event_name` The name of the event that this callback is handling
1710- """
1711- def __call__(self, manager, service_name, event_name):
1712- raise NotImplementedError()
1713-
1714-
1715-class PortManagerCallback(ManagerCallback):
1716- """
1717- Callback class that will open or close ports, for use as either
1718- a start or stop action.
1719- """
1720- def __call__(self, manager, service_name, event_name):
1721- service = manager.get_service(service_name)
1722- new_ports = service.get('ports', [])
1723- port_file = os.path.join(hookenv.charm_dir(), '.{}.ports'.format(service_name))
1724- if os.path.exists(port_file):
1725- with open(port_file) as fp:
1726- old_ports = fp.read().split(',')
1727- for old_port in old_ports:
1728- if bool(old_port):
1729- old_port = int(old_port)
1730- if old_port not in new_ports:
1731- hookenv.close_port(old_port)
1732- with open(port_file, 'w') as fp:
1733- fp.write(','.join(str(port) for port in new_ports))
1734- for port in new_ports:
1735- if event_name == 'start':
1736- hookenv.open_port(port)
1737- elif event_name == 'stop':
1738- hookenv.close_port(port)
1739-
1740-
1741-def service_stop(service_name):
1742- """
1743- Wrapper around host.service_stop to prevent spurious "unknown service"
1744- messages in the logs.
1745- """
1746- if host.service_running(service_name):
1747- host.service_stop(service_name)
1748-
1749-
1750-def service_restart(service_name):
1751- """
1752- Wrapper around host.service_restart to prevent spurious "unknown service"
1753- messages in the logs.
1754- """
1755- if host.service_available(service_name):
1756- if host.service_running(service_name):
1757- host.service_restart(service_name)
1758- else:
1759- host.service_start(service_name)
1760-
1761-
1762-# Convenience aliases
1763-open_ports = close_ports = manage_ports = PortManagerCallback()
1764
1765=== removed file 'hooks/charmhelpers/core/services/helpers.py'
1766--- hooks/charmhelpers/core/services/helpers.py 2015-02-19 16:21:31 +0000
1767+++ hooks/charmhelpers/core/services/helpers.py 1970-01-01 00:00:00 +0000
1768@@ -1,259 +0,0 @@
1769-# Copyright 2014-2015 Canonical Limited.
1770-#
1771-# This file is part of charm-helpers.
1772-#
1773-# charm-helpers is free software: you can redistribute it and/or modify
1774-# it under the terms of the GNU Lesser General Public License version 3 as
1775-# published by the Free Software Foundation.
1776-#
1777-# charm-helpers is distributed in the hope that it will be useful,
1778-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1779-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1780-# GNU Lesser General Public License for more details.
1781-#
1782-# You should have received a copy of the GNU Lesser General Public License
1783-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1784-
1785-import os
1786-import yaml
1787-from charmhelpers.core import hookenv
1788-from charmhelpers.core import templating
1789-
1790-from charmhelpers.core.services.base import ManagerCallback
1791-
1792-
1793-__all__ = ['RelationContext', 'TemplateCallback',
1794- 'render_template', 'template']
1795-
1796-
1797-class RelationContext(dict):
1798- """
1799- Base class for a context generator that gets relation data from juju.
1800-
1801- Subclasses must provide the attributes `name`, which is the name of the
1802- interface of interest, `interface`, which is the type of the interface of
1803- interest, and `required_keys`, which is the set of keys required for the
1804- relation to be considered complete. The data for all interfaces matching
1805- the `name` attribute that are complete will used to populate the dictionary
1806- values (see `get_data`, below).
1807-
1808- The generated context will be namespaced under the relation :attr:`name`,
1809- to prevent potential naming conflicts.
1810-
1811- :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
1812- :param list additional_required_keys: Extend the list of :attr:`required_keys`
1813- """
1814- name = None
1815- interface = None
1816- required_keys = []
1817-
1818- def __init__(self, name=None, additional_required_keys=None):
1819- if name is not None:
1820- self.name = name
1821- if additional_required_keys is not None:
1822- self.required_keys.extend(additional_required_keys)
1823- self.get_data()
1824-
1825- def __bool__(self):
1826- """
1827- Returns True if all of the required_keys are available.
1828- """
1829- return self.is_ready()
1830-
1831- __nonzero__ = __bool__
1832-
1833- def __repr__(self):
1834- return super(RelationContext, self).__repr__()
1835-
1836- def is_ready(self):
1837- """
1838- Returns True if all of the `required_keys` are available from any units.
1839- """
1840- ready = len(self.get(self.name, [])) > 0
1841- if not ready:
1842- hookenv.log('Incomplete relation: {}'.format(self.__class__.__name__), hookenv.DEBUG)
1843- return ready
1844-
1845- def _is_ready(self, unit_data):
1846- """
1847- Helper method that tests a set of relation data and returns True if
1848- all of the `required_keys` are present.
1849- """
1850- return set(unit_data.keys()).issuperset(set(self.required_keys))
1851-
1852- def get_data(self):
1853- """
1854- Retrieve the relation data for each unit involved in a relation and,
1855- if complete, store it in a list under `self[self.name]`. This
1856- is automatically called when the RelationContext is instantiated.
1857-
1858- The units are sorted lexographically first by the service ID, then by
1859- the unit ID. Thus, if an interface has two other services, 'db:1'
1860- and 'db:2', with 'db:1' having two units, 'wordpress/0' and 'wordpress/1',
1861- and 'db:2' having one unit, 'mediawiki/0', all of which have a complete
1862- set of data, the relation data for the units will be stored in the
1863- order: 'wordpress/0', 'wordpress/1', 'mediawiki/0'.
1864-
1865- If you only care about a single unit on the relation, you can just
1866- access it as `{{ interface[0]['key'] }}`. However, if you can at all
1867- support multiple units on a relation, you should iterate over the list,
1868- like::
1869-
1870- {% for unit in interface -%}
1871- {{ unit['key'] }}{% if not loop.last %},{% endif %}
1872- {%- endfor %}
1873-
1874- Note that since all sets of relation data from all related services and
1875- units are in a single list, if you need to know which service or unit a
1876- set of data came from, you'll need to extend this class to preserve
1877- that information.
1878- """
1879- if not hookenv.relation_ids(self.name):
1880- return
1881-
1882- ns = self.setdefault(self.name, [])
1883- for rid in sorted(hookenv.relation_ids(self.name)):
1884- for unit in sorted(hookenv.related_units(rid)):
1885- reldata = hookenv.relation_get(rid=rid, unit=unit)
1886- if self._is_ready(reldata):
1887- ns.append(reldata)
1888-
1889- def provide_data(self):
1890- """
1891- Return data to be relation_set for this interface.
1892- """
1893- return {}
1894-
1895-
1896-class MysqlRelation(RelationContext):
1897- """
1898- Relation context for the `mysql` interface.
1899-
1900- :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
1901- :param list additional_required_keys: Extend the list of :attr:`required_keys`
1902- """
1903- name = 'db'
1904- interface = 'mysql'
1905- required_keys = ['host', 'user', 'password', 'database']
1906-
1907-
1908-class HttpRelation(RelationContext):
1909- """
1910- Relation context for the `http` interface.
1911-
1912- :param str name: Override the relation :attr:`name`, since it can vary from charm to charm
1913- :param list additional_required_keys: Extend the list of :attr:`required_keys`
1914- """
1915- name = 'website'
1916- interface = 'http'
1917- required_keys = ['host', 'port']
1918-
1919- def provide_data(self):
1920- return {
1921- 'host': hookenv.unit_get('private-address'),
1922- 'port': 80,
1923- }
1924-
1925-
1926-class RequiredConfig(dict):
1927- """
1928- Data context that loads config options with one or more mandatory options.
1929-
1930- Once the required options have been changed from their default values, all
1931- config options will be available, namespaced under `config` to prevent
1932- potential naming conflicts (for example, between a config option and a
1933- relation property).
1934-
1935- :param list *args: List of options that must be changed from their default values.
1936- """
1937-
1938- def __init__(self, *args):
1939- self.required_options = args
1940- self['config'] = hookenv.config()
1941- with open(os.path.join(hookenv.charm_dir(), 'config.yaml')) as fp:
1942- self.config = yaml.load(fp).get('options', {})
1943-
1944- def __bool__(self):
1945- for option in self.required_options:
1946- if option not in self['config']:
1947- return False
1948- current_value = self['config'][option]
1949- default_value = self.config[option].get('default')
1950- if current_value == default_value:
1951- return False
1952- if current_value in (None, '') and default_value in (None, ''):
1953- return False
1954- return True
1955-
1956- def __nonzero__(self):
1957- return self.__bool__()
1958-
1959-
1960-class StoredContext(dict):
1961- """
1962- A data context that always returns the data that it was first created with.
1963-
1964- This is useful to do a one-time generation of things like passwords, that
1965- will thereafter use the same value that was originally generated, instead
1966- of generating a new value each time it is run.
1967- """
1968- def __init__(self, file_name, config_data):
1969- """
1970- If the file exists, populate `self` with the data from the file.
1971- Otherwise, populate with the given data and persist it to the file.
1972- """
1973- if os.path.exists(file_name):
1974- self.update(self.read_context(file_name))
1975- else:
1976- self.store_context(file_name, config_data)
1977- self.update(config_data)
1978-
1979- def store_context(self, file_name, config_data):
1980- if not os.path.isabs(file_name):
1981- file_name = os.path.join(hookenv.charm_dir(), file_name)
1982- with open(file_name, 'w') as file_stream:
1983- os.fchmod(file_stream.fileno(), 0o600)
1984- yaml.dump(config_data, file_stream)
1985-
1986- def read_context(self, file_name):
1987- if not os.path.isabs(file_name):
1988- file_name = os.path.join(hookenv.charm_dir(), file_name)
1989- with open(file_name, 'r') as file_stream:
1990- data = yaml.load(file_stream)
1991- if not data:
1992- raise OSError("%s is empty" % file_name)
1993- return data
1994-
1995-
1996-class TemplateCallback(ManagerCallback):
1997- """
1998- Callback class that will render a Jinja2 template, for use as a ready
1999- action.
2000-
2001- :param str source: The template source file, relative to
2002- `$CHARM_DIR/templates`
2003-
2004- :param str target: The target to write the rendered template to
2005- :param str owner: The owner of the rendered file
2006- :param str group: The group of the rendered file
2007- :param int perms: The permissions of the rendered file
2008- """
2009- def __init__(self, source, target,
2010- owner='root', group='root', perms=0o444):
2011- self.source = source
2012- self.target = target
2013- self.owner = owner
2014- self.group = group
2015- self.perms = perms
2016-
2017- def __call__(self, manager, service_name, event_name):
2018- service = manager.get_service(service_name)
2019- context = {}
2020- for ctx in service.get('required_data', []):
2021- context.update(ctx)
2022- templating.render(self.source, self.target, context,
2023- self.owner, self.group, self.perms)
2024-
2025-
2026-# Convenience aliases for templates
2027-render_template = template = TemplateCallback
2028
2029=== removed file 'hooks/charmhelpers/core/strutils.py'
2030--- hooks/charmhelpers/core/strutils.py 2015-02-19 16:21:31 +0000
2031+++ hooks/charmhelpers/core/strutils.py 1970-01-01 00:00:00 +0000
2032@@ -1,42 +0,0 @@
2033-#!/usr/bin/env python
2034-# -*- coding: utf-8 -*-
2035-
2036-# Copyright 2014-2015 Canonical Limited.
2037-#
2038-# This file is part of charm-helpers.
2039-#
2040-# charm-helpers is free software: you can redistribute it and/or modify
2041-# it under the terms of the GNU Lesser General Public License version 3 as
2042-# published by the Free Software Foundation.
2043-#
2044-# charm-helpers is distributed in the hope that it will be useful,
2045-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2046-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2047-# GNU Lesser General Public License for more details.
2048-#
2049-# You should have received a copy of the GNU Lesser General Public License
2050-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2051-
2052-import six
2053-
2054-
2055-def bool_from_string(value):
2056- """Interpret string value as boolean.
2057-
2058- Returns True if value translates to True otherwise False.
2059- """
2060- if isinstance(value, six.string_types):
2061- value = six.text_type(value)
2062- else:
2063- msg = "Unable to interpret non-string value '%s' as boolean" % (value)
2064- raise ValueError(msg)
2065-
2066- value = value.strip().lower()
2067-
2068- if value in ['y', 'yes', 'true', 't']:
2069- return True
2070- elif value in ['n', 'no', 'false', 'f']:
2071- return False
2072-
2073- msg = "Unable to interpret string value '%s' as boolean" % (value)
2074- raise ValueError(msg)
2075
2076=== removed file 'hooks/charmhelpers/core/sysctl.py'
2077--- hooks/charmhelpers/core/sysctl.py 2015-02-19 16:21:31 +0000
2078+++ hooks/charmhelpers/core/sysctl.py 1970-01-01 00:00:00 +0000
2079@@ -1,56 +0,0 @@
2080-#!/usr/bin/env python
2081-# -*- coding: utf-8 -*-
2082-
2083-# Copyright 2014-2015 Canonical Limited.
2084-#
2085-# This file is part of charm-helpers.
2086-#
2087-# charm-helpers is free software: you can redistribute it and/or modify
2088-# it under the terms of the GNU Lesser General Public License version 3 as
2089-# published by the Free Software Foundation.
2090-#
2091-# charm-helpers is distributed in the hope that it will be useful,
2092-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2093-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2094-# GNU Lesser General Public License for more details.
2095-#
2096-# You should have received a copy of the GNU Lesser General Public License
2097-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2098-
2099-import yaml
2100-
2101-from subprocess import check_call
2102-
2103-from charmhelpers.core.hookenv import (
2104- log,
2105- DEBUG,
2106- ERROR,
2107-)
2108-
2109-__author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>'
2110-
2111-
2112-def create(sysctl_dict, sysctl_file):
2113- """Creates a sysctl.conf file from a YAML associative array
2114-
2115- :param sysctl_dict: a YAML-formatted string of sysctl options eg "{ 'kernel.max_pid': 1337 }"
2116- :type sysctl_dict: str
2117- :param sysctl_file: path to the sysctl file to be saved
2118- :type sysctl_file: str or unicode
2119- :returns: None
2120- """
2121- try:
2122- sysctl_dict_parsed = yaml.safe_load(sysctl_dict)
2123- except yaml.YAMLError:
2124- log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict),
2125- level=ERROR)
2126- return
2127-
2128- with open(sysctl_file, "w") as fd:
2129- for key, value in sysctl_dict_parsed.items():
2130- fd.write("{}={}\n".format(key, value))
2131-
2132- log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict_parsed),
2133- level=DEBUG)
2134-
2135- check_call(["sysctl", "-p", sysctl_file])
2136
2137=== removed file 'hooks/charmhelpers/core/templating.py'
2138--- hooks/charmhelpers/core/templating.py 2015-02-19 16:21:31 +0000
2139+++ hooks/charmhelpers/core/templating.py 1970-01-01 00:00:00 +0000
2140@@ -1,68 +0,0 @@
2141-# Copyright 2014-2015 Canonical Limited.
2142-#
2143-# This file is part of charm-helpers.
2144-#
2145-# charm-helpers is free software: you can redistribute it and/or modify
2146-# it under the terms of the GNU Lesser General Public License version 3 as
2147-# published by the Free Software Foundation.
2148-#
2149-# charm-helpers is distributed in the hope that it will be useful,
2150-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2151-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2152-# GNU Lesser General Public License for more details.
2153-#
2154-# You should have received a copy of the GNU Lesser General Public License
2155-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2156-
2157-import os
2158-
2159-from charmhelpers.core import host
2160-from charmhelpers.core import hookenv
2161-
2162-
2163-def render(source, target, context, owner='root', group='root',
2164- perms=0o444, templates_dir=None, encoding='UTF-8'):
2165- """
2166- Render a template.
2167-
2168- The `source` path, if not absolute, is relative to the `templates_dir`.
2169-
2170- The `target` path should be absolute.
2171-
2172- The context should be a dict containing the values to be replaced in the
2173- template.
2174-
2175- The `owner`, `group`, and `perms` options will be passed to `write_file`.
2176-
2177- If omitted, `templates_dir` defaults to the `templates` folder in the charm.
2178-
2179- Note: Using this requires python-jinja2; if it is not installed, calling
2180- this will attempt to use charmhelpers.fetch.apt_install to install it.
2181- """
2182- try:
2183- from jinja2 import FileSystemLoader, Environment, exceptions
2184- except ImportError:
2185- try:
2186- from charmhelpers.fetch import apt_install
2187- except ImportError:
2188- hookenv.log('Could not import jinja2, and could not import '
2189- 'charmhelpers.fetch to install it',
2190- level=hookenv.ERROR)
2191- raise
2192- apt_install('python-jinja2', fatal=True)
2193- from jinja2 import FileSystemLoader, Environment, exceptions
2194-
2195- if templates_dir is None:
2196- templates_dir = os.path.join(hookenv.charm_dir(), 'templates')
2197- loader = Environment(loader=FileSystemLoader(templates_dir))
2198- try:
2199- source = source
2200- template = loader.get_template(source)
2201- except exceptions.TemplateNotFound as e:
2202- hookenv.log('Could not load template %s from %s.' %
2203- (source, templates_dir),
2204- level=hookenv.ERROR)
2205- raise e
2206- content = template.render(context)
2207- host.mkdir(os.path.dirname(target), owner, group, perms=0o755)
2208- host.write_file(target, content.encode(encoding), owner, group, perms)
2209
2210=== removed file 'hooks/charmhelpers/core/unitdata.py'
2211--- hooks/charmhelpers/core/unitdata.py 2015-02-19 16:21:31 +0000
2212+++ hooks/charmhelpers/core/unitdata.py 1970-01-01 00:00:00 +0000
2213@@ -1,477 +0,0 @@
2214-#!/usr/bin/env python
2215-# -*- coding: utf-8 -*-
2216-#
2217-# Copyright 2014-2015 Canonical Limited.
2218-#
2219-# This file is part of charm-helpers.
2220-#
2221-# charm-helpers is free software: you can redistribute it and/or modify
2222-# it under the terms of the GNU Lesser General Public License version 3 as
2223-# published by the Free Software Foundation.
2224-#
2225-# charm-helpers is distributed in the hope that it will be useful,
2226-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2227-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2228-# GNU Lesser General Public License for more details.
2229-#
2230-# You should have received a copy of the GNU Lesser General Public License
2231-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2232-#
2233-#
2234-# Authors:
2235-# Kapil Thangavelu <kapil.foss@gmail.com>
2236-#
2237-"""
2238-Intro
2239------
2240-
2241-A simple way to store state in units. This provides a key value
2242-storage with support for versioned, transactional operation,
2243-and can calculate deltas from previous values to simplify unit logic
2244-when processing changes.
2245-
2246-
2247-Hook Integration
2248-----------------
2249-
2250-There are several extant frameworks for hook execution, including
2251-
2252- - charmhelpers.core.hookenv.Hooks
2253- - charmhelpers.core.services.ServiceManager
2254-
2255-The storage classes are framework agnostic, one simple integration is
2256-via the HookData contextmanager. It will record the current hook
2257-execution environment (including relation data, config data, etc.),
2258-setup a transaction and allow easy access to the changes from
2259-previously seen values. One consequence of the integration is the
2260-reservation of particular keys ('rels', 'unit', 'env', 'config',
2261-'charm_revisions') for their respective values.
2262-
2263-Here's a fully worked integration example using hookenv.Hooks::
2264-
2265- from charmhelper.core import hookenv, unitdata
2266-
2267- hook_data = unitdata.HookData()
2268- db = unitdata.kv()
2269- hooks = hookenv.Hooks()
2270-
2271- @hooks.hook
2272- def config_changed():
2273- # Print all changes to configuration from previously seen
2274- # values.
2275- for changed, (prev, cur) in hook_data.conf.items():
2276- print('config changed', changed,
2277- 'previous value', prev,
2278- 'current value', cur)
2279-
2280- # Get some unit specific bookeeping
2281- if not db.get('pkg_key'):
2282- key = urllib.urlopen('https://example.com/pkg_key').read()
2283- db.set('pkg_key', key)
2284-
2285- # Directly access all charm config as a mapping.
2286- conf = db.getrange('config', True)
2287-
2288- # Directly access all relation data as a mapping
2289- rels = db.getrange('rels', True)
2290-
2291- if __name__ == '__main__':
2292- with hook_data():
2293- hook.execute()
2294-
2295-
2296-A more basic integration is via the hook_scope context manager which simply
2297-manages transaction scope (and records hook name, and timestamp)::
2298-
2299- >>> from unitdata import kv
2300- >>> db = kv()
2301- >>> with db.hook_scope('install'):
2302- ... # do work, in transactional scope.
2303- ... db.set('x', 1)
2304- >>> db.get('x')
2305- 1
2306-
2307-
2308-Usage
2309------
2310-
2311-Values are automatically json de/serialized to preserve basic typing
2312-and complex data struct capabilities (dicts, lists, ints, booleans, etc).
2313-
2314-Individual values can be manipulated via get/set::
2315-
2316- >>> kv.set('y', True)
2317- >>> kv.get('y')
2318- True
2319-
2320- # We can set complex values (dicts, lists) as a single key.
2321- >>> kv.set('config', {'a': 1, 'b': True'})
2322-
2323- # Also supports returning dictionaries as a record which
2324- # provides attribute access.
2325- >>> config = kv.get('config', record=True)
2326- >>> config.b
2327- True
2328-
2329-
2330-Groups of keys can be manipulated with update/getrange::
2331-
2332- >>> kv.update({'z': 1, 'y': 2}, prefix="gui.")
2333- >>> kv.getrange('gui.', strip=True)
2334- {'z': 1, 'y': 2}
2335-
2336-When updating values, its very helpful to understand which values
2337-have actually changed and how have they changed. The storage
2338-provides a delta method to provide for this::
2339-
2340- >>> data = {'debug': True, 'option': 2}
2341- >>> delta = kv.delta(data, 'config.')
2342- >>> delta.debug.previous
2343- None
2344- >>> delta.debug.current
2345- True
2346- >>> delta
2347- {'debug': (None, True), 'option': (None, 2)}
2348-
2349-Note the delta method does not persist the actual change, it needs to
2350-be explicitly saved via 'update' method::
2351-
2352- >>> kv.update(data, 'config.')
2353-
2354-Values modified in the context of a hook scope retain historical values
2355-associated to the hookname.
2356-
2357- >>> with db.hook_scope('config-changed'):
2358- ... db.set('x', 42)
2359- >>> db.gethistory('x')
2360- [(1, u'x', 1, u'install', u'2015-01-21T16:49:30.038372'),
2361- (2, u'x', 42, u'config-changed', u'2015-01-21T16:49:30.038786')]
2362-
2363-"""
2364-
2365-import collections
2366-import contextlib
2367-import datetime
2368-import json
2369-import os
2370-import pprint
2371-import sqlite3
2372-import sys
2373-
2374-__author__ = 'Kapil Thangavelu <kapil.foss@gmail.com>'
2375-
2376-
2377-class Storage(object):
2378- """Simple key value database for local unit state within charms.
2379-
2380- Modifications are automatically committed at hook exit. That's
2381- currently regardless of exit code.
2382-
2383- To support dicts, lists, integer, floats, and booleans values
2384- are automatically json encoded/decoded.
2385- """
2386- def __init__(self, path=None):
2387- self.db_path = path
2388- if path is None:
2389- self.db_path = os.path.join(
2390- os.environ.get('CHARM_DIR', ''), '.unit-state.db')
2391- self.conn = sqlite3.connect('%s' % self.db_path)
2392- self.cursor = self.conn.cursor()
2393- self.revision = None
2394- self._closed = False
2395- self._init()
2396-
2397- def close(self):
2398- if self._closed:
2399- return
2400- self.flush(False)
2401- self.cursor.close()
2402- self.conn.close()
2403- self._closed = True
2404-
2405- def _scoped_query(self, stmt, params=None):
2406- if params is None:
2407- params = []
2408- return stmt, params
2409-
2410- def get(self, key, default=None, record=False):
2411- self.cursor.execute(
2412- *self._scoped_query(
2413- 'select data from kv where key=?', [key]))
2414- result = self.cursor.fetchone()
2415- if not result:
2416- return default
2417- if record:
2418- return Record(json.loads(result[0]))
2419- return json.loads(result[0])
2420-
2421- def getrange(self, key_prefix, strip=False):
2422- stmt = "select key, data from kv where key like '%s%%'" % key_prefix
2423- self.cursor.execute(*self._scoped_query(stmt))
2424- result = self.cursor.fetchall()
2425-
2426- if not result:
2427- return None
2428- if not strip:
2429- key_prefix = ''
2430- return dict([
2431- (k[len(key_prefix):], json.loads(v)) for k, v in result])
2432-
2433- def update(self, mapping, prefix=""):
2434- for k, v in mapping.items():
2435- self.set("%s%s" % (prefix, k), v)
2436-
2437- def unset(self, key):
2438- self.cursor.execute('delete from kv where key=?', [key])
2439- if self.revision and self.cursor.rowcount:
2440- self.cursor.execute(
2441- 'insert into kv_revisions values (?, ?, ?)',
2442- [key, self.revision, json.dumps('DELETED')])
2443-
2444- def set(self, key, value):
2445- serialized = json.dumps(value)
2446-
2447- self.cursor.execute(
2448- 'select data from kv where key=?', [key])
2449- exists = self.cursor.fetchone()
2450-
2451- # Skip mutations to the same value
2452- if exists:
2453- if exists[0] == serialized:
2454- return value
2455-
2456- if not exists:
2457- self.cursor.execute(
2458- 'insert into kv (key, data) values (?, ?)',
2459- (key, serialized))
2460- else:
2461- self.cursor.execute('''
2462- update kv
2463- set data = ?
2464- where key = ?''', [serialized, key])
2465-
2466- # Save
2467- if not self.revision:
2468- return value
2469-
2470- self.cursor.execute(
2471- 'select 1 from kv_revisions where key=? and revision=?',
2472- [key, self.revision])
2473- exists = self.cursor.fetchone()
2474-
2475- if not exists:
2476- self.cursor.execute(
2477- '''insert into kv_revisions (
2478- revision, key, data) values (?, ?, ?)''',
2479- (self.revision, key, serialized))
2480- else:
2481- self.cursor.execute(
2482- '''
2483- update kv_revisions
2484- set data = ?
2485- where key = ?
2486- and revision = ?''',
2487- [serialized, key, self.revision])
2488-
2489- return value
2490-
2491- def delta(self, mapping, prefix):
2492- """
2493- return a delta containing values that have changed.
2494- """
2495- previous = self.getrange(prefix, strip=True)
2496- if not previous:
2497- pk = set()
2498- else:
2499- pk = set(previous.keys())
2500- ck = set(mapping.keys())
2501- delta = DeltaSet()
2502-
2503- # added
2504- for k in ck.difference(pk):
2505- delta[k] = Delta(None, mapping[k])
2506-
2507- # removed
2508- for k in pk.difference(ck):
2509- delta[k] = Delta(previous[k], None)
2510-
2511- # changed
2512- for k in pk.intersection(ck):
2513- c = mapping[k]
2514- p = previous[k]
2515- if c != p:
2516- delta[k] = Delta(p, c)
2517-
2518- return delta
2519-
2520- @contextlib.contextmanager
2521- def hook_scope(self, name=""):
2522- """Scope all future interactions to the current hook execution
2523- revision."""
2524- assert not self.revision
2525- self.cursor.execute(
2526- 'insert into hooks (hook, date) values (?, ?)',
2527- (name or sys.argv[0],
2528- datetime.datetime.utcnow().isoformat()))
2529- self.revision = self.cursor.lastrowid
2530- try:
2531- yield self.revision
2532- self.revision = None
2533- except:
2534- self.flush(False)
2535- self.revision = None
2536- raise
2537- else:
2538- self.flush()
2539-
2540- def flush(self, save=True):
2541- if save:
2542- self.conn.commit()
2543- elif self._closed:
2544- return
2545- else:
2546- self.conn.rollback()
2547-
2548- def _init(self):
2549- self.cursor.execute('''
2550- create table if not exists kv (
2551- key text,
2552- data text,
2553- primary key (key)
2554- )''')
2555- self.cursor.execute('''
2556- create table if not exists kv_revisions (
2557- key text,
2558- revision integer,
2559- data text,
2560- primary key (key, revision)
2561- )''')
2562- self.cursor.execute('''
2563- create table if not exists hooks (
2564- version integer primary key autoincrement,
2565- hook text,
2566- date text
2567- )''')
2568- self.conn.commit()
2569-
2570- def gethistory(self, key, deserialize=False):
2571- self.cursor.execute(
2572- '''
2573- select kv.revision, kv.key, kv.data, h.hook, h.date
2574- from kv_revisions kv,
2575- hooks h
2576- where kv.key=?
2577- and kv.revision = h.version
2578- ''', [key])
2579- if deserialize is False:
2580- return self.cursor.fetchall()
2581- return map(_parse_history, self.cursor.fetchall())
2582-
2583- def debug(self, fh=sys.stderr):
2584- self.cursor.execute('select * from kv')
2585- pprint.pprint(self.cursor.fetchall(), stream=fh)
2586- self.cursor.execute('select * from kv_revisions')
2587- pprint.pprint(self.cursor.fetchall(), stream=fh)
2588-
2589-
2590-def _parse_history(d):
2591- return (d[0], d[1], json.loads(d[2]), d[3],
2592- datetime.datetime.strptime(d[-1], "%Y-%m-%dT%H:%M:%S.%f"))
2593-
2594-
2595-class HookData(object):
2596- """Simple integration for existing hook exec frameworks.
2597-
2598- Records all unit information, and stores deltas for processing
2599- by the hook.
2600-
2601- Sample::
2602-
2603- from charmhelper.core import hookenv, unitdata
2604-
2605- changes = unitdata.HookData()
2606- db = unitdata.kv()
2607- hooks = hookenv.Hooks()
2608-
2609- @hooks.hook
2610- def config_changed():
2611- # View all changes to configuration
2612- for changed, (prev, cur) in changes.conf.items():
2613- print('config changed', changed,
2614- 'previous value', prev,
2615- 'current value', cur)
2616-
2617- # Get some unit specific bookeeping
2618- if not db.get('pkg_key'):
2619- key = urllib.urlopen('https://example.com/pkg_key').read()
2620- db.set('pkg_key', key)
2621-
2622- if __name__ == '__main__':
2623- with changes():
2624- hook.execute()
2625-
2626- """
2627- def __init__(self):
2628- self.kv = kv()
2629- self.conf = None
2630- self.rels = None
2631-
2632- @contextlib.contextmanager
2633- def __call__(self):
2634- from charmhelpers.core import hookenv
2635- hook_name = hookenv.hook_name()
2636-
2637- with self.kv.hook_scope(hook_name):
2638- self._record_charm_version(hookenv.charm_dir())
2639- delta_config, delta_relation = self._record_hook(hookenv)
2640- yield self.kv, delta_config, delta_relation
2641-
2642- def _record_charm_version(self, charm_dir):
2643- # Record revisions.. charm revisions are meaningless
2644- # to charm authors as they don't control the revision.
2645- # so logic dependnent on revision is not particularly
2646- # useful, however it is useful for debugging analysis.
2647- charm_rev = open(
2648- os.path.join(charm_dir, 'revision')).read().strip()
2649- charm_rev = charm_rev or '0'
2650- revs = self.kv.get('charm_revisions', [])
2651- if charm_rev not in revs:
2652- revs.append(charm_rev.strip() or '0')
2653- self.kv.set('charm_revisions', revs)
2654-
2655- def _record_hook(self, hookenv):
2656- data = hookenv.execution_environment()
2657- self.conf = conf_delta = self.kv.delta(data['conf'], 'config')
2658- self.rels = rels_delta = self.kv.delta(data['rels'], 'rels')
2659- self.kv.set('env', data['env'])
2660- self.kv.set('unit', data['unit'])
2661- self.kv.set('relid', data.get('relid'))
2662- return conf_delta, rels_delta
2663-
2664-
2665-class Record(dict):
2666-
2667- __slots__ = ()
2668-
2669- def __getattr__(self, k):
2670- if k in self:
2671- return self[k]
2672- raise AttributeError(k)
2673-
2674-
2675-class DeltaSet(Record):
2676-
2677- __slots__ = ()
2678-
2679-
2680-Delta = collections.namedtuple('Delta', ['previous', 'current'])
2681-
2682-
2683-_KV = None
2684-
2685-
2686-def kv():
2687- global _KV
2688- if _KV is None:
2689- _KV = Storage()
2690- return _KV
2691
2692=== removed directory 'hooks/charmhelpers/fetch'
2693=== removed file 'hooks/charmhelpers/fetch/__init__.py'
2694--- hooks/charmhelpers/fetch/__init__.py 2015-02-19 16:21:31 +0000
2695+++ hooks/charmhelpers/fetch/__init__.py 1970-01-01 00:00:00 +0000
2696@@ -1,439 +0,0 @@
2697-# Copyright 2014-2015 Canonical Limited.
2698-#
2699-# This file is part of charm-helpers.
2700-#
2701-# charm-helpers is free software: you can redistribute it and/or modify
2702-# it under the terms of the GNU Lesser General Public License version 3 as
2703-# published by the Free Software Foundation.
2704-#
2705-# charm-helpers is distributed in the hope that it will be useful,
2706-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2707-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2708-# GNU Lesser General Public License for more details.
2709-#
2710-# You should have received a copy of the GNU Lesser General Public License
2711-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
2712-
2713-import importlib
2714-from tempfile import NamedTemporaryFile
2715-import time
2716-from yaml import safe_load
2717-from charmhelpers.core.host import (
2718- lsb_release
2719-)
2720-import subprocess
2721-from charmhelpers.core.hookenv import (
2722- config,
2723- log,
2724-)
2725-import os
2726-
2727-import six
2728-if six.PY3:
2729- from urllib.parse import urlparse, urlunparse
2730-else:
2731- from urlparse import urlparse, urlunparse
2732-
2733-
2734-CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
2735-deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
2736-"""
2737-PROPOSED_POCKET = """# Proposed
2738-deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
2739-"""
2740-CLOUD_ARCHIVE_POCKETS = {
2741- # Folsom
2742- 'folsom': 'precise-updates/folsom',
2743- 'precise-folsom': 'precise-updates/folsom',
2744- 'precise-folsom/updates': 'precise-updates/folsom',
2745- 'precise-updates/folsom': 'precise-updates/folsom',
2746- 'folsom/proposed': 'precise-proposed/folsom',
2747- 'precise-folsom/proposed': 'precise-proposed/folsom',
2748- 'precise-proposed/folsom': 'precise-proposed/folsom',
2749- # Grizzly
2750- 'grizzly': 'precise-updates/grizzly',
2751- 'precise-grizzly': 'precise-updates/grizzly',
2752- 'precise-grizzly/updates': 'precise-updates/grizzly',
2753- 'precise-updates/grizzly': 'precise-updates/grizzly',
2754- 'grizzly/proposed': 'precise-proposed/grizzly',
2755- 'precise-grizzly/proposed': 'precise-proposed/grizzly',
2756- 'precise-proposed/grizzly': 'precise-proposed/grizzly',
2757- # Havana
2758- 'havana': 'precise-updates/havana',
2759- 'precise-havana': 'precise-updates/havana',
2760- 'precise-havana/updates': 'precise-updates/havana',
2761- 'precise-updates/havana': 'precise-updates/havana',
2762- 'havana/proposed': 'precise-proposed/havana',
2763- 'precise-havana/proposed': 'precise-proposed/havana',
2764- 'precise-proposed/havana': 'precise-proposed/havana',
2765- # Icehouse
2766- 'icehouse': 'precise-updates/icehouse',
2767- 'precise-icehouse': 'precise-updates/icehouse',
2768- 'precise-icehouse/updates': 'precise-updates/icehouse',
2769- 'precise-updates/icehouse': 'precise-updates/icehouse',
2770- 'icehouse/proposed': 'precise-proposed/icehouse',
2771- 'precise-icehouse/proposed': 'precise-proposed/icehouse',
2772- 'precise-proposed/icehouse': 'precise-proposed/icehouse',
2773- # Juno
2774- 'juno': 'trusty-updates/juno',
2775- 'trusty-juno': 'trusty-updates/juno',
2776- 'trusty-juno/updates': 'trusty-updates/juno',
2777- 'trusty-updates/juno': 'trusty-updates/juno',
2778- 'juno/proposed': 'trusty-proposed/juno',
2779- 'trusty-juno/proposed': 'trusty-proposed/juno',
2780- 'trusty-proposed/juno': 'trusty-proposed/juno',
2781- # Kilo
2782- 'kilo': 'trusty-updates/kilo',
2783- 'trusty-kilo': 'trusty-updates/kilo',
2784- 'trusty-kilo/updates': 'trusty-updates/kilo',
2785- 'trusty-updates/kilo': 'trusty-updates/kilo',
2786- 'kilo/proposed': 'trusty-proposed/kilo',
2787- 'trusty-kilo/proposed': 'trusty-proposed/kilo',
2788- 'trusty-proposed/kilo': 'trusty-proposed/kilo',
2789-}
2790-
2791-# The order of this list is very important. Handlers should be listed in from
2792-# least- to most-specific URL matching.
2793-FETCH_HANDLERS = (
2794- 'charmhelpers.fetch.archiveurl.ArchiveUrlFetchHandler',
2795- 'charmhelpers.fetch.bzrurl.BzrUrlFetchHandler',
2796- 'charmhelpers.fetch.giturl.GitUrlFetchHandler',
2797-)
2798-
2799-APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT.
2800-APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks.
2801-APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times.
2802-
2803-
2804-class SourceConfigError(Exception):
2805- pass
2806-
2807-
2808-class UnhandledSource(Exception):
2809- pass
2810-
2811-
2812-class AptLockError(Exception):
2813- pass
2814-
2815-
2816-class BaseFetchHandler(object):
2817-
2818- """Base class for FetchHandler implementations in fetch plugins"""
2819-
2820- def can_handle(self, source):
2821- """Returns True if the source can be handled. Otherwise returns
2822- a string explaining why it cannot"""
2823- return "Wrong source type"
2824-
2825- def install(self, source):
2826- """Try to download and unpack the source. Return the path to the
2827- unpacked files or raise UnhandledSource."""
2828- raise UnhandledSource("Wrong source type {}".format(source))
2829-
2830- def parse_url(self, url):
2831- return urlparse(url)
2832-
2833- def base_url(self, url):
2834- """Return url without querystring or fragment"""
2835- parts = list(self.parse_url(url))
2836- parts[4:] = ['' for i in parts[4:]]
2837- return urlunparse(parts)
2838-
2839-
2840-def filter_installed_packages(packages):
2841- """Returns a list of packages that require installation"""
2842- cache = apt_cache()
2843- _pkgs = []
2844- for package in packages:
2845- try:
2846- p = cache[package]
2847- p.current_ver or _pkgs.append(package)
2848- except KeyError:
2849- log('Package {} has no installation candidate.'.format(package),
2850- level='WARNING')
2851- _pkgs.append(package)
2852- return _pkgs
2853-
2854-
2855-def apt_cache(in_memory=True):
2856- """Build and return an apt cache"""
2857- import apt_pkg
2858- apt_pkg.init()
2859- if in_memory:
2860- apt_pkg.config.set("Dir::Cache::pkgcache", "")
2861- apt_pkg.config.set("Dir::Cache::srcpkgcache", "")
2862- return apt_pkg.Cache()
2863-
2864-
2865-def apt_install(packages, options=None, fatal=False):
2866- """Install one or more packages"""
2867- if options is None:
2868- options = ['--option=Dpkg::Options::=--force-confold']
2869-
2870- cmd = ['apt-get', '--assume-yes']
2871- cmd.extend(options)
2872- cmd.append('install')
2873- if isinstance(packages, six.string_types):
2874- cmd.append(packages)
2875- else:
2876- cmd.extend(packages)
2877- log("Installing {} with options: {}".format(packages,
2878- options))
2879- _run_apt_command(cmd, fatal)
2880-
2881-
2882-def apt_upgrade(options=None, fatal=False, dist=False):
2883- """Upgrade all packages"""
2884- if options is None:
2885- options = ['--option=Dpkg::Options::=--force-confold']
2886-
2887- cmd = ['apt-get', '--assume-yes']
2888- cmd.extend(options)
2889- if dist:
2890- cmd.append('dist-upgrade')
2891- else:
2892- cmd.append('upgrade')
2893- log("Upgrading with options: {}".format(options))
2894- _run_apt_command(cmd, fatal)
2895-
2896-
2897-def apt_update(fatal=False):
2898- """Update local apt cache"""
2899- cmd = ['apt-get', 'update']
2900- _run_apt_command(cmd, fatal)
2901-
2902-
2903-def apt_purge(packages, fatal=False):
2904- """Purge one or more packages"""
2905- cmd = ['apt-get', '--assume-yes', 'purge']
2906- if isinstance(packages, six.string_types):
2907- cmd.append(packages)
2908- else:
2909- cmd.extend(packages)
2910- log("Purging {}".format(packages))
2911- _run_apt_command(cmd, fatal)
2912-
2913-
2914-def apt_hold(packages, fatal=False):
2915- """Hold one or more packages"""
2916- cmd = ['apt-mark', 'hold']
2917- if isinstance(packages, six.string_types):
2918- cmd.append(packages)
2919- else:
2920- cmd.extend(packages)
2921- log("Holding {}".format(packages))
2922-
2923- if fatal:
2924- subprocess.check_call(cmd)
2925- else:
2926- subprocess.call(cmd)
2927-
2928-
2929-def add_source(source, key=None):
2930- """Add a package source to this system.
2931-
2932- @param source: a URL or sources.list entry, as supported by
2933- add-apt-repository(1). Examples::
2934-
2935- ppa:charmers/example
2936- deb https://stub:key@private.example.com/ubuntu trusty main
2937-
2938- In addition:
2939- 'proposed:' may be used to enable the standard 'proposed'
2940- pocket for the release.
2941- 'cloud:' may be used to activate official cloud archive pockets,
2942- such as 'cloud:icehouse'
2943- 'distro' may be used as a noop
2944-
2945- @param key: A key to be added to the system's APT keyring and used
2946- to verify the signatures on packages. Ideally, this should be an
2947- ASCII format GPG public key including the block headers. A GPG key
2948- id may also be used, but be aware that only insecure protocols are
2949- available to retrieve the actual public key from a public keyserver
2950- placing your Juju environment at risk. ppa and cloud archive keys
2951- are securely added automtically, so sould not be provided.
2952- """
2953- if source is None:
2954- log('Source is not present. Skipping')
2955- return
2956-
2957- if (source.startswith('ppa:') or
2958- source.startswith('http') or
2959- source.startswith('deb ') or
2960- source.startswith('cloud-archive:')):
2961- subprocess.check_call(['add-apt-repository', '--yes', source])
2962- elif source.startswith('cloud:'):
2963- apt_install(filter_installed_packages(['ubuntu-cloud-keyring']),
2964- fatal=True)
2965- pocket = source.split(':')[-1]
2966- if pocket not in CLOUD_ARCHIVE_POCKETS:
2967- raise SourceConfigError(
2968- 'Unsupported cloud: source option %s' %
2969- pocket)
2970- actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket]
2971- with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
2972- apt.write(CLOUD_ARCHIVE.format(actual_pocket))
2973- elif source == 'proposed':
2974- release = lsb_release()['DISTRIB_CODENAME']
2975- with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
2976- apt.write(PROPOSED_POCKET.format(release))
2977- elif source == 'distro':
2978- pass
2979- else:
2980- log("Unknown source: {!r}".format(source))
2981-
2982- if key:
2983- if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key:
2984- with NamedTemporaryFile('w+') as key_file:
2985- key_file.write(key)
2986- key_file.flush()
2987- key_file.seek(0)
2988- subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file)
2989- else:
2990- # Note that hkp: is in no way a secure protocol. Using a
2991- # GPG key id is pointless from a security POV unless you
2992- # absolutely trust your network and DNS.
2993- subprocess.check_call(['apt-key', 'adv', '--keyserver',
2994- 'hkp://keyserver.ubuntu.com:80', '--recv',
2995- key])
2996-
2997-
2998-def configure_sources(update=False,
2999- sources_var='install_sources',
3000- keys_var='install_keys'):
3001- """
3002- Configure multiple sources from charm configuration.
3003-
3004- The lists are encoded as yaml fragments in the configuration.
3005- The frament needs to be included as a string. Sources and their
3006- corresponding keys are of the types supported by add_source().
3007-
3008- Example config:
3009- install_sources: |
3010- - "ppa:foo"
3011- - "http://example.com/repo precise main"
3012- install_keys: |
3013- - null
3014- - "a1b2c3d4"
3015-
3016- Note that 'null' (a.k.a. None) should not be quoted.
3017- """
3018- sources = safe_load((config(sources_var) or '').strip()) or []
3019- keys = safe_load((config(keys_var) or '').strip()) or None
3020-
3021- if isinstance(sources, six.string_types):
3022- sources = [sources]
3023-
3024- if keys is None:
3025- for source in sources:
3026- add_source(source, None)
3027- else:
3028- if isinstance(keys, six.string_types):
3029- keys = [keys]
3030-
3031- if len(sources) != len(keys):
3032- raise SourceConfigError(
3033- 'Install sources and keys lists are different lengths')
3034- for source, key in zip(sources, keys):
3035- add_source(source, key)
3036- if update:
3037- apt_update(fatal=True)
3038-
3039-
3040-def install_remote(source, *args, **kwargs):
3041- """
3042- Install a file tree from a remote source
3043-
3044- The specified source should be a url of the form:
3045- scheme://[host]/path[#[option=value][&...]]
3046-
3047- Schemes supported are based on this modules submodules.
3048- Options supported are submodule-specific.
3049- Additional arguments are passed through to the submodule.
3050-
3051- For example::
3052-
3053- dest = install_remote('http://example.com/archive.tgz',
3054- checksum='deadbeef',
3055- hash_type='sha1')
3056-
3057- This will download `archive.tgz`, validate it using SHA1 and, if
3058- the file is ok, extract it and return the directory in which it
3059- was extracted. If the checksum fails, it will raise
3060- :class:`charmhelpers.core.host.ChecksumError`.
3061- """
3062- # We ONLY check for True here because can_handle may return a string
3063- # explaining why it can't handle a given source.
3064- handlers = [h for h in plugins() if h.can_handle(source) is True]
3065- installed_to = None
3066- for handler in handlers:
3067- try:
3068- installed_to = handler.install(source, *args, **kwargs)
3069- except UnhandledSource:
3070- pass
3071- if not installed_to:
3072- raise UnhandledSource("No handler found for source {}".format(source))
3073- return installed_to
3074-
3075-
3076-def install_from_config(config_var_name):
3077- charm_config = config()
3078- source = charm_config[config_var_name]
3079- return install_remote(source)
3080-
3081-
3082-def plugins(fetch_handlers=None):
3083- if not fetch_handlers:
3084- fetch_handlers = FETCH_HANDLERS
3085- plugin_list = []
3086- for handler_name in fetch_handlers:
3087- package, classname = handler_name.rsplit('.', 1)
3088- try:
3089- handler_class = getattr(
3090- importlib.import_module(package),
3091- classname)
3092- plugin_list.append(handler_class())
3093- except (ImportError, AttributeError):
3094- # Skip missing plugins so that they can be ommitted from
3095- # installation if desired
3096- log("FetchHandler {} not found, skipping plugin".format(
3097- handler_name))
3098- return plugin_list
3099-
3100-
3101-def _run_apt_command(cmd, fatal=False):
3102- """
3103- Run an APT command, checking output and retrying if the fatal flag is set
3104- to True.
3105-
3106- :param: cmd: str: The apt command to run.
3107- :param: fatal: bool: Whether the command's output should be checked and
3108- retried.
3109- """
3110- env = os.environ.copy()
3111-
3112- if 'DEBIAN_FRONTEND' not in env:
3113- env['DEBIAN_FRONTEND'] = 'noninteractive'
3114-
3115- if fatal:
3116- retry_count = 0
3117- result = None
3118-
3119- # If the command is considered "fatal", we need to retry if the apt
3120- # lock was not acquired.
3121-
3122- while result is None or result == APT_NO_LOCK:
3123- try:
3124- result = subprocess.check_call(cmd, env=env)
3125- except subprocess.CalledProcessError as e:
3126- retry_count = retry_count + 1
3127- if retry_count > APT_NO_LOCK_RETRY_COUNT:
3128- raise
3129- result = e.returncode
3130- log("Couldn't acquire DPKG lock. Will retry in {} seconds."
3131- "".format(APT_NO_LOCK_RETRY_DELAY))
3132- time.sleep(APT_NO_LOCK_RETRY_DELAY)
3133-
3134- else:
3135- subprocess.call(cmd, env=env)
3136
3137=== removed file 'hooks/charmhelpers/fetch/archiveurl.py'
3138--- hooks/charmhelpers/fetch/archiveurl.py 2015-02-19 16:21:31 +0000
3139+++ hooks/charmhelpers/fetch/archiveurl.py 1970-01-01 00:00:00 +0000
3140@@ -1,161 +0,0 @@
3141-# Copyright 2014-2015 Canonical Limited.
3142-#
3143-# This file is part of charm-helpers.
3144-#
3145-# charm-helpers is free software: you can redistribute it and/or modify
3146-# it under the terms of the GNU Lesser General Public License version 3 as
3147-# published by the Free Software Foundation.
3148-#
3149-# charm-helpers is distributed in the hope that it will be useful,
3150-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3151-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3152-# GNU Lesser General Public License for more details.
3153-#
3154-# You should have received a copy of the GNU Lesser General Public License
3155-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3156-
3157-import os
3158-import hashlib
3159-import re
3160-
3161-from charmhelpers.fetch import (
3162- BaseFetchHandler,
3163- UnhandledSource
3164-)
3165-from charmhelpers.payload.archive import (
3166- get_archive_handler,
3167- extract,
3168-)
3169-from charmhelpers.core.host import mkdir, check_hash
3170-
3171-import six
3172-if six.PY3:
3173- from urllib.request import (
3174- build_opener, install_opener, urlopen, urlretrieve,
3175- HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
3176- )
3177- from urllib.parse import urlparse, urlunparse, parse_qs
3178- from urllib.error import URLError
3179-else:
3180- from urllib import urlretrieve
3181- from urllib2 import (
3182- build_opener, install_opener, urlopen,
3183- HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler,
3184- URLError
3185- )
3186- from urlparse import urlparse, urlunparse, parse_qs
3187-
3188-
3189-def splituser(host):
3190- '''urllib.splituser(), but six's support of this seems broken'''
3191- _userprog = re.compile('^(.*)@(.*)$')
3192- match = _userprog.match(host)
3193- if match:
3194- return match.group(1, 2)
3195- return None, host
3196-
3197-
3198-def splitpasswd(user):
3199- '''urllib.splitpasswd(), but six's support of this is missing'''
3200- _passwdprog = re.compile('^([^:]*):(.*)$', re.S)
3201- match = _passwdprog.match(user)
3202- if match:
3203- return match.group(1, 2)
3204- return user, None
3205-
3206-
3207-class ArchiveUrlFetchHandler(BaseFetchHandler):
3208- """
3209- Handler to download archive files from arbitrary URLs.
3210-
3211- Can fetch from http, https, ftp, and file URLs.
3212-
3213- Can install either tarballs (.tar, .tgz, .tbz2, etc) or zip files.
3214-
3215- Installs the contents of the archive in $CHARM_DIR/fetched/.
3216- """
3217- def can_handle(self, source):
3218- url_parts = self.parse_url(source)
3219- if url_parts.scheme not in ('http', 'https', 'ftp', 'file'):
3220- return "Wrong source type"
3221- if get_archive_handler(self.base_url(source)):
3222- return True
3223- return False
3224-
3225- def download(self, source, dest):
3226- """
3227- Download an archive file.
3228-
3229- :param str source: URL pointing to an archive file.
3230- :param str dest: Local path location to download archive file to.
3231- """
3232- # propogate all exceptions
3233- # URLError, OSError, etc
3234- proto, netloc, path, params, query, fragment = urlparse(source)
3235- if proto in ('http', 'https'):
3236- auth, barehost = splituser(netloc)
3237- if auth is not None:
3238- source = urlunparse((proto, barehost, path, params, query, fragment))
3239- username, password = splitpasswd(auth)
3240- passman = HTTPPasswordMgrWithDefaultRealm()
3241- # Realm is set to None in add_password to force the username and password
3242- # to be used whatever the realm
3243- passman.add_password(None, source, username, password)
3244- authhandler = HTTPBasicAuthHandler(passman)
3245- opener = build_opener(authhandler)
3246- install_opener(opener)
3247- response = urlopen(source)
3248- try:
3249- with open(dest, 'w') as dest_file:
3250- dest_file.write(response.read())
3251- except Exception as e:
3252- if os.path.isfile(dest):
3253- os.unlink(dest)
3254- raise e
3255-
3256- # Mandatory file validation via Sha1 or MD5 hashing.
3257- def download_and_validate(self, url, hashsum, validate="sha1"):
3258- tempfile, headers = urlretrieve(url)
3259- check_hash(tempfile, hashsum, validate)
3260- return tempfile
3261-
3262- def install(self, source, dest=None, checksum=None, hash_type='sha1'):
3263- """
3264- Download and install an archive file, with optional checksum validation.
3265-
3266- The checksum can also be given on the `source` URL's fragment.
3267- For example::
3268-
3269- handler.install('http://example.com/file.tgz#sha1=deadbeef')
3270-
3271- :param str source: URL pointing to an archive file.
3272- :param str dest: Local destination path to install to. If not given,
3273- installs to `$CHARM_DIR/archives/archive_file_name`.
3274- :param str checksum: If given, validate the archive file after download.
3275- :param str hash_type: Algorithm used to generate `checksum`.
3276- Can be any hash alrgorithm supported by :mod:`hashlib`,
3277- such as md5, sha1, sha256, sha512, etc.
3278-
3279- """
3280- url_parts = self.parse_url(source)
3281- dest_dir = os.path.join(os.environ.get('CHARM_DIR'), 'fetched')
3282- if not os.path.exists(dest_dir):
3283- mkdir(dest_dir, perms=0o755)
3284- dld_file = os.path.join(dest_dir, os.path.basename(url_parts.path))
3285- try:
3286- self.download(source, dld_file)
3287- except URLError as e:
3288- raise UnhandledSource(e.reason)
3289- except OSError as e:
3290- raise UnhandledSource(e.strerror)
3291- options = parse_qs(url_parts.fragment)
3292- for key, value in options.items():
3293- if not six.PY3:
3294- algorithms = hashlib.algorithms
3295- else:
3296- algorithms = hashlib.algorithms_available
3297- if key in algorithms:
3298- check_hash(dld_file, value, key)
3299- if checksum:
3300- check_hash(dld_file, checksum, hash_type)
3301- return extract(dld_file, dest)
3302
3303=== removed file 'hooks/charmhelpers/fetch/bzrurl.py'
3304--- hooks/charmhelpers/fetch/bzrurl.py 2015-02-19 16:21:31 +0000
3305+++ hooks/charmhelpers/fetch/bzrurl.py 1970-01-01 00:00:00 +0000
3306@@ -1,78 +0,0 @@
3307-# Copyright 2014-2015 Canonical Limited.
3308-#
3309-# This file is part of charm-helpers.
3310-#
3311-# charm-helpers is free software: you can redistribute it and/or modify
3312-# it under the terms of the GNU Lesser General Public License version 3 as
3313-# published by the Free Software Foundation.
3314-#
3315-# charm-helpers is distributed in the hope that it will be useful,
3316-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3317-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3318-# GNU Lesser General Public License for more details.
3319-#
3320-# You should have received a copy of the GNU Lesser General Public License
3321-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3322-
3323-import os
3324-from charmhelpers.fetch import (
3325- BaseFetchHandler,
3326- UnhandledSource
3327-)
3328-from charmhelpers.core.host import mkdir
3329-
3330-import six
3331-if six.PY3:
3332- raise ImportError('bzrlib does not support Python3')
3333-
3334-try:
3335- from bzrlib.branch import Branch
3336- from bzrlib import bzrdir, workingtree, errors
3337-except ImportError:
3338- from charmhelpers.fetch import apt_install
3339- apt_install("python-bzrlib")
3340- from bzrlib.branch import Branch
3341- from bzrlib import bzrdir, workingtree, errors
3342-
3343-
3344-class BzrUrlFetchHandler(BaseFetchHandler):
3345- """Handler for bazaar branches via generic and lp URLs"""
3346- def can_handle(self, source):
3347- url_parts = self.parse_url(source)
3348- if url_parts.scheme not in ('bzr+ssh', 'lp'):
3349- return False
3350- else:
3351- return True
3352-
3353- def branch(self, source, dest):
3354- url_parts = self.parse_url(source)
3355- # If we use lp:branchname scheme we need to load plugins
3356- if not self.can_handle(source):
3357- raise UnhandledSource("Cannot handle {}".format(source))
3358- if url_parts.scheme == "lp":
3359- from bzrlib.plugin import load_plugins
3360- load_plugins()
3361- try:
3362- local_branch = bzrdir.BzrDir.create_branch_convenience(dest)
3363- except errors.AlreadyControlDirError:
3364- local_branch = Branch.open(dest)
3365- try:
3366- remote_branch = Branch.open(source)
3367- remote_branch.push(local_branch)
3368- tree = workingtree.WorkingTree.open(dest)
3369- tree.update()
3370- except Exception as e:
3371- raise e
3372-
3373- def install(self, source):
3374- url_parts = self.parse_url(source)
3375- branch_name = url_parts.path.strip("/").split("/")[-1]
3376- dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
3377- branch_name)
3378- if not os.path.exists(dest_dir):
3379- mkdir(dest_dir, perms=0o755)
3380- try:
3381- self.branch(source, dest_dir)
3382- except OSError as e:
3383- raise UnhandledSource(e.strerror)
3384- return dest_dir
3385
3386=== removed file 'hooks/charmhelpers/fetch/giturl.py'
3387--- hooks/charmhelpers/fetch/giturl.py 2015-02-19 16:21:31 +0000
3388+++ hooks/charmhelpers/fetch/giturl.py 1970-01-01 00:00:00 +0000
3389@@ -1,71 +0,0 @@
3390-# Copyright 2014-2015 Canonical Limited.
3391-#
3392-# This file is part of charm-helpers.
3393-#
3394-# charm-helpers is free software: you can redistribute it and/or modify
3395-# it under the terms of the GNU Lesser General Public License version 3 as
3396-# published by the Free Software Foundation.
3397-#
3398-# charm-helpers is distributed in the hope that it will be useful,
3399-# but WITHOUT ANY WARRANTY; without even the implied warranty of
3400-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3401-# GNU Lesser General Public License for more details.
3402-#
3403-# You should have received a copy of the GNU Lesser General Public License
3404-# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3405-
3406-import os
3407-from charmhelpers.fetch import (
3408- BaseFetchHandler,
3409- UnhandledSource
3410-)
3411-from charmhelpers.core.host import mkdir
3412-
3413-import six
3414-if six.PY3:
3415- raise ImportError('GitPython does not support Python 3')
3416-
3417-try:
3418- from git import Repo
3419-except ImportError:
3420- from charmhelpers.fetch import apt_install
3421- apt_install("python-git")
3422- from git import Repo
3423-
3424-from git.exc import GitCommandError # noqa E402
3425-
3426-
3427-class GitUrlFetchHandler(BaseFetchHandler):
3428- """Handler for git branches via generic and github URLs"""
3429- def can_handle(self, source):
3430- url_parts = self.parse_url(source)
3431- # TODO (mattyw) no support for ssh git@ yet
3432- if url_parts.scheme not in ('http', 'https', 'git'):
3433- return False
3434- else:
3435- return True
3436-
3437- def clone(self, source, dest, branch):
3438- if not self.can_handle(source):
3439- raise UnhandledSource("Cannot handle {}".format(source))
3440-
3441- repo = Repo.clone_from(source, dest)
3442- repo.git.checkout(branch)
3443-
3444- def install(self, source, branch="master", dest=None):
3445- url_parts = self.parse_url(source)
3446- branch_name = url_parts.path.strip("/").split("/")[-1]
3447- if dest:
3448- dest_dir = os.path.join(dest, branch_name)
3449- else:
3450- dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched",
3451- branch_name)
3452- if not os.path.exists(dest_dir):
3453- mkdir(dest_dir, perms=0o755)
3454- try:
3455- self.clone(source, dest_dir, branch)
3456- except GitCommandError as e:
3457- raise UnhandledSource(e.message)
3458- except OSError as e:
3459- raise UnhandledSource(e.strerror)
3460- return dest_dir
3461
3462=== removed file 'hooks/config-changed'
3463--- hooks/config-changed 2015-02-19 21:19:01 +0000
3464+++ hooks/config-changed 1970-01-01 00:00:00 +0000
3465@@ -1,43 +0,0 @@
3466-#!/usr/bin/python
3467-
3468-#
3469-# Copyright 2015 Canonical Ltd.
3470-#
3471-# Authors:
3472-# Liang Chen <liang.chen@ubuntu.com>
3473-#
3474-import subprocess
3475-import sys
3476-
3477-from charmhelpers.core.hookenv import (
3478- config,
3479- Hooks, UnregisteredHookError,
3480- log, ERROR,
3481-)
3482-
3483-from utils import render_template
3484-
3485-hooks = Hooks()
3486-
3487-
3488-def emit_lxc_br_conf():
3489- lxc_context = {}
3490- if config('new-lxc-network'):
3491- lxc_context['new_network'] = True
3492-
3493- lxc_bridge_conf = "/etc/network/interfaces.d/lxcbr0.cfg"
3494- with open(lxc_bridge_conf, 'w') as lxc_br_conf:
3495- lxc_br_conf.write(render_template('lxc-bridge.conf', lxc_context))
3496-
3497-
3498-@hooks.hook('config-changed')
3499-def config_changed():
3500- emit_lxc_br_conf()
3501- cmd = ['ifup', 'lxcbr0']
3502- subprocess.check_call(cmd)
3503-
3504-if __name__ == '__main__':
3505- try:
3506- hooks.execute(sys.argv)
3507- except UnregisteredHookError as e:
3508- log('Unknown hook {} - skipping.'.format(e))
3509
3510=== removed file 'hooks/hooks.py'
3511--- hooks/hooks.py 2015-02-19 21:19:01 +0000
3512+++ hooks/hooks.py 1970-01-01 00:00:00 +0000
3513@@ -1,45 +0,0 @@
3514-#!/usr/bin/python
3515-
3516-#
3517-# Copyright 2015 Canonical Ltd.
3518-#
3519-# Authors:
3520-# Liang Chen <liang.chen@ubuntu.com>
3521-#
3522-
3523-import subprocess
3524-import sys
3525-
3526-from charmhelpers.core.hookenv import (
3527- config,
3528- Hooks, UnregisteredHookError,
3529- log,
3530-)
3531-
3532-from utils import render_template
3533-
3534-hooks = Hooks()
3535-
3536-
3537-def emit_lxc_br_conf():
3538- lxc_context = {}
3539- if config('new-lxc-network'):
3540- lxc_context['new_network'] = True
3541-
3542- lxc_bridge_conf = "/etc/network/interfaces.d/lxcbr0.cfg"
3543- with open(lxc_bridge_conf, 'w') as lxc_br_conf:
3544- lxc_br_conf.write(render_template('lxc-bridge.conf', lxc_context))
3545-
3546-
3547-@hooks.hook('config-changed')
3548-def config_changed():
3549- emit_lxc_br_conf()
3550- cmd = ['ifup', 'lxcbr0']
3551- subprocess.check_call(cmd)
3552-
3553-
3554-if __name__ == '__main__':
3555- try:
3556- hooks.execute(sys.argv)
3557- except UnregisteredHookError as e:
3558- log('Unknown hook {} - skipping.'.format(e))
3559
3560=== removed file 'hooks/install'
3561--- hooks/install 2012-05-03 22:17:19 +0000
3562+++ hooks/install 1970-01-01 00:00:00 +0000
3563@@ -1,2 +0,0 @@
3564-#!/bin/bash
3565-# Does nothing. - charm proof requires an install hook.
3566
3567=== removed file 'hooks/utils.py'
3568--- hooks/utils.py 2015-02-19 21:19:01 +0000
3569+++ hooks/utils.py 1970-01-01 00:00:00 +0000
3570@@ -1,27 +0,0 @@
3571-#
3572-# Copyright 2015 Canonical Ltd.
3573-#
3574-# Authors:
3575-# Liang Chen <liang.chen@ubuntu.com>
3576-#
3577-
3578-TEMPLATES_DIR = 'templates'
3579-
3580-from charmhelpers.fetch import (
3581- apt_install,
3582- filter_installed_packages
3583-)
3584-
3585-try:
3586- import jinja2
3587-except ImportError:
3588- apt_install(filter_installed_packages(['python-jinja2']),
3589- fatal=True)
3590- import jinja2
3591-
3592-
3593-def render_template(template_name, context, template_dir=TEMPLATES_DIR):
3594- templates = jinja2.Environment(
3595- loader=jinja2.FileSystemLoader(template_dir))
3596- template = templates.get_template(template_name)
3597- return template.render(context)
3598
3599=== modified file 'metadata.yaml'
3600--- metadata.yaml 2013-12-11 21:28:41 +0000
3601+++ metadata.yaml 2015-04-30 13:14:23 +0000
3602@@ -3,5 +3,5 @@
3603 maintainer: Charmers <juju@lists.ubuntu.com>
3604 description: |
3605 This simply deploys Ubuntu Server.
3606-categories:
3607+tags:
3608 - misc
3609
3610=== removed directory 'templates'
3611=== removed file 'templates/lxc-bridge.conf'
3612--- templates/lxc-bridge.conf 2015-02-19 21:19:01 +0000
3613+++ templates/lxc-bridge.conf 1970-01-01 00:00:00 +0000
3614@@ -1,10 +0,0 @@
3615-
3616-{% if new_network -%}
3617-auto eth0
3618-iface eth0 inet manual
3619-
3620-auto lxcbr0
3621-iface lxcbr0 inet dhcp
3622- bridge_ports eth0
3623-
3624-{% endif -%}
3625
3626=== modified file 'tests/00-setup.sh'
3627--- tests/00-setup.sh 2015-02-19 17:29:32 +0000
3628+++ tests/00-setup.sh 2015-04-30 13:14:23 +0000
3629@@ -1,15 +1,9 @@
3630-#!/bin/bash
3631-
3632-# This script runs first to set up the environment for the tests.
3633-
3634-set -x
3635-
3636-# Check if amulet is installed before adding repository and updating apt-get.
3637+#!/bin/bash -x
3638+
3639+# Install amulet if not already installed.
3640 dpkg -s amulet
3641 if [ $? -ne 0 ]; then
3642 sudo add-apt-repository -y ppa:juju/stable
3643- sudo apt-get update -qq
3644+ sudo apt-get update -qq
3645 sudo apt-get install -y amulet
3646 fi
3647-# Install any additional packages needed for tests here.
3648-
3649
3650=== added file 'tests/010_basic_precise'
3651--- tests/010_basic_precise 1970-01-01 00:00:00 +0000
3652+++ tests/010_basic_precise 2015-04-30 13:14:23 +0000
3653@@ -0,0 +1,7 @@
3654+#!/usr/bin/python
3655+"""Amulet tests on a basic ubuntu charm deployment on precise."""
3656+
3657+from basic_deployment import ubuntu_basic_deployment
3658+
3659+if __name__ == '__main__':
3660+ ubuntu_basic_deployment(series='precise')
3661
3662=== added file 'tests/015_basic_trusty'
3663--- tests/015_basic_trusty 1970-01-01 00:00:00 +0000
3664+++ tests/015_basic_trusty 2015-04-30 13:14:23 +0000
3665@@ -0,0 +1,7 @@
3666+#!/usr/bin/python
3667+"""Amulet tests on a basic ubuntu charm deployment on trusty."""
3668+
3669+from basic_deployment import ubuntu_basic_deployment
3670+
3671+if __name__ == '__main__':
3672+ ubuntu_basic_deployment(series='trusty')
3673
3674=== added file 'tests/020_basic_utopic'
3675--- tests/020_basic_utopic 1970-01-01 00:00:00 +0000
3676+++ tests/020_basic_utopic 2015-04-30 13:14:23 +0000
3677@@ -0,0 +1,7 @@
3678+#!/usr/bin/python
3679+"""Amulet tests on a basic ubuntu charm deployment on utopic."""
3680+
3681+from basic_deployment import ubuntu_basic_deployment
3682+
3683+if __name__ == '__main__':
3684+ ubuntu_basic_deployment(series='utopic')
3685
3686=== added file 'tests/025_basic_vivid'
3687--- tests/025_basic_vivid 1970-01-01 00:00:00 +0000
3688+++ tests/025_basic_vivid 2015-04-30 13:14:23 +0000
3689@@ -0,0 +1,7 @@
3690+#!/usr/bin/python
3691+"""Amulet tests on a basic ubuntu charm deployment on trusty."""
3692+
3693+from basic_deployment import ubuntu_basic_deployment
3694+
3695+if __name__ == '__main__':
3696+ ubuntu_basic_deployment(series='vivid')
3697
3698=== removed file 'tests/10-deploy-test.py'
3699--- tests/10-deploy-test.py 2015-03-27 15:26:53 +0000
3700+++ tests/10-deploy-test.py 1970-01-01 00:00:00 +0000
3701@@ -1,90 +0,0 @@
3702-#!/usr/bin/python3
3703-
3704-# This Amulet based tests
3705-# The goal is to ensure the Ubuntu charm
3706-# sucessfully deploys and can be accessed.
3707-# Note the Ubuntu charm does not have any
3708-# relations or config options.
3709-
3710-import amulet
3711-
3712-# Timeout value, in seconds to deploy the environment
3713-seconds = 900
3714-series = 'trusty'
3715-
3716-# Set up the deployer module to interact and set up the environment.
3717-d = amulet.Deployment(series=series)
3718-
3719-# Define the environment in terms of charms, their config, and relations.
3720-
3721-# Add the Ubuntu charm to the deployment.
3722-d.add('ubuntu')
3723-
3724-# Deploy the environment currently defined
3725-try:
3726- # Wait the defined about amount of time to deploy the environment.
3727- # Setup makes sure the services are deployed, related, and in a
3728- # "started" state.
3729- d.setup(timeout=seconds)
3730- # Use a sentry to ensure there are no remaining hooks being execute
3731- # on any of the nodes
3732-except amulet.helpers.TimeoutError:
3733- # Pending the configuration the test will fail or be skipped
3734- # if not deployed properly.
3735- error_message = 'The environment did not deploy in %d seconds.' % seconds
3736- amulet.raise_status(amulet.SKIP, msg=error_message)
3737-except:
3738- # Something else has gone wrong, raise the error so we can see it and this
3739- # will automatically "FAIL" the test.
3740- raise
3741-
3742-# Access the Ubuntu instance to ensure it has been deployed correctly
3743-
3744-# Define the commands to be ran
3745-lsb_command = 'lsb_release -cs'
3746-uname_command = 'uname -a'
3747-lxc_interface_command = 'ifconfig -s lxcbr0'
3748-
3749-print(lsb_command)
3750-# Print the release information
3751-output, code = d.sentry.unit['ubuntu/0'].run(lsb_command)
3752-print(output)
3753-# Confirm the lsb_release command ran successfully
3754-if (code != 0):
3755- message = 'The ' + lsb_command + ' did not return the expected return code of 0.'
3756- amulet.raise_status(amulet.FAIL, msg=message)
3757-else:
3758- if series in output:
3759- print('The series is correct.')
3760- else:
3761- message = 'The series is not correct.'
3762- print(message)
3763- amulet.raise_status(amulet.FAIL, msg=message)
3764-
3765-print(uname_command)
3766-
3767-# Get the uname -a output
3768-output, code = d.sentry.unit['ubuntu/0'].run(uname_command)
3769-print(output)
3770-# Confirm the uname command was ran successfully
3771-if (code != 0):
3772- message = 'The ' + uname_command + ' did not return the expected return code of 0.'
3773- amulet.raise_status(amulet.FAIL, msg=message)
3774-else:
3775- message = 'The ' + uname_command + ' successfully executed.'
3776- print(output)
3777- print(message)
3778-
3779-# Test creating a new network for LXC via new-lxc-network config option.
3780-# Set the new-lxc-network config option to create a new network for LXC.
3781-d.configure('ubuntu', {'new-lxc-network': 'true'})
3782-# Confirm the the lxc bridge interface was sucessfully created
3783-output, code = d.sentry.unit['ubuntu/0'].run(lxc_interface_command)
3784-if (code != 0):
3785- error_message = 'The lxcbr0 interface was not created.'
3786- print(output)
3787- amulet.raise_status(amulet.FAIL, msg=error_message)
3788-else:
3789- message = 'The lxcbr0 interface sucessfully created.'
3790- print(output)
3791- print(message)
3792
3793=== added file 'tests/basic_deployment.py'
3794--- tests/basic_deployment.py 1970-01-01 00:00:00 +0000
3795+++ tests/basic_deployment.py 2015-04-30 13:14:23 +0000
3796@@ -0,0 +1,49 @@
3797+#!/usr/bin/python3
3798+"""
3799+Ubuntu charm functional test. Take note that the Ubuntu
3800+charm does not have any relations or config options
3801+to exercise.
3802+"""
3803+
3804+import amulet
3805+from charmhelpers.contrib.amulet.utils import AmuletUtils
3806+import logging
3807+
3808+
3809+def ubuntu_basic_deployment(series):
3810+ """ Common test routines to run per-series. """
3811+
3812+ # Initialize
3813+ seconds = 900
3814+ u = AmuletUtils(logging.DEBUG)
3815+ d = amulet.Deployment(series=series)
3816+ d.add('ubuntu')
3817+ unit = 'ubuntu/0'
3818+ lsb_command = 'lsb_release -cs'
3819+
3820+ # Deploy services, wait for started state. Fail or skip on timeout.
3821+ try:
3822+ d.setup(timeout=seconds)
3823+ except amulet.helpers.TimeoutError:
3824+ message = 'Deployment timed out ({}s)'.format(seconds)
3825+ amulet.raise_status(amulet.SKIP, msg=message)
3826+ except:
3827+ raise
3828+
3829+ # Confirm Ubuntu release name from the unit.
3830+ u.log.debug('Command: {}'.format(lsb_command))
3831+ output, code = d.sentry.unit[unit].run(lsb_command)
3832+ u.log.debug('Output: {}'.format(output))
3833+
3834+ if (code != 0):
3835+ message = 'Command FAIL: {}'.format(lsb_command)
3836+ u.log.error(message)
3837+ amulet.raise_status(amulet.FAIL, msg=message)
3838+ else:
3839+ if series in output:
3840+ message = 'Series: OK'
3841+ u.log.info(message)
3842+ else:
3843+ message = 'Series: FAIL ({})'.format(output)
3844+ u.log.error(message)
3845+ amulet.raise_status(amulet.FAIL, msg=message)
3846
3847=== added directory 'tests/charmhelpers'
3848=== added file 'tests/charmhelpers/__init__.py'
3849--- tests/charmhelpers/__init__.py 1970-01-01 00:00:00 +0000
3850+++ tests/charmhelpers/__init__.py 2015-04-30 13:14:23 +0000
3851@@ -0,0 +1,38 @@
3852+# Copyright 2014-2015 Canonical Limited.
3853+#
3854+# This file is part of charm-helpers.
3855+#
3856+# charm-helpers is free software: you can redistribute it and/or modify
3857+# it under the terms of the GNU Lesser General Public License version 3 as
3858+# published by the Free Software Foundation.
3859+#
3860+# charm-helpers is distributed in the hope that it will be useful,
3861+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3862+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3863+# GNU Lesser General Public License for more details.
3864+#
3865+# You should have received a copy of the GNU Lesser General Public License
3866+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3867+
3868+# Bootstrap charm-helpers, installing its dependencies if necessary using
3869+# only standard libraries.
3870+import subprocess
3871+import sys
3872+
3873+try:
3874+ import six # flake8: noqa
3875+except ImportError:
3876+ if sys.version_info.major == 2:
3877+ subprocess.check_call(['apt-get', 'install', '-y', 'python-six'])
3878+ else:
3879+ subprocess.check_call(['apt-get', 'install', '-y', 'python3-six'])
3880+ import six # flake8: noqa
3881+
3882+try:
3883+ import yaml # flake8: noqa
3884+except ImportError:
3885+ if sys.version_info.major == 2:
3886+ subprocess.check_call(['apt-get', 'install', '-y', 'python-yaml'])
3887+ else:
3888+ subprocess.check_call(['apt-get', 'install', '-y', 'python3-yaml'])
3889+ import yaml # flake8: noqa
3890
3891=== added directory 'tests/charmhelpers/contrib'
3892=== added file 'tests/charmhelpers/contrib/__init__.py'
3893--- tests/charmhelpers/contrib/__init__.py 1970-01-01 00:00:00 +0000
3894+++ tests/charmhelpers/contrib/__init__.py 2015-04-30 13:14:23 +0000
3895@@ -0,0 +1,15 @@
3896+# Copyright 2014-2015 Canonical Limited.
3897+#
3898+# This file is part of charm-helpers.
3899+#
3900+# charm-helpers is free software: you can redistribute it and/or modify
3901+# it under the terms of the GNU Lesser General Public License version 3 as
3902+# published by the Free Software Foundation.
3903+#
3904+# charm-helpers is distributed in the hope that it will be useful,
3905+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3906+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3907+# GNU Lesser General Public License for more details.
3908+#
3909+# You should have received a copy of the GNU Lesser General Public License
3910+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3911
3912=== added directory 'tests/charmhelpers/contrib/amulet'
3913=== added file 'tests/charmhelpers/contrib/amulet/__init__.py'
3914--- tests/charmhelpers/contrib/amulet/__init__.py 1970-01-01 00:00:00 +0000
3915+++ tests/charmhelpers/contrib/amulet/__init__.py 2015-04-30 13:14:23 +0000
3916@@ -0,0 +1,15 @@
3917+# Copyright 2014-2015 Canonical Limited.
3918+#
3919+# This file is part of charm-helpers.
3920+#
3921+# charm-helpers is free software: you can redistribute it and/or modify
3922+# it under the terms of the GNU Lesser General Public License version 3 as
3923+# published by the Free Software Foundation.
3924+#
3925+# charm-helpers is distributed in the hope that it will be useful,
3926+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3927+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3928+# GNU Lesser General Public License for more details.
3929+#
3930+# You should have received a copy of the GNU Lesser General Public License
3931+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3932
3933=== added file 'tests/charmhelpers/contrib/amulet/deployment.py'
3934--- tests/charmhelpers/contrib/amulet/deployment.py 1970-01-01 00:00:00 +0000
3935+++ tests/charmhelpers/contrib/amulet/deployment.py 2015-04-30 13:14:23 +0000
3936@@ -0,0 +1,93 @@
3937+# Copyright 2014-2015 Canonical Limited.
3938+#
3939+# This file is part of charm-helpers.
3940+#
3941+# charm-helpers is free software: you can redistribute it and/or modify
3942+# it under the terms of the GNU Lesser General Public License version 3 as
3943+# published by the Free Software Foundation.
3944+#
3945+# charm-helpers is distributed in the hope that it will be useful,
3946+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3947+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3948+# GNU Lesser General Public License for more details.
3949+#
3950+# You should have received a copy of the GNU Lesser General Public License
3951+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
3952+
3953+import amulet
3954+import os
3955+import six
3956+
3957+
3958+class AmuletDeployment(object):
3959+ """Amulet deployment.
3960+
3961+ This class provides generic Amulet deployment and test runner
3962+ methods.
3963+ """
3964+
3965+ def __init__(self, series=None):
3966+ """Initialize the deployment environment."""
3967+ self.series = None
3968+
3969+ if series:
3970+ self.series = series
3971+ self.d = amulet.Deployment(series=self.series)
3972+ else:
3973+ self.d = amulet.Deployment()
3974+
3975+ def _add_services(self, this_service, other_services):
3976+ """Add services.
3977+
3978+ Add services to the deployment where this_service is the local charm
3979+ that we're testing and other_services are the other services that
3980+ are being used in the local amulet tests.
3981+ """
3982+ if this_service['name'] != os.path.basename(os.getcwd()):
3983+ s = this_service['name']
3984+ msg = "The charm's root directory name needs to be {}".format(s)
3985+ amulet.raise_status(amulet.FAIL, msg=msg)
3986+
3987+ if 'units' not in this_service:
3988+ this_service['units'] = 1
3989+
3990+ self.d.add(this_service['name'], units=this_service['units'])
3991+
3992+ for svc in other_services:
3993+ if 'location' in svc:
3994+ branch_location = svc['location']
3995+ elif self.series:
3996+ branch_location = 'cs:{}/{}'.format(self.series, svc['name']),
3997+ else:
3998+ branch_location = None
3999+
4000+ if 'units' not in svc:
4001+ svc['units'] = 1
4002+
4003+ self.d.add(svc['name'], charm=branch_location, units=svc['units'])
4004+
4005+ def _add_relations(self, relations):
4006+ """Add all of the relations for the services."""
4007+ for k, v in six.iteritems(relations):
4008+ self.d.relate(k, v)
4009+
4010+ def _configure_services(self, configs):
4011+ """Configure all of the services."""
4012+ for service, config in six.iteritems(configs):
4013+ self.d.configure(service, config)
4014+
4015+ def _deploy(self):
4016+ """Deploy environment and wait for all hooks to finish executing."""
4017+ try:
4018+ self.d.setup(timeout=900)
4019+ self.d.sentry.wait(timeout=900)
4020+ except amulet.helpers.TimeoutError:
4021+ amulet.raise_status(amulet.FAIL, msg="Deployment timed out")
4022+ except Exception:
4023+ raise
4024+
4025+ def run_tests(self):
4026+ """Run all of the methods that are prefixed with 'test_'."""
4027+ for test in dir(self):
4028+ if test.startswith('test_'):
4029+ getattr(self, test)()
4030
4031=== added file 'tests/charmhelpers/contrib/amulet/utils.py'
4032--- tests/charmhelpers/contrib/amulet/utils.py 1970-01-01 00:00:00 +0000
4033+++ tests/charmhelpers/contrib/amulet/utils.py 2015-04-30 13:14:23 +0000
4034@@ -0,0 +1,323 @@
4035+# Copyright 2014-2015 Canonical Limited.
4036+#
4037+# This file is part of charm-helpers.
4038+#
4039+# charm-helpers is free software: you can redistribute it and/or modify
4040+# it under the terms of the GNU Lesser General Public License version 3 as
4041+# published by the Free Software Foundation.
4042+#
4043+# charm-helpers is distributed in the hope that it will be useful,
4044+# but WITHOUT ANY WARRANTY; without even the implied warranty of
4045+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4046+# GNU Lesser General Public License for more details.
4047+#
4048+# You should have received a copy of the GNU Lesser General Public License
4049+# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
4050+
4051+import ConfigParser
4052+import io
4053+import logging
4054+import re
4055+import sys
4056+import time
4057+
4058+import six
4059+
4060+
4061+class AmuletUtils(object):
4062+ """Amulet utilities.
4063+
4064+ This class provides common utility functions that are used by Amulet
4065+ tests.
4066+ """
4067+
4068+ def __init__(self, log_level=logging.ERROR):
4069+ self.log = self.get_logger(level=log_level)
4070+
4071+ def get_logger(self, name="amulet-logger", level=logging.DEBUG):
4072+ """Get a logger object that will log to stdout."""
4073+ log = logging
4074+ logger = log.getLogger(name)
4075+ fmt = log.Formatter("%(asctime)s %(funcName)s "
4076+ "%(levelname)s: %(message)s")
4077+
4078+ handler = log.StreamHandler(stream=sys.stdout)
4079+ handler.setLevel(level)
4080+ handler.setFormatter(fmt)
4081+
4082+ logger.addHandler(handler)
4083+ logger.setLevel(level)
4084+
4085+ return logger
4086+
4087+ def valid_ip(self, ip):
4088+ if re.match(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$", ip):
4089+ return True
4090+ else:
4091+ return False
4092+
4093+ def valid_url(self, url):
4094+ p = re.compile(
4095+ r'^(?:http|ftp)s?://'
4096+ r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}\.?)|' # noqa
4097+ r'localhost|'
4098+ r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'
4099+ r'(?::\d+)?'
4100+ r'(?:/?|[/?]\S+)$',
4101+ re.IGNORECASE)
4102+ if p.match(url):
4103+ return True
4104+ else:
4105+ return False
4106+
4107+ def validate_services(self, commands):
4108+ """Validate services.
4109+
4110+ Verify the specified services are running on the corresponding
4111+ service units.
4112+ """
4113+ for k, v in six.iteritems(commands):
4114+ for cmd in v:
4115+ output, code = k.run(cmd)
4116+ self.log.debug('{} `{}` returned '
4117+ '{}'.format(k.info['unit_name'],
4118+ cmd, code))
4119+ if code != 0:
4120+ return "command `{}` returned {}".format(cmd, str(code))
4121+ return None
4122+
4123+ def _get_config(self, unit, filename):
4124+ """Get a ConfigParser object for parsing a unit's config file."""
4125+ file_contents = unit.file_contents(filename)
4126+
4127+ # NOTE(beisner): by default, ConfigParser does not handle options
4128+ # with no value, such as the flags used in the mysql my.cnf file.
4129+ # https://bugs.python.org/issue7005
4130+ config = ConfigParser.ConfigParser(allow_no_value=True)
4131+ config.readfp(io.StringIO(file_contents))
4132+ return config
4133+
4134+ def validate_config_data(self, sentry_unit, config_file, section,
4135+ expected):
4136+ """Validate config file data.
4137+
4138+ Verify that the specified section of the config file contains
4139+ the expected option key:value pairs.
4140+ """
4141+ config = self._get_config(sentry_unit, config_file)
4142+
4143+ if section != 'DEFAULT' and not config.has_section(section):
4144+ return "section [{}] does not exist".format(section)
4145+
4146+ for k in expected.keys():
4147+ if not config.has_option(section, k):
4148+ return "section [{}] is missing option {}".format(section, k)
4149+ if config.get(section, k) != expected[k]:
4150+ return "section [{}] {}:{} != expected {}:{}".format(
4151+ section, k, config.get(section, k), k, expected[k])
4152+ return None
4153+
4154+ def _validate_dict_data(self, expected, actual):
4155+ """Validate dictionary data.
4156+
4157+ Compare expected dictionary data vs actual dictionary data.
4158+ The values in the 'expected' dictionary can be strings, bools, ints,
4159+ longs, or can be a function that evaluate a variable and returns a
4160+ bool.
4161+ """
4162+ self.log.debug('actual: {}'.format(repr(actual)))
4163+ self.log.debug('expected: {}'.format(repr(expected)))
4164+
4165+ for k, v in six.iteritems(expected):
4166+ if k in actual:
4167+ if (isinstance(v, six.string_types) or
4168+ isinstance(v, bool) or
4169+ isinstance(v, six.integer_types)):
4170+ if v != actual[k]:
4171+ return "{}:{}".format(k, actual[k])
4172+ elif not v(actual[k]):
4173+ return "{}:{}".format(k, actual[k])
4174+ else:
4175+ return "key '{}' does not exist".format(k)
4176+ return None
4177+
4178+ def validate_relation_data(self, sentry_unit, relation, expected):
4179+ """Validate actual relation data based on expected relation data."""
4180+ actual = sentry_unit.relation(relation[0], relation[1])
4181+ return self._validate_dict_data(expected, actual)
4182+
4183+ def _validate_list_data(self, expected, actual):
4184+ """Compare expected list vs actual list data."""
4185+ for e in expected:
4186+ if e not in actual:
4187+ return "expected item {} not found in actual list".format(e)
4188+ return None
4189+
4190+ def not_null(self, string):
4191+ if string is not None:
4192+ return True
4193+ else:
4194+ return False
4195+
4196+ def _get_file_mtime(self, sentry_unit, filename):
4197+ """Get last modification time of file."""
4198+ return sentry_unit.file_stat(filename)['mtime']
4199+
4200+ def _get_dir_mtime(self, sentry_unit, directory):
4201+ """Get last modification time of directory."""
4202+ return sentry_unit.directory_stat(directory)['mtime']
4203+
4204+ def _get_proc_start_time(self, sentry_unit, service, pgrep_full=False):
4205+ """Get process' start time.
4206+
4207+ Determine start time of the process based on the last modification
4208+ time of the /proc/pid directory. If pgrep_full is True, the process
4209+ name is matched against the full command line.
4210+ """
4211+ if pgrep_full:
4212+ cmd = 'pgrep -o -f {}'.format(service)
4213+ else:
4214+ cmd = 'pgrep -o {}'.format(service)
4215+ cmd = cmd + ' | grep -v pgrep || exit 0'
4216+ cmd_out = sentry_unit.run(cmd)
4217+ self.log.debug('CMDout: ' + str(cmd_out))
4218+ if cmd_out[0]:
4219+ self.log.debug('Pid for %s %s' % (service, str(cmd_out[0])))
4220+ proc_dir = '/proc/{}'.format(cmd_out[0].strip())
4221+ return self._get_dir_mtime(sentry_unit, proc_dir)
4222+
4223+ def service_restarted(self, sentry_unit, service, filename,
4224+ pgrep_full=False, sleep_time=20):
4225+ """Check if service was restarted.
4226+
4227+ Compare a service's start time vs a file's last modification time
4228+ (such as a config file for that service) to determine if the service
4229+ has been restarted.
4230+ """
4231+ time.sleep(sleep_time)
4232+ if (self._get_proc_start_time(sentry_unit, service, pgrep_full) >=
4233+ self._get_file_mtime(sentry_unit, filename)):
4234+ return True
4235+ else:
4236+ return False
4237+
4238+ def service_restarted_since(self, sentry_unit, mtime, service,
4239+ pgrep_full=False, sleep_time=20,
4240+ retry_count=2):
4241+ """Check if service was been started after a given time.
4242+
4243+ Args:
4244+ sentry_unit (sentry): The sentry unit to check for the service on
4245+ mtime (float): The epoch time to check against
4246+ service (string): service name to look for in process table
4247+ pgrep_full (boolean): Use full command line search mode with pgrep
4248+ sleep_time (int): Seconds to sleep before looking for process
4249+ retry_count (int): If service is not found, how many times to retry
4250+
4251+ Returns:
4252+ bool: True if service found and its start time it newer than mtime,
4253+ False if service is older than mtime or if service was
4254+ not found.
4255+ """
4256+ self.log.debug('Checking %s restarted since %s' % (service, mtime))
4257+ time.sleep(sleep_time)
4258+ proc_start_time = self._get_proc_start_time(sentry_unit, service,
4259+ pgrep_full)
4260+ while retry_count > 0 and not proc_start_time:
4261+ self.log.debug('No pid file found for service %s, will retry %i '
4262+ 'more times' % (service, retry_count))
4263+ time.sleep(30)
4264+ proc_start_time = self._get_proc_start_time(sentry_unit, service,
4265+ pgrep_full)
4266+ retry_count = retry_count - 1
4267+
4268+ if not proc_start_time:
4269+ self.log.warn('No proc start time found, assuming service did '
4270+ 'not start')
4271+ return False
4272+ if proc_start_time >= mtime:
4273+ self.log.debug('proc start time is newer than provided mtime'
4274+ '(%s >= %s)' % (proc_start_time, mtime))
4275+ return True
4276+ else:
4277+ self.log.warn('proc start time (%s) is older than provided mtime '
4278+ '(%s), service did not restart' % (proc_start_time,
4279+ mtime))
4280+ return False
4281+
4282+ def config_updated_since(self, sentry_unit, filename, mtime,
4283+ sleep_time=20):
4284+ """Check if file was modified after a given time.
4285+
4286+ Args:
4287+ sentry_unit (sentry): The sentry unit to check the file mtime on
4288+ filename (string): The file to check mtime of
4289+ mtime (float): The epoch time to check against
4290+ sleep_time (int): Seconds to sleep before looking for process
4291+
4292+ Returns:
4293+ bool: True if file was modified more recently than mtime, False if
4294+ file was modified before mtime,
4295+ """
4296+ self.log.debug('Checking %s updated since %s' % (filename, mtime))
4297+ time.sleep(sleep_time)
4298+ file_mtime = self._get_file_mtime(sentry_unit, filename)
4299+ if file_mtime >= mtime:
4300+ self.log.debug('File mtime is newer than provided mtime '
4301+ '(%s >= %s)' % (file_mtime, mtime))
4302+ return True
4303+ else:
4304+ self.log.warn('File mtime %s is older than provided mtime %s'
4305+ % (file_mtime, mtime))
4306+ return False
4307+
4308+ def validate_service_config_changed(self, sentry_unit, mtime, service,
4309+ filename, pgrep_full=False,
4310+ sleep_time=20, retry_count=2):
4311+ """Check service and file were updated after mtime
4312+
4313+ Args:
4314+ sentry_unit (sentry): The sentry unit to check for the service on
4315+ mtime (float): The epoch time to check against
4316+ service (string): service name to look for in process table
4317+ filename (string): The file to check mtime of
4318+ pgrep_full (boolean): Use full command line search mode with pgrep
4319+ sleep_time (int): Seconds to sleep before looking for process
4320+ retry_count (int): If service is not found, how many times to retry
4321+
4322+ Typical Usage:
4323+ u = OpenStackAmuletUtils(ERROR)
4324+ ...
4325+ mtime = u.get_sentry_time(self.cinder_sentry)
4326+ self.d.configure('cinder', {'verbose': 'True', 'debug': 'True'})
4327+ if not u.validate_service_config_changed(self.cinder_sentry,
4328+ mtime,
4329+ 'cinder-api',
4330+ '/etc/cinder/cinder.conf')
4331+ amulet.raise_status(amulet.FAIL, msg='update failed')
4332+ Returns:
4333+ bool: True if both service and file where updated/restarted after
4334+ mtime, False if service is older than mtime or if service was
4335+ not found or if filename was modified before mtime.
4336+ """
4337+ self.log.debug('Checking %s restarted since %s' % (service, mtime))
4338+ time.sleep(sleep_time)
4339+ service_restart = self.service_restarted_since(sentry_unit, mtime,
4340+ service,
4341+ pgrep_full=pgrep_full,
4342+ sleep_time=0,
4343+ retry_count=retry_count)
4344+ config_update = self.config_updated_since(sentry_unit, filename, mtime,
4345+ sleep_time=0)
4346+ return service_restart and config_update
4347+
4348+ def get_sentry_time(self, sentry_unit):
4349+ """Return current epoch time on a sentry"""
4350+ cmd = "date +'%s'"
4351+ return float(sentry_unit.run(cmd)[0])
4352+
4353+ def relation_error(self, name, data):
4354+ return 'unexpected relation data in {} - {}'.format(name, data)
4355+
4356+ def endpoint_error(self, name, data):
4357+ return 'unexpected endpoint data in {} - {}'.format(name, data)
4358
4359=== added file 'tests/tests.yaml'
4360--- tests/tests.yaml 1970-01-01 00:00:00 +0000
4361+++ tests/tests.yaml 2015-04-30 13:14:23 +0000
4362@@ -0,0 +1,9 @@
4363+bootstrap: true
4364+reset: true
4365+virtualenv: true
4366+makefile:
4367+ - lint
4368+sources:
4369+ - ppa:juju/stable
4370+packages:
4371+ - amulet

Subscribers

People subscribed via source and target branches

to all changes: