Merge lp:~johnsca/charm-helpers/services-callback-fu into lp:~cf-charmers/charm-helpers/cloud-foundry

Proposed by Cory Johns
Status: Merged
Merged at revision: 182
Proposed branch: lp:~johnsca/charm-helpers/services-callback-fu
Merge into: lp:~cf-charmers/charm-helpers/cloud-foundry
Diff against target: 1640 lines (+866/-582)
10 files modified
charmhelpers/contrib/cloudfoundry/contexts.py (+41/-32)
charmhelpers/core/host.py (+5/-0)
charmhelpers/core/services.py (+316/-79)
charmhelpers/core/templating.py (+38/-145)
test-requirements-tox.txt (+1/-1)
tests/contrib/cloudfoundry/test_render_context.py (+28/-23)
tests/core/test_host.py (+10/-0)
tests/core/test_services.py (+398/-118)
tests/core/test_templating.py (+22/-177)
tox.ini (+7/-7)
To merge this branch: bzr merge lp:~johnsca/charm-helpers/services-callback-fu
Reviewer Review Type Date Requested Status
Cloud Foundry Charmers Pending
Review via email: mp+220733@code.launchpad.net

Description of the change

Callbacks instead of ServiceManagers

Refactored charmhelpers.core.services to use callbacks instead of
service types / managers, to make it easier for charms to specify
custom behavior.

https://codereview.appspot.com/98490043/

To post a comment you must log in.
Revision history for this message
Cory Johns (johnsca) wrote :

Reviewers: mp+220733_code.launchpad.net,

Message:
Please take a look.

Description:
Callbacks instead of ServiceManagers

Refactored charmhelpers.core.services to use callbacks instead of
service types / managers, to make it easier for charms to specify
custom behavior.

https://code.launchpad.net/~johnsca/charm-helpers/services-callback-fu/+merge/220733

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/98490043/

Affected files (+146, -92 lines):
   A [revision details]
   M charmhelpers/core/host.py
   M charmhelpers/core/services.py
   M charmhelpers/core/templating.py
   M tests/core/test_host.py
   M tests/core/test_services.py
   M tox.ini

Revision history for this message
Benjamin Saller (bcsaller) wrote :

LGTM with some trivials

Thanks for this.

https://codereview.appspot.com/98490043/diff/1/charmhelpers/core/host.py
File charmhelpers/core/host.py (right):

https://codereview.appspot.com/98490043/diff/1/charmhelpers/core/host.py#newcode67
charmhelpers/core/host.py:67: """Determine whether a system service is
available"""
'an OS service' or 'an init service' or 'an upstart service'

or something similar, service is very overloaded in this context

https://codereview.appspot.com/98490043/diff/1/charmhelpers/core/services.py
File charmhelpers/core/services.py (right):

https://codereview.appspot.com/98490043/diff/1/charmhelpers/core/services.py#newcode32
charmhelpers/core/services.py:32: restarting the service via Upstart).
Indicate that callback is called with service name,

We should also consider an 'incomplete' event, if a relation is broken
we might stop the service for example.

https://codereview.appspot.com/98490043/diff/1/charmhelpers/core/services.py#newcode35
charmhelpers/core/services.py:35: the service via Upstart).
stop is fine, for a stop hook, but incomplete would be for
relation-broken hooks. We can defer that till we actually intend to
support it though.

https://codereview.appspot.com/98490043/diff/1/charmhelpers/core/services.py#newcode94
charmhelpers/core/services.py:94: for service_name in service_names or
SERVICES.keys():
Handle single argument string calls,
if service_names and isinstance(service_names, str):
    service_names = [service_names]

Makes sense with the 'incomplete' mentioned above

https://codereview.appspot.com/98490043/diff/1/charmhelpers/core/services.py#newcode104
charmhelpers/core/services.py:104: Given the name of a registered
service, return a ServiceManager
This isn't still a ServiceManager as per the comment if I'm reading this
properly.

https://codereview.appspot.com/98490043/

183. By Cory Johns

Updated services.register docstring to clarify when and how the callbacks are called

Revision history for this message
Cory Johns (johnsca) wrote :

On 2014/05/22 23:47:43, benjamin.saller wrote:
> We should also consider an 'incomplete' event, if a relation is broken
we might
> stop the service for example.

> stop is fine, for a stop hook, but incomplete would be for
relation-broken
> hooks. We can defer that till we actually intend to support it though.

I wonder if we should change "complete" to "start," since what we're
really saying is "this code should run when the service is ready to
start." Then, "stop" can be called if stop_services() is called or if a
relationship gets broken such that the service no longer has all of the
valid information ("this code should run when the service has to stop,
for whatever reason").

> Handle single argument string calls,

https://codereview.appspot.com/98490043/diff/1/tests/core/test_services.py#newcode94

`foo(*args)` is a "varargs" definition. It will accept `foo()`,
`foo('bar')`, `foo('bar', 'baz')`, etc. (but not `params = ['bar',
'baz']; foo(params)`; that would have to be written `params = ['bar',
'baz']; foo(*params)`), and `args` will automatically be a list.

https://codereview.appspot.com/98490043/

184. By Cory Johns

Refined services API to deemphasize template management and emphasize service management

Revision history for this message
Cory Johns (johnsca) wrote :
185. By Cory Johns

Fixed missed key change in services API

186. By Cory Johns

Fixed key error checking services.is_ready() before any relations are present

Revision history for this message
Cory Johns (johnsca) wrote :
187. By Cory Johns

Switch services API from module global to instance, per review

Revision history for this message
Cory Johns (johnsca) wrote :
Revision history for this message
Benjamin Saller (bcsaller) wrote :

Comments for this went into the charm, but this LGTM as well, thanks.

https://codereview.appspot.com/98490043/

188. By Cory Johns

Added managed list of ports to services API

Revision history for this message
Cory Johns (johnsca) wrote :
189. By Cory Johns

Removed default target for template helper, as it is too prone to errors

Revision history for this message
Cory Johns (johnsca) wrote :
Revision history for this message
Benjamin Saller (bcsaller) wrote :

Thanks for tracking down the err0r, and yes, explicit win (esp. since we
hit this in practice).

+1

https://codereview.appspot.com/98490043/

Revision history for this message
Cory Johns (johnsca) wrote :

*** Submitted:

Callbacks instead of ServiceManagers

Refactored charmhelpers.core.services to use callbacks instead of
service types / managers, to make it easier for charms to specify
custom behavior.

R=benjamin.saller
CC=
https://codereview.appspot.com/98490043

https://codereview.appspot.com/98490043/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'charmhelpers/contrib/cloudfoundry/contexts.py'
--- charmhelpers/contrib/cloudfoundry/contexts.py 2014-05-16 23:28:28 +0000
+++ charmhelpers/contrib/cloudfoundry/contexts.py 2014-05-29 17:24:14 +0000
@@ -1,55 +1,64 @@
1import os1import os
22import yaml
3from charmhelpers.core.templating import (3
4 ContextGenerator,4from charmhelpers.core.services import RelationContext
5 RelationContext,5
6 StorableContext,6
7)7class StoredContext(dict):
88 """
99 A data context that always returns the data that it was first created with.
10# Stores `config_data` hash into yaml file with `file_name` as a name10 """
11# if `file_name` already exists, then it loads data from `file_name`.
12class StoredContext(ContextGenerator, StorableContext):
13 def __init__(self, file_name, config_data):11 def __init__(self, file_name, config_data):
12 """
13 If the file exists, populate `self` with the data from the file.
14 Otherwise, populate with the given data and persist it to the file.
15 """
14 if os.path.exists(file_name):16 if os.path.exists(file_name):
15 self.data = self.read_context(file_name)17 self.update(self.read_context(file_name))
16 else:18 else:
17 self.store_context(file_name, config_data)19 self.store_context(file_name, config_data)
18 self.data = config_data20 self.update(config_data)
1921
20 def __call__(self):22 def store_context(self, file_name, config_data):
21 return self.data23 with open(file_name, 'w') as file_stream:
2224 yaml.dump(config_data, file_stream)
2325
24class NatsContext(RelationContext):26 def read_context(self, file_name):
27 with open(file_name, 'r') as file_stream:
28 data = yaml.load(file_stream)
29 if not data:
30 raise OSError("%s is empty" % file_name)
31 return data
32
33
34class NatsRelation(RelationContext):
25 interface = 'nats'35 interface = 'nats'
26 required_keys = ['nats_port', 'nats_address', 'nats_user', 'nats_password']36 required_keys = ['nats_port', 'nats_address', 'nats_user', 'nats_password']
2737
2838
29class MysqlDSNContext(RelationContext):39class MysqlRelation(RelationContext):
30 interface = 'db'40 interface = 'db'
31 required_keys = ['user', 'password', 'host', 'database']41 required_keys = ['user', 'password', 'host', 'database']
32 dsn_template = "mysql2://{user}:{password}@{host}:{port}/{database}"42 dsn_template = "mysql2://{user}:{password}@{host}:{port}/{database}"
3343
34 def __call__(self):44 def get_data(self):
35 ctx = RelationContext.__call__(self)45 RelationContext.get_data(self)
36 if ctx:46 if self.is_ready():
37 if 'port' not in ctx:47 if 'port' not in self['db']:
38 ctx['db']['port'] = '3306'48 self['db']['port'] = '3306'
39 ctx['db']['dsn'] = self.dsn_template.format(**ctx['db'])49 self['db']['dsn'] = self.dsn_template.format(**self['db'])
40 return ctx50
4151
4252class RouterRelation(RelationContext):
43class RouterContext(RelationContext):
44 interface = 'router'53 interface = 'router'
45 required_keys = ['domain']54 required_keys = ['domain']
4655
4756
48class LogRouterContext(RelationContext):57class LogRouterRelation(RelationContext):
49 interface = 'logrouter'58 interface = 'logrouter'
50 required_keys = ['shared-secret', 'logrouter-address']59 required_keys = ['shared-secret', 'logrouter-address']
5160
5261
53class LoggregatorContext(RelationContext):62class LoggregatorRelation(RelationContext):
54 interface = 'loggregator'63 interface = 'loggregator'
55 required_keys = ['shared_secret', 'loggregator_address']64 required_keys = ['shared_secret', 'loggregator_address']
5665
=== modified file 'charmhelpers/core/host.py'
--- charmhelpers/core/host.py 2014-05-20 15:40:48 +0000
+++ charmhelpers/core/host.py 2014-05-29 17:24:14 +0000
@@ -63,6 +63,11 @@
63 return False63 return False
6464
6565
66def service_available(service_name):
67 """Determine whether a system service is available"""
68 return service('status', service_name)
69
70
66def adduser(username, password=None, shell='/bin/bash', system_user=False):71def adduser(username, password=None, shell='/bin/bash', system_user=False):
67 """Add a user to the system"""72 """Add a user to the system"""
68 try:73 try:
6974
=== modified file 'charmhelpers/core/services.py'
--- charmhelpers/core/services.py 2014-05-16 21:48:06 +0000
+++ charmhelpers/core/services.py 2014-05-29 17:24:14 +0000
@@ -1,84 +1,321 @@
1import os
2import sys
3from collections import Iterable
1from charmhelpers.core import templating4from charmhelpers.core import templating
2from charmhelpers.core import host5from charmhelpers.core import host
36from charmhelpers.core import hookenv
47
5SERVICES = {}8
69class ServiceManager(object):
710 def __init__(self, services=None):
8def register(services, templates_dir=None):11 """
9 """12 Register a list of services, given their definitions.
10 Register a list of service configs.13
1114 Service definitions are dicts in the following formats (all keys except
12 Service Configs are dicts in the following formats:15 'service' are optional):
1316
14 {17 {
15 "service": <service name>,18 "service": <service name>,
16 "templates": [ {19 "required_data": <list of required data contexts>,
17 'target': <render target of template>,20 "data_ready": <one or more callbacks>,
18 'source': <optional name of template in passed in templates_dir>21 "data_lost": <one or more callbacks>,
19 'file_properties': <optional dict taking owner and octal mode>22 "start": <one or more callbacks>,
20 'contexts': [ context generators, see contexts.py ]23 "stop": <one or more callbacks>,
21 }24 "ports": <list of ports to manage>,
22 ] }25 }
2326
24 Either `source` or `target` must be provided.27 The 'required_data' list should contain dicts of required data (or
2528 dependency managers that act like dicts and know how to collect the data).
26 If 'source' is not provided for a template the templates_dir will29 Only when all items in the 'required_data' list are populated are the list
27 be consulted for ``basename(target).j2``.30 of 'data_ready' and 'start' callbacks executed. See `is_ready()` for more
2831 information.
29 If `target` is not provided, it will be assumed to be32
30 ``/etc/init/<service name>.conf``.33 The 'data_ready' value should be either a single callback, or a list of
31 """34 callbacks, to be called when all items in 'required_data' pass `is_ready()`.
32 for service in services:35 Each callback will be called with the service name as the only parameter.
33 service.setdefault('templates_dir', templates_dir)36 After these all of the 'data_ready' callbacks are called, the 'start'
34 SERVICES[service['service']] = service37 callbacks are fired.
3538
3639 The 'data_lost' value should be either a single callback, or a list of
37def reconfigure_services(restart=True):40 callbacks, to be called when a 'required_data' item no longer passes
38 """41 `is_ready()`. Each callback will be called with the service name as the
39 Update all files for all services and optionally restart them, if ready.42 only parameter. After these all of the 'data_ready' callbacks are called,
40 """43 the 'stop' callbacks are fired.
41 for service_name in SERVICES.keys():44
42 reconfigure_service(service_name, restart=restart)45 The 'start' value should be either a single callback, or a list of
4346 callbacks, to be called when starting the service, after the 'data_ready'
4447 callbacks are complete. Each callback will be called with the service
45def reconfigure_service(service_name, restart=True):48 name as the only parameter. This defaults to
46 """49 `[host.service_start, services.open_ports]`.
47 Update all files for a single service and optionally restart it, if ready.50
48 """51 The 'stop' value should be either a single callback, or a list of
49 service = SERVICES.get(service_name)52 callbacks, to be called when stopping the service. If the service is
50 if not service or service['service'] != service_name:53 being stopped because it no longer has all of its 'required_data', this
51 raise KeyError('Service not registered: %s' % service_name)54 will be called after all of the 'data_lost' callbacks are complete.
5255 Each callback will be called with the service name as the only parameter.
53 manager_type = service.get('type', UpstartService)56 This defaults to `[services.close_ports, host.service_stop]`.
54 manager_type(service).reconfigure(restart)57
5558 The 'ports' value should be a list of ports to manage. The default
5659 'start' handler will open the ports after the service is started,
57def stop_services():60 and the default 'stop' handler will close the ports prior to stopping
58 for service_name in SERVICES.keys():61 the service.
59 if host.service_running(service_name):62
60 host.service_stop(service_name)63
6164 Examples:
6265
63class ServiceTypeManager(object):66 The following registers an Upstart service called bingod that depends on
64 def __init__(self, service_definition):67 a mongodb relation and which runs a custom `db_migrate` function prior to
65 self.service_name = service_definition['service']68 restarting the service, and a Runit serivce called spadesd.
66 self.templates = service_definition['templates']69
67 self.templates_dir = service_definition['templates_dir']70 >>> manager = services.ServiceManager([
6871 ... {
69 def reconfigure(self, restart=True):72 ... 'service': 'bingod',
73 ... 'ports': [80, 443],
74 ... 'required_data': [MongoRelation(), config()],
75 ... 'data_ready': [
76 ... services.template(source='bingod.conf'),
77 ... services.template(source='bingod.ini',
78 ... target='/etc/bingod.ini',
79 ... owner='bingo', perms=0400),
80 ... ],
81 ... },
82 ... {
83 ... 'service': 'spadesd',
84 ... 'data_ready': services.template(source='spadesd_run.j2',
85 ... target='/etc/sv/spadesd/run',
86 ... perms=0555),
87 ... 'start': runit_start,
88 ... 'stop': runit_stop,
89 ... },
90 ... ])
91 ... manager.manage()
92 """
93 self.services = {}
94 for service in services or []:
95 service_name = service['service']
96 self.services[service_name] = service
97
98 def manage(self):
99 """
100 Handle the current hook by doing The Right Thing with the registered services.
101 """
102 hook_name = os.path.basename(sys.argv[0])
103 if hook_name == 'stop':
104 self.stop_services()
105 else:
106 self.reconfigure_services()
107
108 def reconfigure_services(self, *service_names):
109 """
110 Update all files for one or more registered services, and,
111 if ready, optionally restart them.
112
113 If no service names are given, reconfigures all registered services.
114 """
115 for service_name in service_names or self.services.keys():
116 if self.is_ready(service_name):
117 self.fire_event('data_ready', service_name)
118 self.fire_event('start', service_name, default=[
119 host.service_restart,
120 open_ports])
121 self.save_ready(service_name)
122 else:
123 if self.was_ready(service_name):
124 self.fire_event('data_lost', service_name)
125 self.fire_event('stop', service_name, default=[
126 close_ports,
127 host.service_stop])
128 self.save_lost(service_name)
129
130 def stop_services(self, *service_names):
131 """
132 Stop one or more registered services, by name.
133
134 If no service names are given, stops all registered services.
135 """
136 for service_name in service_names or self.services.keys():
137 self.fire_event('stop', service_name, default=[
138 close_ports,
139 host.service_stop])
140
141 def get_service(self, service_name):
142 """
143 Given the name of a registered service, return its service definition.
144 """
145 service = self.services.get(service_name)
146 if not service:
147 raise KeyError('Service not registered: %s' % service_name)
148 return service
149
150 def fire_event(self, event_name, service_name, default=None):
151 """
152 Fire a data_ready, data_lost, start, or stop event on a given service.
153 """
154 service = self.get_service(service_name)
155 callbacks = service.get(event_name, default)
156 if not callbacks:
157 return
158 if not isinstance(callbacks, Iterable):
159 callbacks = [callbacks]
160 for callback in callbacks:
161 if isinstance(callback, ManagerCallback):
162 callback(self, service_name, event_name)
163 else:
164 callback(service_name)
165
166 def is_ready(self, service_name):
167 """
168 Determine if a registered service is ready, by checking its 'required_data'.
169
170 A 'required_data' item can be any mapping type, and is considered ready
171 if `bool(item)` evaluates as True.
172 """
173 service = self.get_service(service_name)
174 reqs = service.get('required_data', [])
175 return all(bool(req) for req in reqs)
176
177 def save_ready(self, service_name):
178 """
179 Save an indicator that the given service is now data_ready.
180 """
181 ready_file = '{}/.ready.{}'.format(hookenv.charm_dir(), service_name)
182 with open(ready_file, 'a'):
183 pass
184
185 def save_lost(self, service_name):
186 """
187 Save an indicator that the given service is no longer data_ready.
188 """
189 ready_file = '{}/.ready.{}'.format(hookenv.charm_dir(), service_name)
190 if os.path.exists(ready_file):
191 os.remove(ready_file)
192
193 def was_ready(self, service_name):
194 """
195 Determine if the given service was previously data_ready.
196 """
197 ready_file = '{}/.ready.{}'.format(hookenv.charm_dir(), service_name)
198 return os.path.exists(ready_file)
199
200
201class RelationContext(dict):
202 """
203 Base class for a context generator that gets relation data from juju.
204
205 Subclasses must provide `interface`, which is the interface type of interest,
206 and `required_keys`, which is the set of keys required for the relation to
207 be considered complete. The first relation for the interface that is complete
208 will be used to populate the data for template.
209
210 The generated context will be namespaced under the interface type, to prevent
211 potential naming conflicts.
212 """
213 interface = None
214 required_keys = []
215
216 def __bool__(self):
217 """
218 Updates the data and returns True if all of the required_keys are available.
219 """
220 self.get_data()
221 return self.is_ready()
222
223 __nonzero__ = __bool__
224
225 def is_ready(self):
226 """
227 Returns True if all of the required_keys are available.
228 """
229 return set(self.get(self.interface, {}).keys()).issuperset(set(self.required_keys))
230
231 def get_data(self):
232 """
233 Retrieve the relation data and store it under `self[self.interface]`.
234
235 If there are more than one units related on the desired interface,
236 then each unit will have its data stored under `self[self.interface][unit_id]`
237 and one of the units with complete information will chosen at random
238 to fill the values at `self[self.interface]`.
239
240
241 For example:
242
243 {
244 'foo': 'bar',
245 'unit/0': {
246 'foo': 'bar',
247 },
248 'unit/1': {
249 'foo': 'baz',
250 },
251 }
252 """
253 if not hookenv.relation_ids(self.interface):
254 return
255
256 ns = self.setdefault(self.interface, {})
257 required = set(self.required_keys)
258 for rid in hookenv.relation_ids(self.interface):
259 for unit in hookenv.related_units(rid):
260 reldata = hookenv.relation_get(rid=rid, unit=unit)
261 unit_ns = ns.setdefault(unit, {})
262 unit_ns.update(reldata)
263 if set(reldata.keys()).issuperset(required):
264 ns.update(reldata)
265
266
267class ManagerCallback(object):
268 """
269 Special case of a callback that takes the `ServiceManager` instance
270 in addition to the service name.
271
272 Subclasses should implement `__call__` which should accept two parameters:
273
274 * `manager` The `ServiceManager` instance
275 * `service_name` The name of the service it's being triggered for
276 * `event_name` The name of the event that this callback is handling
277 """
278 def __call__(self, manager, service_name, event_name):
70 raise NotImplementedError()279 raise NotImplementedError()
71280
72281
73class UpstartService(ServiceTypeManager):282class TemplateCallback(ManagerCallback):
74 def __init__(self, service_definition):283 """
75 super(UpstartService, self).__init__(service_definition)284 Callback class that will render a template, for use as a ready action.
76 for tmpl in self.templates:285
77 if 'target' not in tmpl:286 The `target` param, if omitted, will default to `/etc/init/<service name>`.
78 tmpl['target'] = '/etc/init/%s.conf' % self.service_name287 """
79288 def __init__(self, source, target, owner='root', group='root', perms=0444):
80 def reconfigure(self, restart):289 self.source = source
81 complete = templating.render(self.templates, self.templates_dir)290 self.target = target
82291 self.owner = owner
83 if restart and complete:292 self.group = group
84 host.service_restart(self.service_name)293 self.perms = perms
294
295 def __call__(self, manager, service_name, event_name):
296 service = manager.get_service(service_name)
297 context = {}
298 for ctx in service.get('required_data', []):
299 context.update(ctx)
300 templating.render(self.source, self.target, context,
301 self.owner, self.group, self.perms)
302
303
304class PortManagerCallback(ManagerCallback):
305 """
306 Callback class that will open or close ports, for use as either
307 a start or stop action.
308 """
309 def __call__(self, manager, service_name, event_name):
310 service = manager.get_service(service_name)
311 for port in service.get('ports', []):
312 if event_name == 'start':
313 hookenv.open_port(port)
314 elif event_name == 'stop':
315 hookenv.close_port(port)
316
317
318# Convenience aliases
319template = TemplateCallback
320open_ports = PortManagerCallback()
321close_ports = PortManagerCallback()
85322
=== modified file 'charmhelpers/core/templating.py'
--- charmhelpers/core/templating.py 2014-05-20 15:40:48 +0000
+++ charmhelpers/core/templating.py 2014-05-29 17:24:14 +0000
@@ -1,158 +1,51 @@
1import os1import os
2import yaml
32
4from charmhelpers.core import host3from charmhelpers.core import host
5from charmhelpers.core import hookenv4from charmhelpers.core import hookenv
65
76
8class ContextGenerator(object):7def render(source, target, context, owner='root', group='root', perms=0444, templates_dir=None):
9 """8 """
10 Base interface for template context container generators.9 Render a template.
1110
12 A template context is a dictionary that contains data needed to populate11 The `source` path, if not absolute, is relative to the `templates_dir`.
13 the template. The generator instance should produce the context when
14 called (without arguments) by collecting information from juju (config-get,
15 relation-get, etc), the system, or whatever other sources are appropriate.
16
17 A context generator should only return any values if it has enough information
18 to provide all of its values. Any context that is missing data is considered
19 incomplete and will cause that template to not render until it has all of its
20 necessary data.
21
22 The template may receive several contexts, which will be merged together,
23 so care should be taken in the key names.
24 """
25 def __call__(self):
26 raise NotImplementedError
27
28
29class StorableContext(object):
30 """
31 A mixin for persisting a context to disk.
32 """
33 def store_context(self, file_name, config_data):
34 with open(file_name, 'w') as file_stream:
35 yaml.dump(config_data, file_stream)
36
37 def read_context(self, file_name):
38 with open(file_name, 'r') as file_stream:
39 data = yaml.load(file_stream)
40 if not data:
41 raise OSError("%s is empty" % file_name)
42 return data
43
44
45class ConfigContext(ContextGenerator):
46 """
47 A context generator that generates a context containing all of the
48 juju config values.
49 """
50 def __call__(self):
51 return hookenv.config()
52
53
54class RelationContext(ContextGenerator):
55 """
56 Base class for a context generator that gets relation data from juju.
57
58 Subclasses must provide `interface`, which is the interface type of interest,
59 and `required_keys`, which is the set of keys required for the relation to
60 be considered complete. The first relation for the interface that is complete
61 will be used to populate the data for template.
62
63 The generated context will be namespaced under the interface type, to prevent
64 potential naming conflicts.
65 """
66 interface = None
67 required_keys = []
68
69 def __call__(self):
70 if not hookenv.relation_ids(self.interface):
71 return {}
72
73 ctx = {}
74 for rid in hookenv.relation_ids(self.interface):
75 for unit in hookenv.related_units(rid):
76 reldata = hookenv.relation_get(rid=rid, unit=unit)
77 required = set(self.required_keys)
78 if set(reldata.keys()).issuperset(required):
79 ns = ctx.setdefault(self.interface, {})
80 for k, v in reldata.items():
81 ns[k] = v
82 return ctx
83
84 return {}
85
86
87class StaticContext(ContextGenerator):
88 def __init__(self, data):
89 self.data = data
90
91 def __call__(self):
92 return self.data
93
94
95def _collect_contexts(context_providers):
96 """
97 Helper function to collect and merge contexts from a list of providers.
98
99 If any of the contexts are incomplete (i.e., they return an empty dict),
100 the template is considered incomplete and will not render.
101 """
102 ctx = {}
103 for provider in context_providers:
104 c = provider()
105 if not c:
106 return False
107 ctx.update(c)
108 return ctx
109
110
111def render(template_definitions, templates_dir=None):
112 """
113 Render one or more templates, given a list of template definitions.
114
115 The template definitions should be dicts with the keys: `source`, `target`,
116 `file_properties`, and `contexts`.
117
118 The `source` path, if not absolute, is relative to the `templates_dir`
119 given when the rendered was created. If `source` is not provided
120 for a template the `template_dir` will be consulted for
121 ``basename(target).j2``.
12212
123 The `target` path should be absolute.13 The `target` path should be absolute.
12414
125 The `file_properties` should be a dict optionally containing15 The context should be a dict containing the values to be replaced in the
126 `owner`, `group`, or `perms` options, to be passed to `write_file`.16 template.
12717
128 The `contexts` should be a list containing zero or more ContextGenerators.18 The `owner`, `group`, and `perms` options will be passed to `write_file`.
12919
130 The `template_dir` defaults to `$CHARM_DIR/templates`20 If omitted, `templates_dir` defaults to the `templates` folder in the charm.
13121
132 Returns True if all of the templates were "complete" (i.e., the context22 Note: Using this requires python-jinja2; if it is not installed, calling
133 generators were able to collect the information needed to render the23 this will attempt to use charmhelpers.fetch.apt_install to install it.
134 template) and were rendered.
135 """24 """
136 # lazy import jinja2 in case templating is needed in install hook25 try:
137 from jinja2 import FileSystemLoader, Environment, exceptions26 from jinja2 import FileSystemLoader, Environment, exceptions
138 all_complete = True27 except ImportError:
28 try:
29 from charmhelpers.fetch import apt_install
30 except ImportError:
31 hookenv.log('Could not import jinja2, and could not import '
32 'charmhelpers.fetch to install it',
33 level=hookenv.ERROR)
34 raise
35 apt_install('python-jinja2', fatal=True)
36 from jinja2 import FileSystemLoader, Environment, exceptions
37
139 if templates_dir is None:38 if templates_dir is None:
140 templates_dir = os.path.join(hookenv.charm_dir(), 'templates')39 templates_dir = os.path.join(hookenv.charm_dir(), 'templates')
141 loader = Environment(loader=FileSystemLoader(templates_dir))40 loader = Environment(loader=FileSystemLoader(templates_dir))
142 for tmpl in template_definitions:41 try:
143 ctx = _collect_contexts(tmpl.get('contexts', []))42 source = source
144 if ctx is False:43 template = loader.get_template(source)
145 all_complete = False44 except exceptions.TemplateNotFound as e:
146 continue45 hookenv.log('Could not load template %s from %s.' %
147 try:46 (source, templates_dir),
148 source = tmpl.get('source', os.path.basename(tmpl['target'])+'.j2')47 level=hookenv.ERROR)
149 template = loader.get_template(source)48 raise e
150 except exceptions.TemplateNotFound as e:49 content = template.render(context)
151 hookenv.log('Could not load template %s from %s.' %50 host.mkdir(os.path.dirname(target))
152 (tmpl['source'], templates_dir),51 host.write_file(target, content, owner, group, perms)
153 level=hookenv.ERROR)
154 raise e
155 content = template.render(ctx)
156 host.mkdir(os.path.dirname(tmpl['target']))
157 host.write_file(tmpl['target'], content, **tmpl.get('file_properties', {}))
158 return all_complete
15952
=== modified file 'test-requirements-tox.txt'
--- test-requirements-tox.txt 2014-05-09 16:10:54 +0000
+++ test-requirements-tox.txt 2014-05-29 17:24:14 +0000
@@ -15,4 +15,4 @@
15wsgiref==0.1.215wsgiref==0.1.2
16launchpadlib==1.10.216launchpadlib==1.10.2
17bzr==2.6.017bzr==2.6.0
1818ipdb
1919
=== modified file 'tests/contrib/cloudfoundry/test_render_context.py'
--- tests/contrib/cloudfoundry/test_render_context.py 2014-05-13 20:14:01 +0000
+++ tests/contrib/cloudfoundry/test_render_context.py 2014-05-29 17:24:14 +0000
@@ -5,56 +5,61 @@
5from charmhelpers.contrib.cloudfoundry import contexts5from charmhelpers.contrib.cloudfoundry import contexts
66
77
8class TestNatsContext(unittest.TestCase):8class TestNatsRelation(unittest.TestCase):
99
10 @mock.patch('charmhelpers.core.hookenv.relation_ids')10 @mock.patch('charmhelpers.core.hookenv.relation_ids')
11 def test_nats_context_empty(self, mid):11 def test_nats_relation_empty(self, mid):
12 mid.return_value = None12 mid.return_value = None
13 n = contexts.NatsContext()13 n = contexts.NatsRelation()
14 self.assertEqual(n(), {})14 self.assertEqual(n, {})
1515
16 @mock.patch('charmhelpers.core.hookenv.related_units')16 @mock.patch('charmhelpers.core.hookenv.related_units')
17 @mock.patch('charmhelpers.core.hookenv.relation_ids')17 @mock.patch('charmhelpers.core.hookenv.relation_ids')
18 @mock.patch('charmhelpers.core.hookenv.relation_get')18 @mock.patch('charmhelpers.core.hookenv.relation_get')
19 def test_nats_context_populated(self, mrel, mid, mrelated):19 def test_nats_relation_populated(self, mrel, mid, mrelated):
20 mid.return_value = ['nats']20 mid.return_value = ['nats']
21 mrel.return_value = {'nats_port': 1234, 'nats_address': 'host',21 mrel.return_value = {'nats_port': 1234, 'nats_address': 'host',
22 'nats_user': 'user', 'nats_password': 'password'}22 'nats_user': 'user', 'nats_password': 'password'}
23 mrelated.return_value = ['router/0']23 mrelated.return_value = ['router/0']
24 n = contexts.NatsContext()24 n = contexts.NatsRelation()
25 expected = {'nats': {'nats_port': 1234, 'nats_address': 'host',25 expected = {'nats': {'nats_port': 1234, 'nats_address': 'host',
26 'nats_user': 'user', 'nats_password': 'password'}}26 'nats_user': 'user', 'nats_password': 'password',
27 self.assertEqual(n(), expected)27 'router/0': {'nats_port': 1234, 'nats_address': 'host',
28 'nats_user': 'user', 'nats_password': 'password'}}}
29 self.assertTrue(bool(n))
30 self.assertEqual(n, expected)
2831
29 @mock.patch('charmhelpers.core.hookenv.related_units')32 @mock.patch('charmhelpers.core.hookenv.related_units')
30 @mock.patch('charmhelpers.core.hookenv.relation_ids')33 @mock.patch('charmhelpers.core.hookenv.relation_ids')
31 @mock.patch('charmhelpers.core.hookenv.relation_get')34 @mock.patch('charmhelpers.core.hookenv.relation_get')
32 def test_nats_context_partial(self, mrel, mid, mrelated):35 def test_nats_relation_partial(self, mrel, mid, mrelated):
33 mid.return_value = ['nats']36 mid.return_value = ['nats']
34 mrel.return_value = {'nats_address': 'host'}37 mrel.return_value = {'nats_address': 'host'}
35 mrelated.return_value = ['router/0']38 mrelated.return_value = ['router/0']
36 n = contexts.NatsContext()39 n = contexts.NatsRelation()
37 self.assertEqual(n(), {})40 self.assertEqual(n, {})
3841
3942
40class TestRouterContext(unittest.TestCase):43class TestRouterRelation(unittest.TestCase):
4144
42 @mock.patch('charmhelpers.core.hookenv.relation_ids')45 @mock.patch('charmhelpers.core.hookenv.relation_ids')
43 def test_router_context_empty(self, mid):46 def test_router_relation_empty(self, mid):
44 mid.return_value = None47 mid.return_value = None
45 n = contexts.RouterContext()48 n = contexts.RouterRelation()
46 self.assertEqual(n(), {})49 self.assertEqual(n, {})
4750
48 @mock.patch('charmhelpers.core.hookenv.related_units')51 @mock.patch('charmhelpers.core.hookenv.related_units')
49 @mock.patch('charmhelpers.core.hookenv.relation_ids')52 @mock.patch('charmhelpers.core.hookenv.relation_ids')
50 @mock.patch('charmhelpers.core.hookenv.relation_get')53 @mock.patch('charmhelpers.core.hookenv.relation_get')
51 def test_router_context_populated(self, mrel, mid, mrelated):54 def test_router_relation_populated(self, mrel, mid, mrelated):
52 mid.return_value = ['router']55 mid.return_value = ['router']
53 mrel.return_value = {'domain': 'example.com'}56 mrel.return_value = {'domain': 'example.com'}
54 mrelated.return_value = ['router/0']57 mrelated.return_value = ['router/0']
55 n = contexts.RouterContext()58 n = contexts.RouterRelation()
56 expected = {'router': {'domain': 'example.com'}}59 expected = {'router': {'domain': 'example.com',
57 self.assertEqual(n(), expected)60 'router/0': {'domain': 'example.com'}}}
61 self.assertTrue(bool(n))
62 self.assertEqual(n, expected)
5863
5964
60class TestStoredContext(unittest.TestCase):65class TestStoredContext(unittest.TestCase):
@@ -71,13 +76,13 @@
71 os.unlink(file_name)76 os.unlink(file_name)
72 contexts.StoredContext(file_name, {'key': 'initial_value'})77 contexts.StoredContext(file_name, {'key': 'initial_value'})
73 self.assertTrue(os.path.isfile(file_name))78 self.assertTrue(os.path.isfile(file_name))
74 context = contexts.StoredContext(file_name, {'key': 'random_value'})()79 context = contexts.StoredContext(file_name, {'key': 'random_value'})
75 self.assertIn('key', context)80 self.assertIn('key', context)
76 self.assertEqual(context['key'], 'initial_value')81 self.assertEqual(context['key'], 'initial_value')
7782
78 def test_stored_context_raise(self):83 def test_stored_context_raise(self):
79 _, file_name = tempfile.mkstemp()84 _, file_name = tempfile.mkstemp()
80 with self.assertRaises(OSError) as cm:85 with self.assertRaises(OSError):
81 contexts.StoredContext(file_name, {'key': 'initial_value'})86 contexts.StoredContext(file_name, {'key': 'initial_value'})
82 os.unlink(file_name)87 os.unlink(file_name)
8388
8489
=== modified file 'tests/core/test_host.py'
--- tests/core/test_host.py 2014-05-20 15:40:48 +0000
+++ tests/core/test_host.py 2014-05-29 17:24:14 +0000
@@ -96,6 +96,16 @@
96 service.assert_called_with('reload', service_name)96 service.assert_called_with('reload', service_name)
9797
98 @patch.object(host, 'service')98 @patch.object(host, 'service')
99 def test_service_available(self, service):
100 service_name = 'foo-service'
101 service.side_effect = [True]
102 self.assertTrue(host.service_available(service_name))
103 service.side_effect = [False]
104 self.assertFalse(host.service_available(service_name))
105
106 service.assert_called_with('status', service_name)
107
108 @patch.object(host, 'service')
99 def test_failed_reload_restarts_a_service(self, service):109 def test_failed_reload_restarts_a_service(self, service):
100 service_name = 'foo-service'110 service_name = 'foo-service'
101 service.side_effect = [False, True]111 service.side_effect = [False, True]
102112
=== modified file 'tests/core/test_services.py'
--- tests/core/test_services.py 2014-05-16 21:48:06 +0000
+++ tests/core/test_services.py 2014-05-29 17:24:14 +0000
@@ -1,126 +1,406 @@
1import pkg_resources
2import mock1import mock
3import unittest2import unittest
4from charmhelpers.core import services3from charmhelpers.core import services
54
6TEMPLATES_DIR = pkg_resources.resource_filename(__name__, 'templates')5
76class TestServiceManager(unittest.TestCase):
87 def test_register(self):
9def static_content(data):8 manager = services.ServiceManager([
10 def context():9 {'service': 'service1',
11 return data10 'foo': 'bar'},
12 return context11 {'service': 'service2',
1312 'qux': 'baz'},
1413 ])
15# Sample render context used to test rendering paths14 self.assertEqual(manager.services, {
16default_context = {15 'service1': {'service': 'service1',
17 'nats': {16 'foo': 'bar'},
18 'nats_port': '1234',17 'service2': {'service': 'service2',
19 'nats_host': 'example.com',18 'qux': 'baz'},
20 },19 })
21 'router': {20
22 'domain': 'api.foo.com'21 @mock.patch.object(services.ServiceManager, 'reconfigure_services')
23 }22 @mock.patch.object(services.ServiceManager, 'stop_services')
24}23 @mock.patch.object(services, 'sys')
2524 def test_manage_stop(self, msys, stop_services, reconfigure_services):
2625 manager = services.ServiceManager()
27# Method returning a mutable copy of a default config26 msys.argv = ['charm_dir/hooks/stop']
28# for services.register()27 manager.manage()
29def default_service_config():28 stop_services.assert_called_once_with()
30 return [{29 assert not reconfigure_services.called
31 'service': 'cf-cloudcontroller',30
32 'templates': [31 @mock.patch.object(services.ServiceManager, 'reconfigure_services')
33 {'target': 'cc.yml',32 @mock.patch.object(services.ServiceManager, 'stop_services')
34 'source': 'fake_cc.yml',33 @mock.patch.object(services, 'sys')
35 'file_properties': {},34 def test_manage_other(self, msys, stop_services, reconfigure_services):
36 'contexts': [static_content(default_context)]35 manager = services.ServiceManager()
37 }]36 msys.argv = ['charm_dir/hooks/config-changed']
38 }]37 manager.manage()
3938 assert not stop_services.called
4039 reconfigure_services.assert_called_once_with()
41class TestService(unittest.TestCase):40
41 @mock.patch.object(services.ServiceManager, 'save_ready')
42 @mock.patch.object(services.ServiceManager, 'fire_event')
43 @mock.patch.object(services.ServiceManager, 'is_ready')
44 def test_reconfigure_ready(self, is_ready, fire_event, save_ready):
45 manager = services.ServiceManager([
46 {'service': 'service1'}, {'service': 'service2'}])
47 is_ready.return_value = True
48 manager.reconfigure_services()
49 is_ready.assert_has_calls([
50 mock.call('service1'),
51 mock.call('service2'),
52 ], any_order=True)
53 fire_event.assert_has_calls([
54 mock.call('data_ready', 'service1'),
55 mock.call('start', 'service1', default=[
56 services.host.service_restart,
57 services.open_ports]),
58 ], any_order=False)
59 fire_event.assert_has_calls([
60 mock.call('data_ready', 'service2'),
61 mock.call('start', 'service2', default=[
62 services.host.service_restart,
63 services.open_ports]),
64 ], any_order=False)
65 save_ready.assert_has_calls([
66 mock.call('service1'),
67 mock.call('service2'),
68 ], any_order=True)
69
70 @mock.patch.object(services.ServiceManager, 'save_ready')
71 @mock.patch.object(services.ServiceManager, 'fire_event')
72 @mock.patch.object(services.ServiceManager, 'is_ready')
73 def test_reconfigure_ready_list(self, is_ready, fire_event, save_ready):
74 manager = services.ServiceManager([
75 {'service': 'service1'}, {'service': 'service2'}])
76 is_ready.return_value = True
77 manager.reconfigure_services('service3', 'service4')
78 self.assertEqual(is_ready.call_args_list, [
79 mock.call('service3'),
80 mock.call('service4'),
81 ])
82 self.assertEqual(fire_event.call_args_list, [
83 mock.call('data_ready', 'service3'),
84 mock.call('start', 'service3', default=[
85 services.host.service_restart,
86 services.open_ports]),
87 mock.call('data_ready', 'service4'),
88 mock.call('start', 'service4', default=[
89 services.host.service_restart,
90 services.open_ports]),
91 ])
92 self.assertEqual(save_ready.call_args_list, [
93 mock.call('service3'),
94 mock.call('service4'),
95 ])
96
97 @mock.patch.object(services.ServiceManager, 'save_lost')
98 @mock.patch.object(services.ServiceManager, 'fire_event')
99 @mock.patch.object(services.ServiceManager, 'was_ready')
100 @mock.patch.object(services.ServiceManager, 'is_ready')
101 def test_reconfigure_not_ready(self, is_ready, was_ready, fire_event, save_lost):
102 manager = services.ServiceManager([
103 {'service': 'service1'}, {'service': 'service2'}])
104 is_ready.return_value = False
105 was_ready.return_value = False
106 manager.reconfigure_services()
107 is_ready.assert_has_calls([
108 mock.call('service1'),
109 mock.call('service2'),
110 ], any_order=True)
111 fire_event.assert_has_calls([
112 mock.call('stop', 'service1', default=[
113 services.close_ports,
114 services.host.service_stop]),
115 mock.call('stop', 'service2', default=[
116 services.close_ports,
117 services.host.service_stop]),
118 ], any_order=True)
119 save_lost.assert_has_calls([
120 mock.call('service1'),
121 mock.call('service2'),
122 ], any_order=True)
123
124 @mock.patch.object(services.ServiceManager, 'save_lost')
125 @mock.patch.object(services.ServiceManager, 'fire_event')
126 @mock.patch.object(services.ServiceManager, 'was_ready')
127 @mock.patch.object(services.ServiceManager, 'is_ready')
128 def test_reconfigure_no_longer_ready(self, is_ready, was_ready, fire_event, save_lost):
129 manager = services.ServiceManager([
130 {'service': 'service1'}, {'service': 'service2'}])
131 is_ready.return_value = False
132 was_ready.return_value = True
133 manager.reconfigure_services()
134 is_ready.assert_has_calls([
135 mock.call('service1'),
136 mock.call('service2'),
137 ], any_order=True)
138 fire_event.assert_has_calls([
139 mock.call('data_lost', 'service1'),
140 mock.call('stop', 'service1', default=[
141 services.close_ports,
142 services.host.service_stop]),
143 ], any_order=False)
144 fire_event.assert_has_calls([
145 mock.call('data_lost', 'service2'),
146 mock.call('stop', 'service2', default=[
147 services.close_ports,
148 services.host.service_stop]),
149 ], any_order=False)
150 save_lost.assert_has_calls([
151 mock.call('service1'),
152 mock.call('service2'),
153 ], any_order=True)
154
155 @mock.patch.object(services.ServiceManager, 'fire_event')
156 def test_stop_services(self, fire_event):
157 manager = services.ServiceManager([
158 {'service': 'service1'}, {'service': 'service2'}])
159 manager.stop_services()
160 fire_event.assert_has_calls([
161 mock.call('stop', 'service1', default=[
162 services.close_ports,
163 services.host.service_stop]),
164 mock.call('stop', 'service2', default=[
165 services.close_ports,
166 services.host.service_stop]),
167 ], any_order=True)
168
169 @mock.patch.object(services.ServiceManager, 'fire_event')
170 def test_stop_services_list(self, fire_event):
171 manager = services.ServiceManager([
172 {'service': 'service1'}, {'service': 'service2'}])
173 manager.stop_services('service3', 'service4')
174 self.assertEqual(fire_event.call_args_list, [
175 mock.call('stop', 'service3', default=[
176 services.close_ports,
177 services.host.service_stop]),
178 mock.call('stop', 'service4', default=[
179 services.close_ports,
180 services.host.service_stop]),
181 ])
182
183 def test_get_service(self):
184 service = {'service': 'test', 'test': 'test_service'}
185 manager = services.ServiceManager([service])
186 self.assertEqual(manager.get_service('test'), service)
187
188 def test_get_service_not_registered(self):
189 service = {'service': 'test', 'test': 'test_service'}
190 manager = services.ServiceManager([service])
191 self.assertRaises(KeyError, manager.get_service, 'foo')
192
193 @mock.patch.object(services.ServiceManager, 'get_service')
194 def test_fire_event_default(self, get_service):
195 get_service.return_value = {}
196 cb = mock.Mock()
197 manager = services.ServiceManager()
198 manager.fire_event('event', 'service', cb)
199 cb.assert_called_once_with('service')
200
201 @mock.patch.object(services.ServiceManager, 'get_service')
202 def test_fire_event_default_list(self, get_service):
203 get_service.return_value = {}
204 cb = mock.Mock()
205 manager = services.ServiceManager()
206 manager.fire_event('event', 'service', [cb])
207 cb.assert_called_once_with('service')
208
209 @mock.patch.object(services.ServiceManager, 'get_service')
210 def test_fire_event_simple_callback(self, get_service):
211 cb = mock.Mock()
212 dcb = mock.Mock()
213 get_service.return_value = {'event': cb}
214 manager = services.ServiceManager()
215 manager.fire_event('event', 'service', dcb)
216 assert not dcb.called
217 cb.assert_called_once_with('service')
218
219 @mock.patch.object(services.ServiceManager, 'get_service')
220 def test_fire_event_simple_callback_list(self, get_service):
221 cb = mock.Mock()
222 dcb = mock.Mock()
223 get_service.return_value = {'event': [cb]}
224 manager = services.ServiceManager()
225 manager.fire_event('event', 'service', dcb)
226 assert not dcb.called
227 cb.assert_called_once_with('service')
228
229 @mock.patch.object(services.ManagerCallback, '__call__')
230 @mock.patch.object(services.ServiceManager, 'get_service')
231 def test_fire_event_manager_callback(self, get_service, mcall):
232 cb = services.ManagerCallback()
233 dcb = mock.Mock()
234 get_service.return_value = {'event': cb}
235 manager = services.ServiceManager()
236 manager.fire_event('event', 'service', dcb)
237 assert not dcb.called
238 mcall.assert_called_once_with(manager, 'service', 'event')
239
240 @mock.patch.object(services.ManagerCallback, '__call__')
241 @mock.patch.object(services.ServiceManager, 'get_service')
242 def test_fire_event_manager_callback_list(self, get_service, mcall):
243 cb = services.ManagerCallback()
244 dcb = mock.Mock()
245 get_service.return_value = {'event': [cb]}
246 manager = services.ServiceManager()
247 manager.fire_event('event', 'service', dcb)
248 assert not dcb.called
249 mcall.assert_called_once_with(manager, 'service', 'event')
250
251 @mock.patch.object(services.ServiceManager, 'get_service')
252 def test_is_ready(self, get_service):
253 get_service.side_effect = [
254 {},
255 {'required_data': [True]},
256 {'required_data': [False]},
257 {'required_data': [True, False]},
258 ]
259 manager = services.ServiceManager()
260 assert manager.is_ready('foo')
261 assert manager.is_ready('bar')
262 assert not manager.is_ready('foo')
263 assert not manager.is_ready('foo')
264 get_service.assert_has_calls([mock.call('foo'), mock.call('bar')])
265
266 @mock.patch.object(services.hookenv, 'charm_dir')
267 @mock.patch.object(services, 'open', create=True)
268 def test_save_ready(self, mopen, charm_dir):
269 charm_dir.return_value = 'charm_dir'
270 manager = services.ServiceManager()
271 manager.save_ready('foo')
272 mopen.assert_called_once_with('charm_dir/.ready.foo', 'a')
273
274 @mock.patch.object(services.hookenv, 'charm_dir')
275 @mock.patch('os.remove')
276 @mock.patch('os.path.exists')
277 def test_save_lost(self, exists, remove, charm_dir):
278 charm_dir.return_value = 'charm_dir'
279 manager = services.ServiceManager()
280 manager.save_lost('foo')
281 exists.assert_called_once_with('charm_dir/.ready.foo')
282 remove.assert_called_once_with('charm_dir/.ready.foo')
283
284 @mock.patch.object(services.hookenv, 'charm_dir')
285 @mock.patch('os.path.exists')
286 def test_was_ready(self, exists, charm_dir):
287 charm_dir.return_value = 'charm_dir'
288 manager = services.ServiceManager()
289 manager.was_ready('foo')
290 exists.assert_called_once_with('charm_dir/.ready.foo')
291
292
293class TestRelationContext(unittest.TestCase):
42 def setUp(self):294 def setUp(self):
43 services.SERVICES = {}295 self.context = services.RelationContext()
44296 self.context.interface = 'http'
45 @mock.patch('charmhelpers.core.host.service_restart')297 self.context.required_keys = ['foo', 'bar']
46 @mock.patch('charmhelpers.core.services.templating')298
47 def test_register_no_target(self, mtemplating, mservice_restart):299 @mock.patch.object(services, 'hookenv')
48 config = default_service_config()300 def test_no_relations(self, mhookenv):
49 del config[0]['templates'][0]['target']301 mhookenv.relation_ids.return_value = []
50 services.register(config, TEMPLATES_DIR)302 self.context.get_data()
51 services.reconfigure_services()303 self.assertFalse(self.context.is_ready())
52 self.assertEqual(mtemplating.render.call_args[0][0][0]['target'],304 self.assertEqual(self.context, {})
53 '/etc/init/cf-cloudcontroller.conf')305 mhookenv.relation_ids.assert_called_once_with('http')
54 self.assertEqual(mtemplating.render.call_args[0][1], TEMPLATES_DIR)306
55307 @mock.patch.object(services, 'hookenv')
56 @mock.patch('charmhelpers.core.host.service_restart')308 def test_no_units(self, mhookenv):
57 @mock.patch('charmhelpers.core.services.templating')309 mhookenv.relation_ids.return_value = ['nginx']
58 def test_register_default_tmpl_dir(self, mtemplating, mservice_restart):310 mhookenv.related_units.return_value = []
59 config = default_service_config()311 self.context.get_data()
60 services.register(config)312 self.assertFalse(self.context.is_ready())
61 services.reconfigure_services()313 self.assertEqual(self.context, {'http': {}})
62 self.assertEqual(mtemplating.render.call_args[0][1], None)314
63315 @mock.patch.object(services, 'hookenv')
64 @mock.patch('charmhelpers.core.services.templating')316 def test_incomplete(self, mhookenv):
65 @mock.patch('charmhelpers.core.services.reconfigure_service')317 mhookenv.relation_ids.return_value = ['nginx', 'apache']
66 def test_reconfigure_services(self, mreconfig, mtemplating):318 mhookenv.related_units.side_effect = lambda i: [i+'/0']
67 services.register(default_service_config(), TEMPLATES_DIR)319 mhookenv.relation_get.side_effect = [{}, {'foo': '1'}]
68 services.reconfigure_services()320 self.assertFalse(bool(self.context))
69 mreconfig.assert_called_once_with('cf-cloudcontroller', restart=True)321 self.assertEqual(mhookenv.relation_get.call_args_list, [
70322 mock.call(rid='nginx', unit='nginx/0'),
71 @mock.patch('charmhelpers.core.services.templating')323 mock.call(rid='apache', unit='apache/0'),
72 @mock.patch('charmhelpers.core.host.service_restart')324 ])
73 def test_reconfigure_service_restart(self, mrestart, mtemplating):325
74 services.register(default_service_config(), TEMPLATES_DIR)326 @mock.patch.object(services, 'hookenv')
75 services.reconfigure_service('cf-cloudcontroller', restart=True)327 def test_complete(self, mhookenv):
76 mrestart.assert_called_once_with('cf-cloudcontroller')328 mhookenv.relation_ids.return_value = ['nginx', 'apache', 'tomcat']
77329 mhookenv.related_units.side_effect = lambda i: [i+'/0']
78 @mock.patch('charmhelpers.core.services.templating')330 mhookenv.relation_get.side_effect = [{'foo': '1'}, {'foo': '2', 'bar': '3'}, {}]
79 @mock.patch('charmhelpers.core.host.service_restart')331 self.context.get_data()
80 def test_reconfigure_service_no_restart(self, mrestart, mtemplating):332 self.assertEqual(self.context, {'http': {
81 services.register(default_service_config(), TEMPLATES_DIR)333 'foo': '2',
82 services.reconfigure_service('cf-cloudcontroller', restart=False)334 'bar': '3',
83 self.assertFalse(mrestart.called)335 'nginx/0': {
84336 'foo': '1',
85 @mock.patch('charmhelpers.core.services.templating')337 },
86 @mock.patch('charmhelpers.core.host.service_restart')338 'apache/0': {
87 def test_reconfigure_service_incomplete(self, mrestart, mtemplating):339 'foo': '2',
88 config = default_service_config()340 'bar': '3',
89 mtemplating.render.return_value = False # render fails when incomplete341 },
90 services.register(config, TEMPLATES_DIR)342 'tomcat/0': {
91 services.reconfigure_service('cf-cloudcontroller', restart=True)343 },
92 # verify that we did not restart the service344 }})
93 self.assertFalse(mrestart.called)345 mhookenv.relation_ids.assert_called_with('http')
94346 self.assertEqual(mhookenv.relation_get.call_args_list, [
95 @mock.patch('charmhelpers.core.services.templating')347 mock.call(rid='nginx', unit='nginx/0'),
96 @mock.patch('charmhelpers.core.host.service_restart')348 mock.call(rid='apache', unit='apache/0'),
97 def test_reconfigure_service_no_context(self, mrestart, mtemplating):349 mock.call(rid='tomcat', unit='tomcat/0'),
98 config = default_service_config()350 ])
99 services.register(config, 'foo')351
100 services.reconfigure_service('cf-cloudcontroller', restart=False)352
101 # verify that we called render template with the expected name353class TestTemplateCallback(unittest.TestCase):
102 mtemplating.render.assert_called_once_with(config[0]['templates'], 'foo')354 @mock.patch.object(services, 'templating')
103 self.assertFalse(mrestart.called)355 def test_template_defaults(self, mtemplating):
104 self.assertRaises(KeyError, services.reconfigure_service, 'unknownservice', restart=False)356 manager = mock.Mock(**{'get_service.return_value': {
105357 'required_data': [{'foo': 'bar'}]}})
106 @mock.patch('charmhelpers.core.services.templating')358 self.assertRaises(TypeError, services.template, source='foo.yml')
107 @mock.patch('charmhelpers.core.host.service_restart')359 callback = services.template(source='foo.yml', target='bar.yml')
108 def test_custom_service_type(self, mrestart, mtemplating):360 assert isinstance(callback, services.ManagerCallback)
109 config = default_service_config()361 assert not mtemplating.render.called
110 config[0]['type'] = mock.Mock(name='CustomService')362 callback(manager, 'test', 'event')
111 services.register(config, 'foo')363 mtemplating.render.assert_called_once_with(
112 services.reconfigure_service('cf-cloudcontroller', restart=False)364 'foo.yml', 'bar.yml', {'foo': 'bar'},
113 config[0]['type'].assert_called_once_with(config[0])365 'root', 'root', 0444)
114 config[0]['type'].return_value.reconfigure.assert_called_once_with(False)366
115367 @mock.patch.object(services, 'templating')
116 @mock.patch('charmhelpers.core.services.templating')368 def test_template_explicit(self, mtemplating):
117 @mock.patch('charmhelpers.core.host.service_stop')369 manager = mock.Mock(**{'get_service.return_value': {
118 @mock.patch('charmhelpers.core.host.service_running')370 'required_data': [{'foo': 'bar'}]}})
119 def test_stop_services(self, mrunning, mstop, mtemplating):371 callback = services.template(
120 services.register(default_service_config(), TEMPLATES_DIR)372 source='foo.yml', target='bar.yml',
121 services.stop_services()373 owner='user', group='group', perms=0555
122 mrunning.assert_called_once_with('cf-cloudcontroller')374 )
123 mstop.assert_called_once_with('cf-cloudcontroller')375 assert isinstance(callback, services.ManagerCallback)
376 assert not mtemplating.render.called
377 callback(manager, 'test', 'event')
378 mtemplating.render.assert_called_once_with(
379 'foo.yml', 'bar.yml', {'foo': 'bar'},
380 'user', 'group', 0555)
381
382
383class TestPortsCallback(unittest.TestCase):
384 @mock.patch.object(services, 'hookenv')
385 def test_no_ports(self, hookenv):
386 manager = mock.Mock(**{'get_service.return_value': {}})
387 services.PortManagerCallback()(manager, 'service', 'event')
388 assert not hookenv.open_port.called
389 assert not hookenv.close_port.called
390
391 @mock.patch.object(services, 'hookenv')
392 def test_open_ports(self, hookenv):
393 manager = mock.Mock(**{'get_service.return_value': {'ports': [1, 2]}})
394 services.open_ports(manager, 'service', 'start')
395 hookenv.open_port.has_calls([mock.call(1), mock.call(2)])
396 assert not hookenv.close_port.called
397
398 @mock.patch.object(services, 'hookenv')
399 def test_close_ports(self, hookenv):
400 manager = mock.Mock(**{'get_service.return_value': {'ports': [1, 2]}})
401 services.close_ports(manager, 'service', 'stop')
402 assert not hookenv.open_port.called
403 hookenv.close_port.has_calls([mock.call(1), mock.call(2)])
124404
125if __name__ == '__main__':405if __name__ == '__main__':
126 unittest.main()406 unittest.main()
127407
=== modified file 'tests/core/test_templating.py'
--- tests/core/test_templating.py 2014-05-16 21:48:06 +0000
+++ tests/core/test_templating.py 2014-05-29 17:24:14 +0000
@@ -1,7 +1,7 @@
1import os
2import pkg_resources1import pkg_resources
3import tempfile2import tempfile
4import unittest3import unittest
4import jinja2
55
6import mock6import mock
7from charmhelpers.core import templating7from charmhelpers.core import templating
@@ -10,30 +10,6 @@
10TEMPLATES_DIR = pkg_resources.resource_filename(__name__, 'templates')10TEMPLATES_DIR = pkg_resources.resource_filename(__name__, 'templates')
1111
1212
13def noop():
14 return {}
15
16
17def found():
18 return {'foo': 'bar'}
19
20
21DEFAULT_CONTEXT = {
22 'nats': {
23 'nats_port': '1234',
24 'nats_host': 'example.com',
25 },
26 'router': {
27 'domain': 'api.foo.com'
28 },
29 'nginx_port': 80,
30}
31
32
33def default_context_provider():
34 return DEFAULT_CONTEXT
35
36
37class TestTemplating(unittest.TestCase):13class TestTemplating(unittest.TestCase):
38 def setUp(self):14 def setUp(self):
39 self.charm_dir = pkg_resources.resource_filename(__name__, '')15 self.charm_dir = pkg_resources.resource_filename(__name__, '')
@@ -50,164 +26,33 @@
50 def test_render(self, log, mkdir, fchown):26 def test_render(self, log, mkdir, fchown):
51 _, fn1 = tempfile.mkstemp()27 _, fn1 = tempfile.mkstemp()
52 _, fn2 = tempfile.mkstemp()28 _, fn2 = tempfile.mkstemp()
53 template_definitions = [29 context = {
54 {'source': 'fake_cc.yml',30 'nats': {
55 'target': fn1,31 'nats_port': '1234',
56 'contexts': [default_context_provider]},32 'nats_host': 'example.com',
57 {'source': 'test.conf',33 },
58 'target': fn2,34 'router': {
59 'contexts': [default_context_provider]},35 'domain': 'api.foo.com'
60 ]36 },
61 self.assertTrue(templating.render(template_definitions))37 'nginx_port': 80,
38 }
39 templating.render('fake_cc.yml', fn1, context, templates_dir=TEMPLATES_DIR)
62 contents = open(fn1).read()40 contents = open(fn1).read()
63 self.assertRegexpMatches(contents, 'port: 1234')41 self.assertRegexpMatches(contents, 'port: 1234')
64 self.assertRegexpMatches(contents, 'host: example.com')42 self.assertRegexpMatches(contents, 'host: example.com')
65 self.assertRegexpMatches(contents, 'domain: api.foo.com')43 self.assertRegexpMatches(contents, 'domain: api.foo.com')
44
45 templating.render('test.conf', fn2, context, templates_dir=TEMPLATES_DIR)
66 contents = open(fn2).read()46 contents = open(fn2).read()
67 self.assertRegexpMatches(contents, 'listen 80')47 self.assertRegexpMatches(contents, 'listen 80')
68 self.assertEqual(fchown.call_count, 2)48 self.assertEqual(fchown.call_count, 2)
69 self.assertEqual(mkdir.call_count, 2)49 self.assertEqual(mkdir.call_count, 2)
7050
71 @mock.patch.object(templating.host.os, 'fchown')51 @mock.patch.object(templating, 'hookenv')
72 @mock.patch.object(templating.host, 'mkdir')52 @mock.patch('jinja2.Environment')
73 @mock.patch.object(templating.host, 'log')53 def test_load_error(self, Env, hookenv):
74 def test_render_incomplete(self, log, mkdir, fchown):54 Env().get_template.side_effect = jinja2.exceptions.TemplateNotFound('fake_cc.yml')
75 _, fn1 = tempfile.mkstemp()55 self.assertRaises(
76 _, fn2 = tempfile.mkstemp()56 jinja2.exceptions.TemplateNotFound, templating.render,
77 os.remove(fn1)57 'fake.src', 'fake.tgt', {}, templates_dir='tmpl')
78 os.remove(fn2)58 hookenv.log.assert_called_once_with('Could not load template fake.src from tmpl.', level=hookenv.ERROR)
79 template_definitions = [
80 {'source': 'fake_cc.yml',
81 'target': fn1,
82 'contexts': [lambda: {}]},
83 {'source': 'test.conf',
84 'target': fn2,
85 'contexts': [default_context_provider]},
86 ]
87 self.assertFalse(templating.render(template_definitions))
88 self.assertFalse(os.path.exists(fn1))
89 contents = open(fn2).read()
90 self.assertRegexpMatches(contents, 'listen 80')
91 self.assertEqual(fchown.call_count, 1)
92 self.assertEqual(mkdir.call_count, 1)
93
94 @mock.patch('jinja2.Environment')
95 @mock.patch.object(templating.host, 'mkdir')
96 @mock.patch.object(templating.host, 'write_file')
97 def test_render_no_source(self, write_file, mkdir, Env):
98 template_definitions = [{
99 'target': 'fake_cc.yml',
100 'contexts': [default_context_provider],
101 }]
102 self.assertTrue(templating.render(template_definitions))
103 Env().get_template.assert_called_once_with('fake_cc.yml.j2')
104 Env().get_template.return_value.render.assert_called_once()
105 write_file.assert_called_once()
106
107 @mock.patch('jinja2.Environment')
108 @mock.patch.object(templating.host, 'mkdir')
109 @mock.patch.object(templating.host, 'write_file')
110 def test_render_no_contexts(self, write_file, mkdir, Env):
111 template_definitions = [{
112 'target': 'fake_cc.yml',
113 'contexts': [],
114 }]
115 self.assertTrue(templating.render(template_definitions))
116 Env().get_template.assert_called_once_with('fake_cc.yml.j2')
117 Env().get_template.return_value.render.assert_called_once_with({})
118 write_file.assert_called_once()
119
120 @mock.patch('jinja2.FileSystemLoader')
121 @mock.patch('jinja2.Environment')
122 @mock.patch.object(templating.host, 'mkdir')
123 @mock.patch.object(templating.host, 'write_file')
124 def test_render_implicit_dir(self, write_file, mkdir, Env, FSL):
125 self.charm_dir = 'foo'
126 template_definitions = [{
127 'target': 'fake_cc.yml',
128 'contexts': [default_context_provider],
129 }]
130 self.assertTrue(templating.render(template_definitions))
131 FSL.assert_called_once_with('foo/templates')
132
133 @mock.patch('jinja2.FileSystemLoader')
134 @mock.patch('jinja2.Environment')
135 @mock.patch.object(templating.host, 'mkdir')
136 @mock.patch.object(templating.host, 'write_file')
137 def test_render_explicit_dir(self, write_file, mkdir, Env, FSL):
138 self.charm_dir = 'foo'
139 template_definitions = [{
140 'target': 'fake_cc.yml',
141 'contexts': [default_context_provider],
142 }]
143 self.assertTrue(templating.render(template_definitions, 'bar'))
144 FSL.assert_called_once_with('bar')
145
146 def test_collect_contexts_fail(self):
147 cc = templating._collect_contexts
148 self.assertEqual(cc([]), {})
149 self.assertEqual(cc([noop]), False)
150 self.assertEqual(cc([noop, noop]), False)
151 self.assertEqual(cc([noop, found]), False)
152
153 def test_collect_contexts_found(self):
154 cc = templating._collect_contexts
155 expected = {'foo': 'bar'}
156 self.assertEqual(cc([found]), expected)
157 self.assertEqual(cc([found, found]), expected)
158
159
160class TestConfigContext(unittest.TestCase):
161 @mock.patch('charmhelpers.core.hookenv.config')
162 def test_config_context(self, mconfig):
163 templating.ConfigContext()()
164 self.assertTrue(mconfig.called)
165
166
167class TestStaticContext(unittest.TestCase):
168 def test_static_context(self):
169 a = templating.StaticContext('a')
170 self.assertEqual(a.data, 'a')
171 self.assertEqual(a(), 'a')
172
173
174class TestRelationContext(unittest.TestCase):
175 def setUp(self):
176 self.context_provider = templating.RelationContext()
177 self.context_provider.interface = 'http'
178 self.context_provider.required_keys = ['foo', 'bar']
179
180 @mock.patch.object(templating, 'hookenv')
181 def test_no_relations(self, mhookenv):
182 mhookenv.relation_ids.return_value = []
183 self.assertEqual(self.context_provider(), {})
184 mhookenv.relation_ids.assert_called_once_with('http')
185
186 @mock.patch.object(templating, 'hookenv')
187 def test_no_units(self, mhookenv):
188 mhookenv.relation_ids.return_value = ['nginx']
189 mhookenv.related_units.return_value = []
190 self.assertEqual(self.context_provider(), {})
191
192 @mock.patch.object(templating, 'hookenv')
193 def test_incomplete(self, mhookenv):
194 mhookenv.relation_ids.return_value = ['nginx', 'apache']
195 mhookenv.related_units.side_effect = lambda i: [i+'/0']
196 mhookenv.relation_get.side_effect = [{}, {'foo': '1'}]
197 self.assertEqual(self.context_provider(), {})
198 self.assertEqual(mhookenv.relation_get.call_args_list, [
199 mock.call(rid='nginx', unit='nginx/0'),
200 mock.call(rid='apache', unit='apache/0'),
201 ])
202
203 @mock.patch.object(templating, 'hookenv')
204 def test_complete(self, mhookenv):
205 mhookenv.relation_ids.return_value = ['nginx', 'apache', 'tomcat']
206 mhookenv.related_units.side_effect = lambda i: [i+'/0']
207 mhookenv.relation_get.side_effect = [{'foo': '1'}, {'foo': '2', 'bar': '3'}, {}]
208 self.assertEqual(self.context_provider(), {'http': {'foo': '2', 'bar': '3'}})
209 mhookenv.relation_ids.assert_called_with('http')
210 self.assertEqual(mhookenv.relation_get.call_args_list, [
211 mock.call(rid='nginx', unit='nginx/0'),
212 mock.call(rid='apache', unit='apache/0'),
213 ])
21459
=== modified file 'tox.ini'
--- tox.ini 2014-05-09 16:10:54 +0000
+++ tox.ini 2014-05-29 17:24:14 +0000
@@ -2,13 +2,13 @@
2envlist = py272envlist = py27
33
4[testenv]4[testenv]
5install_command=pip install --pre {opts} 5install_command=pip install --pre {opts}
6 --allow-all-external 6 --allow-all-external
7 --allow-unverified launchpadlib 7 --allow-unverified launchpadlib
8 --allow-unverified python-apt 8 --allow-unverified python-apt
9 --allow-unverified bzr 9 --allow-unverified bzr
10 --allow-unverified lazr.authentication10 --allow-unverified lazr.authentication
11 {packages} 11 {packages}
12deps=-r{toxinidir}/test-requirements-tox.txt12deps=-r{toxinidir}/test-requirements-tox.txt
13commands =13commands =
14 nosetests --nologcapture 14 nosetests --nologcapture {posargs}

Subscribers

People subscribed via source and target branches