Merge lp:~johnsca/charm-helpers/services-framework into lp:charm-helpers

[Cory Johns]

Split services framework off into separate merge proposal, and combined commits, for easier review.

[Cory Johns (johnsca)]

This MP includes the most recent version of the Services framework, which changes the focus of charms from handling charm events (hooks) to describing the data (and sources of that data) required to set up the software and the actions to take when all of the required data is available. It also creates a standard for rendering config and services jobs based on Jinja templates, and manages re-rendering the files and restarting the services when the data changes.

This relieves the charm author of having to do things like keep track of a bunch of .foo flag files for indicating whether or not such-and-such file has been written or such-and-such service has been started.

The docstrings below are fairly complete, and some (somewhat simple) real-world example usages can be found in the Apache Allura charm (http://bazaar.launchpad.net/~johnsca/charms/precise/apache-allura/refactoring-with-tests/files) and the RethinkDB Docker charm (https://github.com/bcsaller/juju-docker/).

[James Troup (elmo)]

Comments inline

[Cory Johns (johnsca)]

Replies / comments inline.

[lp:~johnsca/charm-helpers/services-framework]

180. By Cory Johns on 2014-07-10

Cleanup based on review

[Tim Van Steenburgh (tvansteenburgh)]

+1 LGTM.

This is awesome, I can't wait to try it out! I found a few typos (see inline diff comments), but other than that, this is good-to-go. Tests all pass.

Looking forward to implementing a charm with this to get a better feel for it. You might consider adding more examples to the published docs (http://pythonhosted.org/charmhelpers/) if you get time.

Very nice work!

review: Approve

[lp:~johnsca/charm-helpers/services-framework]

181. By Cory Johns on 2014-08-05

Fixed documentation typos, per review

[Charles Butler (lazypower)]

Looks good to me. I'm on teh band wagon for using this as well.

I see nothing truly heinous in the code base, and considering there are already a slew of CF charms relying on this - I'd like to get more traction. Merging so we can pilot this officially in charms and get a charm-school published on this.

Make sure you're watching the charm-helpers bug tracker for any incoming bugs related to the services framework, and thank you for helping solve some long running issues with a very unique approach to service declaration, implementation, and constraints (on a per service level via relation even!)

+1

review: Approve

=== modified file 'charmhelpers/core/host.py'
--- charmhelpers/core/host.py	2014-07-23 11:18:54 +0000
+++ charmhelpers/core/host.py	2014-08-05 13:03:33 +0000
@@ -12,6 +12,8 @@
 import string
 import subprocess
 import hashlib
+import shutil
+from contextlib import contextmanager
 
 from collections import OrderedDict
 
@@ -52,7 +54,7 @@
 def service_running(service):
     """Determine whether a system service is running"""
     try:
-        output = subprocess.check_output(['service', service, 'status'])
+        output = subprocess.check_output(['service', service, 'status'], stderr=subprocess.STDOUT)
     except subprocess.CalledProcessError:
         return False
     else:
@@ -62,6 +64,16 @@
             return False
 
 
+def service_available(service_name):
+    """Determine whether a system service is available"""
+    try:
+        subprocess.check_output(['service', service_name, 'status'], stderr=subprocess.STDOUT)
+    except subprocess.CalledProcessError:
+        return False
+    else:
+        return True
+
+
 def adduser(username, password=None, shell='/bin/bash', system_user=False):
     """Add a user to the system"""
     try:
@@ -329,3 +341,24 @@
         pkgcache = apt_pkg.Cache()
     pkg = pkgcache[package]
     return apt_pkg.version_compare(pkg.current_ver.ver_str, revno)
+
+
+@contextmanager
+def chdir(d):
+    cur = os.getcwd()
+    try:
+        yield os.chdir(d)
+    finally:
+        os.chdir(cur)
+
+
+def chownr(path, owner, group):
+    uid = pwd.getpwnam(owner).pw_uid
+    gid = grp.getgrnam(group).gr_gid
+
+    for root, dirs, files in os.walk(path):
+        for name in dirs + files:
+            full = os.path.join(root, name)
+            broken_symlink = os.path.lexists(full) and not os.path.exists(full)
+            if not broken_symlink:
+                os.chown(full, uid, gid)

=== added directory 'charmhelpers/core/services'
=== added file 'charmhelpers/core/services/__init__.py'
--- charmhelpers/core/services/__init__.py	1970-01-01 00:00:00 +0000
+++ charmhelpers/core/services/__init__.py	2014-08-05 13:03:33 +0000
@@ -0,0 +1,2 @@
+from .base import *
+from .helpers import *

=== added file 'charmhelpers/core/services/base.py'
--- charmhelpers/core/services/base.py	1970-01-01 00:00:00 +0000
+++ charmhelpers/core/services/base.py	2014-08-05 13:03:33 +0000
@@ -0,0 +1,305 @@
+import os
+import re
+import json
+from collections import Iterable
+
+from charmhelpers.core import host
+from charmhelpers.core import hookenv
+
+
+__all__ = ['ServiceManager', 'ManagerCallback',
+           'PortManagerCallback', 'open_ports', 'close_ports', 'manage_ports',
+           'service_restart', 'service_stop']
+
+
+class ServiceManager(object):
+    def __init__(self, services=None):
+        """
+        Register a list of services, given their definitions.
+
+        Traditional charm authoring is focused on implementing hooks.  That is,
+        the charm author is thinking in terms of "What hook am I handling; what
+        does this hook need to do?"  However, in most cases, the real question
+        should be "Do I have the information I need to configure and start this
+        piece of software and, if so, what are the steps for doing so?"  The
+        ServiceManager framework tries to bring the focus to the data and the
+        setup tasks, in the most declarative way possible.
+
+        Service definitions are dicts in the following formats (all keys except
+        'service' are optional):
+
+            {
+                "service": <service name>,
+                "required_data": <list of required data contexts>,
+                "data_ready": <one or more callbacks>,
+                "data_lost": <one or more callbacks>,
+                "start": <one or more callbacks>,
+                "stop": <one or more callbacks>,
+                "ports": <list of ports to manage>,
+            }
+
+        The 'required_data' list should contain dicts of required data (or
+        dependency managers that act like dicts and know how to collect the data).
+        Only when all items in the 'required_data' list are populated are the list
+        of 'data_ready' and 'start' callbacks executed.  See `is_ready()` for more
+        information.
+
+        The 'data_ready' value should be either a single callback, or a list of
+        callbacks, to be called when all items in 'required_data' pass `is_ready()`.
+        Each callback will be called with the service name as the only parameter.
+        After all of the 'data_ready' callbacks are called, the 'start' callbacks
+        are fired.
+
+        The 'data_lost' value should be either a single callback, or a list of
+        callbacks, to be called when a 'required_data' item no longer passes
+        `is_ready()`.  Each callback will be called with the service name as the
+        only parameter.  After all of the 'data_lost' callbacks are called,
+        the 'stop' callbacks are fired.
+
+        The 'start' value should be either a single callback, or a list of
+        callbacks, to be called when starting the service, after the 'data_ready'
+        callbacks are complete.  Each callback will be called with the service
+        name as the only parameter.  This defaults to
+        `[host.service_start, services.open_ports]`.
+
+        The 'stop' value should be either a single callback, or a list of
+        callbacks, to be called when stopping the service.  If the service is
+        being stopped because it no longer has all of its 'required_data', this
+        will be called after all of the 'data_lost' callbacks are complete.
+        Each callback will be called with the service name as the only parameter.
+        This defaults to `[services.close_ports, host.service_stop]`.
+
+        The 'ports' value should be a list of ports to manage.  The default
+        'start' handler will open the ports after the service is started,
+        and the default 'stop' handler will close the ports prior to stopping
+        the service.
+
+
+        Examples:
+
+        The following registers an Upstart service called bingod that depends on
+        a mongodb relation and which runs a custom `db_migrate` function prior to
+        restarting the service, and a Runit service called spadesd.
+
+            manager = services.ServiceManager([
+                {
+                    'service': 'bingod',
+                    'ports': [80, 443],
+                    'required_data': [MongoRelation(), config(), {'my': 'data'}],
+                    'data_ready': [
+                        services.template(source='bingod.conf'),
+                        services.template(source='bingod.ini',
+                                          target='/etc/bingod.ini',
+                                          owner='bingo', perms=0400),
+                    ],
+                },
+                {
+                    'service': 'spadesd',
+                    'data_ready': services.template(source='spadesd_run.j2',
+                                                    target='/etc/sv/spadesd/run',
+                                                    perms=0555),
+                    'start': runit_start,
+                    'stop': runit_stop,
+                },
+            ])
+            manager.manage()
+        """
+        self._ready_file = os.path.join(hookenv.charm_dir(), 'READY-SERVICES.json')
+        self._ready = None
+        self.services = {}
+        for service in services or []:
+            service_name = service['service']
+            self.services[service_name] = service
+
+    def manage(self):
+        """
+        Handle the current hook by doing The Right Thing with the registered services.
+        """
+        hook_name = hookenv.hook_name()
+        if hook_name == 'stop':
+            self.stop_services()
+        else:
+            self.provide_data()
+            self.reconfigure_services()
+
+    def provide_data(self):
+        hook_name = hookenv.hook_name()
+        for service in self.services.values():
+            for provider in service.get('provided_data', []):
+                if re.match(r'{}-relation-(joined|changed)'.format(provider.name), hook_name):
+                    data = provider.provide_data()
+                    if provider._is_ready(data):
+                        hookenv.relation_set(None, data)
+
+    def reconfigure_services(self, *service_names):
+        """
+        Update all files for one or more registered services, and,
+        if ready, optionally restart them.
+
+        If no service names are given, reconfigures all registered services.
+        """
+        for service_name in service_names or self.services.keys():
+            if self.is_ready(service_name):
+                self.fire_event('data_ready', service_name)
+                self.fire_event('start', service_name, default=[
+                    service_restart,
+                    manage_ports])
+                self.save_ready(service_name)
+            else:
+                if self.was_ready(service_name):
+                    self.fire_event('data_lost', service_name)
+                self.fire_event('stop', service_name, default=[
+                    manage_ports,
+                    service_stop])
+                self.save_lost(service_name)
+
+    def stop_services(self, *service_names):
+        """
+        Stop one or more registered services, by name.
+
+        If no service names are given, stops all registered services.
+        """
+        for service_name in service_names or self.services.keys():
+            self.fire_event('stop', service_name, default=[
+                manage_ports,
+                service_stop])
+
+    def get_service(self, service_name):
+        """
+        Given the name of a registered service, return its service definition.
+        """
+        service = self.services.get(service_name)
+        if not service:
+            raise KeyError('Service not registered: %s' % service_name)
+        return service
+
+    def fire_event(self, event_name, service_name, default=None):
+        """
+        Fire a data_ready, data_lost, start, or stop event on a given service.
+        """
+        service = self.get_service(service_name)
+        callbacks = service.get(event_name, default)
+        if not callbacks:
+            return
+        if not isinstance(callbacks, Iterable):
+            callbacks = [callbacks]
+        for callback in callbacks:
+            if isinstance(callback, ManagerCallback):
+                callback(self, service_name, event_name)
+            else:
+                callback(service_name)
+
+    def is_ready(self, service_name):
+        """
+        Determine if a registered service is ready, by checking its 'required_data'.
+
+        A 'required_data' item can be any mapping type, and is considered ready
+        if `bool(item)` evaluates as True.
+        """
+        service = self.get_service(service_name)
+        reqs = service.get('required_data', [])
+        return all(bool(req) for req in reqs)
+
+    def _load_ready_file(self):
+        if self._ready is not None:
+            return
+        if os.path.exists(self._ready_file):
+            with open(self._ready_file) as fp:
+                self._ready = set(json.load(fp))
+        else:
+            self._ready = set()
+
+    def _save_ready_file(self):
+        if self._ready is None:
+            return
+        with open(self._ready_file, 'w') as fp:
+            json.dump(list(self._ready), fp)
+
+    def save_ready(self, service_name):
+        """
+        Save an indicator that the given service is now data_ready.
+        """
+        self._load_ready_file()
+        self._ready.add(service_name)
+        self._save_ready_file()
+
+    def save_lost(self, service_name):
+        """
+        Save an indicator that the given service is no longer data_ready.
+        """
+        self._load_ready_file()
+        self._ready.discard(service_name)
+        self._save_ready_file()
+
+    def was_ready(self, service_name):
+        """
+        Determine if the given service was previously data_ready.
+        """
+        self._load_ready_file()
+        return service_name in self._ready
+
+
+class ManagerCallback(object):
+    """
+    Special case of a callback that takes the `ServiceManager` instance
+    in addition to the service name.
+
+    Subclasses should implement `__call__` which should accept three parameters:
+
+        * `manager`       The `ServiceManager` instance
+        * `service_name`  The name of the service it's being triggered for
+        * `event_name`    The name of the event that this callback is handling
+    """
+    def __call__(self, manager, service_name, event_name):
+        raise NotImplementedError()
+
+
+class PortManagerCallback(ManagerCallback):
+    """
+    Callback class that will open or close ports, for use as either
+    a start or stop action.
+    """
+    def __call__(self, manager, service_name, event_name):
+        service = manager.get_service(service_name)
+        new_ports = service.get('ports', [])
+        port_file = os.path.join(hookenv.charm_dir(), '.{}.ports'.format(service_name))
+        if os.path.exists(port_file):
+            with open(port_file) as fp:
+                old_ports = fp.read().split(',')
+            for old_port in old_ports:
+                if bool(old_port):
+                    old_port = int(old_port)
+                    if old_port not in new_ports:
+                        hookenv.close_port(old_port)
+        with open(port_file, 'w') as fp:
+            fp.write(','.join(str(port) for port in new_ports))
+        for port in new_ports:
+            if event_name == 'start':
+                hookenv.open_port(port)
+            elif event_name == 'stop':
+                hookenv.close_port(port)
+
+
+def service_stop(service_name):
+    """
+    Wrapper around host.service_stop to prevent spurious "unknown service"
+    messages in the logs.
+    """
+    if host.service_running(service_name):
+        host.service_stop(service_name)
+
+
+def service_restart(service_name):
+    """
+    Wrapper around host.service_restart to prevent spurious "unknown service"
+    messages in the logs.
+    """
+    if host.service_available(service_name):
+        if host.service_running(service_name):
+            host.service_restart(service_name)
+        else:
+            host.service_start(service_name)
+
+
+# Convenience aliases
+open_ports = close_ports = manage_ports = PortManagerCallback()

=== added file 'charmhelpers/core/services/helpers.py'
--- charmhelpers/core/services/helpers.py	1970-01-01 00:00:00 +0000
+++ charmhelpers/core/services/helpers.py	2014-08-05 13:03:33 +0000
@@ -0,0 +1,125 @@
+from charmhelpers.core import hookenv
+from charmhelpers.core import templating
+
+from charmhelpers.core.services.base import ManagerCallback
+
+
+__all__ = ['RelationContext', 'TemplateCallback',
+           'render_template', 'template']
+
+
+class RelationContext(dict):
+    """
+    Base class for a context generator that gets relation data from juju.
+
+    Subclasses must provide the attributes `name`, which is the name of the
+    interface of interest, `interface`, which is the type of the interface of
+    interest, and `required_keys`, which is the set of keys required for the
+    relation to be considered complete.  The data for all interfaces matching
+    the `name` attribute that are complete will used to populate the dictionary
+    values (see `get_data`, below).
+
+    The generated context will be namespaced under the interface type, to prevent
+    potential naming conflicts.
+    """
+    name = None
+    interface = None
+    required_keys = []
+
+    def __init__(self, *args, **kwargs):
+        super(RelationContext, self).__init__(*args, **kwargs)
+        self.get_data()
+
+    def __bool__(self):
+        """
+        Returns True if all of the required_keys are available.
+        """
+        return self.is_ready()
+
+    __nonzero__ = __bool__
+
+    def __repr__(self):
+        return super(RelationContext, self).__repr__()
+
+    def is_ready(self):
+        """
+        Returns True if all of the `required_keys` are available from any units.
+        """
+        ready = len(self.get(self.name, [])) > 0
+        if not ready:
+            hookenv.log('Incomplete relation: {}'.format(self.__class__.__name__), hookenv.DEBUG)
+        return ready
+
+    def _is_ready(self, unit_data):
+        """
+        Helper method that tests a set of relation data and returns True if
+        all of the `required_keys` are present.
+        """
+        return set(unit_data.keys()).issuperset(set(self.required_keys))
+
+    def get_data(self):
+        """
+        Retrieve the relation data for each unit involved in a relation and,
+        if complete, store it in a list under `self[self.name]`.  This
+        is automatically called when the RelationContext is instantiated.
+
+        The units are sorted lexographically first by the service ID, then by
+        the unit ID.  Thus, if an interface has two other services, 'db:1'
+        and 'db:2', with 'db:1' having two units, 'wordpress/0' and 'wordpress/1',
+        and 'db:2' having one unit, 'mediawiki/0', all of which have a complete
+        set of data, the relation data for the units will be stored in the
+        order: 'wordpress/0', 'wordpress/1', 'mediawiki/0'.
+
+        If you only care about a single unit on the relation, you can just
+        access it as `{{ interface[0]['key'] }}`.  However, if you can at all
+        support multiple units on a relation, you should iterate over the list,
+        like:
+
+            {% for unit in interface -%}
+                {{ unit['key'] }}{% if not loop.last %},{% endif %}
+            {%- endfor %}
+
+        Note that since all sets of relation data from all related services and
+        units are in a single list, if you need to know which service or unit a
+        set of data came from, you'll need to extend this class to preserve
+        that information.
+        """
+        if not hookenv.relation_ids(self.name):
+            return
+
+        ns = self.setdefault(self.name, [])
+        for rid in sorted(hookenv.relation_ids(self.name)):
+            for unit in sorted(hookenv.related_units(rid)):
+                reldata = hookenv.relation_get(rid=rid, unit=unit)
+                if self._is_ready(reldata):
+                    ns.append(reldata)
+
+    def provide_data(self):
+        """
+        Return data to be relation_set for this interface.
+        """
+        return {}
+
+
+class TemplateCallback(ManagerCallback):
+    """
+    Callback class that will render a template, for use as a ready action.
+    """
+    def __init__(self, source, target, owner='root', group='root', perms=0444):
+        self.source = source
+        self.target = target
+        self.owner = owner
+        self.group = group
+        self.perms = perms
+
+    def __call__(self, manager, service_name, event_name):
+        service = manager.get_service(service_name)
+        context = {}
+        for ctx in service.get('required_data', []):
+            context.update(ctx)
+        templating.render(self.source, self.target, context,
+                          self.owner, self.group, self.perms)
+
+
+# Convenience aliases for templates
+render_template = template = TemplateCallback

=== added file 'charmhelpers/core/templating.py'
--- charmhelpers/core/templating.py	1970-01-01 00:00:00 +0000
+++ charmhelpers/core/templating.py	2014-08-05 13:03:33 +0000
@@ -0,0 +1,51 @@
+import os
+
+from charmhelpers.core import host
+from charmhelpers.core import hookenv
+
+
+def render(source, target, context, owner='root', group='root', perms=0444, templates_dir=None):
+    """
+    Render a template.
+
+    The `source` path, if not absolute, is relative to the `templates_dir`.
+
+    The `target` path should be absolute.
+
+    The context should be a dict containing the values to be replaced in the
+    template.
+
+    The `owner`, `group`, and `perms` options will be passed to `write_file`.
+
+    If omitted, `templates_dir` defaults to the `templates` folder in the charm.
+
+    Note: Using this requires python-jinja2; if it is not installed, calling
+    this will attempt to use charmhelpers.fetch.apt_install to install it.
+    """
+    try:
+        from jinja2 import FileSystemLoader, Environment, exceptions
+    except ImportError:
+        try:
+            from charmhelpers.fetch import apt_install
+        except ImportError:
+            hookenv.log('Could not import jinja2, and could not import '
+                        'charmhelpers.fetch to install it',
+                        level=hookenv.ERROR)
+            raise
+        apt_install('python-jinja2', fatal=True)
+        from jinja2 import FileSystemLoader, Environment, exceptions
+
+    if templates_dir is None:
+        templates_dir = os.path.join(hookenv.charm_dir(), 'templates')
+    loader = Environment(loader=FileSystemLoader(templates_dir))
+    try:
+        source = source
+        template = loader.get_template(source)
+    except exceptions.TemplateNotFound as e:
+        hookenv.log('Could not load template %s from %s.' %
+                    (source, templates_dir),
+                    level=hookenv.ERROR)
+        raise e
+    content = template.render(context)
+    host.mkdir(os.path.dirname(target))
+    host.write_file(target, content, owner, group, perms)

=== modified file 'test_requirements.txt'
--- test_requirements.txt	2014-06-02 12:18:22 +0000
+++ test_requirements.txt	2014-08-05 13:03:33 +0000
@@ -17,6 +17,5 @@
 Tempita==0.5.1
 bzr+http://bazaar.launchpad.net/~yellow/python-shelltoolbox/trunk@17#egg=shelltoolbox
 http://alastairs-place.net/projects/netifaces/netifaces-0.6.tar.gz
-netaddr==0.7.5
 bzr==2.6.0
 Jinja2==2.7.2

=== added directory 'tests/core/templates'
=== added file 'tests/core/templates/cloud_controller_ng.yml'
--- tests/core/templates/cloud_controller_ng.yml	1970-01-01 00:00:00 +0000
+++ tests/core/templates/cloud_controller_ng.yml	2014-08-05 13:03:33 +0000
@@ -0,0 +1,173 @@
+---
+# TODO cc_ip cc public ip 
+local_route: {{ domain }}
+port: {{ cc_port }}
+pid_filename: /var/vcap/sys/run/cloud_controller_ng/cloud_controller_ng.pid
+development_mode: false
+
+message_bus_servers:
+  - nats://{{ nats['user'] }}:{{ nats['password'] }}@{{ nats['address'] }}:{{ nats['port'] }}
+
+external_domain:
+  - api.{{ domain }}
+
+system_domain_organization: {{ default_organization }}
+system_domain: {{ domain }}
+app_domains: [ {{ domain }} ]
+srv_api_uri: http://api.{{ domain }}
+
+default_app_memory: 1024
+
+cc_partition: default
+
+bootstrap_admin_email: admin@{{ default_organization }}
+
+bulk_api:
+  auth_user: bulk_api
+  auth_password: "Password"
+
+nginx:
+  use_nginx: false
+  instance_socket: "/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock"
+
+index: 1
+name: cloud_controller_ng
+
+info:
+  name: vcap
+  build: "2222"
+  version: 2
+  support_address: http://support.cloudfoundry.com
+  description: Cloud Foundry sponsored by Pivotal
+  api_version: 2.0.0
+
+
+directories:
+ tmpdir: /var/vcap/data/cloud_controller_ng/tmp
+
+
+logging:
+  file: /var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng.log
+
+  syslog: vcap.cloud_controller_ng
+
+  level: debug2
+  max_retries: 1
+
+
+
+
+
+db: &db
+  database: sqlite:///var/lib/cloudfoundry/cfcloudcontroller/db/cc.db
+  max_connections: 25
+  pool_timeout: 10
+  log_level: debug2
+
+
+login:
+  url: http://uaa.{{ domain }}
+
+uaa:
+  url: http://uaa.{{ domain }}
+  resource_id: cloud_controller
+  #symmetric_secret: cc-secret
+  verification_key: |
+    -----BEGIN PUBLIC KEY-----
+    MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDHFr+KICms+tuT1OXJwhCUmR2d
+    KVy7psa8xzElSyzqx7oJyfJ1JZyOzToj9T5SfTIq396agbHJWVfYphNahvZ/7uMX
+    qHxf+ZH9BL1gk9Y6kCnbM5R60gfwjyW1/dQPjOzn9N394zd2FJoFHwdq9Qs0wBug
+    spULZVNRxq7veq/fzwIDAQAB
+    -----END PUBLIC KEY-----
+
+# App staging parameters
+staging:
+  max_staging_runtime: 900
+  auth:
+    user:
+    password: "Password"
+
+maximum_health_check_timeout: 180
+
+runtimes_file: /var/lib/cloudfoundry/cfcloudcontroller/jobs/config/runtimes.yml
+stacks_file: /var/lib/cloudfoundry/cfcloudcontroller/jobs/config/stacks.yml
+
+quota_definitions:
+  free:
+    non_basic_services_allowed: false
+    total_services: 2
+    total_routes: 1000
+    memory_limit: 1024
+  paid:
+    non_basic_services_allowed: true
+    total_services: 32
+    total_routes: 1000
+    memory_limit: 204800
+  runaway:
+    non_basic_services_allowed: true
+    total_services: 500
+    total_routes: 1000
+    memory_limit: 204800
+  trial:
+    non_basic_services_allowed: false
+    total_services: 10
+    memory_limit: 2048
+    total_routes: 1000
+    trial_db_allowed: true
+
+default_quota_definition: free
+
+resource_pool:
+  minimum_size: 65536
+  maximum_size: 536870912
+  resource_directory_key: cc-resources
+
+  cdn:
+    uri:
+    key_pair_id:
+    private_key: ""
+
+  fog_connection: {"provider":"Local","local_root":"/var/vcap/nfs/store"}
+
+packages:
+  app_package_directory_key: cc-packages
+
+  cdn:
+    uri:
+    key_pair_id:
+    private_key: ""
+
+  fog_connection: {"provider":"Local","local_root":"/var/vcap/nfs/store"}
+
+droplets:
+  droplet_directory_key: cc-droplets
+
+  cdn:
+    uri:
+    key_pair_id:
+    private_key: ""
+
+  fog_connection: {"provider":"Local","local_root":"/var/vcap/nfs/store"}
+
+buildpacks:
+  buildpack_directory_key: cc-buildpacks
+
+  cdn:
+    uri:
+    key_pair_id:
+    private_key: ""
+
+  fog_connection: {"provider":"Local","local_root":"/var/vcap/nfs/store"}
+
+db_encryption_key: Password
+
+trial_db:
+  guid: "78ad16cf-3c22-4427-a982-b9d35d746914"
+
+tasks_disabled: false
+hm9000_noop: true
+flapping_crash_count_threshold: 3
+
+disable_custom_buildpacks: false
+
+broker_client_timeout_seconds: 60

=== added file 'tests/core/templates/fake_cc.yml'
--- tests/core/templates/fake_cc.yml	1970-01-01 00:00:00 +0000
+++ tests/core/templates/fake_cc.yml	2014-08-05 13:03:33 +0000
@@ -0,0 +1,3 @@
+host: {{nats['host']}}
+port: {{nats['port']}}
+domain: {{router['domain']}}

=== added file 'tests/core/templates/nginx.conf'
--- tests/core/templates/nginx.conf	1970-01-01 00:00:00 +0000
+++ tests/core/templates/nginx.conf	2014-08-05 13:03:33 +0000
@@ -0,0 +1,154 @@
+# deployment cloudcontroller nginx.conf
+#user  vcap vcap;
+
+error_log /var/vcap/sys/log/nginx_ccng/nginx.error.log;
+pid       /var/vcap/sys/run/nginx_ccng/nginx.pid;
+
+events {
+  worker_connections  8192;
+  use epoll;
+}
+
+http {
+  include       mime.types;
+  default_type  text/html;
+  server_tokens off;
+  variables_hash_max_size 1024;
+
+  log_format main  '$host - [$time_local] '
+                   '"$request" $status $bytes_sent '
+                   '"$http_referer" "$http_#user_agent" '
+                   '$proxy_add_x_forwarded_for response_time:$upstream_response_time';
+
+  access_log  /var/vcap/sys/log/nginx_ccng/nginx.access.log  main;
+
+  sendfile             on;  #enable use of sendfile()
+  tcp_nopush           on;
+  tcp_nodelay          on;  #disable nagel's algorithm
+
+  keepalive_timeout  75 20; #inherited from router
+
+  client_max_body_size 256M; #already enforced upstream/but doesn't hurt.
+
+  upstream cloud_controller {
+    server unix:/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock;
+  }
+
+  server {
+    listen    {{ nginx_port }};
+    server_name  _;
+    server_name_in_redirect off;
+    proxy_send_timeout          300;
+    proxy_read_timeout          300;
+
+    # proxy and log all CC traffic
+    location / {
+      access_log /var/vcap/sys/log/nginx_ccng/nginx.access.log  main;
+      proxy_buffering             off;
+      proxy_set_header            Host $host;
+      proxy_set_header            X-Real_IP $remote_addr;
+      proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;
+      proxy_redirect              off;
+      proxy_connect_timeout       10;
+      proxy_pass                 http://cloud_controller;
+    }
+
+
+    # used for x-accel-redirect uri://location/foo.txt
+    # nginx will serve the file root || location || foo.txt
+    location /droplets/ {
+      internal;
+      root   /var/vcap/nfs/store;
+    }
+
+
+
+    # used for x-accel-redirect uri://location/foo.txt
+    # nginx will serve the file root || location || foo.txt
+    location /cc-packages/ {
+      internal;
+      root   /var/vcap/nfs/store;
+    }
+
+
+    # used for x-accel-redirect uri://location/foo.txt
+    # nginx will serve the file root || location || foo.txt
+    location /cc-droplets/ {
+      internal;
+      root   /var/vcap/nfs/store;
+    }
+
+
+    location ~ (/apps/.*/application|/v2/apps/.*/bits|/services/v\d+/configurations/.*/serialized/data|/v2/buildpacks/.*/bits) {
+      # Pass altered request body to this location
+      upload_pass   @cc_uploads;
+      upload_pass_args on;
+
+      # Store files to this directory
+      upload_store /var/vcap/data/cloud_controller_ng/tmp/uploads;
+
+      # No limit for output body forwarded to CC
+      upload_max_output_body_len 0;
+
+      # Allow uploaded files to be read only by #user
+      #upload_store_access #user:r;
+
+      # Set specified fields in request body
+      upload_set_form_field "${upload_field_name}_name" $upload_file_name;
+      upload_set_form_field "${upload_field_name}_path" $upload_tmp_path;
+
+      #forward the following fields from existing body
+      upload_pass_form_field "^resources$";
+      upload_pass_form_field "^_method$";
+
+      #on any error, delete uploaded files.
+      upload_cleanup 400-505;
+    }
+
+    location ~ /staging/(buildpack_cache|droplets)/.*/upload {
+
+      # Allow download the droplets and buildpacks
+      if ($request_method = GET){
+        proxy_pass http://cloud_controller;
+      }
+
+      # Pass along auth header
+      set $auth_header $upstream_http_x_auth;
+      proxy_set_header Authorization $auth_header;
+
+      # Pass altered request body to this location
+      upload_pass   @cc_uploads;
+
+      # Store files to this directory
+      upload_store /var/vcap/data/cloud_controller_ng/tmp/staged_droplet_uploads;
+
+      # Allow uploaded files to be read only by #user
+      upload_store_access user:r;
+
+      # Set specified fields in request body
+      upload_set_form_field "droplet_path" $upload_tmp_path;
+
+      #on any error, delete uploaded files.
+      upload_cleanup 400-505;
+    }
+
+    # Pass altered request body to a backend
+    location @cc_uploads {
+      proxy_pass http://unix:/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock;
+    }
+
+    location ~ ^/internal_redirect/(.*){
+      # only allow internal redirects
+      internal;
+
+      set $download_url $1;
+
+      #have to manualy pass along auth header
+      set $auth_header $upstream_http_x_auth;
+      proxy_set_header Authorization $auth_header;
+
+      # Download the file and send it to client
+      proxy_pass $download_url;
+    }
+  }
+}

=== added file 'tests/core/templates/test.conf'
--- tests/core/templates/test.conf	1970-01-01 00:00:00 +0000
+++ tests/core/templates/test.conf	2014-08-05 13:03:33 +0000
@@ -0,0 +1,3 @@
+something
+listen {{nginx_port}}
+something else

=== added file 'tests/core/test_services.py'
--- tests/core/test_services.py	1970-01-01 00:00:00 +0000
+++ tests/core/test_services.py	2014-08-05 13:03:33 +0000
@@ -0,0 +1,531 @@
+import mock
+import unittest
+from charmhelpers.core import hookenv
+from charmhelpers.core import services
+
+
+class TestServiceManager(unittest.TestCase):
+    def setUp(self):
+        self.pcharm_dir = mock.patch.object(hookenv, 'charm_dir')
+        self.mcharm_dir = self.pcharm_dir.start()
+        self.mcharm_dir.return_value = 'charm_dir'
+
+    def tearDown(self):
+        self.pcharm_dir.stop()
+
+    def test_register(self):
+        manager = services.ServiceManager([
+            {'service': 'service1',
+             'foo': 'bar'},
+            {'service': 'service2',
+             'qux': 'baz'},
+        ])
+        self.assertEqual(manager.services, {
+            'service1': {'service': 'service1',
+                         'foo': 'bar'},
+            'service2': {'service': 'service2',
+                         'qux': 'baz'},
+        })
+
+    @mock.patch.object(services.ServiceManager, 'reconfigure_services')
+    @mock.patch.object(services.ServiceManager, 'stop_services')
+    @mock.patch.object(hookenv, 'hook_name')
+    def test_manage_stop(self, hook_name, stop_services, reconfigure_services):
+        manager = services.ServiceManager()
+        hook_name.return_value = 'stop'
+        manager.manage()
+        stop_services.assert_called_once_with()
+        assert not reconfigure_services.called
+
+    @mock.patch.object(services.ServiceManager, 'provide_data')
+    @mock.patch.object(services.ServiceManager, 'reconfigure_services')
+    @mock.patch.object(services.ServiceManager, 'stop_services')
+    @mock.patch.object(hookenv, 'hook_name')
+    def test_manage_other(self, hook_name, stop_services, reconfigure_services, provide_data):
+        manager = services.ServiceManager()
+        hook_name.return_value = 'config-changed'
+        manager.manage()
+        assert not stop_services.called
+        reconfigure_services.assert_called_once_with()
+        provide_data.assert_called_once_with()
+
+    @mock.patch.object(services.ServiceManager, 'save_ready')
+    @mock.patch.object(services.ServiceManager, 'fire_event')
+    @mock.patch.object(services.ServiceManager, 'is_ready')
+    def test_reconfigure_ready(self, is_ready, fire_event, save_ready):
+        manager = services.ServiceManager([
+            {'service': 'service1'}, {'service': 'service2'}])
+        is_ready.return_value = True
+        manager.reconfigure_services()
+        is_ready.assert_has_calls([
+            mock.call('service1'),
+            mock.call('service2'),
+        ], any_order=True)
+        fire_event.assert_has_calls([
+            mock.call('data_ready', 'service1'),
+            mock.call('start', 'service1', default=[
+                services.service_restart,
+                services.manage_ports]),
+        ], any_order=False)
+        fire_event.assert_has_calls([
+            mock.call('data_ready', 'service2'),
+            mock.call('start', 'service2', default=[
+                services.service_restart,
+                services.manage_ports]),
+        ], any_order=False)
+        save_ready.assert_has_calls([
+            mock.call('service1'),
+            mock.call('service2'),
+        ], any_order=True)
+
+    @mock.patch.object(services.ServiceManager, 'save_ready')
+    @mock.patch.object(services.ServiceManager, 'fire_event')
+    @mock.patch.object(services.ServiceManager, 'is_ready')
+    def test_reconfigure_ready_list(self, is_ready, fire_event, save_ready):
+        manager = services.ServiceManager([
+            {'service': 'service1'}, {'service': 'service2'}])
+        is_ready.return_value = True
+        manager.reconfigure_services('service3', 'service4')
+        self.assertEqual(is_ready.call_args_list, [
+            mock.call('service3'),
+            mock.call('service4'),
+        ])
+        self.assertEqual(fire_event.call_args_list, [
+            mock.call('data_ready', 'service3'),
+            mock.call('start', 'service3', default=[
+                services.service_restart,
+                services.open_ports]),
+            mock.call('data_ready', 'service4'),
+            mock.call('start', 'service4', default=[
+                services.service_restart,
+                services.open_ports]),
+        ])
+        self.assertEqual(save_ready.call_args_list, [
+            mock.call('service3'),
+            mock.call('service4'),
+        ])
+
+    @mock.patch.object(services.ServiceManager, 'save_lost')
+    @mock.patch.object(services.ServiceManager, 'fire_event')
+    @mock.patch.object(services.ServiceManager, 'was_ready')
+    @mock.patch.object(services.ServiceManager, 'is_ready')
+    def test_reconfigure_not_ready(self, is_ready, was_ready, fire_event, save_lost):
+        manager = services.ServiceManager([
+            {'service': 'service1'}, {'service': 'service2'}])
+        is_ready.return_value = False
+        was_ready.return_value = False
+        manager.reconfigure_services()
+        is_ready.assert_has_calls([
+            mock.call('service1'),
+            mock.call('service2'),
+        ], any_order=True)
+        fire_event.assert_has_calls([
+            mock.call('stop', 'service1', default=[
+                services.close_ports,
+                services.service_stop]),
+            mock.call('stop', 'service2', default=[
+                services.close_ports,
+                services.service_stop]),
+        ], any_order=True)
+        save_lost.assert_has_calls([
+            mock.call('service1'),
+            mock.call('service2'),
+        ], any_order=True)
+
+    @mock.patch.object(services.ServiceManager, 'save_lost')
+    @mock.patch.object(services.ServiceManager, 'fire_event')
+    @mock.patch.object(services.ServiceManager, 'was_ready')
+    @mock.patch.object(services.ServiceManager, 'is_ready')
+    def test_reconfigure_no_longer_ready(self, is_ready, was_ready, fire_event, save_lost):
+        manager = services.ServiceManager([
+            {'service': 'service1'}, {'service': 'service2'}])
+        is_ready.return_value = False
+        was_ready.return_value = True
+        manager.reconfigure_services()
+        is_ready.assert_has_calls([
+            mock.call('service1'),
+            mock.call('service2'),
+        ], any_order=True)
+        fire_event.assert_has_calls([
+            mock.call('data_lost', 'service1'),
+            mock.call('stop', 'service1', default=[
+                services.close_ports,
+                services.service_stop]),
+        ], any_order=False)
+        fire_event.assert_has_calls([
+            mock.call('data_lost', 'service2'),
+            mock.call('stop', 'service2', default=[
+                services.close_ports,
+                services.service_stop]),
+        ], any_order=False)
+        save_lost.assert_has_calls([
+            mock.call('service1'),
+            mock.call('service2'),
+        ], any_order=True)
+
+    @mock.patch.object(services.ServiceManager, 'fire_event')
+    def test_stop_services(self, fire_event):
+        manager = services.ServiceManager([
+            {'service': 'service1'}, {'service': 'service2'}])
+        manager.stop_services()
+        fire_event.assert_has_calls([
+            mock.call('stop', 'service1', default=[
+                services.close_ports,
+                services.service_stop]),
+            mock.call('stop', 'service2', default=[
+                services.close_ports,
+                services.service_stop]),
+        ], any_order=True)
+
+    @mock.patch.object(services.ServiceManager, 'fire_event')
+    def test_stop_services_list(self, fire_event):
+        manager = services.ServiceManager([
+            {'service': 'service1'}, {'service': 'service2'}])
+        manager.stop_services('service3', 'service4')
+        self.assertEqual(fire_event.call_args_list, [
+            mock.call('stop', 'service3', default=[
+                services.close_ports,
+                services.service_stop]),
+            mock.call('stop', 'service4', default=[
+                services.close_ports,
+                services.service_stop]),
+        ])
+
+    def test_get_service(self):
+        service = {'service': 'test', 'test': 'test_service'}
+        manager = services.ServiceManager([service])
+        self.assertEqual(manager.get_service('test'), service)
+
+    def test_get_service_not_registered(self):
+        service = {'service': 'test', 'test': 'test_service'}
+        manager = services.ServiceManager([service])
+        self.assertRaises(KeyError, manager.get_service, 'foo')
+
+    @mock.patch.object(services.ServiceManager, 'get_service')
+    def test_fire_event_default(self, get_service):
+        get_service.return_value = {}
+        cb = mock.Mock()
+        manager = services.ServiceManager()
+        manager.fire_event('event', 'service', cb)
+        cb.assert_called_once_with('service')
+
+    @mock.patch.object(services.ServiceManager, 'get_service')
+    def test_fire_event_default_list(self, get_service):
+        get_service.return_value = {}
+        cb = mock.Mock()
+        manager = services.ServiceManager()
+        manager.fire_event('event', 'service', [cb])
+        cb.assert_called_once_with('service')
+
+    @mock.patch.object(services.ServiceManager, 'get_service')
+    def test_fire_event_simple_callback(self, get_service):
+        cb = mock.Mock()
+        dcb = mock.Mock()
+        get_service.return_value = {'event': cb}
+        manager = services.ServiceManager()
+        manager.fire_event('event', 'service', dcb)
+        assert not dcb.called
+        cb.assert_called_once_with('service')
+
+    @mock.patch.object(services.ServiceManager, 'get_service')
+    def test_fire_event_simple_callback_list(self, get_service):
+        cb = mock.Mock()
+        dcb = mock.Mock()
+        get_service.return_value = {'event': [cb]}
+        manager = services.ServiceManager()
+        manager.fire_event('event', 'service', dcb)
+        assert not dcb.called
+        cb.assert_called_once_with('service')
+
+    @mock.patch.object(services.ManagerCallback, '__call__')
+    @mock.patch.object(services.ServiceManager, 'get_service')
+    def test_fire_event_manager_callback(self, get_service, mcall):
+        cb = services.ManagerCallback()
+        dcb = mock.Mock()
+        get_service.return_value = {'event': cb}
+        manager = services.ServiceManager()
+        manager.fire_event('event', 'service', dcb)
+        assert not dcb.called
+        mcall.assert_called_once_with(manager, 'service', 'event')
+
+    @mock.patch.object(services.ManagerCallback, '__call__')
+    @mock.patch.object(services.ServiceManager, 'get_service')
+    def test_fire_event_manager_callback_list(self, get_service, mcall):
+        cb = services.ManagerCallback()
+        dcb = mock.Mock()
+        get_service.return_value = {'event': [cb]}
+        manager = services.ServiceManager()
+        manager.fire_event('event', 'service', dcb)
+        assert not dcb.called
+        mcall.assert_called_once_with(manager, 'service', 'event')
+
+    @mock.patch.object(services.ServiceManager, 'get_service')
+    def test_is_ready(self, get_service):
+        get_service.side_effect = [
+            {},
+            {'required_data': [True]},
+            {'required_data': [False]},
+            {'required_data': [True, False]},
+        ]
+        manager = services.ServiceManager()
+        assert manager.is_ready('foo')
+        assert manager.is_ready('bar')
+        assert not manager.is_ready('foo')
+        assert not manager.is_ready('foo')
+        get_service.assert_has_calls([mock.call('foo'), mock.call('bar')])
+
+    def test_load_ready_file_short_circuit(self):
+        manager = services.ServiceManager()
+        manager._ready = 'foo'
+        manager._load_ready_file()
+        self.assertEqual(manager._ready, 'foo')
+
+    @mock.patch('os.path.exists')
+    @mock.patch.object(services.base, 'open', create=True)
+    def test_load_ready_file_new(self, mopen, exists):
+        manager = services.ServiceManager()
+        exists.return_value = False
+        manager._load_ready_file()
+        self.assertEqual(manager._ready, set())
+        assert not mopen.called
+
+    @mock.patch('json.load')
+    @mock.patch('os.path.exists')
+    @mock.patch.object(services.base, 'open', create=True)
+    def test_load_ready_file(self, mopen, exists, jload):
+        manager = services.ServiceManager()
+        exists.return_value = True
+        jload.return_value = ['bar']
+        manager._load_ready_file()
+        self.assertEqual(manager._ready, set(['bar']))
+        exists.assert_called_once_with('charm_dir/READY-SERVICES.json')
+        mopen.assert_called_once_with('charm_dir/READY-SERVICES.json')
+
+    @mock.patch('json.dump')
+    @mock.patch.object(services.base, 'open', create=True)
+    def test_save_ready_file(self, mopen, jdump):
+        manager = services.ServiceManager()
+        manager._save_ready_file()
+        assert not mopen.called
+        manager._ready = set(['foo'])
+        manager._save_ready_file()
+        mopen.assert_called_once_with('charm_dir/READY-SERVICES.json', 'w')
+        jdump.assert_called_once_with(['foo'], mopen.return_value.__enter__())
+
+    @mock.patch.object(services.base.ServiceManager, '_save_ready_file')
+    @mock.patch.object(services.base.ServiceManager, '_load_ready_file')
+    def test_save_ready(self, _lrf, _srf):
+        manager = services.ServiceManager()
+        manager._ready = set(['foo'])
+        manager.save_ready('bar')
+        _lrf.assert_called_once_with()
+        self.assertEqual(manager._ready, set(['foo', 'bar']))
+        _srf.assert_called_once_with()
+
+    @mock.patch.object(services.base.ServiceManager, '_save_ready_file')
+    @mock.patch.object(services.base.ServiceManager, '_load_ready_file')
+    def test_save_lost(self, _lrf, _srf):
+        manager = services.ServiceManager()
+        manager._ready = set(['foo', 'bar'])
+        manager.save_lost('bar')
+        _lrf.assert_called_once_with()
+        self.assertEqual(manager._ready, set(['foo']))
+        _srf.assert_called_once_with()
+        manager.save_lost('bar')
+        self.assertEqual(manager._ready, set(['foo']))
+
+    @mock.patch.object(services.base.ServiceManager, '_save_ready_file')
+    @mock.patch.object(services.base.ServiceManager, '_load_ready_file')
+    def test_was_ready(self, _lrf, _srf):
+        manager = services.ServiceManager()
+        manager._ready = set()
+        manager.save_ready('foo')
+        manager.save_ready('bar')
+        assert manager.was_ready('foo')
+        assert manager.was_ready('bar')
+        manager.save_lost('bar')
+        assert manager.was_ready('foo')
+        assert not manager.was_ready('bar')
+
+    @mock.patch.object(services.base.hookenv, 'relation_set')
+    @mock.patch.object(services.base.hookenv, 'hook_name')
+    def test_provide_data_no_match(self, hook_name, relation_set):
+        provider = mock.Mock()
+        provider.name = 'provided'
+        manager = services.ServiceManager([
+            {'service': 'service', 'provided_data': [provider]}
+        ])
+        hook_name.return_value = 'not-provided-relation-joined'
+        manager.provide_data()
+        assert not provider.provide_data.called
+
+        hook_name.return_value = 'provided-relation-broken'
+        manager.provide_data()
+        assert not provider.provide_data.called
+
+    @mock.patch.object(services.base.hookenv, 'relation_set')
+    @mock.patch.object(services.base.hookenv, 'hook_name')
+    def test_provide_data_not_ready(self, hook_name, relation_set):
+        provider = mock.Mock()
+        provider.name = 'provided'
+        data = provider.provide_data.return_value = {'data': True}
+        provider._is_ready.return_value = False
+        manager = services.ServiceManager([
+            {'service': 'service', 'provided_data': [provider]}
+        ])
+        hook_name.return_value = 'provided-relation-joined'
+        manager.provide_data()
+        assert not relation_set.called
+        provider._is_ready.assert_called_once_with(data)
+
+    @mock.patch.object(services.base.hookenv, 'relation_set')
+    @mock.patch.object(services.base.hookenv, 'hook_name')
+    def test_provide_data_ready(self, hook_name, relation_set):
+        provider = mock.Mock()
+        provider.name = 'provided'
+        data = provider.provide_data.return_value = {'data': True}
+        provider._is_ready.return_value = True
+        manager = services.ServiceManager([
+            {'service': 'service', 'provided_data': [provider]}
+        ])
+        hook_name.return_value = 'provided-relation-changed'
+        manager.provide_data()
+        relation_set.assert_called_once_with(None, data)
+
+
+class TestRelationContext(unittest.TestCase):
+    def setUp(self):
+        self.phookenv = mock.patch.object(services.helpers, 'hookenv')
+        self.mhookenv = self.phookenv.start()
+        self.mhookenv.relation_ids.return_value = []
+        self.context = services.RelationContext()
+        self.context.name = 'http'
+        self.context.interface = 'http'
+        self.context.required_keys = ['foo', 'bar']
+        self.mhookenv.reset_mock()
+
+    def tearDown(self):
+        self.phookenv.stop()
+
+    def test_no_relations(self):
+        self.context.get_data()
+        self.assertFalse(self.context.is_ready())
+        self.assertEqual(self.context, {})
+        self.mhookenv.relation_ids.assert_called_once_with('http')
+
+    def test_no_units(self):
+        self.mhookenv.relation_ids.return_value = ['nginx']
+        self.mhookenv.related_units.return_value = []
+        self.context.get_data()
+        self.assertFalse(self.context.is_ready())
+        self.assertEqual(self.context, {'http': []})
+
+    def test_incomplete(self):
+        self.mhookenv.relation_ids.return_value = ['nginx', 'apache']
+        self.mhookenv.related_units.side_effect = lambda i: [i+'/0']
+        self.mhookenv.relation_get.side_effect = [{}, {'foo': '1'}]
+        self.context.get_data()
+        self.assertFalse(bool(self.context))
+        self.assertEqual(self.mhookenv.relation_get.call_args_list, [
+            mock.call(rid='apache', unit='apache/0'),
+            mock.call(rid='nginx', unit='nginx/0'),
+        ])
+
+    def test_complete(self):
+        self.mhookenv.relation_ids.return_value = ['nginx', 'apache', 'tomcat']
+        self.mhookenv.related_units.side_effect = lambda i: [i+'/0']
+        self.mhookenv.relation_get.side_effect = [{'foo': '1'}, {'foo': '2', 'bar': '3'}, {}]
+        self.context.get_data()
+        self.assertTrue(self.context.is_ready())
+        self.assertEqual(self.context, {'http': [
+            {
+                'foo': '2',
+                'bar': '3',
+            },
+        ]})
+        self.mhookenv.relation_ids.assert_called_with('http')
+        self.assertEqual(self.mhookenv.relation_get.call_args_list, [
+            mock.call(rid='apache', unit='apache/0'),
+            mock.call(rid='nginx', unit='nginx/0'),
+            mock.call(rid='tomcat', unit='tomcat/0'),
+        ])
+
+    def test_provide(self):
+        self.assertEqual(self.context.provide_data(), {})
+
+
+class TestTemplateCallback(unittest.TestCase):
+    @mock.patch.object(services.helpers, 'templating')
+    def test_template_defaults(self, mtemplating):
+        manager = mock.Mock(**{'get_service.return_value': {
+            'required_data': [{'foo': 'bar'}]}})
+        self.assertRaises(TypeError, services.template, source='foo.yml')
+        callback = services.template(source='foo.yml', target='bar.yml')
+        assert isinstance(callback, services.ManagerCallback)
+        assert not mtemplating.render.called
+        callback(manager, 'test', 'event')
+        mtemplating.render.assert_called_once_with(
+            'foo.yml', 'bar.yml', {'foo': 'bar'},
+            'root', 'root', 0444)
+
+    @mock.patch.object(services.helpers, 'templating')
+    def test_template_explicit(self, mtemplating):
+        manager = mock.Mock(**{'get_service.return_value': {
+            'required_data': [{'foo': 'bar'}]}})
+        callback = services.template(
+            source='foo.yml', target='bar.yml',
+            owner='user', group='group', perms=0555
+        )
+        assert isinstance(callback, services.ManagerCallback)
+        assert not mtemplating.render.called
+        callback(manager, 'test', 'event')
+        mtemplating.render.assert_called_once_with(
+            'foo.yml', 'bar.yml', {'foo': 'bar'},
+            'user', 'group', 0555)
+
+
+class TestPortsCallback(unittest.TestCase):
+    def setUp(self):
+        self.phookenv = mock.patch.object(services.base, 'hookenv')
+        self.mhookenv = self.phookenv.start()
+        self.mhookenv.relation_ids.return_value = []
+        self.mhookenv.charm_dir.return_value = 'charm_dir'
+        self.popen = mock.patch.object(services.base, 'open', create=True)
+        self.mopen = self.popen.start()
+
+    def tearDown(self):
+        self.phookenv.stop()
+        self.popen.stop()
+
+    def test_no_ports(self):
+        manager = mock.Mock(**{'get_service.return_value': {}})
+        services.PortManagerCallback()(manager, 'service', 'event')
+        assert not self.mhookenv.open_port.called
+        assert not self.mhookenv.close_port.called
+
+    def test_open_ports(self):
+        manager = mock.Mock(**{'get_service.return_value': {'ports': [1, 2]}})
+        services.open_ports(manager, 'service', 'start')
+        self.mhookenv.open_port.has_calls([mock.call(1), mock.call(2)])
+        assert not self.mhookenv.close_port.called
+
+    def test_close_ports(self):
+        manager = mock.Mock(**{'get_service.return_value': {'ports': [1, 2]}})
+        services.close_ports(manager, 'service', 'stop')
+        assert not self.mhookenv.open_port.called
+        self.mhookenv.close_port.has_calls([mock.call(1), mock.call(2)])
+
+    def test_close_old_ports(self):
+        self.mopen.return_value.read.return_value = '10,20'
+        manager = mock.Mock(**{'get_service.return_value': {'ports': [1, 2]}})
+        services.close_ports(manager, 'service', 'stop')
+        assert not self.mhookenv.open_port.called
+        self.mhookenv.close_port.has_calls([
+            mock.call(10),
+            mock.call(20),
+            mock.call(1),
+            mock.call(2)])
+
+
+if __name__ == '__main__':
+    unittest.main()

=== added file 'tests/core/test_templating.py'
--- tests/core/test_templating.py	1970-01-01 00:00:00 +0000
+++ tests/core/test_templating.py	2014-08-05 13:03:33 +0000
@@ -0,0 +1,64 @@
+import os
+import pkg_resources
+import tempfile
+import unittest
+import jinja2
+
+import mock
+from charmhelpers.core import templating
+
+
+TEMPLATES_DIR = pkg_resources.resource_filename(__name__, 'templates')
+
+
+class TestTemplating(unittest.TestCase):
+    def setUp(self):
+        self.charm_dir = pkg_resources.resource_filename(__name__, '')
+        self._charm_dir_patch = mock.patch.object(templating.hookenv, 'charm_dir')
+        self._charm_dir_mock = self._charm_dir_patch.start()
+        self._charm_dir_mock.side_effect = lambda: self.charm_dir
+
+    def tearDown(self):
+        self._charm_dir_patch.stop()
+
+    @mock.patch.object(templating.host.os, 'fchown')
+    @mock.patch.object(templating.host, 'mkdir')
+    @mock.patch.object(templating.host, 'log')
+    def test_render(self, log, mkdir, fchown):
+        _, fn1 = tempfile.mkstemp()
+        _, fn2 = tempfile.mkstemp()
+        try:
+            context = {
+                'nats': {
+                    'port': '1234',
+                    'host': 'example.com',
+                },
+                'router': {
+                    'domain': 'api.foo.com'
+                },
+                'nginx_port': 80,
+            }
+            templating.render('fake_cc.yml', fn1, context, templates_dir=TEMPLATES_DIR)
+            contents = open(fn1).read()
+            self.assertRegexpMatches(contents, 'port: 1234')
+            self.assertRegexpMatches(contents, 'host: example.com')
+            self.assertRegexpMatches(contents, 'domain: api.foo.com')
+
+            templating.render('test.conf', fn2, context, templates_dir=TEMPLATES_DIR)
+            contents = open(fn2).read()
+            self.assertRegexpMatches(contents, 'listen 80')
+            self.assertEqual(fchown.call_count, 2)
+            self.assertEqual(mkdir.call_count, 2)
+        finally:
+            for fn in (fn1, fn2):
+                if os.path.exists(fn):
+                    os.remove(fn)
+
+    @mock.patch.object(templating, 'hookenv')
+    @mock.patch('jinja2.Environment')
+    def test_load_error(self, Env, hookenv):
+        Env().get_template.side_effect = jinja2.exceptions.TemplateNotFound('fake_cc.yml')
+        self.assertRaises(
+            jinja2.exceptions.TemplateNotFound, templating.render,
+            'fake.src', 'fake.tgt', {}, templates_dir='tmpl')
+        hookenv.log.assert_called_once_with('Could not load template fake.src from tmpl.', level=hookenv.ERROR)