Merge lp:~dpb/charms/precise/haproxy/fixes into lp:charms/haproxy

[David Britton]

Entire change is focused on adding "Relation driven proxying", or
"multi-service proxying":

- Added "Relation driven proxying" method for the charm. This allows each
  charm to explicitly hook into haproxy by simply setting realtion variables.
  This also allows haproxy to proxy multiple services per charm, and proxy the
  same service across multiple charms as long as the agree on service names.

- Refactored service dict parsing out of "create_services" and into separate
  method, adding error checking, and allowing the services yaml to come from
  relation settings.

- Fixed website relation to be aware of and make use of get_config_services

- cleaned up some existing pep8 lint on code that I touched

- protect main method to make this testable

- Refactored relation parsing out of "create_services" into it's own method
  "relation_get_all".

- Added unit tests for my changes, and also to test basic functionality of
  configuration and service proxying. `test_hooks.py`, including README.md
  section outlining usage.

- Added README.md with basics of usage of the charm

[Juan L. Negron (negronjl)]

Reviewing this now.

-Juan

[Juan L. Negron (negronjl)]

This looks good to me. Great work on expanding the charm!

Approved.

Merging.

-Juan

review: Approve

=== added file 'README.md'
--- README.md	1970-01-01 00:00:00 +0000
+++ README.md	2013-02-13 23:59:22 +0000
@@ -0,0 +1,119 @@
+Juju charm haproxy
+==================
+
+HAProxy is a free, very fast and reliable solution offering high availability,
+load balancing, and proxying for TCP and HTTP-based applications. It is
+particularly suited for web sites crawling under very high loads while needing
+persistence or Layer7 processing. Supporting tens of thousands of connections
+is clearly realistic with todays hardware. Its mode of operation makes its
+integration into existing architectures very easy and riskless, while still
+offering the possibility not to expose fragile web servers to the Net.
+
+How to deploy the charm
+-----------------------
+    juju deploy haproxy
+    juju deploy my-web-app
+    juju add-relation my-web-app:website haproxy:reverseproxy
+    juju add-unit my-web-app
+    ...
+
+Reverseproxy Relation
+---------------------
+
+The reverse proxy relation is used to distribute connections from one frontend
+port to many backend services (typically different Juju _units_).  You can use
+haproxy just like this, but typically in a production service you would
+frontend this service with apache2 to handle the SSL negotiation, etc.  See
+the "Website Relation" section for more information about that.
+
+When your charm hooks into reverseproxy you have two general approaches
+which can be used to notify haproxy about what services you are running. 
+1) Single-service proxying or 2) Multi-service or relation-driven proxying.
+
+** 1) Single-Service Proxying **
+
+In this case, your website relation will join underneath a single `listen`
+stanza in haproxy.  This stanza will have one `service` entry for each unit
+connecting. By convention, this is typically called "website".  The
+following is an example of a relation-joined or changed hook:
+
+    #!/bin/bash
+    # hooks/website-relation-joined
+
+    relation-set "hostname=$(unit-get private-address)"
+    relation-set "port=80"
+
+    # Set an optional service name, allowing more config-based
+    # customization
+    relation-set "service_name=my_web_app"
+
+If you set the `service_name` relation setting, the configuration `services`
+yaml mapping will be consulted to lookup 3 other options based on service
+name.
+
+  * `{service_name}_servers` - sets the `server` line in the listen stanza
+    explicitly.
+  * `{service_name}_server_options` - Will append to the charm-generated
+    server line for for each joining unit in the reverseproxy relation.
+  * `{service_name}_service_options` - expected to be a list of strings.  Will
+    set each item as an option under the listen stanza.
+
+
+** 2) Relation-Driven Proxying **
+
+In this relation style, your charm should specify these relation settings
+directly as relation variables when joining reverseproxy.  Your charm's
+website-relation-changed hook would look something like this:
+
+    #!/bin/bash
+    # hooks/website-relation-changed
+  
+    host=$(unit-get private-address)
+    port=80
+
+    relation-set "services=
+    - { service_name: my_web_app,
+        service_options: [mode http, balance leastconn],
+        servers: [[my_web_app_1, $host, $port, option httpchk GET / HTTP/1.0],
+                  [... optionally more servers here ...]]}
+    - { ... optionally more services here ... }
+    "
+
+Once set, haproxy will union multiple `servers` stanzas from any units
+joining with the same `service_name` under one listen stanza. 
+`service-options` and `server_options` will be overwritten, so ensure they
+are set uniformly on all services with the same name.
+
+Website Relation
+----------------
+
+The website relation is the other side of haproxy.  It can communicate with
+charms written like apache2 that can act as a front-end for haproxy to take of
+things like ssl encryption.  When joining a service like apache2 on its
+reverseproxy relation, haproxy's website relation will set an `all_services`
+varaible that conforms to the spec layed out in the apache2 charm.
+
+These settings can then be used when crafting your vhost template to make sure
+traffic goes to the correct haproxy listener which will in turn forward the
+traffic to the correct backend server/port
+
+Configuration
+-------------
+Many of the haproxy settings can be altered via the standard juju configuration
+settings.  Please see the config.yaml file as each is fairly clearly documented.
+
+Testing
+-------
+This charm has a simple unit-test program.  Please expand it and make sure new
+changes are covered by simple unit tests.  To run the unit tests:
+
+    sudo apt-get install python-mocker
+    sudo apt-get install python-twisted-core
+    cd hooks; trial test_hooks
+
+TODO:
+-----
+
+  * Expand Single-Service section as I have not tested that mode fully.
+  * Trigger website-relation-changed when the reverse-proxy relation changes
+

=== modified file 'hooks/hooks.py'
--- hooks/hooks.py	2013-01-15 02:04:18 +0000
+++ hooks/hooks.py	2013-02-13 23:59:22 +0000
@@ -19,12 +19,23 @@
 default_haproxy_config_dir = "/etc/haproxy"
 default_haproxy_config = "%s/haproxy.cfg" % default_haproxy_config_dir
 default_haproxy_service_config_dir = "/var/run/haproxy"
-hook_name = os.path.basename(sys.argv[0])
+HOOK_NAME = os.path.basename(sys.argv[0])
 
 ###############################################################################
 # Supporting functions
 ###############################################################################
 
+def unit_get(*args):
+    """Simple wrapper around unit-get, all arguments passed untouched"""
+    get_args = ["unit-get"]
+    get_args.extend(args)
+    return subprocess.check_output(get_args)
+
+def juju_log(*args):
+    """Simple wrapper around juju-log, all arguments are passed untouched"""
+    log_args = ["juju-log"]
+    log_args.extend(args)
+    subprocess.call(log_args)
 
 #------------------------------------------------------------------------------
 # config_get:  Returns a dictionary containing all of the config information
@@ -297,15 +308,41 @@
                                 config_data['monitoring_port'],
                                 monitoring_config))
 
+def get_host_port(services_list):
+    """
+    Given a services list and global juju information, get a host
+    and port for this system.
+    """
+    host = services_list[0]["service_host"]
+    port = int(services_list[0]["service_port"])
+    return (host, port)
 
-#------------------------------------------------------------------------------
-# get_config_services:  Convenience function that returns a list
-#                       of dictionary entries containing all of the services
-#                       configuration
-#------------------------------------------------------------------------------
 def get_config_services():
+    """
+    Return dict of all services in the configuration, and in the relation
+    where appropriate.  If a relation contains a "services" key, read
+    it in as yaml as is the case with the configuration. Set the host and
+    port for any relation initiated service entry as those items cannot be
+    known by the other side of the relation. In the case of a
+    proxy configuration found, ensure the forward for option is set.
+    """
     config_data = config_get()
-    services_list = yaml.load(config_data['services'])
+    config_services_list = yaml.load(config_data['services'])
+    (host, port) = get_host_port(config_services_list)
+    all_relations = relation_get_all("reverseproxy")
+    services_list = []
+    if hasattr(all_relations, "iteritems"):
+        for relid, reldata in all_relations.iteritems():
+            for unit, relation_info in reldata.iteritems():
+                if relation_info.has_key("services"):
+                    rservices = yaml.load(relation_info["services"])
+                    for r in rservices:
+                        r["service_host"] = host
+                        r["service_port"] = port
+                        port += 1
+                    services_list.extend(rservices)
+    if len(services_list) == 0:
+        services_list = config_services_list
     return(services_list)
 
 
@@ -321,79 +358,136 @@
     return(None)
 
 
+def relation_get_all(relation_name):
+    """
+    Iterate through all relations, and return large data structure with the
+    relation data set:
+
+    @param relation_name: The name of the relation to check
+
+    Returns:
+
+        relation_id:
+            unit:
+                key: value
+                key2: value
+    """
+    result = {}
+    try:
+        relids = subprocess.Popen(
+                ['relation-ids', relation_name], stdout=subprocess.PIPE)
+        for relid in [x.strip() for x in relids.stdout]:
+            result[relid] = {}
+            for unit in json.loads(
+                subprocess.check_output(
+                    ['relation-list', '--format=json', '-r', relid])):
+                result[relid][unit] = relation_get(None, unit, relid)
+        return result
+    except Exception, e:
+        subprocess.call(['juju-log', str(e)])
+
+def get_services_dict():
+    """
+    Transform the services list into a dict for easier comprehension,
+    and to ensure that we have only one entry per service type.  If multiple
+    relations specify the same server_name, try to union the servers
+    entries.
+    """
+    services_list = get_config_services()
+    services_dict = {}
+
+    for service_item in services_list:
+        if not hasattr(service_item, "iteritems"):
+            juju_log("Each 'services' entry must be a dict: %s" % service_item)
+            continue;
+        if "service_name" not in service_item:
+            juju_log("Missing 'service_name': %s" % service_item)
+            continue;
+        name = service_item["service_name"]
+        options = service_item["service_options"]
+        if name in services_dict:
+            if "servers" in services_dict[name]:
+                services_dict[name]["servers"].extend(service_item["servers"])
+        else:
+            services_dict[name] = service_item
+        if os.path.exists("%s/%s.is.proxy" % (
+            default_haproxy_service_config_dir, name)):
+            if 'option forwardfor' not in options:
+                options.append("option forwardfor")
+
+    return services_dict
+
+def get_all_services():
+    """
+    Transform a services dict into an "all_services" relation setting expected
+    by apache2.  This is needed to ensure we have the port and hostname setting
+    correct and in the proper format
+    """
+    services = get_services_dict()
+    all_services = []
+    for name in services:
+        s = {"service_name": name,
+             "service_port": services[name]["service_port"]}
+        all_services.append(s)
+    return all_services
+
 #------------------------------------------------------------------------------
 # create_services:  Function that will create the services configuration
 #                   from the config data and/or relation information
 #------------------------------------------------------------------------------
 def create_services():
     services_list = get_config_services()
-    services_dict = {}
-    for service_item in services_list:
-        service_name = service_item['service_name']
-        service_host = service_item['service_host']
-        service_port = service_item['service_port']
-        service_options = service_item['service_options']
-        server_options = service_item['server_options']
-        services_dict[service_name] = {'service_name': service_name,
-                                         'service_host': service_host,
-                                         'service_port': service_port,
-                                         'service_options': service_options,
-                                         'server_options': server_options}
+    services_dict = get_services_dict()
 
-    try:
-        relids = subprocess.Popen(['relation-ids', 'reverseproxy'],
-            stdout=subprocess.PIPE)
-        for relid in [x.strip() for x in relids.stdout]:
-            for unit in json.loads(
-            subprocess.check_output(['relation-list', '--format=json',
-                                     '-r', relid])):
-                relation_info = relation_get(None, unit, relid)
-                unit_name = unit.rpartition('/')[0]
-                if not isinstance(relation_info, dict):
-                    sys.exit(0)
-                # Mandatory switches ( hostname, port )
-                server_name = "%s__%s" % \
-                (relation_info['hostname'].replace('.', '_'),
+    # service definitions overwrites user specified haproxy file in
+    # a pseudo-template form
+    all_relations = relation_get_all("reverseproxy")
+    for relid, reldata in all_relations.iteritems():
+        for unit, relation_info in reldata.iteritems():
+            if not isinstance(relation_info, dict):
+                sys.exit(0)
+            if "services" in relation_info:
+                juju_log("Relation %s has services override defined" % relid)
+                continue;
+            if "hostname" not in relation_info or "port" not in relation_info:
+                juju_log("Relation %s needs hostname and port defined" % relid)
+                continue;
+            juju_service_name = unit.rpartition('/')[0]
+            # Mandatory switches ( hostname, port )
+            server_name = "%s__%s" % (
+                relation_info['hostname'].replace('.', '_'),
                 relation_info['port'])
-                server_ip = relation_info['hostname']
-                server_port = relation_info['port']
-                # Optional switches ( service_name )
-                if 'service_name' in relation_info:
-                    if relation_info['service_name'] in services_dict:
-                        service_name = relation_info['service_name']
-                    else:
-                        subprocess.call([
-                        'juju-log', 'service %s does not exists. ' %
-                        relation_info['service_name']])
-                        sys.exit(1)
-                elif unit_name in services_dict:
-                    service_name = unit_name
-                else:
-                    service_name = services_list[0]['service_name']
-                if os.path.exists("%s/%s.is.proxy" %
-                (default_haproxy_service_config_dir, service_name)):
-                    if 'option forwardfor' not in service_options:
-                        service_options.append("option forwardfor")
-                # Add the server entries
-                if not 'servers' in services_dict[service_name]:
-                    services_dict[service_name]['servers'] = \
-                    [(server_name, server_ip, server_port,
-                    services_dict[service_name]['server_options'])]
-                else:
-                    services_dict[service_name]['servers'].append((
-                    server_name, server_ip, server_port,
-                    services_dict[service_name]['server_options']))
-    except Exception, e:
-        subprocess.call(['juju-log', str(e)])
+            server_ip = relation_info['hostname']
+            server_port = relation_info['port']
+            # Optional switches ( service_name )
+            if 'service_name' in relation_info:
+                if relation_info['service_name'] in services_dict:
+                    service_name = relation_info['service_name']
+                else:
+                    juju_log("service %s does not exist." % (
+                        relation_info['service_name']))
+                    sys.exit(1)
+            elif juju_service_name + '_service' in services_dict:
+                service_name = juju_service_name + '_service'
+            else:
+                service_name = services_list[0]['service_name']
+            # Add the server entries
+            if not 'servers' in services_dict[service_name]:
+                services_dict[service_name]['servers'] = []
+            services_dict[service_name]['servers'].append((
+                server_name, server_ip, server_port,
+                services_dict[service_name]['server_options']))
+
     # Construct the new haproxy.cfg file
     for service in services_dict:
-        print "Service: ", service
+        juju_log("Service: ", service)
         server_entries = None
         if 'servers' in services_dict[service]:
             server_entries = services_dict[service]['servers']
-        with open("%s/%s.service" % (
-        default_haproxy_service_config_dir,
-        services_dict[service]['service_name']), 'w') as service_config:
+        service_config_file = "%s/%s.service" % (
+            default_haproxy_service_config_dir,
+            services_dict[service]['service_name'])
+        with open(service_config_file, 'w') as service_config:
                 service_config.write(
                 create_listen_stanza(services_dict[service]['service_name'],
                                      services_dict[service]['service_host'],
@@ -568,7 +662,6 @@
         return(None)
     default_port = 80
     relation_data = relation_get()
-    config_data = config_get()
 
     # If a specfic service has been asked for then return the ip:port for
     # that service, else pass back the default
@@ -589,9 +682,9 @@
     # hostname
     if my_host == "localhost":
         my_host = socket.gethostname()
-    subprocess.call(['relation-set', 'port=%d' %
-    my_port, 'hostname=%s' % my_host, 'all_services=%s' %
-    config_data['services']])
+    subprocess.call(
+            ['relation-set', 'port=%d' % my_port, 'hostname=%s' % my_host,
+             'all_services=%s' % yaml.dump(get_all_services())])
     if hook_name == "changed":
         if 'is-proxy' in relation_data:
             service_name = "%s__%d" % \
@@ -608,27 +701,28 @@
 ###############################################################################
 # Main section
 ###############################################################################
-if hook_name == "install":
-    install_hook()
-elif hook_name == "config-changed":
-    config_changed()
-    update_nrpe_config()
-elif hook_name == "start":
-    start_hook()
-elif hook_name == "stop":
-    stop_hook()
-elif hook_name == "reverseproxy-relation-broken":
-    config_changed()
-elif hook_name == "reverseproxy-relation-changed":
-    reverseproxy_interface("changed")
-elif hook_name == "reverseproxy-relation-departed":
-    reverseproxy_interface("departed")
-elif hook_name == "website-relation-joined":
-    website_interface("joined")
-elif hook_name == "website-relation-changed":
-    website_interface("changed")
-elif hook_name == "nrpe-external-master-relation-changed":
-    update_nrpe_config()
-else:
-    print "Unknown hook"
-    sys.exit(1)
+if __name__ == "__main__":
+    if HOOK_NAME == "install":
+        install_hook()
+    elif HOOK_NAME == "config-changed":
+        config_changed()
+        update_nrpe_config()
+    elif HOOK_NAME == "start":
+        start_hook()
+    elif HOOK_NAME == "stop":
+        stop_hook()
+    elif HOOK_NAME == "reverseproxy-relation-broken":
+        config_changed()
+    elif HOOK_NAME == "reverseproxy-relation-changed":
+        reverseproxy_interface("changed")
+    elif HOOK_NAME == "reverseproxy-relation-departed":
+        reverseproxy_interface("departed")
+    elif HOOK_NAME == "website-relation-joined":
+        website_interface("joined")
+    elif HOOK_NAME == "website-relation-changed":
+        website_interface("changed")
+    elif HOOK_NAME == "nrpe-external-master-relation-changed":
+        update_nrpe_config()
+    else:
+        print "Unknown hook"
+        sys.exit(1)

=== added file 'hooks/test_hooks.py'
--- hooks/test_hooks.py	1970-01-01 00:00:00 +0000
+++ hooks/test_hooks.py	2013-02-13 23:59:22 +0000
@@ -0,0 +1,272 @@
+import hooks
+import yaml
+from textwrap import dedent
+from mocker import MockerTestCase, ARGS
+
+class JujuHookTest(MockerTestCase):
+
+    def setUp(self):
+        self.config_services = [{
+            "service_name": "haproxy_test",
+            "service_host": "0.0.0.0",
+            "service_port": "88",
+            "service_options": ["balance leastconn"],
+            "server_options": "maxconn 25"}]
+        self.config_services_extended = [
+            {"service_name": "unit_service",
+            "service_host": "supplied-hostname",
+            "service_port": "999",
+            "service_options": ["balance leastconn"],
+            "server_options": "maxconn 99"}]
+        self.relation_services = [
+            {"service_name": "foo_svc",
+            "service_options": ["balance leastconn"],
+            "servers": [("A", "hA", "1", "oA1 oA2")]},
+            {"service_name": "bar_svc",
+            "service_options": ["balance leastconn"],
+            "servers": [
+                ("A", "hA", "1", "oA1 oA2"), ("B", "hB", "2", "oB1 oB2")]}]
+        self.relation_services2 = [
+            {"service_name": "foo_svc",
+            "service_options": ["balance leastconn"],
+            "servers": [("A2", "hA2", "12", "oA12 oA22")]}]
+        hooks.default_haproxy_config_dir = self.makeDir()
+        hooks.default_haproxy_config = self.makeFile()
+        hooks.default_haproxy_service_config_dir = self.makeDir()
+        obj = self.mocker.replace("hooks.juju_log")
+        obj(ARGS)
+        self.mocker.count(0, None)
+        obj = self.mocker.replace("hooks.unit_get")
+        obj("public-address")
+        self.mocker.result("test-host.example.com")
+        self.mocker.count(0, None)
+        self.maxDiff = None
+    
+    def _expect_config_get(self, **kwargs):
+        result = {
+            "default_timeouts": "queue 1000, connect 1000, client 1000, server 1000",
+            "global_log": "127.0.0.1 local0, 127.0.0.1 local1 notice",
+            "global_spread_checks": 0,
+            "monitoring_allowed_cidr": "127.0.0.1/32",
+            "monitoring_username": "haproxy",
+            "default_log": "global",
+            "global_group": "haproxy",
+            "monitoring_stats_refresh": 3,
+            "default_retries": 3,
+            "services": yaml.dump(self.config_services),
+            "global_maxconn": 4096,
+            "global_user": "haproxy",
+            "default_options": "httplog, dontlognull",
+            "monitoring_port": 10000,
+            "global_debug": False,
+            "nagios_context": "juju",
+            "global_quiet": False,
+            "enable_monitoring": False,
+            "monitoring_password": "changeme",
+            "default_mode": "http"}
+        obj = self.mocker.replace("hooks.config_get")
+        obj()
+        result.update(kwargs)
+        self.mocker.result(result)
+        self.mocker.count(1, None)
+
+    def _expect_relation_get_all(self, relation, extra={}):
+        obj = self.mocker.replace("hooks.relation_get_all")
+        obj(relation)
+        relation = {"hostname": "10.0.1.2",
+                    "private-address": "10.0.1.2",
+                    "port": "10000"}
+        relation.update(extra)
+        result = {"1": {"unit/0": relation}}
+        self.mocker.result(result)
+        self.mocker.count(1, None)
+
+    def _expect_relation_get_all_multiple(self, relation_name):
+        obj = self.mocker.replace("hooks.relation_get_all")
+        obj(relation_name)
+        result = {
+                "1": {"unit/0": {
+                    "hostname": "10.0.1.2",
+                    "private-address": "10.0.1.2",
+                    "port": "10000",
+                    "services": yaml.dump(self.relation_services)}},
+                "2": {"unit/1": {
+                    "hostname": "10.0.1.3",
+                    "private-address": "10.0.1.3",
+                    "port": "10001",
+                    "services": yaml.dump(self.relation_services2)}}}
+        self.mocker.result(result)
+        self.mocker.count(1, None)
+
+    def _expect_relation_get_all_with_services(self, relation, extra={}):
+        extra.update({"services": yaml.dump(self.relation_services)})
+        return self._expect_relation_get_all(relation, extra)
+
+    def _expect_relation_get(self):
+        obj = self.mocker.replace("hooks.relation_get")
+        obj()
+        result = {}
+        self.mocker.result(result)
+        self.mocker.count(1, None)
+
+    def _expect_relation_set(self, args):
+        """
+        @param args: list of arguments expected to be passed to relation_set
+        """
+        obj = self.mocker.replace("hooks.relation_set")
+        obj(args)
+        self.relation_set = args
+        self.mocker.count(1,None)
+
+    def test_create_services(self):
+        """
+        Simplest use case, config stanza seeded in config file, server line
+        added through simple relation.  Many servers can join this, but
+        multiple services will not be presented to the outside
+        """
+        self._expect_config_get()
+        self._expect_relation_get_all("reverseproxy")
+        self.mocker.replay()
+        hooks.create_services()
+        services = hooks.load_services()
+        stanza = """\
+            listen haproxy_test 0.0.0.0:88
+                balance leastconn
+                server 10_0_1_2__10000 10.0.1.2:10000 maxconn 25
+
+        """
+        self.assertEquals(services, dedent(stanza))
+
+    def test_create_services_extended_with_relation(self):
+        """
+        This case covers specifying an up-front services file to ha-proxy
+        in the config.  The relation then specifies a singular hostname, 
+        port and server_options setting which is filled into the appropriate
+        haproxy stanza based on multiple criteria.
+        """
+        self._expect_config_get(
+                services=yaml.dump(self.config_services_extended))
+        self._expect_relation_get_all("reverseproxy")
+        self.mocker.replay()
+        hooks.create_services()
+        services = hooks.load_services()
+        stanza = """\
+            listen unit_service supplied-hostname:999
+                balance leastconn
+                server 10_0_1_2__10000 10.0.1.2:10000 maxconn 99
+
+        """
+        self.assertEquals(dedent(stanza), services)
+
+    def test_create_services_pure_relation(self):
+        """
+        In this case, the relation is in control of the haproxy config file.
+        Each relation chooses what server it creates in the haproxy file, it
+        relies on the haproxy service only for the hostname and front-end port.
+        Each member of the relation will put a backend server entry under in
+        the desired stanza.  Each realtion can in fact supply multiple
+        entries from the same juju service unit if desired.
+        """
+        self._expect_config_get()
+        self._expect_relation_get_all_with_services("reverseproxy")
+        self.mocker.replay()
+        hooks.create_services()
+        services = hooks.load_services()
+        stanza = """\
+            listen foo_svc 0.0.0.0:88
+                balance leastconn
+                server A hA:1 oA1 oA2
+        """
+        self.assertIn(dedent(stanza), services)
+        stanza = """\
+            listen bar_svc 0.0.0.0:89
+                balance leastconn
+                server A hA:1 oA1 oA2
+                server B hB:2 oB1 oB2
+        """
+        self.assertIn(dedent(stanza), services)
+
+    def test_create_services_pure_relation_multiple(self):
+        """
+        This is much liek the pure_relation case, where the relation specifies
+        a "services" override.  However, in this case we have multiple relations
+        that partially override each other.  We expect that the created haproxy
+        conf file will combine things appropriately.
+        """
+        self._expect_config_get()
+        self._expect_relation_get_all_multiple("reverseproxy")
+        self.mocker.replay()
+        hooks.create_services()
+        result = hooks.load_services()
+        stanza = """\
+            listen foo_svc 0.0.0.0:88
+                balance leastconn
+                server A hA:1 oA1 oA2
+                server A2 hA2:12 oA12 oA22
+        """
+        self.assertIn(dedent(stanza), result)
+        stanza = """\
+            listen bar_svc 0.0.0.0:89
+                balance leastconn
+                server A hA:1 oA1 oA2
+                server B hB:2 oB1 oB2
+        """
+        self.assertIn(dedent(stanza), result)
+
+    def test_get_config_services_config_only(self):
+        """
+        Attempting to catch the case where a relation is not joined yet
+        """
+        self._expect_config_get()
+        obj = self.mocker.replace("hooks.relation_get_all")
+        obj("reverseproxy")
+        self.mocker.result(None)
+        self.mocker.replay()
+        result = hooks.get_config_services()
+        self.assertEquals(result, self.config_services)
+
+    def test_get_config_services_relation_no_services(self):
+        """
+        If the config specifies services and the realtion does not, just the
+        config services should come through.
+        """
+        self._expect_config_get()
+        self._expect_relation_get_all("reverseproxy")
+        self.mocker.replay()
+        result = hooks.get_config_services()
+        self.assertEquals(result, self.config_services)
+
+    def test_get_config_services_relation_with_services(self):
+        """
+        Testing with both the config and relation providing services should
+        yield the just the relation
+        """
+        self._expect_config_get()
+        self._expect_relation_get_all_with_services("reverseproxy")
+        self.mocker.replay()
+        result = hooks.get_config_services()
+        # Just test "servers" since hostname and port and maybe other keys
+        # will be added by the hook
+        self.assertEquals(result[0]["servers"],
+                self.relation_services[0]["servers"])
+
+    def test_config_generation_indempotent(self):
+        self._expect_config_get()
+        self._expect_relation_get_all_multiple("reverseproxy")
+        self.mocker.replay()
+
+        # Test that we generate the same haproxy.conf file each time
+        hooks.create_services()
+        result1 = hooks.load_services()
+        hooks.create_services()
+        result2 = hooks.load_services()
+        self.assertEqual(result1, result2)
+
+    def test_get_all_services(self):
+        self._expect_config_get()
+        self._expect_relation_get_all_multiple("reverseproxy")
+        self.mocker.replay()
+        baseline = [{"service_name": "foo_svc", "service_port": 88},
+                    {"service_name": "bar_svc", "service_port": 89}]
+        services = hooks.get_all_services()
+        self.assertEqual(baseline, services)

=== modified file 'revision'
--- revision	2013-01-15 02:04:18 +0000
+++ revision	2013-02-13 23:59:22 +0000
@@ -1,1 +1,1 @@
-31
+39