Merge lp:~sidnei/charms/precise/squid-reverseproxy/trunk into lp:charms/squid-reverseproxy

[Sidnei da Silva]

* Greatly improve test coverage

* Allow the use of an X-Balancer-Name header to select which cache_peer backend will be used for a specific request.

  This is useful if you have multiple applications sitting behind a single hostname being directed to the same Squid from an Apache service, using the balancer relation. Since you can't rely on 'dstdomain' anymore, this special header is set by the Apache charm in the balancer, and used instead.

* Support 'all-services' being set in the relation, in the way that the haproxy sets it, in addition to the previously supported 'sitenames' setting. Makes it compatible with the haproxy charm.

* When the list of supported 'sitenames' (computed from dstdomain acls) changes, notify services related via the 'cached-website' relation. This allows to add new services in the haproxy service (or really, any other service related), which notifies the squid service, which then bubbles up to services related via cached-website.

[Mark Mims (mark-mims)]

same as apache... please reserve /tests for charm tests

review: Needs Resubmitting

[James Page (james-page)]

Same comments as for haproxy and apache2; charm branch needs to include all source dependencies.

review: Needs Fixing

[lp:~sidnei/charms/precise/squid-reverseproxy/trunk]

41. By Matthias Arnason on 2013-10-29

[sidnei r=tiaz] Expose 'via' config setting, default to on.

[Marco Ceppi (marcoceppi)]

Hi Sidnei! Thanks for submitting all these improvements and clean up to the squid-reverseproxy charm! The addition of unit tests for the hooks is fantastic and really speaks to our attempts of bringing measurable quality in to the charm store.

I've reviewed everything in the code and the changes look fine, and you've addressed the concerns from previous proposals. However, I have concerns of changing the names of configuration options for a charm. I fear this will break existing deployed charms in an unknown way. I've reached out to the mailing list for a consensus of what to do in this case: https://lists.ubuntu.com/archives/juju/2013-November/003127.html I'll return with a verdict from that post.

review: Needs Information

[Sidnei da Silva (sidnei)]

Thanks! As per my reply on the list, that specific change reverts the config argument to the name it had previous to r31.

The currently published charm as of r42 is in fact broken because the config.yaml has 'nagios_check_url' but the update_nrpe_checks function refer to 'nagios_check_http_params', causing a KeyError.

$ bzr branch lp:charms/squid-reverseproxy
$ cd squid-reverseproxy
$ bzr revno
42
$ bzr grep nagios_check
config.yaml: nagios_check_url:
hooks/hooks.py: (config_data['nagios_check_http_params']))

[Marco Ceppi (marcoceppi)]

Hi Sidnei, thanks for the clarification!

review: Approve

=== modified file '.bzrignore'
--- .bzrignore	2013-02-15 02:21:16 +0000
+++ .bzrignore	2013-10-29 21:07:21 +0000
@@ -1,1 +1,10 @@
-basenode/
+revision
+_trial_temp
+.coverage
+coverage.xml
+*.crt
+*.key
+lib/*
+*.pyc
+exec.d
+build/charm-helpers

=== added file 'Makefile'
--- Makefile	1970-01-01 00:00:00 +0000
+++ Makefile	2013-10-29 21:07:21 +0000
@@ -0,0 +1,39 @@
+PWD := $(shell pwd)
+SOURCEDEPS_DIR ?= $(shell dirname $(PWD))/.sourcecode
+HOOKS_DIR := $(PWD)/hooks
+TEST_PREFIX := PYTHONPATH=$(HOOKS_DIR)
+TEST_DIR := $(PWD)/hooks/tests
+CHARM_DIR := $(PWD)
+PYTHON := /usr/bin/env python
+
+
+build: test lint proof
+
+revision:
+	@test -f revision || echo 0 > revision
+
+proof: revision
+	@echo Proofing charm...
+	@(charm proof $(PWD) || [ $$? -eq 100 ]) && echo OK
+	@test `cat revision` = 0 && rm revision
+
+test:
+	@echo Starting tests...
+	@CHARM_DIR=$(CHARM_DIR) $(TEST_PREFIX) nosetests -s $(TEST_DIR)
+
+lint:
+	@echo Checking for Python syntax...
+	@flake8 $(HOOKS_DIR) --ignore=E123 --exclude=$(HOOKS_DIR)/charmhelpers && echo OK
+
+sourcedeps: $(PWD)/config-manager.txt
+	@echo Updating source dependencies...
+	@$(PYTHON) cm.py -c $(PWD)/config-manager.txt \
+		-p $(SOURCEDEPS_DIR) \
+		-t $(PWD)
+	@$(PYTHON) build/charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py \
+		-c charm-helpers.yaml \
+		-b build/charm-helpers \
+		-d hooks/charmhelpers
+	@echo Do not forget to commit the updated files if any.
+
+.PHONY: revision proof test lint sourcedeps charm-payload

=== added directory 'build'
=== added file 'charm-helpers.yaml'
--- charm-helpers.yaml	1970-01-01 00:00:00 +0000
+++ charm-helpers.yaml	2013-10-29 21:07:21 +0000
@@ -0,0 +1,4 @@
+include:
+    - core
+    - fetch
+    - contrib.charmsupport
\ No newline at end of file

=== added file 'cm.py'
--- cm.py	1970-01-01 00:00:00 +0000
+++ cm.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,193 @@
+# Copyright 2010-2013 Canonical Ltd. All rights reserved.
+import os
+import re
+import sys
+import errno
+import hashlib
+import subprocess
+import optparse
+
+from os import curdir
+from bzrlib.branch import Branch
+from bzrlib.plugin import load_plugins
+load_plugins()
+from bzrlib.plugins.launchpad import account as lp_account
+
+if 'GlobalConfig' in dir(lp_account):
+    from bzrlib.config import LocationConfig as LocationConfiguration
+    _ = LocationConfiguration
+else:
+    from bzrlib.config import LocationStack as LocationConfiguration
+    _ = LocationConfiguration
+
+
+def get_branch_config(config_file):
+    """
+    Retrieves the sourcedeps configuration for an source dir.
+    Returns a dict of (branch, revspec) tuples, keyed by branch name.
+    """
+    branches = {}
+    with open(config_file, 'r') as stream:
+        for line in stream:
+            line = line.split('#')[0].strip()
+            bzr_match = re.match(r'(\S+)\s+'
+                                 'lp:([^;]+)'
+                                 '(?:;revno=(\d+))?', line)
+            if bzr_match:
+                name, branch, revno = bzr_match.group(1, 2, 3)
+                if revno is None:
+                    revspec = -1
+                else:
+                    revspec = revno
+                branches[name] = (branch, revspec)
+                continue
+            dir_match = re.match(r'(\S+)\s+'
+                                 '\(directory\)', line)
+            if dir_match:
+                name = dir_match.group(1)
+                branches[name] = None
+    return branches
+
+
+def main(config_file, parent_dir, target_dir, verbose):
+    """Do the deed."""
+
+    try:
+        os.makedirs(parent_dir)
+    except OSError, e:
+        if e.errno != errno.EEXIST:
+            raise
+
+    branches = sorted(get_branch_config(config_file).items())
+    for branch_name, spec in branches:
+        if spec is None:
+            # It's a directory, just create it and move on.
+            destination_path = os.path.join(target_dir, branch_name)
+            if not os.path.isdir(destination_path):
+                os.makedirs(destination_path)
+            continue
+
+        (quoted_branch_spec, revspec) = spec
+        revno = int(revspec)
+
+        # qualify mirror branch name with hash of remote repo path to deal
+        # with changes to the remote branch URL over time
+        branch_spec_digest = hashlib.sha1(quoted_branch_spec).hexdigest()
+        branch_directory = branch_spec_digest
+
+        source_path = os.path.join(parent_dir, branch_directory)
+        destination_path = os.path.join(target_dir, branch_name)
+
+        # Remove leftover symlinks/stray files.
+        try:
+            os.remove(destination_path)
+        except OSError, e:
+            if e.errno != errno.EISDIR and e.errno != errno.ENOENT:
+                raise
+
+        lp_url = "lp:" + quoted_branch_spec
+
+        # Create the local mirror branch if it doesn't already exist
+        if verbose:
+            sys.stderr.write('%30s: ' % (branch_name,))
+            sys.stderr.flush()
+
+        fresh = False
+        if not os.path.exists(source_path):
+            subprocess.check_call(['bzr', 'branch', '-q', '--no-tree',
+                                   '--', lp_url, source_path])
+            fresh = True
+
+        if not fresh:
+            source_branch = Branch.open(source_path)
+            if revno == -1:
+                orig_branch = Branch.open(lp_url)
+                fresh = source_branch.revno() == orig_branch.revno()
+            else:
+                fresh = source_branch.revno() == revno
+
+        # Freshen the source branch if required.
+        if not fresh:
+            subprocess.check_call(['bzr', 'pull', '-q', '--overwrite', '-r',
+                                   str(revno), '-d', source_path,
+                                   '--', lp_url])
+
+        if os.path.exists(destination_path):
+            # Overwrite the destination with the appropriate revision.
+            subprocess.check_call(['bzr', 'clean-tree', '--force', '-q',
+                                   '--ignored', '-d', destination_path])
+            subprocess.check_call(['bzr', 'pull', '-q', '--overwrite',
+                                   '-r', str(revno),
+                                   '-d', destination_path, '--', source_path])
+        else:
+            # Create a new branch.
+            subprocess.check_call(['bzr', 'branch', '-q', '--hardlink',
+                                   '-r', str(revno),
+                                   '--', source_path, destination_path])
+
+        # Check the state of the destination branch.
+        destination_branch = Branch.open(destination_path)
+        destination_revno = destination_branch.revno()
+
+        if verbose:
+            sys.stderr.write('checked out %4s of %s\n' %
+                             ("r" + str(destination_revno), lp_url))
+            sys.stderr.flush()
+
+        if revno != -1 and destination_revno != revno:
+            raise RuntimeError("Expected revno %d but got revno %d" %
+                               (revno, destination_revno))
+
+if __name__ == '__main__':
+    parser = optparse.OptionParser(
+        usage="%prog [options]",
+        description=(
+            "Add a lightweight checkout in <target> for each "
+            "corresponding file in <parent>."),
+        add_help_option=False)
+    parser.add_option(
+        '-p', '--parent', dest='parent',
+        default=None,
+        help=("The directory of the parent tree."),
+        metavar="DIR")
+    parser.add_option(
+        '-t', '--target', dest='target', default=curdir,
+        help=("The directory of the target tree."),
+        metavar="DIR")
+    parser.add_option(
+        '-c', '--config', dest='config', default=None,
+        help=("The config file to be used for config-manager."),
+        metavar="DIR")
+    parser.add_option(
+        '-q', '--quiet', dest='verbose', action='store_false',
+        help="Be less verbose.")
+    parser.add_option(
+        '-v', '--verbose', dest='verbose', action='store_true',
+        help="Be more verbose.")
+    parser.add_option(
+        '-h', '--help', action='help',
+        help="Show this help message and exit.")
+    parser.set_defaults(verbose=True)
+
+    options, args = parser.parse_args()
+
+    if options.parent is None:
+        options.parent = os.environ.get(
+            "SOURCEDEPS_DIR",
+            os.path.join(curdir, ".sourcecode"))
+
+    if options.target is None:
+        parser.error(
+            "Target directory not specified.")
+
+    if options.config is None:
+        config = [arg for arg in args
+                  if arg != "update"]
+        if not config or len(config) > 1:
+            parser.error("Config not specified")
+        options.config = config[0]
+
+    sys.exit(main(config_file=options.config,
+                  parent_dir=options.parent,
+                  target_dir=options.target,
+                  verbose=options.verbose))

=== added file 'config-manager.txt'
--- config-manager.txt	1970-01-01 00:00:00 +0000
+++ config-manager.txt	2013-10-29 21:07:21 +0000
@@ -0,0 +1,6 @@
+# After making changes to this file, to ensure that your sourcedeps are
+# up-to-date do:
+#
+#   make sourcedeps
+
+./build/charm-helpers                   lp:charm-helpers;revno=70

=== modified file 'config.yaml'
--- config.yaml	2013-01-11 16:49:35 +0000
+++ config.yaml	2013-10-29 21:07:21 +0000
@@ -3,10 +3,22 @@
     type: int
     default: 3128
     description: Squid listening port.
+  port_options:
+    type: string
+    default: accel vhost
+    description: Squid listening port options
   log_format:
     type: string
     default: '%>a %ui %un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh'
     description: Format of the squid log.
+  via:
+    type: string
+    default: 'on'
+    description: Add 'Via' header to outgoing requests.
+  x_balancer_name_allowed:
+    type: boolean
+    default: false
+    description: Route based on X-Balancer-Name header set by Apache charm.
   cache_mem_mb:
     type: int
     default: 256
@@ -14,7 +26,7 @@
   cache_size_mb:
     type: int
     default: 512
-    description: Maximum size of the on-disk object cache (MB).
+    description: Maximum size of the on-disk object cache (MB). Set to zero to disable caching.
   cache_dir:
     type: string
     default: '/var/spool/squid3'
@@ -51,20 +63,45 @@
             juju-squid-0
         If you're running multiple environments with the same services in them
         this allows you to differentiate between them.
-  nagios_check_url:
+  nagios_check_http_params:
     default: ""
     type: string
     description: >
-       The URL to check squid has access to, most likely inside your web server farm
+      The parameters to pass to the nrpe plugin check_http.
   nagios_service_type:
      default: "generic"
      type: string
      description: >
-        What service this component forms part of, e.g. supermassive-squid-cluster. Used by nrpe.
+       What service this component forms part of, e.g. supermassive-squid-cluster. Used by nrpe.
+  package_status:
+     default: "install"
+     type: "string"
+     description: >
+       The status of service-affecting packages will be set to this value in the dpkg database.
+       Useful valid values are "install" and "hold".
   refresh_patterns:
     type: string
     default: ''
     description: >
-      JSON-formatted list of refresh patterns. For example:
-      '{"http://www.ubuntu.com": {"min": 0, "percent": 20, "max": 60}, "http://www.canonical.com": {"min": 0, "percent": 20, "max": 120}}'
+      JSON- or YAML-formatted list of refresh patterns. For example:
+      '{"http://www.ubuntu.com": {"min": 0, "percent": 20, "max": 60},
+        "http://www.canonical.com": {"min": 0, "percent": 20, "max": 120}}'
+  services:
+    default: ''
+    type: string
+    description: |
+        Services definition(s). Although the variable type is a string, this is
+        interpreted by the charm as yaml. To use multiple services within the
+        same instance, specify all of the variables (service_name,
+        service_host, service_port) with a "-" before the first variable,
+        service_name, as below.
 
+        - service_name: example_proxy
+          service_domain: example.com
+          servers:
+          - [foo.internal, 80]
+          - [bar.internal, 80]
+  enable_forward_proxy:
+    default: false
+    type: boolean
+    description: Enables forward proxying

=== added file 'files/check_squidpeers'
--- files/check_squidpeers	1970-01-01 00:00:00 +0000
+++ files/check_squidpeers	2013-10-29 21:07:21 +0000
@@ -0,0 +1,75 @@
+#!/usr/bin/python
+
+from operator import itemgetter
+import re
+import subprocess
+import sys
+
+
+parent_re = re.compile(r'^Parent\s+:\s*(?P<name>\S+)')
+status_re = re.compile(r'^Status\s+:\s*(?P<status>\S+)')
+
+STATUS_UP = 'Up'
+
+
+def parse(output):
+    state = 'header'
+    peers = []
+    for line in output.splitlines():
+        if state == 'header':
+            if not line.strip():
+                state = 'header gap'
+        elif state == 'header gap':
+            if not line.strip():
+                state = 'peer'
+                peers.append(dict())
+            elif line.strip() == "There are no neighbors installed.":
+                return peers
+            else:
+                raise AssertionError('Expecting blank line after header?: {}'.format(line))
+        elif state == 'peer':
+            if not line.strip():
+                peers.append(dict())
+            else:
+                match = parent_re.match(line)
+                if match:
+                    peers[-1]['name'] = match.group('name')
+                match = status_re.match(line)
+                if match:
+                    peers[-1]['up'] = match.group('status') == STATUS_UP
+                    peers[-1]['status'] = match.group('status')
+    return peers
+
+
+def get_status(peers):
+    ok = all(map(itemgetter('up'), peers))
+    if not peers:
+        retcode = 1
+        message = 'Squid has no configured peers.'
+    elif ok:
+        retcode = 0
+        message = 'All peers are UP according to squid.'
+    else:
+        retcode = 2
+        peer_info = ["{}: {}".format(p['name'], p['status']) for p in peers
+                     if not p['up']]
+        message = 'The following peers are not UP according to squid: {}'.format(
+                ", ".join(peer_info))
+    return retcode, message
+
+
+def main():
+    proc = subprocess.Popen(['squidclient', 'mgr:server_list'],
+            stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    stdout, stderr = proc.communicate()
+    if proc.returncode != 0:
+        print("Error running squidclient: %s" % stderr)
+        return 2
+    peers = parse(stdout)
+    retcode, message = get_status(peers)
+    print(message)
+    return retcode
+
+
+if __name__ == '__main__':
+    sys.exit(main())

=== added directory 'hooks/charmhelpers'
=== added file 'hooks/charmhelpers/__init__.py'
=== added directory 'hooks/charmhelpers/contrib'
=== added file 'hooks/charmhelpers/contrib/__init__.py'
=== added directory 'hooks/charmhelpers/contrib/charmsupport'
=== added file 'hooks/charmhelpers/contrib/charmsupport/__init__.py'
=== added file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py'
--- hooks/charmhelpers/contrib/charmsupport/nrpe.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/charmsupport/nrpe.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,218 @@
+"""Compatibility with the nrpe-external-master charm"""
+# Copyright 2012 Canonical Ltd.
+#
+# Authors:
+#  Matthew Wedgwood <[e-mail]>
+
+import subprocess
+import pwd
+import grp
+import os
+import re
+import shlex
+import yaml
+
+from charmhelpers.core.hookenv import (
+    config,
+    local_unit,
+    log,
+    relation_ids,
+    relation_set,
+)
+
+from charmhelpers.core.host import service
+
+# This module adds compatibility with the nrpe-external-master and plain nrpe
+# subordinate charms. To use it in your charm:
+#
+# 1. Update metadata.yaml
+#
+#   provides:
+#     (...)
+#     nrpe-external-master:
+#       interface: nrpe-external-master
+#       scope: container
+#
+#   and/or
+#
+#   provides:
+#     (...)
+#     local-monitors:
+#       interface: local-monitors
+#       scope: container
+
+#
+# 2. Add the following to config.yaml
+#
+#    nagios_context:
+#      default: "juju"
+#      type: string
+#      description: |
+#        Used by the nrpe subordinate charms.
+#        A string that will be prepended to instance name to set the host name
+#        in nagios. So for instance the hostname would be something like:
+#            juju-myservice-0
+#        If you're running multiple environments with the same services in them
+#        this allows you to differentiate between them.
+#
+# 3. Add custom checks (Nagios plugins) to files/nrpe-external-master
+#
+# 4. Update your hooks.py with something like this:
+#
+#    from charmsupport.nrpe import NRPE
+#    (...)
+#    def update_nrpe_config():
+#        nrpe_compat = NRPE()
+#        nrpe_compat.add_check(
+#            shortname = "myservice",
+#            description = "Check MyService",
+#            check_cmd = "check_http -w 2 -c 10 http://localhost"
+#            )
+#        nrpe_compat.add_check(
+#            "myservice_other",
+#            "Check for widget failures",
+#            check_cmd = "/srv/myapp/scripts/widget_check"
+#            )
+#        nrpe_compat.write()
+#
+#    def config_changed():
+#        (...)
+#        update_nrpe_config()
+#
+#    def nrpe_external_master_relation_changed():
+#        update_nrpe_config()
+#
+#    def local_monitors_relation_changed():
+#        update_nrpe_config()
+#
+# 5. ln -s hooks.py nrpe-external-master-relation-changed
+#    ln -s hooks.py local-monitors-relation-changed
+
+
+class CheckException(Exception):
+    pass
+
+
+class Check(object):
+    shortname_re = '[A-Za-z0-9-_]+$'
+    service_template = ("""
+#---------------------------------------------------
+# This file is Juju managed
+#---------------------------------------------------
+define service {{
+    use                             active-service
+    host_name                       {nagios_hostname}
+    service_description             {nagios_hostname}[{shortname}] """
+                        """{description}
+    check_command                   check_nrpe!{command}
+    servicegroups                   {nagios_servicegroup}
+}}
+""")
+
+    def __init__(self, shortname, description, check_cmd):
+        super(Check, self).__init__()
+        # XXX: could be better to calculate this from the service name
+        if not re.match(self.shortname_re, shortname):
+            raise CheckException("shortname must match {}".format(
+                Check.shortname_re))
+        self.shortname = shortname
+        self.command = "check_{}".format(shortname)
+        # Note: a set of invalid characters is defined by the
+        # Nagios server config
+        # The default is: illegal_object_name_chars=`~!$%^&*"|'<>?,()=
+        self.description = description
+        self.check_cmd = self._locate_cmd(check_cmd)
+
+    def _locate_cmd(self, check_cmd):
+        search_path = (
+            '/',
+            os.path.join(os.environ['CHARM_DIR'],
+                         'files/nrpe-external-master'),
+            '/usr/lib/nagios/plugins',
+        )
+        parts = shlex.split(check_cmd)
+        for path in search_path:
+            if os.path.exists(os.path.join(path, parts[0])):
+                command = os.path.join(path, parts[0])
+                if len(parts) > 1:
+                    command += " " + " ".join(parts[1:])
+                return command
+        log('Check command not found: {}'.format(parts[0]))
+        return ''
+
+    def write(self, nagios_context, hostname):
+        nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format(
+            self.command)
+        with open(nrpe_check_file, 'w') as nrpe_check_config:
+            nrpe_check_config.write("# check {}\n".format(self.shortname))
+            nrpe_check_config.write("command[{}]={}\n".format(
+                self.command, self.check_cmd))
+
+        if not os.path.exists(NRPE.nagios_exportdir):
+            log('Not writing service config as {} is not accessible'.format(
+                NRPE.nagios_exportdir))
+        else:
+            self.write_service_config(nagios_context, hostname)
+
+    def write_service_config(self, nagios_context, hostname):
+        for f in os.listdir(NRPE.nagios_exportdir):
+            if re.search('.*{}.cfg'.format(self.command), f):
+                os.remove(os.path.join(NRPE.nagios_exportdir, f))
+
+        templ_vars = {
+            'nagios_hostname': hostname,
+            'nagios_servicegroup': nagios_context,
+            'description': self.description,
+            'shortname': self.shortname,
+            'command': self.command,
+        }
+        nrpe_service_text = Check.service_template.format(**templ_vars)
+        nrpe_service_file = '{}/service__{}_{}.cfg'.format(
+            NRPE.nagios_exportdir, hostname, self.command)
+        with open(nrpe_service_file, 'w') as nrpe_service_config:
+            nrpe_service_config.write(str(nrpe_service_text))
+
+    def run(self):
+        subprocess.call(self.check_cmd)
+
+
+class NRPE(object):
+    nagios_logdir = '/var/log/nagios'
+    nagios_exportdir = '/var/lib/nagios/export'
+    nrpe_confdir = '/etc/nagios/nrpe.d'
+
+    def __init__(self):
+        super(NRPE, self).__init__()
+        self.config = config()
+        self.nagios_context = self.config['nagios_context']
+        self.unit_name = local_unit().replace('/', '-')
+        self.hostname = "{}-{}".format(self.nagios_context, self.unit_name)
+        self.checks = []
+
+    def add_check(self, *args, **kwargs):
+        self.checks.append(Check(*args, **kwargs))
+
+    def write(self):
+        try:
+            nagios_uid = pwd.getpwnam('nagios').pw_uid
+            nagios_gid = grp.getgrnam('nagios').gr_gid
+        except:
+            log("Nagios user not set up, nrpe checks not updated")
+            return
+
+        if not os.path.exists(NRPE.nagios_logdir):
+            os.mkdir(NRPE.nagios_logdir)
+            os.chown(NRPE.nagios_logdir, nagios_uid, nagios_gid)
+
+        nrpe_monitors = {}
+        monitors = {"monitors": {"remote": {"nrpe": nrpe_monitors}}}
+        for nrpecheck in self.checks:
+            nrpecheck.write(self.nagios_context, self.hostname)
+            nrpe_monitors[nrpecheck.shortname] = {
+                "command": nrpecheck.command,
+            }
+
+        service('restart', 'nagios-nrpe-server')
+
+        for rid in relation_ids("local-monitors"):
+            relation_set(relation_id=rid, monitors=yaml.dump(monitors))

=== added file 'hooks/charmhelpers/contrib/charmsupport/volumes.py'
--- hooks/charmhelpers/contrib/charmsupport/volumes.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/charmsupport/volumes.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,156 @@
+'''
+Functions for managing volumes in juju units. One volume is supported per unit.
+Subordinates may have their own storage, provided it is on its own partition.
+
+Configuration stanzas:
+  volume-ephemeral:
+    type: boolean
+    default: true
+    description: >
+      If false, a volume is mounted as sepecified in "volume-map"
+      If true, ephemeral storage will be used, meaning that log data
+         will only exist as long as the machine. YOU HAVE BEEN WARNED.
+  volume-map:
+    type: string
+    default: {}
+    description: >
+      YAML map of units to device names, e.g:
+        "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }"
+      Service units will raise a configure-error if volume-ephemeral
+      is 'true' and no volume-map value is set. Use 'juju set' to set a
+      value and 'juju resolved' to complete configuration.
+
+Usage:
+    from charmsupport.volumes import configure_volume, VolumeConfigurationError
+    from charmsupport.hookenv import log, ERROR
+    def post_mount_hook():
+        stop_service('myservice')
+    def post_mount_hook():
+        start_service('myservice')
+
+    if __name__ == '__main__':
+        try:
+            configure_volume(before_change=pre_mount_hook,
+                             after_change=post_mount_hook)
+        except VolumeConfigurationError:
+            log('Storage could not be configured', ERROR)
+'''
+
+# XXX: Known limitations
+# - fstab is neither consulted nor updated
+
+import os
+from charmhelpers.core import hookenv
+from charmhelpers.core import host
+import yaml
+
+
+MOUNT_BASE = '/srv/juju/volumes'
+
+
+class VolumeConfigurationError(Exception):
+    '''Volume configuration data is missing or invalid'''
+    pass
+
+
+def get_config():
+    '''Gather and sanity-check volume configuration data'''
+    volume_config = {}
+    config = hookenv.config()
+
+    errors = False
+
+    if config.get('volume-ephemeral') in (True, 'True', 'true', 'Yes', 'yes'):
+        volume_config['ephemeral'] = True
+    else:
+        volume_config['ephemeral'] = False
+
+    try:
+        volume_map = yaml.safe_load(config.get('volume-map', '{}'))
+    except yaml.YAMLError as e:
+        hookenv.log("Error parsing YAML volume-map: {}".format(e),
+                    hookenv.ERROR)
+        errors = True
+    if volume_map is None:
+        # probably an empty string
+        volume_map = {}
+    elif not isinstance(volume_map, dict):
+        hookenv.log("Volume-map should be a dictionary, not {}".format(
+            type(volume_map)))
+        errors = True
+
+    volume_config['device'] = volume_map.get(os.environ['JUJU_UNIT_NAME'])
+    if volume_config['device'] and volume_config['ephemeral']:
+        # asked for ephemeral storage but also defined a volume ID
+        hookenv.log('A volume is defined for this unit, but ephemeral '
+                    'storage was requested', hookenv.ERROR)
+        errors = True
+    elif not volume_config['device'] and not volume_config['ephemeral']:
+        # asked for permanent storage but did not define volume ID
+        hookenv.log('Ephemeral storage was requested, but there is no volume '
+                    'defined for this unit.', hookenv.ERROR)
+        errors = True
+
+    unit_mount_name = hookenv.local_unit().replace('/', '-')
+    volume_config['mountpoint'] = os.path.join(MOUNT_BASE, unit_mount_name)
+
+    if errors:
+        return None
+    return volume_config
+
+
+def mount_volume(config):
+    if os.path.exists(config['mountpoint']):
+        if not os.path.isdir(config['mountpoint']):
+            hookenv.log('Not a directory: {}'.format(config['mountpoint']))
+            raise VolumeConfigurationError()
+    else:
+        host.mkdir(config['mountpoint'])
+    if os.path.ismount(config['mountpoint']):
+        unmount_volume(config)
+    if not host.mount(config['device'], config['mountpoint'], persist=True):
+        raise VolumeConfigurationError()
+
+
+def unmount_volume(config):
+    if os.path.ismount(config['mountpoint']):
+        if not host.umount(config['mountpoint'], persist=True):
+            raise VolumeConfigurationError()
+
+
+def managed_mounts():
+    '''List of all mounted managed volumes'''
+    return filter(lambda mount: mount[0].startswith(MOUNT_BASE), host.mounts())
+
+
+def configure_volume(before_change=lambda: None, after_change=lambda: None):
+    '''Set up storage (or don't) according to the charm's volume configuration.
+       Returns the mount point or "ephemeral". before_change and after_change
+       are optional functions to be called if the volume configuration changes.
+    '''
+
+    config = get_config()
+    if not config:
+        hookenv.log('Failed to read volume configuration', hookenv.CRITICAL)
+        raise VolumeConfigurationError()
+
+    if config['ephemeral']:
+        if os.path.ismount(config['mountpoint']):
+            before_change()
+            unmount_volume(config)
+            after_change()
+        return 'ephemeral'
+    else:
+        # persistent storage
+        if os.path.ismount(config['mountpoint']):
+            mounts = dict(managed_mounts())
+            if mounts.get(config['mountpoint']) != config['device']:
+                before_change()
+                unmount_volume(config)
+                mount_volume(config)
+                after_change()
+        else:
+            before_change()
+            mount_volume(config)
+            after_change()
+        return config['mountpoint']

=== added directory 'hooks/charmhelpers/core'
=== added file 'hooks/charmhelpers/core/__init__.py'
=== added file 'hooks/charmhelpers/core/hookenv.py'
--- hooks/charmhelpers/core/hookenv.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/hookenv.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,340 @@
+"Interactions with the Juju environment"
+# Copyright 2013 Canonical Ltd.
+#
+# Authors:
+#  Charm Helpers Developers <[e-mail]>
+
+import os
+import json
+import yaml
+import subprocess
+import UserDict
+
+CRITICAL = "CRITICAL"
+ERROR = "ERROR"
+WARNING = "WARNING"
+INFO = "INFO"
+DEBUG = "DEBUG"
+MARKER = object()
+
+cache = {}
+
+
+def cached(func):
+    ''' Cache return values for multiple executions of func + args
+
+    For example:
+
+        @cached
+        def unit_get(attribute):
+            pass
+
+        unit_get('test')
+
+    will cache the result of unit_get + 'test' for future calls.
+    '''
+    def wrapper(*args, **kwargs):
+        global cache
+        key = str((func, args, kwargs))
+        try:
+            return cache[key]
+        except KeyError:
+            res = func(*args, **kwargs)
+            cache[key] = res
+            return res
+    return wrapper
+
+
+def flush(key):
+    ''' Flushes any entries from function cache where the
+    key is found in the function+args '''
+    flush_list = []
+    for item in cache:
+        if key in item:
+            flush_list.append(item)
+    for item in flush_list:
+        del cache[item]
+
+
+def log(message, level=None):
+    "Write a message to the juju log"
+    command = ['juju-log']
+    if level:
+        command += ['-l', level]
+    command += [message]
+    subprocess.call(command)
+
+
+class Serializable(UserDict.IterableUserDict):
+    "Wrapper, an object that can be serialized to yaml or json"
+
+    def __init__(self, obj):
+        # wrap the object
+        UserDict.IterableUserDict.__init__(self)
+        self.data = obj
+
+    def __getattr__(self, attr):
+        # See if this object has attribute.
+        if attr in ("json", "yaml", "data"):
+            return self.__dict__[attr]
+        # Check for attribute in wrapped object.
+        got = getattr(self.data, attr, MARKER)
+        if got is not MARKER:
+            return got
+        # Proxy to the wrapped object via dict interface.
+        try:
+            return self.data[attr]
+        except KeyError:
+            raise AttributeError(attr)
+
+    def __getstate__(self):
+        # Pickle as a standard dictionary.
+        return self.data
+
+    def __setstate__(self, state):
+        # Unpickle into our wrapper.
+        self.data = state
+
+    def json(self):
+        "Serialize the object to json"
+        return json.dumps(self.data)
+
+    def yaml(self):
+        "Serialize the object to yaml"
+        return yaml.dump(self.data)
+
+
+def execution_environment():
+    """A convenient bundling of the current execution context"""
+    context = {}
+    context['conf'] = config()
+    if relation_id():
+        context['reltype'] = relation_type()
+        context['relid'] = relation_id()
+        context['rel'] = relation_get()
+    context['unit'] = local_unit()
+    context['rels'] = relations()
+    context['env'] = os.environ
+    return context
+
+
+def in_relation_hook():
+    "Determine whether we're running in a relation hook"
+    return 'JUJU_RELATION' in os.environ
+
+
+def relation_type():
+    "The scope for the current relation hook"
+    return os.environ.get('JUJU_RELATION', None)
+
+
+def relation_id():
+    "The relation ID for the current relation hook"
+    return os.environ.get('JUJU_RELATION_ID', None)
+
+
+def local_unit():
+    "Local unit ID"
+    return os.environ['JUJU_UNIT_NAME']
+
+
+def remote_unit():
+    "The remote unit for the current relation hook"
+    return os.environ['JUJU_REMOTE_UNIT']
+
+
+def service_name():
+    "The name service group this unit belongs to"
+    return local_unit().split('/')[0]
+
+
+@cached
+def config(scope=None):
+    "Juju charm configuration"
+    config_cmd_line = ['config-get']
+    if scope is not None:
+        config_cmd_line.append(scope)
+    config_cmd_line.append('--format=json')
+    try:
+        return json.loads(subprocess.check_output(config_cmd_line))
+    except ValueError:
+        return None
+
+
+@cached
+def relation_get(attribute=None, unit=None, rid=None):
+    _args = ['relation-get', '--format=json']
+    if rid:
+        _args.append('-r')
+        _args.append(rid)
+    _args.append(attribute or '-')
+    if unit:
+        _args.append(unit)
+    try:
+        return json.loads(subprocess.check_output(_args))
+    except ValueError:
+        return None
+
+
+def relation_set(relation_id=None, relation_settings={}, **kwargs):
+    relation_cmd_line = ['relation-set']
+    if relation_id is not None:
+        relation_cmd_line.extend(('-r', relation_id))
+    for k, v in (relation_settings.items() + kwargs.items()):
+        if v is None:
+            relation_cmd_line.append('{}='.format(k))
+        else:
+            relation_cmd_line.append('{}={}'.format(k, v))
+    subprocess.check_call(relation_cmd_line)
+    # Flush cache of any relation-gets for local unit
+    flush(local_unit())
+
+
+@cached
+def relation_ids(reltype=None):
+    "A list of relation_ids"
+    reltype = reltype or relation_type()
+    relid_cmd_line = ['relation-ids', '--format=json']
+    if reltype is not None:
+        relid_cmd_line.append(reltype)
+        return json.loads(subprocess.check_output(relid_cmd_line)) or []
+    return []
+
+
+@cached
+def related_units(relid=None):
+    "A list of related units"
+    relid = relid or relation_id()
+    units_cmd_line = ['relation-list', '--format=json']
+    if relid is not None:
+        units_cmd_line.extend(('-r', relid))
+    return json.loads(subprocess.check_output(units_cmd_line)) or []
+
+
+@cached
+def relation_for_unit(unit=None, rid=None):
+    "Get the json represenation of a unit's relation"
+    unit = unit or remote_unit()
+    relation = relation_get(unit=unit, rid=rid)
+    for key in relation:
+        if key.endswith('-list'):
+            relation[key] = relation[key].split()
+    relation['__unit__'] = unit
+    return relation
+
+
+@cached
+def relations_for_id(relid=None):
+    "Get relations of a specific relation ID"
+    relation_data = []
+    relid = relid or relation_ids()
+    for unit in related_units(relid):
+        unit_data = relation_for_unit(unit, relid)
+        unit_data['__relid__'] = relid
+        relation_data.append(unit_data)
+    return relation_data
+
+
+@cached
+def relations_of_type(reltype=None):
+    "Get relations of a specific type"
+    relation_data = []
+    reltype = reltype or relation_type()
+    for relid in relation_ids(reltype):
+        for relation in relations_for_id(relid):
+            relation['__relid__'] = relid
+            relation_data.append(relation)
+    return relation_data
+
+
+@cached
+def relation_types():
+    "Get a list of relation types supported by this charm"
+    charmdir = os.environ.get('CHARM_DIR', '')
+    mdf = open(os.path.join(charmdir, 'metadata.yaml'))
+    md = yaml.safe_load(mdf)
+    rel_types = []
+    for key in ('provides', 'requires', 'peers'):
+        section = md.get(key)
+        if section:
+            rel_types.extend(section.keys())
+    mdf.close()
+    return rel_types
+
+
+@cached
+def relations():
+    rels = {}
+    for reltype in relation_types():
+        relids = {}
+        for relid in relation_ids(reltype):
+            units = {local_unit(): relation_get(unit=local_unit(), rid=relid)}
+            for unit in related_units(relid):
+                reldata = relation_get(unit=unit, rid=relid)
+                units[unit] = reldata
+            relids[relid] = units
+        rels[reltype] = relids
+    return rels
+
+
+def open_port(port, protocol="TCP"):
+    "Open a service network port"
+    _args = ['open-port']
+    _args.append('{}/{}'.format(port, protocol))
+    subprocess.check_call(_args)
+
+
+def close_port(port, protocol="TCP"):
+    "Close a service network port"
+    _args = ['close-port']
+    _args.append('{}/{}'.format(port, protocol))
+    subprocess.check_call(_args)
+
+
+@cached
+def unit_get(attribute):
+    _args = ['unit-get', '--format=json', attribute]
+    try:
+        return json.loads(subprocess.check_output(_args))
+    except ValueError:
+        return None
+
+
+def unit_private_ip():
+    return unit_get('private-address')
+
+
+class UnregisteredHookError(Exception):
+    pass
+
+
+class Hooks(object):
+    def __init__(self):
+        super(Hooks, self).__init__()
+        self._hooks = {}
+
+    def register(self, name, function):
+        self._hooks[name] = function
+
+    def execute(self, args):
+        hook_name = os.path.basename(args[0])
+        if hook_name in self._hooks:
+            self._hooks[hook_name]()
+        else:
+            raise UnregisteredHookError(hook_name)
+
+    def hook(self, *hook_names):
+        def wrapper(decorated):
+            for hook_name in hook_names:
+                self.register(hook_name, decorated)
+            else:
+                self.register(decorated.__name__, decorated)
+                if '_' in decorated.__name__:
+                    self.register(
+                        decorated.__name__.replace('_', '-'), decorated)
+            return decorated
+        return wrapper
+
+
+def charm_dir():
+    return os.environ.get('CHARM_DIR')

=== added file 'hooks/charmhelpers/core/host.py'
--- hooks/charmhelpers/core/host.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/core/host.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,239 @@
+"""Tools for working with the host system"""
+# Copyright 2012 Canonical Ltd.
+#
+# Authors:
+#  Nick Moffitt <[e-mail]>
+#  Matthew Wedgwood <[e-mail]>
+
+import os
+import pwd
+import grp
+import random
+import string
+import subprocess
+import hashlib
+
+from collections import OrderedDict
+
+from hookenv import log
+
+
+def service_start(service_name):
+    service('start', service_name)
+
+
+def service_stop(service_name):
+    service('stop', service_name)
+
+
+def service_restart(service_name):
+    service('restart', service_name)
+
+
+def service_reload(service_name, restart_on_failure=False):
+    if not service('reload', service_name) and restart_on_failure:
+        service('restart', service_name)
+
+
+def service(action, service_name):
+    cmd = ['service', service_name, action]
+    return subprocess.call(cmd) == 0
+
+
+def service_running(service):
+    try:
+        output = subprocess.check_output(['service', service, 'status'])
+    except subprocess.CalledProcessError:
+        return False
+    else:
+        if ("start/running" in output or "is running" in output):
+            return True
+        else:
+            return False
+
+
+def adduser(username, password=None, shell='/bin/bash', system_user=False):
+    """Add a user"""
+    try:
+        user_info = pwd.getpwnam(username)
+        log('user {0} already exists!'.format(username))
+    except KeyError:
+        log('creating user {0}'.format(username))
+        cmd = ['useradd']
+        if system_user or password is None:
+            cmd.append('--system')
+        else:
+            cmd.extend([
+                '--create-home',
+                '--shell', shell,
+                '--password', password,
+            ])
+        cmd.append(username)
+        subprocess.check_call(cmd)
+        user_info = pwd.getpwnam(username)
+    return user_info
+
+
+def add_user_to_group(username, group):
+    """Add a user to a group"""
+    cmd = [
+        'gpasswd', '-a',
+        username,
+        group
+    ]
+    log("Adding user {} to group {}".format(username, group))
+    subprocess.check_call(cmd)
+
+
+def rsync(from_path, to_path, flags='-r', options=None):
+    """Replicate the contents of a path"""
+    options = options or ['--delete', '--executability']
+    cmd = ['/usr/bin/rsync', flags]
+    cmd.extend(options)
+    cmd.append(from_path)
+    cmd.append(to_path)
+    log(" ".join(cmd))
+    return subprocess.check_output(cmd).strip()
+
+
+def symlink(source, destination):
+    """Create a symbolic link"""
+    log("Symlinking {} as {}".format(source, destination))
+    cmd = [
+        'ln',
+        '-sf',
+        source,
+        destination,
+    ]
+    subprocess.check_call(cmd)
+
+
+def mkdir(path, owner='root', group='root', perms=0555, force=False):
+    """Create a directory"""
+    log("Making dir {} {}:{} {:o}".format(path, owner, group,
+                                          perms))
+    uid = pwd.getpwnam(owner).pw_uid
+    gid = grp.getgrnam(group).gr_gid
+    realpath = os.path.abspath(path)
+    if os.path.exists(realpath):
+        if force and not os.path.isdir(realpath):
+            log("Removing non-directory file {} prior to mkdir()".format(path))
+            os.unlink(realpath)
+    else:
+        os.makedirs(realpath, perms)
+    os.chown(realpath, uid, gid)
+
+
+def write_file(path, content, owner='root', group='root', perms=0444):
+    """Create or overwrite a file with the contents of a string"""
+    log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
+    uid = pwd.getpwnam(owner).pw_uid
+    gid = grp.getgrnam(group).gr_gid
+    with open(path, 'w') as target:
+        os.fchown(target.fileno(), uid, gid)
+        os.fchmod(target.fileno(), perms)
+        target.write(content)
+
+
+def mount(device, mountpoint, options=None, persist=False):
+    '''Mount a filesystem'''
+    cmd_args = ['mount']
+    if options is not None:
+        cmd_args.extend(['-o', options])
+    cmd_args.extend([device, mountpoint])
+    try:
+        subprocess.check_output(cmd_args)
+    except subprocess.CalledProcessError, e:
+        log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output))
+        return False
+    if persist:
+        # TODO: update fstab
+        pass
+    return True
+
+
+def umount(mountpoint, persist=False):
+    '''Unmount a filesystem'''
+    cmd_args = ['umount', mountpoint]
+    try:
+        subprocess.check_output(cmd_args)
+    except subprocess.CalledProcessError, e:
+        log('Error unmounting {}\n{}'.format(mountpoint, e.output))
+        return False
+    if persist:
+        # TODO: update fstab
+        pass
+    return True
+
+
+def mounts():
+    '''List of all mounted volumes as [[mountpoint,device],[...]]'''
+    with open('/proc/mounts') as f:
+        # [['/mount/point','/dev/path'],[...]]
+        system_mounts = [m[1::-1] for m in [l.strip().split()
+                                            for l in f.readlines()]]
+    return system_mounts
+
+
+def file_hash(path):
+    ''' Generate a md5 hash of the contents of 'path' or None if not found '''
+    if os.path.exists(path):
+        h = hashlib.md5()
+        with open(path, 'r') as source:
+            h.update(source.read())  # IGNORE:E1101 - it does have update
+        return h.hexdigest()
+    else:
+        return None
+
+
+def restart_on_change(restart_map):
+    ''' Restart services based on configuration files changing
+
+    This function is used a decorator, for example
+
+        @restart_on_change({
+            '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]
+            })
+        def ceph_client_changed():
+            ...
+
+    In this example, the cinder-api and cinder-volume services
+    would be restarted if /etc/ceph/ceph.conf is changed by the
+    ceph_client_changed function.
+    '''
+    def wrap(f):
+        def wrapped_f(*args):
+            checksums = {}
+            for path in restart_map:
+                checksums[path] = file_hash(path)
+            f(*args)
+            restarts = []
+            for path in restart_map:
+                if checksums[path] != file_hash(path):
+                    restarts += restart_map[path]
+            for service_name in list(OrderedDict.fromkeys(restarts)):
+                service('restart', service_name)
+        return wrapped_f
+    return wrap
+
+
+def lsb_release():
+    '''Return /etc/lsb-release in a dict'''
+    d = {}
+    with open('/etc/lsb-release', 'r') as lsb:
+        for l in lsb:
+            k, v = l.split('=')
+            d[k.strip()] = v.strip()
+    return d
+
+
+def pwgen(length=None):
+    '''Generate a random pasword.'''
+    if length is None:
+        length = random.choice(range(35, 45))
+    alphanumeric_chars = [
+        l for l in (string.letters + string.digits)
+        if l not in 'l0QD1vAEIOUaeiou']
+    random_chars = [
+        random.choice(alphanumeric_chars) for _ in range(length)]
+    return(''.join(random_chars))

=== added directory 'hooks/charmhelpers/fetch'
=== added file 'hooks/charmhelpers/fetch/__init__.py'
--- hooks/charmhelpers/fetch/__init__.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/__init__.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,209 @@
+import importlib
+from yaml import safe_load
+from charmhelpers.core.host import (
+    lsb_release
+)
+from urlparse import (
+    urlparse,
+    urlunparse,
+)
+import subprocess
+from charmhelpers.core.hookenv import (
+    config,
+    log,
+)
+import apt_pkg
+
+CLOUD_ARCHIVE = """# Ubuntu Cloud Archive
+deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main
+"""
+PROPOSED_POCKET = """# Proposed
+deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted
+"""
+
+
+def filter_installed_packages(packages):
+    """Returns a list of packages that require installation"""
+    apt_pkg.init()
+    cache = apt_pkg.Cache()
+    _pkgs = []
+    for package in packages:
+        try:
+            p = cache[package]
+            p.current_ver or _pkgs.append(package)
+        except KeyError:
+            log('Package {} has no installation candidate.'.format(package),
+                level='WARNING')
+            _pkgs.append(package)
+    return _pkgs
+
+
+def apt_install(packages, options=None, fatal=False):
+    """Install one or more packages"""
+    options = options or []
+    cmd = ['apt-get', '-y']
+    cmd.extend(options)
+    cmd.append('install')
+    if isinstance(packages, basestring):
+        cmd.append(packages)
+    else:
+        cmd.extend(packages)
+    log("Installing {} with options: {}".format(packages,
+                                                options))
+    if fatal:
+        subprocess.check_call(cmd)
+    else:
+        subprocess.call(cmd)
+
+
+def apt_update(fatal=False):
+    """Update local apt cache"""
+    cmd = ['apt-get', 'update']
+    if fatal:
+        subprocess.check_call(cmd)
+    else:
+        subprocess.call(cmd)
+
+
+def apt_purge(packages, fatal=False):
+    """Purge one or more packages"""
+    cmd = ['apt-get', '-y', 'purge']
+    if isinstance(packages, basestring):
+        cmd.append(packages)
+    else:
+        cmd.extend(packages)
+    log("Purging {}".format(packages))
+    if fatal:
+        subprocess.check_call(cmd)
+    else:
+        subprocess.call(cmd)
+
+
+def add_source(source, key=None):
+    if ((source.startswith('ppa:') or
+         source.startswith('http:'))):
+        subprocess.check_call(['add-apt-repository', '--yes', source])
+    elif source.startswith('cloud:'):
+        apt_install(filter_installed_packages(['ubuntu-cloud-keyring']),
+                    fatal=True)
+        pocket = source.split(':')[-1]
+        with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt:
+            apt.write(CLOUD_ARCHIVE.format(pocket))
+    elif source == 'proposed':
+        release = lsb_release()['DISTRIB_CODENAME']
+        with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt:
+            apt.write(PROPOSED_POCKET.format(release))
+    if key:
+        subprocess.check_call(['apt-key', 'import', key])
+
+
+class SourceConfigError(Exception):
+    pass
+
+
+def configure_sources(update=False,
+                      sources_var='install_sources',
+                      keys_var='install_keys'):
+    """
+    Configure multiple sources from charm configuration
+
+    Example config:
+        install_sources:
+          - "ppa:foo"
+          - "http://example.com/repo precise main"
+        install_keys:
+          - null
+          - "a1b2c3d4"
+
+    Note that 'null' (a.k.a. None) should not be quoted.
+    """
+    sources = safe_load(config(sources_var))
+    keys = safe_load(config(keys_var))
+    if isinstance(sources, basestring) and isinstance(keys, basestring):
+        add_source(sources, keys)
+    else:
+        if not len(sources) == len(keys):
+            msg = 'Install sources and keys lists are different lengths'
+            raise SourceConfigError(msg)
+        for src_num in range(len(sources)):
+            add_source(sources[src_num], keys[src_num])
+    if update:
+        apt_update(fatal=True)
+
+# The order of this list is very important. Handlers should be listed in from
+# least- to most-specific URL matching.
+FETCH_HANDLERS = (
+    'charmhelpers.fetch.archiveurl.ArchiveUrlFetchHandler',
+    'charmhelpers.fetch.bzrurl.BzrUrlFetchHandler',
+)
+
+
+class UnhandledSource(Exception):
+    pass
+
+
+def install_remote(source):
+    """
+    Install a file tree from a remote source
+
+    The specified source should be a url of the form:
+        scheme://[host]/path[#[option=value][&...]]
+
+    Schemes supported are based on this modules submodules
+    Options supported are submodule-specific"""
+    # We ONLY check for True here because can_handle may return a string
+    # explaining why it can't handle a given source.
+    handlers = [h for h in plugins() if h.can_handle(source) is True]
+    installed_to = None
+    for handler in handlers:
+        try:
+            installed_to = handler.install(source)
+        except UnhandledSource:
+            pass
+    if not installed_to:
+        raise UnhandledSource("No handler found for source {}".format(source))
+    return installed_to
+
+
+def install_from_config(config_var_name):
+    charm_config = config()
+    source = charm_config[config_var_name]
+    return install_remote(source)
+
+
+class BaseFetchHandler(object):
+    """Base class for FetchHandler implementations in fetch plugins"""
+    def can_handle(self, source):
+        """Returns True if the source can be handled. Otherwise returns
+        a string explaining why it cannot"""
+        return "Wrong source type"
+
+    def install(self, source):
+        """Try to download and unpack the source. Return the path to the
+        unpacked files or raise UnhandledSource."""
+        raise UnhandledSource("Wrong source type {}".format(source))
+
+    def parse_url(self, url):
+        return urlparse(url)
+
+    def base_url(self, url):
+        """Return url without querystring or fragment"""
+        parts = list(self.parse_url(url))
+        parts[4:] = ['' for i in parts[4:]]
+        return urlunparse(parts)
+
+
+def plugins(fetch_handlers=None):
+    if not fetch_handlers:
+        fetch_handlers = FETCH_HANDLERS
+    plugin_list = []
+    for handler_name in fetch_handlers:
+        package, classname = handler_name.rsplit('.', 1)
+        try:
+            handler_class = getattr(importlib.import_module(package), classname)
+            plugin_list.append(handler_class())
+        except (ImportError, AttributeError):
+            # Skip missing plugins so that they can be ommitted from
+            # installation if desired
+            log("FetchHandler {} not found, skipping plugin".format(handler_name))
+    return plugin_list

=== added file 'hooks/charmhelpers/fetch/archiveurl.py'
--- hooks/charmhelpers/fetch/archiveurl.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/archiveurl.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,48 @@
+import os
+import urllib2
+from charmhelpers.fetch import (
+    BaseFetchHandler,
+    UnhandledSource
+)
+from charmhelpers.payload.archive import (
+    get_archive_handler,
+    extract,
+)
+from charmhelpers.core.host import mkdir
+
+
+class ArchiveUrlFetchHandler(BaseFetchHandler):
+    """Handler for archives via generic URLs"""
+    def can_handle(self, source):
+        url_parts = self.parse_url(source)
+        if url_parts.scheme not in ('http', 'https', 'ftp', 'file'):
+            return "Wrong source type"
+        if get_archive_handler(self.base_url(source)):
+            return True
+        return False
+
+    def download(self, source, dest):
+        # propogate all exceptions
+        # URLError, OSError, etc
+        response = urllib2.urlopen(source)
+        try:
+            with open(dest, 'w') as dest_file:
+                dest_file.write(response.read())
+        except Exception as e:
+            if os.path.isfile(dest):
+                os.unlink(dest)
+            raise e
+
+    def install(self, source):
+        url_parts = self.parse_url(source)
+        dest_dir = os.path.join(os.environ.get('CHARM_DIR'), 'fetched')
+        if not os.path.exists(dest_dir):
+            mkdir(dest_dir, perms=0755)
+        dld_file = os.path.join(dest_dir, os.path.basename(url_parts.path))
+        try:
+            self.download(source, dld_file)
+        except urllib2.URLError as e:
+            raise UnhandledSource(e.reason)
+        except OSError as e:
+            raise UnhandledSource(e.strerror)
+        return extract(dld_file)

=== added file 'hooks/charmhelpers/fetch/bzrurl.py'
--- hooks/charmhelpers/fetch/bzrurl.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/fetch/bzrurl.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,44 @@
+import os
+from bzrlib.branch import Branch
+from charmhelpers.fetch import (
+    BaseFetchHandler,
+    UnhandledSource
+)
+from charmhelpers.core.host import mkdir
+
+
+class BzrUrlFetchHandler(BaseFetchHandler):
+    """Handler for bazaar branches via generic and lp URLs"""
+    def can_handle(self, source):
+        url_parts = self.parse_url(source)
+        if url_parts.scheme not in ('bzr+ssh', 'lp'):
+            return False
+        else:
+            return True
+
+    def branch(self, source, dest):
+        url_parts = self.parse_url(source)
+        # If we use lp:branchname scheme we need to load plugins
+        if not self.can_handle(source):
+            raise UnhandledSource("Cannot handle {}".format(source))
+        if url_parts.scheme == "lp":
+            from bzrlib.plugin import load_plugins
+            load_plugins()
+        try:
+            remote_branch = Branch.open(source)
+            remote_branch.bzrdir.sprout(dest).open_branch()
+        except Exception as e:
+            raise e
+
+    def install(self, source):
+        url_parts = self.parse_url(source)
+        branch_name = url_parts.path.strip("/").split("/")[-1]
+        dest_dir = os.path.join(os.environ.get('CHARM_DIR'), "fetched", branch_name)
+        if not os.path.exists(dest_dir):
+            mkdir(dest_dir, perms=0755)
+        try:
+            self.branch(source, dest_dir)
+        except OSError as e:
+            raise UnhandledSource(e.strerror)
+        return dest_dir
+

=== modified file 'hooks/hooks.py'
--- hooks/hooks.py	2013-01-11 17:00:40 +0000
+++ hooks/hooks.py	2013-10-29 21:07:21 +0000
@@ -1,17 +1,30 @@
 #!/usr/bin/env python
 
-import glob
-import grp
-import json
 import math
 import os
-import pwd
 import random
 import re
 import shutil
 import string
 import subprocess
 import sys
+import yaml
+import collections
+import socket
+
+from charmhelpers.core.hookenv import (
+    config as config_get,
+    log,
+    relation_set,
+    relation_get,
+    relation_ids as get_relation_ids,
+    relations_of_type,
+    related_units,
+    open_port,
+    close_port,
+)
+from charmhelpers.fetch import apt_install
+from charmhelpers.contrib.charmsupport.nrpe import NRPE
 
 ###############################################################################
 # Global variables
@@ -19,146 +32,112 @@
 default_squid3_config_dir = "/etc/squid3"
 default_squid3_config = "%s/squid.conf" % default_squid3_config_dir
 default_squid3_config_cache_dir = "/var/run/squid3"
-hook_name = os.path.basename(sys.argv[0])
+default_nagios_plugin_dir = "/usr/lib/nagios/plugins"
+Server = collections.namedtuple("Server", "name address port options")
+service_affecting_packages = ['squid3']
 
 ###############################################################################
 # Supporting functions
 ###############################################################################
 
 
-#------------------------------------------------------------------------------
-# config_get:  Returns a dictionary containing all of the config information
-#              Optional parameter: scope
-#              scope: limits the scope of the returned configuration to the
-#                     desired config item.
-#------------------------------------------------------------------------------
-def config_get(scope=None):
-    try:
-        config_cmd_line = ['config-get']
-        if scope is not None:
-            config_cmd_line.append(scope)
-        config_cmd_line.append('--format=json')
-        config_data = json.loads(subprocess.check_output(config_cmd_line))
-    except Exception, e:
-        subprocess.call(['juju-log', str(e)])
-        config_data = None
-    finally:
-        return(config_data)
-
-
-#------------------------------------------------------------------------------
-# relation_json:  Returns json-formatted relation data
-#                Optional parameters: scope, relation_id
-#                scope:        limits the scope of the returned data to the
-#                              desired item.
-#                unit_name:    limits the data ( and optionally the scope )
-#                              to the specified unit
-#                relation_id:  specify relation id for out of context usage.
-#------------------------------------------------------------------------------
-def relation_json(scope=None, unit_name=None, relation_id=None):
-    try:
-        relation_cmd_line = ['relation-get', '--format=json']
-        if relation_id is not None:
-            relation_cmd_line.extend(('-r', relation_id))
-        if scope is not None:
-            relation_cmd_line.append(scope)
-        else:
-            relation_cmd_line.append('-')
-        relation_cmd_line.append(unit_name)
-        relation_data = subprocess.check_output(relation_cmd_line)
-    except Exception, e:
-        subprocess.call(['juju-log', str(e)])
-        relation_data = None
-    finally:
-        return(relation_data)
-
-
-#------------------------------------------------------------------------------
-# relation_get:  Returns a dictionary containing the relation information
-#                Optional parameters: scope, relation_id
-#                scope:        limits the scope of the returned data to the
-#                              desired item.
-#                unit_name:    limits the data ( and optionally the scope )
-#                              to the specified unit
-#                relation_id:  specify relation id for out of context usage.
-#------------------------------------------------------------------------------
-def relation_get(scope=None, unit_name=None, relation_id=None):
-    try:
-        relation_data = json.loads(relation_json())
-    except Exception, e:
-        subprocess.call(['juju-log', str(e)])
-        relation_data = None
-    finally:
-        return(relation_data)
-
-
-#------------------------------------------------------------------------------
-# relation_ids:  Returns a list of relation ids
-#                optional parameters: relation_type
-#                relation_type: return relations only of this type
-#------------------------------------------------------------------------------
-def relation_ids(relation_types=['website']):
-    # accept strings or iterators
-    if isinstance(relation_types, basestring):
-        reltypes = [relation_types]
-    else:
-        reltypes = relation_types
-    relids = []
-    for reltype in reltypes:
-        relid_cmd_line = ['relation-ids', '--format=json', reltype]
-        relids.extend(json.loads(subprocess.check_output(relid_cmd_line)))
-    return relids
-
-
-#------------------------------------------------------------------------------
-# relation_get_all:  Returns a dictionary containing the relation information
-#                optional parameters: relation_type
-#                relation_type: limits the scope of the returned data to the
-#                               desired item.
-#------------------------------------------------------------------------------
-def relation_get_all():
-    reldata = {}
-    try:
-        relids = relation_ids()
-        for relid in relids:
-            units_cmd_line = ['relation-list', '--format=json', '-r', relid]
-            units = json.loads(subprocess.check_output(units_cmd_line))
-            for unit in units:
-                reldata[unit] = \
-                    json.loads(relation_json(
-                        relation_id=relid, unit_name=unit))
-                if 'sitenames' in reldata[unit]:
-                    reldata[unit]['sitenames'] = \
-                        reldata[unit]['sitenames'].split()
-                reldata[unit]['relation-id'] = relid
-                reldata[unit]['name'] = unit.replace("/", "_")
-    except Exception, e:
-        subprocess.call(['juju-log', str(e)])
-        reldata = []
-    finally:
-        return(reldata)
-
-
-#------------------------------------------------------------------------------
-# apt_get_install( packages ):  Installs a package
-#------------------------------------------------------------------------------
-def apt_get_install(packages=None):
-    if packages is None:
-        return(False)
-    cmd_line = ['apt-get', '-y', 'install', '-qq']
-    try:
-        cmd_line.extend(packages.split())
-    except AttributeError:
-        cmd_line.extend(packages)
-    return(subprocess.call(cmd_line))
-
-
-#------------------------------------------------------------------------------
-# enable_squid3:  Enable squid3 at boot time
-#------------------------------------------------------------------------------
-def enable_squid3():
-    # squid is enabled at boot time
-    return True
+def get_id(sitename, relation_id, unit_name):
+    unit_name = unit_name.replace('/', '_').replace('.', '_')
+    relation_id = relation_id.replace(':', '_').replace('.', '_')
+    if sitename is None:
+        return relation_id + '__' + unit_name
+    return sitename.replace('.', '_') + '__' + relation_id + '__' + unit_name
+
+
+def get_forward_sites():
+    reldata = []
+    for relid in get_relation_ids('cached-website'):
+        units = related_units(relid)
+        for unit in units:
+            data = relation_get(rid=relid, unit=unit)
+            if 'sitenames' in data:
+                data['sitenames'] = data['sitenames'].split()
+            data['relation-id'] = relid
+            data['name'] = unit.replace("/", "_")
+            reldata.append(data)
+    return reldata
+
+
+def get_reverse_sites():
+    all_sites = {}
+
+    config_data = config_get()
+    config_services = yaml.safe_load(config_data.get("services", "")) or ()
+    server_options_by_site = {}
+
+    for service_item in config_services:
+        site = service_item["service_name"]
+        servers = all_sites.setdefault(site, [])
+        options = " ".join(service_item.get("server_options", []))
+        server_options_by_site[site] = options
+        for host, port in service_item["servers"]:
+            servers.append(
+                Server(name=get_id(None, site, host),
+                       address=host, port=port,
+                       options=options))
+
+    relations = relations_of_type("website")
+
+    for relation_data in relations:
+        unit = relation_data["__unit__"]
+        relation_id = relation_data["__relid__"]
+        if not ("port" in relation_data or "all_services" in relation_data):
+            log("No port in relation data for '%s', skipping." % unit)
+            continue
+
+        if not "private-address" in relation_data:
+            log("No private-address in relation data "
+                "for '%s', skipping." % unit)
+            continue
+
+        for site in relation_data.get("sitenames", "").split():
+            servers = all_sites.setdefault(site, [])
+            servers.append(
+                Server(name=get_id(site, relation_id, unit),
+                       address=relation_data["private-address"],
+                       port=relation_data["port"],
+                       options=server_options_by_site.get(site, '')))
+
+        services = yaml.safe_load(relation_data.get("all_services", "")) or ()
+        for service_item in services:
+            site = service_item["service_name"]
+            servers = all_sites.setdefault(site, [])
+            servers.append(
+                Server(name=get_id(site, relation_id, unit),
+                       address=relation_data["private-address"],
+                       port=service_item["service_port"],
+                       options=server_options_by_site.get(site, '')))
+
+        if not ("sitenames" in relation_data or
+                "all_services" in relation_data):
+            servers = all_sites.setdefault(None, [])
+            servers.append(
+                Server(name=get_id(None, relation_id, unit),
+                       address=relation_data["private-address"],
+                       port=relation_data["port"],
+                       options=server_options_by_site.get(None, '')))
+
+    if len(all_sites) == 0:
+        return
+
+    for site, servers in all_sites.iteritems():
+        servers.sort()
+
+    return all_sites
+
+
+def ensure_package_status(packages, status):
+    if status in ['install', 'hold']:
+        selections = ''.join(['{} {}\n'.format(package, status)
+                              for package in packages])
+        dpkg = subprocess.Popen(['dpkg', '--set-selections'],
+                                stdin=subprocess.PIPE)
+        dpkg.communicate(input=selections)
 
 
 #------------------------------------------------------------------------------
@@ -169,9 +148,8 @@
 #------------------------------------------------------------------------------
 def load_squid3_config(squid3_config_file="/etc/squid3/squid.conf"):
     if os.path.isfile(squid3_config_file):
-        return(open(squid3_config_file).read())
-    else:
-        return(None)
+        return open(squid3_config_file).read()
+    return
 
 
 #------------------------------------------------------------------------------
@@ -183,8 +161,8 @@
 def get_service_ports(squid3_config_file="/etc/squid3/squid.conf"):
     squid3_config = load_squid3_config(squid3_config_file)
     if squid3_config is None:
-        return(None)
-    return(re.findall("http_port ([0-9]+)", squid3_config))
+        return
+    return re.findall("http_port ([0-9]+)", squid3_config)
 
 
 #------------------------------------------------------------------------------
@@ -195,31 +173,9 @@
 def get_sitenames(squid3_config_file="/etc/squid3/squid.conf"):
     squid3_config = load_squid3_config(squid3_config_file)
     if squid3_config is None:
-        return(None)
-    sitenames = re.findall("cache_peer_domain \w+ ([^!].*)", squid3_config)
-    return(list(set(sitenames)))
-
-
-#------------------------------------------------------------------------------
-# open_port:  Convenience function to open a port in juju to
-#             expose a service
-#------------------------------------------------------------------------------
-def open_port(port=None, protocol="TCP"):
-    if port is None:
-        return(None)
-    return(subprocess.call(['/usr/bin/open-port', "%d/%s" %
-                            (int(port), protocol)]))
-
-
-#------------------------------------------------------------------------------
-# close_port:  Convenience function to close a port in juju to
-#              unexpose a service
-#------------------------------------------------------------------------------
-def close_port(port=None, protocol="TCP"):
-    if port is None:
-        return(None)
-    return(subprocess.call(['/usr/bin/close-port', "%d/%s" %
-                            (int(port), protocol)]))
+        return
+    sitenames = re.findall("acl [\w_-]+ dstdomain ([^!].*)", squid3_config)
+    return list(set(sitenames))
 
 
 #------------------------------------------------------------------------------
@@ -229,7 +185,7 @@
 #------------------------------------------------------------------------------
 def update_service_ports(old_service_ports=None, new_service_ports=None):
     if old_service_ports is None or new_service_ports is None:
-        return(None)
+        return
     for port in old_service_ports:
         if port not in new_service_ports:
             close_port(port)
@@ -248,7 +204,7 @@
                           if l not in 'Iil0oO1']
     random_chars = [random.choice(alphanumeric_chars)
                     for i in range(pwd_length)]
-    return(''.join(random_chars))
+    return ''.join(random_chars)
 
 
 #------------------------------------------------------------------------------
@@ -257,27 +213,53 @@
 def construct_squid3_config():
     from jinja2 import Environment, FileSystemLoader
     config_data = config_get()
-    relations = relation_get_all()
+    reverse_sites = get_reverse_sites()
+    only_direct = set()
+    if reverse_sites is not None:
+        for site, servers in reverse_sites.iteritems():
+            if not servers:
+                only_direct.add(site)
+
     if config_data['refresh_patterns']:
-        refresh_patterns = json.loads(config_data['refresh_patterns'])
+        refresh_patterns = yaml.safe_load(config_data['refresh_patterns'])
     else:
         refresh_patterns = {}
-    template_env = Environment(loader=FileSystemLoader(
-        os.path.join(os.environ['CHARM_DIR'], 'templates')))
+    # Use default refresh pattern if specified.
+    if '.' in refresh_patterns:
+        default_refresh_pattern = refresh_patterns.pop('.')
+    else:
+        default_refresh_pattern = {
+            'min': 30,
+            'percent': 20,
+            'max': 4320,
+            'options': [],
+            }
+
+    templates_dir = os.path.join(os.environ['CHARM_DIR'], 'templates')
+    template_env = Environment(loader=FileSystemLoader(templates_dir))
+
     config_data['cache_l1'] = int(math.ceil(math.sqrt(
-        int(config_data['cache_size_mb']) * 1024 / (16 *
-        int(config_data['target_objs_per_dir']) * int(
-            config_data['avg_obj_size_kb'])))))
+        int(config_data['cache_size_mb']) * 1024 / (
+            16 * int(config_data['target_objs_per_dir']) * int(
+                config_data['avg_obj_size_kb'])))))
     config_data['cache_l2'] = config_data['cache_l1'] * 16
     templ_vars = {
         'config': config_data,
-        'relations': relations,
+        'sites': reverse_sites,
+        'forward_relations': get_forward_sites(),
+        'only_direct': only_direct,
         'refresh_patterns': refresh_patterns,
+        'default_refresh_pattern': default_refresh_pattern,
     }
     template = template_env.get_template('main_config.template').\
         render(templ_vars)
+    write_squid3_config('\n'.join(
+        (l.strip() for l in str(template).splitlines())))
+
+
+def write_squid3_config(contents):
     with open(default_squid3_config, 'w') as squid3_config:
-        squid3_config.write(str(template))
+        squid3_config.write(contents)
 
 
 #------------------------------------------------------------------------------
@@ -286,147 +268,192 @@
 #------------------------------------------------------------------------------
 def service_squid3(action=None, squid3_config=default_squid3_config):
     if action is None or squid3_config is None:
-        return(None)
+        return
     elif action == "check":
         check_cmd = ['/usr/sbin/squid3', '-f', squid3_config, '-k', 'parse']
         retVal = subprocess.call(check_cmd)
         if retVal == 1:
-            return(False)
+            return False
         elif retVal == 0:
-            return(True)
+            return True
         else:
-            return(False)
+            return False
     elif action == 'status':
         status = subprocess.check_output(['status', 'squid3'])
         if re.search('running', status) is not None:
-            return(True)
+            return True
         else:
-            return(False)
+            return False
     elif action in ('start', 'stop', 'reload', 'restart'):
         retVal = subprocess.call([action, 'squid3'])
         if retVal == 0:
-            return(True)
+            return True
         else:
-            return(False)
+            return False
+
+
+def install_nrpe_scripts():
+    if not os.path.exists(default_nagios_plugin_dir):
+        os.makedirs(default_nagios_plugin_dir)
+    shutil.copy2('%s/files/check_squidpeers' % (
+        os.environ['CHARM_DIR']),
+        '{}/check_squidpeers'.format(default_nagios_plugin_dir))
 
 
 def update_nrpe_checks():
-    config_data = config_get()
-    try:
-        nagios_uid = pwd.getpwnam('nagios').pw_uid
-        nagios_gid = grp.getgrnam('nagios').gr_gid
-    except:
-        subprocess.call(['juju-log', "Nagios user not setup, exiting"])
-        return
-
-    unit_name = os.environ['JUJU_UNIT_NAME'].replace('/', '-')
-    nrpe_check_file = '/etc/nagios/nrpe.d/check_squidrp.cfg'
-    nagios_hostname = "%s-%s" % (config_data['nagios_context'], unit_name)
-    nagios_logdir = '/var/log/nagios'
-    nagios_exportdir = '/var/lib/nagios/export'
-    nrpe_service_file = \
-        '/var/lib/nagios/export/service__%s_check_squidrp.cfg' % \
-            (nagios_hostname)
-    if not os.path.exists(nagios_logdir):
-        os.mkdir(nagios_logdir)
-        os.chown(nagios_logdir, nagios_uid, nagios_gid)
-    if not os.path.exists(nagios_exportdir):
-        subprocess.call(['juju-log', 'Exiting as %s is not accessible' %
-            (nagios_exportdir)])
-        return
-    for f in os.listdir(nagios_exportdir):
-        if re.search('.*check_squidrp.cfg', f):
-            os.remove(os.path.join(nagios_exportdir, f))
-    from jinja2 import Environment, FileSystemLoader
-    template_env = Environment(
-        loader=FileSystemLoader(os.path.join(os.environ['CHARM_DIR'], 'templates')))
-    templ_vars = {
-        'nagios_hostname': nagios_hostname,
-        'nagios_servicegroup': config_data['nagios_context'],
-    }
-    template = template_env.get_template('nrpe_service.template').\
-        render(templ_vars)
-    with open(nrpe_service_file, 'w') as nrpe_service_config:
-        nrpe_service_config.write(str(template))
-    with open(nrpe_check_file, 'w') as nrpe_check_config:
-        nrpe_check_config.write("# check squid\n")
-        nrpe_check_config.write(
-            "command[check_squidrp]=/usr/lib/nagios/plugins/check_http %s\n" %
-                (config_data['nagios_check_http_params']))
-    if os.path.isfile('/etc/init.d/nagios-nrpe-server'):
-        subprocess.call(['service', 'nagios-nrpe-server', 'reload'])
+    install_nrpe_scripts()
+    nrpe_compat = NRPE()
+    conf = nrpe_compat.config
+    nrpe_compat.add_check(
+        shortname='squidpeers',
+        description='Check Squid Peers',
+        check_cmd='check_squidpeers'
+    )
+    check_http_params = conf.get('nagios_check_http_params')
+    if check_http_params:
+        nrpe_compat.add_check(
+            shortname='squidrp',
+            description='Check Squid',
+            check_cmd='check_http %s' % check_http_params
+        )
+    config_services_str = conf.get('services', '')
+    config_services = yaml.safe_load(config_services_str) or ()
+    for service in config_services:
+        path = service.get('nrpe_check_path')
+        if path is not None:
+            command = 'check_http -I 127.0.0.1 -p 3128 --method=HEAD '
+            service_name = service['service_name']
+            if conf.get('x_balancer_name_allowed'):
+                command += ("-u http://localhost%s "
+                            "-k \\'X-Balancer-Name: %s\\'" % (
+                                path, service_name))
+            else:
+                command += "-u http://%s%s" % (service_name, path)
+            nrpe_compat.add_check(
+                shortname='squid-%s' % service_name.replace(".", "_"),
+                description='Check Squid for site %s' % service_name,
+                check_cmd=command,
+                )
+    nrpe_compat.write()
+
+
+def install_packages():
+    apt_install("squid3 squidclient python-jinja2".split(), fatal=True)
+    ensure_package_status(service_affecting_packages,
+                          config_get('package_status'))
 
 
 ###############################################################################
 # Hook functions
 ###############################################################################
 def install_hook():
-    for f in glob.glob('exec.d/*/charm-pre-install'):
-        if os.path.isfile(f) and os.access(f, os.X_OK):
-            subprocess.check_call(['sh', '-c', f])
     if not os.path.exists(default_squid3_config_dir):
         os.mkdir(default_squid3_config_dir, 0600)
     if not os.path.exists(default_squid3_config_cache_dir):
         os.mkdir(default_squid3_config_cache_dir, 0600)
-    shutil.copy2('%s/files/default.squid3' % (os.environ['CHARM_DIR']), '/etc/default/squid3')
-    return (apt_get_install(
-        "squid3 python-jinja2") == enable_squid3() is not True)
+    shutil.copy2('%s/files/default.squid3' % (
+        os.environ['CHARM_DIR']), '/etc/default/squid3')
+    install_packages()
+    return True
 
 
 def config_changed():
-    current_service_ports = get_service_ports()
+    old_service_ports = get_service_ports()
+    old_sitenames = get_sitenames()
     construct_squid3_config()
     update_nrpe_checks()
+    ensure_package_status(service_affecting_packages,
+                          config_get('package_status'))
 
     if service_squid3("check"):
         updated_service_ports = get_service_ports()
-        update_service_ports(current_service_ports, updated_service_ports)
+        update_service_ports(old_service_ports, updated_service_ports)
         service_squid3("reload")
+        if not (old_sitenames == get_sitenames()):
+            notify_cached_website()
     else:
+        # XXX Ideally the config should be restored to a working state if the
+        # check fails, otherwise an inadvertent reload will cause the service
+        # to be broken.
+        log("squid configuration check failed, exiting")
         sys.exit(1)
 
 
 def start_hook():
     if service_squid3("status"):
-        return(service_squid3("restart"))
+        return service_squid3("restart")
     else:
-        return(service_squid3("start"))
+        return service_squid3("start")
 
 
 def stop_hook():
     if service_squid3("status"):
-        return(service_squid3("stop"))
+        return service_squid3("stop")
 
 
 def website_interface(hook_name=None):
     if hook_name is None:
-        return(None)
-    if hook_name in ["joined", "changed", "broken", "departed"]:
+        return
+    if hook_name in ("joined", "changed", "broken", "departed"):
         config_changed()
 
 
 def cached_website_interface(hook_name=None):
     if hook_name is None:
-        return(None)
-    if hook_name in ["joined", "changed"]:
-        sitenames = ' '.join(get_sitenames())
-        # passing only one port - the first one defined
-        subprocess.call(['relation-set', 'port=%s' % get_service_ports()[0],
-                         'sitenames=%s' % sitenames])
+        return
+    if hook_name in ("joined", "changed"):
+        notify_cached_website(relation_ids=(None,))
+    config_data = config_get()
+    if config_data['enable_forward_proxy']:
+        config_changed()
+
+
+def get_hostname(host=None):
+    my_host = socket.gethostname()
+    if host is None or host == "0.0.0.0":
+        # If the listen ip has been set to 0.0.0.0 then pass back the hostname
+        return socket.getfqdn(my_host)
+    elif host == "localhost":
+        # If the fqdn lookup has returned localhost (lxc setups) then return
+        # hostname
+        return my_host
+    return host
+
+
+def notify_cached_website(relation_ids=None):
+    hostname = get_hostname()
+    # passing only one port - the first one defined
+    port = get_service_ports()[0]
+    sitenames = ' '.join(get_sitenames())
+
+    for rid in relation_ids or get_relation_ids("cached-website"):
+        relation_set(relation_id=rid, port=port,
+                     hostname=hostname,
+                     sitenames=sitenames)
+
+
+def upgrade_charm():
+    # Ensure that all current dependencies are installed.
+    install_packages()
 
 
 ###############################################################################
 # Main section
 ###############################################################################
-def main():
+def main(hook_name):
     if hook_name == "install":
         install_hook()
     elif hook_name == "config-changed":
         config_changed()
+        update_nrpe_checks()
     elif hook_name == "start":
         start_hook()
     elif hook_name == "stop":
         stop_hook()
+    elif hook_name == "upgrade-charm":
+        upgrade_charm()
+        config_changed()
+        update_nrpe_checks()
 
     elif hook_name == "website-relation-joined":
         website_interface("joined")
@@ -446,14 +473,21 @@
     elif hook_name == "cached-website-relation-departed":
         cached_website_interface("departed")
 
-    elif hook_name == "nrpe-external-master-relation-changed":
+    elif hook_name in ("nrpe-external-master-relation-joined",
+                       "local-monitors-relation-joined"):
         update_nrpe_checks()
 
     elif hook_name == "env-dump":
-        print relation_get_all()
+        print relations_of_type()
     else:
         print "Unknown hook"
         sys.exit(1)
 
 if __name__ == '__main__':
-    main()
+    hook_name = os.path.basename(sys.argv[0])
+    # Also support being invoked directly with hook as argument name.
+    if hook_name == "hooks.py":
+        if len(sys.argv) < 2:
+            sys.exit("Missing required hook name argument.")
+        hook_name = sys.argv[1]
+    main(hook_name)

=== modified symlink 'hooks/install' (properties changed: -x to +x)
=== target was u'hooks.py'
--- hooks/install	1970-01-01 00:00:00 +0000
+++ hooks/install	2013-10-29 21:07:21 +0000
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+set -eu
+
+juju-log 'Invoking charm-pre-install hooks'
+[ -d exec.d ] && ( for f in exec.d/*/charm-pre-install; do [ -x $f ] && /bin/sh -c "$f"; done )
+
+juju-log 'Invoking python-based install hook'
+python hooks/hooks.py install

=== added symlink 'hooks/local-monitors-relation-joined'
=== target is u'hooks.py'
=== renamed symlink 'hooks/nrpe-external-master-relation-changed' => 'hooks/nrpe-external-master-relation-joined'
=== removed directory 'hooks/shelltoolbox'
=== removed file 'hooks/shelltoolbox/__init__.py'
--- hooks/shelltoolbox/__init__.py	2012-10-02 09:42:19 +0000
+++ hooks/shelltoolbox/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,662 +0,0 @@
-# Copyright 2012 Canonical Ltd.
-
-# This file is part of python-shell-toolbox.
-#
-# python-shell-toolbox is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by the
-# Free Software Foundation, version 3 of the License.
-#
-# python-shell-toolbox is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
-# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
-# more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with python-shell-toolbox. If not, see <http://www.gnu.org/licenses/>.
-
-"""Helper functions for accessing shell commands in Python."""
-
-__metaclass__ = type
-__all__ = [
-    'apt_get_install',
-    'bzr_whois',
-    'cd',
-    'command',
-    'DictDiffer',
-    'environ',
-    'file_append',
-    'file_prepend',
-    'generate_ssh_keys',
-    'get_su_command',
-    'get_user_home',
-    'get_user_ids',
-    'install_extra_repositories',
-    'join_command',
-    'mkdirs',
-    'run',
-    'Serializer',
-    'script_name',
-    'search_file',
-    'ssh',
-    'su',
-    'user_exists',
-    'wait_for_page_contents',
-    ]
-
-from collections import namedtuple
-from contextlib import contextmanager
-from email.Utils import parseaddr
-import errno
-import json
-import operator
-import os
-import pipes
-import pwd
-import re
-import subprocess
-import sys
-from textwrap import dedent
-import time
-import urllib2
-
-
-Env = namedtuple('Env', 'uid gid home')
-
-
-def apt_get_install(*args, **kwargs):
-    """Install given packages using apt.
-
-    It is possible to pass environment variables to be set during install
-    using keyword arguments.
-
-    :raises: subprocess.CalledProcessError
-    """
-    caller = kwargs.pop('caller', run)
-    debian_frontend = kwargs.pop('DEBIAN_FRONTEND', 'noninteractive')
-    with environ(DEBIAN_FRONTEND=debian_frontend, **kwargs):
-        cmd = ('apt-get', '-y', 'install') + args
-        return caller(*cmd)
-
-
-def bzr_whois(user):
-    """Return full name and email of bzr `user`.
-
-    Return None if the given `user` does not have a bzr user id.
-    """
-    with su(user):
-        try:
-            whoami = run('bzr', 'whoami')
-        except (subprocess.CalledProcessError, OSError):
-            return None
-    return parseaddr(whoami)
-
-
-@contextmanager
-def cd(directory):
-    """A context manager to temporarily change current working dir, e.g.::
-
-        >>> import os
-        >>> os.chdir('/tmp')
-        >>> with cd('/bin'): print os.getcwd()
-        /bin
-        >>> print os.getcwd()
-        /tmp
-    """
-    cwd = os.getcwd()
-    os.chdir(directory)
-    try:
-        yield
-    finally:
-        os.chdir(cwd)
-
-
-def command(*base_args):
-    """Return a callable that will run the given command with any arguments.
-
-    The first argument is the path to the command to run, subsequent arguments
-    are command-line arguments to "bake into" the returned callable.
-
-    The callable runs the given executable and also takes arguments that will
-    be appeneded to the "baked in" arguments.
-
-    For example, this code will list a file named "foo" (if it exists):
-
-        ls_foo = command('/bin/ls', 'foo')
-        ls_foo()
-
-    While this invocation will list "foo" and "bar" (assuming they exist):
-
-        ls_foo('bar')
-    """
-    def callable_command(*args):
-        all_args = base_args + args
-        return run(*all_args)
-
-    return callable_command
-
-
-@contextmanager
-def environ(**kwargs):
-    """A context manager to temporarily change environment variables.
-
-    If an existing environment variable is changed, it is restored during
-    context cleanup::
-
-        >>> import os
-        >>> os.environ['MY_VARIABLE'] = 'foo'
-        >>> with environ(MY_VARIABLE='bar'): print os.getenv('MY_VARIABLE')
-        bar
-        >>> print os.getenv('MY_VARIABLE')
-        foo
-        >>> del os.environ['MY_VARIABLE']
-
-    If we are adding environment variables, they are removed during context
-    cleanup::
-
-        >>> import os
-        >>> with environ(MY_VAR1='foo', MY_VAR2='bar'):
-        ...     print os.getenv('MY_VAR1'), os.getenv('MY_VAR2')
-        foo bar
-        >>> os.getenv('MY_VAR1') == os.getenv('MY_VAR2') == None
-        True
-    """
-    backup = {}
-    for key, value in kwargs.items():
-        backup[key] = os.getenv(key)
-        os.environ[key] = value
-    try:
-        yield
-    finally:
-        for key, value in backup.items():
-            if value is None:
-                del os.environ[key]
-            else:
-                os.environ[key] = value
-
-
-def file_append(filename, line):
-    r"""Append given `line`, if not present, at the end of `filename`.
-
-    Usage example::
-
-        >>> import tempfile
-        >>> f = tempfile.NamedTemporaryFile('w', delete=False)
-        >>> f.write('line1\n')
-        >>> f.close()
-        >>> file_append(f.name, 'new line\n')
-        >>> open(f.name).read()
-        'line1\nnew line\n'
-
-    Nothing happens if the file already contains the given `line`::
-
-        >>> file_append(f.name, 'new line\n')
-        >>> open(f.name).read()
-        'line1\nnew line\n'
-
-    A new line is automatically added before the given `line` if it is not
-    present at the end of current file content::
-
-        >>> import tempfile
-        >>> f = tempfile.NamedTemporaryFile('w', delete=False)
-        >>> f.write('line1')
-        >>> f.close()
-        >>> file_append(f.name, 'new line\n')
-        >>> open(f.name).read()
-        'line1\nnew line\n'
-
-    The file is created if it does not exist::
-
-        >>> import tempfile
-        >>> filename = tempfile.mktemp()
-        >>> file_append(filename, 'line1\n')
-        >>> open(filename).read()
-        'line1\n'
-    """
-    if not line.endswith('\n'):
-        line += '\n'
-    with open(filename, 'a+') as f:
-        lines = f.readlines()
-        if line not in lines:
-            if not lines or lines[-1].endswith('\n'):
-                f.write(line)
-            else:
-                f.write('\n' + line)
-
-
-def file_prepend(filename, line):
-    r"""Insert given `line`, if not present, at the beginning of `filename`.
-
-    Usage example::
-
-        >>> import tempfile
-        >>> f = tempfile.NamedTemporaryFile('w', delete=False)
-        >>> f.write('line1\n')
-        >>> f.close()
-        >>> file_prepend(f.name, 'line0\n')
-        >>> open(f.name).read()
-        'line0\nline1\n'
-
-    If the file starts with the given `line`, nothing happens::
-
-        >>> file_prepend(f.name, 'line0\n')
-        >>> open(f.name).read()
-        'line0\nline1\n'
-
-    If the file contains the given `line`, but not at the beginning,
-    the line is moved on top::
-
-        >>> file_prepend(f.name, 'line1\n')
-        >>> open(f.name).read()
-        'line1\nline0\n'
-    """
-    if not line.endswith('\n'):
-        line += '\n'
-    with open(filename, 'r+') as f:
-        lines = f.readlines()
-        if lines[0] != line:
-            try:
-                lines.remove(line)
-            except ValueError:
-                pass
-            lines.insert(0, line)
-            f.seek(0)
-            f.writelines(lines)
-
-
-def generate_ssh_keys(path, passphrase=''):
-    """Generate ssh key pair, saving them inside the given `directory`.
-
-        >>> generate_ssh_keys('/tmp/id_rsa')
-        0
-        >>> open('/tmp/id_rsa').readlines()[0].strip()
-        '-----BEGIN RSA PRIVATE KEY-----'
-        >>> open('/tmp/id_rsa.pub').read().startswith('ssh-rsa')
-        True
-        >>> os.remove('/tmp/id_rsa')
-        >>> os.remove('/tmp/id_rsa.pub')
-
-    If either of the key files already exist, generate_ssh_keys() will
-    raise an Exception.
-
-    Note that ssh-keygen will prompt if the keyfiles already exist, but
-    when we're using it non-interactively it's better to pre-empt that
-    behaviour.
-
-        >>> with open('/tmp/id_rsa', 'w') as key_file:
-        ...    key_file.write("Don't overwrite me, bro!")
-        >>> generate_ssh_keys('/tmp/id_rsa') # doctest: +ELLIPSIS
-        Traceback (most recent call last):
-        Exception: File /tmp/id_rsa already exists...
-        >>> os.remove('/tmp/id_rsa')
-
-        >>> with open('/tmp/id_rsa.pub', 'w') as key_file:
-        ...    key_file.write("Don't overwrite me, bro!")
-        >>> generate_ssh_keys('/tmp/id_rsa') # doctest: +ELLIPSIS
-        Traceback (most recent call last):
-        Exception: File /tmp/id_rsa.pub already exists...
-        >>> os.remove('/tmp/id_rsa.pub')
-    """
-    if os.path.exists(path):
-        raise Exception("File {} already exists.".format(path))
-    if os.path.exists(path + '.pub'):
-        raise Exception("File {}.pub already exists.".format(path))
-    return subprocess.call([
-        'ssh-keygen', '-q', '-t', 'rsa', '-N', passphrase, '-f', path])
-
-
-def get_su_command(user, args):
-    """Return a command line as a sequence, prepending "su" if necessary.
-
-    This can be used together with `run` when the `su` context manager is not
-    enough (e.g. an external program uses uid rather than euid).
-
-        run(*get_su_command(user, ['bzr', 'whoami']))
-
-    If the su is requested as current user, the arguments are returned as
-    given::
-
-        >>> import getpass
-        >>> current_user = getpass.getuser()
-
-        >>> get_su_command(current_user, ('ls', '-l'))
-        ('ls', '-l')
-
-    Otherwise, "su" is prepended::
-
-        >>> get_su_command('nobody', ('ls', '-l', 'my file'))
-        ('su', 'nobody', '-c', "ls -l 'my file'")
-    """
-    if get_user_ids(user)[0] != os.getuid():
-        args = [i for i in args if i is not None]
-        return ('su', user, '-c', join_command(args))
-    return args
-
-
-def get_user_home(user):
-    """Return the home directory of the given `user`.
-
-        >>> get_user_home('root')
-        '/root'
-
-    If the user does not exist, return a default /home/[username] home::
-
-        >>> get_user_home('_this_user_does_not_exist_')
-        '/home/_this_user_does_not_exist_'
-    """
-    try:
-        return pwd.getpwnam(user).pw_dir
-    except KeyError:
-        return os.path.join(os.path.sep, 'home', user)
-
-
-def get_user_ids(user):
-    """Return the uid and gid of given `user`, e.g.::
-
-        >>> get_user_ids('root')
-        (0, 0)
-    """
-    userdata = pwd.getpwnam(user)
-    return userdata.pw_uid, userdata.pw_gid
-
-
-def install_extra_repositories(*repositories):
-    """Install all of the extra repositories and update apt.
-
-    Given repositories can contain a "{distribution}" placeholder, that will
-    be replaced by current distribution codename.
-
-    :raises: subprocess.CalledProcessError
-    """
-    distribution = run('lsb_release', '-cs').strip()
-    # Starting from Oneiric, `apt-add-repository` is interactive by
-    # default, and requires a "-y" flag to be set.
-    assume_yes = None if distribution == 'lucid' else '-y'
-    for repo in repositories:
-        repository = repo.format(distribution=distribution)
-        run('apt-add-repository', assume_yes, repository)
-    run('apt-get', 'clean')
-    run('apt-get', 'update')
-
-
-def join_command(args):
-    """Return a valid Unix command line from `args`.
-
-        >>> join_command(['ls', '-l'])
-        'ls -l'
-
-    Arguments containing spaces and empty args are correctly quoted::
-
-        >>> join_command(['command', 'arg1', 'arg containing spaces', ''])
-        "command arg1 'arg containing spaces' ''"
-    """
-    return ' '.join(pipes.quote(arg) for arg in args)
-
-
-def mkdirs(*args):
-    """Create leaf directories (given as `args`) and all intermediate ones.
-
-        >>> import tempfile
-        >>> base_dir = tempfile.mktemp(suffix='/')
-        >>> dir1 = tempfile.mktemp(prefix=base_dir)
-        >>> dir2 = tempfile.mktemp(prefix=base_dir)
-        >>> mkdirs(dir1, dir2)
-        >>> os.path.isdir(dir1)
-        True
-        >>> os.path.isdir(dir2)
-        True
-
-    If the leaf directory already exists the function returns without errors::
-
-        >>> mkdirs(dir1)
-
-    An `OSError` is raised if the leaf path exists and it is a file::
-
-        >>> f = tempfile.NamedTemporaryFile(
-        ...     'w', delete=False, prefix=base_dir)
-        >>> f.close()
-        >>> mkdirs(f.name) # doctest: +ELLIPSIS
-        Traceback (most recent call last):
-        OSError: ...
-    """
-    for directory in args:
-        try:
-            os.makedirs(directory)
-        except OSError as err:
-            if err.errno != errno.EEXIST or os.path.isfile(directory):
-                raise
-
-
-def run(*args, **kwargs):
-    """Run the command with the given arguments.
-
-    The first argument is the path to the command to run.
-    Subsequent arguments are command-line arguments to be passed.
-
-    This function accepts all optional keyword arguments accepted by
-    `subprocess.Popen`.
-    """
-    args = [i for i in args if i is not None]
-    pipe = subprocess.PIPE
-    process = subprocess.Popen(
-        args, stdout=kwargs.pop('stdout', pipe),
-        stderr=kwargs.pop('stderr', pipe),
-        close_fds=kwargs.pop('close_fds', True), **kwargs)
-    stdout, stderr = process.communicate()
-    if process.returncode:
-        exception = subprocess.CalledProcessError(
-            process.returncode, repr(args))
-        # The output argument of `CalledProcessError` was introduced in Python
-        # 2.7. Monkey patch the output here to avoid TypeErrors in older
-        # versions of Python, still preserving the output in Python 2.7.
-        exception.output = ''.join(filter(None, [stdout, stderr]))
-        raise exception
-    return stdout
-
-
-def script_name():
-    """Return the name of this script."""
-    return os.path.basename(sys.argv[0])
-
-
-def search_file(regexp, filename):
-    """Return the first line in `filename` that matches `regexp`."""
-    with open(filename) as f:
-        for line in f:
-            if re.search(regexp, line):
-                return line
-
-
-def ssh(location, user=None, key=None, caller=subprocess.call):
-    """Return a callable that can be used to run ssh shell commands.
-
-    The ssh `location` and, optionally, `user` must be given.
-    If the user is None then the current user is used for the connection.
-
-    The callable internally uses the given `caller`::
-
-        >>> def caller(cmd):
-        ...     print tuple(cmd)
-        >>> sshcall = ssh('example.com', 'myuser', caller=caller)
-        >>> root_sshcall = ssh('example.com', caller=caller)
-        >>> sshcall('ls -l') # doctest: +ELLIPSIS
-        ('ssh', '-t', ..., '[e-mail]', '--', 'ls -l')
-        >>> root_sshcall('ls -l') # doctest: +ELLIPSIS
-        ('ssh', '-t', ..., 'example.com', '--', 'ls -l')
-
-    The ssh key path can be optionally provided::
-
-        >>> root_sshcall = ssh('example.com', key='/tmp/foo', caller=caller)
-        >>> root_sshcall('ls -l') # doctest: +ELLIPSIS
-        ('ssh', '-t', ..., '-i', '/tmp/foo', 'example.com', '--', 'ls -l')
-
-    If the ssh command exits with an error code,
-    a `subprocess.CalledProcessError` is raised::
-
-        >>> ssh('loc', caller=lambda cmd: 1)('ls -l') # doctest: +ELLIPSIS
-        Traceback (most recent call last):
-        CalledProcessError: ...
-
-    If ignore_errors is set to True when executing the command, no error
-    will be raised, even if the command itself returns an error code.
-
-        >>> sshcall = ssh('loc', caller=lambda cmd: 1)
-        >>> sshcall('ls -l', ignore_errors=True)
-    """
-    sshcmd = [
-        'ssh',
-        '-t',
-        '-t',  # Yes, this second -t is deliberate. See `man ssh`.
-        '-o', 'StrictHostKeyChecking=no',
-        '-o', 'UserKnownHostsFile=/dev/null',
-        ]
-    if key is not None:
-        sshcmd.extend(['-i', key])
-    if user is not None:
-        location = '{}@{}'.format(user, location)
-    sshcmd.extend([location, '--'])
-
-    def _sshcall(cmd, ignore_errors=False):
-        command = sshcmd + [cmd]
-        retcode = caller(command)
-        if retcode and not ignore_errors:
-            raise subprocess.CalledProcessError(retcode, ' '.join(command))
-
-    return _sshcall
-
-
-@contextmanager
-def su(user):
-    """A context manager to temporarily run the script as a different user."""
-    uid, gid = get_user_ids(user)
-    os.setegid(gid)
-    os.seteuid(uid)
-    home = get_user_home(user)
-    with environ(HOME=home):
-        try:
-            yield Env(uid, gid, home)
-        finally:
-            os.setegid(os.getgid())
-            os.seteuid(os.getuid())
-
-
-def user_exists(username):
-    """Return True if given `username` exists, e.g.::
-
-        >>> user_exists('root')
-        True
-        >>> user_exists('_this_user_does_not_exist_')
-        False
-    """
-    try:
-        pwd.getpwnam(username)
-    except KeyError:
-        return False
-    return True
-
-
-def wait_for_page_contents(url, contents, timeout=120, validate=None):
-    if validate is None:
-        validate = operator.contains
-    start_time = time.time()
-    while True:
-        try:
-            stream = urllib2.urlopen(url)
-        except (urllib2.HTTPError, urllib2.URLError):
-            pass
-        else:
-            page = stream.read()
-            if validate(page, contents):
-                return page
-        if time.time() - start_time >= timeout:
-            raise RuntimeError('timeout waiting for contents of ' + url)
-        time.sleep(0.1)
-
-
-class DictDiffer:
-    """
-    Calculate the difference between two dictionaries as:
-    (1) items added
-    (2) items removed
-    (3) keys same in both but changed values
-    (4) keys same in both and unchanged values
-    """
-
-    # Based on answer by hughdbrown at:
-    # http://stackoverflow.com/questions/1165352
-
-    def __init__(self, current_dict, past_dict):
-        self.current_dict = current_dict
-        self.past_dict = past_dict
-        self.set_current = set(current_dict)
-        self.set_past = set(past_dict)
-        self.intersect = self.set_current.intersection(self.set_past)
-
-    @property
-    def added(self):
-        return self.set_current - self.intersect
-
-    @property
-    def removed(self):
-        return self.set_past - self.intersect
-
-    @property
-    def changed(self):
-        return set(key for key in self.intersect
-                   if self.past_dict[key] != self.current_dict[key])
-    @property
-    def unchanged(self):
-        return set(key for key in self.intersect
-                   if self.past_dict[key] == self.current_dict[key])
-    @property
-    def modified(self):
-        return self.current_dict != self.past_dict
-
-    @property
-    def added_or_changed(self):
-        return self.added.union(self.changed)
-
-    def _changes(self, keys):
-        new = {}
-        old = {}
-        for k in keys:
-            new[k] = self.current_dict.get(k)
-            old[k] = self.past_dict.get(k)
-        return "%s -> %s" % (old, new)
-
-    def __str__(self):
-        if self.modified:
-            s = dedent("""\
-            added: %s
-            removed: %s
-            changed: %s
-            unchanged: %s""") % (
-                self._changes(self.added),
-                self._changes(self.removed),
-                self._changes(self.changed),
-                list(self.unchanged))
-        else:
-            s = "no changes"
-        return s
-
-
-class Serializer:
-    """Handle JSON (de)serialization."""
-
-    def __init__(self, path, default=None, serialize=None, deserialize=None):
-        self.path = path
-        self.default = default or {}
-        self.serialize = serialize or json.dump
-        self.deserialize = deserialize or json.load
-
-    def exists(self):
-        return os.path.exists(self.path)
-
-    def get(self):
-        if self.exists():
-            with open(self.path) as f:
-                return self.deserialize(f)
-        return self.default
-
-    def set(self, data):
-        with open(self.path, 'w') as f:
-            self.serialize(data, f)

=== added directory 'hooks/tests'
=== added file 'hooks/tests/__init__.py'
=== added file 'hooks/tests/test_cached_website_hooks.py'
--- hooks/tests/test_cached_website_hooks.py	1970-01-01 00:00:00 +0000
+++ hooks/tests/test_cached_website_hooks.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,111 @@
+from testtools import TestCase
+from mock import patch, call
+
+import hooks
+
+
+class CachedWebsiteRelationTest(TestCase):
+
+    def setUp(self):
+        super(CachedWebsiteRelationTest, self).setUp()
+        self.notify_cached_website = self.patch_hook("notify_cached_website")
+        self.config_changed = self.patch_hook('config_changed')
+        self.config_get = self.patch_hook('config_get')
+        self.config_get.return_value = {
+            'enable_forward_proxy': False
+        }
+
+    def patch_hook(self, hook_name):
+        mock_controller = patch.object(hooks, hook_name)
+        mock = mock_controller.start()
+        self.addCleanup(mock_controller.stop)
+        return mock
+
+    def test_website_interface_none(self):
+        self.assertEqual(None, hooks.cached_website_interface(hook_name=None))
+        self.assertFalse(self.notify_cached_website.called)
+
+    def test_website_interface_joined(self):
+        hooks.cached_website_interface(hook_name="joined")
+        self.notify_cached_website.assert_called_once_with(
+            relation_ids=(None,))
+        self.assertFalse(self.config_changed.called)
+
+    def test_website_interface_changed(self):
+        hooks.cached_website_interface(hook_name="changed")
+        self.notify_cached_website.assert_called_once_with(
+            relation_ids=(None,))
+        self.assertFalse(self.config_changed.called)
+
+    def test_website_interface_forward_enabled(self):
+        self.config_get.return_value = {
+            'enable_forward_proxy': True
+        }
+        hooks.cached_website_interface(hook_name="changed")
+        self.notify_cached_website.assert_called_once_with(
+            relation_ids=(None,))
+        self.assertTrue(self.config_changed.called)
+
+
+class NotifyCachedWebsiteRelationTest(TestCase):
+
+    def setUp(self):
+        super(NotifyCachedWebsiteRelationTest, self).setUp()
+
+        self.get_service_ports = self.patch_hook("get_service_ports")
+        self.get_sitenames = self.patch_hook("get_sitenames")
+        self.get_relation_ids = self.patch_hook("get_relation_ids")
+        self.get_hostname = self.patch_hook("get_hostname")
+        self.relation_set = self.patch_hook("relation_set")
+
+    def patch_hook(self, hook_name):
+        mock_controller = patch.object(hooks, hook_name)
+        mock = mock_controller.start()
+        self.addCleanup(mock_controller.stop)
+        return mock
+
+    def test_notify_cached_website_no_relation_ids(self):
+        self.get_relation_ids.return_value = ()
+
+        hooks.notify_cached_website()
+
+        self.assertFalse(self.relation_set.called)
+        self.get_relation_ids.assert_called_once_with(
+            "cached-website")
+
+    def test_notify_cached_website_with_default_relation(self):
+        self.get_relation_ids.return_value = ()
+        self.get_hostname.return_value = "foo.local"
+        self.get_service_ports.return_value = (3128,)
+        self.get_sitenames.return_value = ("foo.internal", "bar.internal")
+
+        hooks.notify_cached_website(relation_ids=(None,))
+
+        self.get_hostname.assert_called_once_with()
+        self.relation_set.assert_called_once_with(
+            relation_id=None, port=3128,
+            hostname="foo.local",
+            sitenames="foo.internal bar.internal")
+        self.assertFalse(self.get_relation_ids.called)
+
+    def test_notify_cached_website_with_relations(self):
+        self.get_relation_ids.return_value = ("cached-website:1",
+                                              "cached-website:2")
+        self.get_hostname.return_value = "foo.local"
+        self.get_service_ports.return_value = (3128,)
+        self.get_sitenames.return_value = ("foo.internal", "bar.internal")
+
+        hooks.notify_cached_website()
+
+        self.get_hostname.assert_called_once_with()
+        self.get_relation_ids.assert_called_once_with("cached-website")
+        self.relation_set.assert_has_calls([
+            call.relation_set(
+                relation_id="cached-website:1", port=3128,
+                hostname="foo.local",
+                sitenames="foo.internal bar.internal"),
+            call.relation_set(
+                relation_id="cached-website:2", port=3128,
+                hostname="foo.local",
+                sitenames="foo.internal bar.internal"),
+            ])

=== added file 'hooks/tests/test_config_changed_hooks.py'
--- hooks/tests/test_config_changed_hooks.py	1970-01-01 00:00:00 +0000
+++ hooks/tests/test_config_changed_hooks.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,60 @@
+from testtools import TestCase
+from mock import patch
+
+import hooks
+
+
+class ConfigChangedTest(TestCase):
+
+    def setUp(self):
+        super(ConfigChangedTest, self).setUp()
+        self.get_service_ports = self.patch_hook("get_service_ports")
+        self.get_sitenames = self.patch_hook("get_sitenames")
+        self.construct_squid3_config = self.patch_hook(
+            "construct_squid3_config")
+        self.update_nrpe_checks = self.patch_hook(
+            "update_nrpe_checks")
+        self.update_service_ports = self.patch_hook(
+            "update_service_ports")
+        self.service_squid3 = self.patch_hook(
+            "service_squid3")
+        self.notify_cached_website = self.patch_hook("notify_cached_website")
+        self.log = self.patch_hook("log")
+        self.config_get = self.patch_hook("config_get")
+
+    def patch_hook(self, hook_name):
+        mock_controller = patch.object(hooks, hook_name)
+        mock = mock_controller.start()
+        self.addCleanup(mock_controller.stop)
+        return mock
+
+    def test_config_changed_notify_cached_website_changed_stanzas(self):
+        self.service_squid3.return_value = True
+        self.get_sitenames.side_effect = (
+            ('foo.internal',),
+            ('foo.internal', 'bar.internal'))
+
+        hooks.config_changed()
+
+        self.notify_cached_website.assert_called_once_with()
+
+    def test_config_changed_no_notify_cached_website_not_changed(self):
+        self.service_squid3.return_value = True
+        self.get_sitenames.side_effect = (
+            ('foo.internal',),
+            ('foo.internal',))
+
+        hooks.config_changed()
+
+        self.assertFalse(self.notify_cached_website.called)
+
+    @patch("sys.exit")
+    def test_config_changed_no_notify_cached_website_failed_check(self, exit):
+        self.service_squid3.return_value = False
+        self.get_sitenames.side_effect = (
+            ('foo.internal',),
+            ('foo.internal', 'bar.internal'))
+
+        hooks.config_changed()
+
+        self.assertFalse(self.notify_cached_website.called)

=== added file 'hooks/tests/test_helpers.py'
--- hooks/tests/test_helpers.py	1970-01-01 00:00:00 +0000
+++ hooks/tests/test_helpers.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,415 @@
+import os
+import json
+import yaml
+
+from os.path import dirname
+
+from testtools import TestCase
+from testtools.matchers import AfterPreprocessing, Contains
+from mock import patch
+
+import hooks
+from charmhelpers.core.hookenv import Serializable
+
+
+def normalize_whitespace(data):
+    return ' '.join(chunk for chunk in data.split())
+
+
[e-mail](os.environ, {"CHARM_DIR": dirname(dirname(dirname(__file__)))})
+class SquidConfigTest(TestCase):
+
+    def _assert_contents(self, *expected):
+        def side_effect(got):
+            for chunk in expected:
+                self.assertThat(got, AfterPreprocessing(
+                    normalize_whitespace,
+                    Contains(normalize_whitespace(chunk))))
+        return side_effect
+
+    def _apply_patch(self, name):
+        p = patch(name)
+        mocked_name = p.start()
+        self.addCleanup(p.stop)
+        return mocked_name
+
+    def setUp(self):
+        super(SquidConfigTest, self).setUp()
+        self.get_reverse_sites = self._apply_patch('hooks.get_reverse_sites')
+        self.get_forward_sites = self._apply_patch('hooks.get_forward_sites')
+        self.config_get = self._apply_patch('hooks.config_get')
+        self.write_squid3_config = self._apply_patch(
+            'hooks.write_squid3_config')
+
+        self.get_forward_sites.return_value = {}
+
+    def test_squid_config_no_sites(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": "",
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+            "via": "on",
+        })
+        self.get_reverse_sites.return_value = None
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            always_direct deny all
+            """,
+            """
+            via on
+            """,
+            )
+        hooks.construct_squid3_config()
+
+    def test_squid_config_via_off(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": "",
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+            "via": "off",
+        })
+        self.get_reverse_sites.return_value = None
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            via off
+            """,
+            )
+        hooks.construct_squid3_config()
+
+    def test_squid_config_refresh_pattern_json(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": json.dumps(
+                {"http://www.ubuntu.com":
+                 {"min": 0, "percent": 20, "max": 60}}),
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+        })
+        self.get_reverse_sites.return_value = None
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            refresh_pattern http://www.ubuntu.com 0 20% 60
+            """,
+            )
+        hooks.construct_squid3_config()
+
+    def test_squid_config_refresh_pattern_yaml(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": yaml.dump(
+                {"http://www.ubuntu.com":
+                 {"min": 0, "percent": 20, "max": 60}}),
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+        })
+        self.get_reverse_sites.return_value = None
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            refresh_pattern http://www.ubuntu.com 0 20% 60
+            """,
+            )
+        hooks.construct_squid3_config()
+
+    def test_squid_config_refresh_pattern_options(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": yaml.dump(
+                {"http://www.ubuntu.com":
+                 {"min": 0, "percent": 20, "max": 60,
+                  "options": ["override-lastmod",
+                              "reload-into-ims"]}}),
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+        })
+        self.get_reverse_sites.return_value = None
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            refresh_pattern http://www.ubuntu.com 0 20% 60
+            override-lastmod reload-into-ims
+            refresh_pattern . 30 20% 4320
+            """,
+            )
+        hooks.construct_squid3_config()
+
+    def test_squid_config_refresh_pattern_default(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": yaml.dump(
+                {".":
+                 {"min": 0, "percent": 20, "max": 60,
+                  "options": ["override-lastmod",
+                              "reload-into-ims"]}}),
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+        })
+        self.get_reverse_sites.return_value = None
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            refresh_pattern . 0 20% 60
+            override-lastmod reload-into-ims
+            """,
+            )
+        hooks.construct_squid3_config()
+
+    def test_squid_config_no_sitenames(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": "",
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+        })
+        self.get_reverse_sites.return_value = {
+            None: [
+                hooks.Server("website_1__foo_1", "1.2.3.4", 4242, ''),
+                hooks.Server("website_1__foo_2", "1.2.3.5", 4242, ''),
+                ],
+            }
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            acl no_sitename_acl myport
+            http_access allow accel_ports no_sitename_acl
+            never_direct allow no_sitename_acl
+            """,
+            """
+            cache_peer 1.2.3.4 parent 4242 0 name=website_1__foo_1 no-query
+            no-digest originserver round-robin login=PASS
+            cache_peer_access website_1__foo_1 allow no_sitename_acl
+            cache_peer_access website_1__foo_1 deny all
+            """,
+            """
+            cache_peer 1.2.3.5 parent 4242 0 name=website_1__foo_2 no-query
+            no-digest originserver round-robin login=PASS
+            cache_peer_access website_1__foo_2 allow no_sitename_acl
+            cache_peer_access website_1__foo_2 deny all
+            """
+            )
+        hooks.construct_squid3_config()
+
+    def test_squid_config_with_domain(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": "",
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+        })
+        self.get_reverse_sites.return_value = {
+            "foo.com": [
+                hooks.Server("foo_com__website_1__foo_1",
+                             "1.2.3.4", 4242, "forceddomain=example.com"),
+                hooks.Server("foo_com__website_1__foo_2",
+                             "1.2.3.5", 4242, "forceddomain=example.com"),
+                ],
+            }
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            acl s_1_acl dstdomain foo.com
+            http_access allow accel_ports s_1_acl
+            http_access allow CONNECT SSL_ports s_1_acl
+            always_direct allow CONNECT SSL_ports s_1_acl
+            always_direct deny s_1_acl
+            """,
+            """
+            cache_peer 1.2.3.4 parent 4242 0 name=foo_com__website_1__foo_1
+            no-query no-digest originserver round-robin login=PASS
+            forceddomain=example.com
+            cache_peer_access foo_com__website_1__foo_1 allow s_1_acl
+            cache_peer_access foo_com__website_1__foo_1 deny all
+            """,
+            """
+            cache_peer 1.2.3.5 parent 4242 0 name=foo_com__website_1__foo_2
+            no-query no-digest originserver round-robin login=PASS
+            forceddomain=example.com
+            cache_peer_access foo_com__website_1__foo_2 allow s_1_acl
+            cache_peer_access foo_com__website_1__foo_2 deny all
+            """
+            )
+        hooks.construct_squid3_config()
+
+    def test_with_domain_no_servers_only_direct(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": "",
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+        })
+        self.get_reverse_sites.return_value = {
+            "foo.com": [
+                ],
+            }
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            acl s_1_acl dstdomain foo.com
+            http_access allow accel_ports s_1_acl
+            http_access allow CONNECT SSL_ports s_1_acl
+            always_direct allow s_1_acl
+            """,
+            )
+        hooks.construct_squid3_config()
+
+    def test_with_balancer_no_servers_only_direct(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": "",
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+            "x_balancer_name_allowed": True,
+        })
+        self.get_reverse_sites.return_value = {
+            "foo.com": [],
+            }
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            acl s_1_acl dstdomain foo.com
+            http_access allow accel_ports s_1_acl
+            http_access allow CONNECT SSL_ports s_1_acl
+            always_direct allow s_1_acl
+            """,
+            """
+            acl s_1_balancer req_header X-Balancer-Name foo\.com
+            http_access allow accel_ports s_1_balancer
+            http_access allow CONNECT SSL_ports s_1_balancer
+            always_direct allow s_1_balancer
+            """,
+            )
+        hooks.construct_squid3_config()
+
+    def test_with_balancer_name(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": "",
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+            "x_balancer_name_allowed": True,
+            })
+        self.get_reverse_sites.return_value = {
+            "foo.com": [
+                hooks.Server("foo_com__website_1__foo_1",
+                             "1.2.3.4", 4242, ''),
+                hooks.Server("foo_com__website_1__foo_2",
+                             "1.2.3.5", 4242, ''),
+                ],
+            }
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            acl s_1_acl dstdomain foo.com
+            http_access allow accel_ports s_1_acl
+            http_access allow CONNECT SSL_ports s_1_acl
+            always_direct allow CONNECT SSL_ports s_1_acl
+            always_direct deny s_1_acl
+            """,
+            """
+            acl s_1_balancer req_header X-Balancer-Name foo\.com
+            http_access allow accel_ports s_1_balancer
+            http_access allow CONNECT SSL_ports s_1_balancer
+            always_direct allow CONNECT SSL_ports s_1_balancer
+            always_direct deny s_1_balancer
+            """,
+            """
+            cache_peer 1.2.3.4 parent 4242 0 name=foo_com__website_1__foo_1
+            no-query no-digest originserver round-robin login=PASS
+            cache_peer_access foo_com__website_1__foo_1 allow s_1_acl
+            cache_peer_access foo_com__website_1__foo_1 allow s_1_balancer
+            cache_peer_access foo_com__website_1__foo_1 deny all
+            """,
+            """
+            cache_peer 1.2.3.5 parent 4242 0 name=foo_com__website_1__foo_2
+            no-query no-digest originserver round-robin login=PASS
+            cache_peer_access foo_com__website_1__foo_2 allow s_1_acl
+            """
+            )
+        hooks.construct_squid3_config()
+
+    def test_forward_enabled(self):
+        self.config_get.return_value = Serializable({
+            "enable_forward_proxy": True,
+            "refresh_patterns": "",
+            "cache_size_mb": 1024,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+        })
+        self.get_reverse_sites.return_value = {
+            "foo.com": [],
+            }
+        self.get_forward_sites.return_value = [
+            {'private-address': '1.2.3.4', 'name': 'service_unit_0'},
+            {'private-address': '2.3.4.5', 'name': 'service_unit_1'},
+        ]
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            acl fwd_service_unit_0 src 1.2.3.4
+            http_access allow fwd_service_unit_0
+            http_access allow CONNECT SSL_ports fwd_service_unit_0
+            always_direct allow fwd_service_unit_0
+            always_direct allow CONNECT SSL_ports fwd_service_unit_0
+            acl fwd_service_unit_1 src 2.3.4.5
+            http_access allow fwd_service_unit_1
+            http_access allow CONNECT SSL_ports fwd_service_unit_1
+            always_direct allow fwd_service_unit_1
+            always_direct allow CONNECT SSL_ports fwd_service_unit_1
+            """,
+            """
+            acl s_1_acl dstdomain foo.com
+            http_access allow accel_ports s_1_acl
+            http_access allow CONNECT SSL_ports s_1_acl
+            always_direct allow s_1_acl
+            """
+            )
+        hooks.construct_squid3_config()
+
+    def test_squid_config_cache_enabled(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": "",
+            "cache_size_mb": 1024,
+            "cache_mem_mb": 256,
+            "cache_dir": "/var/run/squid3",
+            "target_objs_per_dir": 16,
+            "avg_obj_size_kb": 4,
+        })
+        self.get_reverse_sites.return_value = None
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            cache_dir aufs /var/run/squid3 1024 32 512
+            cache_mem 256 MB
+            """,
+            )
+        hooks.construct_squid3_config()
+
+    def test_squid_config_cache_disabled(self):
+        self.config_get.return_value = Serializable({
+            "refresh_patterns": "",
+            "cache_size_mb": 0,
+            "target_objs_per_dir": 1024,
+            "avg_obj_size_kb": 1024,
+        })
+        self.get_reverse_sites.return_value = None
+        self.write_squid3_config.side_effect = self._assert_contents(
+            """
+            cache deny all
+            """,
+            )
+        hooks.construct_squid3_config()
+
+
+class HelpersTest(TestCase):
+    def test_gets_config(self):
+        json_string = '{"foo": "BAR"}'
+        with patch('subprocess.check_output') as check_output:
+            check_output.return_value = json_string
+
+            result = hooks.config_get()
+
+            self.assertEqual(result['foo'], 'BAR')
+            check_output.assert_called_with(['config-get', '--format=json'])
+
+    def test_gets_config_with_scope(self):
+        json_string = '{"foo": "BAR"}'
+        with patch('subprocess.check_output') as check_output:
+            check_output.return_value = json_string
+
+            result = hooks.config_get(scope='baz')
+
+            self.assertEqual(result['foo'], 'BAR')
+            check_output.assert_called_with(['config-get', 'baz',
+                                             '--format=json'])

=== added file 'hooks/tests/test_nrpe_hooks.py'
--- hooks/tests/test_nrpe_hooks.py	1970-01-01 00:00:00 +0000
+++ hooks/tests/test_nrpe_hooks.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,271 @@
+import os
+import grp
+import pwd
+import subprocess
+
+from testtools import TestCase
+from mock import patch, call
+
+import hooks
+
+from charmhelpers.contrib.charmsupport import nrpe
+from charmhelpers.core.hookenv import Serializable
+
+
+class NRPERelationTest(TestCase):
+    """Tests for the update_nrpe_checks hook.
+
+    Half of this is already tested in the tests for charmsupport.nrpe, but
+    as the hook in the charm pre-dates that, the tests are left here to ensure
+    backwards-compatibility.
+
+    """
+    patches = {
+        'config': {'object': nrpe},
+        'log': {'object': nrpe},
+        'getpwnam': {'object': pwd},
+        'getgrnam': {'object': grp},
+        'mkdir': {'object': os},
+        'chown': {'object': os},
+        'exists': {'object': os.path},
+        'listdir': {'object': os},
+        'remove': {'object': os},
+        'open': {'object': nrpe, 'create': True},
+        'isfile': {'object': os.path},
+        'call': {'object': subprocess},
+        'install_nrpe_scripts': {'object': hooks},
+        'relation_ids': {'object': nrpe},
+        'relation_set': {'object': nrpe},
+    }
+
+    def setUp(self):
+        super(NRPERelationTest, self).setUp()
+        self.patched = {}
+        # Mock the universe.
+        for attr, data in self.patches.items():
+            create = data.get('create', False)
+            patcher = patch.object(data['object'], attr, create=create)
+            self.patched[attr] = patcher.start()
+            self.addCleanup(patcher.stop)
+        if not 'JUJU_UNIT_NAME' in os.environ:
+            os.environ['JUJU_UNIT_NAME'] = 'test'
+
+    def check_call_counts(self, **kwargs):
+        for attr, expected in kwargs.items():
+            patcher = self.patched[attr]
+            self.assertEqual(expected, patcher.call_count, attr)
+
+    def test_update_nrpe_no_nagios_bails(self):
+        config = {'nagios_context': 'test'}
+        self.patched['config'].return_value = Serializable(config)
+        self.patched['getpwnam'].side_effect = KeyError
+
+        self.assertEqual(None, hooks.update_nrpe_checks())
+
+        expected = 'Nagios user not set up, nrpe checks not updated'
+        self.patched['log'].assert_called_once_with(expected)
+        self.check_call_counts(log=1, config=1, getpwnam=1)
+
+    def test_update_nrpe_removes_existing_config(self):
+        config = {
+            'nagios_context': 'test',
+            'nagios_check_http_params': '-u http://example.com/url',
+        }
+        self.patched['config'].return_value = Serializable(config)
+        self.patched['exists'].return_value = True
+        self.patched['listdir'].return_value = [
+            'foo', 'bar.cfg', 'check_squidrp.cfg']
+
+        self.assertEqual(None, hooks.update_nrpe_checks())
+
+        expected = '/var/lib/nagios/export/check_squidrp.cfg'
+        self.patched['remove'].assert_called_once_with(expected)
+        self.check_call_counts(config=1, getpwnam=1, getgrnam=1,
+                               exists=5, remove=1, open=4, listdir=2)
+
+    def test_update_nrpe_uses_check_squidpeers(self):
+        config = {
+            'nagios_context': 'test',
+        }
+        self.patched['config'].return_value = Serializable(config)
+        self.patched['exists'].return_value = True
+        self.patched['isfile'].return_value = False
+
+        self.assertEqual(None, hooks.update_nrpe_checks())
+        self.assertEqual(2, self.patched['open'].call_count)
+        filename = 'check_squidpeers.cfg'
+
+        service_file_contents = """
+#---------------------------------------------------
+# This file is Juju managed
+#---------------------------------------------------
+define service {
+    use                             active-service
+    host_name                       test-test
+    service_description             test-test[squidpeers] Check Squid Peers
+    check_command                   check_nrpe!check_squidpeers
+    servicegroups                   test
+}
+"""
+        self.patched['open'].assert_has_calls(
+            [call('/etc/nagios/nrpe.d/%s' % filename, 'w'),
+             call('/var/lib/nagios/export/service__test-test_%s' %
+                  filename, 'w'),
+             call().__enter__().write(service_file_contents),
+             call().__enter__().write('# check squidpeers\n'),
+             call().__enter__().write(
+                 'command[check_squidpeers]='
+                 '/check_squidpeers\n')],
+            any_order=True)
+
+        self.check_call_counts(config=1, getpwnam=1, getgrnam=1,
+                               exists=3, open=2, listdir=1)
+
+    def test_update_nrpe_with_check_url(self):
+        config = {
+            'nagios_context': 'test',
+            'nagios_check_http_params': '-u foo -H bar',
+        }
+        self.patched['config'].return_value = Serializable(config)
+        self.patched['exists'].return_value = True
+        self.patched['isfile'].return_value = False
+
+        self.assertEqual(None, hooks.update_nrpe_checks())
+        self.assertEqual(4, self.patched['open'].call_count)
+        filename = 'check_squidrp.cfg'
+
+        service_file_contents = """
+#---------------------------------------------------
+# This file is Juju managed
+#---------------------------------------------------
+define service {
+    use                             active-service
+    host_name                       test-test
+    service_description             test-test[squidrp] Check Squid
+    check_command                   check_nrpe!check_squidrp
+    servicegroups                   test
+}
+"""
+        self.patched['open'].assert_has_calls(
+            [call('/etc/nagios/nrpe.d/%s' % filename, 'w'),
+             call('/var/lib/nagios/export/service__test-test_%s' %
+                  filename, 'w'),
+             call().__enter__().write(service_file_contents),
+             call().__enter__().write('# check squidrp\n'),
+             call().__enter__().write(
+                 'command[check_squidrp]=/check_http -u foo -H bar\n')],
+            any_order=True)
+
+        self.check_call_counts(config=1, getpwnam=1, getgrnam=1,
+                               exists=5, open=4, listdir=2)
+
+    def test_update_nrpe_with_no_check_path(self):
+        config = {
+            'nagios_context': 'test',
+            'services': '- {service_name: i_ytimg_com}',
+        }
+        self.patched['config'].return_value = Serializable(config)
+        self.patched['exists'].return_value = True
+
+        self.assertEqual(None, hooks.update_nrpe_checks())
+
+        self.check_call_counts(config=1, getpwnam=1, getgrnam=1,
+                               exists=3, open=2, listdir=1)
+
+    def test_update_nrpe_with_services_and_host_header(self):
+        config = {
+            'nagios_context': 'test',
+            'services': '- {service_name: i_ytimg_com, nrpe_check_path: /}',
+        }
+        self.patched['config'].return_value = Serializable(config)
+        self.patched['exists'].return_value = True
+
+        self.assertEqual(None, hooks.update_nrpe_checks())
+
+        self.check_call_counts(config=1, getpwnam=1, getgrnam=1,
+                               exists=5, open=4, listdir=2)
+        expected = ('command[check_squid-i_ytimg_com]=/check_http '
+                    '-I 127.0.0.1 -p 3128 --method=HEAD '
+                    '-u http://i_ytimg_com/\n')
+        self.patched['open'].assert_has_calls(
+            [call('/etc/nagios/nrpe.d/check_squid-i_ytimg_com.cfg', 'w'),
+             call().__enter__().write(expected)],
+            any_order=True)
+
+    def test_update_nrpe_with_dotted_service_name_and_host_header(self):
+        config = {
+            'nagios_context': 'test',
+            'services': '- {service_name: i.ytimg.com, nrpe_check_path: /}',
+        }
+        self.patched['config'].return_value = Serializable(config)
+        self.patched['exists'].return_value = True
+
+        self.assertEqual(None, hooks.update_nrpe_checks())
+
+        self.check_call_counts(config=1, getpwnam=1, getgrnam=1,
+                               exists=5, open=4, listdir=2)
+        expected = ('command[check_squid-i_ytimg_com]=/check_http '
+                    '-I 127.0.0.1 -p 3128 --method=HEAD '
+                    '-u http://i.ytimg.com/\n')
+        self.patched['open'].assert_has_calls(
+            [call('/etc/nagios/nrpe.d/check_squid-i_ytimg_com.cfg', 'w'),
+             call().__enter__().write(expected)],
+            any_order=True)
+
+    def test_update_nrpe_with_services_and_balancer_name_header(self):
+        config = {
+            'nagios_context': 'test',
+            'x_balancer_name_allowed': True,
+            'services': '- {service_name: i_ytimg_com, nrpe_check_path: /}',
+        }
+        self.patched['config'].return_value = Serializable(config)
+        self.patched['exists'].return_value = True
+
+        self.assertEqual(None, hooks.update_nrpe_checks())
+
+        self.check_call_counts(config=1, getpwnam=1, getgrnam=1,
+                               exists=5, open=4, listdir=2)
+
+        expected = ('command[check_squid-i_ytimg_com]=/check_http '
+                    '-I 127.0.0.1 -p 3128 --method=HEAD -u http://localhost/ '
+                    "-k 'X-Balancer-Name: i_ytimg_com'\n")
+        self.patched['open'].assert_has_calls(
+            [call('/etc/nagios/nrpe.d/check_squid-i_ytimg_com.cfg', 'w'),
+             call().__enter__().write(expected)],
+            any_order=True)
+
+    def test_update_nrpe_with_services_and_optional_path(self):
+        services = '- {nrpe_check_path: /foo.jpg, service_name: foo_com}\n'
+        config = {
+            'nagios_context': 'test',
+            'services': services,
+        }
+        self.patched['config'].return_value = Serializable(config)
+        self.patched['exists'].return_value = True
+
+        self.assertEqual(None, hooks.update_nrpe_checks())
+
+        self.check_call_counts(config=1, getpwnam=1, getgrnam=1,
+                               exists=5, open=4, listdir=2)
+        expected = ('command[check_squid-foo_com]=/check_http '
+                    '-I 127.0.0.1 -p 3128 --method=HEAD '
+                    '-u http://foo_com/foo.jpg\n')
+        self.patched['open'].assert_has_calls(
+            [call('/etc/nagios/nrpe.d/check_squid-foo_com.cfg', 'w'),
+             call().__enter__().write(expected)],
+            any_order=True)
+
+    def test_update_nrpe_restarts_service(self):
+        config = {
+            'nagios_context': 'test',
+            'nagios_check_http_params': '-u foo -p 3128'
+        }
+        self.patched['config'].return_value = Serializable(config)
+        self.patched['exists'].return_value = True
+
+        self.assertEqual(None, hooks.update_nrpe_checks())
+
+        expected = ['service', 'nagios-nrpe-server', 'restart']
+        self.assertEqual(expected, self.patched['call'].call_args[0][0])
+        self.check_call_counts(config=1, getpwnam=1, getgrnam=1,
+                               exists=5, open=4, listdir=2, call=1)

=== added file 'hooks/tests/test_website_hooks.py'
--- hooks/tests/test_website_hooks.py	1970-01-01 00:00:00 +0000
+++ hooks/tests/test_website_hooks.py	2013-10-29 21:07:21 +0000
@@ -0,0 +1,267 @@
+import yaml
+
+from testtools import TestCase
+from mock import patch
+
+import hooks
+from charmhelpers.core.hookenv import Serializable
+
+
+class WebsiteRelationTest(TestCase):
+
+    def setUp(self):
+        super(WebsiteRelationTest, self).setUp()
+
+        relations_of_type = patch.object(hooks, "relations_of_type")
+        self.relations_of_type = relations_of_type.start()
+        self.addCleanup(relations_of_type.stop)
+
+        config_get = patch.object(hooks, "config_get")
+        self.config_get = config_get.start()
+        self.addCleanup(config_get.stop)
+
+        log = patch.object(hooks, "log")
+        self.log = log.start()
+        self.addCleanup(log.stop)
+
+    def test_relation_data_returns_no_relations(self):
+        self.config_get.return_value = Serializable({})
+        self.relations_of_type.return_value = []
+        self.assertEqual(None, hooks.get_reverse_sites())
+
+    def test_no_port_in_relation_data(self):
+        self.config_get.return_value = Serializable({})
+        self.relations_of_type.return_value = [
+            {"private-address": "1.2.3.4",
+             "__unit__": "foo/1",
+             "__relid__": "website:1"},
+            ]
+        self.assertIs(None, hooks.get_reverse_sites())
+        self.log.assert_called_once_with(
+            "No port in relation data for 'foo/1', skipping.")
+
+    def test_empty_relation_services(self):
+        self.config_get.return_value = Serializable({})
+        self.relations_of_type.return_value = [
+            {"private-address": "1.2.3.4",
+             "__unit__": "foo/1",
+             "__relid__": "website:1",
+             "all_services": "",
+             },
+            ]
+
+        self.assertEqual(None, hooks.get_reverse_sites())
+        self.assertFalse(self.log.called)
+
+    def test_no_port_in_relation_data_ok_with_all_services(self):
+        self.config_get.return_value = Serializable({})
+        self.relations_of_type.return_value = [
+            {"private-address": "1.2.3.4",
+             "__unit__": "foo/1",
+             "__relid__": "website:1",
+             "all_services": yaml.dump([
+                 {"service_name": "foo.internal",
+                  "service_port": 4242},
+                 ]),
+             },
+            ]
+
+        expected = {
+            "foo.internal": [
+                ("foo_internal__website_1__foo_1", "1.2.3.4", 4242, ''),
+                ],
+            }
+        self.assertEqual(expected, hooks.get_reverse_sites())
+        self.assertFalse(self.log.called)
+
+    def test_no_private_address_in_relation_data(self):
+        self.config_get.return_value = Serializable({})
+        self.relations_of_type.return_value = [
+            {"port": 4242,
+             "__unit__": "foo/0",
+             "__relid__": "website:1",
+             },
+            ]
+        self.assertIs(None, hooks.get_reverse_sites())
+        self.log.assert_called_once_with(
+            "No private-address in relation data for 'foo/0', skipping.")
+
+    def test_sitenames_in_relation_data(self):
+        self.config_get.return_value = Serializable({})
+        self.relations_of_type.return_value = [
+            {"private-address": "1.2.3.4",
+             "port": 4242,
+             "__unit__": "foo/1",
+             "__relid__": "website:1",
+             "sitenames": "foo.internal bar.internal"},
+            {"private-address": "1.2.3.5",
+             "port": 4242,
+             "__unit__": "foo/2",
+             "__relid__": "website:1",
+             "sitenames": "foo.internal bar.internal"},
+            ]
+        expected = {
+            "foo.internal": [
+                ("foo_internal__website_1__foo_1", "1.2.3.4", 4242, ''),
+                ("foo_internal__website_1__foo_2", "1.2.3.5", 4242, ''),
+                ],
+            "bar.internal": [
+                ("bar_internal__website_1__foo_1", "1.2.3.4", 4242, ''),
+                ("bar_internal__website_1__foo_2", "1.2.3.5", 4242, ''),
+                ],
+            }
+        self.assertEqual(expected, hooks.get_reverse_sites())
+
+    def test_all_services_in_relation_data(self):
+        self.config_get.return_value = Serializable({})
+        self.relations_of_type.return_value = [
+            {"private-address": "1.2.3.4",
+             "__unit__": "foo/1",
+             "__relid__": "website:1",
+             "all_services": yaml.dump([
+                 {"service_name": "foo.internal",
+                  "service_port": 4242},
+                 {"service_name": "bar.internal",
+                  "service_port": 4243}
+                 ]),
+             },
+            {"private-address": "1.2.3.5",
+             "__unit__": "foo/2",
+             "__relid__": "website:1",
+             "all_services": yaml.dump([
+                 {"service_name": "foo.internal",
+                  "service_port": 4242},
+                 {"service_name": "bar.internal",
+                  "service_port": 4243}
+                 ]),
+             },
+            ]
+        expected = {
+            "foo.internal": [
+                ("foo_internal__website_1__foo_1", "1.2.3.4", 4242, ''),
+                ("foo_internal__website_1__foo_2", "1.2.3.5", 4242, ''),
+                ],
+            "bar.internal": [
+                ("bar_internal__website_1__foo_1", "1.2.3.4", 4243, ''),
+                ("bar_internal__website_1__foo_2", "1.2.3.5", 4243, ''),
+                ],
+            }
+        self.assertEqual(expected, hooks.get_reverse_sites())
+
+    def test_unit_names_in_relation_data(self):
+        self.config_get.return_value = Serializable({})
+        self.relations_of_type.return_value = [
+            {"private-address": "1.2.3.4",
+             "__relid__": "website_1",
+             "__unit__": "foo/1",
+             "port": 4242},
+            {"private-address": "1.2.3.5",
+             "__relid__": "website_1",
+             "__unit__": "foo/2",
+             "port": 4242},
+            ]
+        expected = {
+            None: [
+                ("website_1__foo_1", "1.2.3.4", 4242, ''),
+                ("website_1__foo_2", "1.2.3.5", 4242, ''),
+                ],
+            }
+        self.assertEqual(expected, hooks.get_reverse_sites())
+
+    def test_sites_from_config_no_domain(self):
+        self.relations_of_type.return_value = []
+        self.config_get.return_value = Serializable({
+            "services": yaml.dump([
+                {"service_name": "foo.internal",
+                 "servers": [
+                     ["1.2.3.4", 4242],
+                     ["1.2.3.5", 4242],
+                     ]
+                 }
+                ])})
+        expected = {
+            "foo.internal": [
+                ("foo_internal__1_2_3_4", "1.2.3.4", 4242, ''),
+                ("foo_internal__1_2_3_5", "1.2.3.5", 4242, ''),
+                ],
+            }
+        self.assertEqual(expected, hooks.get_reverse_sites())
+
+    def test_sites_from_config_with_domain(self):
+        self.relations_of_type.return_value = []
+        self.config_get.return_value = Serializable({
+            "services": yaml.dump([
+                {"service_name": "foo.internal",
+                 "server_options": ["forceddomain=example.com"],
+                 "servers": [
+                     ["1.2.3.4", 4242],
+                     ["1.2.3.5", 4242],
+                     ]
+                 }
+                ])})
+        expected = {
+            "foo.internal": [
+                ("foo_internal__1_2_3_4", "1.2.3.4", 4242,
+                 "forceddomain=example.com"),
+                ("foo_internal__1_2_3_5", "1.2.3.5", 4242,
+                 "forceddomain=example.com"),
+                ],
+            }
+        self.assertEqual(expected, hooks.get_reverse_sites())
+
+    def test_sites_from_config_and_relation_with_domain(self):
+        self.relations_of_type.return_value = [
+            {"private-address": "1.2.3.4",
+             "__unit__": "foo/1",
+             "__relid__": "website:1",
+             "all_services": yaml.dump([
+                 {"service_name": "foo.internal",
+                  "service_port": 4242},
+                 {"service_name": "bar.internal",
+                  "service_port": 4243}
+                 ]),
+             },
+            {"private-address": "1.2.3.5",
+             "__unit__": "foo/2",
+             "__relid__": "website:1",
+             "all_services": yaml.dump([
+                 {"service_name": "foo.internal",
+                  "service_port": 4242},
+                 {"service_name": "bar.internal",
+                  "service_port": 4243}
+                 ]),
+             },
+            ]
+        self.config_get.return_value = Serializable({
+            "services": yaml.dump([
+                {"service_name": "foo.internal",
+                 "server_options": ["forceddomain=example.com"],
+                 "servers": [
+                     ["1.2.4.4", 4242],
+                     ["1.2.4.5", 4242],
+                     ]
+                 }
+                ])})
+        expected = {
+            "foo.internal": [
+                ("foo_internal__1_2_4_4", "1.2.4.4", 4242,
+                 "forceddomain=example.com"),
+                ("foo_internal__1_2_4_5", "1.2.4.5", 4242,
+                 "forceddomain=example.com"),
+                ("foo_internal__website_1__foo_1", "1.2.3.4", 4242,
+                 "forceddomain=example.com"),
+                ("foo_internal__website_1__foo_2", "1.2.3.5", 4242,
+                 "forceddomain=example.com"),
+                ],
+            "bar.internal": [
+                ("bar_internal__website_1__foo_1", "1.2.3.4", 4243, ''),
+                ("bar_internal__website_1__foo_2", "1.2.3.5", 4243, ''),
+                ],
+            }
+        self.assertEqual(expected, hooks.get_reverse_sites())
+
+    def test_empty_sites_from_config_no_domain(self):
+        self.relations_of_type.return_value = []
+        self.config_get.return_value = Serializable({
+            "services": ""})
+        self.assertEqual(None, hooks.get_reverse_sites())

=== added symlink 'hooks/upgrade-charm'
=== target is u'hooks.py'
=== modified file 'metadata.yaml'
--- metadata.yaml	2013-07-11 19:15:57 +0000
+++ metadata.yaml	2013-10-29 21:07:21 +0000
@@ -1,6 +1,6 @@
 name: squid-reverseproxy
 summary: Full featured Web Proxy cache (HTTP proxy)
-maintainer: "Matthew Wedgwood <[e-mail]>"
+maintainer: [Matthew Wedgwood <[e-mail]>, Alexander List <[e-mail]>]
 description: >
   Squid is a high-performance proxy caching server for web clients,
   supporting FTP, gopher, and HTTP data objects.  Squid version 3 is a
@@ -23,6 +23,9 @@
   nrpe-external-master:
     interface: nrpe-external-master
     scope: container
+  local-monitors:
+    interface: local-monitors
+    scope: container
 requires:
   website:
     interface: http

=== removed file 'revision'
--- revision	2013-02-15 07:04:29 +0000
+++ revision	1970-01-01 00:00:00 +0000
@@ -1,1 +0,0 @@
-0

=== added file 'setup.cfg'
--- setup.cfg	1970-01-01 00:00:00 +0000
+++ setup.cfg	2013-10-29 21:07:21 +0000
@@ -0,0 +1,4 @@
+[nosetests]
+with-coverage=1
+cover-erase=1
+cover-package=hooks
\ No newline at end of file

=== added file 'tarmac_tests.sh'
--- tarmac_tests.sh	1970-01-01 00:00:00 +0000
+++ tarmac_tests.sh	2013-10-29 21:07:21 +0000
@@ -0,0 +1,6 @@
+#!/bin/sh
+# How the tests are run in Jenkins by Tarmac
+
+set -e
+
+make build

=== modified file 'templates/main_config.template'
--- templates/main_config.template	2013-01-11 16:50:54 +0000
+++ templates/main_config.template	2013-10-29 21:07:21 +0000
@@ -1,10 +1,11 @@
-http_port {{ config.port }}  accel vport=443
+http_port {{ config.port }} {{ config.port_options }}
 
 acl manager proto cache_object
 acl localhost src 127.0.0.1/32
 acl to_localhost dst 127.0.0.0/8
 acl PURGE method PURGE
 acl CONNECT method CONNECT
+acl SSL_ports port 443
 
 {% if config.snmp_community -%}
 acl snmp_access snmp_community {{ config.snmp_community }}
@@ -19,6 +20,7 @@
 snmp_incoming_address {{ config.my_ip_address }}
 {% endif -%}
 
+via {{ config.via }}
 logformat combined {{ config.log_format }}
 access_log /var/log/squid3/access.log combined
 
@@ -26,27 +28,69 @@
 
 coredump_dir {{ config.cache_dir }}
 maximum_object_size {{ config.max_obj_size_kb }} KB
+{% if config.cache_size_mb > 0 -%}
 cache_dir aufs {{ config.cache_dir }} {{ config.cache_size_mb }} {{ config.cache_l1 }} {{ config.cache_l2 }}
-
 cache_mem {{ config.cache_mem_mb }} MB
+{% else -%}
+cache deny all
+{% endif -%}
+
 
 log_mime_hdrs on
 
 acl accel_ports myport {{ config.port }}
 
 {% for rp in refresh_patterns.keys() -%}
-refresh_pattern {{ rp }} {{ refresh_patterns[rp]['min'] }} {{ refresh_patterns[rp]['percent'] }}% {{ refresh_patterns[rp]['max'] }}
-{% endfor -%}
-refresh_pattern .               30      20%     4320
-
-{% for relid in relations.keys() -%}
-{% for sitename in relations[relid].sitenames -%}
-acl {{ relations[relid].name }}_acl dstdomain {{ sitename }}
-{% else -%}
-acl {{ relations[relid].name }}_acl myport {{ config.port }}
-{% endfor -%}
-http_access allow accel_ports {{ relations[relid].name }}_acl
-{% endfor -%}
+refresh_pattern {{ rp }} {{ refresh_patterns[rp]['min'] }} {{ refresh_patterns[rp]['percent'] }}% {{ refresh_patterns[rp]['max'] }} {{ ' '.join(refresh_patterns[rp]['options']) }}
+{% endfor -%}
+refresh_pattern . {{default_refresh_pattern.min}} {{default_refresh_pattern.percent}}% {{default_refresh_pattern.max}} {{ ' '.join(default_refresh_pattern.options) }}
+
+# known services
+{% if sites -%}
+{% for sitename in sites.keys() -%}
+{% if sitename -%}
+{% set site_acl = "s_%s_acl" % loop.index %}
+acl {{ site_acl }} dstdomain {{ sitename }}
+http_access allow accel_ports {{ site_acl }}
+http_access allow CONNECT SSL_ports {{ site_acl }}
+{% if sitename in only_direct -%}
+always_direct allow {{ site_acl }}
+{% else -%}
+always_direct allow CONNECT SSL_ports {{ site_acl }}
+always_direct deny {{ site_acl }}
+{% endif -%}
+{% if config['x_balancer_name_allowed'] -%}
+{% set balancer_acl = "s_%s_balancer" % loop.index %}
+acl {{ balancer_acl }} req_header X-Balancer-Name {{ sitename.replace('.', '\.') }}
+http_access allow accel_ports {{ balancer_acl }}
+http_access allow CONNECT SSL_ports {{ balancer_acl }}
+{% if sitename in only_direct -%}
+always_direct allow {{ balancer_acl }}
+{% else -%}
+always_direct allow CONNECT SSL_ports {{ balancer_acl }}
+always_direct deny {{ balancer_acl }}
+{% endif -%}
+{% endif -%}
+{% else %}  # no sitename
+acl no_sitename_acl myport {{ config.port }}
+http_access allow accel_ports no_sitename_acl
+never_direct allow no_sitename_acl
+{% endif %}
+{% endfor -%}
+{% endif -%}
+
+{% if config.enable_forward_proxy -%}
+# no access retrictions
+{% for relation in forward_relations -%}
+{# acl names are limited to 31 chars (!), so using short "fwd_" prefix #}
+{% set forward_acl = "fwd_%s" % relation['name'] -%}
+acl {{ forward_acl }} src {{ relation['private-address'] }}
+http_access allow {{ forward_acl }}
+http_access allow CONNECT SSL_ports {{ forward_acl }}
+always_direct allow {{ forward_acl }}
+always_direct allow CONNECT SSL_ports {{ forward_acl }}
+{% endfor -%}
+{% endif -%}
 
 http_access allow manager localhost
 http_access deny manager
@@ -57,11 +101,24 @@
 http_access deny accel_ports all
 http_access deny all
 icp_access deny all
+always_direct deny all
 
-{% for relid in relations -%}
-{% if relations[relid].port -%}
-cache_peer {{ relations[relid]['private-address'] }} parent {{ relations[relid].port }} 0 name={{ relations[relid].name }} no-query no-digest originserver round-robin login=PASS
-cache_peer_access {{ relations[relid].name }} allow {{ relations[relid].name }}_acl
-cache_peer_access {{ relations[relid].name }} deny !{{ relations[relid].name }}_acl
-{% endif -%}
-{% endfor -%}
+{% if sites -%}
+{% for sitename in sites.keys() -%}
+{% set sites_loop = loop -%}
+{% for peer in sites[sitename] %}
+{% if sitename -%}
+cache_peer {{ peer.address }} parent {{ peer.port }} 0 name={{ peer.name }} no-query no-digest originserver round-robin login=PASS {{ peer.options }}
+cache_peer_access {{ peer.name }} allow s_{{ sites_loop.index }}_acl
+{% if config['x_balancer_name_allowed'] -%}
+cache_peer_access {{ peer.name }} allow s_{{ sites_loop.index }}_balancer
+{% endif -%}
+cache_peer_access {{ peer.name }} deny all
+{% else %}
+cache_peer {{ peer.address }} parent {{ peer.port }} 0 name={{ peer.name }} no-query no-digest originserver round-robin login=PASS
+cache_peer_access {{ peer.name }} allow no_sitename_acl
+cache_peer_access {{ peer.name }} deny all
+{% endif -%}
+{% endfor -%}
+{% endfor -%}
+{% endif -%}