Merge lp:~brad-marshall/charms/trusty/swift-storage/add-nrpe-checks-fix-rsyncd-conf into lp:~openstack-charmers-archive/charms/trusty/swift-storage/trunk

[Brad Marshall]

This adds support for nrpe-external-master, and adds some basic nrpe checks.

This relies on my nrpe-external-master branch lp:~brad-marshall/charms/trusty/nrpe-external-master/fix-rsyncd-conf.

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_lint_check #986 trusty-swift-storage for brad-marshall mp241483
    LINT FAIL: lint-test failed

LINT Results (max last 5 lines):
  hooks/swift_storage_hooks.py:165:80: E501 line too long (90 > 79 characters)
  hooks/swift_storage_hooks.py:174:22: E251 unexpected spaces around keyword / parameter equals
  hooks/swift_storage_hooks.py:174:24: E251 unexpected spaces around keyword / parameter equals
  hooks/swift_storage_utils.py:267:1: W391 blank line at end of file
  make: *** [lint] Error 1

Full lint test output: http://paste.ubuntu.com/8955703/
Build: http://10.98.191.181:8080/job/charm_lint_check/986/

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_unit_test #821 trusty-swift-storage for brad-marshall mp241483
    UNIT FAIL: unit-test failed

UNIT Results (max last 5 lines):
  swift_storage_utils 113 10 91% 253-254, 259-266
  TOTAL 238 37 84%
  Ran 33 tests in 1.198s
  FAILED (errors=3, failures=2)
  make: *** [unit_test] Error 1

Full unit test output: http://paste.ubuntu.com/8955704/
Build: http://10.98.191.181:8080/job/charm_unit_test/821/

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_amulet_test #366 trusty-swift-storage for brad-marshall mp241483
    AMULET FAIL: amulet-test failed

AMULET Results (max last 5 lines):
  juju-test.conductor DEBUG : Tearing down osci-sv07 juju environment
  juju-test.conductor DEBUG : Calling "juju destroy-environment -y osci-sv07"
  juju-test INFO : Results: 1 passed, 2 failed, 0 errored
  ERROR subprocess encountered error code 2
  make: *** [test] Error 2

Full amulet test output: http://paste.ubuntu.com/8955844/
Build: http://10.98.191.181:8080/job/charm_amulet_test/366/

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_lint_check #1241 trusty-swift-storage for brad-marshall mp241483
    LINT FAIL: lint-test failed

LINT Results (max last 5 lines):
  hooks/swift_storage_hooks.py:165:80: E501 line too long (90 > 79 characters)
  hooks/swift_storage_hooks.py:174:22: E251 unexpected spaces around keyword / parameter equals
  hooks/swift_storage_hooks.py:174:24: E251 unexpected spaces around keyword / parameter equals
  hooks/swift_storage_utils.py:267:1: W391 blank line at end of file
  make: *** [lint] Error 1

Full lint test output: http://paste.ubuntu.com/9281254/
Build: http://10.98.191.181:8080/job/charm_lint_check/1241/

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_unit_test #1075 trusty-swift-storage for brad-marshall mp241483
    UNIT FAIL: unit-test failed

UNIT Results (max last 5 lines):
  swift_storage_utils 113 10 91% 253-254, 259-266
  TOTAL 238 37 84%
  Ran 33 tests in 1.187s
  FAILED (errors=3, failures=2)
  make: *** [unit_test] Error 1

Full unit test output: http://paste.ubuntu.com/9281255/
Build: http://10.98.191.181:8080/job/charm_unit_test/1075/

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_amulet_test #544 trusty-swift-storage for brad-marshall mp241483
    AMULET FAIL: amulet-test failed

AMULET Results (max last 5 lines):
  WARNING cannot delete security group "juju-osci-sv05". Used by another environment?
  WARNING cannot delete security group "juju-osci-sv05-0". Used by another environment?
  juju-test INFO : Results: 1 passed, 2 failed, 0 errored
  ERROR subprocess encountered error code 2
  make: *** [test] Error 2

Full amulet test output: http://paste.ubuntu.com/9281356/
Build: http://10.98.191.181:8080/job/charm_amulet_test/544/

[Liam Young (gnuoy)]

Thanks for this branch. I've taken it and fixed up a few minor bits and proposed it against the 'next' branch. It has now landed into 'next' and will promulgate to staging at the end of the month as part of the 15.01 openstack charm release.

review: Disapprove

=== modified file 'charm-helpers-hooks.yaml'
--- charm-helpers-hooks.yaml	2014-09-23 10:22:22 +0000
+++ charm-helpers-hooks.yaml	2014-11-12 04:10:38 +0000
@@ -10,3 +10,4 @@
         - cluster
     - payload.execd
     - contrib.network.ip
+    - contrib.charmsupport

=== modified file 'config.yaml'
--- config.yaml	2014-10-06 21:45:39 +0000
+++ config.yaml	2014-11-12 04:10:38 +0000
@@ -95,3 +95,17 @@
     type: int
     description: |
       Number of replication workers to spawn.
+  nagios-check-params:
+    default: "-m -r 60 180 10 20"
+    type: string
+    description: String appended to nagios check
+  nagios_context:
+    default: "juju"
+    type: string
+    description: |
+      Used by the nrpe-external-master subordinate charm.
+      A string that will be prepended to instance name to set the host name
+      in nagios. So for instance the hostname would be something like:
+          juju-myservice-0
+      If you're running multiple environments with the same services in them
+      this allows you to differentiate between them.

=== added directory 'files'
=== added directory 'files/nrpe-external-master'
=== added file 'files/nrpe-external-master/check_swift_service'
--- files/nrpe-external-master/check_swift_service	1970-01-01 00:00:00 +0000
+++ files/nrpe-external-master/check_swift_service	2014-11-12 04:10:38 +0000
@@ -0,0 +1,25 @@
+#!/bin/sh
+
+#
+# check_swift_service --- exactly what it says on the tin
+#
+# Copyright 2013 Canonical Ltd.
+#
+# Authors:
+#  Paul Collins <[e-mail]>
+#
+
+STATUS=$(sudo -u swift swift-init status "$1" 2>&1)
+
+case $? in
+    0)
+	echo "OK: ${STATUS}"
+	exit 0
+	;;
+    *)
+	echo "CRITICAL: ${STATUS}"
+	exit 2
+	;;
+esac
+
+exit 1

=== added file 'files/nrpe-external-master/check_swift_storage.py'
--- files/nrpe-external-master/check_swift_storage.py	1970-01-01 00:00:00 +0000
+++ files/nrpe-external-master/check_swift_storage.py	2014-11-12 04:10:38 +0000
@@ -0,0 +1,136 @@
+#!/usr/bin/env python
+
+# Copyright (C) 2014 Canonical
+# All Rights Reserved
+# Author: Jacek Nykis
+
+import sys
+import json
+import urllib2
+import argparse
+import hashlib
+import datetime
+
+STATUS_OK = 0
+STATUS_WARN = 1
+STATUS_CRIT = 2
+STATUS_UNKNOWN = 3
+
+
+def generate_md5(filename):
+    with open(filename, 'rb') as f:
+        md5 = hashlib.md5()
+        buffer = f.read(2 ** 20)
+        while buffer:
+            md5.update(buffer)
+            buffer = f.read(2 ** 20)
+    return md5.hexdigest()
+
+
+def check_md5(base_url):
+    url = base_url + "ringmd5"
+    ringfiles = ["/etc/swift/object.ring.gz",
+                 "/etc/swift/account.ring.gz",
+                 "/etc/swift/container.ring.gz"]
+    results = []
+    try:
+        data = urllib2.urlopen(url).read()
+        j = json.loads(data)
+    except urllib2.URLError:
+        return [(STATUS_UNKNOWN, "Can't open url: {}".format(url))]
+    except ValueError:
+        return [(STATUS_UNKNOWN, "Can't parse status data")]
+
+    for ringfile in ringfiles:
+        try:
+            if generate_md5(ringfile) != j[ringfile]:
+                results.append((STATUS_CRIT,
+                        "Ringfile {} MD5 sum mismatch".format(ringfile)))
+        except IOError:
+            results.append(
+                (STATUS_UNKNOWN, "Can't open ringfile {}".format(ringfile)))
+    if results:
+        return results
+    else:
+        return [(STATUS_OK, "OK")]
+
+
+def check_replication(base_url, limits):
+    types = ["account", "object", "container"]
+    results = []
+    for repl in types:
+        url = base_url + "replication/" + repl
+        try:
+            data = urllib2.urlopen(url).read()
+            j = json.loads(data)
+        except urllib2.URLError:
+            results.append((STATUS_UNKNOWN, "Can't open url: {}".format(url)))
+            continue
+        except ValueError:
+            results.append((STATUS_UNKNOWN, "Can't parse status data"))
+            continue
+
+        if "object_replication_last" in j:
+            repl_last = datetime.datetime.fromtimestamp(j["object_replication_last"])
+        else:
+            repl_last = datetime.datetime.fromtimestamp(j["replication_last"])
+        delta = datetime.datetime.now() - repl_last
+        if delta.seconds >= limits[1]:
+            results.append((STATUS_CRIT,
+                "'{}' replication lag is {} seconds".format(repl, delta.seconds)))
+        elif delta.seconds >= limits[0]:
+            results.append((STATUS_WARN,
+                "'{}' replication lag is {} seconds".format(repl, delta.seconds)))
+        if "replication_stats" in j:
+            errors = j["replication_stats"]["failure"]
+            if errors >= limits[3]:
+                results.append(
+                    (STATUS_CRIT, "{} replication failures".format(errors)))
+            elif errors >= limits[2]:
+                results.append(
+                    (STATUS_WARN, "{} replication failures".format(errors)))
+    if results:
+        return results
+    else:
+        return [(STATUS_OK, "OK")]
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description='Check swift-storage health')
+    parser.add_argument('-H', '--host', dest='host', default='localhost',
+        help='Hostname to query')
+    parser.add_argument('-p', '--port', dest='port', default='6000',
+        type=int, help='Port number')
+    parser.add_argument('-r', '--replication', dest='check_replication',
+        type=int, nargs=4, help='Check replication status',
+        metavar=('lag_warn', 'lag_crit', 'failures_warn', 'failures_crit'))
+    parser.add_argument('-m', '--md5', dest='check_md5', action='store_true',
+        help='Compare server rings md5sum with local copy')
+    args = parser.parse_args()
+
+    if not args.check_replication and not args.check_md5:
+        print "You must use -r or -m switch"
+        sys.exit(STATUS_UNKNOWN)
+
+    base_url = "http://{}:{}/recon/".format(args.host, args.port)
+    results = []
+    if args.check_replication:
+        results.extend(check_replication(base_url, args.check_replication))
+    if args.check_md5:
+        results.extend(check_md5(base_url))
+
+    crits = ';'.join([i[1] for i in results if i[0] == STATUS_CRIT])
+    warns = ';'.join([i[1] for i in results if i[0] == STATUS_WARN])
+    unknowns = ';'.join([i[1] for i in results if i[0] == STATUS_UNKNOWN])
+    if crits:
+        print "CRITICAL: " + crits
+        sys.exit(STATUS_CRIT)
+    elif warns:
+        print "WARNING: " + warns
+        sys.exit(STATUS_WARN)
+    elif unknowns:
+        print "UNKNOWN: " + unknowns
+        sys.exit(STATUS_UNKNOWN)
+    else:
+        print "OK"
+        sys.exit(0)

=== added directory 'files/sudo'
=== added file 'files/sudo/swift-storage'
--- files/sudo/swift-storage	1970-01-01 00:00:00 +0000
+++ files/sudo/swift-storage	2014-11-12 04:10:38 +0000
@@ -0,0 +1,1 @@
+nagios ALL=(swift) NOPASSWD:/usr/bin/swift-init status *

=== added directory 'hooks/charmhelpers/contrib/charmsupport'
=== added file 'hooks/charmhelpers/contrib/charmsupport/__init__.py'
=== added file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py'
--- hooks/charmhelpers/contrib/charmsupport/nrpe.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/charmsupport/nrpe.py	2014-11-12 04:10:38 +0000
@@ -0,0 +1,219 @@
+"""Compatibility with the nrpe-external-master charm"""
+# Copyright 2012 Canonical Ltd.
+#
+# Authors:
+#  Matthew Wedgwood <[e-mail]>
+
+import subprocess
+import pwd
+import grp
+import os
+import re
+import shlex
+import yaml
+
+from charmhelpers.core.hookenv import (
+    config,
+    local_unit,
+    log,
+    relation_ids,
+    relation_set,
+)
+
+from charmhelpers.core.host import service
+
+# This module adds compatibility with the nrpe-external-master and plain nrpe
+# subordinate charms. To use it in your charm:
+#
+# 1. Update metadata.yaml
+#
+#   provides:
+#     (...)
+#     nrpe-external-master:
+#       interface: nrpe-external-master
+#       scope: container
+#
+#   and/or
+#
+#   provides:
+#     (...)
+#     local-monitors:
+#       interface: local-monitors
+#       scope: container
+
+#
+# 2. Add the following to config.yaml
+#
+#    nagios_context:
+#      default: "juju"
+#      type: string
+#      description: |
+#        Used by the nrpe subordinate charms.
+#        A string that will be prepended to instance name to set the host name
+#        in nagios. So for instance the hostname would be something like:
+#            juju-myservice-0
+#        If you're running multiple environments with the same services in them
+#        this allows you to differentiate between them.
+#
+# 3. Add custom checks (Nagios plugins) to files/nrpe-external-master
+#
+# 4. Update your hooks.py with something like this:
+#
+#    from charmsupport.nrpe import NRPE
+#    (...)
+#    def update_nrpe_config():
+#        nrpe_compat = NRPE()
+#        nrpe_compat.add_check(
+#            shortname = "myservice",
+#            description = "Check MyService",
+#            check_cmd = "check_http -w 2 -c 10 http://localhost"
+#            )
+#        nrpe_compat.add_check(
+#            "myservice_other",
+#            "Check for widget failures",
+#            check_cmd = "/srv/myapp/scripts/widget_check"
+#            )
+#        nrpe_compat.write()
+#
+#    def config_changed():
+#        (...)
+#        update_nrpe_config()
+#
+#    def nrpe_external_master_relation_changed():
+#        update_nrpe_config()
+#
+#    def local_monitors_relation_changed():
+#        update_nrpe_config()
+#
+# 5. ln -s hooks.py nrpe-external-master-relation-changed
+#    ln -s hooks.py local-monitors-relation-changed
+
+
+class CheckException(Exception):
+    pass
+
+
+class Check(object):
+    shortname_re = '[A-Za-z0-9-_]+$'
+    service_template = ("""
+#---------------------------------------------------
+# This file is Juju managed
+#---------------------------------------------------
+define service {{
+    use                             active-service
+    host_name                       {nagios_hostname}
+    service_description             {nagios_hostname}[{shortname}] """
+                        """{description}
+    check_command                   check_nrpe!{command}
+    servicegroups                   {nagios_servicegroup}
+}}
+""")
+
+    def __init__(self, shortname, description, check_cmd):
+        super(Check, self).__init__()
+        # XXX: could be better to calculate this from the service name
+        if not re.match(self.shortname_re, shortname):
+            raise CheckException("shortname must match {}".format(
+                Check.shortname_re))
+        self.shortname = shortname
+        self.command = "check_{}".format(shortname)
+        # Note: a set of invalid characters is defined by the
+        # Nagios server config
+        # The default is: illegal_object_name_chars=`~!$%^&*"|'<>?,()=
+        self.description = description
+        self.check_cmd = self._locate_cmd(check_cmd)
+
+    def _locate_cmd(self, check_cmd):
+        search_path = (
+            '/usr/lib/nagios/plugins',
+            '/usr/local/lib/nagios/plugins',
+        )
+        parts = shlex.split(check_cmd)
+        for path in search_path:
+            if os.path.exists(os.path.join(path, parts[0])):
+                command = os.path.join(path, parts[0])
+                if len(parts) > 1:
+                    command += " " + " ".join(parts[1:])
+                return command
+        log('Check command not found: {}'.format(parts[0]))
+        return ''
+
+    def write(self, nagios_context, hostname):
+        nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format(
+            self.command)
+        with open(nrpe_check_file, 'w') as nrpe_check_config:
+            nrpe_check_config.write("# check {}\n".format(self.shortname))
+            nrpe_check_config.write("command[{}]={}\n".format(
+                self.command, self.check_cmd))
+
+        if not os.path.exists(NRPE.nagios_exportdir):
+            log('Not writing service config as {} is not accessible'.format(
+                NRPE.nagios_exportdir))
+        else:
+            self.write_service_config(nagios_context, hostname)
+
+    def write_service_config(self, nagios_context, hostname):
+        for f in os.listdir(NRPE.nagios_exportdir):
+            if re.search('.*{}.cfg'.format(self.command), f):
+                os.remove(os.path.join(NRPE.nagios_exportdir, f))
+
+        templ_vars = {
+            'nagios_hostname': hostname,
+            'nagios_servicegroup': nagios_context,
+            'description': self.description,
+            'shortname': self.shortname,
+            'command': self.command,
+        }
+        nrpe_service_text = Check.service_template.format(**templ_vars)
+        nrpe_service_file = '{}/service__{}_{}.cfg'.format(
+            NRPE.nagios_exportdir, hostname, self.command)
+        with open(nrpe_service_file, 'w') as nrpe_service_config:
+            nrpe_service_config.write(str(nrpe_service_text))
+
+    def run(self):
+        subprocess.call(self.check_cmd)
+
+
+class NRPE(object):
+    nagios_logdir = '/var/log/nagios'
+    nagios_exportdir = '/var/lib/nagios/export'
+    nrpe_confdir = '/etc/nagios/nrpe.d'
+
+    def __init__(self, hostname=None):
+        super(NRPE, self).__init__()
+        self.config = config()
+        self.nagios_context = self.config['nagios_context']
+        self.unit_name = local_unit().replace('/', '-')
+        if hostname:
+            self.hostname = hostname
+        else:
+            self.hostname = "{}-{}".format(self.nagios_context, self.unit_name)
+        self.checks = []
+
+    def add_check(self, *args, **kwargs):
+        self.checks.append(Check(*args, **kwargs))
+
+    def write(self):
+        try:
+            nagios_uid = pwd.getpwnam('nagios').pw_uid
+            nagios_gid = grp.getgrnam('nagios').gr_gid
+        except:
+            log("Nagios user not set up, nrpe checks not updated")
+            return
+
+        if not os.path.exists(NRPE.nagios_logdir):
+            os.mkdir(NRPE.nagios_logdir)
+            os.chown(NRPE.nagios_logdir, nagios_uid, nagios_gid)
+
+        nrpe_monitors = {}
+        monitors = {"monitors": {"remote": {"nrpe": nrpe_monitors}}}
+        for nrpecheck in self.checks:
+            nrpecheck.write(self.nagios_context, self.hostname)
+            nrpe_monitors[nrpecheck.shortname] = {
+                "command": nrpecheck.command,
+            }
+
+        service('restart', 'nagios-nrpe-server')
+
+        for rid in relation_ids("local-monitors"):
+            relation_set(relation_id=rid, monitors=yaml.dump(monitors))

=== added file 'hooks/charmhelpers/contrib/charmsupport/rsync.py'
--- hooks/charmhelpers/contrib/charmsupport/rsync.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/charmsupport/rsync.py	2014-11-12 04:10:38 +0000
@@ -0,0 +1,32 @@
+"""
+Support for rsyncd.conf and using fragments dropped inside /etc/rsync-juju.d
+"""
+import os
+
+from charmhelpers.core.host import (
+    mkdir,
+)
+
+def setup_rsync():
+    '''
+    Ensure all directories required for rsync exist with correct permissions.
+    '''
+    root_dirs = [
+        '/etc/rsync-juju.d',
+    ]
+    [mkdir(d, owner='root', group='root') for d in root_dirs
+     if not os.path.isdir(d)]
+
+    rsyncd_base = """uid = nobody
+gid = nogroup
+pid file = /var/run/rsyncd.pid
+syslog facility = daemon
+socket options = SO_KEEPALIVE
+
+&include /etc/rsync-juju.d
+"""
+
+    f = open('/etc/rsyncd.conf','w')
+    f.write(rsyncd_base)
+    f.close()
+

=== added file 'hooks/charmhelpers/contrib/charmsupport/volumes.py'
--- hooks/charmhelpers/contrib/charmsupport/volumes.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/charmsupport/volumes.py	2014-11-12 04:10:38 +0000
@@ -0,0 +1,156 @@
+'''
+Functions for managing volumes in juju units. One volume is supported per unit.
+Subordinates may have their own storage, provided it is on its own partition.
+
+Configuration stanzas:
+  volume-ephemeral:
+    type: boolean
+    default: true
+    description: >
+      If false, a volume is mounted as sepecified in "volume-map"
+      If true, ephemeral storage will be used, meaning that log data
+         will only exist as long as the machine. YOU HAVE BEEN WARNED.
+  volume-map:
+    type: string
+    default: {}
+    description: >
+      YAML map of units to device names, e.g:
+        "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }"
+      Service units will raise a configure-error if volume-ephemeral
+      is 'true' and no volume-map value is set. Use 'juju set' to set a
+      value and 'juju resolved' to complete configuration.
+
+Usage:
+    from charmsupport.volumes import configure_volume, VolumeConfigurationError
+    from charmsupport.hookenv import log, ERROR
+    def post_mount_hook():
+        stop_service('myservice')
+    def post_mount_hook():
+        start_service('myservice')
+
+    if __name__ == '__main__':
+        try:
+            configure_volume(before_change=pre_mount_hook,
+                             after_change=post_mount_hook)
+        except VolumeConfigurationError:
+            log('Storage could not be configured', ERROR)
+'''
+
+# XXX: Known limitations
+# - fstab is neither consulted nor updated
+
+import os
+from charmhelpers.core import hookenv
+from charmhelpers.core import host
+import yaml
+
+
+MOUNT_BASE = '/srv/juju/volumes'
+
+
+class VolumeConfigurationError(Exception):
+    '''Volume configuration data is missing or invalid'''
+    pass
+
+
+def get_config():
+    '''Gather and sanity-check volume configuration data'''
+    volume_config = {}
+    config = hookenv.config()
+
+    errors = False
+
+    if config.get('volume-ephemeral') in (True, 'True', 'true', 'Yes', 'yes'):
+        volume_config['ephemeral'] = True
+    else:
+        volume_config['ephemeral'] = False
+
+    try:
+        volume_map = yaml.safe_load(config.get('volume-map', '{}'))
+    except yaml.YAMLError as e:
+        hookenv.log("Error parsing YAML volume-map: {}".format(e),
+                    hookenv.ERROR)
+        errors = True
+    if volume_map is None:
+        # probably an empty string
+        volume_map = {}
+    elif not isinstance(volume_map, dict):
+        hookenv.log("Volume-map should be a dictionary, not {}".format(
+            type(volume_map)))
+        errors = True
+
+    volume_config['device'] = volume_map.get(os.environ['JUJU_UNIT_NAME'])
+    if volume_config['device'] and volume_config['ephemeral']:
+        # asked for ephemeral storage but also defined a volume ID
+        hookenv.log('A volume is defined for this unit, but ephemeral '
+                    'storage was requested', hookenv.ERROR)
+        errors = True
+    elif not volume_config['device'] and not volume_config['ephemeral']:
+        # asked for permanent storage but did not define volume ID
+        hookenv.log('Ephemeral storage was requested, but there is no volume '
+                    'defined for this unit.', hookenv.ERROR)
+        errors = True
+
+    unit_mount_name = hookenv.local_unit().replace('/', '-')
+    volume_config['mountpoint'] = os.path.join(MOUNT_BASE, unit_mount_name)
+
+    if errors:
+        return None
+    return volume_config
+
+
+def mount_volume(config):
+    if os.path.exists(config['mountpoint']):
+        if not os.path.isdir(config['mountpoint']):
+            hookenv.log('Not a directory: {}'.format(config['mountpoint']))
+            raise VolumeConfigurationError()
+    else:
+        host.mkdir(config['mountpoint'])
+    if os.path.ismount(config['mountpoint']):
+        unmount_volume(config)
+    if not host.mount(config['device'], config['mountpoint'], persist=True):
+        raise VolumeConfigurationError()
+
+
+def unmount_volume(config):
+    if os.path.ismount(config['mountpoint']):
+        if not host.umount(config['mountpoint'], persist=True):
+            raise VolumeConfigurationError()
+
+
+def managed_mounts():
+    '''List of all mounted managed volumes'''
+    return filter(lambda mount: mount[0].startswith(MOUNT_BASE), host.mounts())
+
+
+def configure_volume(before_change=lambda: None, after_change=lambda: None):
+    '''Set up storage (or don't) according to the charm's volume configuration.
+       Returns the mount point or "ephemeral". before_change and after_change
+       are optional functions to be called if the volume configuration changes.
+    '''
+
+    config = get_config()
+    if not config:
+        hookenv.log('Failed to read volume configuration', hookenv.CRITICAL)
+        raise VolumeConfigurationError()
+
+    if config['ephemeral']:
+        if os.path.ismount(config['mountpoint']):
+            before_change()
+            unmount_volume(config)
+            after_change()
+        return 'ephemeral'
+    else:
+        # persistent storage
+        if os.path.ismount(config['mountpoint']):
+            mounts = dict(managed_mounts())
+            if mounts.get(config['mountpoint']) != config['device']:
+                before_change()
+                unmount_volume(config)
+                mount_volume(config)
+                after_change()
+        else:
+            before_change()
+            mount_volume(config)
+            after_change()
+        return config['mountpoint']

=== added symlink 'hooks/nrpe-external-master-relation-changed'
=== target is u'swift_storage_hooks.py'
=== added symlink 'hooks/nrpe-external-master-relation-joined'
=== target is u'swift_storage_hooks.py'
=== modified file 'hooks/swift_storage_hooks.py'
--- hooks/swift_storage_hooks.py	2014-10-07 08:37:20 +0000
+++ hooks/swift_storage_hooks.py	2014-11-12 04:10:38 +0000
@@ -22,6 +22,8 @@
     log,
     relation_get,
     relation_set,
+    relations_of_type,
+    local_unit,
 )
 
 from charmhelpers.fetch import (
@@ -29,7 +31,7 @@
     apt_update,
     filter_installed_packages
 )
-from charmhelpers.core.host import restart_on_change
+from charmhelpers.core.host import restart_on_change, rsync
 from charmhelpers.payload.execd import execd_preinstall
 
 from charmhelpers.contrib.openstack.utils import (
@@ -39,9 +41,29 @@
 from charmhelpers.contrib.network.ip import (
     get_ipv6_addr
 )
+from swift_storage_utils import (
+    PACKAGES,
+    RESTART_MAP,
+    SWIFT_SVCS,
+    determine_block_devices,
+    do_openstack_upgrade,
+    ensure_swift_directories,
+    fetch_swift_rings,
+    register_configs,
+    save_script_rc,
+    setup_storage,
+    concat_rsync_fragments,
+)
+from charmhelpers.contrib.charmsupport.nrpe import NRPE
+
+from charmhelpers.contrib.charmsupport.rsync import setup_rsync
+
+from distutils.dir_util import mkpath
 
 hooks = Hooks()
 CONFIGS = register_configs()
+NAGIOS_PLUGINS = '/usr/local/lib/nagios/plugins'
+SUDOERS_D = '/etc/sudoers.d'
 
 
 @hooks.hook()
@@ -60,15 +82,22 @@
     if config('prefer-ipv6'):
         assert_charm_supports_ipv6()
 
+    ensure_swift_directories()
+    setup_rsync()
+
     if openstack_upgrade_available('swift'):
         do_openstack_upgrade(configs=CONFIGS)
     CONFIGS.write_all()
+
     save_script_rc()
+    if relations_of_type('nrpe-external-master'):
+    	update_nrpe_config()
 
 
 @hooks.hook('upgrade-charm')
 def upgrade_charm():
     apt_install(filter_installed_packages(PACKAGES), fatal=True)
+    update_nrpe_config()
 
 
 @hooks.hook()
@@ -100,6 +129,53 @@
     fetch_swift_rings(rings_url)
 
 
[e-mail]('nrpe-external-master-relation-joined')
[e-mail]('nrpe-external-master-relation-changed')
+def update_nrpe_config():
+    log('Refreshing nrpe checks')
+    if not os.path.exists(NAGIOS_PLUGINS):
+        mkpath(NAGIOS_PLUGINS)
+    rsync(os.path.join(os.getenv('CHARM_DIR'), 'files', 'nrpe-external-master',
+                       'check_swift_storage.py'),
+          os.path.join(NAGIOS_PLUGINS, 'check_swift_storage.py'))
+    rsync(os.path.join(os.getenv('CHARM_DIR'), 'files', 'nrpe-external-master',
+                       'check_swift_service'),
+          os.path.join(NAGIOS_PLUGINS, 'check_swift_service'))
+    rsync(os.path.join(os.getenv('CHARM_DIR'), 'files', 'sudo',
+                       'swift-storage'),
+          os.path.join(SUDOERS_D, 'swift-storage'))
+    # Find out if nrpe set nagios_hostname
+    hostname = None
+    host_context = None
+    for rel in relations_of_type('nrpe-external-master'):
+        if 'nagios_hostname' in rel:
+            hostname = rel['nagios_hostname']
+            host_context = rel['nagios_host_context']
+            break
+    nrpe = NRPE(hostname=hostname)
+
+    if host_context:
+        current_unit = "%s:%s" % (host_context, local_unit())
+    else:
+        current_unit = local_unit()
+
+    # check the rings and replication
+    nrpe.add_check(
+        shortname='swift_storage',
+        description='Check swift storage ring hashes and replication {%s}' % current_unit,
+        check_cmd='check_swift_storage.py {}'.format(
+            config('nagios-check-params'))
+    )
+    # check services are running
+    for service in SWIFT_SVCS:
+        nrpe.add_check(
+            shortname=service,
+            description='service {%s}' % current_unit,
+            check_cmd = 'check_swift_service %s' % service,
+            )
+    nrpe.write()
+
+
 def main():
     try:
         hooks.execute(sys.argv)

=== modified file 'hooks/swift_storage_utils.py'
--- hooks/swift_storage_utils.py	2014-10-24 14:29:07 +0000
+++ hooks/swift_storage_utils.py	2014-11-12 04:10:38 +0000
@@ -75,8 +75,24 @@
     'swift-object-updater', 'swift-object-replicator'
 ]
 
+SWIFT_SVCS = [
+    'account-auditor',
+    'account-reaper',
+    'account-replicator',
+    'account-server',
+    'container-auditor',
+    'container-replicator',
+    'container-server',
+    'container-sync',
+    'container-updater',
+    'object-auditor',
+    'object-replicator',
+    'object-server',
+    'object-updater',
+    ]
+
 RESTART_MAP = {
-    '/etc/rsyncd.conf': ['rsync'],
+    '/etc/rsync-juju.d/050-swift-storage.conf': ['rsync'],
     '/etc/swift/account-server.conf': ACCOUNT_SVCS,
     '/etc/swift/container-server.conf': CONTAINER_SVCS,
     '/etc/swift/object-server.conf': OBJECT_SVCS,
@@ -104,8 +120,8 @@
                                           openstack_release=release)
     configs.register('/etc/swift/swift.conf',
                      [SwiftStorageContext()])
-    configs.register('/etc/rsyncd.conf',
-                     [RsyncContext()])
+    configs.register('/etc/rsync-juju.d/050-swift-storage.conf',
+                     [RsyncContext(), SwiftStorageServerContext()])
     for server in ['account', 'object', 'container']:
         configs.register('/etc/swift/%s-server.conf' % server,
                          [SwiftStorageServerContext(),
@@ -237,3 +253,15 @@
     if lsb_release()['DISTRIB_CODENAME'].lower() < "trusty":
         raise Exception("IPv6 is not supported in the charms for Ubuntu "
                         "versions less than Trusty 14.04")
+
+
+def concat_rsync_fragments():
+    log('Concatenating rsyncd.d fragments')
+    rsyncd_dir = '/etc/rsyncd.d'
+    rsyncd_conf = ""
+    for filename in sorted(os.listdir(rsyncd_dir)):
+        with open(os.path.join(rsyncd_dir, filename), 'r') as fragment:
+            rsyncd_conf += fragment.read()
+    with open('/etc/rsyncd.conf', 'w') as f:
+        f.write(rsyncd_conf)
+

=== modified file 'metadata.yaml'
--- metadata.yaml	2013-07-11 20:45:19 +0000
+++ metadata.yaml	2014-11-12 04:10:38 +0000
@@ -6,5 +6,8 @@
 categories:
     - file-servers
 provides:
+  nrpe-external-master:
+    interface: nrpe-external-master
+    scope: container
   swift-storage:
     interface: swift

=== added file 'templates/001-baseconfig'
--- templates/001-baseconfig	1970-01-01 00:00:00 +0000
+++ templates/001-baseconfig	2014-11-12 04:10:38 +0000
@@ -0,0 +1,6 @@
+uid = nobody
+gid = nogroup
+pid file = /var/run/rsyncd.pid
+syslog facility = daemon
+socket options = SO_KEEPALIVE
+

=== added file 'templates/050-swift-storage.conf'
--- templates/050-swift-storage.conf	1970-01-01 00:00:00 +0000
+++ templates/050-swift-storage.conf	2014-11-12 04:10:38 +0000
@@ -0,0 +1,24 @@
+[account]
+uid = swift
+guid = swift
+max connections = {{ account_max_connections }}
+path = /srv/node/
+read only = false
+lock file = /var/lock/account.lock
+
+[container]
+uid = swift
+guid = swift
+max connections = {{ container_max_connections }}
+path = /srv/node/
+read only = false
+lock file = /var/lock/container.lock
+
+[object]
+uid = swift
+guid = swift
+max connections = {{ object_max_connections }}
+path = /srv/node/
+read only = false
+lock file = /var/lock/object.lock
+

=== removed file 'templates/rsyncd.conf'
--- templates/rsyncd.conf	2014-09-12 09:17:27 +0000
+++ templates/rsyncd.conf	1970-01-01 00:00:00 +0000
@@ -1,23 +0,0 @@
-uid = swift
-gid = swift
-log file = /var/log/rsyncd.log
-pid file = /var/run/rsyncd.pid
-address = {{ local_ip }}
-
-[account]
-max connections = {{ account_max_connections }}
-path = /srv/node/
-read only = false
-lock file = /var/lock/account.lock
-
-[container]
-max connections = {{ container_max_connections }}
-path = /srv/node/
-read only = false
-lock file = /var/lock/container.lock
-
-[object]
-max connections = {{ object_max_connections }}
-path = /srv/node/
-read only = false
-lock file = /var/lock/object.lock

=== modified file 'unit_tests/test_swift_storage_relations.py'
--- unit_tests/test_swift_storage_relations.py	2014-10-07 08:37:20 +0000
+++ unit_tests/test_swift_storage_relations.py	2014-11-12 04:10:38 +0000
@@ -1,6 +1,6 @@
 from mock import patch, MagicMock
 
-from test_utils import CharmTestCase
+from test_utils import CharmTestCase, patch_open
 
 import swift_storage_utils as utils
 
@@ -21,6 +21,7 @@
     'log',
     'relation_set',
     'relation_get',
+    'relations_of_type',
     # charmhelpers.core.host
     'apt_update',
     'apt_install',
@@ -62,13 +63,19 @@
 
     def test_config_changed_no_upgrade_available(self):
         self.openstack_upgrade_available.return_value = False
-        hooks.config_changed()
+        self.relations_of_type.return_value = False
+        with patch_open() as (_open, _file):
+            _file.read.return_value = "foo"
+            hooks.config_changed()
         self.assertFalse(self.do_openstack_upgrade.called)
         self.assertTrue(self.CONFIGS.write_all.called)
 
     def test_config_changed_upgrade_available(self):
         self.openstack_upgrade_available.return_value = True
-        hooks.config_changed()
+        self.relations_of_type.return_value = False
+        with patch_open() as (_open, _file):
+            _file.read.return_value = "foo"
+            hooks.config_changed()
         self.assertTrue(self.do_openstack_upgrade.called)
         self.assertTrue(self.CONFIGS.write_all.called)
 

=== modified file 'unit_tests/test_swift_storage_utils.py'
--- unit_tests/test_swift_storage_utils.py	2014-10-24 14:33:12 +0000
+++ unit_tests/test_swift_storage_utils.py	2014-11-12 04:10:38 +0000
@@ -85,6 +85,7 @@
             ex_dirs = [
                 call('/etc/swift', owner='swift', group='swift'),
                 call('/var/cache/swift', owner='swift', group='swift'),
+                call('/etc/rsyncd.d', owner='root', group='root'),
                 call('/srv/node', owner='swift', group='swift')
             ]
         self.assertEquals(ex_dirs, self.mkdir.call_args_list)