Merge lp:~brad-marshall/charms/trusty/ceilometer/add-nrpe-checks into lp:~openstack-charmers-archive/charms/trusty/ceilometer/trunk

[Brad Marshall]

Adds nrpe-external-master interface and adds basic nrpe checks.

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_lint_check #992 trusty-ceilometer for brad-marshall mp241498
    LINT FAIL: lint-test failed

LINT Results (max last 5 lines):
ERROR:root:Make target returned non-zero.
  hooks/ceilometer_hooks.py:146:80: E501 line too long (92 > 79 characters)
  hooks/ceilometer_hooks.py:174:22: E251 unexpected spaces around keyword / parameter equals
  hooks/ceilometer_hooks.py:174:24: E251 unexpected spaces around keyword / parameter equals
  make: *** [lint] Error 1

Full lint test output: http://paste.ubuntu.com/8955755/
Build: http://10.98.191.181:8080/job/charm_lint_check/992/

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_unit_test #827 trusty-ceilometer for brad-marshall mp241498
    UNIT FAIL: unit-test failed

UNIT Results (max last 5 lines):
  hooks/ceilometer_utils 58 1 98% 100
  TOTAL 180 24 87%
  Ran 25 tests in 0.729s
  FAILED (errors=3)
  make: *** [test] Error 1

Full unit test output: http://paste.ubuntu.com/8955757/
Build: http://10.98.191.181:8080/job/charm_unit_test/827/

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_amulet_test #372 trusty-ceilometer for brad-marshall mp241498
    AMULET FAIL: amulet-test missing

AMULET Results (max last 5 lines):
INFO:root:Workspace dir: /var/lib/jenkins/workspace/charm_amulet_test
INFO:root:Reading file: Makefile
INFO:root:Searching for: ['@juju test']
INFO:root:Search string not found in makefile target commands.
ERROR:root:No make target was executed.

Full amulet test output: http://paste.ubuntu.com/8955875/
Build: http://10.98.191.181:8080/job/charm_amulet_test/372/

[Liam Young (gnuoy)]

Thank you for the mp, the nagios checks are sorely needed. It looks fine, there just a few things it would be good to get fixed up.

The list of services that comprise a ceilometer deployment are already compiled as part of the service context in ceilometer_utils.py. This includes logic to adjust the list depending on what Openstack release is being deployed. I think this mechanism should be used rather than defining a new list directly in update_nrpe_config()

That being said, it looks like the existing charm has a list of new icehouse packages which are being added to the package list in ceilometer_utils.py but the corresponding services are not being added to the CONFIG_FILES OrderedDict. This means that the ceilometer-alarm* and ceilometer-agent-notification services are not going to be restarted on changes to ceilometer.conf

So, what I think is needed is:
1) Steal the services() method from nova-cloud-controller and use that to get a list of services in update_nrpe_config()
2) Define a list of ICEHOUSE_SERVICES (probably exactly the same as ICEHOUSE_PACKAGES) and conditionally add (depending on ostack release) to the CEILOMETER_CONF service in register_configs():
    if (get_os_codename_install_source(config('openstack-origin'))
            >= 'icehouse'):
        CONFIG_FILES[CEILOMETER_CONF]['services'] = CONFIG_FILES[CEILOMETER_CONF]['services'] + ICEHOUSE_SERVICES

review: Needs Fixing

[Liam Young (gnuoy)]

Also, could you move the check_upstart_job into charmhelpers as it seems to be common across these mps?

[lp:~brad-marshall/charms/trusty/ceilometer/add-nrpe-checks]

62. By Brad Marshall on 2014-11-17

[bradm] Tweaked nagios checks to use functions to pull out services, added checks for sysv init style daemons, added in icehouse daemons, ran pep8 over the whole thing

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_unit_test #917 trusty-ceilometer for brad-marshall mp241498
    UNIT FAIL: unit-test failed

UNIT Results (max last 5 lines):
  hooks/ceilometer_utils 66 6 91% 101, 111, 136-139
  TOTAL 200 41 80%
  Ran 25 tests in 0.747s
  FAILED (errors=3)
  make: *** [test] Error 1

Full unit test output: http://paste.ubuntu.com/9051427/
Build: http://10.98.191.181:8080/job/charm_unit_test/917/

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_amulet_test #425 trusty-ceilometer for brad-marshall mp241498
    AMULET FAIL: amulet-test missing

AMULET Results (max last 5 lines):
INFO:root:Workspace dir: /var/lib/jenkins/workspace/charm_amulet_test
INFO:root:Reading file: Makefile
INFO:root:Searching for: ['@juju test']
INFO:root:Search string not found in makefile target commands.
ERROR:root:No make target was executed.

Full amulet test output: http://paste.ubuntu.com/9051426/
Build: http://10.98.191.181:8080/job/charm_amulet_test/425/

[uosci-testing-bot (uosci-testing-bot)]

UOSCI bot says:
charm_lint_check #1083 trusty-ceilometer for brad-marshall mp241498
    LINT OK: passed

LINT Results (max last 5 lines):
  I: config.yaml: option os-internal-network has no default value
  I: config.yaml: option os-admin-network has no default value
  I: config.yaml: option ssl_ca has no default value
  I: config.yaml: option ssl_cert has no default value
  I: config.yaml: option os-public-network has no default value

Full lint test output: http://paste.ubuntu.com/9051428/
Build: http://10.98.191.181:8080/job/charm_lint_check/1083/

[lp:~brad-marshall/charms/trusty/ceilometer/add-nrpe-checks]

63. By Brad Marshall on 2014-11-17

[bradm] Removed puppet header from nagios_plugin module

64. By Brad Marshall on 2014-11-18

[bradm] Removed nagios check files that were moved to nrpe-external-master charm

Unmerged revisions

=== modified file 'charm-helpers.yaml'
--- charm-helpers.yaml	2014-07-24 10:23:25 +0000
+++ charm-helpers.yaml	2014-11-17 02:28:34 +0000
@@ -7,3 +7,4 @@
     - contrib.hahelpers
     - contrib.storage.linux
     - contrib.network.ip
+    - contrib.charmsupport

=== modified file 'config.yaml'
--- config.yaml	2014-10-01 15:22:55 +0000
+++ config.yaml	2014-11-17 02:28:34 +0000
@@ -58,6 +58,16 @@
     description: |
       SSL CA to use with the certificate and key provided - this is only
       required if you are providing a privately signed ssl_cert and ssl_key.
+  nagios_context:
+    default: "juju"
+    type: string
+    description: |
+      Used by the nrpe-external-master subordinate charm.
+      A string that will be prepended to instance name to set the host name
+      in nagios. So for instance the hostname would be something like:
+          juju-myservice-0
+      If you're running multiple environments with the same services in them
+      this allows you to differentiate between them.
   # Network configuration options
   # by default all access is over 'private-address'
   os-admin-network:
@@ -84,4 +94,3 @@
       192.168.0.0/24)
       .
       This network will be used for public endpoints.
-

=== added directory 'files'
=== added directory 'files/nrpe-external-master'
=== added file 'files/nrpe-external-master/check_exit_status.pl'
--- files/nrpe-external-master/check_exit_status.pl	1970-01-01 00:00:00 +0000
+++ files/nrpe-external-master/check_exit_status.pl	2014-11-17 02:28:34 +0000
@@ -0,0 +1,189 @@
+#!/usr/bin/perl
+################################################################################
+#                                                                              #
+#  Copyright (C) 2011 Chad Columbus <[e-mail]>                     #
+#                                                                              #
+#   This program is free software; you can redistribute it and/or modify       #
+#   it under the terms of the GNU General Public License as published by       #
+#   the Free Software Foundation; either version 2 of the License, or          #
+#   (at your option) any later version.                                        #
+#                                                                              #
+#   This program is distributed in the hope that it will be useful,            #
+#   but WITHOUT ANY WARRANTY; without even the implied warranty of             #
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the              #
+#   GNU General Public License for more details.                               #
+#                                                                              #
+#   You should have received a copy of the GNU General Public License          #
+#   along with this program; if not, write to the Free Software                #
+#   Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA  #
+#                                                                              #
+################################################################################
+
+use strict;
+use Getopt::Std;
+$| = 1;
+
+my %opts;
+getopts('heronp:s:', \%opts);
+
+my $VERSION = "Version 1.0";
+my $AUTHOR = '(c) 2011 Chad Columbus <[e-mail]>';
+
+# Default values:
+my $script_to_check;
+my $pattern = 'is running';
+my $cmd;
+my $message;
+my $error;
+
+# Exit codes
+my $STATE_OK = 0;
+my $STATE_WARNING = 1;
+my $STATE_CRITICAL = 2;
+my $STATE_UNKNOWN = 3;
+
+# Parse command line options
+if ($opts{'h'} || scalar(%opts) == 0) {
+	&print_help();
+	exit($STATE_OK);
+}
+
+# Make sure scipt is provided:
+if ($opts{'s'} eq '') {
+	# Script to run not provided
+	print "\nYou must provide a script to run. Example: -s /etc/init.d/httpd\n";
+	exit($STATE_UNKNOWN);
+} else {
+	$script_to_check = $opts{'s'};
+}
+
+# Make sure only a-z, 0-9, /, _, and - are used in the script.
+if ($script_to_check =~ /[^a-z0-9\_\-\/\.]/) {
+	# Script contains illegal characters exit.
+	print "\nScript to check can only contain Letters, Numbers, Periods, Underscores, Hyphens, and/or Slashes\n";
+	exit($STATE_UNKNOWN);
+}
+
+# See if script is executable
+if (! -x "$script_to_check") {
+	print "\nIt appears you can't execute $script_to_check, $!\n";
+	exit($STATE_UNKNOWN);
+}
+
+# If a pattern is provided use it:
+if ($opts{'p'} ne '') {
+	$pattern = $opts{'p'};
+}
+
+# If -r run command via sudo as root:
+if ($opts{'r'}) {
+	$cmd = "sudo -n $script_to_check status" . ' 2>&1';
+} else {
+	$cmd = "$script_to_check status" . ' 2>&1';
+}
+
+my $cmd_result = `$cmd`;
+chomp($cmd_result);
+if ($cmd_result =~ /sudo/i) {
+	# This means it could not run the sudo command
+	$message = "$script_to_check CRITICAL - Could not run: 'sudo -n $script_to_check status'. Result is $cmd_result";
+	$error = $STATE_UNKNOWN;
+} else {
+	# Check exitstatus instead of output:
+	if ($opts{'e'} == 1) {
+		if ($? != 0) {
+			# error
+			$message = "$script_to_check CRITICAL - Exit code: $?\.";
+			if ($opts{'o'} == 0) {
+				$message .= " $cmd_result";
+			}
+			$error = $STATE_CRITICAL;
+		} else {
+			# success
+			$message = "$script_to_check OK - Exit code: $?\.";
+			if ($opts{'o'} == 0) {
+				$message .= " $cmd_result";
+			}
+			$error = $STATE_OK;
+		}
+	} else {
+		my $not_check = 1;
+		if ($opts{'n'} == 1) { 
+			$not_check = 0;
+		}
+		if (($cmd_result =~ /$pattern/i) == $not_check) {
+			$message = "$script_to_check OK";
+			if ($opts{'o'} == 0) {
+				$message .= " - $cmd_result";
+			}
+			$error = $STATE_OK;
+		} else {
+			$message = "$script_to_check CRITICAL";
+			if ($opts{'o'} == 0) {
+				$message .= " - $cmd_result";
+			}
+			$error = $STATE_CRITICAL;
+		}
+	}
+}
+
+if ($message eq '') {
+	print "Error: program failed in an unknown way\n";
+	exit($STATE_UNKNOWN);
+}
+
+if ($error) {
+	print "$message\n";
+	exit($error);
+} else {
+	# If we get here we are OK
+	print "$message\n";
+	exit($STATE_OK);
+}
+
+####################################
+# Start Subs:
+####################################
+sub print_help() {
+        print << "EOF";
+Check the output or exit status of a script.
+$VERSION
+$AUTHOR
+
+Options:
+-h
+	Print detailed help screen
+
+-s
+	'FULL PATH TO SCRIPT' (required)
+	This is the script to run, the script is designed to run scripts in the
+	/etc/init.d dir (but can run any script) and will call the script with 
+	a 'status' argument. So if you use another script make sure it will
+	work with /path/script status, example: /etc/init.d/httpd status 
+
+-e
+	This is the "exitstaus" flag, it means check the exit status 
+	code instead of looking for a pattern in the output of the script.
+
+-p 'REGEX'
+	This is a pattern to look for in the output of the script to confirm it
+	is running, default is 'is running', but not all init.d scripts output 
+	(iptables), so you can specify an arbitrary pattern.
+	All patterns are case insensitive.
+
+-n 
+	This is the "NOT" flag, it means not the -p pattern, so if you want to
+	make sure the output of the script does NOT contain -p 'REGEX'
+
+-r
+	This is the "ROOT" flag, it means run as root via sudo. You will need a 
+	line in your /etc/sudoers file like:
+	nagios ALL=(root) NOPASSWD: /etc/init.d/* status
+
+-o
+	This is the "SUPPRESS OUTPUT" flag. Some programs have a long output
+	(like iptables), this flag suppresses that output so it is not printed
+	as a part of the nagios message.
+EOF
+}
+

=== added file 'files/nrpe-external-master/check_status_file.py'
--- files/nrpe-external-master/check_status_file.py	1970-01-01 00:00:00 +0000
+++ files/nrpe-external-master/check_status_file.py	2014-11-17 02:28:34 +0000
@@ -0,0 +1,60 @@
+#!/usr/bin/python
+
+#                                       m
+#  mmmm   m   m  mmmm   mmmm    mmm   mm#mm
+#  #" "#  #   #  #" "#  #" "#  #"  #    #
+#  #   #  #   #  #   #  #   #  #""""    #
+#  ##m#"  "mm"#  ##m#"  ##m#"  "#mm"    "mm
+#  #             #      #
+#  "             "      "
+# This file is managed by puppet.  Do not make local changes.
+
+#
+# Copyright 2014 Canonical Ltd.
+#
+# Author: Jacek Nykis <[e-mail]>
+#
+
+import re
+import nagios_plugin
+
+
+def parse_args():
+    import argparse
+
+    parser = argparse.ArgumentParser(
+        description='Read file and return nagios status based on its content',
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+    parser.add_argument('-f', '--status-file', required=True,
+                        help='Status file path')
+    parser.add_argument('-c', '--critical-text', default='CRITICAL',
+                        help='String indicating critical status')
+    parser.add_argument('-w', '--warning-text', default='WARNING',
+                        help='String indicating warning status')
+    parser.add_argument('-o', '--ok-text', default='OK',
+                        help='String indicating OK status')
+    parser.add_argument('-u', '--unknown-text', default='UNKNOWN',
+                        help='String indicating unknown status')
+    return parser.parse_args()
+
+
+def check_status(args):
+    nagios_plugin.check_file_freshness(args.status_file, 43200)
+
+    with open(args.status_file, "r") as f:
+        content = [l.strip() for l in f.readlines()]
+
+    for line in content:
+        if re.search(args.critical_text, line):
+            raise nagios_plugin.CriticalError(line)
+        elif re.search(args.warning_text, line):
+            raise nagios_plugin.WarnError(line)
+        elif re.search(args.unknown_text, line):
+            raise nagios_plugin.UnknownError(line)
+        else:
+            print line
+
+
+if __name__ == '__main__':
+    args = parse_args()
+    nagios_plugin.try_check(check_status, args)

=== added file 'files/nrpe-external-master/check_upstart_job'
--- files/nrpe-external-master/check_upstart_job	1970-01-01 00:00:00 +0000
+++ files/nrpe-external-master/check_upstart_job	2014-11-17 02:28:34 +0000
@@ -0,0 +1,72 @@
+#!/usr/bin/python
+
+#
+# Copyright 2012, 2013 Canonical Ltd.
+#
+# Author: Paul Collins <[e-mail]>
+#
+# Based on http://www.eurion.net/python-snippets/snippet/Upstart%20service%20status.html
+#
+
+import sys
+
+import dbus
+
+
+class Upstart(object):
+    def __init__(self):
+        self._bus = dbus.SystemBus()
+        self._upstart = self._bus.get_object('com.ubuntu.Upstart',
+                                             '/com/ubuntu/Upstart')
+    def get_job(self, job_name):
+        path = self._upstart.GetJobByName(job_name,
+                                          dbus_interface='com.ubuntu.Upstart0_6')
+        return self._bus.get_object('com.ubuntu.Upstart', path)
+
+    def get_properties(self, job):
+        path = job.GetInstance([], dbus_interface='com.ubuntu.Upstart0_6.Job')
+        instance = self._bus.get_object('com.ubuntu.Upstart', path)
+        return instance.GetAll('com.ubuntu.Upstart0_6.Instance',
+                               dbus_interface=dbus.PROPERTIES_IFACE)
+
+    def get_job_instances(self, job_name):
+        job = self.get_job(job_name)
+        paths = job.GetAllInstances([], dbus_interface='com.ubuntu.Upstart0_6.Job')
+        return [self._bus.get_object('com.ubuntu.Upstart', path) for path in paths]
+
+    def get_job_instance_properties(self, job):
+        return job.GetAll('com.ubuntu.Upstart0_6.Instance',
+                          dbus_interface=dbus.PROPERTIES_IFACE)
+
+try:
+    upstart = Upstart()
+    try:
+        job = upstart.get_job(sys.argv[1])        
+        props = upstart.get_properties(job)
+
+        if props['state'] == 'running':
+            print 'OK: %s is running' % sys.argv[1]
+            sys.exit(0)
+        else:
+            print 'CRITICAL: %s is not running' % sys.argv[1]
+            sys.exit(2)
+
+    except dbus.DBusException as e:
+        instances = upstart.get_job_instances(sys.argv[1])
+        propses = [upstart.get_job_instance_properties(instance) for instance in instances]
+        states = dict([(props['name'], props['state']) for props in propses])
+        if len(states) != states.values().count('running'):
+            not_running = []
+            for name in states.keys():
+                if states[name] != 'running':
+                    not_running.append(name)
+            print 'CRITICAL: %d instances of %s not running: %s' % \
+                (len(not_running), sys.argv[1], not_running.join(', '))
+            sys.exit(2)
+        else:
+            print 'OK: %d instances of %s running' % (len(states), sys.argv[1])
+
+except dbus.DBusException as e:
+    print 'CRITICAL: failed to get properties of \'%s\' from upstart' % sys.argv[1]
+    sys.exit(2)
+

=== added file 'files/nrpe-external-master/nagios_plugin.py'
--- files/nrpe-external-master/nagios_plugin.py	1970-01-01 00:00:00 +0000
+++ files/nrpe-external-master/nagios_plugin.py	2014-11-17 02:28:34 +0000
@@ -0,0 +1,78 @@
+#!/usr/bin/env python
+#                                       m
+#  mmmm   m   m  mmmm   mmmm    mmm   mm#mm
+#  #" "#  #   #  #" "#  #" "#  #"  #    #
+#  #   #  #   #  #   #  #   #  #""""    #
+#  ##m#"  "mm"#  ##m#"  ##m#"  "#mm"    "mm
+#  #             #      #
+#  "             "      "
+# This file is managed by puppet.  Do not make local changes.
+
+# Copyright (C) 2005, 2006, 2007, 2012  James Troup <[e-mail]>
+
+import os
+import stat
+import time
+import traceback
+import sys
+
+
+################################################################################
+
+class CriticalError(Exception):
+    """This indicates a critical error."""
+    pass
+
+
+class WarnError(Exception):
+    """This indicates a warning condition."""
+    pass
+
+
+class UnknownError(Exception):
+    """This indicates a unknown error was encountered."""
+    pass
+
+
+def try_check(function, *args, **kwargs):
+    """Perform a check with error/warn/unknown handling."""
+    try:
+        function(*args, **kwargs)
+    except UnknownError, msg:
+        print msg
+        sys.exit(3)
+    except CriticalError, msg:
+        print msg
+        sys.exit(2)
+    except WarnError, msg:
+        print msg
+        sys.exit(1)
+    except:
+        print "%s raised unknown exception '%s'" % (function, sys.exc_info()[0])
+        print '=' * 60
+        traceback.print_exc(file=sys.stdout)
+        print '=' * 60
+        sys.exit(3)
+
+
+################################################################################
+
+def check_file_freshness(filename, newer_than=600):
+    """Check a file exists, is readable and is newer than <n> seconds (where <n> defaults to 600)."""
+    # First check the file exists and is readable
+    if not os.path.exists(filename):
+        raise CriticalError("%s: does not exist." % (filename))
+    if os.access(filename, os.R_OK) == 0:
+        raise CriticalError("%s: is not readable." % (filename))
+
+    # Then ensure the file is up-to-date enough
+    mtime = os.stat(filename)[stat.ST_MTIME]
+    last_modified = time.time() - mtime
+    if last_modified > newer_than:
+        raise CriticalError("%s: was last modified on %s and is too old (> %s seconds)."
+                            % (filename, time.ctime(mtime), newer_than))
+    if last_modified < 0:
+        raise CriticalError("%s: was last modified on %s which is in the future."
+                            % (filename, time.ctime(mtime)))
+
+################################################################################

=== modified file 'hooks/ceilometer_hooks.py'
--- hooks/ceilometer_hooks.py	2014-10-01 15:22:55 +0000
+++ hooks/ceilometer_hooks.py	2014-11-17 02:28:34 +0000
@@ -2,14 +2,17 @@
 
 import base64
 import sys
+import os
 from charmhelpers.fetch import (
     apt_install, filter_installed_packages,
     apt_update
 )
 from charmhelpers.core.hookenv import (
     open_port,
+    local_unit,
     relation_set,
     relation_ids,
+    relations_of_type,
     config,
     Hooks, UnregisteredHookError,
     log
@@ -29,6 +32,7 @@
     CEILOMETER_ROLE,
     register_configs,
     restart_map,
+    services,
     get_ceilometer_context,
     do_openstack_upgrade
 )
@@ -37,6 +41,7 @@
     canonical_url,
     PUBLIC, INTERNAL, ADMIN
 )
+from charmhelpers.contrib.charmsupport.nrpe import NRPE
 
 hooks = Hooks()
 CONFIGS = register_configs()
@@ -89,6 +94,7 @@
 def config_changed():
     if openstack_upgrade_available('ceilometer-common'):
         do_openstack_upgrade(CONFIGS)
+    update_nrpe_config()
     CONFIGS.write_all()
     ceilometer_joined()
     for rid in relation_ids('identity-service'):
@@ -98,6 +104,7 @@
 @hooks.hook('upgrade-charm')
 def upgrade_charm():
     install()
+    update_nrpe_config()
     any_changed()
 
 
@@ -137,6 +144,58 @@
     for relid in relation_ids('ceilometer-service'):
         relation_set(relid, context)
 
+
[e-mail]('nrpe-external-master-relation-joined',
+            'nrpe-external-master-relation-changed')
+def update_nrpe_config():
+    # Find out if nrpe set nagios_hostname
+    hostname = None
+    host_context = None
+    for rel in relations_of_type('nrpe-external-master'):
+        if 'nagios_hostname' in rel:
+            hostname = rel['nagios_hostname']
+            host_context = rel['nagios_host_context']
+            break
+    nrpe = NRPE(hostname=hostname)
+    apt_install('python-dbus')
+
+    if host_context:
+        current_unit = "%s:%s" % (host_context, local_unit())
+    else:
+        current_unit = local_unit()
+
+    services_to_monitor = services()
+
+    for service in services_to_monitor:
+        upstart_init = '/etc/init/%s.conf' % service
+        sysv_init = '/etc/init.d/%s' % service
+        if os.path.exists(upstart_init):
+            nrpe.add_check(
+                shortname=service,
+                description='process check {%s}' % current_unit,
+                check_cmd='check_upstart_job %s' % service,
+                )
+        elif os.path.exists(sysv_init):
+            cronpath = '/etc/cron.d/nagios-service-check-%s' % service
+            checkpath = os.path.join(os.environ['CHARM_DIR'],
+                                     'files/nrpe-external-master',
+                                     'check_exit_status.pl'),
+            cron_template = '*/5 * * * * root \
+%s -s /etc/init.d/%s \
+status > /var/lib/nagios/service-check-%s.txt\n' \
+                            % (checkpath[0], service, service)
+            f = open(cronpath, 'w')
+            f.write(cron_template)
+            f.close()
+            nrpe.add_check(
+                shortname=service,
+                description='process check {%s}' % current_unit,
+                check_cmd='check_status_file.py -f \
+                           /var/lib/nagios/service-check-%s.txt' % service,
+                )
+
+    nrpe.write()
+
 if __name__ == '__main__':
     try:
         hooks.execute(sys.argv)

=== modified file 'hooks/ceilometer_utils.py'
--- hooks/ceilometer_utils.py	2014-10-23 16:03:49 +0000
+++ hooks/ceilometer_utils.py	2014-11-17 02:28:34 +0000
@@ -50,6 +50,12 @@
     'ceilometer-agent-notification'
 ]
 
+ICEHOUSE_SERVICES = [
+    'ceilometer-alarm-notifier',
+    'ceilometer-alarm-evaluator',
+    'ceilometer-agent-notification'
+]
+
 CEILOMETER_ROLE = "ResellerAdmin"
 
 
@@ -90,6 +96,11 @@
     configs = templating.OSConfigRenderer(templates_dir=TEMPLATES,
                                           openstack_release=release)
 
+    if (get_os_codename_install_source(config('openstack-origin'))
+            >= 'icehouse'):
+        CONFIG_FILES[CEILOMETER_CONF]['services'] = \
+            CONFIG_FILES[CEILOMETER_CONF]['services'] + ICEHOUSE_SERVICES
+
     for conf in CONFIG_FILES:
         configs.register(conf, CONFIG_FILES[conf]['hook_contexts'])
 
@@ -120,6 +131,14 @@
     return _map
 
 
+def services():
+    ''' Returns a list of services associate with this charm '''
+    _services = []
+    for v in restart_map().values():
+        _services = _services + v
+    return list(set(_services))
+
+
 def get_ceilometer_context():
     ''' Retrieve a map of all current relation data for agent configuration '''
     ctxt = {}

=== added directory 'hooks/charmhelpers/contrib/charmsupport'
=== added file 'hooks/charmhelpers/contrib/charmsupport/__init__.py'
=== added file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py'
--- hooks/charmhelpers/contrib/charmsupport/nrpe.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/charmsupport/nrpe.py	2014-11-17 02:28:34 +0000
@@ -0,0 +1,222 @@
+"""Compatibility with the nrpe-external-master charm"""
+# Copyright 2012 Canonical Ltd.
+#
+# Authors:
+#  Matthew Wedgwood <[e-mail]>
+
+import subprocess
+import pwd
+import grp
+import os
+import re
+import shlex
+import yaml
+
+from charmhelpers.core.hookenv import (
+    config,
+    local_unit,
+    log,
+    relation_ids,
+    relation_set,
+)
+
+from charmhelpers.core.host import service
+
+# This module adds compatibility with the nrpe-external-master and plain nrpe
+# subordinate charms. To use it in your charm:
+#
+# 1. Update metadata.yaml
+#
+#   provides:
+#     (...)
+#     nrpe-external-master:
+#       interface: nrpe-external-master
+#       scope: container
+#
+#   and/or
+#
+#   provides:
+#     (...)
+#     local-monitors:
+#       interface: local-monitors
+#       scope: container
+
+#
+# 2. Add the following to config.yaml
+#
+#    nagios_context:
+#      default: "juju"
+#      type: string
+#      description: |
+#        Used by the nrpe subordinate charms.
+#        A string that will be prepended to instance name to set the host name
+#        in nagios. So for instance the hostname would be something like:
+#            juju-myservice-0
+#        If you're running multiple environments with the same services in them
+#        this allows you to differentiate between them.
+#
+# 3. Add custom checks (Nagios plugins) to files/nrpe-external-master
+#
+# 4. Update your hooks.py with something like this:
+#
+#    from charmsupport.nrpe import NRPE
+#    (...)
+#    def update_nrpe_config():
+#        nrpe_compat = NRPE()
+#        nrpe_compat.add_check(
+#            shortname = "myservice",
+#            description = "Check MyService",
+#            check_cmd = "check_http -w 2 -c 10 http://localhost"
+#            )
+#        nrpe_compat.add_check(
+#            "myservice_other",
+#            "Check for widget failures",
+#            check_cmd = "/srv/myapp/scripts/widget_check"
+#            )
+#        nrpe_compat.write()
+#
+#    def config_changed():
+#        (...)
+#        update_nrpe_config()
+#
+#    def nrpe_external_master_relation_changed():
+#        update_nrpe_config()
+#
+#    def local_monitors_relation_changed():
+#        update_nrpe_config()
+#
+# 5. ln -s hooks.py nrpe-external-master-relation-changed
+#    ln -s hooks.py local-monitors-relation-changed
+
+
+class CheckException(Exception):
+    pass
+
+
+class Check(object):
+    shortname_re = '[A-Za-z0-9-_]+$'
+    service_template = ("""
+#---------------------------------------------------
+# This file is Juju managed
+#---------------------------------------------------
+define service {{
+    use                             active-service
+    host_name                       {nagios_hostname}
+    service_description             {nagios_hostname}[{shortname}] """
+                        """{description}
+    check_command                   check_nrpe!{command}
+    servicegroups                   {nagios_servicegroup}
+}}
+""")
+
+    def __init__(self, shortname, description, check_cmd):
+        super(Check, self).__init__()
+        # XXX: could be better to calculate this from the service name
+        if not re.match(self.shortname_re, shortname):
+            raise CheckException("shortname must match {}".format(
+                Check.shortname_re))
+        self.shortname = shortname
+        self.command = "check_{}".format(shortname)
+        # Note: a set of invalid characters is defined by the
+        # Nagios server config
+        # The default is: illegal_object_name_chars=`~!$%^&*"|'<>?,()=
+        self.description = description
+        self.check_cmd = self._locate_cmd(check_cmd)
+
+    def _locate_cmd(self, check_cmd):
+        search_path = (
+            '/',
+            os.path.join(os.environ['CHARM_DIR'],
+                         'files/nrpe-external-master'),
+            '/usr/lib/nagios/plugins',
+            '/usr/local/lib/nagios/plugins',
+        )
+        parts = shlex.split(check_cmd)
+        for path in search_path:
+            if os.path.exists(os.path.join(path, parts[0])):
+                command = os.path.join(path, parts[0])
+                if len(parts) > 1:
+                    command += " " + " ".join(parts[1:])
+                return command
+        log('Check command not found: {}'.format(parts[0]))
+        return ''
+
+    def write(self, nagios_context, hostname):
+        nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format(
+            self.command)
+        with open(nrpe_check_file, 'w') as nrpe_check_config:
+            nrpe_check_config.write("# check {}\n".format(self.shortname))
+            nrpe_check_config.write("command[{}]={}\n".format(
+                self.command, self.check_cmd))
+
+        if not os.path.exists(NRPE.nagios_exportdir):
+            log('Not writing service config as {} is not accessible'.format(
+                NRPE.nagios_exportdir))
+        else:
+            self.write_service_config(nagios_context, hostname)
+
+    def write_service_config(self, nagios_context, hostname):
+        for f in os.listdir(NRPE.nagios_exportdir):
+            if re.search('.*{}.cfg'.format(self.command), f):
+                os.remove(os.path.join(NRPE.nagios_exportdir, f))
+
+        templ_vars = {
+            'nagios_hostname': hostname,
+            'nagios_servicegroup': nagios_context,
+            'description': self.description,
+            'shortname': self.shortname,
+            'command': self.command,
+        }
+        nrpe_service_text = Check.service_template.format(**templ_vars)
+        nrpe_service_file = '{}/service__{}_{}.cfg'.format(
+            NRPE.nagios_exportdir, hostname, self.command)
+        with open(nrpe_service_file, 'w') as nrpe_service_config:
+            nrpe_service_config.write(str(nrpe_service_text))
+
+    def run(self):
+        subprocess.call(self.check_cmd)
+
+
+class NRPE(object):
+    nagios_logdir = '/var/log/nagios'
+    nagios_exportdir = '/var/lib/nagios/export'
+    nrpe_confdir = '/etc/nagios/nrpe.d'
+
+    def __init__(self, hostname=None):
+        super(NRPE, self).__init__()
+        self.config = config()
+        self.nagios_context = self.config['nagios_context']
+        self.unit_name = local_unit().replace('/', '-')
+        if hostname:
+            self.hostname = hostname
+        else:
+            self.hostname = "{}-{}".format(self.nagios_context, self.unit_name)
+        self.checks = []
+
+    def add_check(self, *args, **kwargs):
+        self.checks.append(Check(*args, **kwargs))
+
+    def write(self):
+        try:
+            nagios_uid = pwd.getpwnam('nagios').pw_uid
+            nagios_gid = grp.getgrnam('nagios').gr_gid
+        except:
+            log("Nagios user not set up, nrpe checks not updated")
+            return
+
+        if not os.path.exists(NRPE.nagios_logdir):
+            os.mkdir(NRPE.nagios_logdir)
+            os.chown(NRPE.nagios_logdir, nagios_uid, nagios_gid)
+
+        nrpe_monitors = {}
+        monitors = {"monitors": {"remote": {"nrpe": nrpe_monitors}}}
+        for nrpecheck in self.checks:
+            nrpecheck.write(self.nagios_context, self.hostname)
+            nrpe_monitors[nrpecheck.shortname] = {
+                "command": nrpecheck.command,
+            }
+
+        service('restart', 'nagios-nrpe-server')
+
+        for rid in relation_ids("local-monitors"):
+            relation_set(relation_id=rid, monitors=yaml.dump(monitors))

=== added file 'hooks/charmhelpers/contrib/charmsupport/volumes.py'
--- hooks/charmhelpers/contrib/charmsupport/volumes.py	1970-01-01 00:00:00 +0000
+++ hooks/charmhelpers/contrib/charmsupport/volumes.py	2014-11-17 02:28:34 +0000
@@ -0,0 +1,156 @@
+'''
+Functions for managing volumes in juju units. One volume is supported per unit.
+Subordinates may have their own storage, provided it is on its own partition.
+
+Configuration stanzas:
+  volume-ephemeral:
+    type: boolean
+    default: true
+    description: >
+      If false, a volume is mounted as sepecified in "volume-map"
+      If true, ephemeral storage will be used, meaning that log data
+         will only exist as long as the machine. YOU HAVE BEEN WARNED.
+  volume-map:
+    type: string
+    default: {}
+    description: >
+      YAML map of units to device names, e.g:
+        "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }"
+      Service units will raise a configure-error if volume-ephemeral
+      is 'true' and no volume-map value is set. Use 'juju set' to set a
+      value and 'juju resolved' to complete configuration.
+
+Usage:
+    from charmsupport.volumes import configure_volume, VolumeConfigurationError
+    from charmsupport.hookenv import log, ERROR
+    def post_mount_hook():
+        stop_service('myservice')
+    def post_mount_hook():
+        start_service('myservice')
+
+    if __name__ == '__main__':
+        try:
+            configure_volume(before_change=pre_mount_hook,
+                             after_change=post_mount_hook)
+        except VolumeConfigurationError:
+            log('Storage could not be configured', ERROR)
+'''
+
+# XXX: Known limitations
+# - fstab is neither consulted nor updated
+
+import os
+from charmhelpers.core import hookenv
+from charmhelpers.core import host
+import yaml
+
+
+MOUNT_BASE = '/srv/juju/volumes'
+
+
+class VolumeConfigurationError(Exception):
+    '''Volume configuration data is missing or invalid'''
+    pass
+
+
+def get_config():
+    '''Gather and sanity-check volume configuration data'''
+    volume_config = {}
+    config = hookenv.config()
+
+    errors = False
+
+    if config.get('volume-ephemeral') in (True, 'True', 'true', 'Yes', 'yes'):
+        volume_config['ephemeral'] = True
+    else:
+        volume_config['ephemeral'] = False
+
+    try:
+        volume_map = yaml.safe_load(config.get('volume-map', '{}'))
+    except yaml.YAMLError as e:
+        hookenv.log("Error parsing YAML volume-map: {}".format(e),
+                    hookenv.ERROR)
+        errors = True
+    if volume_map is None:
+        # probably an empty string
+        volume_map = {}
+    elif not isinstance(volume_map, dict):
+        hookenv.log("Volume-map should be a dictionary, not {}".format(
+            type(volume_map)))
+        errors = True
+
+    volume_config['device'] = volume_map.get(os.environ['JUJU_UNIT_NAME'])
+    if volume_config['device'] and volume_config['ephemeral']:
+        # asked for ephemeral storage but also defined a volume ID
+        hookenv.log('A volume is defined for this unit, but ephemeral '
+                    'storage was requested', hookenv.ERROR)
+        errors = True
+    elif not volume_config['device'] and not volume_config['ephemeral']:
+        # asked for permanent storage but did not define volume ID
+        hookenv.log('Ephemeral storage was requested, but there is no volume '
+                    'defined for this unit.', hookenv.ERROR)
+        errors = True
+
+    unit_mount_name = hookenv.local_unit().replace('/', '-')
+    volume_config['mountpoint'] = os.path.join(MOUNT_BASE, unit_mount_name)
+
+    if errors:
+        return None
+    return volume_config
+
+
+def mount_volume(config):
+    if os.path.exists(config['mountpoint']):
+        if not os.path.isdir(config['mountpoint']):
+            hookenv.log('Not a directory: {}'.format(config['mountpoint']))
+            raise VolumeConfigurationError()
+    else:
+        host.mkdir(config['mountpoint'])
+    if os.path.ismount(config['mountpoint']):
+        unmount_volume(config)
+    if not host.mount(config['device'], config['mountpoint'], persist=True):
+        raise VolumeConfigurationError()
+
+
+def unmount_volume(config):
+    if os.path.ismount(config['mountpoint']):
+        if not host.umount(config['mountpoint'], persist=True):
+            raise VolumeConfigurationError()
+
+
+def managed_mounts():
+    '''List of all mounted managed volumes'''
+    return filter(lambda mount: mount[0].startswith(MOUNT_BASE), host.mounts())
+
+
+def configure_volume(before_change=lambda: None, after_change=lambda: None):
+    '''Set up storage (or don't) according to the charm's volume configuration.
+       Returns the mount point or "ephemeral". before_change and after_change
+       are optional functions to be called if the volume configuration changes.
+    '''
+
+    config = get_config()
+    if not config:
+        hookenv.log('Failed to read volume configuration', hookenv.CRITICAL)
+        raise VolumeConfigurationError()
+
+    if config['ephemeral']:
+        if os.path.ismount(config['mountpoint']):
+            before_change()
+            unmount_volume(config)
+            after_change()
+        return 'ephemeral'
+    else:
+        # persistent storage
+        if os.path.ismount(config['mountpoint']):
+            mounts = dict(managed_mounts())
+            if mounts.get(config['mountpoint']) != config['device']:
+                before_change()
+                unmount_volume(config)
+                mount_volume(config)
+                after_change()
+        else:
+            before_change()
+            mount_volume(config)
+            after_change()
+        return config['mountpoint']

=== added symlink 'hooks/nrpe-external-master-relation-changed'
=== target is u'ceilometer_hooks.py'
=== added symlink 'hooks/nrpe-external-master-relation-joined'
=== target is u'ceilometer_hooks.py'
=== modified file 'metadata.yaml'
--- metadata.yaml	2013-10-20 22:30:27 +0000
+++ metadata.yaml	2014-11-17 02:28:34 +0000
@@ -12,6 +12,9 @@
   - miscellaneous
   - openstack
 provides:
+  nrpe-external-master:
+    interface: nrpe-external-master
+    scope: container
   ceilometer-service:
     interface: ceilometer
 requires: