Merge lp:~chad.smith/charms/precise/swift-proxy/swift-proxy-ha-with-health into lp:~james-page/charms/precise/swift-proxy/ha-support

[Chad Smith]

This is a merge proposal to establish a potential template for health_script.d, add_to_cluster and remove_from_cluster delivery within the ha-supported openstack charms. The *cluster* simple scripts will be common on each charm that uses corosync configs. The health_scripts.d dir will contain some common scripts and some additional scripts that are appropriate health checks for the specific service.

The basic architecture we are hoping for is an extendable set of run-parts scripts in health_checks.d that can be seeded with juju config environment variables written into a /var/lib/juju/units/<charm_name>/charm/scripts/scriptrc file. Landscape client will do the work of calling these scripts and validating charm service health during openstack service upgrades. Both Jerry and I are tackling the initial push to add health/cluster scripts to the charms. Hopefully the model works and we can iterate on extending health or cluster scripts as service HA configuration gets more complex.

much thanks for the review.

[lp:~chad.smith/charms/precise/swift-proxy/swift-proxy-ha-with-health]

42. By Chad Smith on 2013-03-05

more strict netstat port matching

43. By Chad Smith on 2013-03-06

merge lp:~openstack-charmers/charms/precise/swift-proxy/ha-support resolve conflict

Unmerged revisions

=== modified file 'hooks/lib/openstack_common.py'
--- hooks/lib/openstack_common.py	2013-03-01 23:20:05 +0000
+++ hooks/lib/openstack_common.py	2013-03-06 22:26:24 +0000
@@ -224,3 +224,40 @@
             f.write(src)
     else:
         error_out("Invalid openstack-release specified: %s" % rel)
+
+HAPROXY_CONF = '/etc/haproxy/haproxy.cfg'
+HAPROXY_DEFAULT = '/etc/default/haproxy'
+
+def configure_haproxy(units, service_ports, template_dir=None):
+    template_dir = template_dir or 'templates'
+    import jinja2
+    context = {
+        'units': units,
+        'service_ports': service_ports
+        }
+    templates = jinja2.Environment(
+                    loader=jinja2.FileSystemLoader(template_dir)
+                    )
+    template = templates.get_template(
+                    os.path.basename(HAPROXY_CONF)
+                    )
+    with open(HAPROXY_CONF, 'w') as f:
+        f.write(template.render(context))
+    with open(HAPROXY_DEFAULT, 'w') as f:
+        f.write('ENABLED=1')
+
+def save_script_rc(script_path="scripts/scriptrc", **env_vars):
+    """
+    Write an rc file in the charm-delivered directory containing
+    exported environment variables provided by env_vars. Any charm scripts run
+    outside the juju hook environment can source this scriptrc to obtain
+    updated config information necessary to perform health checks or
+    service changes.
+    """
+    unit_name = os.getenv('JUJU_UNIT_NAME').replace('/', '-')
+    juju_rc_path="/var/lib/juju/units/%s/charm/%s" % (unit_name, script_path)
+    with open(juju_rc_path, 'wb') as rc_script:
+        rc_script.write(
+            "#!/bin/bash\n")
+        [rc_script.write('export %s=%s\n' % (u, p))
+         for u, p in env_vars.iteritems() if u != "script_path"]

=== modified file 'hooks/swift_utils.py'
--- hooks/swift_utils.py	2013-03-04 14:43:54 +0000
+++ hooks/swift_utils.py	2013-03-06 22:26:24 +0000
@@ -171,6 +171,11 @@
         import multiprocessing
         workers = multiprocessing.cpu_count()
 
+    env_vars = {'OPENSTACK_SERVICE_SWIFT': 'proxy-server',
+                'OPENSTACK_PORT_API': bind_port,
+                'OPENSTACK_PORT_MEMCACHED': 11211}
+    openstack.save_script_rc(**env_vars)
+
     ctxt = {
         'proxy_ip': utils.get_host_ip(),
         'bind_port': utils.determine_api_port(bind_port),

=== modified file 'revision'
--- revision	2013-01-29 00:41:52 +0000
+++ revision	2013-03-06 22:26:24 +0000
@@ -1,1 +1,1 @@
-109
+110

=== added directory 'scripts'
=== added file 'scripts/add_to_cluster'
--- scripts/add_to_cluster	1970-01-01 00:00:00 +0000
+++ scripts/add_to_cluster	2013-03-06 22:26:24 +0000
@@ -0,0 +1,2 @@
+#!/bin/bash
+crm node online

=== added directory 'scripts/health_checks.d'
=== added file 'scripts/health_checks.d/service_ports_live'
--- scripts/health_checks.d/service_ports_live	1970-01-01 00:00:00 +0000
+++ scripts/health_checks.d/service_ports_live	2013-03-06 22:26:24 +0000
@@ -0,0 +1,13 @@
+#!/bin/bash
+# Validate that service ports are active
+HEALTH_DIR=`dirname $0`
+SCRIPTS_DIR=`dirname $HEALTH_DIR`
+. $SCRIPTS_DIR/scriptrc
+set -e
+
+# Grab any OPENSTACK_PORT* environment variables
+openstack_ports=`env| awk -F '=' '(/OPENSTACK_PORT/){print $2}'`
+for port in $openstack_ports
+do
+    netstat -ln | grep -q ":$port "
+done

=== added file 'scripts/health_checks.d/service_swift_running'
--- scripts/health_checks.d/service_swift_running	1970-01-01 00:00:00 +0000
+++ scripts/health_checks.d/service_swift_running	2013-03-06 22:26:24 +0000
@@ -0,0 +1,15 @@
+#!/bin/bash
+# Validate that service is running
+HEALTH_DIR=`dirname $0`
+SCRIPTS_DIR=`dirname $HEALTH_DIR`
+. $SCRIPTS_DIR/scriptrc
+set -e
+
+# Grab any OPENSTACK_SWIFT_SERVICE* environment variables
+openstack_service_names=`env| awk -F '=' '(/OPENSTACK_SWIFT_SERVICE/){print $2}'`
+for service_name in $openstack_service_names
+do
+    # Double-negative: we want to ensure swift-init does not return
+    # 'No <service_name> running'
+    swift-init $service_name status 2>/dev/null | grep -vq "No $service_name running"
+done

=== added file 'scripts/remove_from_cluster'
--- scripts/remove_from_cluster	1970-01-01 00:00:00 +0000
+++ scripts/remove_from_cluster	2013-03-06 22:26:24 +0000
@@ -0,0 +1,2 @@
+#!/bin/bash
+crm node standby