Merge lp:~hazmat/charms/precise/mysql/ssl-everywhere into lp:charms/mysql

[Kapil Thangavelu]

SSL Support for MySQL

Adds a few additional configuration options for SSL.

'ssl' is the primary one, it can be 'off', 'on', or 'only'.

If its off, no changes. If its 'on' then ssl support is
enabled and ssl_ca cert is sent to clients along the relation.

If its only, then all clients must connect using ssl.

The server cert/key and ca cert can be provided via config. If
they are not, the service will act as its own ca, and also
in only mode will also generate client certs and pass them
along the relation (the client use of those certs is not
required atm).

https://codereview.appspot.com/68120043/

[Kapil Thangavelu (hazmat)]

Reviewers: mp+207904_code.launchpad.net,

Message:
Please take a look.

Description:
SSL Support for MySQL

Adds a few additional configuration options for SSL.

'ssl' is the primary one, it can be 'off', 'on', or 'only'.

If its off, no changes. If its 'on' then ssl support is
enabled and ssl_ca cert is sent to clients along the relation.

If its only, then all clients must connect using ssl.

The server cert/key and ca cert can be provided via config. If
they are not, the service will act as its own ca, and also
in only mode will also generate client certs and pass them
along the relation (the client use of those certs is not
required atm).

https://code.launchpad.net/~hazmat/charms/precise/mysql/ssl-everywhere/+merge/207904

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/68120043/

Affected files (+483, -135 lines):
   A [revision details]
   M config.yaml
   M hooks/common.py
   M hooks/config-changed
   M hooks/db-relation-joined
   M hooks/shared_db_relations.py
   M hooks/slave-relation-changed
   A hooks/sslca.py
   M revision

[Marco Ceppi (marcoceppi)]

I have some concerns about the way this affects the mysql interface
below. Overall, this is a great change and will bring security to
communication within a deployed environment, I'm open to discussion
about the points below - I simply want to make sure the charm stays as
compatible as possible as it grows.

https://codereview.appspot.com/68120043/diff/1/config.yaml
File config.yaml (right):

https://codereview.appspot.com/68120043/diff/1/config.yaml#newcode38
config.yaml:38: ssl_key:
I don't like that these are _'d while the majority of the configuration
options for MySQL are hypenated (with the exception of vip_iface, and
vip_cidr which I would have rejected had I maintained the charm at the
time). If there isn't a hard requirement for these to be _ then I'd
prefer if they followed the convention of the majority of the charm's
config.

https://codereview.appspot.com/68120043/diff/1/hooks/db-relation-joined
File hooks/db-relation-joined (right):

https://codereview.appspot.com/68120043/diff/1/hooks/db-relation-joined#newcode91
hooks/db-relation-joined:91: cmd.append("%s=%s" % (k, v))
With this, you're now overloading the mysql interface to the point
where, if a user deploys the MySQL charm, configures it for SSL, then
joins to a charm that doesn't know how to speak to MySQL on SSL with on
and off, no change (I assume), with ssl=only that charm just won't work.

Given the nature of this change, it seem more apt that this should be
implemented as a new interface, maybe mysql-ssl - so when you connect on
that interface and no ssl cert/ca/key are provided MySQL generates one,
otherwise uses the values provided by the configuration. In doing so
services can expose their compat with MySQL over SSL without having
cases that charms will break if MySQL is configured in a manner that's
too restrictive.

https://codereview.appspot.com/68120043/

[Kapil Thangavelu (hazmat)]

i disagree, i think that makes for a lousy experience to make a separate
interface, instead of adding ssl to one place now you have to reconfigure
your entire infrastructure to add it later (breaking and readding
relations, etc). Imagine trying to upgrade a production system to take
advantage of ssl.. instead of ugprade charm and config ssl, you now have to
break your entire production system. Its only the config ssl only that
forces the client to ssl. the ssl_ key names are chosen for consistency
with ssl support through the charm ecosystem, there are several others
already using these keys.

-k

On Thu, Feb 27, 2014 at 9:24 AM, Marco Ceppi <email address hidden> wrote:

> I have some concerns about the way this affects the mysql interface
> below. Overall, this is a great change and will bring security to
> communication within a deployed environment, I'm open to discussion
> about the points below - I simply want to make sure the charm stays as
> compatible as possible as it grows.
>
>
> https://codereview.appspot.com/68120043/diff/1/config.yaml
> File config.yaml (right):
>
> https://codereview.appspot.com/68120043/diff/1/config.yaml#newcode38
> config.yaml:38: ssl_key:
> I don't like that these are _'d while the majority of the configuration
> options for MySQL are hypenated (with the exception of vip_iface, and
> vip_cidr which I would have rejected had I maintained the charm at the
> time). If there isn't a hard requirement for these to be _ then I'd
> prefer if they followed the convention of the majority of the charm's
> config.
>
> https://codereview.appspot.com/68120043/diff/1/hooks/db-relation-joined
> File hooks/db-relation-joined (right):
>
>
> https://codereview.appspot.com/68120043/diff/1/hooks/db-relation-joined#newcode91
> hooks/db-relation-joined:91: cmd.append("%s=%s" % (k, v))
> With this, you're now overloading the mysql interface to the point
> where, if a user deploys the MySQL charm, configures it for SSL, then
> joins to a charm that doesn't know how to speak to MySQL on SSL with on
> and off, no change (I assume), with ssl=only that charm just won't work.
>
> Given the nature of this change, it seem more apt that this should be
> implemented as a new interface, maybe mysql-ssl - so when you connect on
> that interface and no ssl cert/ca/key are provided MySQL generates one,
> otherwise uses the values provided by the configuration. In doing so
> services can expose their compat with MySQL over SSL without having
> cases that charms will break if MySQL is configured in a manner that's
> too restrictive.
>
> https://codereview.appspot.com/68120043/
>
> --
>
> https://code.launchpad.net/~hazmat/charms/precise/mysql/ssl-everywhere/+merge/207904
> You are the owner of lp:~hazmat/charms/precise/mysql/ssl-everywhere.
>

[Kapil Thangavelu (hazmat)]

plus with that logic, your talking about 3 extra interfaces to mirror the
three client interfaces we have now on mysql. that seems silly.

On Thu, Feb 27, 2014 at 9:30 AM, Kapil Thangavelu <
<email address hidden>> wrote:

> i disagree, i think that makes for a lousy experience to make a separate
> interface, instead of adding ssl to one place now you have to reconfigure
> your entire infrastructure to add it later (breaking and readding
> relations, etc). Imagine trying to upgrade a production system to take
> advantage of ssl.. instead of ugprade charm and config ssl, you now have to
> break your entire production system. Its only the config ssl only that
> forces the client to ssl. the ssl_ key names are chosen for consistency
> with ssl support through the charm ecosystem, there are several others
> already using these keys.
>
> -k
>
>
>
> On Thu, Feb 27, 2014 at 9:24 AM, Marco Ceppi <email address hidden> wrote:
>
>> I have some concerns about the way this affects the mysql interface
>> below. Overall, this is a great change and will bring security to
>> communication within a deployed environment, I'm open to discussion
>> about the points below - I simply want to make sure the charm stays as
>> compatible as possible as it grows.
>>
>>
>> https://codereview.appspot.com/68120043/diff/1/config.yaml
>> File config.yaml (right):
>>
>> https://codereview.appspot.com/68120043/diff/1/config.yaml#newcode38
>> config.yaml:38: ssl_key:
>> I don't like that these are _'d while the majority of the configuration
>> options for MySQL are hypenated (with the exception of vip_iface, and
>> vip_cidr which I would have rejected had I maintained the charm at the
>> time). If there isn't a hard requirement for these to be _ then I'd
>> prefer if they followed the convention of the majority of the charm's
>> config.
>>
>> https://codereview.appspot.com/68120043/diff/1/hooks/db-relation-joined
>> File hooks/db-relation-joined (right):
>>
>>
>> https://codereview.appspot.com/68120043/diff/1/hooks/db-relation-joined#newcode91
>> hooks/db-relation-joined:91<https://codereview.appspot.com/68120043/diff/1/hooks/db-relation-joined#newcode91hooks/db-relation-joined:91>:
>> cmd.append("%s=%s" % (k, v))
>> With this, you're now overloading the mysql interface to the point
>> where, if a user deploys the MySQL charm, configures it for SSL, then
>> joins to a charm that doesn't know how to speak to MySQL on SSL with on
>> and off, no change (I assume), with ssl=only that charm just won't work.
>>
>> Given the nature of this change, it seem more apt that this should be
>> implemented as a new interface, maybe mysql-ssl - so when you connect on
>> that interface and no ssl cert/ca/key are provided MySQL generates one,
>> otherwise uses the values provided by the configuration. In doing so
>> services can expose their compat with MySQL over SSL without having
>> cases that charms will break if MySQL is configured in a manner that's
>> too restrictive.
>>
>> https://codereview.appspot.com/68120043/
>>
>> --
>>
>> https://code.launchpad.net/~hazmat/charms/precise/mysql/ssl-everywhere/+merge/207904
>> You are the owner of lp:~hazmat/charms/precise/mysql/ssl-everywhere....

[Kapil Thangavelu (hazmat)]

fwiw, i'm refactoring up this branch a bit to use charm-helpers and to cleanly re-configure existing relations when the setting changes at runtime. the core feature functionality is the same though, so the interface extension discussion remains.

Unmerged revisions

118. By Kapil Thangavelu on 2014-02-24

external ca and cert support

117. By Kapil Thangavelu on 2014-02-20

mysql is picky about cert gen, accomodate

116. By Kapil Thangavelu on 2014-02-18

ftest fixes

115. By Kapil Thangavelu on 2014-02-18

off is yaml magic for bool, quote appropriately

114. By Kapil Thangavelu on 2014-02-18

ssl support

=== modified file 'config.yaml'
--- config.yaml	2014-02-04 21:48:28 +0000
+++ config.yaml	2014-02-24 11:48:01 +0000
@@ -31,6 +31,24 @@
         default: 'MIXED'
         type: string
         description: If binlogging is enabled, this is the format that will be used. Ignored when tuning-level == fast.
+    ssl:
+        default: 'off'
+        type: string
+        description: Enable SSL connections on mysql, valid values are 'off', 'on', and 'only'
+    ssl_key:
+        type: string
+        description: private unencrypted key in PEM format (starts "-----BEGIN RSA PRIVATE KEY-----")
+        default: ""
+    ssl_cert:
+        type: string
+        description: X.509 certificate in PEM format (starts "-----BEGIN CERTIFICATE-----")
+        default: ""
+    ssl_ca:
+        type: string
+        description: |
+          Certificate authority cert that signed pem. Optional if the ssl_cert is signed by a ca
+          recognized by the os.
+        default: ""
     vip:
         type: string
         default: ''

=== modified file 'hooks/common.py'
--- hooks/common.py	2013-02-12 17:54:29 +0000
+++ hooks/common.py	2014-02-24 11:48:01 +0000
@@ -1,15 +1,51 @@
 # vim: syntax=python
 
+import base64
 import os
+import json
 import sys
 import MySQLdb
+import sslca
 import subprocess
 import uuid
 
+
 def get_service_user_file(service):
     return '/var/lib/mysql/%s.service_user2' % service
 
 
+def get_service_ca():
+    service_name = os.environ['JUJU_UNIT_NAME'].split('/')[0]
+    ca_path = os.path.join(os.environ['CHARM_DIR'], 'ca')
+    ca = sslca.ServiceCA(service_name, ca_path, sslca.MYSQL_CERT)
+    ca.init()
+    return ca
+
+
+def is_ext_ssl_ca():
+    ssl_key = configs.get('ssl_key')
+    ssl_cert = configs.get('ssl_cert')
+    return all((ssl_key, ssl_cert))
+
+
+def configure_client_ssl():
+    if configs.get('ssl', '') not in ('only', 'on'):
+        return dict()
+    if is_ext_ssl_ca():
+        if configs.get('ssl_ca'):
+            return dict(ssl_ca=base64.b64encode(configs['ssl_ca']))
+        return
+    # self-managed-client-certs
+    ca = get_service_ca()
+    ca_bundle = ca.get_ca_bundle()
+    service_cert, service_key = ca.get_or_create_cert(
+        change_unit.split('/', 1)[0])
+    return dict(
+        ssl_ca=base64.b64encode(ca_bundle),
+        ssl_key=base64.b64encode(service_key),
+        ssl_cert=base64.b64encode(service_cert))
+
+
 def get_service_user(service):
     if service == '':
         return (None, None)
@@ -17,7 +53,8 @@
     if os.path.exists(sfile):
         with open(sfile, 'r') as f:
             return (f.readline().strip(), f.readline().strip())
-    (suser, service_password) = subprocess.check_output(['pwgen', '-N 2', '15']).strip().split("\n")
+    (suser, service_password) = subprocess.check_output(
+        ['pwgen', '-N 2', '15']).strip().split("\n")
     with open(sfile, 'w') as f:
         f.write("%s\n" % suser)
         f.write("%s\n" % service_password)
@@ -55,6 +92,8 @@
 slave = os.path.exists('/var/lib/juju/i.am.a.slave')
 broken_path = '/var/lib/juju/%s.mysql.broken' % database_name
 broken = os.path.exists(broken_path)
+configs = json.loads(subprocess.check_output(['config-get', '--format=json']))
+
 
 def get_db_cursor():
     # Connect to mysql
@@ -95,15 +134,14 @@
     return "GRANT ALL PRIVILEGES ON `{}`".format(db_name) in grants
 
 
-def create_grant(db_name, db_user,
-                 remote_ip, password):
+def create_grant(db_name, db_user, remote_ip, password):
     cursor = get_db_cursor()
+    grant = "GRANT ALL PRIVILEGES ON {}.* TO '{}'@'{}' IDENTIFIED BY '{}'"
+    # http://dev.mysql.com/doc/refman/5.5/en/grant.html
+    if configs.get('ssl') == 'only':
+        grant += ' REQUIRE SSL'
     try:
-        cursor.execute("GRANT ALL PRIVILEGES ON {}.* TO '{}'@'{}' "\
-                       "IDENTIFIED BY '{}'".format(db_name,
-                                                   db_user,
-                                                   remote_ip,
-                                                   password))
+        cursor.execute(grant.format(db_name, db_user, remote_ip, password))
     finally:
         cursor.close()
 
@@ -112,8 +150,8 @@
                   remote_ip):
     cursor = get_db_cursor()
     try:
-        cursor.execute("DROP FROM mysql.user WHERE user='{}' "\
-                       "AND HOST='{}'".format(db_user,
-                                              remote_ip))
+        cursor.execute(
+            "DROP FROM mysql.user WHERE user='{}' AND HOST='{}'".format(
+                db_user, remote_ip))
     finally:
         cursor.close()

=== modified file 'hooks/config-changed'
--- hooks/config-changed	2014-02-15 21:37:45 +0000
+++ hooks/config-changed	2014-02-24 11:48:01 +0000
@@ -10,6 +10,8 @@
 import platform
 from string import upper
 
+from common import get_service_ca, configs
+
 num_re = re.compile('^[0-9]+$')
 
 # There should be a library for this
@@ -48,7 +50,6 @@
 if IS_32BIT_SYSTEM:
     check_call(['juju-log','-l','INFO','32bit system restrictions in play'])
 
-configs=json.loads(check_output(['config-get','--format=json']))
 
 def get_memtotal():
     with open('/proc/meminfo') as meminfo_file:
@@ -153,6 +154,36 @@
 else:
     configs['max-connections'] = 'max_connections = %s' % configs['max-connections']
 
+
+if configs.get('ssl', 'off').strip() in ('on', 'only'):
+    service_name = os.environ.get('JUJU_UNIT_NAME').split('/')[0]
+    ssl_cert, ssl_key, ssl_ca = (
+        configs.get('ssl_cert'),
+        configs.get('ssl_key'),
+        configs.get('ssl_ca'))
+
+    if any((ssl_cert, ssl_key, ssl_ca)):
+        if not all((ssl_cert, ssl_key)):
+            print "ERROR", "SSL Key and cert are both required if either specified"
+            sys.exit(1)
+    else:
+        ca = get_service_ca()
+        ssl_cert, ssl_key = ca.create_certificate(service_name)
+        ssl_ca = ca.get_ca_bundle()
+
+    with open('/etc/mysql/cacert.pem', 'w') as fh:
+        fh.write(ssl_ca)
+    with open('/etc/mysql/server-cert.pem', 'w') as fh:
+        fh.write(ssl_cert)
+    with open('/etc/mysql/server-key.pem', 'w') as fh:
+        fh.write(ssl_key)
+    if ssl_ca:  # In case a user providers key/cert but no ca cause its official.
+        configs['ssl_ca_marker'] = ''
+    if ssl_key:
+        configs['ssl_cert_marker'] = ''
+else:
+    configs['ssl_cert_marker'] = configs['ssl_ca_marker'] = "#"
+
 template="""
 ######################################
 #
@@ -171,7 +202,7 @@
 # You can copy this to one of:
 # - "/etc/mysql/my.cnf" to set global options,
 # - "~/.my.cnf" to set user-specific options.
-# 
+#
 # One can use all long options that the program supports.
 # Run program with --help to get a list of available options and with
 # --print-defaults to see which it would actually understand and use.
@@ -277,9 +308,9 @@
 #
 # For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
 #
-# ssl-ca=/etc/mysql/cacert.pem
-# ssl-cert=/etc/mysql/server-cert.pem
-# ssl-key=/etc/mysql/server-key.pem
+%(ssl_ca_marker)s ssl-ca=/etc/mysql/cacert.pem
+%(ssl_cert_marker)s ssl-cert=/etc/mysql/server-cert.pem
+%(ssl_cert_marker)s ssl-key=/etc/mysql/server-key.pem
 
 
 
@@ -326,7 +357,7 @@
 
 need_restart = False
 for target,content in targets.iteritems():
-    tdir = os.path.dirname(target) 
+    tdir = os.path.dirname(target)
     if len(content) == 0 and os.path.exists(target):
         os.unlink(target)
         need_restart = True

=== modified file 'hooks/db-relation-joined'
--- hooks/db-relation-joined	2012-11-02 06:41:12 +0000
+++ hooks/db-relation-joined	2014-02-24 11:48:01 +0000
@@ -1,22 +1,27 @@
 #!/usr/bin/env python
 
-from common import *
+from common import (
+    change_unit, configs, get_db_cursor, get_service_ca, is_ext_ssl_ca,
+    configure_client_ssl)
 
-import urllib
-import subprocess
+import base64
+import random
 import os
+import pickle
 import sys
 import string
-import random
-import pickle
+import subprocess
+import urllib
+
 
 cursor = get_db_cursor()
 
 admin = os.path.basename(sys.argv[0]) == 'db-admin-relation-joined'
 
 def runsql(sql):
-	print "[%s]" % sql
-	cursor.execute(sql)
+    print "[%s]" % sql
+    cursor.execute(sql)
+
 
 runsql(
     "grant replication client on *.* to `%s` identified by '%s'"  % (
@@ -27,33 +32,33 @@
     try:
         runsql(
             "revoke all on `%s`.* from `%s`"  % (
-            database_name,   
+            database_name,
             user))
     except:
         print "revoke failed, ignoring error"
     runsql(
         "grant select on `%s`.* to `%s`"  % (
-        database_name,   
+        database_name,
         user))
 else:
     if admin:
-        runsql(
-            "grant all privileges on *.* to `%s` identified by '%s'" % (
-            user,
-            service_password))
+        grant = "grant all privileges on *.* to `%s` identified by '%s'"
+        if configs.get('ssl', '') == 'only':
+            grant += " REQUIRE SSL"
+        runsql( grant % (user, service_password))
     else:
-        runsql(
-            "grant all on `%s`.* to `%s` identified by '%s'" % (
-            database_name,
-            user,
-            service_password))
+        grant = "grant all on `%s`.* to `%s` identified by '%s'"
+        if configs.get('ssl', '') == 'only':
+            grant += " REQUIRE SSL"
+        runsql( grant % (database_name, user, service_password))
 
 hostname = subprocess.check_output(['unit-get','private-address']).strip()
 
+
 print str(["relation-set",
  "database=%s" % database_name,
  "user=%s" % user,
- "password=%s" % service_password,
+# "password=%s" % service_password,
  'host=%s' % hostname,
  'slave=%s' % slave])
 
@@ -78,10 +83,11 @@
 cursor.close()
 
 # Store new values in relation settings.
-subprocess.call(
-    ["relation-set",
-     "database=%s" % database_name,
-     "user=%s" % user,
-     "password=%s" % service_password,
-     'host=%s' % hostname,
-     'slave=%s' % slave,])
+cmd = ["relation-set", "database=%s" % database_name, 'host=%s' % hostname,
+       "user=%s" % user, "password=%s" % service_password,
+       'slave=%s' % slave,])
+
+for k, v in configure_client_ssl().items():
+    cmd.append("%s=%s" % (k, v))
+
+subprocess.check_call(cmd)

=== modified file 'hooks/shared_db_relations.py'
--- hooks/shared_db_relations.py	2013-07-10 16:32:52 +0000
+++ hooks/shared_db_relations.py	2014-02-24 11:48:01 +0000
@@ -11,15 +11,22 @@
 from common import (
     database_exists,
     create_database,
+    configure_client_ssl,
     grant_exists,
-    create_grant
-    )
-import subprocess
+    create_grant,
+    configs,
+    change_unit,
+    get_service_ca)
+
+import base64
+import functools
 import json
-import socket
-import os
 import lib.utils as utils
 import lib.cluster_utils as cluster
+import os
+import subprocess
+import socket
+
 
 LEADER_RES = 'res_mysql_vip'
 
@@ -29,44 +36,36 @@
 
 
 def relation_get():
-    return json.loads(subprocess.check_output(
-                        ['relation-get',
-                         '--format',
-                         'json']
-                        )
-                      )
+    return json.loads(
+        subprocess.check_output(['relation-get', '--format', 'json']))
+
+
+local_hostname = utils.unit_get('private-address')
+
+
+def configure_db(hostname, database, username):
+    passwd_file = "/var/lib/mysql/mysql-{}.passwd".format(username)
+    if hostname != local_hostname:
+        remote_ip = socket.gethostbyname(hostname)
+    else:
+        remote_ip = '127.0.0.1'
+
+    if not os.path.exists(passwd_file):
+        password = pwgen()
+        with open(passwd_file, 'w') as pfile:
+            pfile.write(password)
+    else:
+        with open(passwd_file) as pfile:
+            password = pfile.read().strip()
+
+    if not database_exists(database):
+        create_database(database)
+    if not grant_exists(database, username, remote_ip):
+        create_grant(database, username, remote_ip, password)
+        return password
 
 
 def shared_db_changed():
-
-    def configure_db(hostname,
-                     database,
-                     username):
-        passwd_file = "/var/lib/mysql/mysql-{}.passwd"\
-                        .format(username)
-        if hostname != local_hostname:
-            remote_ip = socket.gethostbyname(hostname)
-        else:
-            remote_ip = '127.0.0.1'
-
-        if not os.path.exists(passwd_file):
-            password = pwgen()
-            with open(passwd_file, 'w') as pfile:
-                pfile.write(password)
-        else:
-            with open(passwd_file) as pfile:
-                password = pfile.read().strip()
-
-        if not database_exists(database):
-            create_database(database)
-        if not grant_exists(database,
-                            username,
-                            remote_ip):
-            create_grant(database,
-                         username,
-                         remote_ip, password)
-        return password
-
     if not cluster.eligible_leader(LEADER_RES):
         utils.juju_log('INFO',
                        'MySQL service is peered, bailing shared-db relation'
@@ -74,64 +73,62 @@
         return
 
     settings = relation_get()
-    local_hostname = utils.unit_get('private-address')
-    singleset = set([
-        'database',
-        'username',
-        'hostname'
-        ])
+    singleset = set(['database', 'username', 'hostname'])
+
+    if configs.get('ssl', '').strip() in ('on', 'only'):
+        client_ssl = configure_client_ssl()
+        relation_set = functools.partial(utils.relation_set, **client_ssl)
+    else:
+        relation_set = utils.relation_set
 
     if singleset.issubset(settings):
         # Process a single database configuration
-        password = configure_db(settings['hostname'],
-                                settings['database'],
-                                settings['username'])
+        password = configure_db(
+            settings['hostname'], settings['database'], settings['username'])
         if not cluster.is_clustered():
-            utils.relation_set(db_host=local_hostname,
-                               password=password)
+            relation_set(db_host=local_hostname, password=password)
         else:
-            utils.relation_set(db_host=utils.config_get("vip"),
-                               password=password)
+            relation_set(db_host=utils.config_get("vip"), password=password)
+        return
 
+    # Process multiple database setup requests.
+    # from incoming relation data:
+    #  nova_database=xxx nova_username=xxx nova_hostname=xxx
+    #  quantum_database=xxx quantum_username=xxx quantum_hostname=xxx
+    # create
+    #{
+    #   "nova": {
+    #        "username": xxx,
+    #        "database": xxx,
+    #        "hostname": xxx
+    #    },
+    #    "quantum": {
+    #        "username": xxx,
+    #        "database": xxx,
+    #        "hostname": xxx
+    #    }
+    #}
+    #
+    databases = {}
+    for k, v in settings.iteritems():
+        db = k.split('_')[0]
+        x = '_'.join(k.split('_')[1:])
+        if db not in databases:
+            databases[db] = {}
+        databases[db][x] = v
+    return_data = {}
+    for db in databases:
+        if singleset.issubset(databases[db]):
+            return_data['_'.join([db, 'password'])] = \
+                configure_db(databases[db]['hostname'],
+                             databases[db]['database'],
+                             databases[db]['username'])
+    if len(return_data) > 0:
+        relation_set(**return_data)
+    if not cluster.is_clustered():
+        relation_set(db_host=local_hostname)
     else:
-        # Process multiple database setup requests.
-        # from incoming relation data:
-        #  nova_database=xxx nova_username=xxx nova_hostname=xxx
-        #  quantum_database=xxx quantum_username=xxx quantum_hostname=xxx
-        # create
-        #{
-        #   "nova": {
-        #        "username": xxx,
-        #        "database": xxx,
-        #        "hostname": xxx
-        #    },
-        #    "quantum": {
-        #        "username": xxx,
-        #        "database": xxx,
-        #        "hostname": xxx
-        #    }
-        #}
-        #
-        databases = {}
-        for k, v in settings.iteritems():
-            db = k.split('_')[0]
-            x = '_'.join(k.split('_')[1:])
-            if db not in databases:
-                databases[db] = {}
-            databases[db][x] = v
-        return_data = {}
-        for db in databases:
-            if singleset.issubset(databases[db]):
-                return_data['_'.join([db, 'password'])] = \
-                    configure_db(databases[db]['hostname'],
-                                 databases[db]['database'],
-                                 databases[db]['username'])
-        if len(return_data) > 0:
-            utils.relation_set(**return_data)
-        if not cluster.is_clustered():
-            utils.relation_set(db_host=local_hostname)
-        else:
-            utils.relation_set(db_host=utils.config_get("vip"))
+        relation_set(db_host=utils.config_get("vip"))
 
 hooks = {
     "shared-db-relation-changed": shared_db_changed

=== modified file 'hooks/slave-relation-changed'
--- hooks/slave-relation-changed	2013-02-12 17:54:29 +0000
+++ hooks/slave-relation-changed	2014-02-24 11:48:01 +0000
@@ -87,3 +87,6 @@
 mysql $ROOTARGS -e "START SLAVE"
 mysql $ROOTARGS -e "SHOW SLAVE STATUS"
 touch /var/lib/juju/i.am.a.slave
+
+
+configs=json.loads(check_output(['config-get','--format=json']))

=== added file 'hooks/sslca.py'
--- hooks/sslca.py	1970-01-01 00:00:00 +0000
+++ hooks/sslca.py	2014-02-24 11:48:01 +0000
@@ -0,0 +1,253 @@
+import logging
+import datetime
+import os
+from os.path import join as path_join
+from os.path import exists
+import subprocess
+
+
+log = logging.getLogger("service_ca")
+
+logging.basicConfig(level=logging.DEBUG)
+
+STD_CERT = "standard"
+
+# Mysql server is fairly picky about cert creation
+# and types, spec its creation separately for now.
+MYSQL_CERT = "mysql"
+
+
+class ServiceCA(object):
+
+    default_expiry = str(365 * 2)
+    default_ca_expiry = str(365 * 6)
+
+    def __init__(self, name, ca_dir, cert_type=STD_CERT):
+        self.name = name
+        self.ca_dir = ca_dir
+        self.cert_type = cert_type
+
+    def init(self):
+        log.debug("initializing service ca")
+        if not exists(self.ca_dir):
+            self._init_ca_dir(self.ca_dir)
+            self._init_ca()
+
+    @property
+    def ca_key(self):
+        return path_join(self.ca_dir, 'private', 'cacert.key')
+
+    @property
+    def ca_cert(self):
+        return path_join(self.ca_dir, 'cacert.pem')
+
+    @property
+    def ca_conf(self):
+        return path_join(self.ca_dir, 'ca.cnf')
+
+    @property
+    def signing_conf(self):
+        return path_join(self.ca_dir, 'signing.cnf')
+
+    def _init_ca_dir(self, ca_dir):
+        os.mkdir(ca_dir)
+        for i in ['certs', 'crl', 'newcerts', 'private']:
+            sd = path_join(ca_dir, i)
+            if not exists(sd):
+                os.mkdir(sd)
+
+        if not exists(path_join(ca_dir, 'serial')):
+            with open(path_join(ca_dir, 'serial'), 'wb') as fh:
+                fh.write('02\n')
+
+        if not exists(path_join(ca_dir, 'index.txt')):
+            with open(path_join(ca_dir, 'index.txt'), 'wb') as fh:
+                fh.write('')
+
+    def _init_ca(self):
+        """Generate the root ca's cert and key.
+        """
+        if not exists(path_join(self.ca_dir, 'ca.cnf')):
+            with open(path_join(self.ca_dir, 'ca.cnf'), 'wb') as fh:
+                fh.write(
+                    CA_CONF_TEMPLATE % (self.get_conf_variables()))
+
+        if not exists(path_join(self.ca_dir, 'signing.cnf')):
+            with open(path_join(self.ca_dir, 'signing.cnf'), 'wb') as fh:
+                fh.write(
+                    SIGNING_CONF_TEMPLATE % (self.get_conf_variables()))
+
+        if exists(self.ca_cert) or exists(self.ca_key):
+            raise RuntimeError("Initialized called when CA already exists")
+        cmd = ['openssl', 'req', '-config', self.ca_conf,
+               '-x509', '-nodes', '-newkey', 'rsa',
+               '-days', self.default_ca_expiry,
+               '-keyout', self.ca_key, '-out', self.ca_cert,
+               '-outform', 'PEM']
+        output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
+        log.debug("CA Init:\n %s", output)
+
+    def get_conf_variables(self):
+        return dict(
+            org_name="juju",
+            org_unit_name="%s service" % self.name,
+            common_name=self.name,
+            ca_dir=self.ca_dir)
+
+    def get_or_create_cert(self, common_name):
+        if common_name in self:
+            return self.get_certificate(common_name)
+        return self.create_certificate(common_name)
+
+    def create_certificate(self, common_name):
+        if common_name in self:
+            return self.get_certificate(common_name)
+        key_p = path_join(self.ca_dir, "certs", "%s.key" % common_name)
+        crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
+        csr_p = path_join(self.ca_dir, "certs", "%s.csr" % common_name)
+        self._create_certificate(common_name, key_p, csr_p, crt_p)
+        return self.get_certificate(common_name)
+
+    def get_certificate(self, common_name):
+        if not common_name in self:
+            raise ValueError("No certificate for %s" % common_name)
+        key_p = path_join(self.ca_dir, "certs", "%s.key" % common_name)
+        crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
+        with open(crt_p) as fh:
+            crt = fh.read()
+        with open(key_p) as fh:
+            key = fh.read()
+            # Required for mysql client/server not sure if it matters
+            # to others.
+            #key = key.replace('BEGIN PRIVATE', 'BEGIN RSA PRIVATE')
+            #key = key.replace('END PRIVATE', 'END RSA PRIVATE')
+        return crt, key
+
+    def __contains__(self, common_name):
+        crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
+        return exists(crt_p)
+
+    def _create_certificate(self, common_name, key_p, csr_p, crt_p):
+        template_vars = self.get_conf_variables()
+        template_vars['common_name'] = common_name
+        subj = '/O=%(org_name)s/OU=%(org_unit_name)s/CN=%(common_name)s' % (
+            template_vars)
+
+        log.debug("CA Create Cert %s", common_name)
+        cmd = ['openssl', 'req', '-sha1', '-newkey', 'rsa:2048',
+               '-nodes', '-days', self.default_expiry,
+               '-keyout', key_p, '-out', csr_p, '-subj', subj]
+        subprocess.check_call(cmd)
+        cmd = ['openssl', 'rsa', '-in', key_p, '-out', key_p]
+        subprocess.check_call(cmd)
+
+        log.debug("CA Sign Cert %s", common_name)
+        if self.cert_type == MYSQL_CERT:
+            cmd = ['openssl', 'x509', '-req',
+                   '-in', csr_p, '-days', self.default_expiry,
+                   '-CA', self.ca_cert, '-CAkey', self.ca_key,
+                   '-set_serial', '01', '-out', crt_p]
+        else:
+            cmd = ['openssl', 'ca', '-config', self.signing_conf,
+                   '-extensions', 'req_extensions',
+                   '-days', self.default_expiry, '-notext',
+                   '-in', csr_p, '-out', crt_p, '-subj', subj, '-batch']
+        log.debug("running %s", " ".join(cmd))
+        subprocess.check_call(cmd)
+
+    def get_ca_bundle(self):
+        with open(self.ca_cert) as fh:
+            return fh.read()
+
+
+CA_CONF_TEMPLATE = """
+[ ca ]
+default_ca = CA_default
+
+[ CA_default ]
+dir                     = %(ca_dir)s
+policy                  = policy_match
+database                = $dir/index.txt
+serial                  = $dir/serial
+certs                   = $dir/certs
+crl_dir                 = $dir/crl
+new_certs_dir           = $dir/newcerts
+certificate             = $dir/cacert.pem
+private_key             = $dir/private/cacert.key
+RANDFILE                = $dir/private/.rand
+default_md              = default
+
+[ req ]
+default_bits            = 1024
+default_md              = sha1
+
+prompt                  = no
+distinguished_name      = ca_distinguished_name
+
+x509_extensions         = ca_extensions
+
+[ ca_distinguished_name ]
+organizationName        = %(org_name)s
+organizationalUnitName  = %(org_unit_name)s Certificate Authority
+
+
+[ policy_match ]
+countryName             = optional
+stateOrProvinceName     = optional
+organizationName        = match
+organizationalUnitName  = optional
+commonName              = supplied
+
+[ ca_extensions ]
+basicConstraints        = critical,CA:true
+subjectKeyIdentifier    = hash
+authorityKeyIdentifier  = keyid:always, issuer
+keyUsage                = cRLSign, keyCertSign
+"""
+
+
+SIGNING_CONF_TEMPLATE = """
+[ ca ]
+default_ca = CA_default
+
+[ CA_default ]
+dir                     = %(ca_dir)s
+policy                  = policy_match
+database                = $dir/index.txt
+serial                  = $dir/serial
+certs                   = $dir/certs
+crl_dir                 = $dir/crl
+new_certs_dir           = $dir/newcerts
+certificate             = $dir/cacert.pem
+private_key             = $dir/private/cacert.key
+RANDFILE                = $dir/private/.rand
+default_md              = default
+
+[ req ]
+default_bits            = 1024
+default_md              = sha1
+
+prompt                  = no
+distinguished_name      = req_distinguished_name
+
+x509_extensions         = req_extensions
+
+[ req_distinguished_name ]
+organizationName        = %(org_name)s
+organizationalUnitName  = %(org_unit_name)s machine resources
+commonName              = %(common_name)s
+
+[ policy_match ]
+countryName             = optional
+stateOrProvinceName     = optional
+organizationName        = match
+organizationalUnitName  = optional
+commonName              = supplied
+
+[ req_extensions ]
+basicConstraints        = CA:false
+subjectKeyIdentifier    = hash
+authorityKeyIdentifier  = keyid:always, issuer
+keyUsage                = digitalSignature, keyEncipherment, keyAgreement
+extendedKeyUsage        = serverAuth, clientAuth
+"""

=== modified file 'revision'
--- revision	2014-02-13 18:51:20 +0000
+++ revision	2014-02-24 11:48:01 +0000
@@ -1,1 +1,1 @@
-311
+321