Merge lp:~wesmason/charms/precise/mongodb/add-storage-subordinate-support into lp:charms/mongodb

[Wes Mason]

This adds support for the storage subordinate as a way to automatically handle attaching/mounting volumes from both OpenStack and AWS EBS, by using the data- relation hooks to setup a /srv/juju/mongodb-data mount which is then used by the existing volume management code to symlink in place and rsync any existing MongoDB data over, similar to the way the postgres charm works with the subordinate.

We need this internally to support automated Mongo-on-persistent-volume deploys with juju-deployer.

[Wes Mason (wesmason)]

Bug 1206468 is related to this.

[Stuart Bishop (stub)]

This all seems good to me.

The charm could be tidied up with support of charmhelpers, eg. using @hooks.hook instead of the if/then statement to invoke the correct hook handler. But that is unrelated to this branch, which is nicely contained and doesn't want the scope creep.

review: Approve

=== modified file 'README.md'
--- README.md	2014-02-19 15:47:55 +0000
+++ README.md	2014-06-18 11:23:00 +0000
@@ -164,10 +164,26 @@
 - `run sh.status()`
 You should see your the hosts for your shards in the status output.
 
+### Use the storage subordinate to store mongodb data on a permanent OpenStack or Amazon EBS volume
+
+The [storage](http://manage.jujucharms.com/charms/precise/storage) subordinate and [block-storage-broker](http://manage.jujucharms.com/charms/precise/block-storage-broker) service can automatically handle attaching the volume and mounting it to the unit before MongoDB is setup to use it.
+
+For example if you've created the volumes `vol-id-00001` and `vol-id-00002` and want to attach them to your 2 mongo units, with your OpenStack or AWS credentials in a `credential.yaml` file:
+
+    juju deploy block-storage-broker --config credentials.yaml
+    juju deploy storage
+    juju add-relation block-storage-broker storage
+    juju set storage provider=block-storage-broker
+    juju set volume_map="{mongodb/0: vol-id-00001, mongodb/1: vol-id-00002}"
+    juju add-relation storage mongodb
+
+
+### Use a permanent Openstack volume to store mongodb data. (DEPRECATED)
+
+**Note**: Although these steps will still work they are now deprecated, you should use the storage subordinate above instead.
+
 To deploy mongodb using permanent volume on Openstack, the permanent volume should be attached to the mongodb unit just after the deployment, then the configuration should be updated like follows.
 
-### Use a permanent Openstack volume to store mongodb data.
-
     juju set mongodb volume-dev-regexp="/dev/vdc" volume-map='{"mongodb/0": "vol-id-00000000000000"}' volume-ephemeral-storage=false
 
 ### Backups

=== modified file 'config.yaml'
--- config.yaml	2014-04-11 21:00:49 +0000
+++ config.yaml	2014-06-18 11:23:00 +0000
@@ -168,7 +168,7 @@
     type: int
     description: "Number of backups to keep. Keeps one week's worth by default."
   #------------------------------------------------------------------------
-  # Volume management
+  # Legacy volume management (DEPRECATED)
   #   volume-map, volume-dev_regexp are only used 
   #   if volume-ephemeral-storage == False
   #------------------------------------------------------------------------
@@ -176,6 +176,7 @@
     type: boolean
     default: true
     description: >
+      Deprecated, use the storage subordinate.
       If false, a configure-error state will be raised if
          volume-map[$JUJU_UNIT_NAME] is not set (see "volume-map"
          below) - see "volume-map" below.
@@ -187,6 +188,7 @@
     type: string
     default: ""
     description: >
+      Deprecated, use the storage subordinate.
       YAML map as e.g. "{ mongodb/0: vol-0000010, mongodb/1: vol-0000016 }".
       Service units will raise a "configure-error" condition if no volume-map
       value is set for it - it expects a human to set it properly to resolve it.
@@ -194,6 +196,7 @@
     type: string
     default: "/dev/vd[b-z]"
     description: >
+      Deprecated, use the storage subordinate.
       Block device for attached volumes as seen by the VM, will be "scanned"
       for an unused device when "volume-map" is valid for the unit.
   source:

=== added symlink 'hooks/data-relation-changed'
=== target is u'hooks.py'
=== added symlink 'hooks/data-relation-departed'
=== target is u'hooks.py'
=== added symlink 'hooks/data-relation-joined'
=== target is u'hooks.py'
=== modified file 'hooks/hooks.py'
--- hooks/hooks.py	2014-04-11 21:00:49 +0000
+++ hooks/hooks.py	2014-06-18 11:23:00 +0000
@@ -212,6 +212,20 @@
         return(relation_data)
 
 
+def relation_ids(relation_name=None):
+    juju_log("relation_ids: relation_name: %s" % relation_name)
+    try:
+        relation_cmd_line = ['relation-ids', '--format=json']
+        if relation_name is not None:
+            relation_cmd_line.append(relation_name)
+        relation_data = json.loads(subprocess.check_output(relation_cmd_line))
+    except Exception, e:
+        juju_log(str(e))
+        relation_data = None
+    finally:
+        juju_log("relation_ids %s returns: %s" % (relation_name, relation_data))
+        return(relation_data)
+
 #------------------------------------------------------------------------------
 # open_port:  Convenience function to open a port in juju to
 #             expose a service
@@ -930,7 +944,7 @@
             "'volume-ephemeral-storage' and 'volume-map'")
         sys.exit(1)
     if volume_is_permanent(volid):
-        ## config_changed_volume_apply will stop the service if it founds
+        ## config_changed_volume_apply will stop the service if it finds
         ## it necessary, ie: new volume setup
         if config_changed_volume_apply():
             start_hook()
@@ -1159,6 +1173,30 @@
     return(True)
 
 
+def data_relation_joined():
+    juju_log("data_relation_joined")
+
+    return(relation_set(
+        {
+            'mountpoint': '/srv/juju/mongodb-data'
+        }))
+
+
+def data_relation_changed():
+    juju_log("data_relation_changed")
+
+    if volume_get_id_for_storage_subordinate() is None:
+        juju_log("mountpoint from storage subordinate not ready, let's wait")
+        return(True)
+
+    return(config_changed())
+
+
+def data_relation_departed():
+    juju_log("data_relation_departed")
+    return(config_changed())
+
+
 def configsvr_relation_joined():
     juju_log("configsvr_relation_joined")
     my_hostname = unit_get('public-address')
@@ -1312,10 +1350,37 @@
     return None
 
 
+#------------------------------
+# Returns a stub volume id based on the mountpoint from the storage
+# subordinate relation, if present.
+#
+# @return volid  eg vol-000012345
+#------------------------------
+def volume_get_id_for_storage_subordinate():
+    # storage charm is a subordinate so we should only ever have one
+    # relation_id for the data relation
+    ids = relation_ids('data')
+    if len(ids) > 0:
+        mountpoint = relation_get('mountpoint',
+                                   os.environ['JUJU_UNIT_NAME'],
+                                   ids[0])
+
+        juju_log('mountpoint: %s' % (mountpoint,))
+        if mountpoint and os.path.exists(mountpoint):
+            return mountpoint.split('/')[-1]
+
+
+
 # Do we have a valid storage state?
 # @returns  volid
 #           None    config state is invalid - we should not serve
 def volume_get_volume_id():
+
+
+    volid = volume_get_id_for_storage_subordinate()
+    if volid:
+        return volid
+
     config_data = config_get()
     ephemeral_storage = config_data['volume-ephemeral-storage']
     volid = volume_get_volid_from_volume_map()
@@ -1520,6 +1585,12 @@
         retVal = mongos_relation_changed()
     elif hook_name == "mongos-relation-broken":
         retVal = mongos_relation_broken()
+    elif hook_name == "data-relation-joined":
+        retVal = data_relation_joined()
+    elif hook_name == "data-relation-changed":
+        retVal = data_relation_changed()
+    elif hook_name == "data-relation-departed":
+        retVal = data_relation_departed()
     else:
         print "Unknown hook"
         retVal = False

=== modified file 'metadata.yaml'
--- metadata.yaml	2014-01-09 20:41:59 +0000
+++ metadata.yaml	2014-06-18 11:23:00 +0000
@@ -22,6 +22,10 @@
     interface: mongodb
   configsvr:
     interface: shard
+  data:
+    interface: block-storage
+    scope: container
+    optional: true
 requires:
   mongos-cfg:
     interface: shard