Merge lp:~james-page/charms/raring/ceph-radosgw/trunk into lp:~charmers/charms/precise/ceph-radosgw/trunk

[James Page]

Updates to support newer releases of ceph.

Updates are backwards compatibile with precise/0.48.x series and have been testing using the following combos:

precise/0.48.2
quantal/0.48.2
raring/0.56.1

This branch also adds support for using the RADOS gateway with Openstack Keystone for swift authentication.

[lp:~james-page/charms/raring/ceph-radosgw/trunk]

12. By James Page on 2013-01-24

Improve host ip resolution - fixes issues with maas managed DNS

13. By James Page on 2013-02-07

Allow DNS problems to bubble up and break hooks as things are broken if this
is the case.

[Mark Mims (mark-mims)]

this looks great.

This MP exposes a flaw in our current charm subnmission process. The alias lp:charms/ceph points to lp:~charmers/charms/precise/ceph/trunk. However, the proposed merge is from a raring branch. Should it go to precise, quantal, or should we do a new lp:~charmers/charms/raring/ceph/trunk and leave this merge out of the past series branches?

I'd reject because there's ambiguity in which branch(es) you'd like this merged into... but after talking on IRC, the branches are intended for precise even though they're raring branches. In the future we should submit against a specific series branch if that's possible and what you mean to do... otherwise, at least note the intended destination in the MP somewhere.

Thanks!

review: Approve

=== renamed file 'README' => 'README.md'
--- README	2012-10-10 09:03:18 +0000
+++ README.md	2013-02-07 16:03:24 +0000
@@ -29,15 +29,31 @@
 The gateway can be accessed over port 80 (as show in juju status exposed
 ports).
 
+Access
+======
+
 Note that you will need to login to one of the service units supporting the
 ceph charm to generate some access credentials::
 
    juju ssh ceph/0 \
       'sudo radosgw-admin user create --uid="ubuntu" --display-name="Ubuntu Ceph"'
-   
+
 For security reasons the ceph-radosgw charm is not setup with appropriate
 permissions to administer the ceph cluster.
 
+Keystone Integration
+====================
+
+Ceph >= 0.55 integrates with Openstack Keystone for authentication of Swift requests.
+
+This is enabled by relating the ceph-radosgw service with keystone::
+
+   juju deploy keystone
+   juju add-relation keystone ceph-radosgw
+
+If you try to relate the radosgw to keystone with an earlier version of ceph the hook
+will error out to let you know.
+
 Scale-out
 =========
 
@@ -62,8 +78,7 @@
 Bootnotes
 =========
 
-The Ceph RADOS Gateway makes use of a multiverse package,
-libapache2-mod-fastcgi.   As such it will try to automatically enable the
-multiverse pocket in /etc/apt/sources.list.  Note that there is noting
-'wrong' with multiverse components - they typically have less liberal
-licensing policies or suchlike.
+The Ceph RADOS Gateway makes use of a multiverse package libapache2-mod-fastcgi.
+As such it will try to automatically enable the multiverse pocket in
+/etc/apt/sources.list.  Note that there is noting 'wrong' with multiverse
+components - they typically have less liberal licensing policies or suchlike.

=== modified file 'config.yaml'
--- config.yaml	2012-11-22 10:47:49 +0000
+++ config.yaml	2013-02-07 16:03:24 +0000
@@ -20,3 +20,24 @@
     description: |
       Key ID to import to the apt keyring to support use with arbitary source
       configuration from outside of Launchpad archives or PPA's.
+  # Keystone integration
+  operator-roles:
+    default: "Member,Admin"
+    type: string
+    description: |
+      Comma-separated list of Swift operator roles; used when integrating with
+      OpenStack Keystone.
+  region:
+    default: RegionOne
+    type: string
+    description: |
+      OpenStack region that the RADOS gateway supports; used when integrating with
+      OpenStack Keystone.
+  cache-size:
+    default: 500
+    type: int
+    description: Number of keystone tokens to hold in local cache.
+  revocation-check-interval:
+    default: 600
+    type: int
+    description: Interval between revocation checks to keystone.

=== modified file 'hooks/ceph.py'
--- hooks/ceph.py	2012-10-09 15:12:25 +0000
+++ hooks/ceph.py	2013-02-07 16:03:24 +0000
@@ -12,8 +12,11 @@
 import time
 import utils
 import os
+import apt_pkg as apt
 
-QUORUM = ['leader', 'peon']
+LEADER = 'leader'
+PEON = 'peon'
+QUORUM = [LEADER, PEON]
 
 
 def is_quorum():
@@ -40,6 +43,30 @@
         return False
 
 
+def is_leader():
+    asok = "/var/run/ceph/ceph-mon.{}.asok".format(utils.get_unit_hostname())
+    cmd = [
+        "ceph",
+        "--admin-daemon",
+        asok,
+        "mon_status"
+        ]
+    if os.path.exists(asok):
+        try:
+            result = json.loads(subprocess.check_output(cmd))
+        except subprocess.CalledProcessError:
+            return False
+        except ValueError:
+            # Non JSON response from mon_status
+            return False
+        if result['state'] == LEADER:
+            return True
+        else:
+            return False
+    else:
+        return False
+
+
 def wait_for_quorum():
     while not is_quorum():
         time.sleep(3)
@@ -58,6 +85,12 @@
         # Ignore any errors for this call
         subprocess.call(cmd)
 
+DISK_FORMATS = [
+    'xfs',
+    'ext4',
+    'btrfs'
+    ]
+
 
 def is_osd_disk(dev):
     try:
@@ -72,9 +105,33 @@
         pass
     return False
 
+
+def rescan_osd_devices():
+    cmd = [
+        'udevadm', 'trigger',
+        '--subsystem-match=block', '--action=add'
+        ]
+
+    subprocess.call(cmd)
+
+
+def zap_disk(dev):
+    cmd = ['sgdisk', '--zap-all', dev]
+    subprocess.check_call(cmd)
+
+
 _bootstrap_keyring = "/var/lib/ceph/bootstrap-osd/ceph.keyring"
 
 
+def is_bootstrapped():
+    return os.path.exists(_bootstrap_keyring)
+
+
+def wait_for_bootstrap():
+    while (not is_bootstrapped()):
+        time.sleep(3)
+
+
 def import_osd_bootstrap_key(key):
     if not os.path.exists(_bootstrap_keyring):
         cmd = [
@@ -98,34 +155,7 @@
 
 
 def get_osd_bootstrap_key():
-    cmd = [
-        'ceph',
-        '--name', 'mon.',
-        '--keyring',
-        '/var/lib/ceph/mon/ceph-{}/keyring'.format(
-                                        utils.get_unit_hostname()
-                                        ),
-        'auth', 'get-or-create', 'client.bootstrap-osd',
-        ]
-    # Add capabilities
-    for subsystem, subcaps in _osd_bootstrap_caps.iteritems():
-        cmd.extend([
-            subsystem,
-            '; '.join(subcaps),
-            ])
-    output = subprocess.check_output(cmd).strip()  # IGNORE:E1103
-    # get-or-create appears to have different output depending
-    # on whether its 'get' or 'create'
-    # 'create' just returns the key, 'get' is more verbose and
-    # needs parsing
-    key = None
-    if len(output.splitlines()) == 1:
-        key = output
-    else:
-        for element in output.splitlines():
-            if 'key' in element:
-                key = element.split(' = ')[1].strip()  # IGNORE:E1103
-    return key
+    return get_named_key('bootstrap-osd', _osd_bootstrap_caps)
 
 
 _radosgw_keyring = "/etc/ceph/keyring.rados.gateway"
@@ -150,6 +180,17 @@
 
 
 def get_radosgw_key():
+    return get_named_key('radosgw.gateway', _radosgw_caps)
+
+
+_default_caps = {
+    'mon': ['allow r'],
+    'osd': ['allow rwx']
+    }
+
+
+def get_named_key(name, caps=None):
+    caps = caps or _default_caps
     cmd = [
         'ceph',
         '--name', 'mon.',
@@ -157,10 +198,10 @@
         '/var/lib/ceph/mon/ceph-{}/keyring'.format(
                                         utils.get_unit_hostname()
                                         ),
-        'auth', 'get-or-create', 'client.radosgw.gateway',
+        'auth', 'get-or-create', 'client.{}'.format(name),
         ]
     # Add capabilities
-    for subsystem, subcaps in _radosgw_caps.iteritems():
+    for subsystem, subcaps in caps.iteritems():
         cmd.extend([
             subsystem,
             '; '.join(subcaps),
@@ -178,3 +219,17 @@
             if 'key' in element:
                 key = element.split(' = ')[1].strip()  # IGNORE:E1103
     return key
+
+
+def get_ceph_version(package=None):
+    apt.init()
+    cache = apt.Cache()
+    pkg = cache[package or 'ceph']
+    if pkg.current_ver:
+        return apt.upstream_version(pkg.current_ver.ver_str)
+    else:
+        return None
+
+
+def version_compare(a, b):
+    return apt.version_compare(a, b)

=== modified file 'hooks/hooks.py'
--- hooks/hooks.py	2012-10-19 15:51:24 +0000
+++ hooks/hooks.py	2013-02-07 16:03:24 +0000
@@ -22,6 +22,9 @@
         shutil.copy(x, '/var/www/')
 
 
+NSS_DIR='/var/lib/ceph/nss'
+
+
 def install():
     utils.juju_log('INFO', 'Begin install hook.')
     utils.enable_pocket('multiverse')
@@ -30,6 +33,7 @@
                   'libapache2-mod-fastcgi',
                   'apache2',
                   'ntp')
+    os.makedirs(NSS_DIR)
     utils.juju_log('INFO', 'End install hook.')
 
 
@@ -41,8 +45,17 @@
     cephcontext = {
         'auth_supported': get_auth() or 'none',
         'mon_hosts': ' '.join(get_mon_hosts()),
-        'hostname': utils.get_unit_hostname()
+        'hostname': utils.get_unit_hostname(),
+        'version': ceph.get_ceph_version('radosgw')
         }
+    
+    # Check to ensure that correct version of ceph is 
+    # in use
+    if ceph.get_ceph_version('radosgw') >= "0.55":    
+        # Add keystone configuration if found
+        ks_conf = get_keystone_conf()
+        if ks_conf:
+            cephcontext.update(ks_conf)
 
     with open('/etc/ceph/ceph.conf', 'w') as cephconf:
         cephconf.write(utils.render_template('ceph.conf', cephcontext))
@@ -108,16 +121,33 @@
         for unit in utils.relation_list(relid):
             conf = utils.relation_get(name,
                                       unit, relid)
-            if conf != "":
+            if conf:
                 return conf
     return None
 
+def get_keystone_conf():
+    for relid in utils.relation_ids('identity-service'):
+        for unit in utils.relation_list(relid):
+            ks_auth = {
+                'auth_type': 'keystone',
+                'auth_protocol': 'http',
+                'auth_host': utils.relation_get('auth_host', unit, relid),
+                'auth_port': utils.relation_get('auth_port', unit, relid),
+                'admin_token': utils.relation_get('admin_token', unit, relid),
+                'user_roles': utils.config_get('operator-roles'),
+                'cache_size': utils.config_get('cache-size'),
+                'revocation_check_interval': utils.config_get('revocation-check-interval')
+            }
+            if None not in ks_auth.itervalues():
+                return ks_auth
+    return None
+
 
 def mon_relation():
     utils.juju_log('INFO', 'Begin mon-relation hook.')
     emit_cephconf()
     key = utils.relation_get('radosgw_key')
-    if key != "":
+    if key:
         ceph.import_radosgw_key(key)
         restart()  # TODO figure out a better way todo this
     utils.juju_log('INFO', 'End mon-relation hook.')
@@ -141,7 +171,7 @@
 
 
 def stop():
-    subprocess.call(['service', 'radosgw', 'start'])
+    subprocess.call(['service', 'radosgw', 'stop'])
     utils.expose(port=80)
 
 
@@ -150,6 +180,28 @@
     utils.expose(port=80)
 
 
+def identity_joined(relid=None):
+    if ceph.get_ceph_version('radosgw') < "0.55":
+        utils.juju_log('ERROR',
+                       'Integration with keystone requires ceph >= 0.55')
+        sys.exit(1)
+
+    hostname = utils.unit_get('private-address')
+    admin_url = 'http://{}:80/swift'.format(hostname)
+    internal_url = public_url = '{}/v1'.format(admin_url)
+    utils.relation_set(service='swift',
+                       region=utils.config_get('region'),
+                       public_url=public_url, internal_url=internal_url,
+                       admin_url=admin_url,
+                       requested_roles=utils.config_get('operator-roles'),
+                       rid=relid)
+
+
+def identity_changed():
+    emit_cephconf()
+    restart() 
+
+
 utils.do_hooks({
         'install': install,
         'config-changed': config_changed,
@@ -157,6 +209,8 @@
         'mon-relation-changed': mon_relation,
         'gateway-relation-joined': gateway_relation,
         'upgrade-charm': config_changed,  # same function ATM
+        'identity-service-relation-joined': identity_joined,
+        'identity-service-relation-changed': identity_changed
         })
 
 sys.exit(0)

=== modified file 'hooks/utils.py'
--- hooks/utils.py	2012-11-12 09:46:54 +0000
+++ hooks/utils.py	2013-02-07 16:03:24 +0000
@@ -18,10 +18,12 @@
     hook = os.path.basename(sys.argv[0])
 
     try:
-        hooks[hook]()
+        hook_func = hooks[hook]
     except KeyError:
         juju_log('INFO',
                  "This charm doesn't know how to handle '{}'.".format(hook))
+    else:
+        hook_func()
 
 
 def install(*pkgs):
@@ -42,6 +44,12 @@
     install('python-jinja2')
     import jinja2
 
+try:
+    import dns.resolver
+except ImportError:
+    install('python-dnspython')
+    import dns.resolver
+
 
 def render_template(template_name, context, template_dir=TEMPLATES_DIR):
     templates = jinja2.Environment(
@@ -101,7 +109,6 @@
             else:
                 sources.write(line)
 
-
 # Protocols
 TCP = 'TCP'
 UDP = 'UDP'
@@ -150,15 +157,25 @@
     cmd.append(attribute)
     if unit:
         cmd.append(unit)
-    return subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    value = str(subprocess.check_output(cmd)).strip()
+    if value == "":
+        return None
+    else:
+        return value
 
 
 def relation_set(**kwargs):
     cmd = [
         'relation-set'
         ]
+    args = []
     for k, v in kwargs.items():
-        cmd.append('{}={}'.format(k, v))
+        if k == 'rid':
+            cmd.append('-r')
+            cmd.append(v)
+        else:
+            args.append('{}={}'.format(k, v))
+    cmd += args
     subprocess.check_call(cmd)
 
 
@@ -167,7 +184,11 @@
         'unit-get',
         attribute
         ]
-    return subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    value = str(subprocess.check_output(cmd)).strip()
+    if value == "":
+        return None
+    else:
+        return value
 
 
 def config_get(attribute):
@@ -175,7 +196,11 @@
         'config-get',
         attribute
         ]
-    return subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    value = str(subprocess.check_output(cmd)).strip()
+    if value == "":
+        return None
+    else:
+        return value
 
 
 def get_unit_hostname():
@@ -183,9 +208,13 @@
 
 
 def get_host_ip(hostname=unit_get('private-address')):
-    cmd = [
-        'dig',
-        '+short',
-        hostname
-        ]
-    return subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    try:
+        # Test to see if already an IPv4 address
+        socket.inet_aton(hostname)
+        return hostname
+    except socket.error:
+        # This may throw an NXDOMAIN exception; in which case
+        # things are badly broken so just let it kill the hook
+        answers = dns.resolver.query(hostname, 'A')
+        if answers:
+            return answers[0].address

=== modified file 'metadata.yaml'
--- metadata.yaml	2012-10-08 15:58:16 +0000
+++ metadata.yaml	2013-02-07 16:03:24 +0000
@@ -10,6 +10,8 @@
 requires:
   mon:
     interface: ceph-radosgw
+  identity-service:
+    interface: keystone
 provides:
   gateway:
     interface: http

=== modified file 'revision'
--- revision	2012-11-22 01:09:33 +0000
+++ revision	2013-02-07 16:03:24 +0000
@@ -1,1 +1,1 @@
-15
+21

=== modified file 'templates/ceph.conf'
--- templates/ceph.conf	2012-10-19 15:51:24 +0000
+++ templates/ceph.conf	2013-02-07 16:03:24 +0000
@@ -1,5 +1,11 @@
 [global]
+{% if version < "0.51" %}
  auth supported = {{ auth_supported }}
+{% else %}
+ auth cluster required = {{ auth_supported }}
+ auth service required = {{ auth_supported }}
+ auth client required = {{ auth_supported }}
+{% endif %}
  mon host = {{ mon_hosts }}
 
 [client.radosgw.gateway]
@@ -7,3 +13,14 @@
  keyring = /etc/ceph/keyring.rados.gateway
  rgw socket path = /tmp/radosgw.sock
  log file = /var/log/ceph/radosgw.log
+ # Turn off 100-continue optimization as stock mod_fastcgi
+ # does not support it
+ rgw print continue = false 
+{% if auth_type == 'keystone' %}
+ rgw keystone url = {{ auth_protocol }}://{{ auth_host }}:{{ auth_port }}/
+ rgw keystone admin token = {{ admin_token }}
+ rgw keystone accepted roles = {{ user_roles }}
+ rgw keystone token cache size = {{ cache_size }}
+ rgw keystone revocation interval = {{ revocation_check_interval }}
+#nss db path = /var/lib/ceph/nss
+{% endif %}
\ No newline at end of file