Merge lp:~james-page/charms/raring/ceph/trunk into lp:~charmers/charms/precise/ceph/trunk

[James Page]

Updates to support newer releases of ceph.

Updates are backwards compatibile with precise/0.48.x series and have been testing using the following combos:

precise/0.48.2
quantal/0.48.2
raring/0.56.1

[lp:~james-page/charms/raring/ceph/trunk]

59. By James Page on 2013-01-24

Improve host ip resolution - fixes issues with maas managed DNS

[Jim Baker (jimbaker)]

My only concern is that in the function get_mon_hosts, it is possible that NXDOMAIN (which certainly seems could potentially happen) will result in None:6789 as one of the items in the hosts list:

            hosts.append(
                '{}:6789'.format(utils.get_host_ip(
                                    utils.relation_get('private-address',
                                                       unit, relid)))

This list is then passed on to ceph via /etc/ceph/ceph.conf, so this may be fine. But it also seems potentially problematic.

Maybe instead we should simply not append an unresolved address to hosts. Another possibility is to do a retry on get_host_ip. Alternatively instead of swallowing the error, we could force the juju admin to resolve this problem. That does sound more painful however.

[lp:~james-page/charms/raring/ceph/trunk]

60. By James Page on 2013-02-07

Allow get_host_ip helper to raise exception in the event that a IP address
cannot be resolved for the provided hostname.

[Mark Mims (mark-mims)]

this looks great.

This MP exposes a flaw in our current charm subnmission process. The alias lp:charms/ceph points to lp:~charmers/charms/precise/ceph/trunk. However, the proposed merge is from a raring branch. Should it go to precise, quantal, or should we do a new lp:~charmers/charms/raring/ceph/trunk and leave this merge out of the past series branches?

I'd reject because there's ambiguity in which branch(es) you'd like this merged into... but after talking on IRC, the branches are intended for precise even though they're raring branches. In the future we should submit against a specific series branch if that's possible and what you mean to do... otherwise, at least note the intended destination in the MP somewhere.

Thanks!

review: Approve

=== renamed file 'README' => 'README.md'
=== modified file 'config.yaml'
--- config.yaml	2012-11-22 10:47:03 +0000
+++ config.yaml	2013-02-07 16:02:24 +0000
@@ -39,6 +39,24 @@
       .
       These devices are the range of devices that will be checked for and
       used across all service units.
+  osd-journal:
+    type: string
+    description: |
+      The device to use as a shared journal drive for all OSD's.  By default
+      no journal device will be used.
+      .
+      Only supported with ceph >= 0.55.
+  osd-format:
+    type: string
+    default: xfs
+    description: |
+      Format of filesystem to use for OSD devices; supported formats include:
+      .
+        xfs (Default >= 0.55)
+        ext4 (Only option < 0.55)
+        btrfs (experimental and not recommended)
+      .
+      Only supported with ceph >= 0.55.      
   osd-reformat:
     type: string
     description: |
@@ -47,7 +65,7 @@
       prevent data loss.
       .
       Specifying this option (any value) forces a reformat of any OSD devices
-      found which are not already mounted.       
+      found which are not already mounted.
   ephemeral-unmount:
     type: string
     description: |

=== modified file 'hooks/ceph.py'
--- hooks/ceph.py	2012-10-19 15:50:18 +0000
+++ hooks/ceph.py	2013-02-07 16:02:24 +0000
@@ -12,6 +12,7 @@
 import time
 import utils
 import os
+import apt_pkg as apt
 
 LEADER = 'leader'
 PEON = 'peon'
@@ -84,6 +85,12 @@
         # Ignore any errors for this call
         subprocess.call(cmd)
 
+DISK_FORMATS = [
+    'xfs',
+    'ext4',
+    'btrfs'
+    ]
+
 
 def is_osd_disk(dev):
     try:
@@ -108,6 +115,11 @@
     subprocess.call(cmd)
 
 
+def zap_disk(dev):
+    cmd = ['sgdisk', '--zap-all', dev]
+    subprocess.check_call(cmd)
+
+
 _bootstrap_keyring = "/var/lib/ceph/bootstrap-osd/ceph.keyring"
 
 
@@ -207,3 +219,17 @@
             if 'key' in element:
                 key = element.split(' = ')[1].strip()  # IGNORE:E1103
     return key
+
+
+def get_ceph_version():
+    apt.init()
+    cache = apt.Cache()
+    pkg = cache['ceph']
+    if pkg.current_ver:
+        return apt.upstream_version(pkg.current_ver.ver_str)
+    else:
+        return None
+
+
+def version_compare(a, b):
+    return apt.version_compare(a, b)

=== modified file 'hooks/hooks.py'
--- hooks/hooks.py	2012-11-12 10:00:27 +0000
+++ hooks/hooks.py	2013-02-07 16:02:24 +0000
@@ -19,14 +19,16 @@
 
 
 def install_upstart_scripts():
-    for x in glob.glob('files/upstart/*.conf'):
-        shutil.copy(x, '/etc/init/')
+    # Only install upstart configurations for older versions
+    if ceph.get_ceph_version() < "0.55.1":
+        for x in glob.glob('files/upstart/*.conf'):
+            shutil.copy(x, '/etc/init/')
 
 
 def install():
     utils.juju_log('INFO', 'Begin install hook.')
     utils.configure_source()
-    utils.install('ceph', 'gdisk', 'ntp')
+    utils.install('ceph', 'gdisk', 'ntp', 'btrfs-tools')
     install_upstart_scripts()
     utils.juju_log('INFO', 'End install hook.')
 
@@ -36,35 +38,48 @@
         'auth_supported': utils.config_get('auth-supported'),
         'mon_hosts': ' '.join(get_mon_hosts()),
         'fsid': utils.config_get('fsid'),
+        'version': ceph.get_ceph_version()
         }
 
     with open('/etc/ceph/ceph.conf', 'w') as cephconf:
         cephconf.write(utils.render_template('ceph.conf', cephcontext))
 
+JOURNAL_ZAPPED = '/var/lib/ceph/journal_zapped'
+
 
 def config_changed():
     utils.juju_log('INFO', 'Begin config-changed hook.')
 
     utils.juju_log('INFO', 'Monitor hosts are ' + repr(get_mon_hosts()))
 
-    fsid = utils.config_get('fsid')
-    if fsid == '':
+    # Pre-flight checks
+    if not utils.config_get('fsid'):
         utils.juju_log('CRITICAL', 'No fsid supplied, cannot proceed.')
         sys.exit(1)
-
-    monitor_secret = utils.config_get('monitor-secret')
-    if monitor_secret == '':
+    if not utils.config_get('monitor-secret'):
         utils.juju_log('CRITICAL',
                        'No monitor-secret supplied, cannot proceed.')
         sys.exit(1)
+    if utils.config_get('osd-format') not in ceph.DISK_FORMATS:
+        utils.juju_log('CRITICAL',
+                       'Invalid OSD disk format configuration specified')
+        sys.exit(1)
 
     emit_cephconf()
 
     e_mountpoint = utils.config_get('ephemeral-unmount')
-    if (e_mountpoint != "" and
+    if (e_mountpoint and
         filesystem_mounted(e_mountpoint)):
         subprocess.call(['umount', e_mountpoint])
 
+    osd_journal = utils.config_get('osd-journal')
+    if (osd_journal and
+        not os.path.exists(JOURNAL_ZAPPED) and
+        os.path.exists(osd_journal)):
+        ceph.zap_disk(osd_journal)
+        with open(JOURNAL_ZAPPED, 'w') as zapped:
+            zapped.write('DONE')
+
     for dev in utils.config_get('osd-devices').split(' '):
         osdize(dev)
 
@@ -96,9 +111,22 @@
     return hosts
 
 
+def update_monfs():
+    hostname = utils.get_unit_hostname()
+    monfs = '/var/lib/ceph/mon/ceph-{}'.format(hostname)
+    upstart = '{}/upstart'.format(monfs)
+    if (os.path.exists(monfs) and
+        not os.path.exists(upstart)):
+        # Mark mon as managed by upstart so that
+        # it gets start correctly on reboots
+        with open(upstart, 'w'):
+            pass
+
+
 def bootstrap_monitor_cluster():
     hostname = utils.get_unit_hostname()
     done = '/var/lib/ceph/mon/ceph-{}/done'.format(hostname)
+    upstart = '/var/lib/ceph/mon/ceph-{}/upstart'.format(hostname)
     secret = utils.config_get('monitor-secret')
     keyring = '/var/lib/ceph/tmp/{}.mon.keyring'.format(hostname)
 
@@ -118,6 +146,8 @@
 
             with open(done, 'w'):
                 pass
+            with open(upstart, 'w'):
+                pass
 
             subprocess.check_call(['start', 'ceph-mon-all-starter'])
         except:
@@ -127,7 +157,7 @@
 
 
 def reformat_osd():
-    if utils.config_get('osd-reformat') != "":
+    if utils.config_get('osd-reformat'):
         return True
     else:
         return False
@@ -151,7 +181,23 @@
                        'Looks like {} is in use, skipping.'.format(dev))
         return
 
-    subprocess.call(['ceph-disk-prepare', dev])
+    cmd = ['ceph-disk-prepare']
+    # Later versions of ceph support more options
+    if ceph.get_ceph_version() >= "0.55":
+        osd_format = utils.config_get('osd-format')
+        if osd_format:
+            cmd.append('--fs-type')
+            cmd.append(osd_format)
+        cmd.append(dev)
+        osd_journal = utils.config_get('osd-journal')
+        if (osd_journal and
+            os.path.exists(osd_journal)):
+            cmd.append(osd_journal)
+    else:
+        # Just provide the device - no other options
+        # for older versions of ceph
+        cmd.append(dev)
+    subprocess.call(cmd)
 
 
 def device_mounted(dev):
@@ -272,6 +318,7 @@
     utils.juju_log('INFO', 'Begin upgrade-charm hook.')
     emit_cephconf()
     install_upstart_scripts()
+    update_monfs()
     utils.juju_log('INFO', 'End upgrade-charm hook.')
 
 

=== modified file 'hooks/utils.py'
--- hooks/utils.py	2012-11-12 09:32:06 +0000
+++ hooks/utils.py	2013-02-07 16:02:24 +0000
@@ -11,16 +11,19 @@
 import subprocess
 import socket
 import sys
+import re
 
 
 def do_hooks(hooks):
     hook = os.path.basename(sys.argv[0])
 
     try:
-        hooks[hook]()
+        hook_func = hooks[hook]
     except KeyError:
         juju_log('INFO',
                  "This charm doesn't know how to handle '{}'.".format(hook))
+    else:
+        hook_func()
 
 
 def install(*pkgs):
@@ -41,6 +44,12 @@
     install('python-jinja2')
     import jinja2
 
+try:
+    import dns.resolver
+except ImportError:
+    install('python-dnspython')
+    import dns.resolver
+
 
 def render_template(template_name, context, template_dir=TEMPLATES_DIR):
     templates = jinja2.Environment(
@@ -88,6 +97,18 @@
         ]
     subprocess.check_call(cmd)
 
+
+def enable_pocket(pocket):
+    apt_sources = "/etc/apt/sources.list"
+    with open(apt_sources, "r") as sources:
+        lines = sources.readlines()
+    with open(apt_sources, "w") as sources:
+        for line in lines:
+            if pocket in line:
+                sources.write(re.sub('^# deb', 'deb', line))
+            else:
+                sources.write(line)
+
 # Protocols
 TCP = 'TCP'
 UDP = 'UDP'
@@ -136,7 +157,11 @@
     cmd.append(attribute)
     if unit:
         cmd.append(unit)
-    return subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    value = str(subprocess.check_output(cmd)).strip()
+    if value == "":
+        return None
+    else:
+        return value
 
 
 def relation_set(**kwargs):
@@ -159,7 +184,11 @@
         'unit-get',
         attribute
         ]
-    return subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    value = str(subprocess.check_output(cmd)).strip()
+    if value == "":
+        return None
+    else:
+        return value
 
 
 def config_get(attribute):
@@ -167,7 +196,11 @@
         'config-get',
         attribute
         ]
-    return subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    value = str(subprocess.check_output(cmd)).strip()
+    if value == "":
+        return None
+    else:
+        return value
 
 
 def get_unit_hostname():
@@ -175,9 +208,13 @@
 
 
 def get_host_ip(hostname=unit_get('private-address')):
-    cmd = [
-        'dig',
-        '+short',
-        hostname
-        ]
-    return subprocess.check_output(cmd).strip()  # IGNORE:E1103
+    try:
+        # Test to see if already an IPv4 address
+        socket.inet_aton(hostname)
+        return hostname
+    except socket.error:
+        # This may throw an NXDOMAIN exception; in which case
+        # things are badly broken so just let it kill the hook
+        answers = dns.resolver.query(hostname, 'A')
+        if answers:
+            return answers[0].address

=== modified file 'revision'
--- revision	2012-11-12 10:00:27 +0000
+++ revision	2013-02-07 16:02:24 +0000
@@ -1,1 +1,1 @@
-88
+91

=== modified file 'templates/ceph.conf'
--- templates/ceph.conf	2012-10-18 06:19:52 +0000
+++ templates/ceph.conf	2013-02-07 16:02:24 +0000
@@ -1,5 +1,11 @@
 [global]
+{% if version < "0.51" %}
  auth supported = {{ auth_supported }}
+{% else %}
+ auth cluster required = {{ auth_supported }}
+ auth service required = {{ auth_supported }}
+ auth client required = {{ auth_supported }}
+{% endif %}
  keyring = /etc/ceph/$cluster.$name.keyring
  mon host = {{ mon_hosts }}
  fsid = {{ fsid }}