Merge lp:~fginther/charms/precise/ubuntu-ci-services-itself/rabbitmq-worker-lp into lp:~canonical-ci-engineering/charms/precise/ubuntu-ci-services-itself/lander

[Francis Ginther]

Add the ability to pass launchpad credentials to the worker node, uses a base64 encoded file called "launchpad.credentials".

Add the ability to pass launchpad credentials to the worker node, uses a base64 encoded file called "launchpad.credentials".

[Andy Doan (doanac)]

could we use "unit_config" for this also and include lpcredentials as a value in it? or alternatively just use the unit_config we have now which I think includes all the information needed to create this credentials file?

NOTE: I've proposed moving this charm:

 https://code.launchpad.net/~doanac/ubuntu-ci-services-itself/rabbitwork-local/+merge/203830

you probably want to rebase this MP on to that

[Francis Ginther (fginther)]

Merge into the main project trunk and use unit-config.

[Francis Ginther (fginther)]

"NEED" to merge into the main project and use unit-config. Rejecting this MP.

Unmerged revisions

14. By Francis Ginther on 2014-01-29

Add import base64.

13. By Francis Ginther on 2014-01-26

Add some lp_creds debugging.

12. By Francis Ginther on 2014-01-25

Adjust file path to remove hidden hint.

11. By Francis Ginther on 2014-01-25

Add launchpa credentials file.

10. By Paul Larson on 2014-01-24

add support for installing pip packages

9. By Andy Doan on 2014-01-16

allow user/group for service to be configurable

8. By Andy Doan on 2013-12-10

update how amqp parameters are set

allow the worker to dynamically load the config from a file. If
you don't do this, you have to issue a "stop <service>; start <service>"
in order for upstart to detect a change from a bad config to a good
config

7. By Andy Doan on 2013-12-09

give a limit to the retries for the upstart job

6. By Andy Doan on 2013-12-07

fix quoting issue

5. By Andy Doan on 2013-12-07

fix eol issue on relation_get()

=== added file 'README.ex'
--- README.ex	1970-01-01 00:00:00 +0000
+++ README.ex	2014-01-30 11:47:01 +0000
@@ -0,0 +1,41 @@
+Describe the intended usage of this charm and anything unique about how
+this charm relates to others here. 
+
+This README will be displayed in the Charm Store, it should be either Markdown or RST. Ideal READMEs include instructions on how to use the charm, expected usage, and charm features that your audience might be interested in. For an example of a well written README check out Hadoop: http://jujucharms.com/charms/precise/hadoop
+
+Here's an example you might wish to template off of:
+
+Overview
+--------
+
+This charm provides (service) from (service homepage). Add a description here of what the service itself actually does. 
+
+
+Usage
+-----
+
+Step by step instructions on using the charm:
+
+    juju deploy servicename
+
+and so on. If you're providing a web service or something that the end user needs to go to, tell them here, especially if you're deploying a service that might listen to a non-default port. 
+
+You can then browse to http://ip-address to configure the service. 
+
+Configuration
+-------------
+
+The configuration options will be listed on the charm store, however If you're making assumptions or opinionated decisions in the charm (like setting a default administrator password), you should detail that here so the user knows how to change it immediately, etc.
+
+
+Contact Information
+-------------------
+
+Though this will be listed in the charm store itself don't assume a user will know that, so include that information here:
+
+Author:
+Report bugs at: http://bugs.launchpad.net/charms/+source/charmname
+Location: http://jujucharms.com/charms/distro/charmname
+
+* Be sure to remove the templated parts before submitting to https://launchpad.net/charms for inclusion in the charm store.
+
 
=== added file 'config.yaml'
--- config.yaml	1970-01-01 00:00:00 +0000
+++ config.yaml	2014-01-30 11:47:01 +0000
@@ -0,0 +1,44 @@
+options:
+  branch:
+    type: string
+    description: "BZR branch the service lives in"
+  revno:
+    type: string
+    description: "Revision or tag to branch from"
+  packages:
+    type: string
+    description: "Packages required for this service"
+  pip-packages:
+    type: string
+    description: "Pip packages required for this service"
+  install_root:
+    type: string
+    description: "The root directory the service will be installed in"
+    default: "/srv/"
+  main:
+    type: string
+    description: "The worker script to run as a service. Can include paths relative to local bzr directory"
+
+  uid:
+    type: string
+    default: nobody
+    description: User to run service as
+  gid:
+    type: string
+    default: nogroup
+    description: Group to run service as
+
+  # required for rabbitmq-server charm:
+  amqp-user:
+      type: string
+      default: 'workerbee'
+      description: The user to log into the rabbitMQ server.
+  amqp-vhost:
+      type: string
+      default: '/'
+      description: The vhost in the rabbitMQ server.
+
+  lp_creds:
+      type: string
+      default: ''
+      description: The launchpad credenetials file.
 
=== renamed file 'config.yaml' => 'config.yaml.moved'
=== added directory 'hooks'
=== renamed directory 'hooks' => 'hooks.moved'
=== added symlink 'hooks/amqp-relation-broken'
=== target is u'hooks.py'
=== added symlink 'hooks/amqp-relation-changed'
=== target is u'hooks.py'
=== added symlink 'hooks/amqp-relation-joined'
=== target is u'hooks.py'
=== added symlink 'hooks/config-changed'
=== target is u'hooks.py'
=== added file 'hooks/hooks.py'
--- hooks/hooks.py	1970-01-01 00:00:00 +0000
+++ hooks/hooks.py	2014-01-30 11:47:01 +0000
@@ -0,0 +1,171 @@
+#!/usr/bin/env python
+
+import base64
+import os
+import json
+import subprocess
+import sys
+import textwrap
+
+
+def juju_info(msg):
+    subprocess.check_call(['juju-log', '-l', 'INFO', msg])
+    pass
+
+
+def _config():
+    output = subprocess.check_output(['config-get', '--format=json'])
+    return json.loads(output)
+
+
+def _relation_get(key):
+    return subprocess.check_output(['relation-get', key]).strip()
+
+
+def _relation_set(keyvalues, relation_id=None):
+    args = ['relation-set']
+    if relation_id:
+        args.extend(['-r', relation_id])
+    args.extend(["{}={}".format(k, v or '') for k, v in keyvalues.items()])
+    subprocess.check_call(args)
+
+
+def _service_name(config):
+    unit = os.environ['JUJU_UNIT_NAME'].split('/')[0]
+    for x in (':', '-', '/', '"', "'"):
+        unit = unit.replace(x, '_')
+    return unit
+
+
+def _service_dir(config):
+    return os.path.join(config['install_root'], _service_name(config))
+
+
+def pip_install(package):
+    cmd_line = ['pip', 'install', '-b', '/tmp/']
+    if package.startswith('svn+') or package.startswith('git+') or \
+       package.startswith('hg+') or package.startswith('bzr+'):
+           cmd_line.append('-e')
+    cmd_line.append(package)
+    return(subprocess.call(cmd_line))
+
+
+def config_changed(config):
+    lp_creds = config.get('lp_creds', None)
+    juju_info('lp_creds (raw): %s' % (lp_creds))
+    if lp_creds:
+        lp_creds = str(base64.b64decode(lp_creds))
+        juju_info('lp_creds (decoded): %s' % (lp_creds))
+        with open(os.path.join(_service_dir(config), 'launchpad.credentials'),
+                  'w') as f:
+            f.write(lp_creds)
+
+
+def install(config):
+    pkgs = [x for x in config.get('packages', '').split(' ') if x]
+    pkgs.append('python-amqplib')
+    pkgs.append('python-pip')
+    pkgs.append('bzr')
+
+    juju_info('installing apt packages...')
+    subprocess.check_call(['apt-get', 'install', '-y', '-q'] + pkgs)
+
+    pip_pkgs = [x for x in config.get('pip-packages', '').split(' ') if x]
+    if pip_pkgs:
+        juju_info('installing pip packages...')
+        for package in pip_pkgs:
+            pip_install(package.strip())
+
+    juju_info('grabbing service from bzr...')
+    args = ['bzr', 'branch']
+    rev = config.get('revno', '')
+    if rev:
+        args.extend(['-r', rev])
+    args.append(config['branch'])
+    args.append(_service_dir(config))
+    subprocess.check_call(args)
+
+
+def _create_upstart(config):
+    template = textwrap.dedent('''
+    #--------------------------------------------------------------
+    # This file is managed by the rabbitmq-worker Juju charm
+    #--------------------------------------------------------------
+    description "starts a rabbitmq worker"
+    start on (local-filesystems and net-device-up IFACE=eth0)
+    stop on runlevel [!12345]
+
+    # If the process quits unexpectadly trigger a respawn
+    # give it 15 chances with a 5 second delay in between retries
+    respawn limit 15 5
+
+    setuid {uid}
+    setgid {gid}
+    chdir {sdir}
+
+    exec {main}
+    ''')
+    params = {
+        'main': config['main'],
+        'sdir': _service_dir(config),
+        'uid': config.get('uid', 'nobody'),
+        'gid': config.get('gid', 'nogroup'),
+    }
+    with open('/etc/init/%s.conf' % _service_name(config), 'w') as f:
+        f.write(template.format(**params))
+        # need this so the file is ready for the subprocess call
+        f.flush()
+        os.fsync(f.fileno())
+
+
+def _create_amqp_config(config):
+    with open(os.path.join(_service_dir(config), 'amqp_config.py'), 'w') as f:
+        f.write('# DO NOT EDIT. Generated by restish charm hook\n')
+        f.write('AMQP_USER = "%s"\n' % config['amqp-user'])
+        f.write('AMQP_VHOST = "%s"\n' % config['amqp-vhost'])
+        f.write('AMQP_HOST = "%s"\n' % _relation_get('private-address'))
+        f.write('AMQP_PASSWORD = "%s"\n' % _relation_get('password'))
+
+
+def amqp_relation_joined(config):
+    _relation_set({
+        'username': config['amqp-user'],
+        'vhost': config['amqp-vhost'],
+    })
+
+
+def amqp_relation_changed(config):
+    _create_upstart(config)
+    _create_amqp_config(config)
+    try:
+        subprocess.check_call(['/sbin/restart', _service_name(config)])
+    except subprocess.CalledProcessError:
+        # if it wasn't running, restart fails, so just try to start
+        subprocess.check_call(['/sbin/start', _service_name(config)])
+
+
+def amqp_relation_broken(config):
+    subprocess.call(['/sbin/stop', _service_name(config)])
+    os.unlink(os.path.join(_service_dir(config), 'amqp_config.py'))
+
+
+def main():
+    hook = os.path.basename(sys.argv[0])
+    juju_info("Running hook: %s" % hook)
+
+    hook_py = hook.replace('-', '_')
+    funcs = globals()
+    if hook_py not in funcs:
+        print("Unknown hook: %s" % hook)
+        return 1
+
+    config = _config()
+    try:
+        return funcs[hook_py](config)
+    except subprocess.CalledProcessError as e:
+        juju_info('Error running: %s: %s' % (e.cmd, e.output))
+        return e.returncode
+
+
+if __name__ == '__main__':
+    exit(main())
 
=== added symlink 'hooks/install'
=== target is u'hooks.py'
=== added file 'metadata.yaml'
--- metadata.yaml	1970-01-01 00:00:00 +0000
+++ metadata.yaml	2014-01-30 11:47:01 +0000
@@ -0,0 +1,10 @@
+name: rabbitmq-worker
+summary: Deploy a python-restish service
+maintainer: Andy Doan <[e-mail]>
+description: |
+  Deploys a rabbitmq worker from a given bzr branch.
+categories:
+  - app-servers
+requires:
+  amqp:
+    interface: rabbitmq
 
=== renamed file 'metadata.yaml' => 'metadata.yaml.moved'