Merge lp:~fginther/charms/precise/ubuntu-ci-services-itself/rabbitmq-worker-lp into lp:~canonical-ci-engineering/charms/precise/ubuntu-ci-services-itself/lander

Proposed by Francis Ginther
Status: Rejected
Rejected by: Francis Ginther
Proposed branch: lp:~fginther/charms/precise/ubuntu-ci-services-itself/rabbitmq-worker-lp
Merge into: lp:~canonical-ci-engineering/charms/precise/ubuntu-ci-services-itself/lander
Diff against target: 300 lines (+266/-0) (has conflicts)
4 files modified
README.ex (+41/-0)
config.yaml (+44/-0)
hooks/hooks.py (+171/-0)
metadata.yaml (+10/-0)
Conflict adding file config.yaml.  Moved existing file to config.yaml.moved.
Conflict adding file hooks.  Moved existing file to hooks.moved.
Conflict adding file metadata.yaml.  Moved existing file to metadata.yaml.moved.
To merge this branch: bzr merge lp:~fginther/charms/precise/ubuntu-ci-services-itself/rabbitmq-worker-lp
Reviewer Review Type Date Requested Status
Canonical CI Engineering Pending
Review via email: mp+203930@code.launchpad.net

Commit message

Add the ability to pass launchpad credentials to the worker node, uses a base64 encoded file called "launchpad.credentials".

Description of the change

Add the ability to pass launchpad credentials to the worker node, uses a base64 encoded file called "launchpad.credentials".

To post a comment you must log in.
Revision history for this message
Andy Doan (doanac) wrote :

could we use "unit_config" for this also and include lpcredentials as a value in it? or alternatively just use the unit_config we have now which I think includes all the information needed to create this credentials file?

NOTE: I've proposed moving this charm:

 https://code.launchpad.net/~doanac/ubuntu-ci-services-itself/rabbitwork-local/+merge/203830

you probably want to rebase this MP on to that

Revision history for this message
Francis Ginther (fginther) wrote :

Merge into the main project trunk and use unit-config.

Revision history for this message
Francis Ginther (fginther) wrote :

"NEED" to merge into the main project and use unit-config. Rejecting this MP.

Unmerged revisions

14. By Francis Ginther

Add import base64.

13. By Francis Ginther

Add some lp_creds debugging.

12. By Francis Ginther

Adjust file path to remove hidden hint.

11. By Francis Ginther

Add launchpa credentials file.

10. By Paul Larson

add support for installing pip packages

9. By Andy Doan

allow user/group for service to be configurable

8. By Andy Doan

update how amqp parameters are set

allow the worker to dynamically load the config from a file. If
you don't do this, you have to issue a "stop <service>; start <service>"
in order for upstart to detect a change from a bad config to a good
config

7. By Andy Doan

give a limit to the retries for the upstart job

6. By Andy Doan

fix quoting issue

5. By Andy Doan

fix eol issue on relation_get()

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'README.ex'
2--- README.ex 1970-01-01 00:00:00 +0000
3+++ README.ex 2014-01-30 11:47:01 +0000
4@@ -0,0 +1,41 @@
5+Describe the intended usage of this charm and anything unique about how
6+this charm relates to others here.
7+
8+This README will be displayed in the Charm Store, it should be either Markdown or RST. Ideal READMEs include instructions on how to use the charm, expected usage, and charm features that your audience might be interested in. For an example of a well written README check out Hadoop: http://jujucharms.com/charms/precise/hadoop
9+
10+Here's an example you might wish to template off of:
11+
12+Overview
13+--------
14+
15+This charm provides (service) from (service homepage). Add a description here of what the service itself actually does.
16+
17+
18+Usage
19+-----
20+
21+Step by step instructions on using the charm:
22+
23+ juju deploy servicename
24+
25+and so on. If you're providing a web service or something that the end user needs to go to, tell them here, especially if you're deploying a service that might listen to a non-default port.
26+
27+You can then browse to http://ip-address to configure the service.
28+
29+Configuration
30+-------------
31+
32+The configuration options will be listed on the charm store, however If you're making assumptions or opinionated decisions in the charm (like setting a default administrator password), you should detail that here so the user knows how to change it immediately, etc.
33+
34+
35+Contact Information
36+-------------------
37+
38+Though this will be listed in the charm store itself don't assume a user will know that, so include that information here:
39+
40+Author:
41+Report bugs at: http://bugs.launchpad.net/charms/+source/charmname
42+Location: http://jujucharms.com/charms/distro/charmname
43+
44+* Be sure to remove the templated parts before submitting to https://launchpad.net/charms for inclusion in the charm store.
45+
46
47=== added file 'config.yaml'
48--- config.yaml 1970-01-01 00:00:00 +0000
49+++ config.yaml 2014-01-30 11:47:01 +0000
50@@ -0,0 +1,44 @@
51+options:
52+ branch:
53+ type: string
54+ description: "BZR branch the service lives in"
55+ revno:
56+ type: string
57+ description: "Revision or tag to branch from"
58+ packages:
59+ type: string
60+ description: "Packages required for this service"
61+ pip-packages:
62+ type: string
63+ description: "Pip packages required for this service"
64+ install_root:
65+ type: string
66+ description: "The root directory the service will be installed in"
67+ default: "/srv/"
68+ main:
69+ type: string
70+ description: "The worker script to run as a service. Can include paths relative to local bzr directory"
71+
72+ uid:
73+ type: string
74+ default: nobody
75+ description: User to run service as
76+ gid:
77+ type: string
78+ default: nogroup
79+ description: Group to run service as
80+
81+ # required for rabbitmq-server charm:
82+ amqp-user:
83+ type: string
84+ default: 'workerbee'
85+ description: The user to log into the rabbitMQ server.
86+ amqp-vhost:
87+ type: string
88+ default: '/'
89+ description: The vhost in the rabbitMQ server.
90+
91+ lp_creds:
92+ type: string
93+ default: ''
94+ description: The launchpad credenetials file.
95
96=== renamed file 'config.yaml' => 'config.yaml.moved'
97=== added directory 'hooks'
98=== renamed directory 'hooks' => 'hooks.moved'
99=== added symlink 'hooks/amqp-relation-broken'
100=== target is u'hooks.py'
101=== added symlink 'hooks/amqp-relation-changed'
102=== target is u'hooks.py'
103=== added symlink 'hooks/amqp-relation-joined'
104=== target is u'hooks.py'
105=== added symlink 'hooks/config-changed'
106=== target is u'hooks.py'
107=== added file 'hooks/hooks.py'
108--- hooks/hooks.py 1970-01-01 00:00:00 +0000
109+++ hooks/hooks.py 2014-01-30 11:47:01 +0000
110@@ -0,0 +1,171 @@
111+#!/usr/bin/env python
112+
113+import base64
114+import os
115+import json
116+import subprocess
117+import sys
118+import textwrap
119+
120+
121+def juju_info(msg):
122+ subprocess.check_call(['juju-log', '-l', 'INFO', msg])
123+ pass
124+
125+
126+def _config():
127+ output = subprocess.check_output(['config-get', '--format=json'])
128+ return json.loads(output)
129+
130+
131+def _relation_get(key):
132+ return subprocess.check_output(['relation-get', key]).strip()
133+
134+
135+def _relation_set(keyvalues, relation_id=None):
136+ args = ['relation-set']
137+ if relation_id:
138+ args.extend(['-r', relation_id])
139+ args.extend(["{}={}".format(k, v or '') for k, v in keyvalues.items()])
140+ subprocess.check_call(args)
141+
142+
143+def _service_name(config):
144+ unit = os.environ['JUJU_UNIT_NAME'].split('/')[0]
145+ for x in (':', '-', '/', '"', "'"):
146+ unit = unit.replace(x, '_')
147+ return unit
148+
149+
150+def _service_dir(config):
151+ return os.path.join(config['install_root'], _service_name(config))
152+
153+
154+def pip_install(package):
155+ cmd_line = ['pip', 'install', '-b', '/tmp/']
156+ if package.startswith('svn+') or package.startswith('git+') or \
157+ package.startswith('hg+') or package.startswith('bzr+'):
158+ cmd_line.append('-e')
159+ cmd_line.append(package)
160+ return(subprocess.call(cmd_line))
161+
162+
163+def config_changed(config):
164+ lp_creds = config.get('lp_creds', None)
165+ juju_info('lp_creds (raw): %s' % (lp_creds))
166+ if lp_creds:
167+ lp_creds = str(base64.b64decode(lp_creds))
168+ juju_info('lp_creds (decoded): %s' % (lp_creds))
169+ with open(os.path.join(_service_dir(config), 'launchpad.credentials'),
170+ 'w') as f:
171+ f.write(lp_creds)
172+
173+
174+def install(config):
175+ pkgs = [x for x in config.get('packages', '').split(' ') if x]
176+ pkgs.append('python-amqplib')
177+ pkgs.append('python-pip')
178+ pkgs.append('bzr')
179+
180+ juju_info('installing apt packages...')
181+ subprocess.check_call(['apt-get', 'install', '-y', '-q'] + pkgs)
182+
183+ pip_pkgs = [x for x in config.get('pip-packages', '').split(' ') if x]
184+ if pip_pkgs:
185+ juju_info('installing pip packages...')
186+ for package in pip_pkgs:
187+ pip_install(package.strip())
188+
189+ juju_info('grabbing service from bzr...')
190+ args = ['bzr', 'branch']
191+ rev = config.get('revno', '')
192+ if rev:
193+ args.extend(['-r', rev])
194+ args.append(config['branch'])
195+ args.append(_service_dir(config))
196+ subprocess.check_call(args)
197+
198+
199+def _create_upstart(config):
200+ template = textwrap.dedent('''
201+ #--------------------------------------------------------------
202+ # This file is managed by the rabbitmq-worker Juju charm
203+ #--------------------------------------------------------------
204+ description "starts a rabbitmq worker"
205+ start on (local-filesystems and net-device-up IFACE=eth0)
206+ stop on runlevel [!12345]
207+
208+ # If the process quits unexpectadly trigger a respawn
209+ # give it 15 chances with a 5 second delay in between retries
210+ respawn limit 15 5
211+
212+ setuid {uid}
213+ setgid {gid}
214+ chdir {sdir}
215+
216+ exec {main}
217+ ''')
218+ params = {
219+ 'main': config['main'],
220+ 'sdir': _service_dir(config),
221+ 'uid': config.get('uid', 'nobody'),
222+ 'gid': config.get('gid', 'nogroup'),
223+ }
224+ with open('/etc/init/%s.conf' % _service_name(config), 'w') as f:
225+ f.write(template.format(**params))
226+ # need this so the file is ready for the subprocess call
227+ f.flush()
228+ os.fsync(f.fileno())
229+
230+
231+def _create_amqp_config(config):
232+ with open(os.path.join(_service_dir(config), 'amqp_config.py'), 'w') as f:
233+ f.write('# DO NOT EDIT. Generated by restish charm hook\n')
234+ f.write('AMQP_USER = "%s"\n' % config['amqp-user'])
235+ f.write('AMQP_VHOST = "%s"\n' % config['amqp-vhost'])
236+ f.write('AMQP_HOST = "%s"\n' % _relation_get('private-address'))
237+ f.write('AMQP_PASSWORD = "%s"\n' % _relation_get('password'))
238+
239+
240+def amqp_relation_joined(config):
241+ _relation_set({
242+ 'username': config['amqp-user'],
243+ 'vhost': config['amqp-vhost'],
244+ })
245+
246+
247+def amqp_relation_changed(config):
248+ _create_upstart(config)
249+ _create_amqp_config(config)
250+ try:
251+ subprocess.check_call(['/sbin/restart', _service_name(config)])
252+ except subprocess.CalledProcessError:
253+ # if it wasn't running, restart fails, so just try to start
254+ subprocess.check_call(['/sbin/start', _service_name(config)])
255+
256+
257+def amqp_relation_broken(config):
258+ subprocess.call(['/sbin/stop', _service_name(config)])
259+ os.unlink(os.path.join(_service_dir(config), 'amqp_config.py'))
260+
261+
262+def main():
263+ hook = os.path.basename(sys.argv[0])
264+ juju_info("Running hook: %s" % hook)
265+
266+ hook_py = hook.replace('-', '_')
267+ funcs = globals()
268+ if hook_py not in funcs:
269+ print("Unknown hook: %s" % hook)
270+ return 1
271+
272+ config = _config()
273+ try:
274+ return funcs[hook_py](config)
275+ except subprocess.CalledProcessError as e:
276+ juju_info('Error running: %s: %s' % (e.cmd, e.output))
277+ return e.returncode
278+
279+
280+if __name__ == '__main__':
281+ exit(main())
282
283=== added symlink 'hooks/install'
284=== target is u'hooks.py'
285=== added file 'metadata.yaml'
286--- metadata.yaml 1970-01-01 00:00:00 +0000
287+++ metadata.yaml 2014-01-30 11:47:01 +0000
288@@ -0,0 +1,10 @@
289+name: rabbitmq-worker
290+summary: Deploy a python-restish service
291+maintainer: Andy Doan <andy.doan@ubuntu.com>
292+description: |
293+ Deploys a rabbitmq worker from a given bzr branch.
294+categories:
295+ - app-servers
296+requires:
297+ amqp:
298+ interface: rabbitmq
299
300=== renamed file 'metadata.yaml' => 'metadata.yaml.moved'

Subscribers

People subscribed via source and target branches