Merge lp:~doanac/ubuntu-ci-services-itself/bsbuilder into lp:ubuntu-ci-services-itself

[Andy Doan]

Implements the webservice portion of the branch source builder

This creates a simple app using restish that takes a build_source request and sends it to rabbitmq. On the other end we have a simple worker script, branch-source-builder/run_worker that will wait for messages and print a TODO.

The MP itself is bigger than the service because of the groundwork I did for the restish service.

Implements the webservice portion of the branch source builder

This creates a simple app using restish that takes a build_source request and sends it to rabbitmq. On the other end we have a simple worker script, branch-source-builder/run_worker that will wait for messages and print a TODO.

The MP itself is bigger than the service because of the groundwork I did for the restish service.

[Andy Doan (doanac)]

There are some caveats to getting the juju-deployer portion of this working:

1) the rabbitmq-server charm is broke. It seems to be the .deb itself, but you have to edit /etc/hosts and add the host name to the 127.0.0.1 entry so the install will work. If you are fast and do that before juju-deployer gets to the bsb-rabbit unit the deployer will work. Otherwise you have to make that change, run "juju resolved --retry bsb-rabbit/0". then re-run the deployer script.

2) the bsb-worker script doesn't work out the first time. For some reason even with upstart respawing it, it won't start until you run: sudo stop bsb_worker; sudo start bsb_worker

At this point the service should be running and you can test with:

 curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"source_packages": "todo_source", "ppa": "todo_ppa", "progress_trigger": "todo_trigger"}' http://<BSB RESTISH IP>:8080/api/v1/build_source

then on the bsb_worker node, you'll see a TODO printed in /var/log/upstart/bsb_worker.log

[lp:~doanac/ubuntu-ci-services-itself/bsbuilder]

23. By Andy Doan on 2013-12-10

work around a canonistack bug with seen with rabbitmq-server

[Andy Doan (doanac)]

I've fixed the rabbitmq-server issue mentioned above by creating our own rabbitmq-server charm with a fix.

I've identified the solution to the 2nd problem and should have a minor fix for that ready soon.

[lp:~doanac/ubuntu-ci-services-itself/bsbuilder]

24. By Andy Doan on 2013-12-10

move queue helpers to their own module

This makes the queue helpers more accessible by the run_worker
script w/o making it have to pull in python-restish dependencies.

It also uses the new common configuration file mechanism thats
been changed in the rabbitmq-worker charm to be like the
restish charm.

[Andy Doan (doanac)]

as of revno 24, things "just work".

[Francis Ginther (fginther)]

In addition to eventually refactoring the queue helpers into a common utils directory as you mentioned:

 - when refactoring, be sure to split the tests in branch-source-builder/bsbuilder/tests/test_utils.py.
 - branch-source-builder/setup.py is not executable
 - wsgi.py starts the server on port 8080, but prints '8000'. Is there a guideline for choosing ports? My local jenkins runs on 8080 also so I can't start this locally as is.

Not having done a REST app before, I don't have a lot of input. I'm in favor of how you're doing this, but not familiar with any alternatives. I did spend extra time reviewing the queue setup and like what's going on there.

Want to do some local testing before approving.

[lp:~doanac/ubuntu-ci-services-itself/bsbuilder]

25. By Andy Doan on 2013-12-11

address comments from fginther

make setup.py executable
use argparse for local wsgi server

[Francis Ginther (fginther)]

Was able to deploy successfully after adjusting the juju-deployer file to point to this branch.

Let's go with this and tweak it over time.

review: Approve

=== modified file 'README'
--- README	2013-11-25 17:45:20 +0000
+++ README	2013-12-11 16:52:29 +0000
@@ -4,8 +4,14 @@
 Development environments can be set of using the setup.py files in the
 projects you wish to work on. The easiest approach is to use python-virtualenv.
 Since most projects require the ci-utils project, that should almost always
-get setup first::
-
+get setup first.
+
+There are two types of services written in this project, Django and Restish.
+Development varies slightly between the two.
+
+Django
+~~~~~~
+::
   # setup the ppa-assigner project
   virtualenv /tmp/venv
   . /tmp/venv/bin/activate
@@ -25,6 +31,29 @@
   ./ppa-assigner/manage.py test ppa_assigner.TestApi #test one class
   ./ppa-assigner/manage.py test ppa_assigner.TestApi.testFree # test one method
 
+Restish
+~~~~~~~
+::
+  virtualenv /tmp/venv
+  . /tmp/venv/bin/activate
+  ./branch-source-builder/setup.py develop
+
+Unit-testing can be done with::
+
+  python -m unittest bsbuilder.tests.test_example
+
+Running under the python wsgi server can be done with::
+
+  ./branch-source-builder/bsbuilder/wsgi.py
+
+Running under gunicorn can be done with::
+
+  # need gunicorn:
+  pip install gunicorn
+
+  # TIP: by setting max-requests=1 you can make changes to the source code
+  # and they'll be picked up in the next http request you make!!
+  gunicorn --max-requests 1 bsbuilder.wsgi:app
 
 Juju Testing
 ------------

=== added directory 'branch-source-builder'
=== added directory 'branch-source-builder/bsbuilder'
=== added file 'branch-source-builder/bsbuilder/__init__.py'
--- branch-source-builder/bsbuilder/__init__.py	1970-01-01 00:00:00 +0000
+++ branch-source-builder/bsbuilder/__init__.py	2013-12-11 16:52:29 +0000
@@ -0,0 +1,17 @@
+# Ubuntu CI Services
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# TODO make this probe from changelog or bzr
+__version__ = '0.1'

=== added file 'branch-source-builder/bsbuilder/amqp_utils.py'
--- branch-source-builder/bsbuilder/amqp_utils.py	1970-01-01 00:00:00 +0000
+++ branch-source-builder/bsbuilder/amqp_utils.py	2013-12-11 16:52:29 +0000
@@ -0,0 +1,108 @@
+# Ubuntu CI Services
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import logging
+import os
+import socket
+import time
+
+from amqplib import client_0_8 as amqp
+
+log = logging.getLogger(__name__)
+
+
+def get_config():
+    '''Load the rabbit config created by the restish charm'''
+    config = None
+    try:
+        # try and find the config file, should be in the root of our bzr branch
+        f = os.path.join(os.path.dirname(__file__), '../../amqp_config.py')
+        if os.path.exists(f):
+            import imp
+            config = imp.load_source('amqp_config', f)
+        else:
+            log.warn('No amqp_config found at: %s' % os.path.abspath(f))
+    except:
+        log.exception('ERROR detecting rabbit args')
+    return config
+
+
+def connection(rabbit_config):
+    return amqp.Connection(
+        userid=rabbit_config.AMQP_USER,
+        virtual_host=rabbit_config.AMQP_VHOST,
+        host=rabbit_config.AMQP_HOST,
+        password=rabbit_config.AMQP_PASSWORD
+    )
+
+
+def send(queue, msg):
+    config = get_config()
+    if not config:
+        return 'rabbitmq settings not available.'
+    con = channel = None
+    try:
+        con = connection(config)
+        channel = con.channel()
+        channel.queue_declare(queue=queue, durable=True, auto_delete=False)
+        body = amqp.Message(msg)
+        body.properties['delivery_mode'] = 2  # Persistent
+        channel.basic_publish(body, routing_key=queue)
+    except Exception as e:
+        logging.exception('unable to queue up build_source request')
+        return str(e)
+    finally:
+        if channel:
+            channel.close()
+        if con:
+            con.close()
+
+
+def _run_forever(channel, queue, callback, retry_period=120):
+    tag = channel.basic_consume(callback=callback, queue=queue)
+    try:
+        timeout = time.time()
+        while time.time() < timeout + retry_period:
+            try:
+                channel.wait()
+                timeout = time.time()
+            except (amqp.AMQPConnectionException, socket.error):
+                logging.error('lost connection to Rabbit')
+                # TODO metrics.meter('lost_rabbit_connection')
+                # Don't probe immediately, give the network/process
+                # time to come back.
+                time.sleep(0.1)
+        logging.error('Rabbit did not reappear quickly enough.')
+    except KeyboardInterrupt:
+        pass
+    finally:
+        if channel and channel.is_open:
+            channel.basic_cancel(tag)
+
+
+def process_queue(config, queue, callback):
+    conn = channel = None
+    try:
+        conn = connection(config)
+        channel = conn.channel()
+        channel.queue_declare(queue=queue, durable=True, auto_delete=False)
+        channel.basic_qos(0, 1, False)
+        logging.info('Waiting for messages. ^C to exit.')
+        _run_forever(channel, queue, callback)
+    finally:
+        if channel:
+            channel.close()
+        if conn:
+            conn.close()

=== added directory 'branch-source-builder/bsbuilder/resources'
=== added file 'branch-source-builder/bsbuilder/resources/__init__.py'
=== added file 'branch-source-builder/bsbuilder/resources/root.py'
--- branch-source-builder/bsbuilder/resources/root.py	1970-01-01 00:00:00 +0000
+++ branch-source-builder/bsbuilder/resources/root.py	2013-12-11 16:52:29 +0000
@@ -0,0 +1,30 @@
+# Ubuntu CI Services
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import logging
+
+from restish import resource
+
+from bsbuilder.resources import v1
+
+
+log = logging.getLogger(__name__)
+
+
+class Root(resource.Resource):
+    @resource.child('api/v1')
+    def api(self, request, segments):
+        log.debug('[api]: %s %s', request.url, str(segments))
+        return v1.API()

=== added file 'branch-source-builder/bsbuilder/resources/v1.py'
--- branch-source-builder/bsbuilder/resources/v1.py	1970-01-01 00:00:00 +0000
+++ branch-source-builder/bsbuilder/resources/v1.py	2013-12-11 16:52:29 +0000
@@ -0,0 +1,53 @@
+# Ubuntu CI Services
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import json
+import logging
+
+from restish import http, resource
+
+from bsbuilder import amqp_utils, utils
+
+log = logging.getLogger(__name__)
+
+
+def _status():
+    return utils.json_ok({
+        'rabbit_configured': amqp_utils.get_config() is not None,
+    })
+
+
+def _build_source(source_packages, ppa, progress_trigger):
+    msg = json.dumps({
+        'source_packages': source_packages,
+        'ppa': ppa,
+        'progress_trigger': progress_trigger,
+    })
+    r = amqp_utils.send('bsbuilder', msg)
+    if r:
+        # send only returns something if it an error message
+        r = http.service_unavailable(body=r)
+    return r
+
+
+class API(resource.Resource):
+    @resource.child()
+    def status(self, request, segments):
+        log.debug('[status]: %s %s', request.url, str(segments))
+        return utils.http_get_resource(_status)
+
+    @resource.child()
+    def build_source(self, request, segments):
+        return utils.http_post_resource(_build_source)

=== added directory 'branch-source-builder/bsbuilder/tests'
=== added file 'branch-source-builder/bsbuilder/tests/__init__.py'
=== added file 'branch-source-builder/bsbuilder/tests/test_utils.py'
--- branch-source-builder/bsbuilder/tests/test_utils.py	1970-01-01 00:00:00 +0000
+++ branch-source-builder/bsbuilder/tests/test_utils.py	2013-12-11 16:52:29 +0000
@@ -0,0 +1,165 @@
+# Ubuntu CI Services
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import json
+import socket
+import time
+import unittest
+
+import mock
+import webtest
+
+from restish import resource
+from restish.app import RestishApp
+
+from bsbuilder import amqp_utils, utils
+
+
+def _no_param_action():
+    return utils.json_ok({'func': _no_param_action.__name__})
+
+
+def _two_param_action(p1, p2):
+    return utils.json_ok({'func': _two_param_action.__name__})
+
+
+class TestGetResource(unittest.TestCase):
+    '''Test to ensure the http_get_resource works properly'''
+
+    def setUp(self):
+        super(TestGetResource, self).setUp()
+
+        class TR(resource.Resource):
+            @resource.child()
+            def empty(self, request, segments):
+                return utils.http_get_resource(_no_param_action)
+
+            @resource.child()
+            def two_param_get(self, request, segments):
+                return utils.http_get_resource(_two_param_action)
+
+        app = RestishApp(TR())
+        self.app = webtest.TestApp(app, relative_to='.')
+
+    def testGetNoParam(self):
+        resp = self.app.get('/empty', status=200)
+        data = json.loads(resp.body)
+        self.assertEqual(data['func'], _no_param_action.__name__)
+
+    def testGetTwoParam(self):
+        resp = self.app.get('/two_param_get', {'p1': 1, 'p2': 2}, status=200)
+        data = json.loads(resp.body)
+        self.assertEqual(data['func'], _two_param_action.__name__)
+
+        # ensure a missing parameter returns a 400 error
+        resp = self.app.get('/two_param_get', {'p1': 1}, status=400)
+        self.assertTrue('p2' in resp.body)
+
+    def testLeafEnforced(self):
+        '''ensure we only respond to the leaf and not additional segments'''
+        self.app.get('/empty/', status=404)
+
+    def testAllowedMethods(self):
+        self.app.post_json('/empty', {'key': 'val'}, status=405)
+
+
+class TestPostResource(unittest.TestCase):
+    '''Test to ensure the http_post_resource works properly'''
+
+    def setUp(self):
+        super(TestPostResource, self).setUp()
+
+        class TR(resource.Resource):
+            @resource.child()
+            def empty(self, request, segments):
+                return utils.http_post_resource(_no_param_action)
+
+            @resource.child()
+            def two_param_get(self, request, segments):
+                return utils.http_post_resource(_two_param_action)
+
+        app = RestishApp(TR())
+        self.app = webtest.TestApp(app, relative_to='.')
+
+    def testPostNoParam(self):
+        resp = self.app.post_json('/empty', {}, status=200)
+        data = json.loads(resp.body)
+        self.assertEqual(data['func'], _no_param_action.__name__)
+
+    def testPostTwoParam(self):
+        resp = self.app.post_json('/two_param_get', {'p1': 1, 'p2': 2})
+        data = json.loads(resp.body)
+        self.assertEqual(data['func'], _two_param_action.__name__)
+
+        # ensure a missing parameter returns a 400 error
+        resp = self.app.post_json('/two_param_get', {'p1': 1}, status=400)
+        self.assertTrue('p2' in resp.body)
+
+    def testLeafEnforced(self):
+        '''ensure we only respond to the leaf and not additional segments'''
+        self.app.post_json('/empty/', {}, status=404)
+
+    def testAllowedMethods(self):
+        self.app.get('/empty', {'p1': 1, 'p2': 2}, status=405)
+
+
+class TestAMQP(unittest.TestCase):
+    @mock.patch('bsbuilder.amqp_utils.connection')
+    @mock.patch('bsbuilder.amqp_utils.get_config')
+    def testConnectFailed(self, get_config, connect):
+        '''Ensure a failed queue connection returns an HTTP 503 error'''
+        get_config.return_value = mock.Mock()
+        error = 'mocked test exception'
+        connect.side_effect = RuntimeError(error)
+        r = amqp_utils.send('fake_queue', 'fake_message')
+        self.assertIsNotNone(r)
+        self.assertEqual(error, r)
+
+    @mock.patch('bsbuilder.amqp_utils.connection')
+    @mock.patch('bsbuilder.amqp_utils.get_config')
+    def testSent(self, get_config, connect):
+        '''Test a successful send returns nothing
+
+        There's not much you can test in isolation here, but this gives
+        a bit of a sanity check.
+        '''
+        get_config.return_value = mock.Mock()
+        connect.return_value = mock.Mock()
+        r = amqp_utils.send('fake_queue', 'fake_message')
+        self.assertIsNone(r)
+
+    @mock.patch('bsbuilder.amqp_utils._run_forever')
+    @mock.patch('bsbuilder.amqp_utils.connection')
+    def testProcessQueue(self, connection, run_forever):
+        '''Ensure we close the connection if something fails'''
+        conn = mock.Mock()
+        connection.return_value = conn
+        run_forever.side_effect = RuntimeError
+        with self.assertRaises(RuntimeError):
+            amqp_utils.process_queue(None, None, None)
+        self.assertTrue(conn.close.called)
+
+    def testRunForever(self):
+        '''Ensure this times out after the right amount of time'''
+        channel = mock.Mock()
+        callback = mock.Mock()
+        #wait = mock.Mock()
+        #wait.side_effect = socket.error
+        channel.wait.side_effect = socket.error()
+        start = time.time()
+        retry_period = 2
+        amqp_utils._run_forever(channel, 'foo', callback, retry_period)
+        # see if it was within 1/10 of a second of the retry_period
+        self.assertAlmostEqual(start + retry_period, time.time(), places=1)

=== added file 'branch-source-builder/bsbuilder/tests/test_v1.py'
--- branch-source-builder/bsbuilder/tests/test_v1.py	1970-01-01 00:00:00 +0000
+++ branch-source-builder/bsbuilder/tests/test_v1.py	2013-12-11 16:52:29 +0000
@@ -0,0 +1,70 @@
+# Ubuntu CI Services
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import json
+import unittest
+
+import mock
+import webtest
+
+from bsbuilder import wsgi
+
+
+class TestAPI(unittest.TestCase):
+    '''Test to ensure the v1 API works.'''
+
+    def setUp(self):
+        super(TestAPI, self).setUp()
+        self.app = webtest.TestApp(wsgi.app, relative_to='.')
+
+    @mock.patch('bsbuilder.amqp_utils.get_config')
+    def testStatus(self, get_config):
+        get_config.return_value = None
+        resp = self.app.get('/api/v1/status', status=200)
+        data = json.loads(resp.body)
+        self.assertEqual(False, data['rabbit_configured'])
+
+        get_config.return_value = {'foo': 'bar'}
+        resp = self.app.get('/api/v1/status', status=200)
+        data = json.loads(resp.body)
+        self.assertEqual(True, data['rabbit_configured'])
+
+    @mock.patch('bsbuilder.amqp_utils.get_config')
+    def testBuildSourceUnconfigured(self, get_config):
+        # ensure it fails when not configured
+        get_config.return_value = None
+        params = {
+            'source_packages': 'foo',
+            'ppa': 'foo',
+            'progress_trigger': 'foo',
+        }
+        resp = self.app.post_json('/api/v1/build_source', params, status=503)
+        self.assertTrue('rabbitmq settings not available' in resp.body)
+
+    def testBuildSourceBadParams(self):
+        '''Ensure proper error message is returned for incorrect params.'''
+        resp = self.app.post_json('/api/v1/build_source', {}, status=400)
+        self.assertTrue('Missing required parameters' in resp.body)
+
+    @mock.patch('bsbuilder.amqp_utils.send')
+    def testBuildSource(self, send):
+        send.return_value = None
+        params = {
+            'source_packages': 'foo',
+            'ppa': 'foo',
+            'progress_trigger': 'foo',
+        }
+        r = self.app.post_json('/api/v1/build_source', params, status=204)
+        self.assertEqual('', r.body)

=== added file 'branch-source-builder/bsbuilder/utils.py'
--- branch-source-builder/bsbuilder/utils.py	1970-01-01 00:00:00 +0000
+++ branch-source-builder/bsbuilder/utils.py	2013-12-11 16:52:29 +0000
@@ -0,0 +1,78 @@
+# Ubuntu CI Services
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import inspect
+import logging
+import json
+
+from restish import http, resource
+
+log = logging.getLogger(__name__)
+
+
+def json_ok(data):
+    '''Simple wrapper to return JSON data as an OK response'''
+    data = json.dumps(data) + '\n'  # \n is handy when using curl from CLI
+    return http.ok([('Content-Type', 'application/json')], data)
+
+
+class _BaseResource(resource.Resource):
+    '''Creates a resource that validates arguments by introspcection.
+
+    This class makes it easy to supply a single function with parameters
+    to handle HTTP operations. The GET/POST parameters will be compared to
+    the callback function's parameter list to validate it can be called.
+    '''
+    def __init__(self, callback_func, allowed):
+        super(_BaseResource, self).__init__()
+        self._cb = callback_func
+        self._allowed = allowed
+
+    def _handle(self, req_args):
+        args, vargs, keywords, defaults = inspect.getargspec(self._cb)
+        missing = set(args) - set(req_args.keys())
+        unknown = set(req_args.keys()) - set(args)
+        if unknown:
+            log.warning('Unsupported arguments: %r' % unknown)
+        if missing:
+            body = 'Missing required parameters\n %s\n' % '\n '.join(missing)
+            return http.bad_request([('Content-Type', 'text/plain')], body)
+
+        params = {k: req_args[k] for k in args}
+        return self._cb(**params)
+
+    @resource.GET()
+    def _do_get(self, request):
+        if resource.GET not in self._allowed:
+            return http.method_not_allowed([request.method])
+        return self._handle(request.GET)
+
+    @resource.POST(accept='json')
+    def _do_post(self, request):
+        if resource.POST not in self._allowed:
+            return http.method_not_allowed([request.method])
+        params = json.loads(request.body)
+        resp = self._handle(params)
+        if not resp:
+            resp = http.Response('204 No Content', [], None)
+        return resp
+
+
+def http_get_resource(callback_func):
+    return _BaseResource(callback_func, (resource.GET,))
+
+
+def http_post_resource(callback_func):
+    return _BaseResource(callback_func, (resource.POST,))

=== added file 'branch-source-builder/bsbuilder/wsgi.py'
--- branch-source-builder/bsbuilder/wsgi.py	1970-01-01 00:00:00 +0000
+++ branch-source-builder/bsbuilder/wsgi.py	2013-12-11 16:52:29 +0000
@@ -0,0 +1,48 @@
+#!/usr/bin/env python
+# Ubuntu CI Services
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import logging
+from restish.app import RestishApp
+
+from bsbuilder.resources.root import Root
+
+logging.basicConfig(level=logging.DEBUG)
+
+
+def make_app():
+    """Build the wsgi app object."""
+    app = RestishApp(Root())
+    return app
+
+app = make_app()
+
+if __name__ == '__main__':
+    import argparse
+    from wsgiref.simple_server import make_server
+
+    parser = argparse.ArgumentParser(
+        description='Run webservice in python\'s wsgi reference server.')
+    parser.add_argument('-p', '--port', type=int, default=8080,
+                        help='Port to use. Default=%(default)d')
+    args = parser.parse_args()
+
+    httpd = make_server('', args.port, app)
+    try:
+        print('Running server on port %d...' % args.port)
+        httpd.serve_forever()
+    except KeyboardInterrupt:
+        print('exiting')
+        pass

=== added file 'branch-source-builder/run_worker'
--- branch-source-builder/run_worker	1970-01-01 00:00:00 +0000
+++ branch-source-builder/run_worker	2013-12-11 16:52:29 +0000
@@ -0,0 +1,47 @@
+#!/usr/bin/env python
+# Ubuntu CI Services
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import json
+import logging
+import os
+import sys
+
+
+logging.basicConfig(level=logging.INFO)
+log = logging.getLogger(__name__)
+
+# the worker might not have installed this module, so determine the path
+# and add it, so we can always safely import stuff
+sys.path.append(os.path.join(os.path.dirname(__file__), 'bsbuilder'))
+import amqp_utils
+
+
+def on_message(msg):
+    params = json.loads(msg.body)
+    sources = params['source_packages']
+    ppa = params['ppa']
+    trigger = params['progress_trigger']
+    print('TODO handler message: sources(%s) ppa(%s) trigger(%s)' % (
+          sources, ppa, trigger))
+    # remove from queue so request becomes completed
+    msg.channel.basic_ack(msg.delivery_tag)
+
+
+if __name__ == '__main__':
+    config = amqp_utils.get_config()
+    if not config:
+        exit(1)  # the get_config code prints an error
+    amqp_utils.process_queue(config, 'bsbuilder', on_message)

=== added file 'branch-source-builder/setup.py'
--- branch-source-builder/setup.py	1970-01-01 00:00:00 +0000
+++ branch-source-builder/setup.py	2013-12-11 16:52:29 +0000
@@ -0,0 +1,44 @@
+#!/usr/bin/env python
+# Ubuntu CI Services
+# Copyright 2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+
+from setuptools import find_packages, setup
+
+# ensure find_packages works if our current directory isn't this project
+basedir = os.path.abspath(os.path.dirname(__file__))
+os.chdir(basedir)
+packages = find_packages(basedir)
+
+import bsbuilder
+
+requires = [
+    'restish==0.12.1',
+    'amqplib==1.0.0',
+    'mock==1.0.1',
+    'WebTest==2.0.10',
+]
+
+setup(
+    name='branch-source-builder',
+    version=bsbuilder.__version__,
+    description='Branch/Source Builder component of Ubuntu CI Services',
+    author='Canonical CI Engineering Team',
+    license='AGPL',
+    packages=packages,
+    test_suite='tests',
+    install_requires=requires,
+)

=== added file 'juju-deployer/branch-source-builder.yaml'
--- juju-deployer/branch-source-builder.yaml	1970-01-01 00:00:00 +0000
+++ juju-deployer/branch-source-builder.yaml	2013-12-11 16:52:29 +0000
@@ -0,0 +1,29 @@
+branch-source-builder-staging:
+    series: precise
+    services:
+        bsb-restish:
+            charm: restish
+            branch: lp:~canonical-ci-engineering/charms/precise/ubuntu-ci-services-itself/restish
+            options:
+                branch: lp:ubuntu-ci-services-itself
+                python_path: ./branch-source-builder
+                # need non-default package python-amqplib for this service
+                packages: "python-webtest python-mock python-jinja2 python-amqplib"
+        bsb-gunicorn:
+            charm: gunicorn
+            branch: lp:charms/precise/gunicorn
+            options:
+                wsgi_wsgi_file: bsbuilder.wsgi:app
+        bsb-worker:
+            charm: rabbitmq-worker
+            branch: lp:~canonical-ci-engineering/charms/precise/ubuntu-ci-services-itself/rabbitmq-worker
+            options:
+                branch: lp:ubuntu-ci-services-itself
+                main: ./branch-source-builder/run_worker
+        bsb-rabbit:
+            branch: lp:~canonical-ci-engineering/charms/precise/ubuntu-ci-services-itself/rabbitmq-server
+            charm: rabbitmq
+    relations:
+        - [bsb-restish, bsb-gunicorn]
+        - [bsb-worker, bsb-rabbit]
+        - [bsb-rabbit, bsb-restish]