Merge lp:~frankban/charms/precise/juju-gui/support-firewall into lp:~juju-gui/charms/precise/juju-gui/trunk

[Francesco Banconi]

Avoid installing from PPA if not required.

No longer add the juju-gui PPA by default:
the external repository is added only if required,
i.e. if the legacy server is used or if a branch
is passed to juju-gui-source.

The only missing bit to make the charm work well
from behind a firewall AFAICT is avoiding the release
to be downloaded from Launchpad.

Also included the shelltoolbox file in the charm:
unfortunately the python-shelltoolbox package is
not available on precise. On the other hand, this
allows for getting rid of the bootstrap_utils.py
file, and the install hook now feels cleaner.

Refactoring + some magic removal on the backend
framework. Now it should be less surprising, and
also allows for more customizations, e.g. what
I did in the install method.

Also added missing tests for the backend framework:
those were required in order to increase our control
over what's really happening in the backend "hooks".

Switched to the builtin Tornado server by default.

This diff is very big, I am sorry, but:
- you can ignore the bootstrap_utils removal;
- you can ignore the shelltoolbox.py file: it is
  just a copy of the one present in the raring
  python-shelltoolbox package;
- a lot of code is tests, the rest of the code
  should be quite easy to follow.

QA:
    `make deploy` and watch the logs:
    - no PPA should be installed by default;
    - the deployment succeeds and the GUI works well;
    switch to builtin-server=false and watch the logs:
    - the PPA is installed (and then haproxy, apache...);
    - the config-change hook exits without errors and
      the GUI works well.

Tests:
    `make unittest`
    (I ran the functional tests myself).

Thank you!

https://codereview.appspot.com/14433049/

[Francesco Banconi (frankban)]

Reviewers: mp+189645_code.launchpad.net,

Message:
Please take a look.

Description:
Avoid installing from PPA if not required.

No longer add the juju-gui PPA by default:
the external repository is added only if required,
i.e. if the legacy server is used or if a branch
is passed to juju-gui-source.

The only missing bit to make the charm work well
from behind a firewall AFAICT is avoiding the release
to be downloaded from Launchpad.

Also included the shelltoolbox file in the charm:
unfortunately the python-shelltoolbox package is
not available on precise. On the other hand, this
allows for getting rid of the bootstrap_utils.py
file, and the install hook now feels cleaner.

Refactoring + some magic removal on the backend
framework. Now it should be less surprising, and
also allows for more customizations, e.g. what
I did in the install method.

Also added missing tests for the backend framework:
those were required in order to increase our control
over what's really happening in the backend "hooks".

Switched to the builtin Tornado server by default.

This diff is very big, I am sorry, but:
- you can ignore the bootstrap_utils removal;
- you can ignore the shelltoolbox.py file: it is
   just a copy of the one present in the raring
   python-shelltoolbox package;
- a lot of code is tests, the rest of the code
   should be quite easy to follow.

QA:
     `make deploy` and watch the logs:
     - no PPA should be installed by default;
     - the deployment succeeds and the GUI works well;
     switch to builtin-server=false and watch the logs:
     - the PPA is installed (and then haproxy, apache...);
     - the config-change hook exits without errors and
       the GUI works well.

Tests:
     `make unittest`
     (I ran the functional tests myself).

Thank you!

https://code.launchpad.net/~frankban/charms/precise/juju-gui/support-firewall/+merge/189645

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/14433049/

Affected files (+1261, -365 lines):
   A [revision details]
   M config.yaml
   M hooks/backend.py
   D hooks/bootstrap_utils.py
   M hooks/install
   A hooks/shelltoolbox.py
   M hooks/utils.py
   M revision
   M tests/requirements.pip
   M tests/test_backends.py
   M tests/test_utils.py

[lp:~frankban/charms/precise/juju-gui/support-firewall]

120. By Francesco Banconi on 2013-10-07

Fix functional test.

[Gary Poster (gary)]

Wow! Great tests, and very nice improvement. Code LGTM. I will check
with Rick as to which of us will QA.

Thank you,

Gary

https://codereview.appspot.com/14433049/diff/1/hooks/backend.py
File hooks/backend.py (right):

https://codereview.appspot.com/14433049/diff/1/hooks/backend.py#newcode116
hooks/backend.py:116: debs = ('curl',)
Does this belong somewhere else now? Or can it be deleted?

https://codereview.appspot.com/14433049/diff/1/hooks/backend.py#newcode229
hooks/backend.py:229: If reverse is True, the method is called on the
reversed list of objects.
Delete this line? (functionality is gone)

https://codereview.appspot.com/14433049/diff/1/hooks/backend.py#newcode310
hooks/backend.py:310: """Execute the charm starting steps."""
A subtle thing, but any one of these would read a bit better to me, as
ranked from my personal highest to lowest preference.

Execute the charm's "start" steps.
Execute the charm's starting steps.
Execute the charm-starting steps.

https://codereview.appspot.com/14433049/diff/1/hooks/backend.py#newcode314
hooks/backend.py:314: """Execute the charm stopping steps."""
Perhaps the following?
----
Execute the charm's "stop" steps.

Iterates through the mixins in reverse order.

https://codereview.appspot.com/14433049/diff/1/hooks/backend.py#newcode318
hooks/backend.py:318: """Execute the charm removal steps."""
WDYT?
---
Execute the charm removal steps.

Iterates through the mixins in reverse order.

https://codereview.appspot.com/14433049/diff/1/hooks/shelltoolbox.py
File hooks/shelltoolbox.py (right):

https://codereview.appspot.com/14433049/diff/1/hooks/shelltoolbox.py#newcode3
hooks/shelltoolbox.py:3: # This file is part of python-shell-toolbox.
I included comments about the source in hooks/charmhelpers.py. Check
what I did there. If you like it, include it here; otherwise, feel free
to ignore it.

https://codereview.appspot.com/14433049/

[Richard Harding (rharding)]

LGTM QA ok

For QA I did a deploy, ssh'd into the box to verify that the ppa,
apache/haproxy packages were not installed.

Checked the gui worked fine.

Then changed the config for the server. SSH'd back into the box to
verify that packages were then installed. Gui still worked fine.

https://codereview.appspot.com/14433049/

[lp:~frankban/charms/precise/juju-gui/support-firewall]

121. By Francesco Banconi on 2013-10-08

Changes as per review.

[Francesco Banconi (frankban)]

*** Submitted:

Avoid installing from PPA if not required.

No longer add the juju-gui PPA by default:
the external repository is added only if required,
i.e. if the legacy server is used or if a branch
is passed to juju-gui-source.

The only missing bit to make the charm work well
from behind a firewall AFAICT is avoiding the release
to be downloaded from Launchpad.

Also included the shelltoolbox file in the charm:
unfortunately the python-shelltoolbox package is
not available on precise. On the other hand, this
allows for getting rid of the bootstrap_utils.py
file, and the install hook now feels cleaner.

Refactoring + some magic removal on the backend
framework. Now it should be less surprising, and
also allows for more customizations, e.g. what
I did in the install method.

Also added missing tests for the backend framework:
those were required in order to increase our control
over what's really happening in the backend "hooks".

Switched to the builtin Tornado server by default.

This diff is very big, I am sorry, but:
- you can ignore the bootstrap_utils removal;
- you can ignore the shelltoolbox.py file: it is
   just a copy of the one present in the raring
   python-shelltoolbox package;
- a lot of code is tests, the rest of the code
   should be quite easy to follow.

QA:
     `make deploy` and watch the logs:
     - no PPA should be installed by default;
     - the deployment succeeds and the GUI works well;
     switch to builtin-server=false and watch the logs:
     - the PPA is installed (and then haproxy, apache...);
     - the config-change hook exits without errors and
       the GUI works well.

Tests:
     `make unittest`
     (I ran the functional tests myself).

Thank you!

R=gary.poster, rharding
CC=
https://codereview.appspot.com/14433049

https://codereview.appspot.com/14433049/diff/1/hooks/backend.py
File hooks/backend.py (right):

https://codereview.appspot.com/14433049/diff/1/hooks/backend.py#newcode116
hooks/backend.py:116: debs = ('curl',)
On 2013/10/07 17:01:08, gary.poster wrote:
> Does this belong somewhere else now? Or can it be deleted?

Yeah, I was curious about that dependency too. I found that curl is used
by fetch_gui_release to download a release tarball from Launchpad. So,
formally it belongs here: added a comment describing what curl is used
for.

https://codereview.appspot.com/14433049/diff/1/hooks/backend.py#newcode229
hooks/backend.py:229: If reverse is True, the method is called on the
reversed list of objects.
On 2013/10/07 17:01:08, gary.poster wrote:
> Delete this line? (functionality is gone)

Done thank you, and yes, my first implementation included that kwarg,
but then I realized it was unnecessary.

https://codereview.appspot.com/14433049/diff/1/hooks/backend.py#newcode310
hooks/backend.py:310: """Execute the charm starting steps."""
On 2013/10/07 17:01:08, gary.poster wrote:
> A subtle thing, but any one of these would read a bit better to me, as
ranked
> from my personal highest to lowest preference.

> Execute the charm's "start" steps.
> Execute the charm's starting steps.
> Execute the charm-starting steps.

Done.

https://codereview.appspot.com/14433049/diff/1/hooks/backend.py#newco...

[Francesco Banconi (frankban)]

Hi Gary and Rick, thanks for your reviews!

https://codereview.appspot.com/14433049/

=== modified file 'config.yaml'
--- config.yaml	2013-09-30 20:16:09 +0000
+++ config.yaml	2013-10-08 07:46:30 +0000
@@ -183,7 +183,7 @@
       This is a temporary option: the built-in server will be
       the only server in the future.
     type: boolean
-    default: false
+    default: true
   builtin-server-logging:
     description: |
       Set the GUI server log level. Possible values are debug, info, warning
 
=== modified file 'hooks/backend.py'
--- hooks/backend.py	2013-10-01 17:54:23 +0000
+++ hooks/backend.py	2013-10-08 07:46:30 +0000
@@ -42,8 +42,10 @@
 import os
 import shutil
 
-import charmhelpers
-import shelltoolbox
+from charmhelpers import (
+    log,
+    open_port,
+)
 
 import utils
 
@@ -52,6 +54,7 @@
     """Handle the overall set up and clean up processes."""
 
     def install(self, backend):
+        log('Setting up base dir: {}.'.format(utils.BASE_DIR))
         try:
             os.makedirs(utils.BASE_DIR)
         except OSError as err:
@@ -60,6 +63,7 @@
                 raise
 
     def destroy(self, backend):
+        log('Cleaning up base dir: {}.'.format(utils.BASE_DIR))
         shutil.rmtree(utils.BASE_DIR)
 
 
@@ -109,25 +113,27 @@
 class GuiMixin(object):
     """Install and start the GUI and its dependencies."""
 
+    # The curl package is used to download release tarballs from Launchpad.
     debs = ('curl',)
 
     def install(self, backend):
         """Install the GUI and dependencies."""
-        # If the given installable thing ("backend") requires one or more debs
-        # that are not yet installed, install them.
-        missing = utils.find_missing_packages(*backend.debs)
-        if missing:
-            utils.cmd_log(
-                shelltoolbox.apt_get_install(*backend.debs))
         # If the source setting has changed since the last time this was run,
         # get the code, from either a static release or a branch as specified
         # by the souce setting, and install it.
         if backend.different('juju-gui-source'):
             # Get a tarball somehow.
-            logpath = backend.config['command-log-file']
             origin, version_or_branch = utils.parse_source(
                 backend.config['juju-gui-source'])
             if origin == 'branch':
+                logpath = backend.config['command-log-file']
+                # Make sure we have the required build dependencies.
+                # Note that we also need to add the juju-gui repository
+                # containing our version of nodejs.
+                log('Installing build dependencies.')
+                utils.install_missing_packages(
+                    utils.DEB_BUILD_DEPENDENCIES,
+                    repository=backend.config['repository-location'])
                 branch_url, revision = version_or_branch
                 release_tarball_path = utils.fetch_gui_from_branch(
                     branch_url, revision, logpath)
@@ -138,7 +144,7 @@
             utils.setup_gui(release_tarball_path)
 
     def start(self, backend):
-        charmhelpers.log('Starting Juju GUI.')
+        log('Starting Juju GUI.')
         config = backend.config
         build_dir = utils.compute_build_dir(
             config['juju-gui-debug'], config['serve-tests'])
@@ -151,8 +157,8 @@
             show_get_juju_button=config['show-get-juju-button'],
             password=config.get('password'))
         # Expose the service.
-        charmhelpers.open_port(80)
-        charmhelpers.open_port(443)
+        open_port(80)
+        open_port(443)
 
 
 class ServerInstallMixinBase(object):
@@ -175,6 +181,8 @@
     """Manage haproxy and Apache via Upstart."""
 
     debs = ('apache2', 'haproxy', 'openssl')
+    # We need to add the juju-gui PPA containing our customized haproxy.
+    ppa_required = True
 
     def install(self, backend):
         self._setup_certificates(backend)
@@ -215,32 +223,15 @@
         utils.stop_builtin_server()
 
 
-def chain_methods(name, reverse=False):
-    """Helper to compose a set of mixin objects into a callable.
+def call_methods(objects, name, *args):
+    """For each given object, call, if present, the method named name.
 
-    Each method is called in the context of its mixin instance, and its
-    argument is the Backend instance.
+    Pass the given args.
     """
-    # Chain method calls through all implementing mixins.
-    def method(self):
-        mixins = reversed(self.mixins) if reverse else self.mixins
-        for mixin in mixins:
-            a_callable = getattr(type(mixin), name, None)
-            if a_callable is not None:
-                a_callable(mixin, self)
-    method.__name__ = name
-    return method
-
-
-def merge_properties(name):
-    """Helper to merge one property from mixin objects into a unified set."""
-    @property
-    def method(self):
-        result = set()
-        for mixin in self.mixins:
-            result |= set(getattr(type(mixin), name, frozenset()))
-        return result
-    return method
+    for obj in objects:
+        method = getattr(obj, name, None)
+        if method is not None:
+            method(*args)
 
 
 class Backend(object):
@@ -296,11 +287,39 @@
         current, previous = self.config.get, self.prev_config.get
         return any(current(key) != previous(key) for key in keys)
 
-    # Composed methods.
-    install = chain_methods('install')
-    start = chain_methods('start')
-    stop = chain_methods('stop', reverse=True)
-    destroy = chain_methods('destroy', reverse=True)
-
-    # Merged properties.
-    debs = merge_properties('debs')
+    def get_dependencies(self):
+        """Return a tuple (debs, repository) representing dependencies."""
+        debs = set()
+        needs_ppa = False
+        # Collect the required dependencies and check if adding the juju-gui
+        # PPA is required.
+        for mixin in self.mixins:
+            debs.update(getattr(mixin, 'debs', ()))
+            if getattr(mixin, 'ppa_required', False):
+                needs_ppa = True
+        return debs, self.config['repository-location'] if needs_ppa else None
+
+    def install(self):
+        """Execute the installation steps."""
+        debs, repository = self.get_dependencies()
+        log('Installing dependencies.')
+        utils.install_missing_packages(debs, repository=repository)
+        call_methods(self.mixins, 'install', self)
+
+    def start(self):
+        """Execute the charm's "start" steps."""
+        call_methods(self.mixins, 'start', self)
+
+    def stop(self):
+        """Execute the charm's "stop" steps.
+
+        Iterate through the mixins in reverse order.
+        """
+        call_methods(reversed(self.mixins), 'stop', self)
+
+    def destroy(self):
+        """Execute the charm removal steps.
+
+        Iterate through the mixins in reverse order.
+        """
+        call_methods(reversed(self.mixins), 'destroy', self)
 
=== removed file 'hooks/bootstrap_utils.py'
--- hooks/bootstrap_utils.py	2013-06-11 14:04:04 +0000
+++ hooks/bootstrap_utils.py	1970-01-01 00:00:00 +0000
@@ -1,77 +0,0 @@
-# This file is part of the Juju GUI, which lets users view and manage Juju
-# environments within a graphical interface (https://launchpad.net/juju-gui).
-# Copyright (C) 2012-2013 Canonical Ltd.
-#
-# This program is free software: you can redistribute it and/or modify it under
-# the terms of the GNU Affero General Public License version 3, as published by
-# the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful, but WITHOUT
-# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
-# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-# Affero General Public License for more details.
-#
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-"""
-These are actually maintained in python-shelltoolbox.  Precise does not have
-that package, so we need to bootstrap the process by copying the functions
-we need here.
-"""
-
-import subprocess
-
-
-try:
-    import shelltoolbox
-except ImportError:
-
-    def run(*args, **kwargs):
-        """Run the command with the given arguments.
-
-        The first argument is the path to the command to run.
-        Subsequent arguments are command-line arguments to be passed.
-
-        This function accepts all optional keyword arguments accepted by
-        `subprocess.Popen`.
-        """
-        args = [i for i in args if i is not None]
-        pipe = subprocess.PIPE
-        process = subprocess.Popen(
-            args, stdout=kwargs.pop('stdout', pipe),
-            stderr=kwargs.pop('stderr', pipe),
-            close_fds=kwargs.pop('close_fds', True), **kwargs)
-        stdout, stderr = process.communicate()
-        if process.returncode:
-            exception = subprocess.CalledProcessError(
-                process.returncode, repr(args))
-            # The output argument of `CalledProcessError` was introduced in
-            # Python 2.7. Monkey patch the output here to avoid TypeErrors
-            # in older versions of Python, still preserving the output in
-            # Python 2.7.
-            exception.output = ''.join(filter(None, [stdout, stderr]))
-            raise exception
-        return stdout
-
-    def install_extra_repositories(*repositories):
-        """Install all of the extra repositories and update apt.
-
-        Given repositories can contain a "{distribution}" placeholder,
-        that will be replaced by current distribution codename.
-
-        :raises: subprocess.CalledProcessError
-        """
-        distribution = run('lsb_release', '-cs').strip()
-        # Starting from Oneiric, `apt-add-repository` is interactive by
-        # default, and requires a "-y" flag to be set.
-        assume_yes = None if distribution == 'lucid' else '-y'
-        for repo in repositories:
-            repository = repo.format(distribution=distribution)
-            run('apt-add-repository', assume_yes, repository)
-        run('apt-get', 'clean')
-        run('apt-get', 'update')
-
-else:
-    install_extra_repositories = shelltoolbox.install_extra_repositories
-    run = shelltoolbox.run
 
=== modified file 'hooks/install'
--- hooks/install	2013-10-01 20:53:38 +0000
+++ hooks/install	2013-10-08 07:46:30 +0000
@@ -18,31 +18,27 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import errno
-import json
 import os
 
-# We need to install the Juju PPA, which we will do with a couple of functions
-# that are actually maintained in python-shelltoolbox.
-import bootstrap_utils
-
-
-def get_config():
-    output = bootstrap_utils.run('config-get', '--format', 'json')
-    return json.loads(output)
-
-config = get_config()
-bootstrap_utils.install_extra_repositories(config['repository-location'])
+from charmhelpers import (
+    get_config,
+    log,
+)
+from shelltoolbox import (
+    apt_get_install,
+    run,
+)
+
 
 # Python dependencies must be installed here so that the charm can import and
 # use required libraries.
-PYTHON_DEPENDENCIES = (
-    'python-apt', 'python-launchpadlib', 'python-shelltoolbox',
-    'python-tempita',
-)
-bootstrap_utils.run(*(('apt-get', 'install', '-y') + PYTHON_DEPENDENCIES))
-
-
-from charmhelpers import log
+PYTHON_DEPENDENCIES = ('python-apt', 'python-launchpadlib', 'python-tempita')
+
+log('Installing base Python dependnecies: {}.'.format(
+    ', '.join(PYTHON_DEPENDENCIES)))
+apt_get_install(*PYTHON_DEPENDENCIES)
+
+
 from utils import (
     config_json,
     log_hook,
@@ -60,7 +56,7 @@
         for module in dirnames:
             filename = os.path.join('exec.d', module, 'charm-pre-install')
             try:
-                bootstrap_utils.run(filename)
+                run(filename)
             except OSError, e:
                 # If the exec.d file does not exist or is not runnable or
                 # is not a directory, assume we can recover.  Log the problem
 
=== added file 'hooks/shelltoolbox.py'
--- hooks/shelltoolbox.py	1970-01-01 00:00:00 +0000
+++ hooks/shelltoolbox.py	2013-10-08 07:46:30 +0000
@@ -0,0 +1,669 @@
+# Copyright 2012 Canonical Ltd.
+
+# This file is taken from the python-shelltoolbox package.
+#
+# IMPORTANT: Do not modify this file to add or change functionality.  If you
+# really feel the need to do so, first convert our code to the shelltoolbox
+# library, and modify it instead (or modify the helpers or utils module here,
+# as appropriate).
+#
+# python-shell-toolbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by the
+# Free Software Foundation, version 3 of the License.
+#
+# python-shell-toolbox is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+# more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with python-shell-toolbox. If not, see <http://www.gnu.org/licenses/>.
+
+"""Helper functions for accessing shell commands in Python."""
+
+__metaclass__ = type
+__all__ = [
+    'apt_get_install',
+    'bzr_whois',
+    'cd',
+    'command',
+    'DictDiffer',
+    'environ',
+    'file_append',
+    'file_prepend',
+    'generate_ssh_keys',
+    'get_su_command',
+    'get_user_home',
+    'get_user_ids',
+    'install_extra_repositories',
+    'join_command',
+    'mkdirs',
+    'run',
+    'Serializer',
+    'script_name',
+    'search_file',
+    'ssh',
+    'su',
+    'user_exists',
+    'wait_for_page_contents',
+    ]
+
+from collections import namedtuple
+from contextlib import contextmanager
+from email.Utils import parseaddr
+import errno
+import json
+import operator
+import os
+import pipes
+import pwd
+import re
+import subprocess
+import sys
+from textwrap import dedent
+import time
+import urllib2
+
+
+Env = namedtuple('Env', 'uid gid home')
+
+
+def apt_get_install(*args, **kwargs):
+    """Install given packages using apt.
+
+    It is possible to pass environment variables to be set during install
+    using keyword arguments.
+
+    :raises: subprocess.CalledProcessError
+    """
+    caller = kwargs.pop('caller', run)
+    debian_frontend = kwargs.pop('DEBIAN_FRONTEND', 'noninteractive')
+    with environ(DEBIAN_FRONTEND=debian_frontend, **kwargs):
+        cmd = ('apt-get', '-y', 'install') + args
+        return caller(*cmd)
+
+
+def bzr_whois(user):
+    """Return full name and email of bzr `user`.
+
+    Return None if the given `user` does not have a bzr user id.
+    """
+    with su(user):
+        try:
+            whoami = run('bzr', 'whoami')
+        except (subprocess.CalledProcessError, OSError):
+            return None
+    return parseaddr(whoami)
+
+
+@contextmanager
+def cd(directory):
+    """A context manager to temporarily change current working dir, e.g.::
+
+        >>> import os
+        >>> os.chdir('/tmp')
+        >>> with cd('/bin'): print os.getcwd()
+        /bin
+        >>> print os.getcwd()
+        /tmp
+    """
+    cwd = os.getcwd()
+    os.chdir(directory)
+    try:
+        yield
+    finally:
+        os.chdir(cwd)
+
+
+def command(*base_args):
+    """Return a callable that will run the given command with any arguments.
+
+    The first argument is the path to the command to run, subsequent arguments
+    are command-line arguments to "bake into" the returned callable.
+
+    The callable runs the given executable and also takes arguments that will
+    be appeneded to the "baked in" arguments.
+
+    For example, this code will list a file named "foo" (if it exists):
+
+        ls_foo = command('/bin/ls', 'foo')
+        ls_foo()
+
+    While this invocation will list "foo" and "bar" (assuming they exist):
+
+        ls_foo('bar')
+    """
+    def callable_command(*args):
+        all_args = base_args + args
+        return run(*all_args)
+
+    return callable_command
+
+
+@contextmanager
+def environ(**kwargs):
+    """A context manager to temporarily change environment variables.
+
+    If an existing environment variable is changed, it is restored during
+    context cleanup::
+
+        >>> import os
+        >>> os.environ['MY_VARIABLE'] = 'foo'
+        >>> with environ(MY_VARIABLE='bar'): print os.getenv('MY_VARIABLE')
+        bar
+        >>> print os.getenv('MY_VARIABLE')
+        foo
+        >>> del os.environ['MY_VARIABLE']
+
+    If we are adding environment variables, they are removed during context
+    cleanup::
+
+        >>> import os
+        >>> with environ(MY_VAR1='foo', MY_VAR2='bar'):
+        ...     print os.getenv('MY_VAR1'), os.getenv('MY_VAR2')
+        foo bar
+        >>> os.getenv('MY_VAR1') == os.getenv('MY_VAR2') == None
+        True
+    """
+    backup = {}
+    for key, value in kwargs.items():
+        backup[key] = os.getenv(key)
+        os.environ[key] = value
+    try:
+        yield
+    finally:
+        for key, value in backup.items():
+            if value is None:
+                del os.environ[key]
+            else:
+                os.environ[key] = value
+
+
+def file_append(filename, line):
+    r"""Append given `line`, if not present, at the end of `filename`.
+
+    Usage example::
+
+        >>> import tempfile
+        >>> f = tempfile.NamedTemporaryFile('w', delete=False)
+        >>> f.write('line1\n')
+        >>> f.close()
+        >>> file_append(f.name, 'new line\n')
+        >>> open(f.name).read()
+        'line1\nnew line\n'
+
+    Nothing happens if the file already contains the given `line`::
+
+        >>> file_append(f.name, 'new line\n')
+        >>> open(f.name).read()
+        'line1\nnew line\n'
+
+    A new line is automatically added before the given `line` if it is not
+    present at the end of current file content::
+
+        >>> import tempfile
+        >>> f = tempfile.NamedTemporaryFile('w', delete=False)
+        >>> f.write('line1')
+        >>> f.close()
+        >>> file_append(f.name, 'new line\n')
+        >>> open(f.name).read()
+        'line1\nnew line\n'
+
+    The file is created if it does not exist::
+
+        >>> import tempfile
+        >>> filename = tempfile.mktemp()
+        >>> file_append(filename, 'line1\n')
+        >>> open(filename).read()
+        'line1\n'
+    """
+    if not line.endswith('\n'):
+        line += '\n'
+    with open(filename, 'a+') as f:
+        lines = f.readlines()
+        if line not in lines:
+            if not lines or lines[-1].endswith('\n'):
+                f.write(line)
+            else:
+                f.write('\n' + line)
+
+
+def file_prepend(filename, line):
+    r"""Insert given `line`, if not present, at the beginning of `filename`.
+
+    Usage example::
+
+        >>> import tempfile
+        >>> f = tempfile.NamedTemporaryFile('w', delete=False)
+        >>> f.write('line1\n')
+        >>> f.close()
+        >>> file_prepend(f.name, 'line0\n')
+        >>> open(f.name).read()
+        'line0\nline1\n'
+
+    If the file starts with the given `line`, nothing happens::
+
+        >>> file_prepend(f.name, 'line0\n')
+        >>> open(f.name).read()
+        'line0\nline1\n'
+
+    If the file contains the given `line`, but not at the beginning,
+    the line is moved on top::
+
+        >>> file_prepend(f.name, 'line1\n')
+        >>> open(f.name).read()
+        'line1\nline0\n'
+    """
+    if not line.endswith('\n'):
+        line += '\n'
+    with open(filename, 'r+') as f:
+        lines = f.readlines()
+        if lines[0] != line:
+            try:
+                lines.remove(line)
+            except ValueError:
+                pass
+            lines.insert(0, line)
+            f.seek(0)
+            f.writelines(lines)
+
+
+def generate_ssh_keys(path, passphrase=''):
+    """Generate ssh key pair, saving them inside the given `directory`.
+
+        >>> generate_ssh_keys('/tmp/id_rsa')
+        0
+        >>> open('/tmp/id_rsa').readlines()[0].strip()
+        '-----BEGIN RSA PRIVATE KEY-----'
+        >>> open('/tmp/id_rsa.pub').read().startswith('ssh-rsa')
+        True
+        >>> os.remove('/tmp/id_rsa')
+        >>> os.remove('/tmp/id_rsa.pub')
+
+    If either of the key files already exist, generate_ssh_keys() will
+    raise an Exception.
+
+    Note that ssh-keygen will prompt if the keyfiles already exist, but
+    when we're using it non-interactively it's better to pre-empt that
+    behaviour.
+
+        >>> with open('/tmp/id_rsa', 'w') as key_file:
+        ...    key_file.write("Don't overwrite me, bro!")
+        >>> generate_ssh_keys('/tmp/id_rsa') # doctest: +ELLIPSIS
+        Traceback (most recent call last):
+        Exception: File /tmp/id_rsa already exists...
+        >>> os.remove('/tmp/id_rsa')
+
+        >>> with open('/tmp/id_rsa.pub', 'w') as key_file:
+        ...    key_file.write("Don't overwrite me, bro!")
+        >>> generate_ssh_keys('/tmp/id_rsa') # doctest: +ELLIPSIS
+        Traceback (most recent call last):
+        Exception: File /tmp/id_rsa.pub already exists...
+        >>> os.remove('/tmp/id_rsa.pub')
+    """
+    if os.path.exists(path):
+        raise Exception("File {} already exists.".format(path))
+    if os.path.exists(path + '.pub'):
+        raise Exception("File {}.pub already exists.".format(path))
+    return subprocess.call([
+        'ssh-keygen', '-q', '-t', 'rsa', '-N', passphrase, '-f', path])
+
+
+def get_su_command(user, args):
+    """Return a command line as a sequence, prepending "su" if necessary.
+
+    This can be used together with `run` when the `su` context manager is not
+    enough (e.g. an external program uses uid rather than euid).
+
+        run(*get_su_command(user, ['bzr', 'whoami']))
+
+    If the su is requested as current user, the arguments are returned as
+    given::
+
+        >>> import getpass
+        >>> current_user = getpass.getuser()
+
+        >>> get_su_command(current_user, ('ls', '-l'))
+        ('ls', '-l')
+
+    Otherwise, "su" is prepended::
+
+        >>> get_su_command('nobody', ('ls', '-l', 'my file'))
+        ('su', 'nobody', '-c', "ls -l 'my file'")
+    """
+    if get_user_ids(user)[0] != os.getuid():
+        args = [i for i in args if i is not None]
+        return ('su', user, '-c', join_command(args))
+    return args
+
+
+def get_user_home(user):
+    """Return the home directory of the given `user`.
+
+        >>> get_user_home('root')
+        '/root'
+
+    If the user does not exist, return a default /home/[username] home::
+
+        >>> get_user_home('_this_user_does_not_exist_')
+        '/home/_this_user_does_not_exist_'
+    """
+    try:
+        return pwd.getpwnam(user).pw_dir
+    except KeyError:
+        return os.path.join(os.path.sep, 'home', user)
+
+
+def get_user_ids(user):
+    """Return the uid and gid of given `user`, e.g.::
+
+        >>> get_user_ids('root')
+        (0, 0)
+    """
+    userdata = pwd.getpwnam(user)
+    return userdata.pw_uid, userdata.pw_gid
+
+
+def install_extra_repositories(*repositories):
+    """Install all of the extra repositories and update apt.
+
+    Given repositories can contain a "{distribution}" placeholder, that will
+    be replaced by current distribution codename.
+
+    :raises: subprocess.CalledProcessError
+    """
+    distribution = run('lsb_release', '-cs').strip()
+    # Starting from Oneiric, `apt-add-repository` is interactive by
+    # default, and requires a "-y" flag to be set.
+    assume_yes = None if distribution == 'lucid' else '-y'
+    for repo in repositories:
+        repository = repo.format(distribution=distribution)
+        run('apt-add-repository', assume_yes, repository)
+    run('apt-get', 'clean')
+    run('apt-get', 'update')
+
+
+def join_command(args):
+    """Return a valid Unix command line from `args`.
+
+        >>> join_command(['ls', '-l'])
+        'ls -l'
+
+    Arguments containing spaces and empty args are correctly quoted::
+
+        >>> join_command(['command', 'arg1', 'arg containing spaces', ''])
+        "command arg1 'arg containing spaces' ''"
+    """
+    return ' '.join(pipes.quote(arg) for arg in args)
+
+
+def mkdirs(*args):
+    """Create leaf directories (given as `args`) and all intermediate ones.
+
+        >>> import tempfile
+        >>> base_dir = tempfile.mktemp(suffix='/')
+        >>> dir1 = tempfile.mktemp(prefix=base_dir)
+        >>> dir2 = tempfile.mktemp(prefix=base_dir)
+        >>> mkdirs(dir1, dir2)
+        >>> os.path.isdir(dir1)
+        True
+        >>> os.path.isdir(dir2)
+        True
+
+    If the leaf directory already exists the function returns without errors::
+
+        >>> mkdirs(dir1)
+
+    An `OSError` is raised if the leaf path exists and it is a file::
+
+        >>> f = tempfile.NamedTemporaryFile(
+        ...     'w', delete=False, prefix=base_dir)
+        >>> f.close()
+        >>> mkdirs(f.name) # doctest: +ELLIPSIS
+        Traceback (most recent call last):
+        OSError: ...
+    """
+    for directory in args:
+        try:
+            os.makedirs(directory)
+        except OSError as err:
+            if err.errno != errno.EEXIST or os.path.isfile(directory):
+                raise
+
+
+def run(*args, **kwargs):
+    """Run the command with the given arguments.
+
+    The first argument is the path to the command to run.
+    Subsequent arguments are command-line arguments to be passed.
+
+    This function accepts all optional keyword arguments accepted by
+    `subprocess.Popen`.
+    """
+    args = [i for i in args if i is not None]
+    pipe = subprocess.PIPE
+    process = subprocess.Popen(
+        args, stdout=kwargs.pop('stdout', pipe),
+        stderr=kwargs.pop('stderr', pipe),
+        close_fds=kwargs.pop('close_fds', True), **kwargs)
+    stdout, stderr = process.communicate()
+    if process.returncode:
+        exception = subprocess.CalledProcessError(
+            process.returncode, repr(args))
+        # The output argument of `CalledProcessError` was introduced in Python
+        # 2.7. Monkey patch the output here to avoid TypeErrors in older
+        # versions of Python, still preserving the output in Python 2.7.
+        exception.output = ''.join(filter(None, [stdout, stderr]))
+        raise exception
+    return stdout
+
+
+def script_name():
+    """Return the name of this script."""
+    return os.path.basename(sys.argv[0])
+
+
+def search_file(regexp, filename):
+    """Return the first line in `filename` that matches `regexp`."""
+    with open(filename) as f:
+        for line in f:
+            if re.search(regexp, line):
+                return line
+
+
+def ssh(location, user=None, key=None, caller=subprocess.call):
+    """Return a callable that can be used to run ssh shell commands.
+
+    The ssh `location` and, optionally, `user` must be given.
+    If the user is None then the current user is used for the connection.
+
+    The callable internally uses the given `caller`::
+
+        >>> def caller(cmd):
+        ...     print tuple(cmd)
+        >>> sshcall = ssh('example.com', 'myuser', caller=caller)
+        >>> root_sshcall = ssh('example.com', caller=caller)
+        >>> sshcall('ls -l') # doctest: +ELLIPSIS
+        ('ssh', '-t', ..., '[e-mail]', '--', 'ls -l')
+        >>> root_sshcall('ls -l') # doctest: +ELLIPSIS
+        ('ssh', '-t', ..., 'example.com', '--', 'ls -l')
+
+    The ssh key path can be optionally provided::
+
+        >>> root_sshcall = ssh('example.com', key='/tmp/foo', caller=caller)
+        >>> root_sshcall('ls -l') # doctest: +ELLIPSIS
+        ('ssh', '-t', ..., '-i', '/tmp/foo', 'example.com', '--', 'ls -l')
+
+    If the ssh command exits with an error code,
+    a `subprocess.CalledProcessError` is raised::
+
+        >>> ssh('loc', caller=lambda cmd: 1)('ls -l') # doctest: +ELLIPSIS
+        Traceback (most recent call last):
+        CalledProcessError: ...
+
+    If ignore_errors is set to True when executing the command, no error
+    will be raised, even if the command itself returns an error code.
+
+        >>> sshcall = ssh('loc', caller=lambda cmd: 1)
+        >>> sshcall('ls -l', ignore_errors=True)
+    """
+    sshcmd = [
+        'ssh',
+        '-t',
+        '-t',  # Yes, this second -t is deliberate. See `man ssh`.
+        '-o', 'StrictHostKeyChecking=no',
+        '-o', 'UserKnownHostsFile=/dev/null',
+        ]
+    if key is not None:
+        sshcmd.extend(['-i', key])
+    if user is not None:
+        location = '{}@{}'.format(user, location)
+    sshcmd.extend([location, '--'])
+
+    def _sshcall(cmd, ignore_errors=False):
+        command = sshcmd + [cmd]
+        retcode = caller(command)
+        if retcode and not ignore_errors:
+            raise subprocess.CalledProcessError(retcode, ' '.join(command))
+
+    return _sshcall
+
+
+@contextmanager
+def su(user):
+    """A context manager to temporarily run the script as a different user."""
+    uid, gid = get_user_ids(user)
+    os.setegid(gid)
+    os.seteuid(uid)
+    home = get_user_home(user)
+    with environ(HOME=home):
+        try:
+            yield Env(uid, gid, home)
+        finally:
+            os.setegid(os.getgid())
+            os.seteuid(os.getuid())
+
+
+def user_exists(username):
+    """Return True if given `username` exists, e.g.::
+
+        >>> user_exists('root')
+        True
+        >>> user_exists('_this_user_does_not_exist_')
+        False
+    """
+    try:
+        pwd.getpwnam(username)
+    except KeyError:
+        return False
+    return True
+
+
+def wait_for_page_contents(url, contents, timeout=120, validate=None):
+    if validate is None:
+        validate = operator.contains
+    start_time = time.time()
+    while True:
+        try:
+            stream = urllib2.urlopen(url)
+        except (urllib2.HTTPError, urllib2.URLError):
+            pass
+        else:
+            page = stream.read()
+            if validate(page, contents):
+                return page
+        if time.time() - start_time >= timeout:
+            raise RuntimeError('timeout waiting for contents of ' + url)
+        time.sleep(0.1)
+
+
+class DictDiffer:
+    """
+    Calculate the difference between two dictionaries as:
+    (1) items added
+    (2) items removed
+    (3) keys same in both but changed values
+    (4) keys same in both and unchanged values
+    """
+
+    # Based on answer by hughdbrown at:
+    # http://stackoverflow.com/questions/1165352
+
+    def __init__(self, current_dict, past_dict):
+        self.current_dict = current_dict
+        self.past_dict = past_dict
+        self.set_current = set(current_dict)
+        self.set_past = set(past_dict)
+        self.intersect = self.set_current.intersection(self.set_past)
+
+    @property
+    def added(self):
+        return self.set_current - self.intersect
+
+    @property
+    def removed(self):
+        return self.set_past - self.intersect
+
+    @property
+    def changed(self):
+        return set(key for key in self.intersect
+                   if self.past_dict[key] != self.current_dict[key])
+
+    @property
+    def unchanged(self):
+        return set(key for key in self.intersect
+                   if self.past_dict[key] == self.current_dict[key])
+
+    @property
+    def modified(self):
+        return self.current_dict != self.past_dict
+
+    @property
+    def added_or_changed(self):
+        return self.added.union(self.changed)
+
+    def _changes(self, keys):
+        new = {}
+        old = {}
+        for k in keys:
+            new[k] = self.current_dict.get(k)
+            old[k] = self.past_dict.get(k)
+        return "%s -> %s" % (old, new)
+
+    def __str__(self):
+        if self.modified:
+            s = dedent("""\
+            added: %s
+            removed: %s
+            changed: %s
+            unchanged: %s""") % (
+                self._changes(self.added),
+                self._changes(self.removed),
+                self._changes(self.changed),
+                list(self.unchanged))
+        else:
+            s = "no changes"
+        return s
+
+
+class Serializer:
+    """Handle JSON (de)serialization."""
+
+    def __init__(self, path, default=None, serialize=None, deserialize=None):
+        self.path = path
+        self.default = default or {}
+        self.serialize = serialize or json.dump
+        self.deserialize = deserialize or json.load
+
+    def exists(self):
+        return os.path.exists(self.path)
+
+    def get(self):
+        if self.exists():
+            with open(self.path) as f:
+                return self.deserialize(f)
+        return self.default
+
+    def set(self, data):
+        with open(self.path, 'w') as f:
+            self.serialize(data, f)
 
=== modified file 'hooks/utils.py'
--- hooks/utils.py	2013-10-04 12:53:53 +0000
+++ hooks/utils.py	2013-10-08 07:46:30 +0000
@@ -36,6 +36,7 @@
     'get_npm_cache_archive_url',
     'get_release_file_url',
     'get_zookeeper_address',
+    'install_missing_packages',
     'legacy_juju',
     'log_hook',
     'parse_source',
@@ -78,6 +79,7 @@
     apt_get_install,
     command,
     environ,
+    install_extra_repositories,
     run,
     script_name,
     search_file,
@@ -155,12 +157,6 @@
 results_log = None
 
 
-def _get_build_dependencies():
-    """Install deb dependencies for building."""
-    log('Installing build dependencies.')
-    cmd_log(apt_get_install(*DEB_BUILD_DEPENDENCIES))
-
-
 def get_api_address(unit_dir=None):
     """Return the Juju API address.
 
@@ -623,8 +619,6 @@
 
 def fetch_gui_from_branch(branch_url, revision, logpath):
     """Retrieve the Juju GUI from a branch and build a release archive."""
-    # Make sure we have the needed dependencies.
-    _get_build_dependencies()
     # Inject NPM packages into the cache for faster building.
     prime_npm_cache(get_npm_cache_archive_url())
     # Create a release starting from a branch.
@@ -738,3 +732,20 @@
             continue
         missing.add(pkg_name)
     return missing
+
+
+def install_missing_packages(packages, repository=None):
+    """Install the required debian packages if they are missing.
+
+    If repository is not None, add the given apt repository before installing
+    the dependencies.
+    """
+    missing = find_missing_packages(*packages)
+    if missing:
+        if repository is not None:
+            log('Adding the apt repository {}.'.format(repository))
+            install_extra_repositories(repository)
+        log('Installing deb packages: {}.'.format(', '.join(missing)))
+        cmd_log(apt_get_install(*missing))
+    else:
+        log('No missing deb packages.')
 
=== modified file 'revision'
--- revision	2013-10-04 13:40:26 +0000
+++ revision	2013-10-08 07:46:30 +0000
@@ -1,1 +1,1 @@
-88
+89
 
=== modified file 'tests/20-functional.test'
--- tests/20-functional.test	2013-09-25 16:31:24 +0000
+++ tests/20-functional.test	2013-10-08 07:46:30 +0000
@@ -207,8 +207,11 @@
 
     def test_cache_headers(self):
         # Make sure the correct cache headers are sent.
-        unit_info = self.juju_deploy(
-            self.charm, options={'juju-gui-source': JUJU_GUI_TEST_BRANCH})
+        options = {
+            'builtin-server': False,
+            'juju-gui-source': JUJU_GUI_TEST_BRANCH,
+        }
+        unit_info = self.juju_deploy(self.charm, options=options)
         hostname = unit_info['public-address']
         conn = httplib.HTTPSConnection(hostname)
         conn.request('HEAD', '/')
 
=== modified file 'tests/requirements.pip'
--- tests/requirements.pip	2013-10-04 13:08:59 +0000
+++ tests/requirements.pip	2013-10-08 07:46:30 +0000
@@ -37,10 +37,8 @@
 tornado==3.1.1
 
 # Charm hooks.
--e bzr+http://launchpad.net/charm-tools#egg=charm-tools
 launchpadlib==1.10.2
 python-apt==0.8.5
--e bzr+http://launchpad.net/python-shelltoolbox#egg=python-shelltoolbox
 Tempita==0.5.1
 
 # Charm tests + GUI server tests.
 
=== added symlink 'tests/shelltoolbox.py'
=== target is u'../hooks/shelltoolbox.py'
=== modified file 'tests/test_backends.py'
--- tests/test_backends.py	2013-09-03 08:24:30 +0000
+++ tests/test_backends.py	2013-10-08 07:46:30 +0000
@@ -17,258 +17,492 @@
 """Backend tests."""
 
 
-from collections import defaultdict
-from contextlib import contextmanager
+from contextlib import (
+    contextmanager,
+    nested,
+)
 import os
 import shutil
 import tempfile
 import unittest
 
-import charmhelpers
 import mock
-import shelltoolbox
 
 import backend
 import utils
 
 
-def get_mixin_names(test_backend):
-    return tuple(b.__class__.__name__ for b in test_backend.mixins)
-
-
-class GotEmAllDict(defaultdict):
-    """A dictionary that returns the same default value for all given keys."""
-
-    def get(self, key, default=None):
-        return self.default_factory()
+EXPECTED_PYTHON_LEGACY_DEBS = ('apache2', 'curl', 'haproxy', 'openssl')
+EXPECTED_GO_LEGACY_DEBS = (
+    'apache2', 'curl', 'haproxy', 'openssl', 'python-yaml')
+EXPECTED_PYTHON_BUILTIN_DEBS = (
+    'curl', 'openssl', 'python-bzrlib', 'python-pip')
+EXPECTED_GO_BUILTIN_DEBS = (
+    'curl', 'openssl', 'python-bzrlib', 'python-pip', 'python-yaml')
+
+simulate_pyjuju = mock.patch('utils.legacy_juju', mock.Mock(return_value=True))
+simulate_juju_core = mock.patch(
+    'utils.legacy_juju', mock.Mock(return_value=False))
 
 
 class TestBackendProperties(unittest.TestCase):
     """Ensure the correct mixins and property values are collected."""
 
-    simulate_pyjuju = mock.patch(
-        'utils.legacy_juju', mock.Mock(return_value=True))
-    simulate_juju_core = mock.patch(
-        'utils.legacy_juju', mock.Mock(return_value=False))
+    def assert_mixins(self, expected, backend):
+        """Ensure the given backend includes the expected mixins."""
+        obtained = tuple(mixin.__class__.__name__ for mixin in backend.mixins)
+        self.assertEqual(tuple(expected), obtained)
+
+    def assert_dependencies(self, expected_debs, expected_repository, backend):
+        """Ensure the given backend includes the expected dependencies."""
+        obtained_debs, obtained_repository = backend.get_dependencies()
+        self.assertEqual(set(expected_debs), obtained_debs)
+        self.assertEqual(expected_repository, obtained_repository)
 
     def check_sandbox_mode(self):
         """The backend includes the correct mixins when sandbox mode is active.
         """
-        test_backend = backend.Backend(config={
-            'sandbox': True, 'staging': False, 'builtin-server': False})
-        mixin_names = get_mixin_names(test_backend)
-        self.assertEqual(
-            ('SetUpMixin', 'SandboxMixin', 'GuiMixin', 'HaproxyApacheMixin'),
-            mixin_names)
-        self.assertEqual(
-            frozenset(('apache2', 'curl', 'haproxy', 'openssl')),
-            test_backend.debs)
+        expected_mixins = (
+            'SetUpMixin', 'SandboxMixin', 'GuiMixin', 'HaproxyApacheMixin')
+        config = {
+            'builtin-server': False,
+            'repository-location': 'ppa:my/location',
+            'sandbox': True,
+            'staging': False,
+        }
+        test_backend = backend.Backend(config=config)
+        self.assert_mixins(expected_mixins, test_backend)
+        self.assert_dependencies(
+            EXPECTED_PYTHON_LEGACY_DEBS, 'ppa:my/location', test_backend)
 
     def test_python_staging_backend(self):
         expected_mixins = (
             'SetUpMixin', 'ImprovMixin', 'GuiMixin', 'HaproxyApacheMixin')
-        with self.simulate_pyjuju:
-            test_backend = backend.Backend(config={
-                'sandbox': False, 'staging': True, 'builtin-server': False})
-            mixin_names = get_mixin_names(test_backend)
-            self.assertEqual(expected_mixins, mixin_names)
-            debs = ('apache2', 'curl', 'haproxy', 'openssl', 'zookeeper')
-            self.assertEqual(frozenset(debs), test_backend.debs)
+        config = {
+            'builtin-server': False,
+            'repository-location': 'ppa:my/location',
+            'sandbox': False,
+            'staging': True,
+        }
+        with simulate_pyjuju:
+            test_backend = backend.Backend(config=config)
+            self.assert_mixins(expected_mixins, test_backend)
+            self.assert_dependencies(
+                EXPECTED_PYTHON_LEGACY_DEBS + ('zookeeper',),
+                'ppa:my/location', test_backend)
 
     def test_go_staging_backend(self):
         config = {'sandbox': False, 'staging': True, 'builtin-server': False}
-        with self.simulate_juju_core:
+        with simulate_juju_core:
             with self.assertRaises(ValueError) as context_manager:
                 backend.Backend(config=config)
         error = str(context_manager.exception)
         self.assertEqual('Unable to use staging with go backend', error)
 
     def test_python_sandbox_backend(self):
-        with self.simulate_pyjuju:
+        with simulate_pyjuju:
             self.check_sandbox_mode()
 
     def test_go_sandbox_backend(self):
-        with self.simulate_juju_core:
+        with simulate_juju_core:
             self.check_sandbox_mode()
 
     def test_python_backend(self):
         expected_mixins = (
             'SetUpMixin', 'PythonMixin', 'GuiMixin', 'HaproxyApacheMixin')
-        with self.simulate_pyjuju:
-            test_backend = backend.Backend(config={
-                'sandbox': False, 'staging': False, 'builtin-server': False})
-            mixin_names = get_mixin_names(test_backend)
-            self.assertEqual(expected_mixins, mixin_names)
-            self.assertEqual(
-                frozenset(('apache2', 'curl', 'haproxy', 'openssl')),
-                test_backend.debs)
+        config = {
+            'builtin-server': False,
+            'repository-location': 'ppa:my/location',
+            'sandbox': False,
+            'staging': False,
+        }
+        with simulate_pyjuju:
+            test_backend = backend.Backend(config=config)
+            self.assert_mixins(expected_mixins, test_backend)
+            self.assert_dependencies(
+                EXPECTED_PYTHON_LEGACY_DEBS, 'ppa:my/location', test_backend)
 
     def test_go_backend(self):
-        with self.simulate_juju_core:
-            test_backend = backend.Backend(config={
-                'sandbox': False, 'staging': False, 'builtin-server': False})
-            mixin_names = get_mixin_names(test_backend)
-            self.assertEqual(
-                ('SetUpMixin', 'GoMixin', 'GuiMixin', 'HaproxyApacheMixin'),
-                mixin_names)
-            self.assertEqual(
-                frozenset(
-                    ('apache2', 'curl', 'haproxy', 'openssl', 'python-yaml')),
-                test_backend.debs)
+        expected_mixins = (
+            'SetUpMixin', 'GoMixin', 'GuiMixin', 'HaproxyApacheMixin')
+        config = {
+            'builtin-server': False,
+            'repository-location': 'ppa:my/location',
+            'sandbox': False,
+            'staging': False,
+        }
+        with simulate_juju_core:
+            test_backend = backend.Backend(config=config)
+            self.assert_mixins(expected_mixins, test_backend)
+            self.assert_dependencies(
+                EXPECTED_GO_LEGACY_DEBS, 'ppa:my/location', test_backend)
 
-    def test_builtin_server(self):
+    def test_go_builtin_server(self):
+        config = {
+            'builtin-server': True,
+            'repository-location': 'ppa:my/location',
+            'sandbox': False,
+            'staging': False,
+        }
         expected_mixins = (
             'SetUpMixin', 'GoMixin', 'GuiMixin', 'BuiltinServerMixin')
-        expected_debs = set([
-            'python-pip', 'python-yaml', 'curl', 'openssl', 'python-bzrlib'])
-        with self.simulate_juju_core:
-            test_backend = backend.Backend(config={
-                'sandbox': False, 'staging': False, 'builtin-server': True})
-            mixin_names = get_mixin_names(test_backend)
-            self.assertEqual(expected_mixins, mixin_names)
-            self.assertEqual(expected_debs, test_backend.debs)
+        with simulate_juju_core:
+            test_backend = backend.Backend(config)
+            self.assert_mixins(expected_mixins, test_backend)
+            self.assert_dependencies(
+                EXPECTED_GO_BUILTIN_DEBS, None, test_backend)
+
+    def test_python_builtin_server(self):
+        config = {
+            'builtin-server': True,
+            'repository-location': 'ppa:my/location',
+            'sandbox': False,
+            'staging': False,
+        }
+        expected_mixins = (
+            'SetUpMixin', 'PythonMixin', 'GuiMixin', 'BuiltinServerMixin')
+        with simulate_pyjuju:
+            test_backend = backend.Backend(config)
+            self.assert_mixins(expected_mixins, test_backend)
+            self.assert_dependencies(
+                EXPECTED_PYTHON_BUILTIN_DEBS, None, test_backend)
+
+    def test_sandbox_builtin_server(self):
+        config = {
+            'builtin-server': True,
+            'repository-location': 'ppa:my/location',
+            'sandbox': True,
+            'staging': False,
+        }
+        expected_mixins = (
+            'SetUpMixin', 'SandboxMixin', 'GuiMixin', 'BuiltinServerMixin')
+        with simulate_juju_core:
+            test_backend = backend.Backend(config)
+            self.assert_mixins(expected_mixins, test_backend)
+            self.assert_dependencies(
+                EXPECTED_PYTHON_BUILTIN_DEBS, None, test_backend)
 
 
 class TestBackendCommands(unittest.TestCase):
 
     def setUp(self):
-        self.called = {}
-        self.alwaysFalse = GotEmAllDict(lambda: False)
-        self.alwaysTrue = GotEmAllDict(lambda: True)
-
-        # Monkeypatch functions.
-        self.utils_mocks = {
-            'compute_build_dir': utils.compute_build_dir,
-            'fetch_api': utils.fetch_api,
-            'fetch_gui_from_branch': utils.fetch_gui_from_branch,
-            'fetch_gui_release': utils.fetch_gui_release,
-            'find_missing_packages': utils.find_missing_packages,
-            'get_api_address': utils.get_api_address,
-            'get_npm_cache_archive_url': utils.get_npm_cache_archive_url,
-            'install_builtin_server': utils.install_builtin_server,
-            'parse_source': utils.parse_source,
-            'prime_npm_cache': utils.prime_npm_cache,
-            'remove_apache_setup': utils.remove_apache_setup,
-            'remove_haproxy_setup': utils.remove_haproxy_setup,
-            'save_or_create_certificates': utils.save_or_create_certificates,
-            'setup_apache_config': utils.setup_apache_config,
-            'setup_gui': utils.setup_gui,
-            'setup_haproxy_config': utils.setup_haproxy_config,
-            'start_agent': utils.start_agent,
-            'start_improv': utils.start_improv,
-            'write_builtin_server_startup': utils.write_builtin_server_startup,
-            'write_gui_config': utils.write_gui_config,
-        }
-        self.charmhelpers_mocks = {
-            'log': charmhelpers.log,
-            'open_port': charmhelpers.open_port,
-            'service_control': charmhelpers.service_control,
-        }
-
-        def make_mock_function(name):
-            def mock_function(*args, **kwargs):
-                self.called[name] = True
-                return (None, None)
-            mock_function.__name__ = name
-            return mock_function
-
-        for name in self.utils_mocks.keys():
-            setattr(utils, name, make_mock_function(name))
-        for name in self.charmhelpers_mocks.keys():
-            setattr(charmhelpers, name, make_mock_function(name))
-
-        @contextmanager
-        def mock_su(user):
-            self.called['su'] = True
-            yield
-        self.orig_su = utils.su
-        utils.su = mock_su
-
-        def mock_apt_get_install(*debs):
-            self.called['apt_get_install'] = True
-        self.orig_apt_get_install = shelltoolbox.apt_get_install
-        shelltoolbox.apt_get_install = mock_apt_get_install
-
-        def mock_run(*debs):
-            self.called['run'] = True
-        self.orig_run = shelltoolbox.run
-        shelltoolbox.run = mock_run
-
-        # Monkeypatch directories.
+        # Set up directories.
         self.playground = tempfile.mkdtemp()
-        self.orig_juju_dir = utils.JUJU_AGENT_DIR
-        utils.JUJU_AGENT_DIR = tempfile.mkdtemp(dir=self.playground)
-        self.orig_base_dir = utils.BASE_DIR
-        utils.BASE_DIR = os.path.join(self.playground, 'juju-gui')
-
-    def tearDown(self):
-        # Cleanup directories.
-        shutil.rmtree(self.playground)
-        utils.JUJU_AGENT_DIR = self.orig_juju_dir
-        utils.BASE_DIR = self.orig_base_dir
-        # Undo the monkeypatching.
-        shelltoolbox.run = self.orig_run
-        shelltoolbox.apt_get_install = self.orig_apt_get_install
-        utils.su = self.orig_su
-        for name, orig_fun in self.charmhelpers_mocks.items():
-            setattr(charmhelpers, name, orig_fun)
-        for name, orig_fun in self.utils_mocks.items():
-            setattr(utils, name, orig_fun)
+        self.addCleanup(shutil.rmtree, self.playground)
+        self.base_dir = os.path.join(self.playground, 'juju-gui')
+        self.command_log_file = os.path.join(self.playground, 'logs')
+        self.juju_agent_dir = os.path.join(self.playground, 'juju-agent-dir')
+        self.ssl_cert_path = os.path.join(self.playground, 'ssl-cert-path')
+        # Set up default values.
+        self.juju_api_branch = 'lp:juju-api'
+        self.juju_gui_source = 'stable'
+        self.repository_location = 'ppa:my/location'
+        self.parse_source_return_value = ('stable', None)
+
+    def make_config(self, options=None):
+        """Create and return a backend configuration dict."""
+        config = {
+            'builtin-server': True,
+            'builtin-server-logging': 'info',
+            'charmworld-url': 'http://charmworld.example.com',
+            'command-log-file': self.command_log_file,
+            'default-viewmode': 'sidebar',
+            'ga-key': 'my-key',
+            'juju-api-branch': self.juju_api_branch,
+            'juju-gui-debug': False,
+            'juju-gui-console-enabled': False,
+            'juju-gui-source': self.juju_gui_source,
+            'login-help': 'login-help',
+            'read-only': False,
+            'repository-location': self.repository_location,
+            'sandbox': False,
+            'secure': True,
+            'serve-tests': False,
+            'show-get-juju-button': False,
+            'ssl-cert-path': self.ssl_cert_path,
+            'staging': False,
+        }
+        if options is not None:
+            config.update(options)
+        return config
+
+    @contextmanager
+    def mock_all(self):
+        """Mock all the extrenal functions used by the backend framework."""
+        mock_parse_source = mock.Mock(
+            return_value=self.parse_source_return_value)
+        mocks = {
+            'base_dir': mock.patch('backend.utils.BASE_DIR', self.base_dir),
+            'compute_build_dir': mock.patch('backend.utils.compute_build_dir'),
+            'fetch_api': mock.patch('backend.utils.fetch_api'),
+            'fetch_gui_from_branch': mock.patch(
+                'backend.utils.fetch_gui_from_branch'),
+            'fetch_gui_release': mock.patch('backend.utils.fetch_gui_release'),
+            'install_builtin_server': mock.patch(
+                'backend.utils.install_builtin_server'),
+            'install_missing_packages': mock.patch(
+                'backend.utils.install_missing_packages'),
+            'juju_agent_dir': mock.patch(
+                'backend.utils.JUJU_AGENT_DIR', self.juju_agent_dir),
+            'log': mock.patch('backend.log'),
+            'open_port': mock.patch('backend.open_port'),
+            'parse_source': mock.patch(
+                'backend.utils.parse_source', mock_parse_source),
+            'save_or_create_certificates': mock.patch(
+                'backend.utils.save_or_create_certificates'),
+            'setup_gui': mock.patch('backend.utils.setup_gui'),
+            'start_agent': mock.patch('backend.utils.start_agent'),
+            'start_builtin_server': mock.patch(
+                'backend.utils.start_builtin_server'),
+            'start_haproxy_apache': mock.patch(
+                'backend.utils.start_haproxy_apache'),
+            'stop_agent': mock.patch('backend.utils.stop_agent'),
+            'stop_builtin_server': mock.patch(
+                'backend.utils.stop_builtin_server'),
+            'stop_haproxy_apache': mock.patch(
+                'backend.utils.stop_haproxy_apache'),
+            'write_gui_config': mock.patch('backend.utils.write_gui_config'),
+        }
+        # Note: nested is deprecated for good reasons which do not apply here.
+        # Used here to easily nest a dynamically generated list of context
+        # managers.
+        with nested(*mocks.values()) as context_managers:
+            object_dict = dict(zip(mocks.keys(), context_managers))
+            yield type('Mocks', (object,), object_dict)
+
+    def assert_write_gui_config_called(self, mocks, config):
+        """Ensure the mocked write_gui_config has been properly called."""
+        mocks.write_gui_config.assert_called_once_with(
+            config['juju-gui-console-enabled'], config['login-help'],
+            config['read-only'], config['staging'], config['charmworld-url'],
+            mocks.compute_build_dir(), secure=config['secure'],
+            sandbox=config['sandbox'], ga_key=config['ga-key'],
+            default_viewmode=config['default-viewmode'],
+            show_get_juju_button=config['show-get-juju-button'], password=None)
 
     def test_base_dir_created(self):
-        test_backend = backend.Backend(config=self.alwaysFalse)
-        test_backend.install()
-        self.assertTrue(os.path.isdir(utils.BASE_DIR))
+        # The base Juju GUI directory is correctly created.
+        config = self.make_config()
+        test_backend = backend.Backend(config=config)
+        with self.mock_all():
+            test_backend.install()
+        self.assertTrue(os.path.isdir(self.base_dir))
 
     def test_base_dir_removed(self):
-        test_backend = backend.Backend(config=self.alwaysFalse)
-        test_backend.install()
-        test_backend.destroy()
+        # The base Juju GUI directory is correctly removed.
+        config = self.make_config()
+        test_backend = backend.Backend(config=config)
+        with self.mock_all():
+            test_backend.install()
+            test_backend.destroy()
         self.assertFalse(os.path.exists(utils.BASE_DIR), utils.BASE_DIR)
 
-    def test_install_python(self):
-        test_backend = backend.Backend(config=self.alwaysFalse)
-        test_backend.install()
-        for mocked in (
-            'apt_get_install', 'fetch_api', 'find_missing_packages',
-        ):
-            self.assertTrue(
-                self.called.get(mocked), '{} was not called'.format(mocked))
-
-    def test_install_improv_builtin(self):
-        test_backend = backend.Backend(config=self.alwaysTrue)
-        test_backend.install()
-        for mocked in (
-            'apt_get_install', 'fetch_api', 'find_missing_packages',
-            'install_builtin_server',
-        ):
-            self.assertTrue(
-                self.called.get(mocked), '{} was not called'.format(mocked))
-
-    def test_start_agent(self):
-        test_backend = backend.Backend(config=self.alwaysFalse)
-        test_backend.start()
-        for mocked in (
-            'compute_build_dir', 'open_port', 'setup_apache_config',
-            'setup_haproxy_config', 'start_agent', 'su', 'write_gui_config',
-        ):
-            self.assertTrue(
-                self.called.get(mocked), '{} was not called'.format(mocked))
-
-    def test_start_improv_builtin(self):
-        test_backend = backend.Backend(config=self.alwaysTrue)
-        test_backend.start()
-        for mocked in (
-            'compute_build_dir', 'open_port', 'start_improv', 'su',
-            'write_builtin_server_startup', 'write_gui_config',
-        ):
-            self.assertTrue(
-                self.called.get(mocked), '{} was not called'.format(mocked))
-
-    def test_stop(self):
-        test_backend = backend.Backend(config=self.alwaysFalse)
-        test_backend.stop()
-        self.assertTrue(self.called.get('su'), 'su was not called')
+    def test_install_python_legacy_stable(self):
+        # Install a pyJuju backend with legacy server and stable release.
+        config = self.make_config({'builtin-server': False})
+        with simulate_pyjuju:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.install()
+        mocks.install_missing_packages.assert_called_once_with(
+            set(EXPECTED_PYTHON_LEGACY_DEBS),
+            repository=self.repository_location)
+        mocks.fetch_api.assert_called_once_with(self.juju_api_branch)
+        mocks.parse_source.assert_called_once_with(self.juju_gui_source)
+        mocks.fetch_gui_release.assert_called_once_with(
+            *self.parse_source_return_value)
+        self.assertFalse(mocks.fetch_gui_from_branch.called)
+        mocks.setup_gui.assert_called_once_with(mocks.fetch_gui_release())
+        self.assertFalse(mocks.install_builtin_server.called)
+
+    def test_install_go_legacy_stable(self):
+        # Install a juju-core backend with legacy server and stable release.
+        config = self.make_config({'builtin-server': False})
+        with simulate_juju_core:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.install()
+        mocks.install_missing_packages.assert_called_once_with(
+            set(EXPECTED_GO_LEGACY_DEBS), repository=self.repository_location)
+        self.assertFalse(mocks.fetch_api.called)
+        mocks.parse_source.assert_called_once_with(self.juju_gui_source)
+        mocks.fetch_gui_release.assert_called_once_with(
+            *self.parse_source_return_value)
+        self.assertFalse(mocks.fetch_gui_from_branch.called)
+        mocks.setup_gui.assert_called_once_with(mocks.fetch_gui_release())
+        self.assertFalse(mocks.install_builtin_server.called)
+
+    def test_install_python_builtin_stable(self):
+        # Install a pyJuju backend with builtin server and stable release.
+        config = self.make_config({'builtin-server': True})
+        with simulate_pyjuju:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.install()
+        mocks.install_missing_packages.assert_called_once_with(
+            set(EXPECTED_PYTHON_BUILTIN_DEBS), repository=None)
+        mocks.fetch_api.assert_called_once_with(self.juju_api_branch)
+        mocks.parse_source.assert_called_once_with(self.juju_gui_source)
+        mocks.fetch_gui_release.assert_called_once_with(
+            *self.parse_source_return_value)
+        self.assertFalse(mocks.fetch_gui_from_branch.called)
+        mocks.setup_gui.assert_called_once_with(mocks.fetch_gui_release())
+        mocks.install_builtin_server.assert_called_once_with()
+
+    def test_install_go_builtin_stable(self):
+        # Install a juju-core backend with builtin server and stable release.
+        config = self.make_config({'builtin-server': True})
+        with simulate_juju_core:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.install()
+        mocks.install_missing_packages.assert_called_once_with(
+            set(EXPECTED_GO_BUILTIN_DEBS), repository=None)
+        self.assertFalse(mocks.fetch_api.called)
+        mocks.parse_source.assert_called_once_with(self.juju_gui_source)
+        mocks.fetch_gui_release.assert_called_once_with(
+            *self.parse_source_return_value)
+        self.assertFalse(mocks.fetch_gui_from_branch.called)
+        mocks.setup_gui.assert_called_once_with(mocks.fetch_gui_release())
+        mocks.install_builtin_server.assert_called_once_with()
+
+    def test_install_go_builtin_branch(self):
+        # Install a juju-core backend with builtin server and branch release.
+        self.parse_source_return_value = ('branch', ('lp:juju-gui', 42))
+        expected_calls = [
+            mock.call(set(EXPECTED_GO_BUILTIN_DEBS), repository=None),
+            mock.call(
+                utils.DEB_BUILD_DEPENDENCIES,
+                repository=self.repository_location,
+            ),
+        ]
+        config = self.make_config({'builtin-server': True})
+        with simulate_juju_core:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.install()
+        mocks.install_missing_packages.assert_has_calls(expected_calls)
+        self.assertFalse(mocks.fetch_api.called)
+        mocks.parse_source.assert_called_once_with(self.juju_gui_source)
+        mocks.fetch_gui_from_branch.assert_called_once_with(
+            'lp:juju-gui', 42, self.command_log_file)
+        self.assertFalse(mocks.fetch_gui_release.called)
+        mocks.setup_gui.assert_called_once_with(mocks.fetch_gui_from_branch())
+        mocks.install_builtin_server.assert_called_once_with()
+
+    def test_start_python_legacy(self):
+        # Start a pyJuju backend with legacy server.
+        config = self.make_config({'builtin-server': False})
+        with simulate_pyjuju:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.start()
+        mocks.start_agent.assert_called_once_with(self.ssl_cert_path)
+        mocks.compute_build_dir.assert_called_with(
+            config['juju-gui-debug'], config['serve-tests'])
+        self.assert_write_gui_config_called(mocks, config)
+        mocks.open_port.assert_has_calls([mock.call(80), mock.call(443)])
+        mocks.start_haproxy_apache.assert_called_once_with(
+            mocks.compute_build_dir(), config['serve-tests'],
+            self.ssl_cert_path, config['secure'])
+        self.assertFalse(mocks.start_builtin_server.called)
+
+    def test_start_go_legacy(self):
+        # Start a juju-core backend with legacy server.
+        config = self.make_config({'builtin-server': False})
+        with simulate_juju_core:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.start()
+        self.assertFalse(mocks.start_agent.called)
+        mocks.compute_build_dir.assert_called_with(
+            config['juju-gui-debug'], config['serve-tests'])
+        self.assert_write_gui_config_called(mocks, config)
+        mocks.open_port.assert_has_calls([mock.call(80), mock.call(443)])
+        mocks.start_haproxy_apache.assert_called_once_with(
+            mocks.compute_build_dir(), config['serve-tests'],
+            self.ssl_cert_path, config['secure'])
+        self.assertFalse(mocks.start_builtin_server.called)
+
+    def test_start_python_builtin(self):
+        # Start a pyJuju backend with builtin server.
+        config = self.make_config({'builtin-server': True})
+        with simulate_pyjuju:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.start()
+        mocks.start_agent.assert_called_once_with(self.ssl_cert_path)
+        mocks.compute_build_dir.assert_called_with(
+            config['juju-gui-debug'], config['serve-tests'])
+        self.assert_write_gui_config_called(mocks, config)
+        mocks.open_port.assert_has_calls([mock.call(80), mock.call(443)])
+        mocks.start_builtin_server.assert_called_once_with(
+            mocks.compute_build_dir(), self.ssl_cert_path,
+            config['serve-tests'], config['sandbox'],
+            config['builtin-server-logging'], not config['secure'])
+        self.assertFalse(mocks.start_haproxy_apache.called)
+
+    def test_start_go_builtin(self):
+        # Start a juju-core backend with builtin server.
+        config = self.make_config({'builtin-server': True})
+        with simulate_juju_core:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.start()
+        self.assertFalse(mocks.start_agent.called)
+        mocks.compute_build_dir.assert_called_with(
+            config['juju-gui-debug'], config['serve-tests'])
+        self.assert_write_gui_config_called(mocks, config)
+        mocks.open_port.assert_has_calls([mock.call(80), mock.call(443)])
+        mocks.start_builtin_server.assert_called_once_with(
+            mocks.compute_build_dir(), self.ssl_cert_path,
+            config['serve-tests'], config['sandbox'],
+            config['builtin-server-logging'], not config['secure'])
+        self.assertFalse(mocks.start_haproxy_apache.called)
+
+    def test_stop_python_legacy(self):
+        # Stop a pyJuju backend with legacy server.
+        config = self.make_config({'builtin-server': False})
+        with simulate_pyjuju:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.stop()
+        mocks.stop_agent.assert_called_once_with()
+        mocks.stop_haproxy_apache.assert_called_once_with()
+        self.assertFalse(mocks.stop_builtin_server.called)
+
+    def test_stop_go_legacy(self):
+        # Stop a juju-core backend with legacy server.
+        config = self.make_config({'builtin-server': False})
+        with simulate_juju_core:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.stop()
+        self.assertFalse(mocks.stop_agent.called)
+        mocks.stop_haproxy_apache.assert_called_once_with()
+        self.assertFalse(mocks.stop_builtin_server.called)
+
+    def test_stop_python_builtin(self):
+        # Stop a pyJuju backend with builtin server.
+        config = self.make_config({'builtin-server': True})
+        with simulate_pyjuju:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.stop()
+        mocks.stop_agent.assert_called_once_with()
+        mocks.stop_builtin_server.assert_called_once_with()
+        self.assertFalse(mocks.stop_haproxy_apache.called)
+
+    def test_stop_go_builtin(self):
+        # Stop a juju-core backend with builtin server.
+        config = self.make_config({'builtin-server': True})
+        with simulate_juju_core:
+            test_backend = backend.Backend(config=config)
+            with self.mock_all() as mocks:
+                test_backend.stop()
+        self.assertFalse(mocks.stop_agent.called)
+        mocks.stop_builtin_server.assert_called_once_with()
+        self.assertFalse(mocks.stop_haproxy_apache.called)
 
 
 class TestBackendUtils(unittest.TestCase):
@@ -294,23 +528,28 @@
         self.assertFalse(test_backend.different('staging'))
 
 
-class TestChainMethods(unittest.TestCase):
+class TestCallMethods(unittest.TestCase):
 
     def setUp(self):
         self.called = []
-
-        def method(mixin, backend):
-            self.called.append(mixin.__class__.__name__)
-        mixin1 = type('Mixin1', (object,), {'method': method})()
-        mixin2 = type('Mixin2', (object,), {'method': method})()
-        self.backend = type('Backend', (), {'mixins': (mixin1, mixin2)})()
-
-    def test_chain(self):
-        method = backend.chain_methods('method')
-        method(self.backend)
-        self.assertEqual(['Mixin1', 'Mixin2'], self.called)
-
-    def test_reversed(self):
-        method = backend.chain_methods('method', reverse=True)
-        method(self.backend)
-        self.assertEqual(['Mixin2', 'Mixin1'], self.called)
+        self.objects = [self.make_object('Obj1'), self.make_object('Obj2')]
+
+    def make_object(self, name, has_method=True):
+        """Create and return an test object with the given name."""
+        def method(obj, *args):
+            self.called.append([obj.__class__.__name__, args])
+        object_dict = {'method': method} if has_method else {}
+        return type(name, (object,), object_dict)()
+
+    def test_call(self):
+        # The methods are correctly called.
+        backend.call_methods(self.objects, 'method', 'arg1', 'arg2')
+        expected = [['Obj1', ('arg1', 'arg2')], ['Obj2', ('arg1', 'arg2')]]
+        self.assertEqual(expected, self.called)
+
+    def test_no_method(self):
+        # An object without the method is ignored.
+        self.objects.append(self.make_object('Obj3', has_method=False))
+        backend.call_methods(self.objects, 'method')
+        expected = [['Obj1', ()], ['Obj2', ()]]
+        self.assertEqual(expected, self.called)
 
=== modified file 'tests/test_utils.py'
--- tests/test_utils.py	2013-09-30 20:16:09 +0000
+++ tests/test_utils.py	2013-10-08 07:46:30 +0000
@@ -48,6 +48,7 @@
     parse_source,
     get_npm_cache_archive_url,
     install_builtin_server,
+    install_missing_packages,
     remove_apache_setup,
     remove_haproxy_setup,
     render_to_file,
@@ -868,6 +869,55 @@
         self.assertEqual(0, mock_run.call_count)
 
 
[e-mail]('utils.find_missing_packages')
[e-mail]('utils.install_extra_repositories')
[e-mail]('utils.apt_get_install')
[e-mail]('utils.log')
[e-mail]('utils.cmd_log', mock.Mock())
+class TestInstallMissingPackages(unittest.TestCase):
+
+    packages = ('pkg1', 'pkg2', 'pkg3')
+    repository = 'ppa:my/repository'
+
+    def test_missing(
+            self, mock_log, mock_apt_get_install,
+            mock_install_extra_repositories, mock_find_missing_packages):
+        # The extra repository and packages are correctly installed.
+        repository = self.repository
+        mock_find_missing_packages.return_value = ['pkg1', 'pkg2']
+        install_missing_packages(self.packages, repository=repository)
+        mock_find_missing_packages.assert_called_once_with(*self.packages)
+        mock_install_extra_repositories.assert_called_once_with(repository)
+        mock_apt_get_install.assert_called_once_with('pkg1', 'pkg2')
+        mock_log.assert_has_calls([
+            mock.call('Adding the apt repository ppa:my/repository.'),
+            mock.call('Installing deb packages: pkg1, pkg2.')
+        ])
+
+    def test_missing_no_repository(
+            self, mock_log, mock_apt_get_install,
+            mock_install_extra_repositories, mock_find_missing_packages):
+        # No repositories are installed if not passed.
+        mock_find_missing_packages.return_value = ['pkg1', 'pkg2']
+        install_missing_packages(self.packages)
+        mock_find_missing_packages.assert_called_once_with(*self.packages)
+        self.assertFalse(mock_install_extra_repositories.called)
+        mock_apt_get_install.assert_called_once_with('pkg1', 'pkg2')
+        mock_log.assert_called_once_with(
+            'Installing deb packages: pkg1, pkg2.')
+
+    def test_no_missing(
+            self, mock_log, mock_apt_get_install,
+            mock_install_extra_repositories, mock_find_missing_packages):
+        # Nothing is installed if no missing packages are found.
+        mock_find_missing_packages.return_value = []
+        install_missing_packages(self.packages, repository=self.repository)
+        mock_find_missing_packages.assert_called_once_with(*self.packages)
+        self.assertFalse(mock_install_extra_repositories.called)
+        self.assertFalse(mock_apt_get_install.called)
+        mock_log.assert_called_once_with('No missing deb packages.')
+
+
 class TestNpmCache(unittest.TestCase):
     """To speed building from a branch we prepopulate the NPM cache."""