Merge lp:~frankban/charms/precise/juju-gui/local-releases into lp:~juju-gui/charms/precise/juju-gui/trunk

[Francesco Banconi]

Introduce local GUI releases.

Implemented the new "local" juju-gui-source option.
By default the GUI release is now retrieved from
the charm itself, so that in the deployment process
the charm can avoid connecting to Launchpad.

This should result in the charm being able to be
deployed behind a firewall without the traditional
workarounds. Note that this is not yet demonstrated
and must be QAed in a real firewalled environment.

The process tries is like the following:
- if juju-gui-source is "local", the last tarball
  in the releases dir is installed;
- if juju-gui-source is a stable or development version,
  the charm checks if that version is present in the
  local repository before downloading it from Launchpad.
- the rest of the options should still work as usual
  (but some QA could help, e.g. deploying from a branch).

Also updated the charm documentation and added some
missing tests.

Tests: `make unittest` from the branch root
(I am currently running the functional tests).

QA (assuming 0.10.1 is the latest GUI release):
- `juju bootstrap --debug`;
- `make deploy`;
- check the logs: no PPAs are used, the local release
  is installed;
- `juju set juju-gui juju-gui-source=0.10.0`;
- check the logs: the release is downloaded from
  Launchpad;
- `juju set juju-gui juju-gui-source=0.10.1`;
- check the logs: the charm successfully find
  the 0.10.1 release in the local repository and
  avoids downloading it from Launchpad.
- keep trying to break the charm in all the ways
  you can imagine...

Thank you.

https://codereview.appspot.com/14545044/

[Francesco Banconi (frankban)]

Reviewers: mp+189896_code.launchpad.net,

Message:
Please take a look.

Description:
Introduce local GUI releases.

Implemented the new "local" juju-gui-source option.
By default the GUI release is now retrieved from
the charm itself, so that in the deployment process
the charm can avoid connecting to Launchpad.

This should result in the charm being able to be
deployed behind a firewall without the traditional
workarounds. Note that this is not yet demonstrated
and must be QAed in a real firewalled environment.

The process tries is like the following:
- if juju-gui-source is "local", the last tarball
   in the releases dir is installed;
- if juju-gui-source is a stable or development version,
   the charm checks if that version is present in the
   local repository before downloading it from Launchpad.
- the rest of the options should still work as usual
   (but some QA could help, e.g. deploying from a branch).

Also updated the charm documentation and added some
missing tests.

Tests: `make unittest` from the branch root
(I am currently running the functional tests).

QA (assuming 0.10.1 is the latest GUI release):
- `juju bootstrap --debug`;
- `make deploy`;
- check the logs: no PPAs are used, the local release
   is installed;
- `juju set juju-gui juju-gui-source=0.10.0`;
- check the logs: the release is downloaded from
   Launchpad;
- `juju set juju-gui juju-gui-source=0.10.1`;
- check the logs: the charm successfully find
   the 0.10.1 release in the local repository and
   avoids downloading it from Launchpad.
- keep trying to break the charm in all the ways
   you can imagine...

Thank you.

https://code.launchpad.net/~frankban/charms/precise/juju-gui/local-releases/+merge/189896

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/14545044/

Affected files (+395, -57 lines):
   M HACKING.md
   M README.md
   A [revision details]
   M config.yaml
   M hooks/utils.py
   A releases/juju-gui-0.10.1.tgz
   M revision
   M tests/20-functional.test
   M tests/test_utils.py

[Madison Scott-Clary (makyo)]

LGTM, QA okay.

https://codereview.appspot.com/14545044/

[Gary Poster (gary)]

LGTM with many trivials and one slightly-more-than-trivial-but-not-much.
  Relying on Makyo's QA. Great, thank you!

Gary

https://codereview.appspot.com/14545044/diff/1/config.yaml
File config.yaml (right):

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode20
config.yaml:20: Where to install Juju GUI from. Possible values are:
Possible values are the following.

[changes below will largely be punctuation and capitalization, shifting
semicolons to periods and capitalizing the start of new sentences.]

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode21
config.yaml:21: - 'local' (default): the latest local release will be
deployed. Releases
Capitalize "The"

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode22
config.yaml:22: are stored in the releases directory of this charm;
...charm; -> ...charm.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode23
config.yaml:23: - 'stable' the latest stable release will be deployed;
The latest release from the "stable" series will be deployed.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode24
config.yaml:24: - 'trunk': the latest trunk release will be deployed;
The latest release from the "trunk" series will be deployed. Please
note that this is not a build of the current Juju GUI trunk. For that
functionality, use "lp:juju-gui", as described below.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode25
config.yaml:25: - a stable version (e.g '0.1.0'): the specified stable
version will be
The...

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode28
config.yaml:28: release will be downloaded from Launchpad;
...Launchpad.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode29
config.yaml:29: - a trunk version (e.g '0.1.0+build.1'): the specified
trunk version
The...

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode32
config.yaml:32: release will be downloaded from Launchpad;
...Launchpad.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode33
config.yaml:33: - a Bazaar branch (e.g. 'lp:juju-gui'): a release will
be created and
A release...

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode36
config.yaml:36: revision, e.g. 'lp:juju-gui:42' will checkout revno 42;
...revno 42.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode37
config.yaml:37: - a 'url:' prefixed url (ex: url:http://...) of a
specific location
- a 'url:' prefixed url: The release found at the given URL (ex:
url:http://... or url:file://...) will be deployed.

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py
File hooks/utils.py (right):

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py#newcode165
hooks/utils.py:165: \.tgz # File extension.
would you mind changing this to

     \.(tgz|xz)

per https://bugs.launchpad.net/juju-gui/+bug/1218888 ?

...No, never mind. We can do that in a separate branch, later. It
would be good to decrease the charm size, now that releases will be
included, but not something we have to tackle right now. Stop me! :-)

https://codereview.appspot.com/14545044/diff/1/hooks/utils.py#newcode...

[lp:~frankban/charms/precise/juju-gui/local-releases]

127. By Francesco Banconi on 2013-10-09

Changes as per review.

[Francesco Banconi (frankban)]

*** Submitted:

Introduce local GUI releases.

Implemented the new "local" juju-gui-source option.
By default the GUI release is now retrieved from
the charm itself, so that in the deployment process
the charm can avoid connecting to Launchpad.

This should result in the charm being able to be
deployed behind a firewall without the traditional
workarounds. Note that this is not yet demonstrated
and must be QAed in a real firewalled environment.

The process tries is like the following:
- if juju-gui-source is "local", the last tarball
   in the releases dir is installed;
- if juju-gui-source is a stable or development version,
   the charm checks if that version is present in the
   local repository before downloading it from Launchpad.
- the rest of the options should still work as usual
   (but some QA could help, e.g. deploying from a branch).

Also updated the charm documentation and added some
missing tests.

Tests: `make unittest` from the branch root
(I am currently running the functional tests).

QA (assuming 0.10.1 is the latest GUI release):
- `juju bootstrap --debug`;
- `make deploy`;
- check the logs: no PPAs are used, the local release
   is installed;
- `juju set juju-gui juju-gui-source=0.10.0`;
- check the logs: the release is downloaded from
   Launchpad;
- `juju set juju-gui juju-gui-source=0.10.1`;
- check the logs: the charm successfully find
   the 0.10.1 release in the local repository and
   avoids downloading it from Launchpad.
- keep trying to break the charm in all the ways
   you can imagine...

Thank you.

R=matthew.scott, gary.poster
CC=
https://codereview.appspot.com/14545044

https://codereview.appspot.com/14545044/diff/1/config.yaml
File config.yaml (right):

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode20
config.yaml:20: Where to install Juju GUI from. Possible values are:
On 2013/10/08 18:29:31, gary.poster wrote:
> Possible values are the following.

> [changes below will largely be punctuation and capitalization,
shifting
> semicolons to periods and capitalizing the start of new sentences.]

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode21
config.yaml:21: - 'local' (default): the latest local release will be
deployed. Releases
On 2013/10/08 18:29:31, gary.poster wrote:
> Capitalize "The"

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode22
config.yaml:22: are stored in the releases directory of this charm;
On 2013/10/08 18:29:31, gary.poster wrote:
> ...charm; -> ...charm.

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode23
config.yaml:23: - 'stable' the latest stable release will be deployed;
On 2013/10/08 18:29:31, gary.poster wrote:
> The latest release from the "stable" series will be deployed.

Done.

https://codereview.appspot.com/14545044/diff/1/config.yaml#newcode24
config.yaml:24: - 'trunk': the latest trunk release will be deployed;
On 2013/10/08 18:29:31, gary.poster wrote:
> The latest release from the "trunk" series will be deployed. Please
note that
> this is not a build of the current Juju GUI trunk. For that
functionality, use
> "lp:juju-gui", as described below.

Good clarification! Done.

h...

[Francesco Banconi (frankban)]

Thank you both for the reviews!

https://codereview.appspot.com/14545044/

=== modified file 'HACKING.md'
--- HACKING.md	2013-08-28 14:47:13 +0000
+++ HACKING.md	2013-10-09 08:56:52 +0000
@@ -161,3 +161,17 @@
 
 All of this is described in more detail on the Juju site: this is an
 introduction to the process.
+
+## Upgrading the local releases repository ##
+
+The charm, in the default juju-gui-source configuration ("local"), deploys the
+GUI from the local releases repository. The repository is the `releases`
+directory located in the branch source, and contains GUI releases as tarball
+files. When a new GUI version is released, we want to also update the local
+repository. To do that, just copy the GUI tarball file to the `releases`
+directory, and commit a new revision of the charm.
+It is safe to remove the older files, and it is even recommended: too many
+releases can slow down the deployment process, especially when deploying from
+the local charm.
+Note that at least one release must be always present in the repository,
+otherwise the deployment process will fail.

=== modified file 'README.md'
--- README.md	2013-08-02 14:19:13 +0000
+++ README.md	2013-10-09 08:56:52 +0000
@@ -18,7 +18,7 @@
 
 ## Demo and Staging Servers ##
 
-The Juju GUI runs the Juju Charm Store on 
+The Juju GUI runs the Juju Charm Store on
 [jujucharms.com](http://jujucharms.com).  From there,  you can browse charms,
 try the GUI, and build an example environment to export for use elsewhere.
 
@@ -78,10 +78,29 @@
 
 ### Deploying behind a firewall ###
 
-While we may change this in the future to make firewall deploys simpler, this
-charm pulls the latest release from the Juju GUI page on Launchpad.  If you
-are behind a firewall, however, this may not be available to you.  In this case,
-you can configure the charm to pull the GUI release from a location you specify.
+When using the default options the charm uses the network connection only for
+installing Deb packages from the default Ubuntu repositories. For this reason
+the charm can be deployed behind a firewall in the usual way:
+
+    juju deploy juju-gui
+
+There are situations and customizations in which the charm needs to connect to
+Launchpad:
+
+- juju-gui-source is set to "stable" or "trunk": in this cases the charm pulls
+  the latest stable or development release from Launchpad;
+- juju-gui-source is set to a branch (e.g. "lp:juju-gui"): in this case the
+  charm retrieves a checkout of the specified branch from Launchpad, and adds
+  an external Launchpad PPA to install build dependencies;
+- juju-gui-source is set to a specific version number not available in the
+  local store (i.e. in the releases directory of the deployed charm): in this
+  case the release is downloaded from Launchpad;
+- builtin-server is set to false: in this case the charm adds an external
+  Launchpad PPA to install the legacy server dependencies.
+
+If, for any reason, you need to use the legacy server, it is still possible to
+deploy behind a firewall configuring the charm to pull the GUI release from a
+location you specify.
 
 For both Juju Core and PyJuju, you must simply do the following steps.  Note
 that PyJuju must do these steps, plus another set described further below.
@@ -117,6 +136,18 @@
 
 3. Retry as described in the step 3 above (`juju resolved --retry juju-gui/0`).
 
+### Upgrading the charm behind a firewall ###
+
+When a new version of Juju GUI is released, the charm is updated to include the
+new release in the local releases repository. Assuming the new version is
+1.0.1, after upgrading the charm, it is possible to also upgrade to the newer
+Juju GUI release by running the following:
+
+    juju set juju-gui-source=1.0.1
+
+In this case the new version will be found in the local repository and
+therefore the charm will not attempt to connect to Launchpad.
+
 ### Deploying to a chosen machine ###
 
 The instructions above cause you to use a separate machine to work with the

=== modified file 'config.yaml'
--- config.yaml	2013-10-07 13:26:52 +0000
+++ config.yaml	2013-10-09 08:56:52 +0000
@@ -17,21 +17,29 @@
 options:
   juju-gui-source:
     description: |
-      Where to install Juju GUI from. Possible values are:
-      - 'stable' (default): the latest stable release will be deployed;
-      - 'trunk': the latest trunk release will be deployed;
-      - a stable version (e.g '0.1.0'): the specified stable version will be
-        deployed;
-      - a trunk version (e.g '0.1.0+build.1'): the specified trunk version
-        will be deployed;
-      - a Bazaar branch (e.g. 'lp:juju-gui'): a release will be created and
-        deployed from the specified Bazaar branch. 'http://' prefixed branches
+      Where to install Juju GUI from. Possible values the following.
+      - 'local' (default): The latest local release will be deployed. Releases
+        are stored in the releases directory of this charm.
+      - 'stable': The latest release from the "stable" series will be deployed.
+      - 'trunk': The latest release from the "trunk" series will be deployed.
+        Please note that this is not a build of the current Juju GUI trunk.
+        For that functionality, use "lp:juju-gui", as described below.
+      - a stable version (e.g '0.1.0'): The specified stable version will be
+        deployed. A suitable release is looked up in the local releases
+        repository (see the "local" choice above). If not found locally, the
+        release will be downloaded from Launchpad.
+      - a trunk version (e.g '0.1.0+build.1'): The specified trunk version
+        will be deployed. A suitable release is looked up in the local releases
+        repository (see the "local" choice above). If not found locally, the
+        release will be downloaded from Launchpad.
+      - a Bazaar branch (e.g. 'lp:juju-gui'): A release will be created and
+        deployed from the specified Bazaar branch. "http://"" prefixed branches
         work as well. It is also possible to include the specific branch
-        revision, e.g. 'lp:juju-gui:42' will checkout revno 42.
-      - a 'url:' prefixed url (ex: url:http://...) of a specific location
-        to pull a release from.
+        revision, e.g. "lp:juju-gui:42" will checkout revno 42.
+      - a "url:"" prefixed url: The release found at the given URL
+        (ex: url:http://... or url:file://...) will be deployed.
     type: string
-    default: stable
+    default: local
   juju-gui-debug:
     description: |
       Run Juju GUI in debug mode, serving the uncompressed GUI source files.

=== modified file 'hooks/utils.py'
--- hooks/utils.py	2013-10-07 14:17:11 +0000
+++ hooks/utils.py	2013-10-09 08:56:52 +0000
@@ -27,14 +27,16 @@
     'WEB_PORT',
     'cmd_log',
     'compute_build_dir',
+    'download_release',
     'fetch_api',
     'fetch_gui_from_branch',
     'fetch_gui_release',
     'find_missing_packages',
     'first_path_in_dir',
     'get_api_address',
+    'get_launchpad_release',
     'get_npm_cache_archive_url',
-    'get_release_file_url',
+    'get_release_file_path',
     'get_zookeeper_address',
     'install_missing_packages',
     'legacy_juju',
@@ -60,6 +62,7 @@
 ]
 
 from contextlib import contextmanager
+from distutils.version import LooseVersion
 import errno
 import json
 import os
@@ -68,7 +71,7 @@
 import shutil
 from subprocess import CalledProcessError
 import tempfile
-from urlparse import urlparse
+import urlparse
 
 import apt
 import tempita
@@ -110,6 +113,7 @@
 CONFIG_DIR = os.path.join(CURRENT_DIR, 'config')
 JUJU_AGENT_DIR = os.path.join(BASE_DIR, 'juju')
 JUJU_GUI_DIR = os.path.join(BASE_DIR, 'juju-gui')
+RELEASES_DIR = os.path.join(CURRENT_DIR, 'releases')
 # Builtin server dependencies. The order of these requirements is important.
 SERVER_DEPENDENCIES = (
     'futures-2.1.4.tar.gz',
@@ -153,6 +157,14 @@
     (?::(\d+))?  # Optional branch revision.
     $  # End of line.
 """, re.VERBOSE)
+release_expression = re.compile(r"""
+    juju-gui-  # Juju GUI prefix.
+    (
+        \d+\.\d+\.\d+  # Major, minor, and patch version numbers.
+        (?:\+build\.\d+)?  # Optional bzr revno for development releases.
+    )
+    \.tgz  # File extension.
+""", re.VERBOSE)
 
 results_log = None
 
@@ -200,14 +212,15 @@
             return item
 
 
-def get_release_file_url(project, series_name, release_version):
-    """Return the URL of the release file hosted in Launchpad.
+def get_launchpad_release(project, series_name, release_version):
+    """Return the URL and the name of the release file hosted in Launchpad.
 
     The returned URL points to a release file for the given project, series
     name and release version.
     The argument *project* is a project object as returned by launchpadlib.
     The arguments *series_name* and *release_version* are strings. If
-    *release_version* is None, the URL of the latest release will be returned.
+    *release_version* is None, the URL and file name of the latest release will
+    be returned.
     """
     series = _get_by_attr(project.series, 'name', series_name)
     if series is None:
@@ -223,8 +236,10 @@
         releases = [release]
     for release in releases:
         for file_ in release.files:
-            if str(file_).endswith('.tgz'):
-                return file_.file_link
+            file_url = str(file_)
+            if file_url.endswith('.tgz'):
+                filename = os.path.split(urlparse.urlsplit(file_url).path)[1]
+                return file_.file_link, filename
     raise ValueError('%r: file not found' % release_version)
 
 
@@ -264,6 +279,7 @@
 
     Return a tuple of two elements representing info on how to deploy Juju GUI.
     Examples:
+       - ('local', None): latest local release;
        - ('stable', None): latest stable release;
        - ('stable', '0.1.0'): stable release v0.1.0;
        - ('trunk', None): latest trunk release;
@@ -276,12 +292,12 @@
     if source.startswith('url:'):
         source = source[4:]
         # Support file paths, including relative paths.
-        if urlparse(source).scheme == '':
+        if urlparse.urlparse(source).scheme == '':
             if not source.startswith('/'):
                 source = os.path.join(os.path.abspath(CURRENT_DIR), source)
             source = "file://%s" % source
         return 'url', source
-    if source in ('stable', 'trunk'):
+    if source in ('local', 'stable', 'trunk'):
         return source, None
     match = bzr_url_expression.match(source)
     if match is not None:
@@ -596,7 +612,7 @@
     launchpad = Launchpad.login_anonymously('Juju GUI charm', 'production')
     project = launchpad.projects['juju-gui']
     # Find the URL of the most recently created NPM cache archive.
-    npm_cache_url = get_release_file_url(project, 'npm-cache', None)
+    npm_cache_url, _ = get_launchpad_release(project, 'npm-cache', None)
     return npm_cache_url
 
 
@@ -641,21 +657,74 @@
         os.path.join(juju_gui_source_dir, 'releases'))
 
 
+def download_release(url, filename):
+    """Download a Juju GUI release from the given URL.
+
+    Save the resulting file as filename in the local releases repository.
+    Return the full path of the saved file.
+    """
+    destination = os.path.join(RELEASES_DIR, filename)
+    log('Downloading release file: {} --> {}.'.format(url, destination))
+    cmd_log(run('curl', '-L', '-o', destination, url))
+    return destination
+
+
+def get_release_file_path(version=None):
+    """Return the local path of the release file with the given version.
+
+    If version is None, return the path of the last release.
+    Raise a ValueError if no releases are found in the local repository.
+    """
+    version_path_map = {}
+    # Collect the locally stored releases.
+    for filename in os.listdir(RELEASES_DIR):
+        match = release_expression.match(filename)
+        if match is not None:
+            release_version = match.groups()[0]
+            release_path = os.path.join(RELEASES_DIR, filename)
+            version_path_map[release_version] = release_path
+    # We expect the charm to include at least one release file.
+    if not version_path_map:
+        raise ValueError('Error: no releases found in the charm.')
+    if version is None:
+        # Return the path of the last release.
+        last_version = sorted(version_path_map.keys(), key=LooseVersion)[-1]
+        return version_path_map[last_version]
+    # Return the path of the release with the requested version, or None if
+    # the release is not found.
+    return version_path_map.get(version)
+
+
 def fetch_gui_release(origin, version):
-    """Retrieve a Juju GUI release."""
+    """Retrieve a Juju GUI release. Return the release tarball local path.
+
+    The release file can be retrieved from:
+      - an arbitrary URL (if origin is "url");
+      - the local releases repository (if origin is "local" or if a release
+        version is specified and the corresponding file is present locally);
+      - Launchpad (in all the other cases).
+    """
     log('Retrieving Juju GUI release.')
     if origin == 'url':
-        file_url = version
-    else:
-        # Retrieve a release from Launchpad.
-        launchpad = Launchpad.login_anonymously(
-            'Juju GUI charm', 'production')
-        project = launchpad.projects['juju-gui']
-        file_url = get_release_file_url(project, origin, version)
-    log('Downloading release file from %s.' % file_url)
-    release_tarball = os.path.join(CURRENT_DIR, 'release.tgz')
-    cmd_log(run('curl', '-L', '-o', release_tarball, file_url))
-    return release_tarball
+        return download_release(version, 'url-release.tgz')
+    if origin == 'local':
+        path = get_release_file_path()
+        log('Using a local release: {}'.format(path))
+        return path
+    # Handle "stable" and "trunk" origins.
+    if version is not None:
+        # If the user specified a version, before attempting to download the
+        # requested release from Launchpad, check if that version is already
+        # stored locally.
+        path = get_release_file_path(version)
+        if path is not None:
+            log('Using a local release: {}'.format(path))
+            return path
+    # Retrieve a release from Launchpad.
+    launchpad = Launchpad.login_anonymously('Juju GUI charm', 'production')
+    project = launchpad.projects['juju-gui']
+    url, filename = get_launchpad_release(project, origin, version)
+    return download_release(url, filename)
 
 
 def fetch_api(juju_api_branch):

=== added directory 'releases'
=== added file 'releases/juju-gui-0.10.1.tgz'
Binary files releases/juju-gui-0.10.1.tgz	1970-01-01 00:00:00 +0000 and releases/juju-gui-0.10.1.tgz	2013-10-09 08:56:52 +0000 differ
=== modified file 'revision'
--- revision	2013-10-07 14:18:05 +0000
+++ revision	2013-10-09 08:56:52 +0000
@@ -1,1 +1,1 @@
-89
+90

=== modified file 'tests/20-functional.test'
--- tests/20-functional.test	2013-10-07 16:17:20 +0000
+++ tests/20-functional.test	2013-10-09 08:56:52 +0000
@@ -168,14 +168,25 @@
     def tearDown(self):
         juju_destroy_service(self.charm)
 
-    def test_api_agent(self):
-        # Ensure the Juju GUI and API agent services are correctly set up.
+    def test_local_release(self):
+        # Ensure the Juju GUI and API agent services are correctly set up when
+        # deploying the local release.
         unit_info = self.juju_deploy(self.charm)
         hostname = unit_info['public-address']
         self.navigate_to(hostname)
         self.handle_browser_warning()
         self.assertEnvironmentIsConnected()
 
+    def test_stable_release(self):
+        # Ensure the Juju GUI and API agent services are correctly set up when
+        # deploying the stable release.
+        options = {'juju-gui-source': 'stable'}
+        unit_info = self.juju_deploy(self.charm, options=options)
+        hostname = unit_info['public-address']
+        self.navigate_to(hostname)
+        self.handle_browser_warning()
+        self.assertEnvironmentIsConnected()
+
     @unittest.skipUnless(is_legacy_juju, 'staging only works in pyJuju')
     def test_staging(self):
         # Ensure the Juju GUI and improv services are correctly set up.
@@ -189,8 +200,7 @@
 
     def test_sandbox(self):
         # The GUI is correctly deployed and set up in sandbox mode.
-        unit_info = self.juju_deploy(
-            self.charm, options={'builtin-server': 'true', 'sandbox': 'true'})
+        unit_info = self.juju_deploy(self.charm, options={'sandbox': 'true'})
         hostname = unit_info['public-address']
         self.navigate_to(hostname)
         self.handle_browser_warning()
@@ -198,15 +208,16 @@
 
     def test_branch_source(self):
         # Ensure the Juju GUI is correctly deployed from a Bazaar branch.
-        unit_info = self.juju_deploy(
-            self.charm, options={'juju-gui-source': JUJU_GUI_TEST_BRANCH})
+        options = {'juju-gui-source': JUJU_GUI_TEST_BRANCH}
+        unit_info = self.juju_deploy(self.charm, options=options)
         hostname = unit_info['public-address']
         self.navigate_to(hostname)
         self.handle_browser_warning()
         self.assertEnvironmentIsConnected()
 
-    def test_cache_headers(self):
-        # Make sure the correct cache headers are sent.
+    def test_legacy_server(self):
+        # The legacy apache + haproxy server configuration works correctly.
+        # Also make sure the correct cache headers are sent.
         options = {
             'builtin-server': False,
             'juju-gui-source': JUJU_GUI_TEST_BRANCH,
@@ -236,8 +247,8 @@
 
     def test_nrpe_check_available(self):
         # Make sure the check-app-access.sh script's ADDRESS is available.
-        unit_info = self.juju_deploy(
-            self.charm, options={'juju-gui-source': JUJU_GUI_TEST_BRANCH})
+        options = {'juju-gui-source': JUJU_GUI_TEST_BRANCH}
+        unit_info = self.juju_deploy(self.charm, options=options)
         hostname = unit_info['public-address']
         conn = httplib.HTTPSConnection(hostname)
         # This request matches the ADDRESS var in the script.
@@ -252,7 +263,7 @@
     def setUpClass(cls):
         # Deploy the charm. The resulting service is used by all the tests
         # in this test case.
-        unit_info = juju_deploy(cls.charm, options={'builtin-server': 'true'})
+        unit_info = juju_deploy(cls.charm)
         cls.hostname = unit_info['public-address']
         # The counter is used to produce API request identifiers.
         cls.counter = itertools.count()

=== modified file 'tests/test_utils.py'
--- tests/test_utils.py	2013-10-07 14:17:11 +0000
+++ tests/test_utils.py	2013-10-09 08:56:52 +0000
@@ -39,16 +39,19 @@
     _get_by_attr,
     cmd_log,
     compute_build_dir,
+    download_release,
+    fetch_gui_release,
     first_path_in_dir,
     get_api_address,
-    get_release_file_url,
+    get_launchpad_release,
+    get_npm_cache_archive_url,
+    get_release_file_path,
     get_zookeeper_address,
+    install_builtin_server,
+    install_missing_packages,
     legacy_juju,
     log_hook,
     parse_source,
-    get_npm_cache_archive_url,
-    install_builtin_server,
-    install_missing_packages,
     remove_apache_setup,
     remove_haproxy_setup,
     render_to_file,
@@ -94,6 +97,108 @@
             AttrDict().myattr
 
 
[e-mail]('utils.run')
[e-mail]('utils.log')
[e-mail]('utils.cmd_log', mock.Mock())
+class TestDownloadRelease(unittest.TestCase):
+
+    def test_download(self, mock_log, mock_run):
+        # A release is properly downloaded using curl.
+        url = 'http://download.example.com/release.tgz'
+        filename = 'local-release.tgz'
+        destination = download_release(url, filename)
+        expected_destination = os.path.join(os.getcwd(), 'releases', filename)
+        self.assertEqual(expected_destination, destination)
+        expected_log = 'Downloading release file: {} --> {}.'.format(
+            url, expected_destination)
+        mock_log.assert_called_once_with(expected_log)
+        mock_run.assert_called_once_with(
+            'curl', '-L', '-o', expected_destination, url)
+
+
[e-mail]('utils.log', mock.Mock())
+class TestFetchGuiRelease(unittest.TestCase):
+
+    release_path = '/my/release.tgz'
+
+    @contextmanager
+    def patch_launchpad(self, origin, version):
+        """Mock the functions used to download a release from Launchpad.
+
+        Ensure all the functions are called correctly.
+        """
+        url = 'http://launchpad.example.com/release.tgz/file'
+        filename = 'release.tgz'
+        patch_launchpad = mock.patch('utils.Launchpad')
+        patch_get_launchpad_release = mock.patch(
+            'utils.get_launchpad_release',
+            mock.Mock(return_value=(url, filename)),
+        )
+        patch_download_release = mock.patch(
+            'utils.download_release',
+            mock.Mock(return_value=self.release_path),
+        )
+        with patch_launchpad as mock_launchpad:
+            with patch_get_launchpad_release as mock_get_launchpad_release:
+                with patch_download_release as mock_download_release:
+                    yield
+        login = mock_launchpad.login_anonymously
+        login.assert_called_once_with('Juju GUI charm', 'production')
+        mock_get_launchpad_release.assert_called_once_with(
+            login().projects['juju-gui'], origin, version)
+        mock_download_release.assert_called_once_with(url, filename)
+
+    @mock.patch('utils.download_release')
+    def test_url(self, mock_download_release):
+        # The release is retrieved from an URL.
+        mock_download_release.return_value = self.release_path
+        url = 'http://download.example.com/release.tgz'
+        path = fetch_gui_release('url', url)
+        self.assertEqual(self.release_path, path)
+        mock_download_release.assert_called_once_with(url, 'url-release.tgz')
+
+    @mock.patch('utils.get_release_file_path')
+    def test_local(self, mock_get_release_file_path):
+        # The last local release is requested.
+        mock_get_release_file_path.return_value = self.release_path
+        path = fetch_gui_release('local', None)
+        self.assertEqual(self.release_path, path)
+        mock_get_release_file_path.assert_called_once_with()
+
+    @mock.patch('utils.get_release_file_path')
+    def test_version_found(self, mock_get_release_file_path):
+        # A release version is specified and found locally.
+        mock_get_release_file_path.return_value = self.release_path
+        path = fetch_gui_release('stable', '0.1.42')
+        self.assertEqual(self.release_path, path)
+        mock_get_release_file_path.assert_called_once_with('0.1.42')
+
+    @mock.patch('utils.get_release_file_path')
+    def test_version_not_found(self, mock_get_release_file_path):
+         # A release version is specified but not found locally.
+        mock_get_release_file_path.return_value = None
+        with self.patch_launchpad('stable', '0.1.42'):
+            path = fetch_gui_release('stable', '0.1.42')
+        self.assertEqual(self.release_path, path)
+        mock_get_release_file_path.assert_called_once_with('0.1.42')
+
+    @mock.patch('utils.get_release_file_path')
+    def test_stable(self, mock_get_release_file_path):
+        # The last stable release is requested.
+        with self.patch_launchpad('stable', None):
+            path = fetch_gui_release('stable', None)
+        self.assertEqual(self.release_path, path)
+        self.assertFalse(mock_get_release_file_path.called)
+
+    @mock.patch('utils.get_release_file_path')
+    def test_trunk(self, mock_get_release_file_path):
+        # The last development release is requested.
+        with self.patch_launchpad('trunk', None):
+            path = fetch_gui_release('trunk', None)
+        self.assertEqual(self.release_path, path)
+        self.assertFalse(mock_get_release_file_path.called)
+
+
 class TestFirstPathInDir(unittest.TestCase):
 
     def setUp(self):
@@ -184,6 +289,105 @@
             self.assertRaises(IOError, get_api_address, unit_dir)
 
 
+class TestGetReleaseFilePath(unittest.TestCase):
+
+    def setUp(self):
+        self.playground = tempfile.mkdtemp()
+        self.addCleanup(shutil.rmtree, self.playground)
+
+    def mock_releases_dir(self):
+        """Mock the releases directory."""
+        return mock.patch('utils.RELEASES_DIR', self.playground)
+
+    def assert_path(self, filename, path):
+        """Ensure the absolute path of filename equals the given path."""
+        expected = os.path.join(self.playground, filename)
+        self.assertEqual(expected, path)
+
+    @contextmanager
+    def assert_error(self):
+        """Ensure the code executed in the context block raises a ValueError.
+
+        Also check the error message.
+        """
+        with self.assertRaises(ValueError) as context_manager:
+            yield
+        error = str(context_manager.exception)
+        self.assertEqual('Error: no releases found in the charm.', error)
+
+    def add(self, filename):
+        """Create a release file in the playground directory."""
+        path = os.path.join(self.playground, filename)
+        open(path, 'w').close()
+
+    def test_last_release(self):
+        # The last release is correctly retrieved.
+        self.add('juju-gui-0.12.1.tgz')
+        self.add('juju-gui-1.2.3.tgz')
+        self.add('juju-gui-2.0.0+build.42.tgz')
+        self.add('juju-gui-2.0.1.tgz')
+        with self.mock_releases_dir():
+            path = get_release_file_path()
+        self.assert_path('juju-gui-2.0.1.tgz', path)
+
+    def test_ordering(self):
+        # Release versions are correctly ordered.
+        self.add('juju-gui-0.12.1.tgz')
+        self.add('juju-gui-0.9.1.tgz')
+        with self.mock_releases_dir():
+            path = get_release_file_path()
+        self.assert_path('juju-gui-0.12.1.tgz', path)
+
+    def test_no_releases(self):
+        # A ValueError is raised if no releases are found.
+        with self.mock_releases_dir():
+            with self.assert_error():
+                get_release_file_path()
+
+    def test_no_releases_with_files(self):
+        # A ValueError is raised if no releases are found.
+        # Extraneous files are ignored while looking for releases.
+        self.add('jujugui-1.2.3.tgz')  # Wrong prefix.
+        self.add('juju-gui-1.2.tgz')  # Missing patch version number.
+        self.add('juju-gui-1.2.3.bz2')  # Wrong file extension.
+        self.add('juju-gui-1.2.3.4.tgz')  # Wrong version.
+        self.add('juju-gui-1.2.3.build.42.tgz')  # Missing "+" separator.
+        self.add('juju-gui-1.2.3+built.42.tgz')  # Typo.
+        self.add('juju-gui-1.2.3+build.42.47.tgz')  # Invalid bzr revno.
+        self.add('juju-gui-1.2.3+build.42.bz2')  # Wrong file extension again.
+        with self.mock_releases_dir():
+            with self.assert_error():
+                print get_release_file_path()
+
+    def test_stable_version(self):
+        # A specific stable version is correctly retrieved.
+        self.add('juju-gui-1.2.3.tgz')
+        self.add('juju-gui-2.0.1+build.42.tgz')
+        self.add('juju-gui-2.0.1.tgz')
+        self.add('juju-gui-3.2.1.tgz')
+        with self.mock_releases_dir():
+            path = get_release_file_path('2.0.1')
+        self.assert_path('juju-gui-2.0.1.tgz', path)
+
+    def test_development_version(self):
+        # A specific development version is correctly retrieved.
+        self.add('juju-gui-1.2.3+build.4247.tgz')
+        self.add('juju-gui-2.42.47+build.4247.tgz')
+        self.add('juju-gui-2.42.47.tgz')
+        self.add('juju-gui-3.42.47+build.4247.tgz')
+        with self.mock_releases_dir():
+            path = get_release_file_path('2.42.47+build.4247')
+        self.assert_path('juju-gui-2.42.47+build.4247.tgz', path)
+
+    def test_version_not_found(self):
+        # None is returned if the requested version is not found.
+        self.add('juju-gui-1.2.3.tgz')
+        self.add('juju-GUI-1.42.47.tgz')  # This is not a valid release.
+        with self.mock_releases_dir():
+            path = get_release_file_path('1.42.47')
+        self.assertIsNone(path)
+
+
 class TestLegacyJuju(unittest.TestCase):
 
     def setUp(self):
@@ -261,7 +465,7 @@
         return self.file_link
 
 
-class TestGetReleaseFileUrl(unittest.TestCase):
+class TestGetLaunchpadRelease(unittest.TestCase):
 
     project = AttrDict(
         series=(
@@ -308,43 +512,48 @@
 
     def test_latest_stable_release(self):
         # Ensure the correct URL is returned for the latest stable release.
-        url = get_release_file_url(self.project, 'stable', None)
+        url, name = get_launchpad_release(self.project, 'stable', None)
         self.assertEqual('http://example.com/0.1.1.tgz', url)
+        self.assertEqual('0.1.1.tgz', name)
 
     def test_latest_trunk_release(self):
         # Ensure the correct URL is returned for the latest trunk release.
-        url = get_release_file_url(self.project, 'trunk', None)
+        url, name = get_launchpad_release(self.project, 'trunk', None)
         self.assertEqual('http://example.com/0.1.1+build.1.tgz', url)
+        self.assertEqual('0.1.1+build.1.tgz', name)
 
     def test_specific_stable_release(self):
         # Ensure the correct URL is returned for a specific version of the
         # stable release.
-        url = get_release_file_url(self.project, 'stable', '0.1.0')
+        url, name = get_launchpad_release(self.project, 'stable', '0.1.0')
         self.assertEqual('http://example.com/0.1.0.tgz', url)
+        self.assertEqual('0.1.0.tgz', name)
 
     def test_specific_trunk_release(self):
         # Ensure the correct URL is returned for a specific version of the
         # trunk release.
-        url = get_release_file_url(self.project, 'trunk', '0.1.0+build.1')
+        url, name = get_launchpad_release(
+            self.project, 'trunk', '0.1.0+build.1')
         self.assertEqual('http://example.com/0.1.0+build.1.tgz', url)
+        self.assertEqual('0.1.0+build.1.tgz', name)
 
     def test_series_not_found(self):
         # A ValueError is raised if the series cannot be found.
         with self.assertRaises(ValueError) as cm:
-            get_release_file_url(self.project, 'unstable', None)
+            get_launchpad_release(self.project, 'unstable', None)
         self.assertIn('series not found', str(cm.exception))
 
     def test_no_releases(self):
         # A ValueError is raised if the series does not contain releases.
         project = AttrDict(series=[AttrDict(name='stable', releases=[])])
         with self.assertRaises(ValueError) as cm:
-            get_release_file_url(project, 'stable', None)
+            get_launchpad_release(project, 'stable', None)
         self.assertIn('series does not contain releases', str(cm.exception))
 
     def test_release_not_found(self):
         # A ValueError is raised if the release cannot be found.
         with self.assertRaises(ValueError) as cm:
-            get_release_file_url(self.project, 'stable', '2.0')
+            get_launchpad_release(self.project, 'stable', '2.0')
         self.assertIn('release not found', str(cm.exception))
 
     def test_file_not_found(self):
@@ -358,7 +567,7 @@
             ],
         )
         with self.assertRaises(ValueError) as cm:
-            get_release_file_url(project, 'stable', None)
+            get_launchpad_release(project, 'stable', None)
         self.assertIn('file not found', str(cm.exception))
 
     def test_file_not_found_in_latest_release(self):
@@ -378,8 +587,9 @@
                 ),
             ],
         )
-        url = get_release_file_url(project, 'stable', None)
+        url, name = get_launchpad_release(project, 'stable', None)
         self.assertEqual('http://example.com/0.1.0.tgz', url)
+        self.assertEqual('0.1.0.tgz', name)
 
 
 class TestGetZookeeperAddress(unittest.TestCase):
@@ -452,6 +662,11 @@
         # Restore the original utils.CURRENT_DIR.
         utils.CURRENT_DIR = self.original_current_dir
 
+    def test_latest_local_release(self):
+        # Ensure the latest local release is correctly parsed.
+        expected = ('local', None)
+        self.assertTupleEqual(expected, parse_source('local'))
+
     def test_latest_stable_release(self):
         # Ensure the latest stable release is correctly parsed.
         expected = ('stable', None)