Merge lp:~frankban/juju-quickstart/old-style-bundles-regression into lp:juju-quickstart

[Francesco Banconi]

Fix the old-style bundle regression.

This branch fixes
https://bugs.launchpad.net/juju-quickstart/+bug/1429129
In essence, legacy bundles are converted by the
ingestion process like the following:
- if a basked includes multiple bundles, the resulting
  v4 bundle name is {basket-name}-{bundle-name} for
  each {bundle-name};
- if a basket only includes one bundle, the v4 bundle
  is just {basket-name}.
Previously quickstart always assumed the former: this
branch adds a check for the latter before exiting with
an error.

This branch also introduces a charmstore module in
quickstart. All the interactions between quickstart
and the charm store are now collected in this module.

As part of this refactoring, quickstart is now able
to distinguish HTTP 404 errors and all the other
generic IOErrors that can be raised when connecting
to the network.

Also simplified the logging format and bootstrap logging
earlier in the application execution.

Tests: `make check`.

QA:
install bundles with quickstart:
`devenv/bin/juju-quickstart {bundle}`
Try the following bundles:
- devenv/bin/juju-quickstart mediawiki-single
- devenv/bin/juju-quickstart u/bigdata-dev/apache-analytics-sql
- devenv/bin/juju-quickstart bundle:mediawiki/scalable
- devenv/bin/juju-quickstart bundle:~landscape/landscape-dense-maas/landscape-dense-maas
- devenv/bin/juju-quickstart bundle:django/example-single

Those instead should return errors:
- devenv/bin/juju-quickstart mediawiki/trusty
- devenv/bin/juju-quickstart mediawiki-nosuch
- devenv/bin/juju-quickstart no-such
- devenv/bin/juju-quickstart bundle:no/such
- devenv/bin/juju-quickstart bundle:invalid
- devenv/bin/juju-quickstart bundle:~landscape/landscape-dense-maas/landscape

https://codereview.appspot.com/215070043/

[Francesco Banconi (frankban)]

Reviewers: mp+252353_code.launchpad.net,

Message:
Please take a look.

Description:
Fix the old-style bundle regression.

This branch fixes
https://bugs.launchpad.net/juju-quickstart/+bug/1429129
In essence, legacy bundles are converted by the
ingestion process like the following:
- if a basked includes multiple bundles, the resulting
   v4 bundle name is {basket-name}-{bundle-name} for
   each {bundle-name};
- if a basket only includes one bundle, the v4 bundle
   is just {basket-name}.
Previously quickstart always assumed the former: this
branch adds a check for the latter before exiting with
an error.

This branch also introduces a charmstore module in
quickstart. All the interactions between quickstart
and the charm store are now collected in this module.

As part of this refactoring, quickstart is now able
to distinguish HTTP 404 errors and all the other
generic IOErrors that can be raised when connecting
to the network.

Also simplified the logging format and bootstrap logging
earlier in the application execution.

Tests: `make check`.

QA:
install bundles with quickstart:
`devenv/bin/juju-quickstart {bundle}`
Try the following bundles:
- devenv/bin/juju-quickstart mediawiki-single
- devenv/bin/juju-quickstart u/bigdata-dev/apache-analytics-sql
- devenv/bin/juju-quickstart bundle:mediawiki/scalable
- devenv/bin/juju-quickstart
bundle:~landscape/landscape-dense-maas/landscape-dense-maas
- devenv/bin/juju-quickstart bundle:django/example-single

Those instead should return errors:
- devenv/bin/juju-quickstart mediawiki/trusty
- devenv/bin/juju-quickstart mediawiki-nosuch
- devenv/bin/juju-quickstart no-such
- devenv/bin/juju-quickstart bundle:no/such
- devenv/bin/juju-quickstart bundle:invalid
- devenv/bin/juju-quickstart
bundle:~landscape/landscape-dense-maas/landscape

https://code.launchpad.net/~frankban/juju-quickstart/old-style-bundles-regression/+merge/252353

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/215070043/

Affected files (+856, -266 lines):
   A [revision details]
   M quickstart/__init__.py
   M quickstart/app.py
   A quickstart/charmstore.py
   M quickstart/manage.py
   M quickstart/models/bundles.py
   M quickstart/models/references.py
   M quickstart/netutils.py
   M quickstart/settings.py
   M quickstart/tests/helpers.py
   M quickstart/tests/models/test_bundles.py
   M quickstart/tests/models/test_references.py
   M quickstart/tests/test_app.py
   A quickstart/tests/test_charmstore.py
   M quickstart/tests/test_netutils.py

[Jeff Pihach (hatch)]

LTGM QA OK

https://codereview.appspot.com/215070043/

[Jeff Pihach (hatch)]

Thanks a lot for this fix. really cleans up a lot of the code!

https://codereview.appspot.com/215070043/

[Richard Harding (rharding)]

Thanks for this branch Francesco, I've mainly got one big question on
the legacy version of the call and some minor questions on just code
bits.

Will make sure to look asap in the morning. Let me know if anything here
is unclear.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py
File quickstart/charmstore.py (right):

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode30
quickstart/charmstore.py:30: class NotFoundError(Exception):
can this just use the netutils NotFoundError?

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode34
quickstart/charmstore.py:34: def get(path):
are these internal? Should they be _get prefixed or anything as internal
use methods?

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode45
quickstart/charmstore.py:45: url = settings.CHARMSTORE_API +
path.lstrip('/')
can you confirm that the settings will allow overriding the api endpoint
via env var? Is that documented, maybe in the hacking doc?

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode51
quickstart/charmstore.py:51: raise NotFoundError(msg)
Ok, so this is intentional to tell which kind of "NotFoundError" you've
got by if it's a netutils vs a charmstore.py one? Should we name them
different or just use the traceback?

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode64
quickstart/charmstore.py:64: For instance, to retrieve the hash of a
charm reference, use the following:
here you mention a charm reference but this is particular to bundles.
Should this be bundle reference?

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode78
quickstart/charmstore.py:78: def resolve(name, series=None):
what about specifying the version? The main concern is that if it
doesn't exist that the failure is consistent or clear to the user.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode116
quickstart/charmstore.py:116: def get_legacy_bundle_data(reference):
do we ever get the legacy bundle though now? I thought we only got the
updated bundle and then turned it into legacy format by nesting it
inside another level?

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode134
quickstart/charmstore.py:134: def _retrieve_bundle_data(reference,
path):
get_bundle_data vs _retrieve_bundle_data was a bit unclear to me. Maybe
we can get the word 'parse' into the function name to help clarify that
one fetches while the other actually parses?

https://codereview.appspot.com/215070043/diff/1/quickstart/models/bundles.py
File quickstart/models/bundles.py (right):

https://codereview.appspot.com/215070043/diff/1/quickstart/models/bundles.py#newcode228
quickstart/models/bundles.py:228: data =
charmstore.get_legacy_bundle_data(reference)
I was expecting this to use the same 'get_bundle_data but with the
different reference using only the basket vs getting the orig.yaml file
using the get_legacy_bundle_data. Can you clarify this for me please?

https://codereview.appspot.com/215070043/diff/1/quickstart/models/bundles.py#newcode400
quickstart/mo...

[Brad Crittenden (bac)]

LGTM. Thanks for the branch. Did not QA.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py
File quickstart/charmstore.py (right):

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode68
quickstart/charmstore.py:68: Raise a NotFoundError the an entity with
the given reference cannot be
typo: the an entity

https://codereview.appspot.com/215070043/diff/1/quickstart/models/bundles.py
File quickstart/models/bundles.py (right):

https://codereview.appspot.com/215070043/diff/1/quickstart/models/bundles.py#newcode187
quickstart/models/bundles.py:187:
Thank you for the super-clear regex construction.

https://codereview.appspot.com/215070043/

[lp:~frankban/juju-quickstart/old-style-bundles-regression]

130. By Francesco Banconi on 2015-03-10

Changes as per review.

[Francesco Banconi (frankban)]

Please take a look.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py
File quickstart/charmstore.py (right):

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode30
quickstart/charmstore.py:30: class NotFoundError(Exception):
On 2015/03/10 01:18:32, rharding wrote:
> can this just use the netutils NotFoundError?

It can. But I didn't implement it lije that on purpose.
My idea is that having the charmstore raise a charmstore.NotFoundError
is part of the contract. Instead, exposing its internal dependency on
netutils.urlread is not. This is why we basically mask the underlying
exception.
With this implementation, we discourage people to write something like:

try:
    charmstore.get(...)
except netutils.NotFoundError: # This currently does not work.
    ...

The code above would establish a connection between the charmstore and
the netutils code, which is only an implementation detail. If, for
instance in the future we decide to replace the simplistic code in
netutils with something like requests or similar, the current API
doesn't have to change, i.e. the charmstore would still raise a
charmstore.NotFoundError.
In essence, clients using the charmstore API doesn't have to know (or
import) netutils, and vice versa.

I considered the module namespace to be good enough to distinguish
between netutils.NotFoundError and charmstore.NotFoundError.

That is the reasoning behind this implementation, and I agree it can
seem a repetition, but it's separation of concerns in my idea. Of course
feel free to disagree with the above.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode34
quickstart/charmstore.py:34: def get(path):
On 2015/03/10 01:18:32, rharding wrote:
> are these internal? Should they be _get prefixed or anything as
internal use
> methods?

The idea is that this is public API. A client can use this function when
no higher level calls are available.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode45
quickstart/charmstore.py:45: url = settings.CHARMSTORE_API +
path.lstrip('/')
On 2015/03/10 01:18:32, rharding wrote:
> can you confirm that the settings will allow overriding the api
endpoint via env
> var? Is that documented, maybe in the hacking doc?

Overriding the charmstore API URL is not supported currently. Do we need
to support that? If so, I'll create a card.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode51
quickstart/charmstore.py:51: raise NotFoundError(msg)
On 2015/03/10 01:18:32, rharding wrote:
> Ok, so this is intentional to tell which kind of "NotFoundError"
you've got by
> if it's a netutils vs a charmstore.py one? Should we name them
different or just
> use the traceback?

Not sure if I understand the question about using the traceback. See
above for the goal of having two separate exception types.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode64
quickstart/charmstore.py:64: For instance, to retrieve the hash of a
charm reference, use the following:
On 2015/03/10 01:18:32, rharding wrote:
> here you mention a charm reference but this is particula...

[Richard Harding (rharding)]

On 2015/03/10 10:20:07, frankban wrote:
> Please take a look.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py
> File quickstart/charmstore.py (right):

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode30
> quickstart/charmstore.py:30: class NotFoundError(Exception):
> On 2015/03/10 01:18:32, rharding wrote:
> > can this just use the netutils NotFoundError?

> It can. But I didn't implement it lije that on purpose.
> My idea is that having the charmstore raise a charmstore.NotFoundError
is part
> of the contract. Instead, exposing its internal dependency on
netutils.urlread
> is not. This is why we basically mask the underlying exception.
> With this implementation, we discourage people to write something
like:

> try:
> charmstore.get(...)
> except netutils.NotFoundError: # This currently does not work.
> ...

> The code above would establish a connection between the charmstore and
the
> netutils code, which is only an implementation detail. If, for
instance in the
> future we decide to replace the simplistic code in netutils with
something like
> requests or similar, the current API doesn't have to change, i.e. the
charmstore
> would still raise a charmstore.NotFoundError.
> In essence, clients using the charmstore API doesn't have to know (or
import)
> netutils, and vice versa.

> I considered the module namespace to be good enough to distinguish
between
> netutils.NotFoundError and charmstore.NotFoundError.

> That is the reasoning behind this implementation, and I agree it can
seem a
> repetition, but it's separation of concerns in my idea. Of course feel
free to
> disagree with the above.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode34
> quickstart/charmstore.py:34: def get(path):
> On 2015/03/10 01:18:32, rharding wrote:
> > are these internal? Should they be _get prefixed or anything as
internal use
> > methods?

> The idea is that this is public API. A client can use this function
when no
> higher level calls are available.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode45
> quickstart/charmstore.py:45: url = settings.CHARMSTORE_API +
path.lstrip('/')
> On 2015/03/10 01:18:32, rharding wrote:
> > can you confirm that the settings will allow overriding the api
endpoint via
> env
> > var? Is that documented, maybe in the hacking doc?

> Overriding the charmstore API URL is not supported currently. Do we
need to
> support that? If so, I'll create a card.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode51
> quickstart/charmstore.py:51: raise NotFoundError(msg)
> On 2015/03/10 01:18:32, rharding wrote:
> > Ok, so this is intentional to tell which kind of "NotFoundError"
you've got by
> > if it's a netutils vs a charmstore.py one? Should we name them
different or
> just
> > use the traceback?

> Not sure if I understand the question about using the traceback. See
above for
> the goal of having two separate exception types.

https://codereview.appspot.com/215070043/diff/1/quickstart/charmstore.py#newcode64
> quickstart/charmstore.py:64: For instance, to retrieve the has...

[Francesco Banconi (frankban)]

*** Submitted:

Fix the old-style bundle regression.

This branch fixes
https://bugs.launchpad.net/juju-quickstart/+bug/1429129
In essence, legacy bundles are converted by the
ingestion process like the following:
- if a basked includes multiple bundles, the resulting
   v4 bundle name is {basket-name}-{bundle-name} for
   each {bundle-name};
- if a basket only includes one bundle, the v4 bundle
   is just {basket-name}.
Previously quickstart always assumed the former: this
branch adds a check for the latter before exiting with
an error.

This branch also introduces a charmstore module in
quickstart. All the interactions between quickstart
and the charm store are now collected in this module.

As part of this refactoring, quickstart is now able
to distinguish HTTP 404 errors and all the other
generic IOErrors that can be raised when connecting
to the network.

Also simplified the logging format and bootstrap logging
earlier in the application execution.

Tests: `make check`.

QA:
install bundles with quickstart:
`devenv/bin/juju-quickstart {bundle}`
Try the following bundles:
- devenv/bin/juju-quickstart mediawiki-single
- devenv/bin/juju-quickstart u/bigdata-dev/apache-analytics-sql
- devenv/bin/juju-quickstart bundle:mediawiki/scalable
- devenv/bin/juju-quickstart
bundle:~landscape/landscape-dense-maas/landscape-dense-maas
- devenv/bin/juju-quickstart bundle:django/example-single

Those instead should return errors:
- devenv/bin/juju-quickstart mediawiki/trusty
- devenv/bin/juju-quickstart mediawiki-nosuch
- devenv/bin/juju-quickstart no-such
- devenv/bin/juju-quickstart bundle:no/such
- devenv/bin/juju-quickstart bundle:invalid
- devenv/bin/juju-quickstart
bundle:~landscape/landscape-dense-maas/landscape

R=jeff.pihach, rharding, bac
CC=
https://codereview.appspot.com/215070043

https://codereview.appspot.com/215070043/

[Francesco Banconi (frankban)]

Thanks for the reviews!

https://codereview.appspot.com/215070043/

=== modified file 'quickstart/__init__.py'
--- quickstart/__init__.py	2015-02-27 09:25:33 +0000
+++ quickstart/__init__.py	2015-03-10 10:19:09 +0000
@@ -45,7 +45,7 @@
 Once Juju has been installed, the command can also be run as a juju plugin,
 without the hyphen ("juju quickstart").
 """
-VERSION = (2, 0, 0)
+VERSION = (2, 0, 1)
 
 
 def get_version():

=== modified file 'quickstart/app.py'
--- quickstart/app.py	2015-02-26 11:02:57 +0000
+++ quickstart/app.py	2015-03-10 10:19:09 +0000
@@ -29,6 +29,7 @@
 import jujuclient
 
 from quickstart import (
+    charmstore,
     juju,
     jujutools,
     netutils,
@@ -440,8 +441,9 @@
                 series = settings.JUJU_GUI_SUPPORTED_SERIES[-1]
             try:
                 # Try to get the charm URL from the charm store API.
-                charm_url = netutils.get_charm_url(series)
-            except (IOError, ValueError) as err:
+                charm_url = charmstore.resolve(
+                    settings.JUJU_GUI_CHARM_NAME, series=series)
+            except (charmstore.NotFoundError, IOError, ValueError) as err:
                 # Fall back to the default URL for the current series.
                 msg = 'unable to retrieve the {} charm URL from the API: {}'
                 logging.warn(msg.format(service_name, err))

=== added file 'quickstart/charmstore.py'
--- quickstart/charmstore.py	1970-01-01 00:00:00 +0000
+++ quickstart/charmstore.py	2015-03-10 10:19:09 +0000
@@ -0,0 +1,161 @@
+# This file is part of the Juju Quickstart Plugin, which lets users set up a
+# Juju environment in very few steps (https://launchpad.net/juju-quickstart).
+# Copyright (C) 2015 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License version 3, as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Juju Quickstart utilities for communicating with the Juju charm store."""
+
+from __future__ import unicode_literals
+
+import json
+
+from quickstart import (
+    netutils,
+    serializers,
+    settings,
+)
+
+
+class NotFoundError(Exception):
+    """Represent a not found HTTP error while communicating with the store."""
+
+
+def get(path):
+    """Send a GET request to the charm store at the given path.
+
+    The path must not include the charm store API version identifier. In
+    essence, the path must exclude the "/v4/" fragment.
+
+    Return the string content returned by the charm store.
+    Raise a NotFoundError if a 404 not found response is returned.
+    Raise an IOError if any other problems occur while communicating with the
+    charm store.
+    """
+    url = settings.CHARMSTORE_API + path.lstrip('/')
+    try:
+        return netutils.urlread(url)
+    except netutils.NotFoundError as err:
+        msg = b'charm store resource not found at {}: {}'.format(
+            url.encode('utf-8'), err)
+        raise NotFoundError(msg)
+    except IOError as err:
+        msg = b'cannot communicate with the charm store at {}: {}'.format(
+            url.encode('utf-8'), err)
+        raise IOError(msg)
+
+
+def get_reference(reference, path):
+    """Retrieve the charm store contents for the given reference and path.
+
+    The reference argument identifies a charm or bundle entity and must be an
+    instance of "quickstart.models.references.Reference".
+
+    For instance, to retrieve the hash of a charm reference, use the following:
+
+        hash = get_reference(reference, '/meta/hash')
+
+    Raise a NotFoundError if an entity with the given reference cannot be
+    found in the charm store.
+    Raise an IOError if any other problems occur while communicating with the
+    charm store.
+    """
+    if not path.startswith('/'):
+        path = '/' + path
+    return get(reference.path() + path)
+
+
+def resolve(name, series=None):
+    """Return the fully qualified id of the entity with the given name.
+
+    If the optional series is provided, resolve the entity of the given series.
+
+    Raise a NotFoundError if an entity with the given name and optional series
+    cannot be found in the charm store.
+    Raise an IOError if any other problems occur while communicating with the
+    charm store.
+    Raise a ValueError if the API returns invalid data.
+    """
+    series_part = '' if series is None else '{}/'.format(series)
+    content = get(series_part + name + '/meta/id')
+    data = json.loads(content)
+    entity_id = data.get('Id')
+    if entity_id is None:
+        msg = 'unable to resolve entity id {}{}'.format(series_part, name)
+        raise ValueError(msg.encode('utf-8'))
+    return entity_id
+
+
+def get_bundle_data(reference):
+    """Retrieve and return the bundle data for the given bundle reference.
+
+    The bundle data is returned as a YAML decoded value.
+    The reference argument identifies a bundle entity and must be an instance
+    of "quickstart.models.references.Reference".
+
+    Raise a ValueError if the returned content is not a valid YAML, or if the
+    given reference does not represent a bundle.
+    Raise a NotFoundError if a bundle with the given reference cannot be found
+    in the charm store.
+    Raise an IOError if any other problems occur while communicating with the
+    charm store.
+    """
+    return _retrieve_and_parse_bundle_data(reference, '/archive/bundle.yaml')
+
+
+def get_legacy_bundle_data(reference):
+    """Retrieve and return the bundle legacy data for the given reference.
+
+    The bundle data is returned as a YAML decoded value and represents the
+    legacy bundle with a top level bundle name node.
+    The reference argument identifies a bundle entity and must be an instance
+    of "quickstart.models.references.Reference".
+
+    Raise a ValueError if the returned content is not a valid YAML, or if the
+    given reference does not represent a bundle.
+    Raise a NotFoundError if a bundle with the given reference cannot be found
+    in the charm store.
+    Raise an IOError if any other problems occur while communicating with the
+    charm store.
+    """
+    return _retrieve_and_parse_bundle_data(
+        reference, '/archive/bundles.yaml.orig')
+
+
+def _retrieve_and_parse_bundle_data(reference, path):
+    """Retrieve and parse the bundle YAML for the given reference and path.
+
+    Raise a ValueError if the returned content is not a valid YAML, or if the
+    given reference does not represent a bundle.
+    Raise a NotFoundError if a bundle with the given reference cannot be found
+    in the charm store.
+    Raise an IOError if any other problems occur while communicating with the
+    charm store.
+    """
+    if not reference.is_bundle():
+        raise ValueError(
+            b'expected a bundle, provided charm {}'.format(reference))
+    content = get_reference(reference, path)
+    return load_bundle_yaml(content)
+
+
+def load_bundle_yaml(content):
+    """Deserialize the given YAML encoded bundle content.
+
+    Raise a ValueError if the content is not valid.
+    """
+    try:
+        return serializers.yaml_load(content)
+    except Exception as err:
+        msg = b'unable to parse the bundle content: {}'.format(err)
+        raise ValueError(msg)

=== modified file 'quickstart/manage.py'
--- quickstart/manage.py	2015-02-26 12:42:34 +0000
+++ quickstart/manage.py	2015-03-10 10:19:09 +0000
@@ -294,7 +294,7 @@
         level=level,
         format=(
             '%(asctime)s %(levelname)s '
-            '%(module)s@%(funcName)s:%(lineno)d '
+            '%(module)s:%(lineno)d '
             '%(message)s'
         ),
         datefmt='%H:%M:%S',
@@ -459,9 +459,11 @@
     # Parse the provided arguments.
     options = parser.parse_args()
 
+    # Set up logging.
+    _configure_logging(logging.DEBUG if options.debug else logging.INFO)
+
+    # Validate and add in the platform for convenience.
     _validate_platform(parser, platform)
-
-    # Add in the platform for convenience.
     options.platform = platform
 
     # Convert the provided string arguments to unicode.
@@ -472,8 +474,6 @@
         _validate_bundle(options, parser)
     if options.charm_url is not None:
         _validate_charm_url(options, parser)
-    # Set up logging.
-    _configure_logging(logging.DEBUG if options.debug else logging.INFO)
     return options
 
 

=== modified file 'quickstart/models/bundles.py'
--- quickstart/models/bundles.py	2015-02-26 19:46:48 +0000
+++ quickstart/models/bundles.py	2015-03-10 10:19:09 +0000
@@ -44,8 +44,10 @@
 import collections
 import logging
 import os
+import re
 
 from quickstart import (
+    charmstore,
     netutils,
     serializers,
     settings,
@@ -140,11 +142,7 @@
     """
     if source.startswith('bundle:'):
         # The source refers to an old style bundle URL.
-        reference = references.Reference.from_charmworld_url(source)
-        logging.warn(
-            'this bundle URL is deprecated: please use the new format: '
-            '{}'.format(reference.jujucharms_id()))
-        return _bundle_from_reference(reference)
+        return _bundle_from_charmworld_url(source)
 
     has_extension = source.endswith('.yaml') or source.endswith('.json')
     is_remote = source.startswith('http://') or source.startswith('https://')
@@ -163,30 +161,84 @@
         # No other options are available.
         raise
 
-    if not reference.is_bundle():
-        raise ValueError(
-            b'expected a bundle, provided charm {}'.format(reference))
-
     # The source refers to a bundle URL in jujucharms.com.
-    return _bundle_from_reference(reference)
-
-
-def _bundle_from_reference(reference):
-    """Retrieve bundle YAML contents from its reference in the charm store.
-
-    The path of an entity in the charm store is the fully qualified URL without
-    the schema. The schema is implicitly set to "cs" (charm store entity), e.g.
-    "vivid/django" or "~who/trusty/mediawiki-42".
-
-    Return a Bundle instance which includes the retrieved data and the given
-    reference.
+    try:
+        data = charmstore.get_bundle_data(reference)
+    except charmstore.NotFoundError as err:
+        raise IOError(bytes(err))
+    validate(data)
+    return Bundle(data, reference=reference)
+
+
+# Compile the regular expression used to parse charmworld bundle URLs.
+_charmworld_url_expression = re.compile(r"""
+    ^  # Beginning of the line.
+    (?:bundle:)  # Bundle schema.
+    (?:~({user_pattern})/)?  # Optional user name.
+    ({name_pattern})/  # Basket name.
+    (?:(\d+)/)?  # Optional bundle revision number.
+    ({name_pattern})  # Bundle name.
+    /?  # Optional trailing slash.
+    $  # End of the line.
+""".format(
+    name_pattern=references.NAME_PATTERN,
+    user_pattern=references.USER_PATTERN,
+), re.VERBOSE)
+
+
+def _bundle_from_charmworld_url(url):
+    """Retrieve bundle YAML contents from the given charmworld URL.
+
+    These kind of "bundle:basket/name" URLs were used before the release
+    of the new charm store (API version 4). Possible examples are
+    "bundle:mediawiki/single" or "bundle:~who/wordpress/42/scalable".
+    Note that charmworld URLs always represent a bundle.
+
+    Return a Bundle instance which includes the retrieved data and the bundle
+    reference corresponding to the given charmworld URL.
+    Raise a ValueError if the provided URL is not valid, or if the bundle
+    content is not valid.
     Raise a IOError if a problem is encountered while fetching the YAML
     content from the charm store.
-    Raise a ValueError if the bundle content is not valid.
     """
-    url = settings.CHARMSTORE_API + reference.path() + '/archive/bundle.yaml'
-    content = _retrieve_from_url(url)
-    data = parse_yaml(content)
+    match = _charmworld_url_expression.match(url)
+    if match is None:
+        msg = 'invalid bundle URL: {}'.format(url)
+        raise ValueError(msg.encode('utf-8'))
+    user, basket, revision, name = match.groups()
+    # The legacy bundle ingestion works like the following:
+    # - if a basket includes multiple bundles, the resulting v4 bundle name is
+    #   "{basket-name}-{bundle-name}". In this case, a separate bundle is
+    #   created for each top level name found in the YAML;
+    # - if a basket only includes one bundle, the v4 bundle just uses the
+    #   basket name, because no disambiguation is required.
+    # For this reason, we cannot infer the new bundle identifier from a
+    # charmworld URL: we instead need to try to fetch both references.
+    fullname = '{}-{}'.format(basket, name)
+    reference = references.Reference('cs', user, 'bundle', fullname, revision)
+    try:
+        data = charmstore.get_bundle_data(reference)
+    except charmstore.NotFoundError:
+        # Also try the case in which a single bundle was included in the
+        # legacy YAML. In this case, validating that the name was included
+        # in the original YAML is also required.
+        reference = references.Reference(
+            'cs', user, 'bundle', basket, revision)
+        try:
+            data = charmstore.get_legacy_bundle_data(reference)
+        except charmstore.NotFoundError as err:
+            raise IOError(bytes(err))
+        data = _flatten_data(data, name)
+
+    validate(data)
+    # XXX frankban 2015-02-26: remove this when switching to the new bundle
+    # format. Note that this is monkey patched on purpose: we don't want the
+    # legacy bundle id to be part of the Reference class contract, and we don't
+    # want to keep track of obsolete concepts such as "basket" there.
+    reference.charmworld_id = url[len('bundle:'):]
+    logging.warn(
+        'this bundle URL is deprecated: please use the new format: '
+        '{}'.format(reference.jujucharms_id()))
     return Bundle(data, reference=reference)
 
 
@@ -198,7 +250,7 @@
     """
     try:
         return netutils.urlread(url)
-    except IOError as err:
+    except (netutils.NotFoundError, IOError) as err:
         msg = b'cannot retrieve bundle from remote URL {}: {}'.format(
             url.encode('utf-8'), err)
         raise IOError(msg)
@@ -233,12 +285,28 @@
         - the YAML contents are not properly structured;
         - the bundle does not include services.
     """
-    data = _open_yaml(content)
+    data = charmstore.load_bundle_yaml(content)
     # Validate the bundle data.
     validate(data)
     return data
 
 
+def is_legacy_bundle(data):
+    """Report whether the given bundle data represents a legacy bundle.
+
+    Assume the given data is a dictionary like object.
+    """
+    services = data.get('services')
+    # The internal structure of a bundle in the API version 4 does not include
+    # a wrapping namespace with the bundle name. That's why the check below,
+    # despite its ugliness, is quite effective.
+    if (not services):
+        return True
+    if isinstance(services, collections.Mapping) and ('services' in services):
+        return True
+    return False
+
+
 def _parse_and_flatten_yaml(content, name):
     """Parse and validate the given bundle content.
 
@@ -255,15 +323,29 @@
           than one bundle;
         - the bundle does not include services.
     """
-    data = _open_yaml(content)
-    services = data.get('services')
-    # The internal structure of a bundle in the API version 4 does not include
-    # a wrapping namespace with the bundle name. That's why the check below,
-    # despite its ugliness, is quite effective.
-    if services and 'services' not in services:
-        # This is an API version 4 bundle.
-        validate(data)
-        return data
+    data = charmstore.load_bundle_yaml(content)
+    _ensure_is_dict(data)
+    if is_legacy_bundle(data):
+        data = _flatten_data(data, name)
+    validate(data)
+    return data
+
+
+def _flatten_data(data, name):
+    """Retrieve the bundle content from data for a specific bundle name.
+
+    The returned YAML decoded data represents a new style bundle
+    (API version 4).
+
+    Raise a ValueError if:
+        - the YAML data is not properly structured;
+        - the YAML data does not include any bundles;
+        - the bundle name is specified but not included in the bundle file;
+        - the bundle name is not specified and the bundle file includes more
+          than one bundle;
+        - the bundle does not include services.
+    """
+    _ensure_is_dict(data)
     num_bundles = len(data)
     if not num_bundles:
         raise ValueError(b'no bundles found in the provided list of bundles')
@@ -272,30 +354,14 @@
         if num_bundles > 1:
             msg = 'multiple bundles found ({}) but no bundle name specified'
             raise ValueError(msg.format(names).encode('utf-8'))
-        data = data.values()[0]
-    else:
-        data = data.get(name)
-        if data is None:
+        return data.values()[0]
+    data = data.get(name)
+    if data is None:
+        if num_bundles == 1:
+            msg = 'bundle {} not found, did you mean {}?'
+        else:
             msg = 'bundle {} not found in the provided list of bundles ({})'
-            raise ValueError(msg.format(name, names).encode('utf-8'))
-    validate(data)
-    return data
-
-
-def _open_yaml(content):
-    """Deserialize the given content, that must be a YAML encoded dictionary.
-
-    Raise a ValueError if the content is not valid.
-    """
-    try:
-        data = serializers.yaml_load(content)
-    except Exception as err:
-        msg = b'unable to parse the bundle content: {}'.format(err)
-        raise ValueError(msg)
-    # Ensure the bundle content is well formed.
-    if not isinstance(data, collections.Mapping):
-        msg = 'invalid YAML content: {}'.format(data)
-        raise ValueError(msg.encode('utf-8'))
+        raise ValueError(msg.format(name, names).encode('utf-8'))
     return data
 
 
@@ -312,6 +378,7 @@
         - the YAML contents are not properly structured;
         - the bundle does not include services.
     """
+    _ensure_is_dict(data)
     # Retrieve the bundle services.
     try:
         services = data['services'].keys()
@@ -328,3 +395,13 @@
             b'the provided bundle contains an instance of juju-gui. Juju '
             b'Quickstart will install the latest version of the Juju GUI '
             b'automatically; please remove juju-gui from the bundle')
+
+
+def _ensure_is_dict(data):
+    """Ensure that the given bundle data is a dictionary like object.
+
+    Raise a ValueError otherwise.
+    """
+    if not isinstance(data, collections.Mapping):
+        msg = 'invalid YAML content: {}'.format(data)
+        raise ValueError(msg.encode('utf-8'))

=== modified file 'quickstart/models/references.py'
--- quickstart/models/references.py	2015-02-27 09:25:33 +0000
+++ quickstart/models/references.py	2015-03-10 10:19:09 +0000
@@ -25,29 +25,15 @@
 
 # The following regular expressions are the same used in juju-core: see
 # http://bazaar.launchpad.net/~go-bot/juju-core/trunk/view/head:/charm/url.go.
-_USER_PATTERN = r'[a-z0-9][a-zA-Z0-9+.-]+'
-_SERIES_PATTERN = r'[a-z]+(?:[a-z-]+[a-z])?'
-_NAME_PATTERN = r'[a-z][a-z0-9]*(?:-[a-z0-9]*[a-z][a-z0-9]*)*'
+USER_PATTERN = r'[a-z0-9][a-zA-Z0-9+.-]+'
+SERIES_PATTERN = r'[a-z]+(?:[a-z-]+[a-z])?'
+NAME_PATTERN = r'[a-z][a-z0-9]*(?:-[a-z0-9]*[a-z][a-z0-9]*)*'
 
 # Define the callables used to check if entity reference components are valid.
-_valid_user = re.compile(r'^{}$'.format(_USER_PATTERN)).match
-_valid_series = re.compile(r'^{}$'.format(_SERIES_PATTERN)).match
-_valid_name = re.compile(r'^{}$'.format(_NAME_PATTERN)).match
+_valid_user = re.compile(r'^{}$'.format(USER_PATTERN)).match
+_valid_series = re.compile(r'^{}$'.format(SERIES_PATTERN)).match
+_valid_name = re.compile(r'^{}$'.format(NAME_PATTERN)).match
 
-# Compile the regular expression used to parse charmworld bundle URLs.
-_charmworld_url_expression = re.compile(r"""
-    ^  # Beginning of the line.
-    (?:bundle:)  # Bundle schema.
-    (?:~({user_pattern})/)?  # Optional user name.
-    ({name_pattern})/  # Basket name.
-    (?:(\d+)/)?  # Optional bundle revision number.
-    ({name_pattern})  # Bundle name.
-    /?  # Optional trailing slash.
-    $  # End of the line.
-""".format(
-    name_pattern=_NAME_PATTERN,
-    user_pattern=_USER_PATTERN,
-), re.VERBOSE)
 # Compile the regular expression used to parse new jujucharms entity URLs.
 _jujucharms_url_expression = re.compile(r"""
     ^  # Beginning of the line.
@@ -64,9 +50,9 @@
     $  # End of the line.
 """.format(
     jujucharms=settings.JUJUCHARMS_URL,
-    name_pattern=_NAME_PATTERN,
-    series_pattern=_SERIES_PATTERN,
-    user_pattern=_USER_PATTERN,
+    name_pattern=NAME_PATTERN,
+    series_pattern=SERIES_PATTERN,
+    user_pattern=USER_PATTERN,
 ), re.VERBOSE)
 
 
@@ -101,31 +87,6 @@
         return cls(*_parse_fully_qualified_url(url))
 
     @classmethod
-    def from_charmworld_url(cls, url):
-        """Create and return a Reference from the given charmworld URL.
-
-        These kind of "bundle:basket/name" URLs were used before the release
-        of the new charm store (API version 4). Possible examples are
-        "bundle:mediawiki/single" or "bundle:~who/wordpress/42/scalable".
-        Note that charmworld URLs always represent a bundle.
-
-        Raise a ValueError if the provided URL is not valid.
-        """
-        match = _charmworld_url_expression.match(url)
-        if match is None:
-            msg = 'invalid bundle URL: {}'.format(url)
-            raise ValueError(msg.encode('utf-8'))
-        user, basket, revision, name = match.groups()
-        name = '{}-{}'.format(basket, name)
-        self = cls('cs', user, 'bundle', name, revision)
-        # XXX frankban 2015-02-26: remove this when switching to the new bundle
-        # format. Note that this is monkey patched on purpose: we don't want
-        # the legacy bundle id to be part of this class contract, and we don't
-        # want to keep track of obsolete concepts such as "basket" here.
-        self.charmworld_id = url[len('bundle:'):]
-        return self
-
-    @classmethod
     def from_jujucharms_url(cls, url):
         """Create and return a Reference from the given jujucharms.com URL.
 

=== modified file 'quickstart/netutils.py'
--- quickstart/netutils.py	2015-02-25 15:28:56 +0000
+++ quickstart/netutils.py	2015-03-10 10:19:09 +0000
@@ -18,14 +18,11 @@
 
 from __future__ import unicode_literals
 
-import json
 import httplib
 import logging
 import socket
 import urllib2
 
-from quickstart import settings
-
 
 def check_resolvable(hostname):
     """Check that the hostname can be resolved to a numeric IP address.
@@ -63,31 +60,25 @@
     return None
 
 
-def get_charm_url(series):
-    """Return the charm URL of the latest Juju GUI charm revision.
-
-    Raise an IOError if any problems occur connecting to the API endpoint.
-    Raise a ValueError if the API returns invalid data.
-    """
-    url = '{}{}/{}/meta/id'.format(
-        settings.CHARMSTORE_API, series, settings.JUJU_GUI_CHARM_NAME)
-    data = json.loads(urlread(url))
-    charm_url = data.get('Id')
-    if charm_url is None:
-        raise ValueError(b'unable to find the charm URL')
-    return charm_url
+class NotFoundError(Exception):
+    """Represent a 404 not found HTTP error."""
 
 
 def urlread(url):
     """Open the given URL and return the page contents.
 
-    Raise an IOError if any problems occur.
+    Raise a NotFoundError if the request returns a 404 not found response.
+    Raise an IOError if any other problems occur.
     """
+    logging.debug('sending HTTP GET request to {}'.format(url))
     try:
         response = urllib2.urlopen(url)
+    except urllib2.HTTPError as err:
+        exception = NotFoundError if err.code == 404 else IOError
+        raise exception(bytes(err))
     except urllib2.URLError as err:
         raise IOError(err.reason)
-    except (httplib.HTTPException, socket.error, urllib2.HTTPError) as err:
+    except (httplib.HTTPException, socket.error) as err:
         raise IOError(bytes(err))
     contents = response.read()
     content_type = response.headers['content-type']

=== modified file 'quickstart/settings.py'
--- quickstart/settings.py	2015-02-25 19:26:02 +0000
+++ quickstart/settings.py	2015-03-10 10:19:09 +0000
@@ -37,8 +37,8 @@
 # temporary connection/charm store errors.
 # Keep this list sorted by release date (older first).
 DEFAULT_CHARM_URLS = collections.OrderedDict((
-    ('precise', 'cs:precise/juju-gui-106'),
-    ('trusty', 'cs:trusty/juju-gui-18'),
+    ('precise', 'cs:precise/juju-gui-108'),
+    ('trusty', 'cs:trusty/juju-gui-21'),
 ))
 
 # The quickstart app short description.

=== modified file 'quickstart/tests/helpers.py'
--- quickstart/tests/helpers.py	2015-02-26 19:46:48 +0000
+++ quickstart/tests/helpers.py	2015-03-10 10:19:09 +0000
@@ -299,17 +299,18 @@
 class UrlReadTestsMixin(object):
     """Helpers to mock the quickstart.netutils.urlread helper function."""
 
-    def patch_urlread(self, contents=None, error=False):
+    def patch_urlread(self, contents=None, error=None):
         """Patch the quickstart.netutils.urlread helper function.
 
         If contents is not None, urlread() will return the provided contents.
-        If error is set to True, an IOError will be simulated.
+        Otherwise, if error is not None, the given exception or list of side
+        effects will be simulated.
         """
         mock_urlread = mock.Mock()
         if contents is not None:
             mock_urlread.return_value = contents
-        if error:
-            mock_urlread.side_effect = IOError('bad wolf')
+        elif error is not None:
+            mock_urlread.side_effect = error
         return mock.patch('quickstart.netutils.urlread', mock_urlread)
 
 

=== modified file 'quickstart/tests/models/test_bundles.py'
--- quickstart/tests/models/test_bundles.py	2015-02-26 19:46:48 +0000
+++ quickstart/tests/models/test_bundles.py	2015-03-10 10:19:09 +0000
@@ -21,9 +21,13 @@
 import json
 import unittest
 
+import mock
 import yaml
 
-from quickstart import settings
+from quickstart import (
+    netutils,
+    settings,
+)
 from quickstart.models import (
     bundles,
     references,
@@ -86,6 +90,10 @@
             bundle = bundles.from_source('bundle:mediawiki/single')
         self.assertEqual(self.bundle_data, bundle.data)
         self.assertEqual('cs:bundle/mediawiki-single', bundle.reference.id())
+        # The charmworld id is properly set when passing charmworld URLs.
+        # XXX frankban 2015-03-09: remove this check once we get rid of the
+        # charmworld id concept.
+        self.assertEqual('mediawiki/single', bundle.reference.charmworld_id)
         mock_urlread.assert_called_once_with(
             settings.CHARMSTORE_API +
             'bundle/mediawiki-single/archive/bundle.yaml')
@@ -98,10 +106,45 @@
         self.assertEqual(self.bundle_data, bundle.data)
         self.assertEqual(
             'cs:~who/bundle/mediawiki-single-42', bundle.reference.id())
+        # The charmworld id is properly set when passing charmworld URLs.
+        # XXX frankban 2015-03-09: remove this check once we get rid of the
+        # charmworld id concept.
+        self.assertEqual(
+            '~who/mediawiki/42/single', bundle.reference.charmworld_id)
         mock_urlread.assert_called_once_with(
             settings.CHARMSTORE_API +
             '~who/bundle/mediawiki-single-42/archive/bundle.yaml')
 
+    def test_charmworld_bundle_from_legacy(self):
+        # A bundle instance is properly returned from a charmworld id source
+        # by looking at the legacy YAML stored in the charm store.
+        side_effect = [
+            # A first call urlread returns a not found error.
+            netutils.NotFoundError('boo!'),
+            # A second call succeeds.
+            self.legacy_bundle_content,
+        ]
+        with self.patch_urlread(error=side_effect) as mock_urlread:
+            bundle = bundles.from_source('bundle:mediawiki/bundle1')
+        self.assertEqual(self.bundle_data, bundle.data)
+        self.assertEqual('cs:bundle/mediawiki', bundle.reference.id())
+        # The charmworld id is properly set when passing charmworld URLs.
+        # XXX frankban 2015-03-09: remove this check once we get rid of the
+        # charmworld id concept.
+        self.assertEqual('mediawiki/bundle1', bundle.reference.charmworld_id)
+        # The urlread function has been called two times: the first time
+        # including both the basket and the bundle name, the second time
+        # to retrieve the legacy bundle yaml, only including the basket.
+        self.assertEqual(mock_urlread.call_count, 2)
+        mock_urlread.assert_has_calls([
+            mock.call(
+                settings.CHARMSTORE_API +
+                'bundle/mediawiki-bundle1/archive/bundle.yaml'),
+            mock.call(
+                settings.CHARMSTORE_API +
+                'bundle/mediawiki/archive/bundles.yaml.orig'),
+        ])
+
     def test_charmworld_bundle_deprecation_warning(self):
         # A deprecation warning is printed if the no longer supported
         # charmworld bundle identifiers are used.
@@ -119,22 +162,73 @@
 
     def test_charmworld_bundle_invalid_content(self):
         # A ValueError is raised if the content associated with the given
-        # charmworld URL are not valid.
-        with self.patch_urlread(error=True):
-            with self.assertRaises(IOError) as ctx:
+        # charmworld URL is not valid.
+        with self.patch_urlread(contents='exterminate!'):
+            with self.assert_value_error('invalid YAML content: exterminate!'):
                 bundles.from_source('bundle:mediawiki/single')
-        expected_error = (
-            'cannot retrieve bundle from remote URL '
-            '{}bundle/mediawiki-single/archive/bundle.yaml: '
-            'bad wolf'.format(settings.CHARMSTORE_API))
-        self.assertEqual(expected_error, bytes(ctx.exception))
 
     def test_charmworld_bundle_connection_error(self):
         # An IOError is raised if a connection problem is encountered while
         # retrieving the charmworld bundle.
-        with self.patch_urlread(contents='exterminate!'):
-            with self.assert_value_error('invalid YAML content: exterminate!'):
-                bundles.from_source('bundle:mediawiki/single')
+        with self.patch_urlread(error=IOError('bad wolf')):
+            with self.assertRaises(IOError) as ctx:
+                bundles.from_source('bundle:mediawiki/single')
+        expected_error = (
+            'cannot communicate with the charm store at '
+            '{}bundle/mediawiki-single/archive/bundle.yaml: '
+            'bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+    def test_charmworld_bundle_not_found_error(self):
+        # An IOError is raised if a charmworld bundle is not found.
+        error = netutils.NotFoundError('bad wolf')
+        with self.patch_urlread(error=error) as mock_urlread:
+            with self.assertRaises(IOError) as ctx:
+                bundles.from_source('bundle:mediawiki/single')
+        expected_error = (
+            'charm store resource not found at '
+            '{}bundle/mediawiki/archive/bundles.yaml.orig: '
+            'bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+        # The urlread function has been called two times: the first time
+        # including both the basket and the bundle name, the second time
+        # to retrieve the legacy bundle yaml, only including the basket.
+        self.assertEqual(mock_urlread.call_count, 2)
+        mock_urlread.assert_has_calls([
+            mock.call(
+                settings.CHARMSTORE_API +
+                'bundle/mediawiki-single/archive/bundle.yaml'),
+            mock.call(
+                settings.CHARMSTORE_API +
+                'bundle/mediawiki/archive/bundles.yaml.orig'),
+        ])
+
+    def test_charmworld_bundle_from_legacy_invalid_name(self):
+        # A ValueError is raised if the given bundle name is not found in
+        # the legacy basket.
+        legacy_bundle_content = yaml.safe_dump({'scalable': self.bundle_data})
+        side_effect = [
+            # A first call urlread returns a not found error.
+            netutils.NotFoundError('boo!'),
+            # A second call succeeds.
+            legacy_bundle_content,
+        ]
+        expected_error = 'bundle single not found, did you mean scalable?'
+        with self.patch_urlread(error=side_effect) as mock_urlread:
+            with self.assert_value_error(expected_error):
+                bundles.from_source('bundle:django/single')
+        # The urlread function has been called two times: the first time
+        # including both the basket and the bundle name, the second time
+        # to retrieve the legacy bundle yaml, only including the basket.
+        self.assertEqual(mock_urlread.call_count, 2)
+        mock_urlread.assert_has_calls([
+            mock.call(
+                settings.CHARMSTORE_API +
+                'bundle/django-single/archive/bundle.yaml'),
+            mock.call(
+                settings.CHARMSTORE_API +
+                'bundle/django/archive/bundles.yaml.orig'),
+        ])
 
     def test_jujucharms_bundle(self):
         # A bundle instance is properly returned from a jujucharms.com id.
@@ -173,21 +267,33 @@
     def test_jujucharms_bundle_invalid_content(self):
         # A ValueError is raised if the content associated with the given
         # jujucharms.com URL are not valid.
-        with self.patch_urlread(error=True):
-            with self.assertRaises(IOError) as ctx:
-                bundles.from_source('django/42')
-        expected_error = (
-            'cannot retrieve bundle from remote URL '
-            '{}bundle/django-42/archive/bundle.yaml: '
-            'bad wolf'.format(settings.CHARMSTORE_API))
-        self.assertEqual(expected_error, bytes(ctx.exception))
+        with self.patch_urlread(contents='exterminate!'):
+            with self.assert_value_error('invalid YAML content: exterminate!'):
+                bundles.from_source('wordpress-scalable')
 
     def test_jujucharms_bundle_connection_error(self):
         # An IOError is raised if a connection problem is encountered while
         # retrieving the jujucharms.com bundle.
-        with self.patch_urlread(contents='exterminate!'):
-            with self.assert_value_error('invalid YAML content: exterminate!'):
-                bundles.from_source('wordpress-scalable')
+        with self.patch_urlread(error=IOError('bad wolf')):
+            with self.assertRaises(IOError) as ctx:
+                bundles.from_source('django/42')
+        expected_error = (
+            'cannot communicate with the charm store at '
+            '{}bundle/django-42/archive/bundle.yaml: '
+            'bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+    def test_jujucharms_bundle_not_found_error(self):
+        # An IOError is raised if a connection problem is encountered while
+        # retrieving the jujucharms.com bundle.
+        with self.patch_urlread(error=netutils.NotFoundError('bad wolf')):
+            with self.assertRaises(IOError) as ctx:
+                bundles.from_source('django/42')
+        expected_error = (
+            'charm store resource not found at '
+            '{}bundle/django-42/archive/bundle.yaml: '
+            'bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
 
     def test_local_file(self):
         # A bundle instance can be created from a local file source.
@@ -248,11 +354,20 @@
         with self.assert_value_error(expected_error):
             bundles.from_source(path, 'mybundle')
 
+    def test_local_file_legacy_bundle_single_bundle_name_not_found(self):
+        # A ValueError is raised if a local file contains a legacy version 3
+        # YAML content with one bundle defined and the provided bundle name
+        # does not match.
+        path = self.make_bundle_file({'scalable': self.bundle_data})
+        expected_error = 'bundle mybundle not found, did you mean scalable?'
+        with self.assert_value_error(expected_error):
+            bundles.from_source(path, 'mybundle')
+
     def test_local_file_legacy_bundle_invalid_bundle_content(self):
         # A ValueError is raised if a local file contains an invalid legacy
         # version 3 content.
         path = self.make_bundle_file({'bundle': '42'})
-        expected_error = "unable to retrieve bundle services: '42'"
+        expected_error = "invalid YAML content: 42"
         with self.assert_value_error(expected_error):
             bundles.from_source(path, 'bundle')
 
@@ -280,7 +395,7 @@
     def test_remote_url_not_reachable(self):
         # An IOError is raised if a network problem is encountered while
         # trying to reach the remote URL.
-        with self.patch_urlread(error=True):
+        with self.patch_urlread(error=IOError('bad wolf')):
             with self.assertRaises(IOError) as ctx:
                 bundles.from_source('https://1.2.3.4')
         expected_error = (
@@ -317,6 +432,16 @@
             with self.assert_value_error(expected_error):
                 bundles.from_source('http://1.2.3.4', 'no-such')
 
+    def test_remote_url_legacy_bundle_single_bundle_name_not_found(self):
+        # A ValueError is raised if a remote URL contains a legacy version 3
+        # YAML content with one bundle defined and the provided bundle name
+        # does not match.
+        legacy_bundle_content = yaml.safe_dump({'scalable': self.bundle_data})
+        expected_error = 'bundle no-such not found, did you mean scalable?'
+        with self.patch_urlread(contents=legacy_bundle_content):
+            with self.assert_value_error(expected_error):
+                bundles.from_source('http://1.2.3.4', 'no-such')
+
     def test_remote_url_legacy_bundle_invalid_bundle_content(self):
         # A ValueError is raised if a remote URL contains an invalid legacy
         # version 3 content.
@@ -399,6 +524,22 @@
         self.assertEqual(self.bundle_data, data)
 
 
+class TestIsLegacyBundle(helpers.BundleFileTestsMixin, unittest.TestCase):
+
+    def test_v4_bundle(self):
+        # False is returned for new-style version 4 bundles.
+        self.assertFalse(bundles.is_legacy_bundle(self.bundle_data))
+
+    def test_legacy_bundle(self):
+        # True is returned for legacy bundles.
+        tests = (
+            self.legacy_bundle_data,
+            {'services': {'services': {}}},
+        )
+        for data in tests:
+            self.assertTrue(bundles.is_legacy_bundle(data))
+
+
 class TestValidate(
         helpers.BundleFileTestsMixin, helpers.ValueErrorTestsMixin,
         unittest.TestCase):

=== modified file 'quickstart/tests/models/test_references.py'
--- quickstart/tests/models/test_references.py	2015-02-26 19:46:48 +0000
+++ quickstart/tests/models/test_references.py	2015-03-10 10:19:09 +0000
@@ -318,43 +318,6 @@
             self.assertEqual(expected_ref, ref)
 
 
-class TestReferenceFromCharmworldUrl(
-        helpers.ValueErrorTestsMixin, unittest.TestCase):
-
-    def test_invalid_form(self):
-        # A ValueError is raised if the URL is not valid.
-        expected_error = 'invalid bundle URL: bad wolf'
-        with self.assert_value_error(expected_error):
-            references.Reference.from_charmworld_url('bad wolf')
-
-    def test_success(self):
-        # A reference is correctly created from a charmworld identifier.
-        tests = (
-            ('bundle:~myuser/wordpress/42/single',
-             make_reference(series='bundle', name='wordpress-single')),
-            ('bundle:~myuser/wordpress/single',
-             make_reference(series='bundle', name='wordpress-single',
-                            revision=None)),
-            ('bundle:wordpress/42/single',
-             make_reference(user='', series='bundle',
-                            name='wordpress-single')),
-            ('bundle:wordpress/single',
-             make_reference(user='', series='bundle', name='wordpress-single',
-                            revision=None)),
-        )
-        for url, expected_ref in tests:
-            ref = references.Reference.from_charmworld_url(url)
-            self.assertEqual(expected_ref, ref)
-
-    def test_charmworld_id(self):
-        # The charmworld id is properly set when parsing charmworld URLs.
-        # XXX frankban 2015-02-26: remove this test once we get rid of the
-        # charmworld id concept.
-        ref = references.Reference.from_charmworld_url(
-            'bundle:wordpress/single')
-        self.assertEqual('wordpress/single', ref.charmworld_id)
-
-
 class TestReferenceFromJujucharmsUrl(
         helpers.ValueErrorTestsMixin, unittest.TestCase):
 

=== modified file 'quickstart/tests/test_app.py'
--- quickstart/tests/test_app.py	2015-02-26 19:46:48 +0000
+++ quickstart/tests/test_app.py	2015-03-10 10:19:09 +0000
@@ -930,12 +930,11 @@
         env.get_status.side_effect = side_effect
         return env
 
-    def patch_get_charm_url(self, return_value=None, side_effect=None):
-        """Patch the get_charm_url helper function."""
-        mock_get_charm_url = mock.Mock(
+    def patch_resolve(self, return_value=None, side_effect=None):
+        """Patch the charmstore.resolve function."""
+        mock_resolve = mock.Mock(
             return_value=return_value, side_effect=side_effect)
-        return mock.patch(
-            'quickstart.netutils.get_charm_url', mock_get_charm_url)
+        return mock.patch('quickstart.charmstore.resolve', mock_resolve)
 
     def assert_reference_equal(self, expected_url, ref):
         """Ensure the given reference points to the expected URL."""
@@ -954,8 +953,8 @@
         env_type = 'ec2'
         bootstrap_node_series = 'trusty'
         check_preexisting = False
-        with self.patch_get_charm_url(
-                return_value='cs:trusty/juju-gui-42') as mock_get_charm_url:
+        with self.patch_resolve(
+                return_value='cs:trusty/juju-gui-42') as mock_resolve:
             ref, machine, service_data, unit_data = app.check_environment(
                 env, 'my-gui', charm_url, env_type, bootstrap_node_series,
                 check_preexisting)
@@ -964,7 +963,8 @@
         # The charm URL has been retrieved from the charm store API based on
         # the current bootstrap node series.
         self.assert_reference_equal('cs:trusty/juju-gui-42', ref)
-        mock_get_charm_url.assert_called_once_with(bootstrap_node_series)
+        mock_resolve.assert_called_once_with(
+            settings.JUJU_GUI_CHARM_NAME, series=bootstrap_node_series)
         # Since the bootstrap node series is supported by the GUI charm, the
         # GUI unit can be deployed to machine 0.
         self.assertEqual('0', machine)
@@ -989,8 +989,8 @@
         env_type = 'ec2'
         bootstrap_node_series = 'precise'
         check_preexisting = True
-        with self.patch_get_charm_url(
-                return_value='cs:precise/juju-gui-42') as mock_get_charm_url:
+        with self.patch_resolve(
+                return_value='cs:precise/juju-gui-42') as mock_resolve:
             ref, machine, service_data, unit_data = app.check_environment(
                 env, 'my-gui', charm_url, env_type, bootstrap_node_series,
                 check_preexisting)
@@ -999,7 +999,8 @@
         # The charm URL has been retrieved from the charm store API based on
         # the current bootstrap node series.
         self.assert_reference_equal('cs:precise/juju-gui-42', ref)
-        mock_get_charm_url.assert_called_once_with(bootstrap_node_series)
+        mock_resolve.assert_called_once_with(
+            settings.JUJU_GUI_CHARM_NAME, series=bootstrap_node_series)
         # Since the bootstrap node series is supported by the GUI charm, the
         # GUI unit can be deployed to machine 0.
         self.assertEqual('0', machine)
@@ -1022,7 +1023,7 @@
         env_type = 'ec2'
         bootstrap_node_series = 'precise'
         check_preexisting = True
-        with self.patch_get_charm_url() as mock_get_charm_url:
+        with self.patch_resolve() as mock_resolve:
             ref, machine, service_data, unit_data = app.check_environment(
                 env, 'my-gui', charm_url, env_type, bootstrap_node_series,
                 check_preexisting)
@@ -1030,7 +1031,7 @@
         env.get_status.assert_called_once_with()
         # The charm URL has been retrieved from the environment.
         self.assert_reference_equal('cs:precise/juju-gui-47', ref)
-        self.assertFalse(mock_get_charm_url.called)
+        self.assertFalse(mock_resolve.called)
         # Since the bootstrap node series is supported by the GUI charm, the
         # GUI unit can be safely deployed to machine 0.
         self.assertEqual('0', machine)
@@ -1046,15 +1047,16 @@
         env_type = 'ec2'
         bootstrap_node_series = 'saucy'
         check_preexisting = False
-        with self.patch_get_charm_url(
-                return_value='cs:trusty/juju-gui-42') as mock_get_charm_url:
+        with self.patch_resolve(
+                return_value='cs:trusty/juju-gui-42') as mock_resolve:
             ref, machine, service_data, unit_data = app.check_environment(
                 env, 'my-gui', charm_url, env_type, bootstrap_node_series,
                 check_preexisting)
         # The charm URL has been retrieved from the charm store API using the
         # most recent supported series.
         self.assert_reference_equal('cs:trusty/juju-gui-42', ref)
-        mock_get_charm_url.assert_called_once_with('trusty')
+        mock_resolve.assert_called_once_with(
+            settings.JUJU_GUI_CHARM_NAME, series='trusty')
         # The Juju GUI unit cannot be deployed to saucy machine 0.
         self.assertIsNone(machine)
         # Ensure the function output makes sense.
@@ -1072,7 +1074,7 @@
         env_type = 'local'
         bootstrap_node_series = 'trusty'
         check_preexisting = False
-        with self.patch_get_charm_url(return_value='cs:trusty/juju-gui-42'):
+        with self.patch_resolve(return_value='cs:trusty/juju-gui-42'):
             ref, machine, service_data, unit_data = app.check_environment(
                 env, 'my-gui', charm_url, env_type, bootstrap_node_series,
                 check_preexisting)
@@ -1089,7 +1091,7 @@
         env_type = 'azure'
         bootstrap_node_series = 'trusty'
         check_preexisting = False
-        with self.patch_get_charm_url(return_value='cs:trusty/juju-gui-42'):
+        with self.patch_resolve(return_value='cs:trusty/juju-gui-42'):
             _, machine, _, _ = app.check_environment(
                 env, 'my-gui', charm_url, env_type, bootstrap_node_series,
                 check_preexisting)
@@ -1103,7 +1105,7 @@
         env_type = 'ec2'
         bootstrap_node_series = 'precise'
         check_preexisting = False
-        with self.patch_get_charm_url(side_effect=IOError('boo!')):
+        with self.patch_resolve(side_effect=IOError('boo!')):
             ref, machine, service_data, unit_data = app.check_environment(
                 env, 'my-gui', charm_url, env_type, bootstrap_node_series,
                 check_preexisting)
@@ -1121,7 +1123,7 @@
         env_type = 'ec2'
         bootstrap_node_series = 'saucy'
         check_preexisting = False
-        with self.patch_get_charm_url(side_effect=IOError('boo!')):
+        with self.patch_resolve(side_effect=IOError('boo!')):
             ref, machine, service_data, unit_data = app.check_environment(
                 env, 'my-gui', charm_url, env_type, bootstrap_node_series,
                 check_preexisting)
@@ -1138,13 +1140,13 @@
         env_type = 'ec2'
         bootstrap_node_series = 'trusty'
         check_preexisting = False
-        with self.patch_get_charm_url() as mock_get_charm_url:
+        with self.patch_resolve() as mock_resolve:
             ref, machine, service_data, unit_data = app.check_environment(
                 env, 'my-gui', charm_url, env_type, bootstrap_node_series,
                 check_preexisting)
-        # There is no need to call the charmword API if the charm URL is
+        # There is no need to call the charm store API if the charm URL is
         # provided by the user.
-        self.assertFalse(mock_get_charm_url.called)
+        self.assertFalse(mock_resolve.called)
         # The provided charm URL has been correctly returned.
         self.assert_reference_equal(charm_url, ref)
         # Since the provided charm series is trusty, the charm itself can be
@@ -1165,13 +1167,13 @@
         env_type = 'ec2'
         bootstrap_node_series = 'precise'
         check_preexisting = False
-        with self.patch_get_charm_url() as mock_get_charm_url:
+        with self.patch_resolve() as mock_resolve:
             ref, machine, service_data, unit_data = app.check_environment(
                 env, 'my-gui', charm_url, env_type, bootstrap_node_series,
                 check_preexisting)
-        # There is no need to call the charmword API if the charm URL is
+        # There is no need to call the charm store API if the charm URL is
         # provided by the user.
-        self.assertFalse(mock_get_charm_url.called)
+        self.assertFalse(mock_resolve.called)
         # The provided charm URL has been correctly returned.
         self.assert_reference_equal(charm_url, ref)
         # Since the provided charm series is not precise, the charm must be
@@ -1666,7 +1668,9 @@
         # XXX frankban 2015-02-26: remove this test once we get rid of the
         # charmworld id concept.
         env = mock.Mock()
-        ref = references.Reference.from_charmworld_url('bundle:django/single')
+        ref = references.Reference.from_fully_qualified_url(
+            'cs:bundle/django-single-42')
+        ref.charmworld_id = 'django/single'
         bundle = bundles.Bundle(self.bundle_data, reference=ref)
         app.deploy_bundle(env, bundle)
         env.deploy_bundle.assert_called_once_with(

=== added file 'quickstart/tests/test_charmstore.py'
--- quickstart/tests/test_charmstore.py	1970-01-01 00:00:00 +0000
+++ quickstart/tests/test_charmstore.py	2015-03-10 10:19:09 +0000
@@ -0,0 +1,313 @@
+# This file is part of the Juju Quickstart Plugin, which lets users set up a
+# Juju environment in very few steps (https://launchpad.net/juju-quickstart).
+# Copyright (C) 2015 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License version 3, as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Tests for the Juju Quickstart charm store communication utilities."""
+
+from __future__ import unicode_literals
+
+import unittest
+
+import json
+
+from quickstart import (
+    charmstore,
+    netutils,
+    settings,
+)
+from quickstart.models import references
+from quickstart.tests import helpers
+
+
+class TestNotFoundError(unittest.TestCase):
+
+    def test_error_message(self):
+        # The error message can be properly retrieved.
+        err = charmstore.NotFoundError(b'bad wolf')
+        self.assertEqual('bad wolf', bytes(err))
+
+
+class TestGet(helpers.UrlReadTestsMixin, unittest.TestCase):
+
+    def test_success(self):
+        # A GET request to the charm store is correctly performed.
+        with self.patch_urlread(contents='ok') as mock_urlread:
+            content = charmstore.get('my/path')
+        self.assertEqual('ok', content)
+        mock_urlread.assert_called_once_with(
+            settings.CHARMSTORE_API + 'my/path')
+
+    def test_success_leading_slash(self):
+        # The resulting URL is correctly formatted.
+        with self.patch_urlread() as mock_urlread:
+            charmstore.get('/path/')
+        mock_urlread.assert_called_once_with(settings.CHARMSTORE_API + 'path/')
+
+    def test_io_error(self):
+        # An IOError is raised if a problem is encountered while connecting to
+        # the charm store.
+        with self.patch_urlread(error=IOError('bad wolf')):
+            with self.assertRaises(IOError) as ctx:
+                charmstore.get('/')
+        expected_error = (
+            'cannot communicate with the charm store at '
+            '{}: bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+    def test_not_found_error(self):
+        # A charmstore.NotFoundError is raised if the request returns a 404 not
+        # found response.
+        with self.patch_urlread(error=netutils.NotFoundError('bad wolf')):
+            with self.assertRaises(charmstore.NotFoundError) as ctx:
+                charmstore.get('/no/such')
+        expected_error = (
+            'charm store resource not found at '
+            '{}no/such: bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+
+class TestGetReference(helpers.UrlReadTestsMixin, unittest.TestCase):
+
+    def test_success(self):
+        # A GET request to the charm store is correctly performed using the
+        # given charm or bundle reference.
+        ref = references.Reference.from_jujucharms_url('mediawiki-single')
+        with self.patch_urlread(contents='hash') as mock_urlread:
+            content = charmstore.get_reference(ref, '/meta/hash')
+        self.assertEqual('hash', content)
+        mock_urlread.assert_called_once_with(
+            settings.CHARMSTORE_API + 'bundle/mediawiki-single/meta/hash')
+
+    def test_success_without_leading_slash(self):
+        # The resulting URL is correctly formatted when the static path does
+        # not include a leading slash.
+        ref = references.Reference.from_jujucharms_url('django/trusty/42')
+        with self.patch_urlread() as mock_urlread:
+            charmstore.get_reference(ref, 'expand-id')
+        mock_urlread.assert_called_once_with(
+            settings.CHARMSTORE_API + 'trusty/django-42/expand-id')
+
+    def test_io_error(self):
+        # An IOError is raised if a problem is encountered while connecting to
+        # the charm store.
+        ref = references.Reference.from_jujucharms_url('django/trusty')
+        with self.patch_urlread(error=IOError('bad wolf')):
+            with self.assertRaises(IOError) as ctx:
+                charmstore.get_reference(ref, 'meta/id')
+        expected_error = (
+            'cannot communicate with the charm store at '
+            '{}trusty/django/meta/id: '
+            'bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+    def test_not_found_error(self):
+        # A charmstore.NotFoundError is raised if the reference is not found
+        # in the charm store.
+        ref = references.Reference.from_jujucharms_url('django')
+        with self.patch_urlread(error=netutils.NotFoundError('bad wolf')):
+            with self.assertRaises(charmstore.NotFoundError) as ctx:
+                charmstore.get_reference(ref, '/no/such')
+        expected_error = (
+            'charm store resource not found at '
+            '{}bundle/django/no/such: '
+            'bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+
+class TestResolve(helpers.UrlReadTestsMixin, unittest.TestCase):
+
+    contents = json.dumps({
+        'Id': 'cs:trusty/juju-gui-42',
+        'Series': 'trusty',
+        'Name': 'juju-gui',
+        'Revision': 42,
+    })
+
+    def test_resolved(self):
+        # The resolved entity id is correctly returned.
+        with self.patch_urlread(contents=self.contents) as mock_urlread:
+            entity_id = charmstore.resolve('juju-gui')
+        self.assertEqual('cs:trusty/juju-gui-42', entity_id)
+        mock_urlread.assert_called_once_with(
+            settings.CHARMSTORE_API + 'juju-gui/meta/id')
+
+    def test_resolved_with_series(self):
+        # The resolved entity id is correctly returned when the entity series
+        # is specified.
+        with self.patch_urlread(contents=self.contents) as mock_urlread:
+            entity_id = charmstore.resolve('django', series='vivid')
+        self.assertEqual('cs:trusty/juju-gui-42', entity_id)
+        mock_urlread.assert_called_once_with(
+            settings.CHARMSTORE_API + 'vivid/django/meta/id')
+
+    def test_io_error(self):
+        # IOErrors are properly propagated.
+        with self.patch_urlread(error=IOError('bad wolf')):
+            with self.assertRaises(IOError) as ctx:
+                charmstore.resolve('django')
+        expected_error = (
+            'cannot communicate with the charm store at '
+            '{}django/meta/id: bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+    def test_not_found_error(self):
+        # Not found errors are properly propagated.
+        with self.patch_urlread(error=netutils.NotFoundError('bad wolf')):
+            with self.assertRaises(charmstore.NotFoundError) as ctx:
+                charmstore.resolve('django', series='trusty')
+        expected_error = (
+            'charm store resource not found at '
+            '{}trusty/django/meta/id: '
+            'bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+    def test_value_error(self):
+        # A ValueError is raised if the API response is not valid.
+        contents = json.dumps({'invalid': {}})
+        with self.patch_urlread(contents=contents):
+            with self.assertRaises(ValueError) as ctx:
+                charmstore.resolve('juju-gui', series='trusty')
+        self.assertEqual(
+            'unable to resolve entity id trusty/juju-gui',
+            bytes(ctx.exception))
+
+
+class TestGetBundleData(
+        helpers.BundleFileTestsMixin, helpers.UrlReadTestsMixin,
+        unittest.TestCase):
+
+    def test_data_retrieved(self):
+        # The bundle data is correctly retrieved and parsed.
+        ref = references.Reference.from_jujucharms_url('django/42')
+        with self.patch_urlread(contents=self.bundle_content) as mock_urlread:
+            data = charmstore.get_bundle_data(ref)
+        self.assertEqual(self.bundle_data, data)
+        mock_urlread.assert_called_once_with(
+            settings.CHARMSTORE_API + 'bundle/django-42/archive/bundle.yaml')
+
+    def test_io_error(self):
+        # IOErrors are properly propagated.
+        ref = references.Reference.from_jujucharms_url('mediawiki-single')
+        with self.patch_urlread(error=IOError('bad wolf')):
+            with self.assertRaises(IOError) as ctx:
+                charmstore.get_bundle_data(ref)
+        expected_error = (
+            'cannot communicate with the charm store at '
+            '{}bundle/mediawiki-single/archive/bundle.yaml: '
+            'bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+    def test_not_found_error(self):
+        # Not found errors are properly propagated.
+        ref = references.Reference.from_jujucharms_url('no-such')
+        with self.patch_urlread(error=netutils.NotFoundError('bad wolf')):
+            with self.assertRaises(charmstore.NotFoundError) as ctx:
+                charmstore.get_bundle_data(ref)
+        expected_error = (
+            'charm store resource not found at '
+            '{}bundle/no-such/archive/bundle.yaml: '
+            'bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+    def test_value_error_invalid_data(self):
+        # A ValueError is raised if the API response is not valid.
+        ref = references.Reference.from_jujucharms_url('u/who/django')
+        with self.patch_urlread(contents='invalid: data:'):
+            with self.assertRaises(ValueError) as ctx:
+                charmstore.get_bundle_data(ref)
+        self.assertIn(
+            'unable to parse the bundle content', bytes(ctx.exception))
+
+    def test_value_error_not_a_bundle(self):
+        # A ValueError is raised if the API response is not valid.
+        ref = references.Reference.from_jujucharms_url('django/trusty/47')
+        with self.assertRaises(ValueError) as ctx:
+            charmstore.get_bundle_data(ref)
+        self.assertEqual(
+            'expected a bundle, provided charm cs:trusty/django-47',
+            bytes(ctx.exception))
+
+
+class TestGetLegacyBundleData(
+        helpers.BundleFileTestsMixin, helpers.UrlReadTestsMixin,
+        unittest.TestCase):
+
+    def test_data_retrieved(self):
+        # The legacy bundle data is correctly retrieved and parsed.
+        ref = references.Reference.from_jujucharms_url('u/who/django/42')
+        contents = self.legacy_bundle_content
+        with self.patch_urlread(contents=contents) as mock_urlread:
+            data = charmstore.get_legacy_bundle_data(ref)
+        self.assertEqual(self.legacy_bundle_data, data)
+        mock_urlread.assert_called_once_with(
+            settings.CHARMSTORE_API +
+            '~who/bundle/django-42/archive/bundles.yaml.orig')
+
+    def test_io_error(self):
+        # IOErrors are properly propagated.
+        ref = references.Reference.from_jujucharms_url('mediawiki-single')
+        with self.patch_urlread(error=IOError('bad wolf')):
+            with self.assertRaises(IOError) as ctx:
+                charmstore.get_legacy_bundle_data(ref)
+        expected_error = (
+            'cannot communicate with the charm store at '
+            '{}bundle/mediawiki-single/archive/bundles.yaml.orig: '
+            'bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+    def test_not_found_error(self):
+        # Not found errors are properly propagated.
+        ref = references.Reference.from_jujucharms_url('no-such')
+        with self.patch_urlread(error=netutils.NotFoundError('bad wolf')):
+            with self.assertRaises(charmstore.NotFoundError) as ctx:
+                charmstore.get_legacy_bundle_data(ref)
+        expected_error = (
+            'charm store resource not found at '
+            '{}bundle/no-such/archive/bundles.yaml.orig: '
+            'bad wolf'.format(settings.CHARMSTORE_API))
+        self.assertEqual(expected_error, bytes(ctx.exception))
+
+    def test_value_error_invalid_data(self):
+        # A ValueError is raised if the API response is not valid.
+        ref = references.Reference.from_jujucharms_url('u/who/django')
+        with self.patch_urlread(contents='invalid: data:'):
+            with self.assertRaises(ValueError) as ctx:
+                charmstore.get_legacy_bundle_data(ref)
+        self.assertIn(
+            'unable to parse the bundle content', bytes(ctx.exception))
+
+    def test_value_error_not_a_bundle(self):
+        # A ValueError is raised if the API response is not valid.
+        ref = references.Reference.from_jujucharms_url('django/trusty/47')
+        with self.assertRaises(ValueError) as ctx:
+            charmstore.get_legacy_bundle_data(ref)
+        self.assertEqual(
+            'expected a bundle, provided charm cs:trusty/django-47',
+            bytes(ctx.exception))
+
+
+class TestLoadBundleYaml(helpers.BundleFileTestsMixin, unittest.TestCase):
+
+    def test_valid_bundle_content(self):
+        # The bundle content is correctly loaded.
+        data = charmstore.load_bundle_yaml(self.bundle_content)
+        self.assertEqual(self.bundle_data, data)
+
+    def test_invalid_bundle_content(self):
+        # A ValueError is raised if the bundle content is not valid.
+        with self.assertRaises(ValueError) as ctx:
+            charmstore.load_bundle_yaml('invalid: content:')
+        self.assertIn(
+            'unable to parse the bundle content', bytes(ctx.exception))

=== modified file 'quickstart/tests/test_netutils.py'
--- quickstart/tests/test_netutils.py	2015-01-12 15:07:51 +0000
+++ quickstart/tests/test_netutils.py	2015-03-10 10:19:09 +0000
@@ -19,7 +19,6 @@
 from __future__ import unicode_literals
 
 import httplib
-import json
 import socket
 import unittest
 import urllib2
@@ -103,42 +102,12 @@
             netutils.check_listening('1.2.3.4:17070')
 
 
-class TestGetCharmUrl(helpers.UrlReadTestsMixin, unittest.TestCase):
-
-    def test_charm_url(self):
-        # The Juju GUI charm URL is correctly returned.
-        contents = json.dumps({
-            'Id': 'cs:trusty/juju-gui-42',
-            'Series': 'trusty',
-            'Name': 'juju-gui',
-            'Revision': 42,
-        })
-        with self.patch_urlread(contents=contents) as mock_urlread:
-            charm_url = netutils.get_charm_url('trusty')
-        self.assertEqual('cs:trusty/juju-gui-42', charm_url)
-        mock_urlread.assert_called_once_with(
-            'https://api.jujucharms.com/charmstore/v4/trusty/juju-gui/meta/id')
-
-    def test_io_error(self):
-        # IOErrors are properly propagated.
-        with self.patch_urlread(error=True) as mock_urlread:
-            with self.assertRaises(IOError) as context_manager:
-                netutils.get_charm_url('precise')
-        mock_urlread.assert_called_once_with(
-            'https://api.jujucharms.com/charmstore/v4/precise/juju-gui/meta/id'
-        )
-        self.assertEqual('bad wolf', bytes(context_manager.exception))
-
-    def test_value_error(self):
-        # A ValueError is raised if the API response is not valid.
-        contents = json.dumps({'invalid': {}})
-        with self.patch_urlread(contents=contents) as mock_urlread:
-            with self.assertRaises(ValueError) as context_manager:
-                netutils.get_charm_url('trusty')
-        mock_urlread.assert_called_once_with(
-            'https://api.jujucharms.com/charmstore/v4/trusty/juju-gui/meta/id')
-        self.assertEqual(
-            'unable to find the charm URL', bytes(context_manager.exception))
+class TestNotFoundError(unittest.TestCase):
+
+    def test_error_message(self):
+        # The error message can be properly retrieved.
+        err = netutils.NotFoundError(b'bad wolf')
+        self.assertEqual('bad wolf', bytes(err))
 
 
 class TestUrlread(unittest.TestCase):
@@ -191,7 +160,14 @@
         self.assertIsInstance(contents, unicode)
         mock_urlopen.assert_called_once_with('http://example.com/path/')
 
-    def test_errors(self):
+    def test_logging(self):
+        # The request URL is properly logged.
+        expected_log = 'sending HTTP GET request to http://example.com/path/'
+        with helpers.assert_logs([expected_log], level='debug'):
+            with self.patch_urlopen():
+                netutils.urlread('http://example.com/path/')
+
+    def test_io_errors(self):
         # An IOError is raised if an error occurs connecting to the API.
         errors = {
             'httplib HTTPException': httplib.HTTPException,
@@ -201,7 +177,16 @@
         for message, exception_class in errors.items():
             exception = exception_class(message)
             with self.patch_urlopen(error=exception) as mock_urlopen:
-                with self.assertRaises(IOError) as context_manager:
+                with self.assertRaises(IOError) as ctx:
                     netutils.urlread('http://example.com/path/')
             mock_urlopen.assert_called_once_with('http://example.com/path/')
-            self.assertEqual(message, bytes(context_manager.exception))
+            self.assertEqual(message, bytes(ctx.exception))
+
+    def test_not_found_error(self):
+        # A netutils.NotFoundError is raised if the request returns a 404 not
+        # found response.
+        exception = urllib2.HTTPError('url', 404, 'bad wolf', None, None)
+        with self.patch_urlopen(error=exception):
+            with self.assertRaises(netutils.NotFoundError) as ctx:
+                netutils.urlread('http://example.com/path/')
+        self.assertEqual('HTTP Error 404: bad wolf', bytes(ctx.exception))

=== modified file 'quickstart/utils.py'
--- quickstart/utils.py	2015-02-09 12:34:33 +0000
+++ quickstart/utils.py	2015-03-10 10:19:09 +0000
@@ -88,7 +88,7 @@
     The banner is returned as a string, e.g.:
 
         # This file has been generated by juju quickstart v0.42.0
-        # in date 2013-12-31 23:59:00 UTC.
+        # at 2013-12-31 23:59:00 UTC.
     """
     now = datetime.datetime.utcnow()
     formatted_date = now.isoformat(sep=b' ').split('.')[0]