Merge lp:~dimitern/juju-core/330-general-improvements into lp:~go-bot/juju-core/trunk

[Dimiter Naydenov]

various: production code and logging improvements

Fixed 4 slightly annoying, unrelated minor issues:
1. Refactored production code not to depend on
gocheck (manual provider and apiserver/charms).
As a drive-by fix, I refactored environs/manual
tests to be proper black-box unit tests, rather
than white-box tests in the manual package.
2. Fixed and improved tools download output via
curl and better handling of errors.
3. Added debug logging to environs sshstorage and
httpstorage, and the manual provider.
4. Reformatted and unified generated boilerplate
config across all providers, fixing bug #1221134 in
the process (-e env not properly explained).

Changes tested live with a manual and local environs
with added manually provisioned machines.

While testing I found out and filed this bug #1291292
(basically manual bootstrap is broken with ssl-hostname-
verification set to false, but this is due to a deeper
issue).

I tried to unify and fix when all commands report
"environment not bootstrapped", consistently across
providers, but it turned out like more work than I
originally thought, so I'll do a follow-up on that
(which will entail dropping support for 1.14 environments
without a .jenv file).

https://codereview.appspot.com/72860045/

R=jameinel, rogpeppe

various: production code and logging improvements

Fixed 4 slightly annoying, unrelated minor issues:
1. Refactored production code not to depend on
gocheck (manual provider and apiserver/charms).
As a drive-by fix, I refactored environs/manual
tests to be proper black-box unit tests, rather
than white-box tests in the manual package.
2. Fixed and improved tools download output via
curl and better handling of errors.
3. Added debug logging to environs sshstorage and
httpstorage, and the manual provider.
4. Reformatted and unified generated boilerplate
config across all providers, fixing bug #1221134 in
the process (-e env not properly explained).

Changes tested live with a manual and local environs
with added manually provisioned machines.

While testing I found out and filed this bug #1291292
(basically manual bootstrap is broken with ssl-hostname-
verification set to false, but this is due to a deeper
issue).

I tried to unify and fix when all commands report
"environment not bootstrapped", consistently across
providers, but it turned out like more work than I
originally thought, so I'll do a follow-up on that
(which will entail dropping support for 1.14 environments
without a .jenv file).

https://codereview.appspot.com/72860045/

[Dimiter Naydenov (dimitern)]

Reviewers: mp+210108_code.launchpad.net,

Message:
Please take a look.

Description:
various: production code and logging improvements

Fixed 3 slightly annoying issues:
1. Refactored production code not to depend on
gocheck (manual provider and apiserver/charms).
2. Fixed and improved tools download output via
curl and better handling of errors.
3. Added debug logging to environs sshstorage and
httpstorage, and the manual provider.

Changes tested live with a manual and local environs
with added manually provisioned machines.

https://code.launchpad.net/~dimitern/juju-core/330-general-improvements/+merge/210108

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/72860045/

Affected files (+207, -177 lines):
   A [revision details]
   M environs/bootstrap/state.go
   M environs/cloudinit/cloudinit.go
   M environs/cloudinit/cloudinit_test.go
   M environs/httpstorage/storage.go
   M environs/manual/bootstrap_test.go
   D environs/manual/fakessh.go
   M environs/manual/init.go
   M environs/manual/init_test.go
   M environs/manual/provisioner_test.go
   M environs/sshstorage/storage.go
   M environs/testing/storage.go
   M provider/manual/environ.go
   M state/apiserver/charms.go
   M state/apiserver/charms_test.go
   M state/apiserver/client/client_test.go
   M utils/http.go
   M utils/ssh/ssh.go
   M utils/ssh/testing/fakessh.go
   A utils/storage/storage.go

[Andrew Wilkins (axwalk)]

https://codereview.appspot.com/72860045/diff/1/environs/manual/init.go
File environs/manual/init.go (right):

https://codereview.appspot.com/72860045/diff/1/environs/manual/init.go#newcode27
environs/manual/init.go:27: cmd.Stdin =
strings.NewReader(ssh.CheckProvisionedScript)
utils/ssh is for *general ssh* things. These scripts do not belong
there. I see the case for exposing them, for the test code, but please
keep them out of utils/ssh.

https://codereview.appspot.com/72860045/

[Dimiter Naydenov (dimitern)]

Please take a look.

https://codereview.appspot.com/72860045/diff/1/environs/manual/init.go
File environs/manual/init.go (right):

https://codereview.appspot.com/72860045/diff/1/environs/manual/init.go#newcode27
environs/manual/init.go:27: cmd.Stdin =
strings.NewReader(ssh.CheckProvisionedScript)
On 2014/03/10 01:08:52, axw wrote:
> utils/ssh is for *general ssh* things. These scripts do not belong
there. I see
> the case for exposing them, for the test code, but please keep them
out of
> utils/ssh.

Thanks for the explanation! As discussed on IRC, moved fakessh to
environs/manual/testing and the scripts in environs/manual.

https://codereview.appspot.com/72860045/

[Andrew Wilkins (axwalk)]

Thanks for the changes, and cleaning the code up. Looks much better now
:)

Just a few little things...

https://codereview.appspot.com/72860045/diff/2/environs/bootstrap/state.go
File environs/bootstrap/state.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/bootstrap/state.go#newcode43
environs/bootstrap/state.go:43: logger.Debugf("putting %q to boostrap
storage %#v", StateFile, storage)
s/boostrap/bootstrap/

https://codereview.appspot.com/72860045/diff/2/environs/configstore/disk.go
File environs/configstore/disk.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/configstore/disk.go#newcode29
environs/configstore/disk.go:29: // Exists returns if the default
disk-based environment config storage
returns true iff?

https://codereview.appspot.com/72860045/diff/2/environs/open.go
File environs/open.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/open.go#newcode93
environs/open.go:93: if source != ConfigFromInfo {
Is this right? You're changing behaviour here; the old code only updated
the error if the existing error was non-nil.

https://codereview.appspot.com/72860045/

[John A Meinel (jameinel)]

overall LGTM, though I did have some comments.

https://codereview.appspot.com/72860045/diff/2/environs/open.go
File environs/open.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/open.go#newcode93
environs/open.go:93: if source != ConfigFromInfo {
On 2014/03/11 04:43:22, axw wrote:
> Is this right? You're changing behaviour here; the old code only
updated the
> error if the existing error was non-nil.

So I think this means that if an environ was bootstrapped with 1.14 then
you wouldn't have the .jenv. However, we don't need to preserve compat
with 1.14 if it makes things cleaner.
1.16+ should have always created a .jenv when it bootstrapped.

https://codereview.appspot.com/72860045/diff/2/errors/errors.go
File errors/errors.go (right):

https://codereview.appspot.com/72860045/diff/2/errors/errors.go#newcode6
errors/errors.go:6: import "fmt"
I actually prefer the other form.
Because as soon as we need 2 imports, we want to do it anyway, and that
avoids having the noise of what actually changed confused with the extra
lines.

Certainly not worth going around again, just wanted to give my feeling
that multiline imports are still reasonable to use when you only have
one.

https://codereview.appspot.com/72860045/diff/2/juju/api.go
File juju/api.go (right):

https://codereview.appspot.com/72860045/diff/2/juju/api.go#newcode107
juju/api.go:107: }
I didn't see a change in apiconn_test.go that would actually exercise
this. Can you add one so we don't break it by accident?

https://codereview.appspot.com/72860045/

[Roger Peppe (rogpeppe)]

review so far.

https://codereview.appspot.com/72860045/diff/2/environs/bootstrap/state.go
File environs/bootstrap/state.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/bootstrap/state.go#newcode43
environs/bootstrap/state.go:43: logger.Debugf("putting %q to boostrap
storage %#v", StateFile, storage)
I'd be tempted lose this line entirely - it would be nicer to have
better debugging on the storage put methods. But if we keep it, I'd lose
the %#v - it could be very noisy. %T might be better.

https://codereview.appspot.com/72860045/diff/2/environs/configstore/disk.go
File environs/configstore/disk.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/configstore/disk.go#newcode29
environs/configstore/disk.go:29: // Exists returns if the default
disk-based environment config storage
On 2014/03/11 04:43:22, axw wrote:
> returns true iff?

The canonical form is:

// Exists reports whether ...

I'm having difficulty understanding why this function exists though.
For a start, the name "Exists" doesn't imply in which configstore
implementation the environment info exists.

The only place that it's used in the production code, it seems that it
could be trivially replaced with a call to ReadInfo.

It's easy to "mock" the existence of the environ info without resorting
to replacing this function - just create the info. That way our tests
don't become unnecessarily dependent on internal details of the code.

I suggest removing this functon.

https://codereview.appspot.com/72860045/

[Roger Peppe (rogpeppe)]

A few more suggestions. It's great that we've lost the testing
dependencies from the code, but there are a few pieces I'm concerned
about.

https://codereview.appspot.com/72860045/diff/2/environs/manual/init.go
File environs/manual/init.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/manual/init.go#newcode21
environs/manual/init.go:21: const DetectionScript = `#!/bin/bash
I don't quite see why this and CheckProvisionedScript now need to be
exported.

The only reason that I can see is that manual/testing is a separate
package. Given that it's only imported from environs/manual, I don't see
what value that adds. I'd prefer to avoid unnecessary twisty
dependencies.

https://codereview.appspot.com/72860045/diff/2/environs/open.go
File environs/open.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/open.go#newcode93
environs/open.go:93: if source != ConfigFromInfo {
I think andrew has a point here - if ConfigForName returns a useful
error message, it will be lost.

If we're going to deprecate reading config from environments.yaml,
that deserves its own CL, as there is a substantial amount
of cleanup that can and should happen then.

https://codereview.appspot.com/72860045/diff/2/errors/errors.go
File errors/errors.go (right):

https://codereview.appspot.com/72860045/diff/2/errors/errors.go#newcode6
errors/errors.go:6: import "fmt"
On 2014/03/11 10:06:33, jameinel wrote:
> I actually prefer the other form.
> Because as soon as we need 2 imports, we want to do it anyway, and
that avoids
> having the noise of what actually changed confused with the extra
lines.

> Certainly not worth going around again, just wanted to give my feeling
that
> multiline imports are still reasonable to use when you only have one.

I think we shouldn't care too much. If goimports inserts a single
import, it does so without the ( ), and that's a useful enough tool that
I think it's reasonable to allow its default output through.

https://codereview.appspot.com/72860045/diff/2/juju/api.go
File juju/api.go (right):

https://codereview.appspot.com/72860045/diff/2/juju/api.go#newcode104
juju/api.go:104: if !configstore.Exists(envName) {
As I mentioned in earlier comments, I don't think this is right.

If we want to make it so that we can't open the API without a .jenv
file, that should be a deeper change in this file - there is a fair
amount of logic that can be simplified here.

https://codereview.appspot.com/72860045/diff/2/juju/apiconn_test.go
File juju/apiconn_test.go (right):

https://codereview.appspot.com/72860045/diff/2/juju/apiconn_test.go#newcode257
juju/apiconn_test.go:257: bootstrapEnv(c, &cs.CleanupSuite,
coretesting.SampleEnvName, store)
This is changed only because of the unnecessary Exists patching AFAICS.

https://codereview.appspot.com/72860045/diff/2/juju/conn_test.go
File juju/conn_test.go (right):

https://codereview.appspot.com/72860045/diff/2/juju/conn_test.go#newcode88
juju/conn_test.go:88: s.PatchValue(&configstore.Exists, func(name
string) bool {
This seems wrong, as I've mentioned earlier.
It is not at all clear how patching this value will
affect any of the lines of code below - it's truly
action-at-a-distan...

[Dimiter Naydenov (dimitern)]

Please take a look.

https://codereview.appspot.com/72860045/diff/2/environs/bootstrap/state.go
File environs/bootstrap/state.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/bootstrap/state.go#newcode43
environs/bootstrap/state.go:43: logger.Debugf("putting %q to boostrap
storage %#v", StateFile, storage)
On 2014/03/11 04:43:22, axw wrote:
> s/boostrap/bootstrap/

Done.

https://codereview.appspot.com/72860045/diff/2/environs/bootstrap/state.go#newcode43
environs/bootstrap/state.go:43: logger.Debugf("putting %q to boostrap
storage %#v", StateFile, storage)
On 2014/03/11 11:57:46, rog wrote:
> I'd be tempted lose this line entirely - it would be nicer to have
better
> debugging on the storage put methods. But if we keep it, I'd lose the
%#v - it
> could be very noisy. %T might be better.

Changed to %T as suggested.

https://codereview.appspot.com/72860045/diff/2/environs/configstore/disk.go
File environs/configstore/disk.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/configstore/disk.go#newcode29
environs/configstore/disk.go:29: // Exists returns if the default
disk-based environment config storage
On 2014/03/11 04:43:22, axw wrote:
> returns true iff?

I dropped this entirely.

https://codereview.appspot.com/72860045/diff/2/environs/configstore/disk.go#newcode29
environs/configstore/disk.go:29: // Exists returns if the default
disk-based environment config storage
On 2014/03/11 11:57:46, rog wrote:
> On 2014/03/11 04:43:22, axw wrote:
> > returns true iff?

> The canonical form is:

> // Exists reports whether ...

> I'm having difficulty understanding why this function exists though.
> For a start, the name "Exists" doesn't imply in which configstore
implementation
> the environment info exists.

> The only place that it's used in the production code, it seems that it
could be
> trivially replaced with a call to ReadInfo.

> It's easy to "mock" the existence of the environ info without
resorting to
> replacing this function - just create the info. That way our tests
don't become
> unnecessarily dependent on internal details of the code.

> I suggest removing this functon.

I dropped this entirely.

https://codereview.appspot.com/72860045/diff/2/environs/manual/init.go
File environs/manual/init.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/manual/init.go#newcode21
environs/manual/init.go:21: const DetectionScript = `#!/bin/bash
On 2014/03/11 12:59:34, rog wrote:
> I don't quite see why this and CheckProvisionedScript now need to be
exported.

> The only reason that I can see is that manual/testing is a separate
package.
> Given that it's only imported from environs/manual, I don't see what
value that
> adds. I'd prefer to avoid unnecessary twisty dependencies.

DetectionScript and CheckProvisioned scripts need to be exported,
because they are used by both production code and tests.

https://codereview.appspot.com/72860045/diff/2/environs/open.go
File environs/open.go (right):

https://codereview.appspot.com/72860045/diff/2/environs/open.go#newcode93
environs/open.go:93: if source != ConfigFromInfo {
On 2014/03/11 10:06:33, jameinel wrote:
> On 2014/03/11 04:43:22, axw wrote:...

[John A Meinel (jameinel)]

LGTM, though you should probably wait for Roger since he had the most
comments on it.

https://codereview.appspot.com/72860045/

[Roger Peppe (rogpeppe)]

On 2014/03/12 12:13:00, jameinel wrote:
> LGTM, though you should probably wait for Roger since he had the most
comments
> on it.

LGTM, thanks!

https://codereview.appspot.com/72860045/

[Dimiter Naydenov (dimitern)]

Please take a look.

https://codereview.appspot.com/72860045/

=== modified file 'cmd/juju/destroyenvironment.go'
--- cmd/juju/destroyenvironment.go	2014-02-11 01:35:20 +0000
+++ cmd/juju/destroyenvironment.go	2014-03-12 12:33:26 +0000
@@ -67,7 +67,7 @@
 		}
 		answer := strings.ToLower(scanner.Text())
 		if answer != "y" && answer != "yes" {
-			return errors.New("Environment destruction aborted")
+			return errors.New("environment destruction aborted")
 		}
 	}
 	// If --force is supplied, then don't attempt to use the API.

=== modified file 'cmd/juju/destroyenvironment_test.go'
--- cmd/juju/destroyenvironment_test.go	2014-02-13 02:46:58 +0000
+++ cmd/juju/destroyenvironment_test.go	2014-03-12 12:33:26 +0000
@@ -130,7 +130,7 @@
 	// Ensure confirmation is requested if "-y" is not specified.
 	stdin.WriteString("n")
 	opc, errc := runCommand(ctx, new(DestroyEnvironmentCommand), "dummyenv")
-	c.Check(<-errc, gc.ErrorMatches, "Environment destruction aborted")
+	c.Check(<-errc, gc.ErrorMatches, "environment destruction aborted")
 	c.Check(<-opc, gc.IsNil)
 	c.Check(stdout.String(), gc.Matches, "WARNING!.*dummyenv.*\\(type: dummy\\)(.|\n)*")
 	assertEnvironNotDestroyed(c, env, s.ConfigStore)
@@ -140,7 +140,7 @@
 	stdout.Reset()
 	opc, errc = runCommand(ctx, new(DestroyEnvironmentCommand), "dummyenv")
 	c.Check(<-opc, gc.IsNil)
-	c.Check(<-errc, gc.ErrorMatches, "Environment destruction aborted")
+	c.Check(<-errc, gc.ErrorMatches, "environment destruction aborted")
 	assertEnvironNotDestroyed(c, env, s.ConfigStore)
 
 	// "--yes" passed: no confirmation request.

=== modified file 'cmd/juju/help_topics.go'
--- cmd/juju/help_topics.go	2014-03-04 14:33:43 +0000
+++ cmd/juju/help_topics.go	2014-03-12 12:33:26 +0000
@@ -47,8 +47,8 @@
 
 `
 
-const helpLocalProvider = `  
-The local provider is a linux-only Juju environment that uses LXC containers as
+const helpLocalProvider = `
+The local provider is a Linux-only Juju environment that uses LXC containers as
 a virtual cloud on the local machine.  Because of this, lxc and mongodb are
 required for the local provider to work.  If you don't already have lxc and
 mongodb installed, run the following commands:
@@ -64,14 +64,14 @@
 Now you need to tell Juju to use the local provider and then bootstrap:
 
   juju switch local
-  sudo juju bootstrap
+  juju bootstrap
 
 The first time this runs it might take a bit, as it's doing a netinstall for
 the container, it's around a 300 megabyte download. Subsequent bootstraps
-should be much quicker. 'sudo' is needed because only root can create LXC
-containers. After the initial bootstrap, you do not need 'sudo' anymore,
-except to 'sudo juju destroy-environment' when you want to tear everything
-down.
+should be much quicker. You'll be asked for your 'sudo' password, which is
+needed because only root can create LXC containers. When you need to destroy
+the environment, do 'juju destroy-environment local' and you could be asked
+for your 'sudo' password again.
 
 You deploy charms from the charm store using the following commands:
 
@@ -90,31 +90,35 @@
 
   sample_openstack:
     type: openstack
+
     # Specifies whether the use of a floating IP address is required to
     # give the nodes a public IP address. Some installations assign public
     # IP addresses by default without requiring a floating IP address.
     # use-floating-ip: false
+
     # Specifies whether new machine instances should have the "default"
     # Openstack security group assigned.
     # use-default-secgroup: false
-    admin-secret: 13850d1b9786065cadd0f477e8c97cd3
-    # Globally unique swift bucket name
-    control-bucket: juju-fd6ab8d02393af742bfbe8b9629707ee
+
     # Usually set via the env variable OS_AUTH_URL, but can be specified here
     # auth-url: https://yourkeystoneurl:443/v2.0/
-    # override if your workstation is running a different series to which
-    # you are deploying
+
     # The following are used for userpass authentication (the default)
-    auth-mode: userpass
+    # auth-mode: userpass
+
     # Usually set via the env variable OS_USERNAME, but can be specified here
     # username: <your username>
+
     # Usually set via the env variable OS_PASSWORD, but can be specified here
     # password: <secret>
+
     # Usually set via the env variable OS_TENANT_NAME, but can be specified here
     # tenant-name: <your tenant name>
+
     # Usually set via the env variable OS_REGION_NAME, but can be specified here
     # region: <your region>
 
+If you have set the described OS_* environment variables, you only need "type:".
 References:
 
   http://juju.ubuntu.com/docs/provider-configuration-openstack.html
@@ -125,7 +129,7 @@
 This answer is for generic OpenStack support, if you're using an OpenStack-based
 provider check these questions out for provider-specific information:
 
-  https://juju.ubuntu.com/docs/config-hpcloud.html 
+  https://juju.ubuntu.com/docs/config-hpcloud.html
 
 `
 
@@ -141,11 +145,8 @@
     type: ec2
     # access-key: YOUR-ACCESS-KEY-GOES-HERE
     # secret-key: YOUR-SECRET-KEY-GOES-HERE
-    control-bucket: juju-faefb490d69a41f0a3616a4808e0766b
-    admin-secret: 81a1e7429e6847c4941fda7591246594
 
-See the EC2 provider documentation[2] for more options. The S3 bucket does not
-need to exist already.
+See the EC2 provider documentation[2] for more options.
 
 Note If you already have an AWS account, you can determine your access key by
 visiting your account page[3], clicking "Security Credentials" and then clicking
@@ -183,7 +184,7 @@
 
 See the online help for more information:
 
-  https://juju.ubuntu.com/docs/config-hpcloud.html 
+  https://juju.ubuntu.com/docs/config-hpcloud.html
 `
 
 const helpAzureProvider = `
@@ -191,17 +192,22 @@
 
   sample_azure:
     type: azure
+
     # Location for instances, e.g. West US, North Europe.
     location: West US
+
     # http://msdn.microsoft.com/en-us/library/windowsazure
     # Windows Azure Management info.
     management-subscription-id: 886413e1-3b8a-5382-9b90-0c9aee199e5d
     management-certificate-path: /home/me/azure.pem
+
     # Windows Azure Storage info.
     storage-account-name: juju0useast0
+
     # Override OS image selection with a fixed image for all deployments.
     # Most useful for developers.
     # force-image-name: b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-13_10-amd64-server-DEVELOPMENT-20130713-Juju_ALPHA-en-us-30GB
+
     # Pick a simplestreams stream to select OS images from: daily or released
     # images, or any other stream available on simplestreams.  Leave blank for
     # released images.
@@ -219,7 +225,7 @@
   https://juju.ubuntu.com/docs/config-azure.html
 `
 
-const helpConstraints = `  
+const helpConstraints = `
 Constraints constrain the possible instances that may be started by juju
 commands. They are usually passed as a flag to commands that provision a new
 machine (such as bootstrap, deploy, and add-machine).
@@ -272,7 +278,7 @@
       T terabytes (1024 gigabytes)
       P petabytes (1024 terabytes)
 
-root-disk 
+root-disk
    Root-Disk is a float that defines the amount of space in megabytes that must
    be available in the machine's root partition.  For providers that have
    configurable root disk sizes (such as EC2) an instance with the specified
@@ -280,8 +286,8 @@
    defaults to megabytes and may be specified in the same manner as the mem
    constraint.
 
-container 
-   Container defines that the machine must be a container of the specified type. 
+container
+   Container defines that the machine must be a container of the specified type.
    A container of that type may be created by juju to fulfill the request.
    Currently supported containers:
       none - (default) no container
@@ -294,7 +300,7 @@
    roughly, a single 2007-era Xeon).  Cpu-power is currently only supported by
    the Amazon EC2 environment.
 
-tags    
+tags
    Tags defines the list of tags that the machine must have applied to it.
    Multiple tags must be delimited by a comma. Tags are currently only supported
    by the MaaS environment.
@@ -374,7 +380,7 @@
   other, and the way in which the topology of Services is assembled. The Charm
   defines which Relations a given Service may establish, and what kind of
   interface these Relations require.
- 
+
   In many cases, the establishment of a Relation will result into an actual TCP
   connection being created between the Service Units, but that's not necessarily
   the case. Relations may also be established to inform Services of

=== modified file 'environs/boilerplate_config.go'
--- environs/boilerplate_config.go	2013-10-11 14:05:04 +0000
+++ environs/boilerplate_config.go	2014-03-12 12:33:26 +0000
@@ -12,24 +12,36 @@
 )
 
 var configHeader = `
-# This is the Juju config file, which you can use to specify multiple environments in which to deploy.
-# By default Juju ships AWS (default), HP Cloud, OpenStack.
-# See https://juju.ubuntu.com/docs for more information
+# This is the Juju config file, which you can use to specify multiple
+# environments in which to deploy. By default Juju ships with AWS
+# (default), HP Cloud, OpenStack, Azure, MaaS, Local and Manual
+# providers. See https://juju.ubuntu.com/docs for more information
 
-# An environment configuration must always specify at least the following information:
-#
+# An environment configuration must always specify at least the
+# following information:
 # - name (to identify the environment)
 # - type (to specify the provider)
+# In the following example the name is "myenv" and type is "ec2".
+# myenv:
+#    type: ec2
 
 # Values in <brackets> below need to be filled in by the user.
 # Optional attributes are shown commented out, with
 # a sample value or a value in <brackets>.
 
+# There are several settings supported by all environments, all of which
+# are optional and have specified default values. For more info, see the
+# Juju documentation.
+
 # The default environment is chosen when an environment is not
 # specified using any of the following, in descending order of precedence:
-#   -e or --environment command line parameter.
-#   JUJU_ENV environment variable.
-#   the juju switch command.
+#  1. -e or --environment command line parameter, passed after the command, e.g.
+#     $ juju add-unit -e myenv myservice
+#  2. By setting JUJU_ENV environment variable.
+#  3. Using the juju switch command like this:
+#     $ juju switch myenv
+#
+
 default: amazon
 
 environments:

=== modified file 'environs/bootstrap/bootstrap.go'
--- environs/bootstrap/bootstrap.go	2014-03-05 19:41:34 +0000
+++ environs/bootstrap/bootstrap.go	2014-03-12 12:33:26 +0000
@@ -71,8 +71,9 @@
 	return toolsList, nil
 }
 
-// EnsureNotBootstrapped returns null if the environment is not bootstrapped,
-// and an error if it is or if the function was not able to tell.
+// EnsureNotBootstrapped returns nil if the environment is not
+// bootstrapped, and an error if it is or if the function was not able
+// to tell.
 func EnsureNotBootstrapped(env environs.Environ) error {
 	_, err := LoadState(env.Storage())
 	// If there is no error loading the bootstrap state, then we are

=== modified file 'environs/bootstrap/state.go'
--- environs/bootstrap/state.go	2014-02-25 22:19:30 +0000
+++ environs/bootstrap/state.go	2014-03-12 12:33:26 +0000
@@ -40,6 +40,7 @@
 // putState writes the given data to the state file on the given storage.
 // The file's name is as defined in StateFile.
 func putState(storage storage.StorageWriter, data []byte) error {
+	logger.Debugf("putting %q to bootstrap storage %T", StateFile, storage)
 	return storage.Put(StateFile, bytes.NewBuffer(data), int64(len(data)))
 }
 
@@ -69,6 +70,7 @@
 
 // LoadStateFromURL reads state from the given URL.
 func LoadStateFromURL(url string, disableSSLHostnameVerification bool) (*BootstrapState, error) {
+	logger.Debugf("loading %q from %q", StateFile, url)
 	client := http.DefaultClient
 	if disableSSLHostnameVerification {
 		logger.Infof("hostname SSL verification disabled")

=== modified file 'environs/cloudinit/cloudinit.go'
--- environs/cloudinit/cloudinit.go	2014-03-11 18:33:39 +0000
+++ environs/cloudinit/cloudinit.go	2014-03-12 12:33:26 +0000
@@ -299,9 +299,9 @@
 	if strings.HasPrefix(cfg.Tools.URL, fileSchemePrefix) {
 		copyCmd = fmt.Sprintf("cp %s $bin/tools.tar.gz", shquote(cfg.Tools.URL[len(fileSchemePrefix):]))
 	} else {
-		curlCommand := "curl"
+		curlCommand := "curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s '"
 		if cfg.DisableSSLHostnameVerification {
-			curlCommand = "curl --insecure"
+			curlCommand += " --insecure"
 		}
 		copyCmd = fmt.Sprintf("%s -o $bin/tools.tar.gz %s", curlCommand, shquote(cfg.Tools.URL))
 		c.AddRunCmd(cloudinit.LogProgressCmd("Fetching tools: %s", copyCmd))

=== modified file 'environs/cloudinit/cloudinit_test.go'
--- environs/cloudinit/cloudinit_test.go	2014-03-11 18:33:39 +0000
+++ environs/cloudinit/cloudinit_test.go	2014-03-12 12:33:26 +0000
@@ -124,7 +124,7 @@
 echo 'Fetching tools.*
 bin='/var/lib/juju/tools/1\.2\.3-precise-amd64'
 mkdir -p \$bin
-curl -o \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-precise-amd64\.tgz'
+curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s ' -o \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-precise-amd64\.tgz'
 sha256sum \$bin/tools\.tar\.gz > \$bin/juju1\.2\.3-precise-amd64\.sha256
 grep '1234' \$bin/juju1\.2\.3-precise-amd64.sha256 \|\| \(echo "Tools checksum mismatch"; exit 1\)
 tar zxf \$bin/tools.tar.gz -C \$bin
@@ -195,7 +195,7 @@
 		inexactMatch: true,
 		expectScripts: `
 bin='/var/lib/juju/tools/1\.2\.3-raring-amd64'
-curl -o \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-raring-amd64\.tgz'
+curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s ' -o \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-raring-amd64\.tgz'
 sha256sum \$bin/tools\.tar\.gz > \$bin/juju1\.2\.3-raring-amd64\.sha256
 grep '1234' \$bin/juju1\.2\.3-raring-amd64.sha256 \|\| \(echo "Tools checksum mismatch"; exit 1\)
 rm \$bin/tools\.tar\.gz && rm \$bin/juju1\.2\.3-raring-amd64\.sha256
@@ -243,7 +243,7 @@
 echo 'Fetching tools.*
 bin='/var/lib/juju/tools/1\.2\.3-linux-amd64'
 mkdir -p \$bin
-curl -o \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-linux-amd64\.tgz'
+curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s ' -o \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-linux-amd64\.tgz'
 sha256sum \$bin/tools\.tar\.gz > \$bin/juju1\.2\.3-linux-amd64\.sha256
 grep '1234' \$bin/juju1\.2\.3-linux-amd64.sha256 \|\| \(echo "Tools checksum mismatch"; exit 1\)
 tar zxf \$bin/tools.tar.gz -C \$bin
@@ -324,7 +324,7 @@
 		},
 		inexactMatch: true,
 		expectScripts: `
-curl --insecure -o \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-linux-amd64\.tgz'
+curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s ' --insecure -o \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-linux-amd64\.tgz'
 `,
 	}, {
 		// empty contraints.

=== modified file 'environs/httpstorage/storage.go'
--- environs/httpstorage/storage.go	2014-02-25 22:19:30 +0000
+++ environs/httpstorage/storage.go	2014-03-12 12:33:26 +0000
@@ -16,11 +16,15 @@
 	"strings"
 	"sync"
 
+	"github.com/juju/loggo"
+
 	"launchpad.net/juju-core/environs/storage"
 	coreerrors "launchpad.net/juju-core/errors"
 	"launchpad.net/juju-core/utils"
 )
 
+var logger = loggo.GetLogger("juju.environs.httpstorage")
+
 // storage implements the storage.Storage interface.
 type localStorage struct {
 	addr   string
@@ -46,6 +50,7 @@
 // using TLS. The client is given an authentication key,
 // which the server will verify for Put and Remove* operations.
 func ClientTLS(addr string, caCertPEM []byte, authkey string) (storage.Storage, error) {
+	logger.Debugf("using https storage at %q", addr)
 	caCerts := x509.NewCertPool()
 	if caCertPEM != nil {
 		if !caCerts.AppendCertsFromPEM(caCertPEM) {
@@ -83,6 +88,7 @@
 // responsibility to close it after use. If the name does not
 // exist, it should return a *NotFoundError.
 func (s *localStorage) Get(name string) (io.ReadCloser, error) {
+	logger.Debugf("getting %q from storage", name)
 	url, err := s.URL(name)
 	if err != nil {
 		return nil, err
@@ -167,6 +173,7 @@
 // Put reads from r and writes to the given storage file.
 // The length must be set to the total length of the file.
 func (s *localStorage) Put(name string, r io.Reader, length int64) error {
+	logger.Debugf("putting %q (len %d) to storage", name, length)
 	url, err := s.modURL(name)
 	if err != nil {
 		return err

=== modified file 'environs/jujutest/livetests.go'
--- environs/jujutest/livetests.go	2014-02-20 08:23:40 +0000
+++ environs/jujutest/livetests.go	2014-03-12 12:33:26 +0000
@@ -27,7 +27,6 @@
 	"launchpad.net/juju-core/instance"
 	"launchpad.net/juju-core/juju"
 	"launchpad.net/juju-core/juju/testing"
-	"launchpad.net/juju-core/provider/common"
 	"launchpad.net/juju-core/provider/dummy"
 	"launchpad.net/juju-core/state"
 	"launchpad.net/juju-core/state/api"
@@ -138,7 +137,7 @@
 		c.Assert(err, gc.IsNil)
 	}
 	envtesting.UploadFakeTools(c, t.Env.Storage())
-	err := common.EnsureNotBootstrapped(t.Env)
+	err := bootstrap.EnsureNotBootstrapped(t.Env)
 	c.Assert(err, gc.IsNil)
 	err = bootstrap.Bootstrap(coretesting.Context(c), t.Env, cons)
 	c.Assert(err, gc.IsNil)
@@ -382,7 +381,7 @@
 	// already up, this has been moved into the bootstrap command.
 	t.BootstrapOnce(c)
 
-	err := common.EnsureNotBootstrapped(t.Env)
+	err := bootstrap.EnsureNotBootstrapped(t.Env)
 	c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped")
 
 	c.Logf("destroy env")

=== modified file 'environs/jujutest/tests.go'
--- environs/jujutest/tests.go	2014-02-13 02:46:58 +0000
+++ environs/jujutest/tests.go	2014-03-12 12:33:26 +0000
@@ -21,7 +21,6 @@
 	"launchpad.net/juju-core/errors"
 	"launchpad.net/juju-core/instance"
 	"launchpad.net/juju-core/juju/testing"
-	"launchpad.net/juju-core/provider/common"
 	coretesting "launchpad.net/juju-core/testing"
 	jc "launchpad.net/juju-core/testing/checkers"
 	"launchpad.net/juju-core/testing/testbase"
@@ -132,7 +131,7 @@
 func (t *Tests) TestBootstrap(c *gc.C) {
 	e := t.Prepare(c)
 	envtesting.UploadFakeTools(c, e.Storage())
-	err := common.EnsureNotBootstrapped(e)
+	err := bootstrap.EnsureNotBootstrapped(e)
 	c.Assert(err, gc.IsNil)
 	err = bootstrap.Bootstrap(coretesting.Context(c), e, constraints.Value{})
 	c.Assert(err, gc.IsNil)
@@ -141,12 +140,12 @@
 	c.Check(info.Addrs, gc.Not(gc.HasLen), 0)
 	c.Check(apiInfo.Addrs, gc.Not(gc.HasLen), 0)
 
-	err = common.EnsureNotBootstrapped(e)
+	err = bootstrap.EnsureNotBootstrapped(e)
 	c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped")
 
 	e2 := t.Open(c)
 	envtesting.UploadFakeTools(c, e2.Storage())
-	err = common.EnsureNotBootstrapped(e2)
+	err = bootstrap.EnsureNotBootstrapped(e2)
 	c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped")
 
 	info2, apiInfo2, err := e2.StateInfo()
@@ -160,12 +159,12 @@
 	e3 := t.Prepare(c)
 	envtesting.UploadFakeTools(c, e3.Storage())
 
-	err = common.EnsureNotBootstrapped(e3)
+	err = bootstrap.EnsureNotBootstrapped(e3)
 	c.Assert(err, gc.IsNil)
 	err = bootstrap.Bootstrap(coretesting.Context(c), e3, constraints.Value{})
 	c.Assert(err, gc.IsNil)
 
-	err = common.EnsureNotBootstrapped(e3)
+	err = bootstrap.EnsureNotBootstrapped(e3)
 	c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped")
 }
 

=== modified file 'environs/manual/bootstrap_test.go'
--- environs/manual/bootstrap_test.go	2014-03-04 16:55:38 +0000
+++ environs/manual/bootstrap_test.go	2014-03-12 12:33:26 +0000
@@ -1,7 +1,7 @@
 // Copyright 2013 Canonical Ltd.
 // Licensed under the AGPLv3, see LICENCE file for details.
 
-package manual
+package manual_test
 
 import (
 	"fmt"
@@ -14,6 +14,7 @@
 	"launchpad.net/juju-core/environs/bootstrap"
 	"launchpad.net/juju-core/environs/cloudinit"
 	"launchpad.net/juju-core/environs/filestorage"
+	"launchpad.net/juju-core/environs/manual"
 	"launchpad.net/juju-core/environs/storage"
 	"launchpad.net/juju-core/environs/tools"
 	"launchpad.net/juju-core/instance"
@@ -59,13 +60,13 @@
 	s.env.storage = storage
 }
 
-func (s *bootstrapSuite) getArgs(c *gc.C) BootstrapArgs {
+func (s *bootstrapSuite) getArgs(c *gc.C) manual.BootstrapArgs {
 	hostname, err := os.Hostname()
 	c.Assert(err, gc.IsNil)
 	toolsList, err := tools.FindBootstrapTools(s.Conn.Environ, tools.BootstrapToolsParams{})
 	c.Assert(err, gc.IsNil)
 	arch := "amd64"
-	return BootstrapArgs{
+	return manual.BootstrapArgs{
 		Host:          hostname,
 		DataDir:       "/var/lib/juju",
 		Environ:       s.env,
@@ -83,7 +84,7 @@
 	args.Host = "ubuntu@" + args.Host
 
 	defer fakeSSH{SkipDetection: true}.install(c).Restore()
-	err := Bootstrap(args)
+	err := manual.Bootstrap(args)
 	c.Assert(err, gc.IsNil)
 
 	bootstrapState, err := bootstrap.LoadState(s.env.Storage())
@@ -91,14 +92,14 @@
 	c.Assert(
 		bootstrapState.StateInstances,
 		gc.DeepEquals,
-		[]instance.Id{BootstrapInstanceId},
+		[]instance.Id{manual.BootstrapInstanceId},
 	)
 
 	// Do it all again; this should work, despite the fact that
 	// there's a bootstrap state file. Existence for that is
 	// checked in general bootstrap code (environs/bootstrap).
 	defer fakeSSH{SkipDetection: true}.install(c).Restore()
-	err = Bootstrap(args)
+	err = manual.Bootstrap(args)
 	c.Assert(err, gc.IsNil)
 
 	// We *do* check that the machine has no juju* upstart jobs, though.
@@ -107,15 +108,15 @@
 		SkipDetection:      true,
 		SkipProvisionAgent: true,
 	}.install(c).Restore()
-	err = Bootstrap(args)
-	c.Assert(err, gc.Equals, ErrProvisioned)
+	err = manual.Bootstrap(args)
+	c.Assert(err, gc.Equals, manual.ErrProvisioned)
 }
 
 func (s *bootstrapSuite) TestBootstrapScriptFailure(c *gc.C) {
 	args := s.getArgs(c)
 	args.Host = "ubuntu@" + args.Host
 	defer fakeSSH{SkipDetection: true, ProvisionAgentExitCode: 1}.install(c).Restore()
-	err := Bootstrap(args)
+	err := manual.Bootstrap(args)
 	c.Assert(err, gc.NotNil)
 
 	// Since the script failed, the state file should have been
@@ -127,19 +128,19 @@
 func (s *bootstrapSuite) TestBootstrapEmptyDataDir(c *gc.C) {
 	args := s.getArgs(c)
 	args.DataDir = ""
-	c.Assert(Bootstrap(args), gc.ErrorMatches, "data-dir argument is empty")
+	c.Assert(manual.Bootstrap(args), gc.ErrorMatches, "data-dir argument is empty")
 }
 
 func (s *bootstrapSuite) TestBootstrapEmptyHost(c *gc.C) {
 	args := s.getArgs(c)
 	args.Host = ""
-	c.Assert(Bootstrap(args), gc.ErrorMatches, "host argument is empty")
+	c.Assert(manual.Bootstrap(args), gc.ErrorMatches, "host argument is empty")
 }
 
 func (s *bootstrapSuite) TestBootstrapNilEnviron(c *gc.C) {
 	args := s.getArgs(c)
 	args.Environ = nil
-	c.Assert(Bootstrap(args), gc.ErrorMatches, "environ argument is nil")
+	c.Assert(manual.Bootstrap(args), gc.ErrorMatches, "environ argument is nil")
 }
 
 func (s *bootstrapSuite) TestBootstrapNoMatchingTools(c *gc.C) {
@@ -147,13 +148,13 @@
 	args := s.getArgs(c)
 	args.PossibleTools = nil
 	defer fakeSSH{SkipDetection: true, SkipProvisionAgent: true}.install(c).Restore()
-	c.Assert(Bootstrap(args), gc.ErrorMatches, "possible tools is empty")
+	c.Assert(manual.Bootstrap(args), gc.ErrorMatches, "possible tools is empty")
 
 	// Non-empty list, but none that match the series/arch.
 	args = s.getArgs(c)
 	args.Series = "edgy"
 	defer fakeSSH{SkipDetection: true, SkipProvisionAgent: true}.install(c).Restore()
-	c.Assert(Bootstrap(args), gc.ErrorMatches, "no matching tools available")
+	c.Assert(manual.Bootstrap(args), gc.ErrorMatches, "no matching tools available")
 }
 
 func (s *bootstrapSuite) TestBootstrapToolsFileURL(c *gc.C) {
@@ -170,13 +171,13 @@
 }
 
 func (s *bootstrapSuite) testBootstrapToolsURL(c *gc.C, toolsURL, expectedURL string) {
-	s.PatchValue(&provisionMachineAgent, func(host string, mcfg *cloudinit.MachineConfig, w io.Writer) error {
+	s.PatchValue(manual.ProvisionMachineAgent, func(host string, mcfg *cloudinit.MachineConfig, w io.Writer) error {
 		c.Assert(mcfg.Tools.URL, gc.Equals, expectedURL)
 		return nil
 	})
 	args := s.getArgs(c)
 	args.PossibleTools[0].URL = toolsURL
 	defer fakeSSH{SkipDetection: true}.install(c).Restore()
-	err := Bootstrap(args)
+	err := manual.Bootstrap(args)
 	c.Assert(err, gc.IsNil)
 }

=== modified file 'environs/manual/export_test.go'
--- environs/manual/export_test.go	2014-01-30 03:18:20 +0000
+++ environs/manual/export_test.go	2014-03-12 12:33:26 +0000
@@ -5,4 +5,11 @@
 
 var (
 	InstanceHostAddresses = &instanceHostAddresses
+	ProvisionMachineAgent = &provisionMachineAgent
+	CheckProvisioned      = checkProvisioned
+)
+
+const (
+	DetectionScript        = detectionScript
+	CheckProvisionedScript = checkProvisionedScript
 )

=== removed file 'environs/manual/fakessh.go'
--- environs/manual/fakessh.go	2014-01-13 06:25:28 +0000
+++ environs/manual/fakessh.go	1970-01-01 00:00:00 +0000
@@ -1,87 +0,0 @@
-// Copyright 2013 Canonical Ltd.
-// Licensed under the AGPLv3, see LICENCE file for details.
-
-package manual
-
-import (
-	"strings"
-
-	gc "launchpad.net/gocheck"
-
-	"launchpad.net/juju-core/testing/testbase"
-	sshtesting "launchpad.net/juju-core/utils/ssh/testing"
-)
-
-// installDetectionFakeSSH installs a fake SSH command, which will respond
-// to the series/hardware detection script with the specified
-// series/arch.
-func installDetectionFakeSSH(c *gc.C, series, arch string) testbase.Restorer {
-	if series == "" {
-		series = "precise"
-	}
-	if arch == "" {
-		arch = "amd64"
-	}
-	detectionoutput := strings.Join([]string{
-		series,
-		arch,
-		"MemTotal: 4096 kB",
-		"processor: 0",
-	}, "\n")
-	return sshtesting.InstallFakeSSH(c, detectionScript, detectionoutput, 0)
-}
-
-// FakeSSH wraps the invocation of InstallFakeSSH based on the parameters.
-type fakeSSH struct {
-	Series string
-	Arch   string
-
-	// Provisioned should be set to true if the fakeSSH script
-	// should respond to checkProvisioned with a non-empty result.
-	Provisioned bool
-
-	// exit code for the checkProvisioned script.
-	CheckProvisionedExitCode int
-
-	// exit code for the machine agent provisioning script.
-	ProvisionAgentExitCode int
-
-	// InitUbuntuUser should be set to true if the fakeSSH script
-	// should respond to an attempt to initialise the ubuntu user.
-	InitUbuntuUser bool
-
-	// there are conditions other than error in the above
-	// that might cause provisioning to not go ahead, such
-	// as tools being missing.
-	SkipProvisionAgent bool
-
-	// detection will be skipped if the series/hardware were
-	// detected ahead of time. This should always be set to
-	// true when testing Bootstrap.
-	SkipDetection bool
-}
-
-// install installs fake SSH commands, which will respond to
-// manual provisioning/bootstrapping commands with the specified
-// output and exit codes.
-func (r fakeSSH) install(c *gc.C) testbase.Restorer {
-	var restore testbase.Restorer
-	add := func(input, output interface{}, rc int) {
-		restore = restore.Add(sshtesting.InstallFakeSSH(c, input, output, rc))
-	}
-	if !r.SkipProvisionAgent {
-		add(nil, nil, r.ProvisionAgentExitCode)
-	}
-	if !r.SkipDetection {
-		restore.Add(installDetectionFakeSSH(c, r.Series, r.Arch))
-	}
-	var checkProvisionedOutput interface{}
-	if r.Provisioned {
-		checkProvisionedOutput = "/etc/init/jujud-machine-0.conf"
-	}
-	add(checkProvisionedScript, checkProvisionedOutput, r.CheckProvisionedExitCode)
-	if r.InitUbuntuUser {
-		add("", nil, 0)
-	}
-	return restore
-}

=== added file 'environs/manual/fakessh_test.go'
--- environs/manual/fakessh_test.go	1970-01-01 00:00:00 +0000
+++ environs/manual/fakessh_test.go	2014-03-12 12:33:26 +0000
@@ -0,0 +1,156 @@
+// Copyright 2013 Canonical Ltd.
+// Licensed under the AGPLv3, see LICENCE file for details.
+
+package manual_test
+
+import (
+	"fmt"
+	"io/ioutil"
+	"path/filepath"
+	"strings"
+
+	gc "launchpad.net/gocheck"
+
+	"launchpad.net/juju-core/environs/manual"
+	"launchpad.net/juju-core/testing/testbase"
+)
+
+// sshscript should only print the result on the first execution,
+// to handle the case where it's called multiple times. On
+// subsequent executions, it should find the next 'ssh' in $PATH
+// and exec that.
+var sshscript = `#!/bin/bash --norc
+if [ ! -e "$0.run" ]; then
+    touch "$0.run"
+    if [ -e "$0.expected-input" ]; then
+        diff "$0.expected-input" -
+        exitcode=$?
+        if [ $exitcode -ne 0 ]; then
+            echo "ERROR: did not match expected input" >&2
+            exit $exitcode
+        fi
+        else
+            head >/dev/null
+        fi
+    # stdout
+    %s
+    # stderr
+    %s
+    exit %d
+else
+    export PATH=${PATH#*:}
+    exec ssh $*
+fi`
+
+// installFakeSSH creates a fake "ssh" command in a new $PATH,
+// updates $PATH, and returns a function to reset $PATH to its
+// original value when called.
+//
+// input may be:
+//    - nil (ignore input)
+//    - a string (match input exactly)
+// output may be:
+//    - nil (no output)
+//    - a string (stdout)
+//    - a slice of strings, of length two (stdout, stderr)
+func installFakeSSH(c *gc.C, input, output interface{}, rc int) testbase.Restorer {
+	fakebin := c.MkDir()
+	ssh := filepath.Join(fakebin, "ssh")
+	switch input := input.(type) {
+	case nil:
+	case string:
+		sshexpectedinput := ssh + ".expected-input"
+		err := ioutil.WriteFile(sshexpectedinput, []byte(input), 0644)
+		c.Assert(err, gc.IsNil)
+	default:
+		c.Errorf("input has invalid type: %T", input)
+	}
+	var stdout, stderr string
+	switch output := output.(type) {
+	case nil:
+	case string:
+		stdout = fmt.Sprintf("cat<<EOF\n%s\nEOF", output)
+	case []string:
+		c.Assert(output, gc.HasLen, 2)
+		stdout = fmt.Sprintf("cat<<EOF\n%s\nEOF", output[0])
+		stderr = fmt.Sprintf("cat>&2<<EOF\n%s\nEOF", output[1])
+	}
+	script := fmt.Sprintf(sshscript, stdout, stderr, rc)
+	err := ioutil.WriteFile(ssh, []byte(script), 0777)
+	c.Assert(err, gc.IsNil)
+	return testbase.PatchEnvPathPrepend(fakebin)
+}
+
+// installDetectionFakeSSH installs a fake SSH command, which will respond
+// to the series/hardware detection script with the specified
+// series/arch.
+func installDetectionFakeSSH(c *gc.C, series, arch string) testbase.Restorer {
+	if series == "" {
+		series = "precise"
+	}
+	if arch == "" {
+		arch = "amd64"
+	}
+	detectionoutput := strings.Join([]string{
+		series,
+		arch,
+		"MemTotal: 4096 kB",
+		"processor: 0",
+	}, "\n")
+	return installFakeSSH(c, manual.DetectionScript, detectionoutput, 0)
+}
+
+// fakeSSH wraps the invocation of InstallFakeSSH based on the parameters.
+type fakeSSH struct {
+	Series string
+	Arch   string
+
+	// Provisioned should be set to true if the fakeSSH script
+	// should respond to checkProvisioned with a non-empty result.
+	Provisioned bool
+
+	// exit code for the checkProvisioned script.
+	CheckProvisionedExitCode int
+
+	// exit code for the machine agent provisioning script.
+	ProvisionAgentExitCode int
+
+	// InitUbuntuUser should be set to true if the fakeSSH script
+	// should respond to an attempt to initialise the ubuntu user.
+	InitUbuntuUser bool
+
+	// there are conditions other than error in the above
+	// that might cause provisioning to not go ahead, such
+	// as tools being missing.
+	SkipProvisionAgent bool
+
+	// detection will be skipped if the series/hardware were
+	// detected ahead of time. This should always be set to
+	// true when testing Bootstrap.
+	SkipDetection bool
+}
+
+// install installs fake SSH commands, which will respond to
+// manual provisioning/bootstrapping commands with the specified
+// output and exit codes.
+func (r fakeSSH) install(c *gc.C) testbase.Restorer {
+	var restore testbase.Restorer
+	add := func(input, output interface{}, rc int) {
+		restore = restore.Add(installFakeSSH(c, input, output, rc))
+	}
+	if !r.SkipProvisionAgent {
+		add(nil, nil, r.ProvisionAgentExitCode)
+	}
+	if !r.SkipDetection {
+		restore.Add(installDetectionFakeSSH(c, r.Series, r.Arch))
+	}
+	var checkProvisionedOutput interface{}
+	if r.Provisioned {
+		checkProvisionedOutput = "/etc/init/jujud-machine-0.conf"
+	}
+	add(manual.CheckProvisionedScript, checkProvisionedOutput, r.CheckProvisionedExitCode)
+	if r.InitUbuntuUser {
+		add("", nil, 0)
+	}
+	return restore
+}

=== modified file 'environs/manual/init.go'
--- environs/manual/init.go	2014-03-11 05:55:35 +0000
+++ environs/manual/init.go	2014-03-12 12:33:26 +0000
@@ -16,6 +16,15 @@
 	"launchpad.net/juju-core/utils/ssh"
 )
 
+// detectionScript is the script to run on the remote machine to
+// detect the OS series and hardware characteristics.
+const detectionScript = `#!/bin/bash
+set -e
+lsb_release -cs
+uname -m
+grep MemTotal /proc/meminfo
+cat /proc/cpuinfo`
+
 // checkProvisionedScript is the script to run on the remote machine
 // to check if a machine has already been provisioned.
 //
@@ -132,13 +141,6 @@
 	{regexp.MustCompile("ppc64el|ppc64le"), "ppc64"},
 }
 
-const detectionScript = `#!/bin/bash
-set -e
-lsb_release -cs
-uname -m
-grep MemTotal /proc/meminfo
-cat /proc/cpuinfo`
-
 // InitUbuntuUser adds the ubuntu user if it doesn't
 // already exist, updates its ~/.ssh/authorized_keys,
 // and enables passwordless sudo for it.

=== modified file 'environs/manual/init_test.go'
--- environs/manual/init_test.go	2014-02-07 13:58:06 +0000
+++ environs/manual/init_test.go	2014-03-12 12:33:26 +0000
@@ -1,16 +1,16 @@
 // Copyright 2013 Canonical Ltd.
 // Licensed under the AGPLv3, see LICENCE file for details.
 
-package manual
+package manual_test
 
 import (
 	"strings"
 
 	gc "launchpad.net/gocheck"
 
+	"launchpad.net/juju-core/environs/manual"
 	jc "launchpad.net/juju-core/testing/checkers"
 	"launchpad.net/juju-core/testing/testbase"
-	sshtesting "launchpad.net/juju-core/utils/ssh/testing"
 )
 
 type initialisationSuite struct {
@@ -26,8 +26,8 @@
 		"MemTotal: 4096 kB",
 		"processor: 0",
 	}, "\n")
-	defer sshtesting.InstallFakeSSH(c, detectionScript, response, 0)()
-	_, series, err := DetectSeriesAndHardwareCharacteristics("whatever")
+	defer installFakeSSH(c, manual.DetectionScript, response, 0)()
+	_, series, err := manual.DetectSeriesAndHardwareCharacteristics("whatever")
 	c.Assert(err, gc.IsNil)
 	c.Assert(series, gc.Equals, "edgy")
 }
@@ -41,12 +41,12 @@
 	}, "\n")
 	// if the script fails for whatever reason, then checkProvisioned
 	// will return an error. stderr will be included in the error message.
-	defer sshtesting.InstallFakeSSH(c, detectionScript, []string{scriptResponse, "oh noes"}, 33)()
-	hc, _, err := DetectSeriesAndHardwareCharacteristics("hostname")
+	defer installFakeSSH(c, manual.DetectionScript, []string{scriptResponse, "oh noes"}, 33)()
+	hc, _, err := manual.DetectSeriesAndHardwareCharacteristics("hostname")
 	c.Assert(err, gc.ErrorMatches, "rc: 33 \\(oh noes\\)")
 	// if the script doesn't fail, stderr is simply ignored.
-	defer sshtesting.InstallFakeSSH(c, detectionScript, []string{scriptResponse, "non-empty-stderr"}, 0)()
-	hc, _, err = DetectSeriesAndHardwareCharacteristics("hostname")
+	defer installFakeSSH(c, manual.DetectionScript, []string{scriptResponse, "non-empty-stderr"}, 0)()
+	hc, _, err = manual.DetectSeriesAndHardwareCharacteristics("hostname")
 	c.Assert(err, gc.IsNil)
 	c.Assert(hc.String(), gc.Equals, "arch=arm cpu-cores=1 mem=4M")
 }
@@ -106,52 +106,52 @@
 	for i, test := range tests {
 		c.Logf("test %d: %s", i, test.summary)
 		scriptResponse := strings.Join(test.scriptResponse, "\n")
-		defer sshtesting.InstallFakeSSH(c, detectionScript, scriptResponse, 0)()
-		hc, _, err := DetectSeriesAndHardwareCharacteristics("hostname")
+		defer installFakeSSH(c, manual.DetectionScript, scriptResponse, 0)()
+		hc, _, err := manual.DetectSeriesAndHardwareCharacteristics("hostname")
 		c.Assert(err, gc.IsNil)
 		c.Assert(hc.String(), gc.Equals, test.expectedHc)
 	}
 }
 
 func (s *initialisationSuite) TestCheckProvisioned(c *gc.C) {
-	defer sshtesting.InstallFakeSSH(c, checkProvisionedScript, "", 0)()
-	provisioned, err := checkProvisioned("example.com")
+	defer installFakeSSH(c, manual.CheckProvisionedScript, "", 0)()
+	provisioned, err := manual.CheckProvisioned("example.com")
 	c.Assert(err, gc.IsNil)
 	c.Assert(provisioned, jc.IsFalse)
 
-	defer sshtesting.InstallFakeSSH(c, checkProvisionedScript, "non-empty", 0)()
-	provisioned, err = checkProvisioned("example.com")
+	defer installFakeSSH(c, manual.CheckProvisionedScript, "non-empty", 0)()
+	provisioned, err = manual.CheckProvisioned("example.com")
 	c.Assert(err, gc.IsNil)
 	c.Assert(provisioned, jc.IsTrue)
 
 	// stderr should not affect result.
-	defer sshtesting.InstallFakeSSH(c, checkProvisionedScript, []string{"", "non-empty-stderr"}, 0)()
-	provisioned, err = checkProvisioned("example.com")
+	defer installFakeSSH(c, manual.CheckProvisionedScript, []string{"", "non-empty-stderr"}, 0)()
+	provisioned, err = manual.CheckProvisioned("example.com")
 	c.Assert(err, gc.IsNil)
 	c.Assert(provisioned, jc.IsFalse)
 
 	// if the script fails for whatever reason, then checkProvisioned
 	// will return an error. stderr will be included in the error message.
-	defer sshtesting.InstallFakeSSH(c, checkProvisionedScript, []string{"non-empty-stdout", "non-empty-stderr"}, 255)()
-	_, err = checkProvisioned("example.com")
+	defer installFakeSSH(c, manual.CheckProvisionedScript, []string{"non-empty-stdout", "non-empty-stderr"}, 255)()
+	_, err = manual.CheckProvisioned("example.com")
 	c.Assert(err, gc.ErrorMatches, "rc: 255 \\(non-empty-stderr\\)")
 }
 
 func (s *initialisationSuite) TestInitUbuntuUserNonExisting(c *gc.C) {
-	defer sshtesting.InstallFakeSSH(c, "", "", 0)() // successful creation of ubuntu user
-	defer sshtesting.InstallFakeSSH(c, "", "", 1)() // simulate failure of ubuntu@ login
-	err := InitUbuntuUser("testhost", "testuser", "", nil, nil)
+	defer installFakeSSH(c, "", "", 0)() // successful creation of ubuntu user
+	defer installFakeSSH(c, "", "", 1)() // simulate failure of ubuntu@ login
+	err := manual.InitUbuntuUser("testhost", "testuser", "", nil, nil)
 	c.Assert(err, gc.IsNil)
 }
 
 func (s *initialisationSuite) TestInitUbuntuUserExisting(c *gc.C) {
-	defer sshtesting.InstallFakeSSH(c, "", nil, 0)()
-	InitUbuntuUser("testhost", "testuser", "", nil, nil)
+	defer installFakeSSH(c, "", nil, 0)()
+	manual.InitUbuntuUser("testhost", "testuser", "", nil, nil)
 }
 
 func (s *initialisationSuite) TestInitUbuntuUserError(c *gc.C) {
-	defer sshtesting.InstallFakeSSH(c, "", []string{"", "failed to create ubuntu user"}, 123)()
-	defer sshtesting.InstallFakeSSH(c, "", "", 1)() // simulate failure of ubuntu@ login
-	err := InitUbuntuUser("testhost", "testuser", "", nil, nil)
+	defer installFakeSSH(c, "", []string{"", "failed to create ubuntu user"}, 123)()
+	defer installFakeSSH(c, "", "", 1)() // simulate failure of ubuntu@ login
+	err := manual.InitUbuntuUser("testhost", "testuser", "", nil, nil)
 	c.Assert(err, gc.ErrorMatches, "rc: 123 \\(failed to create ubuntu user\\)")
 }

=== modified file 'environs/manual/provisioner_test.go'
--- environs/manual/provisioner_test.go	2014-02-07 13:58:06 +0000
+++ environs/manual/provisioner_test.go	2014-03-12 12:33:26 +0000
@@ -1,7 +1,7 @@
 // Copyright 2013 Canonical Ltd.
 // Licensed under the AGPLv3, see LICENCE file for details.
 
-package manual
+package manual_test
 
 import (
 	"fmt"
@@ -9,6 +9,7 @@
 
 	gc "launchpad.net/gocheck"
 
+	"launchpad.net/juju-core/environs/manual"
 	envtesting "launchpad.net/juju-core/environs/testing"
 	"launchpad.net/juju-core/instance"
 	"launchpad.net/juju-core/juju/testing"
@@ -25,10 +26,10 @@
 
 var _ = gc.Suite(&provisionerSuite{})
 
-func (s *provisionerSuite) getArgs(c *gc.C) ProvisionMachineArgs {
+func (s *provisionerSuite) getArgs(c *gc.C) manual.ProvisionMachineArgs {
 	hostname, err := os.Hostname()
 	c.Assert(err, gc.IsNil)
-	return ProvisionMachineArgs{
+	return manual.ProvisionMachineArgs{
 		Host:    hostname,
 		EnvName: "dummyenv",
 	}
@@ -50,7 +51,7 @@
 		SkipProvisionAgent: true,
 	}.install(c).Restore()
 	// Attempt to provision a machine with no tools available, expect it to fail.
-	machineId, err := ProvisionMachine(args)
+	machineId, err := manual.ProvisionMachine(args)
 	c.Assert(err, jc.Satisfies, params.IsCodeNotFound)
 	c.Assert(machineId, gc.Equals, "")
 
@@ -68,7 +69,7 @@
 			InitUbuntuUser:         true,
 			ProvisionAgentExitCode: errorCode,
 		}.install(c).Restore()
-		machineId, err = ProvisionMachine(args)
+		machineId, err = manual.ProvisionMachine(args)
 		if errorCode != 0 {
 			c.Assert(err, gc.ErrorMatches, fmt.Sprintf("rc: %d", errorCode))
 			c.Assert(machineId, gc.Equals, "")
@@ -94,8 +95,8 @@
 		SkipDetection:      true,
 		SkipProvisionAgent: true,
 	}.install(c).Restore()
-	_, err = ProvisionMachine(args)
-	c.Assert(err, gc.Equals, ErrProvisioned)
+	_, err = manual.ProvisionMachine(args)
+	c.Assert(err, gc.Equals, manual.ErrProvisioned)
 	defer fakeSSH{
 		Provisioned:              true,
 		CheckProvisionedExitCode: 255,
@@ -103,7 +104,7 @@
 		SkipDetection:            true,
 		SkipProvisionAgent:       true,
 	}.install(c).Restore()
-	_, err = ProvisionMachine(args)
+	_, err = manual.ProvisionMachine(args)
 	c.Assert(err, gc.ErrorMatches, "error checking if provisioned: rc: 255")
 }
 
@@ -115,7 +116,7 @@
 		Arch:           arch,
 		InitUbuntuUser: true,
 	}.install(c).Restore()
-	machineId, err := ProvisionMachine(s.getArgs(c))
+	machineId, err := manual.ProvisionMachine(s.getArgs(c))
 	c.Assert(err, gc.IsNil)
 
 	// Now check what we would've configured it with.

=== added directory 'environs/manual/testing'
=== modified file 'environs/open.go'
--- environs/open.go	2014-02-18 01:29:47 +0000
+++ environs/open.go	2014-03-12 12:33:26 +0000
@@ -46,16 +46,14 @@
 	ConfigFromEnvirons
 )
 
-// ConfigForName returns the configuration for the environment with the
-// given name from the default environments file. If the name is blank,
-// the default environment will be used. If the configuration is not
-// found, an errors.NotFoundError is returned.
-// If the given store contains an entry for the environment
-// and it has associated bootstrap config, that configuration
-// will be returned.
-// ConfigForName also returns where the configuration
-// was sourced from (this is also valid even when there
-// is an error.
+// ConfigForName returns the configuration for the environment with
+// the given name from the default environments file. If the name is
+// blank, the default environment will be used. If the configuration
+// is not found, an errors.NotFoundError is returned. If the given
+// store contains an entry for the environment and it has associated
+// bootstrap config, that configuration will be returned.
+// ConfigForName also returns where the configuration was sourced from
+// (this is also valid even when there is an error.
 func ConfigForName(name string, store configstore.Storage) (*config.Config, ConfigSource, error) {
 	envs, err := ReadEnvirons("")
 	if err != nil {
@@ -86,6 +84,16 @@
 	return cfg, ConfigFromEnvirons, err
 }
 
+// maybeNotBootstrapped takes an error and source, returned by
+// ConfigForName and returns ErrNotBootstrapped if it looks like the
+// environment is not bootstrapped, or err as-is otherwise.
+func maybeNotBootstrapped(err error, source ConfigSource) error {
+	if err != nil && source == ConfigFromEnvirons {
+		return ErrNotBootstrapped
+	}
+	return err
+}
+
 // NewFromName opens the environment with the given
 // name from the default environments file. If the
 // name is blank, the default environment will be used.
@@ -99,16 +107,16 @@
 	// configuration attributes don't exist which
 	// will be filled in by Prepare.
 	cfg, source, err := ConfigForName(name, store)
-	if err != nil && source == ConfigFromEnvirons {
-		err = ErrNotBootstrapped
+	if err := maybeNotBootstrapped(err, source); err != nil {
+		return nil, err
 	}
 	if err != nil {
 		return nil, err
 	}
 
 	env, err := New(cfg)
-	if err != nil && source == ConfigFromEnvirons {
-		err = ErrNotBootstrapped
+	if err := maybeNotBootstrapped(err, source); err != nil {
+		return nil, err
 	}
 	return env, err
 }

=== modified file 'environs/sshstorage/storage.go'
--- environs/sshstorage/storage.go	2014-01-13 06:25:28 +0000
+++ environs/sshstorage/storage.go	2014-03-12 12:33:26 +0000
@@ -16,11 +16,15 @@
 	"strconv"
 	"strings"
 
+	"github.com/juju/loggo"
+
 	coreerrors "launchpad.net/juju-core/errors"
 	"launchpad.net/juju-core/utils"
 	"launchpad.net/juju-core/utils/ssh"
 )
 
+var logger = loggo.GetLogger("juju.environs.sshstorage")
+
 // base64LineLength is the default line length for wrapping
 // output generated by the base64 command line utility.
 const base64LineLength = 76
@@ -240,6 +244,7 @@
 
 // Get implements storage.StorageReader.Get.
 func (s *SSHStorage) Get(name string) (io.ReadCloser, error) {
+	logger.Debugf("getting %q from storage", name)
 	path, err := s.path(name)
 	if err != nil {
 		return nil, err
@@ -305,6 +310,7 @@
 
 // Put implements storage.StorageWriter.Put
 func (s *SSHStorage) Put(name string, r io.Reader, length int64) error {
+	logger.Debugf("putting %q (len %d) to storage", name, length)
 	path, err := s.path(name)
 	if err != nil {
 		return err

=== modified file 'environs/testing/storage.go'
--- environs/testing/storage.go	2014-03-04 16:55:38 +0000
+++ environs/testing/storage.go	2014-03-12 12:33:26 +0000
@@ -4,16 +4,13 @@
 package testing
 
 import (
-	"fmt"
 	"io"
 
 	gc "launchpad.net/gocheck"
 
-	"launchpad.net/juju-core/environs"
 	"launchpad.net/juju-core/environs/filestorage"
 	"launchpad.net/juju-core/environs/httpstorage"
 	"launchpad.net/juju-core/environs/storage"
-	"launchpad.net/juju-core/state"
 )
 
 // CreateLocalTestStorage returns the listener, which needs to be closed, and
@@ -29,17 +26,3 @@
 	closer = listener
 	return
 }
-
-// GetEnvironStorage creates an Environ from the config in state and
-// returns its storage interface.
-func GetEnvironStorage(st *state.State) (storage.Storage, error) {
-	envConfig, err := st.EnvironConfig()
-	if err != nil {
-		return nil, fmt.Errorf("cannot get environment config: %v", err)
-	}
-	env, err := environs.New(envConfig)
-	if err != nil {
-		return nil, fmt.Errorf("cannot access environment: %v", err)
-	}
-	return env.Storage(), nil
-}

=== added file 'environs/utils.go'
--- environs/utils.go	1970-01-01 00:00:00 +0000
+++ environs/utils.go	2014-03-12 12:33:26 +0000
@@ -0,0 +1,22 @@
+package environs
+
+import (
+	"fmt"
+
+	"launchpad.net/juju-core/environs/storage"
+	"launchpad.net/juju-core/state"
+)
+
+// GetStorage creates an Environ from the config in state and returns
+// its storage interface.
+func GetStorage(st *state.State) (storage.Storage, error) {
+	envConfig, err := st.EnvironConfig()
+	if err != nil {
+		return nil, fmt.Errorf("cannot get environment config: %v", err)
+	}
+	env, err := New(envConfig)
+	if err != nil {
+		return nil, fmt.Errorf("cannot access environment: %v", err)
+	}
+	return env.Storage(), nil
+}

=== modified file 'juju/conn_test.go'
--- juju/conn_test.go	2014-03-07 14:15:28 +0000
+++ juju/conn_test.go	2014-03-12 12:33:26 +0000
@@ -124,7 +124,7 @@
 	c.Assert(conn.Environ.Name(), gc.Equals, envName)
 }
 
-func (cs *NewConnSuite) TestConnStateSecretsSideEffect(c *gc.C) {
+func (*NewConnSuite) TestConnStateSecretsSideEffect(c *gc.C) {
 	attrs := dummy.SampleConfig().Merge(coretesting.Attrs{
 		"admin-secret": "side-effect secret",
 		"secret":       "pork",
@@ -175,7 +175,7 @@
 	c.Assert(err, gc.IsNil)
 }
 
-func (cs *NewConnSuite) TestConnStateDoesNotUpdateExistingSecrets(c *gc.C) {
+func (*NewConnSuite) TestConnStateDoesNotUpdateExistingSecrets(c *gc.C) {
 	attrs := dummy.SampleConfig().Merge(coretesting.Attrs{
 		"secret": "pork",
 	})
@@ -212,7 +212,7 @@
 	c.Assert(err, gc.IsNil)
 }
 
-func (cs *NewConnSuite) TestConnWithPassword(c *gc.C) {
+func (*NewConnSuite) TestConnWithPassword(c *gc.C) {
 	attrs := dummy.SampleConfig().Merge(coretesting.Attrs{
 		"admin-secret": "nutkin",
 	})

=== modified file 'provider/azure/config.go'
--- provider/azure/config.go	2014-01-29 00:14:51 +0000
+++ provider/azure/config.go	2014-03-12 12:33:26 +0000
@@ -95,32 +95,40 @@
 	return cfg.Apply(envCfg.attrs)
 }
 
-const boilerplateYAML = `
+var boilerplateYAML = `
 # https://juju.ubuntu.com/docs/config-azure.html
 azure:
     type: azure
 
-    # location specifies the place where instances will be started, for
-    # example: West US, North Europe.
+    # location specifies the place where instances will be started,
+    # for example: West US, North Europe.
+    #
     location: West US
-    
-    # The following attributes specify Windows Azure Management information.
-    # See  http://msdn.microsoft.com/en-us/library/windowsazure
+
+    # The following attributes specify Windows Azure Management
+    # information. See:
+    # http://msdn.microsoft.com/en-us/library/windowsazure
     # for details.
+    #
     management-subscription-id: <00000000-0000-0000-0000-000000000000>
     management-certificate-path: /home/me/azure.pem
 
     # storage-account-name holds Windows Azure Storage info.
+    #
     storage-account-name: abcdefghijkl
 
-    # force-image-name overrides the OS image selection to use
-    # a fixed image for all deployments. Most useful for developers.
+    # force-image-name overrides the OS image selection to use a fixed
+    # image for all deployments. Most useful for developers.
+    #
     # force-image-name: b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-13_10-amd64-server-DEVELOPMENT-20130713-Juju_ALPHA-en-us-30GB
 
-    # image-stream chooses a simplestreams stream to select OS images from,
-    # for example daily or released images (or any other stream available on simplestreams).
+    # image-stream chooses a simplestreams stream to select OS images
+    # from, for example daily or released images (or any other stream
+    # available on simplestreams).
+    #
     # image-stream: "released"
-`
+
+`[1:]
 
 func (prov azureEnvironProvider) BoilerplateConfig() string {
 	return boilerplateYAML

=== modified file 'provider/common/bootstrap.go'
--- provider/common/bootstrap.go	2014-03-05 19:41:34 +0000
+++ provider/common/bootstrap.go	2014-03-12 12:33:26 +0000
@@ -405,18 +405,3 @@
 	}
 	return bootstrap.SetBootstrapTools(env, possibleTools)
 }
-
-// EnsureNotBootstrapped returns null if the environment is not bootstrapped,
-// and an error if it is or if the function was not able to tell.
-func EnsureNotBootstrapped(env environs.Environ) error {
-	_, err := bootstrap.LoadState(env.Storage())
-	// If there is no error loading the bootstrap state, then we are
-	// bootstrapped.
-	if err == nil {
-		return fmt.Errorf("environment is already bootstrapped")
-	}
-	if err == environs.ErrNotBootstrapped {
-		return nil
-	}
-	return err
-}

=== modified file 'provider/ec2/ec2.go'
--- provider/ec2/ec2.go	2014-03-11 07:21:10 +0000
+++ provider/ec2/ec2.go	2014-03-12 12:33:26 +0000
@@ -179,18 +179,25 @@
 # https://juju.ubuntu.com/docs/config-aws.html
 amazon:
     type: ec2
-    # region specifies the ec2 region. It defaults to us-east-1.
+
+    # region specifies the EC2 region. It defaults to us-east-1.
+    #
     # region: us-east-1
+
+    # access-key holds the EC2 access key. It defaults to the
+    # environment variable AWS_ACCESS_KEY_ID.
     #
-    # access-key holds the ec2 access key. It defaults to the environment
-    # variable AWS_ACCESS_KEY_ID.
     # access-key: <secret>
-    #
-    # secret-key holds the ec2 secret key. It defaults to the environment
-    # variable AWS_SECRET_ACCESS_KEY.
-    #
-    # image-stream chooses a simplestreams stream to select OS images from,
-    # for example daily or released images (or any other stream available on simplestreams).
+
+    # secret-key holds the EC2 secret key. It defaults to the
+    # environment variable AWS_SECRET_ACCESS_KEY.
+    #
+    # secret-key: <secret>
+
+    # image-stream chooses a simplestreams stream to select OS images
+    # from, for example daily or released images (or any other stream
+    # available on simplestreams).
+    #
     # image-stream: "released"
 
 `[1:]

=== modified file 'provider/joyent/config.go'
--- provider/joyent/config.go	2014-03-10 00:10:29 +0000
+++ provider/joyent/config.go	2014-03-12 12:33:26 +0000
@@ -14,25 +14,46 @@
 
 // boilerplateConfig will be shown in help output, so please keep it up to
 // date when you change environment configuration below.
-const boilerplateConfig = `joyent:
-  type: joyent
-
-  # SDC config
-  # Can be set via env variables, or specified here
-  # sdc-user: <secret>
-  # Can be set via env variables, or specified here
-  # sdc-key-id: <secret>
-  # region defaults to us-west-1, override if required
-  # sdc-region: us-west-1
-
-  # Manta config
-  # Can be set via env variables, or specified here
-  # manta-user: <secret>
-  # Can be set via env variables, or specified here
-  # manta-key-id: <secret>
-  # region defaults to us-east, override if required
-  # manta-region: us-east
-`
+var boilerplateConfig = `
+joyent:
+    type: joyent
+
+    ## SDC config
+
+    # sdc-user holds the SDC user. It can also be specified in the
+    # SDC_ACCOUNT environment variable.
+    #
+    # sdc-user: <secret>
+
+    # sdc-key-id holds the fingerprint of one of user's SSH keys. It
+    # can also be specified in the SDC_KEY_ID environment variable.
+    #
+    # sdc-key-id: <secret>
+
+    # sdc-region holds the region to use: us-west-1 (default),
+    # us-east-1, us-sw-1, eu-ams-1. Override if required.
+    #
+    # sdc-region: us-west-1
+
+    ## Manta config
+
+    # manta-user holds the user's Manta account name. It can also be
+    #  specified in the MANTA_USER environment variable.
+    #
+    # manta-user: <secret>
+
+    # manta-key-id holds the fingerprint of one of the user's SSH
+    # keys. It can also be specified in the MANTA_KEY_ID environment
+    # variable.
+    #
+    # manta-key-id: <secret>
+
+    # manta-region holds the Manta region to use. It defaults to
+    # us-east, override if required
+    #
+    # manta-region: us-east
+
+`[1:]
 
 const (
 	SdcAccount = "SDC_ACCOUNT"

=== modified file 'provider/local/environprovider.go'
--- provider/local/environprovider.go	2014-03-09 20:48:29 +0000
+++ provider/local/environprovider.go	2014-03-12 12:33:26 +0000
@@ -230,17 +230,23 @@
 # https://juju.ubuntu.com/docs/config-local.html
 local:
     type: local
-    # Override the directory that is used for the storage files and database.
-    # The default location is $JUJU_HOME/<ENV>.
-    
-    # $JUJU_HOME defaults to ~/.juju
+
+    # root-dir holds the directory that is used for the storage files and
+    # database. The default location is $JUJU_HOME/<env-name>.
+    # $JUJU_HOME defaults to ~/.juju. Override if needed.
+    #
     # root-dir: ~/.juju/local
-    
-    # Override the storage port if you have multiple local providers, or if the
-    # default port is used by another program.
+
+    # storage-port holds the port where the local provider starts the
+    # HTTP file server. Override the value if you have multiple local
+    # providers, or if the default port is used by another program.
+    #
     # storage-port: 8040
-    
-    # Override the network bridge if you have changed the default lxc bridge
+
+    # network-bridge holds the name of the LXC network bridge to use.
+    # Override if the default LXC network bridge is different.
+    #
+    #
     # network-bridge: lxcbr0
 
 `[1:]

=== modified file 'provider/maas/environprovider.go'
--- provider/maas/environprovider.go	2014-03-05 19:41:34 +0000
+++ provider/maas/environprovider.go	2014-03-12 12:33:26 +0000
@@ -64,12 +64,14 @@
 # https://juju.ubuntu.com/docs/config-maas.html
 maas:
     type: maas
-  
+
     # maas-server specifies the location of the MAAS server. It must
     # specify the base path.
+    #
     maas-server: 'http://192.168.1.1/MAAS/'
-    
+
     # maas-oauth holds the OAuth credentials from MAAS.
+    #
     maas-oauth: '<add your OAuth credentials from MAAS here>'
 
 `[1:]

=== modified file 'provider/manual/environ.go'
--- provider/manual/environ.go	2014-03-05 19:41:34 +0000
+++ provider/manual/environ.go	2014-03-12 12:33:26 +0000
@@ -188,6 +188,7 @@
 }
 
 var newSSHStorage = func(sshHost, storageDir, storageTmpdir string) (storage.Storage, error) {
+	logger.Debugf("using ssh storage at host %q dir %q", sshHost, storageDir)
 	return sshstorage.NewSSHStorage(sshstorage.NewSSHStorageParams{
 		Host:       sshHost,
 		StorageDir: storageDir,

=== modified file 'provider/openstack/provider.go'
--- provider/openstack/provider.go	2014-03-11 07:21:10 +0000
+++ provider/openstack/provider.go	2014-03-12 12:33:26 +0000
@@ -67,59 +67,75 @@
 # https://juju.ubuntu.com/docs/config-openstack.html
 openstack:
     type: openstack
-    # use-floating-ip specifies whether a floating IP address is required
-    # to give the nodes a public IP address. Some installations assign public IP
-    # addresses by default without requiring a floating IP address.
+
+    # use-floating-ip specifies whether a floating IP address is
+    # required to give the nodes a public IP address. Some
+    # installations assign public IP addresses by default without
+    # requiring a floating IP address.
+    #
     # use-floating-ip: false
 
-    # use-default-secgroup specifies whether new machine instances should have the "default"
-    # Openstack security group assigned.
+    # use-default-secgroup specifies whether new machine instances
+    # should have the "default" Openstack security group assigned.
+    #
     # use-default-secgroup: false
 
-    # network specifies the network label or uuid to bring machines up on, in
-    # the case where multiple networks exist. It may be omitted otherwise.
+    # network specifies the network label or uuid to bring machines up
+    # on, in the case where multiple networks exist. It may be omitted
+    # otherwise.
+    #
     # network: <your network label or uuid>
 
-    # tools-metadata-url specifies the location of the Juju tools and metadata. It defaults to the
-    # global public tools metadata location https://streams.canonical.com/tools.
-    # tools-metadata-url:  https://you-tools-metadata-url
-
-    # image-metadata-url specifies the location of Ubuntu cloud image metadata. It defaults to the
-    # global public image metadata location https://cloud-images.ubuntu.com/releases.
-    # image-metadata-url:  https://you-tools-metadata-url
-
-    # image-stream chooses a simplestreams stream to select OS images from,
-    # for example daily or released images (or any other stream available on simplestreams).
+    # tools-metadata-url specifies the location of the Juju tools and
+    # metadata. It defaults to the global public tools metadata
+    # location https://streams.canonical.com/tools.
+    #
+    # tools-metadata-url:  https://your-tools-metadata-url
+
+    # image-metadata-url specifies the location of Ubuntu cloud image
+    # metadata. It defaults to the global public image metadata
+    # location https://cloud-images.ubuntu.com/releases.
+    #
+    # image-metadata-url:  https://your-tools-metadata-url
+
+    # image-stream chooses a simplestreams stream to select OS images
+    # from, for example daily or released images (or any other stream
+    # available on simplestreams).
+    #
     # image-stream: "released"
 
-    # auth-url defaults to the value of the environment variable OS_AUTH_URL,
-    # but can be specified here.
+    # auth-url defaults to the value of the environment variable
+    # OS_AUTH_URL, but can be specified here.
+    #
     # auth-url: https://yourkeystoneurl:443/v2.0/
 
-    # tenant-name holds the openstack tenant name. It defaults to
-    # the environment variable OS_TENANT_NAME.
+    # tenant-name holds the openstack tenant name. It defaults to the
+    # environment variable OS_TENANT_NAME.
+    #
     # tenant-name: <your tenant name>
 
-    # region holds the openstack region.  It defaults to
-    # the environment variable OS_REGION_NAME.
+    # region holds the openstack region. It defaults to the
+    # environment variable OS_REGION_NAME.
+    #
     # region: <your region>
 
-    # The auth-mode, username and password attributes
-    # are used for userpass authentication (the default).
-
+    # The auth-mode, username and password attributes are used for
+    # userpass authentication (the default).
+    #
     # auth-mode holds the authentication mode. For user-password
-    # authentication, auth-mode should be "userpass" and username
-    # and password should be set appropriately; they default to
-    # the environment variables OS_USERNAME and OS_PASSWORD
-     # respectively.
+    # authentication, auth-mode should be "userpass" and username and
+    # password should be set appropriately; they default to the
+    # environment variables OS_USERNAME and OS_PASSWORD respectively.
+    #
     # auth-mode: userpass
     # username: <your username>
     # password: <secret>
-     
-    # For key-pair authentication, auth-mode should  be "keypair"
-    # and access-key and secret-key should be  set appropriately; they default to
-    # the environment variables OS_ACCESS_KEY and OS_SECRET_KEY
-    # respectively.
+
+    # For key-pair authentication, auth-mode should be "keypair" and
+    # access-key and secret-key should be set appropriately; they
+    # default to the environment variables OS_ACCESS_KEY and
+    # OS_SECRET_KEY respectively.
+    #
     # auth-mode: keypair
     # access-key: <secret>
     # secret-key: <secret>
@@ -127,45 +143,59 @@
 # https://juju.ubuntu.com/docs/config-hpcloud.html
 hpcloud:
     type: openstack
-    
-    # use-floating-ip specifies whether a floating IP address is required
-    # to give the nodes a public IP address. Some installations assign public IP
-    # addresses by default without requiring a floating IP address.
+
+    # use-floating-ip specifies whether a floating IP address is
+    # required to give the nodes a public IP address. Some
+    # installations assign public IP addresses by default without
+    # requiring a floating IP address.
+    #
     # use-floating-ip: false
 
-    # use-default-secgroup specifies whether new machine instances should have the "default"
-    # Openstack security group assigned.
+    # use-default-secgroup specifies whether new machine instances
+    # should have the "default" Openstack security group assigned.
+    #
     # use-default-secgroup: false
 
-    # tenant-name holds the openstack tenant name. In HPCloud, this is 
-    # synonymous with the project-name  It defaults to
-    # the environment variable OS_TENANT_NAME.
+    # tenant-name holds the openstack tenant name. In HPCloud, this is
+    # synonymous with the project-name It defaults to the environment
+    # variable OS_TENANT_NAME.
+    #
     # tenant-name: <your tenant name>
-    
-    # auth-url holds the keystone url for authentication. 
-    # It defaults to the value of the environment variable OS_AUTH_URL.
+
+    # image-stream chooses a simplestreams stream to select OS images
+    # from, for example daily or released images (or any other stream
+    # available on simplestreams).
+    #
+    # image-stream: "released"
+
+    # auth-url holds the keystone url for authentication. It defaults
+    # to the value of the environment variable OS_AUTH_URL.
+    #
     # auth-url: https://region-a.geo-1.identity.hpcloudsvc.com:35357/v2.0/
 
-    # region holds the HP Cloud region (e.g. az-1.region-a.geo-1).  
-    # It defaults to the environment variable OS_REGION_NAME.
+    # region holds the HP Cloud region (e.g. az-1.region-a.geo-1). It
+    # defaults to the environment variable OS_REGION_NAME.
+    #
     # region: <your region>
-    
+
     # auth-mode holds the authentication mode. For user-password
-    # authentication, auth-mode should be "userpass" and username
-    # and password should be set appropriately; they default to
-    # the environment variables OS_USERNAME and OS_PASSWORD
-    # respectively.
+    # authentication, auth-mode should be "userpass" and username and
+    # password should be set appropriately; they default to the
+    # environment variables OS_USERNAME and OS_PASSWORD respectively.
+    #
     # auth-mode: userpass
     # username: <your_username>
     # password: <your_password>
-    
-    # For key-pair authentication, auth-mode should  be "keypair"
-    # and access-key and secret-key should be  set appropriately; they default to
-    # the environment variables OS_ACCESS_KEY and OS_SECRET_KEY
-    # respectively.
+
+    # For key-pair authentication, auth-mode should be "keypair" and
+    # access-key and secret-key should be set appropriately; they
+    # default to the environment variables OS_ACCESS_KEY and
+    # OS_SECRET_KEY respectively.
+    #
     # auth-mode: keypair
     # access-key: <secret>
     # secret-key: <secret>
+
 `[1:]
 }
 

=== modified file 'state/apiserver/charms.go'
--- state/apiserver/charms.go	2014-03-06 18:11:16 +0000
+++ state/apiserver/charms.go	2014-03-12 12:33:26 +0000
@@ -25,7 +25,7 @@
 	"github.com/errgo/errgo"
 
 	"launchpad.net/juju-core/charm"
-	envtesting "launchpad.net/juju-core/environs/testing"
+	"launchpad.net/juju-core/environs"
 	"launchpad.net/juju-core/names"
 	"launchpad.net/juju-core/state"
 	"launchpad.net/juju-core/state/api/params"
@@ -392,7 +392,7 @@
 	if _, err := repackagedArchive.Seek(0, 0); err != nil {
 		return errgo.Annotate(err, "cannot rewind the charm file reader")
 	}
-	storage, err := envtesting.GetEnvironStorage(h.state)
+	storage, err := environs.GetStorage(h.state)
 	if err != nil {
 		return errgo.Annotate(err, "cannot access provider storage")
 	}
@@ -455,7 +455,7 @@
 // saves the corresponding zip archive to the given charmArchivePath.
 func (h *charmsHandler) downloadCharm(name, charmArchivePath string) error {
 	// Get the provider storage.
-	storage, err := envtesting.GetEnvironStorage(h.state)
+	storage, err := environs.GetStorage(h.state)
 	if err != nil {
 		return errgo.Annotate(err, "cannot access provider storage")
 	}

=== modified file 'state/apiserver/charms_test.go'
--- state/apiserver/charms_test.go	2014-02-27 14:11:56 +0000
+++ state/apiserver/charms_test.go	2014-03-12 12:33:26 +0000
@@ -18,7 +18,7 @@
 	gc "launchpad.net/gocheck"
 
 	"launchpad.net/juju-core/charm"
-	envtesting "launchpad.net/juju-core/environs/testing"
+	"launchpad.net/juju-core/environs"
 	jujutesting "launchpad.net/juju-core/juju/testing"
 	"launchpad.net/juju-core/state"
 	"launchpad.net/juju-core/state/api/params"
@@ -168,7 +168,7 @@
 	expectedSHA256, _, err := utils.ReadSHA256(tempFile)
 	c.Assert(err, gc.IsNil)
 	name := charm.Quote(expectedURL.String())
-	storage, err := envtesting.GetEnvironStorage(s.State)
+	storage, err := environs.GetStorage(s.State)
 	c.Assert(err, gc.IsNil)
 	expectedUploadURL, err := storage.URL(name)
 	c.Assert(err, gc.IsNil)
@@ -219,7 +219,7 @@
 	// should succeed, because it was repackaged during upload to
 	// strip nested dirs.
 	archiveName := strings.TrimPrefix(sch.BundleURL().RequestURI(), "/dummyenv/private/")
-	storage, err := envtesting.GetEnvironStorage(s.State)
+	storage, err := environs.GetStorage(s.State)
 	c.Assert(err, gc.IsNil)
 	reader, err := storage.Get(archiveName)
 	c.Assert(err, gc.IsNil)

=== modified file 'state/apiserver/client/client_test.go'
--- state/apiserver/client/client_test.go	2014-03-11 02:53:40 +0000
+++ state/apiserver/client/client_test.go	2014-03-12 12:33:26 +0000
@@ -17,10 +17,10 @@
 	coreCloudinit "launchpad.net/juju-core/cloudinit"
 	"launchpad.net/juju-core/cloudinit/sshinit"
 	"launchpad.net/juju-core/constraints"
+	"launchpad.net/juju-core/environs"
 	"launchpad.net/juju-core/environs/cloudinit"
 	"launchpad.net/juju-core/environs/config"
-	"launchpad.net/juju-core/environs/storage"
-	envtesting "launchpad.net/juju-core/environs/testing"
+	envstorage "launchpad.net/juju-core/environs/storage"
 	ttesting "launchpad.net/juju-core/environs/tools/testing"
 	"launchpad.net/juju-core/errors"
 	"launchpad.net/juju-core/instance"
@@ -1872,7 +1872,7 @@
 	// contains the correct data.
 	sch, err := s.State.Charm(curl)
 	c.Assert(err, gc.IsNil)
-	storage, err := envtesting.GetEnvironStorage(s.State)
+	storage, err := environs.GetStorage(s.State)
 	c.Assert(err, gc.IsNil)
 	uploads, err := storage.List(fmt.Sprintf("%s-%d-", curl.Name, curl.Revision))
 	c.Assert(err, gc.IsNil)
@@ -1938,7 +1938,7 @@
 	}
 }
 
-func (s *clientSuite) assertUploaded(c *gc.C, storage storage.Storage, bundleURL *url.URL, expectedSHA256 string) {
+func (s *clientSuite) assertUploaded(c *gc.C, storage envstorage.Storage, bundleURL *url.URL, expectedSHA256 string) {
 	archiveName := getArchiveName(bundleURL)
 	reader, err := storage.Get(archiveName)
 	c.Assert(err, gc.IsNil)

=== removed file 'utils/ssh/testing/fakessh.go'
--- utils/ssh/testing/fakessh.go	2014-02-18 17:08:55 +0000
+++ utils/ssh/testing/fakessh.go	1970-01-01 00:00:00 +0000
@@ -1,80 +0,0 @@
-// Copyright 2013 Canonical Ltd.
-// Licensed under the AGPLv3, see LICENCE file for details.
-
-package testing
-
-import (
-	"fmt"
-	"io/ioutil"
-	"path/filepath"
-
-	gc "launchpad.net/gocheck"
-
-	"launchpad.net/juju-core/testing/testbase"
-)
-
-// sshscript should only print the result on the first execution,
-// to handle the case where it's called multiple times. On
-// subsequent executions, it should find the next 'ssh' in $PATH
-// and exec that.
-var sshscript = `#!/bin/bash --norc
-if [ ! -e "$0.run" ]; then
-    touch "$0.run"
-    if [ -e "$0.expected-input" ]; then
-        diff "$0.expected-input" -
-        exitcode=$?
-        if [ $exitcode -ne 0 ]; then
-            echo "ERROR: did not match expected input" >&2
-            exit $exitcode
-        fi
-        else
-            head >/dev/null
-        fi
-    # stdout
-    %s
-    # stderr
-    %s
-    exit %d
-else
-    export PATH=${PATH#*:}
-    exec ssh $*
-fi`
-
-// InstallFakeSSH creates a fake "ssh" command in a new $PATH,
-// updates $PATH, and returns a function to reset $PATH to its
-// original value when called.
-//
-// input may be:
-//    - nil (ignore input)
-//    - a string (match input exactly)
-// output may be:
-//    - nil (no output)
-//    - a string (stdout)
-//    - a slice of strings, of length two (stdout, stderr)
-func InstallFakeSSH(c *gc.C, input, output interface{}, rc int) testbase.Restorer {
-	fakebin := c.MkDir()
-	ssh := filepath.Join(fakebin, "ssh")
-	switch input := input.(type) {
-	case nil:
-	case string:
-		sshexpectedinput := ssh + ".expected-input"
-		err := ioutil.WriteFile(sshexpectedinput, []byte(input), 0644)
-		c.Assert(err, gc.IsNil)
-	default:
-		c.Errorf("input has invalid type: %T", input)
-	}
-	var stdout, stderr string
-	switch output := output.(type) {
-	case nil:
-	case string:
-		stdout = fmt.Sprintf("cat<<EOF\n%s\nEOF", output)
-	case []string:
-		c.Assert(output, gc.HasLen, 2)
-		stdout = fmt.Sprintf("cat<<EOF\n%s\nEOF", output[0])
-		stderr = fmt.Sprintf("cat>&2<<EOF\n%s\nEOF", output[1])
-	}
-	script := fmt.Sprintf(sshscript, stdout, stderr, rc)
-	err := ioutil.WriteFile(ssh, []byte(script), 0777)
-	c.Assert(err, gc.IsNil)
-	return testbase.PatchEnvPathPrepend(fakebin)
-}