Merge lp:~axwalk/juju-core/lp1236691-null-provider-default-series-take2 into lp:~go-bot/juju-core/trunk

Proposed by Andrew Wilkins
Status: Merged
Approved by: Andrew Wilkins
Approved revision: no longer in the source branch.
Merged at revision: 2069
Proposed branch: lp:~axwalk/juju-core/lp1236691-null-provider-default-series-take2
Merge into: lp:~go-bot/juju-core/trunk
Diff against target: 2575 lines (+791/-539)
46 files modified
cmd/juju/bootstrap.go (+1/-95)
cmd/juju/bootstrap_test.go (+32/-76)
cmd/juju/upgradejuju.go (+1/-3)
cmd/juju/upgradejuju_test.go (+1/-2)
cmd/jujud/bootstrap.go (+2/-2)
cmd/jujud/bootstrap_test.go (+3/-3)
environs/bootstrap/bootstrap.go (+22/-34)
environs/bootstrap/bootstrap_test.go (+12/-3)
environs/bootstrap/state.go (+97/-0)
environs/bootstrap/state_test.go (+118/-0)
environs/bootstrap/synctools.go (+123/-0)
environs/interface.go (+5/-2)
environs/jujutest/livetests.go (+3/-2)
environs/jujutest/tests.go (+6/-5)
environs/manual/bootstrap.go (+22/-8)
environs/manual/bootstrap_test.go (+47/-8)
environs/manual/detection.go (+6/-4)
environs/manual/detection_test.go (+3/-3)
environs/manual/fakessh.go (+35/-24)
environs/manual/provisioner.go (+1/-1)
environs/manual/provisioner_test.go (+4/-4)
environs/simplestreams/simplestreams.go (+1/-0)
environs/sync/sync.go (+17/-5)
environs/testing/tools.go (+7/-2)
environs/tools/storage.go (+1/-0)
environs/tools/tools.go (+2/-1)
environs/tools/tools_test.go (+2/-2)
provider/azure/environ.go (+2/-2)
provider/azure/environ_test.go (+7/-7)
provider/common/bootstrap.go (+38/-5)
provider/common/bootstrap_test.go (+66/-12)
provider/common/mock_test.go (+27/-0)
provider/common/state.go (+2/-87)
provider/common/state_test.go (+0/-103)
provider/dummy/environs.go (+9/-3)
provider/ec2/ec2.go (+2/-2)
provider/ec2/local_test.go (+2/-3)
provider/local/environ.go (+10/-3)
provider/maas/environ.go (+2/-2)
provider/maas/environ_test.go (+12/-6)
provider/null/config_test.go (+0/-5)
provider/null/environ.go (+16/-5)
provider/null/provider.go (+5/-1)
provider/null/suite_test.go (+14/-0)
provider/openstack/local_test.go (+1/-2)
provider/openstack/provider.go (+2/-2)
To merge this branch: bzr merge lp:~axwalk/juju-core/lp1236691-null-provider-default-series-take2
Reviewer Review Type Date Requested Status
Juju Engineering Pending
Review via email: mp+190032@code.launchpad.net

Commit message

Providers are responsible for selecting tools

This change makes providers responsible for
selecting bootstrap tools, and ensuring they
are available in storage. The null provider is
updated to select the tools appropriate to the
bootstrap host. Other providers use a default
implementation in environs/bootstrap, which
has been extracted from cmd/juju/bootstrap.go.

One test fails due to a logging change. This is
fixed by https://codereview.appspot.com/14430064/.

Fixes #1236691

https://codereview.appspot.com/14433058/

Description of the change

Providers are responsible for selecting tools

This change makes providers responsible for
selecting bootstrap tools, and ensuring they
are available in storage. The null provider is
updated to select the tools appropriate to the
bootstrap host. Other providers use a default
implementation in environs/bootstrap, which
has been extracted from cmd/juju/bootstrap.go.

One test fails due to a logging change. This is
fixed by https://codereview.appspot.com/14430064/.

Fixes #1236691

https://codereview.appspot.com/14433058/

To post a comment you must log in.
Revision history for this message
Andrew Wilkins (axwalk) wrote :

Reviewers: mp+190032_code.launchpad.net,

Message:
Please take a look.

Description:
provider/null: dynamic default-series

default-series is used to decide which
tools to bootstrap with. This is not
necessarily valid for the null provider,
where the boostrap host's architecture
should instead decide which tools to
locate.

So now we assign default-series dynamically,
by performing detection at Prepare time.
We store the detected series and hardware
characteristics, and use them to avoid
repeating the detection at Bootstrap time.

Fixes #1236691

https://code.launchpad.net/~axwalk/juju-core/lp1236691-null-provider-default-series-take2/+merge/190032

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/14433058/

Affected files (+346, -76 lines):
   A [revision details]
   M environs/manual/bootstrap.go
   M environs/manual/bootstrap_test.go
   M environs/manual/detection.go
   M environs/manual/detection_test.go
   M environs/manual/fakessh.go
   M environs/manual/provisioner.go
   M environs/manual/provisioner_test.go
   M provider/null/config.go
   M provider/null/config_test.go
   M provider/null/environ.go
   M provider/null/provider.go
   A provider/null/provider_test.go
   A provider/null/suite_test.go

Revision history for this message
William Reade (fwereade) wrote :

On 2013/10/09 07:13:51, axw wrote:
> Please take a look.

I don't think this is the right approach, I'm afraid. The possibleTools
arg itself only exists because we couldn't justify refactoring any
further towards sanity given time pressure. Basically all this bootstrap
stuff is inside out... environs should generally know what tools they've
got available anyway, and using default-series to pick the bootstrap
node's is pretty dumb and pointless in the first place.

Ping me tomorrow, we can have a long rambling G+ and see what makes
sense.

https://codereview.appspot.com/14433058/

Revision history for this message
Andrew Wilkins (axwalk) wrote :

On 2013/10/09 18:04:30, fwereade wrote:
> On 2013/10/09 07:13:51, axw wrote:
> > Please take a look.

> I don't think this is the right approach, I'm afraid. The
possibleTools arg
> itself only exists because we couldn't justify refactoring any further
towards
> sanity given time pressure. Basically all this bootstrap stuff is
inside out...
> environs should generally know what tools they've got available
anyway, and
> using default-series to pick the bootstrap node's is pretty dumb and
pointless
> in the first place.

Fair enough. I started down the road of adding a "BootstrapSeries"
method to Environ, but started thinking it was a lot of change just for
one provider. But... perhaps this is the right thing to do. Once the
"wire up prechecker" branch is landed we'll have EnvironBase to add
default behaviour to, to keep the ugly down.

> Ping me tomorrow, we can have a long rambling G+ and see what makes
sense.

Sure thing, will be around before the weekly meeting.

https://codereview.appspot.com/14433058/

Revision history for this message
Andrew Wilkins (axwalk) wrote :
Revision history for this message
William Reade (fwereade) wrote :
Download full text (4.0 KiB)

Sorry this has taken so long. I'm a bit uncomfortable about the sheer
amount of stuff we do, and feel like there's something more complex than
it needs to be... can we chat about this? Maybe your tomorrow morning?

https://codereview.appspot.com/14433058/diff/5001/environs/bootstrap/bootstrap.go
File environs/bootstrap/bootstrap.go (right):

https://codereview.appspot.com/14433058/diff/5001/environs/bootstrap/bootstrap.go#newcode17
environs/bootstrap/bootstrap.go:17: var logger =
loggo.GetLogger("juju.environs.bootstrap")
Ha. Thanks.

https://codereview.appspot.com/14433058/diff/5001/environs/bootstrap/bootstrap.go#newcode56
environs/bootstrap/bootstrap.go:56: newVersion, toolsList =
toolsList.Newest()
I'm not 100% sure this is right, but that's because it's not clear where
the tools list comes from and what its properties are. Any reason we
can't get it inside here?

Dimiter's just landing a List.NewestCompatible method, that might be
useful?

https://codereview.appspot.com/14433058/diff/5001/environs/bootstrap/synctools.go
File environs/bootstrap/synctools.go (right):

https://codereview.appspot.com/14433058/diff/5001/environs/bootstrap/synctools.go#newcode42
environs/bootstrap/synctools.go:42: logger.Warningf("no tools found, so
attempting to build and upload new tools")
Does this work with released versions of the local provider? That needs
to upload tools, and would ideally do so *without* the
juju.BootstrapComand level hack.

https://codereview.appspot.com/14433058/diff/5001/environs/bootstrap/synctools.go#newcode117
environs/bootstrap/synctools.go:117: // TODO(axw) have syncOrUpload
return the list of tools in the target, and use that.
There's something still a bit off here, in that the copy-from-local
behaviour of upload is conceptually a sync, but ESCOPE.

https://codereview.appspot.com/14433058/diff/5001/environs/manual/bootstrap.go
File environs/manual/bootstrap.go (right):

https://codereview.appspot.com/14433058/diff/5001/environs/manual/bootstrap.go#newcode39
environs/manual/bootstrap.go:39: HardwareCharacteristics
*instance.HardwareCharacteristics
I'll probably see it in a minute -- but when can they be known ahead of
time ina manual bootstrap?

https://codereview.appspot.com/14433058/diff/5001/provider/local/environ.go
File provider/local/environ.go (right):

https://codereview.appspot.com/14433058/diff/5001/provider/local/environ.go#newcode136
provider/local/environ.go:136: }
This seems to be repeated a lot. Can we pull it out into a common func?

https://codereview.appspot.com/14433058/diff/5001/provider/maas/environ_test.go
File provider/maas/environ_test.go (right):

https://codereview.appspot.com/14433058/diff/5001/provider/maas/environ_test.go#newcode448
provider/maas/environ_test.go:448: "agent-version":
version.Current.Number.String(),
That's what it should always choose anyway.

https://codereview.appspot.com/14433058/diff/5001/provider/null/config_test.go
File provider/null/config_test.go (left):

https://codereview.appspot.com/14433058/diff/5001/provider/null/config_test.go#oldcode26
provider/null/config_test.go:26: gc.TestingT(t)
heh, were we double-running these?

https://codereview.appspot.com/14433058/diff/5001...

Read more...

Revision history for this message
Andrew Wilkins (axwalk) wrote :
Download full text (4.6 KiB)

Please take a look.

https://codereview.appspot.com/14433058/diff/5001/environs/bootstrap/bootstrap.go
File environs/bootstrap/bootstrap.go (right):

https://codereview.appspot.com/14433058/diff/5001/environs/bootstrap/bootstrap.go#newcode56
environs/bootstrap/bootstrap.go:56: newVersion, toolsList =
toolsList.Newest()
On 2013/11/14 09:29:45, fwereade wrote:
> I'm not 100% sure this is right, but that's because it's not clear
where the
> tools list comes from and what its properties are. Any reason we can't
get it
> inside here?

> Dimiter's just landing a List.NewestCompatible method, that might be
useful?

Practically, the list will always come from EnsureToolsAvailabilty. That
function uses environs.FindBootstrapTools, which returns a list of tools
with an appropriate version.

I'm not sure that NewestCompatible is appropriate here.

https://codereview.appspot.com/14433058/diff/5001/environs/bootstrap/synctools.go
File environs/bootstrap/synctools.go (right):

https://codereview.appspot.com/14433058/diff/5001/environs/bootstrap/synctools.go#newcode42
environs/bootstrap/synctools.go:42: logger.Warningf("no tools found, so
attempting to build and upload new tools")
On 2013/11/14 09:29:45, fwereade wrote:
> Does this work with released versions of the local provider? That
needs to
> upload tools, and would ideally do so *without* the
juju.BootstrapComand level
> hack.

There's no change there. AFAICT, the hack in BootstrapCmd isn't
necessary, but I would like to run any change there by thumper, and
preferably do it in another CL.

For posterity: if the hack in BootstrapCmd goes, there's a change
necessary in provider/local/environprovider.go too, to enable
auto-upload. Inside the Open method, the bit that sets agent-version
must be deleted.

https://codereview.appspot.com/14433058/diff/5001/environs/manual/bootstrap.go
File environs/manual/bootstrap.go (right):

https://codereview.appspot.com/14433058/diff/5001/environs/manual/bootstrap.go#newcode39
environs/manual/bootstrap.go:39: HardwareCharacteristics
*instance.HardwareCharacteristics
On 2013/11/14 09:29:45, fwereade wrote:
> I'll probably see it in a minute -- but when can they be known ahead
of time ina
> manual bootstrap?

"Ahead of time" here just means before the environs/manual.Bootstrap
function is called, not "ahead of bootstrap time".

The null/manual provider detects series/hardware in the provider's
Bootstrap method, and then passes it down. See provider/null/environ.go.

https://codereview.appspot.com/14433058/diff/5001/provider/local/environ.go
File provider/local/environ.go (right):

https://codereview.appspot.com/14433058/diff/5001/provider/local/environ.go#newcode136
provider/local/environ.go:136: }
On 2013/11/14 09:29:45, fwereade wrote:
> This seems to be repeated a lot. Can we pull it out into a common
func?

Done. I hadn't done this before, as I didn't want to hide what was
happening. If you think of a better name that describes what's happening
underneath, I'd love to hear it.

https://codereview.appspot.com/14433058/diff/5001/provider/maas/environ_test.go
File provider/maas/environ_test.go (right):

https://codereview.appspot.com/14433058/diff/5001/provider/maas/env...

Read more...

Revision history for this message
Ian Booth (wallyworld) wrote :
Download full text (3.2 KiB)

In general I like that responsibility for setting up tools has been
delegated to the providers. My main concern is that, as implemented, the
bootstrap is not exiting early enough if the environment is already
bootstrapped.

https://codereview.appspot.com/14433058/diff/9001/cmd/juju/bootstrap.go
File cmd/juju/bootstrap.go (right):

https://codereview.appspot.com/14433058/diff/9001/cmd/juju/bootstrap.go#newcode100
cmd/juju/bootstrap.go:100:
One implication of removing this check and running it as part of
bootstrap.Bootstrap() below is that we may well end up uploading tools
only to then tell the user "sorry, already bootstrapped". This check
really does need to be run asap in the bootstrap process, before we do
anything, so that we fail as early as possible.

https://codereview.appspot.com/14433058/diff/9001/environs/bootstrap/bootstrap.go
File environs/bootstrap/bootstrap.go (right):

https://codereview.appspot.com/14433058/diff/9001/environs/bootstrap/bootstrap.go#newcode50
environs/bootstrap/bootstrap.go:50: // and update the agent-version
configuration attribute.
s/update/updates

https://codereview.appspot.com/14433058/diff/9001/environs/bootstrap/bootstrap.go#newcode51
environs/bootstrap/bootstrap.go:51: func SelectBootstrapTools(environ
environs.Environ, toolsList coretools.List) (coretools.List, error) {
could we rename toolsList to availableTools or possibleTools to better
reflect the purpose of the parameter? callers of this method tend to
pass in a possibleTools variable so maybe that is the name to use

https://codereview.appspot.com/14433058/diff/9001/environs/manual/fakessh.go
File environs/manual/fakessh.go (right):

https://codereview.appspot.com/14433058/diff/9001/environs/manual/fakessh.go#newcode1
environs/manual/fakessh.go:1: // Copyright 2013 Canonical Ltd.
Why did this file get renamed from fakessh_test.go?
The contents are only for testing. In general, we only use non
"_test.go" files for testing if there are methods called from outside
the package (I don't think that's the case here?); and if we do, we
usually introduce a "testing" package into which such files are put to
make it clear that the contents are for testing and not production
business logic.

https://codereview.appspot.com/14433058/diff/9001/environs/testing/tools.go
File environs/testing/tools.go (right):

https://codereview.appspot.com/14433058/diff/9001/environs/testing/tools.go#newcode42
environs/testing/tools.go:42: sync.DefaultToolsLocation = c.MkDir() //
stop sync from going to s3
\o/

https://codereview.appspot.com/14433058/diff/9001/provider/common/bootstrap.go
File provider/common/bootstrap.go (right):

https://codereview.appspot.com/14433058/diff/9001/provider/common/bootstrap.go#newcode29
provider/common/bootstrap.go:29: // before potentially uploading any
tools.
Trouble is, we may already have uploaded tools prior to getting to this
point.

https://codereview.appspot.com/14433058/diff/9001/provider/common/bootstrap.go#newcode76
provider/common/bootstrap.go:76: func SetBootstrapTools(env
environs.Environ, series string, arch *string) (coretools.List, error) {
bikeshed - perhaps call this either SetupBootstrapTools or
EnsureBootstrapTools. I think...

Read more...

Revision history for this message
Andrew Wilkins (axwalk) wrote :
Download full text (3.7 KiB)

Please take a look.

https://codereview.appspot.com/14433058/diff/9001/cmd/juju/bootstrap.go
File cmd/juju/bootstrap.go (right):

https://codereview.appspot.com/14433058/diff/9001/cmd/juju/bootstrap.go#newcode100
cmd/juju/bootstrap.go:100:
On 2013/11/17 23:48:46, wallyworld wrote:
> One implication of removing this check and running it as part of
> bootstrap.Bootstrap() below is that we may well end up uploading tools
only to
> then tell the user "sorry, already bootstrapped". This check really
does need to
> be run asap in the bootstrap process, before we do anything, so that
we fail as
> early as possible.

There are a couple of nice things about putting it inside Bootstrap, one
being that Bootstrap fails if the machine is already bootstrapped (no
need to test that property via the CLI command). But you're right, we
shouldn't break behaviour for --upload-tools. If nothing else, it'll
make developers' lives harder.

I'll revert this.

https://codereview.appspot.com/14433058/diff/9001/environs/bootstrap/bootstrap.go
File environs/bootstrap/bootstrap.go (right):

https://codereview.appspot.com/14433058/diff/9001/environs/bootstrap/bootstrap.go#newcode50
environs/bootstrap/bootstrap.go:50: // and update the agent-version
configuration attribute.
On 2013/11/17 23:48:46, wallyworld wrote:
> s/update/updates

Done.

https://codereview.appspot.com/14433058/diff/9001/environs/bootstrap/bootstrap.go#newcode51
environs/bootstrap/bootstrap.go:51: func SelectBootstrapTools(environ
environs.Environ, toolsList coretools.List) (coretools.List, error) {
On 2013/11/17 23:48:46, wallyworld wrote:
> could we rename toolsList to availableTools or possibleTools to better
reflect
> the purpose of the parameter? callers of this method tend to pass in a
> possibleTools variable so maybe that is the name to use

Done.

https://codereview.appspot.com/14433058/diff/9001/environs/manual/fakessh.go
File environs/manual/fakessh.go (right):

https://codereview.appspot.com/14433058/diff/9001/environs/manual/fakessh.go#newcode1
environs/manual/fakessh.go:1: // Copyright 2013 Canonical Ltd.
On 2013/11/17 23:48:46, wallyworld wrote:
> Why did this file get renamed from fakessh_test.go?
> The contents are only for testing. In general, we only use non
"_test.go" files
> for testing if there are methods called from outside the package (I
don't think
> that's the case here?); and if we do, we usually introduce a "testing"
package
> into which such files are put to make it clear that the contents are
for testing
> and not production business logic.

I had intended to use it in a separate package, but didn't end up doing
that. I'll revert this.

BTW: normally I'd put the test code in a separate package, but in this
case it needs to know some internals of the package (or otherwise we
expose parts that are even more private, such as the detection script).

https://codereview.appspot.com/14433058/diff/9001/provider/common/bootstrap.go
File provider/common/bootstrap.go (right):

https://codereview.appspot.com/14433058/diff/9001/provider/common/bootstrap.go#newcode29
provider/common/bootstrap.go:29: // before potentially uploading any
tools.
On 2013/11/17 23:48:46, wallyworld wrote:
> ...

Read more...

Revision history for this message
Ian Booth (wallyworld) wrote :

Thanks :-)
LGTM

https://codereview.appspot.com/14433058/diff/9001/cmd/juju/bootstrap.go
File cmd/juju/bootstrap.go (right):

https://codereview.appspot.com/14433058/diff/9001/cmd/juju/bootstrap.go#newcode100
cmd/juju/bootstrap.go:100:
On 2013/11/18 05:10:01, axw wrote:
> On 2013/11/17 23:48:46, wallyworld wrote:
> > One implication of removing this check and running it as part of
> > bootstrap.Bootstrap() below is that we may well end up uploading
tools only to
> > then tell the user "sorry, already bootstrapped". This check really
does need
> to
> > be run asap in the bootstrap process, before we do anything, so that
we fail
> as
> > early as possible.

> There are a couple of nice things about putting it inside Bootstrap,
one being
> that Bootstrap fails if the machine is already bootstrapped (no need
to test
> that property via the CLI command). But you're right, we shouldn't
break
> behaviour for --upload-tools. If nothing else, it'll make developers'
lives
> harder.

> I'll revert this.

Yeah, I agree it's a bit unfortunate. It's not just developers also -
people like Dave and Marco regularly run from source to get the latest
goodness.

https://codereview.appspot.com/14433058/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'cmd/juju/bootstrap.go'
2--- cmd/juju/bootstrap.go 2013-11-04 02:01:19 +0000
3+++ cmd/juju/bootstrap.go 2013-11-18 06:38:24 +0000
4@@ -5,11 +5,9 @@
5
6 import (
7 "fmt"
8- "io"
9 "strings"
10
11 "launchpad.net/gnuflag"
12- "launchpad.net/loggo"
13
14 "launchpad.net/juju-core/charm"
15 "launchpad.net/juju-core/cmd"
16@@ -19,10 +17,7 @@
17 "launchpad.net/juju-core/environs/config"
18 "launchpad.net/juju-core/environs/configstore"
19 "launchpad.net/juju-core/environs/sync"
20- envtools "launchpad.net/juju-core/environs/tools"
21- "launchpad.net/juju-core/errors"
22 "launchpad.net/juju-core/provider"
23- coretools "launchpad.net/juju-core/tools"
24 "launchpad.net/juju-core/utils/set"
25 "launchpad.net/juju-core/version"
26 )
27@@ -57,7 +52,6 @@
28 Constraints constraints.Value
29 UploadTools bool
30 Series []string
31- Source string
32 }
33
34 func (c *BootstrapCommand) Info() *cmd.Info {
35@@ -73,7 +67,6 @@
36 f.Var(constraints.ConstraintsValue{&c.Constraints}, "constraints", "set environment constraints")
37 f.BoolVar(&c.UploadTools, "upload-tools", false, "upload local version of tools before bootstrapping")
38 f.Var(seriesVar{&c.Series}, "series", "upload tools for supplied comma-separated series list")
39- f.StringVar(&c.Source, "source", "", "local path to use as tools source")
40 }
41
42 func (c *BootstrapCommand) Init(args []string) error {
43@@ -102,8 +95,6 @@
44 return fmt.Errorf("failed to enable bootstrap storage: %v", err)
45 }
46 }
47- // Check to see if the environment is already bootstrapped
48- // before potentially uploading any tools.
49 if err := bootstrap.EnsureNotBootstrapped(environ); err != nil {
50 return err
51 }
52@@ -120,10 +111,6 @@
53 return err
54 }
55 }
56- err = c.ensureToolsAvailability(environ, ctx, c.UploadTools)
57- if err != nil {
58- return err
59- }
60 return bootstrap.Bootstrap(environ, c.Constraints)
61 }
62
63@@ -133,7 +120,7 @@
64 forceVersion := uploadVersion(version.Current.Number, nil)
65 cfg := environ.Config()
66 series := getUploadSeries(cfg, c.Series)
67- agenttools, err := uploadTools(environ.Storage(), &forceVersion, series...)
68+ agenttools, err := sync.Upload(environ.Storage(), &forceVersion, series...)
69 if err != nil {
70 return err
71 }
72@@ -149,87 +136,6 @@
73 return nil
74 }
75
76-const NoToolsMessage = `
77-Juju cannot bootstrap because no tools are available for your environment.
78-An attempt was made to build and upload appropriate tools but this was unsuccessful.
79-
80-`
81-
82-const NoToolsNoUploadMessage = `
83-Juju cannot bootstrap because no tools are available for your environment.
84-In addition, no tools could be located to upload.
85-You may want to use the 'tools-metadata-url' configuration setting to specify the tools location.
86-
87-`
88-
89-func processToolsError(w io.Writer, err *error, uploadAttempted *bool) {
90-
91- if *uploadAttempted && *err != nil {
92- fmt.Fprint(w, NoToolsMessage)
93- } else {
94- if errors.IsNotFoundError(*err) || *err == coretools.ErrNoMatches {
95- fmt.Fprint(w, NoToolsNoUploadMessage)
96- }
97- }
98-}
99-
100-// ensureToolsAvailability verifies the tools are available. If no tools are
101-// found, it will automatically synchronize them.
102-func (c *BootstrapCommand) ensureToolsAvailability(env environs.Environ, ctx *cmd.Context, uploadPerformed bool) (err error) {
103- uploadAttempted := false
104- defer processToolsError(ctx.Stderr, &err, &uploadAttempted)
105- // Capture possible logging while syncing and write it on the screen.
106- loggo.RegisterWriter("bootstrap", cmd.NewCommandLogWriter("juju.environs.sync", ctx.Stdout, ctx.Stderr), loggo.INFO)
107- defer loggo.RemoveWriter("bootstrap")
108-
109- // Try to find bootstrap tools.
110- cfg := env.Config()
111- var vers *version.Number
112- if agentVersion, ok := cfg.AgentVersion(); ok {
113- vers = &agentVersion
114- }
115- logger.Debugf("looking for bootstrap tools")
116- params := envtools.BootstrapToolsParams{
117- Version: vers,
118- Arch: c.Constraints.Arch,
119- AllowRetry: uploadPerformed,
120- }
121- _, err = envtools.FindBootstrapTools(env, params)
122- if err == nil || !errors.IsNotFoundError(err) || uploadPerformed {
123- return err
124- }
125- // If no tools are available, synchronize if necessary.
126- uploadRequired := true
127- if c.Source != "" {
128- // If we are syncing tools and it succeeds, we don't need to upload the tools.
129- uploadRequired = false
130- logger.Warningf("no tools available, attempting to retrieve from %v", c.Source)
131- sctx := &sync.SyncContext{
132- Target: env.Storage(),
133- Source: c.Source,
134- }
135- if err = syncTools(sctx); err != nil {
136- if (err == coretools.ErrNoMatches || err == envtools.ErrNoTools) && vers == nil && version.Current.IsDev() {
137- uploadRequired = true
138- } else {
139- return err
140- }
141- }
142- }
143- // No suitable tools could be synced so try uploading.
144- if uploadRequired {
145- logger.Infof("no tools found, so attempting to build and upload new tools")
146- uploadAttempted = true
147- if err = c.uploadTools(env); err != nil {
148- return err
149- }
150- }
151- // Synchronization/upload done, try again.
152- params.AllowRetry = true
153- _, err = envtools.FindBootstrapTools(env, params)
154- return err
155-}
156-
157 type seriesVar struct {
158 target *[]string
159 }
160
161=== modified file 'cmd/juju/bootstrap_test.go'
162--- cmd/juju/bootstrap_test.go 2013-10-25 22:51:39 +0000
163+++ cmd/juju/bootstrap_test.go 2013-11-18 06:38:24 +0000
164@@ -46,6 +46,10 @@
165 s.LoggingSuite.SetUpTest(c)
166 s.MgoSuite.SetUpTest(c)
167 s.ToolsFixture.SetUpTest(c)
168+
169+ // Set up a local source with tools.
170+ sourceDir := createToolsSource(c, vAll)
171+ s.PatchValue(&sync.DefaultToolsLocation, sourceDir)
172 }
173
174 func (s *BootstrapSuite) TearDownSuite(c *gc.C) {
175@@ -76,24 +80,24 @@
176 var bootstrapRetryTests = []bootstrapRetryTest{{
177 info: "no tools uploaded, first check has no retries; no matching binary in source; sync fails with no second attempt",
178 expectedAllowRetry: []bool{false},
179- err: "no tools available",
180+ err: "cannot find bootstrap tools: no matching tools available",
181 version: "1.16.0-precise-amd64",
182 }, {
183 info: "no tools uploaded, first check has no retries; matching binary in source; check after sync has retries",
184 expectedAllowRetry: []bool{false, true},
185- err: "tools not found",
186+ err: "cannot find bootstrap tools: tools not found",
187 version: "1.16.0-precise-amd64",
188 addVersionToSource: true,
189 }, {
190 info: "no tools uploaded, first check has no retries; no matching binary in source; check after upload has retries",
191 expectedAllowRetry: []bool{false, true},
192- err: "tools not found",
193+ err: "cannot find bootstrap tools: tools not found",
194 version: "1.15.1-precise-amd64", // dev version to force upload
195 }, {
196 info: "new tools uploaded, so we want to allow retries to give them a chance at showing up",
197 args: []string{"--upload-tools"},
198 expectedAllowRetry: []bool{true},
199- err: "tools not found",
200+ err: "cannot find bootstrap tools: no matching tools available",
201 }}
202
203 // Test test checks that bootstrap calls FindTools with the expected allowRetry flag.
204@@ -105,20 +109,20 @@
205 }
206
207 func (s *BootstrapSuite) runAllowRetriesTest(c *gc.C, test bootstrapRetryTest) {
208- var extraVersions []version.Binary
209+ toolsVersions := envtesting.VAll
210 if test.version != "" {
211 testVersion := version.MustParseBinary(test.version)
212 restore := testbase.PatchValue(&version.Current, testVersion)
213 defer restore()
214 if test.addVersionToSource {
215- extraVersions = append(extraVersions, testVersion)
216+ toolsVersions = append([]version.Binary{}, toolsVersions...)
217+ toolsVersions = append(toolsVersions, testVersion)
218 }
219 }
220 _, fake := makeEmptyFakeHome(c)
221 defer fake.Restore()
222- // Create some source tools in a local directory.
223- sourceDir := createToolsSource(c, extraVersions)
224- test.args = append(test.args, []string{"--source", sourceDir}...)
225+ sourceDir := createToolsSource(c, toolsVersions)
226+ s.PatchValue(&sync.DefaultToolsLocation, sourceDir)
227
228 var findToolsRetryValues []bool
229 mockFindTools := func(cloudInst environs.ConfigGetter, majorVersion, minorVersion int,
230@@ -137,9 +141,7 @@
231 }
232
233 func (s *BootstrapSuite) TestTest(c *gc.C) {
234- uploadTools = mockUploadTools
235- defer func() { uploadTools = sync.Upload }()
236-
237+ s.PatchValue(&sync.Upload, mockUploadTools)
238 for i, test := range bootstrapTests {
239 c.Logf("\ntest %d: %s", i, test.info)
240 test.run(c)
241@@ -160,9 +162,6 @@
242 }
243
244 func (test bootstrapTest) run(c *gc.C) {
245- // Prepare a mock storage for testing.
246- sourceDir := createToolsSource(c, vAll)
247-
248 // Create home with dummy provider and remove all
249 // of its envtools.
250 env, fake := makeEmptyFakeHome(c)
251@@ -182,7 +181,6 @@
252 }
253
254 // Run command and check for uploads.
255- test.args = append(test.args, []string{"--source", sourceDir}...)
256 opc, errc := runCommand(nullContext(), new(BootstrapCommand), test.args...)
257 if uploadCount > 0 {
258 for i := 0; i < uploadCount; i++ {
259@@ -303,10 +301,9 @@
260 defer fake.Restore()
261 defaultSeriesVersion := version.Current
262 defaultSeriesVersion.Series = env.Config().DefaultSeries()
263- sourceDir := createToolsSource(c, append(vAll, defaultSeriesVersion))
264
265 ctx := coretesting.Context(c)
266- code := cmd.Main(&BootstrapCommand{}, ctx, []string{"--source", sourceDir})
267+ code := cmd.Main(&BootstrapCommand{}, ctx, nil)
268 c.Check(code, gc.Equals, 0)
269
270 ctx2 := coretesting.Context(c)
271@@ -316,51 +313,10 @@
272 c.Check(coretesting.Stdout(ctx2), gc.Equals, "")
273 }
274
275-func (s *BootstrapSuite) TestAutoSyncLocalSource(c *gc.C) {
276- // Prepare a tools directory for testing and store the
277- // dummy tools in there.
278- sourceDir := createToolsSource(c, vAll)
279-
280- // Change the version and ensure its later restoring.
281- origVersion := version.Current
282- version.Current.Number = version.MustParse("1.2.0")
283- defer func() {
284- version.Current = origVersion
285- }()
286-
287- // Create home with dummy provider and remove all
288- // of its envtools.
289- env, fake := makeEmptyFakeHome(c)
290- defer fake.Restore()
291-
292- // Bootstrap the environment with an invalid source.
293- // The command returns with an error.
294- ctx := coretesting.Context(c)
295- code := cmd.Main(&BootstrapCommand{}, ctx, []string{"--source", c.MkDir()})
296- c.Check(code, gc.Equals, 1)
297-
298- // Now check that there are no tools available.
299- _, err := envtools.FindTools(
300- env, version.Current.Major, version.Current.Minor, coretools.Filter{}, envtools.DoNotAllowRetry)
301- c.Assert(err, gc.FitsTypeOf, errors.NotFoundf(""))
302-
303- // Bootstrap the environment with the valid source. This time
304- // the bootstrapping has to show no error, because the tools
305- // are automatically synchronized.
306- ctx = coretesting.Context(c)
307- code = cmd.Main(&BootstrapCommand{}, ctx, []string{"--source", sourceDir})
308- c.Check(code, gc.Equals, 0)
309-
310- // Now check the available tools which are the 1.2.0 envtools.
311- checkTools(c, env, v120All)
312-}
313-
314-func (s *BootstrapSuite) setupAutoUploadTest(c *gc.C, vers, series string) (environs.Environ, string) {
315- uploadTools = mockUploadTools
316- s.AddCleanup(func(*gc.C) { uploadTools = sync.Upload })
317-
318- // Set up a local source with tools.
319- sourceDir := createToolsSource(c, vAll)
320+func (s *BootstrapSuite) setupAutoUploadTest(c *gc.C, vers, series string) environs.Environ {
321+ s.PatchValue(&sync.Upload, mockUploadTools)
322+ sourceDir := createToolsSource(c, vAll)
323+ s.PatchValue(&sync.DefaultToolsLocation, sourceDir)
324
325 // Change the tools location to be the test location and also
326 // the version and ensure their later restoring.
327@@ -375,7 +331,7 @@
328 // of its envtools.
329 env, fake := makeEmptyFakeHome(c)
330 s.AddCleanup(func(*gc.C) { fake.Restore() })
331- return env, sourceDir
332+ return env
333 }
334
335 func (s *BootstrapSuite) TestAutoUploadAfterFailedSync(c *gc.C) {
336@@ -383,9 +339,9 @@
337 if otherSeries == version.Current.Series {
338 otherSeries = "raring"
339 }
340- env, sourceDir := s.setupAutoUploadTest(c, "1.7.3", otherSeries)
341+ env := s.setupAutoUploadTest(c, "1.7.3", otherSeries)
342 // Run command and check for that upload has been run for tools matching the current juju version.
343- opc, errc := runCommand(nullContext(), new(BootstrapCommand), "--source", sourceDir)
344+ opc, errc := runCommand(nullContext(), new(BootstrapCommand))
345 c.Assert(<-errc, gc.IsNil)
346 c.Assert((<-opc).(dummy.OpPutFile).Env, gc.Equals, "peckham")
347 list, err := envtools.FindTools(env, version.Current.Major, version.Current.Minor, coretools.Filter{}, false)
348@@ -405,20 +361,20 @@
349 }
350
351 func (s *BootstrapSuite) TestAutoUploadOnlyForDev(c *gc.C) {
352- _, sourceDir := s.setupAutoUploadTest(c, "1.8.3", "precise")
353- _, errc := runCommand(nullContext(), new(BootstrapCommand), "--source", sourceDir)
354+ s.setupAutoUploadTest(c, "1.8.3", "precise")
355+ _, errc := runCommand(nullContext(), new(BootstrapCommand))
356 err := <-errc
357- c.Assert(err, gc.ErrorMatches, "no matching tools available")
358+ c.Assert(err, gc.ErrorMatches, "cannot find bootstrap tools: no matching tools available")
359 }
360
361 func (s *BootstrapSuite) TestMissingToolsError(c *gc.C) {
362- _, sourceDir := s.setupAutoUploadTest(c, "1.8.3", "precise")
363+ s.setupAutoUploadTest(c, "1.8.3", "precise")
364 context := coretesting.Context(c)
365- code := cmd.Main(&BootstrapCommand{}, context, []string{"--source", sourceDir})
366+ code := cmd.Main(&BootstrapCommand{}, context, nil)
367 c.Assert(code, gc.Equals, 1)
368 errText := context.Stderr.(*bytes.Buffer).String()
369 errText = strings.Replace(errText, "\n", "", -1)
370- expectedErrText := strings.Replace(fmt.Sprintf(".*%s.*", NoToolsNoUploadMessage), "\n", "", -1)
371+ expectedErrText := "error: cannot find bootstrap tools: no matching tools available"
372 c.Assert(errText, gc.Matches, expectedErrText)
373 }
374
375@@ -427,14 +383,14 @@
376 }
377
378 func (s *BootstrapSuite) TestMissingToolsUploadFailedError(c *gc.C) {
379- _, sourceDir := s.setupAutoUploadTest(c, "1.7.3", "precise")
380- uploadTools = uploadToolsAlwaysFails
381+ s.setupAutoUploadTest(c, "1.7.3", "precise")
382+ s.PatchValue(&sync.Upload, uploadToolsAlwaysFails)
383 context := coretesting.Context(c)
384- code := cmd.Main(&BootstrapCommand{}, context, []string{"--source", sourceDir})
385+ code := cmd.Main(&BootstrapCommand{}, context, nil)
386 c.Assert(code, gc.Equals, 1)
387 errText := context.Stderr.(*bytes.Buffer).String()
388 errText = strings.Replace(errText, "\n", "", -1)
389- expectedErrText := strings.Replace(fmt.Sprintf(".*%s.*", NoToolsMessage), "\n", "", -1)
390+ expectedErrText := "error: cannot find bootstrap tools: an error"
391 c.Assert(errText, gc.Matches, expectedErrText)
392 }
393
394
395=== modified file 'cmd/juju/upgradejuju.go'
396--- cmd/juju/upgradejuju.go 2013-11-14 09:07:52 +0000
397+++ cmd/juju/upgradejuju.go 2013-11-18 06:38:24 +0000
398@@ -31,8 +31,6 @@
399 Series []string
400 }
401
402-var uploadTools = sync.Upload
403-
404 var upgradeJujuDoc = `
405 The upgrade-juju command upgrades a running environment by setting a version
406 number for all juju agents to run. By default, it chooses the most recent
407@@ -226,7 +224,7 @@
408 // include all the extra series we build, so we can set *that* onto
409 // v.available and maybe one day be able to check that a given upgrade
410 // won't leave out-of-date machines lying around, starved of envtools.
411- uploaded, err := uploadTools(storage, &v.chosen, series...)
412+ uploaded, err := sync.Upload(storage, &v.chosen, series...)
413 if err != nil {
414 return err
415 }
416
417=== modified file 'cmd/juju/upgradejuju_test.go'
418--- cmd/juju/upgradejuju_test.go 2013-11-14 09:07:52 +0000
419+++ cmd/juju/upgradejuju_test.go 2013-11-18 06:38:24 +0000
420@@ -273,11 +273,10 @@
421 }
422
423 func (s *UpgradeJujuSuite) TestUpgradeJuju(c *gc.C) {
424+ s.PatchValue(&sync.Upload, mockUploadTools)
425 oldVersion := version.Current
426- uploadTools = mockUploadTools
427 defer func() {
428 version.Current = oldVersion
429- uploadTools = sync.Upload
430 }()
431
432 for i, test := range upgradeJujuTests {
433
434=== modified file 'cmd/jujud/bootstrap.go'
435--- cmd/jujud/bootstrap.go 2013-10-04 17:10:00 +0000
436+++ cmd/jujud/bootstrap.go 2013-11-18 06:38:24 +0000
437@@ -15,10 +15,10 @@
438 "launchpad.net/juju-core/agent"
439 "launchpad.net/juju-core/cmd"
440 "launchpad.net/juju-core/constraints"
441+ "launchpad.net/juju-core/environs/bootstrap"
442 "launchpad.net/juju-core/environs/cloudinit"
443 "launchpad.net/juju-core/environs/config"
444 "launchpad.net/juju-core/instance"
445- "launchpad.net/juju-core/provider/common"
446 "launchpad.net/juju-core/state"
447 )
448
449@@ -62,7 +62,7 @@
450 return fmt.Errorf("cannot read provider-state-url file: %v", err)
451 }
452 stateInfoURL := strings.Split(string(data), "\n")[0]
453- bsState, err := common.LoadStateFromURL(stateInfoURL)
454+ bsState, err := bootstrap.LoadStateFromURL(stateInfoURL)
455 if err != nil {
456 return fmt.Errorf("cannot load state from URL %q (read from %q): %v", stateInfoURL, providerStateURLFile, err)
457 }
458
459=== modified file 'cmd/jujud/bootstrap_test.go'
460--- cmd/jujud/bootstrap_test.go 2013-11-06 13:38:01 +0000
461+++ cmd/jujud/bootstrap_test.go 2013-11-18 06:38:24 +0000
462@@ -13,10 +13,10 @@
463
464 "launchpad.net/juju-core/agent"
465 "launchpad.net/juju-core/constraints"
466+ "launchpad.net/juju-core/environs/bootstrap"
467 "launchpad.net/juju-core/environs/jujutest"
468 "launchpad.net/juju-core/errors"
469 "launchpad.net/juju-core/instance"
470- "launchpad.net/juju-core/provider/common"
471 "launchpad.net/juju-core/provider/dummy"
472 "launchpad.net/juju-core/state"
473 "launchpad.net/juju-core/testing"
474@@ -46,12 +46,12 @@
475 func (s *BootstrapSuite) SetUpSuite(c *gc.C) {
476 s.LoggingSuite.SetUpSuite(c)
477 s.MgoSuite.SetUpSuite(c)
478- stateInfo := common.BootstrapState{
479+ stateInfo := bootstrap.BootstrapState{
480 StateInstances: []instance.Id{instance.Id("dummy.instance.id")},
481 }
482 stateData, err := goyaml.Marshal(stateInfo)
483 c.Assert(err, gc.IsNil)
484- content := map[string]string{"/" + common.StateFile: string(stateData)}
485+ content := map[string]string{"/" + bootstrap.StateFile: string(stateData)}
486 testRoundTripper.Sub = jujutest.NewCannedRoundTripper(content, nil)
487 s.providerStateURLFile = filepath.Join(c.MkDir(), "provider-state-url")
488 providerStateURLFile = s.providerStateURLFile
489
490=== modified file 'environs/bootstrap/bootstrap.go'
491--- environs/bootstrap/bootstrap.go 2013-10-04 16:16:10 +0000
492+++ environs/bootstrap/bootstrap.go 2013-11-18 06:38:24 +0000
493@@ -10,12 +10,11 @@
494
495 "launchpad.net/juju-core/constraints"
496 "launchpad.net/juju-core/environs"
497- "launchpad.net/juju-core/environs/tools"
498- "launchpad.net/juju-core/provider/common"
499+ coretools "launchpad.net/juju-core/tools"
500 "launchpad.net/juju-core/version"
501 )
502
503-var logger = loggo.GetLogger("juju.environs.boostrap")
504+var logger = loggo.GetLogger("juju.environs.bootstrap")
505
506 // Bootstrap bootstraps the given environment. The supplied constraints are
507 // used to provision the instance, and are also set within the bootstrapped
508@@ -43,51 +42,40 @@
509 if err := environs.VerifyStorage(environ.Storage()); err != nil {
510 return err
511 }
512-
513 logger.Infof("bootstrapping environment %q", environ.Name())
514- var vers *version.Number
515- if agentVersion, ok := cfg.AgentVersion(); ok {
516- vers = &agentVersion
517- }
518- params := tools.BootstrapToolsParams{
519- Version: vers,
520- Arch: cons.Arch,
521- }
522- newestTools, err := tools.FindBootstrapTools(environ, params)
523- if err != nil {
524- return fmt.Errorf("cannot find bootstrap tools: %v", err)
525- }
526+ return environ.Bootstrap(cons)
527+}
528
529- // If agent version was not previously known, set it here using the latest compatible tools version.
530- if vers == nil {
531- // We probably still have a mix of versions available; discard older ones
532- // and update environment configuration to use only those remaining.
533- var newVersion version.Number
534- newVersion, newestTools = newestTools.Newest()
535- vers = &newVersion
536- logger.Infof("environs: picked newest version: %s", *vers)
537- cfg, err = cfg.Apply(map[string]interface{}{
538- "agent-version": vers.String(),
539+// SetBootstrapTools returns the newest tools from the given tools list,
540+// and updates the agent-version configuration attribute.
541+func SetBootstrapTools(environ environs.Environ, possibleTools coretools.List) (coretools.List, error) {
542+ if len(possibleTools) == 0 {
543+ return nil, fmt.Errorf("no bootstrap tools available")
544+ }
545+ var newVersion version.Number
546+ newVersion, toolsList := possibleTools.Newest()
547+ logger.Infof("picked newest version: %s", newVersion)
548+ cfg := environ.Config()
549+ if agentVersion, _ := cfg.AgentVersion(); agentVersion != newVersion {
550+ cfg, err := cfg.Apply(map[string]interface{}{
551+ "agent-version": newVersion.String(),
552 })
553 if err == nil {
554 err = environ.SetConfig(cfg)
555 }
556 if err != nil {
557- return fmt.Errorf("failed to update environment configuration: %v", err)
558+ return nil, fmt.Errorf("failed to update environment configuration: %v", err)
559 }
560 }
561- // ensure we have at least one valid tools
562- if len(newestTools) == 0 {
563- return fmt.Errorf("No bootstrap tools found")
564- }
565- return environ.Bootstrap(cons, newestTools)
566+ return toolsList, nil
567 }
568
569 // EnsureNotBootstrapped returns null if the environment is not bootstrapped,
570 // and an error if it is or if the function was not able to tell.
571 func EnsureNotBootstrapped(env environs.Environ) error {
572- _, err := common.LoadState(env.Storage())
573- // If there is no error loading the bootstrap state, then we are bootstrapped.
574+ _, err := LoadState(env.Storage())
575+ // If there is no error loading the bootstrap state, then we are
576+ // bootstrapped.
577 if err == nil {
578 return fmt.Errorf("environment is already bootstrapped")
579 }
580
581=== modified file 'environs/bootstrap/bootstrap_test.go'
582--- environs/bootstrap/bootstrap_test.go 2013-10-24 00:20:59 +0000
583+++ environs/bootstrap/bootstrap_test.go 2013-11-18 06:38:24 +0000
584@@ -21,7 +21,6 @@
585 "launchpad.net/juju-core/provider/dummy"
586 coretesting "launchpad.net/juju-core/testing"
587 "launchpad.net/juju-core/testing/testbase"
588- "launchpad.net/juju-core/tools"
589 "launchpad.net/juju-core/version"
590 )
591
592@@ -183,11 +182,21 @@
593 }
594 }
595
596-func (s *bootstrapSuite) TestBootstrapNeedsTools(c *gc.C) {
597+func (s *bootstrapSuite) TestBootstrapNoTools(c *gc.C) {
598 env := newEnviron("foo", useDefaultKeys)
599 s.setDummyStorage(c, env)
600 envtesting.RemoveFakeTools(c, env.Storage())
601 err := bootstrap.Bootstrap(env, constraints.Value{})
602+ // bootstrap.Bootstrap leaves it to the provider to
603+ // locate bootstrap tools.
604+ c.Assert(err, gc.IsNil)
605+}
606+
607+func (s *bootstrapSuite) TestEnsureToolsAvailability(c *gc.C) {
608+ env := newEnviron("foo", useDefaultKeys)
609+ s.setDummyStorage(c, env)
610+ envtesting.RemoveFakeTools(c, env.Storage())
611+ _, err := bootstrap.EnsureToolsAvailability(env, env.Config().DefaultSeries(), nil)
612 c.Check(err, gc.ErrorMatches, "cannot find bootstrap tools: invalid URL.*")
613 }
614
615@@ -243,7 +252,7 @@
616 return e.name
617 }
618
619-func (e *bootstrapEnviron) Bootstrap(cons constraints.Value, possibleTools tools.List) error {
620+func (e *bootstrapEnviron) Bootstrap(cons constraints.Value) error {
621 e.bootstrapCount++
622 e.constraints = cons
623 return nil
624
625=== added file 'environs/bootstrap/state.go'
626--- environs/bootstrap/state.go 1970-01-01 00:00:00 +0000
627+++ environs/bootstrap/state.go 2013-11-18 06:38:24 +0000
628@@ -0,0 +1,97 @@
629+// Copyright 2012, 2013 Canonical Ltd.
630+// Licensed under the AGPLv3, see LICENCE file for details.
631+
632+package bootstrap
633+
634+import (
635+ "bytes"
636+ "fmt"
637+ "io"
638+ "io/ioutil"
639+ "net/http"
640+
641+ "launchpad.net/goyaml"
642+
643+ "launchpad.net/juju-core/environs"
644+ "launchpad.net/juju-core/environs/storage"
645+ coreerrors "launchpad.net/juju-core/errors"
646+ "launchpad.net/juju-core/instance"
647+)
648+
649+// StateFile is the name of the file where the provider's state is stored.
650+const StateFile = "provider-state"
651+
652+// BootstrapState is the state information that is stored in StateFile.
653+//
654+// Individual providers may define their own state structures instead of
655+// this one, and use their own code for loading and saving those, but this is
656+// the definition that most practically useful providers share unchanged.
657+type BootstrapState struct {
658+ // StateInstances are the state servers.
659+ StateInstances []instance.Id `yaml:"state-instances"`
660+ // Characteristics reflect the hardware each state server is running on.
661+ // This is used at bootstrap time so the state server knows what hardware it has.
662+ // The state *may* be updated later without this information, but by then it's
663+ // served it's purpose.
664+ Characteristics []instance.HardwareCharacteristics `yaml:"characteristics,omitempty"`
665+}
666+
667+// putState writes the given data to the state file on the given storage.
668+// The file's name is as defined in StateFile.
669+func putState(storage storage.StorageWriter, data []byte) error {
670+ return storage.Put(StateFile, bytes.NewBuffer(data), int64(len(data)))
671+}
672+
673+// CreateStateFile creates an empty state file on the given storage, and
674+// returns its URL.
675+func CreateStateFile(storage storage.Storage) (string, error) {
676+ err := putState(storage, []byte{})
677+ if err != nil {
678+ return "", fmt.Errorf("cannot create initial state file: %v", err)
679+ }
680+ return storage.URL(StateFile)
681+}
682+
683+// SaveState writes the given state to the given storage.
684+func SaveState(storage storage.StorageWriter, state *BootstrapState) error {
685+ data, err := goyaml.Marshal(state)
686+ if err != nil {
687+ return err
688+ }
689+ return putState(storage, data)
690+}
691+
692+// LoadStateFromURL reads state from the given URL.
693+func LoadStateFromURL(url string) (*BootstrapState, error) {
694+ resp, err := http.Get(url)
695+ if err != nil {
696+ return nil, err
697+ }
698+ return loadState(resp.Body)
699+}
700+
701+// LoadState reads state from the given storage.
702+func LoadState(stor storage.StorageReader) (*BootstrapState, error) {
703+ r, err := storage.Get(stor, StateFile)
704+ if err != nil {
705+ if coreerrors.IsNotFoundError(err) {
706+ return nil, environs.ErrNotBootstrapped
707+ }
708+ return nil, err
709+ }
710+ return loadState(r)
711+}
712+
713+func loadState(r io.ReadCloser) (*BootstrapState, error) {
714+ defer r.Close()
715+ data, err := ioutil.ReadAll(r)
716+ if err != nil {
717+ return nil, fmt.Errorf("error reading %q: %v", StateFile, err)
718+ }
719+ var state BootstrapState
720+ err = goyaml.Unmarshal(data, &state)
721+ if err != nil {
722+ return nil, fmt.Errorf("error unmarshalling %q: %v", StateFile, err)
723+ }
724+ return &state, nil
725+}
726
727=== added file 'environs/bootstrap/state_test.go'
728--- environs/bootstrap/state_test.go 1970-01-01 00:00:00 +0000
729+++ environs/bootstrap/state_test.go 2013-11-18 06:38:24 +0000
730@@ -0,0 +1,118 @@
731+// Copyright 2013 Canonical Ltd.
732+// Licensed under the AGPLv3, see LICENCE file for details.
733+
734+package bootstrap_test
735+
736+import (
737+ "bytes"
738+ "io/ioutil"
739+
740+ gc "launchpad.net/gocheck"
741+ "launchpad.net/goyaml"
742+
743+ "launchpad.net/juju-core/environs"
744+ "launchpad.net/juju-core/environs/bootstrap"
745+ "launchpad.net/juju-core/environs/storage"
746+ envtesting "launchpad.net/juju-core/environs/testing"
747+ "launchpad.net/juju-core/instance"
748+ "launchpad.net/juju-core/testing/testbase"
749+)
750+
751+type StateSuite struct {
752+ testbase.LoggingSuite
753+}
754+
755+var _ = gc.Suite(&StateSuite{})
756+
757+func (suite *StateSuite) newStorage(c *gc.C) storage.Storage {
758+ closer, stor, _ := envtesting.CreateLocalTestStorage(c)
759+ suite.AddCleanup(func(*gc.C) { closer.Close() })
760+ envtesting.UploadFakeTools(c, stor)
761+ return stor
762+}
763+
764+func (suite *StateSuite) TestCreateStateFileWritesEmptyStateFile(c *gc.C) {
765+ stor := suite.newStorage(c)
766+
767+ url, err := bootstrap.CreateStateFile(stor)
768+ c.Assert(err, gc.IsNil)
769+
770+ reader, err := storage.Get(stor, bootstrap.StateFile)
771+ c.Assert(err, gc.IsNil)
772+ data, err := ioutil.ReadAll(reader)
773+ c.Assert(err, gc.IsNil)
774+ c.Check(string(data), gc.Equals, "")
775+ c.Assert(url, gc.NotNil)
776+ expectedURL, err := stor.URL(bootstrap.StateFile)
777+ c.Assert(err, gc.IsNil)
778+ c.Check(url, gc.Equals, expectedURL)
779+}
780+
781+func (suite *StateSuite) TestSaveStateWritesStateFile(c *gc.C) {
782+ stor := suite.newStorage(c)
783+ arch := "amd64"
784+ state := bootstrap.BootstrapState{
785+ StateInstances: []instance.Id{instance.Id("an-instance-id")},
786+ Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}}
787+ marshaledState, err := goyaml.Marshal(state)
788+ c.Assert(err, gc.IsNil)
789+
790+ err = bootstrap.SaveState(stor, &state)
791+ c.Assert(err, gc.IsNil)
792+
793+ loadedState, err := storage.Get(stor, bootstrap.StateFile)
794+ c.Assert(err, gc.IsNil)
795+ content, err := ioutil.ReadAll(loadedState)
796+ c.Assert(err, gc.IsNil)
797+ c.Check(content, gc.DeepEquals, marshaledState)
798+}
799+
800+func (suite *StateSuite) setUpSavedState(c *gc.C, stor storage.Storage) bootstrap.BootstrapState {
801+ arch := "amd64"
802+ state := bootstrap.BootstrapState{
803+ StateInstances: []instance.Id{instance.Id("an-instance-id")},
804+ Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}}
805+ content, err := goyaml.Marshal(state)
806+ c.Assert(err, gc.IsNil)
807+ err = stor.Put(bootstrap.StateFile, ioutil.NopCloser(bytes.NewReader(content)), int64(len(content)))
808+ c.Assert(err, gc.IsNil)
809+ return state
810+}
811+
812+func (suite *StateSuite) TestLoadStateReadsStateFile(c *gc.C) {
813+ storage := suite.newStorage(c)
814+ state := suite.setUpSavedState(c, storage)
815+ storedState, err := bootstrap.LoadState(storage)
816+ c.Assert(err, gc.IsNil)
817+ c.Check(*storedState, gc.DeepEquals, state)
818+}
819+
820+func (suite *StateSuite) TestLoadStateFromURLReadsStateFile(c *gc.C) {
821+ stor := suite.newStorage(c)
822+ state := suite.setUpSavedState(c, stor)
823+ url, err := stor.URL(bootstrap.StateFile)
824+ c.Assert(err, gc.IsNil)
825+ storedState, err := bootstrap.LoadStateFromURL(url)
826+ c.Assert(err, gc.IsNil)
827+ c.Check(*storedState, gc.DeepEquals, state)
828+}
829+
830+func (suite *StateSuite) TestLoadStateMissingFile(c *gc.C) {
831+ stor := suite.newStorage(c)
832+ _, err := bootstrap.LoadState(stor)
833+ c.Check(err, gc.Equals, environs.ErrNotBootstrapped)
834+}
835+
836+func (suite *StateSuite) TestLoadStateIntegratesWithSaveState(c *gc.C) {
837+ storage := suite.newStorage(c)
838+ arch := "amd64"
839+ state := bootstrap.BootstrapState{
840+ StateInstances: []instance.Id{instance.Id("an-instance-id")},
841+ Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}}
842+ err := bootstrap.SaveState(storage, &state)
843+ c.Assert(err, gc.IsNil)
844+ storedState, err := bootstrap.LoadState(storage)
845+ c.Assert(err, gc.IsNil)
846+
847+ c.Check(*storedState, gc.DeepEquals, state)
848+}
849
850=== added file 'environs/bootstrap/synctools.go'
851--- environs/bootstrap/synctools.go 1970-01-01 00:00:00 +0000
852+++ environs/bootstrap/synctools.go 2013-11-18 06:38:24 +0000
853@@ -0,0 +1,123 @@
854+// Copyright 2012, 2013 Canonical Ltd.
855+// Licensed under the AGPLv3, see LICENCE file for details.
856+
857+package bootstrap
858+
859+import (
860+ "fmt"
861+
862+ "launchpad.net/juju-core/environs"
863+ "launchpad.net/juju-core/environs/config"
864+ "launchpad.net/juju-core/environs/sync"
865+ envtools "launchpad.net/juju-core/environs/tools"
866+ "launchpad.net/juju-core/errors"
867+ coretools "launchpad.net/juju-core/tools"
868+ "launchpad.net/juju-core/utils/set"
869+ "launchpad.net/juju-core/version"
870+)
871+
872+const noToolsMessage = `Juju cannot bootstrap because no tools are available for your environment.
873+An attempt was made to build and upload appropriate tools but this was unsuccessful.
874+`
875+
876+const noToolsNoUploadMessage = `Juju cannot bootstrap because no tools are available for your environment.
877+In addition, no tools could be located to upload.
878+You may want to use the 'tools-metadata-url' configuration setting to specify the tools location.
879+`
880+
881+// syncOrUpload first attempts to synchronize tools from
882+// the default tools source to the environment's storage.
883+//
884+// If synchronization fails due to no matching tools,
885+// a development version of juju is running, and no
886+// agent-version has been specified, then attempt to
887+// build and upload local tools.
888+func syncOrUpload(env environs.Environ, bootstrapSeries string) error {
889+ sctx := &sync.SyncContext{
890+ Target: env.Storage(),
891+ }
892+ err := sync.SyncTools(sctx)
893+ if err == coretools.ErrNoMatches || err == envtools.ErrNoTools {
894+ if _, hasAgentVersion := env.Config().AgentVersion(); !hasAgentVersion && version.Current.IsDev() {
895+ logger.Warningf("no tools found, so attempting to build and upload new tools")
896+ if err = uploadTools(env, bootstrapSeries); err != nil {
897+ logger.Errorf("%s", noToolsMessage)
898+ return err
899+ }
900+ } else {
901+ logger.Errorf("%s", noToolsNoUploadMessage)
902+ }
903+ }
904+ return err
905+}
906+
907+func uploadTools(env environs.Environ, bootstrapSeries string) error {
908+ cfg := env.Config()
909+ uploadVersion := version.Current.Number
910+ uploadVersion.Build++
911+ uploadSeries := set.NewStrings(
912+ bootstrapSeries,
913+ cfg.DefaultSeries(),
914+ config.DefaultSeries,
915+ )
916+ tools, err := sync.Upload(env.Storage(), &uploadVersion, uploadSeries.Values()...)
917+ if err != nil {
918+ return err
919+ }
920+ cfg, err = cfg.Apply(map[string]interface{}{
921+ "agent-version": tools.Version.Number.String(),
922+ })
923+ if err == nil {
924+ err = env.SetConfig(cfg)
925+ }
926+ if err != nil {
927+ return fmt.Errorf("failed to update environment configuration: %v", err)
928+ }
929+ return nil
930+}
931+
932+// EnsureToolsAvailability verifies the tools are available. If no tools are
933+// found, it will automatically synchronize them.
934+func EnsureToolsAvailability(env environs.Environ, series string, arch *string) (coretools.List, error) {
935+ cfg := env.Config()
936+ var vers *version.Number
937+ if agentVersion, ok := cfg.AgentVersion(); ok {
938+ vers = &agentVersion
939+ }
940+
941+ logger.Debugf(
942+ "looking for bootstrap tools: series=%q, arch=%v, version=%v",
943+ series, arch, vers,
944+ )
945+ params := envtools.BootstrapToolsParams{
946+ Version: vers,
947+ Arch: arch,
948+ Series: series,
949+ // If vers.Build>0, the tools may have been uploaded in this session.
950+ // Allow retries, so we wait until the storage has caught up.
951+ AllowRetry: vers != nil && vers.Build > 0,
952+ }
953+ toolsList, err := envtools.FindBootstrapTools(env, params)
954+ if err == nil {
955+ return toolsList, nil
956+ } else if !errors.IsNotFoundError(err) {
957+ return nil, err
958+ }
959+
960+ // No tools available, so synchronize.
961+ logger.Warningf("no tools available, attempting to retrieve from %v", sync.DefaultToolsLocation)
962+ if syncErr := syncOrUpload(env, series); syncErr != nil {
963+ // The target may have tools that don't match, so don't
964+ // return a misleading "no tools found" error.
965+ if syncErr != envtools.ErrNoTools {
966+ err = syncErr
967+ }
968+ return nil, fmt.Errorf("cannot find bootstrap tools: %v", err)
969+ }
970+ // TODO(axw) have syncOrUpload return the list of tools in the target, and use that.
971+ params.AllowRetry = true
972+ if toolsList, err = envtools.FindBootstrapTools(env, params); err != nil {
973+ return nil, fmt.Errorf("cannot find bootstrap tools: %v", err)
974+ }
975+ return toolsList, nil
976+}
977
978=== modified file 'environs/interface.go'
979--- environs/interface.go 2013-10-25 22:24:35 +0000
980+++ environs/interface.go 2013-11-18 06:38:24 +0000
981@@ -10,7 +10,6 @@
982 "launchpad.net/juju-core/instance"
983 "launchpad.net/juju-core/state"
984 "launchpad.net/juju-core/state/api"
985- "launchpad.net/juju-core/tools"
986 )
987
988 // A EnvironProvider represents a computing and storage provider.
989@@ -133,7 +132,11 @@
990 //
991 // The supplied constraints are used to choose the initial instance
992 // specification, and will be stored in the new environment's state.
993- Bootstrap(cons constraints.Value, possibleTools tools.List) error
994+ //
995+ // Bootstrap is responsible for selecting the appropriate tools,
996+ // and setting the agent-version configuration attribute prior to
997+ // bootstrapping the environment.
998+ Bootstrap(cons constraints.Value) error
999
1000 // StateInfo returns information on the state initialized
1001 // by Bootstrap.
1002
1003=== modified file 'environs/jujutest/livetests.go'
1004--- environs/jujutest/livetests.go 2013-10-03 01:28:51 +0000
1005+++ environs/jujutest/livetests.go 2013-11-18 06:38:24 +0000
1006@@ -27,6 +27,7 @@
1007 "launchpad.net/juju-core/instance"
1008 "launchpad.net/juju-core/juju"
1009 "launchpad.net/juju-core/juju/testing"
1010+ "launchpad.net/juju-core/provider/common"
1011 "launchpad.net/juju-core/provider/dummy"
1012 "launchpad.net/juju-core/state"
1013 "launchpad.net/juju-core/state/api"
1014@@ -137,7 +138,7 @@
1015 c.Assert(err, gc.IsNil)
1016 }
1017 envtesting.UploadFakeTools(c, t.Env.Storage())
1018- err := bootstrap.EnsureNotBootstrapped(t.Env)
1019+ err := common.EnsureNotBootstrapped(t.Env)
1020 c.Assert(err, gc.IsNil)
1021 err = bootstrap.Bootstrap(t.Env, cons)
1022 c.Assert(err, gc.IsNil)
1023@@ -391,7 +392,7 @@
1024 // already up, this has been moved into the bootstrap command.
1025 t.BootstrapOnce(c)
1026
1027- err := bootstrap.EnsureNotBootstrapped(t.Env)
1028+ err := common.EnsureNotBootstrapped(t.Env)
1029 c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped")
1030
1031 c.Logf("destroy env")
1032
1033=== modified file 'environs/jujutest/tests.go'
1034--- environs/jujutest/tests.go 2013-10-03 01:28:51 +0000
1035+++ environs/jujutest/tests.go 2013-11-18 06:38:24 +0000
1036@@ -21,6 +21,7 @@
1037 "launchpad.net/juju-core/errors"
1038 "launchpad.net/juju-core/instance"
1039 "launchpad.net/juju-core/juju/testing"
1040+ "launchpad.net/juju-core/provider/common"
1041 coretesting "launchpad.net/juju-core/testing"
1042 jc "launchpad.net/juju-core/testing/checkers"
1043 "launchpad.net/juju-core/testing/testbase"
1044@@ -131,7 +132,7 @@
1045 func (t *Tests) TestBootstrap(c *gc.C) {
1046 e := t.Prepare(c)
1047 envtesting.UploadFakeTools(c, e.Storage())
1048- err := bootstrap.EnsureNotBootstrapped(e)
1049+ err := common.EnsureNotBootstrapped(e)
1050 c.Assert(err, gc.IsNil)
1051 err = bootstrap.Bootstrap(e, constraints.Value{})
1052 c.Assert(err, gc.IsNil)
1053@@ -140,12 +141,12 @@
1054 c.Check(info.Addrs, gc.Not(gc.HasLen), 0)
1055 c.Check(apiInfo.Addrs, gc.Not(gc.HasLen), 0)
1056
1057- err = bootstrap.EnsureNotBootstrapped(e)
1058+ err = common.EnsureNotBootstrapped(e)
1059 c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped")
1060
1061 e2 := t.Open(c)
1062 envtesting.UploadFakeTools(c, e2.Storage())
1063- err = bootstrap.EnsureNotBootstrapped(e2)
1064+ err = common.EnsureNotBootstrapped(e2)
1065 c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped")
1066
1067 info2, apiInfo2, err := e2.StateInfo()
1068@@ -159,12 +160,12 @@
1069 e3 := t.Prepare(c)
1070 envtesting.UploadFakeTools(c, e3.Storage())
1071
1072- err = bootstrap.EnsureNotBootstrapped(e3)
1073+ err = common.EnsureNotBootstrapped(e3)
1074 c.Assert(err, gc.IsNil)
1075 err = bootstrap.Bootstrap(e3, constraints.Value{})
1076 c.Assert(err, gc.IsNil)
1077
1078- err = bootstrap.EnsureNotBootstrapped(e3)
1079+ err = common.EnsureNotBootstrapped(e3)
1080 c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped")
1081 }
1082
1083
1084=== modified file 'environs/manual/bootstrap.go'
1085--- environs/manual/bootstrap.go 2013-11-14 08:27:15 +0000
1086+++ environs/manual/bootstrap.go 2013-11-18 06:38:24 +0000
1087@@ -9,9 +9,9 @@
1088
1089 "launchpad.net/juju-core/constraints"
1090 "launchpad.net/juju-core/environs"
1091+ "launchpad.net/juju-core/environs/bootstrap"
1092 envtools "launchpad.net/juju-core/environs/tools"
1093 "launchpad.net/juju-core/instance"
1094- "launchpad.net/juju-core/provider/common"
1095 "launchpad.net/juju-core/tools"
1096 "launchpad.net/juju-core/worker/localstorage"
1097 )
1098@@ -30,6 +30,13 @@
1099 DataDir string
1100 Environ LocalStorageEnviron
1101 PossibleTools tools.List
1102+
1103+ // If series and hardware characteristics
1104+ // are known ahead of time, they can be
1105+ // set here and Bootstrap will not attempt
1106+ // to detect them again.
1107+ Series string
1108+ HardwareCharacteristics *instance.HardwareCharacteristics
1109 }
1110
1111 func errMachineIdInvalid(machineId string) error {
1112@@ -58,9 +65,16 @@
1113 return ErrProvisioned
1114 }
1115
1116- hc, series, err := detectSeriesAndHardwareCharacteristics(args.Host)
1117- if err != nil {
1118- return fmt.Errorf("error detecting hardware characteristics: %v", err)
1119+ var series string
1120+ var hc instance.HardwareCharacteristics
1121+ if args.Series != "" && args.HardwareCharacteristics != nil {
1122+ series = args.Series
1123+ hc = *args.HardwareCharacteristics
1124+ } else {
1125+ hc, series, err = DetectSeriesAndHardwareCharacteristics(args.Host)
1126+ if err != nil {
1127+ return fmt.Errorf("error detecting hardware characteristics: %v", err)
1128+ }
1129 }
1130
1131 // Filter tools based on detected series/arch.
1132@@ -76,9 +90,9 @@
1133 // Store the state file. If provisioning fails, we'll remove the file.
1134 logger.Infof("Saving bootstrap state file to bootstrap storage")
1135 bootstrapStorage := args.Environ.Storage()
1136- err = common.SaveState(
1137+ err = bootstrap.SaveState(
1138 bootstrapStorage,
1139- &common.BootstrapState{
1140+ &bootstrap.BootstrapState{
1141 StateInstances: []instance.Id{BootstrapInstanceId},
1142 Characteristics: []instance.HardwareCharacteristics{hc},
1143 },
1144@@ -89,7 +103,7 @@
1145 defer func() {
1146 if err != nil {
1147 logger.Errorf("bootstrapping failed, removing state file: %v", err)
1148- bootstrapStorage.Remove(common.StateFile)
1149+ bootstrapStorage.Remove(bootstrap.StateFile)
1150 }
1151 }()
1152
1153@@ -107,7 +121,7 @@
1154 }
1155
1156 // Finally, provision the machine agent.
1157- stateFileURL := fmt.Sprintf("file://%s/%s", storageDir, common.StateFile)
1158+ stateFileURL := fmt.Sprintf("file://%s/%s", storageDir, bootstrap.StateFile)
1159 mcfg := environs.NewBootstrapMachineConfig(stateFileURL)
1160 if args.DataDir != "" {
1161 mcfg.DataDir = args.DataDir
1162
1163=== modified file 'environs/manual/bootstrap_test.go'
1164--- environs/manual/bootstrap_test.go 2013-10-02 10:39:12 +0000
1165+++ environs/manual/bootstrap_test.go 2013-11-18 06:38:24 +0000
1166@@ -9,12 +9,12 @@
1167 gc "launchpad.net/gocheck"
1168
1169 "launchpad.net/juju-core/environs"
1170+ "launchpad.net/juju-core/environs/bootstrap"
1171 "launchpad.net/juju-core/environs/filestorage"
1172 "launchpad.net/juju-core/environs/storage"
1173 "launchpad.net/juju-core/environs/tools"
1174 "launchpad.net/juju-core/instance"
1175 "launchpad.net/juju-core/juju/testing"
1176- "launchpad.net/juju-core/provider/common"
1177 )
1178
1179 type bootstrapSuite struct {
1180@@ -81,11 +81,11 @@
1181 args := s.getArgs(c)
1182 args.Host = "ubuntu@" + args.Host
1183
1184- defer fakeSSH{series: s.Conn.Environ.Config().DefaultSeries()}.install(c).Restore()
1185+ defer fakeSSH{Series: s.Conn.Environ.Config().DefaultSeries()}.install(c).Restore()
1186 err := Bootstrap(args)
1187 c.Assert(err, gc.IsNil)
1188
1189- bootstrapState, err := common.LoadState(s.env.Storage())
1190+ bootstrapState, err := bootstrap.LoadState(s.env.Storage())
1191 c.Assert(err, gc.IsNil)
1192 c.Assert(
1193 bootstrapState.StateInstances,
1194@@ -96,7 +96,7 @@
1195 // Do it all again; this should work, despite the fact that
1196 // there's a bootstrap state file. Existence for that is
1197 // checked in general bootstrap code (environs/bootstrap).
1198- defer fakeSSH{series: s.Conn.Environ.Config().DefaultSeries()}.install(c).Restore()
1199+ defer fakeSSH{Series: s.Conn.Environ.Config().DefaultSeries()}.install(c).Restore()
1200 err = Bootstrap(args)
1201 c.Assert(err, gc.IsNil)
1202
1203@@ -106,17 +106,56 @@
1204 c.Assert(err, gc.Equals, ErrProvisioned)
1205 }
1206
1207+func (s *bootstrapSuite) TestBootstrapSkipDetection(c *gc.C) {
1208+ args := s.getArgs(c)
1209+ hc, err := instance.ParseHardware("arch=amd64")
1210+ c.Assert(err, gc.IsNil)
1211+
1212+ type test struct {
1213+ series string
1214+ hc *instance.HardwareCharacteristics
1215+ }
1216+ tests := []test{{
1217+ series: "",
1218+ hc: nil,
1219+ }, {
1220+ series: "precise",
1221+ hc: nil,
1222+ }, {
1223+ series: "",
1224+ hc: &hc,
1225+ }, {
1226+ series: "precise",
1227+ hc: &hc,
1228+ }}
1229+
1230+ for i, test := range tests {
1231+ c.Logf("test %d: %+v", i, test)
1232+ args.Series = test.series
1233+ args.HardwareCharacteristics = test.hc
1234+ var ssh fakeSSH
1235+ if args.Series != "" && args.HardwareCharacteristics != nil {
1236+ // If neither series nor hardware-characteristics
1237+ // is missing, detection is skipped.
1238+ ssh.SkipDetection = true
1239+ }
1240+ defer ssh.install(c).Restore()
1241+ err = Bootstrap(args)
1242+ c.Assert(err, gc.IsNil)
1243+ }
1244+}
1245+
1246 func (s *bootstrapSuite) TestBootstrapScriptFailure(c *gc.C) {
1247 args := s.getArgs(c)
1248 args.Host = "ubuntu@" + args.Host
1249 series := s.Conn.Environ.Config().DefaultSeries()
1250- defer fakeSSH{series: series, provisionAgentExitCode: 1}.install(c).Restore()
1251+ defer fakeSSH{Series: series, ProvisionAgentExitCode: 1}.install(c).Restore()
1252 err := Bootstrap(args)
1253 c.Assert(err, gc.NotNil)
1254
1255 // Since the script failed, the state file should have been
1256 // removed from storage.
1257- _, err = common.LoadState(s.env.Storage())
1258+ _, err = bootstrap.LoadState(s.env.Storage())
1259 c.Check(err, gc.Equals, environs.ErrNotBootstrapped)
1260 }
1261
1262@@ -143,11 +182,11 @@
1263 args := s.getArgs(c)
1264 args.PossibleTools = nil
1265 series := s.Conn.Environ.Config().DefaultSeries()
1266- defer fakeSSH{series: series, skipProvisionAgent: true}.install(c).Restore()
1267+ defer fakeSSH{Series: series, SkipProvisionAgent: true}.install(c).Restore()
1268 c.Assert(Bootstrap(args), gc.ErrorMatches, "no matching tools available")
1269
1270 // Non-empty list, but none that match the series/arch.
1271- defer fakeSSH{series: "edgy", skipProvisionAgent: true}.install(c).Restore()
1272+ defer fakeSSH{Series: "edgy", SkipProvisionAgent: true}.install(c).Restore()
1273 args = s.getArgs(c)
1274 c.Assert(Bootstrap(args), gc.ErrorMatches, "no matching tools available")
1275 }
1276
1277=== modified file 'environs/manual/detection.go'
1278--- environs/manual/detection.go 2013-09-16 03:24:54 +0000
1279+++ environs/manual/detection.go 2013-11-18 06:38:24 +0000
1280@@ -45,10 +45,12 @@
1281 return provisioned, nil
1282 }
1283
1284-// detectSeriesHardwareCharacteristics detects the OS series
1285-// and hardware characteristics of the remote machine by
1286-// connecting to the machine and executing a bash script.
1287-func detectSeriesAndHardwareCharacteristics(sshHost string) (hc instance.HardwareCharacteristics, series string, err error) {
1288+// DetectSeriesAndHardwareCharacteristics detects the OS
1289+// series and hardware characteristics of the remote machine
1290+// by connecting to the machine and executing a bash script.
1291+//
1292+// The sshHost argument must be a hostname of the form [user@]host.
1293+func DetectSeriesAndHardwareCharacteristics(sshHost string) (hc instance.HardwareCharacteristics, series string, err error) {
1294 logger.Infof("Detecting series and characteristics on %s", sshHost)
1295 cmd := sshCommand(sshHost, "bash")
1296 cmd.Stdin = bytes.NewBufferString(detectionScript)
1297
1298=== modified file 'environs/manual/detection_test.go'
1299--- environs/manual/detection_test.go 2013-09-20 04:41:12 +0000
1300+++ environs/manual/detection_test.go 2013-11-18 06:38:24 +0000
1301@@ -26,14 +26,14 @@
1302 "processor: 0",
1303 }, "\n")
1304 defer installFakeSSH(c, detectionScript, response, 0)()
1305- _, series, err := detectSeriesAndHardwareCharacteristics("whatever")
1306+ _, series, err := DetectSeriesAndHardwareCharacteristics("whatever")
1307 c.Assert(err, gc.IsNil)
1308 c.Assert(series, gc.Equals, "edgy")
1309 }
1310
1311 func (s *detectionSuite) TestDetectionError(c *gc.C) {
1312 defer installFakeSSH(c, detectionScript, "oh noes", 33)()
1313- _, _, err := detectSeriesAndHardwareCharacteristics("whatever")
1314+ _, _, err := DetectSeriesAndHardwareCharacteristics("whatever")
1315 c.Assert(err, gc.ErrorMatches, "exit status 33 \\(oh noes\\)")
1316 }
1317
1318@@ -93,7 +93,7 @@
1319 c.Logf("test %d: %s", i, test.summary)
1320 scriptResponse := strings.Join(test.scriptResponse, "\n")
1321 defer installFakeSSH(c, detectionScript, scriptResponse, 0)()
1322- hc, _, err := detectSeriesAndHardwareCharacteristics("hostname")
1323+ hc, _, err := DetectSeriesAndHardwareCharacteristics("hostname")
1324 c.Assert(err, gc.IsNil)
1325 c.Assert(hc.String(), gc.Equals, test.expectedHc)
1326 }
1327
1328=== renamed file 'environs/manual/fakessh_test.go' => 'environs/manual/fakessh.go'
1329--- environs/manual/fakessh_test.go 2013-10-21 21:49:04 +0000
1330+++ environs/manual/fakessh.go 2013-11-18 06:38:24 +0000
1331@@ -67,29 +67,13 @@
1332 return testbase.PatchEnvironment("PATH", fakebin+":"+os.Getenv("PATH"))
1333 }
1334
1335-// fakeSSH wraps the invocation of installFakeSSH based on the parameters.
1336-type fakeSSH struct {
1337- series string
1338- arch string
1339-
1340- // exit code for the machine agent provisioning script.
1341- provisionAgentExitCode int
1342-
1343- // there are conditions other than error in the above
1344- // that might cause provisioning to not go ahead, such
1345- // as tools being missing.
1346- skipProvisionAgent bool
1347-}
1348-
1349-// install installs fake SSH commands, which will respond to
1350-// manual provisioning/bootstrapping commands with the specified
1351-// output and exit codes.
1352-func (r fakeSSH) install(c *gc.C) testbase.Restorer {
1353- series := r.series
1354+// installDetectionFakeSSH installs a fake SSH command, which will respond
1355+// to the series/hardware detection script with the specified
1356+// series/arch.
1357+func installDetectionFakeSSH(c *gc.C, series, arch string) testbase.Restorer {
1358 if series == "" {
1359 series = "precise"
1360 }
1361- arch := r.arch
1362 if arch == "" {
1363 arch = "amd64"
1364 }
1365@@ -99,14 +83,41 @@
1366 "MemTotal: 4096 kB",
1367 "processor: 0",
1368 }, "\n")
1369+ return installFakeSSH(c, detectionScript, detectionoutput, 0)
1370+}
1371+
1372+// FakeSSH wraps the invocation of installFakeSSH based on the parameters.
1373+type fakeSSH struct {
1374+ Series string
1375+ Arch string
1376+
1377+ // exit code for the machine agent provisioning script.
1378+ ProvisionAgentExitCode int
1379+
1380+ // there are conditions other than error in the above
1381+ // that might cause provisioning to not go ahead, such
1382+ // as tools being missing.
1383+ SkipProvisionAgent bool
1384+
1385+ // detection will be skipped if the series/hardware were
1386+ // detected ahead of time.
1387+ SkipDetection bool
1388+}
1389+
1390+// install installs fake SSH commands, which will respond to
1391+// manual provisioning/bootstrapping commands with the specified
1392+// output and exit codes.
1393+func (r fakeSSH) install(c *gc.C) testbase.Restorer {
1394 var restore testbase.Restorer
1395 add := func(input string, output interface{}, rc int) {
1396 restore = restore.Add(installFakeSSH(c, input, output, rc))
1397 }
1398- if !r.skipProvisionAgent {
1399- add("", nil, r.provisionAgentExitCode)
1400- }
1401- add(detectionScript, detectionoutput, 0)
1402+ if !r.SkipProvisionAgent {
1403+ add("", nil, r.ProvisionAgentExitCode)
1404+ }
1405+ if !r.SkipDetection {
1406+ restore.Add(installDetectionFakeSSH(c, r.Series, r.Arch))
1407+ }
1408 add("", nil, 0) // checkProvisioned
1409 return restore
1410 }
1411
1412=== modified file 'environs/manual/provisioner.go'
1413--- environs/manual/provisioner.go 2013-11-14 08:27:15 +0000
1414+++ environs/manual/provisioner.go 2013-11-18 06:38:24 +0000
1415@@ -137,7 +137,7 @@
1416 return "", "", "", ErrProvisioned
1417 }
1418
1419- hc, series, err := detectSeriesAndHardwareCharacteristics(host)
1420+ hc, series, err := DetectSeriesAndHardwareCharacteristics(host)
1421 if err != nil {
1422 err = fmt.Errorf("error detecting hardware characteristics: %v", err)
1423 return "", "", "", err
1424
1425=== modified file 'environs/manual/provisioner_test.go'
1426--- environs/manual/provisioner_test.go 2013-11-08 02:52:25 +0000
1427+++ environs/manual/provisioner_test.go 2013-11-18 06:38:24 +0000
1428@@ -42,7 +42,7 @@
1429
1430 envtesting.RemoveTools(c, s.Conn.Environ.Storage())
1431 defer fakeSSH{
1432- series: series, arch: arch, skipProvisionAgent: true,
1433+ Series: series, Arch: arch, SkipProvisionAgent: true,
1434 }.install(c).Restore()
1435 // Attempt to provision a machine with no tools available, expect it to fail.
1436 machineId, err := ProvisionMachine(args)
1437@@ -58,9 +58,9 @@
1438 for i, errorCode := range []int{255, 0} {
1439 c.Logf("test %d: code %d", i, errorCode)
1440 defer fakeSSH{
1441- series: series,
1442- arch: arch,
1443- provisionAgentExitCode: errorCode,
1444+ Series: series,
1445+ Arch: arch,
1446+ ProvisionAgentExitCode: errorCode,
1447 }.install(c).Restore()
1448 machineId, err = ProvisionMachine(args)
1449 if errorCode != 0 {
1450
1451=== modified file 'environs/simplestreams/simplestreams.go'
1452--- environs/simplestreams/simplestreams.go 2013-11-05 04:32:17 +0000
1453+++ environs/simplestreams/simplestreams.go 2013-11-18 06:38:24 +0000
1454@@ -970,6 +970,7 @@
1455 if err != nil {
1456 return nil, err
1457 }
1458+ logger.Debugf("metadata: %v", metadata)
1459 matches, err := GetLatestMetadata(metadata, cons, indexRef.Source, indexRef.valueParams.FilterFunc)
1460 if err != nil {
1461 return nil, err
1462
1463=== modified file 'environs/sync/sync.go'
1464--- environs/sync/sync.go 2013-10-30 02:28:40 +0000
1465+++ environs/sync/sync.go 2013-11-18 06:38:24 +0000
1466@@ -12,6 +12,7 @@
1467 "net/http"
1468 "os"
1469 "path/filepath"
1470+ "strings"
1471
1472 "launchpad.net/loggo"
1473
1474@@ -138,10 +139,14 @@
1475
1476 // selectSourceStorage returns a storage reader based on the source setting.
1477 func selectSourceStorage(syncContext *SyncContext) (storage.StorageReader, error) {
1478- if syncContext.Source == "" {
1479- return httpstorage.NewHTTPStorageReader(DefaultToolsLocation), nil
1480- }
1481- return filestorage.NewFileStorageReader(syncContext.Source)
1482+ source := syncContext.Source
1483+ if source == "" {
1484+ source = DefaultToolsLocation
1485+ }
1486+ if strings.HasPrefix(source, "http") {
1487+ return httpstorage.NewHTTPStorageReader(source), nil
1488+ }
1489+ return filestorage.NewFileStorageReader(source)
1490 }
1491
1492 // copyTools copies a set of tools from the source to the target.
1493@@ -199,6 +204,11 @@
1494 return err
1495 }
1496
1497+// UploadFunc is the type of Upload, which may be
1498+// reassigned to control the behaviour of tools
1499+// uploading.
1500+type UploadFunc func(stor storage.Storage, forceVersion *version.Number, series ...string) (*coretools.Tools, error)
1501+
1502 // Upload builds whatever version of launchpad.net/juju-core is in $GOPATH,
1503 // uploads it to the given storage, and returns a Tools instance describing
1504 // them. If forceVersion is not nil, the uploaded tools bundle will report
1505@@ -206,7 +216,9 @@
1506 // of the built tools will be uploaded for use by machines of those series.
1507 // Juju tools built for one series do not necessarily run on another, but this
1508 // func exists only for development use cases.
1509-func Upload(stor storage.Storage, forceVersion *version.Number, fakeSeries ...string) (*coretools.Tools, error) {
1510+var Upload UploadFunc = upload
1511+
1512+func upload(stor storage.Storage, forceVersion *version.Number, fakeSeries ...string) (*coretools.Tools, error) {
1513 // TODO(rog) find binaries from $PATH when not using a development
1514 // version of juju within a $GOPATH.
1515
1516
1517=== modified file 'environs/testing/tools.go'
1518--- environs/testing/tools.go 2013-10-29 22:58:43 +0000
1519+++ environs/testing/tools.go 2013-11-18 06:38:24 +0000
1520@@ -16,6 +16,7 @@
1521 "launchpad.net/juju-core/environs/config"
1522 "launchpad.net/juju-core/environs/simplestreams"
1523 "launchpad.net/juju-core/environs/storage"
1524+ "launchpad.net/juju-core/environs/sync"
1525 envtools "launchpad.net/juju-core/environs/tools"
1526 "launchpad.net/juju-core/log"
1527 "launchpad.net/juju-core/state"
1528@@ -29,16 +30,20 @@
1529 // ToolsFixture is used as a fixture to stub out the default tools URL so we
1530 // don't hit the real internet during tests.
1531 type ToolsFixture struct {
1532- origDefaultURL string
1533- DefaultBaseURL string
1534+ origDefaultURL string
1535+ origDefaultSyncLocation string
1536+ DefaultBaseURL string
1537 }
1538
1539 func (s *ToolsFixture) SetUpTest(c *gc.C) {
1540 s.origDefaultURL = envtools.DefaultBaseURL
1541+ s.origDefaultSyncLocation = sync.DefaultToolsLocation
1542 envtools.DefaultBaseURL = s.DefaultBaseURL
1543+ sync.DefaultToolsLocation = c.MkDir() // stop sync from going to s3
1544 }
1545
1546 func (s *ToolsFixture) TearDownTest(c *gc.C) {
1547+ sync.DefaultToolsLocation = sync.DefaultToolsLocation
1548 envtools.DefaultBaseURL = s.origDefaultURL
1549 }
1550
1551
1552=== modified file 'environs/tools/storage.go'
1553--- environs/tools/storage.go 2013-09-27 02:31:45 +0000
1554+++ environs/tools/storage.go 2013-11-18 06:38:24 +0000
1555@@ -47,6 +47,7 @@
1556 var t coretools.Tools
1557 vers := name[len(toolPrefix) : len(name)-len(toolSuffix)]
1558 if t.Version, err = version.ParseBinary(vers); err != nil {
1559+ logger.Debugf("failed to parse version %q: %v", vers, err)
1560 continue
1561 }
1562 foundAnyTools = true
1563
1564=== modified file 'environs/tools/tools.go'
1565--- environs/tools/tools.go 2013-10-08 06:17:07 +0000
1566+++ environs/tools/tools.go 2013-11-18 06:38:24 +0000
1567@@ -153,6 +153,7 @@
1568 type BootstrapToolsParams struct {
1569 Version *version.Number
1570 Arch *string
1571+ Series string
1572 AllowRetry bool
1573 }
1574
1575@@ -164,7 +165,7 @@
1576 cfg := cloudInst.Config()
1577 cliVersion := version.Current.Number
1578 filter := coretools.Filter{
1579- Series: cfg.DefaultSeries(),
1580+ Series: params.Series,
1581 Arch: stringOrEmpty(params.Arch),
1582 }
1583 if params.Version != nil {
1584
1585=== modified file 'environs/tools/tools_test.go'
1586--- environs/tools/tools_test.go 2013-11-05 04:32:17 +0000
1587+++ environs/tools/tools_test.go 2013-11-18 06:38:24 +0000
1588@@ -291,8 +291,7 @@
1589 for i, test := range envtesting.BootstrapToolsTests {
1590 c.Logf("\ntest %d: %s", i, test.Info)
1591 attrs := map[string]interface{}{
1592- "development": test.Development,
1593- "default-series": test.DefaultSeries,
1594+ "development": test.Development,
1595 }
1596 var agentVersion *version.Number
1597 if test.AgentVersion != version.Zero {
1598@@ -305,6 +304,7 @@
1599
1600 params := envtools.BootstrapToolsParams{
1601 Version: agentVersion,
1602+ Series: test.DefaultSeries,
1603 Arch: &test.Arch,
1604 }
1605 actual, err := envtools.FindBootstrapTools(s.env, params)
1606
1607=== modified file 'provider/azure/environ.go'
1608--- provider/azure/environ.go 2013-11-01 06:20:19 +0000
1609+++ provider/azure/environ.go 2013-11-18 06:38:24 +0000
1610@@ -231,7 +231,7 @@
1611 }
1612
1613 // Bootstrap is specified in the Environ interface.
1614-func (env *azureEnviron) Bootstrap(cons constraints.Value, possibleTools tools.List) (err error) {
1615+func (env *azureEnviron) Bootstrap(cons constraints.Value) (err error) {
1616 // The creation of the affinity group and the virtual network is specific to the Azure provider.
1617 err = env.createAffinityGroup()
1618 if err != nil {
1619@@ -253,7 +253,7 @@
1620 env.deleteVirtualNetwork()
1621 }
1622 }()
1623- err = common.Bootstrap(env, cons, possibleTools)
1624+ err = common.Bootstrap(env, cons)
1625 return err
1626 }
1627
1628
1629=== modified file 'provider/azure/environ_test.go'
1630--- provider/azure/environ_test.go 2013-11-01 06:20:19 +0000
1631+++ provider/azure/environ_test.go 2013-11-18 06:38:24 +0000
1632@@ -21,6 +21,7 @@
1633
1634 "launchpad.net/juju-core/constraints"
1635 "launchpad.net/juju-core/environs"
1636+ "launchpad.net/juju-core/environs/bootstrap"
1637 "launchpad.net/juju-core/environs/config"
1638 "launchpad.net/juju-core/environs/imagemetadata"
1639 "launchpad.net/juju-core/environs/simplestreams"
1640@@ -28,7 +29,6 @@
1641 envtesting "launchpad.net/juju-core/environs/testing"
1642 "launchpad.net/juju-core/environs/tools"
1643 "launchpad.net/juju-core/instance"
1644- "launchpad.net/juju-core/provider/common"
1645 "launchpad.net/juju-core/testing"
1646 jc "launchpad.net/juju-core/testing/checkers"
1647 )
1648@@ -456,9 +456,9 @@
1649 }})
1650 env := makeEnviron(c)
1651 s.setDummyStorage(c, env)
1652- err := common.SaveState(
1653+ err := bootstrap.SaveState(
1654 env.Storage(),
1655- &common.BootstrapState{StateInstances: []instance.Id{instance.Id(instanceID)}})
1656+ &bootstrap.BootstrapState{StateInstances: []instance.Id{instance.Id(instanceID)}})
1657 c.Assert(err, gc.IsNil)
1658
1659 stateInfo, apiInfo, err := env.StateInfo()
1660@@ -787,9 +787,9 @@
1661 env := makeEnviron(c)
1662 s.setDummyStorage(c, env)
1663 // Populate storage.
1664- err := common.SaveState(
1665+ err := bootstrap.SaveState(
1666 env.Storage(),
1667- &common.BootstrapState{StateInstances: []instance.Id{instance.Id("test-id")}})
1668+ &bootstrap.BootstrapState{StateInstances: []instance.Id{instance.Id("test-id")}})
1669 c.Assert(err, gc.IsNil)
1670 responses := []gwacl.DispatcherResponse{
1671 gwacl.NewDispatcherResponse(nil, http.StatusBadRequest, nil),
1672@@ -808,9 +808,9 @@
1673 env := makeEnviron(c)
1674 s.setDummyStorage(c, env)
1675 // Populate storage.
1676- err := common.SaveState(
1677+ err := bootstrap.SaveState(
1678 env.Storage(),
1679- &common.BootstrapState{StateInstances: []instance.Id{instance.Id("test-id")}})
1680+ &bootstrap.BootstrapState{StateInstances: []instance.Id{instance.Id("test-id")}})
1681 c.Assert(err, gc.IsNil)
1682 services := []gwacl.HostedServiceDescriptor{}
1683 responses := getAzureServiceListResponse(c, services)
1684
1685=== modified file 'provider/common/bootstrap.go'
1686--- provider/common/bootstrap.go 2013-10-02 19:30:07 +0000
1687+++ provider/common/bootstrap.go 2013-11-18 06:38:24 +0000
1688@@ -10,6 +10,7 @@
1689
1690 "launchpad.net/juju-core/constraints"
1691 "launchpad.net/juju-core/environs"
1692+ "launchpad.net/juju-core/environs/bootstrap"
1693 "launchpad.net/juju-core/instance"
1694 coretools "launchpad.net/juju-core/tools"
1695 )
1696@@ -19,7 +20,7 @@
1697 // Bootstrap is a common implementation of the Bootstrap method defined on
1698 // environs.Environ; we strongly recommend that this implementation be used
1699 // when writing a new provider.
1700-func Bootstrap(env environs.Environ, cons constraints.Value, possibleTools coretools.List) error {
1701+func Bootstrap(env environs.Environ, cons constraints.Value) error {
1702 // TODO make safe in the case of racing Bootstraps
1703 // If two Bootstraps are called concurrently, there's
1704 // no way to make sure that only one succeeds.
1705@@ -27,12 +28,18 @@
1706 // Create an empty bootstrap state file so we can get its URL.
1707 // It will be updated with the instance id and hardware characteristics
1708 // after the bootstrap instance is started.
1709- stateFileURL, err := CreateStateFile(env.Storage())
1710+ stateFileURL, err := bootstrap.CreateStateFile(env.Storage())
1711 if err != nil {
1712 return err
1713 }
1714 machineConfig := environs.NewBootstrapMachineConfig(stateFileURL)
1715- inst, hw, err := env.StartInstance(cons, possibleTools, machineConfig)
1716+
1717+ selectedTools, err := EnsureBootstrapTools(env, env.Config().DefaultSeries(), cons.Arch)
1718+ if err != nil {
1719+ return err
1720+ }
1721+
1722+ inst, hw, err := env.StartInstance(cons, selectedTools, machineConfig)
1723 if err != nil {
1724 return fmt.Errorf("cannot start bootstrap instance: %v", err)
1725 }
1726@@ -40,9 +47,9 @@
1727 if hw != nil {
1728 characteristics = []instance.HardwareCharacteristics{*hw}
1729 }
1730- err = SaveState(
1731+ err = bootstrap.SaveState(
1732 env.Storage(),
1733- &BootstrapState{
1734+ &bootstrap.BootstrapState{
1735 StateInstances: []instance.Id{inst.Id()},
1736 Characteristics: characteristics,
1737 })
1738@@ -56,3 +63,29 @@
1739 }
1740 return nil
1741 }
1742+
1743+// EnsureBootstrapTools finds tools, syncing with an external tools source as
1744+// necessary; it then selects the newest tools to bootstrap with, and sets
1745+// agent-version.
1746+func EnsureBootstrapTools(env environs.Environ, series string, arch *string) (coretools.List, error) {
1747+ possibleTools, err := bootstrap.EnsureToolsAvailability(env, series, arch)
1748+ if err != nil {
1749+ return nil, err
1750+ }
1751+ return bootstrap.SetBootstrapTools(env, possibleTools)
1752+}
1753+
1754+// EnsureNotBootstrapped returns null if the environment is not bootstrapped,
1755+// and an error if it is or if the function was not able to tell.
1756+func EnsureNotBootstrapped(env environs.Environ) error {
1757+ _, err := bootstrap.LoadState(env.Storage())
1758+ // If there is no error loading the bootstrap state, then we are
1759+ // bootstrapped.
1760+ if err == nil {
1761+ return fmt.Errorf("environment is already bootstrapped")
1762+ }
1763+ if err == environs.ErrNotBootstrapped {
1764+ return nil
1765+ }
1766+ return err
1767+}
1768
1769=== modified file 'provider/common/bootstrap_test.go'
1770--- provider/common/bootstrap_test.go 2013-10-02 00:29:29 +0000
1771+++ provider/common/bootstrap_test.go 2013-11-18 06:38:24 +0000
1772@@ -11,34 +11,79 @@
1773
1774 "launchpad.net/juju-core/constraints"
1775 "launchpad.net/juju-core/environs"
1776+ "launchpad.net/juju-core/environs/bootstrap"
1777 "launchpad.net/juju-core/environs/cloudinit"
1778+ "launchpad.net/juju-core/environs/config"
1779+ "launchpad.net/juju-core/environs/storage"
1780+ envtesting "launchpad.net/juju-core/environs/testing"
1781 "launchpad.net/juju-core/instance"
1782 "launchpad.net/juju-core/provider/common"
1783+ coretesting "launchpad.net/juju-core/testing"
1784 jc "launchpad.net/juju-core/testing/checkers"
1785 "launchpad.net/juju-core/testing/testbase"
1786 "launchpad.net/juju-core/tools"
1787- "launchpad.net/juju-core/version"
1788 )
1789
1790 type BootstrapSuite struct {
1791 testbase.LoggingSuite
1792+ envtesting.ToolsFixture
1793 }
1794
1795 var _ = gc.Suite(&BootstrapSuite{})
1796
1797+type cleaner interface {
1798+ AddCleanup(testbase.CleanupFunc)
1799+}
1800+
1801+func (s *BootstrapSuite) SetUpTest(c *gc.C) {
1802+ s.LoggingSuite.SetUpTest(c)
1803+ s.ToolsFixture.SetUpTest(c)
1804+}
1805+
1806+func (s *BootstrapSuite) TearDownTest(c *gc.C) {
1807+ s.ToolsFixture.TearDownTest(c)
1808+ s.LoggingSuite.TearDownTest(c)
1809+}
1810+
1811+func newStorage(suite cleaner, c *gc.C) storage.Storage {
1812+ closer, stor, _ := envtesting.CreateLocalTestStorage(c)
1813+ suite.AddCleanup(func(*gc.C) { closer.Close() })
1814+ envtesting.UploadFakeTools(c, stor)
1815+ return stor
1816+}
1817+
1818+func minimalConfig(c *gc.C) *config.Config {
1819+ attrs := map[string]interface{}{
1820+ "name": "whatever",
1821+ "type": "anything, really",
1822+ "ca-cert": coretesting.CACert,
1823+ "ca-private-key": coretesting.CAKey,
1824+ }
1825+ cfg, err := config.New(config.UseDefaults, attrs)
1826+ c.Assert(err, gc.IsNil)
1827+ return cfg
1828+}
1829+
1830+func configGetter(c *gc.C) configFunc {
1831+ cfg := minimalConfig(c)
1832+ return func() *config.Config { return cfg }
1833+}
1834+
1835 func (s *BootstrapSuite) TestCannotWriteStateFile(c *gc.C) {
1836- brokenStorage := &mockStorage{putErr: fmt.Errorf("noes!")}
1837+ brokenStorage := &mockStorage{
1838+ Storage: newStorage(s, c),
1839+ putErr: fmt.Errorf("noes!"),
1840+ }
1841 env := &mockEnviron{storage: brokenStorage}
1842- err := common.Bootstrap(env, constraints.Value{}, nil)
1843+ err := common.Bootstrap(env, constraints.Value{})
1844 c.Assert(err, gc.ErrorMatches, "cannot create initial state file: noes!")
1845 }
1846
1847 func (s *BootstrapSuite) TestCannotStartInstance(c *gc.C) {
1848 stor := newStorage(s, c)
1849- checkURL, err := stor.URL(common.StateFile)
1850+ checkURL, err := stor.URL(bootstrap.StateFile)
1851 c.Assert(err, gc.IsNil)
1852 checkCons := constraints.MustParse("mem=8G")
1853- checkTools := tools.List{&tools.Tools{Version: version.Current}}
1854
1855 startInstance := func(
1856 cons constraints.Value, possibleTools tools.List, mcfg *cloudinit.MachineConfig,
1857@@ -46,7 +91,6 @@
1858 instance.Instance, *instance.HardwareCharacteristics, error,
1859 ) {
1860 c.Assert(cons, gc.DeepEquals, checkCons)
1861- c.Assert(possibleTools, gc.DeepEquals, checkTools)
1862 c.Assert(mcfg, gc.DeepEquals, environs.NewBootstrapMachineConfig(checkURL))
1863 return nil, nil, fmt.Errorf("meh, not started")
1864 }
1865@@ -54,9 +98,10 @@
1866 env := &mockEnviron{
1867 storage: stor,
1868 startInstance: startInstance,
1869+ config: configGetter(c),
1870 }
1871
1872- err = common.Bootstrap(env, checkCons, checkTools)
1873+ err = common.Bootstrap(env, checkCons)
1874 c.Assert(err, gc.ErrorMatches, "cannot start bootstrap instance: meh, not started")
1875 }
1876
1877@@ -83,9 +128,10 @@
1878 storage: stor,
1879 startInstance: startInstance,
1880 stopInstances: stopInstances,
1881+ config: configGetter(c),
1882 }
1883
1884- err := common.Bootstrap(env, constraints.Value{}, nil)
1885+ err := common.Bootstrap(env, constraints.Value{})
1886 c.Assert(err, gc.ErrorMatches, "cannot save state: suddenly a wild blah")
1887 c.Assert(stopped, gc.HasLen, 1)
1888 c.Assert(stopped[0].Id(), gc.Equals, instance.Id("i-blah"))
1889@@ -118,9 +164,10 @@
1890 storage: stor,
1891 startInstance: startInstance,
1892 stopInstances: stopInstances,
1893+ config: configGetter(c),
1894 }
1895
1896- err := common.Bootstrap(env, constraints.Value{}, nil)
1897+ err := common.Bootstrap(env, constraints.Value{})
1898 c.Assert(err, gc.ErrorMatches, "cannot save state: suddenly a wild blah")
1899 c.Assert(stopped, gc.HasLen, 1)
1900 c.Assert(stopped[0].Id(), gc.Equals, instance.Id("i-blah"))
1901@@ -144,16 +191,23 @@
1902 return &mockInstance{id: checkInstanceId}, &checkHardware, nil
1903 }
1904
1905+ var getConfigCalled int
1906+ getConfig := func() *config.Config {
1907+ getConfigCalled++
1908+ return minimalConfig(c)
1909+ }
1910+
1911 env := &mockEnviron{
1912 storage: stor,
1913 startInstance: startInstance,
1914+ config: getConfig,
1915 }
1916- err := common.Bootstrap(env, constraints.Value{}, nil)
1917+ err := common.Bootstrap(env, constraints.Value{})
1918 c.Assert(err, gc.IsNil)
1919
1920- savedState, err := common.LoadStateFromURL(checkURL)
1921+ savedState, err := bootstrap.LoadStateFromURL(checkURL)
1922 c.Assert(err, gc.IsNil)
1923- c.Assert(savedState, gc.DeepEquals, &common.BootstrapState{
1924+ c.Assert(savedState, gc.DeepEquals, &bootstrap.BootstrapState{
1925 StateInstances: []instance.Id{instance.Id(checkInstanceId)},
1926 Characteristics: []instance.HardwareCharacteristics{checkHardware},
1927 })
1928
1929=== modified file 'provider/common/mock_test.go'
1930--- provider/common/mock_test.go 2013-10-02 00:29:29 +0000
1931+++ provider/common/mock_test.go 2013-11-18 06:38:24 +0000
1932@@ -9,6 +9,8 @@
1933 "launchpad.net/juju-core/constraints"
1934 "launchpad.net/juju-core/environs"
1935 "launchpad.net/juju-core/environs/cloudinit"
1936+ "launchpad.net/juju-core/environs/config"
1937+ "launchpad.net/juju-core/environs/simplestreams"
1938 "launchpad.net/juju-core/environs/storage"
1939 "launchpad.net/juju-core/instance"
1940 "launchpad.net/juju-core/tools"
1941@@ -17,12 +19,18 @@
1942 type allInstancesFunc func() ([]instance.Instance, error)
1943 type startInstanceFunc func(constraints.Value, tools.List, *cloudinit.MachineConfig) (instance.Instance, *instance.HardwareCharacteristics, error)
1944 type stopInstancesFunc func([]instance.Instance) error
1945+type getToolsSourcesFunc func() ([]simplestreams.DataSource, error)
1946+type configFunc func() *config.Config
1947+type setConfigFunc func(*config.Config) error
1948
1949 type mockEnviron struct {
1950 storage storage.Storage
1951 allInstances allInstancesFunc
1952 startInstance startInstanceFunc
1953 stopInstances stopInstancesFunc
1954+ getToolsSources getToolsSourcesFunc
1955+ config configFunc
1956+ setConfig setConfigFunc
1957 environs.Environ // stub out other methods with panics
1958 }
1959
1960@@ -49,6 +57,25 @@
1961 return env.stopInstances(instances)
1962 }
1963
1964+func (env *mockEnviron) Config() *config.Config {
1965+ return env.config()
1966+}
1967+
1968+func (env *mockEnviron) SetConfig(cfg *config.Config) error {
1969+ if env.setConfig != nil {
1970+ return env.setConfig(cfg)
1971+ }
1972+ return nil
1973+}
1974+
1975+func (env *mockEnviron) GetToolsSources() ([]simplestreams.DataSource, error) {
1976+ if env.getToolsSources != nil {
1977+ return env.getToolsSources()
1978+ }
1979+ datasource := storage.NewStorageSimpleStreamsDataSource(env.Storage(), storage.BaseToolsPath)
1980+ return []simplestreams.DataSource{datasource}, nil
1981+}
1982+
1983 type mockInstance struct {
1984 id string
1985 instance.Instance // stub out other methods with panics
1986
1987=== modified file 'provider/common/state.go'
1988--- provider/common/state.go 2013-10-01 12:03:33 +0000
1989+++ provider/common/state.go 2013-11-18 06:38:24 +0000
1990@@ -4,103 +4,18 @@
1991 package common
1992
1993 import (
1994- "bytes"
1995 "errors"
1996 "fmt"
1997- "io"
1998- "io/ioutil"
1999- "net/http"
2000-
2001- "launchpad.net/goyaml"
2002
2003 "launchpad.net/juju-core/environs"
2004+ "launchpad.net/juju-core/environs/bootstrap"
2005 "launchpad.net/juju-core/environs/config"
2006- "launchpad.net/juju-core/environs/storage"
2007- coreerrors "launchpad.net/juju-core/errors"
2008 "launchpad.net/juju-core/instance"
2009 "launchpad.net/juju-core/log"
2010 "launchpad.net/juju-core/state"
2011 "launchpad.net/juju-core/state/api"
2012 )
2013
2014-// StateFile is the name of the file where the provider's state is stored.
2015-const StateFile = "provider-state"
2016-
2017-// BootstrapState is the state information that is stored in StateFile.
2018-//
2019-// Individual providers may define their own state structures instead of
2020-// this one, and use their own code for loading and saving those, but this is
2021-// the definition that most practically useful providers share unchanged.
2022-type BootstrapState struct {
2023- // StateInstances are the state servers.
2024- StateInstances []instance.Id `yaml:"state-instances"`
2025- // Characteristics reflect the hardware each state server is running on.
2026- // This is used at bootstrap time so the state server knows what hardware it has.
2027- // The state *may* be updated later without this information, but by then it's
2028- // served it's purpose.
2029- Characteristics []instance.HardwareCharacteristics `yaml:"characteristics,omitempty"`
2030-}
2031-
2032-// putState writes the given data to the state file on the given storage.
2033-// The file's name is as defined in StateFile.
2034-func putState(storage storage.StorageWriter, data []byte) error {
2035- return storage.Put(StateFile, bytes.NewBuffer(data), int64(len(data)))
2036-}
2037-
2038-// CreateStateFile creates an empty state file on the given storage, and
2039-// returns its URL.
2040-func CreateStateFile(storage storage.Storage) (string, error) {
2041- err := putState(storage, []byte{})
2042- if err != nil {
2043- return "", fmt.Errorf("cannot create initial state file: %v", err)
2044- }
2045- return storage.URL(StateFile)
2046-}
2047-
2048-// SaveState writes the given state to the given storage.
2049-func SaveState(storage storage.StorageWriter, state *BootstrapState) error {
2050- data, err := goyaml.Marshal(state)
2051- if err != nil {
2052- return err
2053- }
2054- return putState(storage, data)
2055-}
2056-
2057-// LoadStateFromURL reads state from the given URL.
2058-func LoadStateFromURL(url string) (*BootstrapState, error) {
2059- resp, err := http.Get(url)
2060- if err != nil {
2061- return nil, err
2062- }
2063- return loadState(resp.Body)
2064-}
2065-
2066-// LoadState reads state from the given storage.
2067-func LoadState(stor storage.StorageReader) (*BootstrapState, error) {
2068- r, err := storage.Get(stor, StateFile)
2069- if err != nil {
2070- if coreerrors.IsNotFoundError(err) {
2071- return nil, environs.ErrNotBootstrapped
2072- }
2073- return nil, err
2074- }
2075- return loadState(r)
2076-}
2077-
2078-func loadState(r io.ReadCloser) (*BootstrapState, error) {
2079- defer r.Close()
2080- data, err := ioutil.ReadAll(r)
2081- if err != nil {
2082- return nil, fmt.Errorf("error reading %q: %v", StateFile, err)
2083- }
2084- var state BootstrapState
2085- err = goyaml.Unmarshal(data, &state)
2086- if err != nil {
2087- return nil, fmt.Errorf("error unmarshalling %q: %v", StateFile, err)
2088- }
2089- return &state, nil
2090-}
2091-
2092 // getDNSNames queries and returns the DNS names for the given instances,
2093 // ignoring nil instances or ones without DNS names.
2094 func getDNSNames(instances []instance.Instance) []string {
2095@@ -147,7 +62,7 @@
2096 // StateInfo is a reusable implementation of Environ.StateInfo, available to
2097 // providers that also use the other functionality from this file.
2098 func StateInfo(env environs.Environ) (*state.Info, *api.Info, error) {
2099- st, err := LoadState(env.Storage())
2100+ st, err := bootstrap.LoadState(env.Storage())
2101 if err != nil {
2102 return nil, nil, err
2103 }
2104
2105=== modified file 'provider/common/state_test.go'
2106--- provider/common/state_test.go 2013-10-02 10:39:12 +0000
2107+++ provider/common/state_test.go 2013-11-18 06:38:24 +0000
2108@@ -4,16 +4,9 @@
2109 package common_test
2110
2111 import (
2112- "bytes"
2113- "io/ioutil"
2114-
2115 gc "launchpad.net/gocheck"
2116- "launchpad.net/goyaml"
2117
2118- "launchpad.net/juju-core/environs"
2119 "launchpad.net/juju-core/environs/config"
2120- "launchpad.net/juju-core/environs/storage"
2121- envtesting "launchpad.net/juju-core/environs/testing"
2122 "launchpad.net/juju-core/instance"
2123 "launchpad.net/juju-core/provider/common"
2124 "launchpad.net/juju-core/testing"
2125@@ -26,102 +19,6 @@
2126
2127 var _ = gc.Suite(&StateSuite{})
2128
2129-type cleaner interface {
2130- AddCleanup(testbase.CleanupFunc)
2131-}
2132-
2133-func newStorage(suite cleaner, c *gc.C) storage.Storage {
2134- closer, stor, _ := envtesting.CreateLocalTestStorage(c)
2135- suite.AddCleanup(func(*gc.C) { closer.Close() })
2136- return stor
2137-}
2138-
2139-func (suite *StateSuite) TestCreateStateFileWritesEmptyStateFile(c *gc.C) {
2140- stor := newStorage(suite, c)
2141-
2142- url, err := common.CreateStateFile(stor)
2143- c.Assert(err, gc.IsNil)
2144-
2145- reader, err := storage.Get(stor, common.StateFile)
2146- c.Assert(err, gc.IsNil)
2147- data, err := ioutil.ReadAll(reader)
2148- c.Assert(err, gc.IsNil)
2149- c.Check(string(data), gc.Equals, "")
2150- c.Assert(url, gc.NotNil)
2151- expectedURL, err := stor.URL(common.StateFile)
2152- c.Assert(err, gc.IsNil)
2153- c.Check(url, gc.Equals, expectedURL)
2154-}
2155-
2156-func (suite *StateSuite) TestSaveStateWritesStateFile(c *gc.C) {
2157- stor := newStorage(suite, c)
2158- arch := "amd64"
2159- state := common.BootstrapState{
2160- StateInstances: []instance.Id{instance.Id("an-instance-id")},
2161- Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}}
2162- marshaledState, err := goyaml.Marshal(state)
2163- c.Assert(err, gc.IsNil)
2164-
2165- err = common.SaveState(stor, &state)
2166- c.Assert(err, gc.IsNil)
2167-
2168- loadedState, err := storage.Get(stor, common.StateFile)
2169- c.Assert(err, gc.IsNil)
2170- content, err := ioutil.ReadAll(loadedState)
2171- c.Assert(err, gc.IsNil)
2172- c.Check(content, gc.DeepEquals, marshaledState)
2173-}
2174-
2175-func (suite *StateSuite) setUpSavedState(c *gc.C, stor storage.Storage) common.BootstrapState {
2176- arch := "amd64"
2177- state := common.BootstrapState{
2178- StateInstances: []instance.Id{instance.Id("an-instance-id")},
2179- Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}}
2180- content, err := goyaml.Marshal(state)
2181- c.Assert(err, gc.IsNil)
2182- err = stor.Put(common.StateFile, ioutil.NopCloser(bytes.NewReader(content)), int64(len(content)))
2183- c.Assert(err, gc.IsNil)
2184- return state
2185-}
2186-
2187-func (suite *StateSuite) TestLoadStateReadsStateFile(c *gc.C) {
2188- storage := newStorage(suite, c)
2189- state := suite.setUpSavedState(c, storage)
2190- storedState, err := common.LoadState(storage)
2191- c.Assert(err, gc.IsNil)
2192- c.Check(*storedState, gc.DeepEquals, state)
2193-}
2194-
2195-func (suite *StateSuite) TestLoadStateFromURLReadsStateFile(c *gc.C) {
2196- stor := newStorage(suite, c)
2197- state := suite.setUpSavedState(c, stor)
2198- url, err := stor.URL(common.StateFile)
2199- c.Assert(err, gc.IsNil)
2200- storedState, err := common.LoadStateFromURL(url)
2201- c.Assert(err, gc.IsNil)
2202- c.Check(*storedState, gc.DeepEquals, state)
2203-}
2204-
2205-func (suite *StateSuite) TestLoadStateMissingFile(c *gc.C) {
2206- stor := newStorage(suite, c)
2207- _, err := common.LoadState(stor)
2208- c.Check(err, gc.Equals, environs.ErrNotBootstrapped)
2209-}
2210-
2211-func (suite *StateSuite) TestLoadStateIntegratesWithSaveState(c *gc.C) {
2212- storage := newStorage(suite, c)
2213- arch := "amd64"
2214- state := common.BootstrapState{
2215- StateInstances: []instance.Id{instance.Id("an-instance-id")},
2216- Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}}
2217- err := common.SaveState(storage, &state)
2218- c.Assert(err, gc.IsNil)
2219- storedState, err := common.LoadState(storage)
2220- c.Assert(err, gc.IsNil)
2221-
2222- c.Check(*storedState, gc.DeepEquals, state)
2223-}
2224-
2225 func (suite *StateSuite) TestGetDNSNamesAcceptsNil(c *gc.C) {
2226 result := common.GetDNSNames(nil)
2227 c.Check(result, gc.DeepEquals, []string{})
2228
2229=== modified file 'provider/dummy/environs.go'
2230--- provider/dummy/environs.go 2013-11-07 09:09:55 +0000
2231+++ provider/dummy/environs.go 2013-11-18 06:38:24 +0000
2232@@ -37,6 +37,7 @@
2233
2234 "launchpad.net/juju-core/constraints"
2235 "launchpad.net/juju-core/environs"
2236+ "launchpad.net/juju-core/environs/bootstrap"
2237 "launchpad.net/juju-core/environs/cloudinit"
2238 "launchpad.net/juju-core/environs/config"
2239 "launchpad.net/juju-core/environs/imagemetadata"
2240@@ -520,7 +521,12 @@
2241 storage.NewStorageSimpleStreamsDataSource(e.Storage(), storage.BaseToolsPath)}, nil
2242 }
2243
2244-func (e *environ) Bootstrap(cons constraints.Value, possibleTools coretools.List) error {
2245+func (e *environ) Bootstrap(cons constraints.Value) error {
2246+ selectedTools, err := common.EnsureBootstrapTools(e, e.Config().DefaultSeries(), cons.Arch)
2247+ if err != nil {
2248+ return err
2249+ }
2250+
2251 defer delay()
2252 if err := e.checkBroken("Bootstrap"); err != nil {
2253 return err
2254@@ -533,7 +539,7 @@
2255 return fmt.Errorf("no CA certificate in environment configuration")
2256 }
2257
2258- logger.Infof("would pick tools from %s", possibleTools)
2259+ logger.Infof("would pick tools from %s", selectedTools)
2260 cfg, err := environs.BootstrapConfig(e.Config())
2261 if err != nil {
2262 return fmt.Errorf("cannot make bootstrap config: %v", err)
2263@@ -552,7 +558,7 @@
2264 // we need to release the mutex for the save state to work, so regain
2265 // it after the call.
2266 estate.mu.Unlock()
2267- if err := common.SaveState(e.Storage(), &common.BootstrapState{StateInstances: []instance.Id{"localhost"}}); err != nil {
2268+ if err := bootstrap.SaveState(e.Storage(), &bootstrap.BootstrapState{StateInstances: []instance.Id{"localhost"}}); err != nil {
2269 logger.Errorf("failed to save state instances: %v", err)
2270 estate.mu.Lock() // otherwise defered unlock will fail
2271 return err
2272
2273=== modified file 'provider/ec2/ec2.go'
2274--- provider/ec2/ec2.go 2013-11-01 06:20:19 +0000
2275+++ provider/ec2/ec2.go 2013-11-18 06:38:24 +0000
2276@@ -305,8 +305,8 @@
2277 return stor
2278 }
2279
2280-func (e *environ) Bootstrap(cons constraints.Value, possibleTools tools.List) error {
2281- return common.Bootstrap(e, cons, possibleTools)
2282+func (e *environ) Bootstrap(cons constraints.Value) error {
2283+ return common.Bootstrap(e, cons)
2284 }
2285
2286 func (e *environ) StateInfo() (*state.Info, *api.Info, error) {
2287
2288=== modified file 'provider/ec2/local_test.go'
2289--- provider/ec2/local_test.go 2013-11-01 06:20:19 +0000
2290+++ provider/ec2/local_test.go 2013-11-18 06:38:24 +0000
2291@@ -28,7 +28,6 @@
2292 "launchpad.net/juju-core/environs/tools"
2293 "launchpad.net/juju-core/instance"
2294 "launchpad.net/juju-core/juju/testing"
2295- "launchpad.net/juju-core/provider/common"
2296 "launchpad.net/juju-core/provider/ec2"
2297 coretesting "launchpad.net/juju-core/testing"
2298 jc "launchpad.net/juju-core/testing/checkers"
2299@@ -210,7 +209,7 @@
2300 c.Assert(err, gc.IsNil)
2301
2302 // check that the state holds the id of the bootstrap machine.
2303- bootstrapState, err := common.LoadState(env.Storage())
2304+ bootstrapState, err := bootstrap.LoadState(env.Storage())
2305 c.Assert(err, gc.IsNil)
2306 c.Assert(bootstrapState.StateInstances, gc.HasLen, 1)
2307
2308@@ -263,7 +262,7 @@
2309 err = env.Destroy()
2310 c.Assert(err, gc.IsNil)
2311
2312- _, err = common.LoadState(env.Storage())
2313+ _, err = bootstrap.LoadState(env.Storage())
2314 c.Assert(err, gc.NotNil)
2315 }
2316
2317
2318=== modified file 'provider/local/environ.go'
2319--- provider/local/environ.go 2013-11-11 21:38:30 +0000
2320+++ provider/local/environ.go 2013-11-18 06:38:24 +0000
2321@@ -19,6 +19,7 @@
2322 "launchpad.net/juju-core/container"
2323 "launchpad.net/juju-core/container/lxc"
2324 "launchpad.net/juju-core/environs"
2325+ "launchpad.net/juju-core/environs/bootstrap"
2326 "launchpad.net/juju-core/environs/cloudinit"
2327 "launchpad.net/juju-core/environs/config"
2328 "launchpad.net/juju-core/environs/filestorage"
2329@@ -95,7 +96,7 @@
2330 }
2331
2332 // Bootstrap is specified in the Environ interface.
2333-func (env *localEnviron) Bootstrap(cons constraints.Value, possibleTools tools.List) error {
2334+func (env *localEnviron) Bootstrap(cons constraints.Value) error {
2335 if !env.config.runningAsRoot {
2336 return fmt.Errorf("bootstrapping a local environment must be done as root")
2337 }
2338@@ -113,13 +114,19 @@
2339
2340 // Before we write the agent config file, we need to make sure the
2341 // instance is saved in the StateInfo.
2342- if err := common.SaveState(env.Storage(), &common.BootstrapState{
2343+ if err := bootstrap.SaveState(env.Storage(), &bootstrap.BootstrapState{
2344 StateInstances: []instance.Id{bootstrapInstanceId},
2345 }); err != nil {
2346 logger.Errorf("failed to save state instances: %v", err)
2347 return err
2348 }
2349
2350+ vers := version.Current
2351+ selectedTools, err := common.EnsureBootstrapTools(env, vers.Series, &vers.Arch)
2352+ if err != nil {
2353+ return err
2354+ }
2355+
2356 // Need to write out the agent file for machine-0 before initializing
2357 // state, as as part of that process, it will reset the password in the
2358 // agent file.
2359@@ -132,7 +139,7 @@
2360 return err
2361 }
2362
2363- return env.setupLocalMachineAgent(cons, possibleTools)
2364+ return env.setupLocalMachineAgent(cons, selectedTools)
2365 }
2366
2367 // StateInfo is specified in the Environ interface.
2368
2369=== modified file 'provider/maas/environ.go'
2370--- provider/maas/environ.go 2013-11-04 22:24:51 +0000
2371+++ provider/maas/environ.go 2013-11-18 06:38:24 +0000
2372@@ -77,8 +77,8 @@
2373 }
2374
2375 // Bootstrap is specified in the Environ interface.
2376-func (env *maasEnviron) Bootstrap(cons constraints.Value, possibleTools tools.List) error {
2377- return common.Bootstrap(env, cons, possibleTools)
2378+func (env *maasEnviron) Bootstrap(cons constraints.Value) error {
2379+ return common.Bootstrap(env, cons)
2380 }
2381
2382 // StateInfo is specified in the Environ interface.
2383
2384=== modified file 'provider/maas/environ_test.go'
2385--- provider/maas/environ_test.go 2013-11-04 22:24:51 +0000
2386+++ provider/maas/environ_test.go 2013-11-18 06:38:24 +0000
2387@@ -26,7 +26,6 @@
2388 "launchpad.net/juju-core/errors"
2389 "launchpad.net/juju-core/instance"
2390 "launchpad.net/juju-core/juju/testing"
2391- "launchpad.net/juju-core/provider/common"
2392 jc "launchpad.net/juju-core/testing/checkers"
2393 "launchpad.net/juju-core/tools"
2394 "launchpad.net/juju-core/utils"
2395@@ -59,7 +58,7 @@
2396 }
2397
2398 func (suite *environSuite) setupFakeProviderStateFile(c *gc.C) {
2399- suite.testMAASObject.TestServer.NewFile(common.StateFile, []byte("test file content"))
2400+ suite.testMAASObject.TestServer.NewFile(bootstrap.StateFile, []byte("test file content"))
2401 }
2402
2403 func (suite *environSuite) setupFakeTools(c *gc.C) {
2404@@ -216,7 +215,7 @@
2405
2406 // Test the instance id is correctly recorded for the bootstrap node.
2407 // Check that the state holds the id of the bootstrap machine.
2408- stateData, err := common.LoadState(env.Storage())
2409+ stateData, err := bootstrap.LoadState(env.Storage())
2410 c.Assert(err, gc.IsNil)
2411 c.Assert(stateData.StateInstances, gc.HasLen, 1)
2412 insts, err := env.AllInstances()
2413@@ -370,9 +369,9 @@
2414 input := `{"system_id": "system_id", "hostname": "` + hostname + `"}`
2415 node := suite.testMAASObject.TestServer.NewNode(input)
2416 testInstance := &maasInstance{&node, suite.makeEnviron()}
2417- err := common.SaveState(
2418+ err := bootstrap.SaveState(
2419 env.Storage(),
2420- &common.BootstrapState{StateInstances: []instance.Id{testInstance.Id()}})
2421+ &bootstrap.BootstrapState{StateInstances: []instance.Id{testInstance.Id()}})
2422 c.Assert(err, gc.IsNil)
2423
2424 stateInfo, apiInfo, err := env.StateInfo()
2425@@ -429,7 +428,14 @@
2426 env := suite.makeEnviron()
2427 // Can't RemoveAllTools, no public storage.
2428 envtesting.RemoveTools(c, env.Storage())
2429- err := bootstrap.Bootstrap(env, constraints.Value{})
2430+ // Disable auto-uploading by setting the agent version.
2431+ cfg, err := env.Config().Apply(map[string]interface{}{
2432+ "agent-version": version.Current.Number.String(),
2433+ })
2434+ c.Assert(err, gc.IsNil)
2435+ err = env.SetConfig(cfg)
2436+ c.Assert(err, gc.IsNil)
2437+ err = bootstrap.Bootstrap(env, constraints.Value{})
2438 c.Check(err, gc.ErrorMatches, "cannot find bootstrap tools.*")
2439 }
2440
2441
2442=== modified file 'provider/null/config_test.go'
2443--- provider/null/config_test.go 2013-10-04 09:32:51 +0000
2444+++ provider/null/config_test.go 2013-11-18 06:38:24 +0000
2445@@ -6,7 +6,6 @@
2446 import (
2447 "fmt"
2448 "regexp"
2449- stdtesting "testing"
2450
2451 gc "launchpad.net/gocheck"
2452
2453@@ -22,10 +21,6 @@
2454
2455 var _ = gc.Suite(&configSuite{})
2456
2457-func Test(t *stdtesting.T) {
2458- gc.TestingT(t)
2459-}
2460-
2461 func minimalConfigValues() map[string]interface{} {
2462 return map[string]interface{}{
2463 "name": "test",
2464
2465=== modified file 'provider/null/environ.go'
2466--- provider/null/environ.go 2013-10-24 00:20:59 +0000
2467+++ provider/null/environ.go 2013-11-18 06:38:24 +0000
2468@@ -85,12 +85,23 @@
2469 return e.envConfig().Name()
2470 }
2471
2472-func (e *nullEnviron) Bootstrap(_ constraints.Value, possibleTools tools.List) error {
2473+func (e *nullEnviron) Bootstrap(cons constraints.Value) error {
2474+ envConfig := e.envConfig()
2475+ hc, series, err := manual.DetectSeriesAndHardwareCharacteristics(envConfig.sshHost())
2476+ if err != nil {
2477+ return err
2478+ }
2479+ selectedTools, err := common.EnsureBootstrapTools(e, series, hc.Arch)
2480+ if err != nil {
2481+ return err
2482+ }
2483 return manual.Bootstrap(manual.BootstrapArgs{
2484- Host: e.envConfig().sshHost(),
2485- DataDir: dataDir,
2486- Environ: e,
2487- PossibleTools: possibleTools,
2488+ Host: e.envConfig().sshHost(),
2489+ DataDir: dataDir,
2490+ Environ: e,
2491+ PossibleTools: selectedTools,
2492+ Series: series,
2493+ HardwareCharacteristics: &hc,
2494 })
2495 }
2496
2497
2498=== modified file 'provider/null/provider.go'
2499--- provider/null/provider.go 2013-10-08 13:35:41 +0000
2500+++ provider/null/provider.go 2013-11-18 06:38:24 +0000
2501@@ -31,7 +31,11 @@
2502 if err != nil {
2503 return nil, err
2504 }
2505- return &nullEnviron{cfg: envConfig}, nil
2506+ return p.open(envConfig)
2507+}
2508+
2509+func (p nullProvider) open(cfg *environConfig) (environs.Environ, error) {
2510+ return &nullEnviron{cfg: cfg}, nil
2511 }
2512
2513 func checkImmutableString(cfg, old *environConfig, key string) error {
2514
2515=== added file 'provider/null/suite_test.go'
2516--- provider/null/suite_test.go 1970-01-01 00:00:00 +0000
2517+++ provider/null/suite_test.go 2013-11-18 06:38:24 +0000
2518@@ -0,0 +1,14 @@
2519+// Copyright 2013 Canonical Ltd.
2520+// Licensed under the AGPLv3, see LICENCE file for details.
2521+
2522+package null
2523+
2524+import (
2525+ "testing"
2526+
2527+ gc "launchpad.net/gocheck"
2528+)
2529+
2530+func Test(t *testing.T) {
2531+ gc.TestingT(t)
2532+}
2533
2534=== modified file 'provider/openstack/local_test.go'
2535--- provider/openstack/local_test.go 2013-11-05 04:32:17 +0000
2536+++ provider/openstack/local_test.go 2013-11-18 06:38:24 +0000
2537@@ -32,7 +32,6 @@
2538 "launchpad.net/juju-core/environs/tools"
2539 "launchpad.net/juju-core/instance"
2540 "launchpad.net/juju-core/juju/testing"
2541- "launchpad.net/juju-core/provider/common"
2542 "launchpad.net/juju-core/provider/openstack"
2543 coretesting "launchpad.net/juju-core/testing"
2544 jc "launchpad.net/juju-core/testing/checkers"
2545@@ -477,7 +476,7 @@
2546 c.Assert(err, gc.IsNil)
2547
2548 // check that the state holds the id of the bootstrap machine.
2549- stateData, err := common.LoadState(env.Storage())
2550+ stateData, err := bootstrap.LoadState(env.Storage())
2551 c.Assert(err, gc.IsNil)
2552 c.Assert(stateData.StateInstances, gc.HasLen, 1)
2553
2554
2555=== modified file 'provider/openstack/provider.go'
2556--- provider/openstack/provider.go 2013-11-04 02:49:34 +0000
2557+++ provider/openstack/provider.go 2013-11-18 06:38:24 +0000
2558@@ -470,7 +470,7 @@
2559 return stor
2560 }
2561
2562-func (e *environ) Bootstrap(cons constraints.Value, possibleTools tools.List) error {
2563+func (e *environ) Bootstrap(cons constraints.Value) error {
2564 // The client's authentication may have been reset when finding tools if the agent-version
2565 // attribute was updated so we need to re-authenticate. This will be a no-op if already authenticated.
2566 // An authenticated client is needed for the URL() call below.
2567@@ -478,7 +478,7 @@
2568 if err != nil {
2569 return err
2570 }
2571- return common.Bootstrap(e, cons, possibleTools)
2572+ return common.Bootstrap(e, cons)
2573 }
2574
2575 func (e *environ) StateInfo() (*state.Info, *api.Info, error) {

Subscribers

People subscribed via source and target branches

to status/vote changes: