Merge lp:~axwalk/juju-core/lp1236691-null-provider-default-series-take2 into lp:~go-bot/juju-core/trunk
- lp1236691-null-provider-default-series-take2
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Andrew Wilkins |
Approved revision: | no longer in the source branch. |
Merged at revision: | 2069 |
Proposed branch: | lp:~axwalk/juju-core/lp1236691-null-provider-default-series-take2 |
Merge into: | lp:~go-bot/juju-core/trunk |
Diff against target: |
2575 lines (+791/-539) 46 files modified
cmd/juju/bootstrap.go (+1/-95) cmd/juju/bootstrap_test.go (+32/-76) cmd/juju/upgradejuju.go (+1/-3) cmd/juju/upgradejuju_test.go (+1/-2) cmd/jujud/bootstrap.go (+2/-2) cmd/jujud/bootstrap_test.go (+3/-3) environs/bootstrap/bootstrap.go (+22/-34) environs/bootstrap/bootstrap_test.go (+12/-3) environs/bootstrap/state.go (+97/-0) environs/bootstrap/state_test.go (+118/-0) environs/bootstrap/synctools.go (+123/-0) environs/interface.go (+5/-2) environs/jujutest/livetests.go (+3/-2) environs/jujutest/tests.go (+6/-5) environs/manual/bootstrap.go (+22/-8) environs/manual/bootstrap_test.go (+47/-8) environs/manual/detection.go (+6/-4) environs/manual/detection_test.go (+3/-3) environs/manual/fakessh.go (+35/-24) environs/manual/provisioner.go (+1/-1) environs/manual/provisioner_test.go (+4/-4) environs/simplestreams/simplestreams.go (+1/-0) environs/sync/sync.go (+17/-5) environs/testing/tools.go (+7/-2) environs/tools/storage.go (+1/-0) environs/tools/tools.go (+2/-1) environs/tools/tools_test.go (+2/-2) provider/azure/environ.go (+2/-2) provider/azure/environ_test.go (+7/-7) provider/common/bootstrap.go (+38/-5) provider/common/bootstrap_test.go (+66/-12) provider/common/mock_test.go (+27/-0) provider/common/state.go (+2/-87) provider/common/state_test.go (+0/-103) provider/dummy/environs.go (+9/-3) provider/ec2/ec2.go (+2/-2) provider/ec2/local_test.go (+2/-3) provider/local/environ.go (+10/-3) provider/maas/environ.go (+2/-2) provider/maas/environ_test.go (+12/-6) provider/null/config_test.go (+0/-5) provider/null/environ.go (+16/-5) provider/null/provider.go (+5/-1) provider/null/suite_test.go (+14/-0) provider/openstack/local_test.go (+1/-2) provider/openstack/provider.go (+2/-2) |
To merge this branch: | bzr merge lp:~axwalk/juju-core/lp1236691-null-provider-default-series-take2 |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju Engineering | Pending | ||
Review via email:
|
Commit message
Providers are responsible for selecting tools
This change makes providers responsible for
selecting bootstrap tools, and ensuring they
are available in storage. The null provider is
updated to select the tools appropriate to the
bootstrap host. Other providers use a default
implementation in environs/bootstrap, which
has been extracted from cmd/juju/
One test fails due to a logging change. This is
fixed by https:/
Fixes #1236691
Description of the change
Providers are responsible for selecting tools
This change makes providers responsible for
selecting bootstrap tools, and ensuring they
are available in storage. The null provider is
updated to select the tools appropriate to the
bootstrap host. Other providers use a default
implementation in environs/bootstrap, which
has been extracted from cmd/juju/
One test fails due to a logging change. This is
fixed by https:/
Fixes #1236691
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Andrew Wilkins (axwalk) wrote : | # |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
William Reade (fwereade) wrote : | # |
On 2013/10/09 07:13:51, axw wrote:
> Please take a look.
I don't think this is the right approach, I'm afraid. The possibleTools
arg itself only exists because we couldn't justify refactoring any
further towards sanity given time pressure. Basically all this bootstrap
stuff is inside out... environs should generally know what tools they've
got available anyway, and using default-series to pick the bootstrap
node's is pretty dumb and pointless in the first place.
Ping me tomorrow, we can have a long rambling G+ and see what makes
sense.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Andrew Wilkins (axwalk) wrote : | # |
On 2013/10/09 18:04:30, fwereade wrote:
> On 2013/10/09 07:13:51, axw wrote:
> > Please take a look.
> I don't think this is the right approach, I'm afraid. The
possibleTools arg
> itself only exists because we couldn't justify refactoring any further
towards
> sanity given time pressure. Basically all this bootstrap stuff is
inside out...
> environs should generally know what tools they've got available
anyway, and
> using default-series to pick the bootstrap node's is pretty dumb and
pointless
> in the first place.
Fair enough. I started down the road of adding a "BootstrapSeries"
method to Environ, but started thinking it was a lot of change just for
one provider. But... perhaps this is the right thing to do. Once the
"wire up prechecker" branch is landed we'll have EnvironBase to add
default behaviour to, to keep the ugly down.
> Ping me tomorrow, we can have a long rambling G+ and see what makes
sense.
Sure thing, will be around before the weekly meeting.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Andrew Wilkins (axwalk) wrote : | # |
Please take a look.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
William Reade (fwereade) wrote : | # |
Sorry this has taken so long. I'm a bit uncomfortable about the sheer
amount of stuff we do, and feel like there's something more complex than
it needs to be... can we chat about this? Maybe your tomorrow morning?
https:/
File environs/
https:/
environs/
loggo.GetLogger
Ha. Thanks.
https:/
environs/
toolsList.Newest()
I'm not 100% sure this is right, but that's because it's not clear where
the tools list comes from and what its properties are. Any reason we
can't get it inside here?
Dimiter's just landing a List.NewestComp
useful?
https:/
File environs/
https:/
environs/
attempting to build and upload new tools")
Does this work with released versions of the local provider? That needs
to upload tools, and would ideally do so *without* the
juju.BootstrapC
https:/
environs/
return the list of tools in the target, and use that.
There's something still a bit off here, in that the copy-from-local
behaviour of upload is conceptually a sync, but ESCOPE.
https:/
File environs/
https:/
environs/
*instance.
I'll probably see it in a minute -- but when can they be known ahead of
time ina manual bootstrap?
https:/
File provider/
https:/
provider/
This seems to be repeated a lot. Can we pull it out into a common func?
https:/
File provider/
https:/
provider/
version.
That's what it should always choose anyway.
https:/
File provider/
https:/
provider/
heh, were we double-running these?
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Andrew Wilkins (axwalk) wrote : | # |
Please take a look.
https:/
File environs/
https:/
environs/
toolsList.Newest()
On 2013/11/14 09:29:45, fwereade wrote:
> I'm not 100% sure this is right, but that's because it's not clear
where the
> tools list comes from and what its properties are. Any reason we can't
get it
> inside here?
> Dimiter's just landing a List.NewestComp
useful?
Practically, the list will always come from EnsureToolsAvai
function uses environs.
with an appropriate version.
I'm not sure that NewestCompatible is appropriate here.
https:/
File environs/
https:/
environs/
attempting to build and upload new tools")
On 2013/11/14 09:29:45, fwereade wrote:
> Does this work with released versions of the local provider? That
needs to
> upload tools, and would ideally do so *without* the
juju.BootstrapC
> hack.
There's no change there. AFAICT, the hack in BootstrapCmd isn't
necessary, but I would like to run any change there by thumper, and
preferably do it in another CL.
For posterity: if the hack in BootstrapCmd goes, there's a change
necessary in provider/
auto-upload. Inside the Open method, the bit that sets agent-version
must be deleted.
https:/
File environs/
https:/
environs/
*instance.
On 2013/11/14 09:29:45, fwereade wrote:
> I'll probably see it in a minute -- but when can they be known ahead
of time ina
> manual bootstrap?
"Ahead of time" here just means before the environs/
function is called, not "ahead of bootstrap time".
The null/manual provider detects series/hardware in the provider's
Bootstrap method, and then passes it down. See provider/
https:/
File provider/
https:/
provider/
On 2013/11/14 09:29:45, fwereade wrote:
> This seems to be repeated a lot. Can we pull it out into a common
func?
Done. I hadn't done this before, as I didn't want to hide what was
happening. If you think of a better name that describes what's happening
underneath, I'd love to hear it.
https:/
File provider/
https:/
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Ian Booth (wallyworld) wrote : | # |
In general I like that responsibility for setting up tools has been
delegated to the providers. My main concern is that, as implemented, the
bootstrap is not exiting early enough if the environment is already
bootstrapped.
https:/
File cmd/juju/
https:/
cmd/juju/
One implication of removing this check and running it as part of
bootstrap.
only to then tell the user "sorry, already bootstrapped". This check
really does need to be run asap in the bootstrap process, before we do
anything, so that we fail as early as possible.
https:/
File environs/
https:/
environs/
configuration attribute.
s/update/updates
https:/
environs/
environs.Environ, toolsList coretools.List) (coretools.List, error) {
could we rename toolsList to availableTools or possibleTools to better
reflect the purpose of the parameter? callers of this method tend to
pass in a possibleTools variable so maybe that is the name to use
https:/
File environs/
https:/
environs/
Why did this file get renamed from fakessh_test.go?
The contents are only for testing. In general, we only use non
"_test.go" files for testing if there are methods called from outside
the package (I don't think that's the case here?); and if we do, we
usually introduce a "testing" package into which such files are put to
make it clear that the contents are for testing and not production
business logic.
https:/
File environs/
https:/
environs/
stop sync from going to s3
\o/
https:/
File provider/
https:/
provider/
tools.
Trouble is, we may already have uploaded tools prior to getting to this
point.
https:/
provider/
environs.Environ, series string, arch *string) (coretools.List, error) {
bikeshed - perhaps call this either SetupBootstrapTools or
EnsureBootstrap
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Andrew Wilkins (axwalk) wrote : | # |
Please take a look.
https:/
File cmd/juju/
https:/
cmd/juju/
On 2013/11/17 23:48:46, wallyworld wrote:
> One implication of removing this check and running it as part of
> bootstrap.
only to
> then tell the user "sorry, already bootstrapped". This check really
does need to
> be run asap in the bootstrap process, before we do anything, so that
we fail as
> early as possible.
There are a couple of nice things about putting it inside Bootstrap, one
being that Bootstrap fails if the machine is already bootstrapped (no
need to test that property via the CLI command). But you're right, we
shouldn't break behaviour for --upload-tools. If nothing else, it'll
make developers' lives harder.
I'll revert this.
https:/
File environs/
https:/
environs/
configuration attribute.
On 2013/11/17 23:48:46, wallyworld wrote:
> s/update/updates
Done.
https:/
environs/
environs.Environ, toolsList coretools.List) (coretools.List, error) {
On 2013/11/17 23:48:46, wallyworld wrote:
> could we rename toolsList to availableTools or possibleTools to better
reflect
> the purpose of the parameter? callers of this method tend to pass in a
> possibleTools variable so maybe that is the name to use
Done.
https:/
File environs/
https:/
environs/
On 2013/11/17 23:48:46, wallyworld wrote:
> Why did this file get renamed from fakessh_test.go?
> The contents are only for testing. In general, we only use non
"_test.go" files
> for testing if there are methods called from outside the package (I
don't think
> that's the case here?); and if we do, we usually introduce a "testing"
package
> into which such files are put to make it clear that the contents are
for testing
> and not production business logic.
I had intended to use it in a separate package, but didn't end up doing
that. I'll revert this.
BTW: normally I'd put the test code in a separate package, but in this
case it needs to know some internals of the package (or otherwise we
expose parts that are even more private, such as the detection script).
https:/
File provider/
https:/
provider/
tools.
On 2013/11/17 23:48:46, wallyworld wrote:
> ...
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Ian Booth (wallyworld) wrote : | # |
Thanks :-)
LGTM
https:/
File cmd/juju/
https:/
cmd/juju/
On 2013/11/18 05:10:01, axw wrote:
> On 2013/11/17 23:48:46, wallyworld wrote:
> > One implication of removing this check and running it as part of
> > bootstrap.
tools only to
> > then tell the user "sorry, already bootstrapped". This check really
does need
> to
> > be run asap in the bootstrap process, before we do anything, so that
we fail
> as
> > early as possible.
> There are a couple of nice things about putting it inside Bootstrap,
one being
> that Bootstrap fails if the machine is already bootstrapped (no need
to test
> that property via the CLI command). But you're right, we shouldn't
break
> behaviour for --upload-tools. If nothing else, it'll make developers'
lives
> harder.
> I'll revert this.
Yeah, I agree it's a bit unfortunate. It's not just developers also -
people like Dave and Marco regularly run from source to get the latest
goodness.
Preview Diff
1 | === modified file 'cmd/juju/bootstrap.go' |
2 | --- cmd/juju/bootstrap.go 2013-11-04 02:01:19 +0000 |
3 | +++ cmd/juju/bootstrap.go 2013-11-18 06:38:24 +0000 |
4 | @@ -5,11 +5,9 @@ |
5 | |
6 | import ( |
7 | "fmt" |
8 | - "io" |
9 | "strings" |
10 | |
11 | "launchpad.net/gnuflag" |
12 | - "launchpad.net/loggo" |
13 | |
14 | "launchpad.net/juju-core/charm" |
15 | "launchpad.net/juju-core/cmd" |
16 | @@ -19,10 +17,7 @@ |
17 | "launchpad.net/juju-core/environs/config" |
18 | "launchpad.net/juju-core/environs/configstore" |
19 | "launchpad.net/juju-core/environs/sync" |
20 | - envtools "launchpad.net/juju-core/environs/tools" |
21 | - "launchpad.net/juju-core/errors" |
22 | "launchpad.net/juju-core/provider" |
23 | - coretools "launchpad.net/juju-core/tools" |
24 | "launchpad.net/juju-core/utils/set" |
25 | "launchpad.net/juju-core/version" |
26 | ) |
27 | @@ -57,7 +52,6 @@ |
28 | Constraints constraints.Value |
29 | UploadTools bool |
30 | Series []string |
31 | - Source string |
32 | } |
33 | |
34 | func (c *BootstrapCommand) Info() *cmd.Info { |
35 | @@ -73,7 +67,6 @@ |
36 | f.Var(constraints.ConstraintsValue{&c.Constraints}, "constraints", "set environment constraints") |
37 | f.BoolVar(&c.UploadTools, "upload-tools", false, "upload local version of tools before bootstrapping") |
38 | f.Var(seriesVar{&c.Series}, "series", "upload tools for supplied comma-separated series list") |
39 | - f.StringVar(&c.Source, "source", "", "local path to use as tools source") |
40 | } |
41 | |
42 | func (c *BootstrapCommand) Init(args []string) error { |
43 | @@ -102,8 +95,6 @@ |
44 | return fmt.Errorf("failed to enable bootstrap storage: %v", err) |
45 | } |
46 | } |
47 | - // Check to see if the environment is already bootstrapped |
48 | - // before potentially uploading any tools. |
49 | if err := bootstrap.EnsureNotBootstrapped(environ); err != nil { |
50 | return err |
51 | } |
52 | @@ -120,10 +111,6 @@ |
53 | return err |
54 | } |
55 | } |
56 | - err = c.ensureToolsAvailability(environ, ctx, c.UploadTools) |
57 | - if err != nil { |
58 | - return err |
59 | - } |
60 | return bootstrap.Bootstrap(environ, c.Constraints) |
61 | } |
62 | |
63 | @@ -133,7 +120,7 @@ |
64 | forceVersion := uploadVersion(version.Current.Number, nil) |
65 | cfg := environ.Config() |
66 | series := getUploadSeries(cfg, c.Series) |
67 | - agenttools, err := uploadTools(environ.Storage(), &forceVersion, series...) |
68 | + agenttools, err := sync.Upload(environ.Storage(), &forceVersion, series...) |
69 | if err != nil { |
70 | return err |
71 | } |
72 | @@ -149,87 +136,6 @@ |
73 | return nil |
74 | } |
75 | |
76 | -const NoToolsMessage = ` |
77 | -Juju cannot bootstrap because no tools are available for your environment. |
78 | -An attempt was made to build and upload appropriate tools but this was unsuccessful. |
79 | - |
80 | -` |
81 | - |
82 | -const NoToolsNoUploadMessage = ` |
83 | -Juju cannot bootstrap because no tools are available for your environment. |
84 | -In addition, no tools could be located to upload. |
85 | -You may want to use the 'tools-metadata-url' configuration setting to specify the tools location. |
86 | - |
87 | -` |
88 | - |
89 | -func processToolsError(w io.Writer, err *error, uploadAttempted *bool) { |
90 | - |
91 | - if *uploadAttempted && *err != nil { |
92 | - fmt.Fprint(w, NoToolsMessage) |
93 | - } else { |
94 | - if errors.IsNotFoundError(*err) || *err == coretools.ErrNoMatches { |
95 | - fmt.Fprint(w, NoToolsNoUploadMessage) |
96 | - } |
97 | - } |
98 | -} |
99 | - |
100 | -// ensureToolsAvailability verifies the tools are available. If no tools are |
101 | -// found, it will automatically synchronize them. |
102 | -func (c *BootstrapCommand) ensureToolsAvailability(env environs.Environ, ctx *cmd.Context, uploadPerformed bool) (err error) { |
103 | - uploadAttempted := false |
104 | - defer processToolsError(ctx.Stderr, &err, &uploadAttempted) |
105 | - // Capture possible logging while syncing and write it on the screen. |
106 | - loggo.RegisterWriter("bootstrap", cmd.NewCommandLogWriter("juju.environs.sync", ctx.Stdout, ctx.Stderr), loggo.INFO) |
107 | - defer loggo.RemoveWriter("bootstrap") |
108 | - |
109 | - // Try to find bootstrap tools. |
110 | - cfg := env.Config() |
111 | - var vers *version.Number |
112 | - if agentVersion, ok := cfg.AgentVersion(); ok { |
113 | - vers = &agentVersion |
114 | - } |
115 | - logger.Debugf("looking for bootstrap tools") |
116 | - params := envtools.BootstrapToolsParams{ |
117 | - Version: vers, |
118 | - Arch: c.Constraints.Arch, |
119 | - AllowRetry: uploadPerformed, |
120 | - } |
121 | - _, err = envtools.FindBootstrapTools(env, params) |
122 | - if err == nil || !errors.IsNotFoundError(err) || uploadPerformed { |
123 | - return err |
124 | - } |
125 | - // If no tools are available, synchronize if necessary. |
126 | - uploadRequired := true |
127 | - if c.Source != "" { |
128 | - // If we are syncing tools and it succeeds, we don't need to upload the tools. |
129 | - uploadRequired = false |
130 | - logger.Warningf("no tools available, attempting to retrieve from %v", c.Source) |
131 | - sctx := &sync.SyncContext{ |
132 | - Target: env.Storage(), |
133 | - Source: c.Source, |
134 | - } |
135 | - if err = syncTools(sctx); err != nil { |
136 | - if (err == coretools.ErrNoMatches || err == envtools.ErrNoTools) && vers == nil && version.Current.IsDev() { |
137 | - uploadRequired = true |
138 | - } else { |
139 | - return err |
140 | - } |
141 | - } |
142 | - } |
143 | - // No suitable tools could be synced so try uploading. |
144 | - if uploadRequired { |
145 | - logger.Infof("no tools found, so attempting to build and upload new tools") |
146 | - uploadAttempted = true |
147 | - if err = c.uploadTools(env); err != nil { |
148 | - return err |
149 | - } |
150 | - } |
151 | - // Synchronization/upload done, try again. |
152 | - params.AllowRetry = true |
153 | - _, err = envtools.FindBootstrapTools(env, params) |
154 | - return err |
155 | -} |
156 | - |
157 | type seriesVar struct { |
158 | target *[]string |
159 | } |
160 | |
161 | === modified file 'cmd/juju/bootstrap_test.go' |
162 | --- cmd/juju/bootstrap_test.go 2013-10-25 22:51:39 +0000 |
163 | +++ cmd/juju/bootstrap_test.go 2013-11-18 06:38:24 +0000 |
164 | @@ -46,6 +46,10 @@ |
165 | s.LoggingSuite.SetUpTest(c) |
166 | s.MgoSuite.SetUpTest(c) |
167 | s.ToolsFixture.SetUpTest(c) |
168 | + |
169 | + // Set up a local source with tools. |
170 | + sourceDir := createToolsSource(c, vAll) |
171 | + s.PatchValue(&sync.DefaultToolsLocation, sourceDir) |
172 | } |
173 | |
174 | func (s *BootstrapSuite) TearDownSuite(c *gc.C) { |
175 | @@ -76,24 +80,24 @@ |
176 | var bootstrapRetryTests = []bootstrapRetryTest{{ |
177 | info: "no tools uploaded, first check has no retries; no matching binary in source; sync fails with no second attempt", |
178 | expectedAllowRetry: []bool{false}, |
179 | - err: "no tools available", |
180 | + err: "cannot find bootstrap tools: no matching tools available", |
181 | version: "1.16.0-precise-amd64", |
182 | }, { |
183 | info: "no tools uploaded, first check has no retries; matching binary in source; check after sync has retries", |
184 | expectedAllowRetry: []bool{false, true}, |
185 | - err: "tools not found", |
186 | + err: "cannot find bootstrap tools: tools not found", |
187 | version: "1.16.0-precise-amd64", |
188 | addVersionToSource: true, |
189 | }, { |
190 | info: "no tools uploaded, first check has no retries; no matching binary in source; check after upload has retries", |
191 | expectedAllowRetry: []bool{false, true}, |
192 | - err: "tools not found", |
193 | + err: "cannot find bootstrap tools: tools not found", |
194 | version: "1.15.1-precise-amd64", // dev version to force upload |
195 | }, { |
196 | info: "new tools uploaded, so we want to allow retries to give them a chance at showing up", |
197 | args: []string{"--upload-tools"}, |
198 | expectedAllowRetry: []bool{true}, |
199 | - err: "tools not found", |
200 | + err: "cannot find bootstrap tools: no matching tools available", |
201 | }} |
202 | |
203 | // Test test checks that bootstrap calls FindTools with the expected allowRetry flag. |
204 | @@ -105,20 +109,20 @@ |
205 | } |
206 | |
207 | func (s *BootstrapSuite) runAllowRetriesTest(c *gc.C, test bootstrapRetryTest) { |
208 | - var extraVersions []version.Binary |
209 | + toolsVersions := envtesting.VAll |
210 | if test.version != "" { |
211 | testVersion := version.MustParseBinary(test.version) |
212 | restore := testbase.PatchValue(&version.Current, testVersion) |
213 | defer restore() |
214 | if test.addVersionToSource { |
215 | - extraVersions = append(extraVersions, testVersion) |
216 | + toolsVersions = append([]version.Binary{}, toolsVersions...) |
217 | + toolsVersions = append(toolsVersions, testVersion) |
218 | } |
219 | } |
220 | _, fake := makeEmptyFakeHome(c) |
221 | defer fake.Restore() |
222 | - // Create some source tools in a local directory. |
223 | - sourceDir := createToolsSource(c, extraVersions) |
224 | - test.args = append(test.args, []string{"--source", sourceDir}...) |
225 | + sourceDir := createToolsSource(c, toolsVersions) |
226 | + s.PatchValue(&sync.DefaultToolsLocation, sourceDir) |
227 | |
228 | var findToolsRetryValues []bool |
229 | mockFindTools := func(cloudInst environs.ConfigGetter, majorVersion, minorVersion int, |
230 | @@ -137,9 +141,7 @@ |
231 | } |
232 | |
233 | func (s *BootstrapSuite) TestTest(c *gc.C) { |
234 | - uploadTools = mockUploadTools |
235 | - defer func() { uploadTools = sync.Upload }() |
236 | - |
237 | + s.PatchValue(&sync.Upload, mockUploadTools) |
238 | for i, test := range bootstrapTests { |
239 | c.Logf("\ntest %d: %s", i, test.info) |
240 | test.run(c) |
241 | @@ -160,9 +162,6 @@ |
242 | } |
243 | |
244 | func (test bootstrapTest) run(c *gc.C) { |
245 | - // Prepare a mock storage for testing. |
246 | - sourceDir := createToolsSource(c, vAll) |
247 | - |
248 | // Create home with dummy provider and remove all |
249 | // of its envtools. |
250 | env, fake := makeEmptyFakeHome(c) |
251 | @@ -182,7 +181,6 @@ |
252 | } |
253 | |
254 | // Run command and check for uploads. |
255 | - test.args = append(test.args, []string{"--source", sourceDir}...) |
256 | opc, errc := runCommand(nullContext(), new(BootstrapCommand), test.args...) |
257 | if uploadCount > 0 { |
258 | for i := 0; i < uploadCount; i++ { |
259 | @@ -303,10 +301,9 @@ |
260 | defer fake.Restore() |
261 | defaultSeriesVersion := version.Current |
262 | defaultSeriesVersion.Series = env.Config().DefaultSeries() |
263 | - sourceDir := createToolsSource(c, append(vAll, defaultSeriesVersion)) |
264 | |
265 | ctx := coretesting.Context(c) |
266 | - code := cmd.Main(&BootstrapCommand{}, ctx, []string{"--source", sourceDir}) |
267 | + code := cmd.Main(&BootstrapCommand{}, ctx, nil) |
268 | c.Check(code, gc.Equals, 0) |
269 | |
270 | ctx2 := coretesting.Context(c) |
271 | @@ -316,51 +313,10 @@ |
272 | c.Check(coretesting.Stdout(ctx2), gc.Equals, "") |
273 | } |
274 | |
275 | -func (s *BootstrapSuite) TestAutoSyncLocalSource(c *gc.C) { |
276 | - // Prepare a tools directory for testing and store the |
277 | - // dummy tools in there. |
278 | - sourceDir := createToolsSource(c, vAll) |
279 | - |
280 | - // Change the version and ensure its later restoring. |
281 | - origVersion := version.Current |
282 | - version.Current.Number = version.MustParse("1.2.0") |
283 | - defer func() { |
284 | - version.Current = origVersion |
285 | - }() |
286 | - |
287 | - // Create home with dummy provider and remove all |
288 | - // of its envtools. |
289 | - env, fake := makeEmptyFakeHome(c) |
290 | - defer fake.Restore() |
291 | - |
292 | - // Bootstrap the environment with an invalid source. |
293 | - // The command returns with an error. |
294 | - ctx := coretesting.Context(c) |
295 | - code := cmd.Main(&BootstrapCommand{}, ctx, []string{"--source", c.MkDir()}) |
296 | - c.Check(code, gc.Equals, 1) |
297 | - |
298 | - // Now check that there are no tools available. |
299 | - _, err := envtools.FindTools( |
300 | - env, version.Current.Major, version.Current.Minor, coretools.Filter{}, envtools.DoNotAllowRetry) |
301 | - c.Assert(err, gc.FitsTypeOf, errors.NotFoundf("")) |
302 | - |
303 | - // Bootstrap the environment with the valid source. This time |
304 | - // the bootstrapping has to show no error, because the tools |
305 | - // are automatically synchronized. |
306 | - ctx = coretesting.Context(c) |
307 | - code = cmd.Main(&BootstrapCommand{}, ctx, []string{"--source", sourceDir}) |
308 | - c.Check(code, gc.Equals, 0) |
309 | - |
310 | - // Now check the available tools which are the 1.2.0 envtools. |
311 | - checkTools(c, env, v120All) |
312 | -} |
313 | - |
314 | -func (s *BootstrapSuite) setupAutoUploadTest(c *gc.C, vers, series string) (environs.Environ, string) { |
315 | - uploadTools = mockUploadTools |
316 | - s.AddCleanup(func(*gc.C) { uploadTools = sync.Upload }) |
317 | - |
318 | - // Set up a local source with tools. |
319 | - sourceDir := createToolsSource(c, vAll) |
320 | +func (s *BootstrapSuite) setupAutoUploadTest(c *gc.C, vers, series string) environs.Environ { |
321 | + s.PatchValue(&sync.Upload, mockUploadTools) |
322 | + sourceDir := createToolsSource(c, vAll) |
323 | + s.PatchValue(&sync.DefaultToolsLocation, sourceDir) |
324 | |
325 | // Change the tools location to be the test location and also |
326 | // the version and ensure their later restoring. |
327 | @@ -375,7 +331,7 @@ |
328 | // of its envtools. |
329 | env, fake := makeEmptyFakeHome(c) |
330 | s.AddCleanup(func(*gc.C) { fake.Restore() }) |
331 | - return env, sourceDir |
332 | + return env |
333 | } |
334 | |
335 | func (s *BootstrapSuite) TestAutoUploadAfterFailedSync(c *gc.C) { |
336 | @@ -383,9 +339,9 @@ |
337 | if otherSeries == version.Current.Series { |
338 | otherSeries = "raring" |
339 | } |
340 | - env, sourceDir := s.setupAutoUploadTest(c, "1.7.3", otherSeries) |
341 | + env := s.setupAutoUploadTest(c, "1.7.3", otherSeries) |
342 | // Run command and check for that upload has been run for tools matching the current juju version. |
343 | - opc, errc := runCommand(nullContext(), new(BootstrapCommand), "--source", sourceDir) |
344 | + opc, errc := runCommand(nullContext(), new(BootstrapCommand)) |
345 | c.Assert(<-errc, gc.IsNil) |
346 | c.Assert((<-opc).(dummy.OpPutFile).Env, gc.Equals, "peckham") |
347 | list, err := envtools.FindTools(env, version.Current.Major, version.Current.Minor, coretools.Filter{}, false) |
348 | @@ -405,20 +361,20 @@ |
349 | } |
350 | |
351 | func (s *BootstrapSuite) TestAutoUploadOnlyForDev(c *gc.C) { |
352 | - _, sourceDir := s.setupAutoUploadTest(c, "1.8.3", "precise") |
353 | - _, errc := runCommand(nullContext(), new(BootstrapCommand), "--source", sourceDir) |
354 | + s.setupAutoUploadTest(c, "1.8.3", "precise") |
355 | + _, errc := runCommand(nullContext(), new(BootstrapCommand)) |
356 | err := <-errc |
357 | - c.Assert(err, gc.ErrorMatches, "no matching tools available") |
358 | + c.Assert(err, gc.ErrorMatches, "cannot find bootstrap tools: no matching tools available") |
359 | } |
360 | |
361 | func (s *BootstrapSuite) TestMissingToolsError(c *gc.C) { |
362 | - _, sourceDir := s.setupAutoUploadTest(c, "1.8.3", "precise") |
363 | + s.setupAutoUploadTest(c, "1.8.3", "precise") |
364 | context := coretesting.Context(c) |
365 | - code := cmd.Main(&BootstrapCommand{}, context, []string{"--source", sourceDir}) |
366 | + code := cmd.Main(&BootstrapCommand{}, context, nil) |
367 | c.Assert(code, gc.Equals, 1) |
368 | errText := context.Stderr.(*bytes.Buffer).String() |
369 | errText = strings.Replace(errText, "\n", "", -1) |
370 | - expectedErrText := strings.Replace(fmt.Sprintf(".*%s.*", NoToolsNoUploadMessage), "\n", "", -1) |
371 | + expectedErrText := "error: cannot find bootstrap tools: no matching tools available" |
372 | c.Assert(errText, gc.Matches, expectedErrText) |
373 | } |
374 | |
375 | @@ -427,14 +383,14 @@ |
376 | } |
377 | |
378 | func (s *BootstrapSuite) TestMissingToolsUploadFailedError(c *gc.C) { |
379 | - _, sourceDir := s.setupAutoUploadTest(c, "1.7.3", "precise") |
380 | - uploadTools = uploadToolsAlwaysFails |
381 | + s.setupAutoUploadTest(c, "1.7.3", "precise") |
382 | + s.PatchValue(&sync.Upload, uploadToolsAlwaysFails) |
383 | context := coretesting.Context(c) |
384 | - code := cmd.Main(&BootstrapCommand{}, context, []string{"--source", sourceDir}) |
385 | + code := cmd.Main(&BootstrapCommand{}, context, nil) |
386 | c.Assert(code, gc.Equals, 1) |
387 | errText := context.Stderr.(*bytes.Buffer).String() |
388 | errText = strings.Replace(errText, "\n", "", -1) |
389 | - expectedErrText := strings.Replace(fmt.Sprintf(".*%s.*", NoToolsMessage), "\n", "", -1) |
390 | + expectedErrText := "error: cannot find bootstrap tools: an error" |
391 | c.Assert(errText, gc.Matches, expectedErrText) |
392 | } |
393 | |
394 | |
395 | === modified file 'cmd/juju/upgradejuju.go' |
396 | --- cmd/juju/upgradejuju.go 2013-11-14 09:07:52 +0000 |
397 | +++ cmd/juju/upgradejuju.go 2013-11-18 06:38:24 +0000 |
398 | @@ -31,8 +31,6 @@ |
399 | Series []string |
400 | } |
401 | |
402 | -var uploadTools = sync.Upload |
403 | - |
404 | var upgradeJujuDoc = ` |
405 | The upgrade-juju command upgrades a running environment by setting a version |
406 | number for all juju agents to run. By default, it chooses the most recent |
407 | @@ -226,7 +224,7 @@ |
408 | // include all the extra series we build, so we can set *that* onto |
409 | // v.available and maybe one day be able to check that a given upgrade |
410 | // won't leave out-of-date machines lying around, starved of envtools. |
411 | - uploaded, err := uploadTools(storage, &v.chosen, series...) |
412 | + uploaded, err := sync.Upload(storage, &v.chosen, series...) |
413 | if err != nil { |
414 | return err |
415 | } |
416 | |
417 | === modified file 'cmd/juju/upgradejuju_test.go' |
418 | --- cmd/juju/upgradejuju_test.go 2013-11-14 09:07:52 +0000 |
419 | +++ cmd/juju/upgradejuju_test.go 2013-11-18 06:38:24 +0000 |
420 | @@ -273,11 +273,10 @@ |
421 | } |
422 | |
423 | func (s *UpgradeJujuSuite) TestUpgradeJuju(c *gc.C) { |
424 | + s.PatchValue(&sync.Upload, mockUploadTools) |
425 | oldVersion := version.Current |
426 | - uploadTools = mockUploadTools |
427 | defer func() { |
428 | version.Current = oldVersion |
429 | - uploadTools = sync.Upload |
430 | }() |
431 | |
432 | for i, test := range upgradeJujuTests { |
433 | |
434 | === modified file 'cmd/jujud/bootstrap.go' |
435 | --- cmd/jujud/bootstrap.go 2013-10-04 17:10:00 +0000 |
436 | +++ cmd/jujud/bootstrap.go 2013-11-18 06:38:24 +0000 |
437 | @@ -15,10 +15,10 @@ |
438 | "launchpad.net/juju-core/agent" |
439 | "launchpad.net/juju-core/cmd" |
440 | "launchpad.net/juju-core/constraints" |
441 | + "launchpad.net/juju-core/environs/bootstrap" |
442 | "launchpad.net/juju-core/environs/cloudinit" |
443 | "launchpad.net/juju-core/environs/config" |
444 | "launchpad.net/juju-core/instance" |
445 | - "launchpad.net/juju-core/provider/common" |
446 | "launchpad.net/juju-core/state" |
447 | ) |
448 | |
449 | @@ -62,7 +62,7 @@ |
450 | return fmt.Errorf("cannot read provider-state-url file: %v", err) |
451 | } |
452 | stateInfoURL := strings.Split(string(data), "\n")[0] |
453 | - bsState, err := common.LoadStateFromURL(stateInfoURL) |
454 | + bsState, err := bootstrap.LoadStateFromURL(stateInfoURL) |
455 | if err != nil { |
456 | return fmt.Errorf("cannot load state from URL %q (read from %q): %v", stateInfoURL, providerStateURLFile, err) |
457 | } |
458 | |
459 | === modified file 'cmd/jujud/bootstrap_test.go' |
460 | --- cmd/jujud/bootstrap_test.go 2013-11-06 13:38:01 +0000 |
461 | +++ cmd/jujud/bootstrap_test.go 2013-11-18 06:38:24 +0000 |
462 | @@ -13,10 +13,10 @@ |
463 | |
464 | "launchpad.net/juju-core/agent" |
465 | "launchpad.net/juju-core/constraints" |
466 | + "launchpad.net/juju-core/environs/bootstrap" |
467 | "launchpad.net/juju-core/environs/jujutest" |
468 | "launchpad.net/juju-core/errors" |
469 | "launchpad.net/juju-core/instance" |
470 | - "launchpad.net/juju-core/provider/common" |
471 | "launchpad.net/juju-core/provider/dummy" |
472 | "launchpad.net/juju-core/state" |
473 | "launchpad.net/juju-core/testing" |
474 | @@ -46,12 +46,12 @@ |
475 | func (s *BootstrapSuite) SetUpSuite(c *gc.C) { |
476 | s.LoggingSuite.SetUpSuite(c) |
477 | s.MgoSuite.SetUpSuite(c) |
478 | - stateInfo := common.BootstrapState{ |
479 | + stateInfo := bootstrap.BootstrapState{ |
480 | StateInstances: []instance.Id{instance.Id("dummy.instance.id")}, |
481 | } |
482 | stateData, err := goyaml.Marshal(stateInfo) |
483 | c.Assert(err, gc.IsNil) |
484 | - content := map[string]string{"/" + common.StateFile: string(stateData)} |
485 | + content := map[string]string{"/" + bootstrap.StateFile: string(stateData)} |
486 | testRoundTripper.Sub = jujutest.NewCannedRoundTripper(content, nil) |
487 | s.providerStateURLFile = filepath.Join(c.MkDir(), "provider-state-url") |
488 | providerStateURLFile = s.providerStateURLFile |
489 | |
490 | === modified file 'environs/bootstrap/bootstrap.go' |
491 | --- environs/bootstrap/bootstrap.go 2013-10-04 16:16:10 +0000 |
492 | +++ environs/bootstrap/bootstrap.go 2013-11-18 06:38:24 +0000 |
493 | @@ -10,12 +10,11 @@ |
494 | |
495 | "launchpad.net/juju-core/constraints" |
496 | "launchpad.net/juju-core/environs" |
497 | - "launchpad.net/juju-core/environs/tools" |
498 | - "launchpad.net/juju-core/provider/common" |
499 | + coretools "launchpad.net/juju-core/tools" |
500 | "launchpad.net/juju-core/version" |
501 | ) |
502 | |
503 | -var logger = loggo.GetLogger("juju.environs.boostrap") |
504 | +var logger = loggo.GetLogger("juju.environs.bootstrap") |
505 | |
506 | // Bootstrap bootstraps the given environment. The supplied constraints are |
507 | // used to provision the instance, and are also set within the bootstrapped |
508 | @@ -43,51 +42,40 @@ |
509 | if err := environs.VerifyStorage(environ.Storage()); err != nil { |
510 | return err |
511 | } |
512 | - |
513 | logger.Infof("bootstrapping environment %q", environ.Name()) |
514 | - var vers *version.Number |
515 | - if agentVersion, ok := cfg.AgentVersion(); ok { |
516 | - vers = &agentVersion |
517 | - } |
518 | - params := tools.BootstrapToolsParams{ |
519 | - Version: vers, |
520 | - Arch: cons.Arch, |
521 | - } |
522 | - newestTools, err := tools.FindBootstrapTools(environ, params) |
523 | - if err != nil { |
524 | - return fmt.Errorf("cannot find bootstrap tools: %v", err) |
525 | - } |
526 | + return environ.Bootstrap(cons) |
527 | +} |
528 | |
529 | - // If agent version was not previously known, set it here using the latest compatible tools version. |
530 | - if vers == nil { |
531 | - // We probably still have a mix of versions available; discard older ones |
532 | - // and update environment configuration to use only those remaining. |
533 | - var newVersion version.Number |
534 | - newVersion, newestTools = newestTools.Newest() |
535 | - vers = &newVersion |
536 | - logger.Infof("environs: picked newest version: %s", *vers) |
537 | - cfg, err = cfg.Apply(map[string]interface{}{ |
538 | - "agent-version": vers.String(), |
539 | +// SetBootstrapTools returns the newest tools from the given tools list, |
540 | +// and updates the agent-version configuration attribute. |
541 | +func SetBootstrapTools(environ environs.Environ, possibleTools coretools.List) (coretools.List, error) { |
542 | + if len(possibleTools) == 0 { |
543 | + return nil, fmt.Errorf("no bootstrap tools available") |
544 | + } |
545 | + var newVersion version.Number |
546 | + newVersion, toolsList := possibleTools.Newest() |
547 | + logger.Infof("picked newest version: %s", newVersion) |
548 | + cfg := environ.Config() |
549 | + if agentVersion, _ := cfg.AgentVersion(); agentVersion != newVersion { |
550 | + cfg, err := cfg.Apply(map[string]interface{}{ |
551 | + "agent-version": newVersion.String(), |
552 | }) |
553 | if err == nil { |
554 | err = environ.SetConfig(cfg) |
555 | } |
556 | if err != nil { |
557 | - return fmt.Errorf("failed to update environment configuration: %v", err) |
558 | + return nil, fmt.Errorf("failed to update environment configuration: %v", err) |
559 | } |
560 | } |
561 | - // ensure we have at least one valid tools |
562 | - if len(newestTools) == 0 { |
563 | - return fmt.Errorf("No bootstrap tools found") |
564 | - } |
565 | - return environ.Bootstrap(cons, newestTools) |
566 | + return toolsList, nil |
567 | } |
568 | |
569 | // EnsureNotBootstrapped returns null if the environment is not bootstrapped, |
570 | // and an error if it is or if the function was not able to tell. |
571 | func EnsureNotBootstrapped(env environs.Environ) error { |
572 | - _, err := common.LoadState(env.Storage()) |
573 | - // If there is no error loading the bootstrap state, then we are bootstrapped. |
574 | + _, err := LoadState(env.Storage()) |
575 | + // If there is no error loading the bootstrap state, then we are |
576 | + // bootstrapped. |
577 | if err == nil { |
578 | return fmt.Errorf("environment is already bootstrapped") |
579 | } |
580 | |
581 | === modified file 'environs/bootstrap/bootstrap_test.go' |
582 | --- environs/bootstrap/bootstrap_test.go 2013-10-24 00:20:59 +0000 |
583 | +++ environs/bootstrap/bootstrap_test.go 2013-11-18 06:38:24 +0000 |
584 | @@ -21,7 +21,6 @@ |
585 | "launchpad.net/juju-core/provider/dummy" |
586 | coretesting "launchpad.net/juju-core/testing" |
587 | "launchpad.net/juju-core/testing/testbase" |
588 | - "launchpad.net/juju-core/tools" |
589 | "launchpad.net/juju-core/version" |
590 | ) |
591 | |
592 | @@ -183,11 +182,21 @@ |
593 | } |
594 | } |
595 | |
596 | -func (s *bootstrapSuite) TestBootstrapNeedsTools(c *gc.C) { |
597 | +func (s *bootstrapSuite) TestBootstrapNoTools(c *gc.C) { |
598 | env := newEnviron("foo", useDefaultKeys) |
599 | s.setDummyStorage(c, env) |
600 | envtesting.RemoveFakeTools(c, env.Storage()) |
601 | err := bootstrap.Bootstrap(env, constraints.Value{}) |
602 | + // bootstrap.Bootstrap leaves it to the provider to |
603 | + // locate bootstrap tools. |
604 | + c.Assert(err, gc.IsNil) |
605 | +} |
606 | + |
607 | +func (s *bootstrapSuite) TestEnsureToolsAvailability(c *gc.C) { |
608 | + env := newEnviron("foo", useDefaultKeys) |
609 | + s.setDummyStorage(c, env) |
610 | + envtesting.RemoveFakeTools(c, env.Storage()) |
611 | + _, err := bootstrap.EnsureToolsAvailability(env, env.Config().DefaultSeries(), nil) |
612 | c.Check(err, gc.ErrorMatches, "cannot find bootstrap tools: invalid URL.*") |
613 | } |
614 | |
615 | @@ -243,7 +252,7 @@ |
616 | return e.name |
617 | } |
618 | |
619 | -func (e *bootstrapEnviron) Bootstrap(cons constraints.Value, possibleTools tools.List) error { |
620 | +func (e *bootstrapEnviron) Bootstrap(cons constraints.Value) error { |
621 | e.bootstrapCount++ |
622 | e.constraints = cons |
623 | return nil |
624 | |
625 | === added file 'environs/bootstrap/state.go' |
626 | --- environs/bootstrap/state.go 1970-01-01 00:00:00 +0000 |
627 | +++ environs/bootstrap/state.go 2013-11-18 06:38:24 +0000 |
628 | @@ -0,0 +1,97 @@ |
629 | +// Copyright 2012, 2013 Canonical Ltd. |
630 | +// Licensed under the AGPLv3, see LICENCE file for details. |
631 | + |
632 | +package bootstrap |
633 | + |
634 | +import ( |
635 | + "bytes" |
636 | + "fmt" |
637 | + "io" |
638 | + "io/ioutil" |
639 | + "net/http" |
640 | + |
641 | + "launchpad.net/goyaml" |
642 | + |
643 | + "launchpad.net/juju-core/environs" |
644 | + "launchpad.net/juju-core/environs/storage" |
645 | + coreerrors "launchpad.net/juju-core/errors" |
646 | + "launchpad.net/juju-core/instance" |
647 | +) |
648 | + |
649 | +// StateFile is the name of the file where the provider's state is stored. |
650 | +const StateFile = "provider-state" |
651 | + |
652 | +// BootstrapState is the state information that is stored in StateFile. |
653 | +// |
654 | +// Individual providers may define their own state structures instead of |
655 | +// this one, and use their own code for loading and saving those, but this is |
656 | +// the definition that most practically useful providers share unchanged. |
657 | +type BootstrapState struct { |
658 | + // StateInstances are the state servers. |
659 | + StateInstances []instance.Id `yaml:"state-instances"` |
660 | + // Characteristics reflect the hardware each state server is running on. |
661 | + // This is used at bootstrap time so the state server knows what hardware it has. |
662 | + // The state *may* be updated later without this information, but by then it's |
663 | + // served it's purpose. |
664 | + Characteristics []instance.HardwareCharacteristics `yaml:"characteristics,omitempty"` |
665 | +} |
666 | + |
667 | +// putState writes the given data to the state file on the given storage. |
668 | +// The file's name is as defined in StateFile. |
669 | +func putState(storage storage.StorageWriter, data []byte) error { |
670 | + return storage.Put(StateFile, bytes.NewBuffer(data), int64(len(data))) |
671 | +} |
672 | + |
673 | +// CreateStateFile creates an empty state file on the given storage, and |
674 | +// returns its URL. |
675 | +func CreateStateFile(storage storage.Storage) (string, error) { |
676 | + err := putState(storage, []byte{}) |
677 | + if err != nil { |
678 | + return "", fmt.Errorf("cannot create initial state file: %v", err) |
679 | + } |
680 | + return storage.URL(StateFile) |
681 | +} |
682 | + |
683 | +// SaveState writes the given state to the given storage. |
684 | +func SaveState(storage storage.StorageWriter, state *BootstrapState) error { |
685 | + data, err := goyaml.Marshal(state) |
686 | + if err != nil { |
687 | + return err |
688 | + } |
689 | + return putState(storage, data) |
690 | +} |
691 | + |
692 | +// LoadStateFromURL reads state from the given URL. |
693 | +func LoadStateFromURL(url string) (*BootstrapState, error) { |
694 | + resp, err := http.Get(url) |
695 | + if err != nil { |
696 | + return nil, err |
697 | + } |
698 | + return loadState(resp.Body) |
699 | +} |
700 | + |
701 | +// LoadState reads state from the given storage. |
702 | +func LoadState(stor storage.StorageReader) (*BootstrapState, error) { |
703 | + r, err := storage.Get(stor, StateFile) |
704 | + if err != nil { |
705 | + if coreerrors.IsNotFoundError(err) { |
706 | + return nil, environs.ErrNotBootstrapped |
707 | + } |
708 | + return nil, err |
709 | + } |
710 | + return loadState(r) |
711 | +} |
712 | + |
713 | +func loadState(r io.ReadCloser) (*BootstrapState, error) { |
714 | + defer r.Close() |
715 | + data, err := ioutil.ReadAll(r) |
716 | + if err != nil { |
717 | + return nil, fmt.Errorf("error reading %q: %v", StateFile, err) |
718 | + } |
719 | + var state BootstrapState |
720 | + err = goyaml.Unmarshal(data, &state) |
721 | + if err != nil { |
722 | + return nil, fmt.Errorf("error unmarshalling %q: %v", StateFile, err) |
723 | + } |
724 | + return &state, nil |
725 | +} |
726 | |
727 | === added file 'environs/bootstrap/state_test.go' |
728 | --- environs/bootstrap/state_test.go 1970-01-01 00:00:00 +0000 |
729 | +++ environs/bootstrap/state_test.go 2013-11-18 06:38:24 +0000 |
730 | @@ -0,0 +1,118 @@ |
731 | +// Copyright 2013 Canonical Ltd. |
732 | +// Licensed under the AGPLv3, see LICENCE file for details. |
733 | + |
734 | +package bootstrap_test |
735 | + |
736 | +import ( |
737 | + "bytes" |
738 | + "io/ioutil" |
739 | + |
740 | + gc "launchpad.net/gocheck" |
741 | + "launchpad.net/goyaml" |
742 | + |
743 | + "launchpad.net/juju-core/environs" |
744 | + "launchpad.net/juju-core/environs/bootstrap" |
745 | + "launchpad.net/juju-core/environs/storage" |
746 | + envtesting "launchpad.net/juju-core/environs/testing" |
747 | + "launchpad.net/juju-core/instance" |
748 | + "launchpad.net/juju-core/testing/testbase" |
749 | +) |
750 | + |
751 | +type StateSuite struct { |
752 | + testbase.LoggingSuite |
753 | +} |
754 | + |
755 | +var _ = gc.Suite(&StateSuite{}) |
756 | + |
757 | +func (suite *StateSuite) newStorage(c *gc.C) storage.Storage { |
758 | + closer, stor, _ := envtesting.CreateLocalTestStorage(c) |
759 | + suite.AddCleanup(func(*gc.C) { closer.Close() }) |
760 | + envtesting.UploadFakeTools(c, stor) |
761 | + return stor |
762 | +} |
763 | + |
764 | +func (suite *StateSuite) TestCreateStateFileWritesEmptyStateFile(c *gc.C) { |
765 | + stor := suite.newStorage(c) |
766 | + |
767 | + url, err := bootstrap.CreateStateFile(stor) |
768 | + c.Assert(err, gc.IsNil) |
769 | + |
770 | + reader, err := storage.Get(stor, bootstrap.StateFile) |
771 | + c.Assert(err, gc.IsNil) |
772 | + data, err := ioutil.ReadAll(reader) |
773 | + c.Assert(err, gc.IsNil) |
774 | + c.Check(string(data), gc.Equals, "") |
775 | + c.Assert(url, gc.NotNil) |
776 | + expectedURL, err := stor.URL(bootstrap.StateFile) |
777 | + c.Assert(err, gc.IsNil) |
778 | + c.Check(url, gc.Equals, expectedURL) |
779 | +} |
780 | + |
781 | +func (suite *StateSuite) TestSaveStateWritesStateFile(c *gc.C) { |
782 | + stor := suite.newStorage(c) |
783 | + arch := "amd64" |
784 | + state := bootstrap.BootstrapState{ |
785 | + StateInstances: []instance.Id{instance.Id("an-instance-id")}, |
786 | + Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}} |
787 | + marshaledState, err := goyaml.Marshal(state) |
788 | + c.Assert(err, gc.IsNil) |
789 | + |
790 | + err = bootstrap.SaveState(stor, &state) |
791 | + c.Assert(err, gc.IsNil) |
792 | + |
793 | + loadedState, err := storage.Get(stor, bootstrap.StateFile) |
794 | + c.Assert(err, gc.IsNil) |
795 | + content, err := ioutil.ReadAll(loadedState) |
796 | + c.Assert(err, gc.IsNil) |
797 | + c.Check(content, gc.DeepEquals, marshaledState) |
798 | +} |
799 | + |
800 | +func (suite *StateSuite) setUpSavedState(c *gc.C, stor storage.Storage) bootstrap.BootstrapState { |
801 | + arch := "amd64" |
802 | + state := bootstrap.BootstrapState{ |
803 | + StateInstances: []instance.Id{instance.Id("an-instance-id")}, |
804 | + Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}} |
805 | + content, err := goyaml.Marshal(state) |
806 | + c.Assert(err, gc.IsNil) |
807 | + err = stor.Put(bootstrap.StateFile, ioutil.NopCloser(bytes.NewReader(content)), int64(len(content))) |
808 | + c.Assert(err, gc.IsNil) |
809 | + return state |
810 | +} |
811 | + |
812 | +func (suite *StateSuite) TestLoadStateReadsStateFile(c *gc.C) { |
813 | + storage := suite.newStorage(c) |
814 | + state := suite.setUpSavedState(c, storage) |
815 | + storedState, err := bootstrap.LoadState(storage) |
816 | + c.Assert(err, gc.IsNil) |
817 | + c.Check(*storedState, gc.DeepEquals, state) |
818 | +} |
819 | + |
820 | +func (suite *StateSuite) TestLoadStateFromURLReadsStateFile(c *gc.C) { |
821 | + stor := suite.newStorage(c) |
822 | + state := suite.setUpSavedState(c, stor) |
823 | + url, err := stor.URL(bootstrap.StateFile) |
824 | + c.Assert(err, gc.IsNil) |
825 | + storedState, err := bootstrap.LoadStateFromURL(url) |
826 | + c.Assert(err, gc.IsNil) |
827 | + c.Check(*storedState, gc.DeepEquals, state) |
828 | +} |
829 | + |
830 | +func (suite *StateSuite) TestLoadStateMissingFile(c *gc.C) { |
831 | + stor := suite.newStorage(c) |
832 | + _, err := bootstrap.LoadState(stor) |
833 | + c.Check(err, gc.Equals, environs.ErrNotBootstrapped) |
834 | +} |
835 | + |
836 | +func (suite *StateSuite) TestLoadStateIntegratesWithSaveState(c *gc.C) { |
837 | + storage := suite.newStorage(c) |
838 | + arch := "amd64" |
839 | + state := bootstrap.BootstrapState{ |
840 | + StateInstances: []instance.Id{instance.Id("an-instance-id")}, |
841 | + Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}} |
842 | + err := bootstrap.SaveState(storage, &state) |
843 | + c.Assert(err, gc.IsNil) |
844 | + storedState, err := bootstrap.LoadState(storage) |
845 | + c.Assert(err, gc.IsNil) |
846 | + |
847 | + c.Check(*storedState, gc.DeepEquals, state) |
848 | +} |
849 | |
850 | === added file 'environs/bootstrap/synctools.go' |
851 | --- environs/bootstrap/synctools.go 1970-01-01 00:00:00 +0000 |
852 | +++ environs/bootstrap/synctools.go 2013-11-18 06:38:24 +0000 |
853 | @@ -0,0 +1,123 @@ |
854 | +// Copyright 2012, 2013 Canonical Ltd. |
855 | +// Licensed under the AGPLv3, see LICENCE file for details. |
856 | + |
857 | +package bootstrap |
858 | + |
859 | +import ( |
860 | + "fmt" |
861 | + |
862 | + "launchpad.net/juju-core/environs" |
863 | + "launchpad.net/juju-core/environs/config" |
864 | + "launchpad.net/juju-core/environs/sync" |
865 | + envtools "launchpad.net/juju-core/environs/tools" |
866 | + "launchpad.net/juju-core/errors" |
867 | + coretools "launchpad.net/juju-core/tools" |
868 | + "launchpad.net/juju-core/utils/set" |
869 | + "launchpad.net/juju-core/version" |
870 | +) |
871 | + |
872 | +const noToolsMessage = `Juju cannot bootstrap because no tools are available for your environment. |
873 | +An attempt was made to build and upload appropriate tools but this was unsuccessful. |
874 | +` |
875 | + |
876 | +const noToolsNoUploadMessage = `Juju cannot bootstrap because no tools are available for your environment. |
877 | +In addition, no tools could be located to upload. |
878 | +You may want to use the 'tools-metadata-url' configuration setting to specify the tools location. |
879 | +` |
880 | + |
881 | +// syncOrUpload first attempts to synchronize tools from |
882 | +// the default tools source to the environment's storage. |
883 | +// |
884 | +// If synchronization fails due to no matching tools, |
885 | +// a development version of juju is running, and no |
886 | +// agent-version has been specified, then attempt to |
887 | +// build and upload local tools. |
888 | +func syncOrUpload(env environs.Environ, bootstrapSeries string) error { |
889 | + sctx := &sync.SyncContext{ |
890 | + Target: env.Storage(), |
891 | + } |
892 | + err := sync.SyncTools(sctx) |
893 | + if err == coretools.ErrNoMatches || err == envtools.ErrNoTools { |
894 | + if _, hasAgentVersion := env.Config().AgentVersion(); !hasAgentVersion && version.Current.IsDev() { |
895 | + logger.Warningf("no tools found, so attempting to build and upload new tools") |
896 | + if err = uploadTools(env, bootstrapSeries); err != nil { |
897 | + logger.Errorf("%s", noToolsMessage) |
898 | + return err |
899 | + } |
900 | + } else { |
901 | + logger.Errorf("%s", noToolsNoUploadMessage) |
902 | + } |
903 | + } |
904 | + return err |
905 | +} |
906 | + |
907 | +func uploadTools(env environs.Environ, bootstrapSeries string) error { |
908 | + cfg := env.Config() |
909 | + uploadVersion := version.Current.Number |
910 | + uploadVersion.Build++ |
911 | + uploadSeries := set.NewStrings( |
912 | + bootstrapSeries, |
913 | + cfg.DefaultSeries(), |
914 | + config.DefaultSeries, |
915 | + ) |
916 | + tools, err := sync.Upload(env.Storage(), &uploadVersion, uploadSeries.Values()...) |
917 | + if err != nil { |
918 | + return err |
919 | + } |
920 | + cfg, err = cfg.Apply(map[string]interface{}{ |
921 | + "agent-version": tools.Version.Number.String(), |
922 | + }) |
923 | + if err == nil { |
924 | + err = env.SetConfig(cfg) |
925 | + } |
926 | + if err != nil { |
927 | + return fmt.Errorf("failed to update environment configuration: %v", err) |
928 | + } |
929 | + return nil |
930 | +} |
931 | + |
932 | +// EnsureToolsAvailability verifies the tools are available. If no tools are |
933 | +// found, it will automatically synchronize them. |
934 | +func EnsureToolsAvailability(env environs.Environ, series string, arch *string) (coretools.List, error) { |
935 | + cfg := env.Config() |
936 | + var vers *version.Number |
937 | + if agentVersion, ok := cfg.AgentVersion(); ok { |
938 | + vers = &agentVersion |
939 | + } |
940 | + |
941 | + logger.Debugf( |
942 | + "looking for bootstrap tools: series=%q, arch=%v, version=%v", |
943 | + series, arch, vers, |
944 | + ) |
945 | + params := envtools.BootstrapToolsParams{ |
946 | + Version: vers, |
947 | + Arch: arch, |
948 | + Series: series, |
949 | + // If vers.Build>0, the tools may have been uploaded in this session. |
950 | + // Allow retries, so we wait until the storage has caught up. |
951 | + AllowRetry: vers != nil && vers.Build > 0, |
952 | + } |
953 | + toolsList, err := envtools.FindBootstrapTools(env, params) |
954 | + if err == nil { |
955 | + return toolsList, nil |
956 | + } else if !errors.IsNotFoundError(err) { |
957 | + return nil, err |
958 | + } |
959 | + |
960 | + // No tools available, so synchronize. |
961 | + logger.Warningf("no tools available, attempting to retrieve from %v", sync.DefaultToolsLocation) |
962 | + if syncErr := syncOrUpload(env, series); syncErr != nil { |
963 | + // The target may have tools that don't match, so don't |
964 | + // return a misleading "no tools found" error. |
965 | + if syncErr != envtools.ErrNoTools { |
966 | + err = syncErr |
967 | + } |
968 | + return nil, fmt.Errorf("cannot find bootstrap tools: %v", err) |
969 | + } |
970 | + // TODO(axw) have syncOrUpload return the list of tools in the target, and use that. |
971 | + params.AllowRetry = true |
972 | + if toolsList, err = envtools.FindBootstrapTools(env, params); err != nil { |
973 | + return nil, fmt.Errorf("cannot find bootstrap tools: %v", err) |
974 | + } |
975 | + return toolsList, nil |
976 | +} |
977 | |
978 | === modified file 'environs/interface.go' |
979 | --- environs/interface.go 2013-10-25 22:24:35 +0000 |
980 | +++ environs/interface.go 2013-11-18 06:38:24 +0000 |
981 | @@ -10,7 +10,6 @@ |
982 | "launchpad.net/juju-core/instance" |
983 | "launchpad.net/juju-core/state" |
984 | "launchpad.net/juju-core/state/api" |
985 | - "launchpad.net/juju-core/tools" |
986 | ) |
987 | |
988 | // A EnvironProvider represents a computing and storage provider. |
989 | @@ -133,7 +132,11 @@ |
990 | // |
991 | // The supplied constraints are used to choose the initial instance |
992 | // specification, and will be stored in the new environment's state. |
993 | - Bootstrap(cons constraints.Value, possibleTools tools.List) error |
994 | + // |
995 | + // Bootstrap is responsible for selecting the appropriate tools, |
996 | + // and setting the agent-version configuration attribute prior to |
997 | + // bootstrapping the environment. |
998 | + Bootstrap(cons constraints.Value) error |
999 | |
1000 | // StateInfo returns information on the state initialized |
1001 | // by Bootstrap. |
1002 | |
1003 | === modified file 'environs/jujutest/livetests.go' |
1004 | --- environs/jujutest/livetests.go 2013-10-03 01:28:51 +0000 |
1005 | +++ environs/jujutest/livetests.go 2013-11-18 06:38:24 +0000 |
1006 | @@ -27,6 +27,7 @@ |
1007 | "launchpad.net/juju-core/instance" |
1008 | "launchpad.net/juju-core/juju" |
1009 | "launchpad.net/juju-core/juju/testing" |
1010 | + "launchpad.net/juju-core/provider/common" |
1011 | "launchpad.net/juju-core/provider/dummy" |
1012 | "launchpad.net/juju-core/state" |
1013 | "launchpad.net/juju-core/state/api" |
1014 | @@ -137,7 +138,7 @@ |
1015 | c.Assert(err, gc.IsNil) |
1016 | } |
1017 | envtesting.UploadFakeTools(c, t.Env.Storage()) |
1018 | - err := bootstrap.EnsureNotBootstrapped(t.Env) |
1019 | + err := common.EnsureNotBootstrapped(t.Env) |
1020 | c.Assert(err, gc.IsNil) |
1021 | err = bootstrap.Bootstrap(t.Env, cons) |
1022 | c.Assert(err, gc.IsNil) |
1023 | @@ -391,7 +392,7 @@ |
1024 | // already up, this has been moved into the bootstrap command. |
1025 | t.BootstrapOnce(c) |
1026 | |
1027 | - err := bootstrap.EnsureNotBootstrapped(t.Env) |
1028 | + err := common.EnsureNotBootstrapped(t.Env) |
1029 | c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped") |
1030 | |
1031 | c.Logf("destroy env") |
1032 | |
1033 | === modified file 'environs/jujutest/tests.go' |
1034 | --- environs/jujutest/tests.go 2013-10-03 01:28:51 +0000 |
1035 | +++ environs/jujutest/tests.go 2013-11-18 06:38:24 +0000 |
1036 | @@ -21,6 +21,7 @@ |
1037 | "launchpad.net/juju-core/errors" |
1038 | "launchpad.net/juju-core/instance" |
1039 | "launchpad.net/juju-core/juju/testing" |
1040 | + "launchpad.net/juju-core/provider/common" |
1041 | coretesting "launchpad.net/juju-core/testing" |
1042 | jc "launchpad.net/juju-core/testing/checkers" |
1043 | "launchpad.net/juju-core/testing/testbase" |
1044 | @@ -131,7 +132,7 @@ |
1045 | func (t *Tests) TestBootstrap(c *gc.C) { |
1046 | e := t.Prepare(c) |
1047 | envtesting.UploadFakeTools(c, e.Storage()) |
1048 | - err := bootstrap.EnsureNotBootstrapped(e) |
1049 | + err := common.EnsureNotBootstrapped(e) |
1050 | c.Assert(err, gc.IsNil) |
1051 | err = bootstrap.Bootstrap(e, constraints.Value{}) |
1052 | c.Assert(err, gc.IsNil) |
1053 | @@ -140,12 +141,12 @@ |
1054 | c.Check(info.Addrs, gc.Not(gc.HasLen), 0) |
1055 | c.Check(apiInfo.Addrs, gc.Not(gc.HasLen), 0) |
1056 | |
1057 | - err = bootstrap.EnsureNotBootstrapped(e) |
1058 | + err = common.EnsureNotBootstrapped(e) |
1059 | c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped") |
1060 | |
1061 | e2 := t.Open(c) |
1062 | envtesting.UploadFakeTools(c, e2.Storage()) |
1063 | - err = bootstrap.EnsureNotBootstrapped(e2) |
1064 | + err = common.EnsureNotBootstrapped(e2) |
1065 | c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped") |
1066 | |
1067 | info2, apiInfo2, err := e2.StateInfo() |
1068 | @@ -159,12 +160,12 @@ |
1069 | e3 := t.Prepare(c) |
1070 | envtesting.UploadFakeTools(c, e3.Storage()) |
1071 | |
1072 | - err = bootstrap.EnsureNotBootstrapped(e3) |
1073 | + err = common.EnsureNotBootstrapped(e3) |
1074 | c.Assert(err, gc.IsNil) |
1075 | err = bootstrap.Bootstrap(e3, constraints.Value{}) |
1076 | c.Assert(err, gc.IsNil) |
1077 | |
1078 | - err = bootstrap.EnsureNotBootstrapped(e3) |
1079 | + err = common.EnsureNotBootstrapped(e3) |
1080 | c.Assert(err, gc.ErrorMatches, "environment is already bootstrapped") |
1081 | } |
1082 | |
1083 | |
1084 | === modified file 'environs/manual/bootstrap.go' |
1085 | --- environs/manual/bootstrap.go 2013-11-14 08:27:15 +0000 |
1086 | +++ environs/manual/bootstrap.go 2013-11-18 06:38:24 +0000 |
1087 | @@ -9,9 +9,9 @@ |
1088 | |
1089 | "launchpad.net/juju-core/constraints" |
1090 | "launchpad.net/juju-core/environs" |
1091 | + "launchpad.net/juju-core/environs/bootstrap" |
1092 | envtools "launchpad.net/juju-core/environs/tools" |
1093 | "launchpad.net/juju-core/instance" |
1094 | - "launchpad.net/juju-core/provider/common" |
1095 | "launchpad.net/juju-core/tools" |
1096 | "launchpad.net/juju-core/worker/localstorage" |
1097 | ) |
1098 | @@ -30,6 +30,13 @@ |
1099 | DataDir string |
1100 | Environ LocalStorageEnviron |
1101 | PossibleTools tools.List |
1102 | + |
1103 | + // If series and hardware characteristics |
1104 | + // are known ahead of time, they can be |
1105 | + // set here and Bootstrap will not attempt |
1106 | + // to detect them again. |
1107 | + Series string |
1108 | + HardwareCharacteristics *instance.HardwareCharacteristics |
1109 | } |
1110 | |
1111 | func errMachineIdInvalid(machineId string) error { |
1112 | @@ -58,9 +65,16 @@ |
1113 | return ErrProvisioned |
1114 | } |
1115 | |
1116 | - hc, series, err := detectSeriesAndHardwareCharacteristics(args.Host) |
1117 | - if err != nil { |
1118 | - return fmt.Errorf("error detecting hardware characteristics: %v", err) |
1119 | + var series string |
1120 | + var hc instance.HardwareCharacteristics |
1121 | + if args.Series != "" && args.HardwareCharacteristics != nil { |
1122 | + series = args.Series |
1123 | + hc = *args.HardwareCharacteristics |
1124 | + } else { |
1125 | + hc, series, err = DetectSeriesAndHardwareCharacteristics(args.Host) |
1126 | + if err != nil { |
1127 | + return fmt.Errorf("error detecting hardware characteristics: %v", err) |
1128 | + } |
1129 | } |
1130 | |
1131 | // Filter tools based on detected series/arch. |
1132 | @@ -76,9 +90,9 @@ |
1133 | // Store the state file. If provisioning fails, we'll remove the file. |
1134 | logger.Infof("Saving bootstrap state file to bootstrap storage") |
1135 | bootstrapStorage := args.Environ.Storage() |
1136 | - err = common.SaveState( |
1137 | + err = bootstrap.SaveState( |
1138 | bootstrapStorage, |
1139 | - &common.BootstrapState{ |
1140 | + &bootstrap.BootstrapState{ |
1141 | StateInstances: []instance.Id{BootstrapInstanceId}, |
1142 | Characteristics: []instance.HardwareCharacteristics{hc}, |
1143 | }, |
1144 | @@ -89,7 +103,7 @@ |
1145 | defer func() { |
1146 | if err != nil { |
1147 | logger.Errorf("bootstrapping failed, removing state file: %v", err) |
1148 | - bootstrapStorage.Remove(common.StateFile) |
1149 | + bootstrapStorage.Remove(bootstrap.StateFile) |
1150 | } |
1151 | }() |
1152 | |
1153 | @@ -107,7 +121,7 @@ |
1154 | } |
1155 | |
1156 | // Finally, provision the machine agent. |
1157 | - stateFileURL := fmt.Sprintf("file://%s/%s", storageDir, common.StateFile) |
1158 | + stateFileURL := fmt.Sprintf("file://%s/%s", storageDir, bootstrap.StateFile) |
1159 | mcfg := environs.NewBootstrapMachineConfig(stateFileURL) |
1160 | if args.DataDir != "" { |
1161 | mcfg.DataDir = args.DataDir |
1162 | |
1163 | === modified file 'environs/manual/bootstrap_test.go' |
1164 | --- environs/manual/bootstrap_test.go 2013-10-02 10:39:12 +0000 |
1165 | +++ environs/manual/bootstrap_test.go 2013-11-18 06:38:24 +0000 |
1166 | @@ -9,12 +9,12 @@ |
1167 | gc "launchpad.net/gocheck" |
1168 | |
1169 | "launchpad.net/juju-core/environs" |
1170 | + "launchpad.net/juju-core/environs/bootstrap" |
1171 | "launchpad.net/juju-core/environs/filestorage" |
1172 | "launchpad.net/juju-core/environs/storage" |
1173 | "launchpad.net/juju-core/environs/tools" |
1174 | "launchpad.net/juju-core/instance" |
1175 | "launchpad.net/juju-core/juju/testing" |
1176 | - "launchpad.net/juju-core/provider/common" |
1177 | ) |
1178 | |
1179 | type bootstrapSuite struct { |
1180 | @@ -81,11 +81,11 @@ |
1181 | args := s.getArgs(c) |
1182 | args.Host = "ubuntu@" + args.Host |
1183 | |
1184 | - defer fakeSSH{series: s.Conn.Environ.Config().DefaultSeries()}.install(c).Restore() |
1185 | + defer fakeSSH{Series: s.Conn.Environ.Config().DefaultSeries()}.install(c).Restore() |
1186 | err := Bootstrap(args) |
1187 | c.Assert(err, gc.IsNil) |
1188 | |
1189 | - bootstrapState, err := common.LoadState(s.env.Storage()) |
1190 | + bootstrapState, err := bootstrap.LoadState(s.env.Storage()) |
1191 | c.Assert(err, gc.IsNil) |
1192 | c.Assert( |
1193 | bootstrapState.StateInstances, |
1194 | @@ -96,7 +96,7 @@ |
1195 | // Do it all again; this should work, despite the fact that |
1196 | // there's a bootstrap state file. Existence for that is |
1197 | // checked in general bootstrap code (environs/bootstrap). |
1198 | - defer fakeSSH{series: s.Conn.Environ.Config().DefaultSeries()}.install(c).Restore() |
1199 | + defer fakeSSH{Series: s.Conn.Environ.Config().DefaultSeries()}.install(c).Restore() |
1200 | err = Bootstrap(args) |
1201 | c.Assert(err, gc.IsNil) |
1202 | |
1203 | @@ -106,17 +106,56 @@ |
1204 | c.Assert(err, gc.Equals, ErrProvisioned) |
1205 | } |
1206 | |
1207 | +func (s *bootstrapSuite) TestBootstrapSkipDetection(c *gc.C) { |
1208 | + args := s.getArgs(c) |
1209 | + hc, err := instance.ParseHardware("arch=amd64") |
1210 | + c.Assert(err, gc.IsNil) |
1211 | + |
1212 | + type test struct { |
1213 | + series string |
1214 | + hc *instance.HardwareCharacteristics |
1215 | + } |
1216 | + tests := []test{{ |
1217 | + series: "", |
1218 | + hc: nil, |
1219 | + }, { |
1220 | + series: "precise", |
1221 | + hc: nil, |
1222 | + }, { |
1223 | + series: "", |
1224 | + hc: &hc, |
1225 | + }, { |
1226 | + series: "precise", |
1227 | + hc: &hc, |
1228 | + }} |
1229 | + |
1230 | + for i, test := range tests { |
1231 | + c.Logf("test %d: %+v", i, test) |
1232 | + args.Series = test.series |
1233 | + args.HardwareCharacteristics = test.hc |
1234 | + var ssh fakeSSH |
1235 | + if args.Series != "" && args.HardwareCharacteristics != nil { |
1236 | + // If neither series nor hardware-characteristics |
1237 | + // is missing, detection is skipped. |
1238 | + ssh.SkipDetection = true |
1239 | + } |
1240 | + defer ssh.install(c).Restore() |
1241 | + err = Bootstrap(args) |
1242 | + c.Assert(err, gc.IsNil) |
1243 | + } |
1244 | +} |
1245 | + |
1246 | func (s *bootstrapSuite) TestBootstrapScriptFailure(c *gc.C) { |
1247 | args := s.getArgs(c) |
1248 | args.Host = "ubuntu@" + args.Host |
1249 | series := s.Conn.Environ.Config().DefaultSeries() |
1250 | - defer fakeSSH{series: series, provisionAgentExitCode: 1}.install(c).Restore() |
1251 | + defer fakeSSH{Series: series, ProvisionAgentExitCode: 1}.install(c).Restore() |
1252 | err := Bootstrap(args) |
1253 | c.Assert(err, gc.NotNil) |
1254 | |
1255 | // Since the script failed, the state file should have been |
1256 | // removed from storage. |
1257 | - _, err = common.LoadState(s.env.Storage()) |
1258 | + _, err = bootstrap.LoadState(s.env.Storage()) |
1259 | c.Check(err, gc.Equals, environs.ErrNotBootstrapped) |
1260 | } |
1261 | |
1262 | @@ -143,11 +182,11 @@ |
1263 | args := s.getArgs(c) |
1264 | args.PossibleTools = nil |
1265 | series := s.Conn.Environ.Config().DefaultSeries() |
1266 | - defer fakeSSH{series: series, skipProvisionAgent: true}.install(c).Restore() |
1267 | + defer fakeSSH{Series: series, SkipProvisionAgent: true}.install(c).Restore() |
1268 | c.Assert(Bootstrap(args), gc.ErrorMatches, "no matching tools available") |
1269 | |
1270 | // Non-empty list, but none that match the series/arch. |
1271 | - defer fakeSSH{series: "edgy", skipProvisionAgent: true}.install(c).Restore() |
1272 | + defer fakeSSH{Series: "edgy", SkipProvisionAgent: true}.install(c).Restore() |
1273 | args = s.getArgs(c) |
1274 | c.Assert(Bootstrap(args), gc.ErrorMatches, "no matching tools available") |
1275 | } |
1276 | |
1277 | === modified file 'environs/manual/detection.go' |
1278 | --- environs/manual/detection.go 2013-09-16 03:24:54 +0000 |
1279 | +++ environs/manual/detection.go 2013-11-18 06:38:24 +0000 |
1280 | @@ -45,10 +45,12 @@ |
1281 | return provisioned, nil |
1282 | } |
1283 | |
1284 | -// detectSeriesHardwareCharacteristics detects the OS series |
1285 | -// and hardware characteristics of the remote machine by |
1286 | -// connecting to the machine and executing a bash script. |
1287 | -func detectSeriesAndHardwareCharacteristics(sshHost string) (hc instance.HardwareCharacteristics, series string, err error) { |
1288 | +// DetectSeriesAndHardwareCharacteristics detects the OS |
1289 | +// series and hardware characteristics of the remote machine |
1290 | +// by connecting to the machine and executing a bash script. |
1291 | +// |
1292 | +// The sshHost argument must be a hostname of the form [user@]host. |
1293 | +func DetectSeriesAndHardwareCharacteristics(sshHost string) (hc instance.HardwareCharacteristics, series string, err error) { |
1294 | logger.Infof("Detecting series and characteristics on %s", sshHost) |
1295 | cmd := sshCommand(sshHost, "bash") |
1296 | cmd.Stdin = bytes.NewBufferString(detectionScript) |
1297 | |
1298 | === modified file 'environs/manual/detection_test.go' |
1299 | --- environs/manual/detection_test.go 2013-09-20 04:41:12 +0000 |
1300 | +++ environs/manual/detection_test.go 2013-11-18 06:38:24 +0000 |
1301 | @@ -26,14 +26,14 @@ |
1302 | "processor: 0", |
1303 | }, "\n") |
1304 | defer installFakeSSH(c, detectionScript, response, 0)() |
1305 | - _, series, err := detectSeriesAndHardwareCharacteristics("whatever") |
1306 | + _, series, err := DetectSeriesAndHardwareCharacteristics("whatever") |
1307 | c.Assert(err, gc.IsNil) |
1308 | c.Assert(series, gc.Equals, "edgy") |
1309 | } |
1310 | |
1311 | func (s *detectionSuite) TestDetectionError(c *gc.C) { |
1312 | defer installFakeSSH(c, detectionScript, "oh noes", 33)() |
1313 | - _, _, err := detectSeriesAndHardwareCharacteristics("whatever") |
1314 | + _, _, err := DetectSeriesAndHardwareCharacteristics("whatever") |
1315 | c.Assert(err, gc.ErrorMatches, "exit status 33 \\(oh noes\\)") |
1316 | } |
1317 | |
1318 | @@ -93,7 +93,7 @@ |
1319 | c.Logf("test %d: %s", i, test.summary) |
1320 | scriptResponse := strings.Join(test.scriptResponse, "\n") |
1321 | defer installFakeSSH(c, detectionScript, scriptResponse, 0)() |
1322 | - hc, _, err := detectSeriesAndHardwareCharacteristics("hostname") |
1323 | + hc, _, err := DetectSeriesAndHardwareCharacteristics("hostname") |
1324 | c.Assert(err, gc.IsNil) |
1325 | c.Assert(hc.String(), gc.Equals, test.expectedHc) |
1326 | } |
1327 | |
1328 | === renamed file 'environs/manual/fakessh_test.go' => 'environs/manual/fakessh.go' |
1329 | --- environs/manual/fakessh_test.go 2013-10-21 21:49:04 +0000 |
1330 | +++ environs/manual/fakessh.go 2013-11-18 06:38:24 +0000 |
1331 | @@ -67,29 +67,13 @@ |
1332 | return testbase.PatchEnvironment("PATH", fakebin+":"+os.Getenv("PATH")) |
1333 | } |
1334 | |
1335 | -// fakeSSH wraps the invocation of installFakeSSH based on the parameters. |
1336 | -type fakeSSH struct { |
1337 | - series string |
1338 | - arch string |
1339 | - |
1340 | - // exit code for the machine agent provisioning script. |
1341 | - provisionAgentExitCode int |
1342 | - |
1343 | - // there are conditions other than error in the above |
1344 | - // that might cause provisioning to not go ahead, such |
1345 | - // as tools being missing. |
1346 | - skipProvisionAgent bool |
1347 | -} |
1348 | - |
1349 | -// install installs fake SSH commands, which will respond to |
1350 | -// manual provisioning/bootstrapping commands with the specified |
1351 | -// output and exit codes. |
1352 | -func (r fakeSSH) install(c *gc.C) testbase.Restorer { |
1353 | - series := r.series |
1354 | +// installDetectionFakeSSH installs a fake SSH command, which will respond |
1355 | +// to the series/hardware detection script with the specified |
1356 | +// series/arch. |
1357 | +func installDetectionFakeSSH(c *gc.C, series, arch string) testbase.Restorer { |
1358 | if series == "" { |
1359 | series = "precise" |
1360 | } |
1361 | - arch := r.arch |
1362 | if arch == "" { |
1363 | arch = "amd64" |
1364 | } |
1365 | @@ -99,14 +83,41 @@ |
1366 | "MemTotal: 4096 kB", |
1367 | "processor: 0", |
1368 | }, "\n") |
1369 | + return installFakeSSH(c, detectionScript, detectionoutput, 0) |
1370 | +} |
1371 | + |
1372 | +// FakeSSH wraps the invocation of installFakeSSH based on the parameters. |
1373 | +type fakeSSH struct { |
1374 | + Series string |
1375 | + Arch string |
1376 | + |
1377 | + // exit code for the machine agent provisioning script. |
1378 | + ProvisionAgentExitCode int |
1379 | + |
1380 | + // there are conditions other than error in the above |
1381 | + // that might cause provisioning to not go ahead, such |
1382 | + // as tools being missing. |
1383 | + SkipProvisionAgent bool |
1384 | + |
1385 | + // detection will be skipped if the series/hardware were |
1386 | + // detected ahead of time. |
1387 | + SkipDetection bool |
1388 | +} |
1389 | + |
1390 | +// install installs fake SSH commands, which will respond to |
1391 | +// manual provisioning/bootstrapping commands with the specified |
1392 | +// output and exit codes. |
1393 | +func (r fakeSSH) install(c *gc.C) testbase.Restorer { |
1394 | var restore testbase.Restorer |
1395 | add := func(input string, output interface{}, rc int) { |
1396 | restore = restore.Add(installFakeSSH(c, input, output, rc)) |
1397 | } |
1398 | - if !r.skipProvisionAgent { |
1399 | - add("", nil, r.provisionAgentExitCode) |
1400 | - } |
1401 | - add(detectionScript, detectionoutput, 0) |
1402 | + if !r.SkipProvisionAgent { |
1403 | + add("", nil, r.ProvisionAgentExitCode) |
1404 | + } |
1405 | + if !r.SkipDetection { |
1406 | + restore.Add(installDetectionFakeSSH(c, r.Series, r.Arch)) |
1407 | + } |
1408 | add("", nil, 0) // checkProvisioned |
1409 | return restore |
1410 | } |
1411 | |
1412 | === modified file 'environs/manual/provisioner.go' |
1413 | --- environs/manual/provisioner.go 2013-11-14 08:27:15 +0000 |
1414 | +++ environs/manual/provisioner.go 2013-11-18 06:38:24 +0000 |
1415 | @@ -137,7 +137,7 @@ |
1416 | return "", "", "", ErrProvisioned |
1417 | } |
1418 | |
1419 | - hc, series, err := detectSeriesAndHardwareCharacteristics(host) |
1420 | + hc, series, err := DetectSeriesAndHardwareCharacteristics(host) |
1421 | if err != nil { |
1422 | err = fmt.Errorf("error detecting hardware characteristics: %v", err) |
1423 | return "", "", "", err |
1424 | |
1425 | === modified file 'environs/manual/provisioner_test.go' |
1426 | --- environs/manual/provisioner_test.go 2013-11-08 02:52:25 +0000 |
1427 | +++ environs/manual/provisioner_test.go 2013-11-18 06:38:24 +0000 |
1428 | @@ -42,7 +42,7 @@ |
1429 | |
1430 | envtesting.RemoveTools(c, s.Conn.Environ.Storage()) |
1431 | defer fakeSSH{ |
1432 | - series: series, arch: arch, skipProvisionAgent: true, |
1433 | + Series: series, Arch: arch, SkipProvisionAgent: true, |
1434 | }.install(c).Restore() |
1435 | // Attempt to provision a machine with no tools available, expect it to fail. |
1436 | machineId, err := ProvisionMachine(args) |
1437 | @@ -58,9 +58,9 @@ |
1438 | for i, errorCode := range []int{255, 0} { |
1439 | c.Logf("test %d: code %d", i, errorCode) |
1440 | defer fakeSSH{ |
1441 | - series: series, |
1442 | - arch: arch, |
1443 | - provisionAgentExitCode: errorCode, |
1444 | + Series: series, |
1445 | + Arch: arch, |
1446 | + ProvisionAgentExitCode: errorCode, |
1447 | }.install(c).Restore() |
1448 | machineId, err = ProvisionMachine(args) |
1449 | if errorCode != 0 { |
1450 | |
1451 | === modified file 'environs/simplestreams/simplestreams.go' |
1452 | --- environs/simplestreams/simplestreams.go 2013-11-05 04:32:17 +0000 |
1453 | +++ environs/simplestreams/simplestreams.go 2013-11-18 06:38:24 +0000 |
1454 | @@ -970,6 +970,7 @@ |
1455 | if err != nil { |
1456 | return nil, err |
1457 | } |
1458 | + logger.Debugf("metadata: %v", metadata) |
1459 | matches, err := GetLatestMetadata(metadata, cons, indexRef.Source, indexRef.valueParams.FilterFunc) |
1460 | if err != nil { |
1461 | return nil, err |
1462 | |
1463 | === modified file 'environs/sync/sync.go' |
1464 | --- environs/sync/sync.go 2013-10-30 02:28:40 +0000 |
1465 | +++ environs/sync/sync.go 2013-11-18 06:38:24 +0000 |
1466 | @@ -12,6 +12,7 @@ |
1467 | "net/http" |
1468 | "os" |
1469 | "path/filepath" |
1470 | + "strings" |
1471 | |
1472 | "launchpad.net/loggo" |
1473 | |
1474 | @@ -138,10 +139,14 @@ |
1475 | |
1476 | // selectSourceStorage returns a storage reader based on the source setting. |
1477 | func selectSourceStorage(syncContext *SyncContext) (storage.StorageReader, error) { |
1478 | - if syncContext.Source == "" { |
1479 | - return httpstorage.NewHTTPStorageReader(DefaultToolsLocation), nil |
1480 | - } |
1481 | - return filestorage.NewFileStorageReader(syncContext.Source) |
1482 | + source := syncContext.Source |
1483 | + if source == "" { |
1484 | + source = DefaultToolsLocation |
1485 | + } |
1486 | + if strings.HasPrefix(source, "http") { |
1487 | + return httpstorage.NewHTTPStorageReader(source), nil |
1488 | + } |
1489 | + return filestorage.NewFileStorageReader(source) |
1490 | } |
1491 | |
1492 | // copyTools copies a set of tools from the source to the target. |
1493 | @@ -199,6 +204,11 @@ |
1494 | return err |
1495 | } |
1496 | |
1497 | +// UploadFunc is the type of Upload, which may be |
1498 | +// reassigned to control the behaviour of tools |
1499 | +// uploading. |
1500 | +type UploadFunc func(stor storage.Storage, forceVersion *version.Number, series ...string) (*coretools.Tools, error) |
1501 | + |
1502 | // Upload builds whatever version of launchpad.net/juju-core is in $GOPATH, |
1503 | // uploads it to the given storage, and returns a Tools instance describing |
1504 | // them. If forceVersion is not nil, the uploaded tools bundle will report |
1505 | @@ -206,7 +216,9 @@ |
1506 | // of the built tools will be uploaded for use by machines of those series. |
1507 | // Juju tools built for one series do not necessarily run on another, but this |
1508 | // func exists only for development use cases. |
1509 | -func Upload(stor storage.Storage, forceVersion *version.Number, fakeSeries ...string) (*coretools.Tools, error) { |
1510 | +var Upload UploadFunc = upload |
1511 | + |
1512 | +func upload(stor storage.Storage, forceVersion *version.Number, fakeSeries ...string) (*coretools.Tools, error) { |
1513 | // TODO(rog) find binaries from $PATH when not using a development |
1514 | // version of juju within a $GOPATH. |
1515 | |
1516 | |
1517 | === modified file 'environs/testing/tools.go' |
1518 | --- environs/testing/tools.go 2013-10-29 22:58:43 +0000 |
1519 | +++ environs/testing/tools.go 2013-11-18 06:38:24 +0000 |
1520 | @@ -16,6 +16,7 @@ |
1521 | "launchpad.net/juju-core/environs/config" |
1522 | "launchpad.net/juju-core/environs/simplestreams" |
1523 | "launchpad.net/juju-core/environs/storage" |
1524 | + "launchpad.net/juju-core/environs/sync" |
1525 | envtools "launchpad.net/juju-core/environs/tools" |
1526 | "launchpad.net/juju-core/log" |
1527 | "launchpad.net/juju-core/state" |
1528 | @@ -29,16 +30,20 @@ |
1529 | // ToolsFixture is used as a fixture to stub out the default tools URL so we |
1530 | // don't hit the real internet during tests. |
1531 | type ToolsFixture struct { |
1532 | - origDefaultURL string |
1533 | - DefaultBaseURL string |
1534 | + origDefaultURL string |
1535 | + origDefaultSyncLocation string |
1536 | + DefaultBaseURL string |
1537 | } |
1538 | |
1539 | func (s *ToolsFixture) SetUpTest(c *gc.C) { |
1540 | s.origDefaultURL = envtools.DefaultBaseURL |
1541 | + s.origDefaultSyncLocation = sync.DefaultToolsLocation |
1542 | envtools.DefaultBaseURL = s.DefaultBaseURL |
1543 | + sync.DefaultToolsLocation = c.MkDir() // stop sync from going to s3 |
1544 | } |
1545 | |
1546 | func (s *ToolsFixture) TearDownTest(c *gc.C) { |
1547 | + sync.DefaultToolsLocation = sync.DefaultToolsLocation |
1548 | envtools.DefaultBaseURL = s.origDefaultURL |
1549 | } |
1550 | |
1551 | |
1552 | === modified file 'environs/tools/storage.go' |
1553 | --- environs/tools/storage.go 2013-09-27 02:31:45 +0000 |
1554 | +++ environs/tools/storage.go 2013-11-18 06:38:24 +0000 |
1555 | @@ -47,6 +47,7 @@ |
1556 | var t coretools.Tools |
1557 | vers := name[len(toolPrefix) : len(name)-len(toolSuffix)] |
1558 | if t.Version, err = version.ParseBinary(vers); err != nil { |
1559 | + logger.Debugf("failed to parse version %q: %v", vers, err) |
1560 | continue |
1561 | } |
1562 | foundAnyTools = true |
1563 | |
1564 | === modified file 'environs/tools/tools.go' |
1565 | --- environs/tools/tools.go 2013-10-08 06:17:07 +0000 |
1566 | +++ environs/tools/tools.go 2013-11-18 06:38:24 +0000 |
1567 | @@ -153,6 +153,7 @@ |
1568 | type BootstrapToolsParams struct { |
1569 | Version *version.Number |
1570 | Arch *string |
1571 | + Series string |
1572 | AllowRetry bool |
1573 | } |
1574 | |
1575 | @@ -164,7 +165,7 @@ |
1576 | cfg := cloudInst.Config() |
1577 | cliVersion := version.Current.Number |
1578 | filter := coretools.Filter{ |
1579 | - Series: cfg.DefaultSeries(), |
1580 | + Series: params.Series, |
1581 | Arch: stringOrEmpty(params.Arch), |
1582 | } |
1583 | if params.Version != nil { |
1584 | |
1585 | === modified file 'environs/tools/tools_test.go' |
1586 | --- environs/tools/tools_test.go 2013-11-05 04:32:17 +0000 |
1587 | +++ environs/tools/tools_test.go 2013-11-18 06:38:24 +0000 |
1588 | @@ -291,8 +291,7 @@ |
1589 | for i, test := range envtesting.BootstrapToolsTests { |
1590 | c.Logf("\ntest %d: %s", i, test.Info) |
1591 | attrs := map[string]interface{}{ |
1592 | - "development": test.Development, |
1593 | - "default-series": test.DefaultSeries, |
1594 | + "development": test.Development, |
1595 | } |
1596 | var agentVersion *version.Number |
1597 | if test.AgentVersion != version.Zero { |
1598 | @@ -305,6 +304,7 @@ |
1599 | |
1600 | params := envtools.BootstrapToolsParams{ |
1601 | Version: agentVersion, |
1602 | + Series: test.DefaultSeries, |
1603 | Arch: &test.Arch, |
1604 | } |
1605 | actual, err := envtools.FindBootstrapTools(s.env, params) |
1606 | |
1607 | === modified file 'provider/azure/environ.go' |
1608 | --- provider/azure/environ.go 2013-11-01 06:20:19 +0000 |
1609 | +++ provider/azure/environ.go 2013-11-18 06:38:24 +0000 |
1610 | @@ -231,7 +231,7 @@ |
1611 | } |
1612 | |
1613 | // Bootstrap is specified in the Environ interface. |
1614 | -func (env *azureEnviron) Bootstrap(cons constraints.Value, possibleTools tools.List) (err error) { |
1615 | +func (env *azureEnviron) Bootstrap(cons constraints.Value) (err error) { |
1616 | // The creation of the affinity group and the virtual network is specific to the Azure provider. |
1617 | err = env.createAffinityGroup() |
1618 | if err != nil { |
1619 | @@ -253,7 +253,7 @@ |
1620 | env.deleteVirtualNetwork() |
1621 | } |
1622 | }() |
1623 | - err = common.Bootstrap(env, cons, possibleTools) |
1624 | + err = common.Bootstrap(env, cons) |
1625 | return err |
1626 | } |
1627 | |
1628 | |
1629 | === modified file 'provider/azure/environ_test.go' |
1630 | --- provider/azure/environ_test.go 2013-11-01 06:20:19 +0000 |
1631 | +++ provider/azure/environ_test.go 2013-11-18 06:38:24 +0000 |
1632 | @@ -21,6 +21,7 @@ |
1633 | |
1634 | "launchpad.net/juju-core/constraints" |
1635 | "launchpad.net/juju-core/environs" |
1636 | + "launchpad.net/juju-core/environs/bootstrap" |
1637 | "launchpad.net/juju-core/environs/config" |
1638 | "launchpad.net/juju-core/environs/imagemetadata" |
1639 | "launchpad.net/juju-core/environs/simplestreams" |
1640 | @@ -28,7 +29,6 @@ |
1641 | envtesting "launchpad.net/juju-core/environs/testing" |
1642 | "launchpad.net/juju-core/environs/tools" |
1643 | "launchpad.net/juju-core/instance" |
1644 | - "launchpad.net/juju-core/provider/common" |
1645 | "launchpad.net/juju-core/testing" |
1646 | jc "launchpad.net/juju-core/testing/checkers" |
1647 | ) |
1648 | @@ -456,9 +456,9 @@ |
1649 | }}) |
1650 | env := makeEnviron(c) |
1651 | s.setDummyStorage(c, env) |
1652 | - err := common.SaveState( |
1653 | + err := bootstrap.SaveState( |
1654 | env.Storage(), |
1655 | - &common.BootstrapState{StateInstances: []instance.Id{instance.Id(instanceID)}}) |
1656 | + &bootstrap.BootstrapState{StateInstances: []instance.Id{instance.Id(instanceID)}}) |
1657 | c.Assert(err, gc.IsNil) |
1658 | |
1659 | stateInfo, apiInfo, err := env.StateInfo() |
1660 | @@ -787,9 +787,9 @@ |
1661 | env := makeEnviron(c) |
1662 | s.setDummyStorage(c, env) |
1663 | // Populate storage. |
1664 | - err := common.SaveState( |
1665 | + err := bootstrap.SaveState( |
1666 | env.Storage(), |
1667 | - &common.BootstrapState{StateInstances: []instance.Id{instance.Id("test-id")}}) |
1668 | + &bootstrap.BootstrapState{StateInstances: []instance.Id{instance.Id("test-id")}}) |
1669 | c.Assert(err, gc.IsNil) |
1670 | responses := []gwacl.DispatcherResponse{ |
1671 | gwacl.NewDispatcherResponse(nil, http.StatusBadRequest, nil), |
1672 | @@ -808,9 +808,9 @@ |
1673 | env := makeEnviron(c) |
1674 | s.setDummyStorage(c, env) |
1675 | // Populate storage. |
1676 | - err := common.SaveState( |
1677 | + err := bootstrap.SaveState( |
1678 | env.Storage(), |
1679 | - &common.BootstrapState{StateInstances: []instance.Id{instance.Id("test-id")}}) |
1680 | + &bootstrap.BootstrapState{StateInstances: []instance.Id{instance.Id("test-id")}}) |
1681 | c.Assert(err, gc.IsNil) |
1682 | services := []gwacl.HostedServiceDescriptor{} |
1683 | responses := getAzureServiceListResponse(c, services) |
1684 | |
1685 | === modified file 'provider/common/bootstrap.go' |
1686 | --- provider/common/bootstrap.go 2013-10-02 19:30:07 +0000 |
1687 | +++ provider/common/bootstrap.go 2013-11-18 06:38:24 +0000 |
1688 | @@ -10,6 +10,7 @@ |
1689 | |
1690 | "launchpad.net/juju-core/constraints" |
1691 | "launchpad.net/juju-core/environs" |
1692 | + "launchpad.net/juju-core/environs/bootstrap" |
1693 | "launchpad.net/juju-core/instance" |
1694 | coretools "launchpad.net/juju-core/tools" |
1695 | ) |
1696 | @@ -19,7 +20,7 @@ |
1697 | // Bootstrap is a common implementation of the Bootstrap method defined on |
1698 | // environs.Environ; we strongly recommend that this implementation be used |
1699 | // when writing a new provider. |
1700 | -func Bootstrap(env environs.Environ, cons constraints.Value, possibleTools coretools.List) error { |
1701 | +func Bootstrap(env environs.Environ, cons constraints.Value) error { |
1702 | // TODO make safe in the case of racing Bootstraps |
1703 | // If two Bootstraps are called concurrently, there's |
1704 | // no way to make sure that only one succeeds. |
1705 | @@ -27,12 +28,18 @@ |
1706 | // Create an empty bootstrap state file so we can get its URL. |
1707 | // It will be updated with the instance id and hardware characteristics |
1708 | // after the bootstrap instance is started. |
1709 | - stateFileURL, err := CreateStateFile(env.Storage()) |
1710 | + stateFileURL, err := bootstrap.CreateStateFile(env.Storage()) |
1711 | if err != nil { |
1712 | return err |
1713 | } |
1714 | machineConfig := environs.NewBootstrapMachineConfig(stateFileURL) |
1715 | - inst, hw, err := env.StartInstance(cons, possibleTools, machineConfig) |
1716 | + |
1717 | + selectedTools, err := EnsureBootstrapTools(env, env.Config().DefaultSeries(), cons.Arch) |
1718 | + if err != nil { |
1719 | + return err |
1720 | + } |
1721 | + |
1722 | + inst, hw, err := env.StartInstance(cons, selectedTools, machineConfig) |
1723 | if err != nil { |
1724 | return fmt.Errorf("cannot start bootstrap instance: %v", err) |
1725 | } |
1726 | @@ -40,9 +47,9 @@ |
1727 | if hw != nil { |
1728 | characteristics = []instance.HardwareCharacteristics{*hw} |
1729 | } |
1730 | - err = SaveState( |
1731 | + err = bootstrap.SaveState( |
1732 | env.Storage(), |
1733 | - &BootstrapState{ |
1734 | + &bootstrap.BootstrapState{ |
1735 | StateInstances: []instance.Id{inst.Id()}, |
1736 | Characteristics: characteristics, |
1737 | }) |
1738 | @@ -56,3 +63,29 @@ |
1739 | } |
1740 | return nil |
1741 | } |
1742 | + |
1743 | +// EnsureBootstrapTools finds tools, syncing with an external tools source as |
1744 | +// necessary; it then selects the newest tools to bootstrap with, and sets |
1745 | +// agent-version. |
1746 | +func EnsureBootstrapTools(env environs.Environ, series string, arch *string) (coretools.List, error) { |
1747 | + possibleTools, err := bootstrap.EnsureToolsAvailability(env, series, arch) |
1748 | + if err != nil { |
1749 | + return nil, err |
1750 | + } |
1751 | + return bootstrap.SetBootstrapTools(env, possibleTools) |
1752 | +} |
1753 | + |
1754 | +// EnsureNotBootstrapped returns null if the environment is not bootstrapped, |
1755 | +// and an error if it is or if the function was not able to tell. |
1756 | +func EnsureNotBootstrapped(env environs.Environ) error { |
1757 | + _, err := bootstrap.LoadState(env.Storage()) |
1758 | + // If there is no error loading the bootstrap state, then we are |
1759 | + // bootstrapped. |
1760 | + if err == nil { |
1761 | + return fmt.Errorf("environment is already bootstrapped") |
1762 | + } |
1763 | + if err == environs.ErrNotBootstrapped { |
1764 | + return nil |
1765 | + } |
1766 | + return err |
1767 | +} |
1768 | |
1769 | === modified file 'provider/common/bootstrap_test.go' |
1770 | --- provider/common/bootstrap_test.go 2013-10-02 00:29:29 +0000 |
1771 | +++ provider/common/bootstrap_test.go 2013-11-18 06:38:24 +0000 |
1772 | @@ -11,34 +11,79 @@ |
1773 | |
1774 | "launchpad.net/juju-core/constraints" |
1775 | "launchpad.net/juju-core/environs" |
1776 | + "launchpad.net/juju-core/environs/bootstrap" |
1777 | "launchpad.net/juju-core/environs/cloudinit" |
1778 | + "launchpad.net/juju-core/environs/config" |
1779 | + "launchpad.net/juju-core/environs/storage" |
1780 | + envtesting "launchpad.net/juju-core/environs/testing" |
1781 | "launchpad.net/juju-core/instance" |
1782 | "launchpad.net/juju-core/provider/common" |
1783 | + coretesting "launchpad.net/juju-core/testing" |
1784 | jc "launchpad.net/juju-core/testing/checkers" |
1785 | "launchpad.net/juju-core/testing/testbase" |
1786 | "launchpad.net/juju-core/tools" |
1787 | - "launchpad.net/juju-core/version" |
1788 | ) |
1789 | |
1790 | type BootstrapSuite struct { |
1791 | testbase.LoggingSuite |
1792 | + envtesting.ToolsFixture |
1793 | } |
1794 | |
1795 | var _ = gc.Suite(&BootstrapSuite{}) |
1796 | |
1797 | +type cleaner interface { |
1798 | + AddCleanup(testbase.CleanupFunc) |
1799 | +} |
1800 | + |
1801 | +func (s *BootstrapSuite) SetUpTest(c *gc.C) { |
1802 | + s.LoggingSuite.SetUpTest(c) |
1803 | + s.ToolsFixture.SetUpTest(c) |
1804 | +} |
1805 | + |
1806 | +func (s *BootstrapSuite) TearDownTest(c *gc.C) { |
1807 | + s.ToolsFixture.TearDownTest(c) |
1808 | + s.LoggingSuite.TearDownTest(c) |
1809 | +} |
1810 | + |
1811 | +func newStorage(suite cleaner, c *gc.C) storage.Storage { |
1812 | + closer, stor, _ := envtesting.CreateLocalTestStorage(c) |
1813 | + suite.AddCleanup(func(*gc.C) { closer.Close() }) |
1814 | + envtesting.UploadFakeTools(c, stor) |
1815 | + return stor |
1816 | +} |
1817 | + |
1818 | +func minimalConfig(c *gc.C) *config.Config { |
1819 | + attrs := map[string]interface{}{ |
1820 | + "name": "whatever", |
1821 | + "type": "anything, really", |
1822 | + "ca-cert": coretesting.CACert, |
1823 | + "ca-private-key": coretesting.CAKey, |
1824 | + } |
1825 | + cfg, err := config.New(config.UseDefaults, attrs) |
1826 | + c.Assert(err, gc.IsNil) |
1827 | + return cfg |
1828 | +} |
1829 | + |
1830 | +func configGetter(c *gc.C) configFunc { |
1831 | + cfg := minimalConfig(c) |
1832 | + return func() *config.Config { return cfg } |
1833 | +} |
1834 | + |
1835 | func (s *BootstrapSuite) TestCannotWriteStateFile(c *gc.C) { |
1836 | - brokenStorage := &mockStorage{putErr: fmt.Errorf("noes!")} |
1837 | + brokenStorage := &mockStorage{ |
1838 | + Storage: newStorage(s, c), |
1839 | + putErr: fmt.Errorf("noes!"), |
1840 | + } |
1841 | env := &mockEnviron{storage: brokenStorage} |
1842 | - err := common.Bootstrap(env, constraints.Value{}, nil) |
1843 | + err := common.Bootstrap(env, constraints.Value{}) |
1844 | c.Assert(err, gc.ErrorMatches, "cannot create initial state file: noes!") |
1845 | } |
1846 | |
1847 | func (s *BootstrapSuite) TestCannotStartInstance(c *gc.C) { |
1848 | stor := newStorage(s, c) |
1849 | - checkURL, err := stor.URL(common.StateFile) |
1850 | + checkURL, err := stor.URL(bootstrap.StateFile) |
1851 | c.Assert(err, gc.IsNil) |
1852 | checkCons := constraints.MustParse("mem=8G") |
1853 | - checkTools := tools.List{&tools.Tools{Version: version.Current}} |
1854 | |
1855 | startInstance := func( |
1856 | cons constraints.Value, possibleTools tools.List, mcfg *cloudinit.MachineConfig, |
1857 | @@ -46,7 +91,6 @@ |
1858 | instance.Instance, *instance.HardwareCharacteristics, error, |
1859 | ) { |
1860 | c.Assert(cons, gc.DeepEquals, checkCons) |
1861 | - c.Assert(possibleTools, gc.DeepEquals, checkTools) |
1862 | c.Assert(mcfg, gc.DeepEquals, environs.NewBootstrapMachineConfig(checkURL)) |
1863 | return nil, nil, fmt.Errorf("meh, not started") |
1864 | } |
1865 | @@ -54,9 +98,10 @@ |
1866 | env := &mockEnviron{ |
1867 | storage: stor, |
1868 | startInstance: startInstance, |
1869 | + config: configGetter(c), |
1870 | } |
1871 | |
1872 | - err = common.Bootstrap(env, checkCons, checkTools) |
1873 | + err = common.Bootstrap(env, checkCons) |
1874 | c.Assert(err, gc.ErrorMatches, "cannot start bootstrap instance: meh, not started") |
1875 | } |
1876 | |
1877 | @@ -83,9 +128,10 @@ |
1878 | storage: stor, |
1879 | startInstance: startInstance, |
1880 | stopInstances: stopInstances, |
1881 | + config: configGetter(c), |
1882 | } |
1883 | |
1884 | - err := common.Bootstrap(env, constraints.Value{}, nil) |
1885 | + err := common.Bootstrap(env, constraints.Value{}) |
1886 | c.Assert(err, gc.ErrorMatches, "cannot save state: suddenly a wild blah") |
1887 | c.Assert(stopped, gc.HasLen, 1) |
1888 | c.Assert(stopped[0].Id(), gc.Equals, instance.Id("i-blah")) |
1889 | @@ -118,9 +164,10 @@ |
1890 | storage: stor, |
1891 | startInstance: startInstance, |
1892 | stopInstances: stopInstances, |
1893 | + config: configGetter(c), |
1894 | } |
1895 | |
1896 | - err := common.Bootstrap(env, constraints.Value{}, nil) |
1897 | + err := common.Bootstrap(env, constraints.Value{}) |
1898 | c.Assert(err, gc.ErrorMatches, "cannot save state: suddenly a wild blah") |
1899 | c.Assert(stopped, gc.HasLen, 1) |
1900 | c.Assert(stopped[0].Id(), gc.Equals, instance.Id("i-blah")) |
1901 | @@ -144,16 +191,23 @@ |
1902 | return &mockInstance{id: checkInstanceId}, &checkHardware, nil |
1903 | } |
1904 | |
1905 | + var getConfigCalled int |
1906 | + getConfig := func() *config.Config { |
1907 | + getConfigCalled++ |
1908 | + return minimalConfig(c) |
1909 | + } |
1910 | + |
1911 | env := &mockEnviron{ |
1912 | storage: stor, |
1913 | startInstance: startInstance, |
1914 | + config: getConfig, |
1915 | } |
1916 | - err := common.Bootstrap(env, constraints.Value{}, nil) |
1917 | + err := common.Bootstrap(env, constraints.Value{}) |
1918 | c.Assert(err, gc.IsNil) |
1919 | |
1920 | - savedState, err := common.LoadStateFromURL(checkURL) |
1921 | + savedState, err := bootstrap.LoadStateFromURL(checkURL) |
1922 | c.Assert(err, gc.IsNil) |
1923 | - c.Assert(savedState, gc.DeepEquals, &common.BootstrapState{ |
1924 | + c.Assert(savedState, gc.DeepEquals, &bootstrap.BootstrapState{ |
1925 | StateInstances: []instance.Id{instance.Id(checkInstanceId)}, |
1926 | Characteristics: []instance.HardwareCharacteristics{checkHardware}, |
1927 | }) |
1928 | |
1929 | === modified file 'provider/common/mock_test.go' |
1930 | --- provider/common/mock_test.go 2013-10-02 00:29:29 +0000 |
1931 | +++ provider/common/mock_test.go 2013-11-18 06:38:24 +0000 |
1932 | @@ -9,6 +9,8 @@ |
1933 | "launchpad.net/juju-core/constraints" |
1934 | "launchpad.net/juju-core/environs" |
1935 | "launchpad.net/juju-core/environs/cloudinit" |
1936 | + "launchpad.net/juju-core/environs/config" |
1937 | + "launchpad.net/juju-core/environs/simplestreams" |
1938 | "launchpad.net/juju-core/environs/storage" |
1939 | "launchpad.net/juju-core/instance" |
1940 | "launchpad.net/juju-core/tools" |
1941 | @@ -17,12 +19,18 @@ |
1942 | type allInstancesFunc func() ([]instance.Instance, error) |
1943 | type startInstanceFunc func(constraints.Value, tools.List, *cloudinit.MachineConfig) (instance.Instance, *instance.HardwareCharacteristics, error) |
1944 | type stopInstancesFunc func([]instance.Instance) error |
1945 | +type getToolsSourcesFunc func() ([]simplestreams.DataSource, error) |
1946 | +type configFunc func() *config.Config |
1947 | +type setConfigFunc func(*config.Config) error |
1948 | |
1949 | type mockEnviron struct { |
1950 | storage storage.Storage |
1951 | allInstances allInstancesFunc |
1952 | startInstance startInstanceFunc |
1953 | stopInstances stopInstancesFunc |
1954 | + getToolsSources getToolsSourcesFunc |
1955 | + config configFunc |
1956 | + setConfig setConfigFunc |
1957 | environs.Environ // stub out other methods with panics |
1958 | } |
1959 | |
1960 | @@ -49,6 +57,25 @@ |
1961 | return env.stopInstances(instances) |
1962 | } |
1963 | |
1964 | +func (env *mockEnviron) Config() *config.Config { |
1965 | + return env.config() |
1966 | +} |
1967 | + |
1968 | +func (env *mockEnviron) SetConfig(cfg *config.Config) error { |
1969 | + if env.setConfig != nil { |
1970 | + return env.setConfig(cfg) |
1971 | + } |
1972 | + return nil |
1973 | +} |
1974 | + |
1975 | +func (env *mockEnviron) GetToolsSources() ([]simplestreams.DataSource, error) { |
1976 | + if env.getToolsSources != nil { |
1977 | + return env.getToolsSources() |
1978 | + } |
1979 | + datasource := storage.NewStorageSimpleStreamsDataSource(env.Storage(), storage.BaseToolsPath) |
1980 | + return []simplestreams.DataSource{datasource}, nil |
1981 | +} |
1982 | + |
1983 | type mockInstance struct { |
1984 | id string |
1985 | instance.Instance // stub out other methods with panics |
1986 | |
1987 | === modified file 'provider/common/state.go' |
1988 | --- provider/common/state.go 2013-10-01 12:03:33 +0000 |
1989 | +++ provider/common/state.go 2013-11-18 06:38:24 +0000 |
1990 | @@ -4,103 +4,18 @@ |
1991 | package common |
1992 | |
1993 | import ( |
1994 | - "bytes" |
1995 | "errors" |
1996 | "fmt" |
1997 | - "io" |
1998 | - "io/ioutil" |
1999 | - "net/http" |
2000 | - |
2001 | - "launchpad.net/goyaml" |
2002 | |
2003 | "launchpad.net/juju-core/environs" |
2004 | + "launchpad.net/juju-core/environs/bootstrap" |
2005 | "launchpad.net/juju-core/environs/config" |
2006 | - "launchpad.net/juju-core/environs/storage" |
2007 | - coreerrors "launchpad.net/juju-core/errors" |
2008 | "launchpad.net/juju-core/instance" |
2009 | "launchpad.net/juju-core/log" |
2010 | "launchpad.net/juju-core/state" |
2011 | "launchpad.net/juju-core/state/api" |
2012 | ) |
2013 | |
2014 | -// StateFile is the name of the file where the provider's state is stored. |
2015 | -const StateFile = "provider-state" |
2016 | - |
2017 | -// BootstrapState is the state information that is stored in StateFile. |
2018 | -// |
2019 | -// Individual providers may define their own state structures instead of |
2020 | -// this one, and use their own code for loading and saving those, but this is |
2021 | -// the definition that most practically useful providers share unchanged. |
2022 | -type BootstrapState struct { |
2023 | - // StateInstances are the state servers. |
2024 | - StateInstances []instance.Id `yaml:"state-instances"` |
2025 | - // Characteristics reflect the hardware each state server is running on. |
2026 | - // This is used at bootstrap time so the state server knows what hardware it has. |
2027 | - // The state *may* be updated later without this information, but by then it's |
2028 | - // served it's purpose. |
2029 | - Characteristics []instance.HardwareCharacteristics `yaml:"characteristics,omitempty"` |
2030 | -} |
2031 | - |
2032 | -// putState writes the given data to the state file on the given storage. |
2033 | -// The file's name is as defined in StateFile. |
2034 | -func putState(storage storage.StorageWriter, data []byte) error { |
2035 | - return storage.Put(StateFile, bytes.NewBuffer(data), int64(len(data))) |
2036 | -} |
2037 | - |
2038 | -// CreateStateFile creates an empty state file on the given storage, and |
2039 | -// returns its URL. |
2040 | -func CreateStateFile(storage storage.Storage) (string, error) { |
2041 | - err := putState(storage, []byte{}) |
2042 | - if err != nil { |
2043 | - return "", fmt.Errorf("cannot create initial state file: %v", err) |
2044 | - } |
2045 | - return storage.URL(StateFile) |
2046 | -} |
2047 | - |
2048 | -// SaveState writes the given state to the given storage. |
2049 | -func SaveState(storage storage.StorageWriter, state *BootstrapState) error { |
2050 | - data, err := goyaml.Marshal(state) |
2051 | - if err != nil { |
2052 | - return err |
2053 | - } |
2054 | - return putState(storage, data) |
2055 | -} |
2056 | - |
2057 | -// LoadStateFromURL reads state from the given URL. |
2058 | -func LoadStateFromURL(url string) (*BootstrapState, error) { |
2059 | - resp, err := http.Get(url) |
2060 | - if err != nil { |
2061 | - return nil, err |
2062 | - } |
2063 | - return loadState(resp.Body) |
2064 | -} |
2065 | - |
2066 | -// LoadState reads state from the given storage. |
2067 | -func LoadState(stor storage.StorageReader) (*BootstrapState, error) { |
2068 | - r, err := storage.Get(stor, StateFile) |
2069 | - if err != nil { |
2070 | - if coreerrors.IsNotFoundError(err) { |
2071 | - return nil, environs.ErrNotBootstrapped |
2072 | - } |
2073 | - return nil, err |
2074 | - } |
2075 | - return loadState(r) |
2076 | -} |
2077 | - |
2078 | -func loadState(r io.ReadCloser) (*BootstrapState, error) { |
2079 | - defer r.Close() |
2080 | - data, err := ioutil.ReadAll(r) |
2081 | - if err != nil { |
2082 | - return nil, fmt.Errorf("error reading %q: %v", StateFile, err) |
2083 | - } |
2084 | - var state BootstrapState |
2085 | - err = goyaml.Unmarshal(data, &state) |
2086 | - if err != nil { |
2087 | - return nil, fmt.Errorf("error unmarshalling %q: %v", StateFile, err) |
2088 | - } |
2089 | - return &state, nil |
2090 | -} |
2091 | - |
2092 | // getDNSNames queries and returns the DNS names for the given instances, |
2093 | // ignoring nil instances or ones without DNS names. |
2094 | func getDNSNames(instances []instance.Instance) []string { |
2095 | @@ -147,7 +62,7 @@ |
2096 | // StateInfo is a reusable implementation of Environ.StateInfo, available to |
2097 | // providers that also use the other functionality from this file. |
2098 | func StateInfo(env environs.Environ) (*state.Info, *api.Info, error) { |
2099 | - st, err := LoadState(env.Storage()) |
2100 | + st, err := bootstrap.LoadState(env.Storage()) |
2101 | if err != nil { |
2102 | return nil, nil, err |
2103 | } |
2104 | |
2105 | === modified file 'provider/common/state_test.go' |
2106 | --- provider/common/state_test.go 2013-10-02 10:39:12 +0000 |
2107 | +++ provider/common/state_test.go 2013-11-18 06:38:24 +0000 |
2108 | @@ -4,16 +4,9 @@ |
2109 | package common_test |
2110 | |
2111 | import ( |
2112 | - "bytes" |
2113 | - "io/ioutil" |
2114 | - |
2115 | gc "launchpad.net/gocheck" |
2116 | - "launchpad.net/goyaml" |
2117 | |
2118 | - "launchpad.net/juju-core/environs" |
2119 | "launchpad.net/juju-core/environs/config" |
2120 | - "launchpad.net/juju-core/environs/storage" |
2121 | - envtesting "launchpad.net/juju-core/environs/testing" |
2122 | "launchpad.net/juju-core/instance" |
2123 | "launchpad.net/juju-core/provider/common" |
2124 | "launchpad.net/juju-core/testing" |
2125 | @@ -26,102 +19,6 @@ |
2126 | |
2127 | var _ = gc.Suite(&StateSuite{}) |
2128 | |
2129 | -type cleaner interface { |
2130 | - AddCleanup(testbase.CleanupFunc) |
2131 | -} |
2132 | - |
2133 | -func newStorage(suite cleaner, c *gc.C) storage.Storage { |
2134 | - closer, stor, _ := envtesting.CreateLocalTestStorage(c) |
2135 | - suite.AddCleanup(func(*gc.C) { closer.Close() }) |
2136 | - return stor |
2137 | -} |
2138 | - |
2139 | -func (suite *StateSuite) TestCreateStateFileWritesEmptyStateFile(c *gc.C) { |
2140 | - stor := newStorage(suite, c) |
2141 | - |
2142 | - url, err := common.CreateStateFile(stor) |
2143 | - c.Assert(err, gc.IsNil) |
2144 | - |
2145 | - reader, err := storage.Get(stor, common.StateFile) |
2146 | - c.Assert(err, gc.IsNil) |
2147 | - data, err := ioutil.ReadAll(reader) |
2148 | - c.Assert(err, gc.IsNil) |
2149 | - c.Check(string(data), gc.Equals, "") |
2150 | - c.Assert(url, gc.NotNil) |
2151 | - expectedURL, err := stor.URL(common.StateFile) |
2152 | - c.Assert(err, gc.IsNil) |
2153 | - c.Check(url, gc.Equals, expectedURL) |
2154 | -} |
2155 | - |
2156 | -func (suite *StateSuite) TestSaveStateWritesStateFile(c *gc.C) { |
2157 | - stor := newStorage(suite, c) |
2158 | - arch := "amd64" |
2159 | - state := common.BootstrapState{ |
2160 | - StateInstances: []instance.Id{instance.Id("an-instance-id")}, |
2161 | - Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}} |
2162 | - marshaledState, err := goyaml.Marshal(state) |
2163 | - c.Assert(err, gc.IsNil) |
2164 | - |
2165 | - err = common.SaveState(stor, &state) |
2166 | - c.Assert(err, gc.IsNil) |
2167 | - |
2168 | - loadedState, err := storage.Get(stor, common.StateFile) |
2169 | - c.Assert(err, gc.IsNil) |
2170 | - content, err := ioutil.ReadAll(loadedState) |
2171 | - c.Assert(err, gc.IsNil) |
2172 | - c.Check(content, gc.DeepEquals, marshaledState) |
2173 | -} |
2174 | - |
2175 | -func (suite *StateSuite) setUpSavedState(c *gc.C, stor storage.Storage) common.BootstrapState { |
2176 | - arch := "amd64" |
2177 | - state := common.BootstrapState{ |
2178 | - StateInstances: []instance.Id{instance.Id("an-instance-id")}, |
2179 | - Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}} |
2180 | - content, err := goyaml.Marshal(state) |
2181 | - c.Assert(err, gc.IsNil) |
2182 | - err = stor.Put(common.StateFile, ioutil.NopCloser(bytes.NewReader(content)), int64(len(content))) |
2183 | - c.Assert(err, gc.IsNil) |
2184 | - return state |
2185 | -} |
2186 | - |
2187 | -func (suite *StateSuite) TestLoadStateReadsStateFile(c *gc.C) { |
2188 | - storage := newStorage(suite, c) |
2189 | - state := suite.setUpSavedState(c, storage) |
2190 | - storedState, err := common.LoadState(storage) |
2191 | - c.Assert(err, gc.IsNil) |
2192 | - c.Check(*storedState, gc.DeepEquals, state) |
2193 | -} |
2194 | - |
2195 | -func (suite *StateSuite) TestLoadStateFromURLReadsStateFile(c *gc.C) { |
2196 | - stor := newStorage(suite, c) |
2197 | - state := suite.setUpSavedState(c, stor) |
2198 | - url, err := stor.URL(common.StateFile) |
2199 | - c.Assert(err, gc.IsNil) |
2200 | - storedState, err := common.LoadStateFromURL(url) |
2201 | - c.Assert(err, gc.IsNil) |
2202 | - c.Check(*storedState, gc.DeepEquals, state) |
2203 | -} |
2204 | - |
2205 | -func (suite *StateSuite) TestLoadStateMissingFile(c *gc.C) { |
2206 | - stor := newStorage(suite, c) |
2207 | - _, err := common.LoadState(stor) |
2208 | - c.Check(err, gc.Equals, environs.ErrNotBootstrapped) |
2209 | -} |
2210 | - |
2211 | -func (suite *StateSuite) TestLoadStateIntegratesWithSaveState(c *gc.C) { |
2212 | - storage := newStorage(suite, c) |
2213 | - arch := "amd64" |
2214 | - state := common.BootstrapState{ |
2215 | - StateInstances: []instance.Id{instance.Id("an-instance-id")}, |
2216 | - Characteristics: []instance.HardwareCharacteristics{{Arch: &arch}}} |
2217 | - err := common.SaveState(storage, &state) |
2218 | - c.Assert(err, gc.IsNil) |
2219 | - storedState, err := common.LoadState(storage) |
2220 | - c.Assert(err, gc.IsNil) |
2221 | - |
2222 | - c.Check(*storedState, gc.DeepEquals, state) |
2223 | -} |
2224 | - |
2225 | func (suite *StateSuite) TestGetDNSNamesAcceptsNil(c *gc.C) { |
2226 | result := common.GetDNSNames(nil) |
2227 | c.Check(result, gc.DeepEquals, []string{}) |
2228 | |
2229 | === modified file 'provider/dummy/environs.go' |
2230 | --- provider/dummy/environs.go 2013-11-07 09:09:55 +0000 |
2231 | +++ provider/dummy/environs.go 2013-11-18 06:38:24 +0000 |
2232 | @@ -37,6 +37,7 @@ |
2233 | |
2234 | "launchpad.net/juju-core/constraints" |
2235 | "launchpad.net/juju-core/environs" |
2236 | + "launchpad.net/juju-core/environs/bootstrap" |
2237 | "launchpad.net/juju-core/environs/cloudinit" |
2238 | "launchpad.net/juju-core/environs/config" |
2239 | "launchpad.net/juju-core/environs/imagemetadata" |
2240 | @@ -520,7 +521,12 @@ |
2241 | storage.NewStorageSimpleStreamsDataSource(e.Storage(), storage.BaseToolsPath)}, nil |
2242 | } |
2243 | |
2244 | -func (e *environ) Bootstrap(cons constraints.Value, possibleTools coretools.List) error { |
2245 | +func (e *environ) Bootstrap(cons constraints.Value) error { |
2246 | + selectedTools, err := common.EnsureBootstrapTools(e, e.Config().DefaultSeries(), cons.Arch) |
2247 | + if err != nil { |
2248 | + return err |
2249 | + } |
2250 | + |
2251 | defer delay() |
2252 | if err := e.checkBroken("Bootstrap"); err != nil { |
2253 | return err |
2254 | @@ -533,7 +539,7 @@ |
2255 | return fmt.Errorf("no CA certificate in environment configuration") |
2256 | } |
2257 | |
2258 | - logger.Infof("would pick tools from %s", possibleTools) |
2259 | + logger.Infof("would pick tools from %s", selectedTools) |
2260 | cfg, err := environs.BootstrapConfig(e.Config()) |
2261 | if err != nil { |
2262 | return fmt.Errorf("cannot make bootstrap config: %v", err) |
2263 | @@ -552,7 +558,7 @@ |
2264 | // we need to release the mutex for the save state to work, so regain |
2265 | // it after the call. |
2266 | estate.mu.Unlock() |
2267 | - if err := common.SaveState(e.Storage(), &common.BootstrapState{StateInstances: []instance.Id{"localhost"}}); err != nil { |
2268 | + if err := bootstrap.SaveState(e.Storage(), &bootstrap.BootstrapState{StateInstances: []instance.Id{"localhost"}}); err != nil { |
2269 | logger.Errorf("failed to save state instances: %v", err) |
2270 | estate.mu.Lock() // otherwise defered unlock will fail |
2271 | return err |
2272 | |
2273 | === modified file 'provider/ec2/ec2.go' |
2274 | --- provider/ec2/ec2.go 2013-11-01 06:20:19 +0000 |
2275 | +++ provider/ec2/ec2.go 2013-11-18 06:38:24 +0000 |
2276 | @@ -305,8 +305,8 @@ |
2277 | return stor |
2278 | } |
2279 | |
2280 | -func (e *environ) Bootstrap(cons constraints.Value, possibleTools tools.List) error { |
2281 | - return common.Bootstrap(e, cons, possibleTools) |
2282 | +func (e *environ) Bootstrap(cons constraints.Value) error { |
2283 | + return common.Bootstrap(e, cons) |
2284 | } |
2285 | |
2286 | func (e *environ) StateInfo() (*state.Info, *api.Info, error) { |
2287 | |
2288 | === modified file 'provider/ec2/local_test.go' |
2289 | --- provider/ec2/local_test.go 2013-11-01 06:20:19 +0000 |
2290 | +++ provider/ec2/local_test.go 2013-11-18 06:38:24 +0000 |
2291 | @@ -28,7 +28,6 @@ |
2292 | "launchpad.net/juju-core/environs/tools" |
2293 | "launchpad.net/juju-core/instance" |
2294 | "launchpad.net/juju-core/juju/testing" |
2295 | - "launchpad.net/juju-core/provider/common" |
2296 | "launchpad.net/juju-core/provider/ec2" |
2297 | coretesting "launchpad.net/juju-core/testing" |
2298 | jc "launchpad.net/juju-core/testing/checkers" |
2299 | @@ -210,7 +209,7 @@ |
2300 | c.Assert(err, gc.IsNil) |
2301 | |
2302 | // check that the state holds the id of the bootstrap machine. |
2303 | - bootstrapState, err := common.LoadState(env.Storage()) |
2304 | + bootstrapState, err := bootstrap.LoadState(env.Storage()) |
2305 | c.Assert(err, gc.IsNil) |
2306 | c.Assert(bootstrapState.StateInstances, gc.HasLen, 1) |
2307 | |
2308 | @@ -263,7 +262,7 @@ |
2309 | err = env.Destroy() |
2310 | c.Assert(err, gc.IsNil) |
2311 | |
2312 | - _, err = common.LoadState(env.Storage()) |
2313 | + _, err = bootstrap.LoadState(env.Storage()) |
2314 | c.Assert(err, gc.NotNil) |
2315 | } |
2316 | |
2317 | |
2318 | === modified file 'provider/local/environ.go' |
2319 | --- provider/local/environ.go 2013-11-11 21:38:30 +0000 |
2320 | +++ provider/local/environ.go 2013-11-18 06:38:24 +0000 |
2321 | @@ -19,6 +19,7 @@ |
2322 | "launchpad.net/juju-core/container" |
2323 | "launchpad.net/juju-core/container/lxc" |
2324 | "launchpad.net/juju-core/environs" |
2325 | + "launchpad.net/juju-core/environs/bootstrap" |
2326 | "launchpad.net/juju-core/environs/cloudinit" |
2327 | "launchpad.net/juju-core/environs/config" |
2328 | "launchpad.net/juju-core/environs/filestorage" |
2329 | @@ -95,7 +96,7 @@ |
2330 | } |
2331 | |
2332 | // Bootstrap is specified in the Environ interface. |
2333 | -func (env *localEnviron) Bootstrap(cons constraints.Value, possibleTools tools.List) error { |
2334 | +func (env *localEnviron) Bootstrap(cons constraints.Value) error { |
2335 | if !env.config.runningAsRoot { |
2336 | return fmt.Errorf("bootstrapping a local environment must be done as root") |
2337 | } |
2338 | @@ -113,13 +114,19 @@ |
2339 | |
2340 | // Before we write the agent config file, we need to make sure the |
2341 | // instance is saved in the StateInfo. |
2342 | - if err := common.SaveState(env.Storage(), &common.BootstrapState{ |
2343 | + if err := bootstrap.SaveState(env.Storage(), &bootstrap.BootstrapState{ |
2344 | StateInstances: []instance.Id{bootstrapInstanceId}, |
2345 | }); err != nil { |
2346 | logger.Errorf("failed to save state instances: %v", err) |
2347 | return err |
2348 | } |
2349 | |
2350 | + vers := version.Current |
2351 | + selectedTools, err := common.EnsureBootstrapTools(env, vers.Series, &vers.Arch) |
2352 | + if err != nil { |
2353 | + return err |
2354 | + } |
2355 | + |
2356 | // Need to write out the agent file for machine-0 before initializing |
2357 | // state, as as part of that process, it will reset the password in the |
2358 | // agent file. |
2359 | @@ -132,7 +139,7 @@ |
2360 | return err |
2361 | } |
2362 | |
2363 | - return env.setupLocalMachineAgent(cons, possibleTools) |
2364 | + return env.setupLocalMachineAgent(cons, selectedTools) |
2365 | } |
2366 | |
2367 | // StateInfo is specified in the Environ interface. |
2368 | |
2369 | === modified file 'provider/maas/environ.go' |
2370 | --- provider/maas/environ.go 2013-11-04 22:24:51 +0000 |
2371 | +++ provider/maas/environ.go 2013-11-18 06:38:24 +0000 |
2372 | @@ -77,8 +77,8 @@ |
2373 | } |
2374 | |
2375 | // Bootstrap is specified in the Environ interface. |
2376 | -func (env *maasEnviron) Bootstrap(cons constraints.Value, possibleTools tools.List) error { |
2377 | - return common.Bootstrap(env, cons, possibleTools) |
2378 | +func (env *maasEnviron) Bootstrap(cons constraints.Value) error { |
2379 | + return common.Bootstrap(env, cons) |
2380 | } |
2381 | |
2382 | // StateInfo is specified in the Environ interface. |
2383 | |
2384 | === modified file 'provider/maas/environ_test.go' |
2385 | --- provider/maas/environ_test.go 2013-11-04 22:24:51 +0000 |
2386 | +++ provider/maas/environ_test.go 2013-11-18 06:38:24 +0000 |
2387 | @@ -26,7 +26,6 @@ |
2388 | "launchpad.net/juju-core/errors" |
2389 | "launchpad.net/juju-core/instance" |
2390 | "launchpad.net/juju-core/juju/testing" |
2391 | - "launchpad.net/juju-core/provider/common" |
2392 | jc "launchpad.net/juju-core/testing/checkers" |
2393 | "launchpad.net/juju-core/tools" |
2394 | "launchpad.net/juju-core/utils" |
2395 | @@ -59,7 +58,7 @@ |
2396 | } |
2397 | |
2398 | func (suite *environSuite) setupFakeProviderStateFile(c *gc.C) { |
2399 | - suite.testMAASObject.TestServer.NewFile(common.StateFile, []byte("test file content")) |
2400 | + suite.testMAASObject.TestServer.NewFile(bootstrap.StateFile, []byte("test file content")) |
2401 | } |
2402 | |
2403 | func (suite *environSuite) setupFakeTools(c *gc.C) { |
2404 | @@ -216,7 +215,7 @@ |
2405 | |
2406 | // Test the instance id is correctly recorded for the bootstrap node. |
2407 | // Check that the state holds the id of the bootstrap machine. |
2408 | - stateData, err := common.LoadState(env.Storage()) |
2409 | + stateData, err := bootstrap.LoadState(env.Storage()) |
2410 | c.Assert(err, gc.IsNil) |
2411 | c.Assert(stateData.StateInstances, gc.HasLen, 1) |
2412 | insts, err := env.AllInstances() |
2413 | @@ -370,9 +369,9 @@ |
2414 | input := `{"system_id": "system_id", "hostname": "` + hostname + `"}` |
2415 | node := suite.testMAASObject.TestServer.NewNode(input) |
2416 | testInstance := &maasInstance{&node, suite.makeEnviron()} |
2417 | - err := common.SaveState( |
2418 | + err := bootstrap.SaveState( |
2419 | env.Storage(), |
2420 | - &common.BootstrapState{StateInstances: []instance.Id{testInstance.Id()}}) |
2421 | + &bootstrap.BootstrapState{StateInstances: []instance.Id{testInstance.Id()}}) |
2422 | c.Assert(err, gc.IsNil) |
2423 | |
2424 | stateInfo, apiInfo, err := env.StateInfo() |
2425 | @@ -429,7 +428,14 @@ |
2426 | env := suite.makeEnviron() |
2427 | // Can't RemoveAllTools, no public storage. |
2428 | envtesting.RemoveTools(c, env.Storage()) |
2429 | - err := bootstrap.Bootstrap(env, constraints.Value{}) |
2430 | + // Disable auto-uploading by setting the agent version. |
2431 | + cfg, err := env.Config().Apply(map[string]interface{}{ |
2432 | + "agent-version": version.Current.Number.String(), |
2433 | + }) |
2434 | + c.Assert(err, gc.IsNil) |
2435 | + err = env.SetConfig(cfg) |
2436 | + c.Assert(err, gc.IsNil) |
2437 | + err = bootstrap.Bootstrap(env, constraints.Value{}) |
2438 | c.Check(err, gc.ErrorMatches, "cannot find bootstrap tools.*") |
2439 | } |
2440 | |
2441 | |
2442 | === modified file 'provider/null/config_test.go' |
2443 | --- provider/null/config_test.go 2013-10-04 09:32:51 +0000 |
2444 | +++ provider/null/config_test.go 2013-11-18 06:38:24 +0000 |
2445 | @@ -6,7 +6,6 @@ |
2446 | import ( |
2447 | "fmt" |
2448 | "regexp" |
2449 | - stdtesting "testing" |
2450 | |
2451 | gc "launchpad.net/gocheck" |
2452 | |
2453 | @@ -22,10 +21,6 @@ |
2454 | |
2455 | var _ = gc.Suite(&configSuite{}) |
2456 | |
2457 | -func Test(t *stdtesting.T) { |
2458 | - gc.TestingT(t) |
2459 | -} |
2460 | - |
2461 | func minimalConfigValues() map[string]interface{} { |
2462 | return map[string]interface{}{ |
2463 | "name": "test", |
2464 | |
2465 | === modified file 'provider/null/environ.go' |
2466 | --- provider/null/environ.go 2013-10-24 00:20:59 +0000 |
2467 | +++ provider/null/environ.go 2013-11-18 06:38:24 +0000 |
2468 | @@ -85,12 +85,23 @@ |
2469 | return e.envConfig().Name() |
2470 | } |
2471 | |
2472 | -func (e *nullEnviron) Bootstrap(_ constraints.Value, possibleTools tools.List) error { |
2473 | +func (e *nullEnviron) Bootstrap(cons constraints.Value) error { |
2474 | + envConfig := e.envConfig() |
2475 | + hc, series, err := manual.DetectSeriesAndHardwareCharacteristics(envConfig.sshHost()) |
2476 | + if err != nil { |
2477 | + return err |
2478 | + } |
2479 | + selectedTools, err := common.EnsureBootstrapTools(e, series, hc.Arch) |
2480 | + if err != nil { |
2481 | + return err |
2482 | + } |
2483 | return manual.Bootstrap(manual.BootstrapArgs{ |
2484 | - Host: e.envConfig().sshHost(), |
2485 | - DataDir: dataDir, |
2486 | - Environ: e, |
2487 | - PossibleTools: possibleTools, |
2488 | + Host: e.envConfig().sshHost(), |
2489 | + DataDir: dataDir, |
2490 | + Environ: e, |
2491 | + PossibleTools: selectedTools, |
2492 | + Series: series, |
2493 | + HardwareCharacteristics: &hc, |
2494 | }) |
2495 | } |
2496 | |
2497 | |
2498 | === modified file 'provider/null/provider.go' |
2499 | --- provider/null/provider.go 2013-10-08 13:35:41 +0000 |
2500 | +++ provider/null/provider.go 2013-11-18 06:38:24 +0000 |
2501 | @@ -31,7 +31,11 @@ |
2502 | if err != nil { |
2503 | return nil, err |
2504 | } |
2505 | - return &nullEnviron{cfg: envConfig}, nil |
2506 | + return p.open(envConfig) |
2507 | +} |
2508 | + |
2509 | +func (p nullProvider) open(cfg *environConfig) (environs.Environ, error) { |
2510 | + return &nullEnviron{cfg: cfg}, nil |
2511 | } |
2512 | |
2513 | func checkImmutableString(cfg, old *environConfig, key string) error { |
2514 | |
2515 | === added file 'provider/null/suite_test.go' |
2516 | --- provider/null/suite_test.go 1970-01-01 00:00:00 +0000 |
2517 | +++ provider/null/suite_test.go 2013-11-18 06:38:24 +0000 |
2518 | @@ -0,0 +1,14 @@ |
2519 | +// Copyright 2013 Canonical Ltd. |
2520 | +// Licensed under the AGPLv3, see LICENCE file for details. |
2521 | + |
2522 | +package null |
2523 | + |
2524 | +import ( |
2525 | + "testing" |
2526 | + |
2527 | + gc "launchpad.net/gocheck" |
2528 | +) |
2529 | + |
2530 | +func Test(t *testing.T) { |
2531 | + gc.TestingT(t) |
2532 | +} |
2533 | |
2534 | === modified file 'provider/openstack/local_test.go' |
2535 | --- provider/openstack/local_test.go 2013-11-05 04:32:17 +0000 |
2536 | +++ provider/openstack/local_test.go 2013-11-18 06:38:24 +0000 |
2537 | @@ -32,7 +32,6 @@ |
2538 | "launchpad.net/juju-core/environs/tools" |
2539 | "launchpad.net/juju-core/instance" |
2540 | "launchpad.net/juju-core/juju/testing" |
2541 | - "launchpad.net/juju-core/provider/common" |
2542 | "launchpad.net/juju-core/provider/openstack" |
2543 | coretesting "launchpad.net/juju-core/testing" |
2544 | jc "launchpad.net/juju-core/testing/checkers" |
2545 | @@ -477,7 +476,7 @@ |
2546 | c.Assert(err, gc.IsNil) |
2547 | |
2548 | // check that the state holds the id of the bootstrap machine. |
2549 | - stateData, err := common.LoadState(env.Storage()) |
2550 | + stateData, err := bootstrap.LoadState(env.Storage()) |
2551 | c.Assert(err, gc.IsNil) |
2552 | c.Assert(stateData.StateInstances, gc.HasLen, 1) |
2553 | |
2554 | |
2555 | === modified file 'provider/openstack/provider.go' |
2556 | --- provider/openstack/provider.go 2013-11-04 02:49:34 +0000 |
2557 | +++ provider/openstack/provider.go 2013-11-18 06:38:24 +0000 |
2558 | @@ -470,7 +470,7 @@ |
2559 | return stor |
2560 | } |
2561 | |
2562 | -func (e *environ) Bootstrap(cons constraints.Value, possibleTools tools.List) error { |
2563 | +func (e *environ) Bootstrap(cons constraints.Value) error { |
2564 | // The client's authentication may have been reset when finding tools if the agent-version |
2565 | // attribute was updated so we need to re-authenticate. This will be a no-op if already authenticated. |
2566 | // An authenticated client is needed for the URL() call below. |
2567 | @@ -478,7 +478,7 @@ |
2568 | if err != nil { |
2569 | return err |
2570 | } |
2571 | - return common.Bootstrap(e, cons, possibleTools) |
2572 | + return common.Bootstrap(e, cons) |
2573 | } |
2574 | |
2575 | func (e *environ) StateInfo() (*state.Info, *api.Info, error) { |
Reviewers: mp+190032_ code.launchpad. net,
Message:
Please take a look.
Description:
provider/null: dynamic default-series
default-series is used to decide which
tools to bootstrap with. This is not
necessarily valid for the null provider,
where the boostrap host's architecture
should instead decide which tools to
locate.
So now we assign default-series dynamically,
by performing detection at Prepare time.
We store the detected series and hardware
characteristics, and use them to avoid
repeating the detection at Bootstrap time.
Fixes #1236691
https:/ /code.launchpad .net/~axwalk/ juju-core/ lp1236691- null-provider- default- series- take2/+ merge/190032
(do not edit description out of merge proposal)
Please review this at https:/ /codereview. appspot. com/14433058/
Affected files (+346, -76 lines): manual/ bootstrap. go manual/ bootstrap_ test.go manual/ detection. go manual/ detection_ test.go manual/ fakessh. go manual/ provisioner. go manual/ provisioner_ test.go null/config. go null/config_ test.go null/environ. go null/provider. go null/provider_ test.go null/suite_ test.go
A [revision details]
M environs/
M environs/
M environs/
M environs/
M environs/
M environs/
M environs/
M provider/
M provider/
M provider/
M provider/
A provider/
A provider/