Merge lp:~wallyworld/juju-core/bootstrap-tools-fixes into lp:~go-bot/juju-core/trunk
- bootstrap-tools-fixes
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Ian Booth |
Approved revision: | no longer in the source branch. |
Merged at revision: | 2440 |
Proposed branch: | lp:~wallyworld/juju-core/bootstrap-tools-fixes |
Merge into: | lp:~go-bot/juju-core/trunk |
Diff against target: |
1543 lines (+539/-188) 32 files modified
agent/tools/tools_test.go (+1/-1) cmd/juju/bootstrap.go (+1/-41) cmd/juju/bootstrap_test.go (+59/-20) cmd/juju/upgradejuju.go (+3/-2) cmd/package_test.go (+1/-1) environs/bootstrap/bootstrap_test.go (+198/-11) environs/bootstrap/synctools.go (+93/-46) environs/interface.go (+12/-1) environs/manual/init.go (+6/-6) environs/testing/tools.go (+18/-13) environs/tools/tools_test.go (+1/-1) juju/arch/arch.go (+13/-15) juju/arch/arch_test.go (+5/-9) provider/azure/environ.go (+8/-3) provider/azure/environ_test.go (+7/-0) provider/common/bootstrap.go (+6/-5) provider/common/bootstrap_test.go (+1/-1) provider/common/mock_test.go (+4/-0) provider/dummy/environs.go (+6/-0) provider/ec2/ec2.go (+5/-0) provider/ec2/local_test.go (+7/-0) provider/joyent/environ.go (+6/-0) provider/local/environ.go (+7/-0) provider/local/environ_test.go (+13/-0) provider/maas/environ.go (+7/-0) provider/maas/environ_test.go (+10/-2) provider/maas/environ_whitebox_test.go (+5/-1) provider/manual/environ.go (+11/-0) provider/manual/environ_test.go (+10/-0) provider/openstack/local_test.go (+7/-0) provider/openstack/provider.go (+5/-0) version/version.go (+3/-9) |
To merge this branch: | bzr merge lp:~wallyworld/juju-core/bootstrap-tools-fixes |
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju Engineering | Pending | ||
Review via email: mp+211449@code.launchpad.net |
Commit message
Fix bootstrap tools issues
These bugs are addressed:
lp:1227722 - tools for wrong arch
lp:1282870 - cannot use constraints to select tools
lp:1282869 - bad failure mode for incorrect arch
tl;dr; you can bootstrap environments which run on
architectures which are different to the client
machine architcture, so long as tools are available.
eg ppc client boottrapping an environment on EC2.
Other error conditions are also addressed eg you cannot
upload tools to a cloud if that cloud does not support
the client machine architecture, since that's what the
uploaded tools would be built for. You also cannot
specify an arch constraint different to your client
machine if you are uploading tools.
A lot of the tools logic was duplicated. It has been
refactoroed to used shared code. There's still more
to do however since it's still possible to do bad things
with upgrade-juju.
Description of the change
Fix bootstrap tools issues
These bugs are addressed:
lp:1227722 - tools for wrong arch
lp:1282870 - cannot use constraints to select tools
lp:1282869 - bad failure mode for incorrect arch
tl;dr; you can bootstrap environments which run on
architectures which are different to the client
machine architcture, so long as tools are available.
eg ppc client boottrapping an environment on EC2.
Other error conditions are also addressed eg you cannot
upload tools to a cloud if that cloud does not support
the client machine architecture, since that's what the
uploaded tools would be built for. You also cannot
specify an arch constraint different to your client
machine if you are uploading tools.
A lot of the tools logic was duplicated. It has been
refactoroed to used shared code. There's still more
to do however since it's still possible to do bad things
with upgrade-juju.
Ian Booth (wallyworld) wrote : | # |
John A Meinel (jameinel) wrote : | # |
I like this patch a lot. The only thing that concerns me is that all of
the "SupportedArchi
is going to vary a lot by cloud installation (especially for MaaS and
Openstack).
I'm fine landing the patch as this, but we'll definitely need to add
some bugs about how we're going to enable other Architectures, or
probing for Architectures.
Maybe change MaaS to be more lenient at least. I'd hate for us to
release this and MaaS doesn't support any of the arm platforms. (I'm
pretty sure the hyperscale work was all done with armhf or arm64
machines.)
LGTM with some discussion.
https:/
File cmd/juju/
https:/
cmd/juju/
c.Constraints.Arch, true, c.Series...)
This file looks amazing! :)
https:/
File environs/
https:/
environs/
version already set.
"if agent version is already set"
https:/
File environs/
https:/
environs/
I'm a little surprised that we have something called "validate*" and we
are doing more validation before we get to it.
Would it make more sense to move the series validation checks into
validateUploadA
https:/
File environs/
https:/
environs/
numerous tests which use it in cmd/juju and bootstrapSuite.
I'm not sure if the comment here is relevant.
I agree that it is effectively tested, but having a couple of direct
tests can also help define what *this* part of the code is meant to be
doing in isolation.
https:/
environs/
*config.Config, series []string) []string {
I think this should be called "SeriesToUpload".
The problem is that "UploadTools" actually does an Upload, but
UploadSeries is just determining what list of series we would like to
use, so I'd rather see it named something that doesn't look like an
action.
https:/
File provider/
https:/
provider/
Is there any way we could detect this from the Provider rather than hard
coding it inside Juju?
It seems like if a cloud start supporting a new arch which we already
build tools for, then we...
Ian Booth (wallyworld) wrote : | # |
On 2014/03/18 06:29:10, jameinel wrote:
> I like this patch a lot. The only thing that concerns me is that all
of the
> "SupportedArchi
is going to
> vary a lot by cloud installation (especially for MaaS and Openstack).
> I'm fine landing the patch as this, but we'll definitely need to add
some bugs
> about how we're going to enable other Architectures, or probing for
> Architectures.
> Maybe change MaaS to be more lenient at least. I'd hate for us to
release this
> and MaaS doesn't support any of the arm platforms. (I'm pretty sure
the
> hyperscale work was all done with armhf or arm64 machines.)
I agree and I should have been more verbose in my covering letter. The
plan is indeed for the SupportedArchit
environs.Environ to probe for what archietcures can be run. But I'm not
sure just yet how to do that an what form it will actually take. We were
effectively previously hardcoding the types of tools we were searching
for so yeah, it's not a regression as such. I wanted to land as is
because there's plenty of value already and we can iterate. There's also
more work required to consolidate the upload/sync tools stuff. It's
still a bit all over the place.
John A Meinel (jameinel) wrote : | # |
So I think we worked out that we want to default to being more permissive on MaaS and other providers (allow any possible arch), and to try to get the information from simplestreams data (since that is what we need to start an instance anyway).
And that we want to use version.
Ian Booth (wallyworld) wrote : | # |
Please take a look.
https:/
File environs/
https:/
environs/
version already set.
On 2014/03/18 06:29:11, jameinel wrote:
> "if agent version is already set"
Done.
https:/
File environs/
https:/
environs/
On 2014/03/18 06:29:11, jameinel wrote:
> I'm a little surprised that we have something called "validate*" and
we are
> doing more validation before we get to it.
> Would it make more sense to move the series validation checks into
> validateUploadA
Different purposes - the series check just ensures the bootstrapSeries
parameter is syntactically correct. The validateUploadA
semantic checking to ensure the upload can proceed based on business
rules eg arches must match. The series check is not expected to fail and
is a failsafe with a terse error message. The upload validation error is
quite verbose since it is plausible that failure will occur and the user
needs feedback. So I'd prefer to keep as is but am happy to discuss if
you want.
https:/
File environs/
https:/
environs/
numerous tests which use it in cmd/juju and bootstrapSuite.
On 2014/03/18 06:29:11, jameinel wrote:
> I'm not sure if the comment here is relevant.
> I agree that it is effectively tested, but having a couple of direct
tests can
> also help define what *this* part of the code is meant to be doing in
isolation.
Ok. The functionality was previously not exported. It was only exported
when I removed a bunch of duplicate code. But I'll add a direct test.
https:/
environs/
*config.Config, series []string) []string {
On 2014/03/18 06:29:11, jameinel wrote:
> I think this should be called "SeriesToUpload".
> The problem is that "UploadTools" actually does an Upload, but
UploadSeries is
> just determining what list of series we would like to use, so I'd
rather see it
> named something that doesn't look like an action.
Done.
https:/
File provider/
https:/
provider/
On 2014/03/18 06:29:11, jameinel wrote:
> Is there any way we could detect this from the Provider rather than
hard coding
> it inside Juju?
> It seems like if a cloud start supporting a new arch which we already
build
> tools for, then we'll have to issue a .patch rele...
Go Bot (go-bot) wrote : | # |
The attempt to merge lp:~wallyworld/juju-core/bootstrap-tools-fixes into lp:juju-core failed. Below is the output from the failed tests.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
-------
FAIL: bootstrap_
[LOG] 38.70757 DEBUG juju.environs.tools no architecture specified when finding tools, looking for any
[LOG] 38.70759 DEBUG juju.environs.tools no series specified when finding tools, looking for any
[LOG] 38.70764 DEBUG juju.environs.
[LOG] 38.70769 DEBUG juju.environs.
[LOG] 38.70773 DEBUG juju.environs.
[LOG] 38.70777 DEBUG juju.environs.
[LOG] 38.70971 INFO juju.environs.tools Writing tools/streams/
[LOG] 38.70999 INFO juju.environs.tools Writing tools/streams/
[LOG] 38.71128 DEBUG juju.environs.tools no architecture specified when finding tools, looking for any
[LOG] 38.71129 DEBUG juju.environs.tools no series specified when finding tools, looking for any
[LOG] 38.71135 DEBUG juju.environs.
[LOG] 38.71140 DEBUG juju.environs.
[LOG] 38.71143 DEBUG juju.environs.
Preview Diff
1 | === modified file 'agent/tools/tools_test.go' |
2 | --- agent/tools/tools_test.go 2014-01-21 05:25:30 +0000 |
3 | +++ agent/tools/tools_test.go 2014-03-19 02:34:50 +0000 |
4 | @@ -39,7 +39,7 @@ |
5 | // resulting slice has that prefix removed to keep the output short. |
6 | c.Assert(testbase.FindJujuCoreImports(c, "launchpad.net/juju-core/agent/tools"), |
7 | gc.DeepEquals, |
8 | - []string{"tools", "utils/set", "version"}) |
9 | + []string{"juju/arch", "tools", "utils/set", "version"}) |
10 | } |
11 | |
12 | const toolsFile = "downloaded-tools.txt" |
13 | |
14 | === modified file 'cmd/juju/bootstrap.go' |
15 | --- cmd/juju/bootstrap.go 2014-03-10 00:45:41 +0000 |
16 | +++ cmd/juju/bootstrap.go 2014-03-19 02:34:50 +0000 |
17 | @@ -13,15 +13,10 @@ |
18 | "launchpad.net/juju-core/charm" |
19 | "launchpad.net/juju-core/cmd" |
20 | "launchpad.net/juju-core/constraints" |
21 | - "launchpad.net/juju-core/environs" |
22 | "launchpad.net/juju-core/environs/bootstrap" |
23 | - "launchpad.net/juju-core/environs/config" |
24 | "launchpad.net/juju-core/environs/imagemetadata" |
25 | - "launchpad.net/juju-core/environs/sync" |
26 | "launchpad.net/juju-core/environs/tools" |
27 | "launchpad.net/juju-core/provider" |
28 | - "launchpad.net/juju-core/utils/set" |
29 | - "launchpad.net/juju-core/version" |
30 | ) |
31 | |
32 | const bootstrapDoc = ` |
33 | @@ -127,7 +122,7 @@ |
34 | c.UploadTools = true |
35 | } |
36 | if c.UploadTools { |
37 | - err = c.uploadTools(environ) |
38 | + err = bootstrap.UploadTools(environ, c.Constraints.Arch, true, c.Series...) |
39 | if err != nil { |
40 | return err |
41 | } |
42 | @@ -135,28 +130,6 @@ |
43 | return bootstrap.Bootstrap(ctx, environ, c.Constraints) |
44 | } |
45 | |
46 | -func (c *BootstrapCommand) uploadTools(environ environs.Environ) error { |
47 | - // Force version.Current, for consistency with subsequent upgrade-juju |
48 | - // (see UpgradeJujuCommand). |
49 | - forceVersion := uploadVersion(version.Current.Number, nil) |
50 | - cfg := environ.Config() |
51 | - series := getUploadSeries(cfg, c.Series) |
52 | - agenttools, err := sync.Upload(environ.Storage(), &forceVersion, series...) |
53 | - if err != nil { |
54 | - return err |
55 | - } |
56 | - cfg, err = cfg.Apply(map[string]interface{}{ |
57 | - "agent-version": agenttools.Version.Number.String(), |
58 | - }) |
59 | - if err == nil { |
60 | - err = environ.SetConfig(cfg) |
61 | - } |
62 | - if err != nil { |
63 | - return fmt.Errorf("failed to update environment configuration: %v", err) |
64 | - } |
65 | - return nil |
66 | -} |
67 | - |
68 | type seriesVar struct { |
69 | target *[]string |
70 | } |
71 | @@ -175,16 +148,3 @@ |
72 | func (v seriesVar) String() string { |
73 | return strings.Join(*v.target, ",") |
74 | } |
75 | - |
76 | -// getUploadSeries returns the supplied series with duplicates removed if |
77 | -// non-empty; otherwise it returns a default list of series we should |
78 | -// probably upload, based on cfg. |
79 | -func getUploadSeries(cfg *config.Config, series []string) []string { |
80 | - unique := set.NewStrings(series...) |
81 | - if unique.IsEmpty() { |
82 | - unique.Add(version.Current.Series) |
83 | - unique.Add(config.DefaultSeries) |
84 | - unique.Add(cfg.DefaultSeries()) |
85 | - } |
86 | - return unique.Values() |
87 | -} |
88 | |
89 | === modified file 'cmd/juju/bootstrap_test.go' |
90 | --- cmd/juju/bootstrap_test.go 2014-03-13 23:30:56 +0000 |
91 | +++ cmd/juju/bootstrap_test.go 2014-03-19 02:34:50 +0000 |
92 | @@ -26,6 +26,7 @@ |
93 | envtools "launchpad.net/juju-core/environs/tools" |
94 | ttesting "launchpad.net/juju-core/environs/tools/testing" |
95 | "launchpad.net/juju-core/errors" |
96 | + "launchpad.net/juju-core/juju/arch" |
97 | "launchpad.net/juju-core/provider/dummy" |
98 | coretesting "launchpad.net/juju-core/testing" |
99 | "launchpad.net/juju-core/testing/testbase" |
100 | @@ -81,27 +82,30 @@ |
101 | addVersionToSource bool |
102 | } |
103 | |
104 | +var noToolsAvailableMessage = "cannot upload bootstrap tools: Juju cannot bootstrap because no tools are available for your environment.*" |
105 | +var toolsNotFoundMessage = "cannot find bootstrap tools: tools not found" |
106 | + |
107 | var bootstrapRetryTests = []bootstrapRetryTest{{ |
108 | - info: "no tools uploaded, first check has no retries; no matching binary in source; sync fails with no second attempt", |
109 | + info: "no tools uploaded, first check has no retries; no matching binary in source; no second attempt", |
110 | expectedAllowRetry: []bool{false}, |
111 | - err: "cannot find bootstrap tools: no matching tools available", |
112 | + err: noToolsAvailableMessage, |
113 | version: "1.16.0-precise-amd64", |
114 | }, { |
115 | - info: "no tools uploaded, first check has no retries; matching binary in source; check after sync has retries", |
116 | + info: "no tools uploaded, first check has no retries; matching binary in source; check after upload has retries", |
117 | expectedAllowRetry: []bool{false, true}, |
118 | - err: "cannot find bootstrap tools: tools not found", |
119 | - version: "1.16.0-precise-amd64", |
120 | + err: toolsNotFoundMessage, |
121 | + version: "1.17.0-precise-amd64", // dev version to force upload |
122 | addVersionToSource: true, |
123 | }, { |
124 | info: "no tools uploaded, first check has no retries; no matching binary in source; check after upload has retries", |
125 | expectedAllowRetry: []bool{false, true}, |
126 | - err: "cannot find bootstrap tools: tools not found", |
127 | + err: toolsNotFoundMessage, |
128 | version: "1.15.1-precise-amd64", // dev version to force upload |
129 | }, { |
130 | info: "new tools uploaded, so we want to allow retries to give them a chance at showing up", |
131 | args: []string{"--upload-tools"}, |
132 | expectedAllowRetry: []bool{true}, |
133 | - err: "cannot find bootstrap tools: no matching tools available", |
134 | + err: noToolsAvailableMessage, |
135 | }} |
136 | |
137 | // Test test checks that bootstrap calls FindTools with the expected allowRetry flag. |
138 | @@ -141,7 +145,8 @@ |
139 | _, errc := runCommand(nullContext(), new(BootstrapCommand), test.args...) |
140 | err := <-errc |
141 | c.Check(findToolsRetryValues, gc.DeepEquals, test.expectedAllowRetry) |
142 | - c.Check(err, gc.ErrorMatches, test.err) |
143 | + stripped := strings.Replace(err.Error(), "\n", "", -1) |
144 | + c.Check(stripped, gc.Matches, test.err) |
145 | } |
146 | |
147 | // mockUploadTools simulates the effect of tools.Upload, but skips the time- |
148 | @@ -187,6 +192,7 @@ |
149 | // will be uploaded before running the test. |
150 | uploads []string |
151 | constraints constraints.Value |
152 | + hostArch string |
153 | } |
154 | |
155 | func (test bootstrapTest) run(c *gc.C) { |
156 | @@ -201,6 +207,14 @@ |
157 | defer func() { version.Current = origVersion }() |
158 | } |
159 | |
160 | + if test.hostArch != "" { |
161 | + origVersion := arch.HostArch |
162 | + arch.HostArch = func() string { |
163 | + return test.hostArch |
164 | + } |
165 | + defer func() { arch.HostArch = origVersion }() |
166 | + } |
167 | + |
168 | uploadCount := len(test.uploads) |
169 | if uploadCount == 0 { |
170 | usefulVersion := version.Current |
171 | @@ -210,6 +224,17 @@ |
172 | |
173 | // Run command and check for uploads. |
174 | opc, errc := runCommand(nullContext(), new(BootstrapCommand), test.args...) |
175 | + // Check for remaining operations/errors. |
176 | + if test.err != "" { |
177 | + err := <-errc |
178 | + stripped := strings.Replace(err.Error(), "\n", "", -1) |
179 | + c.Check(stripped, gc.Matches, test.err) |
180 | + return |
181 | + } |
182 | + if !c.Check(<-errc, gc.IsNil) { |
183 | + return |
184 | + } |
185 | + |
186 | if uploadCount > 0 { |
187 | for i := 0; i < uploadCount; i++ { |
188 | c.Check((<-opc).(dummy.OpPutFile).Env, gc.Equals, "peckham") |
189 | @@ -227,15 +252,6 @@ |
190 | c.Check(found, gc.Equals, true) |
191 | } |
192 | } |
193 | - |
194 | - // Check for remaining operations/errors. |
195 | - if test.err != "" { |
196 | - c.Check(<-errc, gc.ErrorMatches, test.err) |
197 | - return |
198 | - } |
199 | - if !c.Check(<-errc, gc.IsNil) { |
200 | - return |
201 | - } |
202 | if len(test.uploads) > 0 { |
203 | indexFile := (<-opc).(dummy.OpPutFile) |
204 | c.Check(indexFile.FileName, gc.Equals, "tools/streams/v1/index.json") |
205 | @@ -302,6 +318,17 @@ |
206 | "1.2.3.1-precise-amd64", // from environs/config.DefaultSeries |
207 | }, |
208 | }, { |
209 | + info: "--upload-tools uses arch from constraint if it matches current version", |
210 | + version: "1.3.3-saucy-ppc64", |
211 | + hostArch: "ppc64", |
212 | + args: []string{"--upload-tools", "--constraints", "arch=ppc64"}, |
213 | + uploads: []string{ |
214 | + "1.3.3.1-saucy-ppc64", // from version.Current |
215 | + "1.3.3.1-raring-ppc64", // from env.Config().DefaultSeries() |
216 | + "1.3.3.1-precise-ppc64", // from environs/config.DefaultSeries |
217 | + }, |
218 | + constraints: constraints.MustParse("arch=ppc64"), |
219 | +}, { |
220 | info: "--upload-tools only uploads each file once", |
221 | version: "1.2.3-precise-amd64", |
222 | args: []string{"--upload-tools"}, |
223 | @@ -315,6 +342,17 @@ |
224 | args: []string{"--upload-tools", "--series", "ping,ping,pong"}, |
225 | err: `invalid series "ping"`, |
226 | }, { |
227 | + info: "--upload-tools rejects mismatched arch", |
228 | + version: "1.3.3-saucy-amd64", |
229 | + args: []string{"--upload-tools", "--constraints", "arch=ppc64"}, |
230 | + err: `cannot build tools for "ppc64" using a machine running on "amd64"`, |
231 | +}, { |
232 | + info: "--upload-tools rejects non-supported arch", |
233 | + version: "1.3.3-saucy-arm64", |
234 | + hostArch: "arm64", |
235 | + args: []string{"--upload-tools"}, |
236 | + err: `environment "peckham" of type dummy does not support instances running on "arm64"`, |
237 | +}, { |
238 | info: "--upload-tools always bumps build number", |
239 | version: "1.2.3.4-raring-amd64", |
240 | args: []string{"--upload-tools"}, |
241 | @@ -493,7 +531,8 @@ |
242 | s.setupAutoUploadTest(c, "1.8.3", "precise") |
243 | _, errc := runCommand(nullContext(), new(BootstrapCommand)) |
244 | err := <-errc |
245 | - c.Assert(err, gc.ErrorMatches, "cannot find bootstrap tools: no matching tools available") |
246 | + stripped := strings.Replace(err.Error(), "\n", "", -1) |
247 | + c.Assert(stripped, gc.Matches, noToolsAvailableMessage) |
248 | } |
249 | |
250 | func (s *BootstrapSuite) TestMissingToolsError(c *gc.C) { |
251 | @@ -503,7 +542,7 @@ |
252 | c.Assert(code, gc.Equals, 1) |
253 | errText := context.Stderr.(*bytes.Buffer).String() |
254 | errText = strings.Replace(errText, "\n", "", -1) |
255 | - expectedErrText := "error: cannot find bootstrap tools: no matching tools available" |
256 | + expectedErrText := "error: cannot upload bootstrap tools: Juju cannot bootstrap because no tools are available for your environment.*" |
257 | c.Assert(errText, gc.Matches, expectedErrText) |
258 | } |
259 | |
260 | @@ -519,7 +558,7 @@ |
261 | c.Assert(code, gc.Equals, 1) |
262 | errText := context.Stderr.(*bytes.Buffer).String() |
263 | errText = strings.Replace(errText, "\n", "", -1) |
264 | - expectedErrText := "error: cannot find bootstrap tools: an error" |
265 | + expectedErrText := "error: cannot upload bootstrap tools: an error" |
266 | c.Assert(errText, gc.Matches, expectedErrText) |
267 | } |
268 | |
269 | |
270 | === modified file 'cmd/juju/upgradejuju.go' |
271 | --- cmd/juju/upgradejuju.go 2014-03-13 10:40:55 +0000 |
272 | +++ cmd/juju/upgradejuju.go 2014-03-19 02:34:50 +0000 |
273 | @@ -13,6 +13,7 @@ |
274 | |
275 | "launchpad.net/juju-core/cmd" |
276 | "launchpad.net/juju-core/environs" |
277 | + "launchpad.net/juju-core/environs/bootstrap" |
278 | "launchpad.net/juju-core/environs/config" |
279 | "launchpad.net/juju-core/environs/storage" |
280 | "launchpad.net/juju-core/environs/sync" |
281 | @@ -141,7 +142,7 @@ |
282 | return err |
283 | } |
284 | if c.UploadTools { |
285 | - series := getUploadSeries(cfg, c.Series) |
286 | + series := bootstrap.SeriesToUpload(cfg, c.Series) |
287 | if err := context.uploadTools(series); err != nil { |
288 | return err |
289 | } |
290 | @@ -395,7 +396,7 @@ |
291 | return err |
292 | } |
293 | if c.UploadTools { |
294 | - series := getUploadSeries(cfg, c.Series) |
295 | + series := bootstrap.SeriesToUpload(cfg, c.Series) |
296 | if err := context.uploadTools1dot16(env.Storage(), series); err != nil { |
297 | return err |
298 | } |
299 | |
300 | === modified file 'cmd/package_test.go' |
301 | --- cmd/package_test.go 2014-01-22 22:48:54 +0000 |
302 | +++ cmd/package_test.go 2014-03-19 02:34:50 +0000 |
303 | @@ -23,5 +23,5 @@ |
304 | // really be moved into "juju/osenv". |
305 | c.Assert(testbase.FindJujuCoreImports(c, "launchpad.net/juju-core/cmd"), |
306 | gc.DeepEquals, |
307 | - []string{"juju/osenv", "names", "version"}) |
308 | + []string{"juju/arch", "juju/osenv", "names", "version"}) |
309 | } |
310 | |
311 | === modified file 'environs/bootstrap/bootstrap_test.go' |
312 | --- environs/bootstrap/bootstrap_test.go 2014-03-03 10:10:44 +0000 |
313 | +++ environs/bootstrap/bootstrap_test.go 2014-03-19 02:34:50 +0000 |
314 | @@ -5,6 +5,7 @@ |
315 | |
316 | import ( |
317 | "fmt" |
318 | + "strings" |
319 | stdtesting "testing" |
320 | |
321 | gc "launchpad.net/gocheck" |
322 | @@ -14,10 +15,13 @@ |
323 | "launchpad.net/juju-core/environs/bootstrap" |
324 | "launchpad.net/juju-core/environs/config" |
325 | "launchpad.net/juju-core/environs/configstore" |
326 | + "launchpad.net/juju-core/environs/filestorage" |
327 | "launchpad.net/juju-core/environs/simplestreams" |
328 | "launchpad.net/juju-core/environs/storage" |
329 | + "launchpad.net/juju-core/environs/sync" |
330 | envtesting "launchpad.net/juju-core/environs/testing" |
331 | envtools "launchpad.net/juju-core/environs/tools" |
332 | + "launchpad.net/juju-core/juju/arch" |
333 | "launchpad.net/juju-core/provider/dummy" |
334 | coretesting "launchpad.net/juju-core/testing" |
335 | "launchpad.net/juju-core/testing/testbase" |
336 | @@ -54,7 +58,7 @@ |
337 | } |
338 | |
339 | func (s *bootstrapSuite) TestBootstrapNeedsSettings(c *gc.C) { |
340 | - env := newEnviron("bar", noKeysDefined) |
341 | + env := newEnviron("bar", noKeysDefined, nil) |
342 | s.setDummyStorage(c, env) |
343 | fixEnv := func(key string, value interface{}) { |
344 | cfg, err := env.Config().Apply(map[string]interface{}{ |
345 | @@ -88,7 +92,7 @@ |
346 | } |
347 | |
348 | func (s *bootstrapSuite) TestBootstrapEmptyConstraints(c *gc.C) { |
349 | - env := newEnviron("foo", useDefaultKeys) |
350 | + env := newEnviron("foo", useDefaultKeys, nil) |
351 | s.setDummyStorage(c, env) |
352 | err := bootstrap.Bootstrap(coretesting.Context(c), env, constraints.Value{}) |
353 | c.Assert(err, gc.IsNil) |
354 | @@ -97,7 +101,7 @@ |
355 | } |
356 | |
357 | func (s *bootstrapSuite) TestBootstrapSpecifiedConstraints(c *gc.C) { |
358 | - env := newEnviron("foo", useDefaultKeys) |
359 | + env := newEnviron("foo", useDefaultKeys, nil) |
360 | s.setDummyStorage(c, env) |
361 | cons := constraints.MustParse("cpu-cores=2 mem=4G") |
362 | err := bootstrap.Bootstrap(coretesting.Context(c), env, cons) |
363 | @@ -135,10 +139,18 @@ |
364 | DefaultSeries: "precise", |
365 | Arch: "i386", |
366 | Expect: []version.Binary{envtesting.V120p32}, |
367 | + }, { |
368 | + Info: "dev cli has different arch to available", |
369 | + Available: envtesting.V1all, |
370 | + CliVersion: envtesting.V310qppc64, |
371 | + DefaultSeries: "precise", |
372 | + Expect: []version.Binary{envtesting.V3101qppc64}, |
373 | }} |
374 | |
375 | func (s *bootstrapSuite) TestBootstrapTools(c *gc.C) { |
376 | allTests := append(envtesting.BootstrapToolsTests, bootstrapSetAgentVersionTests...) |
377 | + // version.Current is set in the loop so ensure it is restored later. |
378 | + s.PatchValue(&version.Current, version.Current) |
379 | for i, test := range allTests { |
380 | c.Logf("\ntest %d: %s", i, test.Info) |
381 | dummy.Reset() |
382 | @@ -166,8 +178,9 @@ |
383 | cons = constraints.MustParse("arch=" + test.Arch) |
384 | } |
385 | err = bootstrap.Bootstrap(coretesting.Context(c), env, cons) |
386 | - if test.Err != nil { |
387 | - c.Check(err, gc.ErrorMatches, ".*"+test.Err.Error()) |
388 | + if test.Err != "" { |
389 | + stripped := strings.Replace(err.Error(), "\n", "", -1) |
390 | + c.Check(stripped, gc.Matches, ".*"+stripped) |
391 | continue |
392 | } else { |
393 | c.Check(err, gc.IsNil) |
394 | @@ -185,7 +198,7 @@ |
395 | } |
396 | |
397 | func (s *bootstrapSuite) TestBootstrapNoTools(c *gc.C) { |
398 | - env := newEnviron("foo", useDefaultKeys) |
399 | + env := newEnviron("foo", useDefaultKeys, nil) |
400 | s.setDummyStorage(c, env) |
401 | envtesting.RemoveFakeTools(c, env.Storage()) |
402 | err := bootstrap.Bootstrap(coretesting.Context(c), env, constraints.Value{}) |
403 | @@ -194,12 +207,182 @@ |
404 | c.Assert(err, gc.IsNil) |
405 | } |
406 | |
407 | +func (s *bootstrapSuite) TestEnsureToolsAvailabilityIncompatibleHostArch(c *gc.C) { |
408 | + // Host runs amd64, want ppc64 tools. |
409 | + s.PatchValue(&arch.HostArch, func() string { |
410 | + return "amd64" |
411 | + }) |
412 | + env := newEnviron("foo", useDefaultKeys, nil) |
413 | + s.setDummyStorage(c, env) |
414 | + envtesting.RemoveFakeTools(c, env.Storage()) |
415 | + arch := "ppc64" |
416 | + _, err := bootstrap.EnsureToolsAvailability(env, env.Config().DefaultSeries(), &arch) |
417 | + c.Assert(err, gc.NotNil) |
418 | + stripped := strings.Replace(err.Error(), "\n", "", -1) |
419 | + c.Assert(stripped, |
420 | + gc.Matches, |
421 | + `cannot upload bootstrap tools: cannot build tools for "ppc64" using a machine running on "amd64"`) |
422 | +} |
423 | + |
424 | +func (s *bootstrapSuite) TestEnsureToolsAvailabilityIncompatibleTargetArch(c *gc.C) { |
425 | + // Host runs ppc64, environment only supports amd64, arm64. |
426 | + s.PatchValue(&arch.HostArch, func() string { |
427 | + return "ppc64" |
428 | + }) |
429 | + env := newEnviron("foo", useDefaultKeys, nil) |
430 | + s.setDummyStorage(c, env) |
431 | + envtesting.RemoveFakeTools(c, env.Storage()) |
432 | + _, err := bootstrap.EnsureToolsAvailability(env, env.Config().DefaultSeries(), nil) |
433 | + c.Assert(err, gc.NotNil) |
434 | + stripped := strings.Replace(err.Error(), "\n", "", -1) |
435 | + c.Assert(stripped, |
436 | + gc.Matches, |
437 | + `cannot upload bootstrap tools: environment "foo" of type dummy does not support instances running on "ppc64"`) |
438 | +} |
439 | + |
440 | +func (s *bootstrapSuite) TestEnsureToolsAvailabilityAgentVersionAlreadySet(c *gc.C) { |
441 | + // Can't upload tools if agent version already set. |
442 | + env := newEnviron("foo", useDefaultKeys, map[string]interface{}{"agent-version": "1.16.0"}) |
443 | + s.setDummyStorage(c, env) |
444 | + envtesting.RemoveFakeTools(c, env.Storage()) |
445 | + _, err := bootstrap.EnsureToolsAvailability(env, env.Config().DefaultSeries(), nil) |
446 | + c.Assert(err, gc.NotNil) |
447 | + stripped := strings.Replace(err.Error(), "\n", "", -1) |
448 | + c.Assert(stripped, |
449 | + gc.Matches, |
450 | + "cannot upload bootstrap tools: Juju cannot bootstrap because no tools are available for your environment.*") |
451 | +} |
452 | + |
453 | +func (s *bootstrapSuite) TestEnsureToolsAvailabilityNonDevVersion(c *gc.C) { |
454 | + // Can't upload tools for released versions. |
455 | + s.PatchValue(&version.Current, version.MustParseBinary("1.18.0-trusty-arm64")) |
456 | + env := newEnviron("foo", useDefaultKeys, nil) |
457 | + s.setDummyStorage(c, env) |
458 | + envtesting.RemoveFakeTools(c, env.Storage()) |
459 | + _, err := bootstrap.EnsureToolsAvailability(env, env.Config().DefaultSeries(), nil) |
460 | + c.Assert(err, gc.NotNil) |
461 | + stripped := strings.Replace(err.Error(), "\n", "", -1) |
462 | + c.Assert(stripped, |
463 | + gc.Matches, |
464 | + "cannot upload bootstrap tools: Juju cannot bootstrap because no tools are available for your environment.*") |
465 | +} |
466 | + |
467 | +// getMockBuildTools returns a sync.BuildToolsTarballFunc implementation which generates |
468 | +// a fake tools tarball. |
469 | +func (s *bootstrapSuite) getMockBuildTools(c *gc.C) sync.BuildToolsTarballFunc { |
470 | + toolsDir := c.MkDir() |
471 | + return func(forceVersion *version.Number) (*sync.BuiltTools, error) { |
472 | + // UploadFakeToolsVersions requires a storage to write to. |
473 | + stor, err := filestorage.NewFileStorageWriter(toolsDir) |
474 | + c.Assert(err, gc.IsNil) |
475 | + vers := version.Current |
476 | + if forceVersion != nil { |
477 | + vers.Number = *forceVersion |
478 | + } |
479 | + versions := []version.Binary{vers} |
480 | + uploadedTools, err := envtesting.UploadFakeToolsVersions(stor, versions...) |
481 | + c.Assert(err, gc.IsNil) |
482 | + agentTools := uploadedTools[0] |
483 | + return &sync.BuiltTools{ |
484 | + Dir: toolsDir, |
485 | + StorageName: envtools.StorageName(vers), |
486 | + Version: vers, |
487 | + Size: agentTools.Size, |
488 | + Sha256Hash: agentTools.SHA256, |
489 | + }, nil |
490 | + } |
491 | +} |
492 | + |
493 | func (s *bootstrapSuite) TestEnsureToolsAvailability(c *gc.C) { |
494 | - env := newEnviron("foo", useDefaultKeys) |
495 | + existingToolsVersion := version.MustParseBinary("1.19.0-trusty-amd64") |
496 | + s.PatchValue(&version.Current, existingToolsVersion) |
497 | + env := newEnviron("foo", useDefaultKeys, nil) |
498 | + s.setDummyStorage(c, env) |
499 | + // At this point, as a result of setDummyStorage, env has tools for amd64 uploaded. |
500 | + // Set version.Current to be arm64 to simulate a different CLI version. |
501 | + cliVersion := version.Current |
502 | + cliVersion.Arch = "arm64" |
503 | + version.Current = cliVersion |
504 | + s.PatchValue(&sync.BuildToolsTarball, s.getMockBuildTools(c)) |
505 | + // Host runs arm64, environment supports arm64. |
506 | + s.PatchValue(&arch.HostArch, func() string { |
507 | + return "arm64" |
508 | + }) |
509 | + arch := "arm64" |
510 | + agentTools, err := bootstrap.EnsureToolsAvailability(env, env.Config().DefaultSeries(), &arch) |
511 | + c.Assert(err, gc.IsNil) |
512 | + c.Assert(agentTools, gc.HasLen, 1) |
513 | + expectedVers := version.Current |
514 | + expectedVers.Number.Build++ |
515 | + expectedVers.Series = env.Config().DefaultSeries() |
516 | + c.Assert(agentTools[0].Version, gc.DeepEquals, expectedVers) |
517 | +} |
518 | + |
519 | +func (s *bootstrapSuite) TestSeriesToUpload(c *gc.C) { |
520 | + vers := version.Current |
521 | + vers.Series = "quantal" |
522 | + s.PatchValue(&version.Current, vers) |
523 | + env := newEnviron("foo", useDefaultKeys, nil) |
524 | + cfg := env.Config() |
525 | + c.Assert(bootstrap.SeriesToUpload(cfg, nil), gc.DeepEquals, []string{"quantal", "precise"}) |
526 | + c.Assert(bootstrap.SeriesToUpload(cfg, []string{"quantal"}), gc.DeepEquals, []string{"quantal"}) |
527 | + env = newEnviron("foo", useDefaultKeys, map[string]interface{}{"default-series": "lucid"}) |
528 | + cfg = env.Config() |
529 | + c.Assert(bootstrap.SeriesToUpload(cfg, nil), gc.DeepEquals, []string{"quantal", "precise", "lucid"}) |
530 | +} |
531 | + |
532 | +func (s *bootstrapSuite) assertUploadTools(c *gc.C, vers version.Binary, allowRelease bool, errMessage string) { |
533 | + s.PatchValue(&version.Current, vers) |
534 | + // If we allow released tools to be uploaded, the build number is incremented so in that case |
535 | + // we need to ensure the environment is set up to allow dev tools to be used. |
536 | + env := newEnviron("foo", useDefaultKeys, map[string]interface{}{"development": allowRelease}) |
537 | s.setDummyStorage(c, env) |
538 | envtesting.RemoveFakeTools(c, env.Storage()) |
539 | - _, err := bootstrap.EnsureToolsAvailability(env, env.Config().DefaultSeries(), nil) |
540 | - c.Check(err, gc.ErrorMatches, ".*no tools available") |
541 | + |
542 | + // At this point, as a result of setDummyStorage, env has tools for amd64 uploaded. |
543 | + // Set version.Current to be arm64 to simulate a different CLI version. |
544 | + cliVersion := version.Current |
545 | + cliVersion.Arch = "arm64" |
546 | + version.Current = cliVersion |
547 | + s.PatchValue(&sync.BuildToolsTarball, s.getMockBuildTools(c)) |
548 | + // Host runs arm64, environment supports arm64. |
549 | + s.PatchValue(&arch.HostArch, func() string { |
550 | + return "arm64" |
551 | + }) |
552 | + arch := "arm64" |
553 | + err := bootstrap.UploadTools(env, &arch, allowRelease, "precise") |
554 | + if errMessage != "" { |
555 | + stripped := strings.Replace(err.Error(), "\n", "", -1) |
556 | + c.Assert(stripped, gc.Matches, errMessage) |
557 | + return |
558 | + } |
559 | + c.Assert(err, gc.IsNil) |
560 | + params := envtools.BootstrapToolsParams{ |
561 | + Arch: &arch, |
562 | + Series: "precise", |
563 | + } |
564 | + agentTools, err := envtools.FindBootstrapTools(env, params) |
565 | + c.Assert(err, gc.IsNil) |
566 | + c.Assert(agentTools, gc.HasLen, 1) |
567 | + expectedVers := vers |
568 | + expectedVers.Number.Build++ |
569 | + expectedVers.Series = "precise" |
570 | + c.Assert(agentTools[0].Version, gc.DeepEquals, expectedVers) |
571 | +} |
572 | + |
573 | +func (s *bootstrapSuite) TestUploadTools(c *gc.C) { |
574 | + vers := version.MustParseBinary("1.19.0-trusty-arm64") |
575 | + s.assertUploadTools(c, vers, false, "") |
576 | +} |
577 | + |
578 | +func (s *bootstrapSuite) TestUploadToolsReleaseVersionAllowed(c *gc.C) { |
579 | + vers := version.MustParseBinary("1.18.0-trusty-arm64") |
580 | + s.assertUploadTools(c, vers, true, "") |
581 | +} |
582 | + |
583 | +func (s *bootstrapSuite) TestUploadToolsReleaseVersionDisallowed(c *gc.C) { |
584 | + vers := version.MustParseBinary("1.18.0-trusty-arm64") |
585 | + s.assertUploadTools(c, vers, false, "Juju cannot bootstrap because no tools are available for your environment.*") |
586 | } |
587 | |
588 | type bootstrapEnviron struct { |
589 | @@ -222,8 +405,8 @@ |
590 | storage.NewStorageSimpleStreamsDataSource("cloud storage", e.Storage(), storage.BaseToolsPath)}, nil |
591 | } |
592 | |
593 | -func newEnviron(name string, defaultKeys bool) *bootstrapEnviron { |
594 | - m := dummy.SampleConfig() |
595 | +func newEnviron(name string, defaultKeys bool, extraAttrs map[string]interface{}) *bootstrapEnviron { |
596 | + m := dummy.SampleConfig().Merge(extraAttrs) |
597 | if !defaultKeys { |
598 | m = m.Delete( |
599 | "ca-cert", |
600 | @@ -273,3 +456,7 @@ |
601 | func (e *bootstrapEnviron) Storage() storage.Storage { |
602 | return e.storage |
603 | } |
604 | + |
605 | +func (e *bootstrapEnviron) SupportedArchitectures() ([]string, error) { |
606 | + return []string{"amd64", "arm64"}, nil |
607 | +} |
608 | |
609 | === modified file 'environs/bootstrap/synctools.go' |
610 | --- environs/bootstrap/synctools.go 2014-01-20 23:32:00 +0000 |
611 | +++ environs/bootstrap/synctools.go 2014-03-19 02:34:50 +0000 |
612 | @@ -8,9 +8,11 @@ |
613 | |
614 | "launchpad.net/juju-core/environs" |
615 | "launchpad.net/juju-core/environs/config" |
616 | + "launchpad.net/juju-core/environs/simplestreams" |
617 | "launchpad.net/juju-core/environs/sync" |
618 | envtools "launchpad.net/juju-core/environs/tools" |
619 | "launchpad.net/juju-core/errors" |
620 | + "launchpad.net/juju-core/juju/arch" |
621 | coretools "launchpad.net/juju-core/tools" |
622 | "launchpad.net/juju-core/utils/set" |
623 | "launchpad.net/juju-core/version" |
624 | @@ -21,46 +23,29 @@ |
625 | ` |
626 | |
627 | const noToolsNoUploadMessage = `Juju cannot bootstrap because no tools are available for your environment. |
628 | -In addition, no tools could be located to upload. |
629 | You may want to use the 'tools-metadata-url' configuration setting to specify the tools location. |
630 | ` |
631 | |
632 | -// syncOrUpload first attempts to synchronize tools from |
633 | -// the default tools source to the environment's storage. |
634 | -// |
635 | -// If synchronization fails due to no matching tools, |
636 | -// a development version of juju is running, and no |
637 | -// agent-version has been specified, then attempt to |
638 | -// build and upload local tools. |
639 | -func syncOrUpload(env environs.Environ, bootstrapSeries string) error { |
640 | - sctx := &sync.SyncContext{ |
641 | - Target: env.Storage(), |
642 | - } |
643 | - err := sync.SyncTools(sctx) |
644 | - if err == coretools.ErrNoMatches || err == envtools.ErrNoTools { |
645 | - if _, hasAgentVersion := env.Config().AgentVersion(); !hasAgentVersion && version.Current.IsDev() { |
646 | - logger.Warningf("no tools found, so attempting to build and upload new tools") |
647 | - if err = uploadTools(env, bootstrapSeries); err != nil { |
648 | - logger.Errorf("%s", noToolsMessage) |
649 | - return err |
650 | - } |
651 | - } else { |
652 | - logger.Errorf("%s", noToolsNoUploadMessage) |
653 | +// UploadTools uploads tools for the specified series and any other relevant series to |
654 | +// the environment storage, after which it sets the agent-version. |
655 | +func UploadTools(env environs.Environ, toolsArch *string, allowRelease bool, bootstrapSeries ...string) error { |
656 | + logger.Infof("checking that upload is possible") |
657 | + // Check the series are valid. |
658 | + for _, series := range bootstrapSeries { |
659 | + if _, err := simplestreams.SeriesVersion(series); err != nil { |
660 | + return err |
661 | } |
662 | } |
663 | - return err |
664 | -} |
665 | + // See that we are allowed to upload the tools. |
666 | + if err := validateUploadAllowed(env, toolsArch, allowRelease); err != nil { |
667 | + return err |
668 | + } |
669 | |
670 | -func uploadTools(env environs.Environ, bootstrapSeries string) error { |
671 | cfg := env.Config() |
672 | - uploadVersion := version.Current.Number |
673 | - uploadVersion.Build++ |
674 | - uploadSeries := set.NewStrings( |
675 | - bootstrapSeries, |
676 | - cfg.DefaultSeries(), |
677 | - config.DefaultSeries, |
678 | - ) |
679 | - tools, err := sync.Upload(env.Storage(), &uploadVersion, uploadSeries.Values()...) |
680 | + forceVersion := uploadVersion(version.Current.Number, nil) |
681 | + uploadSeries := SeriesToUpload(cfg, bootstrapSeries) |
682 | + logger.Infof("uploading tools for series %s", uploadSeries) |
683 | + tools, err := sync.Upload(env.Storage(), &forceVersion, uploadSeries...) |
684 | if err != nil { |
685 | return err |
686 | } |
687 | @@ -76,9 +61,74 @@ |
688 | return nil |
689 | } |
690 | |
691 | +// uploadVersion returns a copy of the supplied version with a build number |
692 | +// higher than any of the supplied tools that share its major, minor and patch. |
693 | +func uploadVersion(vers version.Number, existing coretools.List) version.Number { |
694 | + vers.Build++ |
695 | + for _, t := range existing { |
696 | + if t.Version.Major != vers.Major || t.Version.Minor != vers.Minor || t.Version.Patch != vers.Patch { |
697 | + continue |
698 | + } |
699 | + if t.Version.Build >= vers.Build { |
700 | + vers.Build = t.Version.Build + 1 |
701 | + } |
702 | + } |
703 | + return vers |
704 | +} |
705 | + |
706 | +// SeriesToUpload returns the supplied series with duplicates removed if |
707 | +// non-empty; otherwise it returns a default list of series we should |
708 | +// probably upload, based on cfg. |
709 | +func SeriesToUpload(cfg *config.Config, series []string) []string { |
710 | + unique := set.NewStrings(series...) |
711 | + if unique.IsEmpty() { |
712 | + unique.Add(version.Current.Series) |
713 | + unique.Add(config.DefaultSeries) |
714 | + unique.Add(cfg.DefaultSeries()) |
715 | + } |
716 | + return unique.Values() |
717 | +} |
718 | + |
719 | +// validateUploadAllowed returns an error if an attempt to upload tools should |
720 | +// not be allowed. |
721 | +func validateUploadAllowed(env environs.Environ, toolsArch *string, allowRelease bool) error { |
722 | + // First, check that there isn't already an agent version specified, and that we |
723 | + // are running a development version. |
724 | + if _, hasAgentVersion := env.Config().AgentVersion(); hasAgentVersion || (!allowRelease && !version.Current.IsDev()) { |
725 | + return fmt.Errorf(noToolsNoUploadMessage) |
726 | + } |
727 | + |
728 | + // Now check that the architecture for which we are setting up an |
729 | + // environment matches that from which we are bootstrapping. |
730 | + hostArch := arch.HostArch() |
731 | + // We can't build tools for a different architecture if one is specified. |
732 | + if toolsArch != nil && *toolsArch != hostArch { |
733 | + return fmt.Errorf("cannot build tools for %q using a machine running on %q", *toolsArch, hostArch) |
734 | + } |
735 | + // If no architecture is specified, ensure the target provider supports instances matching our architecture. |
736 | + supportedArchitectures, err := env.SupportedArchitectures() |
737 | + if err != nil { |
738 | + return fmt.Errorf( |
739 | + "no packaged tools available and cannot determine environment's supported architectures: %v", err) |
740 | + } |
741 | + archSupported := false |
742 | + for _, arch := range supportedArchitectures { |
743 | + if hostArch == arch { |
744 | + archSupported = true |
745 | + break |
746 | + } |
747 | + } |
748 | + if !archSupported { |
749 | + envType := env.Config().Type() |
750 | + return fmt.Errorf( |
751 | + "environment %q of type %s does not support instances running on %q", env.Name(), envType, hostArch) |
752 | + } |
753 | + return nil |
754 | +} |
755 | + |
756 | // EnsureToolsAvailability verifies the tools are available. If no tools are |
757 | // found, it will automatically synchronize them. |
758 | -func EnsureToolsAvailability(env environs.Environ, series string, arch *string) (coretools.List, error) { |
759 | +func EnsureToolsAvailability(env environs.Environ, series string, toolsArch *string) (coretools.List, error) { |
760 | cfg := env.Config() |
761 | var vers *version.Number |
762 | if agentVersion, ok := cfg.AgentVersion(); ok { |
763 | @@ -87,11 +137,11 @@ |
764 | |
765 | logger.Debugf( |
766 | "looking for bootstrap tools: series=%q, arch=%v, version=%v", |
767 | - series, arch, vers, |
768 | + series, toolsArch, vers, |
769 | ) |
770 | params := envtools.BootstrapToolsParams{ |
771 | Version: vers, |
772 | - Arch: arch, |
773 | + Arch: toolsArch, |
774 | Series: series, |
775 | // If vers.Build>0, the tools may have been uploaded in this session. |
776 | // Allow retries, so we wait until the storage has caught up. |
777 | @@ -104,17 +154,14 @@ |
778 | return nil, err |
779 | } |
780 | |
781 | - // No tools available, so synchronize. |
782 | - logger.Warningf("no tools available, attempting to retrieve from %v", envtools.DefaultBaseURL) |
783 | - if syncErr := syncOrUpload(env, series); syncErr != nil { |
784 | - // The target may have tools that don't match, so don't |
785 | - // return a misleading "no tools found" error. |
786 | - if syncErr != envtools.ErrNoTools { |
787 | - err = syncErr |
788 | - } |
789 | - return nil, fmt.Errorf("cannot find bootstrap tools: %v", err) |
790 | + // No tools available so our only hope is to build locally and upload. |
791 | + logger.Warningf("no prepackaged tools available") |
792 | + uploadSeries := SeriesToUpload(cfg, nil) |
793 | + if err := UploadTools(env, toolsArch, false, append(uploadSeries, series)...); err != nil { |
794 | + logger.Errorf("%s", noToolsMessage) |
795 | + return nil, fmt.Errorf("cannot upload bootstrap tools: %v", err) |
796 | } |
797 | - // TODO(axw) have syncOrUpload return the list of tools in the target, and use that. |
798 | + // TODO(axw) have uploadTools return the list of tools in the target, and use that. |
799 | params.AllowRetry = true |
800 | if toolsList, err = envtools.FindBootstrapTools(env, params); err != nil { |
801 | return nil, fmt.Errorf("cannot find bootstrap tools: %v", err) |
802 | |
803 | === modified file 'environs/interface.go' |
804 | --- environs/interface.go 2014-02-20 08:23:40 +0000 |
805 | +++ environs/interface.go 2014-03-19 02:34:50 +0000 |
806 | @@ -15,6 +15,14 @@ |
807 | "launchpad.net/juju-core/state/api" |
808 | ) |
809 | |
810 | +// EnvironCapability implements access to metadata about the capabilities |
811 | +// of an environment. |
812 | +type EnvironCapability interface { |
813 | + // SupportedArchitectures returns the image architectures which can |
814 | + // be hosted by this environment. |
815 | + SupportedArchitectures() ([]string, error) |
816 | +} |
817 | + |
818 | // A EnvironProvider represents a computing and storage provider. |
819 | type EnvironProvider interface { |
820 | // Prepare prepares an environment for use. Any additional |
821 | @@ -63,7 +71,7 @@ |
822 | Storage() storage.Storage |
823 | } |
824 | |
825 | -// ConfigGetter implements access to an environments configuration. |
826 | +// ConfigGetter implements access to an environment's configuration. |
827 | type ConfigGetter interface { |
828 | // Config returns the configuration data with which the Environ was created. |
829 | // Note that this is not necessarily current; the canonical location |
830 | @@ -116,6 +124,9 @@ |
831 | // ConfigGetter allows the retrieval of the configuration data. |
832 | ConfigGetter |
833 | |
834 | + // EnvironCapability allows access to this environment's capabilities. |
835 | + EnvironCapability |
836 | + |
837 | // SetConfig updates the Environ's configuration. |
838 | // |
839 | // Calls to SetConfig do not affect the configuration of |
840 | |
841 | === modified file 'environs/manual/init.go' |
842 | --- environs/manual/init.go 2014-03-17 03:45:21 +0000 |
843 | +++ environs/manual/init.go 2014-03-19 02:34:50 +0000 |
844 | @@ -57,10 +57,13 @@ |
845 | return provisioned, nil |
846 | } |
847 | |
848 | -// DetectSeriesAndHardwareCharacteristics detects the OS |
849 | +// Patch for testing. |
850 | +var DetectSeriesAndHardwareCharacteristics = detectSeriesAndHardwareCharacteristics |
851 | + |
852 | +// detectSeriesAndHardwareCharacteristics detects the OS |
853 | // series and hardware characteristics of the remote machine |
854 | // by connecting to the machine and executing a bash script. |
855 | -func DetectSeriesAndHardwareCharacteristics(host string) (hc instance.HardwareCharacteristics, series string, err error) { |
856 | +func detectSeriesAndHardwareCharacteristics(host string) (hc instance.HardwareCharacteristics, series string, err error) { |
857 | logger.Infof("Detecting series and characteristics on %s", host) |
858 | cmd := ssh.Command("ubuntu@"+host, []string{"/bin/bash"}, nil) |
859 | var stdout, stderr bytes.Buffer |
860 | @@ -76,10 +79,7 @@ |
861 | lines := strings.Split(stdout.String(), "\n") |
862 | series = strings.TrimSpace(lines[0]) |
863 | |
864 | - arch, err := arch.NormaliseArch(lines[1]) |
865 | - if err != nil { |
866 | - return hc, "", err |
867 | - } |
868 | + arch := arch.NormaliseArch(lines[1]) |
869 | hc.Arch = &arch |
870 | |
871 | // HardwareCharacteristics wants memory in megabytes, |
872 | |
873 | === modified file 'environs/testing/tools.go' |
874 | --- environs/testing/tools.go 2014-03-17 06:05:54 +0000 |
875 | +++ environs/testing/tools.go 2014-03-19 02:34:50 +0000 |
876 | @@ -269,6 +269,9 @@ |
877 | V220q64 = version.MustParseBinary("2.2.0-quantal-amd64") |
878 | V220all = []version.Binary{V220p64, V220p32, V220q64, V220q32} |
879 | VAll = append(V1all, V220all...) |
880 | + |
881 | + V310qppc64 = version.MustParseBinary("3.1.0-quantal-ppc64") |
882 | + V3101qppc64 = version.MustParseBinary("3.1.0.1-quantal-ppc64") |
883 | ) |
884 | |
885 | type BootstrapToolsTest struct { |
886 | @@ -280,15 +283,17 @@ |
887 | Development bool |
888 | Arch string |
889 | Expect []version.Binary |
890 | - Err error |
891 | + Err string |
892 | } |
893 | |
894 | +var noToolsMessage = "Juju cannot bootstrap because no tools are available for your environment.*" |
895 | + |
896 | var BootstrapToolsTests = []BootstrapToolsTest{ |
897 | { |
898 | Info: "no tools at all", |
899 | CliVersion: V100p64, |
900 | DefaultSeries: "precise", |
901 | - Err: coretools.ErrNoMatches, |
902 | + Err: noToolsMessage, |
903 | }, { |
904 | Info: "released cli: use newest compatible release version", |
905 | Available: VAll, |
906 | @@ -345,70 +350,70 @@ |
907 | Available: V220all, |
908 | CliVersion: V100p64, |
909 | DefaultSeries: "precise", |
910 | - Err: coretools.ErrNoMatches, |
911 | + Err: noToolsMessage, |
912 | }, { |
913 | Info: "released cli: minor upgrades bad", |
914 | Available: V120all, |
915 | CliVersion: V100p64, |
916 | DefaultSeries: "precise", |
917 | - Err: coretools.ErrNoMatches, |
918 | + Err: noToolsMessage, |
919 | }, { |
920 | Info: "released cli: major downgrades bad", |
921 | Available: V100Xall, |
922 | CliVersion: V220p64, |
923 | DefaultSeries: "precise", |
924 | - Err: coretools.ErrNoMatches, |
925 | + Err: noToolsMessage, |
926 | }, { |
927 | Info: "released cli: minor downgrades bad", |
928 | Available: V100Xall, |
929 | CliVersion: V120p64, |
930 | DefaultSeries: "quantal", |
931 | - Err: coretools.ErrNoMatches, |
932 | + Err: noToolsMessage, |
933 | }, { |
934 | Info: "released cli: no matching series", |
935 | Available: VAll, |
936 | CliVersion: V100p64, |
937 | DefaultSeries: "raring", |
938 | - Err: coretools.ErrNoMatches, |
939 | + Err: noToolsMessage, |
940 | }, { |
941 | Info: "released cli: no matching arches", |
942 | Available: VAll, |
943 | CliVersion: V100p64, |
944 | DefaultSeries: "precise", |
945 | Arch: "arm", |
946 | - Err: coretools.ErrNoMatches, |
947 | + Err: noToolsMessage, |
948 | }, { |
949 | Info: "released cli: specific bad major 1", |
950 | Available: VAll, |
951 | CliVersion: V220p64, |
952 | AgentVersion: V120, |
953 | DefaultSeries: "precise", |
954 | - Err: coretools.ErrNoMatches, |
955 | + Err: noToolsMessage, |
956 | }, { |
957 | Info: "released cli: specific bad major 2", |
958 | Available: VAll, |
959 | CliVersion: V120p64, |
960 | AgentVersion: V220, |
961 | DefaultSeries: "precise", |
962 | - Err: coretools.ErrNoMatches, |
963 | + Err: noToolsMessage, |
964 | }, { |
965 | Info: "released cli: ignore dev tools 1", |
966 | Available: V110all, |
967 | CliVersion: V100p64, |
968 | DefaultSeries: "precise", |
969 | - Err: coretools.ErrNoMatches, |
970 | + Err: noToolsMessage, |
971 | }, { |
972 | Info: "released cli: ignore dev tools 2", |
973 | Available: V110all, |
974 | CliVersion: V120p64, |
975 | DefaultSeries: "precise", |
976 | - Err: coretools.ErrNoMatches, |
977 | + Err: noToolsMessage, |
978 | }, { |
979 | Info: "released cli: ignore dev tools 3", |
980 | Available: []version.Binary{V1001p64}, |
981 | CliVersion: V100p64, |
982 | DefaultSeries: "precise", |
983 | - Err: coretools.ErrNoMatches, |
984 | + Err: noToolsMessage, |
985 | }, { |
986 | Info: "released cli with dev setting respects agent-version", |
987 | Available: VAll, |
988 | |
989 | === modified file 'environs/tools/tools_test.go' |
990 | --- environs/tools/tools_test.go 2014-03-13 07:54:56 +0000 |
991 | +++ environs/tools/tools_test.go 2014-03-19 02:34:50 +0000 |
992 | @@ -240,7 +240,7 @@ |
993 | Arch: &test.Arch, |
994 | } |
995 | actual, err := envtools.FindBootstrapTools(s.env, params) |
996 | - if test.Err != nil { |
997 | + if test.Err != "" { |
998 | if len(actual) > 0 { |
999 | c.Logf(actual.String()) |
1000 | } |
1001 | |
1002 | === modified file 'juju/arch/arch.go' |
1003 | --- juju/arch/arch.go 2014-03-17 03:38:08 +0000 |
1004 | +++ juju/arch/arch.go 2014-03-19 02:34:50 +0000 |
1005 | @@ -4,11 +4,9 @@ |
1006 | package arch |
1007 | |
1008 | import ( |
1009 | - "fmt" |
1010 | "regexp" |
1011 | + "runtime" |
1012 | "strings" |
1013 | - |
1014 | - "launchpad.net/juju-core/utils" |
1015 | ) |
1016 | |
1017 | // The following constants define the machine architectures supported by Juju. |
1018 | @@ -36,32 +34,32 @@ |
1019 | arch string |
1020 | }{ |
1021 | {regexp.MustCompile("amd64|x86_64"), AMD64}, |
1022 | - {regexp.MustCompile("i[3-9]86"), I386}, |
1023 | + {regexp.MustCompile("i?[3-9]86"), I386}, |
1024 | {regexp.MustCompile("armv.*"), ARM}, |
1025 | {regexp.MustCompile("aarch64"), ARM64}, |
1026 | {regexp.MustCompile("ppc64el|ppc64le"), PPC64}, |
1027 | } |
1028 | |
1029 | +// Override for testing. |
1030 | +var HostArch = hostArch |
1031 | + |
1032 | // HostArch returns the Juju architecture of the machine on which it is run. |
1033 | -func HostArch() (string, error) { |
1034 | - rawArch, err := utils.RunCommand("uname", "-m") |
1035 | - if err != nil { |
1036 | - return "", err |
1037 | - } |
1038 | - return NormaliseArch(rawArch) |
1039 | +func hostArch() string { |
1040 | + return NormaliseArch(runtime.GOARCH) |
1041 | } |
1042 | |
1043 | -// NormaliseArch returns the Juju architecture corresponding to the |
1044 | -// output of `uname -m`. |
1045 | -func NormaliseArch(rawArch string) (string, error) { |
1046 | +// NormaliseArch returns the Juju architecture corresponding to a machine's |
1047 | +// reported architecture. The Juju architecture is used to filter simple |
1048 | +// streams lookup of tools and images. |
1049 | +func NormaliseArch(rawArch string) string { |
1050 | rawArch = strings.TrimSpace(rawArch) |
1051 | for _, re := range archREs { |
1052 | if re.Match([]byte(rawArch)) { |
1053 | - return re.arch, nil |
1054 | + return re.arch |
1055 | break |
1056 | } |
1057 | } |
1058 | - return "", fmt.Errorf("unrecognised architecture: %s", rawArch) |
1059 | + return rawArch |
1060 | } |
1061 | |
1062 | // IsSupportedArch returns true if arch is one supported by Juju. |
1063 | |
1064 | === modified file 'juju/arch/arch_test.go' |
1065 | --- juju/arch/arch_test.go 2014-03-17 03:45:21 +0000 |
1066 | +++ juju/arch/arch_test.go 2014-03-19 02:34:50 +0000 |
1067 | @@ -18,8 +18,7 @@ |
1068 | var _ = gc.Suite(&archSuite{}) |
1069 | |
1070 | func (s *archSuite) TestHostArch(c *gc.C) { |
1071 | - a, err := arch.HostArch() |
1072 | - c.Assert(err, gc.IsNil) |
1073 | + a := arch.HostArch() |
1074 | c.Assert(arch.IsSupportedArch(a), jc.IsTrue) |
1075 | } |
1076 | |
1077 | @@ -28,9 +27,10 @@ |
1078 | raw string |
1079 | arch string |
1080 | }{ |
1081 | - {"invalid", ""}, |
1082 | + {"windows", "windows"}, |
1083 | {"amd64", "amd64"}, |
1084 | {"x86_64", "amd64"}, |
1085 | + {"386", "i386"}, |
1086 | {"i386", "i386"}, |
1087 | {"i486", "i386"}, |
1088 | {"armv", "arm"}, |
1089 | @@ -39,12 +39,8 @@ |
1090 | {"ppc64el", "ppc64"}, |
1091 | {"ppc64le", "ppc64"}, |
1092 | } { |
1093 | - arch, err := arch.NormaliseArch(test.raw) |
1094 | - if test.arch == "" { |
1095 | - c.Check(err, gc.ErrorMatches, "unrecognised architecture:.*") |
1096 | - } else { |
1097 | - c.Check(arch, gc.Equals, test.arch) |
1098 | - } |
1099 | + arch := arch.NormaliseArch(test.raw) |
1100 | + c.Check(arch, gc.Equals, test.arch) |
1101 | } |
1102 | } |
1103 | |
1104 | |
1105 | === modified file 'provider/azure/environ.go' |
1106 | --- provider/azure/environ.go 2014-03-17 03:45:21 +0000 |
1107 | +++ provider/azure/environ.go 2014-03-19 02:34:50 +0000 |
1108 | @@ -302,9 +302,6 @@ |
1109 | return req, nil |
1110 | } |
1111 | |
1112 | -// supportedArches lists the CPU architectures supported by Azure. |
1113 | -var supportedArches = []string{arch.AMD64, arch.I386} |
1114 | - |
1115 | // newHostedService creates a hosted service. It will make up a unique name, |
1116 | // starting with the given prefix. |
1117 | func newHostedService(azure *gwacl.ManagementAPI, prefix string, affinityGroupName string, location string) (*gwacl.CreateHostedService, error) { |
1118 | @@ -322,6 +319,14 @@ |
1119 | return svc, nil |
1120 | } |
1121 | |
1122 | +// supportedArches lists the CPU architectures supported by Azure. |
1123 | +var supportedArches = []string{arch.AMD64, arch.I386} |
1124 | + |
1125 | +// SupportedArchitectures is specified on the EnvironCapability interface. |
1126 | +func (*azureEnviron) SupportedArchitectures() ([]string, error) { |
1127 | + return supportedArches, nil |
1128 | +} |
1129 | + |
1130 | // selectInstanceTypeAndImage returns the appropriate instance-type name and |
1131 | // the OS image name for launching a virtual machine with the given parameters. |
1132 | func (env *azureEnviron) selectInstanceTypeAndImage(cons constraints.Value, series, location string) (string, string, error) { |
1133 | |
1134 | === modified file 'provider/azure/environ_test.go' |
1135 | --- provider/azure/environ_test.go 2014-03-13 07:54:56 +0000 |
1136 | +++ provider/azure/environ_test.go 2014-03-19 02:34:50 +0000 |
1137 | @@ -163,6 +163,13 @@ |
1138 | return gwacl.PatchManagementAPIResponses(responses) |
1139 | } |
1140 | |
1141 | +func (*environSuite) TestSupportedArchitectures(c *gc.C) { |
1142 | + env := makeEnviron(c) |
1143 | + a, err := env.SupportedArchitectures() |
1144 | + c.Assert(err, gc.IsNil) |
1145 | + c.Assert(a, gc.DeepEquals, []string{"amd64", "i386"}) |
1146 | +} |
1147 | + |
1148 | func (suite *environSuite) TestGetEnvPrefixContainsEnvName(c *gc.C) { |
1149 | env := makeEnviron(c) |
1150 | c.Check(strings.Contains(env.getEnvPrefix(), env.Name()), jc.IsTrue) |
1151 | |
1152 | === modified file 'provider/common/bootstrap.go' |
1153 | --- provider/common/bootstrap.go 2014-03-17 08:25:06 +0000 |
1154 | +++ provider/common/bootstrap.go 2014-03-19 02:34:50 +0000 |
1155 | @@ -41,6 +41,12 @@ |
1156 | var inst instance.Instance |
1157 | defer func() { handleBootstrapError(err, ctx, inst, env) }() |
1158 | |
1159 | + // First thing, ensure we have tools otherwise there's no point. |
1160 | + selectedTools, err := EnsureBootstrapTools(env, env.Config().DefaultSeries(), cons.Arch) |
1161 | + if err != nil { |
1162 | + return err |
1163 | + } |
1164 | + |
1165 | // Get the bootstrap SSH client. Do this early, so we know |
1166 | // not to bother with any of the below if we can't finish the job. |
1167 | client := ssh.DefaultClient |
1168 | @@ -64,11 +70,6 @@ |
1169 | } |
1170 | machineConfig := environs.NewBootstrapMachineConfig(stateFileURL, privateKey) |
1171 | |
1172 | - selectedTools, err := EnsureBootstrapTools(env, env.Config().DefaultSeries(), cons.Arch) |
1173 | - if err != nil { |
1174 | - return err |
1175 | - } |
1176 | - |
1177 | fmt.Fprintln(ctx.GetStderr(), "Launching instance") |
1178 | inst, hw, err := env.StartInstance(environs.StartInstanceParams{ |
1179 | Constraints: cons, |
1180 | |
1181 | === modified file 'provider/common/bootstrap_test.go' |
1182 | --- provider/common/bootstrap_test.go 2014-03-13 07:54:56 +0000 |
1183 | +++ provider/common/bootstrap_test.go 2014-03-19 02:34:50 +0000 |
1184 | @@ -80,7 +80,7 @@ |
1185 | Storage: newStorage(s, c), |
1186 | putErr: fmt.Errorf("noes!"), |
1187 | } |
1188 | - env := &mockEnviron{storage: brokenStorage} |
1189 | + env := &mockEnviron{storage: brokenStorage, config: configGetter(c)} |
1190 | ctx := coretesting.Context(c) |
1191 | err := common.Bootstrap(ctx, env, constraints.Value{}) |
1192 | c.Assert(err, gc.ErrorMatches, "cannot create initial state file: noes!") |
1193 | |
1194 | === modified file 'provider/common/mock_test.go' |
1195 | --- provider/common/mock_test.go 2014-03-13 05:09:14 +0000 |
1196 | +++ provider/common/mock_test.go 2014-03-19 02:34:50 +0000 |
1197 | @@ -38,6 +38,10 @@ |
1198 | return "mock environment" |
1199 | } |
1200 | |
1201 | +func (*mockEnviron) SupportedArchitectures() ([]string, error) { |
1202 | + return []string{"amd64", "arm64"}, nil |
1203 | +} |
1204 | + |
1205 | func (env *mockEnviron) Storage() storage.Storage { |
1206 | return env.storage |
1207 | } |
1208 | |
1209 | === modified file 'provider/dummy/environs.go' |
1210 | --- provider/dummy/environs.go 2014-03-13 05:09:14 +0000 |
1211 | +++ provider/dummy/environs.go 2014-03-19 02:34:50 +0000 |
1212 | @@ -44,6 +44,7 @@ |
1213 | "launchpad.net/juju-core/environs/storage" |
1214 | "launchpad.net/juju-core/environs/tools" |
1215 | "launchpad.net/juju-core/instance" |
1216 | + "launchpad.net/juju-core/juju/arch" |
1217 | "launchpad.net/juju-core/names" |
1218 | "launchpad.net/juju-core/provider" |
1219 | "launchpad.net/juju-core/provider/common" |
1220 | @@ -524,6 +525,11 @@ |
1221 | return e.name |
1222 | } |
1223 | |
1224 | +// SupportedArchitectures is specified on the EnvironCapability interface. |
1225 | +func (*environ) SupportedArchitectures() ([]string, error) { |
1226 | + return []string{arch.AMD64, arch.PPC64}, nil |
1227 | +} |
1228 | + |
1229 | // GetImageSources returns a list of sources which are used to search for simplestreams image metadata. |
1230 | func (e *environ) GetImageSources() ([]simplestreams.DataSource, error) { |
1231 | return []simplestreams.DataSource{ |
1232 | |
1233 | === modified file 'provider/ec2/ec2.go' |
1234 | --- provider/ec2/ec2.go 2014-03-17 03:45:21 +0000 |
1235 | +++ provider/ec2/ec2.go 2014-03-19 02:34:50 +0000 |
1236 | @@ -337,6 +337,11 @@ |
1237 | return common.StateInfo(e) |
1238 | } |
1239 | |
1240 | +// SupportedArchitectures is specified on the EnvironCapability interface. |
1241 | +func (*environ) SupportedArchitectures() ([]string, error) { |
1242 | + return supportedArches, nil |
1243 | +} |
1244 | + |
1245 | // MetadataLookupParams returns parameters which are used to query simplestreams metadata. |
1246 | func (e *environ) MetadataLookupParams(region string) (*simplestreams.MetadataLookupParams, error) { |
1247 | if region == "" { |
1248 | |
1249 | === modified file 'provider/ec2/local_test.go' |
1250 | --- provider/ec2/local_test.go 2014-03-13 20:52:41 +0000 |
1251 | +++ provider/ec2/local_test.go 2014-03-19 02:34:50 +0000 |
1252 | @@ -399,6 +399,13 @@ |
1253 | c.Assert(strings.Contains(url, ec2.ControlBucketName(env)+"/tools"), jc.IsTrue) |
1254 | } |
1255 | |
1256 | +func (t *localServerSuite) TestSupportedArchitectures(c *gc.C) { |
1257 | + env := t.Prepare(c) |
1258 | + a, err := env.SupportedArchitectures() |
1259 | + c.Assert(err, gc.IsNil) |
1260 | + c.Assert(a, gc.DeepEquals, []string{"amd64", "i386"}) |
1261 | +} |
1262 | + |
1263 | // localNonUSEastSuite is similar to localServerSuite but the S3 mock server |
1264 | // behaves as if it is not in the us-east region. |
1265 | type localNonUSEastSuite struct { |
1266 | |
1267 | === modified file 'provider/joyent/environ.go' |
1268 | --- provider/joyent/environ.go 2014-03-10 00:10:29 +0000 |
1269 | +++ provider/joyent/environ.go 2014-03-19 02:34:50 +0000 |
1270 | @@ -10,6 +10,7 @@ |
1271 | "launchpad.net/juju-core/environs" |
1272 | "launchpad.net/juju-core/environs/config" |
1273 | "launchpad.net/juju-core/environs/storage" |
1274 | + "launchpad.net/juju-core/juju/arch" |
1275 | "launchpad.net/juju-core/provider/common" |
1276 | "launchpad.net/juju-core/state" |
1277 | "launchpad.net/juju-core/state/api" |
1278 | @@ -45,6 +46,11 @@ |
1279 | return providerInstance |
1280 | } |
1281 | |
1282 | +// SupportedArchitectures is specified on the EnvironCapability interface. |
1283 | +func (*environ) SupportedArchitectures() ([]string, error) { |
1284 | + return []string{arch.AMD64}, nil |
1285 | +} |
1286 | + |
1287 | func (env *environ) SetConfig(cfg *config.Config) error { |
1288 | env.lock.Lock() |
1289 | defer env.lock.Unlock() |
1290 | |
1291 | === modified file 'provider/local/environ.go' |
1292 | --- provider/local/environ.go 2014-03-18 03:51:15 +0000 |
1293 | +++ provider/local/environ.go 2014-03-19 02:34:50 +0000 |
1294 | @@ -33,6 +33,7 @@ |
1295 | "launchpad.net/juju-core/environs/storage" |
1296 | envtools "launchpad.net/juju-core/environs/tools" |
1297 | "launchpad.net/juju-core/instance" |
1298 | + "launchpad.net/juju-core/juju/arch" |
1299 | "launchpad.net/juju-core/juju/osenv" |
1300 | "launchpad.net/juju-core/provider/common" |
1301 | "launchpad.net/juju-core/state" |
1302 | @@ -70,6 +71,12 @@ |
1303 | storage.NewStorageSimpleStreamsDataSource("cloud storage", e.Storage(), storage.BaseToolsPath)}, nil |
1304 | } |
1305 | |
1306 | +// SupportedArchitectures is specified on the EnvironCapability interface. |
1307 | +func (*localEnviron) SupportedArchitectures() ([]string, error) { |
1308 | + localArch := arch.HostArch() |
1309 | + return []string{localArch}, nil |
1310 | +} |
1311 | + |
1312 | // Name is specified in the Environ interface. |
1313 | func (env *localEnviron) Name() string { |
1314 | return env.name |
1315 | |
1316 | === modified file 'provider/local/environ_test.go' |
1317 | --- provider/local/environ_test.go 2014-03-13 07:54:56 +0000 |
1318 | +++ provider/local/environ_test.go 2014-03-19 02:34:50 +0000 |
1319 | @@ -20,6 +20,7 @@ |
1320 | "launchpad.net/juju-core/environs/jujutest" |
1321 | envtesting "launchpad.net/juju-core/environs/testing" |
1322 | "launchpad.net/juju-core/environs/tools" |
1323 | + "launchpad.net/juju-core/juju/arch" |
1324 | "launchpad.net/juju-core/juju/osenv" |
1325 | "launchpad.net/juju-core/provider/local" |
1326 | "launchpad.net/juju-core/state/api/params" |
1327 | @@ -77,6 +78,18 @@ |
1328 | c.Assert(strings.Contains(url, "/tools"), jc.IsTrue) |
1329 | } |
1330 | |
1331 | +func (*environSuite) TestSupportedArchitectures(c *gc.C) { |
1332 | + testConfig := minimalConfig(c) |
1333 | + environ, err := local.Provider.Open(testConfig) |
1334 | + c.Assert(err, gc.IsNil) |
1335 | + c.Assert(err, gc.IsNil) |
1336 | + arches, err := environ.SupportedArchitectures() |
1337 | + c.Assert(err, gc.IsNil) |
1338 | + for _, a := range arches { |
1339 | + c.Assert(arch.IsSupportedArch(a), jc.IsTrue) |
1340 | + } |
1341 | +} |
1342 | + |
1343 | type localJujuTestSuite struct { |
1344 | baseProviderSuite |
1345 | jujutest.Tests |
1346 | |
1347 | === modified file 'provider/maas/environ.go' |
1348 | --- provider/maas/environ.go 2014-03-14 13:57:26 +0000 |
1349 | +++ provider/maas/environ.go 2014-03-19 02:34:50 +0000 |
1350 | @@ -22,6 +22,7 @@ |
1351 | "launchpad.net/juju-core/environs/storage" |
1352 | envtools "launchpad.net/juju-core/environs/tools" |
1353 | "launchpad.net/juju-core/instance" |
1354 | + "launchpad.net/juju-core/juju/arch" |
1355 | "launchpad.net/juju-core/provider/common" |
1356 | "launchpad.net/juju-core/state" |
1357 | "launchpad.net/juju-core/state/api" |
1358 | @@ -130,6 +131,12 @@ |
1359 | return nil |
1360 | } |
1361 | |
1362 | +// SupportedArchitectures is specified on the EnvironCapability interface. |
1363 | +func (*maasEnviron) SupportedArchitectures() ([]string, error) { |
1364 | + // TODO(wallyworld) - how to find out what architectures a MAAS environ supports |
1365 | + return arch.AllSupportedArches, nil |
1366 | +} |
1367 | + |
1368 | // getMAASClient returns a MAAS client object to use for a request, in a |
1369 | // lock-protected fashion. |
1370 | func (env *maasEnviron) getMAASClient() *gomaasapi.MAASObject { |
1371 | |
1372 | === modified file 'provider/maas/environ_test.go' |
1373 | --- provider/maas/environ_test.go 2013-12-06 11:00:58 +0000 |
1374 | +++ provider/maas/environ_test.go 2014-03-19 02:34:50 +0000 |
1375 | @@ -11,6 +11,7 @@ |
1376 | |
1377 | "launchpad.net/juju-core/environs/config" |
1378 | envtesting "launchpad.net/juju-core/environs/testing" |
1379 | + "launchpad.net/juju-core/juju/arch" |
1380 | "launchpad.net/juju-core/provider/maas" |
1381 | coretesting "launchpad.net/juju-core/testing" |
1382 | "launchpad.net/juju-core/testing/testbase" |
1383 | @@ -56,8 +57,6 @@ |
1384 | s.LoggingSuite.TearDownSuite(c) |
1385 | } |
1386 | |
1387 | -var _ = gc.Suite(&environSuite{}) |
1388 | - |
1389 | func getSimpleTestConfig(c *gc.C, extraAttrs coretesting.Attrs) *config.Config { |
1390 | attrs := coretesting.FakeConfig() |
1391 | attrs["type"] = "maas" |
1392 | @@ -195,3 +194,12 @@ |
1393 | c.Check(err, gc.IsNil) |
1394 | c.Check(env.Name(), gc.Equals, "testenv") |
1395 | } |
1396 | + |
1397 | +func (*environSuite) TestSupportedArchitectures(c *gc.C) { |
1398 | + cfg := getSimpleTestConfig(c, nil) |
1399 | + env, err := maas.NewEnviron(cfg) |
1400 | + c.Assert(err, gc.IsNil) |
1401 | + a, err := env.SupportedArchitectures() |
1402 | + c.Assert(err, gc.IsNil) |
1403 | + c.Assert(a, gc.DeepEquals, arch.AllSupportedArches) |
1404 | +} |
1405 | |
1406 | === modified file 'provider/maas/environ_whitebox_test.go' |
1407 | --- provider/maas/environ_whitebox_test.go 2014-03-13 07:54:56 +0000 |
1408 | +++ provider/maas/environ_whitebox_test.go 2014-03-19 02:34:50 +0000 |
1409 | @@ -9,6 +9,7 @@ |
1410 | "fmt" |
1411 | "io/ioutil" |
1412 | "net/url" |
1413 | + "strings" |
1414 | |
1415 | jc "github.com/juju/testing/checkers" |
1416 | gc "launchpad.net/gocheck" |
1417 | @@ -385,7 +386,10 @@ |
1418 | err = env.SetConfig(cfg) |
1419 | c.Assert(err, gc.IsNil) |
1420 | err = bootstrap.Bootstrap(coretesting.Context(c), env, constraints.Value{}) |
1421 | - c.Check(err, gc.ErrorMatches, "cannot find bootstrap tools.*") |
1422 | + stripped := strings.Replace(err.Error(), "\n", "", -1) |
1423 | + c.Check(stripped, |
1424 | + gc.Matches, |
1425 | + "cannot upload bootstrap tools: Juju cannot bootstrap because no tools are available for your environment.*") |
1426 | } |
1427 | |
1428 | func (suite *environSuite) TestBootstrapFailsIfNoNodes(c *gc.C) { |
1429 | |
1430 | === modified file 'provider/manual/environ.go' |
1431 | --- provider/manual/environ.go 2014-03-13 05:09:14 +0000 |
1432 | +++ provider/manual/environ.go 2014-03-19 02:34:50 +0000 |
1433 | @@ -89,6 +89,17 @@ |
1434 | return e.envConfig().Name() |
1435 | } |
1436 | |
1437 | +// SupportedArchitectures is specified on the EnvironCapability interface. |
1438 | +func (e *manualEnviron) SupportedArchitectures() ([]string, error) { |
1439 | + envConfig := e.envConfig() |
1440 | + host := envConfig.bootstrapHost() |
1441 | + hc, _, err := manual.DetectSeriesAndHardwareCharacteristics(host) |
1442 | + if err != nil { |
1443 | + return nil, err |
1444 | + } |
1445 | + return []string{*hc.Arch}, nil |
1446 | +} |
1447 | + |
1448 | func (e *manualEnviron) Bootstrap(ctx environs.BootstrapContext, cons constraints.Value) error { |
1449 | // Set "use-sshstorage" to false, so agents know not to use sshstorage. |
1450 | cfg, err := e.Config().Apply(map[string]interface{}{"use-sshstorage": false}) |
1451 | |
1452 | === modified file 'provider/manual/environ_test.go' |
1453 | --- provider/manual/environ_test.go 2014-03-13 07:54:56 +0000 |
1454 | +++ provider/manual/environ_test.go 2014-03-19 02:34:50 +0000 |
1455 | @@ -129,3 +129,13 @@ |
1456 | c.Assert(err, gc.IsNil) |
1457 | c.Assert(strings.Contains(url, "/tools"), jc.IsTrue) |
1458 | } |
1459 | + |
1460 | +func (s *environSuite) TestSupportedArchitectures(c *gc.C) { |
1461 | + s.PatchValue(&manual.DetectSeriesAndHardwareCharacteristics, func(host string) (instance.HardwareCharacteristics, string, error) { |
1462 | + c.Assert(host, gc.Equals, "hostname") |
1463 | + return instance.MustParseHardware("arch=arm64"), "precise", nil |
1464 | + }) |
1465 | + a, err := s.env.SupportedArchitectures() |
1466 | + c.Assert(err, gc.IsNil) |
1467 | + c.Assert(a, gc.DeepEquals, []string{"arm64"}) |
1468 | +} |
1469 | |
1470 | === modified file 'provider/openstack/local_test.go' |
1471 | --- provider/openstack/local_test.go 2014-03-13 07:54:56 +0000 |
1472 | +++ provider/openstack/local_test.go 2014-03-19 02:34:50 +0000 |
1473 | @@ -587,6 +587,13 @@ |
1474 | c.Assert(err, gc.IsNil) |
1475 | } |
1476 | |
1477 | +func (s *localServerSuite) TestSupportedArchitectures(c *gc.C) { |
1478 | + env := s.Open(c) |
1479 | + a, err := env.SupportedArchitectures() |
1480 | + c.Assert(err, gc.IsNil) |
1481 | + c.Assert(a, gc.DeepEquals, []string{"amd64", "arm", "arm64", "ppc64"}) |
1482 | +} |
1483 | + |
1484 | func (s *localServerSuite) TestFindImageBadDefaultImage(c *gc.C) { |
1485 | // Prevent falling over to the public datasource. |
1486 | s.PatchValue(&imagemetadata.DefaultBaseURL, "") |
1487 | |
1488 | === modified file 'provider/openstack/provider.go' |
1489 | --- provider/openstack/provider.go 2014-03-17 03:45:21 +0000 |
1490 | +++ provider/openstack/provider.go 2014-03-19 02:34:50 +0000 |
1491 | @@ -520,6 +520,11 @@ |
1492 | return e.name |
1493 | } |
1494 | |
1495 | +// SupportedArchitectures is specified on the EnvironCapability interface. |
1496 | +func (*environ) SupportedArchitectures() ([]string, error) { |
1497 | + return supportedArches, nil |
1498 | +} |
1499 | + |
1500 | func (e *environ) Storage() storage.Storage { |
1501 | e.ecfgMutex.Lock() |
1502 | stor := e.storageUnlocked |
1503 | |
1504 | === modified file 'version/version.go' |
1505 | --- version/version.go 2014-03-14 16:45:23 +0000 |
1506 | +++ version/version.go 2014-03-19 02:34:50 +0000 |
1507 | @@ -12,11 +12,12 @@ |
1508 | "os" |
1509 | "path/filepath" |
1510 | "regexp" |
1511 | - "runtime" |
1512 | "strconv" |
1513 | "strings" |
1514 | |
1515 | "labix.org/v2/mgo/bson" |
1516 | + |
1517 | + "launchpad.net/juju-core/juju/arch" |
1518 | ) |
1519 | |
1520 | // The presence and format of this constant is very important. |
1521 | @@ -34,7 +35,7 @@ |
1522 | var Current = Binary{ |
1523 | Number: MustParse(version), |
1524 | Series: readSeries(lsbReleaseFile), |
1525 | - Arch: ubuntuArch(runtime.GOARCH), |
1526 | + Arch: arch.HostArch(), |
1527 | } |
1528 | |
1529 | func init() { |
1530 | @@ -340,13 +341,6 @@ |
1531 | return "" |
1532 | } |
1533 | |
1534 | -func ubuntuArch(arch string) string { |
1535 | - if arch == "386" { |
1536 | - arch = "i386" |
1537 | - } |
1538 | - return arch |
1539 | -} |
1540 | - |
1541 | // ParseMajorMinor takes an argument of the form "major.minor" and returns ints major and minor. |
1542 | func ParseMajorMinor(vers string) (int, int, error) { |
1543 | parts := strings.Split(vers, ".") |
Reviewers: mp+211449_ code.launchpad. net,
Message:
Please take a look.
Description:
Fix bootstrap tools issues
These bugs are addressed:
lp:1227722 - tools for wrong arch
lp:1282870 - cannot use constraints to select tools
lp:1282869 - bad failure mode for incorrect arch
tl;dr; you can bootstrap environments which run on
architectures which are different to the client
machine architcture, so long as tools are available.
eg ppc client boottrapping an environment on EC2.
Other error conditions are also addressed eg you cannot
upload tools to a cloud if that cloud does not support
the client machine architecture, since that's what the
uploaded tools would be built for. You also cannot
specify an arch constraint different to your client
machine if you are uploading tools.
A lot of the tools logic was duplicated. It has been
refactoroed to used shared code. There's still more
to do however since it's still possible to do bad things
with upgrade-juju.
https:/ /code.launchpad .net/~wallyworl d/juju- core/bootstrap- tools-fixes/ +merge/ 211449
(do not edit description out of merge proposal)
Please review this at https:/ /codereview. appspot. com/77270043/
Affected files (+474, -151 lines): bootstrap. go bootstrap_ test.go upgradejuju. go bootstrap/ bootstrap_ test.go bootstrap/ synctools. go interface. go manual/ init.go testing/ tools.go tools/tools_ test.go azure/environ. go azure/environ_ test.go common/ bootstrap. go common/ bootstrap_ test.go common/ mock_test. go dummy/environs. go ec2/local_ test.go joyent/ environ. go local/environ. go local/environ_ test.go maas/environ. go maas/environ_ test.go maas/environ_ whitebox_ test.go manual/ environ. go manual/ environ_ test.go openstack/ local_test. go openstack/ provider. go
A [revision details]
M cmd/juju/
M cmd/juju/
M cmd/juju/
M environs/
M environs/
M environs/
M environs/
M environs/
M environs/
M juju/arch/arch.go
M provider/
M provider/
M provider/
M provider/
M provider/
M provider/
M provider/ec2/ec2.go
M provider/
M provider/
M provider/
M provider/
M provider/
M provider/
M provider/
M provider/
M provider/
M provider/
M provider/