Merge lp:~axwalk/juju-core/manual-bootstrap into lp:~go-bot/juju-core/trunk
- manual-bootstrap
- Merge into trunk
| Status: | Merged |
|---|---|
| Approved by: | Andrew Wilkins |
| Approved revision: | no longer in the source branch. |
| Merged at revision: | 1850 |
| Proposed branch: | lp:~axwalk/juju-core/manual-bootstrap |
| Merge into: | lp:~go-bot/juju-core/trunk |
| Diff against target: |
1104 lines (+616/-155) 16 files modified
cmd/juju/bootstrap.go (+19/-0) cmd/jujud/machine.go (+1/-1) environs/interface.go (+11/-0) environs/manual/agent.go (+28/-40) environs/manual/bootstrap.go (+135/-0) environs/manual/bootstrap_test.go (+164/-0) environs/manual/detection.go (+14/-3) environs/manual/detection_test.go (+7/-63) environs/manual/fakessh_test.go (+112/-0) environs/manual/provisioner.go (+17/-10) environs/manual/provisioner_test.go (+28/-23) environs/manual/tools.go (+24/-0) testing/testbase/patch.go (+12/-0) testing/testbase/patch_test.go (+9/-0) worker/localstorage/config.go (+14/-0) worker/localstorage/worker.go (+21/-15) |
| To merge this branch: | bzr merge lp:~axwalk/juju-core/manual-bootstrap |
| Related bugs: |
| Reviewer | Review Type | Date Requested | Status |
|---|---|---|---|
| Juju Engineering | Pending | ||
|
Review via email:
|
|||
Commit message
Implement manual bootstrapping
environs/bootstrap:
- Added BootstrapStorage interface. If an Environ
implements this interface, its BootstrapStorage
will be used by cmd/juju bootstrap.
cmd/juju:
- Updated bootstrap subcommand to use
environs/
as described above.
provider/
- Added LocalStorageConfig interface, which an
Environ may implement to describe the configuration
of a "localstorage".
environs/
- Added NewFileStorage (i.e. implement StorageWrite)
for testing in environs/manual.
environs/manual:
- Refactored existing code to support bootstrap machines.
- Added "Bootstrap" function to manually bootstrap an
existing machine.
To manually bootstrap, you must provide an environment that
implements the new LocalStorageConfig and BootstrapStorage
interfaces. In other words, you can (currently) only manually
bootstrap a machine which is expected to manage the storage
for its environment. This may be relaxed later if needed.
Next up is the null provider, which uses this new Bootstrap
function. I need to address one last issue with it first:
the null provider mustn't narrow the tools to default-series.
Description of the change
Implement manual bootstrapping
environs/bootstrap:
- Added BootstrapStorage interface. If an Environ
implements this interface, its BootstrapStorage
will be used by cmd/juju bootstrap.
cmd/juju:
- Updated bootstrap subcommand to use
environs/
as described above.
provider/
- Added LocalStorageConfig interface, which an
Environ may implement to describe the configuration
of a "localstorage".
environs/
- Added NewFileStorage (i.e. implement StorageWrite)
for testing in environs/manual.
environs/manual:
- Refactored existing code to support bootstrap machines.
- Added "Bootstrap" function to manually bootstrap an
existing machine.
To manually bootstrap, you must provide an environment that
implements the new LocalStorageConfig and BootstrapStorage
interfaces. In other words, you can (currently) only manually
bootstrap a machine which is expected to manage the storage
for its environment. This may be relaxed later if needed.
Next up is the null provider, which uses this new Bootstrap
function. I need to address one last issue with it first:
the null provider mustn't narrow the tools to default-series.
| Andrew Wilkins (axwalk) wrote : | # |
| Andrew Wilkins (axwalk) wrote : | # |
Please take a look.
| William Reade (fwereade) wrote : | # |
Looking very nice; just a few questions, and a suggestion that you
consider moving the local storage worker into the top-level worker
package, rather than hiding it away under local.
https:/
File cmd/juju/
https:/
cmd/juju/
environ.
Would that maybe be ".(BootstrapSto
experimenting with things like:
type HasBootstrapStorage interface {
BootstrapS
}
...which IMO leads at least to more-readable casts. YMMV.
https:/
File environs/
https:/
environs/
length)
As suggested in the previous review, please copy the whole lot somewhere
out of the way and move atomically into place on success; in this case I
think we've definitely vulnerable to errors with longer-term
consequences.
https:/
File environs/
https:/
environs/
This is really nice; can we produce some output while we're doing so? I
guess it'll have to be straight to the log for now, but a few INFO
messages would be helpful.
https:/
environs/
Don't think so...
https:/
File environs/
https:/
environs/
gc.ErrorMatches, "Environ argument is nil")
FWIW, I'm kinda ok with panics for unexpected nils.
https:/
environs/
Eyebrow slightly raised here... I'm not sure that's actually valid. Do
we initialize the machine id counter in state such that all subsequent
numbers are higher?
https:/
environs/
checkProvisione
This code block is looking awfully familiar now. Can we tidy it up a
little?
https:/
File environs/
https:/
environs/
Why would we be getting multiple arches out of FindInstanceTools? Aren't
we passing one in?
| Andrew Wilkins (axwalk) wrote : | # |
Thank you for the review. I have moved provider/
worker/
https:/
File cmd/juju/
https:/
cmd/juju/
environ.
On 2013/09/11 13:51:21, fwereade wrote:
> Would that maybe be ".(BootstrapSto
Yes, that's a bit better, thanks.
> Elsewhere, we're experimenting with
> things like:
> type HasBootstrapStorage interface {
> BootstrapStorage() (Storage, error)
> }
> ...which IMO leads at least to more-readable casts. YMMV.
I'm not a big fan of the "HasX" style. IMO, the name should represent
the behaviour or role, not something an implementer has.
https:/
File environs/
https:/
environs/
length)
On 2013/09/11 13:51:21, fwereade wrote:
> As suggested in the previous review, please copy the whole lot
somewhere out of
> the way and move atomically into place on success; in this case I
think we've
> definitely vulnerable to errors with longer-term consequences.
Done. It's a bit simpler here :)
https:/
File environs/
https:/
environs/
On 2013/09/11 13:51:21, fwereade wrote:
> This is really nice; can we produce some output while we're doing so?
I guess
> it'll have to be straight to the log for now, but a few INFO messages
would be
> helpful.
Sure. I've added some logger.Infofs to checkProvisioned,
detectSeriesAnd
purposely not added Errorfs, as error results will be logged upstream.
https:/
environs/
On 2013/09/11 13:51:21, fwereade wrote:
> Don't think so...
Oops! Nor do I. I had that in there from early on, and forgot to take it
out :)
https:/
File environs/
https:/
environs/
gc.ErrorMatches, "Environ argument is nil")
On 2013/09/11 13:51:21, fwereade wrote:
> FWIW, I'm kinda ok with panics for unexpected nils.
Noted.
https:/
environs/
On 2013/09/11 13:51:21, fwereade wrote:
> Eyebrow slightly raised here... I'm not sure that's actually valid. Do
we
> initialize the machine id counter in state such that all subsequent
numbers are
> higher?
The reason for t...
| Andrew Wilkins (axwalk) wrote : | # |
Please take a look.
| William Reade (fwereade) wrote : | # |
This is great, thanks. Modulo the /tmp-filesystem issue, this LGTM, on
the understanding that the local storage is not yet fit for purpose and
needs some concept of authentication [0] before we can actually let
people use this feature.
Let me know if I've completely misunderstood the overlap between
SSHStorage and LocalStorage, which is not a blocker for this CL but must
be resolved in a followup (if it's how I understood it to be).
[0] or to not be exposed, which we *might* be able to get away with if
we always used SSH storage from outside..?
https:/
File cmd/juju/
https:/
cmd/juju/
environ.
On 2013/09/12 07:59:00, axw1 wrote:
> On 2013/09/11 13:51:21, fwereade wrote:
> > Would that maybe be ".(BootstrapSto
> Yes, that's a bit better, thanks.
> > Elsewhere, we're experimenting with
> > things like:
> >
> > type HasBootstrapStorage interface {
> > BootstrapStorage() (Storage, error)
> > }
> >
> > ...which IMO leads at least to more-readable casts. YMMV.
> I'm not a big fan of the "HasX" style. IMO, the name should represent
the
> behaviour or role, not something an implementer has.
Your mileage may vary :). I'm not really sold on either way tbh.
https:/
File environs/
https:/
environs/
On 2013/09/12 07:59:00, axw1 wrote:
> On 2013/09/11 13:51:21, fwereade wrote:
> > Eyebrow slightly raised here... I'm not sure that's actually valid.
Do we
> > initialize the machine id counter in state such that all subsequent
numbers
> are
> > higher?
> The reason for this test is that Environ.Bootstrap is provided with a
machine ID
> to use. manual.Bootstrap is invoked via the null provider's Bootstrap
method.
> Thus, the test exists so as not to make assumptions about being on
machine "0".
> Whether or not the counter is incremented... I don't think that can be
its
> concern, if it's been provided with the ID.
Yeah, this isn't your problem. I think it still is a problem that we can
allow bootstrap with arbitrary ids, but that other bits of the system
assign special meaning to "0" (it somehow seems that every time we
eliminate a special case for 0 we end up with a new one somewhere else).
https:/
File cmd/jujud/
https:/
cmd/jujud/
"launchpad.
drop the explicit name?
https:/
File environs/
https:/
environs/
| Andrew Wilkins (axwalk) wrote : | # |
Haven't pushed changes yet, need to sort out the storage issues. Here's
my answers anyway.
https:/
File cmd/jujud/
https:/
cmd/jujud/
"launchpad.
On 2013/09/13 09:24:59, fwereade wrote:
> drop the explicit name?
Done.
https:/
File environs/
https:/
environs/
(environs.Storage, error) {
On 2013/09/13 09:24:59, fwereade wrote:
> High-level question while I think of it: does the SSHStorage on
bootstrap get
> reused but backed with a FileStorage? I sort of imagine so, in which
case, cool
> ...but is it possible for there to be overlap? ie should we be sharing
the
> flocking everywhere?
> or am I just fantasizing?
filestorage is only used for testing at the moment.
environs/
storage directory. However, there is no overlap in use;
environs/sshstorage is only used to write the tools and bootstrap state
file. *If* there's a failure bootstrapping, the bootstrap storage may
also be used to remove the state file.
Since there's no overlap, there's no need to share the locking. I did
just realise that I totally broke the manual bootstrap code though, when
I changed sshstorage's directory structure. I'll have to revisit that
before merging.
https:/
environs/
ioutil.TempFile("", "juju-filestora
On 2013/09/13 09:24:59, fwereade wrote:
> Same different-
This is only for testing, but I will fix it anyway. I think localstorage
ought to be renamed httpstorage, and have the backend contain one of
these.
https:/
File environs/
https:/
environs/
On 2013/09/13 09:24:59, fwereade wrote:
> OK, I think my problem here is maybe that it's not *just* about
bootstrap
> storage, right? It's the same storage we always use, but this is the
path to it
> that's accessible from a client machine. Yes?
> So, at least in the short term, we *will* need SSHStorage and
LocalStorage to
> coexist.
They're pointing to the same location, but they're never used
concurrently.
https:/
File worker/
https:/
worker/
On 2013/09/13 09:24:59, fwereade wrote:
> ah, I see. Can we call this localstorage please?
Sorry, fixed.
https:/
| Roger Peppe (rogpeppe) wrote : | # |
Looks good. Some minor comments and suggestions below.
https:/
File cmd/juju/
https:/
cmd/juju/
bootstrapStorage}
nice
https:/
File environs/
https:/
environs/
that returns a environs.Storage that may
I think I'd define this inside environs/
This package doesn't define or use it, and that file
is a useful reference for people implementing providers.
https:/
File environs/
https:/
environs/
(environs.Storage, error) {
Doc comment please.
Also, wouldn't it make more sense to define NewFileStorageR
terms of NewFileStorage? That way you wouldn't need the dynamic type
conversion, as Storage can be assigned to StorageReader.
For future consideration, I think filestorage.New and
filestorage.
filestorage.
https:/
File environs/
https:/
environs/
s/machineenv/
https:/
File environs/
https:/
environs/
argument is empty")
Errors don't generally start with a capital letter.
Also, if it's only used once, I wouldn't bother defining a variable for
it.
https:/
environs/
argument is nil")
ditto
https:/
environs/
s/will/will be/ ?
https:/
environs/
provisioned: %v", err)
"failed to check provisioned status: %v" ?
https:/
environs/
Why not defer this after the call to SaveState and thereby lose the need
for the savedState variable?
https:/
| Tim Penhey (thumper) wrote : | # |
Still some changes needed here I think
https:/
File cmd/juju/
https:/
cmd/juju/
bootstrapStorage}
If all we are caring about is the storage, why are we changing the
environ?
Does it not make sense to change the things that depend on the storage
to just depend on the storage and not the entire environment, then you
don't need to mock out the Storage method.
Obviously here the important thing is that we want to use a particular
storage for bootstrap, so I'd expect to be assigning the
bootstrapStorage to a storage variable, not mocking out the method with
a different structure.
https:/
File environs/
https:/
environs/
that returns a environs.Storage that may
On 2013/09/14 07:28:03, rog wrote:
> I think I'd define this inside environs/
> This package doesn't define or use it, and that file
> is a useful reference for people implementing providers.
Agreed. It is something that is used by the environment, so it makes
more sense to live there.
https:/
File environs/
https:/
environs/
ioutil.TempFile("", "juju-filestora
On 2013/09/13 09:41:26, axw1 wrote:
> On 2013/09/13 09:24:59, fwereade wrote:
> > Same different-
> This is only for testing, but I will fix it anyway. I think
localstorage ought
> to be renamed httpstorage, and have the backend contain one of these.
That sounds like a good idea IMO
https:/
File environs/
https:/
environs/
args.envcfg, constraints.
I fear that using names like "envcfg" means that we have more and more
assumed knowledge encoded into our variable names. Here I like some
parts of the Zen of Python "Explicit is better than implicit", yes it is
python, but I think the principle holds.
https:/
File environs/
https:/
environs/
Arg, here I know that I'm going to clash on names with Rog, but IMO
BootstrapSto
is a terrible name.
HasBootstrap
makes a lot more sense to me. Firstly you can look at the name and know
what is going on, and we aren't inventing words to go by the weird...
| Andrew Wilkins (axwalk) wrote : | # |
https:/
File cmd/juju/
https:/
cmd/juju/
bootstrapStorage}
On 2013/09/16 03:02:08, thumper wrote:
> If all we are caring about is the storage, why are we changing the
environ?
> Does it not make sense to change the things that depend on the storage
to just
> depend on the storage and not the entire environment, then you don't
need to
> mock out the Storage method.
The storage must be passed down to environs/bootstrap Bootstrap, and
then to environs/tools FindTools. Possibly more places. Those things
also want an Environ (in the case of FindTools, it's hidden in an
interface conversion for the legacy fallback codepath).
Bootstrap storage must be used for *everything* until the environment is
bootstrapped. Leaving a method for acquiring broken/unusable storage
seems like a mistake to me.
> Obviously here the important thing is that we want to use a particular
storage
> for bootstrap, so I'd expect to be assigning the bootstrapStorage to a
storage
> variable, not mocking out the method with a different structure.
https:/
File environs/
https:/
environs/
that returns a environs.Storage that may
On 2013/09/14 07:28:03, rog wrote:
> I think I'd define this inside environs/
> This package doesn't define or use it, and that file
> is a useful reference for people implementing providers.
Fair point, done.
https:/
File environs/
https:/
environs/
(environs.Storage, error) {
On 2013/09/14 07:28:03, rog wrote:
> Doc comment please.
Done. Seems someone else came along and wrote a writer constructor in
the mean time. It lacks a doc comment, so I'll add to that :)
> Also, wouldn't it make more sense to define NewFileStorageR
terms of
> NewFileStorage? That way you wouldn't need the dynamic type
conversion, as
> Storage can be assigned to StorageReader.
That would simplify the code here. My rationale for doing it this way is
that it limits exposure.
IMO, you shouldn't be able to convert a StorageReader to a Storage,
unless you originally created it with NewStorage and converted it to
StorageReader.
> For future consideration, I think filestorage.New and
filestorage.
> would be more idiomatic names than filestorage.
> filestorage.
I do agree, but sadly it seems there's no consensus.
https:/
environs/
ioutil.TempFile...
| Andrew Wilkins (axwalk) wrote : | # |
Please take a look.
| Roger Peppe (rogpeppe) wrote : | # |
https:/
File environs/
https:/
environs/
On 2013/09/16 04:56:48, axw1 wrote:
> On 2013/09/16 03:02:08, thumper wrote:
> > Arg, here I know that I'm going to clash on names with Rog, but IMO
> > BootstrapStorager
> > is a terrible name.
> > HasBootstrapStorage
> > makes a lot more sense to me. Firstly you can look at the name and
know what
> is
> > going on, and we aren't inventing words to go by the weird idea that
> interfaces
> > should end with "er".
> >
> > bootstrapStorage, ok := sometype.
> >
> > reads ok to me (FSVO ok given the syntax of the type assertion).
> Copy and paste of my reply to William: I'm not a big fan of the "HasX"
style.
> IMO, the name should represent the behaviour or role, not something an
> implementer has.
+lots.
I'm not keen on "BootstrapStorager" as a name either,
but I think it's important that interface names are nouns
not predicates - an interface name should represent what the
object *is*, not what it *can be*.
| Tim Penhey (thumper) wrote : | # |
On 2013/09/16 16:51:34, rog wrote:
https:/
> File environs/
https:/
> environs/
> On 2013/09/16 04:56:48, axw1 wrote:
> > On 2013/09/16 03:02:08, thumper wrote:
> > > Arg, here I know that I'm going to clash on names with Rog, but
IMO
> > > BootstrapStorager
> > > is a terrible name.
> > > HasBootstrapStorage
> > > makes a lot more sense to me. Firstly you can look at the name
and know
> what
> > is
> > > going on, and we aren't inventing words to go by the weird idea
that
> > interfaces
> > > should end with "er".
> > >
> > > bootstrapStorage, ok := sometype.
> > >
> > > reads ok to me (FSVO ok given the syntax of the type assertion).
> >
> > Copy and paste of my reply to William: I'm not a big fan of the
"HasX" style.
> > IMO, the name should represent the behaviour or role, not something
an
> > implementer has.
> +lots.
> I'm not keen on "BootstrapStorager" as a name either,
> but I think it's important that interface names are nouns
> not predicates - an interface name should represent what the
> object *is*, not what it *can be*.
Why not then just call the interface "BootstrapStorage".
| Roger Peppe (rogpeppe) wrote : | # |
On 16 September 2013 22:01, <email address hidden> wrote:
> Why not then just call the interface "BootstrapStorage".
That would be appropriate for the storage itself, but this
interface represents an object that can be asked *for* the storage,
which doesn't seem quite the same thing to me.
| William Reade (fwereade) wrote : | # |
If the only remaining contentious point is (Has)BootstrapS
then I call dealer's choice and LGTM whatever he picks; I don't think
any of the names are great, but I also don't think any of them are so
bad as to unacceptably reduce the quality of the codebase ;).
If there are other problems, carry on as usual, ofc, but let's not block
this on naming arguments.
| Go Bot (go-bot) wrote : | # |
The attempt to merge lp:~axwalk/juju-core/manual-bootstrap into lp:juju-core failed. Below is the output from the failed tests.
environs/
# launchpad.
cmd/juju/
cmd/juju/
Preview Diff
| 1 | === modified file 'cmd/juju/bootstrap.go' | |||
| 2 | --- cmd/juju/bootstrap.go 2013-09-20 00:52:13 +0000 | |||
| 3 | +++ cmd/juju/bootstrap.go 2013-09-20 05:05:14 +0000 | |||
| 4 | @@ -17,6 +17,7 @@ | |||
| 5 | 17 | "launchpad.net/juju-core/environs" | 17 | "launchpad.net/juju-core/environs" |
| 6 | 18 | "launchpad.net/juju-core/environs/bootstrap" | 18 | "launchpad.net/juju-core/environs/bootstrap" |
| 7 | 19 | "launchpad.net/juju-core/environs/config" | 19 | "launchpad.net/juju-core/environs/config" |
| 8 | 20 | "launchpad.net/juju-core/environs/storage" | ||
| 9 | 20 | "launchpad.net/juju-core/environs/sync" | 21 | "launchpad.net/juju-core/environs/sync" |
| 10 | 21 | envtools "launchpad.net/juju-core/environs/tools" | 22 | envtools "launchpad.net/juju-core/environs/tools" |
| 11 | 22 | "launchpad.net/juju-core/errors" | 23 | "launchpad.net/juju-core/errors" |
| 12 | @@ -68,6 +69,15 @@ | |||
| 13 | 68 | if err != nil { | 69 | if err != nil { |
| 14 | 69 | return err | 70 | return err |
| 15 | 70 | } | 71 | } |
| 16 | 72 | // If the environment has a special bootstrap Storage, use it wherever | ||
| 17 | 73 | // we'd otherwise use environ.Storage. | ||
| 18 | 74 | if bs, ok := environ.(environs.BootstrapStorager); ok { | ||
| 19 | 75 | bootstrapStorage, err := bs.BootstrapStorage() | ||
| 20 | 76 | if err != nil { | ||
| 21 | 77 | return fmt.Errorf("failed to acquire bootstrap storage: %v", err) | ||
| 22 | 78 | } | ||
| 23 | 79 | environ = &bootstrapStorageEnviron{environ, bootstrapStorage} | ||
| 24 | 80 | } | ||
| 25 | 71 | // TODO: if in verbose mode, write out to Stdout if a new cert was created. | 81 | // TODO: if in verbose mode, write out to Stdout if a new cert was created. |
| 26 | 72 | _, err = environs.EnsureCertificate(environ, environs.WriteCertAndKey) | 82 | _, err = environs.EnsureCertificate(environ, environs.WriteCertAndKey) |
| 27 | 73 | if err != nil { | 83 | if err != nil { |
| 28 | @@ -222,3 +232,12 @@ | |||
| 29 | 222 | } | 232 | } |
| 30 | 223 | return unique.Values() | 233 | return unique.Values() |
| 31 | 224 | } | 234 | } |
| 32 | 235 | |||
| 33 | 236 | type bootstrapStorageEnviron struct { | ||
| 34 | 237 | environs.Environ | ||
| 35 | 238 | bootstrapStorage storage.Storage | ||
| 36 | 239 | } | ||
| 37 | 240 | |||
| 38 | 241 | func (b *bootstrapStorageEnviron) Storage() storage.Storage { | ||
| 39 | 242 | return b.bootstrapStorage | ||
| 40 | 243 | } | ||
| 41 | 225 | 244 | ||
| 42 | === modified file 'cmd/jujud/machine.go' | |||
| 43 | --- cmd/jujud/machine.go 2013-09-18 06:27:52 +0000 | |||
| 44 | +++ cmd/jujud/machine.go 2013-09-20 05:05:14 +0000 | |||
| 45 | @@ -20,7 +20,6 @@ | |||
| 46 | 20 | "launchpad.net/juju-core/log" | 20 | "launchpad.net/juju-core/log" |
| 47 | 21 | "launchpad.net/juju-core/names" | 21 | "launchpad.net/juju-core/names" |
| 48 | 22 | "launchpad.net/juju-core/provider" | 22 | "launchpad.net/juju-core/provider" |
| 49 | 23 | localstorage "launchpad.net/juju-core/provider/local/storage" | ||
| 50 | 24 | "launchpad.net/juju-core/state" | 23 | "launchpad.net/juju-core/state" |
| 51 | 25 | "launchpad.net/juju-core/state/api/params" | 24 | "launchpad.net/juju-core/state/api/params" |
| 52 | 26 | "launchpad.net/juju-core/state/apiserver" | 25 | "launchpad.net/juju-core/state/apiserver" |
| 53 | @@ -29,6 +28,7 @@ | |||
| 54 | 29 | "launchpad.net/juju-core/worker/cleaner" | 28 | "launchpad.net/juju-core/worker/cleaner" |
| 55 | 30 | "launchpad.net/juju-core/worker/deployer" | 29 | "launchpad.net/juju-core/worker/deployer" |
| 56 | 31 | "launchpad.net/juju-core/worker/firewaller" | 30 | "launchpad.net/juju-core/worker/firewaller" |
| 57 | 31 | "launchpad.net/juju-core/worker/localstorage" | ||
| 58 | 32 | "launchpad.net/juju-core/worker/logger" | 32 | "launchpad.net/juju-core/worker/logger" |
| 59 | 33 | "launchpad.net/juju-core/worker/machiner" | 33 | "launchpad.net/juju-core/worker/machiner" |
| 60 | 34 | "launchpad.net/juju-core/worker/minunitsworker" | 34 | "launchpad.net/juju-core/worker/minunitsworker" |
| 61 | 35 | 35 | ||
| 62 | === modified file 'environs/interface.go' | |||
| 63 | --- environs/interface.go 2013-09-18 07:13:09 +0000 | |||
| 64 | +++ environs/interface.go 2013-09-20 05:05:14 +0000 | |||
| 65 | @@ -68,6 +68,17 @@ | |||
| 66 | 68 | PublicStorage() storage.StorageReader | 68 | PublicStorage() storage.StorageReader |
| 67 | 69 | } | 69 | } |
| 68 | 70 | 70 | ||
| 69 | 71 | // BootstrapStorager is an interface that returns a environs.Storage that may | ||
| 70 | 72 | // be used before the bootstrap machine agent has been provisioned. | ||
| 71 | 73 | // | ||
| 72 | 74 | // This is useful for environments where the storage is managed by the machine | ||
| 73 | 75 | // agent once bootstrapped. | ||
| 74 | 76 | type BootstrapStorager interface { | ||
| 75 | 77 | // BootstrapStorager returns an environs.Storage that may be used while | ||
| 76 | 78 | // bootstrapping a machine. | ||
| 77 | 79 | BootstrapStorage() (storage.Storage, error) | ||
| 78 | 80 | } | ||
| 79 | 81 | |||
| 80 | 71 | // ConfigGetter implements access to an environments configuration. | 82 | // ConfigGetter implements access to an environments configuration. |
| 81 | 72 | type ConfigGetter interface { | 83 | type ConfigGetter interface { |
| 82 | 73 | // Config returns the configuration data with which the Environ was created. | 84 | // Config returns the configuration data with which the Environ was created. |
| 83 | 74 | 85 | ||
| 84 | === modified file 'environs/manual/agent.go' | |||
| 85 | --- environs/manual/agent.go 2013-09-13 08:39:36 +0000 | |||
| 86 | +++ environs/manual/agent.go 2013-09-20 05:05:14 +0000 | |||
| 87 | @@ -14,7 +14,7 @@ | |||
| 88 | 14 | "launchpad.net/juju-core/constraints" | 14 | "launchpad.net/juju-core/constraints" |
| 89 | 15 | "launchpad.net/juju-core/environs" | 15 | "launchpad.net/juju-core/environs" |
| 90 | 16 | "launchpad.net/juju-core/environs/cloudinit" | 16 | "launchpad.net/juju-core/environs/cloudinit" |
| 92 | 17 | envtools "launchpad.net/juju-core/environs/tools" | 17 | "launchpad.net/juju-core/environs/config" |
| 93 | 18 | "launchpad.net/juju-core/provider" | 18 | "launchpad.net/juju-core/provider" |
| 94 | 19 | "launchpad.net/juju-core/state" | 19 | "launchpad.net/juju-core/state" |
| 95 | 20 | "launchpad.net/juju-core/state/api" | 20 | "launchpad.net/juju-core/state/api" |
| 96 | @@ -23,21 +23,27 @@ | |||
| 97 | 23 | ) | 23 | ) |
| 98 | 24 | 24 | ||
| 99 | 25 | type provisionMachineAgentArgs struct { | 25 | type provisionMachineAgentArgs struct { |
| 110 | 26 | host string | 26 | host string |
| 111 | 27 | dataDir string | 27 | dataDir string |
| 112 | 28 | env environs.Environ | 28 | bootstrap bool |
| 113 | 29 | machine *state.Machine | 29 | environConfig *config.Config |
| 114 | 30 | nonce string | 30 | machineId string |
| 115 | 31 | stateInfo *state.Info | 31 | nonce string |
| 116 | 32 | apiInfo *api.Info | 32 | stateFileURL string |
| 117 | 33 | series string | 33 | stateInfo *state.Info |
| 118 | 34 | arch string | 34 | apiInfo *api.Info |
| 119 | 35 | tools *tools.Tools | 35 | tools *tools.Tools |
| 120 | 36 | |||
| 121 | 37 | // agentEnv is an optional map of | ||
| 122 | 38 | // arbitrary key/value pairs to pass | ||
| 123 | 39 | // into the machine agent. | ||
| 124 | 40 | agentEnv map[string]string | ||
| 125 | 36 | } | 41 | } |
| 126 | 37 | 42 | ||
| 127 | 38 | // provisionMachineAgent connects to a machine over SSH, | 43 | // provisionMachineAgent connects to a machine over SSH, |
| 128 | 39 | // copies across the tools, and installs a machine agent. | 44 | // copies across the tools, and installs a machine agent. |
| 129 | 40 | func provisionMachineAgent(args provisionMachineAgentArgs) error { | 45 | func provisionMachineAgent(args provisionMachineAgentArgs) error { |
| 130 | 46 | logger.Infof("Provisioning machine agent on %s", args.host) | ||
| 131 | 41 | script, err := provisionMachineAgentScript(args) | 47 | script, err := provisionMachineAgentScript(args) |
| 132 | 42 | if err != nil { | 48 | if err != nil { |
| 133 | 43 | return err | 49 | return err |
| 134 | @@ -58,28 +64,27 @@ | |||
| 135 | 58 | // provisionMachineAgentScript generates the script necessary | 64 | // provisionMachineAgentScript generates the script necessary |
| 136 | 59 | // to install a machine agent on the specified host. | 65 | // to install a machine agent on the specified host. |
| 137 | 60 | func provisionMachineAgentScript(args provisionMachineAgentArgs) (string, error) { | 66 | func provisionMachineAgentScript(args provisionMachineAgentArgs) (string, error) { |
| 138 | 61 | tools := args.tools | ||
| 139 | 62 | if tools == nil { | ||
| 140 | 63 | var err error | ||
| 141 | 64 | tools, err = findMachineAgentTools(args.env, args.series, args.arch) | ||
| 142 | 65 | if err != nil { | ||
| 143 | 66 | return "", err | ||
| 144 | 67 | } | ||
| 145 | 68 | } | ||
| 146 | 69 | |||
| 147 | 70 | // We generate a cloud-init config, which we'll then pull the runcmds | 67 | // We generate a cloud-init config, which we'll then pull the runcmds |
| 148 | 71 | // and prerequisite packages out of. Rather than generating a cloud-config, | 68 | // and prerequisite packages out of. Rather than generating a cloud-config, |
| 149 | 72 | // we'll just generate a shell script. | 69 | // we'll just generate a shell script. |
| 151 | 73 | mcfg := environs.NewMachineConfig(args.machine.Id(), args.nonce, args.stateInfo, args.apiInfo) | 70 | var mcfg *cloudinit.MachineConfig |
| 152 | 71 | if args.bootstrap { | ||
| 153 | 72 | mcfg = environs.NewBootstrapMachineConfig(args.machineId, args.nonce) | ||
| 154 | 73 | } else { | ||
| 155 | 74 | mcfg = environs.NewMachineConfig(args.machineId, args.nonce, args.stateInfo, args.apiInfo) | ||
| 156 | 75 | } | ||
| 157 | 74 | if args.dataDir != "" { | 76 | if args.dataDir != "" { |
| 158 | 75 | mcfg.DataDir = args.dataDir | 77 | mcfg.DataDir = args.dataDir |
| 159 | 76 | } | 78 | } |
| 162 | 77 | mcfg.Tools = tools | 79 | mcfg.Tools = args.tools |
| 163 | 78 | err := environs.FinishMachineConfig(mcfg, args.env.Config(), constraints.Value{}) | 80 | err := environs.FinishMachineConfig(mcfg, args.environConfig, constraints.Value{}) |
| 164 | 79 | if err != nil { | 81 | if err != nil { |
| 165 | 80 | return "", err | 82 | return "", err |
| 166 | 81 | } | 83 | } |
| 167 | 82 | mcfg.AgentEnvironment[agent.ProviderType] = provider.Null | 84 | mcfg.AgentEnvironment[agent.ProviderType] = provider.Null |
| 168 | 85 | for k, v := range args.agentEnv { | ||
| 169 | 86 | mcfg.AgentEnvironment[k] = v | ||
| 170 | 87 | } | ||
| 171 | 83 | cloudcfg := corecloudinit.New() | 88 | cloudcfg := corecloudinit.New() |
| 172 | 84 | if cloudcfg, err = cloudinit.Configure(mcfg, cloudcfg); err != nil { | 89 | if cloudcfg, err = cloudinit.Configure(mcfg, cloudcfg); err != nil { |
| 173 | 85 | return "", err | 90 | return "", err |
| 174 | @@ -116,20 +121,3 @@ | |||
| 175 | 116 | script = append(head, tail...) | 121 | script = append(head, tail...) |
| 176 | 117 | return strings.Join(script, "\n"), nil | 122 | return strings.Join(script, "\n"), nil |
| 177 | 118 | } | 123 | } |
| 178 | 119 | |||
| 179 | 120 | func findMachineAgentTools(env environs.Environ, series, arch string) (*tools.Tools, error) { | ||
| 180 | 121 | agentVersion, ok := env.Config().AgentVersion() | ||
| 181 | 122 | if !ok { | ||
| 182 | 123 | return nil, fmt.Errorf("no agent version set in environment configuration") | ||
| 183 | 124 | } | ||
| 184 | 125 | possibleTools, err := envtools.FindInstanceTools(env, agentVersion, series, &arch) | ||
| 185 | 126 | if err != nil { | ||
| 186 | 127 | return nil, err | ||
| 187 | 128 | } | ||
| 188 | 129 | arches := possibleTools.Arches() | ||
| 189 | 130 | possibleTools, err = possibleTools.Match(tools.Filter{Arch: arch}) | ||
| 190 | 131 | if err != nil { | ||
| 191 | 132 | return nil, fmt.Errorf("chosen architecture %v not present in %v", arch, arches) | ||
| 192 | 133 | } | ||
| 193 | 134 | return possibleTools[0], nil | ||
| 194 | 135 | } | ||
| 195 | 136 | 124 | ||
| 196 | === added file 'environs/manual/bootstrap.go' | |||
| 197 | --- environs/manual/bootstrap.go 1970-01-01 00:00:00 +0000 | |||
| 198 | +++ environs/manual/bootstrap.go 2013-09-20 05:05:14 +0000 | |||
| 199 | @@ -0,0 +1,135 @@ | |||
| 200 | 1 | // Copyright 2013 Canonical Ltd. | ||
| 201 | 2 | // Licensed under the AGPLv3, see LICENCE file for details. | ||
| 202 | 3 | |||
| 203 | 4 | package manual | ||
| 204 | 5 | |||
| 205 | 6 | import ( | ||
| 206 | 7 | "errors" | ||
| 207 | 8 | "fmt" | ||
| 208 | 9 | |||
| 209 | 10 | "launchpad.net/juju-core/agent" | ||
| 210 | 11 | "launchpad.net/juju-core/environs" | ||
| 211 | 12 | envtools "launchpad.net/juju-core/environs/tools" | ||
| 212 | 13 | "launchpad.net/juju-core/instance" | ||
| 213 | 14 | "launchpad.net/juju-core/names" | ||
| 214 | 15 | "launchpad.net/juju-core/provider" | ||
| 215 | 16 | "launchpad.net/juju-core/state" | ||
| 216 | 17 | "launchpad.net/juju-core/tools" | ||
| 217 | 18 | "launchpad.net/juju-core/worker/localstorage" | ||
| 218 | 19 | ) | ||
| 219 | 20 | |||
| 220 | 21 | const BootstrapInstanceId = instance.Id(manualInstancePrefix) | ||
| 221 | 22 | |||
| 222 | 23 | // LocalStorageEnviron is an Environ where the bootstrap node | ||
| 223 | 24 | // manages its own local storage. | ||
| 224 | 25 | type LocalStorageEnviron interface { | ||
| 225 | 26 | environs.Environ | ||
| 226 | 27 | environs.BootstrapStorager | ||
| 227 | 28 | localstorage.LocalStorageConfig | ||
| 228 | 29 | } | ||
| 229 | 30 | |||
| 230 | 31 | type BootstrapArgs struct { | ||
| 231 | 32 | Host string | ||
| 232 | 33 | Environ LocalStorageEnviron | ||
| 233 | 34 | MachineId string | ||
| 234 | 35 | PossibleTools tools.List | ||
| 235 | 36 | } | ||
| 236 | 37 | |||
| 237 | 38 | // TODO(axw) make this configurable? | ||
| 238 | 39 | const dataDir = "/var/lib/juju" | ||
| 239 | 40 | |||
| 240 | 41 | func errMachineIdInvalid(machineId string) error { | ||
| 241 | 42 | return fmt.Errorf("%q is not a valid machine ID", machineId) | ||
| 242 | 43 | } | ||
| 243 | 44 | |||
| 244 | 45 | // NewManualBootstrapEnviron wraps a LocalStorageEnviron with another which | ||
| 245 | 46 | // overrides the Bootstrap method; when Bootstrap is invoked, the specified | ||
| 246 | 47 | // host will be manually bootstrapped. | ||
| 247 | 48 | func Bootstrap(args BootstrapArgs) (err error) { | ||
| 248 | 49 | if args.Host == "" { | ||
| 249 | 50 | return errors.New("host argument is empty") | ||
| 250 | 51 | } | ||
| 251 | 52 | if args.Environ == nil { | ||
| 252 | 53 | return errors.New("environ argument is nil") | ||
| 253 | 54 | } | ||
| 254 | 55 | if !names.IsMachine(args.MachineId) { | ||
| 255 | 56 | return errMachineIdInvalid(args.MachineId) | ||
| 256 | 57 | } | ||
| 257 | 58 | |||
| 258 | 59 | provisioned, err := checkProvisioned(args.Host) | ||
| 259 | 60 | if err != nil { | ||
| 260 | 61 | return fmt.Errorf("failed to check provisioned status: %v", err) | ||
| 261 | 62 | } | ||
| 262 | 63 | if provisioned { | ||
| 263 | 64 | return ErrProvisioned | ||
| 264 | 65 | } | ||
| 265 | 66 | |||
| 266 | 67 | bootstrapStorage, err := args.Environ.BootstrapStorage() | ||
| 267 | 68 | if err != nil { | ||
| 268 | 69 | return err | ||
| 269 | 70 | } | ||
| 270 | 71 | |||
| 271 | 72 | hc, series, err := detectSeriesAndHardwareCharacteristics(args.Host) | ||
| 272 | 73 | if err != nil { | ||
| 273 | 74 | return fmt.Errorf("error detecting hardware characteristics: %v", err) | ||
| 274 | 75 | } | ||
| 275 | 76 | |||
| 276 | 77 | // Filter tools based on detected series/arch. | ||
| 277 | 78 | logger.Infof("Filtering possible tools: %v", args.PossibleTools) | ||
| 278 | 79 | possibleTools, err := args.PossibleTools.Match(tools.Filter{ | ||
| 279 | 80 | Arch: *hc.Arch, | ||
| 280 | 81 | Series: series, | ||
| 281 | 82 | }) | ||
| 282 | 83 | if err != nil { | ||
| 283 | 84 | return err | ||
| 284 | 85 | } | ||
| 285 | 86 | |||
| 286 | 87 | // Store the state file. If provisioning fails, we'll remove the file. | ||
| 287 | 88 | logger.Infof("Saving bootstrap state file to bootstrap storage") | ||
| 288 | 89 | err = provider.SaveState( | ||
| 289 | 90 | bootstrapStorage, | ||
| 290 | 91 | &provider.BootstrapState{ | ||
| 291 | 92 | StateInstances: []instance.Id{BootstrapInstanceId}, | ||
| 292 | 93 | Characteristics: []instance.HardwareCharacteristics{hc}, | ||
| 293 | 94 | }, | ||
| 294 | 95 | ) | ||
| 295 | 96 | if err != nil { | ||
| 296 | 97 | return err | ||
| 297 | 98 | } | ||
| 298 | 99 | defer func() { | ||
| 299 | 100 | if err != nil { | ||
| 300 | 101 | logger.Errorf("bootstrapping failed, removing state file: %v", err) | ||
| 301 | 102 | bootstrapStorage.Remove(provider.StateFile) | ||
| 302 | 103 | } | ||
| 303 | 104 | }() | ||
| 304 | 105 | |||
| 305 | 106 | // Get a file:// scheme tools URL for the tools, which will have been | ||
| 306 | 107 | // copied to the remote machine's storage directory. | ||
| 307 | 108 | tools := *possibleTools[0] | ||
| 308 | 109 | storageDir := args.Environ.StorageDir() | ||
| 309 | 110 | toolsStorageName := envtools.StorageName(tools.Version) | ||
| 310 | 111 | tools.URL = fmt.Sprintf("file://%s/%s", storageDir, toolsStorageName) | ||
| 311 | 112 | |||
| 312 | 113 | // Add the local storage configuration. | ||
| 313 | 114 | agentEnv := map[string]string{ | ||
| 314 | 115 | agent.StorageAddr: args.Environ.StorageAddr(), | ||
| 315 | 116 | agent.StorageDir: storageDir, | ||
| 316 | 117 | agent.SharedStorageAddr: args.Environ.SharedStorageAddr(), | ||
| 317 | 118 | agent.SharedStorageDir: args.Environ.SharedStorageDir(), | ||
| 318 | 119 | } | ||
| 319 | 120 | |||
| 320 | 121 | // Finally, provision the machine agent. | ||
| 321 | 122 | stateFileURL := fmt.Sprintf("file://%s/%s", storageDir, provider.StateFile) | ||
| 322 | 123 | err = provisionMachineAgent(provisionMachineAgentArgs{ | ||
| 323 | 124 | host: args.Host, | ||
| 324 | 125 | dataDir: dataDir, | ||
| 325 | 126 | environConfig: args.Environ.Config(), | ||
| 326 | 127 | stateFileURL: stateFileURL, | ||
| 327 | 128 | machineId: args.MachineId, | ||
| 328 | 129 | bootstrap: true, | ||
| 329 | 130 | nonce: state.BootstrapNonce, | ||
| 330 | 131 | tools: &tools, | ||
| 331 | 132 | agentEnv: agentEnv, | ||
| 332 | 133 | }) | ||
| 333 | 134 | return err | ||
| 334 | 135 | } | ||
| 335 | 0 | 136 | ||
| 336 | === added file 'environs/manual/bootstrap_test.go' | |||
| 337 | --- environs/manual/bootstrap_test.go 1970-01-01 00:00:00 +0000 | |||
| 338 | +++ environs/manual/bootstrap_test.go 2013-09-20 05:05:14 +0000 | |||
| 339 | @@ -0,0 +1,164 @@ | |||
| 340 | 1 | // Copyright 2013 Canonical Ltd. | ||
| 341 | 2 | // Licensed under the AGPLv3, see LICENCE file for details. | ||
| 342 | 3 | |||
| 343 | 4 | package manual | ||
| 344 | 5 | |||
| 345 | 6 | import ( | ||
| 346 | 7 | "os" | ||
| 347 | 8 | |||
| 348 | 9 | gc "launchpad.net/gocheck" | ||
| 349 | 10 | |||
| 350 | 11 | "launchpad.net/juju-core/environs" | ||
| 351 | 12 | "launchpad.net/juju-core/environs/filestorage" | ||
| 352 | 13 | "launchpad.net/juju-core/environs/storage" | ||
| 353 | 14 | "launchpad.net/juju-core/environs/tools" | ||
| 354 | 15 | coreerrors "launchpad.net/juju-core/errors" | ||
| 355 | 16 | "launchpad.net/juju-core/instance" | ||
| 356 | 17 | "launchpad.net/juju-core/juju/testing" | ||
| 357 | 18 | "launchpad.net/juju-core/provider" | ||
| 358 | 19 | jc "launchpad.net/juju-core/testing/checkers" | ||
| 359 | 20 | ) | ||
| 360 | 21 | |||
| 361 | 22 | type bootstrapSuite struct { | ||
| 362 | 23 | testing.JujuConnSuite | ||
| 363 | 24 | env *localStorageEnviron | ||
| 364 | 25 | } | ||
| 365 | 26 | |||
| 366 | 27 | var _ = gc.Suite(&bootstrapSuite{}) | ||
| 367 | 28 | |||
| 368 | 29 | type localStorageEnviron struct { | ||
| 369 | 30 | environs.Environ | ||
| 370 | 31 | storage storage.Storage | ||
| 371 | 32 | storageAddr string | ||
| 372 | 33 | storageDir string | ||
| 373 | 34 | sharedStorageAddr string | ||
| 374 | 35 | sharedStorageDir string | ||
| 375 | 36 | } | ||
| 376 | 37 | |||
| 377 | 38 | func (e *localStorageEnviron) BootstrapStorage() (storage.Storage, error) { | ||
| 378 | 39 | return e.storage, nil | ||
| 379 | 40 | } | ||
| 380 | 41 | |||
| 381 | 42 | func (e *localStorageEnviron) StorageAddr() string { | ||
| 382 | 43 | return e.storageAddr | ||
| 383 | 44 | } | ||
| 384 | 45 | |||
| 385 | 46 | func (e *localStorageEnviron) StorageDir() string { | ||
| 386 | 47 | return e.storageDir | ||
| 387 | 48 | } | ||
| 388 | 49 | |||
| 389 | 50 | func (e *localStorageEnviron) SharedStorageAddr() string { | ||
| 390 | 51 | return e.sharedStorageAddr | ||
| 391 | 52 | } | ||
| 392 | 53 | |||
| 393 | 54 | func (e *localStorageEnviron) SharedStorageDir() string { | ||
| 394 | 55 | return e.sharedStorageDir | ||
| 395 | 56 | } | ||
| 396 | 57 | |||
| 397 | 58 | func (s *bootstrapSuite) SetUpTest(c *gc.C) { | ||
| 398 | 59 | s.JujuConnSuite.SetUpTest(c) | ||
| 399 | 60 | s.env = &localStorageEnviron{ | ||
| 400 | 61 | Environ: s.Conn.Environ, | ||
| 401 | 62 | storageDir: c.MkDir(), | ||
| 402 | 63 | } | ||
| 403 | 64 | storage, err := filestorage.NewFileStorageWriter(s.env.storageDir, filestorage.UseDefaultTmpDir) | ||
| 404 | 65 | c.Assert(err, gc.IsNil) | ||
| 405 | 66 | s.env.storage = storage | ||
| 406 | 67 | } | ||
| 407 | 68 | |||
| 408 | 69 | func (s *bootstrapSuite) getArgs(c *gc.C) BootstrapArgs { | ||
| 409 | 70 | hostname, err := os.Hostname() | ||
| 410 | 71 | c.Assert(err, gc.IsNil) | ||
| 411 | 72 | toolsList, err := tools.FindBootstrapTools(s.Conn.Environ, tools.BootstrapToolsParams{}) | ||
| 412 | 73 | c.Assert(err, gc.IsNil) | ||
| 413 | 74 | return BootstrapArgs{ | ||
| 414 | 75 | Host: hostname, | ||
| 415 | 76 | Environ: s.env, | ||
| 416 | 77 | MachineId: "0", | ||
| 417 | 78 | PossibleTools: toolsList, | ||
| 418 | 79 | } | ||
| 419 | 80 | } | ||
| 420 | 81 | |||
| 421 | 82 | func (s *bootstrapSuite) TestBootstrap(c *gc.C) { | ||
| 422 | 83 | args := s.getArgs(c) | ||
| 423 | 84 | args.Host = "ubuntu@" + args.Host | ||
| 424 | 85 | |||
| 425 | 86 | defer fakeSSH{series: s.Conn.Environ.Config().DefaultSeries()}.install(c).Restore() | ||
| 426 | 87 | err := Bootstrap(args) | ||
| 427 | 88 | c.Assert(err, gc.IsNil) | ||
| 428 | 89 | |||
| 429 | 90 | bootstrapState, err := provider.LoadState(s.env.storage) | ||
| 430 | 91 | c.Assert(err, gc.IsNil) | ||
| 431 | 92 | c.Assert( | ||
| 432 | 93 | bootstrapState.StateInstances, | ||
| 433 | 94 | gc.DeepEquals, | ||
| 434 | 95 | []instance.Id{BootstrapInstanceId}, | ||
| 435 | 96 | ) | ||
| 436 | 97 | |||
| 437 | 98 | // Do it all again; this should work, despite the fact that | ||
| 438 | 99 | // there's a bootstrap state file. Existence for that is | ||
| 439 | 100 | // checked in general bootstrap code (environs/bootstrap). | ||
| 440 | 101 | defer fakeSSH{series: s.Conn.Environ.Config().DefaultSeries()}.install(c).Restore() | ||
| 441 | 102 | err = Bootstrap(args) | ||
| 442 | 103 | c.Assert(err, gc.IsNil) | ||
| 443 | 104 | |||
| 444 | 105 | // We *do* check that the machine has no juju* upstart jobs, though. | ||
| 445 | 106 | defer installFakeSSH(c, "", "/etc/init/jujud-machine-0.conf", 0).Restore() | ||
| 446 | 107 | err = Bootstrap(args) | ||
| 447 | 108 | c.Assert(err, gc.Equals, ErrProvisioned) | ||
| 448 | 109 | } | ||
| 449 | 110 | |||
| 450 | 111 | func (s *bootstrapSuite) TestBootstrapScriptFailure(c *gc.C) { | ||
| 451 | 112 | args := s.getArgs(c) | ||
| 452 | 113 | args.Host = "ubuntu@" + args.Host | ||
| 453 | 114 | series := s.Conn.Environ.Config().DefaultSeries() | ||
| 454 | 115 | defer fakeSSH{series: series, provisionAgentExitCode: 1}.install(c).Restore() | ||
| 455 | 116 | err := Bootstrap(args) | ||
| 456 | 117 | c.Assert(err, gc.NotNil) | ||
| 457 | 118 | |||
| 458 | 119 | // Since the script failed, the state file should have been | ||
| 459 | 120 | // removed from storage. | ||
| 460 | 121 | _, err = provider.LoadState(s.env.storage) | ||
| 461 | 122 | c.Assert(err, jc.Satisfies, coreerrors.IsNotBootstrapped) | ||
| 462 | 123 | } | ||
| 463 | 124 | |||
| 464 | 125 | func (s *bootstrapSuite) TestBootstrapEmptyHost(c *gc.C) { | ||
| 465 | 126 | args := s.getArgs(c) | ||
| 466 | 127 | args.Host = "" | ||
| 467 | 128 | c.Assert(Bootstrap(args), gc.ErrorMatches, "host argument is empty") | ||
| 468 | 129 | } | ||
| 469 | 130 | |||
| 470 | 131 | func (s *bootstrapSuite) TestBootstrapNilEnviron(c *gc.C) { | ||
| 471 | 132 | args := s.getArgs(c) | ||
| 472 | 133 | args.Environ = nil | ||
| 473 | 134 | c.Assert(Bootstrap(args), gc.ErrorMatches, "environ argument is nil") | ||
| 474 | 135 | } | ||
| 475 | 136 | |||
| 476 | 137 | func (s *bootstrapSuite) TestBootstrapInvalidMachineId(c *gc.C) { | ||
| 477 | 138 | args := s.getArgs(c) | ||
| 478 | 139 | args.MachineId = "" | ||
| 479 | 140 | c.Assert(Bootstrap(args), gc.ErrorMatches, `"" is not a valid machine ID`) | ||
| 480 | 141 | args.MachineId = "bahhumbug" | ||
| 481 | 142 | c.Assert(Bootstrap(args), gc.ErrorMatches, `"bahhumbug" is not a valid machine ID`) | ||
| 482 | 143 | } | ||
| 483 | 144 | |||
| 484 | 145 | func (s *bootstrapSuite) TestBootstrapAlternativeMachineId(c *gc.C) { | ||
| 485 | 146 | args := s.getArgs(c) | ||
| 486 | 147 | args.MachineId = "1" | ||
| 487 | 148 | defer fakeSSH{series: s.Conn.Environ.Config().DefaultSeries()}.install(c).Restore() | ||
| 488 | 149 | c.Assert(Bootstrap(args), gc.IsNil) | ||
| 489 | 150 | } | ||
| 490 | 151 | |||
| 491 | 152 | func (s *bootstrapSuite) TestBootstrapNoMatchingTools(c *gc.C) { | ||
| 492 | 153 | // Empty tools list. | ||
| 493 | 154 | args := s.getArgs(c) | ||
| 494 | 155 | args.PossibleTools = nil | ||
| 495 | 156 | series := s.Conn.Environ.Config().DefaultSeries() | ||
| 496 | 157 | defer fakeSSH{series: series, skipProvisionAgent: true}.install(c).Restore() | ||
| 497 | 158 | c.Assert(Bootstrap(args), gc.ErrorMatches, "no matching tools available") | ||
| 498 | 159 | |||
| 499 | 160 | // Non-empty list, but none that match the series/arch. | ||
| 500 | 161 | defer fakeSSH{series: "edgy", skipProvisionAgent: true}.install(c).Restore() | ||
| 501 | 162 | args = s.getArgs(c) | ||
| 502 | 163 | c.Assert(Bootstrap(args), gc.ErrorMatches, "no matching tools available") | ||
| 503 | 164 | } | ||
| 504 | 0 | 165 | ||
| 505 | === modified file 'environs/manual/detection.go' | |||
| 506 | --- environs/manual/detection.go 2013-09-12 02:04:05 +0000 | |||
| 507 | +++ environs/manual/detection.go 2013-09-20 05:05:14 +0000 | |||
| 508 | @@ -11,6 +11,7 @@ | |||
| 509 | 11 | "strings" | 11 | "strings" |
| 510 | 12 | 12 | ||
| 511 | 13 | "launchpad.net/juju-core/instance" | 13 | "launchpad.net/juju-core/instance" |
| 512 | 14 | "launchpad.net/juju-core/utils" | ||
| 513 | 14 | ) | 15 | ) |
| 514 | 15 | 16 | ||
| 515 | 16 | // checkProvisionedScript is the script to run on the remote machine | 17 | // checkProvisionedScript is the script to run on the remote machine |
| 516 | @@ -23,9 +24,9 @@ | |||
| 517 | 23 | // checkProvisioned checks if any juju upstart jobs already | 24 | // checkProvisioned checks if any juju upstart jobs already |
| 518 | 24 | // exist on the host machine. | 25 | // exist on the host machine. |
| 519 | 25 | func checkProvisioned(sshHost string) (bool, error) { | 26 | func checkProvisioned(sshHost string) (bool, error) { |
| 521 | 26 | cmd := sshCommand(sshHost, "bash") | 27 | logger.Infof("Checking if %s is already provisioned", sshHost) |
| 522 | 28 | cmd := sshCommand(sshHost, fmt.Sprintf("bash -c %s", utils.ShQuote(checkProvisionedScript))) | ||
| 523 | 27 | var stdout, stderr bytes.Buffer | 29 | var stdout, stderr bytes.Buffer |
| 524 | 28 | cmd.Stdin = bytes.NewBufferString(checkProvisionedScript) | ||
| 525 | 29 | cmd.Stdout = &stdout | 30 | cmd.Stdout = &stdout |
| 526 | 30 | cmd.Stderr = &stderr | 31 | cmd.Stderr = &stderr |
| 527 | 31 | if err := cmd.Run(); err != nil { | 32 | if err := cmd.Run(); err != nil { |
| 528 | @@ -34,13 +35,21 @@ | |||
| 529 | 34 | } | 35 | } |
| 530 | 35 | return false, err | 36 | return false, err |
| 531 | 36 | } | 37 | } |
| 533 | 37 | return len(strings.TrimSpace(stdout.String())) > 0, nil | 38 | output := strings.TrimSpace(stdout.String()) |
| 534 | 39 | provisioned := len(output) > 0 | ||
| 535 | 40 | if provisioned { | ||
| 536 | 41 | logger.Infof("%s is already provisioned [%q]", sshHost, output) | ||
| 537 | 42 | } else { | ||
| 538 | 43 | logger.Infof("%s is not provisioned", sshHost) | ||
| 539 | 44 | } | ||
| 540 | 45 | return provisioned, nil | ||
| 541 | 38 | } | 46 | } |
| 542 | 39 | 47 | ||
| 543 | 40 | // detectSeriesHardwareCharacteristics detects the OS series | 48 | // detectSeriesHardwareCharacteristics detects the OS series |
| 544 | 41 | // and hardware characteristics of the remote machine by | 49 | // and hardware characteristics of the remote machine by |
| 545 | 42 | // connecting to the machine and executing a bash script. | 50 | // connecting to the machine and executing a bash script. |
| 546 | 43 | func detectSeriesAndHardwareCharacteristics(sshHost string) (hc instance.HardwareCharacteristics, series string, err error) { | 51 | func detectSeriesAndHardwareCharacteristics(sshHost string) (hc instance.HardwareCharacteristics, series string, err error) { |
| 547 | 52 | logger.Infof("Detecting series and characteristics on %s", sshHost) | ||
| 548 | 44 | cmd := sshCommand(sshHost, "bash") | 53 | cmd := sshCommand(sshHost, "bash") |
| 549 | 45 | cmd.Stdin = bytes.NewBufferString(detectionScript) | 54 | cmd.Stdin = bytes.NewBufferString(detectionScript) |
| 550 | 46 | out, err := cmd.CombinedOutput() | 55 | out, err := cmd.CombinedOutput() |
| 551 | @@ -101,6 +110,7 @@ | |||
| 552 | 101 | } | 110 | } |
| 553 | 102 | 111 | ||
| 554 | 103 | // TODO(axw) calculate CpuPower. What algorithm do we use? | 112 | // TODO(axw) calculate CpuPower. What algorithm do we use? |
| 555 | 113 | logger.Infof("series: %s, characteristics: %s", series, hc) | ||
| 556 | 104 | return hc, series, nil | 114 | return hc, series, nil |
| 557 | 105 | } | 115 | } |
| 558 | 106 | 116 | ||
| 559 | @@ -116,6 +126,7 @@ | |||
| 560 | 116 | } | 126 | } |
| 561 | 117 | 127 | ||
| 562 | 118 | const detectionScript = `#!/bin/bash | 128 | const detectionScript = `#!/bin/bash |
| 563 | 129 | set -e | ||
| 564 | 119 | lsb_release -cs | 130 | lsb_release -cs |
| 565 | 120 | uname -m | 131 | uname -m |
| 566 | 121 | grep MemTotal /proc/meminfo | 132 | grep MemTotal /proc/meminfo |
| 567 | 122 | 133 | ||
| 568 | === modified file 'environs/manual/detection_test.go' | |||
| 569 | --- environs/manual/detection_test.go 2013-09-20 02:53:59 +0000 | |||
| 570 | +++ environs/manual/detection_test.go 2013-09-20 05:05:14 +0000 | |||
| 571 | @@ -4,10 +4,6 @@ | |||
| 572 | 4 | package manual | 4 | package manual |
| 573 | 5 | 5 | ||
| 574 | 6 | import ( | 6 | import ( |
| 575 | 7 | "fmt" | ||
| 576 | 8 | "io/ioutil" | ||
| 577 | 9 | "os" | ||
| 578 | 10 | "path/filepath" | ||
| 579 | 11 | "strings" | 7 | "strings" |
| 580 | 12 | 8 | ||
| 581 | 13 | gc "launchpad.net/gocheck" | 9 | gc "launchpad.net/gocheck" |
| 582 | @@ -22,58 +18,6 @@ | |||
| 583 | 22 | 18 | ||
| 584 | 23 | var _ = gc.Suite(&detectionSuite{}) | 19 | var _ = gc.Suite(&detectionSuite{}) |
| 585 | 24 | 20 | ||
| 586 | 25 | // sshscript should only print the result on the first execution, | ||
| 587 | 26 | // to handle the case where it's called multiple times. On | ||
| 588 | 27 | // subsequent executions, it should find the next 'ssh' in $PATH | ||
| 589 | 28 | // and exec that. | ||
| 590 | 29 | var sshscript = `#!/bin/bash | ||
| 591 | 30 | if [ ! -e "$0.run" ]; then | ||
| 592 | 31 | touch "$0.run" | ||
| 593 | 32 | diff "$0.expected-input" - | ||
| 594 | 33 | exitcode=$? | ||
| 595 | 34 | if [ $exitcode -ne 0 ]; then | ||
| 596 | 35 | echo "ERROR: did not match expected input" >&2 | ||
| 597 | 36 | exit $exitcode | ||
| 598 | 37 | fi | ||
| 599 | 38 | # stdout | ||
| 600 | 39 | %s | ||
| 601 | 40 | # stderr | ||
| 602 | 41 | %s | ||
| 603 | 42 | exit %d | ||
| 604 | 43 | else | ||
| 605 | 44 | export PATH=${PATH#*:} | ||
| 606 | 45 | exec ssh $* | ||
| 607 | 46 | fi` | ||
| 608 | 47 | |||
| 609 | 48 | // sshresponse creates a fake "ssh" command in a new $PATH, | ||
| 610 | 49 | // updates $PATH, and returns a function to reset $PATH to | ||
| 611 | 50 | // its original value when called. | ||
| 612 | 51 | // | ||
| 613 | 52 | // output may be: | ||
| 614 | 53 | // - nil (no output) | ||
| 615 | 54 | // - a string (stdout) | ||
| 616 | 55 | // - a slice of strings, of length two (stdout, stderr) | ||
| 617 | 56 | func sshresponse(c *gc.C, input string, output interface{}, rc int) func() { | ||
| 618 | 57 | fakebin := c.MkDir() | ||
| 619 | 58 | ssh := filepath.Join(fakebin, "ssh") | ||
| 620 | 59 | sshexpectedinput := ssh + ".expected-input" | ||
| 621 | 60 | var stdout, stderr string | ||
| 622 | 61 | switch output := output.(type) { | ||
| 623 | 62 | case nil: | ||
| 624 | 63 | case string: | ||
| 625 | 64 | stdout = fmt.Sprintf("cat<<EOF\n%s\nEOF", output) | ||
| 626 | 65 | case []string: | ||
| 627 | 66 | stdout = fmt.Sprintf("cat<<EOF\n%s\nEOF", output[0]) | ||
| 628 | 67 | stderr = fmt.Sprintf("cat>&2<<EOF\n%s\nEOF", output[1]) | ||
| 629 | 68 | } | ||
| 630 | 69 | script := fmt.Sprintf(sshscript, stdout, stderr, rc) | ||
| 631 | 70 | err := ioutil.WriteFile(ssh, []byte(script), 0777) | ||
| 632 | 71 | c.Assert(err, gc.IsNil) | ||
| 633 | 72 | err = ioutil.WriteFile(sshexpectedinput, []byte(input), 0644) | ||
| 634 | 73 | c.Assert(err, gc.IsNil) | ||
| 635 | 74 | return testbase.PatchEnvironment("PATH", fakebin+":"+os.Getenv("PATH")) | ||
| 636 | 75 | } | ||
| 637 | 76 | |||
| 638 | 77 | func (s *detectionSuite) TestDetectSeries(c *gc.C) { | 21 | func (s *detectionSuite) TestDetectSeries(c *gc.C) { |
| 639 | 78 | response := strings.Join([]string{ | 22 | response := strings.Join([]string{ |
| 640 | 79 | "edgy", | 23 | "edgy", |
| 641 | @@ -81,14 +25,14 @@ | |||
| 642 | 81 | "MemTotal: 4096 kB", | 25 | "MemTotal: 4096 kB", |
| 643 | 82 | "processor: 0", | 26 | "processor: 0", |
| 644 | 83 | }, "\n") | 27 | }, "\n") |
| 646 | 84 | defer sshresponse(c, detectionScript, response, 0)() | 28 | defer installFakeSSH(c, detectionScript, response, 0)() |
| 647 | 85 | _, series, err := detectSeriesAndHardwareCharacteristics("whatever") | 29 | _, series, err := detectSeriesAndHardwareCharacteristics("whatever") |
| 648 | 86 | c.Assert(err, gc.IsNil) | 30 | c.Assert(err, gc.IsNil) |
| 649 | 87 | c.Assert(series, gc.Equals, "edgy") | 31 | c.Assert(series, gc.Equals, "edgy") |
| 650 | 88 | } | 32 | } |
| 651 | 89 | 33 | ||
| 652 | 90 | func (s *detectionSuite) TestDetectionError(c *gc.C) { | 34 | func (s *detectionSuite) TestDetectionError(c *gc.C) { |
| 654 | 91 | defer sshresponse(c, detectionScript, "oh noes", 33)() | 35 | defer installFakeSSH(c, detectionScript, "oh noes", 33)() |
| 655 | 92 | _, _, err := detectSeriesAndHardwareCharacteristics("whatever") | 36 | _, _, err := detectSeriesAndHardwareCharacteristics("whatever") |
| 656 | 93 | c.Assert(err, gc.ErrorMatches, "exit status 33 \\(oh noes\\)") | 37 | c.Assert(err, gc.ErrorMatches, "exit status 33 \\(oh noes\\)") |
| 657 | 94 | } | 38 | } |
| 658 | @@ -148,7 +92,7 @@ | |||
| 659 | 148 | for i, test := range tests { | 92 | for i, test := range tests { |
| 660 | 149 | c.Logf("test %d: %s", i, test.summary) | 93 | c.Logf("test %d: %s", i, test.summary) |
| 661 | 150 | scriptResponse := strings.Join(test.scriptResponse, "\n") | 94 | scriptResponse := strings.Join(test.scriptResponse, "\n") |
| 663 | 151 | defer sshresponse(c, detectionScript, scriptResponse, 0)() | 95 | defer installFakeSSH(c, detectionScript, scriptResponse, 0)() |
| 664 | 152 | hc, _, err := detectSeriesAndHardwareCharacteristics("hostname") | 96 | hc, _, err := detectSeriesAndHardwareCharacteristics("hostname") |
| 665 | 153 | c.Assert(err, gc.IsNil) | 97 | c.Assert(err, gc.IsNil) |
| 666 | 154 | c.Assert(hc.String(), gc.Equals, test.expectedHc) | 98 | c.Assert(hc.String(), gc.Equals, test.expectedHc) |
| 667 | @@ -156,25 +100,25 @@ | |||
| 668 | 156 | } | 100 | } |
| 669 | 157 | 101 | ||
| 670 | 158 | func (s *detectionSuite) TestCheckProvisioned(c *gc.C) { | 102 | func (s *detectionSuite) TestCheckProvisioned(c *gc.C) { |
| 672 | 159 | defer sshresponse(c, checkProvisionedScript, "", 0)() | 103 | defer installFakeSSH(c, "", "", 0)() |
| 673 | 160 | provisioned, err := checkProvisioned("example.com") | 104 | provisioned, err := checkProvisioned("example.com") |
| 674 | 161 | c.Assert(err, gc.IsNil) | 105 | c.Assert(err, gc.IsNil) |
| 675 | 162 | c.Assert(provisioned, jc.IsFalse) | 106 | c.Assert(provisioned, jc.IsFalse) |
| 676 | 163 | 107 | ||
| 678 | 164 | defer sshresponse(c, checkProvisionedScript, "non-empty", 0)() | 108 | defer installFakeSSH(c, "", "non-empty", 0)() |
| 679 | 165 | provisioned, err = checkProvisioned("example.com") | 109 | provisioned, err = checkProvisioned("example.com") |
| 680 | 166 | c.Assert(err, gc.IsNil) | 110 | c.Assert(err, gc.IsNil) |
| 681 | 167 | c.Assert(provisioned, jc.IsTrue) | 111 | c.Assert(provisioned, jc.IsTrue) |
| 682 | 168 | 112 | ||
| 683 | 169 | // stderr should not affect result. | 113 | // stderr should not affect result. |
| 685 | 170 | defer sshresponse(c, checkProvisionedScript, []string{"", "non-empty-stderr"}, 0)() | 114 | defer installFakeSSH(c, "", []string{"", "non-empty-stderr"}, 0)() |
| 686 | 171 | provisioned, err = checkProvisioned("example.com") | 115 | provisioned, err = checkProvisioned("example.com") |
| 687 | 172 | c.Assert(err, gc.IsNil) | 116 | c.Assert(err, gc.IsNil) |
| 688 | 173 | c.Assert(provisioned, jc.IsFalse) | 117 | c.Assert(provisioned, jc.IsFalse) |
| 689 | 174 | 118 | ||
| 690 | 175 | // if the script fails for whatever reason, then checkProvisioned | 119 | // if the script fails for whatever reason, then checkProvisioned |
| 691 | 176 | // will return an error. stderr will be included in the error message. | 120 | // will return an error. stderr will be included in the error message. |
| 693 | 177 | defer sshresponse(c, checkProvisionedScript, []string{"non-empty-stdout", "non-empty-stderr"}, 255)() | 121 | defer installFakeSSH(c, "", []string{"non-empty-stdout", "non-empty-stderr"}, 255)() |
| 694 | 178 | _, err = checkProvisioned("example.com") | 122 | _, err = checkProvisioned("example.com") |
| 695 | 179 | c.Assert(err, gc.ErrorMatches, "exit status 255 \\(non-empty-stderr\\)") | 123 | c.Assert(err, gc.ErrorMatches, "exit status 255 \\(non-empty-stderr\\)") |
| 696 | 180 | } | 124 | } |
| 697 | 181 | 125 | ||
| 698 | === added file 'environs/manual/fakessh_test.go' | |||
| 699 | --- environs/manual/fakessh_test.go 1970-01-01 00:00:00 +0000 | |||
| 700 | +++ environs/manual/fakessh_test.go 2013-09-20 05:05:14 +0000 | |||
| 701 | @@ -0,0 +1,112 @@ | |||
| 702 | 1 | // Copyright 2013 Canonical Ltd. | ||
| 703 | 2 | // Licensed under the AGPLv3, see LICENCE file for details. | ||
| 704 | 3 | |||
| 705 | 4 | package manual | ||
| 706 | 5 | |||
| 707 | 6 | import ( | ||
| 708 | 7 | "fmt" | ||
| 709 | 8 | "io/ioutil" | ||
| 710 | 9 | "os" | ||
| 711 | 10 | "path/filepath" | ||
| 712 | 11 | "strings" | ||
| 713 | 12 | |||
| 714 | 13 | gc "launchpad.net/gocheck" | ||
| 715 | 14 | |||
| 716 | 15 | "launchpad.net/juju-core/testing/testbase" | ||
| 717 | 16 | ) | ||
| 718 | 17 | |||
| 719 | 18 | // sshscript should only print the result on the first execution, | ||
| 720 | 19 | // to handle the case where it's called multiple times. On | ||
| 721 | 20 | // subsequent executions, it should find the next 'ssh' in $PATH | ||
| 722 | 21 | // and exec that. | ||
| 723 | 22 | var sshscript = `#!/bin/bash | ||
| 724 | 23 | if [ ! -e "$0.run" ]; then | ||
| 725 | 24 | touch "$0.run" | ||
| 726 | 25 | diff "$0.expected-input" - | ||
| 727 | 26 | exitcode=$? | ||
| 728 | 27 | if [ $exitcode -ne 0 ]; then | ||
| 729 | 28 | echo "ERROR: did not match expected input" >&2 | ||
| 730 | 29 | exit $exitcode | ||
| 731 | 30 | fi | ||
| 732 | 31 | # stdout | ||
| 733 | 32 | %s | ||
| 734 | 33 | # stderr | ||
| 735 | 34 | %s | ||
| 736 | 35 | exit %d | ||
| 737 | 36 | else | ||
| 738 | 37 | export PATH=${PATH#*:} | ||
| 739 | 38 | exec ssh $* | ||
| 740 | 39 | fi` | ||
| 741 | 40 | |||
| 742 | 41 | // installFakeSSH creates a fake "ssh" command in a new $PATH, | ||
| 743 | 42 | // updates $PATH, and returns a function to reset $PATH to | ||
| 744 | 43 | // its original value when called. | ||
| 745 | 44 | // | ||
| 746 | 45 | // output may be: | ||
| 747 | 46 | // - nil (no output) | ||
| 748 | 47 | // - a string (stdout) | ||
| 749 | 48 | // - a slice of strings, of length two (stdout, stderr) | ||
| 750 | 49 | func installFakeSSH(c *gc.C, input string, output interface{}, rc int) testbase.Restorer { | ||
| 751 | 50 | fakebin := c.MkDir() | ||
| 752 | 51 | ssh := filepath.Join(fakebin, "ssh") | ||
| 753 | 52 | sshexpectedinput := ssh + ".expected-input" | ||
| 754 | 53 | var stdout, stderr string | ||
| 755 | 54 | switch output := output.(type) { | ||
| 756 | 55 | case nil: | ||
| 757 | 56 | case string: | ||
| 758 | 57 | stdout = fmt.Sprintf("cat<<EOF\n%s\nEOF", output) | ||
| 759 | 58 | case []string: | ||
| 760 | 59 | stdout = fmt.Sprintf("cat<<EOF\n%s\nEOF", output[0]) | ||
| 761 | 60 | stderr = fmt.Sprintf("cat>&2<<EOF\n%s\nEOF", output[1]) | ||
| 762 | 61 | } | ||
| 763 | 62 | script := fmt.Sprintf(sshscript, stdout, stderr, rc) | ||
| 764 | 63 | err := ioutil.WriteFile(ssh, []byte(script), 0777) | ||
| 765 | 64 | c.Assert(err, gc.IsNil) | ||
| 766 | 65 | err = ioutil.WriteFile(sshexpectedinput, []byte(input), 0644) | ||
| 767 | 66 | c.Assert(err, gc.IsNil) | ||
| 768 | 67 | return testbase.PatchEnvironment("PATH", fakebin+":"+os.Getenv("PATH")) | ||
| 769 | 68 | } | ||
| 770 | 69 | |||
| 771 | 70 | // fakeSSH wraps the invocation of installFakeSSH based on the parameters. | ||
| 772 | 71 | type fakeSSH struct { | ||
| 773 | 72 | series string | ||
| 774 | 73 | arch string | ||
| 775 | 74 | |||
| 776 | 75 | // exit code for the machine agent provisioning script. | ||
| 777 | 76 | provisionAgentExitCode int | ||
| 778 | 77 | |||
| 779 | 78 | // there are conditions other than error in the above | ||
| 780 | 79 | // that might cause provisioning to not go ahead, such | ||
| 781 | 80 | // as tools being missing. | ||
| 782 | 81 | skipProvisionAgent bool | ||
| 783 | 82 | } | ||
| 784 | 83 | |||
| 785 | 84 | // install installs fake SSH commands, which will respond to | ||
| 786 | 85 | // manual provisioning/bootstrapping commands with the specified | ||
| 787 | 86 | // output and exit codes. | ||
| 788 | 87 | func (r fakeSSH) install(c *gc.C) testbase.Restorer { | ||
| 789 | 88 | series := r.series | ||
| 790 | 89 | if series == "" { | ||
| 791 | 90 | series = "precise" | ||
| 792 | 91 | } | ||
| 793 | 92 | arch := r.arch | ||
| 794 | 93 | if arch == "" { | ||
| 795 | 94 | arch = "amd64" | ||
| 796 | 95 | } | ||
| 797 | 96 | detectionoutput := strings.Join([]string{ | ||
| 798 | 97 | series, | ||
| 799 | 98 | arch, | ||
| 800 | 99 | "MemTotal: 4096 kB", | ||
| 801 | 100 | "processor: 0", | ||
| 802 | 101 | }, "\n") | ||
| 803 | 102 | var restore testbase.Restorer | ||
| 804 | 103 | add := func(input string, output interface{}, rc int) { | ||
| 805 | 104 | restore = restore.Add(installFakeSSH(c, input, output, rc)) | ||
| 806 | 105 | } | ||
| 807 | 106 | if !r.skipProvisionAgent { | ||
| 808 | 107 | add("", nil, r.provisionAgentExitCode) | ||
| 809 | 108 | } | ||
| 810 | 109 | add(detectionScript, detectionoutput, 0) | ||
| 811 | 110 | add("", nil, 0) // checkProvisioned | ||
| 812 | 111 | return restore | ||
| 813 | 112 | } | ||
| 814 | 0 | 113 | ||
| 815 | === modified file 'environs/manual/provisioner.go' | |||
| 816 | --- environs/manual/provisioner.go 2013-09-18 04:43:04 +0000 | |||
| 817 | +++ environs/manual/provisioner.go 2013-09-20 05:05:14 +0000 | |||
| 818 | @@ -104,6 +104,14 @@ | |||
| 819 | 104 | return nil, err | 104 | return nil, err |
| 820 | 105 | } | 105 | } |
| 821 | 106 | 106 | ||
| 822 | 107 | tools := args.Tools | ||
| 823 | 108 | if tools == nil { | ||
| 824 | 109 | tools, err = findInstanceTools(env, series, *hc.Arch) | ||
| 825 | 110 | if err != nil { | ||
| 826 | 111 | return nil, err | ||
| 827 | 112 | } | ||
| 828 | 113 | } | ||
| 829 | 114 | |||
| 830 | 107 | // Generate a unique nonce for the machine. | 115 | // Generate a unique nonce for the machine. |
| 831 | 108 | uuid, err := utils.NewUUID() | 116 | uuid, err := utils.NewUUID() |
| 832 | 109 | if err != nil { | 117 | if err != nil { |
| 833 | @@ -137,16 +145,15 @@ | |||
| 834 | 137 | 145 | ||
| 835 | 138 | // Finally, provision the machine agent. | 146 | // Finally, provision the machine agent. |
| 836 | 139 | err = provisionMachineAgent(provisionMachineAgentArgs{ | 147 | err = provisionMachineAgent(provisionMachineAgentArgs{ |
| 847 | 140 | host: args.Host, | 148 | host: args.Host, |
| 848 | 141 | dataDir: args.DataDir, | 149 | dataDir: args.DataDir, |
| 849 | 142 | env: env, | 150 | environConfig: env.Config(), |
| 850 | 143 | machine: m, | 151 | machineId: m.Id(), |
| 851 | 144 | nonce: nonce, | 152 | bootstrap: false, |
| 852 | 145 | stateInfo: stateInfo, | 153 | nonce: nonce, |
| 853 | 146 | apiInfo: apiInfo, | 154 | stateInfo: stateInfo, |
| 854 | 147 | series: series, | 155 | apiInfo: apiInfo, |
| 855 | 148 | arch: *hc.Arch, | 156 | tools: tools, |
| 846 | 149 | tools: args.Tools, | ||
| 856 | 150 | }) | 157 | }) |
| 857 | 151 | if err != nil { | 158 | if err != nil { |
| 858 | 152 | return m, err | 159 | return m, err |
| 859 | 153 | 160 | ||
| 860 | === modified file 'environs/manual/provisioner_test.go' | |||
| 861 | --- environs/manual/provisioner_test.go 2013-09-20 00:33:38 +0000 | |||
| 862 | +++ environs/manual/provisioner_test.go 2013-09-20 05:05:14 +0000 | |||
| 863 | @@ -6,13 +6,15 @@ | |||
| 864 | 6 | import ( | 6 | import ( |
| 865 | 7 | "fmt" | 7 | "fmt" |
| 866 | 8 | "os" | 8 | "os" |
| 867 | 9 | "strings" | ||
| 868 | 10 | 9 | ||
| 869 | 11 | gc "launchpad.net/gocheck" | 10 | gc "launchpad.net/gocheck" |
| 870 | 12 | 11 | ||
| 872 | 13 | "launchpad.net/juju-core/environs/tools" | 12 | "launchpad.net/juju-core/environs/storage" |
| 873 | 13 | envtesting "launchpad.net/juju-core/environs/testing" | ||
| 874 | 14 | "launchpad.net/juju-core/instance" | 14 | "launchpad.net/juju-core/instance" |
| 875 | 15 | "launchpad.net/juju-core/juju/testing" | 15 | "launchpad.net/juju-core/juju/testing" |
| 876 | 16 | jc "launchpad.net/juju-core/testing/checkers" | ||
| 877 | 17 | "launchpad.net/juju-core/version" | ||
| 878 | 16 | ) | 18 | ) |
| 879 | 17 | 19 | ||
| 880 | 18 | type provisionerSuite struct { | 20 | type provisionerSuite struct { |
| 881 | @@ -31,32 +33,35 @@ | |||
| 882 | 31 | } | 33 | } |
| 883 | 32 | 34 | ||
| 884 | 33 | func (s *provisionerSuite) TestProvisionMachine(c *gc.C) { | 35 | func (s *provisionerSuite) TestProvisionMachine(c *gc.C) { |
| 892 | 34 | // Prepare a mock ssh response for the detection phase. | 36 | const series = "precise" |
| 893 | 35 | detectionoutput := strings.Join([]string{ | 37 | const arch = "amd64" |
| 887 | 36 | "edgy", | ||
| 888 | 37 | "armv4", | ||
| 889 | 38 | "MemTotal: 4096 kB", | ||
| 890 | 39 | "processor: 0", | ||
| 891 | 40 | }, "\n") | ||
| 894 | 41 | 38 | ||
| 895 | 42 | args := s.getArgs(c) | 39 | args := s.getArgs(c) |
| 896 | 43 | hostname := args.Host | 40 | hostname := args.Host |
| 897 | 44 | args.Host = "ubuntu@" + args.Host | 41 | args.Host = "ubuntu@" + args.Host |
| 898 | 45 | 42 | ||
| 901 | 46 | defer sshresponse(c, detectionScript, detectionoutput, 0)() | 43 | envtesting.RemoveTools(c, s.Conn.Environ.Storage()) |
| 902 | 47 | defer sshresponse(c, checkProvisionedScript, "", 0)() | 44 | envtesting.RemoveTools(c, s.Conn.Environ.PublicStorage().(storage.Storage)) |
| 903 | 45 | defer fakeSSH{ | ||
| 904 | 46 | series: series, arch: arch, skipProvisionAgent: true, | ||
| 905 | 47 | }.install(c).Restore() | ||
| 906 | 48 | m, err := ProvisionMachine(args) | 48 | m, err := ProvisionMachine(args) |
| 908 | 49 | c.Assert(err, gc.ErrorMatches, "no matching tools available") | 49 | c.Assert(err, gc.ErrorMatches, "no tools available") |
| 909 | 50 | c.Assert(m, gc.IsNil) | 50 | c.Assert(m, gc.IsNil) |
| 910 | 51 | 51 | ||
| 914 | 52 | toolsList, err := tools.FindBootstrapTools(s.Conn.Environ, tools.BootstrapToolsParams{}) | 52 | cfg := s.Conn.Environ.Config() |
| 915 | 53 | c.Assert(err, gc.IsNil) | 53 | number, ok := cfg.AgentVersion() |
| 916 | 54 | args.Tools = toolsList[0] | 54 | c.Assert(ok, jc.IsTrue) |
| 917 | 55 | binVersion := version.Binary{number, series, arch} | ||
| 918 | 56 | envtesting.UploadFakeToolsVersion(c, s.Conn.Environ.Storage(), binVersion) | ||
| 919 | 55 | 57 | ||
| 924 | 56 | for _, errorCode := range []int{255, 0} { | 58 | for i, errorCode := range []int{255, 0} { |
| 925 | 57 | defer sshresponse(c, "", "", errorCode)() // executing script | 59 | c.Logf("test %d: code %d", i, errorCode) |
| 926 | 58 | defer sshresponse(c, detectionScript, detectionoutput, 0)() | 60 | defer fakeSSH{ |
| 927 | 59 | defer sshresponse(c, checkProvisionedScript, "", 0)() | 61 | series: series, |
| 928 | 62 | arch: arch, | ||
| 929 | 63 | provisionAgentExitCode: errorCode, | ||
| 930 | 64 | }.install(c).Restore() | ||
| 931 | 60 | m, err = ProvisionMachine(args) | 65 | m, err = ProvisionMachine(args) |
| 932 | 61 | if errorCode != 0 { | 66 | if errorCode != 0 { |
| 933 | 62 | c.Assert(err, gc.ErrorMatches, fmt.Sprintf("exit status %d", errorCode)) | 67 | c.Assert(err, gc.ErrorMatches, fmt.Sprintf("exit status %d", errorCode)) |
| 934 | @@ -64,9 +69,9 @@ | |||
| 935 | 64 | } else { | 69 | } else { |
| 936 | 65 | c.Assert(err, gc.IsNil) | 70 | c.Assert(err, gc.IsNil) |
| 937 | 66 | c.Assert(m, gc.NotNil) | 71 | c.Assert(m, gc.NotNil) |
| 939 | 67 | // machine ID will be 2, not 1. Even though we failed and the | 72 | // machine ID will be incremented. Even though we failed and the |
| 940 | 68 | // machine is removed, the ID is not reused. | 73 | // machine is removed, the ID is not reused. |
| 942 | 69 | c.Assert(m.Id(), gc.Equals, "2") | 74 | c.Assert(m.Id(), gc.Equals, fmt.Sprint(i)) |
| 943 | 70 | instanceId, err := m.InstanceId() | 75 | instanceId, err := m.InstanceId() |
| 944 | 71 | c.Assert(err, gc.IsNil) | 76 | c.Assert(err, gc.IsNil) |
| 945 | 72 | c.Assert(instanceId, gc.Equals, instance.Id("manual:"+hostname)) | 77 | c.Assert(instanceId, gc.Equals, instance.Id("manual:"+hostname)) |
| 946 | @@ -75,10 +80,10 @@ | |||
| 947 | 75 | 80 | ||
| 948 | 76 | // Attempting to provision a machine twice should fail. We effect | 81 | // Attempting to provision a machine twice should fail. We effect |
| 949 | 77 | // this by checking for existing juju upstart configurations. | 82 | // this by checking for existing juju upstart configurations. |
| 951 | 78 | defer sshresponse(c, checkProvisionedScript, "/etc/init/jujud-machine-0.conf", 0)() | 83 | defer installFakeSSH(c, "", "/etc/init/jujud-machine-0.conf", 0)() |
| 952 | 79 | _, err = ProvisionMachine(args) | 84 | _, err = ProvisionMachine(args) |
| 953 | 80 | c.Assert(err, gc.Equals, ErrProvisioned) | 85 | c.Assert(err, gc.Equals, ErrProvisioned) |
| 955 | 81 | defer sshresponse(c, checkProvisionedScript, "/etc/init/jujud-machine-0.conf", 255)() | 86 | defer installFakeSSH(c, "", "/etc/init/jujud-machine-0.conf", 255)() |
| 956 | 82 | _, err = ProvisionMachine(args) | 87 | _, err = ProvisionMachine(args) |
| 957 | 83 | c.Assert(err, gc.ErrorMatches, "error checking if provisioned: exit status 255") | 88 | c.Assert(err, gc.ErrorMatches, "error checking if provisioned: exit status 255") |
| 958 | 84 | } | 89 | } |
| 959 | 85 | 90 | ||
| 960 | === added file 'environs/manual/tools.go' | |||
| 961 | --- environs/manual/tools.go 1970-01-01 00:00:00 +0000 | |||
| 962 | +++ environs/manual/tools.go 2013-09-20 05:05:14 +0000 | |||
| 963 | @@ -0,0 +1,24 @@ | |||
| 964 | 1 | // Copyright 2013 Canonical Ltd. | ||
| 965 | 2 | // Licensed under the AGPLv3, see LICENCE file for details. | ||
| 966 | 3 | |||
| 967 | 4 | package manual | ||
| 968 | 5 | |||
| 969 | 6 | import ( | ||
| 970 | 7 | "fmt" | ||
| 971 | 8 | |||
| 972 | 9 | "launchpad.net/juju-core/environs" | ||
| 973 | 10 | envtools "launchpad.net/juju-core/environs/tools" | ||
| 974 | 11 | "launchpad.net/juju-core/tools" | ||
| 975 | 12 | ) | ||
| 976 | 13 | |||
| 977 | 14 | func findInstanceTools(env environs.Environ, series, arch string) (*tools.Tools, error) { | ||
| 978 | 15 | agentVersion, ok := env.Config().AgentVersion() | ||
| 979 | 16 | if !ok { | ||
| 980 | 17 | return nil, fmt.Errorf("no agent version set in environment configuration") | ||
| 981 | 18 | } | ||
| 982 | 19 | possibleTools, err := envtools.FindInstanceTools(env, agentVersion, series, &arch) | ||
| 983 | 20 | if err != nil { | ||
| 984 | 21 | return nil, err | ||
| 985 | 22 | } | ||
| 986 | 23 | return possibleTools[0], nil | ||
| 987 | 24 | } | ||
| 988 | 0 | 25 | ||
| 989 | === modified file 'testing/testbase/patch.go' | |||
| 990 | --- testing/testbase/patch.go 2013-09-20 02:10:46 +0000 | |||
| 991 | +++ testing/testbase/patch.go 2013-09-20 05:05:14 +0000 | |||
| 992 | @@ -12,6 +12,18 @@ | |||
| 993 | 12 | // to restore some previous state. | 12 | // to restore some previous state. |
| 994 | 13 | type Restorer func() | 13 | type Restorer func() |
| 995 | 14 | 14 | ||
| 996 | 15 | // Add returns a Restorer that restores first f1 | ||
| 997 | 16 | // and then f. It is valid to call this on a nil | ||
| 998 | 17 | // Restorer. | ||
| 999 | 18 | func (f Restorer) Add(f1 Restorer) Restorer { | ||
| 1000 | 19 | return func() { | ||
| 1001 | 20 | f1.Restore() | ||
| 1002 | 21 | if f != nil { | ||
| 1003 | 22 | f.Restore() | ||
| 1004 | 23 | } | ||
| 1005 | 24 | } | ||
| 1006 | 25 | } | ||
| 1007 | 26 | |||
| 1008 | 15 | // Restore restores some previous state. | 27 | // Restore restores some previous state. |
| 1009 | 16 | func (r Restorer) Restore() { | 28 | func (r Restorer) Restore() { |
| 1010 | 17 | r() | 29 | r() |
| 1011 | 18 | 30 | ||
| 1012 | === modified file 'testing/testbase/patch_test.go' | |||
| 1013 | --- testing/testbase/patch_test.go 2013-09-20 02:10:46 +0000 | |||
| 1014 | +++ testing/testbase/patch_test.go 2013-09-20 05:05:14 +0000 | |||
| 1015 | @@ -74,3 +74,12 @@ | |||
| 1016 | 74 | c.Check(os.Getenv(envName), gc.Equals, "initial") | 74 | c.Check(os.Getenv(envName), gc.Equals, "initial") |
| 1017 | 75 | os.Setenv(envName, oldValue) | 75 | os.Setenv(envName, oldValue) |
| 1018 | 76 | } | 76 | } |
| 1019 | 77 | |||
| 1020 | 78 | func (*PatchEnvironmentSuite) TestRestorerAdd(c *gc.C) { | ||
| 1021 | 79 | var order []string | ||
| 1022 | 80 | first := testbase.Restorer(func() { order = append(order, "first") }) | ||
| 1023 | 81 | second := testbase.Restorer(func() { order = append(order, "second") }) | ||
| 1024 | 82 | restore := first.Add(second) | ||
| 1025 | 83 | restore() | ||
| 1026 | 84 | c.Assert(order, gc.DeepEquals, []string{"second", "first"}) | ||
| 1027 | 85 | } | ||
| 1028 | 77 | 86 | ||
| 1029 | === renamed directory 'provider/local/storage' => 'worker/localstorage' | |||
| 1030 | === added file 'worker/localstorage/config.go' | |||
| 1031 | --- worker/localstorage/config.go 1970-01-01 00:00:00 +0000 | |||
| 1032 | +++ worker/localstorage/config.go 2013-09-20 05:05:14 +0000 | |||
| 1033 | @@ -0,0 +1,14 @@ | |||
| 1034 | 1 | // Copyright 2013 Canonical Ltd. | ||
| 1035 | 2 | // Licensed under the AGPLv3, see LICENCE file for details. | ||
| 1036 | 3 | |||
| 1037 | 4 | package localstorage | ||
| 1038 | 5 | |||
| 1039 | 6 | // LocalStorageConfig is an interface that, if implemented, may be used | ||
| 1040 | 7 | // to configure a machine agent for use with the localstorage worker in | ||
| 1041 | 8 | // this package. | ||
| 1042 | 9 | type LocalStorageConfig interface { | ||
| 1043 | 10 | StorageDir() string | ||
| 1044 | 11 | StorageAddr() string | ||
| 1045 | 12 | SharedStorageDir() string | ||
| 1046 | 13 | SharedStorageAddr() string | ||
| 1047 | 14 | } | ||
| 1048 | 0 | 15 | ||
| 1049 | === modified file 'worker/localstorage/worker.go' | |||
| 1050 | --- provider/local/storage/worker.go 2013-09-19 03:14:56 +0000 | |||
| 1051 | +++ worker/localstorage/worker.go 2013-09-20 05:05:14 +0000 | |||
| 1052 | @@ -1,4 +1,7 @@ | |||
| 1054 | 1 | package storage | 1 | // Copyright 2013 Canonical Ltd. |
| 1055 | 2 | // Licensed under the AGPLv3, see LICENCE file for details. | ||
| 1056 | 3 | |||
| 1057 | 4 | package localstorage | ||
| 1058 | 2 | 5 | ||
| 1059 | 3 | import ( | 6 | import ( |
| 1060 | 4 | "launchpad.net/loggo" | 7 | "launchpad.net/loggo" |
| 1061 | @@ -10,7 +13,7 @@ | |||
| 1062 | 10 | "launchpad.net/juju-core/worker" | 13 | "launchpad.net/juju-core/worker" |
| 1063 | 11 | ) | 14 | ) |
| 1064 | 12 | 15 | ||
| 1066 | 13 | var logger = loggo.GetLogger("juju.local.storage") | 16 | var logger = loggo.GetLogger("juju.worker.localstorage") |
| 1067 | 14 | 17 | ||
| 1068 | 15 | type storageWorker struct { | 18 | type storageWorker struct { |
| 1069 | 16 | config agent.Config | 19 | config agent.Config |
| 1070 | @@ -55,19 +58,22 @@ | |||
| 1071 | 55 | 58 | ||
| 1072 | 56 | sharedStorageDir := s.config.Value(agent.SharedStorageDir) | 59 | sharedStorageDir := s.config.Value(agent.SharedStorageDir) |
| 1073 | 57 | sharedStorageAddr := s.config.Value(agent.SharedStorageAddr) | 60 | sharedStorageAddr := s.config.Value(agent.SharedStorageAddr) |
| 1087 | 58 | logger.Infof("serving %s on %s", sharedStorageDir, sharedStorageAddr) | 61 | if sharedStorageAddr != "" && sharedStorageDir != "" { |
| 1088 | 59 | 62 | logger.Infof("serving %s on %s", sharedStorageDir, sharedStorageAddr) | |
| 1089 | 60 | sharedStorage, err := filestorage.NewFileStorageWriter(sharedStorageDir, filestorage.UseDefaultTmpDir) | 63 | sharedStorage, err := filestorage.NewFileStorageWriter(sharedStorageDir, filestorage.UseDefaultTmpDir) |
| 1090 | 61 | if err != nil { | 64 | if err != nil { |
| 1091 | 62 | logger.Errorf("error with local storage: %v", err) | 65 | logger.Errorf("error with local storage: %v", err) |
| 1092 | 63 | return err | 66 | return err |
| 1093 | 64 | } | 67 | } |
| 1094 | 65 | sharedStorageListener, err := httpstorage.Serve(sharedStorageAddr, sharedStorage) | 68 | sharedStorageListener, err := httpstorage.Serve(sharedStorageAddr, sharedStorage) |
| 1095 | 66 | if err != nil { | 69 | if err != nil { |
| 1096 | 67 | logger.Errorf("error with local storage: %v", err) | 70 | logger.Errorf("error with local storage: %v", err) |
| 1097 | 68 | return err | 71 | return err |
| 1098 | 69 | } | 72 | } |
| 1099 | 70 | defer sharedStorageListener.Close() | 73 | defer sharedStorageListener.Close() |
| 1100 | 74 | } else { | ||
| 1101 | 75 | logger.Infof("no shared storage: dir=%q addr=%q", sharedStorageDir, sharedStorageAddr) | ||
| 1102 | 76 | } | ||
| 1103 | 71 | 77 | ||
| 1104 | 72 | logger.Infof("storage routines started, awaiting death") | 78 | logger.Infof("storage routines started, awaiting death") |
| 1105 | 73 | 79 | ||
Reviewers: mp+184714_
Message:
Please take a look.
Description:
Implement manual bootstrapping
environs/bootstrap:
- Added BootstrapStorage interface. If an Environ
implements this interface, its BootstrapStorage
will be used by cmd/juju bootstrap.
cmd/juju:
- Updated bootstrap subcommand to use
environs/
as described above.
provider/
- Added LocalStorageConfig interface, which an
Environ may implement to describe the configuration
of a "localstorage".
environs/
- Added NewFileStorage (i.e. implement StorageWrite)
for testing in environs/manual.
environs/manual:
- Refactored existing code to support bootstrap machines.
- Added "Bootstrap" function to manually bootstrap an
existing machine.
To manually bootstrap, you must provide an environment that
implements the new LocalStorageConfig and BootstrapStorage
interfaces. In other words, you can (currently) only manually
bootstrap a machine which is expected to manage the storage
for its environment. This may be relaxed later if needed.
Next up is the null provider, which uses this new Bootstrap
function. I need to address one last issue with it first:
the null provider mustn't narrow the tools to default-series.
https:/
(do not edit description out of merge proposal)
Please review this at https:/
Affected files (+525, -53 lines):
A [revision details]
M cmd/juju/
M environs/
M environs/
M environs/
A environs/
A environs/
M environs/
M environs/
A environs/
A provider/
M provider/