Merge lp:~axwalk/juju-core/synchronous-bootstrap into lp:~go-bot/juju-core/trunk

Proposed by Andrew Wilkins
Status: Merged
Approved by: Andrew Wilkins
Approved revision: no longer in the source branch.
Merged at revision: 2116
Proposed branch: lp:~axwalk/juju-core/synchronous-bootstrap
Merge into: lp:~go-bot/juju-core/trunk
Diff against target: 1532 lines (+680/-120)
30 files modified
cloudinit/cloudinit_test.go (+31/-0)
cloudinit/options.go (+21/-0)
cloudinit/progress.go (+43/-0)
cloudinit/progress_test.go (+27/-0)
cloudinit/sshinit/configure.go (+45/-29)
cloudinit/sshinit/configure_test.go (+68/-18)
cloudinit/sshinit/suite_test.go (+14/-0)
cmd/juju/debughooks.go (+1/-1)
cmd/juju/debughooks_test.go (+3/-3)
cmd/juju/scp.go (+2/-4)
cmd/juju/scp_test.go (+1/-1)
cmd/juju/ssh.go (+8/-4)
cmd/juju/ssh_test.go (+7/-7)
environs/bootstrap/state.go (+5/-0)
environs/bootstrap/state_test.go (+21/-0)
environs/cloudinit.go (+10/-3)
environs/cloudinit/cloudinit.go (+56/-8)
environs/cloudinit/cloudinit_test.go (+8/-0)
environs/cloudinit_test.go (+38/-4)
environs/manual/bootstrap.go (+1/-1)
environs/manual/detection.go (+3/-2)
environs/manual/provisioner.go (+14/-1)
environs/sshstorage/storage.go (+4/-4)
environs/testing/bootstrap.go (+26/-0)
provider/common/bootstrap.go (+163/-8)
provider/common/bootstrap_test.go (+3/-0)
provider/ec2/local_test.go (+16/-10)
provider/maas/maas_test.go (+2/-0)
provider/openstack/local_test.go (+7/-0)
utils/ssh/ssh.go (+32/-12)
To merge this branch: bzr merge lp:~axwalk/juju-core/synchronous-bootstrap
Reviewer Review Type Date Requested Status
Juju Engineering Pending
Review via email: mp+196058@code.launchpad.net

Commit message

Implement synchronous bootstrap

environs/manual has been further refactored to
split the "manual cloud-init over SSH" out into
a separate package (cloudinit/sshinit).

provider/common now starts an instance with the
basic cloud-init only (SSH keys + logging), then
waits for a DNS name, waits to be able to connect
via SSH, and then defers to sshinit to execute the
remaining cloud-init steps.

If the user hits Ctrl-C, SIGINT will terminate
the SSH connection, and the bootstrap process will
attempt to clean up by stopping the instance and
removing the state file (if the instance could be
cleanly stopped). We also ignore SIGINT in the
juju process so a second Ctrl-C will not terminate
the cleanup procedure.

Fixes #1224230

https://codereview.appspot.com/30190043/

Description of the change

Implement synchronous bootstrap

environs/manual has been further refactored to
split the "manual cloud-init over SSH" out into
a separate package (cloudinit/sshinit).

provider/common now starts an instance with the
basic cloud-init only (SSH keys + logging), then
waits for a DNS name, waits to be able to connect
via SSH, and then defers to sshinit to execute the
remaining cloud-init steps.

If the user hits Ctrl-C, SIGINT will terminate
the SSH connection, and the bootstrap process will
attempt to clean up by stopping the instance and
removing the state file (if the instance could be
cleanly stopped). We also ignore SIGINT in the
juju process so a second Ctrl-C will not terminate
the cleanup procedure.

Fixes #1224230

https://codereview.appspot.com/30190043/

To post a comment you must log in.
Revision history for this message
Andrew Wilkins (axwalk) wrote :

Reviewers: mp+196058_code.launchpad.net,

Message:
Please take a look.

Description:
Implement synchronous bootstrap

environs/manual has been further refactored to
split the "manual cloud-init over SSH" out into
a separate package (cloudinit/sshinit).

provider/common now starts an instance with the
basic cloud-init only (SSH keys + logging), then
waits for a DNS name, waits to be able to connect
via SSH, and then defers to sshinit to execute the
remaining cloud-init steps.

If the user hits Ctrl-C, SIGINT will terminate
the SSH connection, and the bootstrap process will
attempt to clean up by stopping the instance and
removing the state file (if the instance could be
cleanly stopped). We also ignore SIGINT in the
juju process so a second Ctrl-C will not terminate
the cleanup procedure.

Fixes #1224230

https://code.launchpad.net/~axwalk/juju-core/synchronous-bootstrap/+merge/196058

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/30190043/

Affected files (+334, -88 lines):
   A [revision details]
   M cloudinit/options.go
   M cloudinit/sshinit/configure.go
   M cloudinit/sshinit/configure_test.go
   A cloudinit/sshinit/suite_test.go
   M cmd/juju/bootstrap.go
   M environs/bootstrap/state.go
   M environs/bootstrap/state_test.go
   M environs/cloudinit.go
   M environs/cloudinit/cloudinit.go
   M environs/cloudinit_test.go
   M environs/manual/bootstrap.go
   M environs/manual/detection.go
   M environs/manual/provisioner.go
   M provider/common/bootstrap.go
   M provider/common/bootstrap_test.go
   M provider/ec2/local_test.go
   M provider/maas/maas_test.go
   M provider/openstack/local_test.go
   M utils/ssh/ssh.go

Revision history for this message
Ian Booth (wallyworld) wrote :

This looks pretty solid. I like how manual provisioning and cloud
provisioning now share more code.

Before landing, please test live on each of the providers.

I have a nagging concern about the finishBootstrap step. WaitDNS and the
ssh connect retry should work most if not all of the time, but there are
times when canonistack for example is reaaaally slow to start an
instance. It worries me that we could needlessly kill bootstrap if we
think we have waited long enough but the provider is just slow.
Thoughts?

https://codereview.appspot.com/30190043/diff/1/cloudinit/sshinit/configure_test.go
File cloudinit/sshinit/configure_test.go (left):

https://codereview.appspot.com/30190043/diff/1/cloudinit/sshinit/configure_test.go#oldcode3
cloudinit/sshinit/configure_test.go:3:
Is there a test that "apt-get -y upgrade" is only used when asked for? I
can't see it if there is. I think this is a pretty important change so
warrants an explicit test.

https://codereview.appspot.com/30190043/diff/1/cmd/juju/bootstrap.go
File cmd/juju/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/1/cmd/juju/bootstrap.go#newcode126
cmd/juju/bootstrap.go:126: signal.Notify(make(chan os.Signal),
os.Interrupt)
Just a thought. One other option is to install a handler which prints a
message informing the user that their Ctrl-C cannot terminate the cmd
because cleanup is being done.

https://codereview.appspot.com/30190043/diff/1/environs/cloudinit.go
File environs/cloudinit.go (right):

https://codereview.appspot.com/30190043/diff/1/environs/cloudinit.go#newcode135
environs/cloudinit.go:135: // and setup the SSH keys. The rest we leave
to environs/manual.
environs/manual or cloudinit/sshinit?

https://codereview.appspot.com/30190043/diff/1/environs/cloudinit_test.go
File environs/cloudinit_test.go (right):

https://codereview.appspot.com/30190043/diff/1/environs/cloudinit_test.go#newcode207
environs/cloudinit_test.go:207: // for MAAS. MAAS needs to configure and
thren bounce the
typo

https://codereview.appspot.com/30190043/diff/1/provider/common/bootstrap.go
File provider/common/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/1/provider/common/bootstrap.go#newcode35
provider/common/bootstrap.go:35: defer func() {
Could we break this logic out into a separate function outside of
Bootstrap for clarity. The func could be

handleBootstrapError(err error, inst instance.Instance)

https://codereview.appspot.com/30190043/diff/1/provider/common/bootstrap.go#newcode95
provider/common/bootstrap.go:95: func TestingDisableFinishBootstrap()
func() {
I was going to suggest move this to an export_tests.go file but I see
it's called outside this package. Bollocks.

https://codereview.appspot.com/30190043/diff/1/provider/common/bootstrap.go#newcode104
provider/common/bootstrap.go:104: func TestingPatchFinishBootstrap(f
func(instance.Instance, *cloudinit.MachineConfig) error) func() {
Does this need to be exported? It's only called from the above.

https://codereview.appspot.com/30190043/

Revision history for this message
Andrew Wilkins (axwalk) wrote :
Download full text (3.4 KiB)

I've tested against ec2, will do openstack and azure later. I don't have
access to a MAAS installation, so that may take a little while.

I've not made any changes to the signal handling yet, but I will do so.
I need to think a bit more about how to handle it.

https://codereview.appspot.com/30190043/diff/1/cloudinit/sshinit/configure_test.go
File cloudinit/sshinit/configure_test.go (left):

https://codereview.appspot.com/30190043/diff/1/cloudinit/sshinit/configure_test.go#oldcode3
cloudinit/sshinit/configure_test.go:3:
On 2013/11/22 01:54:00, wallyworld wrote:
> Is there a test that "apt-get -y upgrade" is only used when asked for?
I can't
> see it if there is. I think this is a pretty important change so
warrants an
> explicit test.

Added a test, also for apt-get update.

https://codereview.appspot.com/30190043/diff/1/cmd/juju/bootstrap.go
File cmd/juju/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/1/cmd/juju/bootstrap.go#newcode126
cmd/juju/bootstrap.go:126: signal.Notify(make(chan os.Signal),
os.Interrupt)
On 2013/11/22 01:54:00, wallyworld wrote:
> Just a thought. One other option is to install a handler which prints
a message
> informing the user that their Ctrl-C cannot terminate the cmd because
cleanup is
> being done.

Yeah, that's not a bad idea. The only catch is, what do we do before
finish bootstrap is entered? That's got me thinking about your concerns
some more: we need to be able to cancel the WaitDNS/ssh phase too, I
think, and that's not currently handled.

I think what I'll have to do is move the signal handling inside
finishBootstrap. This is in the same realm as direct-os.Stdin access, so
it should probably (eventually? now?) go into some kind of OS-context
structure that gets passed to the Environment (or maybe just to
Bootstrap).

https://codereview.appspot.com/30190043/diff/1/environs/cloudinit.go
File environs/cloudinit.go (right):

https://codereview.appspot.com/30190043/diff/1/environs/cloudinit.go#newcode135
environs/cloudinit.go:135: // and setup the SSH keys. The rest we leave
to environs/manual.
On 2013/11/22 01:54:00, wallyworld wrote:
> environs/manual or cloudinit/sshinit?

cloudinit/sshinit; fixed, thanks

https://codereview.appspot.com/30190043/diff/1/environs/cloudinit_test.go
File environs/cloudinit_test.go (right):

https://codereview.appspot.com/30190043/diff/1/environs/cloudinit_test.go#newcode207
environs/cloudinit_test.go:207: // for MAAS. MAAS needs to configure and
thren bounce the
On 2013/11/22 01:54:00, wallyworld wrote:
> typo

Done.

https://codereview.appspot.com/30190043/diff/1/provider/common/bootstrap.go
File provider/common/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/1/provider/common/bootstrap.go#newcode35
provider/common/bootstrap.go:35: defer func() {
On 2013/11/22 01:54:00, wallyworld wrote:
> Could we break this logic out into a separate function outside of
Bootstrap for
> clarity. The func could be

> handleBootstrapError(err error, inst instance.Instance)

Done.

https://codereview.appspot.com/30190043/diff/1/provider/common/bootstrap.go#newcode104
provider/common/bootstrap.go:104: func TestingPatchFinishBootstrap(f
func(instance.Inst...

Read more...

Revision history for this message
Andrew Wilkins (axwalk) wrote :
Revision history for this message
Ian Booth (wallyworld) wrote :

Thanks for dealing with the finishBootstrap issues - I think it's much
better now that it allows the process to complete in it's own time, but
still allows a Ctrl-C. The code looks ok from a reading it perspective,
and you said you've tested it live :-) One final suggestion - after each
"tick" in the for loops while waiting for dns and ssh, should we print a
"." or something so the user knows the system is alive. Every one second
would be too often so perhaps after each 10 1 second timeouts we could
do it?

We do need to test on maas before landing.

LGTM after a MAAS test and final move of those test helpers to a testing
package.

https://codereview.appspot.com/30190043/diff/20001/cloudinit/sshinit/configure_test.go
File cloudinit/sshinit/configure_test.go (right):

https://codereview.appspot.com/30190043/diff/20001/cloudinit/sshinit/configure_test.go#newcode136
cloudinit/sshinit/configure_test.go:136: assertScriptMatches(c, cfg,
"(.|\n)*apt-get -y update(.|\n)*", false)
To prevent typos causing the test to pass, I'd love to see the regexp
string which is cut and pasted be put into a const or var.

https://codereview.appspot.com/30190043/diff/20001/cloudinit/sshinit/configure_test.go#newcode149
cloudinit/sshinit/configure_test.go:149: assertScriptMatches(c, cfg,
"(.|\n)*apt-get -y upgrade(.|\n)*", false)
ditto

https://codereview.appspot.com/30190043/diff/20001/provider/common/bootstrap.go
File provider/common/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/20001/provider/common/bootstrap.go#newcode37
provider/common/bootstrap.go:37: // TODO(axw) 2013-11-22 #XXX
a bug number would be great here

https://codereview.appspot.com/30190043/diff/20001/provider/common/bootstrap.go#newcode90
provider/common/bootstrap.go:90: fmt.Fprintln(ctx.Stderr, "Stopping
instance...")
pedant alert - I think we should use case consistently for out messages.
"Stopping...." vs "cleaning up..." etc

https://codereview.appspot.com/30190043/

Revision history for this message
Andrew Wilkins (axwalk) wrote :

On 2013/11/22 06:37:49, wallyworld wrote:
> Thanks for dealing with the finishBootstrap issues - I think it's much
better
> now that it allows the process to complete in it's own time, but still
allows a
> Ctrl-C. The code looks ok from a reading it perspective, and you said
you've
> tested it live :-) One final suggestion - after each "tick" in the for
loops
> while waiting for dns and ssh, should we print a "." or something so
the user
> knows the system is alive. Every one second would be too often so
perhaps after
> each 10 1 second timeouts we could do it?

I've added the dots. I think 1 second is fine; the DNS wait is quick
anyway. The difference in rate should also give an indication of the
relative speed of the two phases.

> We do need to test on maas before landing.

> LGTM after a MAAS test and final move of those test helpers to a
testing
> package.

Thanks, I will do that. I might just run this by thumper (and fwereade?)
as well, as I know he was interested in it to begin with.

https://codereview.appspot.com/30190043/diff/20001/cloudinit/sshinit/configure_test.go
> File cloudinit/sshinit/configure_test.go (right):

https://codereview.appspot.com/30190043/diff/20001/cloudinit/sshinit/configure_test.go#newcode136
> cloudinit/sshinit/configure_test.go:136: assertScriptMatches(c, cfg,
> "(.|\n)*apt-get -y update(.|\n)*", false)
> To prevent typos causing the test to pass, I'd love to see the regexp
string
> which is cut and pasted be put into a const or var.

https://codereview.appspot.com/30190043/diff/20001/cloudinit/sshinit/configure_test.go#newcode149
> cloudinit/sshinit/configure_test.go:149: assertScriptMatches(c, cfg,
> "(.|\n)*apt-get -y upgrade(.|\n)*", false)
> ditto

https://codereview.appspot.com/30190043/diff/20001/provider/common/bootstrap.go
> File provider/common/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/20001/provider/common/bootstrap.go#newcode37
> provider/common/bootstrap.go:37: // TODO(axw) 2013-11-22 #XXX
> a bug number would be great here

https://codereview.appspot.com/30190043/diff/20001/provider/common/bootstrap.go#newcode90
> provider/common/bootstrap.go:90: fmt.Fprintln(ctx.Stderr, "Stopping
> instance...")
> pedant alert - I think we should use case consistently for out
messages.
> "Stopping...." vs "cleaning up..." etc

https://codereview.appspot.com/30190043/

Revision history for this message
Andrew Wilkins (axwalk) wrote :

Please take a look.

https://codereview.appspot.com/30190043/diff/20001/cloudinit/sshinit/configure_test.go
File cloudinit/sshinit/configure_test.go (right):

https://codereview.appspot.com/30190043/diff/20001/cloudinit/sshinit/configure_test.go#newcode136
cloudinit/sshinit/configure_test.go:136: assertScriptMatches(c, cfg,
"(.|\n)*apt-get -y update(.|\n)*", false)
On 2013/11/22 06:37:50, wallyworld wrote:
> To prevent typos causing the test to pass, I'd love to see the regexp
string
> which is cut and pasted be put into a const or var.

Done.

https://codereview.appspot.com/30190043/diff/20001/cloudinit/sshinit/configure_test.go#newcode149
cloudinit/sshinit/configure_test.go:149: assertScriptMatches(c, cfg,
"(.|\n)*apt-get -y upgrade(.|\n)*", false)
On 2013/11/22 06:37:50, wallyworld wrote:
> ditto

Done.

https://codereview.appspot.com/30190043/diff/20001/provider/common/bootstrap.go
File provider/common/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/20001/provider/common/bootstrap.go#newcode37
provider/common/bootstrap.go:37: // TODO(axw) 2013-11-22 #XXX
On 2013/11/22 06:37:50, wallyworld wrote:
> a bug number would be great here

Done.

https://codereview.appspot.com/30190043/diff/20001/provider/common/bootstrap.go#newcode90
provider/common/bootstrap.go:90: fmt.Fprintln(ctx.Stderr, "Stopping
instance...")
On 2013/11/22 06:37:50, wallyworld wrote:
> pedant alert - I think we should use case consistently for out
messages.
> "Stopping...." vs "cleaning up..." etc

Absolutely - just an oversight, thanks for picking it up.

https://codereview.appspot.com/30190043/

Revision history for this message
John A Meinel (jameinel) wrote :

I tried this, and I *really* like the first steps where it is starting
to connect, letting you know what is going on, etc. However, once it can
SSH in, it just dumps all of cloud-init-output.log to stdout which seems
very overly verbose. I personally would like to know a *summary* of what
it is doing (installing packages) rather than 100s of lines about the
exact thing it is doing.

Thoughts?

https://codereview.appspot.com/30190043/

Revision history for this message
Andrew Wilkins (axwalk) wrote :

On 2013/11/24 08:11:01, jameinel wrote:
> I tried this, and I *really* like the first steps where it is starting
to
> connect, letting you know what is going on, etc. However, once it can
SSH in, it
> just dumps all of cloud-init-output.log to stdout which seems very
overly
> verbose. I personally would like to know a *summary* of what it is
doing
> (installing packages) rather than 100s of lines about the exact thing
it is
> doing.

> Thoughts?

I agree. It's maybe slightly tricky to implement, as it really is just
executing one big script over SSH. I'll have to do some investigation to
see what's practical.

https://codereview.appspot.com/30190043/

Revision history for this message
Andrew Wilkins (axwalk) wrote :
Revision history for this message
William Reade (fwereade) wrote :
Download full text (3.7 KiB)

This is *great*. Thanks very much. It would be really nice to have an
initial line saying "Launching instance", followed by the actual
instance id that got started -- and maybe to finish by running juju
status -- but those are just details.

So LGTM modulo trivials below, assuming jam is happy; and a few followup
questions:

1) Can we drop the secrets dance soon? (I guess we still need it for the
first connect to 1.16 via state :/... but maybe we don't need it for the
API)

2) How close are we to writing the state server address directly into
the .jenv, so we get fast API connections from the word go? I'd be
really keen to see that followup soon :).

3) Which SSH commands do we still use? I think it's just debug-log that
goes over the API now, so ssh, scp, and debug-hooks are all still SSHy
AIUI -- can we consolidate those bits now? Any I didn't think of?

4) How hard will it be to extract the manual-provisioning code such that
it's possible to run a "juju register" command on an instance that you
want to provision and already have direct access to? I think we touched
on this a while back, but I'm not sure we came to any conclusion.

https://codereview.appspot.com/30190043/diff/60001/cloudinit/sshinit/configure_test.go
File cloudinit/sshinit/configure_test.go (right):

https://codereview.appspot.com/30190043/diff/60001/cloudinit/sshinit/configure_test.go#newcode36
cloudinit/sshinit/configure_test.go:36:
environs.RegisterProvider("sshinit", &testProvider{})
Can we stick an explicit "test" into the name somewhere please?

https://codereview.appspot.com/30190043/diff/60001/environs/bootstrap/state_test.go
File environs/bootstrap/state_test.go (right):

https://codereview.appspot.com/30190043/diff/60001/environs/bootstrap/state_test.go#newcode59
environs/bootstrap/state_test.go:59: c.Assert(err, gc.IsNil) // doesn't
exist, juju don't carea
s/carea/care/

https://codereview.appspot.com/30190043/diff/60001/provider/common/bootstrap.go
File provider/common/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/60001/provider/common/bootstrap.go#newcode40
provider/common/bootstrap.go:40: // Std{in,out,err}, and interrupt
signal handling.
I'd like this to be an imminent followup, please :).

https://codereview.appspot.com/30190043/diff/60001/provider/common/bootstrap.go#newcode61
provider/common/bootstrap.go:61: inst, hw, err = env.StartInstance(cons,
selectedTools, machineConfig)
Wouldn't this still work with :=, even given inst? Maybe I'm still
sluggish after lunch...

https://codereview.appspot.com/30190043/diff/60001/provider/common/testing/bootstrap.go
File provider/common/testing/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/60001/provider/common/testing/bootstrap.go#newcode17
provider/common/testing/bootstrap.go:17: f :=
func(*common.BootstrapContext, instance.Instance,
*cloudinit.MachineConfig) error {
Let's log something in here so that there's some way of diagnosing what
happened if Disable gets called in a surprising situation.

https://codereview.appspot.com/30190043/diff/60001/provider/maas/maas_test.go
File provider/maas/maas_test.go (right):

https://codereview.appspot.com/30190043/diff/60001/provider...

Read more...

Revision history for this message
William Reade (fwereade) wrote :

Oh! And, can we run in-environment storage everywhere soon? This doesn't
handle the bootstrap-state case ofc, but it would be great to keep
charms and tools inside the environment.

https://codereview.appspot.com/30190043/

Revision history for this message
John A Meinel (jameinel) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

...

> 3) Which SSH commands do we still use? I think it's just debug-log
> that goes over the API now, so ssh, scp, and debug-hooks are all
> still SSHy AIUI -- can we consolidate those bits now? Any I didn't
> think of?

Right now debug-log still goes via ssh + tail, though Frank is working
on moving us to an actual API call.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlKdYRYACgkQJdeBCYSNAAPUsgCfTgnsDcRvG+/CyNLYoptrrENk
sSAAn2dC2nbwNAT76g3n701gOdt1Yt9c
=X9FF
-----END PGP SIGNATURE-----

Revision history for this message
Andrew Wilkins (axwalk) wrote :
Download full text (4.8 KiB)

On 2013/12/02 14:54:09, fwereade wrote:
> This is *great*. Thanks very much. It would be really nice to have an
initial
> line saying "Launching instance", followed by the actual instance id
that got
> started -- and maybe to finish by running juju status -- but those are
just
> details.

I've added the "Launching instance" message. Leaving the status bit for
now; that can easily be added in afterwards, and is somewhat separate
from this CL.

> So LGTM modulo trivials below, assuming jam is happy; and a few
followup
> questions:

> 1) Can we drop the secrets dance soon? (I guess we still need it for
the first
> connect to 1.16 via state :/... but maybe we don't need it for the
API)

Yes, I intend to kill it soon.

> 2) How close are we to writing the state server address directly into
the .jenv,
> so we get fast API connections from the word go? I'd be really keen to
see that
> followup soon :).

I don't know, that wasn't part of my plan. Sounds easy enough.

> 3) Which SSH commands do we still use? I think it's just debug-log
that goes
> over the API now, so ssh, scp, and debug-hooks are all still SSHy AIUI
-- can we
> consolidate those bits now? Any I didn't think of?

debug-log still uses SSH; it only uses the API for getting the address.
IIANM, TheMue is looking at changing this.
ssh, scp, debug-* all use SSH, as do environs/manual,
environs/sshstorage, and cloudinit/sshinit. I've changed them all to use
utils/ssh.

> 4) How hard will it be to extract the manual-provisioning code such
that it's
> possible to run a "juju register" command on an instance that you want
to
> provision and already have direct access to? I think we touched on
this a while
> back, but I'm not sure we came to any conclusion.

The SSH-bits could easily be pretty easily swapped to use os/exec
Command, so it's all on localhost. The challenging bit will be how to
package configuration in a nice way.

> Oh! And, can we run in-environment storage everywhere soon? This
doesn't handle
> the bootstrap-state case ofc, but it would be great to keep charms and
tools
> inside the environment.

I can still see one major problem, which is that you couldn't then
sync-tools before bootstrapping. Other than that, I think it should be
doable.

https://codereview.appspot.com/30190043/diff/60001/cloudinit/sshinit/configure_test.go
> File cloudinit/sshinit/configure_test.go (right):

https://codereview.appspot.com/30190043/diff/60001/cloudinit/sshinit/configure_test.go#newcode36
> cloudinit/sshinit/configure_test.go:36:
environs.RegisterProvider("sshinit",
> &testProvider{})
> Can we stick an explicit "test" into the name somewhere please?

https://codereview.appspot.com/30190043/diff/60001/environs/bootstrap/state_test.go
> File environs/bootstrap/state_test.go (right):

https://codereview.appspot.com/30190043/diff/60001/environs/bootstrap/state_test.go#newcode59
> environs/bootstrap/state_test.go:59: c.Assert(err, gc.IsNil) //
doesn't exist,
> juju don't carea
> s/carea/care/

https://codereview.appspot.com/30190043/diff/60001/provider/common/bootstrap.go
> File provider/common/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/60001/provider/common/bootstrap....

Read more...

Revision history for this message
Andrew Wilkins (axwalk) wrote :

Please take a look.

https://codereview.appspot.com/30190043/diff/60001/cloudinit/sshinit/configure_test.go
File cloudinit/sshinit/configure_test.go (right):

https://codereview.appspot.com/30190043/diff/60001/cloudinit/sshinit/configure_test.go#newcode36
cloudinit/sshinit/configure_test.go:36:
environs.RegisterProvider("sshinit", &testProvider{})
On 2013/12/02 14:54:09, fwereade wrote:
> Can we stick an explicit "test" into the name somewhere please?

Done.

https://codereview.appspot.com/30190043/diff/60001/environs/bootstrap/state_test.go
File environs/bootstrap/state_test.go (right):

https://codereview.appspot.com/30190043/diff/60001/environs/bootstrap/state_test.go#newcode59
environs/bootstrap/state_test.go:59: c.Assert(err, gc.IsNil) // doesn't
exist, juju don't carea
On 2013/12/02 14:54:09, fwereade wrote:
> s/carea/care/

Done.

https://codereview.appspot.com/30190043/diff/60001/provider/common/bootstrap.go
File provider/common/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/60001/provider/common/bootstrap.go#newcode40
provider/common/bootstrap.go:40: // Std{in,out,err}, and interrupt
signal handling.
On 2013/12/02 14:54:09, fwereade wrote:
> I'd like this to be an imminent followup, please :).

No problems.

https://codereview.appspot.com/30190043/diff/60001/provider/common/bootstrap.go#newcode61
provider/common/bootstrap.go:61: inst, hw, err = env.StartInstance(cons,
selectedTools, machineConfig)
On 2013/12/02 14:54:09, fwereade wrote:
> Wouldn't this still work with :=, even given inst? Maybe I'm still
sluggish
> after lunch...

No, you're right, that'll work. Fixed.

https://codereview.appspot.com/30190043/diff/60001/provider/common/testing/bootstrap.go
File provider/common/testing/bootstrap.go (right):

https://codereview.appspot.com/30190043/diff/60001/provider/common/testing/bootstrap.go#newcode17
provider/common/testing/bootstrap.go:17: f :=
func(*common.BootstrapContext, instance.Instance,
*cloudinit.MachineConfig) error {
On 2013/12/02 14:54:09, fwereade wrote:
> Let's log something in here so that there's some way of diagnosing
what happened
> if Disable gets called in a surprising situation.

Done.

https://codereview.appspot.com/30190043/diff/60001/provider/maas/maas_test.go
File provider/maas/maas_test.go (right):

https://codereview.appspot.com/30190043/diff/60001/provider/maas/maas_test.go#newcode14
provider/maas/maas_test.go:14: commontesting
"launchpad.net/juju-core/provider/common/testing"
On 2013/12/02 14:54:09, fwereade wrote:
> Oh, a thought. Might DisableFinishBootstrap be happier in
environs/testing?

Hmm yeah that's fine, and stops the package proliferation. Moved.

https://codereview.appspot.com/30190043/diff/60001/utils/ssh/ssh.go
File utils/ssh/ssh.go (left):

https://codereview.appspot.com/30190043/diff/60001/utils/ssh/ssh.go#oldcode15
utils/ssh/ssh.go:15: // Move this to a common package for use in
cmd/juju, and others.
On 2013/12/02 14:54:09, fwereade wrote:
> Can the SSHy bits in cmd/juju use this now?

Done. Also updated environs/sshstorage, and environs/manual. Had to
tweak utils/ssh a bit, and add an ScpCommand function.

https://codereview.appspot.com/30190043/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'cloudinit/cloudinit_test.go'
--- cloudinit/cloudinit_test.go 2013-11-26 12:24:48 +0000
+++ cloudinit/cloudinit_test.go 2013-12-03 06:15:09 +0000
@@ -269,6 +269,37 @@
269 c.Assert(cfg.Packages(), gc.DeepEquals, []string{"a b c", "d!"})269 c.Assert(cfg.Packages(), gc.DeepEquals, []string{"a b c", "d!"})
270}270}
271271
272func (S) TestSetOutput(c *gc.C) {
273 type test struct {
274 kind cloudinit.OutputKind
275 stdout string
276 stderr string
277 }
278 tests := []test{{
279 cloudinit.OutAll, "a", "",
280 }, {
281 cloudinit.OutAll, "", "b",
282 }, {
283 cloudinit.OutInit, "a", "b",
284 }, {
285 cloudinit.OutAll, "a", "b",
286 }, {
287 cloudinit.OutAll, "", "",
288 }}
289
290 cfg := cloudinit.New()
291 stdout, stderr := cfg.Output(cloudinit.OutAll)
292 c.Assert(stdout, gc.Equals, "")
293 c.Assert(stderr, gc.Equals, "")
294 for i, t := range tests {
295 c.Logf("test %d: %+v", i, t)
296 cfg.SetOutput(t.kind, t.stdout, t.stderr)
297 stdout, stderr = cfg.Output(t.kind)
298 c.Assert(stdout, gc.Equals, t.stdout)
299 c.Assert(stderr, gc.Equals, t.stderr)
300 }
301}
302
272//#cloud-config303//#cloud-config
273//packages:304//packages:
274//- juju305//- juju
275306
=== modified file 'cloudinit/options.go'
--- cloudinit/options.go 2013-11-26 12:24:48 +0000
+++ cloudinit/options.go 2013-12-03 06:15:09 +0000
@@ -31,6 +31,13 @@
31 cfg.set("apt_upgrade", yes, yes)31 cfg.set("apt_upgrade", yes, yes)
32}32}
3333
34// AptUpgrade returns the value set by SetAptUpgrade, or
35// false if no call to SetAptUpgrade has been made.
36func (cfg *Config) AptUpgrade() bool {
37 update, _ := cfg.attrs["apt_upgrade"].(bool)
38 return update
39}
40
34// SetUpdate sets whether cloud-init runs "apt-get update"41// SetUpdate sets whether cloud-init runs "apt-get update"
35// on first boot.42// on first boot.
36func (cfg *Config) SetAptUpdate(yes bool) {43func (cfg *Config) SetAptUpdate(yes bool) {
@@ -233,6 +240,20 @@
233 cfg.attrs["output"] = out240 cfg.attrs["output"] = out
234}241}
235242
243// Output returns the output destination passed to SetOutput for
244// the specified output kind.
245func (cfg *Config) Output(kind OutputKind) (stdout, stderr string) {
246 if out, ok := cfg.attrs["output"].(map[string]interface{}); ok {
247 switch out := out[string(kind)].(type) {
248 case string:
249 stdout = out
250 case []string:
251 stdout, stderr = out[0], out[1]
252 }
253 }
254 return stdout, stderr
255}
256
236// AddSSHKey adds a pre-generated ssh key to the257// AddSSHKey adds a pre-generated ssh key to the
237// server keyring. Keys that are added like this will be258// server keyring. Keys that are added like this will be
238// written to /etc/ssh and new random keys will not259// written to /etc/ssh and new random keys will not
239260
=== added file 'cloudinit/progress.go'
--- cloudinit/progress.go 1970-01-01 00:00:00 +0000
+++ cloudinit/progress.go 2013-12-03 06:15:09 +0000
@@ -0,0 +1,43 @@
1// Copyright 2013 Canonical Ltd.
2// Licensed under the AGPLv3, see LICENCE file for details.
3
4package cloudinit
5
6import (
7 "fmt"
8
9 "launchpad.net/juju-core/utils"
10)
11
12// progressFd is the file descriptor to which progress is logged.
13// This is necessary so that we can redirect all non-progress
14// related stderr away.
15//
16// Note, from the Bash manual:
17// "Redirections using file descriptors greater than 9 should be
18// used with care, as they may conflict with file descriptors the
19// shell uses internally."
20const progressFd = 9
21
22// InitProgressCmd will return a command to initialise progress
23// reporting, sending messages to stderr. If LogProgressCmd is
24// used in a script, InitProgressCmd MUST be executed beforehand.
25//
26// The returned command is idempotent; this is important, to
27// allow a script to be embedded in another with stderr redirected,
28// in which case InitProgressCmd must precede the redirection.
29func InitProgressCmd() string {
30 return fmt.Sprintf("test -e /proc/self/fd/%d || exec %d>&2", progressFd, progressFd)
31}
32
33// LogProgressCmd will return a command to log the specified progress
34// message to stderr; the resultant command should be added to the
35// configuration as a runcmd or bootcmd as appropriate.
36//
37// If there are any uses of LogProgressCmd in a configuration, the
38// configuration MUST precede the command with the result of
39// InitProgressCmd.
40func LogProgressCmd(format string, args ...interface{}) string {
41 msg := utils.ShQuote(fmt.Sprintf(format, args...))
42 return fmt.Sprintf("echo %s >&%d", msg, progressFd)
43}
044
=== added file 'cloudinit/progress_test.go'
--- cloudinit/progress_test.go 1970-01-01 00:00:00 +0000
+++ cloudinit/progress_test.go 2013-12-03 06:15:09 +0000
@@ -0,0 +1,27 @@
1// Copyright 2011, 2012, 2013 Canonical Ltd.
2// Licensed under the AGPLv3, see LICENCE file for details.
3
4package cloudinit_test
5
6import (
7 "regexp"
8
9 gc "launchpad.net/gocheck"
10
11 "launchpad.net/juju-core/cloudinit"
12)
13
14type progressSuite struct{}
15
16var _ = gc.Suite(&progressSuite{})
17
18func (*progressSuite) TestProgressCmds(c *gc.C) {
19 initCmd := cloudinit.InitProgressCmd()
20 re := regexp.MustCompile(`test -e /proc/self/fd/([0-9]+) \|\| exec ([0-9]+)>&2`)
21 submatch := re.FindStringSubmatch(initCmd)
22 c.Assert(submatch, gc.HasLen, 3)
23 c.Assert(submatch[0], gc.Equals, initCmd)
24 c.Assert(submatch[1], gc.Equals, submatch[2])
25 logCmd := cloudinit.LogProgressCmd("he'llo\"!")
26 c.Assert(logCmd, gc.Equals, `echo 'he'"'"'llo"!' >&`+submatch[1])
27}
028
=== added directory 'cloudinit/sshinit'
=== renamed file 'environs/manual/agent.go' => 'cloudinit/sshinit/configure.go'
--- environs/manual/agent.go 2013-11-14 08:27:15 +0000
+++ cloudinit/sshinit/configure.go 2013-12-03 06:15:09 +0000
@@ -1,7 +1,7 @@
1// Copyright 2013 Canonical Ltd.1// Copyright 2013 Canonical Ltd.
2// Licensed under the AGPLv3, see LICENCE file for details.2// Licensed under the AGPLv3, see LICENCE file for details.
33
4package manual4package sshinit
55
6import (6import (
7 "encoding/base64"7 "encoding/base64"
@@ -9,25 +9,29 @@
9 "os"9 "os"
10 "strings"10 "strings"
1111
12 corecloudinit "launchpad.net/juju-core/cloudinit"12 "launchpad.net/loggo"
13 "launchpad.net/juju-core/environs/cloudinit"13
14 "launchpad.net/juju-core/cloudinit"
14 "launchpad.net/juju-core/utils"15 "launchpad.net/juju-core/utils"
16 "launchpad.net/juju-core/utils/ssh"
15)17)
1618
17// ProvisionMachineAgent connects to a machine over SSH,19var logger = loggo.GetLogger("juju.cloudinit.sshinit")
18// and installs a machine agent.20
19func ProvisionMachineAgent(host string, mcfg *cloudinit.MachineConfig) error {21// Configure connects to the specified host over SSH,
22// and executes a script that carries out cloud-config.
23func Configure(host string, cfg *cloudinit.Config) error {
20 logger.Infof("Provisioning machine agent on %s", host)24 logger.Infof("Provisioning machine agent on %s", host)
21 script, err := provisionMachineAgentScript(mcfg)25 script, err := generateScript(cfg)
22 if err != nil {26 if err != nil {
23 return err27 return err
24 }28 }
25 scriptBase64 := base64.StdEncoding.EncodeToString([]byte(script))29 scriptBase64 := base64.StdEncoding.EncodeToString([]byte(script))
26 script = fmt.Sprintf(`F=$(mktemp); echo %s | base64 -d > $F; . $F`, scriptBase64)30 script = fmt.Sprintf(`F=$(mktemp); echo %s | base64 -d > $F; . $F`, scriptBase64)
27 cmd := sshCommand(31 cmd := ssh.Command(
28 host,32 host,
29 fmt.Sprintf("sudo bash -c '%s'", script),33 []string{"sudo", fmt.Sprintf("bash -c '%s'", script)},
30 allocateTTY,34 ssh.AllocateTTY,
31 )35 )
32 cmd.Stdout = os.Stdout36 cmd.Stdout = os.Stdout
33 cmd.Stderr = os.Stderr37 cmd.Stderr = os.Stderr
@@ -35,17 +39,9 @@
35 return cmd.Run()39 return cmd.Run()
36}40}
3741
38// provisionMachineAgentScript generates the script necessary42// generateScript generates the script that applies
39// to install a machine agent with the provided MachineConfig.43// the specified cloud-config.
40func provisionMachineAgentScript(mcfg *cloudinit.MachineConfig) (string, error) {44func generateScript(cloudcfg *cloudinit.Config) (string, error) {
41 // We generate a cloud-init config, which we'll then pull the runcmds
42 // and prerequisite packages out of. Rather than generating a cloud-config,
43 // we'll just generate a shell script.
44 cloudcfg := corecloudinit.New()
45 if err := cloudinit.Configure(mcfg, cloudcfg); err != nil {
46 return "", err
47 }
48
49 // TODO(axw): 2013-08-23 bug 121577745 // TODO(axw): 2013-08-23 bug 1215777
50 // Carry out configuration for ssh-keys-per-user,46 // Carry out configuration for ssh-keys-per-user,
51 // machine-updates-authkeys, using cloud-init config.47 // machine-updates-authkeys, using cloud-init config.
@@ -78,23 +74,38 @@
78 // We prepend "set -xe". This is already in runcmds,74 // We prepend "set -xe". This is already in runcmds,
79 // but added here to avoid relying on that to be75 // but added here to avoid relying on that to be
80 // invariant.76 // invariant.
81 script := []string{"#!/bin/bash", "set -xe"}77 script := []string{"#!/bin/bash", "set -e"}
78 // We must initialise progress reporting before entering
79 // the subshell and redirecting stderr.
80 script = append(script, cloudinit.InitProgressCmd())
81 stdout, stderr := cloudcfg.Output(cloudinit.OutAll)
82 script = append(script, "(")
83 if stderr != "" {
84 script = append(script, "(")
85 }
82 script = append(script, bootcmds...)86 script = append(script, bootcmds...)
83 script = append(script, pkgcmds...)87 script = append(script, pkgcmds...)
84 script = append(script, runcmds...)88 script = append(script, runcmds...)
89 if stderr != "" {
90 script = append(script, ") "+stdout)
91 script = append(script, ") "+stderr)
92 } else {
93 script = append(script, ") "+stdout+" 2>&1")
94 }
85 return strings.Join(script, "\n"), nil95 return strings.Join(script, "\n"), nil
86}96}
8797
88// addPackageCommands returns a slice of commands that, when run,98// addPackageCommands returns a slice of commands that, when run,
89// will add the required apt repositories and packages.99// will add the required apt repositories and packages.
90func addPackageCommands(cfg *corecloudinit.Config) ([]string, error) {100func addPackageCommands(cfg *cloudinit.Config) ([]string, error) {
91 var cmds []string101 var cmds []string
92 if len(cfg.AptSources()) > 0 {102 if len(cfg.AptSources()) > 0 {
93 // Ensure apt-add-repository is available.103 // Ensure add-apt-repository is available.
104 cmds = append(cmds, cloudinit.LogProgressCmd("Installing add-apt-repository"))
94 cmds = append(cmds, "apt-get -y install python-software-properties")105 cmds = append(cmds, "apt-get -y install python-software-properties")
95 }106 }
96 for _, src := range cfg.AptSources() {107 for _, src := range cfg.AptSources() {
97 // PPA keys are obtained by apt-add-repository, from launchpad.108 // PPA keys are obtained by add-apt-repository, from launchpad.
98 if !strings.HasPrefix(src.Source, "ppa:") {109 if !strings.HasPrefix(src.Source, "ppa:") {
99 if src.Key != "" {110 if src.Key != "" {
100 key := utils.ShQuote(src.Key)111 key := utils.ShQuote(src.Key)
@@ -102,14 +113,19 @@
102 cmds = append(cmds, cmd)113 cmds = append(cmds, cmd)
103 }114 }
104 }115 }
105 cmds = append(cmds, "apt-add-repository -y "+utils.ShQuote(src.Source))116 cmds = append(cmds, cloudinit.LogProgressCmd("Adding apt repository: %s", src.Source))
117 cmds = append(cmds, "add-apt-repository -y "+utils.ShQuote(src.Source))
106 }118 }
107 if len(cfg.AptSources()) > 0 {119 if len(cfg.AptSources()) > 0 || cfg.AptUpdate() {
120 cmds = append(cmds, cloudinit.LogProgressCmd("Running apt-get update"))
108 cmds = append(cmds, "apt-get -y update")121 cmds = append(cmds, "apt-get -y update")
109 }122 }
110 // Note: explicitly ignoring apt_upgrade, so as not to trample the target123 if cfg.AptUpgrade() {
111 // machine's existing configuration.124 cmds = append(cmds, cloudinit.LogProgressCmd("Running apt-get upgrade"))
125 cmds = append(cmds, "apt-get -y upgrade")
126 }
112 for _, pkg := range cfg.Packages() {127 for _, pkg := range cfg.Packages() {
128 cmds = append(cmds, cloudinit.LogProgressCmd("Installing package: %s", pkg))
113 cmd := fmt.Sprintf("apt-get -y install %s", utils.ShQuote(pkg))129 cmd := fmt.Sprintf("apt-get -y install %s", utils.ShQuote(pkg))
114 cmds = append(cmds, cmd)130 cmds = append(cmds, cmd)
115 }131 }
116132
=== renamed file 'environs/manual/agent_test.go' => 'cloudinit/sshinit/configure_test.go'
--- environs/manual/agent_test.go 2013-11-14 08:27:15 +0000
+++ cloudinit/sshinit/configure_test.go 2013-12-03 06:15:09 +0000
@@ -1,35 +1,46 @@
1// Copyright 2013 Canonical Ltd.1// Copyright 2013 Canonical Ltd.
2// Licensed under the AGPLv3, see LICENCE file for details.2// Licensed under the AGPLv3, see LICENCE file for details.
33
4package manual4package sshinit
55
6import (6import (
7 gc "launchpad.net/gocheck"7 gc "launchpad.net/gocheck"
88
9 "launchpad.net/juju-core/cloudinit"
9 "launchpad.net/juju-core/constraints"10 "launchpad.net/juju-core/constraints"
10 "launchpad.net/juju-core/environs"11 "launchpad.net/juju-core/environs"
11 "launchpad.net/juju-core/environs/cloudinit"12 envcloudinit "launchpad.net/juju-core/environs/cloudinit"
12 "launchpad.net/juju-core/environs/config"13 "launchpad.net/juju-core/environs/config"
13 envtools "launchpad.net/juju-core/environs/tools"14 envtools "launchpad.net/juju-core/environs/tools"
14 _ "launchpad.net/juju-core/provider/dummy"
15 coretesting "launchpad.net/juju-core/testing"15 coretesting "launchpad.net/juju-core/testing"
16 "launchpad.net/juju-core/testing/testbase"16 "launchpad.net/juju-core/testing/testbase"
17 "launchpad.net/juju-core/tools"17 "launchpad.net/juju-core/tools"
18 "launchpad.net/juju-core/version"18 "launchpad.net/juju-core/version"
19)19)
2020
21type agentSuite struct {21type configureSuite struct {
22 testbase.LoggingSuite22 testbase.LoggingSuite
23}23}
2424
25var _ = gc.Suite(&agentSuite{})25var _ = gc.Suite(&configureSuite{})
2626
27func dummyConfig(c *gc.C, stateServer bool, vers version.Binary) *config.Config {27type testProvider struct {
28 environs.EnvironProvider
29}
30
31func (p *testProvider) SecretAttrs(cfg *config.Config) (map[string]string, error) {
32 return map[string]string{}, nil
33}
34
35func init() {
36 environs.RegisterProvider("sshinit_test", &testProvider{})
37}
38
39func testConfig(c *gc.C, stateServer bool, vers version.Binary) *config.Config {
28 testConfig, err := config.New(config.UseDefaults, coretesting.FakeConfig())40 testConfig, err := config.New(config.UseDefaults, coretesting.FakeConfig())
29 c.Assert(err, gc.IsNil)41 c.Assert(err, gc.IsNil)
30 testConfig, err = testConfig.Apply(map[string]interface{}{42 testConfig, err = testConfig.Apply(map[string]interface{}{
31 "type": "dummy",43 "type": "sshinit_test",
32 "state-server": stateServer,
33 "default-series": vers.Series,44 "default-series": vers.Series,
34 "agent-version": vers.Number.String(),45 "agent-version": vers.Number.String(),
35 })46 })
@@ -37,8 +48,8 @@
37 return testConfig48 return testConfig
38}49}
3950
40func (s *agentSuite) getMachineConfig(c *gc.C, stateServer bool, vers version.Binary) *cloudinit.MachineConfig {51func (s *configureSuite) getCloudConfig(c *gc.C, stateServer bool, vers version.Binary) *cloudinit.Config {
41 var mcfg *cloudinit.MachineConfig52 var mcfg *envcloudinit.MachineConfig
42 if stateServer {53 if stateServer {
43 mcfg = environs.NewBootstrapMachineConfig("http://whatever/dotcom")54 mcfg = environs.NewBootstrapMachineConfig("http://whatever/dotcom")
44 } else {55 } else {
@@ -48,10 +59,13 @@
48 Version: vers,59 Version: vers,
49 URL: "file:///var/lib/juju/storage/" + envtools.StorageName(vers),60 URL: "file:///var/lib/juju/storage/" + envtools.StorageName(vers),
50 }61 }
51 environConfig := dummyConfig(c, stateServer, vers)62 environConfig := testConfig(c, stateServer, vers)
52 err := environs.FinishMachineConfig(mcfg, environConfig, constraints.Value{})63 err := environs.FinishMachineConfig(mcfg, environConfig, constraints.Value{})
53 c.Assert(err, gc.IsNil)64 c.Assert(err, gc.IsNil)
54 return mcfg65 cloudcfg := cloudinit.New()
66 err = envcloudinit.Configure(mcfg, cloudcfg)
67 c.Assert(err, gc.IsNil)
68 return cloudcfg
55}69}
5670
57var allSeries = [...]string{"precise", "quantal", "raring", "saucy"}71var allSeries = [...]string{"precise", "quantal", "raring", "saucy"}
@@ -63,10 +77,10 @@
63 return gc.Not(checker)77 return gc.Not(checker)
64}78}
6579
66func (s *agentSuite) TestAptSources(c *gc.C) {80func (s *configureSuite) TestAptSources(c *gc.C) {
67 for _, series := range allSeries {81 for _, series := range allSeries {
68 vers := version.MustParseBinary("1.16.0-" + series + "-amd64")82 vers := version.MustParseBinary("1.16.0-" + series + "-amd64")
69 script, err := provisionMachineAgentScript(s.getMachineConfig(c, true, vers))83 script, err := generateScript(s.getCloudConfig(c, true, vers))
70 c.Assert(err, gc.IsNil)84 c.Assert(err, gc.IsNil)
7185
72 // Only Precise requires the cloud-tools pocket.86 // Only Precise requires the cloud-tools pocket.
@@ -82,7 +96,7 @@
82 c.Assert(96 c.Assert(
83 script,97 script,
84 checkIff(gc.Matches, needsCloudTools),98 checkIff(gc.Matches, needsCloudTools),
85 "(.|\n)*apt-add-repository.*cloud-tools(.|\n)*",99 "(.|\n)*add-apt-repository.*cloud-tools(.|\n)*",
86 )100 )
87101
88 // Only Quantal requires the PPA (for mongo).102 // Only Quantal requires the PPA (for mongo).
@@ -90,10 +104,10 @@
90 c.Assert(104 c.Assert(
91 script,105 script,
92 checkIff(gc.Matches, needsJujuPPA),106 checkIff(gc.Matches, needsJujuPPA),
93 "(.|\n)*apt-add-repository.*ppa:juju/stable(.|\n)*",107 "(.|\n)*add-apt-repository.*ppa:juju/stable(.|\n)*",
94 )108 )
95109
96 // Only install python-software-properties (apt-add-repository)110 // Only install python-software-properties (add-apt-repository)
97 // if we need to.111 // if we need to.
98 c.Assert(112 c.Assert(
99 script,113 script,
@@ -102,3 +116,39 @@
102 )116 )
103 }117 }
104}118}
119
120func assertScriptMatches(c *gc.C, cfg *cloudinit.Config, pattern string, match bool) {
121 script, err := generateScript(cfg)
122 c.Assert(err, gc.IsNil)
123 checker := gc.Matches
124 if !match {
125 checker = gc.Not(checker)
126 }
127 c.Assert(script, checker, pattern)
128}
129
130func (s *configureSuite) TestAptUpdate(c *gc.C) {
131 // apt-get update is run if either AptUpdate is set,
132 // or apt sources are defined.
133 const aptGetUpdatePattern = "(.|\n)*apt-get -y update(.|\n)*"
134 cfg := cloudinit.New()
135 c.Assert(cfg.AptUpdate(), gc.Equals, false)
136 c.Assert(cfg.AptSources(), gc.HasLen, 0)
137 assertScriptMatches(c, cfg, aptGetUpdatePattern, false)
138 cfg.SetAptUpdate(true)
139 assertScriptMatches(c, cfg, aptGetUpdatePattern, true)
140 cfg.SetAptUpdate(false)
141 cfg.AddAptSource("source", "key")
142 assertScriptMatches(c, cfg, aptGetUpdatePattern, true)
143}
144
145func (s *configureSuite) TestAptUpgrade(c *gc.C) {
146 // apt-get upgrade is only run if AptUpgrade is set.
147 const aptGetUpgradePattern = "(.|\n)*apt-get -y upgrade(.|\n)*"
148 cfg := cloudinit.New()
149 cfg.SetAptUpdate(true)
150 cfg.AddAptSource("source", "key")
151 assertScriptMatches(c, cfg, aptGetUpgradePattern, false)
152 cfg.SetAptUpgrade(true)
153 assertScriptMatches(c, cfg, aptGetUpgradePattern, true)
154}
105155
=== added file 'cloudinit/sshinit/suite_test.go'
--- cloudinit/sshinit/suite_test.go 1970-01-01 00:00:00 +0000
+++ cloudinit/sshinit/suite_test.go 2013-12-03 06:15:09 +0000
@@ -0,0 +1,14 @@
1// Copyright 2013 Canonical Ltd.
2// Licensed under the AGPLv3, see LICENCE file for details.
3
4package sshinit
5
6import (
7 stdtesting "testing"
8
9 gc "launchpad.net/gocheck"
10)
11
12func Test(t *stdtesting.T) {
13 gc.TestingT(t)
14}
015
=== modified file 'cmd/juju/debughooks.go'
--- cmd/juju/debughooks.go 2013-11-28 11:57:57 +0000
+++ cmd/juju/debughooks.go 2013-12-03 06:15:09 +0000
@@ -146,7 +146,7 @@
146 debugctx := unitdebug.NewHooksContext(c.Target)146 debugctx := unitdebug.NewHooksContext(c.Target)
147 script := base64.StdEncoding.EncodeToString([]byte(unitdebug.ClientScript(debugctx, c.hooks)))147 script := base64.StdEncoding.EncodeToString([]byte(unitdebug.ClientScript(debugctx, c.hooks)))
148 innercmd := fmt.Sprintf(`F=$(mktemp); echo %s | base64 -d > $F; . $F`, script)148 innercmd := fmt.Sprintf(`F=$(mktemp); echo %s | base64 -d > $F; . $F`, script)
149 args := []string{"--", fmt.Sprintf("sudo /bin/bash -c '%s'", innercmd)}149 args := []string{fmt.Sprintf("sudo /bin/bash -c '%s'", innercmd)}
150 c.Args = args150 c.Args = args
151 return c.SSHCommand.Run(ctx)151 return c.SSHCommand.Run(ctx)
152}152}
153153
=== modified file 'cmd/juju/debughooks_test.go'
--- cmd/juju/debughooks_test.go 2013-11-08 04:45:20 +0000
+++ cmd/juju/debughooks_test.go 2013-12-03 06:15:09 +0000
@@ -19,7 +19,7 @@
19 SSHCommonSuite19 SSHCommonSuite
20}20}
2121
22const debugHooksArgs = `-l ubuntu -t ` + commonArgs22const debugHooksArgs = sshArgs
2323
24var debugHooksTests = []struct {24var debugHooksTests = []struct {
25 info string25 info string
@@ -29,10 +29,10 @@
29 stderr string29 stderr string
30}{{30}{{
31 args: []string{"mysql/0"},31 args: []string{"mysql/0"},
32 result: regexp.QuoteMeta(debugHooksArgs + "dummyenv-0.dns -- sudo /bin/bash -c 'F=$(mktemp); echo IyEvYmluL2Jhc2gKKAojIExvY2sgdGhlIGp1anUtPHVuaXQ+LWRlYnVnIGxvY2tmaWxlLgpmbG9jayAtbiA4IHx8IChlY2hvICJGYWlsZWQgdG8gYWNxdWlyZSAvdG1wL2p1anUtdW5pdC1teXNxbC0wLWRlYnVnLWhvb2tzOiB1bml0IGlzIGFscmVhZHkgYmVpbmcgZGVidWdnZWQiIDI+JjE7IGV4aXQgMSkKKAojIENsb3NlIHRoZSBpbmhlcml0ZWQgbG9jayBGRCwgb3IgdG11eCB3aWxsIGtlZXAgaXQgb3Blbi4KZXhlYyA4PiYtCgojIFdyaXRlIG91dCB0aGUgZGVidWctaG9va3MgYXJncy4KZWNobyAiZTMwSyIgfCBiYXNlNjQgLWQgPiAvdG1wL2p1anUtdW5pdC1teXNxbC0wLWRlYnVnLWhvb2tzCgojIExvY2sgdGhlIGp1anUtPHVuaXQ+LWRlYnVnLWV4aXQgbG9ja2ZpbGUuCmZsb2NrIC1uIDkgfHwgZXhpdCAxCgojIFdhaXQgZm9yIHRtdXggdG8gYmUgaW5zdGFsbGVkLgp3aGlsZSBbICEgLWYgL3Vzci9iaW4vdG11eCBdOyBkbwogICAgc2xlZXAgMQpkb25lCgppZiBbICEgLWYgfi8udG11eC5jb25mIF07IHRoZW4KICAgICAgICBpZiBbIC1mIC91c3Ivc2hhcmUvYnlvYnUvcHJvZmlsZXMvdG11eCBdOyB0aGVuCiAgICAgICAgICAgICAgICAjIFVzZSBieW9idS90bXV4IHByb2ZpbGUgZm9yIGZhbWlsaWFyIGtleWJpbmRpbmdzIGFuZCBicmFuZGluZwogICAgICAgICAgICAgICAgZWNobyAic291cmNlLWZpbGUgL3Vzci9zaGFyZS9ieW9idS9wcm9maWxlcy90bXV4IiA+IH4vLnRtdXguY29uZgogICAgICAgIGVsc2UKICAgICAgICAgICAgICAgICMgT3RoZXJ3aXNlLCB1c2UgdGhlIGxlZ2FjeSBqdWp1L3RtdXggY29uZmlndXJhdGlvbgogICAgICAgICAgICAgICAgY2F0ID4gfi8udG11eC5jb25mIDw8RU5ECiAgICAgICAgICAgICAgICAKIyBTdGF0dXMgYmFyCnNldC1vcHRpb24gLWcgc3RhdHVzLWJnIGJsYWNrCnNldC1vcHRpb24gLWcgc3RhdHVzLWZnIHdoaXRlCgpzZXQtd2luZG93LW9wdGlvbiAtZyB3aW5kb3ctc3RhdHVzLWN1cnJlbnQtYmcgcmVkCnNldC13aW5kb3ctb3B0aW9uIC1nIHdpbmRvdy1zdGF0dXMtY3VycmVudC1hdHRyIGJyaWdodAoKc2V0LW9wdGlvbiAtZyBzdGF0dXMtcmlnaHQgJycKCiMgUGFuZXMKc2V0LW9wdGlvbiAtZyBwYW5lLWJvcmRlci1mZyB3aGl0ZQpzZXQtb3B0aW9uIC1nIHBhbmUtYWN0aXZlLWJvcmRlci1mZyB3aGl0ZQoKIyBNb25pdG9yIGFjdGl2aXR5IG9uIHdpbmRvd3MKc2V0LXdpbmRvdy1vcHRpb24gLWcgbW9uaXRvci1hY3Rpdml0eSBvbgoKIyBTY3JlZW4gYmluZGluZ3MsIHNpbmNlIHBlb3BsZSBhcmUgbW9yZSBmYW1pbGlhciB3aXRoIHRoYXQuCnNldC1vcHRpb24gLWcgcHJlZml4IEMtYQpiaW5kIEMtYSBsYXN0LXdpbmRvdwpiaW5kIGEgc2VuZC1rZXkgQy1hCgpiaW5kIHwgc3BsaXQtd2luZG93IC1oCmJpbmQgLSBzcGxpdC13aW5kb3cgLXYKCiMgRml4IENUUkwtUEdVUC9QR0RPV04gZm9yIHZpbQpzZXQtd2luZG93LW9wdGlvbiAtZyB4dGVybS1rZXlzIG9uCgojIFByZXZlbnQgRVNDIGtleSBmcm9tIGFkZGluZyBkZWxheSBhbmQgYnJlYWtpbmcgVmltJ3MgRVNDID4gYXJyb3cga2V5CnNldC1vcHRpb24gLXMgZXNjYXBlLXRpbWUgMAoKRU5ECiAgICAgICAgZmkKZmkKCigKICAgICMgQ2xvc2UgdGhlIGluaGVyaXRlZCBsb2NrIEZELCBvciB0bXV4IHdpbGwga2VlcCBpdCBvcGVuLgogICAgZXhlYyA5PiYtCiAgICBleGVjIHRtdXggbmV3LXNlc3Npb24gLXMgbXlzcWwvMAopCikgOT4vdG1wL2p1anUtdW5pdC1teXNxbC0wLWRlYnVnLWhvb2tzLWV4aXQKKSA4Pi90bXAvanVqdS11bml0LW15c3FsLTAtZGVidWctaG9va3MKZXhpdCAkPwo= | base64 -d > $F; . $F'\n"),32 result: regexp.QuoteMeta(debugHooksArgs + "ubuntu@dummyenv-0.dns -- sudo /bin/bash -c 'F=$(mktemp); echo IyEvYmluL2Jhc2gKKAojIExvY2sgdGhlIGp1anUtPHVuaXQ+LWRlYnVnIGxvY2tmaWxlLgpmbG9jayAtbiA4IHx8IChlY2hvICJGYWlsZWQgdG8gYWNxdWlyZSAvdG1wL2p1anUtdW5pdC1teXNxbC0wLWRlYnVnLWhvb2tzOiB1bml0IGlzIGFscmVhZHkgYmVpbmcgZGVidWdnZWQiIDI+JjE7IGV4aXQgMSkKKAojIENsb3NlIHRoZSBpbmhlcml0ZWQgbG9jayBGRCwgb3IgdG11eCB3aWxsIGtlZXAgaXQgb3Blbi4KZXhlYyA4PiYtCgojIFdyaXRlIG91dCB0aGUgZGVidWctaG9va3MgYXJncy4KZWNobyAiZTMwSyIgfCBiYXNlNjQgLWQgPiAvdG1wL2p1anUtdW5pdC1teXNxbC0wLWRlYnVnLWhvb2tzCgojIExvY2sgdGhlIGp1anUtPHVuaXQ+LWRlYnVnLWV4aXQgbG9ja2ZpbGUuCmZsb2NrIC1uIDkgfHwgZXhpdCAxCgojIFdhaXQgZm9yIHRtdXggdG8gYmUgaW5zdGFsbGVkLgp3aGlsZSBbICEgLWYgL3Vzci9iaW4vdG11eCBdOyBkbwogICAgc2xlZXAgMQpkb25lCgppZiBbICEgLWYgfi8udG11eC5jb25mIF07IHRoZW4KICAgICAgICBpZiBbIC1mIC91c3Ivc2hhcmUvYnlvYnUvcHJvZmlsZXMvdG11eCBdOyB0aGVuCiAgICAgICAgICAgICAgICAjIFVzZSBieW9idS90bXV4IHByb2ZpbGUgZm9yIGZhbWlsaWFyIGtleWJpbmRpbmdzIGFuZCBicmFuZGluZwogICAgICAgICAgICAgICAgZWNobyAic291cmNlLWZpbGUgL3Vzci9zaGFyZS9ieW9idS9wcm9maWxlcy90bXV4IiA+IH4vLnRtdXguY29uZgogICAgICAgIGVsc2UKICAgICAgICAgICAgICAgICMgT3RoZXJ3aXNlLCB1c2UgdGhlIGxlZ2FjeSBqdWp1L3RtdXggY29uZmlndXJhdGlvbgogICAgICAgICAgICAgICAgY2F0ID4gfi8udG11eC5jb25mIDw8RU5ECiAgICAgICAgICAgICAgICAKIyBTdGF0dXMgYmFyCnNldC1vcHRpb24gLWcgc3RhdHVzLWJnIGJsYWNrCnNldC1vcHRpb24gLWcgc3RhdHVzLWZnIHdoaXRlCgpzZXQtd2luZG93LW9wdGlvbiAtZyB3aW5kb3ctc3RhdHVzLWN1cnJlbnQtYmcgcmVkCnNldC13aW5kb3ctb3B0aW9uIC1nIHdpbmRvdy1zdGF0dXMtY3VycmVudC1hdHRyIGJyaWdodAoKc2V0LW9wdGlvbiAtZyBzdGF0dXMtcmlnaHQgJycKCiMgUGFuZXMKc2V0LW9wdGlvbiAtZyBwYW5lLWJvcmRlci1mZyB3aGl0ZQpzZXQtb3B0aW9uIC1nIHBhbmUtYWN0aXZlLWJvcmRlci1mZyB3aGl0ZQoKIyBNb25pdG9yIGFjdGl2aXR5IG9uIHdpbmRvd3MKc2V0LXdpbmRvdy1vcHRpb24gLWcgbW9uaXRvci1hY3Rpdml0eSBvbgoKIyBTY3JlZW4gYmluZGluZ3MsIHNpbmNlIHBlb3BsZSBhcmUgbW9yZSBmYW1pbGlhciB3aXRoIHRoYXQuCnNldC1vcHRpb24gLWcgcHJlZml4IEMtYQpiaW5kIEMtYSBsYXN0LXdpbmRvdwpiaW5kIGEgc2VuZC1rZXkgQy1hCgpiaW5kIHwgc3BsaXQtd2luZG93IC1oCmJpbmQgLSBzcGxpdC13aW5kb3cgLXYKCiMgRml4IENUUkwtUEdVUC9QR0RPV04gZm9yIHZpbQpzZXQtd2luZG93LW9wdGlvbiAtZyB4dGVybS1rZXlzIG9uCgojIFByZXZlbnQgRVNDIGtleSBmcm9tIGFkZGluZyBkZWxheSBhbmQgYnJlYWtpbmcgVmltJ3MgRVNDID4gYXJyb3cga2V5CnNldC1vcHRpb24gLXMgZXNjYXBlLXRpbWUgMAoKRU5ECiAgICAgICAgZmkKZmkKCigKICAgICMgQ2xvc2UgdGhlIGluaGVyaXRlZCBsb2NrIEZELCBvciB0bXV4IHdpbGwga2VlcCBpdCBvcGVuLgogICAgZXhlYyA5PiYtCiAgICBleGVjIHRtdXggbmV3LXNlc3Npb24gLXMgbXlzcWwvMAopCikgOT4vdG1wL2p1anUtdW5pdC1teXNxbC0wLWRlYnVnLWhvb2tzLWV4aXQKKSA4Pi90bXAvanVqdS11bml0LW15c3FsLTAtZGVidWctaG9va3MKZXhpdCAkPwo= | base64 -d > $F; . $F'\n"),
33}, {33}, {
34 args: []string{"mongodb/1"},34 args: []string{"mongodb/1"},
35 result: regexp.QuoteMeta(debugHooksArgs + "dummyenv-2.dns -- sudo /bin/bash -c 'F=$(mktemp); echo IyEvYmluL2Jhc2gKKAojIExvY2sgdGhlIGp1anUtPHVuaXQ+LWRlYnVnIGxvY2tmaWxlLgpmbG9jayAtbiA4IHx8IChlY2hvICJGYWlsZWQgdG8gYWNxdWlyZSAvdG1wL2p1anUtdW5pdC1tb25nb2RiLTEtZGVidWctaG9va3M6IHVuaXQgaXMgYWxyZWFkeSBiZWluZyBkZWJ1Z2dlZCIgMj4mMTsgZXhpdCAxKQooCiMgQ2xvc2UgdGhlIGluaGVyaXRlZCBsb2NrIEZELCBvciB0bXV4IHdpbGwga2VlcCBpdCBvcGVuLgpleGVjIDg+Ji0KCiMgV3JpdGUgb3V0IHRoZSBkZWJ1Zy1ob29rcyBhcmdzLgplY2hvICJlMzBLIiB8IGJhc2U2NCAtZCA+IC90bXAvanVqdS11bml0LW1vbmdvZGItMS1kZWJ1Zy1ob29rcwoKIyBMb2NrIHRoZSBqdWp1LTx1bml0Pi1kZWJ1Zy1leGl0IGxvY2tmaWxlLgpmbG9jayAtbiA5IHx8IGV4aXQgMQoKIyBXYWl0IGZvciB0bXV4IHRvIGJlIGluc3RhbGxlZC4Kd2hpbGUgWyAhIC1mIC91c3IvYmluL3RtdXggXTsgZG8KICAgIHNsZWVwIDEKZG9uZQoKaWYgWyAhIC1mIH4vLnRtdXguY29uZiBdOyB0aGVuCiAgICAgICAgaWYgWyAtZiAvdXNyL3NoYXJlL2J5b2J1L3Byb2ZpbGVzL3RtdXggXTsgdGhlbgogICAgICAgICAgICAgICAgIyBVc2UgYnlvYnUvdG11eCBwcm9maWxlIGZvciBmYW1pbGlhciBrZXliaW5kaW5ncyBhbmQgYnJhbmRpbmcKICAgICAgICAgICAgICAgIGVjaG8gInNvdXJjZS1maWxlIC91c3Ivc2hhcmUvYnlvYnUvcHJvZmlsZXMvdG11eCIgPiB+Ly50bXV4LmNvbmYKICAgICAgICBlbHNlCiAgICAgICAgICAgICAgICAjIE90aGVyd2lzZSwgdXNlIHRoZSBsZWdhY3kganVqdS90bXV4IGNvbmZpZ3VyYXRpb24KICAgICAgICAgICAgICAgIGNhdCA+IH4vLnRtdXguY29uZiA8PEVORAogICAgICAgICAgICAgICAgCiMgU3RhdHVzIGJhcgpzZXQtb3B0aW9uIC1nIHN0YXR1cy1iZyBibGFjawpzZXQtb3B0aW9uIC1nIHN0YXR1cy1mZyB3aGl0ZQoKc2V0LXdpbmRvdy1vcHRpb24gLWcgd2luZG93LXN0YXR1cy1jdXJyZW50LWJnIHJlZApzZXQtd2luZG93LW9wdGlvbiAtZyB3aW5kb3ctc3RhdHVzLWN1cnJlbnQtYXR0ciBicmlnaHQKCnNldC1vcHRpb24gLWcgc3RhdHVzLXJpZ2h0ICcnCgojIFBhbmVzCnNldC1vcHRpb24gLWcgcGFuZS1ib3JkZXItZmcgd2hpdGUKc2V0LW9wdGlvbiAtZyBwYW5lLWFjdGl2ZS1ib3JkZXItZmcgd2hpdGUKCiMgTW9uaXRvciBhY3Rpdml0eSBvbiB3aW5kb3dzCnNldC13aW5kb3ctb3B0aW9uIC1nIG1vbml0b3ItYWN0aXZpdHkgb24KCiMgU2NyZWVuIGJpbmRpbmdzLCBzaW5jZSBwZW9wbGUgYXJlIG1vcmUgZmFtaWxpYXIgd2l0aCB0aGF0LgpzZXQtb3B0aW9uIC1nIHByZWZpeCBDLWEKYmluZCBDLWEgbGFzdC13aW5kb3cKYmluZCBhIHNlbmQta2V5IEMtYQoKYmluZCB8IHNwbGl0LXdpbmRvdyAtaApiaW5kIC0gc3BsaXQtd2luZG93IC12CgojIEZpeCBDVFJMLVBHVVAvUEdET1dOIGZvciB2aW0Kc2V0LXdpbmRvdy1vcHRpb24gLWcgeHRlcm0ta2V5cyBvbgoKIyBQcmV2ZW50IEVTQyBrZXkgZnJvbSBhZGRpbmcgZGVsYXkgYW5kIGJyZWFraW5nIFZpbSdzIEVTQyA+IGFycm93IGtleQpzZXQtb3B0aW9uIC1zIGVzY2FwZS10aW1lIDAKCkVORAogICAgICAgIGZpCmZpCgooCiAgICAjIENsb3NlIHRoZSBpbmhlcml0ZWQgbG9jayBGRCwgb3IgdG11eCB3aWxsIGtlZXAgaXQgb3Blbi4KICAgIGV4ZWMgOT4mLQogICAgZXhlYyB0bXV4IG5ldy1zZXNzaW9uIC1zIG1vbmdvZGIvMQopCikgOT4vdG1wL2p1anUtdW5pdC1tb25nb2RiLTEtZGVidWctaG9va3MtZXhpdAopIDg+L3RtcC9qdWp1LXVuaXQtbW9uZ29kYi0xLWRlYnVnLWhvb2tzCmV4aXQgJD8K | base64 -d > $F; . $F'\n"),35 result: regexp.QuoteMeta(debugHooksArgs + "ubuntu@dummyenv-2.dns -- sudo /bin/bash -c 'F=$(mktemp); echo IyEvYmluL2Jhc2gKKAojIExvY2sgdGhlIGp1anUtPHVuaXQ+LWRlYnVnIGxvY2tmaWxlLgpmbG9jayAtbiA4IHx8IChlY2hvICJGYWlsZWQgdG8gYWNxdWlyZSAvdG1wL2p1anUtdW5pdC1tb25nb2RiLTEtZGVidWctaG9va3M6IHVuaXQgaXMgYWxyZWFkeSBiZWluZyBkZWJ1Z2dlZCIgMj4mMTsgZXhpdCAxKQooCiMgQ2xvc2UgdGhlIGluaGVyaXRlZCBsb2NrIEZELCBvciB0bXV4IHdpbGwga2VlcCBpdCBvcGVuLgpleGVjIDg+Ji0KCiMgV3JpdGUgb3V0IHRoZSBkZWJ1Zy1ob29rcyBhcmdzLgplY2hvICJlMzBLIiB8IGJhc2U2NCAtZCA+IC90bXAvanVqdS11bml0LW1vbmdvZGItMS1kZWJ1Zy1ob29rcwoKIyBMb2NrIHRoZSBqdWp1LTx1bml0Pi1kZWJ1Zy1leGl0IGxvY2tmaWxlLgpmbG9jayAtbiA5IHx8IGV4aXQgMQoKIyBXYWl0IGZvciB0bXV4IHRvIGJlIGluc3RhbGxlZC4Kd2hpbGUgWyAhIC1mIC91c3IvYmluL3RtdXggXTsgZG8KICAgIHNsZWVwIDEKZG9uZQoKaWYgWyAhIC1mIH4vLnRtdXguY29uZiBdOyB0aGVuCiAgICAgICAgaWYgWyAtZiAvdXNyL3NoYXJlL2J5b2J1L3Byb2ZpbGVzL3RtdXggXTsgdGhlbgogICAgICAgICAgICAgICAgIyBVc2UgYnlvYnUvdG11eCBwcm9maWxlIGZvciBmYW1pbGlhciBrZXliaW5kaW5ncyBhbmQgYnJhbmRpbmcKICAgICAgICAgICAgICAgIGVjaG8gInNvdXJjZS1maWxlIC91c3Ivc2hhcmUvYnlvYnUvcHJvZmlsZXMvdG11eCIgPiB+Ly50bXV4LmNvbmYKICAgICAgICBlbHNlCiAgICAgICAgICAgICAgICAjIE90aGVyd2lzZSwgdXNlIHRoZSBsZWdhY3kganVqdS90bXV4IGNvbmZpZ3VyYXRpb24KICAgICAgICAgICAgICAgIGNhdCA+IH4vLnRtdXguY29uZiA8PEVORAogICAgICAgICAgICAgICAgCiMgU3RhdHVzIGJhcgpzZXQtb3B0aW9uIC1nIHN0YXR1cy1iZyBibGFjawpzZXQtb3B0aW9uIC1nIHN0YXR1cy1mZyB3aGl0ZQoKc2V0LXdpbmRvdy1vcHRpb24gLWcgd2luZG93LXN0YXR1cy1jdXJyZW50LWJnIHJlZApzZXQtd2luZG93LW9wdGlvbiAtZyB3aW5kb3ctc3RhdHVzLWN1cnJlbnQtYXR0ciBicmlnaHQKCnNldC1vcHRpb24gLWcgc3RhdHVzLXJpZ2h0ICcnCgojIFBhbmVzCnNldC1vcHRpb24gLWcgcGFuZS1ib3JkZXItZmcgd2hpdGUKc2V0LW9wdGlvbiAtZyBwYW5lLWFjdGl2ZS1ib3JkZXItZmcgd2hpdGUKCiMgTW9uaXRvciBhY3Rpdml0eSBvbiB3aW5kb3dzCnNldC13aW5kb3ctb3B0aW9uIC1nIG1vbml0b3ItYWN0aXZpdHkgb24KCiMgU2NyZWVuIGJpbmRpbmdzLCBzaW5jZSBwZW9wbGUgYXJlIG1vcmUgZmFtaWxpYXIgd2l0aCB0aGF0LgpzZXQtb3B0aW9uIC1nIHByZWZpeCBDLWEKYmluZCBDLWEgbGFzdC13aW5kb3cKYmluZCBhIHNlbmQta2V5IEMtYQoKYmluZCB8IHNwbGl0LXdpbmRvdyAtaApiaW5kIC0gc3BsaXQtd2luZG93IC12CgojIEZpeCBDVFJMLVBHVVAvUEdET1dOIGZvciB2aW0Kc2V0LXdpbmRvdy1vcHRpb24gLWcgeHRlcm0ta2V5cyBvbgoKIyBQcmV2ZW50IEVTQyBrZXkgZnJvbSBhZGRpbmcgZGVsYXkgYW5kIGJyZWFraW5nIFZpbSdzIEVTQyA+IGFycm93IGtleQpzZXQtb3B0aW9uIC1zIGVzY2FwZS10aW1lIDAKCkVORAogICAgICAgIGZpCmZpCgooCiAgICAjIENsb3NlIHRoZSBpbmhlcml0ZWQgbG9jayBGRCwgb3IgdG11eCB3aWxsIGtlZXAgaXQgb3Blbi4KICAgIGV4ZWMgOT4mLQogICAgZXhlYyB0bXV4IG5ldy1zZXNzaW9uIC1zIG1vbmdvZGIvMQopCikgOT4vdG1wL2p1anUtdW5pdC1tb25nb2RiLTEtZGVidWctaG9va3MtZXhpdAopIDg+L3RtcC9qdWp1LXVuaXQtbW9uZ29kYi0xLWRlYnVnLWhvb2tzCmV4aXQgJD8K | base64 -d > $F; . $F'\n"),
36}, {36}, {
37 info: `"*" is a valid hook name: it means hook everything`,37 info: `"*" is a valid hook name: it means hook everything`,
38 args: []string{"mysql/0", "*"},38 args: []string{"mysql/0", "*"},
3939
=== modified file 'cmd/juju/scp.go'
--- cmd/juju/scp.go 2013-11-20 13:29:15 +0000
+++ cmd/juju/scp.go 2013-12-03 06:15:09 +0000
@@ -5,10 +5,10 @@
55
6import (6import (
7 "errors"7 "errors"
8 "os/exec"
9 "strings"8 "strings"
109
11 "launchpad.net/juju-core/cmd"10 "launchpad.net/juju-core/cmd"
11 "launchpad.net/juju-core/utils/ssh"
12)12)
1313
14// SCPCommand is responsible for launching a scp command to copy files to/from remote machine(s)14// SCPCommand is responsible for launching a scp command to copy files to/from remote machine(s)
@@ -80,9 +80,7 @@
80 }80 }
81 }81 }
8282
83 args := []string{"-o", "StrictHostKeyChecking no", "-o", "PasswordAuthentication no"}83 cmd := ssh.ScpCommand(c.Args[0], c.Args[1], ssh.NoPasswordAuthentication)
84 args = append(args, c.Args...)
85 cmd := exec.Command("scp", args...)
86 cmd.Stdin = ctx.Stdin84 cmd.Stdin = ctx.Stdin
87 cmd.Stdout = ctx.Stdout85 cmd.Stdout = ctx.Stdout
88 cmd.Stderr = ctx.Stderr86 cmd.Stderr = ctx.Stderr
8987
=== modified file 'cmd/juju/scp_test.go'
--- cmd/juju/scp_test.go 2013-11-08 04:45:20 +0000
+++ cmd/juju/scp_test.go 2013-12-03 06:15:09 +0000
@@ -39,7 +39,7 @@
39 },39 },
40 {40 {
41 []string{"a", "b", "mysql/0"},41 []string{"a", "b", "mysql/0"},
42 commonArgs + "a b mysql/0\n",42 commonArgs + "a b\n",
43 },43 },
44 {44 {
45 []string{"mongodb/1:foo", "mongodb/0:"},45 []string{"mongodb/1:foo", "mongodb/0:"},
4646
=== modified file 'cmd/juju/ssh.go'
--- cmd/juju/ssh.go 2013-11-27 12:02:11 +0000
+++ cmd/juju/ssh.go 2013-12-03 06:15:09 +0000
@@ -6,7 +6,6 @@
6import (6import (
7 "errors"7 "errors"
8 "fmt"8 "fmt"
9 "os/exec"
10 "time"9 "time"
1110
12 "launchpad.net/juju-core/cmd"11 "launchpad.net/juju-core/cmd"
@@ -16,6 +15,7 @@
16 "launchpad.net/juju-core/rpc"15 "launchpad.net/juju-core/rpc"
17 "launchpad.net/juju-core/state/api"16 "launchpad.net/juju-core/state/api"
18 "launchpad.net/juju-core/utils"17 "launchpad.net/juju-core/utils"
18 "launchpad.net/juju-core/utils/ssh"
19)19)
2020
21// SSHCommand is responsible for launching a ssh shell on a given unit or machine.21// SSHCommand is responsible for launching a ssh shell on a given unit or machine.
@@ -82,9 +82,13 @@
82 if err != nil {82 if err != nil {
83 return err83 return err
84 }84 }
85 args := []string{"-l", "ubuntu", "-t", "-o", "StrictHostKeyChecking no", "-o", "PasswordAuthentication no", host}85 args := c.Args
86 args = append(args, c.Args...)86 if len(args) > 0 && args[0] == "--" {
87 cmd := exec.Command("ssh", args...)87 // utils/ssh adds "--"; we will continue to accept
88 // it from the CLI for backwards compatibility.
89 args = args[1:]
90 }
91 cmd := ssh.Command("ubuntu@"+host, args, ssh.NoPasswordAuthentication, ssh.AllocateTTY)
88 cmd.Stdin = ctx.Stdin92 cmd.Stdin = ctx.Stdin
89 cmd.Stdout = ctx.Stdout93 cmd.Stdout = ctx.Stdout
90 cmd.Stderr = ctx.Stderr94 cmd.Stderr = ctx.Stderr
9195
=== modified file 'cmd/juju/ssh_test.go'
--- cmd/juju/ssh_test.go 2013-11-08 04:45:20 +0000
+++ cmd/juju/ssh_test.go 2013-12-03 06:15:09 +0000
@@ -60,7 +60,7 @@
6060
61const (61const (
62 commonArgs = `-o StrictHostKeyChecking no -o PasswordAuthentication no `62 commonArgs = `-o StrictHostKeyChecking no -o PasswordAuthentication no `
63 sshArgs = `-l ubuntu -t ` + commonArgs63 sshArgs = commonArgs + `-t `
64)64)
6565
66var sshTests = []struct {66var sshTests = []struct {
@@ -69,30 +69,30 @@
69}{69}{
70 {70 {
71 []string{"ssh", "0"},71 []string{"ssh", "0"},
72 sshArgs + "dummyenv-0.dns\n",72 sshArgs + "ubuntu@dummyenv-0.dns\n",
73 },73 },
74 // juju ssh 0 'uname -a'74 // juju ssh 0 'uname -a'
75 {75 {
76 []string{"ssh", "0", "uname -a"},76 []string{"ssh", "0", "uname -a"},
77 sshArgs + "dummyenv-0.dns uname -a\n",77 sshArgs + "ubuntu@dummyenv-0.dns -- uname -a\n",
78 },78 },
79 // juju ssh 0 -- uname -a79 // juju ssh 0 -- uname -a
80 {80 {
81 []string{"ssh", "0", "--", "uname", "-a"},81 []string{"ssh", "0", "--", "uname", "-a"},
82 sshArgs + "dummyenv-0.dns -- uname -a\n",82 sshArgs + "ubuntu@dummyenv-0.dns -- uname -a\n",
83 },83 },
84 // juju ssh 0 uname -a84 // juju ssh 0 uname -a
85 {85 {
86 []string{"ssh", "0", "uname", "-a"},86 []string{"ssh", "0", "uname", "-a"},
87 sshArgs + "dummyenv-0.dns uname -a\n",87 sshArgs + "ubuntu@dummyenv-0.dns -- uname -a\n",
88 },88 },
89 {89 {
90 []string{"ssh", "mysql/0"},90 []string{"ssh", "mysql/0"},
91 sshArgs + "dummyenv-0.dns\n",91 sshArgs + "ubuntu@dummyenv-0.dns\n",
92 },92 },
93 {93 {
94 []string{"ssh", "mongodb/1"},94 []string{"ssh", "mongodb/1"},
95 sshArgs + "dummyenv-2.dns\n",95 sshArgs + "ubuntu@dummyenv-2.dns\n",
96 },96 },
97}97}
9898
9999
=== modified file 'environs/bootstrap/state.go'
--- environs/bootstrap/state.go 2013-11-18 04:53:44 +0000
+++ environs/bootstrap/state.go 2013-12-03 06:15:09 +0000
@@ -52,6 +52,11 @@
52 return storage.URL(StateFile)52 return storage.URL(StateFile)
53}53}
5454
55// DeleteStateFile deletes the state file on the given storage.
56func DeleteStateFile(storage storage.Storage) error {
57 return storage.Remove(StateFile)
58}
59
55// SaveState writes the given state to the given storage.60// SaveState writes the given state to the given storage.
56func SaveState(storage storage.StorageWriter, state *BootstrapState) error {61func SaveState(storage storage.StorageWriter, state *BootstrapState) error {
57 data, err := goyaml.Marshal(state)62 data, err := goyaml.Marshal(state)
5863
=== modified file 'environs/bootstrap/state_test.go'
--- environs/bootstrap/state_test.go 2013-11-18 04:53:44 +0000
+++ environs/bootstrap/state_test.go 2013-12-03 06:15:09 +0000
@@ -6,6 +6,8 @@
6import (6import (
7 "bytes"7 "bytes"
8 "io/ioutil"8 "io/ioutil"
9 "os"
10 "path/filepath"
911
10 gc "launchpad.net/gocheck"12 gc "launchpad.net/gocheck"
11 "launchpad.net/goyaml"13 "launchpad.net/goyaml"
@@ -15,6 +17,7 @@
15 "launchpad.net/juju-core/environs/storage"17 "launchpad.net/juju-core/environs/storage"
16 envtesting "launchpad.net/juju-core/environs/testing"18 envtesting "launchpad.net/juju-core/environs/testing"
17 "launchpad.net/juju-core/instance"19 "launchpad.net/juju-core/instance"
20 jc "launchpad.net/juju-core/testing/checkers"
18 "launchpad.net/juju-core/testing/testbase"21 "launchpad.net/juju-core/testing/testbase"
19)22)
2023
@@ -48,6 +51,24 @@
48 c.Check(url, gc.Equals, expectedURL)51 c.Check(url, gc.Equals, expectedURL)
49}52}
5053
54func (suite *StateSuite) TestDeleteStateFile(c *gc.C) {
55 closer, stor, dataDir := envtesting.CreateLocalTestStorage(c)
56 defer closer.Close()
57
58 err := bootstrap.DeleteStateFile(stor)
59 c.Assert(err, gc.IsNil) // doesn't exist, juju don't care
60
61 _, err = bootstrap.CreateStateFile(stor)
62 c.Assert(err, gc.IsNil)
63 _, err = os.Stat(filepath.Join(dataDir, bootstrap.StateFile))
64 c.Assert(err, gc.IsNil)
65
66 err = bootstrap.DeleteStateFile(stor)
67 c.Assert(err, gc.IsNil)
68 _, err = os.Stat(filepath.Join(dataDir, bootstrap.StateFile))
69 c.Assert(err, jc.Satisfies, os.IsNotExist)
70}
71
51func (suite *StateSuite) TestSaveStateWritesStateFile(c *gc.C) {72func (suite *StateSuite) TestSaveStateWritesStateFile(c *gc.C) {
52 stor := suite.newStorage(c)73 stor := suite.newStorage(c)
53 arch := "amd64"74 arch := "amd64"
5475
=== modified file 'environs/cloudinit.go'
--- environs/cloudinit.go 2013-11-26 12:24:48 +0000
+++ environs/cloudinit.go 2013-12-03 06:15:09 +0000
@@ -133,9 +133,16 @@
133 for _, script := range additionalScripts {133 for _, script := range additionalScripts {
134 cloudcfg.AddRunCmd(script)134 cloudcfg.AddRunCmd(script)
135 }135 }
136 err := cloudinit.Configure(cfg, cloudcfg)136 // When bootstrapping, we only want to apt-get update/upgrade
137 if err != nil {137 // and setup the SSH keys. The rest we leave to cloudinit/sshinit.
138 return nil, err138 if cfg.StateServer {
139 if err := cloudinit.ConfigureBasic(cfg, cloudcfg); err != nil {
140 return nil, err
141 }
142 } else {
143 if err := cloudinit.Configure(cfg, cloudcfg); err != nil {
144 return nil, err
145 }
139 }146 }
140 data, err := cloudcfg.Render()147 data, err := cloudcfg.Render()
141 logger.Tracef("Generated cloud init:\n%s", string(data))148 logger.Tracef("Generated cloud init:\n%s", string(data))
142149
=== modified file 'environs/cloudinit/cloudinit.go'
--- environs/cloudinit/cloudinit.go 2013-11-26 12:24:48 +0000
+++ environs/cloudinit/cloudinit.go 2013-12-03 06:15:09 +0000
@@ -132,16 +132,60 @@
132// Configure updates the provided cloudinit.Config with132// Configure updates the provided cloudinit.Config with
133// configuration to initialize a Juju machine agent.133// configuration to initialize a Juju machine agent.
134func Configure(cfg *MachineConfig, c *cloudinit.Config) error {134func Configure(cfg *MachineConfig, c *cloudinit.Config) error {
135 if err := ConfigureBasic(cfg, c); err != nil {
136 return err
137 }
138 return ConfigureJuju(cfg, c)
139}
140
141const cloudInitOutputLog = "/var/log/cloud-init-output.log"
142
143// ConfigureBasic updates the provided cloudinit.Config with
144// basic configuration to initialise an OS image, such that it can
145// be connected to via SSH, and log to a standard location.
146//
147// Any potentially failing operation should not be added to the
148// configuration, but should instead be done in ConfigureJuju.
149//
150// Note: we don't do apt update/upgrade here so as not to have to wait on
151// apt to finish when performing the second half of image initialisation.
152// Doing it later brings the benefit of feedback in the face of errors,
153// but adds to the running time of initialisation due to lack of activity
154// between image bringup and start of agent installation.
155func ConfigureBasic(cfg *MachineConfig, c *cloudinit.Config) error {
156 c.AddSSHAuthorizedKeys(cfg.AuthorizedKeys)
157 c.SetOutput(cloudinit.OutAll, "| tee -a "+cloudInitOutputLog, "")
158 return nil
159}
160
161// ConfigureJuju updates the provided cloudinit.Config with configuration
162// to initialise a Juju machine agent.
163func ConfigureJuju(cfg *MachineConfig, c *cloudinit.Config) error {
135 if err := verifyConfig(cfg); err != nil {164 if err := verifyConfig(cfg); err != nil {
136 return err165 return err
137 }166 }
138167
139 // General options.168 // Initialise progress reporting. We need to do separately for runcmd
169 // and (possibly, below) for bootcmd, as they may be run in different
170 // shell sessions.
171 initProgressCmd := cloudinit.InitProgressCmd()
172 c.AddRunCmd(initProgressCmd)
173
174 // If we're doing synchronous bootstrap or manual provisioning, then
175 // ConfigureBasic won't have been invoked; thus, the output log won't
176 // have been set. We don't want to show the log to the user, so simply
177 // append to the log file rather than teeing.
178 if stdout, _ := c.Output(cloudinit.OutAll); stdout == "" {
179 c.SetOutput(cloudinit.OutAll, ">> "+cloudInitOutputLog, "")
180 c.AddBootCmd(initProgressCmd)
181 c.AddBootCmd(cloudinit.LogProgressCmd("Logging to %s on remote host", cloudInitOutputLog))
182 }
183
184 // Bring packages up-to-date.
185 c.SetAptUpdate(true)
140 c.SetAptUpgrade(true)186 c.SetAptUpgrade(true)
141 c.SetAptUpdate(true)
142 c.SetOutput(cloudinit.OutAll, "| tee -a /var/log/cloud-init-output.log", "")
143 c.AddSSHAuthorizedKeys(cfg.AuthorizedKeys)
144187
188 // juju requires git for managing charm directories.
145 c.AddPackage("git")189 c.AddPackage("git")
146190
147 c.AddScripts(191 c.AddScripts(
@@ -149,17 +193,18 @@
149 fmt.Sprintf("mkdir -p %s", cfg.DataDir),193 fmt.Sprintf("mkdir -p %s", cfg.DataDir),
150 "mkdir -p /var/log/juju")194 "mkdir -p /var/log/juju")
151195
152 wgetCommand := "wget"
153 if cfg.DisableSSLHostnameVerification {
154 wgetCommand = "wget --no-check-certificate"
155 }
156 // Make a directory for the tools to live in, then fetch the196 // Make a directory for the tools to live in, then fetch the
157 // tools and unarchive them into it.197 // tools and unarchive them into it.
158 var copyCmd string198 var copyCmd string
159 if strings.HasPrefix(cfg.Tools.URL, fileSchemePrefix) {199 if strings.HasPrefix(cfg.Tools.URL, fileSchemePrefix) {
160 copyCmd = fmt.Sprintf("cp %s $bin/tools.tar.gz", shquote(cfg.Tools.URL[len(fileSchemePrefix):]))200 copyCmd = fmt.Sprintf("cp %s $bin/tools.tar.gz", shquote(cfg.Tools.URL[len(fileSchemePrefix):]))
161 } else {201 } else {
202 wgetCommand := "wget"
203 if cfg.DisableSSLHostnameVerification {
204 wgetCommand = "wget --no-check-certificate"
205 }
162 copyCmd = fmt.Sprintf("%s --no-verbose -O $bin/tools.tar.gz %s", wgetCommand, shquote(cfg.Tools.URL))206 copyCmd = fmt.Sprintf("%s --no-verbose -O $bin/tools.tar.gz %s", wgetCommand, shquote(cfg.Tools.URL))
207 c.AddRunCmd(cloudinit.LogProgressCmd("Fetching tools: %s", copyCmd))
163 }208 }
164 toolsJson, err := json.Marshal(cfg.Tools)209 toolsJson, err := json.Marshal(cfg.Tools)
165 if err != nil {210 if err != nil {
@@ -231,6 +276,7 @@
231 if cons != "" {276 if cons != "" {
232 cons = " --constraints " + shquote(cons)277 cons = " --constraints " + shquote(cons)
233 }278 }
279 c.AddRunCmd(cloudinit.LogProgressCmd("Bootstrapping Juju machine agent"))
234 c.AddScripts(280 c.AddScripts(
235 fmt.Sprintf("echo %s > %s", shquote(cfg.StateInfoURL), BootstrapStateURLFile),281 fmt.Sprintf("echo %s > %s", shquote(cfg.StateInfoURL), BootstrapStateURLFile),
236 // The bootstrapping is always run with debug on.282 // The bootstrapping is always run with debug on.
@@ -340,6 +386,7 @@
340 if err != nil {386 if err != nil {
341 return fmt.Errorf("cannot make cloud-init upstart script for the %s agent: %v", tag, err)387 return fmt.Errorf("cannot make cloud-init upstart script for the %s agent: %v", tag, err)
342 }388 }
389 c.AddRunCmd(cloudinit.LogProgressCmd("Starting Juju machine agent (%s)", name))
343 c.AddScripts(cmds...)390 c.AddScripts(cmds...)
344 return nil391 return nil
345}392}
@@ -361,6 +408,7 @@
361 if err != nil {408 if err != nil {
362 return fmt.Errorf("cannot make cloud-init upstart script for the state database: %v", err)409 return fmt.Errorf("cannot make cloud-init upstart script for the state database: %v", err)
363 }410 }
411 c.AddRunCmd(cloudinit.LogProgressCmd("Starting MongoDB server (%s)", name))
364 c.AddScripts(cmds...)412 c.AddScripts(cmds...)
365 return nil413 return nil
366}414}
367415
=== modified file 'environs/cloudinit/cloudinit_test.go'
--- environs/cloudinit/cloudinit_test.go 2013-11-26 12:24:48 +0000
+++ environs/cloudinit/cloudinit_test.go 2013-12-03 06:15:09 +0000
@@ -88,9 +88,11 @@
88 setEnvConfig: true,88 setEnvConfig: true,
89 expectScripts: `89 expectScripts: `
90echo ENABLE_MONGODB="no" > /etc/default/mongodb90echo ENABLE_MONGODB="no" > /etc/default/mongodb
91test -e /proc/self/fd/9 \|\| exec 9>&2
91set -xe92set -xe
92mkdir -p /var/lib/juju93mkdir -p /var/lib/juju
93mkdir -p /var/log/juju94mkdir -p /var/log/juju
95echo 'Fetching tools.*
94bin='/var/lib/juju/tools/1\.2\.3-precise-amd64'96bin='/var/lib/juju/tools/1\.2\.3-precise-amd64'
95mkdir -p \$bin97mkdir -p \$bin
96wget --no-verbose -O \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-precise-amd64\.tgz'98wget --no-verbose -O \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-precise-amd64\.tgz'
@@ -114,6 +116,7 @@
114dd bs=1M count=1 if=/dev/zero of=/var/lib/juju/db/journal/prealloc\.0116dd bs=1M count=1 if=/dev/zero of=/var/lib/juju/db/journal/prealloc\.0
115dd bs=1M count=1 if=/dev/zero of=/var/lib/juju/db/journal/prealloc\.1117dd bs=1M count=1 if=/dev/zero of=/var/lib/juju/db/journal/prealloc\.1
116dd bs=1M count=1 if=/dev/zero of=/var/lib/juju/db/journal/prealloc\.2118dd bs=1M count=1 if=/dev/zero of=/var/lib/juju/db/journal/prealloc\.2
119echo 'Starting MongoDB server \(juju-db\)'.*
117cat >> /etc/init/juju-db\.conf << 'EOF'\\ndescription "juju state database"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 65000 65000\\nlimit nproc 20000 20000\\n\\nexec /usr/bin/mongod --auth --dbpath=/var/lib/juju/db --sslOnNormalPorts --sslPEMKeyFile '/var/lib/juju/server\.pem' --sslPEMKeyPassword ignored --bind_ip 0\.0\.0\.0 --port 37017 --noprealloc --syslog --smallfiles\\nEOF\\n120cat >> /etc/init/juju-db\.conf << 'EOF'\\ndescription "juju state database"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 65000 65000\\nlimit nproc 20000 20000\\n\\nexec /usr/bin/mongod --auth --dbpath=/var/lib/juju/db --sslOnNormalPorts --sslPEMKeyFile '/var/lib/juju/server\.pem' --sslPEMKeyPassword ignored --bind_ip 0\.0\.0\.0 --port 37017 --noprealloc --syslog --smallfiles\\nEOF\\n
118start juju-db121start juju-db
119mkdir -p '/var/lib/juju/agents/bootstrap'122mkdir -p '/var/lib/juju/agents/bootstrap'
@@ -121,10 +124,12 @@
121printf '%s\\n' '.*' > '/var/lib/juju/agents/bootstrap/format'124printf '%s\\n' '.*' > '/var/lib/juju/agents/bootstrap/format'
122install -m 600 /dev/null '/var/lib/juju/agents/bootstrap/agent\.conf'125install -m 600 /dev/null '/var/lib/juju/agents/bootstrap/agent\.conf'
123printf '%s\\n' '.*' > '/var/lib/juju/agents/bootstrap/agent\.conf'126printf '%s\\n' '.*' > '/var/lib/juju/agents/bootstrap/agent\.conf'
127echo 'Bootstrapping Juju machine agent'.*
124echo 'some-url' > /tmp/provider-state-url128echo 'some-url' > /tmp/provider-state-url
125/var/lib/juju/tools/1\.2\.3-precise-amd64/jujud bootstrap-state --data-dir '/var/lib/juju' --env-config '[^']*' --constraints 'mem=2048M' --debug129/var/lib/juju/tools/1\.2\.3-precise-amd64/jujud bootstrap-state --data-dir '/var/lib/juju' --env-config '[^']*' --constraints 'mem=2048M' --debug
126rm -rf '/var/lib/juju/agents/bootstrap'130rm -rf '/var/lib/juju/agents/bootstrap'
127ln -s 1\.2\.3-precise-amd64 '/var/lib/juju/tools/machine-0'131ln -s 1\.2\.3-precise-amd64 '/var/lib/juju/tools/machine-0'
132echo 'Starting Juju machine agent \(jujud-machine-0\)'.*
128cat >> /etc/init/jujud-machine-0\.conf << 'EOF'\\ndescription "juju machine-0 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-0/jujud machine --data-dir '/var/lib/juju' --machine-id 0 --debug >> /var/log/juju/machine-0\.log 2>&1\\nEOF\\n133cat >> /etc/init/jujud-machine-0\.conf << 'EOF'\\ndescription "juju machine-0 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-0/jujud machine --data-dir '/var/lib/juju' --machine-id 0 --debug >> /var/log/juju/machine-0\.log 2>&1\\nEOF\\n
129start jujud-machine-0134start jujud-machine-0
130`,135`,
@@ -193,9 +198,11 @@
193 SyslogPort: 514,198 SyslogPort: 514,
194 },199 },
195 expectScripts: `200 expectScripts: `
201test -e /proc/self/fd/9 \|\| exec 9>&2
196set -xe202set -xe
197mkdir -p /var/lib/juju203mkdir -p /var/lib/juju
198mkdir -p /var/log/juju204mkdir -p /var/log/juju
205echo 'Fetching tools.*
199bin='/var/lib/juju/tools/1\.2\.3-linux-amd64'206bin='/var/lib/juju/tools/1\.2\.3-linux-amd64'
200mkdir -p \$bin207mkdir -p \$bin
201wget --no-verbose -O \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-linux-amd64\.tgz'208wget --no-verbose -O \$bin/tools\.tar\.gz 'http://foo\.com/tools/releases/juju1\.2\.3-linux-amd64\.tgz'
@@ -213,6 +220,7 @@
213install -m 600 /dev/null '/var/lib/juju/agents/machine-99/agent\.conf'220install -m 600 /dev/null '/var/lib/juju/agents/machine-99/agent\.conf'
214printf '%s\\n' '.*' > '/var/lib/juju/agents/machine-99/agent\.conf'221printf '%s\\n' '.*' > '/var/lib/juju/agents/machine-99/agent\.conf'
215ln -s 1\.2\.3-linux-amd64 '/var/lib/juju/tools/machine-99'222ln -s 1\.2\.3-linux-amd64 '/var/lib/juju/tools/machine-99'
223echo 'Starting Juju machine agent \(jujud-machine-99\)'.*
216cat >> /etc/init/jujud-machine-99\.conf << 'EOF'\\ndescription "juju machine-99 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-99/jujud machine --data-dir '/var/lib/juju' --machine-id 99 --debug >> /var/log/juju/machine-99\.log 2>&1\\nEOF\\n224cat >> /etc/init/jujud-machine-99\.conf << 'EOF'\\ndescription "juju machine-99 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-99/jujud machine --data-dir '/var/lib/juju' --machine-id 99 --debug >> /var/log/juju/machine-99\.log 2>&1\\nEOF\\n
217start jujud-machine-99225start jujud-machine-99
218`,226`,
219227
=== modified file 'environs/cloudinit_test.go'
--- environs/cloudinit_test.go 2013-11-20 04:28:46 +0000
+++ environs/cloudinit_test.go 2013-12-03 06:15:09 +0000
@@ -139,7 +139,15 @@
139 c.Assert(err, gc.NotNil)139 c.Assert(err, gc.NotNil)
140}140}
141141
142func (*CloudInitSuite) TestUserData(c *gc.C) {142func (s *CloudInitSuite) TestUserData(c *gc.C) {
143 s.testUserData(c, false)
144}
145
146func (s *CloudInitSuite) TestStateServerUserData(c *gc.C) {
147 s.testUserData(c, true)
148}
149
150func (*CloudInitSuite) testUserData(c *gc.C, stateServer bool) {
143 testJujuHome := c.MkDir()151 testJujuHome := c.MkDir()
144 defer config.SetJujuHome(config.SetJujuHome(testJujuHome))152 defer config.SetJujuHome(config.SetJujuHome(testJujuHome))
145 tools := &tools.Tools{153 tools := &tools.Tools{
@@ -156,20 +164,25 @@
156 StateServerCert: []byte(testing.ServerCert),164 StateServerCert: []byte(testing.ServerCert),
157 StateServerKey: []byte(testing.ServerKey),165 StateServerKey: []byte(testing.ServerKey),
158 StateInfo: &state.Info{166 StateInfo: &state.Info{
167 Addrs: []string{"127.0.0.1:1234"},
159 Password: "pw1",168 Password: "pw1",
160 CACert: []byte("CA CERT\n" + testing.CACert),169 CACert: []byte("CA CERT\n" + testing.CACert),
170 Tag: "machine-10",
161 },171 },
162 APIInfo: &api.Info{172 APIInfo: &api.Info{
173 Addrs: []string{"127.0.0.1:1234"},
163 Password: "pw2",174 Password: "pw2",
164 CACert: []byte("CA CERT\n" + testing.CACert),175 CACert: []byte("CA CERT\n" + testing.CACert),
176 Tag: "machine-10",
165 },177 },
166 DataDir: environs.DataDir,178 DataDir: environs.DataDir,
167 Config: envConfig,179 Config: envConfig,
168 StatePort: envConfig.StatePort(),180 StatePort: envConfig.StatePort(),
169 APIPort: envConfig.APIPort(),181 APIPort: envConfig.APIPort(),
170 SyslogPort: envConfig.SyslogPort(),182 SyslogPort: envConfig.SyslogPort(),
171 StateServer: true,183 StateServer: stateServer,
172 AgentEnvironment: map[string]string{agent.ProviderType: "dummy"},184 AgentEnvironment: map[string]string{agent.ProviderType: "dummy"},
185 AuthorizedKeys: "wheredidileavemykeys",
173 }186 }
174 script1 := "script1"187 script1 := "script1"
175 script2 := "script2"188 script2 := "script2"
@@ -184,11 +197,32 @@
184 err = goyaml.Unmarshal(unzipped, &config)197 err = goyaml.Unmarshal(unzipped, &config)
185 c.Assert(err, gc.IsNil)198 c.Assert(err, gc.IsNil)
186199
187 // Just check that the cloudinit config looks good.
188 c.Check(config["apt_upgrade"], gc.Equals, true)
189 // The scripts given to userData where added as the first200 // The scripts given to userData where added as the first
190 // commands to be run.201 // commands to be run.
191 runCmd := config["runcmd"].([]interface{})202 runCmd := config["runcmd"].([]interface{})
192 c.Check(runCmd[0], gc.Equals, script1)203 c.Check(runCmd[0], gc.Equals, script1)
193 c.Check(runCmd[1], gc.Equals, script2)204 c.Check(runCmd[1], gc.Equals, script2)
205
206 if stateServer {
207 // The cloudinit config should have nothing but the basics:
208 // SSH authorized keys, the additional runcmds, and log output.
209 //
210 // Note: the additional runcmds *do* belong here, at least
211 // for MAAS. MAAS needs to configure and then bounce the
212 // network interfaces, which would sever the SSH connection
213 // in the synchronous bootstrap phase.
214 c.Check(config, gc.DeepEquals, map[interface{}]interface{}{
215 "output": map[interface{}]interface{}{
216 "all": "| tee -a /var/log/cloud-init-output.log",
217 },
218 "runcmd": []interface{}{"script1", "script2"},
219 "ssh_authorized_keys": []interface{}{"wheredidileavemykeys"},
220 })
221 } else {
222 // Just check that the cloudinit config looks good,
223 // and that there are more runcmds than the additional
224 // ones we passed into ComposeUserData.
225 c.Check(config["apt_upgrade"], gc.Equals, true)
226 c.Check(len(runCmd) > 2, jc.IsTrue)
227 }
194}228}
195229
=== modified file 'environs/manual/bootstrap.go'
--- environs/manual/bootstrap.go 2013-11-18 05:41:36 +0000
+++ environs/manual/bootstrap.go 2013-12-03 06:15:09 +0000
@@ -134,5 +134,5 @@
134 for k, v := range agentEnv {134 for k, v := range agentEnv {
135 mcfg.AgentEnvironment[k] = v135 mcfg.AgentEnvironment[k] = v
136 }136 }
137 return ProvisionMachineAgent(args.Host, mcfg)137 return provisionMachineAgent(args.Host, mcfg)
138}138}
139139
=== modified file 'environs/manual/detection.go'
--- environs/manual/detection.go 2013-10-09 06:54:15 +0000
+++ environs/manual/detection.go 2013-12-03 06:15:09 +0000
@@ -12,6 +12,7 @@
1212
13 "launchpad.net/juju-core/instance"13 "launchpad.net/juju-core/instance"
14 "launchpad.net/juju-core/utils"14 "launchpad.net/juju-core/utils"
15 "launchpad.net/juju-core/utils/ssh"
15)16)
1617
17// checkProvisionedScript is the script to run on the remote machine18// checkProvisionedScript is the script to run on the remote machine
@@ -25,7 +26,7 @@
25// exist on the host machine.26// exist on the host machine.
26func checkProvisioned(sshHost string) (bool, error) {27func checkProvisioned(sshHost string) (bool, error) {
27 logger.Infof("Checking if %s is already provisioned", sshHost)28 logger.Infof("Checking if %s is already provisioned", sshHost)
28 cmd := sshCommand(sshHost, fmt.Sprintf("bash -c %s", utils.ShQuote(checkProvisionedScript)))29 cmd := ssh.Command(sshHost, []string{"bash", "-c", utils.ShQuote(checkProvisionedScript)})
29 var stdout, stderr bytes.Buffer30 var stdout, stderr bytes.Buffer
30 cmd.Stdout = &stdout31 cmd.Stdout = &stdout
31 cmd.Stderr = &stderr32 cmd.Stderr = &stderr
@@ -52,7 +53,7 @@
52// The sshHost argument must be a hostname of the form [user@]host.53// The sshHost argument must be a hostname of the form [user@]host.
53func DetectSeriesAndHardwareCharacteristics(sshHost string) (hc instance.HardwareCharacteristics, series string, err error) {54func DetectSeriesAndHardwareCharacteristics(sshHost string) (hc instance.HardwareCharacteristics, series string, err error) {
54 logger.Infof("Detecting series and characteristics on %s", sshHost)55 logger.Infof("Detecting series and characteristics on %s", sshHost)
55 cmd := sshCommand(sshHost, "bash")56 cmd := ssh.Command(sshHost, []string{"bash"})
56 cmd.Stdin = bytes.NewBufferString(detectionScript)57 cmd.Stdin = bytes.NewBufferString(detectionScript)
57 out, err := cmd.CombinedOutput()58 out, err := cmd.CombinedOutput()
58 if err != nil {59 if err != nil {
5960
=== modified file 'environs/manual/provisioner.go'
--- environs/manual/provisioner.go 2013-11-26 14:06:20 +0000
+++ environs/manual/provisioner.go 2013-12-03 06:15:09 +0000
@@ -11,6 +11,8 @@
1111
12 "launchpad.net/loggo"12 "launchpad.net/loggo"
1313
14 coreCloudinit "launchpad.net/juju-core/cloudinit"
15 "launchpad.net/juju-core/cloudinit/sshinit"
14 "launchpad.net/juju-core/constraints"16 "launchpad.net/juju-core/constraints"
15 "launchpad.net/juju-core/environs"17 "launchpad.net/juju-core/environs"
16 "launchpad.net/juju-core/environs/cloudinit"18 "launchpad.net/juju-core/environs/cloudinit"
@@ -89,7 +91,7 @@
89 }91 }
9092
91 // Finally, provision the machine agent.93 // Finally, provision the machine agent.
92 err = ProvisionMachineAgent(args.Host, mcfg)94 err = provisionMachineAgent(args.Host, mcfg)
93 if err != nil {95 if err != nil {
94 return machineId, err96 return machineId, err
95 }97 }
@@ -202,3 +204,14 @@
202 }204 }
203 return mcfg, nil205 return mcfg, nil
204}206}
207
208func provisionMachineAgent(host string, mcfg *cloudinit.MachineConfig) error {
209 cloudcfg := coreCloudinit.New()
210 if err := cloudinit.ConfigureJuju(mcfg, cloudcfg); err != nil {
211 return err
212 }
213 // Explicitly disabling apt_upgrade so as not to trample
214 // the target machine's existing configuration.
215 cloudcfg.SetAptUpgrade(false)
216 return sshinit.Configure(host, cloudcfg)
217}
205218
=== modified file 'environs/sshstorage/storage.go'
--- environs/sshstorage/storage.go 2013-10-10 01:31:43 +0000
+++ environs/sshstorage/storage.go 2013-12-03 06:15:09 +0000
@@ -20,6 +20,7 @@
2020
21 coreerrors "launchpad.net/juju-core/errors"21 coreerrors "launchpad.net/juju-core/errors"
22 "launchpad.net/juju-core/utils"22 "launchpad.net/juju-core/utils"
23 "launchpad.net/juju-core/utils/ssh"
23)24)
2425
25// base64LineLength is the default line length for wrapping26// base64LineLength is the default line length for wrapping
@@ -44,12 +45,11 @@
44}45}
4546
46var sshCommand = func(host string, tty bool, command string) *exec.Cmd {47var sshCommand = func(host string, tty bool, command string) *exec.Cmd {
47 sshArgs := []string{host}48 var options []ssh.Option
48 if tty {49 if tty {
49 sshArgs = append(sshArgs, "-t")50 options = append(options, ssh.AllocateTTY)
50 }51 }
51 sshArgs = append(sshArgs, "--", command)52 return ssh.Command(host, []string{command}, options...)
52 return exec.Command("ssh", sshArgs...)
53}53}
5454
55type flockmode string55type flockmode string
5656
=== added file 'environs/testing/bootstrap.go'
--- environs/testing/bootstrap.go 1970-01-01 00:00:00 +0000
+++ environs/testing/bootstrap.go 2013-12-03 06:15:09 +0000
@@ -0,0 +1,26 @@
1// Copyright 2013 Canonical Ltd.
2// Licensed under the AGPLv3, see LICENCE file for details.
3
4package testing
5
6import (
7 "launchpad.net/loggo"
8
9 "launchpad.net/juju-core/environs/cloudinit"
10 "launchpad.net/juju-core/instance"
11 "launchpad.net/juju-core/provider/common"
12 "launchpad.net/juju-core/testing/testbase"
13)
14
15var logger = loggo.GetLogger("juju.environs.testing")
16
17// DisableFinishBootstrap disables common.FinishBootstrap so that tests
18// do not attempt to SSH to non-existent machines. The result is a function
19// that restores finishBootstrap.
20func DisableFinishBootstrap() func() {
21 f := func(*common.BootstrapContext, instance.Instance, *cloudinit.MachineConfig) error {
22 logger.Warningf("provider/common.FinishBootstrap is disabled")
23 return nil
24 }
25 return testbase.PatchValue(&common.FinishBootstrap, f)
26}
027
=== modified file 'provider/common/bootstrap.go'
--- provider/common/bootstrap.go 2013-11-18 04:53:44 +0000
+++ provider/common/bootstrap.go 2013-12-03 06:15:09 +0000
@@ -5,12 +5,21 @@
55
6import (6import (
7 "fmt"7 "fmt"
8 "net"
9 "os"
10 "os/signal"
11 "sync"
12 "time"
813
9 "launchpad.net/loggo"14 "launchpad.net/loggo"
15 "launchpad.net/tomb"
1016
17 coreCloudinit "launchpad.net/juju-core/cloudinit"
18 "launchpad.net/juju-core/cloudinit/sshinit"
11 "launchpad.net/juju-core/constraints"19 "launchpad.net/juju-core/constraints"
12 "launchpad.net/juju-core/environs"20 "launchpad.net/juju-core/environs"
13 "launchpad.net/juju-core/environs/bootstrap"21 "launchpad.net/juju-core/environs/bootstrap"
22 "launchpad.net/juju-core/environs/cloudinit"
14 "launchpad.net/juju-core/instance"23 "launchpad.net/juju-core/instance"
15 coretools "launchpad.net/juju-core/tools"24 coretools "launchpad.net/juju-core/tools"
16)25)
@@ -20,11 +29,20 @@
20// Bootstrap is a common implementation of the Bootstrap method defined on29// Bootstrap is a common implementation of the Bootstrap method defined on
21// environs.Environ; we strongly recommend that this implementation be used30// environs.Environ; we strongly recommend that this implementation be used
22// when writing a new provider.31// when writing a new provider.
23func Bootstrap(env environs.Environ, cons constraints.Value) error {32func Bootstrap(env environs.Environ, cons constraints.Value) (err error) {
24 // TODO make safe in the case of racing Bootstraps33 // TODO make safe in the case of racing Bootstraps
25 // If two Bootstraps are called concurrently, there's34 // If two Bootstraps are called concurrently, there's
26 // no way to make sure that only one succeeds.35 // no way to make sure that only one succeeds.
2736
37 // TODO(axw) 2013-11-22 #1237736
38 // Modify environs/Environ Bootstrap method signature
39 // to take a new context structure, which contains
40 // Std{in,out,err}, and interrupt signal handling.
41 ctx := BootstrapContext{Stderr: os.Stderr}
42
43 var inst instance.Instance
44 defer func() { handleBootstrapError(err, &ctx, inst, env) }()
45
28 // Create an empty bootstrap state file so we can get its URL.46 // Create an empty bootstrap state file so we can get its URL.
29 // It will be updated with the instance id and hardware characteristics47 // It will be updated with the instance id and hardware characteristics
30 // after the bootstrap instance is started.48 // after the bootstrap instance is started.
@@ -39,10 +57,13 @@
39 return err57 return err
40 }58 }
4159
60 fmt.Fprintln(ctx.Stderr, "Launching instance")
42 inst, hw, err := env.StartInstance(cons, selectedTools, machineConfig)61 inst, hw, err := env.StartInstance(cons, selectedTools, machineConfig)
43 if err != nil {62 if err != nil {
44 return fmt.Errorf("cannot start bootstrap instance: %v", err)63 return fmt.Errorf("cannot start bootstrap instance: %v", err)
45 }64 }
65 fmt.Fprintf(ctx.Stderr, " - %s\n", inst.Id())
66
46 var characteristics []instance.HardwareCharacteristics67 var characteristics []instance.HardwareCharacteristics
47 if hw != nil {68 if hw != nil {
48 characteristics = []instance.HardwareCharacteristics{*hw}69 characteristics = []instance.HardwareCharacteristics{*hw}
@@ -54,14 +75,148 @@
54 Characteristics: characteristics,75 Characteristics: characteristics,
55 })76 })
56 if err != nil {77 if err != nil {
57 stoperr := env.StopInstances([]instance.Instance{inst})78 return fmt.Errorf("cannot save state: %v", err)
58 if stoperr != nil {79 }
59 // Failure upon failure. Log it, but return the original error.80 return FinishBootstrap(&ctx, inst, machineConfig)
81}
82
83// handelBootstrapError cleans up after a failed bootstrap.
84func handleBootstrapError(err error, ctx *BootstrapContext, inst instance.Instance, env environs.Environ) {
85 if err == nil {
86 return
87 }
88 ctx.SetInterruptHandler(func() {
89 fmt.Fprintln(ctx.Stderr, "Cleaning up failed bootstrap")
90 })
91 if inst != nil {
92 fmt.Fprintln(ctx.Stderr, "Stopping instance...")
93 if stoperr := env.StopInstances([]instance.Instance{inst}); stoperr != nil {
60 logger.Errorf("cannot stop failed bootstrap instance %q: %v", inst.Id(), stoperr)94 logger.Errorf("cannot stop failed bootstrap instance %q: %v", inst.Id(), stoperr)
61 }95 } else {
62 return fmt.Errorf("cannot save state: %v", err)96 // set to nil so we know we can safely delete the state file
63 }97 inst = nil
64 return nil98 }
99 }
100 // We only delete the bootstrap state file if either we didn't
101 // start an instance, or we managed to cleanly stop it.
102 if inst == nil {
103 if rmerr := bootstrap.DeleteStateFile(env.Storage()); rmerr != nil {
104 logger.Errorf("cannot delete bootstrap state file: %v", rmerr)
105 }
106 }
107 ctx.SetInterruptHandler(nil)
108}
109
110// FinishBootstrap completes the bootstrap process by connecting
111// to the instance via SSH and carrying out the cloud-config.
112//
113// Note: FinishBootstrap is exposed so it can be replaced for testing.
114var FinishBootstrap = func(ctx *BootstrapContext, inst instance.Instance, machineConfig *cloudinit.MachineConfig) error {
115 var t tomb.Tomb
116 ctx.SetInterruptHandler(func() { t.Killf("interrupted") })
117 dnsName, err := waitSSH(ctx, inst, &t)
118 if err != nil {
119 return err
120 }
121 // Bootstrap is synchronous, and will spawn a subprocess
122 // to complete the procedure. If the user hits Ctrl-C,
123 // SIGINT is sent to the foreground process attached to
124 // the terminal, which will be the ssh subprocess at that
125 // point.
126 ctx.SetInterruptHandler(func() {})
127 cloudcfg := coreCloudinit.New()
128 if err := cloudinit.ConfigureJuju(machineConfig, cloudcfg); err != nil {
129 return err
130 }
131 return sshinit.Configure("ubuntu@"+dnsName, cloudcfg)
132}
133
134// waitSSH waits for the instance to be assigned a DNS
135// entry, then waits until we can connect to it via SSH.
136func waitSSH(ctx *BootstrapContext, inst instance.Instance, t *tomb.Tomb) (dnsName string, err error) {
137 defer t.Done()
138
139 // Wait for a DNS name.
140 fmt.Fprint(ctx.Stderr, "Waiting for DNS name")
141 for {
142 fmt.Fprintf(ctx.Stderr, ".")
143 dnsName, err = inst.DNSName()
144 if err == nil {
145 break
146 } else if err != instance.ErrNoDNSName {
147 fmt.Fprintln(ctx.Stderr)
148 return "", t.Killf("getting DNS name: %v", err)
149 }
150 select {
151 case <-time.After(1 * time.Second):
152 case <-t.Dying():
153 fmt.Fprintln(ctx.Stderr)
154 return "", t.Err()
155 }
156 }
157 fmt.Fprintf(ctx.Stderr, "\n - %v\n", dnsName)
158
159 // Wait until we can open a connection to port 22.
160 fmt.Fprintf(ctx.Stderr, "Attempting to connect to %s:22", dnsName)
161 for {
162 fmt.Fprintf(ctx.Stderr, ".")
163 conn, err := net.DialTimeout("tcp", dnsName+":22", 5*time.Second)
164 if err == nil {
165 conn.Close()
166 fmt.Fprintln(ctx.Stderr)
167 return dnsName, nil
168 } else {
169 logger.Debugf("connection failed: %v", err)
170 }
171 select {
172 case <-time.After(5 * time.Second):
173 case <-t.Dying():
174 return "", t.Err()
175 }
176 }
177}
178
179// TODO(axw) move this to environs; see
180// comment near the top of common.Bootstrap.
181type BootstrapContext struct {
182 once sync.Once
183 handlerchan chan func()
184
185 Stderr *os.File
186}
187
188func (ctx *BootstrapContext) SetInterruptHandler(f func()) {
189 ctx.once.Do(ctx.initHandler)
190 ctx.handlerchan <- f
191}
192
193func (ctx *BootstrapContext) initHandler() {
194 ctx.handlerchan = make(chan func())
195 go ctx.handleInterrupt()
196}
197
198func (ctx *BootstrapContext) handleInterrupt() {
199 signalchan := make(chan os.Signal, 1)
200 var s chan os.Signal
201 var handler func()
202 for {
203 select {
204 case handler = <-ctx.handlerchan:
205 if handler == nil {
206 if s != nil {
207 signal.Stop(signalchan)
208 s = nil
209 }
210 } else {
211 if s == nil {
212 s = signalchan
213 signal.Notify(signalchan, os.Interrupt)
214 }
215 }
216 case <-s:
217 handler()
218 }
219 }
65}220}
66221
67// EnsureBootstrapTools finds tools, syncing with an external tools source as222// EnsureBootstrapTools finds tools, syncing with an external tools source as
68223
=== modified file 'provider/common/bootstrap_test.go'
--- provider/common/bootstrap_test.go 2013-11-18 04:53:44 +0000
+++ provider/common/bootstrap_test.go 2013-12-03 06:15:09 +0000
@@ -197,6 +197,9 @@
197 return minimalConfig(c)197 return minimalConfig(c)
198 }198 }
199199
200 restore := envtesting.DisableFinishBootstrap()
201 defer restore()
202
200 env := &mockEnviron{203 env := &mockEnviron{
201 storage: stor,204 storage: stor,
202 startInstance: startInstance,205 startInstance: startInstance,
203206
=== modified file 'provider/ec2/local_test.go'
--- provider/ec2/local_test.go 2013-11-18 05:41:36 +0000
+++ provider/ec2/local_test.go 2013-12-03 06:15:09 +0000
@@ -222,26 +222,28 @@
222222
223 // check that the user data is configured to start zookeeper223 // check that the user data is configured to start zookeeper
224 // and the machine and provisioning agents.224 // and the machine and provisioning agents.
225 // check that the user data is configured to only configure
226 // authorized SSH keys and set the log output; everything
227 // else happens after the machine is brought up.
225 inst := t.srv.ec2srv.Instance(string(insts[0].Id()))228 inst := t.srv.ec2srv.Instance(string(insts[0].Id()))
226 c.Assert(inst, gc.NotNil)229 c.Assert(inst, gc.NotNil)
227 bootstrapDNS, err := insts[0].DNSName()230 bootstrapDNS, err := insts[0].DNSName()
228 c.Assert(err, gc.IsNil)231 c.Assert(err, gc.IsNil)
229 c.Assert(bootstrapDNS, gc.Not(gc.Equals), "")232 c.Assert(bootstrapDNS, gc.Not(gc.Equals), "")
230
231 userData, err := utils.Gunzip(inst.UserData)233 userData, err := utils.Gunzip(inst.UserData)
232 c.Assert(err, gc.IsNil)234 c.Assert(err, gc.IsNil)
233 c.Logf("first instance: UserData: %q", userData)235 c.Logf("first instance: UserData: %q", userData)
234 var userDataMap map[interface{}]interface{}236 var userDataMap map[interface{}]interface{}
235 err = goyaml.Unmarshal(userData, &userDataMap)237 err = goyaml.Unmarshal(userData, &userDataMap)
236 c.Assert(err, gc.IsNil)238 c.Assert(err, gc.IsNil)
237 CheckPackage(c, userDataMap, "git", true)239 c.Assert(userDataMap, gc.DeepEquals, map[interface{}]interface{}{
238 CheckScripts(c, userDataMap, "jujud bootstrap-state", true)240 "output": map[interface{}]interface{}{
239 // TODO check for provisioning agent241 "all": "| tee -a /var/log/cloud-init-output.log",
240 // TODO check for machine agent242 },
243 "ssh_authorized_keys": []interface{}{"my-keys"},
244 })
241245
242 // check that a new instance will be started without246 // check that a new instance will be started with a machine agent
243 // zookeeper, with a machine agent, and without a
244 // provisioning agent.
245 inst1, hc := testing.AssertStartInstance(c, env, "1")247 inst1, hc := testing.AssertStartInstance(c, env, "1")
246 c.Check(*hc.Arch, gc.Equals, "amd64")248 c.Check(*hc.Arch, gc.Equals, "amd64")
247 c.Check(*hc.Mem, gc.Equals, uint64(1740))249 c.Check(*hc.Mem, gc.Equals, uint64(1740))
@@ -255,9 +257,11 @@
255 userDataMap = nil257 userDataMap = nil
256 err = goyaml.Unmarshal(userData, &userDataMap)258 err = goyaml.Unmarshal(userData, &userDataMap)
257 c.Assert(err, gc.IsNil)259 c.Assert(err, gc.IsNil)
258 CheckPackage(c, userDataMap, "zookeeperd", false)260 CheckPackage(c, userDataMap, "git", true)
261 CheckPackage(c, userDataMap, "mongodb-server", false)
262 CheckScripts(c, userDataMap, "jujud bootstrap-state", false)
263 CheckScripts(c, userDataMap, "/var/lib/juju/agents/machine-1/agent.conf", true)
259 // TODO check for provisioning agent264 // TODO check for provisioning agent
260 // TODO check for machine agent
261265
262 err = env.Destroy()266 err = env.Destroy()
263 c.Assert(err, gc.IsNil)267 c.Assert(err, gc.IsNil)
@@ -408,7 +412,9 @@
408 ec2.UseTestInstanceTypeData(ec2.TestInstanceTypeCosts)412 ec2.UseTestInstanceTypeData(ec2.TestInstanceTypeCosts)
409 ec2.UseTestRegionData(ec2.TestRegions)413 ec2.UseTestRegionData(ec2.TestRegions)
410 restoreTimeouts := envtesting.PatchAttemptStrategies(ec2.ShortAttempt, ec2.StorageAttempt)414 restoreTimeouts := envtesting.PatchAttemptStrategies(ec2.ShortAttempt, ec2.StorageAttempt)
415 restoreFinishBootstrap := envtesting.DisableFinishBootstrap()
411 return func() {416 return func() {
417 restoreFinishBootstrap()
412 restoreTimeouts()418 restoreTimeouts()
413 ec2.UseTestImageData(nil)419 ec2.UseTestImageData(nil)
414 ec2.UseTestInstanceTypeData(nil)420 ec2.UseTestInstanceTypeData(nil)
415421
=== modified file 'provider/maas/maas_test.go'
--- provider/maas/maas_test.go 2013-11-26 12:24:48 +0000
+++ provider/maas/maas_test.go 2013-12-03 06:15:09 +0000
@@ -33,6 +33,8 @@
33 s.LoggingSuite.SetUpSuite(c)33 s.LoggingSuite.SetUpSuite(c)
34 TestMAASObject := gomaasapi.NewTestMAAS("1.0")34 TestMAASObject := gomaasapi.NewTestMAAS("1.0")
35 s.testMAASObject = TestMAASObject35 s.testMAASObject = TestMAASObject
36 restoreFinishBootstrap := envtesting.DisableFinishBootstrap()
37 s.AddSuiteCleanup(func(*gc.C) { restoreFinishBootstrap() })
36}38}
3739
38func (s *providerSuite) SetUpTest(c *gc.C) {40func (s *providerSuite) SetUpTest(c *gc.C) {
3941
=== modified file 'provider/openstack/local_test.go'
--- provider/openstack/local_test.go 2013-11-18 05:41:36 +0000
+++ provider/openstack/local_test.go 2013-12-03 06:15:09 +0000
@@ -167,6 +167,8 @@
167 s.srv.start(c, s.cred)167 s.srv.start(c, s.cred)
168 s.LiveTests.SetUpSuite(c)168 s.LiveTests.SetUpSuite(c)
169 openstack.UseTestImageData(openstack.ImageMetadataStorage(s.Env), s.cred)169 openstack.UseTestImageData(openstack.ImageMetadataStorage(s.Env), s.cred)
170 restoreFinishBootstrap := envtesting.DisableFinishBootstrap()
171 s.AddSuiteCleanup(func(*gc.C) { restoreFinishBootstrap() })
170}172}
171173
172func (s *localLiveSuite) TearDownSuite(c *gc.C) {174func (s *localLiveSuite) TearDownSuite(c *gc.C) {
@@ -203,6 +205,8 @@
203func (s *localServerSuite) SetUpSuite(c *gc.C) {205func (s *localServerSuite) SetUpSuite(c *gc.C) {
204 s.LoggingSuite.SetUpSuite(c)206 s.LoggingSuite.SetUpSuite(c)
205 s.Tests.SetUpSuite(c)207 s.Tests.SetUpSuite(c)
208 restoreFinishBootstrap := envtesting.DisableFinishBootstrap()
209 s.AddSuiteCleanup(func(*gc.C) { restoreFinishBootstrap() })
206 c.Logf("Running local tests")210 c.Logf("Running local tests")
207}211}
208212
@@ -740,6 +744,9 @@
740}744}
741745
742func (s *localHTTPSServerSuite) TestCanBootstrap(c *gc.C) {746func (s *localHTTPSServerSuite) TestCanBootstrap(c *gc.C) {
747 restoreFinishBootstrap := envtesting.DisableFinishBootstrap()
748 defer restoreFinishBootstrap()
749
743 // For testing, we create a storage instance to which is uploaded tools and image metadata.750 // For testing, we create a storage instance to which is uploaded tools and image metadata.
744 metadataStorage := openstack.MetadataStorage(s.env)751 metadataStorage := openstack.MetadataStorage(s.env)
745 url, err := metadataStorage.URL("")752 url, err := metadataStorage.URL("")
746753
=== added directory 'utils/ssh'
=== renamed file 'environs/manual/ssh.go' => 'utils/ssh/ssh.go'
--- environs/manual/ssh.go 2013-09-12 02:04:05 +0000
+++ utils/ssh/ssh.go 2013-12-03 06:15:09 +0000
@@ -1,25 +1,45 @@
1// Copyright 2013 Canonical Ltd.1// Copyright 2013 Canonical Ltd.
2// Licensed under the AGPLv3, see LICENCE file for details.2// Licensed under the AGPLv3, see LICENCE file for details.
33
4package manual4package ssh
55
6import (6import (
7 "os/exec"7 "os/exec"
8)8)
99
10type sshOption []string10type Option []string
1111
12var allocateTTY sshOption = []string{"-t"}12var (
1313 commonOptions Option = []string{"-o", "StrictHostKeyChecking no"}
14// TODO(axw) 2013-09-12 bug #122423014
15// Move this to a common package for use in cmd/juju, and others.15 // AllocateTTY forces pseudo-TTY allocation, which is required,
16var commonSSHOptions = []string{"-o", "StrictHostKeyChecking no"}16 // for example, for sudo password prompts on the target host.
1717 AllocateTTY Option = []string{"-t"}
18func sshCommand(host string, command string, options ...sshOption) *exec.Cmd {18
19 args := append([]string{}, commonSSHOptions...)19 // NoPasswordAuthentication disallows password-based authentication.
20 NoPasswordAuthentication Option = []string{"-o", "PasswordAuthentication no"}
21)
22
23// Command initialises an os/exec.Cmd to execute the native ssh program.
24func Command(host string, command []string, options ...Option) *exec.Cmd {
25 args := append([]string{}, commonOptions...)
20 for _, option := range options {26 for _, option := range options {
21 args = append(args, option...)27 args = append(args, option...)
22 }28 }
23 args = append(args, host, "--", command)29 args = append(args, host)
30 if len(command) > 0 {
31 args = append(args, "--")
32 args = append(args, command...)
33 }
24 return exec.Command("ssh", args...)34 return exec.Command("ssh", args...)
25}35}
36
37// ScpCommand initialises an os/exec.Cmd to execute the native scp program.
38func ScpCommand(source, destination string, options ...Option) *exec.Cmd {
39 args := append([]string{}, commonOptions...)
40 for _, option := range options {
41 args = append(args, option...)
42 }
43 args = append(args, source, destination)
44 return exec.Command("scp", args...)
45}

Subscribers

People subscribed via source and target branches

to status/vote changes: