Merge lp:~axwalk/juju-core/ssh-gocrypto-client into lp:~go-bot/juju-core/trunk

[Andrew Wilkins]

Introduce go.crypto/ssh support

A new "client key directory" is introduced,
which may contain one or more SSH key pairs.
The client key directory is initialised to
$JUJU_HOME/ssh in juju/conn.go. If the
directory does not exist, it is created with
a new 2048-bit RSA keypair.

If the default SSH client is not available
at bootstrap time, then a go.crypto/ssh client
will be used; this client will attempt to use
each of the key pairs it finds in the client
key directory.

The OpenSSH client (in utils/ssh/ssh_openssh.go)
is modified to use any of the default SSH
identities, or any of the ones found in the
client key directory.

Fixes #1263851

https://codereview.appspot.com/48370043/

[Andrew Wilkins (axwalk)]

Reviewers: mp+200628_code.launchpad.net,

Message:
Please take a look.

Description:
Introduce go.crypto/ssh support

A new "client key directory" is introduced,
which may contain one or more SSH key pairs.
The client key directory is initialised to
$JUJU_HOME/ssh in juju/conn.go. If the
directory does not exist, it is created with
a new 2048-bit RSA keypair.

If the default SSH client is not available
at bootstrap time, then a go.crypto/ssh client
will be used; this client will attempt to use
each of the key pairs it finds in the client
key directory.

The OpenSSH client (in utils/ssh/ssh_openssh.go)
is modified to use any of the default SSH
identities, or any of the ones found in the
client key directory.

Fixes #1263851

https://code.launchpad.net/~axwalk/juju-core/ssh-gocrypto-client/+merge/200628

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/48370043/

Affected files (+1116, -188 lines):
   A [revision details]
   M cloudinit/sshinit/configure.go
   M cmd/juju/scp.go
   M cmd/juju/scp_test.go
   M cmd/juju/ssh.go
   M cmd/juju/ssh_test.go
   M environs/config/authkeys.go
   M environs/manual/fakessh.go
   M environs/manual/init.go
   M environs/manual/init_test.go
   M environs/manual/provisioner.go
   M environs/sshstorage/storage.go
   M environs/sshstorage/storage_test.go
   M environs/testing/bootstrap.go
   M juju/conn.go
   M juju/conn_test.go
   M provider/common/bootstrap.go
   M provider/common/bootstrap_test.go
   M utils/ssh/authorisedkeys.go
   A utils/ssh/clientkeys.go
   A utils/ssh/clientkeys_test.go
   M utils/ssh/ssh.go
   A utils/ssh/ssh_gocrypto.go
   A utils/ssh/ssh_openssh.go
   M utils/ssh/ssh_test.go
   M utils/trivial.go
   M utils/trivial_test.go

[Andrew Wilkins (axwalk)]

On 2014/01/07 06:38:02, axw wrote:
> Please take a look.

I'm going to split this up, it's rather large. I would appreciate
comments on direction, though.

https://codereview.appspot.com/48370043/

[Tim Penhey (thumper)]

I think the general approach is good, but there will be issues with the
identity file that my branch has introduced.

https://codereview.appspot.com/48370043/diff/1/cloudinit/sshinit/configure.go
File cloudinit/sshinit/configure.go (right):

https://codereview.appspot.com/48370043/diff/1/cloudinit/sshinit/configure.go#newcode54
cloudinit/sshinit/configure.go:54: cmd.Stdin = strings.NewReader(script)
One thing I noticed was the need for a carriage return on the end.
Does the script end with a new line?

https://codereview.appspot.com/48370043/diff/1/environs/config/authkeys.go
File environs/config/authkeys.go (right):

https://codereview.appspot.com/48370043/diff/1/environs/config/authkeys.go#newcode40
environs/config/authkeys.go:40: var err error
Wondering if this function should perhaps be moved into utils/ssh?

https://codereview.appspot.com/48370043/diff/1/environs/manual/fakessh.go
File environs/manual/fakessh.go (right):

https://codereview.appspot.com/48370043/diff/1/environs/manual/fakessh.go#newcode33
environs/manual/fakessh.go:33: head >/dev/null
Given the number of different places we mock out the ssh executable,
wondering if we should have some standard place for things like this,
perhaps "testing/ssh" ?

https://codereview.appspot.com/48370043/diff/1/environs/sshstorage/storage.go
File environs/sshstorage/storage.go (right):

https://codereview.appspot.com/48370043/diff/1/environs/sshstorage/storage.go#newcode48
environs/sshstorage/storage.go:48: return ssh.Command(host,
[]string{"bash", "-c", command}, &options)
bash or /bin/bash ??

We have both in the codebase, and I think we should be consistent with
ourselves.

Personally I don't care too much which we choose, just that we have one.

https://codereview.appspot.com/48370043/diff/1/utils/ssh/ssh.go
File utils/ssh/ssh.go (right):

https://codereview.appspot.com/48370043/diff/1/utils/ssh/ssh.go#newcode18
utils/ssh/ssh.go:18: passwordAuthDisabled bool
hmm.. one of my branches adds an identity file option using the old way.
  This will need to be taken into account some how.

https://codereview.appspot.com/48370043/diff/1/utils/ssh/ssh_gocrypto.go
File utils/ssh/ssh_gocrypto.go (right):

https://codereview.appspot.com/48370043/diff/1/utils/ssh/ssh_gocrypto.go#newcode39
utils/ssh/ssh_gocrypto.go:39: func (c *GoCryptoClient) Command(host
string, command []string, options *Options) *Cmd {
Lots of public functions missing docs.

https://codereview.appspot.com/48370043/diff/1/utils/trivial.go
File utils/trivial.go (right):

https://codereview.appspot.com/48370043/diff/1/utils/trivial.go#newcode66
utils/trivial.go:66: // quotes as necessary; each argument is
single-quoted.
this isn't escaping slashes... is it?

https://codereview.appspot.com/48370043/

Unmerged revisions

2165. By Andrew Wilkins on 2014-01-07

Fixes, finishing touches

2164. By Andrew Wilkins on 2014-01-07

Merge with trunk

Also fix various tests.

2163. By Andrew Wilkins on 2014-01-06

utils/ssh: use ssh.GenerateKey

2162. By Andrew Wilkins on 2014-01-06

Merge with trunk

2161. By Andrew Wilkins on 2014-01-06

Missed changes

2160. By Andrew Wilkins on 2014-01-06

Auto-generate SSH client keys

At initialisation time, Juju now creates $JUJU_HOME/ssh
if it doesn't exist, and populates it with a new key pair.
This key pair will be used if the client cannot connect
with any other key pair on the system (or if there are none).

The utils/ssh package now falls back to the go.crypto/ssh
client, which will use the auto-generated key.

2159. By Andrew Wilkins on 2014-01-02

Enable bootstrapping via go.crypto/ssh

If the default SSH client is not available
at bootstrap time, then a new key pair is
generated (in memory), and go.crypto/ssh
is used to bootstrap with that. The private
key is not recorded anywhere.

2158. By Andrew Wilkins on 2013-12-23

Merge with trunk

2157. By Andrew Wilkins on 2013-12-23

Merge with trunk

2156. By Andrew Wilkins on 2013-12-23

utils/ssh: go.crypto and PuTTY clients

=== modified file 'cloudinit/sshinit/configure.go'
--- cloudinit/sshinit/configure.go	2014-01-03 05:14:11 +0000
+++ cloudinit/sshinit/configure.go	2014-01-07 06:33:34 +0000
@@ -4,7 +4,6 @@
 package sshinit
 
 import (
-	"encoding/base64"
 	"fmt"
 	"io"
 	"strings"
@@ -22,6 +21,10 @@
 	// Host is the host to configure, in the format [user@]hostname.
 	Host string
 
+	// Client is the SSH client to connect with.
+	// If Client is nil, ssh.DefaultClient will be used.
+	Client ssh.Client
+
 	// Config is the cloudinit config to carry out.
 	Config *cloudinit.Config
 
@@ -37,13 +40,18 @@
 	if err != nil {
 		return err
 	}
-	scriptBase64 := base64.StdEncoding.EncodeToString([]byte(script))
-	script = fmt.Sprintf(`F=$(mktemp); echo %s | base64 -d > $F; . $F`, scriptBase64)
-	cmd := ssh.Command(
+	var options ssh.Options
+	options.DisablePasswordAuthentication()
+	client := params.Client
+	if client == nil {
+		client = ssh.DefaultClient
+	}
+	cmd := client.Command(
 		params.Host,
-		[]string{"sudo", "-n", fmt.Sprintf("bash -c '%s'", script)},
-		ssh.NoPasswordAuthentication,
+		[]string{"sudo", "bash"},
+		&options,
 	)
+	cmd.Stdin = strings.NewReader(script)
 	cmd.Stderr = params.Stderr
 	return cmd.Run()
 }

=== modified file 'cmd/juju/scp.go'
--- cmd/juju/scp.go	2013-12-03 06:04:43 +0000
+++ cmd/juju/scp.go	2014-01-07 06:33:34 +0000
@@ -80,9 +80,7 @@
 		}
 	}
 
-	cmd := ssh.ScpCommand(c.Args[0], c.Args[1], ssh.NoPasswordAuthentication)
-	cmd.Stdin = ctx.Stdin
-	cmd.Stdout = ctx.Stdout
-	cmd.Stderr = ctx.Stderr
-	return cmd.Run()
+	var options ssh.Options
+	options.DisablePasswordAuthentication()
+	return ssh.Copy(c.Args[0], c.Args[1], &options)
 }

=== modified file 'cmd/juju/scp_test.go'
--- cmd/juju/scp_test.go	2013-12-03 14:19:47 +0000
+++ cmd/juju/scp_test.go	2014-01-07 06:33:34 +0000
@@ -6,7 +6,9 @@
 import (
 	"bytes"
 	"fmt"
+	"io/ioutil"
 	"net/url"
+	"path/filepath"
 
 	gc "launchpad.net/gocheck"
 
@@ -69,7 +71,11 @@
 		ctx := coretesting.Context(c)
 		code := cmd.Main(&SCPCommand{}, ctx, t.args)
 		c.Check(code, gc.Equals, 0)
+		// we suppress stdout from scp.
 		c.Check(ctx.Stderr.(*bytes.Buffer).String(), gc.Equals, "")
-		c.Check(ctx.Stdout.(*bytes.Buffer).String(), gc.Equals, t.result)
+		c.Check(ctx.Stdout.(*bytes.Buffer).String(), gc.Equals, "")
+		data, err := ioutil.ReadFile(filepath.Join(s.bin, "scp.args"))
+		c.Assert(err, gc.IsNil)
+		c.Assert(string(data), gc.Equals, t.result)
 	}
 }

=== modified file 'cmd/juju/ssh.go'
--- cmd/juju/ssh.go	2013-12-17 18:21:26 +0000
+++ cmd/juju/ssh.go	2014-01-07 06:33:34 +0000
@@ -88,7 +88,10 @@
 		// it from the CLI for backwards compatibility.
 		args = args[1:]
 	}
-	cmd := ssh.Command("ubuntu@"+host, args, ssh.NoPasswordAuthentication, ssh.AllocateTTY)
+	var options ssh.Options
+	options.EnablePTY()
+	options.DisablePasswordAuthentication()
+	cmd := ssh.Command("ubuntu@"+host, args, &options)
 	cmd.Stdin = ctx.Stdin
 	cmd.Stdout = ctx.Stdout
 	cmd.Stderr = ctx.Stderr

=== modified file 'cmd/juju/ssh_test.go'
--- cmd/juju/ssh_test.go	2013-12-03 14:19:47 +0000
+++ cmd/juju/ssh_test.go	2014-01-07 06:33:34 +0000
@@ -28,23 +28,21 @@
 
 type SSHCommonSuite struct {
 	testing.JujuConnSuite
-	oldpath string
+	bin string
 }
 
 // fakecommand outputs its arguments to stdout for verification
 var fakecommand = `#!/bin/bash
 
-echo $@
+echo $@ | tee $0.args
 `
 
 func (s *SSHCommonSuite) SetUpTest(c *gc.C) {
 	s.JujuConnSuite.SetUpTest(c)
-
-	path := c.MkDir()
-	s.oldpath = os.Getenv("PATH")
-	os.Setenv("PATH", path+":"+s.oldpath)
+	s.bin = c.MkDir()
+	s.PatchEnvironment("PATH", s.bin+":"+os.Getenv("PATH"))
 	for _, name := range []string{"ssh", "scp"} {
-		f, err := os.OpenFile(filepath.Join(path, name), os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0777)
+		f, err := os.OpenFile(filepath.Join(s.bin, name), os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0777)
 		c.Assert(err, gc.IsNil)
 		_, err = f.Write([]byte(fakecommand))
 		c.Assert(err, gc.IsNil)
@@ -53,11 +51,6 @@
 	}
 }
 
-func (s *SSHCommonSuite) TearDownTest(c *gc.C) {
-	os.Setenv("PATH", s.oldpath)
-	s.JujuConnSuite.TearDownTest(c)
-}
-
 const (
 	commonArgs = `-o StrictHostKeyChecking no -o PasswordAuthentication no `
 	sshArgs    = commonArgs + `-t `

=== modified file 'environs/config/authkeys.go'
--- environs/config/authkeys.go	2014-01-03 03:35:25 +0000
+++ environs/config/authkeys.go	2014-01-07 06:33:34 +0000
@@ -15,6 +15,8 @@
 
 	"launchpad.net/juju-core/cert"
 	"launchpad.net/juju-core/juju/osenv"
+	"launchpad.net/juju-core/utils"
+	"launchpad.net/juju-core/utils/ssh"
 )
 
 func expandTilde(f string) string {
@@ -35,18 +37,24 @@
 // to $HOME/.ssh.
 func ReadAuthorizedKeys(path string) (string, error) {
 	var files []string
+	var err error
 	if path == "" {
-		files = []string{"id_dsa.pub", "id_rsa.pub", "identity.pub"}
+		files, err = ssh.PublicKeyFiles()
 	} else {
-		files = []string{path}
+		path, err = utils.NormalizePath(path)
+		if err == nil {
+			if !filepath.IsAbs(path) {
+				path = filepath.Join(osenv.Home(), ".ssh", path)
+			}
+			files = append(files, path)
+		}
+	}
+	if err != nil {
+		return "", err
 	}
 	var firstError error
 	var keyData []byte
 	for _, f := range files {
-		f = expandTilde(f)
-		if !filepath.IsAbs(f) {
-			f = filepath.Join(osenv.Home(), ".ssh", f)
-		}
 		data, err := ioutil.ReadFile(f)
 		if err != nil {
 			if firstError == nil && !os.IsNotExist(err) {

=== modified file 'environs/manual/fakessh.go'
--- environs/manual/fakessh.go	2014-01-06 07:17:46 +0000
+++ environs/manual/fakessh.go	2014-01-07 06:33:34 +0000
@@ -22,11 +22,15 @@
 var sshscript = `#!/bin/bash --norc
 if [ ! -e "$0.run" ]; then
     touch "$0.run"
-    diff "$0.expected-input" -
-    exitcode=$?
-    if [ $exitcode -ne 0 ]; then
-        echo "ERROR: did not match expected input" >&2
-        exit $exitcode
+    if [ -e "$0.expected-input" ]; then
+        diff "$0.expected-input" - >&2
+        exitcode=$?
+        if [ $exitcode -ne 0 ]; then
+            echo "ERROR: did not match expected input" >&2
+            exit $exitcode
+        fi
+    else
+        head >/dev/null
     fi
     # stdout
     %s
@@ -42,14 +46,21 @@
 // updates $PATH, and returns a function to reset $PATH to
 // its original value when called.
 //
+// input may be:
+//    - nil (ignore input)
+//    - a string (match input exactly)
 // output may be:
 //    - nil (no output)
 //    - a string (stdout)
 //    - a slice of strings, of length two (stdout, stderr)
-func installFakeSSH(c *gc.C, input string, output interface{}, rc int) testbase.Restorer {
+func installFakeSSH(c *gc.C, input, output interface{}, rc int) testbase.Restorer {
 	fakebin := c.MkDir()
 	ssh := filepath.Join(fakebin, "ssh")
-	sshexpectedinput := ssh + ".expected-input"
+	if input, ok := input.(string); ok {
+		sshexpectedinput := ssh + ".expected-input"
+		err := ioutil.WriteFile(sshexpectedinput, []byte(input), 0644)
+		c.Assert(err, gc.IsNil)
+	}
 	var stdout, stderr string
 	switch output := output.(type) {
 	case nil:
@@ -62,8 +73,6 @@
 	script := fmt.Sprintf(sshscript, stdout, stderr, rc)
 	err := ioutil.WriteFile(ssh, []byte(script), 0777)
 	c.Assert(err, gc.IsNil)
-	err = ioutil.WriteFile(sshexpectedinput, []byte(input), 0644)
-	c.Assert(err, gc.IsNil)
 	return testbase.PatchEnvironment("PATH", fakebin+":"+os.Getenv("PATH"))
 }
 
@@ -121,11 +130,11 @@
 // output and exit codes.
 func (r fakeSSH) install(c *gc.C) testbase.Restorer {
 	var restore testbase.Restorer
-	add := func(input string, output interface{}, rc int) {
+	add := func(input, output interface{}, rc int) {
 		restore = restore.Add(installFakeSSH(c, input, output, rc))
 	}
 	if !r.SkipProvisionAgent {
-		add("", nil, r.ProvisionAgentExitCode)
+		add(nil, nil, r.ProvisionAgentExitCode)
 	}
 	if !r.SkipDetection {
 		restore.Add(installDetectionFakeSSH(c, r.Series, r.Arch))
@@ -134,7 +143,7 @@
 	if r.Provisioned {
 		checkProvisionedOutput = "/etc/init/jujud-machine-0.conf"
 	}
-	add("", checkProvisionedOutput, r.CheckProvisionedExitCode)
+	add(checkProvisionedScript, checkProvisionedOutput, r.CheckProvisionedExitCode)
 	if r.InitUbuntuUser {
 		add("", nil, 0)
 	}

=== modified file 'environs/manual/init.go'
--- environs/manual/init.go	2014-01-03 05:14:11 +0000
+++ environs/manual/init.go	2014-01-07 06:33:34 +0000
@@ -27,8 +27,11 @@
 // exist on the host machine.
 func checkProvisioned(host string) (bool, error) {
 	logger.Infof("Checking if %s is already provisioned", host)
-	cmd := ssh.Command("ubuntu@"+host, []string{"bash", "-c", utils.ShQuote(checkProvisionedScript)}, ssh.NoPasswordAuthentication)
+	var options ssh.Options
+	options.DisablePasswordAuthentication()
+	cmd := ssh.Command("ubuntu@"+host, []string{"bash"}, &options)
 	var stdout, stderr bytes.Buffer
+	cmd.Stdin = strings.NewReader(checkProvisionedScript)
 	cmd.Stdout = &stdout
 	cmd.Stderr = &stderr
 	if err := cmd.Run(); err != nil {
@@ -52,7 +55,9 @@
 // by connecting to the machine and executing a bash script.
 func DetectSeriesAndHardwareCharacteristics(host string) (hc instance.HardwareCharacteristics, series string, err error) {
 	logger.Infof("Detecting series and characteristics on %s", host)
-	cmd := ssh.Command("ubuntu@"+host, []string{"bash"}, ssh.NoPasswordAuthentication)
+	var options ssh.Options
+	options.DisablePasswordAuthentication()
+	cmd := ssh.Command("ubuntu@"+host, []string{"bash"}, &options)
 	var stdout, stderr bytes.Buffer
 	cmd.Stdout = &stdout
 	cmd.Stderr = &stderr
@@ -160,8 +165,11 @@
 	//
 	// Note that we explicitly do not allocate a PTY, so we
 	// get a failure if sudo prompts.
-	cmd := ssh.Command("ubuntu@"+host, []string{"sudo", "-n", "true"}, ssh.NoPasswordAuthentication)
+	var options ssh.Options
+	options.DisablePasswordAuthentication()
+	cmd := ssh.Command("ubuntu@"+host, []string{"sudo", "-n", "true"}, &options)
 	if cmd.Run() == nil {
+		logger.Infof("ubuntu user is already initialised")
 		return nil
 	}
 
@@ -171,7 +179,9 @@
 		host = login + "@" + host
 	}
 	script := fmt.Sprintf(initUbuntuScript, utils.ShQuote(authorizedKeys))
-	cmd = ssh.Command(host, []string{"sudo", "bash -c " + utils.ShQuote(script)}, ssh.AllocateTTY)
+	options = ssh.Options{} // reset options
+	options.EnablePTY()
+	cmd = ssh.Command(host, []string{"sudo", "bash -c " + utils.ShQuote(script)}, &options)
 	var stderr bytes.Buffer
 	cmd.Stdin = stdin
 	cmd.Stdout = stdout // for sudo prompt

=== modified file 'environs/manual/init_test.go'
--- environs/manual/init_test.go	2014-01-03 05:14:11 +0000
+++ environs/manual/init_test.go	2014-01-07 06:33:34 +0000
@@ -113,25 +113,25 @@
 }
 
 func (s *initialisationSuite) TestCheckProvisioned(c *gc.C) {
-	defer installFakeSSH(c, "", "", 0)()
+	defer installFakeSSH(c, checkProvisionedScript, "", 0)()
 	provisioned, err := checkProvisioned("example.com")
 	c.Assert(err, gc.IsNil)
 	c.Assert(provisioned, jc.IsFalse)
 
-	defer installFakeSSH(c, "", "non-empty", 0)()
+	defer installFakeSSH(c, checkProvisionedScript, "non-empty", 0)()
 	provisioned, err = checkProvisioned("example.com")
 	c.Assert(err, gc.IsNil)
 	c.Assert(provisioned, jc.IsTrue)
 
 	// stderr should not affect result.
-	defer installFakeSSH(c, "", []string{"", "non-empty-stderr"}, 0)()
+	defer installFakeSSH(c, checkProvisionedScript, []string{"", "non-empty-stderr"}, 0)()
 	provisioned, err = checkProvisioned("example.com")
 	c.Assert(err, gc.IsNil)
 	c.Assert(provisioned, jc.IsFalse)
 
 	// if the script fails for whatever reason, then checkProvisioned
 	// will return an error. stderr will be included in the error message.
-	defer installFakeSSH(c, "", []string{"non-empty-stdout", "non-empty-stderr"}, 255)()
+	defer installFakeSSH(c, checkProvisionedScript, []string{"non-empty-stdout", "non-empty-stderr"}, 255)()
 	_, err = checkProvisioned("example.com")
 	c.Assert(err, gc.ErrorMatches, "exit status 255 \\(non-empty-stderr\\)")
 }

=== modified file 'environs/manual/provisioner.go'
--- environs/manual/provisioner.go	2014-01-03 05:14:11 +0000
+++ environs/manual/provisioner.go	2014-01-07 06:33:34 +0000
@@ -82,9 +82,9 @@
 	}()
 
 	// Create the "ubuntu" user and initialise passwordless sudo. We populate
-	// the ubuntu user's authorized_keys file with the public keys in the current
-	// user's ~/.ssh directory. The authenticationworker will later update the
-	// ubuntu user's authorized_keys.
+	// the ubuntu user's authorized_keys file with the public keys in the
+	// current user's ~/.ssh directory. The authenticationworker will later
+	// update the ubuntu user's authorized_keys.
 	user, host := splitUserHost(args.Host)
 	authorizedKeys, err := config.ReadAuthorizedKeys("")
 	if err := InitUbuntuUser(host, user, authorizedKeys, args.Stdin, args.Stdout); err != nil {

=== modified file 'environs/sshstorage/storage.go'
--- environs/sshstorage/storage.go	2014-01-03 05:14:11 +0000
+++ environs/sshstorage/storage.go	2014-01-07 06:33:34 +0000
@@ -11,7 +11,6 @@
 	"fmt"
 	"io"
 	"io/ioutil"
-	"os/exec"
 	"path"
 	"sort"
 	"strconv"
@@ -37,14 +36,16 @@
 	remotepath string
 	tmpdir     string
 
-	cmd     *exec.Cmd
+	cmd     *ssh.Cmd
 	stdin   io.WriteCloser
 	stdout  io.ReadCloser
 	scanner *bufio.Scanner
 }
 
-var sshCommand = func(host string, command string) *exec.Cmd {
-	return ssh.Command(host, []string{command}, ssh.NoPasswordAuthentication)
+var sshCommand = func(host string, command string) *ssh.Cmd {
+	var options ssh.Options
+	options.DisablePasswordAuthentication()
+	return ssh.Command(host, []string{"bash", "-c", command}, &options)
 }
 
 type flockmode string
@@ -87,9 +88,10 @@
 		utils.ShQuote(params.TmpDir),
 	)
 
-	cmd := sshCommand(params.Host, fmt.Sprintf("sudo -n bash -c %s", utils.ShQuote(script)))
+	cmd := sshCommand(params.Host, "sudo -n bash")
 	var stderr bytes.Buffer
 	cmd.Stderr = &stderr
+	cmd.Stdin = strings.NewReader(script)
 	if err := cmd.Run(); err != nil {
 		err = fmt.Errorf("failed to create storage dir: %v (%v)", err, strings.TrimSpace(stderr.String()))
 		return nil, err

=== modified file 'environs/sshstorage/storage_test.go'
--- environs/sshstorage/storage_test.go	2014-01-03 05:14:11 +0000
+++ environs/sshstorage/storage_test.go	2014-01-07 06:33:34 +0000
@@ -22,22 +22,27 @@
 	jc "launchpad.net/juju-core/testing/checkers"
 	"launchpad.net/juju-core/testing/testbase"
 	"launchpad.net/juju-core/utils"
+	"launchpad.net/juju-core/utils/ssh"
 )
 
 type storageSuite struct {
 	testbase.LoggingSuite
+	bin string // temporary bin dir
 }
 
 var _ = gc.Suite(&storageSuite{})
 
-func sshCommandTesting(host string, command string) *exec.Cmd {
-	cmd := exec.Command("bash", "-c", command)
-	uid := fmt.Sprint(os.Getuid())
-	gid := fmt.Sprint(os.Getgid())
-	defer testbase.PatchEnvironment("SUDO_UID", uid)()
-	defer testbase.PatchEnvironment("SUDO_GID", gid)()
-	cmd.Env = os.Environ()
-	return cmd
+func (s *storageSuite) sshCommand(host string, command string) *ssh.Cmd {
+	script := []byte("#!/bin/bash\n" + command)
+	err := ioutil.WriteFile(filepath.Join(s.bin, "ssh"), script, 0755)
+	if err != nil {
+		panic(err)
+	}
+	client, err := ssh.NewOpenSSHClient()
+	if err != nil {
+		panic(err)
+	}
+	return client.Command(host, []string{command}, nil)
 }
 
 func newSSHStorage(host, storageDir, tmpDir string) (*SSHStorage, error) {
@@ -59,19 +64,22 @@
 	flockBin, err = exec.LookPath("flock")
 	c.Assert(err, gc.IsNil)
 
-	bin := c.MkDir()
-	restoreEnv := testbase.PatchEnvironment("PATH", bin+":"+os.Getenv("PATH"))
+	s.bin = c.MkDir()
+	restoreEnv := testbase.PatchEnvironment("PATH", s.bin+":"+os.Getenv("PATH"))
 	s.AddSuiteCleanup(func(*gc.C) { restoreEnv() })
 
-	// Create a "sudo" command which shifts away the "-n" and executes the remaining args.
-	err = ioutil.WriteFile(filepath.Join(bin, "sudo"), []byte("#!/bin/sh\nshift; exec \"$@\""), 0755)
+	// Create a "sudo" command which shifts away the "-n", sets
+	// SUDO_UID/SUDO_GID, and executes the remaining args.
+	err = ioutil.WriteFile(filepath.Join(s.bin, "sudo"), []byte(
+		"#!/bin/sh\nshift; export SUDO_UID=`id -u` SUDO_GID=`id -g`; exec \"$@\"",
+	), 0755)
 	c.Assert(err, gc.IsNil)
-	restoreSshCommand := testbase.PatchValue(&sshCommand, sshCommandTesting)
+	restoreSshCommand := testbase.PatchValue(&sshCommand, s.sshCommand)
 	s.AddSuiteCleanup(func(*gc.C) { restoreSshCommand() })
 
 	// Create a new "flock" which calls the original, but in non-blocking mode.
 	data := []byte(fmt.Sprintf("#!/bin/sh\nexec %s --nonblock \"$@\"", flockBin))
-	err = ioutil.WriteFile(filepath.Join(bin, "flock"), data, 0755)
+	err = ioutil.WriteFile(filepath.Join(s.bin, "flock"), data, 0755)
 	c.Assert(err, gc.IsNil)
 }
 
@@ -167,18 +175,18 @@
 	//  3: second "install"
 	//  4: touch
 	var invocations int
-	badSshCommand := func(host string, command string) *exec.Cmd {
+	badSshCommand := func(host string, command string) *ssh.Cmd {
 		invocations++
 		switch invocations {
 		case 1, 3:
-			return exec.Command("true")
+			return s.sshCommand(host, "true")
 		case 2:
 			// Note: must close stdin before responding the first time, or
 			// the second command will race with closing stdin, and may
 			// flush first.
-			return exec.Command("bash", "-c", "head -n 1 > /dev/null; exec 0<&-; echo JUJU-RC: 0; echo blah blah; echo more")
+			return s.sshCommand(host, "head -n 1 > /dev/null; exec 0<&-; echo JUJU-RC: 0; echo blah blah; echo more")
 		case 4:
-			return exec.Command("bash", "-c", `head -n 1 > /dev/null; echo "Hey it's JUJU-RC: , but not at the beginning of the line"; echo more`)
+			return s.sshCommand(host, `head -n 1 > /dev/null; echo "Hey it's JUJU-RC: , but not at the beginning of the line"; echo more`)
 		default:
 			c.Errorf("unexpected invocation: #%d, %s", invocations, command)
 			return nil

=== modified file 'environs/testing/bootstrap.go'
--- environs/testing/bootstrap.go	2013-12-20 02:38:56 +0000
+++ environs/testing/bootstrap.go	2014-01-07 06:33:34 +0000
@@ -14,6 +14,7 @@
 	"launchpad.net/juju-core/instance"
 	"launchpad.net/juju-core/provider/common"
 	"launchpad.net/juju-core/testing/testbase"
+	"launchpad.net/juju-core/utils/ssh"
 )
 
 var logger = loggo.GetLogger("juju.environs.testing")
@@ -22,7 +23,7 @@
 // do not attempt to SSH to non-existent machines. The result is a function
 // that restores finishBootstrap.
 func DisableFinishBootstrap() func() {
-	f := func(environs.BootstrapContext, instance.Instance, *cloudinit.MachineConfig) error {
+	f := func(environs.BootstrapContext, ssh.Client, instance.Instance, *cloudinit.MachineConfig) error {
 		logger.Warningf("provider/common.FinishBootstrap is disabled")
 		return nil
 	}

=== modified file 'juju/conn.go'
--- juju/conn.go	2013-12-19 21:17:26 +0000
+++ juju/conn.go	2014-01-07 06:33:34 +0000
@@ -12,7 +12,6 @@
 	"io/ioutil"
 	"net/url"
 	"os"
-	"path/filepath"
 	"time"
 
 	"launchpad.net/juju-core/charm"
@@ -21,9 +20,9 @@
 	"launchpad.net/juju-core/environs/configstore"
 	"launchpad.net/juju-core/errors"
 	"launchpad.net/juju-core/juju/osenv"
-	"launchpad.net/juju-core/log"
 	"launchpad.net/juju-core/state"
 	"launchpad.net/juju-core/utils"
+	"launchpad.net/juju-core/utils/ssh"
 )
 
 // Conn holds a connection to a juju environment and its
@@ -59,7 +58,7 @@
 	opts := state.DefaultDialOpts()
 	st, err := state.Open(info, opts)
 	if errors.IsUnauthorizedError(err) {
-		log.Noticef("juju: authorization error while connecting to state server; retrying")
+		logger.Infof("juju: authorization error while connecting to state server; retrying")
 		// We can't connect with the administrator password,;
 		// perhaps this was the first connection and the
 		// password has not been changed yet.
@@ -233,7 +232,7 @@
 		return nil, err
 	}
 	stor := conn.Environ.Storage()
-	log.Infof("writing charm to storage [%d bytes]", size)
+	logger.Infof("writing charm to storage [%d bytes]", size)
 	if err := stor.Put(name, f, size); err != nil {
 		return nil, fmt.Errorf("cannot put charm: %v", err)
 	}
@@ -245,7 +244,7 @@
 	if err != nil {
 		return nil, fmt.Errorf("cannot parse storage URL: %v", err)
 	}
-	log.Infof("adding charm to state")
+	logger.Infof("adding charm to state")
 	sch, err := conn.State.AddCharm(ch, curl, u, digest)
 	if err != nil {
 		return nil, fmt.Errorf("cannot add charm: %v", err)
@@ -253,8 +252,8 @@
 	return sch, nil
 }
 
-// InitJujuHome initializes the charm and environs/config packages to use
-// default paths based on the $JUJU_HOME or $HOME environment variables.
+// InitJujuHome initializes the charm, environs/config and utils/ssh packages
+// to use default paths based on the $JUJU_HOME or $HOME environment variables.
 // This function should be called before calling NewConn or Conn.Deploy.
 func InitJujuHome() error {
 	jujuHome := osenv.JujuHomeDir()
@@ -263,6 +262,9 @@
 			"cannot determine juju home, required environment variables are not set")
 	}
 	config.SetJujuHome(jujuHome)
-	charm.CacheDir = filepath.Join(jujuHome, "charmcache")
+	charm.CacheDir = config.JujuHomePath("charmcache")
+	if err := ssh.InitClientKeyDir(config.JujuHomePath("ssh")); err != nil {
+		return fmt.Errorf("cannot initialise ssh client key directory: %v", err)
+	}
 	return nil
 }

=== modified file 'juju/conn_test.go'
--- juju/conn_test.go	2013-12-20 09:25:57 +0000
+++ juju/conn_test.go	2014-01-07 06:33:34 +0000
@@ -657,18 +657,20 @@
 }
 
 func (s *InitJujuHomeSuite) TestJujuHome(c *gc.C) {
-	os.Setenv("JUJU_HOME", "/my/juju/home")
+	jujuHome := c.MkDir()
+	os.Setenv("JUJU_HOME", jujuHome)
 	err := juju.InitJujuHome()
 	c.Assert(err, gc.IsNil)
-	c.Assert(config.JujuHome(), gc.Equals, "/my/juju/home")
+	c.Assert(config.JujuHome(), gc.Equals, jujuHome)
 }
 
 func (s *InitJujuHomeSuite) TestHome(c *gc.C) {
+	osHome := c.MkDir()
 	os.Setenv("JUJU_HOME", "")
-	osenv.SetHome("/my/home/")
+	osenv.SetHome(osHome)
 	err := juju.InitJujuHome()
 	c.Assert(err, gc.IsNil)
-	c.Assert(config.JujuHome(), gc.Equals, "/my/home/.juju")
+	c.Assert(config.JujuHome(), gc.Equals, filepath.Join(osHome, ".juju"))
 }
 
 func (s *InitJujuHomeSuite) TestError(c *gc.C) {
@@ -679,9 +681,10 @@
 }
 
 func (s *InitJujuHomeSuite) TestCacheDir(c *gc.C) {
-	os.Setenv("JUJU_HOME", "/foo/bar")
+	jujuHome := c.MkDir()
+	os.Setenv("JUJU_HOME", jujuHome)
 	c.Assert(charm.CacheDir, gc.Equals, "")
 	err := juju.InitJujuHome()
 	c.Assert(err, gc.IsNil)
-	c.Assert(charm.CacheDir, gc.Equals, "/foo/bar/charmcache")
+	c.Assert(charm.CacheDir, gc.Equals, filepath.Join(jujuHome, "charmcache"))
 }

=== modified file 'provider/common/bootstrap.go'
--- provider/common/bootstrap.go	2014-01-06 07:38:53 +0000
+++ provider/common/bootstrap.go	2014-01-07 06:33:34 +0000
@@ -39,6 +39,15 @@
 	var inst instance.Instance
 	defer func() { handleBootstrapError(err, ctx, inst, env) }()
 
+	// Get the bootstrap SSH client. Do this early, so we know
+	// not to bother with any of the below if we can't finish the job.
+	client := ssh.DefaultClient
+	if client == nil {
+		// This should never happen: if we don't have OpenSSH, then
+		// go.crypto/ssh should be used with an auto-generated key.
+		return fmt.Errorf("no SSH client available")
+	}
+
 	// Create an empty bootstrap state file so we can get its URL.
 	// It will be updated with the instance id and hardware characteristics
 	// after the bootstrap instance is started.
@@ -78,7 +87,8 @@
 	if err != nil {
 		return fmt.Errorf("cannot save state: %v", err)
 	}
-	return FinishBootstrap(ctx, inst, machineConfig)
+
+	return FinishBootstrap(ctx, client, inst, machineConfig)
 }
 
 // GenerateSystemSSHKey creates a new key for the system identity. The
@@ -143,7 +153,7 @@
 // to the instance via SSH and carrying out the cloud-config.
 //
 // Note: FinishBootstrap is exposed so it can be replaced for testing.
-var FinishBootstrap = func(ctx environs.BootstrapContext, inst instance.Instance, machineConfig *cloudinit.MachineConfig) error {
+var FinishBootstrap = func(ctx environs.BootstrapContext, client ssh.Client, inst instance.Instance, machineConfig *cloudinit.MachineConfig) error {
 	interrupted := make(chan os.Signal, 1)
 	ctx.InterruptNotify(interrupted)
 	defer ctx.StopInterruptNotify(interrupted)
@@ -168,7 +178,7 @@
 	// TODO: jam 2013-12-04 bug #1257649
 	// It would be nice if users had some controll over their bootstrap
 	// timeout, since it is unlikely to be a perfect match for all clouds.
-	addr, err := waitSSH(ctx, interrupted, checkNonceCommand, inst, DefaultBootstrapSSHTimeout())
+	addr, err := waitSSH(ctx, interrupted, client, checkNonceCommand, inst, DefaultBootstrapSSHTimeout())
 	if err != nil {
 		return err
 	}
@@ -184,6 +194,7 @@
 	}
 	return sshinit.Configure(sshinit.ConfigureParams{
 		Host:   "ubuntu@" + addr,
+		Client: client,
 		Config: cloudcfg,
 		Stderr: ctx.Stderr(),
 	})
@@ -229,6 +240,9 @@
 type hostChecker struct {
 	addr instance.Address
 
+	// client is the ssh.Client to use to check the host.
+	client ssh.Client
+
 	// checkDelay is the amount of time to wait between retries.
 	checkDelay time.Duration
 
@@ -256,7 +270,7 @@
 	var lastErr error
 	for {
 		go func() {
-			done <- connectSSH(hc.addr.Value, hc.checkHostScript)
+			done <- connectSSH(hc.client, hc.addr.Value, hc.checkHostScript)
 		}()
 		select {
 		case <-hc.closed:
@@ -267,6 +281,7 @@
 			if lastErr == nil {
 				return hc, nil
 			}
+			logger.Errorf("failed check for %q: %v", hc.addr.Value, lastErr)
 		}
 		select {
 		case <-hc.closed:
@@ -279,6 +294,8 @@
 type parallelHostChecker struct {
 	*parallel.Try
 
+	client ssh.Client
+
 	stderr io.Writer
 
 	// active is a map of adresses to channels for addresses actively
@@ -305,6 +322,7 @@
 		hc := &hostChecker{
 			addr:            addr,
 			checkDelay:      p.checkDelay,
+			client:          p.client,
 			checkHostScript: p.checkHostScript,
 			closed:          closed,
 		}
@@ -329,10 +347,11 @@
 
 // connectSSH is called to connect to the specified host and
 // execute the "checkHostScript" bash script on it.
-var connectSSH = func(host, checkHostScript string) error {
-	cmd := ssh.Command("ubuntu@"+host, []string{
-		"/bin/bash", "-c", utils.ShQuote(checkHostScript),
-	}, ssh.NoPasswordAuthentication)
+var connectSSH = func(client ssh.Client, host, checkHostScript string) error {
+	var options ssh.Options
+	options.DisablePasswordAuthentication()
+	cmd := client.Command("ubuntu@"+host, []string{"bash"}, &options)
+	cmd.Stdin = strings.NewReader(checkHostScript)
 	output, err := cmd.CombinedOutput()
 	if err != nil && len(output) > 0 {
 		err = fmt.Errorf("%s", strings.TrimSpace(string(output)))
@@ -349,7 +368,7 @@
 // the presence of a file on the machine that contains the
 // machine's nonce. The "checkHostScript" is a bash script
 // that performs this file check.
-func waitSSH(ctx environs.BootstrapContext, interrupted <-chan os.Signal, checkHostScript string, inst addresser, timeout SSHTimeoutOpts) (addr string, err error) {
+func waitSSH(ctx environs.BootstrapContext, interrupted <-chan os.Signal, client ssh.Client, checkHostScript string, inst addresser, timeout SSHTimeoutOpts) (addr string, err error) {
 	globalTimeout := time.After(timeout.Timeout)
 	pollAddresses := time.NewTimer(0)
 
@@ -360,6 +379,7 @@
 		Try:             parallel.NewTry(0, nil),
 		stderr:          ctx.Stderr(),
 		active:          make(map[instance.Address]chan struct{}),
+		client:          client,
 		checkDelay:      timeout.ConnectDelay,
 		checkHostScript: checkHostScript,
 	}

=== modified file 'provider/common/bootstrap_test.go'
--- provider/common/bootstrap_test.go	2014-01-06 01:38:49 +0000
+++ provider/common/bootstrap_test.go	2014-01-07 06:33:34 +0000
@@ -25,6 +25,7 @@
 	jc "launchpad.net/juju-core/testing/checkers"
 	"launchpad.net/juju-core/testing/testbase"
 	"launchpad.net/juju-core/tools"
+	"launchpad.net/juju-core/utils/ssh"
 )
 
 type BootstrapSuite struct {
@@ -41,7 +42,7 @@
 func (s *BootstrapSuite) SetUpTest(c *gc.C) {
 	s.LoggingSuite.SetUpTest(c)
 	s.ToolsFixture.SetUpTest(c)
-	s.PatchValue(common.ConnectSSH, func(host, checkHostScript string) error {
+	s.PatchValue(common.ConnectSSH, func(_ ssh.Client, host, checkHostScript string) error {
 		return fmt.Errorf("mock connection failure to %s", host)
 	})
 }
@@ -268,7 +269,7 @@
 
 func (s *BootstrapSuite) TestWaitSSHTimesOutWaitingForAddresses(c *gc.C) {
 	ctx, stderr := bootstrapContext(c)
-	_, err := common.WaitSSH(ctx, nil, "/bin/true", neverAddresses{}, testSSHTimeout)
+	_, err := common.WaitSSH(ctx, nil, ssh.DefaultClient, "/bin/true", neverAddresses{}, testSSHTimeout)
 	c.Check(err, gc.ErrorMatches, `waited for `+testSSHTimeout.Timeout.String()+` without getting any addresses`)
 	c.Check(stderr.String(), gc.Matches, "Waiting for address\n")
 }
@@ -280,7 +281,7 @@
 		<-time.After(2 * time.Millisecond)
 		interrupted <- os.Interrupt
 	}()
-	_, err := common.WaitSSH(ctx, interrupted, "/bin/true", neverAddresses{}, testSSHTimeout)
+	_, err := common.WaitSSH(ctx, interrupted, ssh.DefaultClient, "/bin/true", neverAddresses{}, testSSHTimeout)
 	c.Check(err, gc.ErrorMatches, "interrupted")
 	c.Check(stderr.String(), gc.Matches, "Waiting for address\n")
 }
@@ -295,7 +296,7 @@
 
 func (s *BootstrapSuite) TestWaitSSHStopsOnBadError(c *gc.C) {
 	ctx, stderr := bootstrapContext(c)
-	_, err := common.WaitSSH(ctx, nil, "/bin/true", brokenAddresses{}, testSSHTimeout)
+	_, err := common.WaitSSH(ctx, nil, ssh.DefaultClient, "/bin/true", brokenAddresses{}, testSSHTimeout)
 	c.Check(err, gc.ErrorMatches, "getting addresses: Addresses will never work")
 	c.Check(stderr.String(), gc.Equals, "Waiting for address\n")
 }
@@ -312,7 +313,7 @@
 func (s *BootstrapSuite) TestWaitSSHTimesOutWaitingForDial(c *gc.C) {
 	ctx, stderr := bootstrapContext(c)
 	// 0.x.y.z addresses are always invalid
-	_, err := common.WaitSSH(ctx, nil, "/bin/true", &neverOpensPort{addr: "0.1.2.3"}, testSSHTimeout)
+	_, err := common.WaitSSH(ctx, nil, ssh.DefaultClient, "/bin/true", &neverOpensPort{addr: "0.1.2.3"}, testSSHTimeout)
 	c.Check(err, gc.ErrorMatches,
 		`waited for `+testSSHTimeout.Timeout.String()+` without being able to connect: mock connection failure to 0.1.2.3`)
 	c.Check(stderr.String(), gc.Matches,
@@ -342,7 +343,7 @@
 	timeout := testSSHTimeout
 	timeout.Timeout = 1 * time.Minute
 	interrupted := make(chan os.Signal, 1)
-	_, err := common.WaitSSH(ctx, interrupted, "", &interruptOnDial{name: "0.1.2.3", interrupted: interrupted}, timeout)
+	_, err := common.WaitSSH(ctx, interrupted, ssh.DefaultClient, "", &interruptOnDial{name: "0.1.2.3", interrupted: interrupted}, timeout)
 	c.Check(err, gc.ErrorMatches, "interrupted")
 	// Exact timing is imprecise but it should have tried a few times before being killed
 	c.Check(stderr.String(), gc.Matches,
@@ -371,7 +372,7 @@
 
 func (s *BootstrapSuite) TestWaitSSHRefreshAddresses(c *gc.C) {
 	ctx, stderr := bootstrapContext(c)
-	_, err := common.WaitSSH(ctx, nil, "", &addressesChange{addrs: [][]string{
+	_, err := common.WaitSSH(ctx, nil, ssh.DefaultClient, "", &addressesChange{addrs: [][]string{
 		nil,
 		nil,
 		[]string{"0.1.2.3"},

=== modified file 'utils/ssh/authorisedkeys.go'
--- utils/ssh/authorisedkeys.go	2013-12-23 05:18:58 +0000
+++ utils/ssh/authorisedkeys.go	2014-01-07 06:33:34 +0000
@@ -19,7 +19,7 @@
 	"launchpad.net/juju-core/utils"
 )
 
-var logger = loggo.GetLogger("juju.ssh")
+var logger = loggo.GetLogger("juju.utils.ssh")
 
 type ListMode bool
 

=== added file 'utils/ssh/clientkeys.go'
--- utils/ssh/clientkeys.go	1970-01-01 00:00:00 +0000
+++ utils/ssh/clientkeys.go	2014-01-07 06:33:34 +0000
@@ -0,0 +1,193 @@
+// Copyright 2014 Canonical Ltd.
+// Licensed under the AGPLv3, see LICENCE file for details.
+
+package ssh
+
+import (
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strings"
+	"sync"
+
+	"code.google.com/p/go.crypto/ssh"
+
+	"launchpad.net/juju-core/utils"
+)
+
+const clientKeyName = "juju_id_rsa"
+
+var (
+	clientKeyMutex sync.Mutex
+
+	// clientKeyDir may be set to a filesystem directory
+	// in which additional private and public keys will be
+	// located. SSH client implementations will use these
+	// keys as well as any defaults found in ~/.ssh.
+	clientKeyDir string
+
+	// clientPrivateKeys is a cached slice of ssh.Signers,
+	// which gets recomputed when clientKeyDir changes.
+	clientPrivateKeys []ssh.Signer
+)
+
+// InitClientKeyDir initialises a directory in which
+// client keys may be located; the directory is created
+// if it does not already exist, and populated with a
+// new public/private key.
+func InitClientKeyDir(dir string) error {
+	clientKeyMutex.Lock()
+	defer clientKeyMutex.Unlock()
+	if dir == "" {
+		// Primarily for testing.
+		clientKeyDir = ""
+		return nil
+	}
+	if dir == clientKeyDir {
+		return nil
+	}
+	dir, err := utils.NormalizePath(dir)
+	if err != nil {
+		return err
+	}
+	if _, err := os.Stat(dir); err == nil {
+		// If the directory exists without keys,
+		// the user must remove the directory.
+		if err = loadClientKeys(dir); err == nil {
+			clientKeyDir = dir
+		}
+		return err
+	}
+	if err := os.MkdirAll(dir, 0700); err != nil {
+		return err
+	}
+	err = generateClientKey(dir)
+	if err != nil {
+		os.RemoveAll(dir)
+		return err
+	}
+	clientKeyDir = dir
+	return nil
+}
+
+func generateClientKey(dir string) error {
+	private, public, err := GenerateKey("juju-client-key")
+	if err != nil {
+		return err
+	}
+	clientPrivateKey, err := ssh.ParsePrivateKey([]byte(private))
+	if err != nil {
+		return err
+	}
+	privkeyFilename := filepath.Join(dir, clientKeyName)
+	if err = ioutil.WriteFile(privkeyFilename, []byte(private), 0600); err != nil {
+		return err
+	}
+	if err := ioutil.WriteFile(privkeyFilename+".pub", []byte(public), 0600); err != nil {
+		os.Remove(privkeyFilename)
+		return err
+	}
+	clientPrivateKeys = []ssh.Signer{clientPrivateKey}
+	return nil
+}
+
+func loadClientKeys(dir string) error {
+	publicKeyFiles, err := clientPublicKeyFiles(dir)
+	if err != nil {
+		return err
+	}
+	privateKeyFiles := privateKeyFiles(publicKeyFiles)
+	clientPrivateKeysLocal := make([]ssh.Signer, len(privateKeyFiles))
+	for i, filename := range privateKeyFiles {
+		data, err := ioutil.ReadFile(filename)
+		if err != nil {
+			return err
+		}
+		clientPrivateKeysLocal[i], err = ssh.ParsePrivateKey(data)
+		if err != nil {
+			return fmt.Errorf("parsing key file %q: %v", filename, err)
+		}
+	}
+	clientPrivateKeys = clientPrivateKeysLocal
+	return nil
+}
+
+// getClientPrivateKeys returns the private keys in the client key dir.
+func getClientPrivateKeys() (signers []ssh.Signer) {
+	clientKeyMutex.Lock()
+	signers = append(signers, clientPrivateKeys...)
+	clientKeyMutex.Unlock()
+	return signers
+}
+
+// PrivateKeyFiles returns the filenames of private SSH keys for the
+// current user. The private keys are made up of the files with
+// a corresponding public key (i.e. the private key filename + ".pub").
+func PrivateKeyFiles() ([]string, error) {
+	pubkeys, err := PublicKeyFiles()
+	if err != nil {
+		return nil, err
+	}
+	return privateKeyFiles(pubkeys), nil
+}
+
+func privateKeyFiles(pubkeys []string) []string {
+	privkeys := make([]string, 0, len(pubkeys))
+	for _, pubkey := range pubkeys {
+		privkey := pubkey[:len(pubkey)-len(".pub")]
+		if _, err := os.Stat(privkey); err == nil {
+			privkeys = append(privkeys, privkey)
+		}
+	}
+	return privkeys
+}
+
+// PublicKeyFiles returns the filenames of the public SSH keys for the
+// current user. The public keys are made up of the files
+// ~/.ssh/{id_rsa,id_dsa,identity}.pub, as well as *.pub in the client
+// key directory initialised with InitClientKeyDir ($JUJU_HOME/ssh).
+func PublicKeyFiles() ([]string, error) {
+	var keys []string
+	sshDir, err := utils.NormalizePath("~/.ssh")
+	if err != nil {
+		return nil, err
+	}
+	for _, file := range []string{"id_dsa.pub", "id_rsa.pub", "identity.pub"} {
+		key := filepath.Join(sshDir, file)
+		if _, err := os.Stat(key); err == nil {
+			keys = append(keys, key)
+		}
+	}
+	clientKeyMutex.Lock()
+	defer clientKeyMutex.Unlock()
+	keys2, err := clientPublicKeyFiles(clientKeyDir)
+	if err != nil {
+		return nil, err
+	}
+	return append(keys, keys2...), nil
+}
+
+// clientPublicKeyFiles returns the filenames of public SSH keys
+// in the specified directory.
+func clientPublicKeyFiles(clientKeyDir string) ([]string, error) {
+	if clientKeyDir == "" {
+		return nil, nil
+	}
+	var keys []string
+	dir, err := os.Open(clientKeyDir)
+	if err != nil {
+		return nil, err
+	}
+	names, err := dir.Readdirnames(-1)
+	dir.Close()
+	if err != nil {
+		return nil, err
+	}
+	for _, name := range names {
+		if strings.HasSuffix(name, ".pub") {
+			keys = append(keys, filepath.Join(dir.Name(), name))
+		}
+	}
+	return keys, nil
+}

=== added file 'utils/ssh/clientkeys_test.go'
--- utils/ssh/clientkeys_test.go	1970-01-01 00:00:00 +0000
+++ utils/ssh/clientkeys_test.go	2014-01-07 06:33:34 +0000
@@ -0,0 +1,139 @@
+// Copyright 2013 Canonical Ltd.
+// Licensed under the AGPLv3, see LICENCE file for details.
+
+package ssh_test
+
+import (
+	"io/ioutil"
+	"os"
+	"path/filepath"
+
+	gc "launchpad.net/gocheck"
+
+	"launchpad.net/juju-core/testing"
+	jc "launchpad.net/juju-core/testing/checkers"
+	"launchpad.net/juju-core/testing/testbase"
+	"launchpad.net/juju-core/utils"
+	"launchpad.net/juju-core/utils/ssh"
+)
+
+var defaultIdentities = []string{"~/.ssh/identity", "~/.ssh/id_rsa", "~/.ssh/id_dsa"}
+
+type ClientKeysSuite struct {
+	testbase.LoggingSuite
+}
+
+var _ = gc.Suite(&ClientKeysSuite{})
+
+func (s *ClientKeysSuite) SetUpTest(c *gc.C) {
+	s.LoggingSuite.SetUpTest(c)
+	fakeHome := testing.MakeFakeHomeNoEnvironments(c)
+	s.AddCleanup(func(*gc.C) { fakeHome.Restore() })
+	s.AddCleanup(func(*gc.C) { ssh.InitClientKeyDir("") })
+
+	// fake home is created with a public key file; kill it.
+	err := os.RemoveAll(testing.HomePath(".ssh"))
+	c.Assert(err, gc.IsNil)
+	err = os.RemoveAll(testing.HomePath("juju", "ssh"))
+	c.Assert(err, gc.IsNil)
+}
+
+// createEmptyFile creates an empty file, relative to the fake home.
+func createEmptyFile(c *gc.C, file string) string {
+	file, err := utils.NormalizePath(file)
+	c.Assert(err, gc.IsNil)
+	err = os.MkdirAll(filepath.Dir(file), 0700)
+	c.Assert(err, gc.IsNil)
+	err = ioutil.WriteFile(file, nil, 0600)
+	c.Assert(err, gc.IsNil)
+	return file
+}
+
+func checkFiles(c *gc.C, obtained, expected []string) {
+	var err error
+	for i, e := range expected {
+		expected[i], err = utils.NormalizePath(e)
+		c.Assert(err, gc.IsNil)
+	}
+	c.Assert(obtained, jc.SameContents, expected)
+}
+
+func checkPublicKeyFiles(c *gc.C, expected ...string) {
+	keys, err := ssh.PublicKeyFiles()
+	c.Assert(err, gc.IsNil)
+	checkFiles(c, keys, expected)
+}
+
+func checkPrivateKeyFiles(c *gc.C, expected ...string) {
+	keys, err := ssh.PrivateKeyFiles()
+	c.Assert(err, gc.IsNil)
+	checkFiles(c, keys, expected)
+}
+
+func (s *ClientKeysSuite) TestDefaultIdentities(c *gc.C) {
+	// No .ssh dir, no .juju/ssh dir: no keys.
+	checkPrivateKeyFiles(c)
+	checkPublicKeyFiles(c)
+
+	// Only three whitelisted filenames get picked up from
+	// ~/.ssh: identity.pub, id_rsa.pub, and id_dsa.pub.
+	expectedPrivate := make([]string, len(defaultIdentities))
+	expectedPublic := make([]string, len(defaultIdentities))
+	for i, ident := range defaultIdentities {
+		expectedPrivate[i] = ident
+		expectedPublic[i] = ident + ".pub"
+		createEmptyFile(c, expectedPrivate[i])
+		createEmptyFile(c, expectedPublic[i])
+	}
+	createEmptyFile(c, "~/.ssh/identity2.pub")
+	checkPublicKeyFiles(c, expectedPublic...)
+	checkPrivateKeyFiles(c, expectedPrivate...)
+}
+
+func (s *ClientKeysSuite) TestPublicKeyFiles(c *gc.C) {
+	expected := make([]string, len(defaultIdentities))
+	for i, ident := range defaultIdentities {
+		expected[i] = ident + ".pub"
+		createEmptyFile(c, expected[i])
+	}
+	checkPublicKeyFiles(c, expected...)
+
+	// InitClientKeyDir will create the specified directory
+	// and populate it with a key pair.
+	err := ssh.InitClientKeyDir("~/.juju/ssh")
+	c.Assert(err, gc.IsNil)
+	expected = append(expected, "~/.juju/ssh/juju_id_rsa.pub")
+	checkPublicKeyFiles(c, expected...)
+
+	// All files ending with .pub in the client key dir get picked up.
+	createEmptyFile(c, "~/.juju/ssh/identity2.pub")
+	expected = append(expected, "~/.juju/ssh/identity2.pub")
+	checkPublicKeyFiles(c, expected...)
+}
+
+func (s *ClientKeysSuite) TestPrivateKeyFiles(c *gc.C) {
+	// Create various public keys. Then, go through each
+	// and create the corresponding private key, verifying
+	// that PrivateKeyFiles returns them.
+	for _, ident := range defaultIdentities {
+		createEmptyFile(c, ident+".pub")
+	}
+	err := ssh.InitClientKeyDir("~/.juju/ssh")
+	c.Assert(err, gc.IsNil)
+	err = os.Remove(testing.HomePath(".juju", "ssh", "juju_id_rsa"))
+	c.Assert(err, gc.IsNil)
+	createEmptyFile(c, "~/.juju/ssh/identity2.pub")
+
+	// No private keys yet.
+	checkPrivateKeyFiles(c)
+
+	pubkeys, err := ssh.PublicKeyFiles()
+	c.Assert(err, gc.IsNil)
+	expected := make([]string, 0, len(pubkeys))
+	for _, pubkey := range pubkeys {
+		privkey := pubkey[:len(pubkey)-len(".pub")]
+		createEmptyFile(c, privkey)
+		expected = append(expected, privkey)
+	}
+	checkPrivateKeyFiles(c, expected...)
+}

=== modified file 'utils/ssh/ssh.go'
--- utils/ssh/ssh.go	2013-12-17 08:43:06 +0000
+++ utils/ssh/ssh.go	2014-01-07 06:33:34 +0000
@@ -4,69 +4,152 @@
 // Package ssh contains utilities for dealing with SSH connections,
 // key management, and so on. All SSH-based command executions in
 // Juju should use the Command/ScpCommand functions in this package.
-//
-// TODO(axw) use PuTTY/plink if it's available on Windows.
-// TODO(axw) fallback to go.crypto/ssh if no native client is available.
 package ssh
 
 import (
-	"os"
-	"os/exec"
-)
-
-type Option []string
-
-var (
-	commonOptions Option = []string{"-o", "StrictHostKeyChecking no"}
-
-	// AllocateTTY forces pseudo-TTY allocation, which is required,
-	// for example, for sudo password prompts on the target host.
-	AllocateTTY Option = []string{"-t"}
-
-	// NoPasswordAuthentication disallows password-based authentication.
-	NoPasswordAuthentication Option = []string{"-o", "PasswordAuthentication no"}
-)
-
-// sshpassWrap wraps the command/args with sshpass if it is found in $PATH
-// and the SSHPASS environment variable is set. Otherwise, the original
-// command/args are returned.
-func sshpassWrap(cmd string, args []string) (string, []string) {
-	if os.Getenv("SSHPASS") != "" {
-		if path, err := exec.LookPath("sshpass"); err == nil {
-			return path, append([]string{"-e", cmd}, args...)
-		}
-	}
-	return cmd, args
-}
-
-// Command initialises an os/exec.Cmd to execute the native ssh program.
-//
-// If the SSHPASS environment variable is set, and the sshpass program
-// is available in $PATH, then the ssh command will be run with "sshpass -e".
-func Command(host string, command []string, options ...Option) *exec.Cmd {
-	args := append([]string{}, commonOptions...)
-	for _, option := range options {
-		args = append(args, option...)
-	}
-	args = append(args, host)
-	if len(command) > 0 {
-		args = append(args, "--")
-		args = append(args, command...)
-	}
-	bin, args := sshpassWrap("ssh", args)
-	return exec.Command(bin, args...)
-}
-
-// ScpCommand initialises an os/exec.Cmd to execute the native scp program.
-//
-// If the SSHPASS environment variable is set, and the sshpass program
-// is available in $PATH, then the scp command will be run with "sshpass -e".
-func ScpCommand(source, destination string, options ...Option) *exec.Cmd {
-	args := append([]string{}, commonOptions...)
-	for _, option := range options {
-		args = append(args, option...)
-	}
-	args = append(args, source, destination)
-	bin, args := sshpassWrap("scp", args)
-	return exec.Command(bin, args...)
+	"bytes"
+	"errors"
+	"io"
+)
+
+// Options is a client-implementation independent SSH options set.
+type Options struct {
+	allocatePTY          bool
+	passwordAuthDisabled bool
+}
+
+// EnablePTY forces the allocation of a pseudo-TTY.
+//
+// Forcing a pseudo-TTY is required, for example, for sudo
+// prompts on the target host.
+func (o *Options) EnablePTY() {
+	o.allocatePTY = true
+}
+
+// DisablePasswordAuthentication prevents the SSH
+// client from prompting the user for a password.
+func (o *Options) DisablePasswordAuthentication() {
+	o.passwordAuthDisabled = true
+}
+
+// Client is an interface for SSH clients to implement
+type Client interface {
+	// Command returns a Command for executing a command
+	// on the specified host. Each Command is executed
+	// within its own SSH session.
+	Command(host string, command []string, options *Options) *Cmd
+
+	// Copy copies a file between the local host and
+	// target host. Paths are specified in the scp format,
+	// [[user@]host:]path.
+	Copy(source, dest string, options *Options) error
+}
+
+type Cmd struct {
+	Stdin  io.Reader
+	Stdout io.Writer
+	Stderr io.Writer
+	impl   command
+}
+
+func (c *Cmd) CombinedOutput() ([]byte, error) {
+	if c.Stdout != nil {
+		return nil, errors.New("ssh: Stdout already set")
+	}
+	if c.Stderr != nil {
+		return nil, errors.New("ssh: Stderr already set")
+	}
+	var b bytes.Buffer
+	c.Stdout = &b
+	c.Stderr = &b
+	err := c.Run()
+	return b.Bytes(), err
+}
+
+func (c *Cmd) Output() ([]byte, error) {
+	if c.Stdout != nil {
+		return nil, errors.New("ssh: Stdout already set")
+	}
+	var b bytes.Buffer
+	c.Stdout = &b
+	err := c.Run()
+	return b.Bytes(), err
+}
+
+func (c *Cmd) Run() error {
+	if err := c.Start(); err != nil {
+		return err
+	}
+	return c.Wait()
+}
+
+func (c *Cmd) Start() error {
+	c.impl.SetStdio(c.Stdin, c.Stdout, c.Stderr)
+	return c.impl.Start()
+}
+
+func (c *Cmd) Wait() error {
+	return c.impl.Wait()
+}
+
+func (c *Cmd) StdinPipe() (io.WriteCloser, error) {
+	wc, r, err := c.impl.StdinPipe()
+	if err != nil {
+		return nil, err
+	}
+	c.Stdin = r
+	return wc, nil
+}
+
+func (c *Cmd) StdoutPipe() (io.ReadCloser, error) {
+	rc, w, err := c.impl.StdoutPipe()
+	if err != nil {
+		return nil, err
+	}
+	c.Stdout = w
+	return rc, nil
+}
+
+func (c *Cmd) StderrPipe() (io.ReadCloser, error) {
+	rc, w, err := c.impl.StderrPipe()
+	if err != nil {
+		return nil, err
+	}
+	c.Stderr = w
+	return rc, nil
+}
+
+// command is an implementation-specific representation of a
+// command prepared to execute against a specific host.
+type command interface {
+	Start() error
+	Wait() error
+	SetStdio(stdin io.Reader, stdout, stderr io.Writer)
+	StdinPipe() (io.WriteCloser, io.Reader, error)
+	StdoutPipe() (io.ReadCloser, io.Writer, error)
+	StderrPipe() (io.ReadCloser, io.Writer, error)
+}
+
+// DefaultClient is the default SSH client for the process.
+//
+// If OpenSSH is available, that will be used. Otherwise,
+// the embedded go.crypto/ssh client will be used. We
+// We cannot use PuTTY due to reliance on disabling strict
+// host-key checking.
+var DefaultClient Client
+
+func init() {
+	if client, err := NewOpenSSHClient(); err == nil {
+		DefaultClient = client
+	} else if client, err := NewGoCryptoClient(); err == nil {
+		DefaultClient = client
+	}
+}
+
+func Command(host string, command []string, options *Options) *Cmd {
+	return DefaultClient.Command(host, command, options)
+}
+
+func Copy(source, dest string, options *Options) error {
+	return DefaultClient.Copy(source, dest, options)
 }

=== added file 'utils/ssh/ssh_gocrypto.go'
--- utils/ssh/ssh_gocrypto.go	1970-01-01 00:00:00 +0000
+++ utils/ssh/ssh_gocrypto.go	2014-01-07 06:33:34 +0000
@@ -0,0 +1,197 @@
+// Copyright 2013 Canonical Ltd.
+// Licensed under the AGPLv3, see LICENCE file for details.
+
+package ssh
+
+import (
+	"fmt"
+	"io"
+	"io/ioutil"
+	"os/user"
+	"strings"
+
+	"code.google.com/p/go.crypto/ssh"
+
+	"launchpad.net/juju-core/utils"
+)
+
+// GoCryptoClient is an implementation of Client that
+// uses the embedded go.crypto/ssh SSH client.
+//
+// GoCryptoClient is intentionally limited in the
+// functionality that it enables, as it is currently
+// intended to be used only for non-interactive command
+// execution.
+type GoCryptoClient struct {
+	signers []ssh.Signer
+}
+
+// NewGoCryptoClient creates a new GoCryptoClient.
+//
+// If no signers are specified, NewGoCryptoClient will
+// use the private key generated by InitClientKeyDir.
+func NewGoCryptoClient(signers ...ssh.Signer) (*GoCryptoClient, error) {
+	var c GoCryptoClient
+	c.signers = signers
+	return &c, nil
+}
+
+func (c *GoCryptoClient) Command(host string, command []string, options *Options) *Cmd {
+	shellCommand := utils.CommandString(command...)
+	return &Cmd{impl: &goCryptoCommand{
+		signers: c.signers,
+		host:    host,
+		command: shellCommand,
+	}}
+}
+
+func (c *GoCryptoClient) Copy(source, dest string, options *Options) error {
+	return fmt.Errorf("Copy not implemented")
+}
+
+type goCryptoCommand struct {
+	signers []ssh.Signer
+	host    string
+	command string
+	stdin   io.Reader
+	stdout  io.Writer
+	stderr  io.Writer
+	conn    *ssh.ClientConn
+	sess    *ssh.Session
+}
+
+func (c *goCryptoCommand) ensureSession() (*ssh.Session, error) {
+	if c.sess != nil {
+		return c.sess, nil
+	}
+	username, host := splitUserHost(c.host)
+	if username == "" {
+		currentUser, err := user.Current()
+		if err != nil {
+			return nil, fmt.Errorf("getting current user: %v", err)
+		}
+		username = currentUser.Username
+	}
+	if len(c.signers) == 0 {
+		c.signers = getClientPrivateKeys()
+		if len(c.signers) == 0 {
+			return nil, fmt.Errorf("client private key not set")
+		}
+	}
+	config := &ssh.ClientConfig{
+		User: username,
+		Auth: []ssh.ClientAuth{
+			ssh.ClientAuthKeyring(keyring{c.signers}),
+		},
+	}
+	conn, err := ssh.Dial("tcp", host+":22", config)
+	if err != nil {
+		return nil, err
+	}
+	sess, err := conn.NewSession()
+	if err != nil {
+		conn.Close()
+		return nil, err
+	}
+	c.conn = conn
+	c.sess = sess
+	c.sess.Stdin = c.stdin
+	c.sess.Stdout = c.stdout
+	c.sess.Stderr = c.stderr
+	return sess, nil
+}
+
+func (c *goCryptoCommand) Start() error {
+	sess, err := c.ensureSession()
+	if err != nil {
+		return err
+	}
+	if c.command == "" {
+		return sess.Shell()
+	}
+	return sess.Start(c.command)
+}
+
+func (c *goCryptoCommand) Close() error {
+	if c.sess == nil {
+		return nil
+	}
+	err0 := c.sess.Close()
+	err1 := c.conn.Close()
+	if err0 == nil {
+		err0 = err1
+	}
+	c.sess = nil
+	c.conn = nil
+	return err0
+}
+
+func (c *goCryptoCommand) Wait() error {
+	if c.sess == nil {
+		return fmt.Errorf("Command has not been started")
+	}
+	err := c.sess.Wait()
+	c.Close()
+	return err
+}
+
+func (c *goCryptoCommand) SetStdio(stdin io.Reader, stdout, stderr io.Writer) {
+	c.stdin = stdin
+	c.stdout = stdout
+	c.stderr = stderr
+}
+
+func (c *goCryptoCommand) StdinPipe() (io.WriteCloser, io.Reader, error) {
+	sess, err := c.ensureSession()
+	if err != nil {
+		return nil, nil, err
+	}
+	wc, err := sess.StdinPipe()
+	return wc, sess.Stdin, err
+}
+
+func (c *goCryptoCommand) StdoutPipe() (io.ReadCloser, io.Writer, error) {
+	sess, err := c.ensureSession()
+	if err != nil {
+		return nil, nil, err
+	}
+	wc, err := sess.StdoutPipe()
+	return ioutil.NopCloser(wc), sess.Stdout, err
+}
+
+func (c *goCryptoCommand) StderrPipe() (io.ReadCloser, io.Writer, error) {
+	sess, err := c.ensureSession()
+	if err != nil {
+		return nil, nil, err
+	}
+	wc, err := sess.StderrPipe()
+	return ioutil.NopCloser(wc), sess.Stderr, err
+}
+
+// keyring implements ssh.ClientKeyring
+type keyring struct {
+	signers []ssh.Signer
+}
+
+func (k keyring) Key(i int) (ssh.PublicKey, error) {
+	if i < 0 || i >= len(k.signers) {
+		// nil key marks the end of the keyring; must not return an error.
+		return nil, nil
+	}
+	return k.signers[i].PublicKey(), nil
+}
+
+func (k keyring) Sign(i int, rand io.Reader, data []byte) ([]byte, error) {
+	if i < 0 || i >= len(k.signers) {
+		return nil, fmt.Errorf("no key at position %d", i)
+	}
+	return k.signers[i].Sign(rand, data)
+}
+
+func splitUserHost(s string) (user, host string) {
+	userHost := strings.SplitN(s, "@", 2)
+	if len(userHost) == 2 {
+		return userHost[0], userHost[1]
+	}
+	return "", userHost[0]
+}

=== added file 'utils/ssh/ssh_openssh.go'
--- utils/ssh/ssh_openssh.go	1970-01-01 00:00:00 +0000
+++ utils/ssh/ssh_openssh.go	2014-01-07 06:33:34 +0000
@@ -0,0 +1,180 @@
+// Copyright 2013 Canonical Ltd.
+// Licensed under the AGPLv3, see LICENCE file for details.
+
+package ssh
+
+import (
+	"bytes"
+	"fmt"
+	"io"
+	"os"
+	"os/exec"
+	"strings"
+)
+
+var opensshCommonOptions = []string{"-o", "StrictHostKeyChecking no"}
+
+// OpenSSHClient is an implementation of Client that
+// uses the ssh and scp executables found in $PATH.
+type OpenSSHClient struct{}
+
+func NewOpenSSHClient() (*OpenSSHClient, error) {
+	var c OpenSSHClient
+	if _, err := exec.LookPath("ssh"); err != nil {
+		return nil, err
+	}
+	if _, err := exec.LookPath("scp"); err != nil {
+		return nil, err
+	}
+	return &c, nil
+}
+
+// sshpassWrap wraps the command/args with sshpass if it is found in $PATH
+// and the SSHPASS environment variable is set. Otherwise, the original
+// command/args are returned.
+func sshpassWrap(cmd string, args []string) (string, []string) {
+	if os.Getenv("SSHPASS") != "" {
+		if path, err := exec.LookPath("sshpass"); err == nil {
+			return path, append([]string{"-e", cmd}, args...)
+		}
+	}
+	return cmd, args
+}
+
+func opensshOptions(options *Options) []string {
+	args := append([]string{}, opensshCommonOptions...)
+	if options == nil {
+		return args
+	}
+	if options.passwordAuthDisabled {
+		args = append(args, "-o", "PasswordAuthentication no")
+	}
+	if options.allocatePTY {
+		args = append(args, "-t")
+	}
+	return args
+}
+
+func identityArgs() ([]string, error) {
+	ids, err := PrivateKeyFiles()
+	if err != nil {
+		return nil, err
+	}
+	args := make([]string, len(ids)*2)
+	for i, id := range ids {
+		args[2*i] = "-i"
+		args[2*i+1] = id
+	}
+	return args, nil
+}
+
+func (c *OpenSSHClient) Command(host string, command []string, options *Options) *Cmd {
+	args := opensshOptions(options)
+	args = append(args, host)
+	if len(command) > 0 {
+		args = append(args, "--")
+		args = append(args, command...)
+	}
+	return &Cmd{impl: &opensshCmd{args: args}}
+}
+
+func (c *OpenSSHClient) Copy(source, dest string, options *Options) error {
+	args := opensshOptions(options)
+	idArgs, err := identityArgs()
+	if err != nil {
+		return err
+	}
+	args = append(args, idArgs...)
+	args = append(args, source, dest)
+	bin, args := sshpassWrap("scp", args)
+	cmd := exec.Command(bin, args...)
+	var stderr bytes.Buffer
+	cmd.Stderr = &stderr
+	if err := cmd.Run(); err != nil {
+		stderr := strings.TrimSpace(stderr.String())
+		if len(stderr) > 0 {
+			err = fmt.Errorf("%v (%v)", err, stderr)
+		}
+		return err
+	}
+	return nil
+}
+
+type opensshCmd struct {
+	*exec.Cmd
+
+	args   []string
+	stdin  io.Reader
+	stdout io.Writer
+	stderr io.Writer
+}
+
+func (c *opensshCmd) ensureCmd() error {
+	if c.Cmd != nil {
+		return nil
+	}
+	idArgs, err := identityArgs()
+	if err != nil {
+		return err
+	}
+	c.args = append(idArgs, c.args...)
+	bin, args := sshpassWrap("ssh", c.args)
+	c.Cmd = exec.Command(bin, args...)
+	return nil
+}
+
+func (c *opensshCmd) Start() error {
+	if err := c.ensureCmd(); err != nil {
+		return err
+	}
+	c.Cmd.Stdin = c.stdin
+	c.Cmd.Stdout = c.stdout
+	c.Cmd.Stderr = c.stderr
+	return c.Cmd.Start()
+}
+
+func (c *opensshCmd) Wait() error {
+	if err := c.ensureCmd(); err != nil {
+		return err
+	}
+	return c.Cmd.Wait()
+}
+
+func (c *opensshCmd) SetStdio(stdin io.Reader, stdout, stderr io.Writer) {
+	c.stdin = stdin
+	c.stdout = stdout
+	c.stderr = stderr
+}
+
+func (c *opensshCmd) StdinPipe() (io.WriteCloser, io.Reader, error) {
+	if err := c.ensureCmd(); err != nil {
+		return nil, nil, err
+	}
+	wc, err := c.Cmd.StdinPipe()
+	if err != nil {
+		return nil, nil, err
+	}
+	return wc, c.Stdin, nil
+}
+
+func (c *opensshCmd) StdoutPipe() (io.ReadCloser, io.Writer, error) {
+	if err := c.ensureCmd(); err != nil {
+		return nil, nil, err
+	}
+	rc, err := c.Cmd.StdoutPipe()
+	if err != nil {
+		return nil, nil, err
+	}
+	return rc, c.Stdout, nil
+}
+
+func (c *opensshCmd) StderrPipe() (io.ReadCloser, io.Writer, error) {
+	if err := c.ensureCmd(); err != nil {
+		return nil, nil, err
+	}
+	rc, err := c.Cmd.StderrPipe()
+	if err != nil {
+		return nil, nil, err
+	}
+	return rc, c.Stderr, nil
+}

=== modified file 'utils/ssh/ssh_test.go'
--- utils/ssh/ssh_test.go	2013-12-17 08:43:06 +0000
+++ utils/ssh/ssh_test.go	2014-01-07 06:33:34 +0000
@@ -7,9 +7,11 @@
 	"io/ioutil"
 	"os"
 	"path/filepath"
+	"strings"
 
 	gc "launchpad.net/gocheck"
 
+	"launchpad.net/juju-core/testing"
 	"launchpad.net/juju-core/testing/testbase"
 	"launchpad.net/juju-core/utils/ssh"
 )
@@ -18,49 +20,78 @@
 	testbase.LoggingSuite
 	testbin string
 	fakessh string
+	fakescp string
+	client  ssh.Client
 }
 
+const echoCommandScript = "#!/bin/sh\necho $0 $*"
+
 var _ = gc.Suite(&SSHCommandSuite{})
 
 func (s *SSHCommandSuite) SetUpTest(c *gc.C) {
 	s.LoggingSuite.SetUpTest(c)
+	fakeHome := testing.MakeFakeHomeNoEnvironments(c)
+	s.AddCleanup(func(*gc.C) { fakeHome.Restore() })
+	s.AddCleanup(func(*gc.C) { ssh.InitClientKeyDir("") })
 	s.testbin = c.MkDir()
 	s.fakessh = filepath.Join(s.testbin, "ssh")
-	err := ioutil.WriteFile(s.fakessh, nil, 0755)
+	s.fakescp = filepath.Join(s.testbin, "scp")
+	err := ioutil.WriteFile(s.fakessh, []byte(echoCommandScript), 0755)
+	c.Assert(err, gc.IsNil)
+	err = ioutil.WriteFile(s.fakescp, []byte(echoCommandScript), 0755)
 	c.Assert(err, gc.IsNil)
 	s.PatchEnvironment("PATH", s.testbin)
+	s.client, err = ssh.NewOpenSSHClient()
+	c.Assert(err, gc.IsNil)
 }
 
 func (s *SSHCommandSuite) TestCommand(c *gc.C) {
-	s.assertCommandArgs(c, "localhost", []string{"echo", "123"}, []string{
-		"ssh", "-o", "StrictHostKeyChecking no", "localhost", "--", "echo", "123",
-	})
+	s.assertCommandArgs(c, "localhost", []string{"echo", "123"},
+		s.fakessh+" -o StrictHostKeyChecking no localhost -- echo 123",
+	)
 }
 
-func (s *SSHCommandSuite) assertCommandArgs(c *gc.C, hostname string, command []string, expected []string) {
-	cmd := ssh.Command("localhost", []string{"echo", "123"})
+func (s *SSHCommandSuite) assertCommandArgs(c *gc.C, hostname string, command []string, expected string) {
+	cmd := s.client.Command("localhost", []string{"echo", "123"}, nil)
 	c.Assert(cmd, gc.NotNil)
-	c.Assert(cmd.Args, gc.DeepEquals, expected)
+	out, err := cmd.Output()
+	c.Assert(err, gc.IsNil)
+	c.Assert(strings.TrimSpace(string(out)), gc.Equals, expected)
 }
 
 func (s *SSHCommandSuite) TestCommandSSHPass(c *gc.C) {
 	// First create a fake sshpass, but don't set SSHPASS
 	fakesshpass := filepath.Join(s.testbin, "sshpass")
-	err := ioutil.WriteFile(fakesshpass, nil, 0755)
-	s.assertCommandArgs(c, "localhost", []string{"echo", "123"}, []string{
-		"ssh", "-o", "StrictHostKeyChecking no", "localhost", "--", "echo", "123",
-	})
+	err := ioutil.WriteFile(fakesshpass, []byte(echoCommandScript), 0755)
+	s.assertCommandArgs(c, "localhost", []string{"echo", "123"},
+		s.fakessh+" -o StrictHostKeyChecking no localhost -- echo 123",
+	)
 
 	// Now set SSHPASS.
 	s.PatchEnvironment("SSHPASS", "anyoldthing")
-	s.assertCommandArgs(c, "localhost", []string{"echo", "123"}, []string{
-		fakesshpass, "-e", "ssh", "-o", "StrictHostKeyChecking no", "localhost", "--", "echo", "123",
-	})
+	s.assertCommandArgs(c, "localhost", []string{"echo", "123"},
+		fakesshpass+" -e ssh -o StrictHostKeyChecking no localhost -- echo 123",
+	)
 
 	// Finally, remove sshpass from $PATH.
 	err = os.Remove(fakesshpass)
 	c.Assert(err, gc.IsNil)
-	s.assertCommandArgs(c, "localhost", []string{"echo", "123"}, []string{
-		"ssh", "-o", "StrictHostKeyChecking no", "localhost", "--", "echo", "123",
-	})
+	s.assertCommandArgs(c, "localhost", []string{"echo", "123"},
+		s.fakessh+" -o StrictHostKeyChecking no localhost -- echo 123",
+	)
+}
+
+func (s *SSHCommandSuite) TestCommandSSHIdentities(c *gc.C) {
+	// No keys? No -i args.
+	s.assertCommandArgs(c, "localhost", []string{"echo", "123"},
+		s.fakessh+" -o StrictHostKeyChecking no localhost -- echo 123",
+	)
+	// Otherwise, there are as many -i args as there are identity files.
+	id0 := createEmptyFile(c, "~/.ssh/id_rsa")
+	err := ssh.InitClientKeyDir("~/.juju/ssh")
+	c.Assert(err, gc.IsNil)
+	id1 := testing.HomePath(".juju", "ssh", "juju_id_rsa")
+	s.assertCommandArgs(c, "localhost", []string{"echo", "123"},
+		s.fakessh+" -i "+id0+" -i "+id1+" -o StrictHostKeyChecking no localhost -- echo 123",
+	)
 }

=== modified file 'utils/trivial.go'
--- utils/trivial.go	2013-10-10 20:58:54 +0000
+++ utils/trivial.go	2014-01-07 06:33:34 +0000
@@ -61,6 +61,16 @@
 	return `'` + strings.Replace(s, `'`, `'"'"'`, -1) + `'`
 }
 
+// CommandString flattens a sequence of command arguments into a
+// string suitable for executing in a shell, escaping slashes and
+// quotes as necessary; each argument is single-quoted.
+func CommandString(args ...string) string {
+	for i, arg := range args {
+		args[i] = "'" + strings.Replace(arg, "'", "'\\''", -1) + "'"
+	}
+	return strings.Join(args, " ")
+}
+
 // Gzip compresses the given data.
 func Gzip(data []byte) []byte {
 	var buf bytes.Buffer

=== modified file 'utils/trivial_test.go'
--- utils/trivial_test.go	2013-11-05 05:40:47 +0000
+++ utils/trivial_test.go	2014-01-07 06:33:34 +0000
@@ -47,3 +47,24 @@
 	c.Assert(err, gc.IsNil)
 	c.Assert(data1, gc.DeepEquals, data)
 }
+
+func (utilsSuite) TestCommandString(c *gc.C) {
+	type test struct {
+		args     []string
+		expected string
+	}
+	tests := []test{
+		{nil, ""},
+		{[]string{"a"}, "'a'"},
+		{[]string{"a", "'b'"}, "'a' ''\\''b'\\'''"},
+		{[]string{"a b"}, `'a b'`},
+		{[]string{"a", `"b"`}, `'a' '"b"'`},
+		{[]string{"a", `"b\"`}, `'a' '"b\"'`},
+		{[]string{"a\n"}, "'a\n'"},
+	}
+	for i, test := range tests {
+		c.Logf("test %d: %q", i, test.args)
+		result := utils.CommandString(test.args...)
+		c.Assert(result, gc.Equals, test.expected)
+	}
+}