Merge lp:~wallyworld/juju-core/improve-image-metadata-command into lp:~go-bot/juju-core/trunk

[Ian Booth]

Improve the image metadata generation command

juju metadata generate-image was originally written
as a developer tool to aid simplestreams development.
It now is useful for setting up private clouds so has
been given a facelift to make it more user friendly.
The key feature is it will now use the current environment
to pick up most of what it needs, and the user just has to
supply the image id and architecture (if something different
to amd64 is required).

https://codereview.appspot.com/14502059/

Improve the image metadata generation command

juju metadata generate-image was originally written
as a developer tool to aid simplestreams development.
It now is useful for setting up private clouds so has
been given a facelift to make it more user friendly.
The key feature is it will now use the current environment
to pick up most of what it needs, and the user just has to
supply the image id and architecture (if something different
to amd64 is required).

https://codereview.appspot.com/14502059/

[Ian Booth (wallyworld)]

Reviewers: mp+190517_code.launchpad.net,

Message:
Please take a look.

Description:
Improve the image metadata generation command

juju metadata generate-image was originally written
as a developer tool to aid simplestreams development.
It now is useful for setting up private clouds so has
been given a facelift to make it more user friendly.
The key feature is it will now use the current environment
to pick up most of what it needs, and the user just has to
supply the image id and architecture (if something different
to amd64 is required).

https://code.launchpad.net/~wallyworld/juju-core/improve-image-metadata-command/+merge/190517

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/14502059/

Affected files (+202, -133 lines):
   A [revision details]
   M cmd/plugins/juju-metadata/imagemetadata.go
   M cmd/plugins/juju-metadata/imagemetadata_test.go
   M cmd/plugins/juju-metadata/validateimagemetadata_test.go
   M environs/imagemetadata/generate.go

[Andrew Wilkins (axwalk)]

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata.go
File cmd/plugins/juju-metadata/imagemetadata.go (right):

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata.go#newcode72
cmd/plugins/juju-metadata/imagemetadata.go:72: return
fmt.Errorf("environment %q cannot provide region and endpoint",
environ.Name())
What if the user has specified -s, -r, and -e on the command line? Is
this
really an error then?

Just a thought: perhaps ImageMetadataCommand could implement HasRegion.
If environ doesn't implement HasRegion, fall back to
ImageMetadataCommand.

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata_test.go
File cmd/plugins/juju-metadata/imagemetadata_test.go (right):

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata_test.go#newcode20
cmd/plugins/juju-metadata/imagemetadata_test.go:20: "path/filepath"
Move me please

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata_test.go#newcode42
cmd/plugins/juju-metadata/imagemetadata_test.go:42: restore :=
testbase.PatchEnvironment("AWS_ACCESS_KEY_ID", "access")
You can just use s.PatchEnvironment here, which wraps
testbase.PatchEnvironment & AddCleanup.

https://codereview.appspot.com/14502059/

[Ian Booth (wallyworld)]

Please take a look.

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata.go
File cmd/plugins/juju-metadata/imagemetadata.go (right):

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata.go#newcode72
cmd/plugins/juju-metadata/imagemetadata.go:72: return
fmt.Errorf("environment %q cannot provide region and endpoint",
environ.Name())
On 2013/10/15 07:08:34, axw wrote:
> What if the user has specified -s, -r, and -e on the command line? Is
this
> really an error then?

Hmmm. I thought yes when I wrote this. But I changed it so that it works
more as expected.

> Just a thought: perhaps ImageMetadataCommand could implement
HasRegion. If
> environ doesn't implement HasRegion, fall back to
ImageMetadataCommand.

See my changes - I think it's ok as is.

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata_test.go
File cmd/plugins/juju-metadata/imagemetadata_test.go (right):

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata_test.go#newcode20
cmd/plugins/juju-metadata/imagemetadata_test.go:20: "path/filepath"
On 2013/10/15 07:08:34, axw wrote:
> Move me please

Done.

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata_test.go#newcode42
cmd/plugins/juju-metadata/imagemetadata_test.go:42: restore :=
testbase.PatchEnvironment("AWS_ACCESS_KEY_ID", "access")
On 2013/10/15 07:08:34, axw wrote:
> You can just use s.PatchEnvironment here, which wraps
testbase.PatchEnvironment
> & AddCleanup.

Done.

https://codereview.appspot.com/14502059/

[Andrew Wilkins (axwalk)]

On 2013/10/16 02:57:49, wallyworld wrote:
> Please take a look.

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata.go
> File cmd/plugins/juju-metadata/imagemetadata.go (right):

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata.go#newcode72
> cmd/plugins/juju-metadata/imagemetadata.go:72: return
fmt.Errorf("environment %q
> cannot provide region and endpoint", environ.Name())
> On 2013/10/15 07:08:34, axw wrote:
> > What if the user has specified -s, -r, and -e on the command line?
Is this
> > really an error then?
> >

> Hmmm. I thought yes when I wrote this. But I changed it so that it
works more as
> expected.

> > Just a thought: perhaps ImageMetadataCommand could implement
HasRegion. If
> > environ doesn't implement HasRegion, fall back to
ImageMetadataCommand.

> See my changes - I think it's ok as is.

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata_test.go
> File cmd/plugins/juju-metadata/imagemetadata_test.go (right):

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata_test.go#newcode20
> cmd/plugins/juju-metadata/imagemetadata_test.go:20: "path/filepath"
> On 2013/10/15 07:08:34, axw wrote:
> > Move me please

> Done.

https://codereview.appspot.com/14502059/diff/1/cmd/plugins/juju-metadata/imagemetadata_test.go#newcode42
> cmd/plugins/juju-metadata/imagemetadata_test.go:42: restore :=
> testbase.PatchEnvironment("AWS_ACCESS_KEY_ID", "access")
> On 2013/10/15 07:08:34, axw wrote:
> > You can just use s.PatchEnvironment here, which wraps
> testbase.PatchEnvironment
> > & AddCleanup.

> Done.

Thanks, LGTM

https://codereview.appspot.com/14502059/

=== modified file 'cmd/plugins/juju-metadata/imagemetadata.go'
--- cmd/plugins/juju-metadata/imagemetadata.go	2013-08-07 04:37:43 +0000
+++ cmd/plugins/juju-metadata/imagemetadata.go	2013-10-16 02:57:31 +0000
@@ -5,51 +5,102 @@
 
 import (
 	"fmt"
-	"strings"
+	"os"
+	"path/filepath"
 
 	"launchpad.net/gnuflag"
-	"launchpad.net/goose/identity"
 
 	"launchpad.net/juju-core/cmd"
+	"launchpad.net/juju-core/environs"
 	"launchpad.net/juju-core/environs/config"
+	"launchpad.net/juju-core/environs/configstore"
 	"launchpad.net/juju-core/environs/imagemetadata"
 	"launchpad.net/juju-core/environs/simplestreams"
 )
 
-// ImageMetadataCommand is used to write out a boilerplate environments.yaml file.
+// ImageMetadataCommand is used to write out simplestreams image metadata information.
 type ImageMetadataCommand struct {
-	cmd.CommandBase
-	Name     string
-	Series   string
-	Arch     string
-	ImageId  string
-	Region   string
-	Endpoint string
+	cmd.EnvCommandBase
+	Dir            string
+	Series         string
+	Arch           string
+	ImageId        string
+	Region         string
+	Endpoint       string
+	privateStorage string
 }
 
+var imageMetadataDoc = `
+generate-image creates simplestreams image metadata for the specified cloud.
+
+The cloud specification comes from the current Juju environment, as specified in
+the usual way from either ~/.juju/environments.yaml, the -e option, or JUJU_ENV.
+
+Using command arguments, it is possible to override cloud attributes region, endpoint, and series.
+By default, "amd64" is used for the architecture but this may also be changed.
+`
+
 func (c *ImageMetadataCommand) Info() *cmd.Info {
 	return &cmd.Info{
 		Name:    "generate-image",
 		Purpose: "generate simplestreams image metadata",
+		Doc:     imageMetadataDoc,
 	}
 }
 
 func (c *ImageMetadataCommand) SetFlags(f *gnuflag.FlagSet) {
-	f.StringVar(&c.Series, "s", "precise", "the charm series")
+	c.EnvCommandBase.SetFlags(f)
+	f.StringVar(&c.Series, "s", "", "the charm series")
 	f.StringVar(&c.Arch, "a", "amd64", "the image achitecture")
-	f.StringVar(&c.Name, "n", "", "the cloud name, as a prefix for the generated file names")
+	f.StringVar(&c.Dir, "d", "", "the destination directory in which to place the metadata files")
 	f.StringVar(&c.ImageId, "i", "", "the image id")
 	f.StringVar(&c.Region, "r", "", "the region")
 	f.StringVar(&c.Endpoint, "u", "", "the cloud endpoint (for Openstack, this is the Identity Service endpoint)")
 }
 
 func (c *ImageMetadataCommand) Init(args []string) error {
-	cred := identity.CredentialsFromEnv()
-	if c.Region == "" {
-		c.Region = cred.Region
-	}
-	if c.Endpoint == "" {
-		c.Endpoint = cred.URL
+	c.privateStorage = "<private storage name>"
+	var environ environs.Environ
+	if store, err := configstore.Default(); err == nil {
+		if environ, err = environs.PrepareFromName(c.EnvName, store); err == nil {
+			logger.Infof("creating image metadata for environment %q", environ.Name())
+			// If the user has not specified region and endpoint, try and get it from the environment.
+			if c.Region == "" || c.Endpoint == "" {
+				var cloudSpec simplestreams.CloudSpec
+				if inst, ok := environ.(simplestreams.HasRegion); ok {
+					if cloudSpec, err = inst.Region(); err != nil {
+						return err
+					}
+				} else {
+					return fmt.Errorf("environment %q cannot provide region and endpoint", environ.Name())
+				}
+				// If only one of region or endpoint is provided, that is a problem.
+				if cloudSpec.Region != cloudSpec.Endpoint && (cloudSpec.Region == "" || cloudSpec.Endpoint == "") {
+					return fmt.Errorf("cannot generate metadata without a complete cloud configuration")
+				}
+				if c.Region == "" {
+					c.Region = cloudSpec.Region
+				}
+				if c.Endpoint == "" {
+					c.Endpoint = cloudSpec.Endpoint
+				}
+			}
+			cfg := environ.Config()
+			if c.Series == "" {
+				c.Series = cfg.DefaultSeries()
+			}
+			if v, ok := cfg.AllAttrs()["control-bucket"]; ok {
+				c.privateStorage = v.(string)
+			}
+		} else {
+			logger.Warningf("environment %q could not be opened: %v", c.EnvName, err)
+		}
+	}
+	if environ == nil {
+		logger.Infof("no environment found, creating image metadata using user supplied data")
+	}
+	if c.Series == "" {
+		c.Series = config.DefaultSeries
 	}
 	if c.ImageId == "" {
 		return fmt.Errorf("image id must be specified")
@@ -60,10 +111,36 @@
 	if c.Endpoint == "" {
 		return fmt.Errorf("cloud endpoint URL must be specified")
 	}
+	if c.Dir == "" {
+		logger.Infof("no destination directory specified, using current directory")
+		var err error
+		if c.Dir, err = os.Getwd(); err != nil {
+			return err
+		}
+	}
 
 	return cmd.CheckEmpty(args)
 }
 
+var helpDoc = `
+image metadata files have been written to:
+%q.
+For Juju to use this metadata, the files need to be put into the
+image metadata search path. There are 2 options:
+
+1. Use tools-url in $JUJU_HOME/environments.yaml
+Configure a http server to serve the contents of
+%q
+and set the value of tools-url accordingly.
+
+2. Upload the contents of
+%q
+to your cloud's private storage (for ec2 and openstack).
+eg for openstack
+"cd %s; swift upload %s streams/v1/*"
+
+`
+
 func (c *ImageMetadataCommand) Run(context *cmd.Context) error {
 	out := context.Stdout
 
@@ -75,14 +152,11 @@
 		Region:   c.Region,
 		Endpoint: c.Endpoint,
 	}
-	files, err := imagemetadata.Boilerplate(c.Name, c.Series, &im, &cloudSpec)
+	_, err := imagemetadata.GenerateMetadata(c.Series, &im, &cloudSpec, c.Dir)
 	if err != nil {
-		return fmt.Errorf("boilerplate image metadata files could not be created: %v", err)
+		return fmt.Errorf("image metadata files could not be created: %v", err)
 	}
-	fmt.Fprintf(
-		out,
-		"Boilerplate image metadata files %q have been written to %s.\n", strings.Join(files, ", "), config.JujuHome())
-	fmt.Fprintf(out, `Copy the files to the path "streams/v1" in your cloud's public bucket.`)
-	fmt.Fprintln(out, "")
+	dest := filepath.Join(c.Dir, "streams", "v1")
+	fmt.Fprintf(out, fmt.Sprintf(helpDoc, dest, c.Dir, c.Dir, c.Dir, c.privateStorage))
 	return nil
 }

=== modified file 'cmd/plugins/juju-metadata/imagemetadata_test.go'
--- cmd/plugins/juju-metadata/imagemetadata_test.go	2013-07-30 23:18:16 +0000
+++ cmd/plugins/juju-metadata/imagemetadata_test.go	2013-10-16 02:57:31 +0000
@@ -9,26 +9,45 @@
 	"fmt"
 	"io/ioutil"
 	"os"
+	"path/filepath"
 	"strings"
 
 	gc "launchpad.net/gocheck"
 
 	"launchpad.net/juju-core/cmd"
 	"launchpad.net/juju-core/testing"
+	jc "launchpad.net/juju-core/testing/checkers"
+	"launchpad.net/juju-core/testing/testbase"
 )
 
 type ImageMetadataSuite struct {
+	testbase.LoggingSuite
 	environ []string
+	home    *testing.FakeHome
+	dir     string
 }
 
 var _ = gc.Suite(&ImageMetadataSuite{})
 
 func (s *ImageMetadataSuite) SetUpSuite(c *gc.C) {
+	s.LoggingSuite.SetUpSuite(c)
 	s.environ = os.Environ()
 }
 
 func (s *ImageMetadataSuite) SetUpTest(c *gc.C) {
+	s.LoggingSuite.SetUpTest(c)
 	os.Clearenv()
+	s.dir = c.MkDir()
+	// Create a fake certificate so azure test environment can be opened.
+	certfile, err := ioutil.TempFile(s.dir, "")
+	c.Assert(err, gc.IsNil)
+	filename := certfile.Name()
+	err = ioutil.WriteFile(filename, []byte("test certificate"), 0644)
+	c.Assert(err, gc.IsNil)
+	envConfig := strings.Replace(metadataTestEnvConfig, "/home/me/azure.pem", filename, -1)
+	s.home = testing.MakeFakeHome(c, envConfig)
+	s.PatchEnvironment("AWS_ACCESS_KEY_ID", "access")
+	s.PatchEnvironment("AWS_SECRET_ACCESS_KEY", "secret")
 }
 
 func (s *ImageMetadataSuite) TearDownTest(c *gc.C) {
@@ -36,6 +55,8 @@
 		kv := strings.SplitN(envstring, "=", 2)
 		os.Setenv(kv[0], kv[1])
 	}
+	s.home.Restore()
+	s.LoggingSuite.TearDownTest(c)
 }
 
 var seriesVersions map[string]string = map[string]string{
@@ -43,31 +64,46 @@
 	"raring":  "13.04",
 }
 
-func (*ImageMetadataSuite) assertCommandOutput(c *gc.C, errOut, series, arch, indexFileName, imageFileName string) {
+type expectedMetadata struct {
+	series   string
+	arch     string
+	region   string
+	endpoint string
+}
+
+func (s *ImageMetadataSuite) assertCommandOutput(c *gc.C, expected expectedMetadata, errOut, indexFileName, imageFileName string) {
+	if expected.region == "" {
+		expected.region = "region"
+	}
+	if expected.endpoint == "" {
+		expected.endpoint = "endpoint"
+	}
 	strippedOut := strings.Replace(errOut, "\n", "", -1)
-	c.Check(strippedOut, gc.Matches, `Boilerplate image metadata files.*have been written.*Copy the files.*`)
-	indexpath := testing.HomePath(".juju", indexFileName)
+	c.Check(strippedOut, gc.Matches, `image metadata files have been written to.*`)
+	indexpath := filepath.Join(s.dir, "streams", "v1", indexFileName)
 	data, err := ioutil.ReadFile(indexpath)
 	c.Assert(err, gc.IsNil)
+	content := string(data)
 	var indices interface{}
 	err = json.Unmarshal(data, &indices)
 	c.Assert(err, gc.IsNil)
 	c.Assert(indices.(map[string]interface{})["format"], gc.Equals, "index:1.0")
-	prodId := fmt.Sprintf("com.ubuntu.cloud:server:%s:%s", seriesVersions[series], arch)
-	c.Assert(strings.Contains(string(data), prodId), gc.Equals, true)
-	c.Assert(strings.Contains(string(data), `"region": "region"`), gc.Equals, true)
-	c.Assert(strings.Contains(string(data), `"endpoint": "endpoint"`), gc.Equals, true)
-	c.Assert(strings.Contains(string(data), fmt.Sprintf(`"path": "streams/v1/%s"`, imageFileName)), gc.Equals, true)
+	prodId := fmt.Sprintf("com.ubuntu.cloud:server:%s:%s", seriesVersions[expected.series], expected.arch)
+	c.Assert(strings.Contains(content, prodId), jc.IsTrue)
+	c.Assert(strings.Contains(content, fmt.Sprintf(`"region": %q`, expected.region)), jc.IsTrue)
+	c.Assert(strings.Contains(content, fmt.Sprintf(`"endpoint": %q`, expected.endpoint)), jc.IsTrue)
+	c.Assert(strings.Contains(content, fmt.Sprintf(`"path": "streams/v1/%s"`, imageFileName)), jc.IsTrue)
 
-	imagepath := testing.HomePath(".juju", imageFileName)
+	imagepath := filepath.Join(s.dir, "streams", "v1", imageFileName)
 	data, err = ioutil.ReadFile(imagepath)
 	c.Assert(err, gc.IsNil)
+	content = string(data)
 	var images interface{}
 	err = json.Unmarshal(data, &images)
 	c.Assert(err, gc.IsNil)
 	c.Assert(images.(map[string]interface{})["format"], gc.Equals, "products:1.0")
-	c.Assert(strings.Contains(string(data), prodId), gc.Equals, true)
-	c.Assert(strings.Contains(string(data), `"id": "1234"`), gc.Equals, true)
+	c.Assert(strings.Contains(content, prodId), gc.Equals, true)
+	c.Assert(strings.Contains(content, `"id": "1234"`), gc.Equals, true)
 }
 
 const (
@@ -76,71 +112,92 @@
 )
 
 func (s *ImageMetadataSuite) TestImageMetadataFilesNoEnv(c *gc.C) {
-	defer testing.MakeEmptyFakeHome(c).Restore()
-
-	ctx := testing.Context(c)
-	code := cmd.Main(
-		&ImageMetadataCommand{}, ctx, []string{"-i", "1234", "-r", "region", "-a", "arch", "-u", "endpoint", "-s", "raring"})
-	c.Assert(code, gc.Equals, 0)
-	errOut := ctx.Stdout.(*bytes.Buffer).String()
-	s.assertCommandOutput(c, errOut, "raring", "arch", defaultIndexFileName, defaultImageFileName)
-}
-
-func (s *ImageMetadataSuite) TestImageMetadataFilesWithName(c *gc.C) {
-	defer testing.MakeEmptyFakeHome(c).Restore()
-
-	ctx := testing.Context(c)
-	code := cmd.Main(
-		&ImageMetadataCommand{}, ctx, []string{"-n", "foo", "-i", "1234", "-r", "region", "-a", "arch", "-u", "endpoint", "-s", "raring"})
-	c.Assert(code, gc.Equals, 0)
-	errOut := ctx.Stdout.(*bytes.Buffer).String()
-	s.assertCommandOutput(c, errOut, "raring", "arch", "foo-"+defaultIndexFileName, "foo-"+defaultImageFileName)
+	ctx := testing.Context(c)
+	code := cmd.Main(
+		&ImageMetadataCommand{}, ctx, []string{
+			"-d", s.dir, "-i", "1234", "-r", "region", "-a", "arch", "-u", "endpoint", "-s", "raring"})
+	c.Assert(code, gc.Equals, 0)
+	errOut := ctx.Stdout.(*bytes.Buffer).String()
+	expected := expectedMetadata{
+		series: "raring",
+		arch:   "arch",
+	}
+	s.assertCommandOutput(c, expected, errOut, defaultIndexFileName, defaultImageFileName)
 }
 
 func (s *ImageMetadataSuite) TestImageMetadataFilesDefaultArch(c *gc.C) {
-	defer testing.MakeEmptyFakeHome(c).Restore()
-
 	ctx := testing.Context(c)
 	code := cmd.Main(
-		&ImageMetadataCommand{}, ctx, []string{"-i", "1234", "-r", "region", "-u", "endpoint", "-s", "raring"})
+		&ImageMetadataCommand{}, ctx, []string{
+			"-d", s.dir, "-i", "1234", "-r", "region", "-u", "endpoint", "-s", "raring"})
 	c.Assert(code, gc.Equals, 0)
 	errOut := ctx.Stdout.(*bytes.Buffer).String()
-	s.assertCommandOutput(c, errOut, "raring", "amd64", defaultIndexFileName, defaultImageFileName)
+	expected := expectedMetadata{
+		series: "raring",
+		arch:   "amd64",
+	}
+	s.assertCommandOutput(c, expected, errOut, defaultIndexFileName, defaultImageFileName)
 }
 
 func (s *ImageMetadataSuite) TestImageMetadataFilesDefaultSeries(c *gc.C) {
-	defer testing.MakeEmptyFakeHome(c).Restore()
-
-	ctx := testing.Context(c)
-	code := cmd.Main(
-		&ImageMetadataCommand{}, ctx, []string{"-i", "1234", "-r", "region", "-a", "arch", "-u", "endpoint"})
-	c.Assert(code, gc.Equals, 0)
-	errOut := ctx.Stdout.(*bytes.Buffer).String()
-	s.assertCommandOutput(c, errOut, "precise", "arch", defaultIndexFileName, defaultImageFileName)
-}
-
-func (s *ImageMetadataSuite) TestImageMetadataFilesUsingEnvRegion(c *gc.C) {
-	defer testing.MakeEmptyFakeHome(c).Restore()
-
-	os.Setenv("OS_REGION_NAME", "region")
-	ctx := testing.Context(c)
-	code := cmd.Main(
-		&ImageMetadataCommand{}, ctx, []string{"-i", "1234", "-u", "endpoint"})
-	c.Assert(code, gc.Equals, 0)
-	errOut := ctx.Stdout.(*bytes.Buffer).String()
-	s.assertCommandOutput(c, errOut, "precise", "amd64", defaultIndexFileName, defaultImageFileName)
-}
-
-func (s *ImageMetadataSuite) TestImageMetadataFilesUsingEnvEndpoint(c *gc.C) {
-	defer testing.MakeEmptyFakeHome(c).Restore()
-
-	os.Setenv("OS_AUTH_URL", "endpoint")
-	ctx := testing.Context(c)
-	code := cmd.Main(
-		&ImageMetadataCommand{}, ctx, []string{"-i", "1234", "-r", "region"})
-	c.Assert(code, gc.Equals, 0)
-	errOut := ctx.Stdout.(*bytes.Buffer).String()
-	s.assertCommandOutput(c, errOut, "precise", "amd64", defaultIndexFileName, defaultImageFileName)
+	ctx := testing.Context(c)
+	code := cmd.Main(
+		&ImageMetadataCommand{}, ctx, []string{
+			"-d", s.dir, "-i", "1234", "-r", "region", "-a", "arch", "-u", "endpoint"})
+	c.Assert(code, gc.Equals, 0)
+	errOut := ctx.Stdout.(*bytes.Buffer).String()
+	expected := expectedMetadata{
+		series: "precise",
+		arch:   "arch",
+	}
+	s.assertCommandOutput(c, expected, errOut, defaultIndexFileName, defaultImageFileName)
+}
+
+func (s *ImageMetadataSuite) TestImageMetadataFilesUsingEnv(c *gc.C) {
+	ctx := testing.Context(c)
+	code := cmd.Main(
+		&ImageMetadataCommand{}, ctx, []string{"-d", s.dir, "-e", "ec2", "-i", "1234"})
+	c.Assert(code, gc.Equals, 0)
+	errOut := ctx.Stdout.(*bytes.Buffer).String()
+	expected := expectedMetadata{
+		series:   "precise",
+		arch:     "amd64",
+		region:   "us-east-1",
+		endpoint: "https://ec2.us-east-1.amazonaws.com",
+	}
+	s.assertCommandOutput(c, expected, errOut, defaultIndexFileName, defaultImageFileName)
+}
+
+func (s *ImageMetadataSuite) TestImageMetadataFilesUsingEnvWithRegionOverride(c *gc.C) {
+	ctx := testing.Context(c)
+	code := cmd.Main(
+		&ImageMetadataCommand{}, ctx, []string{
+			"-d", s.dir, "-e", "ec2", "-r", "us-west-1", "-u", "https://ec2.us-west-1.amazonaws.com", "-i", "1234"})
+	c.Assert(code, gc.Equals, 0)
+	errOut := ctx.Stdout.(*bytes.Buffer).String()
+	expected := expectedMetadata{
+		series:   "precise",
+		arch:     "amd64",
+		region:   "us-west-1",
+		endpoint: "https://ec2.us-west-1.amazonaws.com",
+	}
+	s.assertCommandOutput(c, expected, errOut, defaultIndexFileName, defaultImageFileName)
+}
+
+func (s *ImageMetadataSuite) TestImageMetadataFilesUsingEnvWithNoHasRegion(c *gc.C) {
+	ctx := testing.Context(c)
+	code := cmd.Main(
+		&ImageMetadataCommand{}, ctx, []string{
+			"-d", s.dir, "-e", "azure", "-r", "region", "-u", "endpoint", "-i", "1234"})
+	c.Assert(code, gc.Equals, 0)
+	errOut := ctx.Stdout.(*bytes.Buffer).String()
+	expected := expectedMetadata{
+		series:   "raring",
+		arch:     "amd64",
+		region:   "region",
+		endpoint: "endpoint",
+	}
+	s.assertCommandOutput(c, expected, errOut, defaultIndexFileName, defaultImageFileName)
 }
 
 type errTestParams struct {
@@ -160,6 +217,10 @@
 		// Missing endpoint
 		args: []string{"-i", "1234", "-u", "endpoint", "-a", "arch", "-s", "precise"},
 	},
+	{
+		// Missing endpoint/region for environment with no HasRegion interface
+		args: []string{"-i", "1234", "-e", "azure"},
+	},
 }
 
 func (s *ImageMetadataSuite) TestImageMetadataBadArgs(c *gc.C) {

=== modified file 'cmd/plugins/juju-metadata/validateimagemetadata_test.go'
--- cmd/plugins/juju-metadata/validateimagemetadata_test.go	2013-10-10 11:40:54 +0000
+++ cmd/plugins/juju-metadata/validateimagemetadata_test.go	2013-10-16 02:57:31 +0000
@@ -11,7 +11,6 @@
 	gc "launchpad.net/gocheck"
 
 	"launchpad.net/juju-core/cmd"
-	"launchpad.net/juju-core/environs/config"
 	"launchpad.net/juju-core/environs/imagemetadata"
 	"launchpad.net/juju-core/environs/simplestreams"
 	coretesting "launchpad.net/juju-core/testing"
@@ -20,7 +19,8 @@
 
 type ValidateImageMetadataSuite struct {
 	testbase.LoggingSuite
-	home *coretesting.FakeHome
+	home        *coretesting.FakeHome
+	metadataDir string
 }
 
 var _ = gc.Suite(&ValidateImageMetadataSuite{})
@@ -73,7 +73,7 @@
 		Region:   region,
 		Endpoint: endpoint,
 	}
-	_, err := imagemetadata.MakeBoilerplate("", series, &im, &cloudSpec, false)
+	_, err := imagemetadata.GenerateMetadata(series, &im, &cloudSpec, s.metadataDir)
 	if err != nil {
 		return err
 	}
@@ -84,13 +84,21 @@
 environments:
     ec2:
         type: ec2
-        control-bucket: foo
         default-series: precise
         region: us-east-1
+
+    azure:
+        type: azure
+        default-series: raring
+        location: US West
+        management-subscription-id: foo
+        storage-account-name: bar
+        management-certificate-path: /home/me/azure.pem
 `
 
 func (s *ValidateImageMetadataSuite) SetUpTest(c *gc.C) {
 	s.LoggingSuite.SetUpTest(c)
+	s.metadataDir = c.MkDir()
 	s.home = coretesting.MakeFakeHome(c, metadataTestEnvConfig)
 	restore := testbase.PatchEnvironment("AWS_ACCESS_KEY_ID", "access")
 	s.AddCleanup(func(*gc.C) { restore() })
@@ -115,9 +123,8 @@
 func (s *ValidateImageMetadataSuite) TestEc2LocalMetadataUsingEnvironment(c *gc.C) {
 	s.setupEc2LocalMetadata(c, "us-east-1")
 	ctx := coretesting.Context(c)
-	metadataDir := config.JujuHomePath("")
 	code := cmd.Main(
-		&ValidateImageMetadataCommand{}, ctx, []string{"-e", "ec2", "-d", metadataDir},
+		&ValidateImageMetadataCommand{}, ctx, []string{"-e", "ec2", "-d", s.metadataDir},
 	)
 	c.Assert(code, gc.Equals, 0)
 	errOut := ctx.Stdout.(*bytes.Buffer).String()
@@ -128,11 +135,12 @@
 func (s *ValidateImageMetadataSuite) TestEc2LocalMetadataUsingIncompleteEnvironment(c *gc.C) {
 	testbase.PatchEnvironment("AWS_ACCESS_KEY_ID", "")
 	testbase.PatchEnvironment("AWS_SECRET_ACCESS_KEY", "")
+	testbase.PatchEnvironment("EC2_ACCESS_KEY", "")
+	testbase.PatchEnvironment("EC2_SECRET_KEY", "")
 	s.setupEc2LocalMetadata(c, "us-east-1")
 	ctx := coretesting.Context(c)
-	metadataDir := config.JujuHomePath("")
 	code := cmd.Main(
-		&ValidateImageMetadataCommand{}, ctx, []string{"-e", "ec2", "-d", metadataDir},
+		&ValidateImageMetadataCommand{}, ctx, []string{"-e", "ec2", "-d", s.metadataDir},
 	)
 	c.Assert(code, gc.Equals, 1)
 	errOut := ctx.Stderr.(*bytes.Buffer).String()
@@ -143,11 +151,10 @@
 func (s *ValidateImageMetadataSuite) TestEc2LocalMetadataWithManualParams(c *gc.C) {
 	s.setupEc2LocalMetadata(c, "us-west-1")
 	ctx := coretesting.Context(c)
-	metadataDir := config.JujuHomePath("")
 	code := cmd.Main(
 		&ValidateImageMetadataCommand{}, ctx, []string{
 			"-p", "ec2", "-s", "precise", "-r", "us-west-1",
-			"-u", "https://ec2.us-west-1.amazonaws.com", "-d", metadataDir},
+			"-u", "https://ec2.us-west-1.amazonaws.com", "-d", s.metadataDir},
 	)
 	c.Assert(code, gc.Equals, 0)
 	errOut := ctx.Stdout.(*bytes.Buffer).String()
@@ -158,17 +165,16 @@
 func (s *ValidateImageMetadataSuite) TestEc2LocalMetadataNoMatch(c *gc.C) {
 	s.setupEc2LocalMetadata(c, "us-east-1")
 	ctx := coretesting.Context(c)
-	metadataDir := config.JujuHomePath("")
 	code := cmd.Main(
 		&ValidateImageMetadataCommand{}, ctx, []string{
 			"-p", "ec2", "-s", "raring", "-r", "us-west-1",
-			"-u", "https://ec2.us-west-1.amazonaws.com", "-d", metadataDir},
+			"-u", "https://ec2.us-west-1.amazonaws.com", "-d", s.metadataDir},
 	)
 	c.Assert(code, gc.Equals, 1)
 	code = cmd.Main(
 		&ValidateImageMetadataCommand{}, ctx, []string{
 			"-p", "ec2", "-s", "precise", "-r", "region",
-			"-u", "https://ec2.region.amazonaws.com", "-d", metadataDir},
+			"-u", "https://ec2.region.amazonaws.com", "-d", s.metadataDir},
 	)
 	c.Assert(code, gc.Equals, 1)
 }
@@ -176,11 +182,10 @@
 func (s *ValidateImageMetadataSuite) TestOpenstackLocalMetadataWithManualParams(c *gc.C) {
 	s.makeLocalMetadata(c, "1234", "region-2", "raring", "some-auth-url")
 	ctx := coretesting.Context(c)
-	metadataDir := config.JujuHomePath("")
 	code := cmd.Main(
 		&ValidateImageMetadataCommand{}, ctx, []string{
 			"-p", "openstack", "-s", "raring", "-r", "region-2",
-			"-u", "some-auth-url", "-d", metadataDir},
+			"-u", "some-auth-url", "-d", s.metadataDir},
 	)
 	c.Assert(code, gc.Equals, 0)
 	errOut := ctx.Stdout.(*bytes.Buffer).String()
@@ -191,17 +196,16 @@
 func (s *ValidateImageMetadataSuite) TestOpenstackLocalMetadataNoMatch(c *gc.C) {
 	s.makeLocalMetadata(c, "1234", "region-2", "raring", "some-auth-url")
 	ctx := coretesting.Context(c)
-	metadataDir := config.JujuHomePath("")
 	code := cmd.Main(
 		&ValidateImageMetadataCommand{}, ctx, []string{
 			"-p", "openstack", "-s", "precise", "-r", "region-2",
-			"-u", "some-auth-url", "-d", metadataDir},
+			"-u", "some-auth-url", "-d", s.metadataDir},
 	)
 	c.Assert(code, gc.Equals, 1)
 	code = cmd.Main(
 		&ValidateImageMetadataCommand{}, ctx, []string{
 			"-p", "openstack", "-s", "raring", "-r", "region-3",
-			"-u", "some-auth-url", "-d", metadataDir},
+			"-u", "some-auth-url", "-d", s.metadataDir},
 	)
 	c.Assert(code, gc.Equals, 1)
 }

=== renamed file 'environs/imagemetadata/boilerplate.go' => 'environs/imagemetadata/generate.go'
--- environs/imagemetadata/boilerplate.go	2013-08-07 04:34:02 +0000
+++ environs/imagemetadata/generate.go	2013-10-16 02:57:31 +0000
@@ -12,38 +12,25 @@
 	"text/template"
 	"time"
 
-	"launchpad.net/juju-core/environs/config"
 	"launchpad.net/juju-core/environs/simplestreams"
 )
 
 const (
 	defaultIndexFileName = "index.json"
 	defaultImageFileName = "imagemetadata.json"
-	streamsDir           = "streams/v1"
 )
 
-// Boilerplate generates some basic simplestreams metadata using the specified cloud and image details.
-// If name is non-empty, it will be used as a prefix for the names of the generated index and image files.
-func Boilerplate(name, series string, im *ImageMetadata, cloudSpec *simplestreams.CloudSpec) ([]string, error) {
-	return MakeBoilerplate(name, series, im, cloudSpec, true)
-}
-
-// MakeBoilerplate exists so it can be called by tests. See Boilerplate above. It provides an option to retain
-// the streams directories when writing the generated metadata files.
-func MakeBoilerplate(name, series string, im *ImageMetadata, cloudSpec *simplestreams.CloudSpec, flattenPath bool) ([]string, error) {
+// GenerateMetadata generates some basic simplestreams metadata using the specified cloud and image details.
+func GenerateMetadata(series string, im *ImageMetadata, cloudSpec *simplestreams.CloudSpec, dest string) ([]string, error) {
 	indexFileName := defaultIndexFileName
 	imageFileName := defaultImageFileName
-	if name != "" {
-		indexFileName = fmt.Sprintf("%s-%s", name, indexFileName)
-		imageFileName = fmt.Sprintf("%s-%s", name, imageFileName)
-	}
 	now := time.Now()
 	imparams := imageMetadataParams{
 		Id:            im.Id,
 		Arch:          im.Arch,
 		Region:        cloudSpec.Region,
 		URL:           cloudSpec.Endpoint,
-		Path:          streamsDir,
+		Path:          "streams/v1",
 		ImageFileName: imageFileName,
 		Updated:       now.Format(time.RFC1123Z),
 		VersionKey:    now.Format("20060102"),
@@ -55,14 +42,12 @@
 		return nil, fmt.Errorf("invalid series %q", series)
 	}
 
-	if !flattenPath {
-		streamsPath := config.JujuHomePath(streamsDir)
-		if err = os.MkdirAll(streamsPath, 0755); err != nil {
-			return nil, err
-		}
-		indexFileName = filepath.Join(streamsDir, indexFileName)
-		imageFileName = filepath.Join(streamsDir, imageFileName)
+	streamsPath := filepath.Join(dest, "streams", "v1")
+	if err = os.MkdirAll(streamsPath, 0755); err != nil {
+		return nil, err
 	}
+	indexFileName = filepath.Join(streamsPath, indexFileName)
+	imageFileName = filepath.Join(streamsPath, imageFileName)
 	err = writeJsonFile(imparams, indexFileName, indexBoilerplate)
 	if err != nil {
 		return nil, err
@@ -94,8 +79,7 @@
 		panic(fmt.Errorf("cannot generate %s metdata: %v", filename, err))
 	}
 	data := metadata.Bytes()
-	path := config.JujuHomePath(filename)
-	if err := ioutil.WriteFile(path, data, 0666); err != nil {
+	if err := ioutil.WriteFile(filename, data, 0666); err != nil {
 		return err
 	}
 	return nil

=== modified file 'environs/imagemetadata/validation_test.go'
--- environs/imagemetadata/validation_test.go	2013-09-24 08:01:52 +0000
+++ environs/imagemetadata/validation_test.go	2013-10-16 02:57:31 +0000
@@ -6,16 +6,14 @@
 import (
 	gc "launchpad.net/gocheck"
 
-	"launchpad.net/juju-core/environs/config"
 	"launchpad.net/juju-core/environs/imagemetadata"
 	"launchpad.net/juju-core/environs/simplestreams"
-	coretesting "launchpad.net/juju-core/testing"
 	"launchpad.net/juju-core/testing/testbase"
 )
 
 type ValidateSuite struct {
 	testbase.LoggingSuite
-	home *coretesting.FakeHome
+	metadataDir string
 }
 
 var _ = gc.Suite(&ValidateSuite{})
@@ -29,7 +27,7 @@
 		Region:   region,
 		Endpoint: endpoint,
 	}
-	_, err := imagemetadata.MakeBoilerplate("", series, &im, &cloudSpec, false)
+	_, err := imagemetadata.GenerateMetadata(series, &im, &cloudSpec, s.metadataDir)
 	if err != nil {
 		return err
 	}
@@ -38,23 +36,17 @@
 
 func (s *ValidateSuite) SetUpTest(c *gc.C) {
 	s.LoggingSuite.SetUpTest(c)
-	s.home = coretesting.MakeEmptyFakeHome(c)
-}
-
-func (s *ValidateSuite) TearDownTest(c *gc.C) {
-	s.home.Restore()
-	s.LoggingSuite.TearDownTest(c)
+	s.metadataDir = c.MkDir()
 }
 
 func (s *ValidateSuite) TestMatch(c *gc.C) {
 	s.makeLocalMetadata(c, "1234", "region-2", "raring", "some-auth-url")
-	metadataDir := config.JujuHomePath("")
 	params := &simplestreams.MetadataLookupParams{
 		Region:        "region-2",
 		Series:        "raring",
 		Architectures: []string{"amd64"},
 		Endpoint:      "some-auth-url",
-		Sources:       []simplestreams.DataSource{simplestreams.NewURLDataSource("file://"+metadataDir, simplestreams.VerifySSLHostnames)},
+		Sources:       []simplestreams.DataSource{simplestreams.NewURLDataSource("file://"+s.metadataDir, simplestreams.VerifySSLHostnames)},
 	}
 	imageIds, err := imagemetadata.ValidateImageMetadata(params)
 	c.Assert(err, gc.IsNil)
@@ -63,13 +55,12 @@
 
 func (s *ValidateSuite) TestNoMatch(c *gc.C) {
 	s.makeLocalMetadata(c, "1234", "region-2", "raring", "some-auth-url")
-	metadataDir := config.JujuHomePath("")
 	params := &simplestreams.MetadataLookupParams{
 		Region:        "region-2",
 		Series:        "precise",
 		Architectures: []string{"amd64"},
 		Endpoint:      "some-auth-url",
-		Sources:       []simplestreams.DataSource{simplestreams.NewURLDataSource("file://"+metadataDir, simplestreams.VerifySSLHostnames)},
+		Sources:       []simplestreams.DataSource{simplestreams.NewURLDataSource("file://"+s.metadataDir, simplestreams.VerifySSLHostnames)},
 	}
 	_, err := imagemetadata.ValidateImageMetadata(params)
 	c.Assert(err, gc.Not(gc.IsNil))