Merge lp:~maas-maintainers/juju-core/maas-provider-skeleton into lp:~juju/juju-core/trunk

Proposed by Jeroen T. Vermeulen
Status: Merged
Approved by: Jeroen T. Vermeulen
Approved revision: no longer in the source branch.
Merged at revision: 1150
Proposed branch: lp:~maas-maintainers/juju-core/maas-provider-skeleton
Merge into: lp:~juju/juju-core/trunk
Diff against target: 2475 lines (+2313/-13)
21 files modified
cmd/builddb/main.go (+4/-2)
cmd/juju/main.go (+2/-5)
cmd/jujud/main.go (+2/-4)
environs/all/all.go (+8/-0)
environs/cloudinit/cloudinit.go (+5/-2)
environs/cloudinit/cloudinit_test.go (+33/-0)
environs/maas/config.go (+72/-0)
environs/maas/config_test.go (+94/-0)
environs/maas/environ.go (+519/-0)
environs/maas/environ_test.go (+379/-0)
environs/maas/environprovider.go (+81/-0)
environs/maas/environprovider_test.go (+94/-0)
environs/maas/instance.go (+63/-0)
environs/maas/instance_test.go (+51/-0)
environs/maas/maas_test.go (+32/-0)
environs/maas/state.go (+47/-0)
environs/maas/state_test.go (+23/-0)
environs/maas/storage.go (+207/-0)
environs/maas/storage_test.go (+385/-0)
environs/maas/util.go (+83/-0)
environs/maas/util_test.go (+129/-0)
To merge this branch: bzr merge lp:~maas-maintainers/juju-core/maas-provider-skeleton
Reviewer Review Type Date Requested Status
Julian Edwards (community) Approve
William Reade (community) Approve
Review via email: mp+157025@code.launchpad.net

Commit message

Feature branch integration: MAAS provider.

Description of the change

What you see here is not yet a complete implementation of the MAAS provider. It's an ongoing feature branch that we would like to integrate at this point, before the interfaces diverge further. We apologize for the long diff! We kept it separate so that you wouldn't have to review all the intermediate steps.

I also offer my apologies for submitting this for review in the usual Canonical style rather than going through lbox like the juju-core team does. I tried to submit through lbox but it seems to have problems with team-owned branches. We work against this branch as a team, and plan to go through the review collaboratively, so taking ownership was not the sensible thing to do. Pushing a personal copy (like lbox did at one point) seemed like a recipe for confusion.

We're on a very short schedule with a very long branch, so reviewers, we would like to ask you: please keep a careful distinction between things that absolutely need fixing before the code can land, and things that need fixing before the code can be used or maintained. Landing is the first priority, and we can track and resolve the other issues on an ongoing basis. Landing first will allow upcoming refactorings in the main codebase to take this new provider into account, and so save us all time.

As is the practice for other providers, we based much of this work on the EC2 provider. But MAAS is a different beast, and so we didn't copy as much code unchanged as you'd see in the OpenStack provider. With Bootstrap() in particular we had to simplify code in order to track what data goes where, and be sure it suited our needs. The same refactoring would work for the other providers, so we also posted it on the juju-dev mailing list.

A few more things you'll want to know before reading the code:
 * Instance IDs in the MAAS provider are implemented as node URIs on the MAAS API.
 * We require that each environment have its own MAAS user.
 * A MAAS provides file storage in separate per-user namespaces (there is no global namespace).

To post a comment you must log in.
Revision history for this message
William Reade (fwereade) wrote :

> so we didn't copy as much code unchanged as you'd see in the OpenStack provider.

/me cheers, dances, gets down to reviewing

Revision history for this message
William Reade (fwereade) wrote :

Preliminary comments on the Environ...

Any points not badged with [FIX] should be considered usability/maintainability issues of varying importance.

1) _ "launchpad.net/juju-core/environs/maas"

Consider adding an environs/all package, that registers all providers except the dummy.

2) envCfg.attrs = v.(map[string]interface{})

Hmm... the attrs are all stored in the Config() anyway. Why are we duplicating them?

3) s/ecfgMutext/ecfgMutex/

4) quiesceStateFile [FIX if not necessary]

Only necessary if maas storage is not consistent.

5) uploadTools() [FIX: not necessary]

(Bootstrap no longer accepts uploadTools)

6) for a := longAttempt.Start(); len(stateAddrs) == 0 && a.Next(); { [FIX as (4)]

7) var err error = fmt.Errorf("(no error)")

I think this would be much clearer if we just break on err == nil inside the loop

8) StateInfo()

can't this use instances()?

9) environs.HighestVersion in StartInstance [FIX unless not unique]

This is wrong, but it might still match other providers.

10) environ.StopInstances([]environs.Instance{&instance})

Shouldn't this be a release?

Revision history for this message
William Reade (fwereade) wrote :

11) "maas-oauth": schema.String(),

This should probably be a custom schema.Checker

12) validCfg, err := prov.Validate(cfg, nil)

thumper's just landing some changes around this area: there's another method you need to call in Validate, IIRC, that checks old/new configs match sanely (no changing env names, for example).

13) "ca-private-key": testing.CAKey,

I'd have authorized-keys and admin-secret in here rather than duplicated in every test. While you're at it, please add `"agent-version": "1.2.3",` or something (it'll fit better with an upcoming branch of mine).

14) inst, err := env.startBootstrapNode(tools,

An upcoming branch will change this almost beyond recognition, but only for the better. BTW, is constraints support expected imminently? If not, please log.Warningf that they're ignored.

15) env.ecfgUnlocked = ecfg

There should be some validation that the two configs match, per thumper's branch. If we forgot it elsewhere, which we might have done, don't consider this a blocker.

16) environ.StopInstances([]environs.Instance{&instance})

upgrading (10) to a FIX

17) flags := environs.HighestVersion

crossgrading (9) to a changing-beyond-recognition

18) _, err := blah();\n if err != nil {

several places:

if _, err := blah(); err != nil {

19) client := environ.maasClientUnlocked.GetSubObject("nodes/") [FIX]

The intended model is that Environ methods be safe from any goroutine. Use a maasClient() helper throughout to get a client safely. (see `// To properly observe e.storageUnlocked `)

It may very well be the case that there's actually no point making Environs goroutine-safe, but I'm not going to analyse that now.

20) errSetConfig := env.SetConfig(cfg2)

Changing names is actively bad behaviour, please stick to sensible changes like default-series and authorized-keys :).

21) package maas

I don't really like having all these tests internal -- most of them should really go into maas_test, I think

22) func fakeWriteCertAndKey(

unnecessary?

23) === added file 'environs/maas/environprovider.go'

(and similar small files with clear purposes) yay! I like this, we should do it more ;)

24) func createTempFile(c *C

create it inside a c.MkDir() and it'll be cleaned up for you

25) info.serializeYAML()

Please use goyaml.Marshal and export/tag the struct fields where necessary

~~~~~~~~~~~~~~

Thanks, and sorry for the delay. This is looking really solid so far; if it's actually spinning up environments and deploying units now, then I'd like to get it in as soon as possible and starting working on rationalizing the testing then. I have no problem with the coverage, just of where the tests are (and that none of the jujutest tests have been hooked).

review: Approve
Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

On point 12, thumper explains:

The new method is environs.config.Validate(). Our Validate() should upcall to that, so that it can do its basic non-specific verification of the proposed config changes. See ec2/config.go, and similar for the openstack and dummy providers.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

A note on point 15:

«15) env.ecfgUnlocked = ecfg

«There should be some validation that the two configs match, per thumper's branch. If we forgot it elsewhere, which we might have done, don't consider this a blocker.»

The preceding call to maasEnvironProvider.newConfig() calls Validate(), which in turn will validate the state transition. So that should do the trick, right?

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

A note on points 10 & 16:

«10) environ.StopInstances([]environs.Instance{&instance})

«Shouldn't this be a release?»

«16) environ.StopInstances([]environs.Instance{&instance})

«upgrading (10) to a FIX»

It may just be me but I have no idea at all what you mean! How is a release related to the action of stopping instances? I can't find anything in the other providers that might make it clearer either.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

Note about points 9 and 17:

«9) environs.HighestVersion in StartInstance [FIX unless not unique]

«This is wrong, but it might still match other providers.»

«17) flags := environs.HighestVersion

«crossgrading (9) to a changing-beyond-recognition»

It's still the same in the EC2 provider, so leaving this be for now.

Revision history for this message
Raphaël Badin (rvb) wrote :

> Thanks, and sorry for the delay. This is looking really solid so far; if it's actually spinning up environments and
> deploying units now, then I'd like to get it in as soon as possible and starting working on rationalizing the testing
> then. I have no problem with the coverage, just of where the tests are (and that none of the jujutest tests have been
> hooked).

Thanks a lot for the reviews. We're in the process of fixing the issues you mentioned in your reviews but yes, we've been able to deploy a working mediawiki service in the MAAS lab today: http://paste.ubuntu.com/5697858/

Revision history for this message
Raphaël Badin (rvb) wrote :

> 8) StateInfo()
>
> can't this use instances()?

Do you mean you'd prefer StateInfo() to use the internal method env.instances() instead of env.Instances()?

Revision history for this message
William Reade (fwereade) wrote :

26) forgot to mention this: env.Provider().(*maasEnvironProvider)

just use providerInstance?

27) var _ environs.EnvironProvider = (*maasEnvironProvider)(nil)

no fields, no pointer receivers -- you may as well assert that the value implements EnvironProvider, use the value everywhere, and just forget about the &s and *s.

Revision history for this message
William Reade (fwereade) wrote :

> A note on point 15:
>
> «15) env.ecfgUnlocked = ecfg
>
> «There should be some validation that the two configs match, per thumper's
> branch. If we forgot it elsewhere, which we might have done, don't consider
> this a blocker.»
>
> The preceding call to maasEnvironProvider.newConfig() calls Validate(), which
> in turn will validate the state transition. So that should do the trick,
> right?

I don't think so... ISTM that we would validate against nil, rather than against the old config.

Revision history for this message
William Reade (fwereade) wrote :

> A note on points 10 & 16:
>
> «10) environ.StopInstances([]environs.Instance{&instance})
>
> «Shouldn't this be a release?»
>
> «16) environ.StopInstances([]environs.Instance{&instance})
>
> «upgrading (10) to a FIX»
>
> It may just be me but I have no idea at all what you mean! How is a release
> related to the action of stopping instances? I can't find anything in the
> other providers that might make it clearer either.

Ah, sorry -- I'm saying that if start has failed we should certainly release the nodes back to the pool if we can, but it's confusing to stop something that was never actually started.

Revision history for this message
William Reade (fwereade) wrote :

> Note about points 9 and 17:
>
> «9) environs.HighestVersion in StartInstance [FIX unless not unique]
>
> «This is wrong, but it might still match other providers.»
>
> «17) flags := environs.HighestVersion
>
> «crossgrading (9) to a changing-beyond-recognition»
>
> It's still the same in the EC2 provider, so leaving this be for now.

It'll be a race to land then, still a WIP on my end. If you win, I'm pretty sure it'll be np for me; if I win it should be simple for you but I'm always here to discuss if there are complications.

Revision history for this message
William Reade (fwereade) wrote :

> > 8) StateInfo()
> >
> > can't this use instances()?
>
> Do you mean you'd prefer StateInfo() to use the internal method
> env.instances() instead of env.Instances()?

It was really just a question, there may be some detail that makes it inappropriate, but at first blush it seemed like the ideal candidate. Your judgment call.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

I don't think instances() would make a good replacement for Instances(). It's basically the same thing, except instances() has a quirk to its API that made it suit the implementation of multiple exported functions. Instances() says what we want more precisely and with less risk of accident.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

More notes! On point #2 this time:

«2) envCfg.attrs = v.(map[string]interface{})

«Hmm... the attrs are all stored in the Config() anyway. Why are we duplicating them?»

Is there any more convenient access path (from outside the config package) than through configchecker.Coerce(cfg.UnknownAttrs(), nil)? If we skip the Coerce(), my understanding is that the variables won't be appropriately typed. Not an issue now, but very possibly problematic in the future. And UnknownAttrs() tediously duplicates attributes, so we might as well keep the result around.

Actually we're just doing the same thing as the EC2 provider here, only more explicitly — so maybe this was always an issue but it went unnoticed before. Here's how the EC2 code sets the equivalent "attrs" field:

        ecfg := &environConfig{cfg, v.(map[string]interface{})}

(And "v" is produced in the same way that we do it.)

Does this make sense? Or is there still a change that needs to be made here?

Revision history for this message
Julian Edwards (julian-edwards) :
review: Approve
Revision history for this message
William Reade (fwereade) wrote :

> Actually we're just doing the same thing as the EC2 provider here, only more explicitly — so maybe this was always an issue but it went unnoticed before. Here's how the EC2 code sets the equivalent "attrs" field:
>
> ecfg := &environConfig{cfg, v.(map[string]interface{})}
>
> (And "v" is produced in the same way that we do it.)
>
> Does this make sense? Or is there still a change that needs to be made here?

I think it's probably OK. Which is to say that environment configuration remains hairy and problematic and weird, because it's trying to be at *least* 3 different things, all of which have different sets of valid keys and values in various situations, but it's not very amenable to fixing at the moment.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'cmd/builddb/main.go'
--- cmd/builddb/main.go 2013-04-11 15:46:31 +0000
+++ cmd/builddb/main.go 2013-04-12 10:00:43 +0000
@@ -11,9 +11,11 @@
11 "os"11 "os"
12 "path/filepath"12 "path/filepath"
13 "time"13 "time"
14)
1415
15 // Register the provider16// Import the providers.
16 _ "launchpad.net/juju-core/environs/ec2"17import (
18 _ "launchpad.net/juju-core/environs/all"
17)19)
1820
19func main() {21func main() {
2022
=== modified file 'cmd/juju/main.go'
--- cmd/juju/main.go 2013-04-09 22:56:52 +0000
+++ cmd/juju/main.go 2013-04-12 10:00:43 +0000
@@ -7,12 +7,9 @@
7 "os"7 "os"
8)8)
99
10// When we import an environment provider implementation10// Import the providers.
11// here, it will register itself with environs, and hence
12// be available to the juju command.
13import (11import (
14 _ "launchpad.net/juju-core/environs/ec2"12 _ "launchpad.net/juju-core/environs/all"
15 _ "launchpad.net/juju-core/environs/openstack"
16)13)
1714
18var jujuDoc = `15var jujuDoc = `
1916
=== modified file 'cmd/jujud/main.go'
--- cmd/jujud/main.go 2013-03-26 20:17:23 +0000
+++ cmd/jujud/main.go 2013-04-12 10:00:43 +0000
@@ -9,11 +9,9 @@
9 "path/filepath"9 "path/filepath"
10)10)
1111
12// When we import an environment provider implementation12// Import the providers.
13// here, it will register itself with environs.
14import (13import (
15 _ "launchpad.net/juju-core/environs/ec2"14 _ "launchpad.net/juju-core/environs/all"
16 _ "launchpad.net/juju-core/environs/openstack"
17)15)
1816
19var jujudDoc = `17var jujudDoc = `
2018
=== added directory 'environs/all'
=== added file 'environs/all/all.go'
--- environs/all/all.go 1970-01-01 00:00:00 +0000
+++ environs/all/all.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,8 @@
1package all
2
3// Register all the available providers.
4import (
5 _ "launchpad.net/juju-core/environs/ec2"
6 _ "launchpad.net/juju-core/environs/maas"
7 _ "launchpad.net/juju-core/environs/openstack"
8)
09
=== modified file 'environs/cloudinit/cloudinit.go'
--- environs/cloudinit/cloudinit.go 2013-04-11 02:13:40 +0000
+++ environs/cloudinit/cloudinit.go 2013-04-12 10:00:43 +0000
@@ -98,11 +98,14 @@
98}98}
9999
100func New(cfg *MachineConfig) (*cloudinit.Config, error) {100func New(cfg *MachineConfig) (*cloudinit.Config, error) {
101 c := cloudinit.New()
102 return Configure(cfg, c)
103}
104
105func Configure(cfg *MachineConfig, c *cloudinit.Config) (*cloudinit.Config, error) {
101 if err := verifyConfig(cfg); err != nil {106 if err := verifyConfig(cfg); err != nil {
102 return nil, err107 return nil, err
103 }108 }
104 c := cloudinit.New()
105
106 c.AddSSHAuthorizedKeys(cfg.AuthorizedKeys)109 c.AddSSHAuthorizedKeys(cfg.AuthorizedKeys)
107 c.AddPackage("git")110 c.AddPackage("git")
108111
109112
=== modified file 'environs/cloudinit/cloudinit_test.go'
--- environs/cloudinit/cloudinit_test.go 2013-04-11 07:54:47 +0000
+++ environs/cloudinit/cloudinit_test.go 2013-04-12 10:00:43 +0000
@@ -4,6 +4,7 @@
4 "encoding/base64"4 "encoding/base64"
5 . "launchpad.net/gocheck"5 . "launchpad.net/gocheck"
6 "launchpad.net/goyaml"6 "launchpad.net/goyaml"
7 cloudinit_core "launchpad.net/juju-core/cloudinit"
7 "launchpad.net/juju-core/constraints"8 "launchpad.net/juju-core/constraints"
8 "launchpad.net/juju-core/environs/cloudinit"9 "launchpad.net/juju-core/environs/cloudinit"
9 "launchpad.net/juju-core/environs/config"10 "launchpad.net/juju-core/environs/config"
@@ -264,6 +265,38 @@
264 }265 }
265}266}
266267
268func (*cloudinitSuite) TestCloudInitConfigure(c *C) {
269 for i, test := range cloudinitTests {
270 test.cfg.Config = minimalConfig(c)
271 c.Logf("test %d (Configure)", i)
272 cloudcfg := cloudinit_core.New()
273 ci, err := cloudinit.Configure(&test.cfg, cloudcfg)
274 c.Assert(err, IsNil)
275 c.Check(ci, NotNil)
276 }
277}
278
279func (*cloudinitSuite) TestCloudInitConfigureUsesGivenConfig(c *C) {
280 // Create a simple cloudinit config with a 'runcmd' statement.
281 cloudcfg := cloudinit_core.New()
282 script := "test script"
283 cloudcfg.AddRunCmd(script)
284 cloudinitTests[0].cfg.Config = minimalConfig(c)
285 ci, err := cloudinit.Configure(&cloudinitTests[0].cfg, cloudcfg)
286 c.Assert(err, IsNil)
287 c.Check(ci, NotNil)
288 data, err := ci.Render()
289 c.Assert(err, IsNil)
290
291 ciContent := make(map[interface{}]interface{})
292 err = goyaml.Unmarshal(data, &ciContent)
293 c.Assert(err, IsNil)
294 // The 'runcmd' statement is at the beginning of the list
295 // of 'runcmd' statements.
296 runCmd := ciContent["runcmd"].([]interface{})
297 c.Check(runCmd[0], Equals, script)
298}
299
267func getScripts(x map[interface{}]interface{}) []string {300func getScripts(x map[interface{}]interface{}) []string {
268 var scripts []string301 var scripts []string
269 for _, s := range x["runcmd"].([]interface{}) {302 for _, s := range x["runcmd"].([]interface{}) {
270303
=== added directory 'environs/maas'
=== added file 'environs/maas/config.go'
--- environs/maas/config.go 1970-01-01 00:00:00 +0000
+++ environs/maas/config.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,72 @@
1package maas
2
3import (
4 "errors"
5 "fmt"
6 "launchpad.net/juju-core/environs/config"
7 "launchpad.net/juju-core/schema"
8 "net/url"
9 "strings"
10)
11
12var maasConfigChecker = schema.StrictFieldMap(
13 schema.Fields{
14 "maas-server": schema.String(),
15 // maas-oauth is a colon-separated triplet of:
16 // consumer-key:resource-token:resource-secret
17 "maas-oauth": schema.String(),
18 },
19 schema.Defaults{},
20)
21
22type maasEnvironConfig struct {
23 *config.Config
24 attrs map[string]interface{}
25}
26
27func (cfg *maasEnvironConfig) MAASServer() string {
28 return cfg.attrs["maas-server"].(string)
29}
30
31func (cfg *maasEnvironConfig) MAASOAuth() string {
32 return cfg.attrs["maas-oauth"].(string)
33}
34
35func (prov maasEnvironProvider) newConfig(cfg *config.Config) (*maasEnvironConfig, error) {
36 validCfg, err := prov.Validate(cfg, nil)
37 if err != nil {
38 return nil, err
39 }
40 result := new(maasEnvironConfig)
41 result.Config = validCfg
42 result.attrs = validCfg.UnknownAttrs()
43 return result, nil
44}
45
46var errMalformedMaasOAuth = errors.New("malformed maas-oauth (3 items separated by colons)")
47
48func (prov maasEnvironProvider) Validate(cfg, oldCfg *config.Config) (*config.Config, error) {
49 // Validate base configuration change before validating MAAS specifics.
50 err := config.Validate(cfg, oldCfg)
51 if err != nil {
52 return nil, err
53 }
54
55 v, err := maasConfigChecker.Coerce(cfg.UnknownAttrs(), nil)
56 if err != nil {
57 return nil, err
58 }
59 envCfg := new(maasEnvironConfig)
60 envCfg.Config = cfg
61 envCfg.attrs = v.(map[string]interface{})
62 server := envCfg.MAASServer()
63 serverURL, err := url.Parse(server)
64 if err != nil || serverURL.Scheme == "" || serverURL.Host == "" {
65 return nil, fmt.Errorf("malformed maas-server URL '%v': %s", server, err)
66 }
67 oauth := envCfg.MAASOAuth()
68 if strings.Count(oauth, ":") != 2 {
69 return nil, errMalformedMaasOAuth
70 }
71 return cfg.Apply(envCfg.attrs)
72}
073
=== added file 'environs/maas/config_test.go'
--- environs/maas/config_test.go 1970-01-01 00:00:00 +0000
+++ environs/maas/config_test.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,94 @@
1package maas
2
3import (
4 . "launchpad.net/gocheck"
5 "launchpad.net/juju-core/environs"
6 "launchpad.net/juju-core/testing"
7 "launchpad.net/juju-core/version"
8)
9
10type ConfigSuite struct{}
11
12var _ = Suite(new(ConfigSuite))
13
14// copyAttrs copies values from src into dest. If src contains a key that was
15// already in dest, its value in dest will still be updated to the one from
16// src.
17func copyAttrs(src, dest map[string]interface{}) {
18 for k, v := range src {
19 dest[k] = v
20 }
21}
22
23// newConfig creates a MAAS environment config from attributes.
24func newConfig(values map[string]interface{}) (*maasEnvironConfig, error) {
25 defaults := map[string]interface{}{
26 "name": "testenv",
27 "type": "maas",
28 "admin-secret": "ssshhhhhh",
29 "authorized-keys": "I-am-not-a-real-key",
30 "agent-version": version.CurrentNumber().String(),
31 // These are not needed by MAAS, but juju-core breaks without them. Needs
32 // fixing there.
33 "ca-cert": testing.CACert,
34 "ca-private-key": testing.CAKey,
35 }
36 cfg := make(map[string]interface{})
37 copyAttrs(defaults, cfg)
38 copyAttrs(values, cfg)
39 env, err := environs.NewFromAttrs(cfg)
40 if err != nil {
41 return nil, err
42 }
43 return env.(*maasEnviron).ecfg(), nil
44}
45
46func (ConfigSuite) TestParsesMAASSettings(c *C) {
47 server := "http://maas.example.com/maas/"
48 oauth := "consumer-key:resource-token:resource-secret"
49 ecfg, err := newConfig(map[string]interface{}{
50 "maas-server": server,
51 "maas-oauth": oauth,
52 })
53 c.Assert(err, IsNil)
54 c.Check(ecfg.MAASServer(), Equals, server)
55 c.Check(ecfg.MAASOAuth(), DeepEquals, oauth)
56}
57
58func (ConfigSuite) TestChecksWellFormedMaasServer(c *C) {
59 _, err := newConfig(map[string]interface{}{
60 "maas-server": "This should have been a URL.",
61 "maas-oauth": "consumer-key:resource-token:resource-secret",
62 })
63 c.Assert(err, NotNil)
64 c.Check(err, ErrorMatches, ".*malformed maas-server.*")
65}
66
67func (ConfigSuite) TestChecksWellFormedMaasOAuth(c *C) {
68 _, err := newConfig(map[string]interface{}{
69 "maas-server": "http://maas.example.com/maas/",
70 "maas-oauth": "This should have been a 3-part token.",
71 })
72 c.Assert(err, NotNil)
73 c.Check(err, ErrorMatches, ".*malformed maas-oauth.*")
74}
75
76func (ConfigSuite) TestValidateUpcallsEnvironsConfigValidate(c *C) {
77 // The base Validate() function will not allow an environment to
78 // change its name. Trigger that error so as to prove that the
79 // environment provider's Validate() calls the base Validate().
80 baseAttrs := map[string]interface{}{
81 "maas-server": "http://maas.example.com/maas/",
82 "maas-oauth": "consumer-key:resource-token:resource-secret",
83 }
84 oldCfg, err := newConfig(baseAttrs)
85 c.Assert(err, IsNil)
86 newName := oldCfg.Name() + "-but-different"
87 newCfg, err := oldCfg.Apply(map[string]interface{}{"name": newName})
88 c.Assert(err, IsNil)
89
90 _, err = maasEnvironProvider{}.Validate(newCfg, oldCfg.Config)
91
92 c.Assert(err, NotNil)
93 c.Check(err, ErrorMatches, ".*cannot change name.*")
94}
095
=== added file 'environs/maas/environ.go'
--- environs/maas/environ.go 1970-01-01 00:00:00 +0000
+++ environs/maas/environ.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,519 @@
1package maas
2
3import (
4 "encoding/base64"
5 "errors"
6 "fmt"
7 "launchpad.net/gomaasapi"
8 "launchpad.net/juju-core/constraints"
9 "launchpad.net/juju-core/environs"
10 "launchpad.net/juju-core/environs/cloudinit"
11 "launchpad.net/juju-core/environs/config"
12 "launchpad.net/juju-core/log"
13 "launchpad.net/juju-core/state"
14 "launchpad.net/juju-core/state/api"
15 "launchpad.net/juju-core/state/api/params"
16 "launchpad.net/juju-core/trivial"
17 "launchpad.net/juju-core/version"
18 "net/url"
19 "sync"
20 "time"
21)
22
23const (
24 mgoPort = 37017
25 apiPort = 17070
26 jujuDataDir = "/var/lib/juju"
27 // We're using v1.0 of the MAAS API.
28 apiVersion = "1.0"
29)
30
31var mgoPortSuffix = fmt.Sprintf(":%d", mgoPort)
32var apiPortSuffix = fmt.Sprintf(":%d", apiPort)
33
34var longAttempt = trivial.AttemptStrategy{
35 Total: 3 * time.Minute,
36 Delay: 1 * time.Second,
37}
38
39type maasEnviron struct {
40 name string
41
42 // ecfgMutex protects the *Unlocked fields below.
43 ecfgMutex sync.Mutex
44
45 ecfgUnlocked *maasEnvironConfig
46 maasClientUnlocked *gomaasapi.MAASObject
47 storageUnlocked environs.Storage
48}
49
50var _ environs.Environ = (*maasEnviron)(nil)
51
52var couldNotAllocate = errors.New("Could not allocate MAAS environment object.")
53
54func NewEnviron(cfg *config.Config) (*maasEnviron, error) {
55 env := new(maasEnviron)
56 if env == nil {
57 return nil, couldNotAllocate
58 }
59 err := env.SetConfig(cfg)
60 if err != nil {
61 return nil, err
62 }
63 env.storageUnlocked = NewStorage(env)
64 return env, nil
65}
66
67func (env *maasEnviron) Name() string {
68 return env.name
69}
70
71// TODO: this code is cargo-culted from the openstack/ec2 providers.
72func (env *maasEnviron) findTools() (*state.Tools, error) {
73 flags := environs.HighestVersion | environs.CompatVersion
74 v := version.Current
75 v.Series = env.Config().DefaultSeries()
76 return environs.FindTools(env, v, flags)
77}
78
79// makeMachineConfig sets up a basic machine configuration for use with
80// userData(). You may still need to supply more information, but this takes
81// care of the fixed entries and the ones that are always needed.
82func (env *maasEnviron) makeMachineConfig(machineID, machineNonce string, stateInfo *state.Info, apiInfo *api.Info, tools *state.Tools) *cloudinit.MachineConfig {
83 return &cloudinit.MachineConfig{
84 // Fixed entries.
85 MongoPort: mgoPort,
86 APIPort: apiPort,
87 DataDir: jujuDataDir,
88
89 // Entries based purely on what's in the environment.
90 AuthorizedKeys: env.ecfg().AuthorizedKeys(),
91
92 // Parameter entries.
93 MachineId: machineID,
94 MachineNonce: machineNonce,
95 StateInfo: stateInfo,
96 APIInfo: apiInfo,
97 Tools: tools,
98 }
99}
100
101// startBootstrapNode starts the juju bootstrap node for this environment.
102func (env *maasEnviron) startBootstrapNode(tools *state.Tools, cert, key []byte, password string) (environs.Instance, error) {
103 config, err := environs.BootstrapConfig(env.Provider(), env.Config(), tools)
104 if err != nil {
105 return nil, fmt.Errorf("unable to determine initial configuration: %v", err)
106 }
107 caCert, hasCert := env.Config().CACert()
108 if !hasCert {
109 return nil, fmt.Errorf("no CA certificate in environment configuration")
110 }
111 stateInfo := state.Info{
112 Password: trivial.PasswordHash(password),
113 CACert: caCert,
114 }
115 apiInfo := api.Info{
116 Password: trivial.PasswordHash(password),
117 CACert: caCert,
118 }
119
120 // The bootstrap instance gets machine id "0". This is not related to
121 // instance ids or MAAS system ids. Juju assigns the machine ID.
122 const machineID = "0"
123
124 mcfg := env.makeMachineConfig(machineID, state.BootstrapNonce, &stateInfo, &apiInfo, tools)
125 mcfg.StateServer = true
126 mcfg.StateServerCert = cert
127 mcfg.StateServerKey = key
128 mcfg.Config = config
129
130 inst, err := env.obtainNode(machineID, &stateInfo, &apiInfo, tools, mcfg)
131 if err != nil {
132 return nil, fmt.Errorf("cannot start bootstrap instance: %v", err)
133 }
134 return inst, nil
135}
136
137// Bootstrap is specified in the Environ interface.
138func (env *maasEnviron) Bootstrap(cons constraints.Value, stateServerCert, stateServerKey []byte) error {
139 constraints := cons.String()
140 if constraints != "" {
141 log.Warningf("ignoring constraints '%s' (not implemented)", constraints)
142 }
143
144 // This was all cargo-culted from the EC2 provider.
145 password := env.Config().AdminSecret()
146 if password == "" {
147 return fmt.Errorf("admin-secret is required for bootstrap")
148 }
149 log.Debugf("environs/maas: bootstrapping environment %q.", env.Name())
150 tools, err := env.findTools()
151 if err != nil {
152 return err
153 }
154 inst, err := env.startBootstrapNode(tools, stateServerCert, stateServerKey, password)
155 if err != nil {
156 return err
157 }
158 err = env.saveState(&bootstrapState{StateInstances: []state.InstanceId{inst.Id()}})
159 if err != nil {
160 env.releaseInstance(inst)
161 return fmt.Errorf("cannot save state: %v", err)
162 }
163
164 // TODO make safe in the case of racing Bootstraps
165 // If two Bootstraps are called concurrently, there's
166 // no way to make sure that only one succeeds.
167
168 return nil
169}
170
171// StateInfo is specified in the Environ interface.
172func (env *maasEnviron) StateInfo() (*state.Info, *api.Info, error) {
173 // This code is cargo-culted from the openstack/ec2 providers.
174 // It's a bit unclear what the "longAttempt" loop is actually for
175 // but this should probably be refactored outside of the provider
176 // code.
177 st, err := env.loadState()
178 if err != nil {
179 return nil, nil, err
180 }
181 cert, hasCert := env.Config().CACert()
182 if !hasCert {
183 return nil, nil, fmt.Errorf("no CA certificate in environment configuration")
184 }
185 var stateAddrs []string
186 var apiAddrs []string
187 // Wait for the DNS names of any of the instances
188 // to become available.
189 log.Debugf("environs/maas: waiting for DNS name(s) of state server instances %v", st.StateInstances)
190 for a := longAttempt.Start(); len(stateAddrs) == 0 && a.Next(); {
191 insts, err := env.Instances(st.StateInstances)
192 if err != nil && err != environs.ErrPartialInstances {
193 log.Debugf("error getting state instance: %v", err.Error())
194 return nil, nil, err
195 }
196 log.Debugf("started processing instances: %#v", insts)
197 for _, inst := range insts {
198 if inst == nil {
199 continue
200 }
201 name, err := inst.DNSName()
202 if err != nil {
203 continue
204 }
205 if name != "" {
206 stateAddrs = append(stateAddrs, name+mgoPortSuffix)
207 apiAddrs = append(apiAddrs, name+apiPortSuffix)
208 }
209 }
210 }
211 if len(stateAddrs) == 0 {
212 return nil, nil, fmt.Errorf("timed out waiting for mgo address from %v", st.StateInstances)
213 }
214 return &state.Info{
215 Addrs: stateAddrs,
216 CACert: cert,
217 }, &api.Info{
218 Addrs: apiAddrs,
219 CACert: cert,
220 }, nil
221}
222
223// ecfg returns the environment's maasEnvironConfig, and protects it with a
224// mutex.
225func (env *maasEnviron) ecfg() *maasEnvironConfig {
226 env.ecfgMutex.Lock()
227 defer env.ecfgMutex.Unlock()
228 return env.ecfgUnlocked
229}
230
231// Config is specified in the Environ interface.
232func (env *maasEnviron) Config() *config.Config {
233 return env.ecfg().Config
234}
235
236// SetConfig is specified in the Environ interface.
237func (env *maasEnviron) SetConfig(cfg *config.Config) error {
238 env.ecfgMutex.Lock()
239 defer env.ecfgMutex.Unlock()
240
241 // The new config has already been validated by itself, but now we
242 // validate the transition from the old config to the new.
243 var oldCfg *config.Config
244 if env.ecfgUnlocked != nil {
245 oldCfg = env.ecfgUnlocked.Config
246 }
247 cfg, err := env.Provider().Validate(cfg, oldCfg)
248 if err != nil {
249 return err
250 }
251
252 ecfg, err := providerInstance.newConfig(cfg)
253 if err != nil {
254 return err
255 }
256
257 env.name = cfg.Name()
258 env.ecfgUnlocked = ecfg
259
260 authClient, err := gomaasapi.NewAuthenticatedClient(ecfg.MAASServer(), ecfg.MAASOAuth(), apiVersion)
261 if err != nil {
262 return err
263 }
264 env.maasClientUnlocked = gomaasapi.NewMAAS(*authClient)
265
266 return nil
267}
268
269// getMAASClient returns a MAAS client object to use for a request, in a
270// lock-protected fashioon.
271func (env *maasEnviron) getMAASClient() *gomaasapi.MAASObject {
272 env.ecfgMutex.Lock()
273 defer env.ecfgMutex.Unlock()
274
275 return env.maasClientUnlocked
276}
277
278// acquireNode allocates a node from the MAAS.
279func (environ *maasEnviron) acquireNode() (gomaasapi.MAASObject, error) {
280 retry := trivial.AttemptStrategy{
281 Total: 5 * time.Second,
282 Delay: 200 * time.Millisecond,
283 }
284 var result gomaasapi.JSONObject
285 var err error
286 for a := retry.Start(); a.Next(); {
287 client := environ.getMAASClient().GetSubObject("nodes/")
288 result, err = client.CallPost("acquire", nil)
289 if err == nil {
290 break
291 }
292 }
293 if err != nil {
294 return gomaasapi.MAASObject{}, err
295 }
296 node, err := result.GetMAASObject()
297 if err != nil {
298 msg := fmt.Errorf("unexpected result from 'acquire' on MAAS API: %v", err)
299 return gomaasapi.MAASObject{}, msg
300 }
301 return node, nil
302}
303
304// startNode installs and boots a node.
305func (environ *maasEnviron) startNode(node gomaasapi.MAASObject, tools *state.Tools, userdata []byte) error {
306 retry := trivial.AttemptStrategy{
307 Total: 5 * time.Second,
308 Delay: 200 * time.Millisecond,
309 }
310 userDataParam := base64.StdEncoding.EncodeToString(userdata)
311 params := url.Values{
312 "distro_series": {tools.Series},
313 "user_data": {userDataParam},
314 }
315 // Initialize err to a non-nil value as a sentinel for the following
316 // loop.
317 err := fmt.Errorf("(no error)")
318 for a := retry.Start(); a.Next() && err != nil; {
319 _, err = node.CallPost("start", params)
320 }
321 return err
322}
323
324// obtainNode allocates and starts a MAAS node. It is used both for the
325// implementation of StartInstance, and to initialize the bootstrap node.
326func (environ *maasEnviron) obtainNode(machineId string, stateInfo *state.Info, apiInfo *api.Info, tools *state.Tools, mcfg *cloudinit.MachineConfig) (*maasInstance, error) {
327
328 log.Debugf("environs/maas: starting machine %s in $q running tools version %q from %q", machineId, environ.name, tools.Binary, tools.URL)
329
330 node, err := environ.acquireNode()
331 if err != nil {
332 return nil, fmt.Errorf("cannot run instances: %v", err)
333 }
334
335 hostname, err := node.GetField("hostname")
336 if err != nil {
337 return nil, err
338 }
339 instance := maasInstance{&node, environ}
340 info := machineInfo{string(instance.Id()), hostname}
341 runCmd, err := info.cloudinitRunCmd()
342 if err != nil {
343 return nil, err
344 }
345 userdata, err := userData(mcfg, runCmd)
346 if err != nil {
347 msg := fmt.Errorf("could not compose userdata for bootstrap node: %v", err)
348 return nil, msg
349 }
350 err = environ.startNode(node, tools, userdata)
351 if err != nil {
352 environ.releaseInstance(&instance)
353 return nil, fmt.Errorf("cannot start instance: %v", err)
354 }
355 log.Debugf("environs/maas: started instance %q", instance.Id())
356 return &instance, nil
357}
358
359// StartInstance is specified in the Environ interface.
360func (environ *maasEnviron) StartInstance(machineID, machineNonce string, series string, cons constraints.Value, stateInfo *state.Info, apiInfo *api.Info) (environs.Instance, error) {
361 // TODO: Support series and constraints. They were added to the
362 // interface after we implemented.
363 flags := environs.HighestVersion | environs.CompatVersion
364 var err error
365 tools, err := environs.FindTools(environ, version.Current, flags)
366 if err != nil {
367 return nil, err
368 }
369
370 mcfg := environ.makeMachineConfig(machineID, machineNonce, stateInfo, apiInfo, tools)
371 return environ.obtainNode(machineID, stateInfo, apiInfo, tools, mcfg)
372}
373
374// StopInstances is specified in the Environ interface.
375func (environ *maasEnviron) StopInstances(instances []environs.Instance) error {
376 // Shortcut to exit quickly if 'instances' is an empty slice or nil.
377 if len(instances) == 0 {
378 return nil
379 }
380 // Tell MAAS to release each of the instances. If there are errors,
381 // return only the first one (but release all instances regardless).
382 // Note that releasing instances also turns them off.
383 var firstErr error
384 for _, instance := range instances {
385 err := environ.releaseInstance(instance)
386 if firstErr == nil {
387 firstErr = err
388 }
389 }
390 return firstErr
391}
392
393// releaseInstance releases a single instance.
394func (environ *maasEnviron) releaseInstance(inst environs.Instance) error {
395 maasInst := inst.(*maasInstance)
396 maasObj := maasInst.maasObject
397 _, err := maasObj.CallPost("release", nil)
398 if err != nil {
399 log.Debugf("environs/maas: error releasing instance %v", maasInst)
400 }
401 return err
402}
403
404// Instances returns the environs.Instance objects corresponding to the given
405// slice of state.InstanceId. Similar to what the ec2 provider does,
406// Instances returns nil if the given slice is empty or nil.
407func (environ *maasEnviron) Instances(ids []state.InstanceId) ([]environs.Instance, error) {
408 if len(ids) == 0 {
409 return nil, nil
410 }
411 return environ.instances(ids)
412}
413
414// instances is an internal method which returns the instances matching the
415// given instance ids or all the instances if 'ids' is empty.
416// If the some of the intances could not be found, it returns the instance
417// that could be found plus the error environs.ErrPartialInstances in the error
418// return.
419func (environ *maasEnviron) instances(ids []state.InstanceId) ([]environs.Instance, error) {
420 nodeListing := environ.getMAASClient().GetSubObject("nodes")
421 filter := getSystemIdValues(ids)
422 listNodeObjects, err := nodeListing.CallGet("list", filter)
423 if err != nil {
424 return nil, err
425 }
426 listNodes, err := listNodeObjects.GetArray()
427 if err != nil {
428 return nil, err
429 }
430 instances := make([]environs.Instance, len(listNodes))
431 for index, nodeObj := range listNodes {
432 node, err := nodeObj.GetMAASObject()
433 if err != nil {
434 return nil, err
435 }
436 instances[index] = &maasInstance{
437 maasObject: &node,
438 environ: environ,
439 }
440 }
441 if len(ids) != 0 && len(ids) != len(instances) {
442 return instances, environs.ErrPartialInstances
443 }
444 return instances, nil
445}
446
447// AllInstances returns all the environs.Instance in this provider.
448func (environ *maasEnviron) AllInstances() ([]environs.Instance, error) {
449 return environ.instances(nil)
450}
451
452// Storage is defined by the Environ interface.
453func (env *maasEnviron) Storage() environs.Storage {
454 env.ecfgMutex.Lock()
455 defer env.ecfgMutex.Unlock()
456 return env.storageUnlocked
457}
458
459// PublicStorage is defined by the Environ interface.
460func (env *maasEnviron) PublicStorage() environs.StorageReader {
461 // MAAS does not have a shared storage.
462 return environs.EmptyStorage
463}
464
465func (environ *maasEnviron) Destroy(ensureInsts []environs.Instance) error {
466 log.Debugf("environs/maas: destroying environment %q", environ.name)
467 insts, err := environ.AllInstances()
468 if err != nil {
469 return fmt.Errorf("cannot get instances: %v", err)
470 }
471 found := make(map[state.InstanceId]bool)
472 for _, inst := range insts {
473 found[inst.Id()] = true
474 }
475
476 // Add any instances we've been told about but haven't yet shown
477 // up in the instance list.
478 for _, inst := range ensureInsts {
479 id := inst.Id()
480 if !found[id] {
481 insts = append(insts, inst)
482 found[id] = true
483 }
484 }
485 err = environ.StopInstances(insts)
486 if err != nil {
487 return err
488 }
489
490 // To properly observe e.storageUnlocked we need to get its value while
491 // holding e.ecfgMutex. e.Storage() does this for us, then we convert
492 // back to the (*storage) to access the private deleteAll() method.
493 st := environ.Storage().(*maasStorage)
494 return st.deleteAll()
495}
496
497func (*maasEnviron) AssignmentPolicy() state.AssignmentPolicy {
498 return state.AssignUnused
499}
500
501// MAAS does not do firewalling so these port methods do nothing.
502func (*maasEnviron) OpenPorts([]params.Port) error {
503 log.Debugf("environs/maas: unimplemented OpenPorts() called")
504 return nil
505}
506
507func (*maasEnviron) ClosePorts([]params.Port) error {
508 log.Debugf("environs/maas: unimplemented ClosePorts() called")
509 return nil
510}
511
512func (*maasEnviron) Ports() ([]params.Port, error) {
513 log.Debugf("environs/maas: unimplemented Ports() called")
514 return []params.Port{}, nil
515}
516
517func (*maasEnviron) Provider() environs.EnvironProvider {
518 return &providerInstance
519}
0520
=== added file 'environs/maas/environ_test.go'
--- environs/maas/environ_test.go 1970-01-01 00:00:00 +0000
+++ environs/maas/environ_test.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,379 @@
1package maas
2
3import (
4 "encoding/base64"
5 . "launchpad.net/gocheck"
6 "launchpad.net/gomaasapi"
7 "launchpad.net/goyaml"
8 "launchpad.net/juju-core/constraints"
9 "launchpad.net/juju-core/environs"
10 "launchpad.net/juju-core/environs/config"
11 envtesting "launchpad.net/juju-core/environs/testing"
12 "launchpad.net/juju-core/state"
13 "launchpad.net/juju-core/testing"
14 "launchpad.net/juju-core/trivial"
15 "launchpad.net/juju-core/version"
16)
17
18type EnvironSuite struct {
19 ProviderSuite
20}
21
22var _ = Suite(new(EnvironSuite))
23
24// getTestConfig creates a customized sample MAAS provider configuration.
25func getTestConfig(name, server, oauth, secret string) *config.Config {
26 ecfg, err := newConfig(map[string]interface{}{
27 "name": name,
28 "maas-server": server,
29 "maas-oauth": oauth,
30 "admin-secret": secret,
31 "authorized-keys": "I-am-not-a-real-key",
32 })
33 if err != nil {
34 panic(err)
35 }
36 return ecfg.Config
37}
38
39// makeEnviron creates a functional maasEnviron for a test. Its configuration
40// is a bit arbitrary and none of the test code's business.
41func (suite *EnvironSuite) makeEnviron() *maasEnviron {
42 config, err := config.New(map[string]interface{}{
43 "name": suite.environ.Name(),
44 "type": "maas",
45 "admin-secret": "local-secret",
46 "authorized-keys": "foo",
47 "agent-version": version.CurrentNumber().String(),
48 "maas-oauth": "a:b:c",
49 "maas-server": suite.testMAASObject.TestServer.URL,
50 // These are not needed by MAAS, but juju-core breaks without them. Needs
51 // fixing there.
52 "ca-cert": testing.CACert,
53 "ca-private-key": testing.CAKey,
54 })
55 if err != nil {
56 panic(err)
57 }
58 env, err := NewEnviron(config)
59 if err != nil {
60 panic(err)
61 }
62 return env
63}
64
65func (suite *EnvironSuite) setupFakeProviderStateFile(c *C) {
66 suite.testMAASObject.TestServer.NewFile("provider-state", []byte("test file content"))
67}
68
69func (suite *EnvironSuite) setupFakeTools(c *C) {
70 storage := NewStorage(suite.environ)
71 envtesting.PutFakeTools(c, storage)
72}
73
74func (EnvironSuite) TestSetConfigValidatesFirst(c *C) {
75 // SetConfig() validates the config change and disallows, for example,
76 // changes in the environment name.
77 server := "http://maas.example.com"
78 oauth := "a:b:c"
79 secret := "pssst"
80 oldCfg := getTestConfig("old-name", server, oauth, secret)
81 newCfg := getTestConfig("new-name", server, oauth, secret)
82 env, err := NewEnviron(oldCfg)
83 c.Assert(err, IsNil)
84
85 // SetConfig() fails, even though both the old and the new config are
86 // individually valid.
87 err = env.SetConfig(newCfg)
88 c.Assert(err, NotNil)
89 c.Check(err, ErrorMatches, ".*cannot change name.*")
90
91 // The old config is still in place. The new config never took effect.
92 c.Check(env.Name(), Equals, "old-name")
93}
94
95func (EnvironSuite) TestSetConfigUpdatesConfig(c *C) {
96 name := "test env"
97 cfg := getTestConfig(name, "http://maas2.example.com", "a:b:c", "secret")
98 env, err := NewEnviron(cfg)
99 c.Check(err, IsNil)
100 c.Check(env.name, Equals, "test env")
101
102 anotherServer := "http://maas.example.com"
103 anotherOauth := "c:d:e"
104 anotherSecret := "secret2"
105 cfg2 := getTestConfig(name, anotherServer, anotherOauth, anotherSecret)
106 errSetConfig := env.SetConfig(cfg2)
107 c.Check(errSetConfig, IsNil)
108 c.Check(env.name, Equals, name)
109 authClient, _ := gomaasapi.NewAuthenticatedClient(anotherServer, anotherOauth, apiVersion)
110 maas := gomaasapi.NewMAAS(*authClient)
111 MAASServer := env.maasClientUnlocked
112 c.Check(MAASServer, DeepEquals, maas)
113}
114
115func (EnvironSuite) TestNewEnvironSetsConfig(c *C) {
116 name := "test env"
117 cfg := getTestConfig(name, "http://maas.example.com", "a:b:c", "secret")
118
119 env, err := NewEnviron(cfg)
120
121 c.Check(err, IsNil)
122 c.Check(env.name, Equals, name)
123}
124
125func (suite *EnvironSuite) TestInstancesReturnsInstances(c *C) {
126 input := `{"system_id": "test"}`
127 node := suite.testMAASObject.TestServer.NewNode(input)
128 resourceURI, _ := node.GetField("resource_uri")
129 instanceIds := []state.InstanceId{state.InstanceId(resourceURI)}
130
131 instances, err := suite.environ.Instances(instanceIds)
132
133 c.Check(err, IsNil)
134 c.Check(len(instances), Equals, 1)
135 c.Check(string(instances[0].Id()), Equals, resourceURI)
136}
137
138func (suite *EnvironSuite) TestInstancesReturnsNilIfEmptyParameter(c *C) {
139 // Instances returns nil if the given parameter is empty.
140 input := `{"system_id": "test"}`
141 suite.testMAASObject.TestServer.NewNode(input)
142 instances, err := suite.environ.Instances([]state.InstanceId{})
143
144 c.Check(err, IsNil)
145 c.Check(instances, IsNil)
146}
147
148func (suite *EnvironSuite) TestInstancesReturnsNilIfNilParameter(c *C) {
149 // Instances returns nil if the given parameter is nil.
150 input := `{"system_id": "test"}`
151 suite.testMAASObject.TestServer.NewNode(input)
152 instances, err := suite.environ.Instances(nil)
153
154 c.Check(err, IsNil)
155 c.Check(instances, IsNil)
156}
157
158func (suite *EnvironSuite) TestAllInstancesReturnsAllInstances(c *C) {
159 input := `{"system_id": "test"}`
160 node := suite.testMAASObject.TestServer.NewNode(input)
161 resourceURI, _ := node.GetField("resource_uri")
162
163 instances, err := suite.environ.AllInstances()
164
165 c.Check(err, IsNil)
166 c.Check(len(instances), Equals, 1)
167 c.Check(string(instances[0].Id()), Equals, resourceURI)
168}
169
170func (suite *EnvironSuite) TestAllInstancesReturnsEmptySliceIfNoInstance(c *C) {
171 instances, err := suite.environ.AllInstances()
172
173 c.Check(err, IsNil)
174 c.Check(len(instances), Equals, 0)
175}
176
177func (suite *EnvironSuite) TestInstancesReturnsErrorIfPartialInstances(c *C) {
178 input1 := `{"system_id": "test"}`
179 node1 := suite.testMAASObject.TestServer.NewNode(input1)
180 resourceURI1, _ := node1.GetField("resource_uri")
181 input2 := `{"system_id": "test2"}`
182 suite.testMAASObject.TestServer.NewNode(input2)
183 instanceId1 := state.InstanceId(resourceURI1)
184 instanceId2 := state.InstanceId("unknown systemID")
185 instanceIds := []state.InstanceId{instanceId1, instanceId2}
186
187 instances, err := suite.environ.Instances(instanceIds)
188
189 c.Check(err, Equals, environs.ErrPartialInstances)
190 c.Check(len(instances), Equals, 1)
191 c.Check(string(instances[0].Id()), Equals, resourceURI1)
192}
193
194func (suite *EnvironSuite) TestStorageReturnsStorage(c *C) {
195 env := suite.makeEnviron()
196 storage := env.Storage()
197 c.Check(storage, NotNil)
198 // The Storage object is really a maasStorage.
199 specificStorage := storage.(*maasStorage)
200 // Its environment pointer refers back to its environment.
201 c.Check(specificStorage.environUnlocked, Equals, env)
202}
203
204func (suite *EnvironSuite) TestPublicStorageReturnsEmptyStorage(c *C) {
205 env := suite.makeEnviron()
206 storage := env.PublicStorage()
207 c.Assert(storage, NotNil)
208 c.Check(storage, Equals, environs.EmptyStorage)
209}
210
211func decodeUserData(userData string) ([]byte, error) {
212 data, err := base64.StdEncoding.DecodeString(userData)
213 if err != nil {
214 return []byte(""), err
215 }
216 return trivial.Gunzip(data)
217}
218
219func (suite *EnvironSuite) TestStartInstanceStartsInstance(c *C) {
220 suite.setupFakeTools(c)
221 env := suite.makeEnviron()
222 // Create node 0: it will be used as the bootstrap node.
223 suite.testMAASObject.TestServer.NewNode(`{"system_id": "node0", "hostname": "host0"}`)
224 err := environs.Bootstrap(env, constraints.Value{})
225 c.Assert(err, IsNil)
226 // The bootstrap node has been started.
227 operations := suite.testMAASObject.TestServer.NodeOperations()
228 actions, found := operations["node0"]
229 c.Check(found, Equals, true)
230 c.Check(actions, DeepEquals, []string{"start"})
231
232 // Create node 1: it will be used as instance number 1.
233 suite.testMAASObject.TestServer.NewNode(`{"system_id": "node1", "hostname": "host1"}`)
234 stateInfo, apiInfo, err := env.StateInfo()
235 c.Assert(err, IsNil)
236 stateInfo.Tag = "machine-1"
237 apiInfo.Tag = "machine-1"
238 series := version.Current.Series
239 nonce := "12345"
240 instance, err := env.StartInstance("1", nonce, series, constraints.Value{}, stateInfo, apiInfo)
241 c.Assert(err, IsNil)
242 c.Check(instance, NotNil)
243
244 // The instance number 1 has been started.
245 actions, found = operations["node1"]
246 c.Assert(found, Equals, true)
247 c.Check(actions, DeepEquals, []string{"start"})
248
249 // The value of the "user data" parameter used when starting the node
250 // contains the run cmd used to write the machine information onto
251 // the node's filesystem.
252 requestValues := suite.testMAASObject.TestServer.NodeOperationRequestValues()
253 nodeRequestValues, found := requestValues["node1"]
254 c.Assert(found, Equals, true)
255 userData := nodeRequestValues[0].Get("user_data")
256 decodedUserData, err := decodeUserData(userData)
257 c.Assert(err, IsNil)
258 info := machineInfo{string(instance.Id()), "host1"}
259 cloudinitRunCmd, err := info.cloudinitRunCmd()
260 c.Assert(err, IsNil)
261 data, err := goyaml.Marshal(cloudinitRunCmd)
262 c.Assert(err, IsNil)
263 c.Check(string(decodedUserData), Matches, "(.|\n)*"+string(data)+"(\n|.)*")
264}
265
266func (suite *EnvironSuite) getInstance(systemId string) *maasInstance {
267 input := `{"system_id": "` + systemId + `"}`
268 node := suite.testMAASObject.TestServer.NewNode(input)
269 return &maasInstance{&node, suite.environ}
270}
271
272func (suite *EnvironSuite) TestStopInstancesReturnsIfParameterEmpty(c *C) {
273 suite.getInstance("test1")
274
275 err := suite.environ.StopInstances([]environs.Instance{})
276 c.Check(err, IsNil)
277 operations := suite.testMAASObject.TestServer.NodeOperations()
278 c.Check(operations, DeepEquals, map[string][]string{})
279}
280
281func (suite *EnvironSuite) TestStopInstancesStopsAndReleasesInstances(c *C) {
282 instance1 := suite.getInstance("test1")
283 instance2 := suite.getInstance("test2")
284 suite.getInstance("test3")
285 instances := []environs.Instance{instance1, instance2}
286
287 err := suite.environ.StopInstances(instances)
288
289 c.Check(err, IsNil)
290 operations := suite.testMAASObject.TestServer.NodeOperations()
291 expectedOperations := map[string][]string{"test1": {"release"}, "test2": {"release"}}
292 c.Check(operations, DeepEquals, expectedOperations)
293}
294
295func (suite *EnvironSuite) TestStateInfo(c *C) {
296 env := suite.makeEnviron()
297 hostname := "test"
298 input := `{"system_id": "system_id", "hostname": "` + hostname + `"}`
299 node := suite.testMAASObject.TestServer.NewNode(input)
300 instance := &maasInstance{&node, suite.environ}
301 err := env.saveState(&bootstrapState{StateInstances: []state.InstanceId{instance.Id()}})
302 c.Assert(err, IsNil)
303
304 stateInfo, apiInfo, err := env.StateInfo()
305
306 c.Assert(err, IsNil)
307 c.Assert(stateInfo.Addrs, DeepEquals, []string{hostname + mgoPortSuffix})
308 c.Assert(apiInfo.Addrs, DeepEquals, []string{hostname + apiPortSuffix})
309}
310
311func (suite *EnvironSuite) TestStateInfoFailsIfNoStateInstances(c *C) {
312 env := suite.makeEnviron()
313
314 _, _, err := env.StateInfo()
315
316 c.Check(err, FitsTypeOf, &environs.NotFoundError{})
317}
318
319func (suite *EnvironSuite) TestDestroy(c *C) {
320 env := suite.makeEnviron()
321 suite.getInstance("test1")
322 instance := suite.getInstance("test2")
323 data := makeRandomBytes(10)
324 suite.testMAASObject.TestServer.NewFile("filename", data)
325 storage := env.Storage()
326
327 err := env.Destroy([]environs.Instance{instance})
328
329 c.Check(err, IsNil)
330 // Instances have been stopped.
331 operations := suite.testMAASObject.TestServer.NodeOperations()
332 expectedOperations := map[string][]string{"test1": {"release"}, "test2": {"release"}}
333 c.Check(operations, DeepEquals, expectedOperations)
334 // Files have been cleaned up.
335 listing, err := storage.List("")
336 c.Assert(err, IsNil)
337 c.Check(listing, DeepEquals, []string{})
338}
339
340// It would be nice if we could unit-test Bootstrap() in more detail, but
341// at the time of writing that would require more support from gomaasapi's
342// testing service than we have.
343func (suite *EnvironSuite) TestBootstrapSucceeds(c *C) {
344 suite.setupFakeTools(c)
345 env := suite.makeEnviron()
346 suite.testMAASObject.TestServer.NewNode(`{"system_id": "thenode", "hostname": "host"}`)
347 cert := []byte{1, 2, 3}
348 key := []byte{4, 5, 6}
349
350 err := env.Bootstrap(constraints.Value{}, cert, key)
351 c.Assert(err, IsNil)
352}
353
354func (suite *EnvironSuite) TestBootstrapFailsIfNoNodes(c *C) {
355 suite.setupFakeTools(c)
356 env := suite.makeEnviron()
357 cert := []byte{1, 2, 3}
358 key := []byte{4, 5, 6}
359 err := env.Bootstrap(constraints.Value{}, cert, key)
360 // Since there are no nodes, the attempt to allocate one returns a
361 // 409: Conflict.
362 c.Check(err, ErrorMatches, ".*409.*")
363}
364
365func (suite *EnvironSuite) TestBootstrapIntegratesWithEnvirons(c *C) {
366 suite.setupFakeTools(c)
367 env := suite.makeEnviron()
368 suite.testMAASObject.TestServer.NewNode(`{"system_id": "bootstrapnode", "hostname": "host"}`)
369
370 // environs.Bootstrap calls Environ.Bootstrap. This works.
371 err := environs.Bootstrap(env, constraints.Value{})
372 c.Assert(err, IsNil)
373}
374
375func (suite *EnvironSuite) TestAssignmentPolicy(c *C) {
376 env := suite.makeEnviron()
377
378 c.Check(env.AssignmentPolicy(), Equals, state.AssignUnused)
379}
0380
=== added file 'environs/maas/environprovider.go'
--- environs/maas/environprovider.go 1970-01-01 00:00:00 +0000
+++ environs/maas/environprovider.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,81 @@
1package maas
2
3import (
4 "launchpad.net/juju-core/environs"
5 "launchpad.net/juju-core/environs/config"
6 "launchpad.net/juju-core/log"
7 "launchpad.net/juju-core/state"
8)
9
10type maasEnvironProvider struct{}
11
12var _ environs.EnvironProvider = (*maasEnvironProvider)(nil)
13
14var providerInstance maasEnvironProvider
15
16func init() {
17 environs.RegisterProvider("maas", maasEnvironProvider{})
18}
19
20func (maasEnvironProvider) Open(cfg *config.Config) (environs.Environ, error) {
21 log.Debugf("environs/maas: opening environment %q.", cfg.Name())
22 return NewEnviron(cfg)
23}
24
25// Boilerplate config YAML. Don't mess with the indentation or add newlines!
26const boilerplateYAML = `maas:
27 type: maas
28 # Change this to where your MAAS server lives. It must specify the base path.
29 maas-server: 'http://192.168.1.1/MAAS/'
30 maas-oauth: '<add your OAuth credentials from MAAS here>'
31 admin-secret: {{rand}}
32 default-series: precise
33 authorized-keys-path: ~/.ssh/authorized_keys # or any file you want.
34 # Or:
35 # authorized-keys: ssh-rsa keymaterialhere
36`
37
38// BoilerplateConfig is specified in the EnvironProvider interface.
39func (maasEnvironProvider) BoilerplateConfig() string {
40 return boilerplateYAML
41}
42
43// SecretAttrs is specified in the EnvironProvider interface.
44func (prov maasEnvironProvider) SecretAttrs(cfg *config.Config) (map[string]interface{}, error) {
45 secretAttrs := make(map[string]interface{})
46 maasCfg, err := prov.newConfig(cfg)
47 if err != nil {
48 return nil, err
49 }
50 secretAttrs["maas-oauth"] = maasCfg.MAASOAuth()
51 return secretAttrs, nil
52}
53
54func (maasEnvironProvider) hostname() (string, error) {
55 info := machineInfo{}
56 err := info.load()
57 if err != nil {
58 return "", err
59 }
60 return info.Hostname, nil
61}
62
63// PublicAddress is specified in the EnvironProvider interface.
64func (prov maasEnvironProvider) PublicAddress() (string, error) {
65 return prov.hostname()
66}
67
68// PrivateAddress is specified in the EnvironProvider interface.
69func (prov maasEnvironProvider) PrivateAddress() (string, error) {
70 return prov.hostname()
71}
72
73// InstanceId is specified in the EnvironProvider interface.
74func (maasEnvironProvider) InstanceId() (state.InstanceId, error) {
75 info := machineInfo{}
76 err := info.load()
77 if err != nil {
78 return "", err
79 }
80 return state.InstanceId(info.InstanceId), nil
81}
082
=== added file 'environs/maas/environprovider_test.go'
--- environs/maas/environprovider_test.go 1970-01-01 00:00:00 +0000
+++ environs/maas/environprovider_test.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,94 @@
1package maas
2
3import (
4 "io/ioutil"
5 . "launchpad.net/gocheck"
6 "launchpad.net/goyaml"
7 "launchpad.net/juju-core/environs/config"
8 "launchpad.net/juju-core/state"
9)
10
11type EnvironProviderSuite struct {
12 ProviderSuite
13}
14
15var _ = Suite(new(EnvironProviderSuite))
16
17func (suite *EnvironProviderSuite) TestSecretAttrsReturnsSensitiveMAASAttributes(c *C) {
18 testJujuHome := c.MkDir()
19 defer config.SetJujuHome(config.SetJujuHome(testJujuHome))
20 const oauth = "aa:bb:cc"
21 attrs := map[string]interface{}{
22 "maas-oauth": oauth,
23 "maas-server": "http://maas.example.com/maas/",
24 "name": "wheee",
25 "type": "maas",
26 "authorized-keys": "I-am-not-a-real-key",
27 }
28 config, err := config.New(attrs)
29 c.Assert(err, IsNil)
30
31 secretAttrs, err := suite.environ.Provider().SecretAttrs(config)
32 c.Assert(err, IsNil)
33
34 expectedAttrs := map[string]interface{}{"maas-oauth": oauth}
35 c.Check(secretAttrs, DeepEquals, expectedAttrs)
36}
37
38// create a temporary file with the given content. The file will be cleaned
39// up at the end of the test calling this method.
40func createTempFile(c *C, content []byte) string {
41 file, err := ioutil.TempFile(c.MkDir(), "")
42 c.Assert(err, IsNil)
43 filename := file.Name()
44 err = ioutil.WriteFile(filename, content, 0644)
45 c.Assert(err, IsNil)
46 return filename
47}
48
49// InstanceId returns the instanceId of the machine read from the file
50// _MAASInstanceFilename.
51func (suite *EnvironProviderSuite) TestInstanceIdReadsInstanceIdFromMachineFile(c *C) {
52 instanceId := "instance-id"
53 info := machineInfo{instanceId, "hostname"}
54 yaml, err := goyaml.Marshal(info)
55 c.Assert(err, IsNil)
56 // Create a temporary file to act as the file where the instanceID
57 // is stored.
58 filename := createTempFile(c, yaml)
59 // "Monkey patch" the value of _MAASInstanceFilename with the path
60 // to the temporary file.
61 old_MAASInstanceFilename := _MAASInstanceFilename
62 _MAASInstanceFilename = filename
63 defer func() { _MAASInstanceFilename = old_MAASInstanceFilename }()
64
65 provider := suite.environ.Provider()
66 returnedInstanceId, err := provider.InstanceId()
67 c.Assert(err, IsNil)
68 c.Check(returnedInstanceId, Equals, state.InstanceId(instanceId))
69}
70
71// PublicAddress and PrivateAddress return the hostname of the machine read
72// from the file _MAASInstanceFilename.
73func (suite *EnvironProviderSuite) TestPrivatePublicAddressReadsHostnameFromMachineFile(c *C) {
74 hostname := "myhostname"
75 info := machineInfo{"instance-id", hostname}
76 yaml, err := goyaml.Marshal(info)
77 c.Assert(err, IsNil)
78 // Create a temporary file to act as the file where the instanceID
79 // is stored.
80 filename := createTempFile(c, yaml)
81 // "Monkey patch" the value of _MAASInstanceFilename with the path
82 // to the temporary file.
83 old_MAASInstanceFilename := _MAASInstanceFilename
84 _MAASInstanceFilename = filename
85 defer func() { _MAASInstanceFilename = old_MAASInstanceFilename }()
86
87 provider := suite.environ.Provider()
88 publicAddress, err := provider.PublicAddress()
89 c.Assert(err, IsNil)
90 c.Check(publicAddress, Equals, hostname)
91 privateAddress, err := provider.PrivateAddress()
92 c.Assert(err, IsNil)
93 c.Check(privateAddress, Equals, hostname)
94}
095
=== added file 'environs/maas/instance.go'
--- environs/maas/instance.go 1970-01-01 00:00:00 +0000
+++ environs/maas/instance.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,63 @@
1package maas
2
3import (
4 "launchpad.net/gomaasapi"
5 "launchpad.net/juju-core/environs"
6 "launchpad.net/juju-core/log"
7 "launchpad.net/juju-core/state"
8 "launchpad.net/juju-core/state/api/params"
9)
10
11type maasInstance struct {
12 maasObject *gomaasapi.MAASObject
13 environ *maasEnviron
14}
15
16var _ environs.Instance = (*maasInstance)(nil)
17
18func (instance *maasInstance) Id() state.InstanceId {
19 // Use the node's 'resource_uri' value.
20 return state.InstanceId((*instance.maasObject).URI().String())
21}
22
23// refreshInstance refreshes the instance with the most up-to-date information
24// from the MAAS server.
25func (instance *maasInstance) refreshInstance() error {
26 insts, err := instance.environ.Instances([]state.InstanceId{instance.Id()})
27 if err != nil {
28 return err
29 }
30 newMaasObject := insts[0].(*maasInstance).maasObject
31 instance.maasObject = newMaasObject
32 return nil
33}
34
35func (instance *maasInstance) DNSName() (string, error) {
36 hostname, err := (*instance.maasObject).GetField("hostname")
37 if err != nil {
38 return "", err
39 }
40 return hostname, nil
41}
42
43func (instance *maasInstance) WaitDNSName() (string, error) {
44 // A MAAS nodes gets his DNS name when it's created. WaitDNSName,
45 // (same as DNSName) just returns the hostname of the node.
46 return instance.DNSName()
47}
48
49// MAAS does not do firewalling so these port methods do nothing.
50func (instance *maasInstance) OpenPorts(machineId string, ports []params.Port) error {
51 log.Debugf("environs/maas: unimplemented OpenPorts() called")
52 return nil
53}
54
55func (instance *maasInstance) ClosePorts(machineId string, ports []params.Port) error {
56 log.Debugf("environs/maas: unimplemented ClosePorts() called")
57 return nil
58}
59
60func (instance *maasInstance) Ports(machineId string) ([]params.Port, error) {
61 log.Debugf("environs/maas: unimplemented Ports() called")
62 return []params.Port{}, nil
63}
064
=== added file 'environs/maas/instance_test.go'
--- environs/maas/instance_test.go 1970-01-01 00:00:00 +0000
+++ environs/maas/instance_test.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,51 @@
1package maas
2
3import (
4 . "launchpad.net/gocheck"
5)
6
7type InstanceTest struct {
8 ProviderSuite
9}
10
11var _ = Suite(&InstanceTest{})
12
13func (s *InstanceTest) TestId(c *C) {
14 jsonValue := `{"system_id": "system_id", "test": "test"}`
15 obj := s.testMAASObject.TestServer.NewNode(jsonValue)
16 resourceURI, _ := obj.GetField("resource_uri")
17 instance := maasInstance{&obj, s.environ}
18
19 c.Check(string(instance.Id()), Equals, resourceURI)
20}
21
22func (s *InstanceTest) TestRefreshInstance(c *C) {
23 jsonValue := `{"system_id": "system_id", "test": "test"}`
24 obj := s.testMAASObject.TestServer.NewNode(jsonValue)
25 s.testMAASObject.TestServer.ChangeNode("system_id", "test2", "test2")
26 instance := maasInstance{&obj, s.environ}
27
28 err := instance.refreshInstance()
29
30 c.Check(err, IsNil)
31 testField, err := (*instance.maasObject).GetField("test2")
32 c.Check(err, IsNil)
33 c.Check(testField, Equals, "test2")
34}
35
36func (s *InstanceTest) TestDNSName(c *C) {
37 jsonValue := `{"hostname": "DNS name", "system_id": "system_id"}`
38 obj := s.testMAASObject.TestServer.NewNode(jsonValue)
39 instance := maasInstance{&obj, s.environ}
40
41 dnsName, err := instance.DNSName()
42
43 c.Check(err, IsNil)
44 c.Check(dnsName, Equals, "DNS name")
45
46 // WaitDNSName() currently simply calls DNSName().
47 dnsName, err = instance.WaitDNSName()
48
49 c.Check(err, IsNil)
50 c.Check(dnsName, Equals, "DNS name")
51}
052
=== added file 'environs/maas/maas_test.go'
--- environs/maas/maas_test.go 1970-01-01 00:00:00 +0000
+++ environs/maas/maas_test.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,32 @@
1package maas
2
3import (
4 . "launchpad.net/gocheck"
5 "launchpad.net/gomaasapi"
6 "testing"
7)
8
9func TestMAAS(t *testing.T) {
10 TestingT(t)
11}
12
13type ProviderSuite struct {
14 environ *maasEnviron
15 testMAASObject *gomaasapi.TestMAASObject
16}
17
18var _ = Suite(&ProviderSuite{})
19
20func (s *ProviderSuite) SetUpSuite(c *C) {
21 TestMAASObject := gomaasapi.NewTestMAAS("1.0")
22 s.testMAASObject = TestMAASObject
23 s.environ = &maasEnviron{name: "test env", maasClientUnlocked: &TestMAASObject.MAASObject}
24}
25
26func (s *ProviderSuite) TearDownTest(c *C) {
27 s.testMAASObject.TestServer.Clear()
28}
29
30func (s *ProviderSuite) TearDownSuite(c *C) {
31 s.testMAASObject.Close()
32}
033
=== added file 'environs/maas/state.go'
--- environs/maas/state.go 1970-01-01 00:00:00 +0000
+++ environs/maas/state.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,47 @@
1package maas
2
3import (
4 "bytes"
5 "fmt"
6 "io/ioutil"
7 "launchpad.net/goyaml"
8 "launchpad.net/juju-core/state"
9)
10
11const stateFile = "provider-state"
12
13// Persistent environment state. An environment needs to know what instances
14// it manages.
15type bootstrapState struct {
16 StateInstances []state.InstanceId `yaml:"state-instances"`
17}
18
19// saveState writes the environment's state to the provider-state file stored
20// in the environment's storage.
21func (env *maasEnviron) saveState(state *bootstrapState) error {
22 data, err := goyaml.Marshal(state)
23 if err != nil {
24 return err
25 }
26 buf := bytes.NewBuffer(data)
27 return env.Storage().Put(stateFile, buf, int64(len(data)))
28}
29
30// loadState reads the environment's state from storage.
31func (env *maasEnviron) loadState() (*bootstrapState, error) {
32 r, err := env.Storage().Get(stateFile)
33 if err != nil {
34 return nil, err
35 }
36 defer r.Close()
37 data, err := ioutil.ReadAll(r)
38 if err != nil {
39 return nil, fmt.Errorf("error reading %q: %v", stateFile, err)
40 }
41 var state bootstrapState
42 err = goyaml.Unmarshal(data, &state)
43 if err != nil {
44 return nil, fmt.Errorf("error unmarshalling %q: %v", stateFile, err)
45 }
46 return &state, nil
47}
048
=== added file 'environs/maas/state_test.go'
--- environs/maas/state_test.go 1970-01-01 00:00:00 +0000
+++ environs/maas/state_test.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,23 @@
1package maas
2
3import (
4 . "launchpad.net/gocheck"
5 "launchpad.net/juju-core/environs"
6)
7
8type StateSuite struct {
9 ProviderSuite
10}
11
12var _ = Suite(new(StateSuite))
13
14func (suite *StateSuite) TestLoadStateReturnsNotFoundPointerForMissingFile(c *C) {
15 serverURL := suite.testMAASObject.URL().String()
16 config := getTestConfig("loadState-test", serverURL, "a:b:c", "foo")
17 env, err := NewEnviron(config)
18 c.Assert(err, IsNil)
19
20 _, err = env.loadState()
21
22 c.Check(err, FitsTypeOf, &environs.NotFoundError{})
23}
024
=== added file 'environs/maas/storage.go'
--- environs/maas/storage.go 1970-01-01 00:00:00 +0000
+++ environs/maas/storage.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,207 @@
1package maas
2
3import (
4 "bytes"
5 "encoding/base64"
6 "fmt"
7 "io"
8 "io/ioutil"
9 "launchpad.net/gomaasapi"
10 "launchpad.net/juju-core/environs"
11 "net/url"
12 "sort"
13 "sync"
14)
15
16type maasStorage struct {
17 // Mutex protects the "*Unlocked" fields.
18 sync.Mutex
19
20 // The Environ that this Storage is for.
21 environUnlocked *maasEnviron
22
23 // Reference to the URL on the API where files are stored.
24 maasClientUnlocked gomaasapi.MAASObject
25}
26
27var _ environs.Storage = (*maasStorage)(nil)
28
29func NewStorage(env *maasEnviron) environs.Storage {
30 storage := new(maasStorage)
31 storage.environUnlocked = env
32 storage.maasClientUnlocked = env.getMAASClient().GetSubObject("files")
33 return storage
34}
35
36// getSnapshot returns a consistent copy of a maasStorage. Use this if you
37// need a consistent view of the object's entire state, without having to
38// lock the object the whole time.
39//
40// An easy mistake to make with "defer" is to keep holding a lock without
41// realizing it, while you go on to block on http requests or other slow
42// things that don't actually require the lock. In most cases you can just
43// create a snapshot first (releasing the lock immediately) and then do the
44// rest of the work with the snapshot.
45func (stor *maasStorage) getSnapshot() *maasStorage {
46 stor.Lock()
47 defer stor.Unlock()
48
49 return &maasStorage{
50 environUnlocked: stor.environUnlocked,
51 maasClientUnlocked: stor.maasClientUnlocked,
52 }
53}
54
55// addressFileObject creates a MAASObject pointing to a given file.
56// Takes out a lock on the storage object to get a consistent view.
57func (stor *maasStorage) addressFileObject(name string) gomaasapi.MAASObject {
58 stor.Lock()
59 defer stor.Unlock()
60 return stor.maasClientUnlocked.GetSubObject(name)
61}
62
63// retrieveFileObject retrieves the information of the named file, including
64// its download URL and its contents, as a MAASObject.
65//
66// This may return many different errors, but specifically, it returns
67// (a pointer to) environs.NotFoundError if the file did not exist.
68//
69// The function takes out a lock on the storage object.
70func (stor *maasStorage) retrieveFileObject(name string) (gomaasapi.MAASObject, error) {
71 obj, err := stor.addressFileObject(name).Get()
72 if err != nil {
73 noObj := gomaasapi.MAASObject{}
74 serverErr, ok := err.(gomaasapi.ServerError)
75 if ok && serverErr.StatusCode == 404 {
76 msg := fmt.Errorf("file '%s' not found", name)
77 return noObj, &environs.NotFoundError{msg}
78 }
79 msg := fmt.Errorf("could not access file '%s': %v", name, err)
80 return noObj, msg
81 }
82 return obj, nil
83}
84
85// Get is specified in the Storage interface.
86func (stor *maasStorage) Get(name string) (io.ReadCloser, error) {
87 fileObj, err := stor.retrieveFileObject(name)
88 if err != nil {
89 return nil, err
90 }
91 data, err := fileObj.GetField("content")
92 if err != nil {
93 return nil, fmt.Errorf("could not extract file content for %s: %v", name, err)
94 }
95 buf, err := base64.StdEncoding.DecodeString(data)
96 if err != nil {
97 return nil, fmt.Errorf("bad data in file '%s': %v", name, err)
98 }
99 return ioutil.NopCloser(bytes.NewReader(buf)), nil
100}
101
102// extractFilenames returns the filenames from a "list" operation on the
103// MAAS API, sorted by name.
104func (stor *maasStorage) extractFilenames(listResult gomaasapi.JSONObject) ([]string, error) {
105 list, err := listResult.GetArray()
106 if err != nil {
107 return nil, err
108 }
109 result := make([]string, len(list))
110 for index, entry := range list {
111 file, err := entry.GetMap()
112 if err != nil {
113 return nil, err
114 }
115 filename, err := file["filename"].GetString()
116 if err != nil {
117 return nil, err
118 }
119 result[index] = filename
120 }
121 sort.Strings(result)
122 return result, nil
123}
124
125// List is specified in the Storage interface.
126func (stor *maasStorage) List(prefix string) ([]string, error) {
127 params := make(url.Values)
128 params.Add("prefix", prefix)
129 snapshot := stor.getSnapshot()
130 obj, err := snapshot.maasClientUnlocked.CallGet("list", params)
131 if err != nil {
132 return nil, err
133 }
134 return snapshot.extractFilenames(obj)
135}
136
137// URL is specified in the Storage interface.
138func (stor *maasStorage) URL(name string) (string, error) {
139 fileObj, err := stor.retrieveFileObject(name)
140 if err != nil {
141 return "", err
142 }
143 uri, err := fileObj.GetField("anon_resource_uri")
144 if err != nil {
145 msg := fmt.Errorf("could not get file's download URL (may be an outdated MAAS): %s", err)
146 return "", msg
147 }
148
149 partialURL, err := url.Parse(uri)
150 if err != nil {
151 return "", err
152 }
153 fullURL := fileObj.URL().ResolveReference(partialURL)
154 return fullURL.String(), nil
155}
156
157// Put is specified in the Storage interface.
158func (stor *maasStorage) Put(name string, r io.Reader, length int64) error {
159 data, err := ioutil.ReadAll(io.LimitReader(r, length))
160 if err != nil {
161 return err
162 }
163 params := url.Values{"filename": {name}}
164 files := map[string][]byte{"file": data}
165 snapshot := stor.getSnapshot()
166 _, err = snapshot.maasClientUnlocked.CallPostFiles("add", params, files)
167 return err
168}
169
170// Remove is specified in the Storage interface.
171func (stor *maasStorage) Remove(name string) error {
172 // The only thing that can go wrong here, really, is that the file
173 // does not exist. But deletion is idempotent: deleting a file that
174 // is no longer there anyway is success, not failure.
175 stor.getSnapshot().maasClientUnlocked.GetSubObject(name).Delete()
176 return nil
177}
178
179func (stor *maasStorage) deleteAll() error {
180 names, err := stor.List("")
181 if err != nil {
182 return err
183 }
184 // Remove all the objects in parallel so that we incur less round-trips.
185 // If we're in danger of having hundreds of objects,
186 // we'll want to change this to limit the number
187 // of concurrent operations.
188 var wg sync.WaitGroup
189 wg.Add(len(names))
190 errc := make(chan error, len(names))
191 for _, name := range names {
192 name := name
193 go func() {
194 defer wg.Done()
195 if err := stor.Remove(name); err != nil {
196 errc <- err
197 }
198 }()
199 }
200 wg.Wait()
201 select {
202 case err := <-errc:
203 return fmt.Errorf("cannot delete all provider state: %v", err)
204 default:
205 }
206 return nil
207}
0208
=== added file 'environs/maas/storage_test.go'
--- environs/maas/storage_test.go 1970-01-01 00:00:00 +0000
+++ environs/maas/storage_test.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,385 @@
1package maas
2
3import (
4 "bytes"
5 "encoding/base64"
6 "io/ioutil"
7 . "launchpad.net/gocheck"
8 "launchpad.net/gomaasapi"
9 "launchpad.net/juju-core/environs"
10 "math/rand"
11 "net/http"
12 "net/url"
13 "sync"
14)
15
16type StorageSuite struct {
17 ProviderSuite
18}
19
20var _ = Suite(new(StorageSuite))
21
22// makeStorage creates a MAAS storage object for the running test.
23func (s *StorageSuite) makeStorage(name string) *maasStorage {
24 maasobj := s.testMAASObject.MAASObject
25 env := maasEnviron{name: name, maasClientUnlocked: &maasobj}
26 return NewStorage(&env).(*maasStorage)
27}
28
29// makeRandomBytes returns an array of arbitrary byte values.
30func makeRandomBytes(length int) []byte {
31 data := make([]byte, length)
32 for index := range data {
33 data[index] = byte(rand.Intn(256))
34 }
35 return data
36}
37
38// fakeStoredFile creates a file directly in the (simulated) MAAS file store.
39// It will contain an arbitrary amount of random data. The contents are also
40// returned.
41//
42// If you want properly random data here, initialize the randomizer first.
43// Or don't, if you want consistent (and debuggable) results.
44func (s *StorageSuite) fakeStoredFile(storage environs.Storage, name string) gomaasapi.MAASObject {
45 data := makeRandomBytes(rand.Intn(10))
46 return s.testMAASObject.TestServer.NewFile(name, data)
47}
48
49func (s *StorageSuite) TestGetSnapshotCreatesClone(c *C) {
50 original := s.makeStorage("storage-name")
51 snapshot := original.getSnapshot()
52 c.Check(snapshot.environUnlocked, Equals, original.environUnlocked)
53 c.Check(snapshot.maasClientUnlocked.URL().String(), Equals, original.maasClientUnlocked.URL().String())
54 // Snapshotting locks the original internally, but does not leave
55 // either the original or the snapshot locked.
56 unlockedMutexValue := sync.Mutex{}
57 c.Check(original.Mutex, Equals, unlockedMutexValue)
58 c.Check(snapshot.Mutex, Equals, unlockedMutexValue)
59}
60
61func (s *StorageSuite) TestGetRetrievesFile(c *C) {
62 const filename = "stored-data"
63 storage := s.makeStorage("get-retrieves-file")
64 file := s.fakeStoredFile(storage, filename)
65 base64Content, err := file.GetField("content")
66 c.Assert(err, IsNil)
67 content, err := base64.StdEncoding.DecodeString(base64Content)
68 c.Assert(err, IsNil)
69
70 reader, err := storage.Get(filename)
71 c.Assert(err, IsNil)
72 defer reader.Close()
73
74 buf, err := ioutil.ReadAll(reader)
75 c.Assert(err, IsNil)
76 c.Check(len(buf), Equals, len(content))
77 c.Check(buf, DeepEquals, content)
78}
79
80func (s *StorageSuite) TestRetrieveFileObjectReturnsFileObject(c *C) {
81 const filename = "myfile"
82 stor := s.makeStorage("rfo-test")
83 file := s.fakeStoredFile(stor, filename)
84 fileURI, err := file.GetField("anon_resource_uri")
85 c.Assert(err, IsNil)
86 fileContent, err := file.GetField("content")
87 c.Assert(err, IsNil)
88
89 obj, err := stor.retrieveFileObject(filename)
90 c.Assert(err, IsNil)
91
92 uri, err := obj.GetField("anon_resource_uri")
93 c.Assert(err, IsNil)
94 c.Check(uri, Equals, fileURI)
95 content, err := obj.GetField("content")
96 c.Check(content, Equals, fileContent)
97}
98
99func (s *StorageSuite) TestRetrieveFileObjectReturnsNotFoundForMissingFile(c *C) {
100 stor := s.makeStorage("rfo-test")
101 _, err := stor.retrieveFileObject("nonexistent-file")
102 c.Assert(err, NotNil)
103 c.Check(err, FitsTypeOf, &environs.NotFoundError{})
104}
105
106func (s *StorageSuite) TestRetrieveFileObjectEscapesName(c *C) {
107 const filename = "#a?b c&d%e!"
108 data := []byte("File contents here")
109 stor := s.makeStorage("rfo-test")
110 err := stor.Put(filename, bytes.NewReader(data), int64(len(data)))
111 c.Assert(err, IsNil)
112
113 obj, err := stor.retrieveFileObject(filename)
114 c.Assert(err, IsNil)
115
116 base64Content, err := obj.GetField("content")
117 c.Assert(err, IsNil)
118 content, err := base64.StdEncoding.DecodeString(base64Content)
119 c.Assert(err, IsNil)
120 c.Check(content, DeepEquals, data)
121}
122
123func (s *StorageSuite) TestFileContentsAreBinary(c *C) {
124 const filename = "myfile.bin"
125 data := []byte{0, 1, 255, 2, 254, 3}
126 stor := s.makeStorage("binary-test")
127
128 err := stor.Put(filename, bytes.NewReader(data), int64(len(data)))
129 c.Assert(err, IsNil)
130 file, err := stor.Get(filename)
131 c.Assert(err, IsNil)
132 content, err := ioutil.ReadAll(file)
133 c.Assert(err, IsNil)
134
135 c.Check(content, DeepEquals, data)
136}
137
138func (s *StorageSuite) TestGetReturnsNotFoundErrorIfNotFound(c *C) {
139 const filename = "lost-data"
140 storage := NewStorage(s.environ)
141 _, err := storage.Get(filename)
142 c.Assert(err, FitsTypeOf, &environs.NotFoundError{})
143}
144
145func (s *StorageSuite) TestListReturnsEmptyIfNoFilesStored(c *C) {
146 storage := NewStorage(s.environ)
147 listing, err := storage.List("")
148 c.Assert(err, IsNil)
149 c.Check(listing, DeepEquals, []string{})
150}
151
152func (s *StorageSuite) TestListReturnsAllFilesIfPrefixEmpty(c *C) {
153 storage := NewStorage(s.environ)
154 files := []string{"1a", "2b", "3c"}
155 for _, name := range files {
156 s.fakeStoredFile(storage, name)
157 }
158
159 listing, err := storage.List("")
160 c.Assert(err, IsNil)
161 c.Check(listing, DeepEquals, files)
162}
163
164func (s *StorageSuite) TestListSortsResults(c *C) {
165 storage := NewStorage(s.environ)
166 files := []string{"4d", "1a", "3c", "2b"}
167 for _, name := range files {
168 s.fakeStoredFile(storage, name)
169 }
170
171 listing, err := storage.List("")
172 c.Assert(err, IsNil)
173 c.Check(listing, DeepEquals, []string{"1a", "2b", "3c", "4d"})
174}
175
176func (s *StorageSuite) TestListReturnsNoFilesIfNoFilesMatchPrefix(c *C) {
177 storage := NewStorage(s.environ)
178 s.fakeStoredFile(storage, "foo")
179
180 listing, err := storage.List("bar")
181 c.Assert(err, IsNil)
182 c.Check(listing, DeepEquals, []string{})
183}
184
185func (s *StorageSuite) TestListReturnsOnlyFilesWithMatchingPrefix(c *C) {
186 storage := NewStorage(s.environ)
187 s.fakeStoredFile(storage, "abc")
188 s.fakeStoredFile(storage, "xyz")
189
190 listing, err := storage.List("x")
191 c.Assert(err, IsNil)
192 c.Check(listing, DeepEquals, []string{"xyz"})
193}
194
195func (s *StorageSuite) TestListMatchesPrefixOnly(c *C) {
196 storage := NewStorage(s.environ)
197 s.fakeStoredFile(storage, "abc")
198 s.fakeStoredFile(storage, "xabc")
199
200 listing, err := storage.List("a")
201 c.Assert(err, IsNil)
202 c.Check(listing, DeepEquals, []string{"abc"})
203}
204
205func (s *StorageSuite) TestListOperatesOnFlatNamespace(c *C) {
206 storage := NewStorage(s.environ)
207 s.fakeStoredFile(storage, "a/b/c/d")
208
209 listing, err := storage.List("a/b")
210 c.Assert(err, IsNil)
211 c.Check(listing, DeepEquals, []string{"a/b/c/d"})
212}
213
214// getFileAtURL requests, and returns, the file at the given URL.
215func getFileAtURL(fileURL string) ([]byte, error) {
216 response, err := http.Get(fileURL)
217 if err != nil {
218 return nil, err
219 }
220 body, err := ioutil.ReadAll(response.Body)
221 if err != nil {
222 return nil, err
223 }
224 return body, nil
225}
226
227func (s *StorageSuite) TestURLReturnsURLCorrespondingToFile(c *C) {
228 const filename = "my-file.txt"
229 storage := NewStorage(s.environ).(*maasStorage)
230 file := s.fakeStoredFile(storage, filename)
231 // The file contains an anon_resource_uri, which lacks a network part
232 // (but will probably contain a query part). anonURL will be the
233 // file's full URL.
234 anonURI, err := file.GetField("anon_resource_uri")
235 c.Assert(err, IsNil)
236 parsedURI, err := url.Parse(anonURI)
237 c.Assert(err, IsNil)
238 anonURL := storage.maasClientUnlocked.URL().ResolveReference(parsedURI)
239 c.Assert(err, IsNil)
240
241 fileURL, err := storage.URL(filename)
242 c.Assert(err, IsNil)
243
244 c.Check(fileURL, NotNil)
245 c.Check(fileURL, Equals, anonURL.String())
246}
247
248func (s *StorageSuite) TestPutStoresRetrievableFile(c *C) {
249 const filename = "broken-toaster.jpg"
250 contents := []byte("Contents here")
251 length := int64(len(contents))
252 storage := NewStorage(s.environ)
253
254 err := storage.Put(filename, bytes.NewReader(contents), length)
255
256 reader, err := storage.Get(filename)
257 c.Assert(err, IsNil)
258 defer reader.Close()
259
260 buf, err := ioutil.ReadAll(reader)
261 c.Assert(err, IsNil)
262 c.Check(buf, DeepEquals, contents)
263}
264
265func (s *StorageSuite) TestPutOverwritesFile(c *C) {
266 const filename = "foo.bar"
267 storage := NewStorage(s.environ)
268 s.fakeStoredFile(storage, filename)
269 newContents := []byte("Overwritten")
270
271 err := storage.Put(filename, bytes.NewReader(newContents), int64(len(newContents)))
272 c.Assert(err, IsNil)
273
274 reader, err := storage.Get(filename)
275 c.Assert(err, IsNil)
276 defer reader.Close()
277
278 buf, err := ioutil.ReadAll(reader)
279 c.Assert(err, IsNil)
280 c.Check(len(buf), Equals, len(newContents))
281 c.Check(buf, DeepEquals, newContents)
282}
283
284func (s *StorageSuite) TestPutStopsAtGivenLength(c *C) {
285 const filename = "xyzzyz.2.xls"
286 const length = 5
287 contents := []byte("abcdefghijklmnopqrstuvwxyz")
288 storage := NewStorage(s.environ)
289
290 err := storage.Put(filename, bytes.NewReader(contents), length)
291 c.Assert(err, IsNil)
292
293 reader, err := storage.Get(filename)
294 c.Assert(err, IsNil)
295 defer reader.Close()
296
297 buf, err := ioutil.ReadAll(reader)
298 c.Assert(err, IsNil)
299 c.Check(len(buf), Equals, length)
300}
301
302func (s *StorageSuite) TestPutToExistingFileTruncatesAtGivenLength(c *C) {
303 const filename = "a-file-which-is-mine"
304 oldContents := []byte("abcdefghijklmnopqrstuvwxyz")
305 newContents := []byte("xyz")
306 storage := NewStorage(s.environ)
307 err := storage.Put(filename, bytes.NewReader(oldContents), int64(len(oldContents)))
308 c.Assert(err, IsNil)
309
310 err = storage.Put(filename, bytes.NewReader(newContents), int64(len(newContents)))
311 c.Assert(err, IsNil)
312
313 reader, err := storage.Get(filename)
314 c.Assert(err, IsNil)
315 defer reader.Close()
316
317 buf, err := ioutil.ReadAll(reader)
318 c.Assert(err, IsNil)
319 c.Check(len(buf), Equals, len(newContents))
320 c.Check(buf, DeepEquals, newContents)
321}
322
323func (s *StorageSuite) TestRemoveDeletesFile(c *C) {
324 const filename = "doomed.txt"
325 storage := NewStorage(s.environ)
326 s.fakeStoredFile(storage, filename)
327
328 err := storage.Remove(filename)
329 c.Assert(err, IsNil)
330
331 _, err = storage.Get(filename)
332 c.Assert(err, FitsTypeOf, &environs.NotFoundError{})
333
334 listing, err := storage.List(filename)
335 c.Assert(err, IsNil)
336 c.Assert(listing, DeepEquals, []string{})
337}
338
339func (s *StorageSuite) TestRemoveIsIdempotent(c *C) {
340 const filename = "half-a-file"
341 storage := NewStorage(s.environ)
342 s.fakeStoredFile(storage, filename)
343
344 err := storage.Remove(filename)
345 c.Assert(err, IsNil)
346
347 err = storage.Remove(filename)
348 c.Assert(err, IsNil)
349}
350
351func (s *StorageSuite) TestNamesMayHaveSlashes(c *C) {
352 const filename = "name/with/slashes"
353 content := []byte("File contents")
354 storage := NewStorage(s.environ)
355
356 err := storage.Put(filename, bytes.NewReader(content), int64(len(content)))
357 c.Assert(err, IsNil)
358
359 // There's not much we can say about the anonymous URL, except that
360 // we get one.
361 anonURL, err := storage.URL(filename)
362 c.Assert(err, IsNil)
363 c.Check(anonURL, Matches, "http[s]*://.*")
364
365 reader, err := storage.Get(filename)
366 c.Assert(err, IsNil)
367 defer reader.Close()
368 data, err := ioutil.ReadAll(reader)
369 c.Assert(err, IsNil)
370 c.Check(data, DeepEquals, content)
371}
372
373func (s *StorageSuite) TestDeleteAllDeletesAllFiles(c *C) {
374 storage := s.makeStorage("get-retrieves-file")
375 const filename1 = "stored-data1"
376 s.fakeStoredFile(storage, filename1)
377 const filename2 = "stored-data2"
378 s.fakeStoredFile(storage, filename2)
379
380 err := storage.deleteAll()
381 c.Assert(err, IsNil)
382 listing, err := storage.List("")
383 c.Assert(err, IsNil)
384 c.Assert(listing, DeepEquals, []string{})
385}
0386
=== added file 'environs/maas/util.go'
--- environs/maas/util.go 1970-01-01 00:00:00 +0000
+++ environs/maas/util.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,83 @@
1package maas
2
3import (
4 "fmt"
5 "launchpad.net/goyaml"
6 cloudinit_core "launchpad.net/juju-core/cloudinit"
7 "launchpad.net/juju-core/environs/cloudinit"
8 "launchpad.net/juju-core/log"
9 "launchpad.net/juju-core/state"
10 "launchpad.net/juju-core/trivial"
11 "net/url"
12 "strings"
13)
14
15// extractSystemId extracts the 'system_id' part from an InstanceId.
16// "/MAAS/api/1.0/nodes/system_id/" => "system_id"
17func extractSystemId(instanceId state.InstanceId) string {
18 trimmed := strings.TrimRight(string(instanceId), "/")
19 split := strings.Split(trimmed, "/")
20 return split[len(split)-1]
21}
22
23// getSystemIdValues returns a url.Values object with all the 'system_ids'
24// from the given instanceIds stored under the key 'id'. This is used
25// to filter out instances when listing the nodes objects.
26func getSystemIdValues(instanceIds []state.InstanceId) url.Values {
27 values := url.Values{}
28 for _, instanceId := range instanceIds {
29 values.Add("id", extractSystemId(instanceId))
30 }
31 return values
32}
33
34// userData returns a zipped cloudinit config.
35func userData(cfg *cloudinit.MachineConfig, scripts ...string) ([]byte, error) {
36 cloudcfg := cloudinit_core.New()
37 for _, script := range scripts {
38 cloudcfg.AddRunCmd(script)
39 }
40 cloudcfg, err := cloudinit.Configure(cfg, cloudcfg)
41 if err != nil {
42 return nil, err
43 }
44 data, err := cloudcfg.Render()
45 if err != nil {
46 return nil, err
47 }
48 cdata := trivial.Gzip(data)
49 log.Debugf("environs/maas: maas user data; %d bytes", len(cdata))
50 return cdata, nil
51}
52
53// machineInfo is the structure used to pass information between the provider
54// and the agent running on a node.
55// When a node is started, the provider code creates a machineInfo object
56// containing information about the node being started and configures
57// cloudinit to get a YAML representation of that object written on the node's
58// filesystem during its first startup. That file is then read by the juju
59// agent running on the node and converted back into a machineInfo object.
60type machineInfo struct {
61 InstanceId string `yaml:,omitempty`
62 Hostname string `yaml:,omitempty`
63}
64
65var _MAASInstanceFilename = jujuDataDir + "/MAASmachine.txt"
66
67// cloudinitRunCmd returns the shell command that, when run, will create the
68// "machine info" file containing the instanceId and the hostname of a machine.
69// That command is destined to be used by cloudinit.
70func (info *machineInfo) cloudinitRunCmd() (string, error) {
71 yaml, err := goyaml.Marshal(info)
72 if err != nil {
73 return "", err
74 }
75 script := fmt.Sprintf(`mkdir -p %s; echo -n %s > %s`, trivial.ShQuote(jujuDataDir), trivial.ShQuote(string(yaml)), trivial.ShQuote(_MAASInstanceFilename))
76 return script, nil
77}
78
79// load loads the "machine info" file and parse the content into the info
80// object.
81func (info *machineInfo) load() error {
82 return trivial.ReadYaml(_MAASInstanceFilename, info)
83}
084
=== added file 'environs/maas/util_test.go'
--- environs/maas/util_test.go 1970-01-01 00:00:00 +0000
+++ environs/maas/util_test.go 2013-04-12 10:00:43 +0000
@@ -0,0 +1,129 @@
1package maas
2
3import (
4 "fmt"
5 . "launchpad.net/gocheck"
6 "launchpad.net/goyaml"
7 "launchpad.net/juju-core/environs/cloudinit"
8 "launchpad.net/juju-core/environs/config"
9 "launchpad.net/juju-core/state"
10 "launchpad.net/juju-core/state/api"
11 "launchpad.net/juju-core/testing"
12 "launchpad.net/juju-core/trivial"
13 "launchpad.net/juju-core/version"
14)
15
16type UtilSuite struct{}
17
18var _ = Suite(&UtilSuite{})
19
20func (s *UtilSuite) TestExtractSystemId(c *C) {
21 instanceId := state.InstanceId("/MAAS/api/1.0/nodes/system_id/")
22
23 systemId := extractSystemId(instanceId)
24
25 c.Check(systemId, Equals, "system_id")
26}
27
28func (s *UtilSuite) TestGetSystemIdValues(c *C) {
29 instanceId1 := state.InstanceId("/MAAS/api/1.0/nodes/system_id1/")
30 instanceId2 := state.InstanceId("/MAAS/api/1.0/nodes/system_id2/")
31 instanceIds := []state.InstanceId{instanceId1, instanceId2}
32
33 values := getSystemIdValues(instanceIds)
34
35 c.Check(values["id"], DeepEquals, []string{"system_id1", "system_id2"})
36}
37
38func (s *UtilSuite) TestUserData(c *C) {
39 testJujuHome := c.MkDir()
40 defer config.SetJujuHome(config.SetJujuHome(testJujuHome))
41 tools := &state.Tools{
42 URL: "http://foo.com/tools/juju1.2.3-linux-amd64.tgz",
43 Binary: version.MustParseBinary("1.2.3-linux-amd64"),
44 }
45 envConfig, err := config.New(map[string]interface{}{
46 "type": "maas",
47 "name": "foo",
48 "default-series": "series",
49 "authorized-keys": "keys",
50 "ca-cert": testing.CACert,
51 })
52 c.Assert(err, IsNil)
53
54 cfg := &cloudinit.MachineConfig{
55 MachineId: "10",
56 MachineNonce: "5432",
57 Tools: tools,
58 StateServerCert: []byte(testing.ServerCert),
59 StateServerKey: []byte(testing.ServerKey),
60 StateInfo: &state.Info{
61 Password: "pw1",
62 CACert: []byte("CA CERT\n" + testing.CACert),
63 },
64 APIInfo: &api.Info{
65 Password: "pw2",
66 CACert: []byte("CA CERT\n" + testing.CACert),
67 },
68 DataDir: jujuDataDir,
69 MongoPort: mgoPort,
70 Config: envConfig,
71 APIPort: apiPort,
72 StateServer: true,
73 }
74 script1 := "script1"
75 script2 := "script2"
76 scripts := []string{script1, script2}
77 result, err := userData(cfg, scripts...)
78 c.Assert(err, IsNil)
79
80 unzipped, err := trivial.Gunzip(result)
81 c.Assert(err, IsNil)
82
83 config := make(map[interface{}]interface{})
84 err = goyaml.Unmarshal(unzipped, &config)
85 c.Assert(err, IsNil)
86
87 // Just check that the cloudinit config looks good.
88 c.Check(config["apt_upgrade"], Equals, true)
89 // The scripts given to userData where added as the first
90 // commands to be run.
91 runCmd := config["runcmd"].([]interface{})
92 c.Check(runCmd[0], Equals, script1)
93 c.Check(runCmd[1], Equals, script2)
94}
95
96func (s *UtilSuite) TestMachineInfoCloudinitRunCmd(c *C) {
97 instanceId := "instanceId"
98 hostname := "hostname"
99 filename := "path/to/file"
100 old_MAASInstanceFilename := _MAASInstanceFilename
101 _MAASInstanceFilename = filename
102 defer func() { _MAASInstanceFilename = old_MAASInstanceFilename }()
103 info := machineInfo{instanceId, hostname}
104
105 script, err := info.cloudinitRunCmd()
106
107 c.Assert(err, IsNil)
108 yaml, err := goyaml.Marshal(info)
109 c.Assert(err, IsNil)
110 expected := fmt.Sprintf("mkdir -p '%s'; echo -n '%s' > '%s'", jujuDataDir, yaml, filename)
111 c.Check(script, Equals, expected)
112}
113
114func (s *UtilSuite) TestMachineInfoLoad(c *C) {
115 instanceId := "instanceId"
116 hostname := "hostname"
117 yaml := fmt.Sprintf("instanceid: %s\nhostname: %s\n", instanceId, hostname)
118 filename := createTempFile(c, []byte(yaml))
119 old_MAASInstanceFilename := _MAASInstanceFilename
120 _MAASInstanceFilename = filename
121 defer func() { _MAASInstanceFilename = old_MAASInstanceFilename }()
122 info := machineInfo{}
123
124 err := info.load()
125
126 c.Assert(err, IsNil)
127 c.Check(info.InstanceId, Equals, instanceId)
128 c.Check(info.Hostname, Equals, hostname)
129}

Subscribers

People subscribed via source and target branches