Merge lp:~maas-maintainers/juju-core/maas-provider-skeleton into lp:~juju/juju-core/trunk

Proposed by Jeroen T. Vermeulen
Status: Merged
Approved by: Jeroen T. Vermeulen
Approved revision: no longer in the source branch.
Merged at revision: 1150
Proposed branch: lp:~maas-maintainers/juju-core/maas-provider-skeleton
Merge into: lp:~juju/juju-core/trunk
Diff against target: 2475 lines (+2313/-13)
21 files modified
cmd/builddb/main.go (+4/-2)
cmd/juju/main.go (+2/-5)
cmd/jujud/main.go (+2/-4)
environs/all/all.go (+8/-0)
environs/cloudinit/cloudinit.go (+5/-2)
environs/cloudinit/cloudinit_test.go (+33/-0)
environs/maas/config.go (+72/-0)
environs/maas/config_test.go (+94/-0)
environs/maas/environ.go (+519/-0)
environs/maas/environ_test.go (+379/-0)
environs/maas/environprovider.go (+81/-0)
environs/maas/environprovider_test.go (+94/-0)
environs/maas/instance.go (+63/-0)
environs/maas/instance_test.go (+51/-0)
environs/maas/maas_test.go (+32/-0)
environs/maas/state.go (+47/-0)
environs/maas/state_test.go (+23/-0)
environs/maas/storage.go (+207/-0)
environs/maas/storage_test.go (+385/-0)
environs/maas/util.go (+83/-0)
environs/maas/util_test.go (+129/-0)
To merge this branch: bzr merge lp:~maas-maintainers/juju-core/maas-provider-skeleton
Reviewer Review Type Date Requested Status
Julian Edwards (community) Approve
William Reade (community) Approve
Review via email: mp+157025@code.launchpad.net

Commit message

Feature branch integration: MAAS provider.

Description of the change

What you see here is not yet a complete implementation of the MAAS provider. It's an ongoing feature branch that we would like to integrate at this point, before the interfaces diverge further. We apologize for the long diff! We kept it separate so that you wouldn't have to review all the intermediate steps.

I also offer my apologies for submitting this for review in the usual Canonical style rather than going through lbox like the juju-core team does. I tried to submit through lbox but it seems to have problems with team-owned branches. We work against this branch as a team, and plan to go through the review collaboratively, so taking ownership was not the sensible thing to do. Pushing a personal copy (like lbox did at one point) seemed like a recipe for confusion.

We're on a very short schedule with a very long branch, so reviewers, we would like to ask you: please keep a careful distinction between things that absolutely need fixing before the code can land, and things that need fixing before the code can be used or maintained. Landing is the first priority, and we can track and resolve the other issues on an ongoing basis. Landing first will allow upcoming refactorings in the main codebase to take this new provider into account, and so save us all time.

As is the practice for other providers, we based much of this work on the EC2 provider. But MAAS is a different beast, and so we didn't copy as much code unchanged as you'd see in the OpenStack provider. With Bootstrap() in particular we had to simplify code in order to track what data goes where, and be sure it suited our needs. The same refactoring would work for the other providers, so we also posted it on the juju-dev mailing list.

A few more things you'll want to know before reading the code:
 * Instance IDs in the MAAS provider are implemented as node URIs on the MAAS API.
 * We require that each environment have its own MAAS user.
 * A MAAS provides file storage in separate per-user namespaces (there is no global namespace).

To post a comment you must log in.
Revision history for this message
William Reade (fwereade) wrote :

> so we didn't copy as much code unchanged as you'd see in the OpenStack provider.

/me cheers, dances, gets down to reviewing

Revision history for this message
William Reade (fwereade) wrote :

Preliminary comments on the Environ...

Any points not badged with [FIX] should be considered usability/maintainability issues of varying importance.

1) _ "launchpad.net/juju-core/environs/maas"

Consider adding an environs/all package, that registers all providers except the dummy.

2) envCfg.attrs = v.(map[string]interface{})

Hmm... the attrs are all stored in the Config() anyway. Why are we duplicating them?

3) s/ecfgMutext/ecfgMutex/

4) quiesceStateFile [FIX if not necessary]

Only necessary if maas storage is not consistent.

5) uploadTools() [FIX: not necessary]

(Bootstrap no longer accepts uploadTools)

6) for a := longAttempt.Start(); len(stateAddrs) == 0 && a.Next(); { [FIX as (4)]

7) var err error = fmt.Errorf("(no error)")

I think this would be much clearer if we just break on err == nil inside the loop

8) StateInfo()

can't this use instances()?

9) environs.HighestVersion in StartInstance [FIX unless not unique]

This is wrong, but it might still match other providers.

10) environ.StopInstances([]environs.Instance{&instance})

Shouldn't this be a release?

Revision history for this message
William Reade (fwereade) wrote :

11) "maas-oauth": schema.String(),

This should probably be a custom schema.Checker

12) validCfg, err := prov.Validate(cfg, nil)

thumper's just landing some changes around this area: there's another method you need to call in Validate, IIRC, that checks old/new configs match sanely (no changing env names, for example).

13) "ca-private-key": testing.CAKey,

I'd have authorized-keys and admin-secret in here rather than duplicated in every test. While you're at it, please add `"agent-version": "1.2.3",` or something (it'll fit better with an upcoming branch of mine).

14) inst, err := env.startBootstrapNode(tools,

An upcoming branch will change this almost beyond recognition, but only for the better. BTW, is constraints support expected imminently? If not, please log.Warningf that they're ignored.

15) env.ecfgUnlocked = ecfg

There should be some validation that the two configs match, per thumper's branch. If we forgot it elsewhere, which we might have done, don't consider this a blocker.

16) environ.StopInstances([]environs.Instance{&instance})

upgrading (10) to a FIX

17) flags := environs.HighestVersion

crossgrading (9) to a changing-beyond-recognition

18) _, err := blah();\n if err != nil {

several places:

if _, err := blah(); err != nil {

19) client := environ.maasClientUnlocked.GetSubObject("nodes/") [FIX]

The intended model is that Environ methods be safe from any goroutine. Use a maasClient() helper throughout to get a client safely. (see `// To properly observe e.storageUnlocked `)

It may very well be the case that there's actually no point making Environs goroutine-safe, but I'm not going to analyse that now.

20) errSetConfig := env.SetConfig(cfg2)

Changing names is actively bad behaviour, please stick to sensible changes like default-series and authorized-keys :).

21) package maas

I don't really like having all these tests internal -- most of them should really go into maas_test, I think

22) func fakeWriteCertAndKey(

unnecessary?

23) === added file 'environs/maas/environprovider.go'

(and similar small files with clear purposes) yay! I like this, we should do it more ;)

24) func createTempFile(c *C

create it inside a c.MkDir() and it'll be cleaned up for you

25) info.serializeYAML()

Please use goyaml.Marshal and export/tag the struct fields where necessary

~~~~~~~~~~~~~~

Thanks, and sorry for the delay. This is looking really solid so far; if it's actually spinning up environments and deploying units now, then I'd like to get it in as soon as possible and starting working on rationalizing the testing then. I have no problem with the coverage, just of where the tests are (and that none of the jujutest tests have been hooked).

review: Approve
Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

On point 12, thumper explains:

The new method is environs.config.Validate(). Our Validate() should upcall to that, so that it can do its basic non-specific verification of the proposed config changes. See ec2/config.go, and similar for the openstack and dummy providers.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

A note on point 15:

«15) env.ecfgUnlocked = ecfg

«There should be some validation that the two configs match, per thumper's branch. If we forgot it elsewhere, which we might have done, don't consider this a blocker.»

The preceding call to maasEnvironProvider.newConfig() calls Validate(), which in turn will validate the state transition. So that should do the trick, right?

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

A note on points 10 & 16:

«10) environ.StopInstances([]environs.Instance{&instance})

«Shouldn't this be a release?»

«16) environ.StopInstances([]environs.Instance{&instance})

«upgrading (10) to a FIX»

It may just be me but I have no idea at all what you mean! How is a release related to the action of stopping instances? I can't find anything in the other providers that might make it clearer either.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

Note about points 9 and 17:

«9) environs.HighestVersion in StartInstance [FIX unless not unique]

«This is wrong, but it might still match other providers.»

«17) flags := environs.HighestVersion

«crossgrading (9) to a changing-beyond-recognition»

It's still the same in the EC2 provider, so leaving this be for now.

Revision history for this message
Raphaël Badin (rvb) wrote :

> Thanks, and sorry for the delay. This is looking really solid so far; if it's actually spinning up environments and
> deploying units now, then I'd like to get it in as soon as possible and starting working on rationalizing the testing
> then. I have no problem with the coverage, just of where the tests are (and that none of the jujutest tests have been
> hooked).

Thanks a lot for the reviews. We're in the process of fixing the issues you mentioned in your reviews but yes, we've been able to deploy a working mediawiki service in the MAAS lab today: http://paste.ubuntu.com/5697858/

Revision history for this message
Raphaël Badin (rvb) wrote :

> 8) StateInfo()
>
> can't this use instances()?

Do you mean you'd prefer StateInfo() to use the internal method env.instances() instead of env.Instances()?

Revision history for this message
William Reade (fwereade) wrote :

26) forgot to mention this: env.Provider().(*maasEnvironProvider)

just use providerInstance?

27) var _ environs.EnvironProvider = (*maasEnvironProvider)(nil)

no fields, no pointer receivers -- you may as well assert that the value implements EnvironProvider, use the value everywhere, and just forget about the &s and *s.

Revision history for this message
William Reade (fwereade) wrote :

> A note on point 15:
>
> «15) env.ecfgUnlocked = ecfg
>
> «There should be some validation that the two configs match, per thumper's
> branch. If we forgot it elsewhere, which we might have done, don't consider
> this a blocker.»
>
> The preceding call to maasEnvironProvider.newConfig() calls Validate(), which
> in turn will validate the state transition. So that should do the trick,
> right?

I don't think so... ISTM that we would validate against nil, rather than against the old config.

Revision history for this message
William Reade (fwereade) wrote :

> A note on points 10 & 16:
>
> «10) environ.StopInstances([]environs.Instance{&instance})
>
> «Shouldn't this be a release?»
>
> «16) environ.StopInstances([]environs.Instance{&instance})
>
> «upgrading (10) to a FIX»
>
> It may just be me but I have no idea at all what you mean! How is a release
> related to the action of stopping instances? I can't find anything in the
> other providers that might make it clearer either.

Ah, sorry -- I'm saying that if start has failed we should certainly release the nodes back to the pool if we can, but it's confusing to stop something that was never actually started.

Revision history for this message
William Reade (fwereade) wrote :

> Note about points 9 and 17:
>
> «9) environs.HighestVersion in StartInstance [FIX unless not unique]
>
> «This is wrong, but it might still match other providers.»
>
> «17) flags := environs.HighestVersion
>
> «crossgrading (9) to a changing-beyond-recognition»
>
> It's still the same in the EC2 provider, so leaving this be for now.

It'll be a race to land then, still a WIP on my end. If you win, I'm pretty sure it'll be np for me; if I win it should be simple for you but I'm always here to discuss if there are complications.

Revision history for this message
William Reade (fwereade) wrote :

> > 8) StateInfo()
> >
> > can't this use instances()?
>
> Do you mean you'd prefer StateInfo() to use the internal method
> env.instances() instead of env.Instances()?

It was really just a question, there may be some detail that makes it inappropriate, but at first blush it seemed like the ideal candidate. Your judgment call.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

I don't think instances() would make a good replacement for Instances(). It's basically the same thing, except instances() has a quirk to its API that made it suit the implementation of multiple exported functions. Instances() says what we want more precisely and with less risk of accident.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

More notes! On point #2 this time:

«2) envCfg.attrs = v.(map[string]interface{})

«Hmm... the attrs are all stored in the Config() anyway. Why are we duplicating them?»

Is there any more convenient access path (from outside the config package) than through configchecker.Coerce(cfg.UnknownAttrs(), nil)? If we skip the Coerce(), my understanding is that the variables won't be appropriately typed. Not an issue now, but very possibly problematic in the future. And UnknownAttrs() tediously duplicates attributes, so we might as well keep the result around.

Actually we're just doing the same thing as the EC2 provider here, only more explicitly — so maybe this was always an issue but it went unnoticed before. Here's how the EC2 code sets the equivalent "attrs" field:

        ecfg := &environConfig{cfg, v.(map[string]interface{})}

(And "v" is produced in the same way that we do it.)

Does this make sense? Or is there still a change that needs to be made here?

Revision history for this message
Julian Edwards (julian-edwards) :
review: Approve
Revision history for this message
William Reade (fwereade) wrote :

> Actually we're just doing the same thing as the EC2 provider here, only more explicitly — so maybe this was always an issue but it went unnoticed before. Here's how the EC2 code sets the equivalent "attrs" field:
>
> ecfg := &environConfig{cfg, v.(map[string]interface{})}
>
> (And "v" is produced in the same way that we do it.)
>
> Does this make sense? Or is there still a change that needs to be made here?

I think it's probably OK. Which is to say that environment configuration remains hairy and problematic and weird, because it's trying to be at *least* 3 different things, all of which have different sets of valid keys and values in various situations, but it's not very amenable to fixing at the moment.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'cmd/builddb/main.go'
2--- cmd/builddb/main.go 2013-04-11 15:46:31 +0000
3+++ cmd/builddb/main.go 2013-04-12 10:00:43 +0000
4@@ -11,9 +11,11 @@
5 "os"
6 "path/filepath"
7 "time"
8+)
9
10- // Register the provider
11- _ "launchpad.net/juju-core/environs/ec2"
12+// Import the providers.
13+import (
14+ _ "launchpad.net/juju-core/environs/all"
15 )
16
17 func main() {
18
19=== modified file 'cmd/juju/main.go'
20--- cmd/juju/main.go 2013-04-09 22:56:52 +0000
21+++ cmd/juju/main.go 2013-04-12 10:00:43 +0000
22@@ -7,12 +7,9 @@
23 "os"
24 )
25
26-// When we import an environment provider implementation
27-// here, it will register itself with environs, and hence
28-// be available to the juju command.
29+// Import the providers.
30 import (
31- _ "launchpad.net/juju-core/environs/ec2"
32- _ "launchpad.net/juju-core/environs/openstack"
33+ _ "launchpad.net/juju-core/environs/all"
34 )
35
36 var jujuDoc = `
37
38=== modified file 'cmd/jujud/main.go'
39--- cmd/jujud/main.go 2013-03-26 20:17:23 +0000
40+++ cmd/jujud/main.go 2013-04-12 10:00:43 +0000
41@@ -9,11 +9,9 @@
42 "path/filepath"
43 )
44
45-// When we import an environment provider implementation
46-// here, it will register itself with environs.
47+// Import the providers.
48 import (
49- _ "launchpad.net/juju-core/environs/ec2"
50- _ "launchpad.net/juju-core/environs/openstack"
51+ _ "launchpad.net/juju-core/environs/all"
52 )
53
54 var jujudDoc = `
55
56=== added directory 'environs/all'
57=== added file 'environs/all/all.go'
58--- environs/all/all.go 1970-01-01 00:00:00 +0000
59+++ environs/all/all.go 2013-04-12 10:00:43 +0000
60@@ -0,0 +1,8 @@
61+package all
62+
63+// Register all the available providers.
64+import (
65+ _ "launchpad.net/juju-core/environs/ec2"
66+ _ "launchpad.net/juju-core/environs/maas"
67+ _ "launchpad.net/juju-core/environs/openstack"
68+)
69
70=== modified file 'environs/cloudinit/cloudinit.go'
71--- environs/cloudinit/cloudinit.go 2013-04-11 02:13:40 +0000
72+++ environs/cloudinit/cloudinit.go 2013-04-12 10:00:43 +0000
73@@ -98,11 +98,14 @@
74 }
75
76 func New(cfg *MachineConfig) (*cloudinit.Config, error) {
77+ c := cloudinit.New()
78+ return Configure(cfg, c)
79+}
80+
81+func Configure(cfg *MachineConfig, c *cloudinit.Config) (*cloudinit.Config, error) {
82 if err := verifyConfig(cfg); err != nil {
83 return nil, err
84 }
85- c := cloudinit.New()
86-
87 c.AddSSHAuthorizedKeys(cfg.AuthorizedKeys)
88 c.AddPackage("git")
89
90
91=== modified file 'environs/cloudinit/cloudinit_test.go'
92--- environs/cloudinit/cloudinit_test.go 2013-04-11 07:54:47 +0000
93+++ environs/cloudinit/cloudinit_test.go 2013-04-12 10:00:43 +0000
94@@ -4,6 +4,7 @@
95 "encoding/base64"
96 . "launchpad.net/gocheck"
97 "launchpad.net/goyaml"
98+ cloudinit_core "launchpad.net/juju-core/cloudinit"
99 "launchpad.net/juju-core/constraints"
100 "launchpad.net/juju-core/environs/cloudinit"
101 "launchpad.net/juju-core/environs/config"
102@@ -264,6 +265,38 @@
103 }
104 }
105
106+func (*cloudinitSuite) TestCloudInitConfigure(c *C) {
107+ for i, test := range cloudinitTests {
108+ test.cfg.Config = minimalConfig(c)
109+ c.Logf("test %d (Configure)", i)
110+ cloudcfg := cloudinit_core.New()
111+ ci, err := cloudinit.Configure(&test.cfg, cloudcfg)
112+ c.Assert(err, IsNil)
113+ c.Check(ci, NotNil)
114+ }
115+}
116+
117+func (*cloudinitSuite) TestCloudInitConfigureUsesGivenConfig(c *C) {
118+ // Create a simple cloudinit config with a 'runcmd' statement.
119+ cloudcfg := cloudinit_core.New()
120+ script := "test script"
121+ cloudcfg.AddRunCmd(script)
122+ cloudinitTests[0].cfg.Config = minimalConfig(c)
123+ ci, err := cloudinit.Configure(&cloudinitTests[0].cfg, cloudcfg)
124+ c.Assert(err, IsNil)
125+ c.Check(ci, NotNil)
126+ data, err := ci.Render()
127+ c.Assert(err, IsNil)
128+
129+ ciContent := make(map[interface{}]interface{})
130+ err = goyaml.Unmarshal(data, &ciContent)
131+ c.Assert(err, IsNil)
132+ // The 'runcmd' statement is at the beginning of the list
133+ // of 'runcmd' statements.
134+ runCmd := ciContent["runcmd"].([]interface{})
135+ c.Check(runCmd[0], Equals, script)
136+}
137+
138 func getScripts(x map[interface{}]interface{}) []string {
139 var scripts []string
140 for _, s := range x["runcmd"].([]interface{}) {
141
142=== added directory 'environs/maas'
143=== added file 'environs/maas/config.go'
144--- environs/maas/config.go 1970-01-01 00:00:00 +0000
145+++ environs/maas/config.go 2013-04-12 10:00:43 +0000
146@@ -0,0 +1,72 @@
147+package maas
148+
149+import (
150+ "errors"
151+ "fmt"
152+ "launchpad.net/juju-core/environs/config"
153+ "launchpad.net/juju-core/schema"
154+ "net/url"
155+ "strings"
156+)
157+
158+var maasConfigChecker = schema.StrictFieldMap(
159+ schema.Fields{
160+ "maas-server": schema.String(),
161+ // maas-oauth is a colon-separated triplet of:
162+ // consumer-key:resource-token:resource-secret
163+ "maas-oauth": schema.String(),
164+ },
165+ schema.Defaults{},
166+)
167+
168+type maasEnvironConfig struct {
169+ *config.Config
170+ attrs map[string]interface{}
171+}
172+
173+func (cfg *maasEnvironConfig) MAASServer() string {
174+ return cfg.attrs["maas-server"].(string)
175+}
176+
177+func (cfg *maasEnvironConfig) MAASOAuth() string {
178+ return cfg.attrs["maas-oauth"].(string)
179+}
180+
181+func (prov maasEnvironProvider) newConfig(cfg *config.Config) (*maasEnvironConfig, error) {
182+ validCfg, err := prov.Validate(cfg, nil)
183+ if err != nil {
184+ return nil, err
185+ }
186+ result := new(maasEnvironConfig)
187+ result.Config = validCfg
188+ result.attrs = validCfg.UnknownAttrs()
189+ return result, nil
190+}
191+
192+var errMalformedMaasOAuth = errors.New("malformed maas-oauth (3 items separated by colons)")
193+
194+func (prov maasEnvironProvider) Validate(cfg, oldCfg *config.Config) (*config.Config, error) {
195+ // Validate base configuration change before validating MAAS specifics.
196+ err := config.Validate(cfg, oldCfg)
197+ if err != nil {
198+ return nil, err
199+ }
200+
201+ v, err := maasConfigChecker.Coerce(cfg.UnknownAttrs(), nil)
202+ if err != nil {
203+ return nil, err
204+ }
205+ envCfg := new(maasEnvironConfig)
206+ envCfg.Config = cfg
207+ envCfg.attrs = v.(map[string]interface{})
208+ server := envCfg.MAASServer()
209+ serverURL, err := url.Parse(server)
210+ if err != nil || serverURL.Scheme == "" || serverURL.Host == "" {
211+ return nil, fmt.Errorf("malformed maas-server URL '%v': %s", server, err)
212+ }
213+ oauth := envCfg.MAASOAuth()
214+ if strings.Count(oauth, ":") != 2 {
215+ return nil, errMalformedMaasOAuth
216+ }
217+ return cfg.Apply(envCfg.attrs)
218+}
219
220=== added file 'environs/maas/config_test.go'
221--- environs/maas/config_test.go 1970-01-01 00:00:00 +0000
222+++ environs/maas/config_test.go 2013-04-12 10:00:43 +0000
223@@ -0,0 +1,94 @@
224+package maas
225+
226+import (
227+ . "launchpad.net/gocheck"
228+ "launchpad.net/juju-core/environs"
229+ "launchpad.net/juju-core/testing"
230+ "launchpad.net/juju-core/version"
231+)
232+
233+type ConfigSuite struct{}
234+
235+var _ = Suite(new(ConfigSuite))
236+
237+// copyAttrs copies values from src into dest. If src contains a key that was
238+// already in dest, its value in dest will still be updated to the one from
239+// src.
240+func copyAttrs(src, dest map[string]interface{}) {
241+ for k, v := range src {
242+ dest[k] = v
243+ }
244+}
245+
246+// newConfig creates a MAAS environment config from attributes.
247+func newConfig(values map[string]interface{}) (*maasEnvironConfig, error) {
248+ defaults := map[string]interface{}{
249+ "name": "testenv",
250+ "type": "maas",
251+ "admin-secret": "ssshhhhhh",
252+ "authorized-keys": "I-am-not-a-real-key",
253+ "agent-version": version.CurrentNumber().String(),
254+ // These are not needed by MAAS, but juju-core breaks without them. Needs
255+ // fixing there.
256+ "ca-cert": testing.CACert,
257+ "ca-private-key": testing.CAKey,
258+ }
259+ cfg := make(map[string]interface{})
260+ copyAttrs(defaults, cfg)
261+ copyAttrs(values, cfg)
262+ env, err := environs.NewFromAttrs(cfg)
263+ if err != nil {
264+ return nil, err
265+ }
266+ return env.(*maasEnviron).ecfg(), nil
267+}
268+
269+func (ConfigSuite) TestParsesMAASSettings(c *C) {
270+ server := "http://maas.example.com/maas/"
271+ oauth := "consumer-key:resource-token:resource-secret"
272+ ecfg, err := newConfig(map[string]interface{}{
273+ "maas-server": server,
274+ "maas-oauth": oauth,
275+ })
276+ c.Assert(err, IsNil)
277+ c.Check(ecfg.MAASServer(), Equals, server)
278+ c.Check(ecfg.MAASOAuth(), DeepEquals, oauth)
279+}
280+
281+func (ConfigSuite) TestChecksWellFormedMaasServer(c *C) {
282+ _, err := newConfig(map[string]interface{}{
283+ "maas-server": "This should have been a URL.",
284+ "maas-oauth": "consumer-key:resource-token:resource-secret",
285+ })
286+ c.Assert(err, NotNil)
287+ c.Check(err, ErrorMatches, ".*malformed maas-server.*")
288+}
289+
290+func (ConfigSuite) TestChecksWellFormedMaasOAuth(c *C) {
291+ _, err := newConfig(map[string]interface{}{
292+ "maas-server": "http://maas.example.com/maas/",
293+ "maas-oauth": "This should have been a 3-part token.",
294+ })
295+ c.Assert(err, NotNil)
296+ c.Check(err, ErrorMatches, ".*malformed maas-oauth.*")
297+}
298+
299+func (ConfigSuite) TestValidateUpcallsEnvironsConfigValidate(c *C) {
300+ // The base Validate() function will not allow an environment to
301+ // change its name. Trigger that error so as to prove that the
302+ // environment provider's Validate() calls the base Validate().
303+ baseAttrs := map[string]interface{}{
304+ "maas-server": "http://maas.example.com/maas/",
305+ "maas-oauth": "consumer-key:resource-token:resource-secret",
306+ }
307+ oldCfg, err := newConfig(baseAttrs)
308+ c.Assert(err, IsNil)
309+ newName := oldCfg.Name() + "-but-different"
310+ newCfg, err := oldCfg.Apply(map[string]interface{}{"name": newName})
311+ c.Assert(err, IsNil)
312+
313+ _, err = maasEnvironProvider{}.Validate(newCfg, oldCfg.Config)
314+
315+ c.Assert(err, NotNil)
316+ c.Check(err, ErrorMatches, ".*cannot change name.*")
317+}
318
319=== added file 'environs/maas/environ.go'
320--- environs/maas/environ.go 1970-01-01 00:00:00 +0000
321+++ environs/maas/environ.go 2013-04-12 10:00:43 +0000
322@@ -0,0 +1,519 @@
323+package maas
324+
325+import (
326+ "encoding/base64"
327+ "errors"
328+ "fmt"
329+ "launchpad.net/gomaasapi"
330+ "launchpad.net/juju-core/constraints"
331+ "launchpad.net/juju-core/environs"
332+ "launchpad.net/juju-core/environs/cloudinit"
333+ "launchpad.net/juju-core/environs/config"
334+ "launchpad.net/juju-core/log"
335+ "launchpad.net/juju-core/state"
336+ "launchpad.net/juju-core/state/api"
337+ "launchpad.net/juju-core/state/api/params"
338+ "launchpad.net/juju-core/trivial"
339+ "launchpad.net/juju-core/version"
340+ "net/url"
341+ "sync"
342+ "time"
343+)
344+
345+const (
346+ mgoPort = 37017
347+ apiPort = 17070
348+ jujuDataDir = "/var/lib/juju"
349+ // We're using v1.0 of the MAAS API.
350+ apiVersion = "1.0"
351+)
352+
353+var mgoPortSuffix = fmt.Sprintf(":%d", mgoPort)
354+var apiPortSuffix = fmt.Sprintf(":%d", apiPort)
355+
356+var longAttempt = trivial.AttemptStrategy{
357+ Total: 3 * time.Minute,
358+ Delay: 1 * time.Second,
359+}
360+
361+type maasEnviron struct {
362+ name string
363+
364+ // ecfgMutex protects the *Unlocked fields below.
365+ ecfgMutex sync.Mutex
366+
367+ ecfgUnlocked *maasEnvironConfig
368+ maasClientUnlocked *gomaasapi.MAASObject
369+ storageUnlocked environs.Storage
370+}
371+
372+var _ environs.Environ = (*maasEnviron)(nil)
373+
374+var couldNotAllocate = errors.New("Could not allocate MAAS environment object.")
375+
376+func NewEnviron(cfg *config.Config) (*maasEnviron, error) {
377+ env := new(maasEnviron)
378+ if env == nil {
379+ return nil, couldNotAllocate
380+ }
381+ err := env.SetConfig(cfg)
382+ if err != nil {
383+ return nil, err
384+ }
385+ env.storageUnlocked = NewStorage(env)
386+ return env, nil
387+}
388+
389+func (env *maasEnviron) Name() string {
390+ return env.name
391+}
392+
393+// TODO: this code is cargo-culted from the openstack/ec2 providers.
394+func (env *maasEnviron) findTools() (*state.Tools, error) {
395+ flags := environs.HighestVersion | environs.CompatVersion
396+ v := version.Current
397+ v.Series = env.Config().DefaultSeries()
398+ return environs.FindTools(env, v, flags)
399+}
400+
401+// makeMachineConfig sets up a basic machine configuration for use with
402+// userData(). You may still need to supply more information, but this takes
403+// care of the fixed entries and the ones that are always needed.
404+func (env *maasEnviron) makeMachineConfig(machineID, machineNonce string, stateInfo *state.Info, apiInfo *api.Info, tools *state.Tools) *cloudinit.MachineConfig {
405+ return &cloudinit.MachineConfig{
406+ // Fixed entries.
407+ MongoPort: mgoPort,
408+ APIPort: apiPort,
409+ DataDir: jujuDataDir,
410+
411+ // Entries based purely on what's in the environment.
412+ AuthorizedKeys: env.ecfg().AuthorizedKeys(),
413+
414+ // Parameter entries.
415+ MachineId: machineID,
416+ MachineNonce: machineNonce,
417+ StateInfo: stateInfo,
418+ APIInfo: apiInfo,
419+ Tools: tools,
420+ }
421+}
422+
423+// startBootstrapNode starts the juju bootstrap node for this environment.
424+func (env *maasEnviron) startBootstrapNode(tools *state.Tools, cert, key []byte, password string) (environs.Instance, error) {
425+ config, err := environs.BootstrapConfig(env.Provider(), env.Config(), tools)
426+ if err != nil {
427+ return nil, fmt.Errorf("unable to determine initial configuration: %v", err)
428+ }
429+ caCert, hasCert := env.Config().CACert()
430+ if !hasCert {
431+ return nil, fmt.Errorf("no CA certificate in environment configuration")
432+ }
433+ stateInfo := state.Info{
434+ Password: trivial.PasswordHash(password),
435+ CACert: caCert,
436+ }
437+ apiInfo := api.Info{
438+ Password: trivial.PasswordHash(password),
439+ CACert: caCert,
440+ }
441+
442+ // The bootstrap instance gets machine id "0". This is not related to
443+ // instance ids or MAAS system ids. Juju assigns the machine ID.
444+ const machineID = "0"
445+
446+ mcfg := env.makeMachineConfig(machineID, state.BootstrapNonce, &stateInfo, &apiInfo, tools)
447+ mcfg.StateServer = true
448+ mcfg.StateServerCert = cert
449+ mcfg.StateServerKey = key
450+ mcfg.Config = config
451+
452+ inst, err := env.obtainNode(machineID, &stateInfo, &apiInfo, tools, mcfg)
453+ if err != nil {
454+ return nil, fmt.Errorf("cannot start bootstrap instance: %v", err)
455+ }
456+ return inst, nil
457+}
458+
459+// Bootstrap is specified in the Environ interface.
460+func (env *maasEnviron) Bootstrap(cons constraints.Value, stateServerCert, stateServerKey []byte) error {
461+ constraints := cons.String()
462+ if constraints != "" {
463+ log.Warningf("ignoring constraints '%s' (not implemented)", constraints)
464+ }
465+
466+ // This was all cargo-culted from the EC2 provider.
467+ password := env.Config().AdminSecret()
468+ if password == "" {
469+ return fmt.Errorf("admin-secret is required for bootstrap")
470+ }
471+ log.Debugf("environs/maas: bootstrapping environment %q.", env.Name())
472+ tools, err := env.findTools()
473+ if err != nil {
474+ return err
475+ }
476+ inst, err := env.startBootstrapNode(tools, stateServerCert, stateServerKey, password)
477+ if err != nil {
478+ return err
479+ }
480+ err = env.saveState(&bootstrapState{StateInstances: []state.InstanceId{inst.Id()}})
481+ if err != nil {
482+ env.releaseInstance(inst)
483+ return fmt.Errorf("cannot save state: %v", err)
484+ }
485+
486+ // TODO make safe in the case of racing Bootstraps
487+ // If two Bootstraps are called concurrently, there's
488+ // no way to make sure that only one succeeds.
489+
490+ return nil
491+}
492+
493+// StateInfo is specified in the Environ interface.
494+func (env *maasEnviron) StateInfo() (*state.Info, *api.Info, error) {
495+ // This code is cargo-culted from the openstack/ec2 providers.
496+ // It's a bit unclear what the "longAttempt" loop is actually for
497+ // but this should probably be refactored outside of the provider
498+ // code.
499+ st, err := env.loadState()
500+ if err != nil {
501+ return nil, nil, err
502+ }
503+ cert, hasCert := env.Config().CACert()
504+ if !hasCert {
505+ return nil, nil, fmt.Errorf("no CA certificate in environment configuration")
506+ }
507+ var stateAddrs []string
508+ var apiAddrs []string
509+ // Wait for the DNS names of any of the instances
510+ // to become available.
511+ log.Debugf("environs/maas: waiting for DNS name(s) of state server instances %v", st.StateInstances)
512+ for a := longAttempt.Start(); len(stateAddrs) == 0 && a.Next(); {
513+ insts, err := env.Instances(st.StateInstances)
514+ if err != nil && err != environs.ErrPartialInstances {
515+ log.Debugf("error getting state instance: %v", err.Error())
516+ return nil, nil, err
517+ }
518+ log.Debugf("started processing instances: %#v", insts)
519+ for _, inst := range insts {
520+ if inst == nil {
521+ continue
522+ }
523+ name, err := inst.DNSName()
524+ if err != nil {
525+ continue
526+ }
527+ if name != "" {
528+ stateAddrs = append(stateAddrs, name+mgoPortSuffix)
529+ apiAddrs = append(apiAddrs, name+apiPortSuffix)
530+ }
531+ }
532+ }
533+ if len(stateAddrs) == 0 {
534+ return nil, nil, fmt.Errorf("timed out waiting for mgo address from %v", st.StateInstances)
535+ }
536+ return &state.Info{
537+ Addrs: stateAddrs,
538+ CACert: cert,
539+ }, &api.Info{
540+ Addrs: apiAddrs,
541+ CACert: cert,
542+ }, nil
543+}
544+
545+// ecfg returns the environment's maasEnvironConfig, and protects it with a
546+// mutex.
547+func (env *maasEnviron) ecfg() *maasEnvironConfig {
548+ env.ecfgMutex.Lock()
549+ defer env.ecfgMutex.Unlock()
550+ return env.ecfgUnlocked
551+}
552+
553+// Config is specified in the Environ interface.
554+func (env *maasEnviron) Config() *config.Config {
555+ return env.ecfg().Config
556+}
557+
558+// SetConfig is specified in the Environ interface.
559+func (env *maasEnviron) SetConfig(cfg *config.Config) error {
560+ env.ecfgMutex.Lock()
561+ defer env.ecfgMutex.Unlock()
562+
563+ // The new config has already been validated by itself, but now we
564+ // validate the transition from the old config to the new.
565+ var oldCfg *config.Config
566+ if env.ecfgUnlocked != nil {
567+ oldCfg = env.ecfgUnlocked.Config
568+ }
569+ cfg, err := env.Provider().Validate(cfg, oldCfg)
570+ if err != nil {
571+ return err
572+ }
573+
574+ ecfg, err := providerInstance.newConfig(cfg)
575+ if err != nil {
576+ return err
577+ }
578+
579+ env.name = cfg.Name()
580+ env.ecfgUnlocked = ecfg
581+
582+ authClient, err := gomaasapi.NewAuthenticatedClient(ecfg.MAASServer(), ecfg.MAASOAuth(), apiVersion)
583+ if err != nil {
584+ return err
585+ }
586+ env.maasClientUnlocked = gomaasapi.NewMAAS(*authClient)
587+
588+ return nil
589+}
590+
591+// getMAASClient returns a MAAS client object to use for a request, in a
592+// lock-protected fashioon.
593+func (env *maasEnviron) getMAASClient() *gomaasapi.MAASObject {
594+ env.ecfgMutex.Lock()
595+ defer env.ecfgMutex.Unlock()
596+
597+ return env.maasClientUnlocked
598+}
599+
600+// acquireNode allocates a node from the MAAS.
601+func (environ *maasEnviron) acquireNode() (gomaasapi.MAASObject, error) {
602+ retry := trivial.AttemptStrategy{
603+ Total: 5 * time.Second,
604+ Delay: 200 * time.Millisecond,
605+ }
606+ var result gomaasapi.JSONObject
607+ var err error
608+ for a := retry.Start(); a.Next(); {
609+ client := environ.getMAASClient().GetSubObject("nodes/")
610+ result, err = client.CallPost("acquire", nil)
611+ if err == nil {
612+ break
613+ }
614+ }
615+ if err != nil {
616+ return gomaasapi.MAASObject{}, err
617+ }
618+ node, err := result.GetMAASObject()
619+ if err != nil {
620+ msg := fmt.Errorf("unexpected result from 'acquire' on MAAS API: %v", err)
621+ return gomaasapi.MAASObject{}, msg
622+ }
623+ return node, nil
624+}
625+
626+// startNode installs and boots a node.
627+func (environ *maasEnviron) startNode(node gomaasapi.MAASObject, tools *state.Tools, userdata []byte) error {
628+ retry := trivial.AttemptStrategy{
629+ Total: 5 * time.Second,
630+ Delay: 200 * time.Millisecond,
631+ }
632+ userDataParam := base64.StdEncoding.EncodeToString(userdata)
633+ params := url.Values{
634+ "distro_series": {tools.Series},
635+ "user_data": {userDataParam},
636+ }
637+ // Initialize err to a non-nil value as a sentinel for the following
638+ // loop.
639+ err := fmt.Errorf("(no error)")
640+ for a := retry.Start(); a.Next() && err != nil; {
641+ _, err = node.CallPost("start", params)
642+ }
643+ return err
644+}
645+
646+// obtainNode allocates and starts a MAAS node. It is used both for the
647+// implementation of StartInstance, and to initialize the bootstrap node.
648+func (environ *maasEnviron) obtainNode(machineId string, stateInfo *state.Info, apiInfo *api.Info, tools *state.Tools, mcfg *cloudinit.MachineConfig) (*maasInstance, error) {
649+
650+ log.Debugf("environs/maas: starting machine %s in $q running tools version %q from %q", machineId, environ.name, tools.Binary, tools.URL)
651+
652+ node, err := environ.acquireNode()
653+ if err != nil {
654+ return nil, fmt.Errorf("cannot run instances: %v", err)
655+ }
656+
657+ hostname, err := node.GetField("hostname")
658+ if err != nil {
659+ return nil, err
660+ }
661+ instance := maasInstance{&node, environ}
662+ info := machineInfo{string(instance.Id()), hostname}
663+ runCmd, err := info.cloudinitRunCmd()
664+ if err != nil {
665+ return nil, err
666+ }
667+ userdata, err := userData(mcfg, runCmd)
668+ if err != nil {
669+ msg := fmt.Errorf("could not compose userdata for bootstrap node: %v", err)
670+ return nil, msg
671+ }
672+ err = environ.startNode(node, tools, userdata)
673+ if err != nil {
674+ environ.releaseInstance(&instance)
675+ return nil, fmt.Errorf("cannot start instance: %v", err)
676+ }
677+ log.Debugf("environs/maas: started instance %q", instance.Id())
678+ return &instance, nil
679+}
680+
681+// StartInstance is specified in the Environ interface.
682+func (environ *maasEnviron) StartInstance(machineID, machineNonce string, series string, cons constraints.Value, stateInfo *state.Info, apiInfo *api.Info) (environs.Instance, error) {
683+ // TODO: Support series and constraints. They were added to the
684+ // interface after we implemented.
685+ flags := environs.HighestVersion | environs.CompatVersion
686+ var err error
687+ tools, err := environs.FindTools(environ, version.Current, flags)
688+ if err != nil {
689+ return nil, err
690+ }
691+
692+ mcfg := environ.makeMachineConfig(machineID, machineNonce, stateInfo, apiInfo, tools)
693+ return environ.obtainNode(machineID, stateInfo, apiInfo, tools, mcfg)
694+}
695+
696+// StopInstances is specified in the Environ interface.
697+func (environ *maasEnviron) StopInstances(instances []environs.Instance) error {
698+ // Shortcut to exit quickly if 'instances' is an empty slice or nil.
699+ if len(instances) == 0 {
700+ return nil
701+ }
702+ // Tell MAAS to release each of the instances. If there are errors,
703+ // return only the first one (but release all instances regardless).
704+ // Note that releasing instances also turns them off.
705+ var firstErr error
706+ for _, instance := range instances {
707+ err := environ.releaseInstance(instance)
708+ if firstErr == nil {
709+ firstErr = err
710+ }
711+ }
712+ return firstErr
713+}
714+
715+// releaseInstance releases a single instance.
716+func (environ *maasEnviron) releaseInstance(inst environs.Instance) error {
717+ maasInst := inst.(*maasInstance)
718+ maasObj := maasInst.maasObject
719+ _, err := maasObj.CallPost("release", nil)
720+ if err != nil {
721+ log.Debugf("environs/maas: error releasing instance %v", maasInst)
722+ }
723+ return err
724+}
725+
726+// Instances returns the environs.Instance objects corresponding to the given
727+// slice of state.InstanceId. Similar to what the ec2 provider does,
728+// Instances returns nil if the given slice is empty or nil.
729+func (environ *maasEnviron) Instances(ids []state.InstanceId) ([]environs.Instance, error) {
730+ if len(ids) == 0 {
731+ return nil, nil
732+ }
733+ return environ.instances(ids)
734+}
735+
736+// instances is an internal method which returns the instances matching the
737+// given instance ids or all the instances if 'ids' is empty.
738+// If the some of the intances could not be found, it returns the instance
739+// that could be found plus the error environs.ErrPartialInstances in the error
740+// return.
741+func (environ *maasEnviron) instances(ids []state.InstanceId) ([]environs.Instance, error) {
742+ nodeListing := environ.getMAASClient().GetSubObject("nodes")
743+ filter := getSystemIdValues(ids)
744+ listNodeObjects, err := nodeListing.CallGet("list", filter)
745+ if err != nil {
746+ return nil, err
747+ }
748+ listNodes, err := listNodeObjects.GetArray()
749+ if err != nil {
750+ return nil, err
751+ }
752+ instances := make([]environs.Instance, len(listNodes))
753+ for index, nodeObj := range listNodes {
754+ node, err := nodeObj.GetMAASObject()
755+ if err != nil {
756+ return nil, err
757+ }
758+ instances[index] = &maasInstance{
759+ maasObject: &node,
760+ environ: environ,
761+ }
762+ }
763+ if len(ids) != 0 && len(ids) != len(instances) {
764+ return instances, environs.ErrPartialInstances
765+ }
766+ return instances, nil
767+}
768+
769+// AllInstances returns all the environs.Instance in this provider.
770+func (environ *maasEnviron) AllInstances() ([]environs.Instance, error) {
771+ return environ.instances(nil)
772+}
773+
774+// Storage is defined by the Environ interface.
775+func (env *maasEnviron) Storage() environs.Storage {
776+ env.ecfgMutex.Lock()
777+ defer env.ecfgMutex.Unlock()
778+ return env.storageUnlocked
779+}
780+
781+// PublicStorage is defined by the Environ interface.
782+func (env *maasEnviron) PublicStorage() environs.StorageReader {
783+ // MAAS does not have a shared storage.
784+ return environs.EmptyStorage
785+}
786+
787+func (environ *maasEnviron) Destroy(ensureInsts []environs.Instance) error {
788+ log.Debugf("environs/maas: destroying environment %q", environ.name)
789+ insts, err := environ.AllInstances()
790+ if err != nil {
791+ return fmt.Errorf("cannot get instances: %v", err)
792+ }
793+ found := make(map[state.InstanceId]bool)
794+ for _, inst := range insts {
795+ found[inst.Id()] = true
796+ }
797+
798+ // Add any instances we've been told about but haven't yet shown
799+ // up in the instance list.
800+ for _, inst := range ensureInsts {
801+ id := inst.Id()
802+ if !found[id] {
803+ insts = append(insts, inst)
804+ found[id] = true
805+ }
806+ }
807+ err = environ.StopInstances(insts)
808+ if err != nil {
809+ return err
810+ }
811+
812+ // To properly observe e.storageUnlocked we need to get its value while
813+ // holding e.ecfgMutex. e.Storage() does this for us, then we convert
814+ // back to the (*storage) to access the private deleteAll() method.
815+ st := environ.Storage().(*maasStorage)
816+ return st.deleteAll()
817+}
818+
819+func (*maasEnviron) AssignmentPolicy() state.AssignmentPolicy {
820+ return state.AssignUnused
821+}
822+
823+// MAAS does not do firewalling so these port methods do nothing.
824+func (*maasEnviron) OpenPorts([]params.Port) error {
825+ log.Debugf("environs/maas: unimplemented OpenPorts() called")
826+ return nil
827+}
828+
829+func (*maasEnviron) ClosePorts([]params.Port) error {
830+ log.Debugf("environs/maas: unimplemented ClosePorts() called")
831+ return nil
832+}
833+
834+func (*maasEnviron) Ports() ([]params.Port, error) {
835+ log.Debugf("environs/maas: unimplemented Ports() called")
836+ return []params.Port{}, nil
837+}
838+
839+func (*maasEnviron) Provider() environs.EnvironProvider {
840+ return &providerInstance
841+}
842
843=== added file 'environs/maas/environ_test.go'
844--- environs/maas/environ_test.go 1970-01-01 00:00:00 +0000
845+++ environs/maas/environ_test.go 2013-04-12 10:00:43 +0000
846@@ -0,0 +1,379 @@
847+package maas
848+
849+import (
850+ "encoding/base64"
851+ . "launchpad.net/gocheck"
852+ "launchpad.net/gomaasapi"
853+ "launchpad.net/goyaml"
854+ "launchpad.net/juju-core/constraints"
855+ "launchpad.net/juju-core/environs"
856+ "launchpad.net/juju-core/environs/config"
857+ envtesting "launchpad.net/juju-core/environs/testing"
858+ "launchpad.net/juju-core/state"
859+ "launchpad.net/juju-core/testing"
860+ "launchpad.net/juju-core/trivial"
861+ "launchpad.net/juju-core/version"
862+)
863+
864+type EnvironSuite struct {
865+ ProviderSuite
866+}
867+
868+var _ = Suite(new(EnvironSuite))
869+
870+// getTestConfig creates a customized sample MAAS provider configuration.
871+func getTestConfig(name, server, oauth, secret string) *config.Config {
872+ ecfg, err := newConfig(map[string]interface{}{
873+ "name": name,
874+ "maas-server": server,
875+ "maas-oauth": oauth,
876+ "admin-secret": secret,
877+ "authorized-keys": "I-am-not-a-real-key",
878+ })
879+ if err != nil {
880+ panic(err)
881+ }
882+ return ecfg.Config
883+}
884+
885+// makeEnviron creates a functional maasEnviron for a test. Its configuration
886+// is a bit arbitrary and none of the test code's business.
887+func (suite *EnvironSuite) makeEnviron() *maasEnviron {
888+ config, err := config.New(map[string]interface{}{
889+ "name": suite.environ.Name(),
890+ "type": "maas",
891+ "admin-secret": "local-secret",
892+ "authorized-keys": "foo",
893+ "agent-version": version.CurrentNumber().String(),
894+ "maas-oauth": "a:b:c",
895+ "maas-server": suite.testMAASObject.TestServer.URL,
896+ // These are not needed by MAAS, but juju-core breaks without them. Needs
897+ // fixing there.
898+ "ca-cert": testing.CACert,
899+ "ca-private-key": testing.CAKey,
900+ })
901+ if err != nil {
902+ panic(err)
903+ }
904+ env, err := NewEnviron(config)
905+ if err != nil {
906+ panic(err)
907+ }
908+ return env
909+}
910+
911+func (suite *EnvironSuite) setupFakeProviderStateFile(c *C) {
912+ suite.testMAASObject.TestServer.NewFile("provider-state", []byte("test file content"))
913+}
914+
915+func (suite *EnvironSuite) setupFakeTools(c *C) {
916+ storage := NewStorage(suite.environ)
917+ envtesting.PutFakeTools(c, storage)
918+}
919+
920+func (EnvironSuite) TestSetConfigValidatesFirst(c *C) {
921+ // SetConfig() validates the config change and disallows, for example,
922+ // changes in the environment name.
923+ server := "http://maas.example.com"
924+ oauth := "a:b:c"
925+ secret := "pssst"
926+ oldCfg := getTestConfig("old-name", server, oauth, secret)
927+ newCfg := getTestConfig("new-name", server, oauth, secret)
928+ env, err := NewEnviron(oldCfg)
929+ c.Assert(err, IsNil)
930+
931+ // SetConfig() fails, even though both the old and the new config are
932+ // individually valid.
933+ err = env.SetConfig(newCfg)
934+ c.Assert(err, NotNil)
935+ c.Check(err, ErrorMatches, ".*cannot change name.*")
936+
937+ // The old config is still in place. The new config never took effect.
938+ c.Check(env.Name(), Equals, "old-name")
939+}
940+
941+func (EnvironSuite) TestSetConfigUpdatesConfig(c *C) {
942+ name := "test env"
943+ cfg := getTestConfig(name, "http://maas2.example.com", "a:b:c", "secret")
944+ env, err := NewEnviron(cfg)
945+ c.Check(err, IsNil)
946+ c.Check(env.name, Equals, "test env")
947+
948+ anotherServer := "http://maas.example.com"
949+ anotherOauth := "c:d:e"
950+ anotherSecret := "secret2"
951+ cfg2 := getTestConfig(name, anotherServer, anotherOauth, anotherSecret)
952+ errSetConfig := env.SetConfig(cfg2)
953+ c.Check(errSetConfig, IsNil)
954+ c.Check(env.name, Equals, name)
955+ authClient, _ := gomaasapi.NewAuthenticatedClient(anotherServer, anotherOauth, apiVersion)
956+ maas := gomaasapi.NewMAAS(*authClient)
957+ MAASServer := env.maasClientUnlocked
958+ c.Check(MAASServer, DeepEquals, maas)
959+}
960+
961+func (EnvironSuite) TestNewEnvironSetsConfig(c *C) {
962+ name := "test env"
963+ cfg := getTestConfig(name, "http://maas.example.com", "a:b:c", "secret")
964+
965+ env, err := NewEnviron(cfg)
966+
967+ c.Check(err, IsNil)
968+ c.Check(env.name, Equals, name)
969+}
970+
971+func (suite *EnvironSuite) TestInstancesReturnsInstances(c *C) {
972+ input := `{"system_id": "test"}`
973+ node := suite.testMAASObject.TestServer.NewNode(input)
974+ resourceURI, _ := node.GetField("resource_uri")
975+ instanceIds := []state.InstanceId{state.InstanceId(resourceURI)}
976+
977+ instances, err := suite.environ.Instances(instanceIds)
978+
979+ c.Check(err, IsNil)
980+ c.Check(len(instances), Equals, 1)
981+ c.Check(string(instances[0].Id()), Equals, resourceURI)
982+}
983+
984+func (suite *EnvironSuite) TestInstancesReturnsNilIfEmptyParameter(c *C) {
985+ // Instances returns nil if the given parameter is empty.
986+ input := `{"system_id": "test"}`
987+ suite.testMAASObject.TestServer.NewNode(input)
988+ instances, err := suite.environ.Instances([]state.InstanceId{})
989+
990+ c.Check(err, IsNil)
991+ c.Check(instances, IsNil)
992+}
993+
994+func (suite *EnvironSuite) TestInstancesReturnsNilIfNilParameter(c *C) {
995+ // Instances returns nil if the given parameter is nil.
996+ input := `{"system_id": "test"}`
997+ suite.testMAASObject.TestServer.NewNode(input)
998+ instances, err := suite.environ.Instances(nil)
999+
1000+ c.Check(err, IsNil)
1001+ c.Check(instances, IsNil)
1002+}
1003+
1004+func (suite *EnvironSuite) TestAllInstancesReturnsAllInstances(c *C) {
1005+ input := `{"system_id": "test"}`
1006+ node := suite.testMAASObject.TestServer.NewNode(input)
1007+ resourceURI, _ := node.GetField("resource_uri")
1008+
1009+ instances, err := suite.environ.AllInstances()
1010+
1011+ c.Check(err, IsNil)
1012+ c.Check(len(instances), Equals, 1)
1013+ c.Check(string(instances[0].Id()), Equals, resourceURI)
1014+}
1015+
1016+func (suite *EnvironSuite) TestAllInstancesReturnsEmptySliceIfNoInstance(c *C) {
1017+ instances, err := suite.environ.AllInstances()
1018+
1019+ c.Check(err, IsNil)
1020+ c.Check(len(instances), Equals, 0)
1021+}
1022+
1023+func (suite *EnvironSuite) TestInstancesReturnsErrorIfPartialInstances(c *C) {
1024+ input1 := `{"system_id": "test"}`
1025+ node1 := suite.testMAASObject.TestServer.NewNode(input1)
1026+ resourceURI1, _ := node1.GetField("resource_uri")
1027+ input2 := `{"system_id": "test2"}`
1028+ suite.testMAASObject.TestServer.NewNode(input2)
1029+ instanceId1 := state.InstanceId(resourceURI1)
1030+ instanceId2 := state.InstanceId("unknown systemID")
1031+ instanceIds := []state.InstanceId{instanceId1, instanceId2}
1032+
1033+ instances, err := suite.environ.Instances(instanceIds)
1034+
1035+ c.Check(err, Equals, environs.ErrPartialInstances)
1036+ c.Check(len(instances), Equals, 1)
1037+ c.Check(string(instances[0].Id()), Equals, resourceURI1)
1038+}
1039+
1040+func (suite *EnvironSuite) TestStorageReturnsStorage(c *C) {
1041+ env := suite.makeEnviron()
1042+ storage := env.Storage()
1043+ c.Check(storage, NotNil)
1044+ // The Storage object is really a maasStorage.
1045+ specificStorage := storage.(*maasStorage)
1046+ // Its environment pointer refers back to its environment.
1047+ c.Check(specificStorage.environUnlocked, Equals, env)
1048+}
1049+
1050+func (suite *EnvironSuite) TestPublicStorageReturnsEmptyStorage(c *C) {
1051+ env := suite.makeEnviron()
1052+ storage := env.PublicStorage()
1053+ c.Assert(storage, NotNil)
1054+ c.Check(storage, Equals, environs.EmptyStorage)
1055+}
1056+
1057+func decodeUserData(userData string) ([]byte, error) {
1058+ data, err := base64.StdEncoding.DecodeString(userData)
1059+ if err != nil {
1060+ return []byte(""), err
1061+ }
1062+ return trivial.Gunzip(data)
1063+}
1064+
1065+func (suite *EnvironSuite) TestStartInstanceStartsInstance(c *C) {
1066+ suite.setupFakeTools(c)
1067+ env := suite.makeEnviron()
1068+ // Create node 0: it will be used as the bootstrap node.
1069+ suite.testMAASObject.TestServer.NewNode(`{"system_id": "node0", "hostname": "host0"}`)
1070+ err := environs.Bootstrap(env, constraints.Value{})
1071+ c.Assert(err, IsNil)
1072+ // The bootstrap node has been started.
1073+ operations := suite.testMAASObject.TestServer.NodeOperations()
1074+ actions, found := operations["node0"]
1075+ c.Check(found, Equals, true)
1076+ c.Check(actions, DeepEquals, []string{"start"})
1077+
1078+ // Create node 1: it will be used as instance number 1.
1079+ suite.testMAASObject.TestServer.NewNode(`{"system_id": "node1", "hostname": "host1"}`)
1080+ stateInfo, apiInfo, err := env.StateInfo()
1081+ c.Assert(err, IsNil)
1082+ stateInfo.Tag = "machine-1"
1083+ apiInfo.Tag = "machine-1"
1084+ series := version.Current.Series
1085+ nonce := "12345"
1086+ instance, err := env.StartInstance("1", nonce, series, constraints.Value{}, stateInfo, apiInfo)
1087+ c.Assert(err, IsNil)
1088+ c.Check(instance, NotNil)
1089+
1090+ // The instance number 1 has been started.
1091+ actions, found = operations["node1"]
1092+ c.Assert(found, Equals, true)
1093+ c.Check(actions, DeepEquals, []string{"start"})
1094+
1095+ // The value of the "user data" parameter used when starting the node
1096+ // contains the run cmd used to write the machine information onto
1097+ // the node's filesystem.
1098+ requestValues := suite.testMAASObject.TestServer.NodeOperationRequestValues()
1099+ nodeRequestValues, found := requestValues["node1"]
1100+ c.Assert(found, Equals, true)
1101+ userData := nodeRequestValues[0].Get("user_data")
1102+ decodedUserData, err := decodeUserData(userData)
1103+ c.Assert(err, IsNil)
1104+ info := machineInfo{string(instance.Id()), "host1"}
1105+ cloudinitRunCmd, err := info.cloudinitRunCmd()
1106+ c.Assert(err, IsNil)
1107+ data, err := goyaml.Marshal(cloudinitRunCmd)
1108+ c.Assert(err, IsNil)
1109+ c.Check(string(decodedUserData), Matches, "(.|\n)*"+string(data)+"(\n|.)*")
1110+}
1111+
1112+func (suite *EnvironSuite) getInstance(systemId string) *maasInstance {
1113+ input := `{"system_id": "` + systemId + `"}`
1114+ node := suite.testMAASObject.TestServer.NewNode(input)
1115+ return &maasInstance{&node, suite.environ}
1116+}
1117+
1118+func (suite *EnvironSuite) TestStopInstancesReturnsIfParameterEmpty(c *C) {
1119+ suite.getInstance("test1")
1120+
1121+ err := suite.environ.StopInstances([]environs.Instance{})
1122+ c.Check(err, IsNil)
1123+ operations := suite.testMAASObject.TestServer.NodeOperations()
1124+ c.Check(operations, DeepEquals, map[string][]string{})
1125+}
1126+
1127+func (suite *EnvironSuite) TestStopInstancesStopsAndReleasesInstances(c *C) {
1128+ instance1 := suite.getInstance("test1")
1129+ instance2 := suite.getInstance("test2")
1130+ suite.getInstance("test3")
1131+ instances := []environs.Instance{instance1, instance2}
1132+
1133+ err := suite.environ.StopInstances(instances)
1134+
1135+ c.Check(err, IsNil)
1136+ operations := suite.testMAASObject.TestServer.NodeOperations()
1137+ expectedOperations := map[string][]string{"test1": {"release"}, "test2": {"release"}}
1138+ c.Check(operations, DeepEquals, expectedOperations)
1139+}
1140+
1141+func (suite *EnvironSuite) TestStateInfo(c *C) {
1142+ env := suite.makeEnviron()
1143+ hostname := "test"
1144+ input := `{"system_id": "system_id", "hostname": "` + hostname + `"}`
1145+ node := suite.testMAASObject.TestServer.NewNode(input)
1146+ instance := &maasInstance{&node, suite.environ}
1147+ err := env.saveState(&bootstrapState{StateInstances: []state.InstanceId{instance.Id()}})
1148+ c.Assert(err, IsNil)
1149+
1150+ stateInfo, apiInfo, err := env.StateInfo()
1151+
1152+ c.Assert(err, IsNil)
1153+ c.Assert(stateInfo.Addrs, DeepEquals, []string{hostname + mgoPortSuffix})
1154+ c.Assert(apiInfo.Addrs, DeepEquals, []string{hostname + apiPortSuffix})
1155+}
1156+
1157+func (suite *EnvironSuite) TestStateInfoFailsIfNoStateInstances(c *C) {
1158+ env := suite.makeEnviron()
1159+
1160+ _, _, err := env.StateInfo()
1161+
1162+ c.Check(err, FitsTypeOf, &environs.NotFoundError{})
1163+}
1164+
1165+func (suite *EnvironSuite) TestDestroy(c *C) {
1166+ env := suite.makeEnviron()
1167+ suite.getInstance("test1")
1168+ instance := suite.getInstance("test2")
1169+ data := makeRandomBytes(10)
1170+ suite.testMAASObject.TestServer.NewFile("filename", data)
1171+ storage := env.Storage()
1172+
1173+ err := env.Destroy([]environs.Instance{instance})
1174+
1175+ c.Check(err, IsNil)
1176+ // Instances have been stopped.
1177+ operations := suite.testMAASObject.TestServer.NodeOperations()
1178+ expectedOperations := map[string][]string{"test1": {"release"}, "test2": {"release"}}
1179+ c.Check(operations, DeepEquals, expectedOperations)
1180+ // Files have been cleaned up.
1181+ listing, err := storage.List("")
1182+ c.Assert(err, IsNil)
1183+ c.Check(listing, DeepEquals, []string{})
1184+}
1185+
1186+// It would be nice if we could unit-test Bootstrap() in more detail, but
1187+// at the time of writing that would require more support from gomaasapi's
1188+// testing service than we have.
1189+func (suite *EnvironSuite) TestBootstrapSucceeds(c *C) {
1190+ suite.setupFakeTools(c)
1191+ env := suite.makeEnviron()
1192+ suite.testMAASObject.TestServer.NewNode(`{"system_id": "thenode", "hostname": "host"}`)
1193+ cert := []byte{1, 2, 3}
1194+ key := []byte{4, 5, 6}
1195+
1196+ err := env.Bootstrap(constraints.Value{}, cert, key)
1197+ c.Assert(err, IsNil)
1198+}
1199+
1200+func (suite *EnvironSuite) TestBootstrapFailsIfNoNodes(c *C) {
1201+ suite.setupFakeTools(c)
1202+ env := suite.makeEnviron()
1203+ cert := []byte{1, 2, 3}
1204+ key := []byte{4, 5, 6}
1205+ err := env.Bootstrap(constraints.Value{}, cert, key)
1206+ // Since there are no nodes, the attempt to allocate one returns a
1207+ // 409: Conflict.
1208+ c.Check(err, ErrorMatches, ".*409.*")
1209+}
1210+
1211+func (suite *EnvironSuite) TestBootstrapIntegratesWithEnvirons(c *C) {
1212+ suite.setupFakeTools(c)
1213+ env := suite.makeEnviron()
1214+ suite.testMAASObject.TestServer.NewNode(`{"system_id": "bootstrapnode", "hostname": "host"}`)
1215+
1216+ // environs.Bootstrap calls Environ.Bootstrap. This works.
1217+ err := environs.Bootstrap(env, constraints.Value{})
1218+ c.Assert(err, IsNil)
1219+}
1220+
1221+func (suite *EnvironSuite) TestAssignmentPolicy(c *C) {
1222+ env := suite.makeEnviron()
1223+
1224+ c.Check(env.AssignmentPolicy(), Equals, state.AssignUnused)
1225+}
1226
1227=== added file 'environs/maas/environprovider.go'
1228--- environs/maas/environprovider.go 1970-01-01 00:00:00 +0000
1229+++ environs/maas/environprovider.go 2013-04-12 10:00:43 +0000
1230@@ -0,0 +1,81 @@
1231+package maas
1232+
1233+import (
1234+ "launchpad.net/juju-core/environs"
1235+ "launchpad.net/juju-core/environs/config"
1236+ "launchpad.net/juju-core/log"
1237+ "launchpad.net/juju-core/state"
1238+)
1239+
1240+type maasEnvironProvider struct{}
1241+
1242+var _ environs.EnvironProvider = (*maasEnvironProvider)(nil)
1243+
1244+var providerInstance maasEnvironProvider
1245+
1246+func init() {
1247+ environs.RegisterProvider("maas", maasEnvironProvider{})
1248+}
1249+
1250+func (maasEnvironProvider) Open(cfg *config.Config) (environs.Environ, error) {
1251+ log.Debugf("environs/maas: opening environment %q.", cfg.Name())
1252+ return NewEnviron(cfg)
1253+}
1254+
1255+// Boilerplate config YAML. Don't mess with the indentation or add newlines!
1256+const boilerplateYAML = `maas:
1257+ type: maas
1258+ # Change this to where your MAAS server lives. It must specify the base path.
1259+ maas-server: 'http://192.168.1.1/MAAS/'
1260+ maas-oauth: '<add your OAuth credentials from MAAS here>'
1261+ admin-secret: {{rand}}
1262+ default-series: precise
1263+ authorized-keys-path: ~/.ssh/authorized_keys # or any file you want.
1264+ # Or:
1265+ # authorized-keys: ssh-rsa keymaterialhere
1266+`
1267+
1268+// BoilerplateConfig is specified in the EnvironProvider interface.
1269+func (maasEnvironProvider) BoilerplateConfig() string {
1270+ return boilerplateYAML
1271+}
1272+
1273+// SecretAttrs is specified in the EnvironProvider interface.
1274+func (prov maasEnvironProvider) SecretAttrs(cfg *config.Config) (map[string]interface{}, error) {
1275+ secretAttrs := make(map[string]interface{})
1276+ maasCfg, err := prov.newConfig(cfg)
1277+ if err != nil {
1278+ return nil, err
1279+ }
1280+ secretAttrs["maas-oauth"] = maasCfg.MAASOAuth()
1281+ return secretAttrs, nil
1282+}
1283+
1284+func (maasEnvironProvider) hostname() (string, error) {
1285+ info := machineInfo{}
1286+ err := info.load()
1287+ if err != nil {
1288+ return "", err
1289+ }
1290+ return info.Hostname, nil
1291+}
1292+
1293+// PublicAddress is specified in the EnvironProvider interface.
1294+func (prov maasEnvironProvider) PublicAddress() (string, error) {
1295+ return prov.hostname()
1296+}
1297+
1298+// PrivateAddress is specified in the EnvironProvider interface.
1299+func (prov maasEnvironProvider) PrivateAddress() (string, error) {
1300+ return prov.hostname()
1301+}
1302+
1303+// InstanceId is specified in the EnvironProvider interface.
1304+func (maasEnvironProvider) InstanceId() (state.InstanceId, error) {
1305+ info := machineInfo{}
1306+ err := info.load()
1307+ if err != nil {
1308+ return "", err
1309+ }
1310+ return state.InstanceId(info.InstanceId), nil
1311+}
1312
1313=== added file 'environs/maas/environprovider_test.go'
1314--- environs/maas/environprovider_test.go 1970-01-01 00:00:00 +0000
1315+++ environs/maas/environprovider_test.go 2013-04-12 10:00:43 +0000
1316@@ -0,0 +1,94 @@
1317+package maas
1318+
1319+import (
1320+ "io/ioutil"
1321+ . "launchpad.net/gocheck"
1322+ "launchpad.net/goyaml"
1323+ "launchpad.net/juju-core/environs/config"
1324+ "launchpad.net/juju-core/state"
1325+)
1326+
1327+type EnvironProviderSuite struct {
1328+ ProviderSuite
1329+}
1330+
1331+var _ = Suite(new(EnvironProviderSuite))
1332+
1333+func (suite *EnvironProviderSuite) TestSecretAttrsReturnsSensitiveMAASAttributes(c *C) {
1334+ testJujuHome := c.MkDir()
1335+ defer config.SetJujuHome(config.SetJujuHome(testJujuHome))
1336+ const oauth = "aa:bb:cc"
1337+ attrs := map[string]interface{}{
1338+ "maas-oauth": oauth,
1339+ "maas-server": "http://maas.example.com/maas/",
1340+ "name": "wheee",
1341+ "type": "maas",
1342+ "authorized-keys": "I-am-not-a-real-key",
1343+ }
1344+ config, err := config.New(attrs)
1345+ c.Assert(err, IsNil)
1346+
1347+ secretAttrs, err := suite.environ.Provider().SecretAttrs(config)
1348+ c.Assert(err, IsNil)
1349+
1350+ expectedAttrs := map[string]interface{}{"maas-oauth": oauth}
1351+ c.Check(secretAttrs, DeepEquals, expectedAttrs)
1352+}
1353+
1354+// create a temporary file with the given content. The file will be cleaned
1355+// up at the end of the test calling this method.
1356+func createTempFile(c *C, content []byte) string {
1357+ file, err := ioutil.TempFile(c.MkDir(), "")
1358+ c.Assert(err, IsNil)
1359+ filename := file.Name()
1360+ err = ioutil.WriteFile(filename, content, 0644)
1361+ c.Assert(err, IsNil)
1362+ return filename
1363+}
1364+
1365+// InstanceId returns the instanceId of the machine read from the file
1366+// _MAASInstanceFilename.
1367+func (suite *EnvironProviderSuite) TestInstanceIdReadsInstanceIdFromMachineFile(c *C) {
1368+ instanceId := "instance-id"
1369+ info := machineInfo{instanceId, "hostname"}
1370+ yaml, err := goyaml.Marshal(info)
1371+ c.Assert(err, IsNil)
1372+ // Create a temporary file to act as the file where the instanceID
1373+ // is stored.
1374+ filename := createTempFile(c, yaml)
1375+ // "Monkey patch" the value of _MAASInstanceFilename with the path
1376+ // to the temporary file.
1377+ old_MAASInstanceFilename := _MAASInstanceFilename
1378+ _MAASInstanceFilename = filename
1379+ defer func() { _MAASInstanceFilename = old_MAASInstanceFilename }()
1380+
1381+ provider := suite.environ.Provider()
1382+ returnedInstanceId, err := provider.InstanceId()
1383+ c.Assert(err, IsNil)
1384+ c.Check(returnedInstanceId, Equals, state.InstanceId(instanceId))
1385+}
1386+
1387+// PublicAddress and PrivateAddress return the hostname of the machine read
1388+// from the file _MAASInstanceFilename.
1389+func (suite *EnvironProviderSuite) TestPrivatePublicAddressReadsHostnameFromMachineFile(c *C) {
1390+ hostname := "myhostname"
1391+ info := machineInfo{"instance-id", hostname}
1392+ yaml, err := goyaml.Marshal(info)
1393+ c.Assert(err, IsNil)
1394+ // Create a temporary file to act as the file where the instanceID
1395+ // is stored.
1396+ filename := createTempFile(c, yaml)
1397+ // "Monkey patch" the value of _MAASInstanceFilename with the path
1398+ // to the temporary file.
1399+ old_MAASInstanceFilename := _MAASInstanceFilename
1400+ _MAASInstanceFilename = filename
1401+ defer func() { _MAASInstanceFilename = old_MAASInstanceFilename }()
1402+
1403+ provider := suite.environ.Provider()
1404+ publicAddress, err := provider.PublicAddress()
1405+ c.Assert(err, IsNil)
1406+ c.Check(publicAddress, Equals, hostname)
1407+ privateAddress, err := provider.PrivateAddress()
1408+ c.Assert(err, IsNil)
1409+ c.Check(privateAddress, Equals, hostname)
1410+}
1411
1412=== added file 'environs/maas/instance.go'
1413--- environs/maas/instance.go 1970-01-01 00:00:00 +0000
1414+++ environs/maas/instance.go 2013-04-12 10:00:43 +0000
1415@@ -0,0 +1,63 @@
1416+package maas
1417+
1418+import (
1419+ "launchpad.net/gomaasapi"
1420+ "launchpad.net/juju-core/environs"
1421+ "launchpad.net/juju-core/log"
1422+ "launchpad.net/juju-core/state"
1423+ "launchpad.net/juju-core/state/api/params"
1424+)
1425+
1426+type maasInstance struct {
1427+ maasObject *gomaasapi.MAASObject
1428+ environ *maasEnviron
1429+}
1430+
1431+var _ environs.Instance = (*maasInstance)(nil)
1432+
1433+func (instance *maasInstance) Id() state.InstanceId {
1434+ // Use the node's 'resource_uri' value.
1435+ return state.InstanceId((*instance.maasObject).URI().String())
1436+}
1437+
1438+// refreshInstance refreshes the instance with the most up-to-date information
1439+// from the MAAS server.
1440+func (instance *maasInstance) refreshInstance() error {
1441+ insts, err := instance.environ.Instances([]state.InstanceId{instance.Id()})
1442+ if err != nil {
1443+ return err
1444+ }
1445+ newMaasObject := insts[0].(*maasInstance).maasObject
1446+ instance.maasObject = newMaasObject
1447+ return nil
1448+}
1449+
1450+func (instance *maasInstance) DNSName() (string, error) {
1451+ hostname, err := (*instance.maasObject).GetField("hostname")
1452+ if err != nil {
1453+ return "", err
1454+ }
1455+ return hostname, nil
1456+}
1457+
1458+func (instance *maasInstance) WaitDNSName() (string, error) {
1459+ // A MAAS nodes gets his DNS name when it's created. WaitDNSName,
1460+ // (same as DNSName) just returns the hostname of the node.
1461+ return instance.DNSName()
1462+}
1463+
1464+// MAAS does not do firewalling so these port methods do nothing.
1465+func (instance *maasInstance) OpenPorts(machineId string, ports []params.Port) error {
1466+ log.Debugf("environs/maas: unimplemented OpenPorts() called")
1467+ return nil
1468+}
1469+
1470+func (instance *maasInstance) ClosePorts(machineId string, ports []params.Port) error {
1471+ log.Debugf("environs/maas: unimplemented ClosePorts() called")
1472+ return nil
1473+}
1474+
1475+func (instance *maasInstance) Ports(machineId string) ([]params.Port, error) {
1476+ log.Debugf("environs/maas: unimplemented Ports() called")
1477+ return []params.Port{}, nil
1478+}
1479
1480=== added file 'environs/maas/instance_test.go'
1481--- environs/maas/instance_test.go 1970-01-01 00:00:00 +0000
1482+++ environs/maas/instance_test.go 2013-04-12 10:00:43 +0000
1483@@ -0,0 +1,51 @@
1484+package maas
1485+
1486+import (
1487+ . "launchpad.net/gocheck"
1488+)
1489+
1490+type InstanceTest struct {
1491+ ProviderSuite
1492+}
1493+
1494+var _ = Suite(&InstanceTest{})
1495+
1496+func (s *InstanceTest) TestId(c *C) {
1497+ jsonValue := `{"system_id": "system_id", "test": "test"}`
1498+ obj := s.testMAASObject.TestServer.NewNode(jsonValue)
1499+ resourceURI, _ := obj.GetField("resource_uri")
1500+ instance := maasInstance{&obj, s.environ}
1501+
1502+ c.Check(string(instance.Id()), Equals, resourceURI)
1503+}
1504+
1505+func (s *InstanceTest) TestRefreshInstance(c *C) {
1506+ jsonValue := `{"system_id": "system_id", "test": "test"}`
1507+ obj := s.testMAASObject.TestServer.NewNode(jsonValue)
1508+ s.testMAASObject.TestServer.ChangeNode("system_id", "test2", "test2")
1509+ instance := maasInstance{&obj, s.environ}
1510+
1511+ err := instance.refreshInstance()
1512+
1513+ c.Check(err, IsNil)
1514+ testField, err := (*instance.maasObject).GetField("test2")
1515+ c.Check(err, IsNil)
1516+ c.Check(testField, Equals, "test2")
1517+}
1518+
1519+func (s *InstanceTest) TestDNSName(c *C) {
1520+ jsonValue := `{"hostname": "DNS name", "system_id": "system_id"}`
1521+ obj := s.testMAASObject.TestServer.NewNode(jsonValue)
1522+ instance := maasInstance{&obj, s.environ}
1523+
1524+ dnsName, err := instance.DNSName()
1525+
1526+ c.Check(err, IsNil)
1527+ c.Check(dnsName, Equals, "DNS name")
1528+
1529+ // WaitDNSName() currently simply calls DNSName().
1530+ dnsName, err = instance.WaitDNSName()
1531+
1532+ c.Check(err, IsNil)
1533+ c.Check(dnsName, Equals, "DNS name")
1534+}
1535
1536=== added file 'environs/maas/maas_test.go'
1537--- environs/maas/maas_test.go 1970-01-01 00:00:00 +0000
1538+++ environs/maas/maas_test.go 2013-04-12 10:00:43 +0000
1539@@ -0,0 +1,32 @@
1540+package maas
1541+
1542+import (
1543+ . "launchpad.net/gocheck"
1544+ "launchpad.net/gomaasapi"
1545+ "testing"
1546+)
1547+
1548+func TestMAAS(t *testing.T) {
1549+ TestingT(t)
1550+}
1551+
1552+type ProviderSuite struct {
1553+ environ *maasEnviron
1554+ testMAASObject *gomaasapi.TestMAASObject
1555+}
1556+
1557+var _ = Suite(&ProviderSuite{})
1558+
1559+func (s *ProviderSuite) SetUpSuite(c *C) {
1560+ TestMAASObject := gomaasapi.NewTestMAAS("1.0")
1561+ s.testMAASObject = TestMAASObject
1562+ s.environ = &maasEnviron{name: "test env", maasClientUnlocked: &TestMAASObject.MAASObject}
1563+}
1564+
1565+func (s *ProviderSuite) TearDownTest(c *C) {
1566+ s.testMAASObject.TestServer.Clear()
1567+}
1568+
1569+func (s *ProviderSuite) TearDownSuite(c *C) {
1570+ s.testMAASObject.Close()
1571+}
1572
1573=== added file 'environs/maas/state.go'
1574--- environs/maas/state.go 1970-01-01 00:00:00 +0000
1575+++ environs/maas/state.go 2013-04-12 10:00:43 +0000
1576@@ -0,0 +1,47 @@
1577+package maas
1578+
1579+import (
1580+ "bytes"
1581+ "fmt"
1582+ "io/ioutil"
1583+ "launchpad.net/goyaml"
1584+ "launchpad.net/juju-core/state"
1585+)
1586+
1587+const stateFile = "provider-state"
1588+
1589+// Persistent environment state. An environment needs to know what instances
1590+// it manages.
1591+type bootstrapState struct {
1592+ StateInstances []state.InstanceId `yaml:"state-instances"`
1593+}
1594+
1595+// saveState writes the environment's state to the provider-state file stored
1596+// in the environment's storage.
1597+func (env *maasEnviron) saveState(state *bootstrapState) error {
1598+ data, err := goyaml.Marshal(state)
1599+ if err != nil {
1600+ return err
1601+ }
1602+ buf := bytes.NewBuffer(data)
1603+ return env.Storage().Put(stateFile, buf, int64(len(data)))
1604+}
1605+
1606+// loadState reads the environment's state from storage.
1607+func (env *maasEnviron) loadState() (*bootstrapState, error) {
1608+ r, err := env.Storage().Get(stateFile)
1609+ if err != nil {
1610+ return nil, err
1611+ }
1612+ defer r.Close()
1613+ data, err := ioutil.ReadAll(r)
1614+ if err != nil {
1615+ return nil, fmt.Errorf("error reading %q: %v", stateFile, err)
1616+ }
1617+ var state bootstrapState
1618+ err = goyaml.Unmarshal(data, &state)
1619+ if err != nil {
1620+ return nil, fmt.Errorf("error unmarshalling %q: %v", stateFile, err)
1621+ }
1622+ return &state, nil
1623+}
1624
1625=== added file 'environs/maas/state_test.go'
1626--- environs/maas/state_test.go 1970-01-01 00:00:00 +0000
1627+++ environs/maas/state_test.go 2013-04-12 10:00:43 +0000
1628@@ -0,0 +1,23 @@
1629+package maas
1630+
1631+import (
1632+ . "launchpad.net/gocheck"
1633+ "launchpad.net/juju-core/environs"
1634+)
1635+
1636+type StateSuite struct {
1637+ ProviderSuite
1638+}
1639+
1640+var _ = Suite(new(StateSuite))
1641+
1642+func (suite *StateSuite) TestLoadStateReturnsNotFoundPointerForMissingFile(c *C) {
1643+ serverURL := suite.testMAASObject.URL().String()
1644+ config := getTestConfig("loadState-test", serverURL, "a:b:c", "foo")
1645+ env, err := NewEnviron(config)
1646+ c.Assert(err, IsNil)
1647+
1648+ _, err = env.loadState()
1649+
1650+ c.Check(err, FitsTypeOf, &environs.NotFoundError{})
1651+}
1652
1653=== added file 'environs/maas/storage.go'
1654--- environs/maas/storage.go 1970-01-01 00:00:00 +0000
1655+++ environs/maas/storage.go 2013-04-12 10:00:43 +0000
1656@@ -0,0 +1,207 @@
1657+package maas
1658+
1659+import (
1660+ "bytes"
1661+ "encoding/base64"
1662+ "fmt"
1663+ "io"
1664+ "io/ioutil"
1665+ "launchpad.net/gomaasapi"
1666+ "launchpad.net/juju-core/environs"
1667+ "net/url"
1668+ "sort"
1669+ "sync"
1670+)
1671+
1672+type maasStorage struct {
1673+ // Mutex protects the "*Unlocked" fields.
1674+ sync.Mutex
1675+
1676+ // The Environ that this Storage is for.
1677+ environUnlocked *maasEnviron
1678+
1679+ // Reference to the URL on the API where files are stored.
1680+ maasClientUnlocked gomaasapi.MAASObject
1681+}
1682+
1683+var _ environs.Storage = (*maasStorage)(nil)
1684+
1685+func NewStorage(env *maasEnviron) environs.Storage {
1686+ storage := new(maasStorage)
1687+ storage.environUnlocked = env
1688+ storage.maasClientUnlocked = env.getMAASClient().GetSubObject("files")
1689+ return storage
1690+}
1691+
1692+// getSnapshot returns a consistent copy of a maasStorage. Use this if you
1693+// need a consistent view of the object's entire state, without having to
1694+// lock the object the whole time.
1695+//
1696+// An easy mistake to make with "defer" is to keep holding a lock without
1697+// realizing it, while you go on to block on http requests or other slow
1698+// things that don't actually require the lock. In most cases you can just
1699+// create a snapshot first (releasing the lock immediately) and then do the
1700+// rest of the work with the snapshot.
1701+func (stor *maasStorage) getSnapshot() *maasStorage {
1702+ stor.Lock()
1703+ defer stor.Unlock()
1704+
1705+ return &maasStorage{
1706+ environUnlocked: stor.environUnlocked,
1707+ maasClientUnlocked: stor.maasClientUnlocked,
1708+ }
1709+}
1710+
1711+// addressFileObject creates a MAASObject pointing to a given file.
1712+// Takes out a lock on the storage object to get a consistent view.
1713+func (stor *maasStorage) addressFileObject(name string) gomaasapi.MAASObject {
1714+ stor.Lock()
1715+ defer stor.Unlock()
1716+ return stor.maasClientUnlocked.GetSubObject(name)
1717+}
1718+
1719+// retrieveFileObject retrieves the information of the named file, including
1720+// its download URL and its contents, as a MAASObject.
1721+//
1722+// This may return many different errors, but specifically, it returns
1723+// (a pointer to) environs.NotFoundError if the file did not exist.
1724+//
1725+// The function takes out a lock on the storage object.
1726+func (stor *maasStorage) retrieveFileObject(name string) (gomaasapi.MAASObject, error) {
1727+ obj, err := stor.addressFileObject(name).Get()
1728+ if err != nil {
1729+ noObj := gomaasapi.MAASObject{}
1730+ serverErr, ok := err.(gomaasapi.ServerError)
1731+ if ok && serverErr.StatusCode == 404 {
1732+ msg := fmt.Errorf("file '%s' not found", name)
1733+ return noObj, &environs.NotFoundError{msg}
1734+ }
1735+ msg := fmt.Errorf("could not access file '%s': %v", name, err)
1736+ return noObj, msg
1737+ }
1738+ return obj, nil
1739+}
1740+
1741+// Get is specified in the Storage interface.
1742+func (stor *maasStorage) Get(name string) (io.ReadCloser, error) {
1743+ fileObj, err := stor.retrieveFileObject(name)
1744+ if err != nil {
1745+ return nil, err
1746+ }
1747+ data, err := fileObj.GetField("content")
1748+ if err != nil {
1749+ return nil, fmt.Errorf("could not extract file content for %s: %v", name, err)
1750+ }
1751+ buf, err := base64.StdEncoding.DecodeString(data)
1752+ if err != nil {
1753+ return nil, fmt.Errorf("bad data in file '%s': %v", name, err)
1754+ }
1755+ return ioutil.NopCloser(bytes.NewReader(buf)), nil
1756+}
1757+
1758+// extractFilenames returns the filenames from a "list" operation on the
1759+// MAAS API, sorted by name.
1760+func (stor *maasStorage) extractFilenames(listResult gomaasapi.JSONObject) ([]string, error) {
1761+ list, err := listResult.GetArray()
1762+ if err != nil {
1763+ return nil, err
1764+ }
1765+ result := make([]string, len(list))
1766+ for index, entry := range list {
1767+ file, err := entry.GetMap()
1768+ if err != nil {
1769+ return nil, err
1770+ }
1771+ filename, err := file["filename"].GetString()
1772+ if err != nil {
1773+ return nil, err
1774+ }
1775+ result[index] = filename
1776+ }
1777+ sort.Strings(result)
1778+ return result, nil
1779+}
1780+
1781+// List is specified in the Storage interface.
1782+func (stor *maasStorage) List(prefix string) ([]string, error) {
1783+ params := make(url.Values)
1784+ params.Add("prefix", prefix)
1785+ snapshot := stor.getSnapshot()
1786+ obj, err := snapshot.maasClientUnlocked.CallGet("list", params)
1787+ if err != nil {
1788+ return nil, err
1789+ }
1790+ return snapshot.extractFilenames(obj)
1791+}
1792+
1793+// URL is specified in the Storage interface.
1794+func (stor *maasStorage) URL(name string) (string, error) {
1795+ fileObj, err := stor.retrieveFileObject(name)
1796+ if err != nil {
1797+ return "", err
1798+ }
1799+ uri, err := fileObj.GetField("anon_resource_uri")
1800+ if err != nil {
1801+ msg := fmt.Errorf("could not get file's download URL (may be an outdated MAAS): %s", err)
1802+ return "", msg
1803+ }
1804+
1805+ partialURL, err := url.Parse(uri)
1806+ if err != nil {
1807+ return "", err
1808+ }
1809+ fullURL := fileObj.URL().ResolveReference(partialURL)
1810+ return fullURL.String(), nil
1811+}
1812+
1813+// Put is specified in the Storage interface.
1814+func (stor *maasStorage) Put(name string, r io.Reader, length int64) error {
1815+ data, err := ioutil.ReadAll(io.LimitReader(r, length))
1816+ if err != nil {
1817+ return err
1818+ }
1819+ params := url.Values{"filename": {name}}
1820+ files := map[string][]byte{"file": data}
1821+ snapshot := stor.getSnapshot()
1822+ _, err = snapshot.maasClientUnlocked.CallPostFiles("add", params, files)
1823+ return err
1824+}
1825+
1826+// Remove is specified in the Storage interface.
1827+func (stor *maasStorage) Remove(name string) error {
1828+ // The only thing that can go wrong here, really, is that the file
1829+ // does not exist. But deletion is idempotent: deleting a file that
1830+ // is no longer there anyway is success, not failure.
1831+ stor.getSnapshot().maasClientUnlocked.GetSubObject(name).Delete()
1832+ return nil
1833+}
1834+
1835+func (stor *maasStorage) deleteAll() error {
1836+ names, err := stor.List("")
1837+ if err != nil {
1838+ return err
1839+ }
1840+ // Remove all the objects in parallel so that we incur less round-trips.
1841+ // If we're in danger of having hundreds of objects,
1842+ // we'll want to change this to limit the number
1843+ // of concurrent operations.
1844+ var wg sync.WaitGroup
1845+ wg.Add(len(names))
1846+ errc := make(chan error, len(names))
1847+ for _, name := range names {
1848+ name := name
1849+ go func() {
1850+ defer wg.Done()
1851+ if err := stor.Remove(name); err != nil {
1852+ errc <- err
1853+ }
1854+ }()
1855+ }
1856+ wg.Wait()
1857+ select {
1858+ case err := <-errc:
1859+ return fmt.Errorf("cannot delete all provider state: %v", err)
1860+ default:
1861+ }
1862+ return nil
1863+}
1864
1865=== added file 'environs/maas/storage_test.go'
1866--- environs/maas/storage_test.go 1970-01-01 00:00:00 +0000
1867+++ environs/maas/storage_test.go 2013-04-12 10:00:43 +0000
1868@@ -0,0 +1,385 @@
1869+package maas
1870+
1871+import (
1872+ "bytes"
1873+ "encoding/base64"
1874+ "io/ioutil"
1875+ . "launchpad.net/gocheck"
1876+ "launchpad.net/gomaasapi"
1877+ "launchpad.net/juju-core/environs"
1878+ "math/rand"
1879+ "net/http"
1880+ "net/url"
1881+ "sync"
1882+)
1883+
1884+type StorageSuite struct {
1885+ ProviderSuite
1886+}
1887+
1888+var _ = Suite(new(StorageSuite))
1889+
1890+// makeStorage creates a MAAS storage object for the running test.
1891+func (s *StorageSuite) makeStorage(name string) *maasStorage {
1892+ maasobj := s.testMAASObject.MAASObject
1893+ env := maasEnviron{name: name, maasClientUnlocked: &maasobj}
1894+ return NewStorage(&env).(*maasStorage)
1895+}
1896+
1897+// makeRandomBytes returns an array of arbitrary byte values.
1898+func makeRandomBytes(length int) []byte {
1899+ data := make([]byte, length)
1900+ for index := range data {
1901+ data[index] = byte(rand.Intn(256))
1902+ }
1903+ return data
1904+}
1905+
1906+// fakeStoredFile creates a file directly in the (simulated) MAAS file store.
1907+// It will contain an arbitrary amount of random data. The contents are also
1908+// returned.
1909+//
1910+// If you want properly random data here, initialize the randomizer first.
1911+// Or don't, if you want consistent (and debuggable) results.
1912+func (s *StorageSuite) fakeStoredFile(storage environs.Storage, name string) gomaasapi.MAASObject {
1913+ data := makeRandomBytes(rand.Intn(10))
1914+ return s.testMAASObject.TestServer.NewFile(name, data)
1915+}
1916+
1917+func (s *StorageSuite) TestGetSnapshotCreatesClone(c *C) {
1918+ original := s.makeStorage("storage-name")
1919+ snapshot := original.getSnapshot()
1920+ c.Check(snapshot.environUnlocked, Equals, original.environUnlocked)
1921+ c.Check(snapshot.maasClientUnlocked.URL().String(), Equals, original.maasClientUnlocked.URL().String())
1922+ // Snapshotting locks the original internally, but does not leave
1923+ // either the original or the snapshot locked.
1924+ unlockedMutexValue := sync.Mutex{}
1925+ c.Check(original.Mutex, Equals, unlockedMutexValue)
1926+ c.Check(snapshot.Mutex, Equals, unlockedMutexValue)
1927+}
1928+
1929+func (s *StorageSuite) TestGetRetrievesFile(c *C) {
1930+ const filename = "stored-data"
1931+ storage := s.makeStorage("get-retrieves-file")
1932+ file := s.fakeStoredFile(storage, filename)
1933+ base64Content, err := file.GetField("content")
1934+ c.Assert(err, IsNil)
1935+ content, err := base64.StdEncoding.DecodeString(base64Content)
1936+ c.Assert(err, IsNil)
1937+
1938+ reader, err := storage.Get(filename)
1939+ c.Assert(err, IsNil)
1940+ defer reader.Close()
1941+
1942+ buf, err := ioutil.ReadAll(reader)
1943+ c.Assert(err, IsNil)
1944+ c.Check(len(buf), Equals, len(content))
1945+ c.Check(buf, DeepEquals, content)
1946+}
1947+
1948+func (s *StorageSuite) TestRetrieveFileObjectReturnsFileObject(c *C) {
1949+ const filename = "myfile"
1950+ stor := s.makeStorage("rfo-test")
1951+ file := s.fakeStoredFile(stor, filename)
1952+ fileURI, err := file.GetField("anon_resource_uri")
1953+ c.Assert(err, IsNil)
1954+ fileContent, err := file.GetField("content")
1955+ c.Assert(err, IsNil)
1956+
1957+ obj, err := stor.retrieveFileObject(filename)
1958+ c.Assert(err, IsNil)
1959+
1960+ uri, err := obj.GetField("anon_resource_uri")
1961+ c.Assert(err, IsNil)
1962+ c.Check(uri, Equals, fileURI)
1963+ content, err := obj.GetField("content")
1964+ c.Check(content, Equals, fileContent)
1965+}
1966+
1967+func (s *StorageSuite) TestRetrieveFileObjectReturnsNotFoundForMissingFile(c *C) {
1968+ stor := s.makeStorage("rfo-test")
1969+ _, err := stor.retrieveFileObject("nonexistent-file")
1970+ c.Assert(err, NotNil)
1971+ c.Check(err, FitsTypeOf, &environs.NotFoundError{})
1972+}
1973+
1974+func (s *StorageSuite) TestRetrieveFileObjectEscapesName(c *C) {
1975+ const filename = "#a?b c&d%e!"
1976+ data := []byte("File contents here")
1977+ stor := s.makeStorage("rfo-test")
1978+ err := stor.Put(filename, bytes.NewReader(data), int64(len(data)))
1979+ c.Assert(err, IsNil)
1980+
1981+ obj, err := stor.retrieveFileObject(filename)
1982+ c.Assert(err, IsNil)
1983+
1984+ base64Content, err := obj.GetField("content")
1985+ c.Assert(err, IsNil)
1986+ content, err := base64.StdEncoding.DecodeString(base64Content)
1987+ c.Assert(err, IsNil)
1988+ c.Check(content, DeepEquals, data)
1989+}
1990+
1991+func (s *StorageSuite) TestFileContentsAreBinary(c *C) {
1992+ const filename = "myfile.bin"
1993+ data := []byte{0, 1, 255, 2, 254, 3}
1994+ stor := s.makeStorage("binary-test")
1995+
1996+ err := stor.Put(filename, bytes.NewReader(data), int64(len(data)))
1997+ c.Assert(err, IsNil)
1998+ file, err := stor.Get(filename)
1999+ c.Assert(err, IsNil)
2000+ content, err := ioutil.ReadAll(file)
2001+ c.Assert(err, IsNil)
2002+
2003+ c.Check(content, DeepEquals, data)
2004+}
2005+
2006+func (s *StorageSuite) TestGetReturnsNotFoundErrorIfNotFound(c *C) {
2007+ const filename = "lost-data"
2008+ storage := NewStorage(s.environ)
2009+ _, err := storage.Get(filename)
2010+ c.Assert(err, FitsTypeOf, &environs.NotFoundError{})
2011+}
2012+
2013+func (s *StorageSuite) TestListReturnsEmptyIfNoFilesStored(c *C) {
2014+ storage := NewStorage(s.environ)
2015+ listing, err := storage.List("")
2016+ c.Assert(err, IsNil)
2017+ c.Check(listing, DeepEquals, []string{})
2018+}
2019+
2020+func (s *StorageSuite) TestListReturnsAllFilesIfPrefixEmpty(c *C) {
2021+ storage := NewStorage(s.environ)
2022+ files := []string{"1a", "2b", "3c"}
2023+ for _, name := range files {
2024+ s.fakeStoredFile(storage, name)
2025+ }
2026+
2027+ listing, err := storage.List("")
2028+ c.Assert(err, IsNil)
2029+ c.Check(listing, DeepEquals, files)
2030+}
2031+
2032+func (s *StorageSuite) TestListSortsResults(c *C) {
2033+ storage := NewStorage(s.environ)
2034+ files := []string{"4d", "1a", "3c", "2b"}
2035+ for _, name := range files {
2036+ s.fakeStoredFile(storage, name)
2037+ }
2038+
2039+ listing, err := storage.List("")
2040+ c.Assert(err, IsNil)
2041+ c.Check(listing, DeepEquals, []string{"1a", "2b", "3c", "4d"})
2042+}
2043+
2044+func (s *StorageSuite) TestListReturnsNoFilesIfNoFilesMatchPrefix(c *C) {
2045+ storage := NewStorage(s.environ)
2046+ s.fakeStoredFile(storage, "foo")
2047+
2048+ listing, err := storage.List("bar")
2049+ c.Assert(err, IsNil)
2050+ c.Check(listing, DeepEquals, []string{})
2051+}
2052+
2053+func (s *StorageSuite) TestListReturnsOnlyFilesWithMatchingPrefix(c *C) {
2054+ storage := NewStorage(s.environ)
2055+ s.fakeStoredFile(storage, "abc")
2056+ s.fakeStoredFile(storage, "xyz")
2057+
2058+ listing, err := storage.List("x")
2059+ c.Assert(err, IsNil)
2060+ c.Check(listing, DeepEquals, []string{"xyz"})
2061+}
2062+
2063+func (s *StorageSuite) TestListMatchesPrefixOnly(c *C) {
2064+ storage := NewStorage(s.environ)
2065+ s.fakeStoredFile(storage, "abc")
2066+ s.fakeStoredFile(storage, "xabc")
2067+
2068+ listing, err := storage.List("a")
2069+ c.Assert(err, IsNil)
2070+ c.Check(listing, DeepEquals, []string{"abc"})
2071+}
2072+
2073+func (s *StorageSuite) TestListOperatesOnFlatNamespace(c *C) {
2074+ storage := NewStorage(s.environ)
2075+ s.fakeStoredFile(storage, "a/b/c/d")
2076+
2077+ listing, err := storage.List("a/b")
2078+ c.Assert(err, IsNil)
2079+ c.Check(listing, DeepEquals, []string{"a/b/c/d"})
2080+}
2081+
2082+// getFileAtURL requests, and returns, the file at the given URL.
2083+func getFileAtURL(fileURL string) ([]byte, error) {
2084+ response, err := http.Get(fileURL)
2085+ if err != nil {
2086+ return nil, err
2087+ }
2088+ body, err := ioutil.ReadAll(response.Body)
2089+ if err != nil {
2090+ return nil, err
2091+ }
2092+ return body, nil
2093+}
2094+
2095+func (s *StorageSuite) TestURLReturnsURLCorrespondingToFile(c *C) {
2096+ const filename = "my-file.txt"
2097+ storage := NewStorage(s.environ).(*maasStorage)
2098+ file := s.fakeStoredFile(storage, filename)
2099+ // The file contains an anon_resource_uri, which lacks a network part
2100+ // (but will probably contain a query part). anonURL will be the
2101+ // file's full URL.
2102+ anonURI, err := file.GetField("anon_resource_uri")
2103+ c.Assert(err, IsNil)
2104+ parsedURI, err := url.Parse(anonURI)
2105+ c.Assert(err, IsNil)
2106+ anonURL := storage.maasClientUnlocked.URL().ResolveReference(parsedURI)
2107+ c.Assert(err, IsNil)
2108+
2109+ fileURL, err := storage.URL(filename)
2110+ c.Assert(err, IsNil)
2111+
2112+ c.Check(fileURL, NotNil)
2113+ c.Check(fileURL, Equals, anonURL.String())
2114+}
2115+
2116+func (s *StorageSuite) TestPutStoresRetrievableFile(c *C) {
2117+ const filename = "broken-toaster.jpg"
2118+ contents := []byte("Contents here")
2119+ length := int64(len(contents))
2120+ storage := NewStorage(s.environ)
2121+
2122+ err := storage.Put(filename, bytes.NewReader(contents), length)
2123+
2124+ reader, err := storage.Get(filename)
2125+ c.Assert(err, IsNil)
2126+ defer reader.Close()
2127+
2128+ buf, err := ioutil.ReadAll(reader)
2129+ c.Assert(err, IsNil)
2130+ c.Check(buf, DeepEquals, contents)
2131+}
2132+
2133+func (s *StorageSuite) TestPutOverwritesFile(c *C) {
2134+ const filename = "foo.bar"
2135+ storage := NewStorage(s.environ)
2136+ s.fakeStoredFile(storage, filename)
2137+ newContents := []byte("Overwritten")
2138+
2139+ err := storage.Put(filename, bytes.NewReader(newContents), int64(len(newContents)))
2140+ c.Assert(err, IsNil)
2141+
2142+ reader, err := storage.Get(filename)
2143+ c.Assert(err, IsNil)
2144+ defer reader.Close()
2145+
2146+ buf, err := ioutil.ReadAll(reader)
2147+ c.Assert(err, IsNil)
2148+ c.Check(len(buf), Equals, len(newContents))
2149+ c.Check(buf, DeepEquals, newContents)
2150+}
2151+
2152+func (s *StorageSuite) TestPutStopsAtGivenLength(c *C) {
2153+ const filename = "xyzzyz.2.xls"
2154+ const length = 5
2155+ contents := []byte("abcdefghijklmnopqrstuvwxyz")
2156+ storage := NewStorage(s.environ)
2157+
2158+ err := storage.Put(filename, bytes.NewReader(contents), length)
2159+ c.Assert(err, IsNil)
2160+
2161+ reader, err := storage.Get(filename)
2162+ c.Assert(err, IsNil)
2163+ defer reader.Close()
2164+
2165+ buf, err := ioutil.ReadAll(reader)
2166+ c.Assert(err, IsNil)
2167+ c.Check(len(buf), Equals, length)
2168+}
2169+
2170+func (s *StorageSuite) TestPutToExistingFileTruncatesAtGivenLength(c *C) {
2171+ const filename = "a-file-which-is-mine"
2172+ oldContents := []byte("abcdefghijklmnopqrstuvwxyz")
2173+ newContents := []byte("xyz")
2174+ storage := NewStorage(s.environ)
2175+ err := storage.Put(filename, bytes.NewReader(oldContents), int64(len(oldContents)))
2176+ c.Assert(err, IsNil)
2177+
2178+ err = storage.Put(filename, bytes.NewReader(newContents), int64(len(newContents)))
2179+ c.Assert(err, IsNil)
2180+
2181+ reader, err := storage.Get(filename)
2182+ c.Assert(err, IsNil)
2183+ defer reader.Close()
2184+
2185+ buf, err := ioutil.ReadAll(reader)
2186+ c.Assert(err, IsNil)
2187+ c.Check(len(buf), Equals, len(newContents))
2188+ c.Check(buf, DeepEquals, newContents)
2189+}
2190+
2191+func (s *StorageSuite) TestRemoveDeletesFile(c *C) {
2192+ const filename = "doomed.txt"
2193+ storage := NewStorage(s.environ)
2194+ s.fakeStoredFile(storage, filename)
2195+
2196+ err := storage.Remove(filename)
2197+ c.Assert(err, IsNil)
2198+
2199+ _, err = storage.Get(filename)
2200+ c.Assert(err, FitsTypeOf, &environs.NotFoundError{})
2201+
2202+ listing, err := storage.List(filename)
2203+ c.Assert(err, IsNil)
2204+ c.Assert(listing, DeepEquals, []string{})
2205+}
2206+
2207+func (s *StorageSuite) TestRemoveIsIdempotent(c *C) {
2208+ const filename = "half-a-file"
2209+ storage := NewStorage(s.environ)
2210+ s.fakeStoredFile(storage, filename)
2211+
2212+ err := storage.Remove(filename)
2213+ c.Assert(err, IsNil)
2214+
2215+ err = storage.Remove(filename)
2216+ c.Assert(err, IsNil)
2217+}
2218+
2219+func (s *StorageSuite) TestNamesMayHaveSlashes(c *C) {
2220+ const filename = "name/with/slashes"
2221+ content := []byte("File contents")
2222+ storage := NewStorage(s.environ)
2223+
2224+ err := storage.Put(filename, bytes.NewReader(content), int64(len(content)))
2225+ c.Assert(err, IsNil)
2226+
2227+ // There's not much we can say about the anonymous URL, except that
2228+ // we get one.
2229+ anonURL, err := storage.URL(filename)
2230+ c.Assert(err, IsNil)
2231+ c.Check(anonURL, Matches, "http[s]*://.*")
2232+
2233+ reader, err := storage.Get(filename)
2234+ c.Assert(err, IsNil)
2235+ defer reader.Close()
2236+ data, err := ioutil.ReadAll(reader)
2237+ c.Assert(err, IsNil)
2238+ c.Check(data, DeepEquals, content)
2239+}
2240+
2241+func (s *StorageSuite) TestDeleteAllDeletesAllFiles(c *C) {
2242+ storage := s.makeStorage("get-retrieves-file")
2243+ const filename1 = "stored-data1"
2244+ s.fakeStoredFile(storage, filename1)
2245+ const filename2 = "stored-data2"
2246+ s.fakeStoredFile(storage, filename2)
2247+
2248+ err := storage.deleteAll()
2249+ c.Assert(err, IsNil)
2250+ listing, err := storage.List("")
2251+ c.Assert(err, IsNil)
2252+ c.Assert(listing, DeepEquals, []string{})
2253+}
2254
2255=== added file 'environs/maas/util.go'
2256--- environs/maas/util.go 1970-01-01 00:00:00 +0000
2257+++ environs/maas/util.go 2013-04-12 10:00:43 +0000
2258@@ -0,0 +1,83 @@
2259+package maas
2260+
2261+import (
2262+ "fmt"
2263+ "launchpad.net/goyaml"
2264+ cloudinit_core "launchpad.net/juju-core/cloudinit"
2265+ "launchpad.net/juju-core/environs/cloudinit"
2266+ "launchpad.net/juju-core/log"
2267+ "launchpad.net/juju-core/state"
2268+ "launchpad.net/juju-core/trivial"
2269+ "net/url"
2270+ "strings"
2271+)
2272+
2273+// extractSystemId extracts the 'system_id' part from an InstanceId.
2274+// "/MAAS/api/1.0/nodes/system_id/" => "system_id"
2275+func extractSystemId(instanceId state.InstanceId) string {
2276+ trimmed := strings.TrimRight(string(instanceId), "/")
2277+ split := strings.Split(trimmed, "/")
2278+ return split[len(split)-1]
2279+}
2280+
2281+// getSystemIdValues returns a url.Values object with all the 'system_ids'
2282+// from the given instanceIds stored under the key 'id'. This is used
2283+// to filter out instances when listing the nodes objects.
2284+func getSystemIdValues(instanceIds []state.InstanceId) url.Values {
2285+ values := url.Values{}
2286+ for _, instanceId := range instanceIds {
2287+ values.Add("id", extractSystemId(instanceId))
2288+ }
2289+ return values
2290+}
2291+
2292+// userData returns a zipped cloudinit config.
2293+func userData(cfg *cloudinit.MachineConfig, scripts ...string) ([]byte, error) {
2294+ cloudcfg := cloudinit_core.New()
2295+ for _, script := range scripts {
2296+ cloudcfg.AddRunCmd(script)
2297+ }
2298+ cloudcfg, err := cloudinit.Configure(cfg, cloudcfg)
2299+ if err != nil {
2300+ return nil, err
2301+ }
2302+ data, err := cloudcfg.Render()
2303+ if err != nil {
2304+ return nil, err
2305+ }
2306+ cdata := trivial.Gzip(data)
2307+ log.Debugf("environs/maas: maas user data; %d bytes", len(cdata))
2308+ return cdata, nil
2309+}
2310+
2311+// machineInfo is the structure used to pass information between the provider
2312+// and the agent running on a node.
2313+// When a node is started, the provider code creates a machineInfo object
2314+// containing information about the node being started and configures
2315+// cloudinit to get a YAML representation of that object written on the node's
2316+// filesystem during its first startup. That file is then read by the juju
2317+// agent running on the node and converted back into a machineInfo object.
2318+type machineInfo struct {
2319+ InstanceId string `yaml:,omitempty`
2320+ Hostname string `yaml:,omitempty`
2321+}
2322+
2323+var _MAASInstanceFilename = jujuDataDir + "/MAASmachine.txt"
2324+
2325+// cloudinitRunCmd returns the shell command that, when run, will create the
2326+// "machine info" file containing the instanceId and the hostname of a machine.
2327+// That command is destined to be used by cloudinit.
2328+func (info *machineInfo) cloudinitRunCmd() (string, error) {
2329+ yaml, err := goyaml.Marshal(info)
2330+ if err != nil {
2331+ return "", err
2332+ }
2333+ script := fmt.Sprintf(`mkdir -p %s; echo -n %s > %s`, trivial.ShQuote(jujuDataDir), trivial.ShQuote(string(yaml)), trivial.ShQuote(_MAASInstanceFilename))
2334+ return script, nil
2335+}
2336+
2337+// load loads the "machine info" file and parse the content into the info
2338+// object.
2339+func (info *machineInfo) load() error {
2340+ return trivial.ReadYaml(_MAASInstanceFilename, info)
2341+}
2342
2343=== added file 'environs/maas/util_test.go'
2344--- environs/maas/util_test.go 1970-01-01 00:00:00 +0000
2345+++ environs/maas/util_test.go 2013-04-12 10:00:43 +0000
2346@@ -0,0 +1,129 @@
2347+package maas
2348+
2349+import (
2350+ "fmt"
2351+ . "launchpad.net/gocheck"
2352+ "launchpad.net/goyaml"
2353+ "launchpad.net/juju-core/environs/cloudinit"
2354+ "launchpad.net/juju-core/environs/config"
2355+ "launchpad.net/juju-core/state"
2356+ "launchpad.net/juju-core/state/api"
2357+ "launchpad.net/juju-core/testing"
2358+ "launchpad.net/juju-core/trivial"
2359+ "launchpad.net/juju-core/version"
2360+)
2361+
2362+type UtilSuite struct{}
2363+
2364+var _ = Suite(&UtilSuite{})
2365+
2366+func (s *UtilSuite) TestExtractSystemId(c *C) {
2367+ instanceId := state.InstanceId("/MAAS/api/1.0/nodes/system_id/")
2368+
2369+ systemId := extractSystemId(instanceId)
2370+
2371+ c.Check(systemId, Equals, "system_id")
2372+}
2373+
2374+func (s *UtilSuite) TestGetSystemIdValues(c *C) {
2375+ instanceId1 := state.InstanceId("/MAAS/api/1.0/nodes/system_id1/")
2376+ instanceId2 := state.InstanceId("/MAAS/api/1.0/nodes/system_id2/")
2377+ instanceIds := []state.InstanceId{instanceId1, instanceId2}
2378+
2379+ values := getSystemIdValues(instanceIds)
2380+
2381+ c.Check(values["id"], DeepEquals, []string{"system_id1", "system_id2"})
2382+}
2383+
2384+func (s *UtilSuite) TestUserData(c *C) {
2385+ testJujuHome := c.MkDir()
2386+ defer config.SetJujuHome(config.SetJujuHome(testJujuHome))
2387+ tools := &state.Tools{
2388+ URL: "http://foo.com/tools/juju1.2.3-linux-amd64.tgz",
2389+ Binary: version.MustParseBinary("1.2.3-linux-amd64"),
2390+ }
2391+ envConfig, err := config.New(map[string]interface{}{
2392+ "type": "maas",
2393+ "name": "foo",
2394+ "default-series": "series",
2395+ "authorized-keys": "keys",
2396+ "ca-cert": testing.CACert,
2397+ })
2398+ c.Assert(err, IsNil)
2399+
2400+ cfg := &cloudinit.MachineConfig{
2401+ MachineId: "10",
2402+ MachineNonce: "5432",
2403+ Tools: tools,
2404+ StateServerCert: []byte(testing.ServerCert),
2405+ StateServerKey: []byte(testing.ServerKey),
2406+ StateInfo: &state.Info{
2407+ Password: "pw1",
2408+ CACert: []byte("CA CERT\n" + testing.CACert),
2409+ },
2410+ APIInfo: &api.Info{
2411+ Password: "pw2",
2412+ CACert: []byte("CA CERT\n" + testing.CACert),
2413+ },
2414+ DataDir: jujuDataDir,
2415+ MongoPort: mgoPort,
2416+ Config: envConfig,
2417+ APIPort: apiPort,
2418+ StateServer: true,
2419+ }
2420+ script1 := "script1"
2421+ script2 := "script2"
2422+ scripts := []string{script1, script2}
2423+ result, err := userData(cfg, scripts...)
2424+ c.Assert(err, IsNil)
2425+
2426+ unzipped, err := trivial.Gunzip(result)
2427+ c.Assert(err, IsNil)
2428+
2429+ config := make(map[interface{}]interface{})
2430+ err = goyaml.Unmarshal(unzipped, &config)
2431+ c.Assert(err, IsNil)
2432+
2433+ // Just check that the cloudinit config looks good.
2434+ c.Check(config["apt_upgrade"], Equals, true)
2435+ // The scripts given to userData where added as the first
2436+ // commands to be run.
2437+ runCmd := config["runcmd"].([]interface{})
2438+ c.Check(runCmd[0], Equals, script1)
2439+ c.Check(runCmd[1], Equals, script2)
2440+}
2441+
2442+func (s *UtilSuite) TestMachineInfoCloudinitRunCmd(c *C) {
2443+ instanceId := "instanceId"
2444+ hostname := "hostname"
2445+ filename := "path/to/file"
2446+ old_MAASInstanceFilename := _MAASInstanceFilename
2447+ _MAASInstanceFilename = filename
2448+ defer func() { _MAASInstanceFilename = old_MAASInstanceFilename }()
2449+ info := machineInfo{instanceId, hostname}
2450+
2451+ script, err := info.cloudinitRunCmd()
2452+
2453+ c.Assert(err, IsNil)
2454+ yaml, err := goyaml.Marshal(info)
2455+ c.Assert(err, IsNil)
2456+ expected := fmt.Sprintf("mkdir -p '%s'; echo -n '%s' > '%s'", jujuDataDir, yaml, filename)
2457+ c.Check(script, Equals, expected)
2458+}
2459+
2460+func (s *UtilSuite) TestMachineInfoLoad(c *C) {
2461+ instanceId := "instanceId"
2462+ hostname := "hostname"
2463+ yaml := fmt.Sprintf("instanceid: %s\nhostname: %s\n", instanceId, hostname)
2464+ filename := createTempFile(c, []byte(yaml))
2465+ old_MAASInstanceFilename := _MAASInstanceFilename
2466+ _MAASInstanceFilename = filename
2467+ defer func() { _MAASInstanceFilename = old_MAASInstanceFilename }()
2468+ info := machineInfo{}
2469+
2470+ err := info.load()
2471+
2472+ c.Assert(err, IsNil)
2473+ c.Check(info.InstanceId, Equals, instanceId)
2474+ c.Check(info.Hostname, Equals, hostname)
2475+}

Subscribers

People subscribed via source and target branches