Merge lp:~thumper/juju-core/new-agent-format into lp:~go-bot/juju-core/trunk
- new-agent-format
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Tim Penhey |
Approved revision: | no longer in the source branch. |
Merged at revision: | 1802 |
Proposed branch: | lp:~thumper/juju-core/new-agent-format |
Merge into: | lp:~go-bot/juju-core/trunk |
Prerequisite: | lp:~thumper/juju-core/pass-through-agent-config |
Diff against target: |
1845 lines (+786/-177) 25 files modified
agent/agent.go (+94/-8) agent/format-1.12.go (+4/-2) agent/format-1.12_whitebox_test.go (+0/-9) agent/format-1.16.go (+207/-0) agent/format-1.16_whitebox_test.go (+143/-0) agent/format.go (+52/-3) agent/format_whitebox_test.go (+75/-2) cmd/jujud/machine.go (+8/-9) container/lxc/lxc.go (+0/-4) environs/cloudinit.go (+5/-4) environs/cloudinit/cloudinit.go (+5/-4) environs/cloudinit/cloudinit_test.go (+46/-34) environs/cloudinit_test.go (+14/-11) environs/manual/agent.go (+2/-2) juju/osenv/vars.go (+8/-10) juju/testing/conn.go (+16/-0) provider/local/environ.go (+10/-7) provider/local/storage/worker.go (+9/-10) provider/maas/environ.go (+2/-2) worker/deployer/simple.go (+9/-3) worker/deployer/simple_test.go (+4/-0) worker/provisioner/lxc-broker.go (+11/-11) worker/provisioner/lxc-broker_test.go (+25/-9) worker/provisioner/provisioner.go (+17/-15) worker/provisioner/provisioner_test.go (+20/-18) |
To merge this branch: | bzr merge lp:~thumper/juju-core/new-agent-format |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju Engineering | Pending | ||
Review via email: mp+183558@code.launchpad.net |
Commit message
Introduce the format 1.16 for agent confg.
With adding the new format, a migrate method was added
to the formatter interface to provide a defined place
to do additional upgrade data munging as needed.
When an old format config is read, it is immediately
migrated and written out using the current format.
The new format allows storing of key/value pairs. These
are now used instead of OS environment variables for
passing on the provider type, lxc bridge, and local
storage information.
Cloudinit no longer writes upstart scripts with extra
environment variables for the machine agents, but instead
writes the extra variables to the agent config. Due to
this change, the MachineEnvironment param was changed
to AgentEnvironment.
Also the StatePort was never read, so it has been removed
from information that is saved.
The cloudinit tests were modified to be agnostic to the
actual format of the agent.conf files.
Description of the change
Introduce the format 1.16 for agent confg.
With adding the new format, a migrate method was added
to the formatter interface to provide a defined place
to do additional upgrade data munging as needed.
When an old format config is read, it is immediately
migrated and written out using the current format.
The new format allows storing of key/value pairs. These
are now used instead of OS environment variables for
passing on the provider type, lxc bridge, and local
storage information.
Cloudinit no longer writes upstart scripts with extra
environment variables for the machine agents, but instead
writes the extra variables to the agent config. Due to
this change, the MachineEnvironment param was changed
to AgentEnvironment.
Also the StatePort was never read, so it has been removed
from information that is saved.
The cloudinit tests were modified to be agnostic to the
actual format of the agent.conf files.
Previous review found here:
https:/
Tim Penhey (thumper) wrote : | # |
Ian Booth (wallyworld) wrote : | # |
Typo:
50 +// most recent state to disk. Since we different agent configs for each
64 + values map[string]string
I'm a little -1 on the name "values". "values" is kinda meaningless. Perhaps "params"? Or "configParams"?
154 +func (c *configInternal) Value(key string) string {
Similarly, "SetParam" perhaps? etc
605 + currentFormat = format_1_16
606 + previousFormat = format_1_12
I'm not sure about this - what happens if we have another new format? Then we have more than one previous format. Shouldn't we iterate over the previous formats till we find one that suits? Now all we do is assume it's the one previous format if loading with the new format fails.
There are white box tests for the config migration etc, but it may be good to have tests for the provisioner etc where an old format is present initially and after starting the format migration is validated? Or perhaps there are such tests and I missed them?
William Reade (fwereade) wrote : | # |
ISTM that Configs are passed around a bit casually, and certainly not
goroutine-safely, and that this will be a timebomb even if the usage
patterns today (we only write on startup) make it theoretically safe.
Discuss...
https:/
File agent/agent.go (right):
https:/
agent/agent.go:85: // Set the value for the specified key.
It's nice to do complete sentences starting with the name of the thing
being documented.
And, man, the lack of Get prefixes is starting to bug me. It just feels
like wilful character-saving at the cost of consistency and clarity...
https:/
agent/agent.go:97: // written to disk, the mutex should be locked prior
to generating any disk
prior to generating disk state, or prior to reading the source state
from which the future disk state will be generated?
https:/
agent/agent.go:217: // that this should not be called more than once by
an agent, I feel that
Hmm... how far do we expect these to propagate? Will we be cloning them?
If so, surely writes will never be safe... and if not, we'll need to
mutex everything we do with them. Right?
William Reade (fwereade) wrote : | # |
WIP due to inactivity
Tim Penhey (thumper) wrote : | # |
Please take a look.
https:/
File agent/agent.go (right):
https:/
agent/agent.go:85: // Set the value for the specified key.
On 2013/09/05 07:05:54, fwereade wrote:
> It's nice to do complete sentences starting with the name of the thing
being
> documented.
> And, man, the lack of Get prefixes is starting to bug me. It just
feels like
> wilful character-saving at the cost of consistency and clarity...
Done.
https:/
agent/agent.go:97: // written to disk, the mutex should be locked prior
to generating any disk
On 2013/09/05 07:05:54, fwereade wrote:
> prior to generating disk state, or prior to reading the source state
from which
> the future disk state will be generated?
Yes, this is what I mean. Effectively noting that it should be locked
at the start of the method, not just prior to writing.
https:/
agent/agent.go:217: // that this should not be called more than once by
an agent, I feel that
On 2013/09/05 07:05:54, fwereade wrote:
> Hmm... how far do we expect these to propagate? Will we be cloning
them? If so,
> surely writes will never be safe... and if not, we'll need to mutex
everything
> we do with them. Right?
We only need to mutex the mutable values, which is the values map. This
is now done. There is only one agent config, and it isn't regenerated,
only read once, and now passed around. With the addition of the mutex
around the setting and getting of the arbitrary values, this is now
thread (go routine) safe.
William Reade (fwereade) wrote : | # |
On 2013/09/12 02:21:06, thumper wrote:
> Please take a look.
> https:/
> File agent/agent.go (right):
https:/
> agent/agent.go:85: // Set the value for the specified key.
> On 2013/09/05 07:05:54, fwereade wrote:
> > It's nice to do complete sentences starting with the name of the
thing being
> > documented.
> >
> > And, man, the lack of Get prefixes is starting to bug me. It just
feels like
> > wilful character-saving at the cost of consistency and clarity...
> Done.
https:/
> agent/agent.go:97: // written to disk, the mutex should be locked
prior to
> generating any disk
> On 2013/09/05 07:05:54, fwereade wrote:
> > prior to generating disk state, or prior to reading the source state
from
> which
> > the future disk state will be generated?
> Yes, this is what I mean. Effectively noting that it should be locked
at the
> start of the method, not just prior to writing.
OK, cool. Thanks.
https:/
> agent/agent.go:217: // that this should not be called more than once
by an
> agent, I feel that
> On 2013/09/05 07:05:54, fwereade wrote:
> > Hmm... how far do we expect these to propagate? Will we be cloning
them? If
> so,
> > surely writes will never be safe... and if not, we'll need to mutex
everything
> > we do with them. Right?
> We only need to mutex the mutable values, which is the values map.
This is now
> done. There is only one agent config, and it isn't regenerated, only
read once,
> and now passed around. With the addition of the mutex around the
setting and
> getting of the arbitrary values, this is now thread (go routine) safe.
Ah, ok. I'm a little concerned about likely future changes in which we
need, eg, an api worker to write out the state info for a machine that's
become a manager node. I guess it's not something we need to worry about
yet, but I'm still worried that it will not be obvious when the time
comes that we *do* need to worry. Will review properly imminently.
William Reade (fwereade) wrote : | # |
LGTM; comments but no blockers.
https:/
File agent/agent.go (right):
https:/
agent/agent.go:103: // when mutliple go-routines may be adding things to
the agent config, and
mutliple
https:/
agent/agent.go:111: // map. Retrieving and setting values here are
protected by the mutex.
To get the read/write thing right across the board, I think we have to
go with a model whereby you tell the config to persist by passing in a
mutate func... but even then, I remain nervous about the N cached copies
of mutually diverging state.
I'm starting to get the impression that this is maybe a case for the
Forbidden Design Pattern... but I can also appreciate that what we have
here is progress even if it's not perfection. Would you just rewrite the
comment above to make the limits of the current model clear and
explicit?
https:/
File agent/format-
https:/
agent/format-
I agree that we should; or possibly that we should be atomically
swapping a symlink. That's surely out of scope, because of backward
compatibility if nothing else, but it deserves its own bug.
https:/
File worker/
https:/
worker/
I feel like we ought to be able to get the id (tag better perhaps?) out
of the agent config.
https:/
worker/
agenttools.
I feel like most of agenttools should really be methods on agentConfig.
YMMV, not one for this CL, etc.
Roger Peppe (rogpeppe) wrote : | # |
A couple of suggestions based on a very quick skim.
https:/
File agent/format-
https:/
agent/format-
base64.
can't we just use string here and below?
the certificates and keys are defined to be in PEM format (it is, or was
at least, even documented in this packages as such), which is ASCII
(and mostly base64 itself).
https:/
File agent/format.go (right):
https:/
agent/format.go:33: formatFilename = "format"
Why do we need the file format to be in a separate file?
Can't we just store the format inside the file itself
(some care would obviously need to be taken when first introducing
the format specifier, and that the file format is sufficiently
general to cater for different formats).
Then we don't need to worry about the config file getting
out of step with its purported format.
Something like:
type agentConfig struct {
Format string
Contents string
}
should do the job (it's a pity that goyaml doesn't have the equivalent
of json.RawMessage).
Tim Penhey (thumper) wrote : | # |
https:/
File agent/agent.go (right):
https:/
agent/agent.go:103: // when mutliple go-routines may be adding things to
the agent config, and
On 2013/09/12 13:21:44, fwereade wrote:
> mutliple
Done.
https:/
agent/agent.go:111: // map. Retrieving and setting values here are
protected by the mutex.
On 2013/09/12 13:21:44, fwereade wrote:
> To get the read/write thing right across the board, I think we have to
go with a
> model whereby you tell the config to persist by passing in a mutate
func... but
> even then, I remain nervous about the N cached copies of mutually
diverging
> state.
> I'm starting to get the impression that this is maybe a case for the
Forbidden
> Design Pattern... but I can also appreciate that what we have here is
progress
> even if it's not perfection. Would you just rewrite the comment above
to make
> the limits of the current model clear and explicit?
I'm not sure I agree here. The agent configuration is defined at
creation time, and the only mutable values are in the values map.
Synchronisation around these and the write methods are sufficient.
Clearly if we are adding other mutating functions to the interface,
these should also be protected. We don't have N cached copies mutually
diverging. We have one instance used in multiple places.
I'll add a bit more to the comment here, but I'm not clear on where you
feel the issue is.
https:/
File agent/format-
https:/
agent/format-
base64.
On 2013/09/12 14:50:58, rog wrote:
> can't we just use string here and below?
> the certificates and keys are defined to be in PEM format (it is, or
was at
> least, even documented in this packages as such), which is ASCII
> (and mostly base64 itself).
This was done to just take the []byte safely into a string.
If it was a string before, why are we passing it around as a byte slice?
We can always change future formats to do this if we decide we want to
change.
https:/
agent/format-
On 2013/09/12 13:21:44, fwereade wrote:
> I agree that we should; or possibly that we should be atomically
swapping a
> symlink. That's surely out of scope, because of backward compatibility
if
> nothing else, but it deserves its own bug.
Agree that it is out of scope for this branch, filed a bug and linked
it.
https:/
File agent/format.go (right):
https:/
agent/format.go:33: formatFilename = "format"
On 2013/09/12 14:50:58, rog wrote:
> Why do we need the file format to be in a separate file?
> Can't we just store the format inside the file itself
> (some care would obviously need to be taken when first introducing...
Preview Diff
1 | === modified file 'agent/agent.go' |
2 | --- agent/agent.go 2013-09-03 02:23:44 +0000 |
3 | +++ agent/agent.go 2013-09-12 22:52:57 +0000 |
4 | @@ -7,6 +7,7 @@ |
5 | "fmt" |
6 | "path" |
7 | "regexp" |
8 | + "sync" |
9 | |
10 | "launchpad.net/loggo" |
11 | |
12 | @@ -20,6 +21,26 @@ |
13 | |
14 | var logger = loggo.GetLogger("juju.agent") |
15 | |
16 | +const ( |
17 | + LxcBridge = "LXC_BRIDGE" |
18 | + ProviderType = "PROVIDER_TYPE" |
19 | + ContainerType = "CONTAINER_TYPE" |
20 | + StorageDir = "STORAGE_DIR" |
21 | + StorageAddr = "STORAGE_ADDR" |
22 | + SharedStorageDir = "SHARED_STORAGE_DIR" |
23 | + SharedStorageAddr = "SHARED_STORAGE_ADDR" |
24 | +) |
25 | + |
26 | +// The Config interface is the sole way that the agent gets access to the |
27 | +// configuration information for the machine and unit agents. There should |
28 | +// only be one instance of a config object for any given agent, and this |
29 | +// interface is passed between multiple go routines. The mutable methods are |
30 | +// protected by a mutex, and it is expected that the caller doesn't modify any |
31 | +// slice that may be returned. |
32 | +// |
33 | +// NOTE: should new mutating methods be added to this interface, consideration |
34 | +// is needed around the synchronisation as a single instance is used in |
35 | +// multiple go routines. |
36 | type Config interface { |
37 | // DataDir returns the data directory. Each agent has a subdirectory |
38 | // containing the configuration files. |
39 | @@ -68,11 +89,33 @@ |
40 | |
41 | // APIServerDetails returns the details needed to run an API server. |
42 | APIServerDetails() (port int, cert, key []byte) |
43 | + |
44 | + // Value returns the value associated with the key, or an empty string if |
45 | + // the key is not found. |
46 | + Value(key string) string |
47 | + |
48 | + // SetValue updates the value for the specified key. |
49 | + SetValue(key, value string) |
50 | } |
51 | |
52 | // Ensure that the configInternal struct implements the Config interface. |
53 | var _ Config = (*configInternal)(nil) |
54 | |
55 | +// The configMutex should be locked before any writing to disk during the |
56 | +// write commands, and unlocked when the writing is complete. This process |
57 | +// wide lock should stop any unintended concurrent writes. This may happen |
58 | +// when multiple go-routines may be adding things to the agent config, and |
59 | +// wanting to persist them to disk. To ensure that the correct data is written |
60 | +// to disk, the mutex should be locked prior to generating any disk state. |
61 | +// This way calls that might get interleaved would always write the most |
62 | +// recent state to disk. Since we have different agent configs for each |
63 | +// agent, and there is only one process for each agent, a simple mutex is |
64 | +// enough for concurrency. The mutex should also be locked around any access |
65 | +// to mutable values, either setting or getting. The only mutable value is |
66 | +// the values map. Retrieving and setting values here are protected by the |
67 | +// mutex. New mutating methods should also be synchronized using this mutex. |
68 | +var configMutex sync.Mutex |
69 | + |
70 | type connectionDetails struct { |
71 | addresses []string |
72 | password string |
73 | @@ -88,8 +131,8 @@ |
74 | oldPassword string |
75 | stateServerCert []byte |
76 | stateServerKey []byte |
77 | - statePort int |
78 | apiPort int |
79 | + values map[string]string |
80 | } |
81 | |
82 | type AgentConfigParams struct { |
83 | @@ -100,6 +143,7 @@ |
84 | StateAddresses []string |
85 | APIAddresses []string |
86 | CACert []byte |
87 | + Values map[string]string |
88 | } |
89 | |
90 | func newConfig(params AgentConfigParams) (*configInternal, error) { |
91 | @@ -123,6 +167,7 @@ |
92 | nonce: params.Nonce, |
93 | caCert: params.CACert, |
94 | oldPassword: params.Password, |
95 | + values: params.Values, |
96 | } |
97 | if len(params.StateAddresses) > 0 { |
98 | config.stateDetails = &connectionDetails{ |
99 | @@ -137,6 +182,9 @@ |
100 | if err := config.check(); err != nil { |
101 | return nil, err |
102 | } |
103 | + if config.values == nil { |
104 | + config.values = make(map[string]string) |
105 | + } |
106 | return config, nil |
107 | } |
108 | |
109 | @@ -168,7 +216,6 @@ |
110 | } |
111 | config.stateServerCert = params.StateServerCert |
112 | config.stateServerKey = params.StateServerKey |
113 | - config.statePort = params.StatePort |
114 | config.apiPort = params.APIPort |
115 | return config, nil |
116 | } |
117 | @@ -181,8 +228,22 @@ |
118 | // ReadConf reads configuration data for the given |
119 | // entity from the given data directory. |
120 | func ReadConf(dataDir, tag string) (Config, error) { |
121 | + // Even though the ReadConf is done at the start of the agent loading, and |
122 | + // that this should not be called more than once by an agent, I feel that |
123 | + // not locking the mutex that is used to protect writes is wrong. |
124 | + configMutex.Lock() |
125 | + defer configMutex.Unlock() |
126 | dir := Dir(dataDir, tag) |
127 | - config, err := currentFormatter.read(dir) |
128 | + format, err := readFormat(dir) |
129 | + if err != nil { |
130 | + return nil, err |
131 | + } |
132 | + logger.Debugf("Reading agent config, format: %s", format) |
133 | + formatter, err := newFormatter(format) |
134 | + if err != nil { |
135 | + return nil, err |
136 | + } |
137 | + config, err := formatter.read(dir) |
138 | if err != nil { |
139 | return nil, err |
140 | } |
141 | @@ -190,12 +251,18 @@ |
142 | if err := config.check(); err != nil { |
143 | return nil, err |
144 | } |
145 | + |
146 | + if format != currentFormat { |
147 | + // Migrate the config to the new format. |
148 | + currentFormatter.migrate(config) |
149 | + // Write the content out in the new format. |
150 | + if err := currentFormatter.write(config); err != nil { |
151 | + logger.Errorf("Problem writing the agent config out in format: %s, %v", currentFormat, err) |
152 | + return nil, err |
153 | + } |
154 | + } |
155 | + |
156 | return config, nil |
157 | - // Read the format for the dir. |
158 | - // Create a formatter for the format |
159 | - // Read in the config |
160 | - // If the format isn't the current format, call the upgrade function, and |
161 | - // write out using the new formatter. |
162 | } |
163 | |
164 | func requiredError(what string) error { |
165 | @@ -222,6 +289,22 @@ |
166 | return result |
167 | } |
168 | |
169 | +func (c *configInternal) Value(key string) string { |
170 | + configMutex.Lock() |
171 | + defer configMutex.Unlock() |
172 | + return c.values[key] |
173 | +} |
174 | + |
175 | +func (c *configInternal) SetValue(key, value string) { |
176 | + configMutex.Lock() |
177 | + defer configMutex.Unlock() |
178 | + if value == "" { |
179 | + delete(c.values, key) |
180 | + } else { |
181 | + c.values[key] = value |
182 | + } |
183 | +} |
184 | + |
185 | func (c *configInternal) APIServerDetails() (port int, cert, key []byte) { |
186 | return c.apiPort, c.stateServerCert, c.stateServerKey |
187 | } |
188 | @@ -308,6 +391,9 @@ |
189 | |
190 | // Write writes the agent configuration. |
191 | func (c *configInternal) Write() error { |
192 | + // Lock is taken prior to generating any content to write. |
193 | + configMutex.Lock() |
194 | + defer configMutex.Unlock() |
195 | return currentFormatter.write(c) |
196 | } |
197 | |
198 | |
199 | === modified file 'agent/format-1.12.go' |
200 | --- agent/format-1.12.go 2013-09-02 03:54:20 +0000 |
201 | +++ agent/format-1.12.go 2013-09-12 22:52:57 +0000 |
202 | @@ -102,8 +102,8 @@ |
203 | oldPassword: conf.OldPassword, |
204 | stateServerCert: conf.StateServerCert, |
205 | stateServerKey: conf.StateServerKey, |
206 | - statePort: conf.StatePort, |
207 | apiPort: conf.APIPort, |
208 | + values: map[string]string{}, |
209 | }, nil |
210 | } |
211 | |
212 | @@ -111,7 +111,6 @@ |
213 | format := &format_1_12Serialization{ |
214 | StateServerCert: config.stateServerCert, |
215 | StateServerKey: config.stateServerKey, |
216 | - StatePort: config.statePort, |
217 | APIPort: config.apiPort, |
218 | OldPassword: config.oldPassword, |
219 | MachineNonce: config.nonce, |
220 | @@ -173,3 +172,6 @@ |
221 | addCommand(`printf '%%s\n' %s > %s`, utils.ShQuote(string(data)), filename) |
222 | return commands, nil |
223 | } |
224 | + |
225 | +func (*formatter_1_12) migrate(config *configInternal) { |
226 | +} |
227 | |
228 | === modified file 'agent/format-1.12_whitebox_test.go' |
229 | --- agent/format-1.12_whitebox_test.go 2013-09-02 03:57:53 +0000 |
230 | +++ agent/format-1.12_whitebox_test.go 2013-09-12 22:52:57 +0000 |
231 | @@ -20,15 +20,6 @@ |
232 | |
233 | var _ = gc.Suite(&format_1_12Suite{}) |
234 | |
235 | -var agentParams = AgentConfigParams{ |
236 | - Tag: "omg", |
237 | - Password: "sekrit", |
238 | - CACert: []byte("ca cert"), |
239 | - StateAddresses: []string{"localhost:1234"}, |
240 | - APIAddresses: []string{"localhost:1235"}, |
241 | - Nonce: "a nonce", |
242 | -} |
243 | - |
244 | func (s *format_1_12Suite) newConfig(c *gc.C) *configInternal { |
245 | params := agentParams |
246 | params.DataDir = c.MkDir() |
247 | |
248 | === added file 'agent/format-1.16.go' |
249 | --- agent/format-1.16.go 1970-01-01 00:00:00 +0000 |
250 | +++ agent/format-1.16.go 2013-09-12 22:52:57 +0000 |
251 | @@ -0,0 +1,207 @@ |
252 | +// Copyright 2013 Canonical Ltd. |
253 | +// Licensed under the AGPLv3, see LICENCE file for details. |
254 | + |
255 | +package agent |
256 | + |
257 | +import ( |
258 | + "encoding/base64" |
259 | + "io/ioutil" |
260 | + "os" |
261 | + "path" |
262 | + |
263 | + "launchpad.net/goyaml" |
264 | + |
265 | + "launchpad.net/juju-core/juju/osenv" |
266 | +) |
267 | + |
268 | +const ( |
269 | + format_1_16 = "format 1.16" |
270 | + // Old environment variables that are now stored in agent config. |
271 | + JujuLxcBridge = "JUJU_LXC_BRIDGE" |
272 | + JujuProviderType = "JUJU_PROVIDER_TYPE" |
273 | + JujuStorageDir = "JUJU_STORAGE_DIR" |
274 | + JujuStorageAddr = "JUJU_STORAGE_ADDR" |
275 | + JujuSharedStorageDir = "JUJU_SHARED_STORAGE_DIR" |
276 | + JujuSharedStorageAddr = "JUJU_SHARED_STORAGE_ADDR" |
277 | +) |
278 | + |
279 | +// formatter_1_16 is the formatter for the 1.16 format. |
280 | +type formatter_1_16 struct { |
281 | +} |
282 | + |
283 | +// format_1_16Serialization holds information for a given agent. |
284 | +type format_1_16Serialization struct { |
285 | + Tag string |
286 | + Nonce string |
287 | + // CACert is base64 encoded |
288 | + CACert string |
289 | + StateAddresses []string `yaml:",omitempty"` |
290 | + StatePassword string `yaml:",omitempty"` |
291 | + |
292 | + APIAddresses []string `yaml:",omitempty"` |
293 | + APIPassword string `yaml:",omitempty"` |
294 | + |
295 | + OldPassword string |
296 | + Values map[string]string |
297 | + |
298 | + // Only state server machines have these next three items |
299 | + StateServerCert string `yaml:",omitempty"` |
300 | + StateServerKey string `yaml:",omitempty"` |
301 | + APIPort int `yaml:",omitempty"` |
302 | +} |
303 | + |
304 | +// Ensure that the formatter_1_16 struct implements the formatter interface. |
305 | +var _ formatter = (*formatter_1_16)(nil) |
306 | + |
307 | +func (*formatter_1_16) configFile(dirName string) string { |
308 | + return path.Join(dirName, "agent.conf") |
309 | +} |
310 | + |
311 | +// decode64 makes sure that for an empty string we have a nil slice, not an |
312 | +// empty slice, which is what the base64 DecodeString function returns. |
313 | +func (*formatter_1_16) decode64(value string) (result []byte, err error) { |
314 | + if value != "" { |
315 | + result, err = base64.StdEncoding.DecodeString(value) |
316 | + } |
317 | + return |
318 | +} |
319 | + |
320 | +func (formatter *formatter_1_16) read(dirName string) (*configInternal, error) { |
321 | + data, err := ioutil.ReadFile(formatter.configFile(dirName)) |
322 | + if err != nil { |
323 | + return nil, err |
324 | + } |
325 | + var format format_1_16Serialization |
326 | + if err := goyaml.Unmarshal(data, &format); err != nil { |
327 | + return nil, err |
328 | + } |
329 | + caCert, err := formatter.decode64(format.CACert) |
330 | + if err != nil { |
331 | + return nil, err |
332 | + } |
333 | + stateServerCert, err := formatter.decode64(format.StateServerCert) |
334 | + if err != nil { |
335 | + return nil, err |
336 | + } |
337 | + stateServerKey, err := formatter.decode64(format.StateServerKey) |
338 | + if err != nil { |
339 | + return nil, err |
340 | + } |
341 | + config := &configInternal{ |
342 | + tag: format.Tag, |
343 | + nonce: format.Nonce, |
344 | + caCert: caCert, |
345 | + oldPassword: format.OldPassword, |
346 | + stateServerCert: stateServerCert, |
347 | + stateServerKey: stateServerKey, |
348 | + apiPort: format.APIPort, |
349 | + values: format.Values, |
350 | + } |
351 | + if len(format.StateAddresses) > 0 { |
352 | + config.stateDetails = &connectionDetails{ |
353 | + format.StateAddresses, |
354 | + format.StatePassword, |
355 | + } |
356 | + } |
357 | + if len(format.APIAddresses) > 0 { |
358 | + config.apiDetails = &connectionDetails{ |
359 | + format.APIAddresses, |
360 | + format.APIPassword, |
361 | + } |
362 | + } |
363 | + return config, nil |
364 | +} |
365 | + |
366 | +func (formatter *formatter_1_16) makeFormat(config *configInternal) *format_1_16Serialization { |
367 | + format := &format_1_16Serialization{ |
368 | + Tag: config.tag, |
369 | + Nonce: config.nonce, |
370 | + CACert: base64.StdEncoding.EncodeToString(config.caCert), |
371 | + OldPassword: config.oldPassword, |
372 | + StateServerCert: base64.StdEncoding.EncodeToString(config.stateServerCert), |
373 | + StateServerKey: base64.StdEncoding.EncodeToString(config.stateServerKey), |
374 | + APIPort: config.apiPort, |
375 | + Values: config.values, |
376 | + } |
377 | + if config.stateDetails != nil { |
378 | + format.StateAddresses = config.stateDetails.addresses |
379 | + format.StatePassword = config.stateDetails.password |
380 | + } |
381 | + if config.apiDetails != nil { |
382 | + format.APIAddresses = config.apiDetails.addresses |
383 | + format.APIPassword = config.apiDetails.password |
384 | + } |
385 | + return format |
386 | +} |
387 | + |
388 | +func (formatter *formatter_1_16) write(config *configInternal) error { |
389 | + dirName := config.Dir() |
390 | + conf := formatter.makeFormat(config) |
391 | + data, err := goyaml.Marshal(conf) |
392 | + if err != nil { |
393 | + return err |
394 | + } |
395 | + // TODO(thumper): 2013-09-13 bug 1224725 |
396 | + // We should be writing to a temp dir first then rename. |
397 | + // Writing the format file makes sure that dirName exists. We should |
398 | + // really be writing the format and new config files into a separate |
399 | + // directory, and renaming the directory, and moving the old agend |
400 | + // directory to ".old". |
401 | + if err := writeFormatFile(dirName, format_1_16); err != nil { |
402 | + return err |
403 | + } |
404 | + newFile := path.Join(dirName, "agent.conf-new") |
405 | + if err := ioutil.WriteFile(newFile, data, 0600); err != nil { |
406 | + return err |
407 | + } |
408 | + if err := os.Rename(newFile, formatter.configFile(dirName)); err != nil { |
409 | + return err |
410 | + } |
411 | + return nil |
412 | +} |
413 | + |
414 | +func (formatter *formatter_1_16) writeCommands(config *configInternal) ([]string, error) { |
415 | + dirName := config.Dir() |
416 | + conf := formatter.makeFormat(config) |
417 | + data, err := goyaml.Marshal(conf) |
418 | + if err != nil { |
419 | + return nil, err |
420 | + } |
421 | + commands := writeCommandsForFormat(dirName, format_1_16) |
422 | + commands = append(commands, |
423 | + writeFileCommands(formatter.configFile(dirName), string(data), 0600)...) |
424 | + return commands, nil |
425 | +} |
426 | + |
427 | +func (*formatter_1_16) migrate(config *configInternal) { |
428 | + for _, name := range []struct { |
429 | + environment string |
430 | + config string |
431 | + }{{ |
432 | + JujuProviderType, |
433 | + ProviderType, |
434 | + }, { |
435 | + osenv.JujuContainerType, |
436 | + ContainerType, |
437 | + }, { |
438 | + JujuLxcBridge, |
439 | + LxcBridge, |
440 | + }, { |
441 | + JujuStorageDir, |
442 | + StorageDir, |
443 | + }, { |
444 | + JujuStorageAddr, |
445 | + StorageAddr, |
446 | + }, { |
447 | + JujuSharedStorageDir, |
448 | + SharedStorageDir, |
449 | + }, { |
450 | + JujuSharedStorageAddr, |
451 | + SharedStorageAddr, |
452 | + }} { |
453 | + value := os.Getenv(name.environment) |
454 | + if value != "" { |
455 | + config.values[name.config] = value |
456 | + } |
457 | + } |
458 | +} |
459 | |
460 | === added file 'agent/format-1.16_whitebox_test.go' |
461 | --- agent/format-1.16_whitebox_test.go 1970-01-01 00:00:00 +0000 |
462 | +++ agent/format-1.16_whitebox_test.go 2013-09-12 22:52:57 +0000 |
463 | @@ -0,0 +1,143 @@ |
464 | +// Copyright 2013 Canonical Ltd. |
465 | +// Licensed under the AGPLv3, see LICENCE file for details. |
466 | + |
467 | +// The format tests are white box tests, meaning that the tests are in the |
468 | +// same package as the code, as all the format details are internal to the |
469 | +// package. |
470 | + |
471 | +package agent |
472 | + |
473 | +import ( |
474 | + "os" |
475 | + "path" |
476 | + |
477 | + gc "launchpad.net/gocheck" |
478 | + |
479 | + "launchpad.net/juju-core/juju/osenv" |
480 | + "launchpad.net/juju-core/testing" |
481 | + jc "launchpad.net/juju-core/testing/checkers" |
482 | +) |
483 | + |
484 | +type format_1_16Suite struct { |
485 | + testing.LoggingSuite |
486 | + formatter formatter_1_16 |
487 | +} |
488 | + |
489 | +var _ = gc.Suite(&format_1_16Suite{}) |
490 | + |
491 | +func (s *format_1_16Suite) newConfig(c *gc.C) *configInternal { |
492 | + params := agentParams |
493 | + params.DataDir = c.MkDir() |
494 | + config, err := newConfig(params) |
495 | + c.Assert(err, gc.IsNil) |
496 | + return config |
497 | +} |
498 | + |
499 | +func (s *format_1_16Suite) TestWriteAgentConfig(c *gc.C) { |
500 | + config := s.newConfig(c) |
501 | + err := s.formatter.write(config) |
502 | + c.Assert(err, gc.IsNil) |
503 | + |
504 | + expectedLocation := path.Join(config.Dir(), "agent.conf") |
505 | + fileInfo, err := os.Stat(expectedLocation) |
506 | + c.Assert(err, gc.IsNil) |
507 | + c.Assert(fileInfo.Mode().IsRegular(), jc.IsTrue) |
508 | + c.Assert(fileInfo.Mode().Perm(), gc.Equals, os.FileMode(0600)) |
509 | + c.Assert(fileInfo.Size(), jc.GreaterThan, 0) |
510 | + |
511 | + formatLocation := path.Join(config.Dir(), formatFilename) |
512 | + fileInfo, err = os.Stat(formatLocation) |
513 | + c.Assert(err, gc.IsNil) |
514 | + c.Assert(fileInfo.Mode().IsRegular(), jc.IsTrue) |
515 | + c.Assert(fileInfo.Mode().Perm(), gc.Equals, os.FileMode(0644)) |
516 | + c.Assert(fileInfo.Size(), jc.GreaterThan, 0) |
517 | + |
518 | + formatContent, err := readFormat(config.Dir()) |
519 | + c.Assert(formatContent, gc.Equals, format_1_16) |
520 | +} |
521 | + |
522 | +func (s *format_1_16Suite) assertWriteAndRead(c *gc.C, config *configInternal) { |
523 | + err := s.formatter.write(config) |
524 | + c.Assert(err, gc.IsNil) |
525 | + // The readConfig is missing the dataDir initially. |
526 | + readConfig, err := s.formatter.read(config.Dir()) |
527 | + c.Assert(err, gc.IsNil) |
528 | + c.Assert(readConfig.dataDir, gc.Equals, "") |
529 | + // This is put in by the ReadConf method that we are avoiding using |
530 | + // becuase it will have side-effects soon around migrating configs. |
531 | + readConfig.dataDir = config.dataDir |
532 | + c.Assert(readConfig, gc.DeepEquals, config) |
533 | +} |
534 | + |
535 | +func (s *format_1_16Suite) TestRead(c *gc.C) { |
536 | + config := s.newConfig(c) |
537 | + s.assertWriteAndRead(c, config) |
538 | +} |
539 | + |
540 | +func (s *format_1_16Suite) TestWriteCommands(c *gc.C) { |
541 | + config := s.newConfig(c) |
542 | + commands, err := s.formatter.writeCommands(config) |
543 | + c.Assert(err, gc.IsNil) |
544 | + c.Assert(commands, gc.HasLen, 5) |
545 | + c.Assert(commands[0], gc.Matches, `mkdir -p '\S+/agents/omg'`) |
546 | + c.Assert(commands[1], gc.Matches, `install -m 644 /dev/null '\S+/agents/omg/format'`) |
547 | + c.Assert(commands[2], gc.Matches, `printf '%s\\n' '.*' > '\S+/agents/omg/format'`) |
548 | + c.Assert(commands[3], gc.Matches, `install -m 600 /dev/null '\S+/agents/omg/agent.conf'`) |
549 | + c.Assert(commands[4], gc.Matches, `printf '%s\\n' '(.|\n)*' > '\S+/agents/omg/agent.conf'`) |
550 | +} |
551 | + |
552 | +func (s *format_1_16Suite) TestReadWriteStateConfig(c *gc.C) { |
553 | + stateParams := StateMachineConfigParams{ |
554 | + AgentConfigParams: agentParams, |
555 | + StateServerCert: []byte("some special cert"), |
556 | + StateServerKey: []byte("a special key"), |
557 | + StatePort: 12345, |
558 | + APIPort: 23456, |
559 | + } |
560 | + stateParams.DataDir = c.MkDir() |
561 | + stateParams.Values = map[string]string{"foo": "bar", "wibble": "wobble"} |
562 | + configInterface, err := NewStateMachineConfig(stateParams) |
563 | + c.Assert(err, gc.IsNil) |
564 | + config, ok := configInterface.(*configInternal) |
565 | + c.Assert(ok, jc.IsTrue) |
566 | + |
567 | + s.assertWriteAndRead(c, config) |
568 | +} |
569 | + |
570 | +func (s *format_1_16Suite) TestMigrate(c *gc.C) { |
571 | + defer testing.PatchEnvironment(JujuLxcBridge, "lxc bridge")() |
572 | + defer testing.PatchEnvironment(JujuProviderType, "provider type")() |
573 | + defer testing.PatchEnvironment(osenv.JujuContainerType, "container type")() |
574 | + defer testing.PatchEnvironment(JujuStorageDir, "storage dir")() |
575 | + defer testing.PatchEnvironment(JujuStorageAddr, "storage addr")() |
576 | + defer testing.PatchEnvironment(JujuSharedStorageDir, "shared storage dir")() |
577 | + defer testing.PatchEnvironment(JujuSharedStorageAddr, "shared storage addr")() |
578 | + |
579 | + config := s.newConfig(c) |
580 | + s.formatter.migrate(config) |
581 | + |
582 | + expected := map[string]string{ |
583 | + LxcBridge: "lxc bridge", |
584 | + ProviderType: "provider type", |
585 | + ContainerType: "container type", |
586 | + StorageDir: "storage dir", |
587 | + StorageAddr: "storage addr", |
588 | + SharedStorageDir: "shared storage dir", |
589 | + SharedStorageAddr: "shared storage addr", |
590 | + } |
591 | + |
592 | + c.Assert(config.values, gc.DeepEquals, expected) |
593 | +} |
594 | + |
595 | +func (s *format_1_16Suite) TestMigrateOnlySetsExisting(c *gc.C) { |
596 | + defer testing.PatchEnvironment(JujuProviderType, "provider type")() |
597 | + |
598 | + config := s.newConfig(c) |
599 | + s.formatter.migrate(config) |
600 | + |
601 | + expected := map[string]string{ |
602 | + ProviderType: "provider type", |
603 | + } |
604 | + |
605 | + c.Assert(config.values, gc.DeepEquals, expected) |
606 | +} |
607 | |
608 | === modified file 'agent/format.go' |
609 | --- agent/format.go 2013-09-02 03:54:20 +0000 |
610 | +++ agent/format.go 2013-09-12 22:52:57 +0000 |
611 | @@ -5,6 +5,12 @@ |
612 | |
613 | import ( |
614 | "fmt" |
615 | + "io/ioutil" |
616 | + "os" |
617 | + "path" |
618 | + "strings" |
619 | + |
620 | + "launchpad.net/juju-core/utils" |
621 | ) |
622 | |
623 | // The format file in the agent config directory is used to identify the |
624 | @@ -25,10 +31,14 @@ |
625 | |
626 | const ( |
627 | formatFilename = "format" |
628 | - currentFormat = format_1_12 |
629 | + currentFormat = format_1_16 |
630 | + previousFormat = format_1_12 |
631 | ) |
632 | |
633 | -var currentFormatter = &formatter_1_12{} |
634 | +var ( |
635 | + currentFormatter = &formatter_1_16{} |
636 | + previousFormatter = &formatter_1_12{} |
637 | +) |
638 | |
639 | // The formatter defines the two methods needed by the formatters for |
640 | // translating to and from the internal, format agnostic, structure. |
641 | @@ -36,16 +46,55 @@ |
642 | read(dirName string) (*configInternal, error) |
643 | write(config *configInternal) error |
644 | writeCommands(config *configInternal) ([]string, error) |
645 | + // Migrate is called when upgrading from the previous format to the current format. |
646 | + migrate(config *configInternal) |
647 | +} |
648 | + |
649 | +func formatFile(dirName string) string { |
650 | + return path.Join(dirName, formatFilename) |
651 | } |
652 | |
653 | func readFormat(dirName string) (string, error) { |
654 | - return currentFormat, nil |
655 | + contents, err := ioutil.ReadFile(formatFile(dirName)) |
656 | + // Once the previousFormat is defined to have a format file (1.16 or |
657 | + // above), not finding a format file should be a real error. |
658 | + if err != nil { |
659 | + return previousFormat, nil |
660 | + } |
661 | + return strings.TrimSpace(string(contents)), nil |
662 | } |
663 | |
664 | func newFormatter(format string) (formatter, error) { |
665 | switch format { |
666 | case currentFormat: |
667 | return currentFormatter, nil |
668 | + case previousFormat: |
669 | + return previousFormatter, nil |
670 | } |
671 | return nil, fmt.Errorf("unknown agent config format") |
672 | } |
673 | + |
674 | +func writeFormatFile(dirName string, format string) error { |
675 | + if err := os.MkdirAll(dirName, 0755); err != nil { |
676 | + return err |
677 | + } |
678 | + newFile := path.Join(dirName, formatFilename+"-new") |
679 | + if err := ioutil.WriteFile(newFile, []byte(format+"\n"), 0644); err != nil { |
680 | + return err |
681 | + } |
682 | + return os.Rename(newFile, formatFile(dirName)) |
683 | +} |
684 | + |
685 | +func writeFileCommands(filename, contents string, permission int) []string { |
686 | + quotedFilename := utils.ShQuote(filename) |
687 | + return []string{ |
688 | + fmt.Sprintf("install -m %o /dev/null %s", permission, quotedFilename), |
689 | + fmt.Sprintf(`printf '%%s\n' %s > %s`, utils.ShQuote(contents), quotedFilename), |
690 | + } |
691 | +} |
692 | + |
693 | +func writeCommandsForFormat(dirName, format string) []string { |
694 | + commands := []string{"mkdir -p " + utils.ShQuote(dirName)} |
695 | + commands = append(commands, writeFileCommands(formatFile(dirName), format, 0644)...) |
696 | + return commands |
697 | +} |
698 | |
699 | === modified file 'agent/format_whitebox_test.go' |
700 | --- agent/format_whitebox_test.go 2013-09-02 03:57:53 +0000 |
701 | +++ agent/format_whitebox_test.go 2013-09-12 22:52:57 +0000 |
702 | @@ -4,6 +4,9 @@ |
703 | package agent |
704 | |
705 | import ( |
706 | + "io/ioutil" |
707 | + "path" |
708 | + |
709 | gc "launchpad.net/gocheck" |
710 | |
711 | "launchpad.net/juju-core/testing" |
712 | @@ -15,9 +18,35 @@ |
713 | |
714 | var _ = gc.Suite(&formatSuite{}) |
715 | |
716 | +// The agentParams are used by the specific formatter whitebox tests, and is |
717 | +// located here for easy reuse. |
718 | +var agentParams = AgentConfigParams{ |
719 | + Tag: "omg", |
720 | + Password: "sekrit", |
721 | + CACert: []byte("ca cert"), |
722 | + StateAddresses: []string{"localhost:1234"}, |
723 | + APIAddresses: []string{"localhost:1235"}, |
724 | + Nonce: "a nonce", |
725 | +} |
726 | + |
727 | +func (*formatSuite) TestReadFormatEmptyDir(c *gc.C) { |
728 | + // Since the previous format didn't have a format file, a missing format |
729 | + // should return the previous format. Once we are over the hump of |
730 | + // missing format files, a missing format file should generate an error. |
731 | + dir := c.MkDir() |
732 | + format, err := readFormat(dir) |
733 | + c.Assert(format, gc.Equals, previousFormat) |
734 | + c.Assert(err, gc.IsNil) |
735 | +} |
736 | + |
737 | func (*formatSuite) TestReadFormat(c *gc.C) { |
738 | - format, err := readFormat("ignored") |
739 | - c.Assert(format, gc.Equals, currentFormat) |
740 | + dir := c.MkDir() |
741 | + // Make sure that the write adds the carriage return to show that |
742 | + // this is stripped off for the returned format. |
743 | + err := ioutil.WriteFile(path.Join(dir, formatFilename), []byte("some format\n"), 0644) |
744 | + c.Assert(err, gc.IsNil) |
745 | + format, err := readFormat(dir) |
746 | + c.Assert(format, gc.Equals, "some format") |
747 | c.Assert(err, gc.IsNil) |
748 | } |
749 | |
750 | @@ -26,7 +55,51 @@ |
751 | c.Assert(formatter, gc.NotNil) |
752 | c.Assert(err, gc.IsNil) |
753 | |
754 | + formatter, err = newFormatter(previousFormat) |
755 | + c.Assert(formatter, gc.NotNil) |
756 | + c.Assert(err, gc.IsNil) |
757 | + |
758 | formatter, err = newFormatter("other") |
759 | c.Assert(formatter, gc.IsNil) |
760 | c.Assert(err, gc.ErrorMatches, "unknown agent config format") |
761 | } |
762 | + |
763 | +func (*formatSuite) TestWriteFormat(c *gc.C) { |
764 | + dir := c.MkDir() |
765 | + testDir := path.Join(dir, "test") |
766 | + err := writeFormatFile(testDir, "some format") |
767 | + c.Assert(err, gc.IsNil) |
768 | + format, err := readFormat(testDir) |
769 | + c.Assert(format, gc.Equals, "some format") |
770 | + c.Assert(err, gc.IsNil) |
771 | + // Make sure the carriage return is there as it makes catting the file nicer. |
772 | + content, err := ioutil.ReadFile(path.Join(testDir, formatFilename)) |
773 | + c.Assert(err, gc.IsNil) |
774 | + c.Assert(string(content), gc.Equals, "some format\n") |
775 | +} |
776 | + |
777 | +func (*formatSuite) TestWriteCommandsForFormat(c *gc.C) { |
778 | + dir := c.MkDir() |
779 | + testDir := path.Join(dir, "test") |
780 | + commands := writeCommandsForFormat(testDir, "some format") |
781 | + c.Assert(commands, gc.HasLen, 3) |
782 | + c.Assert(commands[0], gc.Matches, `mkdir -p \S+`) |
783 | + c.Assert(commands[1], gc.Matches, `install -m 644 /dev/null '\S+/format'`) |
784 | + c.Assert(commands[2], gc.Matches, `printf '%s\\n' '.*' > '\S+/format'`) |
785 | +} |
786 | + |
787 | +func (*formatSuite) TestReadPreviousFormatWritesNew(c *gc.C) { |
788 | + params := agentParams |
789 | + params.DataDir = c.MkDir() |
790 | + config, err := newConfig(params) |
791 | + c.Assert(err, gc.IsNil) |
792 | + |
793 | + err = previousFormatter.write(config) |
794 | + c.Assert(err, gc.IsNil) |
795 | + |
796 | + _, err = ReadConf(params.DataDir, params.Tag) |
797 | + c.Assert(err, gc.IsNil) |
798 | + format, err := readFormat(config.Dir()) |
799 | + c.Assert(err, gc.IsNil) |
800 | + c.Assert(format, gc.Equals, currentFormat) |
801 | +} |
802 | |
803 | === modified file 'cmd/jujud/machine.go' |
804 | --- cmd/jujud/machine.go 2013-09-12 14:43:51 +0000 |
805 | +++ cmd/jujud/machine.go 2013-09-12 22:52:57 +0000 |
806 | @@ -12,10 +12,10 @@ |
807 | "launchpad.net/gnuflag" |
808 | "launchpad.net/tomb" |
809 | |
810 | + "launchpad.net/juju-core/agent" |
811 | "launchpad.net/juju-core/charm" |
812 | "launchpad.net/juju-core/cmd" |
813 | "launchpad.net/juju-core/instance" |
814 | - "launchpad.net/juju-core/juju/osenv" |
815 | "launchpad.net/juju-core/log" |
816 | "launchpad.net/juju-core/names" |
817 | "launchpad.net/juju-core/provider" |
818 | @@ -204,15 +204,14 @@ |
819 | // StateJobs returns a worker running all the workers that require |
820 | // a *state.State cofnnection. |
821 | func (a *MachineAgent) StateWorker() (worker.Worker, error) { |
822 | - st, entity, err := openState(a.Conf.config, a) |
823 | + agentConfig := a.Conf.config |
824 | + st, entity, err := openState(agentConfig, a) |
825 | if err != nil { |
826 | return nil, err |
827 | } |
828 | reportOpenedState(st) |
829 | m := entity.(*state.Machine) |
830 | - // TODO(rog) use more discriminating test for errors |
831 | - // rather than taking everything down indiscriminately. |
832 | - dataDir := a.Conf.dataDir |
833 | + |
834 | runner := newRunner(connectionIsFatal(st), moreImportant) |
835 | // At this stage, since we don't embed lxc containers, just start an lxc |
836 | // provisioner task for non-lxc containers. Since we have only LXC |
837 | @@ -222,18 +221,18 @@ |
838 | // containers, it is likely that we will want an LXC provisioner on a KVM |
839 | // machine, and once we get nested LXC containers, we can remove this |
840 | // check. |
841 | - providerType := os.Getenv(osenv.JujuProviderType) |
842 | + providerType := agentConfig.Value(agent.ProviderType) |
843 | if providerType != provider.Local && m.ContainerType() != instance.LXC { |
844 | workerName := fmt.Sprintf("%s-provisioner", provisioner.LXC) |
845 | runner.StartWorker(workerName, func() (worker.Worker, error) { |
846 | - return provisioner.NewProvisioner(provisioner.LXC, st, a.MachineId, dataDir), nil |
847 | + return provisioner.NewProvisioner(provisioner.LXC, st, a.MachineId, agentConfig), nil |
848 | }) |
849 | } |
850 | // Take advantage of special knowledge here in that we will only ever want |
851 | // the storage provider on one machine, and that is the "bootstrap" node. |
852 | if providerType == provider.Local && m.Id() == bootstrapMachineId { |
853 | runner.StartWorker("local-storage", func() (worker.Worker, error) { |
854 | - return localstorage.NewWorker(), nil |
855 | + return localstorage.NewWorker(agentConfig), nil |
856 | }) |
857 | } |
858 | for _, job := range m.Jobs() { |
859 | @@ -242,7 +241,7 @@ |
860 | // Implemented in APIWorker. |
861 | case state.JobManageEnviron: |
862 | runner.StartWorker("environ-provisioner", func() (worker.Worker, error) { |
863 | - return provisioner.NewProvisioner(provisioner.ENVIRON, st, a.MachineId, dataDir), nil |
864 | + return provisioner.NewProvisioner(provisioner.ENVIRON, st, a.MachineId, agentConfig), nil |
865 | }) |
866 | runner.StartWorker("firewaller", func() (worker.Worker, error) { |
867 | return firewaller.NewFirewaller(st), nil |
868 | |
869 | === modified file 'container/lxc/lxc.go' |
870 | --- container/lxc/lxc.go 2013-08-27 23:07:55 +0000 |
871 | +++ container/lxc/lxc.go 2013-09-12 22:52:57 +0000 |
872 | @@ -19,7 +19,6 @@ |
873 | "launchpad.net/juju-core/environs/cloudinit" |
874 | "launchpad.net/juju-core/environs/config" |
875 | "launchpad.net/juju-core/instance" |
876 | - "launchpad.net/juju-core/juju/osenv" |
877 | "launchpad.net/juju-core/names" |
878 | "launchpad.net/juju-core/state" |
879 | "launchpad.net/juju-core/state/api" |
880 | @@ -340,9 +339,6 @@ |
881 | if err := environs.FinishMachineConfig(machineConfig, environConfig, constraints.Value{}); err != nil { |
882 | return nil, err |
883 | } |
884 | - // TODO(thumper): 2013-08-28 bug 1217614 |
885 | - // The machine envronment config values are being moved to the agent config. |
886 | - machineConfig.MachineEnvironment[osenv.JujuContainerType] = string(instance.LXC) |
887 | cloudConfig, err := cloudinit.New(machineConfig) |
888 | if err != nil { |
889 | return nil, err |
890 | |
891 | === modified file 'environs/cloudinit.go' |
892 | --- environs/cloudinit.go 2013-08-22 07:42:43 +0000 |
893 | +++ environs/cloudinit.go 2013-09-12 22:52:57 +0000 |
894 | @@ -6,11 +6,11 @@ |
895 | import ( |
896 | "fmt" |
897 | |
898 | + "launchpad.net/juju-core/agent" |
899 | coreCloudinit "launchpad.net/juju-core/cloudinit" |
900 | "launchpad.net/juju-core/constraints" |
901 | "launchpad.net/juju-core/environs/cloudinit" |
902 | "launchpad.net/juju-core/environs/config" |
903 | - "launchpad.net/juju-core/juju/osenv" |
904 | "launchpad.net/juju-core/state" |
905 | "launchpad.net/juju-core/state/api" |
906 | "launchpad.net/juju-core/utils" |
907 | @@ -70,10 +70,11 @@ |
908 | return fmt.Errorf("environment configuration has no authorized-keys") |
909 | } |
910 | mcfg.AuthorizedKeys = authKeys |
911 | - if mcfg.MachineEnvironment == nil { |
912 | - mcfg.MachineEnvironment = make(map[string]string) |
913 | + if mcfg.AgentEnvironment == nil { |
914 | + mcfg.AgentEnvironment = make(map[string]string) |
915 | } |
916 | - mcfg.MachineEnvironment[osenv.JujuProviderType] = cfg.Type() |
917 | + mcfg.AgentEnvironment[agent.ProviderType] = cfg.Type() |
918 | + mcfg.AgentEnvironment[agent.ContainerType] = string(mcfg.MachineContainerType) |
919 | if !mcfg.StateServer { |
920 | return nil |
921 | } |
922 | |
923 | === modified file 'environs/cloudinit/cloudinit.go' |
924 | --- environs/cloudinit/cloudinit.go 2013-09-11 20:16:49 +0000 |
925 | +++ environs/cloudinit/cloudinit.go 2013-09-12 22:52:57 +0000 |
926 | @@ -93,9 +93,9 @@ |
927 | // commands cannot work. |
928 | AuthorizedKeys string |
929 | |
930 | - // MachineEnvironment defines additional environment variables to set in |
931 | - // the machine agent upstart script. |
932 | - MachineEnvironment map[string]string |
933 | + // AgentEnvironment defines additional configuration variables to set in |
934 | + // the machine agent config. |
935 | + AgentEnvironment map[string]string |
936 | |
937 | // Config holds the initial environment configuration. |
938 | Config *config.Config |
939 | @@ -262,6 +262,7 @@ |
940 | StateAddresses: cfg.stateHostAddrs(), |
941 | APIAddresses: cfg.apiHostAddrs(), |
942 | CACert: cfg.StateInfo.CACert, |
943 | + Values: cfg.AgentEnvironment, |
944 | } |
945 | if cfg.StateServer { |
946 | return agent.NewStateMachineConfig( |
947 | @@ -300,7 +301,7 @@ |
948 | c.AddScripts(fmt.Sprintf("ln -s %v %s", cfg.Tools.Version, shquote(toolsDir))) |
949 | |
950 | name := "jujud-" + tag |
951 | - conf := upstart.MachineAgentUpstartService(name, toolsDir, cfg.DataDir, "/var/log/juju/", tag, machineId, logConfig, cfg.MachineEnvironment) |
952 | + conf := upstart.MachineAgentUpstartService(name, toolsDir, cfg.DataDir, "/var/log/juju/", tag, machineId, logConfig, nil) |
953 | cmds, err := conf.InstallCommands() |
954 | if err != nil { |
955 | return fmt.Errorf("cannot make cloud-init upstart script for the %s agent: %v", tag, err) |
956 | |
957 | === modified file 'environs/cloudinit/cloudinit_test.go' |
958 | --- environs/cloudinit/cloudinit_test.go 2013-09-12 14:16:48 +0000 |
959 | +++ environs/cloudinit/cloudinit_test.go 2013-09-12 22:52:57 +0000 |
960 | @@ -11,12 +11,12 @@ |
961 | gc "launchpad.net/gocheck" |
962 | "launchpad.net/goyaml" |
963 | |
964 | + "launchpad.net/juju-core/agent" |
965 | coreCloudinit "launchpad.net/juju-core/cloudinit" |
966 | "launchpad.net/juju-core/constraints" |
967 | "launchpad.net/juju-core/environs" |
968 | "launchpad.net/juju-core/environs/cloudinit" |
969 | "launchpad.net/juju-core/environs/config" |
970 | - "launchpad.net/juju-core/juju/osenv" |
971 | "launchpad.net/juju-core/state" |
972 | "launchpad.net/juju-core/state/api" |
973 | "launchpad.net/juju-core/testing" |
974 | @@ -52,9 +52,9 @@ |
975 | { |
976 | // precise state server |
977 | cfg: cloudinit.MachineConfig{ |
978 | - MachineId: "0", |
979 | - AuthorizedKeys: "sshkey1", |
980 | - MachineEnvironment: map[string]string{osenv.JujuProviderType: "dummy"}, |
981 | + MachineId: "0", |
982 | + AuthorizedKeys: "sshkey1", |
983 | + AgentEnvironment: map[string]string{agent.ProviderType: "dummy"}, |
984 | // precise currently needs mongo from PPA |
985 | Tools: newSimpleTools("1.2.3-precise-amd64"), |
986 | StateServer: true, |
987 | @@ -90,8 +90,10 @@ |
988 | printf '%s\\n' '/var/log/juju/\*\.log {\\n daily\\n minsize 5M\\n maxsize 50M\\n copytruncate\\n rotate 7\\n missingok\\n compress\\n delaycompress\\n}' > '/etc/logrotate\.d/juju' |
989 | restart rsyslog |
990 | mkdir -p '/var/lib/juju/agents/machine-0' |
991 | +install -m 644 /dev/null '/var/lib/juju/agents/machine-0/format' |
992 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/machine-0/format' |
993 | install -m 600 /dev/null '/var/lib/juju/agents/machine-0/agent\.conf' |
994 | -printf '%s\\n' 'stateservercert:\\n[^']+stateserverkey:\\n[^']+stateport: 37017\\napiport: 17070\\noldpassword: arble\\nmachinenonce: FAKE_NONCE\\nstateinfo:\\n addrs:\\n - localhost:37017\\n cacert:\\n[^']+ tag: machine-0\\n password: ""\\noldapipassword: ""\\napiinfo:\\n addrs:\\n - localhost:17070\\n cacert:\\n[^']+ tag: machine-0\\n password: ""\\n' > '/var/lib/juju/agents/machine-0/agent\.conf' |
995 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/machine-0/agent\.conf' |
996 | install -m 600 /dev/null '/var/lib/juju/server\.pem' |
997 | printf '%s\\n' 'SERVER CERT\\n[^']*SERVER KEY\\n[^']*' > '/var/lib/juju/server\.pem' |
998 | mkdir -p /var/lib/juju/db/journal |
999 | @@ -102,21 +104,23 @@ |
1000 | cat >> /etc/init/juju-db\.conf << 'EOF'\\ndescription "juju state database"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 65000 65000\\nlimit nproc 20000 20000\\n\\nexec /usr/bin/mongod --auth --dbpath=/var/lib/juju/db --sslOnNormalPorts --sslPEMKeyFile '/var/lib/juju/server\.pem' --sslPEMKeyPassword ignored --bind_ip 0\.0\.0\.0 --port 37017 --noprealloc --syslog --smallfiles\\nEOF\\n |
1001 | start juju-db |
1002 | mkdir -p '/var/lib/juju/agents/bootstrap' |
1003 | +install -m 644 /dev/null '/var/lib/juju/agents/bootstrap/format' |
1004 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/bootstrap/format' |
1005 | install -m 600 /dev/null '/var/lib/juju/agents/bootstrap/agent\.conf' |
1006 | -printf '%s\\n' 'stateservercert:\\n[^']+stateserverkey:\\n[^']+stateport: 37017\\napiport: 17070\\noldpassword: arble\\nmachinenonce: FAKE_NONCE\\nstateinfo:\\n addrs:\\n - localhost:37017\\n cacert:\\n[^']+ tag: bootstrap\\n password: ""\\noldapipassword: ""\\napiinfo:\\n addrs:\\n - localhost:17070\\n cacert:\\n[^']+ tag: bootstrap\\n password: ""\\n' > '/var/lib/juju/agents/bootstrap/agent\.conf' |
1007 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/bootstrap/agent\.conf' |
1008 | echo 'some-url' > /tmp/provider-state-url |
1009 | /var/lib/juju/tools/1\.2\.3-precise-amd64/jujud bootstrap-state --data-dir '/var/lib/juju' --env-config '[^']*' --constraints 'mem=2048M' --debug |
1010 | rm -rf '/var/lib/juju/agents/bootstrap' |
1011 | ln -s 1\.2\.3-precise-amd64 '/var/lib/juju/tools/machine-0' |
1012 | -cat >> /etc/init/jujud-machine-0\.conf << 'EOF'\\ndescription "juju machine-0 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\nenv JUJU_PROVIDER_TYPE="dummy"\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-0/jujud machine --log-file '/var/log/juju/machine-0\.log' --data-dir '/var/lib/juju' --machine-id 0 --debug >> /var/log/juju/machine-0\.log 2>&1\\nEOF\\n |
1013 | +cat >> /etc/init/jujud-machine-0\.conf << 'EOF'\\ndescription "juju machine-0 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-0/jujud machine --log-file '/var/log/juju/machine-0\.log' --data-dir '/var/lib/juju' --machine-id 0 --debug >> /var/log/juju/machine-0\.log 2>&1\\nEOF\\n |
1014 | start jujud-machine-0 |
1015 | `, |
1016 | }, { |
1017 | // raring state server |
1018 | cfg: cloudinit.MachineConfig{ |
1019 | - MachineId: "0", |
1020 | - AuthorizedKeys: "sshkey1", |
1021 | - MachineEnvironment: map[string]string{osenv.JujuProviderType: "dummy"}, |
1022 | + MachineId: "0", |
1023 | + AuthorizedKeys: "sshkey1", |
1024 | + AgentEnvironment: map[string]string{agent.ProviderType: "dummy"}, |
1025 | // raring provides mongo in the archive |
1026 | Tools: newSimpleTools("1.2.3-raring-amd64"), |
1027 | StateServer: true, |
1028 | @@ -152,8 +156,10 @@ |
1029 | printf '%s\\n' '/var/log/juju/\*\.log {\\n daily\\n minsize 5M\\n maxsize 50M\\n copytruncate\\n rotate 7\\n missingok\\n compress\\n delaycompress\\n}' > '/etc/logrotate\.d/juju' |
1030 | restart rsyslog |
1031 | mkdir -p '/var/lib/juju/agents/machine-0' |
1032 | +install -m 644 /dev/null '/var/lib/juju/agents/machine-0/format' |
1033 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/machine-0/format' |
1034 | install -m 600 /dev/null '/var/lib/juju/agents/machine-0/agent\.conf' |
1035 | -printf '%s\\n' 'stateservercert:\\n[^']+stateserverkey:\\n[^']+stateport: 37017\\napiport: 17070\\noldpassword: arble\\nmachinenonce: FAKE_NONCE\\nstateinfo:\\n addrs:\\n - localhost:37017\\n cacert:\\n[^']+ tag: machine-0\\n password: ""\\noldapipassword: ""\\napiinfo:\\n addrs:\\n - localhost:17070\\n cacert:\\n[^']+ tag: machine-0\\n password: ""\\n' > '/var/lib/juju/agents/machine-0/agent\.conf' |
1036 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/machine-0/agent\.conf' |
1037 | install -m 600 /dev/null '/var/lib/juju/server\.pem' |
1038 | printf '%s\\n' 'SERVER CERT\\n[^']*SERVER KEY\\n[^']*' > '/var/lib/juju/server\.pem' |
1039 | mkdir -p /var/lib/juju/db/journal |
1040 | @@ -164,24 +170,26 @@ |
1041 | cat >> /etc/init/juju-db\.conf << 'EOF'\\ndescription "juju state database"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 65000 65000\\nlimit nproc 20000 20000\\n\\nexec /usr/bin/mongod --auth --dbpath=/var/lib/juju/db --sslOnNormalPorts --sslPEMKeyFile '/var/lib/juju/server\.pem' --sslPEMKeyPassword ignored --bind_ip 0\.0\.0\.0 --port 37017 --noprealloc --syslog --smallfiles\\nEOF\\n |
1042 | start juju-db |
1043 | mkdir -p '/var/lib/juju/agents/bootstrap' |
1044 | +install -m 644 /dev/null '/var/lib/juju/agents/bootstrap/format' |
1045 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/bootstrap/format' |
1046 | install -m 600 /dev/null '/var/lib/juju/agents/bootstrap/agent\.conf' |
1047 | -printf '%s\\n' 'stateservercert:\\n[^']+stateserverkey:\\n[^']+stateport: 37017\\napiport: 17070\\noldpassword: arble\\nmachinenonce: FAKE_NONCE\\nstateinfo:\\n addrs:\\n - localhost:37017\\n cacert:\\n[^']+ tag: bootstrap\\n password: ""\\noldapipassword: ""\\napiinfo:\\n addrs:\\n - localhost:17070\\n cacert:\\n[^']+ tag: bootstrap\\n password: ""\\n' > '/var/lib/juju/agents/bootstrap/agent\.conf' |
1048 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/bootstrap/agent\.conf' |
1049 | echo 'some-url' > /tmp/provider-state-url |
1050 | /var/lib/juju/tools/1\.2\.3-raring-amd64/jujud bootstrap-state --data-dir '/var/lib/juju' --env-config '[^']*' --constraints 'mem=2048M' --debug |
1051 | rm -rf '/var/lib/juju/agents/bootstrap' |
1052 | ln -s 1\.2\.3-raring-amd64 '/var/lib/juju/tools/machine-0' |
1053 | -cat >> /etc/init/jujud-machine-0\.conf << 'EOF'\\ndescription "juju machine-0 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\nenv JUJU_PROVIDER_TYPE="dummy"\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-0/jujud machine --log-file '/var/log/juju/machine-0\.log' --data-dir '/var/lib/juju' --machine-id 0 --debug >> /var/log/juju/machine-0\.log 2>&1\\nEOF\\n |
1054 | +cat >> /etc/init/jujud-machine-0\.conf << 'EOF'\\ndescription "juju machine-0 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-0/jujud machine --log-file '/var/log/juju/machine-0\.log' --data-dir '/var/lib/juju' --machine-id 0 --debug >> /var/log/juju/machine-0\.log 2>&1\\nEOF\\n |
1055 | start jujud-machine-0 |
1056 | `, |
1057 | }, { |
1058 | cfg: cloudinit.MachineConfig{ |
1059 | - MachineId: "99", |
1060 | - AuthorizedKeys: "sshkey1", |
1061 | - MachineEnvironment: map[string]string{osenv.JujuProviderType: "dummy"}, |
1062 | - DataDir: environs.DataDir, |
1063 | - StateServer: false, |
1064 | - Tools: newSimpleTools("1.2.3-linux-amd64"), |
1065 | - MachineNonce: "FAKE_NONCE", |
1066 | + MachineId: "99", |
1067 | + AuthorizedKeys: "sshkey1", |
1068 | + AgentEnvironment: map[string]string{agent.ProviderType: "dummy"}, |
1069 | + DataDir: environs.DataDir, |
1070 | + StateServer: false, |
1071 | + Tools: newSimpleTools("1.2.3-linux-amd64"), |
1072 | + MachineNonce: "FAKE_NONCE", |
1073 | StateInfo: &state.Info{ |
1074 | Addrs: []string{"state-addr.testing.invalid:12345"}, |
1075 | Tag: "machine-99", |
1076 | @@ -209,10 +217,12 @@ |
1077 | printf '%s\\n' '/var/log/juju/\*\.log {\\n daily\\n minsize 5M\\n maxsize 50M\\n copytruncate\\n rotate 7\\n missingok\\n compress\\n delaycompress\\n}' > '/etc/logrotate\.d/juju' |
1078 | restart rsyslog |
1079 | mkdir -p '/var/lib/juju/agents/machine-99' |
1080 | +install -m 644 /dev/null '/var/lib/juju/agents/machine-99/format' |
1081 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/machine-99/format' |
1082 | install -m 600 /dev/null '/var/lib/juju/agents/machine-99/agent\.conf' |
1083 | -printf '%s\\n' 'oldpassword: arble\\nmachinenonce: FAKE_NONCE\\nstateinfo:\\n addrs:\\n - state-addr\.testing\.invalid:12345\\n cacert:\\n[^']+ tag: machine-99\\n password: ""\\noldapipassword: ""\\napiinfo:\\n addrs:\\n - state-addr\.testing\.invalid:54321\\n cacert:\\n[^']+ tag: machine-99\\n password: ""\\n' > '/var/lib/juju/agents/machine-99/agent\.conf' |
1084 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/machine-99/agent\.conf' |
1085 | ln -s 1\.2\.3-linux-amd64 '/var/lib/juju/tools/machine-99' |
1086 | -cat >> /etc/init/jujud-machine-99\.conf << 'EOF'\\ndescription "juju machine-99 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\nenv JUJU_PROVIDER_TYPE="dummy"\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-99/jujud machine --log-file '/var/log/juju/machine-99\.log' --data-dir '/var/lib/juju' --machine-id 99 --debug >> /var/log/juju/machine-99\.log 2>&1\\nEOF\\n |
1087 | +cat >> /etc/init/jujud-machine-99\.conf << 'EOF'\\ndescription "juju machine-99 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-99/jujud machine --log-file '/var/log/juju/machine-99\.log' --data-dir '/var/lib/juju' --machine-id 99 --debug >> /var/log/juju/machine-99\.log 2>&1\\nEOF\\n |
1088 | start jujud-machine-99 |
1089 | `, |
1090 | }, { |
1091 | @@ -220,7 +230,7 @@ |
1092 | MachineId: "2/lxc/1", |
1093 | MachineContainerType: "lxc", |
1094 | AuthorizedKeys: "sshkey1", |
1095 | - MachineEnvironment: map[string]string{osenv.JujuProviderType: "dummy"}, |
1096 | + AgentEnvironment: map[string]string{agent.ProviderType: "dummy"}, |
1097 | DataDir: environs.DataDir, |
1098 | StateServer: false, |
1099 | Tools: newSimpleTools("1.2.3-linux-amd64"), |
1100 | @@ -252,10 +262,12 @@ |
1101 | printf '%s\\n' '/var/log/juju/\*\.log {\\n daily\\n minsize 5M\\n maxsize 50M\\n copytruncate\\n rotate 7\\n missingok\\n compress\\n delaycompress\\n}' > '/etc/logrotate\.d/juju' |
1102 | restart rsyslog |
1103 | mkdir -p '/var/lib/juju/agents/machine-2-lxc-1' |
1104 | +install -m 644 /dev/null '/var/lib/juju/agents/machine-2-lxc-1/format' |
1105 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/machine-2-lxc-1/format' |
1106 | install -m 600 /dev/null '/var/lib/juju/agents/machine-2-lxc-1/agent\.conf' |
1107 | -printf '%s\\n' 'oldpassword: arble\\nmachinenonce: FAKE_NONCE\\nstateinfo:\\n addrs:\\n - state-addr\.testing\.invalid:12345\\n cacert:\\n[^']+ tag: machine-2-lxc-1\\n password: ""\\noldapipassword: ""\\napiinfo:\\n addrs:\\n - state-addr\.testing\.invalid:54321\\n cacert:\\n[^']+ tag: machine-2-lxc-1\\n password: ""\\n' > '/var/lib/juju/agents/machine-2-lxc-1/agent\.conf' |
1108 | +printf '%s\\n' '.*' > '/var/lib/juju/agents/machine-2-lxc-1/agent\.conf' |
1109 | ln -s 1\.2\.3-linux-amd64 '/var/lib/juju/tools/machine-2-lxc-1' |
1110 | -cat >> /etc/init/jujud-machine-2-lxc-1\.conf << 'EOF'\\ndescription "juju machine-2-lxc-1 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\nenv JUJU_PROVIDER_TYPE="dummy"\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-2-lxc-1/jujud machine --log-file '/var/log/juju/machine-2-lxc-1\.log' --data-dir '/var/lib/juju' --machine-id 2/lxc/1 --debug >> /var/log/juju/machine-2-lxc-1\.log 2>&1\\nEOF\\n |
1111 | +cat >> /etc/init/jujud-machine-2-lxc-1\.conf << 'EOF'\\ndescription "juju machine-2-lxc-1 agent"\\nauthor "Juju Team <juju@lists\.ubuntu\.com>"\\nstart on runlevel \[2345\]\\nstop on runlevel \[!2345\]\\nrespawn\\nnormal exit 0\\n\\nlimit nofile 20000 20000\\n\\nexec /var/lib/juju/tools/machine-2-lxc-1/jujud machine --log-file '/var/log/juju/machine-2-lxc-1\.log' --data-dir '/var/lib/juju' --machine-id 2/lxc/1 --debug >> /var/log/juju/machine-2-lxc-1\.log 2>&1\\nEOF\\n |
1112 | start jujud-machine-2-lxc-1 |
1113 | `, |
1114 | }, |
1115 | @@ -565,15 +577,15 @@ |
1116 | // checked for by NewCloudInit. |
1117 | func (*cloudinitSuite) TestCloudInitVerify(c *gc.C) { |
1118 | cfg := &cloudinit.MachineConfig{ |
1119 | - StateServer: true, |
1120 | - StateServerCert: serverCert, |
1121 | - StateServerKey: serverKey, |
1122 | - StatePort: 1234, |
1123 | - APIPort: 1235, |
1124 | - MachineId: "99", |
1125 | - Tools: newSimpleTools("9.9.9-linux-arble"), |
1126 | - AuthorizedKeys: "sshkey1", |
1127 | - MachineEnvironment: map[string]string{osenv.JujuProviderType: "dummy"}, |
1128 | + StateServer: true, |
1129 | + StateServerCert: serverCert, |
1130 | + StateServerKey: serverKey, |
1131 | + StatePort: 1234, |
1132 | + APIPort: 1235, |
1133 | + MachineId: "99", |
1134 | + Tools: newSimpleTools("9.9.9-linux-arble"), |
1135 | + AuthorizedKeys: "sshkey1", |
1136 | + AgentEnvironment: map[string]string{agent.ProviderType: "dummy"}, |
1137 | StateInfo: &state.Info{ |
1138 | Addrs: []string{"host:98765"}, |
1139 | CACert: []byte(testing.CACert), |
1140 | |
1141 | === modified file 'environs/cloudinit_test.go' |
1142 | --- environs/cloudinit_test.go 2013-09-12 12:38:04 +0000 |
1143 | +++ environs/cloudinit_test.go 2013-09-12 22:52:57 +0000 |
1144 | @@ -9,12 +9,12 @@ |
1145 | gc "launchpad.net/gocheck" |
1146 | "launchpad.net/goyaml" |
1147 | |
1148 | + "launchpad.net/juju-core/agent" |
1149 | "launchpad.net/juju-core/cert" |
1150 | "launchpad.net/juju-core/constraints" |
1151 | "launchpad.net/juju-core/environs" |
1152 | "launchpad.net/juju-core/environs/cloudinit" |
1153 | "launchpad.net/juju-core/environs/config" |
1154 | - "launchpad.net/juju-core/juju/osenv" |
1155 | "launchpad.net/juju-core/provider/dummy" |
1156 | "launchpad.net/juju-core/state" |
1157 | "launchpad.net/juju-core/state/api" |
1158 | @@ -52,10 +52,13 @@ |
1159 | err = environs.FinishMachineConfig(mcfg, cfg, constraints.Value{}) |
1160 | c.Assert(err, gc.IsNil) |
1161 | c.Assert(mcfg, gc.DeepEquals, &cloudinit.MachineConfig{ |
1162 | - AuthorizedKeys: "we-are-the-keys", |
1163 | - MachineEnvironment: map[string]string{osenv.JujuProviderType: "dummy"}, |
1164 | - StateInfo: &state.Info{Tag: "not touched"}, |
1165 | - APIInfo: &api.Info{Tag: "not touched"}, |
1166 | + AuthorizedKeys: "we-are-the-keys", |
1167 | + AgentEnvironment: map[string]string{ |
1168 | + agent.ProviderType: "dummy", |
1169 | + agent.ContainerType: "", |
1170 | + }, |
1171 | + StateInfo: &state.Info{Tag: "not touched"}, |
1172 | + APIInfo: &api.Info{Tag: "not touched"}, |
1173 | }) |
1174 | } |
1175 | |
1176 | @@ -128,12 +131,12 @@ |
1177 | Password: "pw2", |
1178 | CACert: []byte("CA CERT\n" + testing.CACert), |
1179 | }, |
1180 | - DataDir: environs.DataDir, |
1181 | - Config: envConfig, |
1182 | - StatePort: envConfig.StatePort(), |
1183 | - APIPort: envConfig.APIPort(), |
1184 | - StateServer: true, |
1185 | - MachineEnvironment: map[string]string{osenv.JujuProviderType: "dummy"}, |
1186 | + DataDir: environs.DataDir, |
1187 | + Config: envConfig, |
1188 | + StatePort: envConfig.StatePort(), |
1189 | + APIPort: envConfig.APIPort(), |
1190 | + StateServer: true, |
1191 | + AgentEnvironment: map[string]string{agent.ProviderType: "dummy"}, |
1192 | } |
1193 | script1 := "script1" |
1194 | script2 := "script2" |
1195 | |
1196 | === modified file 'environs/manual/agent.go' |
1197 | --- environs/manual/agent.go 2013-09-03 09:03:02 +0000 |
1198 | +++ environs/manual/agent.go 2013-09-12 22:52:57 +0000 |
1199 | @@ -10,12 +10,12 @@ |
1200 | "os/exec" |
1201 | "strings" |
1202 | |
1203 | + "launchpad.net/juju-core/agent" |
1204 | corecloudinit "launchpad.net/juju-core/cloudinit" |
1205 | "launchpad.net/juju-core/constraints" |
1206 | "launchpad.net/juju-core/environs" |
1207 | "launchpad.net/juju-core/environs/cloudinit" |
1208 | envtools "launchpad.net/juju-core/environs/tools" |
1209 | - "launchpad.net/juju-core/juju/osenv" |
1210 | "launchpad.net/juju-core/provider" |
1211 | "launchpad.net/juju-core/state" |
1212 | "launchpad.net/juju-core/state/api" |
1213 | @@ -81,7 +81,7 @@ |
1214 | if err != nil { |
1215 | return "", err |
1216 | } |
1217 | - mcfg.MachineEnvironment[osenv.JujuProviderType] = provider.Null |
1218 | + mcfg.AgentEnvironment[agent.ProviderType] = provider.Null |
1219 | cloudcfg := corecloudinit.New() |
1220 | if cloudcfg, err = cloudinit.Configure(mcfg, cloudcfg); err != nil { |
1221 | return "", err |
1222 | |
1223 | === modified file 'juju/osenv/vars.go' |
1224 | --- juju/osenv/vars.go 2013-09-03 15:38:37 +0000 |
1225 | +++ juju/osenv/vars.go 2013-09-12 22:52:57 +0000 |
1226 | @@ -10,16 +10,14 @@ |
1227 | ) |
1228 | |
1229 | const ( |
1230 | - JujuEnv = "JUJU_ENV" |
1231 | - JujuHome = "JUJU_HOME" |
1232 | - JujuRepository = "JUJU_REPOSITORY" |
1233 | - JujuLxcBridge = "JUJU_LXC_BRIDGE" |
1234 | - JujuProviderType = "JUJU_PROVIDER_TYPE" |
1235 | - JujuContainerType = "JUJU_CONTAINER_TYPE" |
1236 | - JujuStorageDir = "JUJU_STORAGE_DIR" |
1237 | - JujuStorageAddr = "JUJU_STORAGE_ADDR" |
1238 | - JujuSharedStorageDir = "JUJU_SHARED_STORAGE_DIR" |
1239 | - JujuSharedStorageAddr = "JUJU_SHARED_STORAGE_ADDR" |
1240 | + JujuEnv = "JUJU_ENV" |
1241 | + JujuHome = "JUJU_HOME" |
1242 | + JujuRepository = "JUJU_REPOSITORY" |
1243 | + // TODO(thumper): 2013-09-02 bug 1219630 |
1244 | + // As much as I'd like to remove JujuContainerType now, it is still |
1245 | + // needed as MAAS still needs it at this stage, and we can't fix |
1246 | + // everything at once. |
1247 | + JujuContainerType = "JUJU_CONTAINER_TYPE" |
1248 | ) |
1249 | |
1250 | // JujuHome returns the directory where juju should store application-specific files |
1251 | |
1252 | === modified file 'juju/testing/conn.go' |
1253 | --- juju/testing/conn.go 2013-09-12 14:16:48 +0000 |
1254 | +++ juju/testing/conn.go 2013-09-12 22:52:57 +0000 |
1255 | @@ -12,6 +12,7 @@ |
1256 | gc "launchpad.net/gocheck" |
1257 | "launchpad.net/goyaml" |
1258 | |
1259 | + "launchpad.net/juju-core/agent" |
1260 | "launchpad.net/juju-core/charm" |
1261 | "launchpad.net/juju-core/constraints" |
1262 | "launchpad.net/juju-core/environs" |
1263 | @@ -309,3 +310,18 @@ |
1264 | c.Assert(err, gc.IsNil) |
1265 | return sch |
1266 | } |
1267 | + |
1268 | +func (s *JujuConnSuite) AgentConfigForTag(c *gc.C, tag string) agent.Config { |
1269 | + config, err := agent.NewAgentConfig( |
1270 | + agent.AgentConfigParams{ |
1271 | + DataDir: s.DataDir(), |
1272 | + Tag: tag, |
1273 | + Password: "dummy-secret", |
1274 | + Nonce: "nonce", |
1275 | + StateAddresses: s.StateInfo(c).Addrs, |
1276 | + APIAddresses: s.APIInfo(c).Addrs, |
1277 | + CACert: []byte(testing.CACert), |
1278 | + }) |
1279 | + c.Assert(err, gc.IsNil) |
1280 | + return config |
1281 | +} |
1282 | |
1283 | === modified file 'provider/local/environ.go' |
1284 | --- provider/local/environ.go 2013-09-11 12:18:04 +0000 |
1285 | +++ provider/local/environ.go 2013-09-12 22:52:57 +0000 |
1286 | @@ -464,13 +464,8 @@ |
1287 | logDir := env.config.logDir() |
1288 | logConfig := "--debug" // TODO(thumper): specify loggo config |
1289 | machineEnvironment := map[string]string{ |
1290 | - "USER": env.config.user, |
1291 | - "HOME": osenv.Home(), |
1292 | - osenv.JujuProviderType: env.config.Type(), |
1293 | - osenv.JujuStorageDir: env.config.storageDir(), |
1294 | - osenv.JujuStorageAddr: env.config.storageAddr(), |
1295 | - osenv.JujuSharedStorageDir: env.config.sharedStorageDir(), |
1296 | - osenv.JujuSharedStorageAddr: env.config.sharedStorageAddr(), |
1297 | + "USER": env.config.user, |
1298 | + "HOME": osenv.Home(), |
1299 | } |
1300 | agentService := upstart.MachineAgentUpstartService( |
1301 | env.machineAgentServiceName(), |
1302 | @@ -496,6 +491,13 @@ |
1303 | // wouldn't get this far. |
1304 | cfg := env.config.Config |
1305 | caCert, _ := cfg.CACert() |
1306 | + agentValues := map[string]string{ |
1307 | + agent.ProviderType: env.config.Type(), |
1308 | + agent.StorageDir: env.config.storageDir(), |
1309 | + agent.StorageAddr: env.config.storageAddr(), |
1310 | + agent.SharedStorageDir: env.config.sharedStorageDir(), |
1311 | + agent.SharedStorageAddr: env.config.sharedStorageAddr(), |
1312 | + } |
1313 | // NOTE: the state address HAS to be localhost, otherwise the mongo |
1314 | // initialization fails. There is some magic code somewhere in the mongo |
1315 | // connection code that treats connections from localhost as special, and |
1316 | @@ -513,6 +515,7 @@ |
1317 | StateAddresses: []string{stateAddress}, |
1318 | APIAddresses: []string{apiAddress}, |
1319 | CACert: caCert, |
1320 | + Values: agentValues, |
1321 | }, |
1322 | StateServerCert: cert, |
1323 | StateServerKey: key, |
1324 | |
1325 | === modified file 'provider/local/storage/worker.go' |
1326 | --- provider/local/storage/worker.go 2013-07-31 05:11:24 +0000 |
1327 | +++ provider/local/storage/worker.go 2013-09-12 22:52:57 +0000 |
1328 | @@ -1,24 +1,23 @@ |
1329 | package storage |
1330 | |
1331 | import ( |
1332 | - "os" |
1333 | - |
1334 | "launchpad.net/loggo" |
1335 | "launchpad.net/tomb" |
1336 | |
1337 | + "launchpad.net/juju-core/agent" |
1338 | "launchpad.net/juju-core/environs/localstorage" |
1339 | - "launchpad.net/juju-core/juju/osenv" |
1340 | "launchpad.net/juju-core/worker" |
1341 | ) |
1342 | |
1343 | var logger = loggo.GetLogger("juju.local.storage") |
1344 | |
1345 | type storageWorker struct { |
1346 | - tomb tomb.Tomb |
1347 | + config agent.Config |
1348 | + tomb tomb.Tomb |
1349 | } |
1350 | |
1351 | -func NewWorker() worker.Worker { |
1352 | - w := &storageWorker{} |
1353 | +func NewWorker(config agent.Config) worker.Worker { |
1354 | + w := &storageWorker{config: config} |
1355 | go func() { |
1356 | defer w.tomb.Done() |
1357 | w.tomb.Kill(w.waitForDeath()) |
1358 | @@ -37,8 +36,8 @@ |
1359 | } |
1360 | |
1361 | func (s *storageWorker) waitForDeath() error { |
1362 | - storageDir := os.Getenv(osenv.JujuStorageDir) |
1363 | - storageAddr := os.Getenv(osenv.JujuStorageAddr) |
1364 | + storageDir := s.config.Value(agent.StorageDir) |
1365 | + storageAddr := s.config.Value(agent.StorageAddr) |
1366 | logger.Infof("serving %s on %s", storageDir, storageAddr) |
1367 | |
1368 | storageListener, err := localstorage.Serve(storageAddr, storageDir) |
1369 | @@ -48,8 +47,8 @@ |
1370 | } |
1371 | defer storageListener.Close() |
1372 | |
1373 | - sharedStorageDir := os.Getenv(osenv.JujuSharedStorageDir) |
1374 | - sharedStorageAddr := os.Getenv(osenv.JujuSharedStorageAddr) |
1375 | + sharedStorageDir := s.config.Value(agent.SharedStorageDir) |
1376 | + sharedStorageAddr := s.config.Value(agent.SharedStorageAddr) |
1377 | logger.Infof("serving %s on %s", sharedStorageDir, sharedStorageAddr) |
1378 | |
1379 | sharedStorageListener, err := localstorage.Serve(sharedStorageAddr, sharedStorageDir) |
1380 | |
1381 | === modified file 'provider/maas/environ.go' |
1382 | --- provider/maas/environ.go 2013-09-10 03:43:51 +0000 |
1383 | +++ provider/maas/environ.go 2013-09-12 22:52:57 +0000 |
1384 | @@ -12,6 +12,7 @@ |
1385 | |
1386 | "launchpad.net/gomaasapi" |
1387 | |
1388 | + "launchpad.net/juju-core/agent" |
1389 | "launchpad.net/juju-core/constraints" |
1390 | "launchpad.net/juju-core/environs" |
1391 | "launchpad.net/juju-core/environs/cloudinit" |
1392 | @@ -20,7 +21,6 @@ |
1393 | "launchpad.net/juju-core/environs/simplestreams" |
1394 | envtools "launchpad.net/juju-core/environs/tools" |
1395 | "launchpad.net/juju-core/instance" |
1396 | - "launchpad.net/juju-core/juju/osenv" |
1397 | "launchpad.net/juju-core/provider" |
1398 | "launchpad.net/juju-core/state" |
1399 | "launchpad.net/juju-core/state/api" |
1400 | @@ -257,7 +257,7 @@ |
1401 | // TODO(thumper): 2013-08-28 bug 1217614 |
1402 | // The machine envronment config values are being moved to the agent config. |
1403 | // Explicitly specify that the lxc containers use the network bridge defined above. |
1404 | - machineConfig.MachineEnvironment[osenv.JujuLxcBridge] = "br0" |
1405 | + machineConfig.AgentEnvironment[agent.LxcBridge] = "br0" |
1406 | userdata, err := environs.ComposeUserData( |
1407 | machineConfig, |
1408 | runCmd, |
1409 | |
1410 | === modified file 'worker/deployer/simple.go' |
1411 | --- worker/deployer/simple.go 2013-09-02 05:41:29 +0000 |
1412 | +++ worker/deployer/simple.go 2013-09-12 22:52:57 +0000 |
1413 | @@ -94,6 +94,7 @@ |
1414 | return err |
1415 | } |
1416 | logger.Debugf("API addresses: %q", apiAddrs) |
1417 | + containerType := ctx.agentConfig.Value(agent.ContainerType) |
1418 | conf, err := agent.NewAgentConfig( |
1419 | agent.AgentConfigParams{ |
1420 | DataDir: dataDir, |
1421 | @@ -103,6 +104,9 @@ |
1422 | StateAddresses: stateAddrs, |
1423 | APIAddresses: apiAddrs, |
1424 | CACert: ctx.agentConfig.CACert(), |
1425 | + Values: map[string]string{ |
1426 | + agent.ContainerType: containerType, |
1427 | + }, |
1428 | }) |
1429 | if err != nil { |
1430 | return err |
1431 | @@ -132,15 +136,17 @@ |
1432 | "--unit-name", unitName, |
1433 | "--debug", // TODO: propagate debug state sensibly |
1434 | }, " ") |
1435 | + // TODO(thumper): 2013-09-02 bug 1219630 |
1436 | + // As much as I'd like to remove JujuContainerType now, it is still |
1437 | + // needed as MAAS still needs it at this stage, and we can't fix |
1438 | + // everything at once. |
1439 | uconf := &upstart.Conf{ |
1440 | Service: *svc, |
1441 | Desc: "juju unit agent for " + unitName, |
1442 | Cmd: cmd, |
1443 | Out: logPath, |
1444 | - // Propagate the provider type and container type enviroment variables. |
1445 | Env: map[string]string{ |
1446 | - osenv.JujuProviderType: os.Getenv(osenv.JujuProviderType), |
1447 | - osenv.JujuContainerType: os.Getenv(osenv.JujuContainerType), |
1448 | + osenv.JujuContainerType: containerType, |
1449 | }, |
1450 | } |
1451 | return uconf.Install() |
1452 | |
1453 | === modified file 'worker/deployer/simple_test.go' |
1454 | --- worker/deployer/simple_test.go 2013-09-02 06:03:14 +0000 |
1455 | +++ worker/deployer/simple_test.go 2013-09-12 22:52:57 +0000 |
1456 | @@ -304,6 +304,10 @@ |
1457 | return []byte(testing.CACert) |
1458 | } |
1459 | |
1460 | +func (mock *mockConfig) Value(_ string) string { |
1461 | + return "" |
1462 | +} |
1463 | + |
1464 | func agentConfig(tag, datadir string) agent.Config { |
1465 | return &mockConfig{tag: tag, datadir: datadir} |
1466 | } |
1467 | |
1468 | === modified file 'worker/provisioner/lxc-broker.go' |
1469 | --- worker/provisioner/lxc-broker.go 2013-08-22 22:24:54 +0000 |
1470 | +++ worker/provisioner/lxc-broker.go 2013-09-12 22:52:57 +0000 |
1471 | @@ -4,17 +4,15 @@ |
1472 | package provisioner |
1473 | |
1474 | import ( |
1475 | - "os" |
1476 | - |
1477 | "launchpad.net/loggo" |
1478 | |
1479 | + "launchpad.net/juju-core/agent" |
1480 | "launchpad.net/juju-core/constraints" |
1481 | "launchpad.net/juju-core/container/lxc" |
1482 | "launchpad.net/juju-core/environs" |
1483 | "launchpad.net/juju-core/environs/cloudinit" |
1484 | "launchpad.net/juju-core/environs/config" |
1485 | "launchpad.net/juju-core/instance" |
1486 | - "launchpad.net/juju-core/juju/osenv" |
1487 | "launchpad.net/juju-core/tools" |
1488 | ) |
1489 | |
1490 | @@ -23,18 +21,20 @@ |
1491 | var _ environs.InstanceBroker = (*lxcBroker)(nil) |
1492 | var _ tools.HasTools = (*lxcBroker)(nil) |
1493 | |
1494 | -func NewLxcBroker(config *config.Config, tools *tools.Tools) environs.InstanceBroker { |
1495 | +func NewLxcBroker(config *config.Config, tools *tools.Tools, agentConfig agent.Config) environs.InstanceBroker { |
1496 | return &lxcBroker{ |
1497 | - manager: lxc.NewContainerManager(lxc.ManagerConfig{Name: "juju"}), |
1498 | - config: config, |
1499 | - tools: tools, |
1500 | + manager: lxc.NewContainerManager(lxc.ManagerConfig{Name: "juju"}), |
1501 | + config: config, |
1502 | + tools: tools, |
1503 | + agentConfig: agentConfig, |
1504 | } |
1505 | } |
1506 | |
1507 | type lxcBroker struct { |
1508 | - manager lxc.ContainerManager |
1509 | - config *config.Config |
1510 | - tools *tools.Tools |
1511 | + manager lxc.ContainerManager |
1512 | + config *config.Config |
1513 | + tools *tools.Tools |
1514 | + agentConfig agent.Config |
1515 | } |
1516 | |
1517 | func (broker *lxcBroker) Tools() tools.List { |
1518 | @@ -49,7 +49,7 @@ |
1519 | lxcLogger.Infof("starting lxc container for machineId: %s", machineId) |
1520 | |
1521 | // Default to using the host network until we can configure. |
1522 | - bridgeDevice := os.Getenv(osenv.JujuLxcBridge) |
1523 | + bridgeDevice := broker.agentConfig.Value(agent.LxcBridge) |
1524 | if bridgeDevice == "" { |
1525 | bridgeDevice = lxc.DefaultLxcBridge |
1526 | } |
1527 | |
1528 | === modified file 'worker/provisioner/lxc-broker_test.go' |
1529 | --- worker/provisioner/lxc-broker_test.go 2013-08-22 22:24:54 +0000 |
1530 | +++ worker/provisioner/lxc-broker_test.go 2013-09-12 22:52:57 +0000 |
1531 | @@ -12,6 +12,7 @@ |
1532 | |
1533 | gc "launchpad.net/gocheck" |
1534 | |
1535 | + "launchpad.net/juju-core/agent" |
1536 | agenttools "launchpad.net/juju-core/agent/tools" |
1537 | "launchpad.net/juju-core/constraints" |
1538 | "launchpad.net/juju-core/container/lxc" |
1539 | @@ -19,8 +20,8 @@ |
1540 | "launchpad.net/juju-core/environs" |
1541 | "launchpad.net/juju-core/environs/config" |
1542 | "launchpad.net/juju-core/instance" |
1543 | - "launchpad.net/juju-core/juju/osenv" |
1544 | jujutesting "launchpad.net/juju-core/juju/testing" |
1545 | + "launchpad.net/juju-core/names" |
1546 | "launchpad.net/juju-core/provider" |
1547 | "launchpad.net/juju-core/state" |
1548 | coretesting "launchpad.net/juju-core/testing" |
1549 | @@ -38,7 +39,8 @@ |
1550 | |
1551 | type lxcBrokerSuite struct { |
1552 | lxcSuite |
1553 | - broker environs.InstanceBroker |
1554 | + broker environs.InstanceBroker |
1555 | + agentConfig agent.Config |
1556 | } |
1557 | |
1558 | var _ = gc.Suite(&lxcBrokerSuite{}) |
1559 | @@ -77,7 +79,19 @@ |
1560 | Version: version.MustParseBinary("2.3.4-foo-bar"), |
1561 | URL: "http://tools.testing.invalid/2.3.4-foo-bar.tgz", |
1562 | } |
1563 | - s.broker = provisioner.NewLxcBroker(coretesting.EnvironConfig(c), tools) |
1564 | + config := coretesting.EnvironConfig(c) |
1565 | + var err error |
1566 | + s.agentConfig, err = agent.NewAgentConfig( |
1567 | + agent.AgentConfigParams{ |
1568 | + DataDir: "/not/used/here", |
1569 | + Tag: "tag", |
1570 | + Password: "dummy-secret", |
1571 | + Nonce: "nonce", |
1572 | + APIAddresses: []string{"10.0.0.1:1234"}, |
1573 | + CACert: []byte(coretesting.CACert), |
1574 | + }) |
1575 | + c.Assert(err, gc.IsNil) |
1576 | + s.broker = provisioner.NewLxcBroker(config, tools, s.agentConfig) |
1577 | } |
1578 | |
1579 | func (s *lxcBrokerSuite) startInstance(c *gc.C, machineId string) instance.Instance { |
1580 | @@ -106,7 +120,7 @@ |
1581 | } |
1582 | |
1583 | func (s *lxcBrokerSuite) TestStartInstanceWithBridgeEnviron(c *gc.C) { |
1584 | - defer coretesting.PatchEnvironment(osenv.JujuLxcBridge, "br0")() |
1585 | + s.agentConfig.SetValue(agent.LxcBridge, "br0") |
1586 | machineId := "1/lxc/0" |
1587 | lxc := s.startInstance(c, machineId) |
1588 | c.Assert(lxc.Id(), gc.Equals, instance.Id("juju-machine-1-lxc-0")) |
1589 | @@ -231,17 +245,19 @@ |
1590 | s.CommonProvisionerSuite.TearDownTest(c) |
1591 | } |
1592 | |
1593 | -func (s *lxcProvisionerSuite) newLxcProvisioner() *provisioner.Provisioner { |
1594 | - return provisioner.NewProvisioner(provisioner.LXC, s.State, s.machineId, s.DataDir()) |
1595 | +func (s *lxcProvisionerSuite) newLxcProvisioner(c *gc.C) *provisioner.Provisioner { |
1596 | + machineTag := names.MachineTag(s.machineId) |
1597 | + agentConfig := s.AgentConfigForTag(c, machineTag) |
1598 | + return provisioner.NewProvisioner(provisioner.LXC, s.State, s.machineId, agentConfig) |
1599 | } |
1600 | |
1601 | func (s *lxcProvisionerSuite) TestProvisionerStartStop(c *gc.C) { |
1602 | - p := s.newLxcProvisioner() |
1603 | + p := s.newLxcProvisioner(c) |
1604 | c.Assert(p.Stop(), gc.IsNil) |
1605 | } |
1606 | |
1607 | func (s *lxcProvisionerSuite) TestDoesNotStartEnvironMachines(c *gc.C) { |
1608 | - p := s.newLxcProvisioner() |
1609 | + p := s.newLxcProvisioner(c) |
1610 | defer stop(c, p) |
1611 | |
1612 | // Check that an instance is not provisioned when the machine is created. |
1613 | @@ -264,7 +280,7 @@ |
1614 | } |
1615 | |
1616 | func (s *lxcProvisionerSuite) TestContainerStartedAndStopped(c *gc.C) { |
1617 | - p := s.newLxcProvisioner() |
1618 | + p := s.newLxcProvisioner(c) |
1619 | defer stop(c, p) |
1620 | |
1621 | container := s.addContainer(c) |
1622 | |
1623 | === modified file 'worker/provisioner/provisioner.go' |
1624 | --- worker/provisioner/provisioner.go 2013-08-22 22:24:54 +0000 |
1625 | +++ worker/provisioner/provisioner.go 2013-09-12 22:52:57 +0000 |
1626 | @@ -10,6 +10,7 @@ |
1627 | "launchpad.net/loggo" |
1628 | "launchpad.net/tomb" |
1629 | |
1630 | + "launchpad.net/juju-core/agent" |
1631 | agenttools "launchpad.net/juju-core/agent/tools" |
1632 | "launchpad.net/juju-core/environs" |
1633 | "launchpad.net/juju-core/environs/config" |
1634 | @@ -34,13 +35,13 @@ |
1635 | |
1636 | // Provisioner represents a running provisioning worker. |
1637 | type Provisioner struct { |
1638 | - pt ProvisionerType |
1639 | - st *state.State |
1640 | - machineId string // Which machine runs the provisioner. |
1641 | - dataDir string |
1642 | - machine *state.Machine |
1643 | - environ environs.Environ |
1644 | - tomb tomb.Tomb |
1645 | + pt ProvisionerType |
1646 | + st *state.State |
1647 | + machineId string // Which machine runs the provisioner. |
1648 | + machine *state.Machine |
1649 | + environ environs.Environ |
1650 | + agentConfig agent.Config |
1651 | + tomb tomb.Tomb |
1652 | |
1653 | configObserver |
1654 | } |
1655 | @@ -62,12 +63,12 @@ |
1656 | // NewProvisioner returns a new Provisioner. When new machines |
1657 | // are added to the state, it allocates instances from the environment |
1658 | // and allocates them to the new machines. |
1659 | -func NewProvisioner(pt ProvisionerType, st *state.State, machineId, dataDir string) *Provisioner { |
1660 | +func NewProvisioner(pt ProvisionerType, st *state.State, machineId string, agentConfig agent.Config) *Provisioner { |
1661 | p := &Provisioner{ |
1662 | - pt: pt, |
1663 | - st: st, |
1664 | - machineId: machineId, |
1665 | - dataDir: dataDir, |
1666 | + pt: pt, |
1667 | + st: st, |
1668 | + machineId: machineId, |
1669 | + agentConfig: agentConfig, |
1670 | } |
1671 | go func() { |
1672 | defer p.tomb.Done() |
1673 | @@ -166,15 +167,16 @@ |
1674 | logger.Errorf("cannot get tools from machine for lxc broker") |
1675 | return nil, err |
1676 | } |
1677 | - return NewLxcBroker(config, tools), nil |
1678 | + return NewLxcBroker(config, tools, p.agentConfig), nil |
1679 | } |
1680 | return nil, fmt.Errorf("unknown provisioner type") |
1681 | } |
1682 | |
1683 | func (p *Provisioner) getAgentTools() (*coretools.Tools, error) { |
1684 | - tools, err := agenttools.ReadTools(p.dataDir, version.Current) |
1685 | + dataDir := p.agentConfig.DataDir() |
1686 | + tools, err := agenttools.ReadTools(dataDir, version.Current) |
1687 | if err != nil { |
1688 | - logger.Errorf("cannot read agent tools from %q", p.dataDir) |
1689 | + logger.Errorf("cannot read agent tools from %q", dataDir) |
1690 | return nil, err |
1691 | } |
1692 | return tools, nil |
1693 | |
1694 | === modified file 'worker/provisioner/provisioner_test.go' |
1695 | --- worker/provisioner/provisioner_test.go 2013-08-30 18:35:54 +0000 |
1696 | +++ worker/provisioner/provisioner_test.go 2013-09-12 22:52:57 +0000 |
1697 | @@ -285,12 +285,14 @@ |
1698 | }) |
1699 | } |
1700 | |
1701 | -func (s *ProvisionerSuite) newEnvironProvisioner(machineId string) *provisioner.Provisioner { |
1702 | - return provisioner.NewProvisioner(provisioner.ENVIRON, s.State, machineId, "") |
1703 | +func (s *ProvisionerSuite) newEnvironProvisioner(c *gc.C, machineId string) *provisioner.Provisioner { |
1704 | + machineTag := names.MachineTag(machineId) |
1705 | + agentConfig := s.AgentConfigForTag(c, machineTag) |
1706 | + return provisioner.NewProvisioner(provisioner.ENVIRON, s.State, machineId, agentConfig) |
1707 | } |
1708 | |
1709 | func (s *ProvisionerSuite) TestProvisionerStartStop(c *gc.C) { |
1710 | - p := s.newEnvironProvisioner("0") |
1711 | + p := s.newEnvironProvisioner(c, "0") |
1712 | c.Assert(p.Stop(), gc.IsNil) |
1713 | } |
1714 | |
1715 | @@ -304,7 +306,7 @@ |
1716 | } |
1717 | |
1718 | func (s *ProvisionerSuite) TestSimple(c *gc.C) { |
1719 | - p := s.newEnvironProvisioner("0") |
1720 | + p := s.newEnvironProvisioner(c, "0") |
1721 | defer stop(c, p) |
1722 | |
1723 | // Check that an instance is provisioned when the machine is created... |
1724 | @@ -327,14 +329,14 @@ |
1725 | c.Assert(err, gc.IsNil) |
1726 | |
1727 | // Start a provisioner and check those constraints are used. |
1728 | - p := s.newEnvironProvisioner("0") |
1729 | + p := s.newEnvironProvisioner(c, "0") |
1730 | defer stop(c, p) |
1731 | s.checkStartInstanceCustom(c, m, "pork", cons) |
1732 | } |
1733 | |
1734 | func (s *ProvisionerSuite) TestProvisionerSetsErrorStatusWhenStartInstanceFailed(c *gc.C) { |
1735 | brokenMsg := breakDummyProvider(c, s.State, "StartInstance") |
1736 | - p := s.newEnvironProvisioner("0") |
1737 | + p := s.newEnvironProvisioner(c, "0") |
1738 | defer stop(c, p) |
1739 | |
1740 | // Check that an instance is not provisioned when the machine is created... |
1741 | @@ -362,13 +364,13 @@ |
1742 | |
1743 | // Restart the PA to make sure the machine is skipped again. |
1744 | stop(c, p) |
1745 | - p = s.newEnvironProvisioner("0") |
1746 | + p = s.newEnvironProvisioner(c, "0") |
1747 | defer stop(c, p) |
1748 | s.checkNoOperations(c) |
1749 | } |
1750 | |
1751 | func (s *ProvisionerSuite) TestProvisioningDoesNotOccurForContainers(c *gc.C) { |
1752 | - p := s.newEnvironProvisioner("0") |
1753 | + p := s.newEnvironProvisioner(c, "0") |
1754 | defer stop(c, p) |
1755 | |
1756 | // create a machine to host the container. |
1757 | @@ -401,7 +403,7 @@ |
1758 | err := s.invalidateEnvironment(c) |
1759 | c.Assert(err, gc.IsNil) |
1760 | |
1761 | - p := s.newEnvironProvisioner("0") |
1762 | + p := s.newEnvironProvisioner(c, "0") |
1763 | defer stop(c, p) |
1764 | |
1765 | // try to create a machine |
1766 | @@ -416,7 +418,7 @@ |
1767 | err := s.invalidateEnvironment(c) |
1768 | c.Assert(err, gc.IsNil) |
1769 | |
1770 | - p := s.newEnvironProvisioner("0") |
1771 | + p := s.newEnvironProvisioner(c, "0") |
1772 | defer stop(c, p) |
1773 | |
1774 | // try to create a machine |
1775 | @@ -433,7 +435,7 @@ |
1776 | } |
1777 | |
1778 | func (s *ProvisionerSuite) TestProvisioningDoesOccurAfterInvalidEnvironmentPublished(c *gc.C) { |
1779 | - p := s.newEnvironProvisioner("0") |
1780 | + p := s.newEnvironProvisioner(c, "0") |
1781 | defer stop(c, p) |
1782 | |
1783 | // place a new machine into the state |
1784 | @@ -454,7 +456,7 @@ |
1785 | } |
1786 | |
1787 | func (s *ProvisionerSuite) TestProvisioningDoesNotProvisionTheSameMachineAfterRestart(c *gc.C) { |
1788 | - p := s.newEnvironProvisioner("0") |
1789 | + p := s.newEnvironProvisioner(c, "0") |
1790 | defer stop(c, p) |
1791 | |
1792 | // create a machine |
1793 | @@ -464,7 +466,7 @@ |
1794 | |
1795 | // restart the PA |
1796 | stop(c, p) |
1797 | - p = s.newEnvironProvisioner("0") |
1798 | + p = s.newEnvironProvisioner(c, "0") |
1799 | defer stop(c, p) |
1800 | |
1801 | // check that there is only one machine known |
1802 | @@ -478,7 +480,7 @@ |
1803 | } |
1804 | |
1805 | func (s *ProvisionerSuite) TestProvisioningStopsInstances(c *gc.C) { |
1806 | - p := s.newEnvironProvisioner("0") |
1807 | + p := s.newEnvironProvisioner(c, "0") |
1808 | defer stop(c, p) |
1809 | |
1810 | // create a machine |
1811 | @@ -500,14 +502,14 @@ |
1812 | c.Assert(m1.Remove(), gc.IsNil) |
1813 | |
1814 | // start a new provisioner to shut them both down |
1815 | - p = s.newEnvironProvisioner("0") |
1816 | + p = s.newEnvironProvisioner(c, "0") |
1817 | defer stop(c, p) |
1818 | s.checkStopInstances(c, i0, i1) |
1819 | s.waitRemoved(c, m0) |
1820 | } |
1821 | |
1822 | func (s *ProvisionerSuite) TestDyingMachines(c *gc.C) { |
1823 | - p := s.newEnvironProvisioner("0") |
1824 | + p := s.newEnvironProvisioner(c, "0") |
1825 | defer stop(c, p) |
1826 | |
1827 | // provision a machine |
1828 | @@ -527,7 +529,7 @@ |
1829 | c.Assert(err, gc.IsNil) |
1830 | |
1831 | // start the provisioner and wait for it to reap the useless machine |
1832 | - p = s.newEnvironProvisioner("0") |
1833 | + p = s.newEnvironProvisioner(c, "0") |
1834 | defer stop(c, p) |
1835 | s.checkNoOperations(c) |
1836 | s.waitRemoved(c, m1) |
1837 | @@ -539,7 +541,7 @@ |
1838 | } |
1839 | |
1840 | func (s *ProvisionerSuite) TestProvisioningRecoversAfterInvalidEnvironmentPublished(c *gc.C) { |
1841 | - p := s.newEnvironProvisioner("0") |
1842 | + p := s.newEnvironProvisioner(c, "0") |
1843 | defer stop(c, p) |
1844 | |
1845 | // place a new machine into the state |
Reviewers: mp+183558_ code.launchpad. net,
Message:
Please take a look.
Description:
Introduce the format 1.16 for agent confg.
With adding the new format, a migrate method was added
to the formatter interface to provide a defined place
to do additional upgrade data munging as needed.
When an old format config is read, it is immediately
migrated and written out using the current format.
The new format allows storing of key/value pairs. These
are now used instead of OS environment variables for
passing on the provider type, lxc bridge, and local
storage information.
Cloudinit no longer writes upstart scripts with extra
environment variables for the machine agents, but instead
writes the extra variables to the agent config. Due to
this change, the MachineEnvironment param was changed
to AgentEnvironment.
Also the StatePort was never read, so it has been removed
from information that is saved.
The cloudinit tests were modified to be agnostic to the
actual format of the agent.conf files.
Previous review found here: /codereview. appspot. com/13421043/
https:/
https:/ /code.launchpad .net/~thumper/ juju-core/ new-agent- format/ +merge/ 183558
Requires: /code.launchpad .net/~thumper/ juju-core/ pass-through- agent-config/ +merge/ 183543
https:/
(do not edit description out of merge proposal)
Please review this at https:/ /codereview. appspot. com/13481043/
Affected files: 1.12.go 1.12_whitebox_ test.go 1.16.go 1.16_whitebox_ test.go whitebox_ test.go machine. go lxc/lxc. go cloudinit. go cloudinit/ cloudinit. go cloudinit/ cloudinit_ test.go cloudinit_ test.go manual/ agent.go conn.go local/environ. go local/storage/ worker. go maas/environ. go deployer/ simple. go deployer/ simple_ test.go provisioner/ lxc-broker. go provisioner/ lxc-broker_ test.go provisioner/ provisioner. go provisioner/ provisioner_ test.go
A [revision details]
M agent/agent.go
M agent/format-
M agent/format-
A agent/format-
A agent/format-
M agent/format.go
M agent/format_
M cmd/jujud/
M container/
M environs/
M environs/
M environs/
M environs/
M environs/
M juju/osenv/vars.go
M juju/testing/
M provider/
M provider/
M provider/
M worker/
M worker/
M worker/
M worker/
M worker/
M worker/