Merge lp:~axwalk/juju-core/local-provider-environs-cloudinit into lp:~go-bot/juju-core/trunk

Proposed by Andrew Wilkins
Status: Merged
Approved by: Andrew Wilkins
Approved revision: no longer in the source branch.
Merged at revision: 2250
Proposed branch: lp:~axwalk/juju-core/local-provider-environs-cloudinit
Merge into: lp:~go-bot/juju-core/trunk
Diff against target: 1458 lines (+383/-487)
14 files modified
agent/agent.go (+1/-0)
cmd/jujud/machine.go (+10/-0)
environs/cloudinit.go (+20/-2)
environs/cloudinit/cloudinit.go (+90/-51)
environs/cloudinit/cloudinit_test.go (+83/-36)
environs/cloudinit_test.go (+12/-8)
provider/azure/customdata_test.go (+9/-6)
provider/local/config.go (+7/-36)
provider/local/config_test.go (+9/-23)
provider/local/environ.go (+92/-301)
provider/local/environ_test.go (+18/-4)
provider/local/environprovider.go (+12/-0)
provider/local/export_test.go (+6/-12)
provider/local/instance.go (+14/-8)
To merge this branch: bzr merge lp:~axwalk/juju-core/local-provider-environs-cloudinit
Reviewer Review Type Date Requested Status
Juju Engineering Pending
Review via email: mp+202791@code.launchpad.net

Commit message

local: use sudo internally

The local provider has been updated to do several things:
- use environs/cloudinit to bootstrap
- rely on the terminationworker to cleanup
- prevent bootstrap if the user is root (forcing good practice)
- use sudo internally, in bootstrap/destroy

Bootstrap now generates a script, a la manual bootstrap,
and executes it with "sudo bash". Destroy works by repeating
the "juju destroy-environment" command under sudo, which is
necessary to destroy any leftover containers.

https://codereview.appspot.com/55880043/

Description of the change

local: use sudo internally

The local provider has been updated to do several things:
- use environs/cloudinit to bootstrap
- rely on the terminationworker to cleanup
- prevent bootstrap if the user is root (forcing good practice)
- use sudo internally, in bootstrap/destroy

Bootstrap now generates a script, a la manual bootstrap,
and executes it with "sudo bash". Destroy works by repeating
the "juju destroy-environment" command under sudo, which is
necessary to destroy any leftover containers.

https://codereview.appspot.com/55880043/

To post a comment you must log in.
Revision history for this message
Andrew Wilkins (axwalk) wrote :

Reviewers: mp+202791_code.launchpad.net,

Message:
Please take a look.

Description:
local: use sudo internally

The local provider has been updated to do several things:
- use environs/cloudinit to bootstrap
- rely on the terminationworker to cleanup
- prevent bootstrap if the user is root (forcing good practice)
- use sudo internally, in bootstrap/destroy

Bootstrap now generates a script, a la manual bootstrap,
and executes it with "sudo bash". Destroy works by repeating
the "juju destroy-environment" command under sudo, which is
necessary to destroy any leftover containers.

https://code.launchpad.net/~axwalk/juju-core/local-provider-environs-cloudinit/+merge/202791

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/55880043/

Affected files (+351, -477 lines):
   A [revision details]
   M agent/agent.go
   M cmd/jujud/machine.go
   M environs/cloudinit.go
   M environs/cloudinit/cloudinit.go
   M environs/cloudinit/cloudinit_test.go
   M environs/cloudinit_test.go
   M provider/azure/customdata_test.go
   M provider/local/config.go
   M provider/local/config_test.go
   M provider/local/environ.go
   M provider/local/environ_test.go
   M provider/local/environprovider.go
   M provider/local/export_test.go
   M provider/local/instance.go

Revision history for this message
Michael Nelson (michael.nelson) wrote :

https://codereview.appspot.com/55880043/diff/1/provider/local/environ.go
File provider/local/environ.go (right):

https://codereview.appspot.com/55880043/diff/1/provider/local/environ.go#newcode402
provider/local/environ.go:402: cmd := exec.Command("sudo",
append([]string{juju}, os.Args[1:]...)...)
Not a review, just a question: If I'm using /home/me/golang/bin/juju
destroy-environment (whether via PATH or explicitly), I think "sudo juju
..." won't necessarily use the same binary - at least, I was caught by
that recently. It might be better to get the actual binary name instead
of []string{juju}?

https://codereview.appspot.com/55880043/

Revision history for this message
Andrew Wilkins (axwalk) wrote :

https://codereview.appspot.com/55880043/diff/1/provider/local/environ.go
File provider/local/environ.go (right):

https://codereview.appspot.com/55880043/diff/1/provider/local/environ.go#newcode402
provider/local/environ.go:402: cmd := exec.Command("sudo",
append([]string{juju}, os.Args[1:]...)...)
On 2014/01/23 10:54:28, Michael Nelson wrote:
> Not a review, just a question: If I'm using /home/me/golang/bin/juju
> destroy-environment (whether via PATH or explicitly), I think "sudo
juju ..."
> won't necessarily use the same binary - at least, I was caught by that
recently.

Right, this is one of the problems with using sudo externally: the
environment isn't (by default) propagated through sudo. People have had
to work around this by either using "sudo -E ..." or "sudo `which juju`
...".

> It might be better to get the actual binary name instead of
[]string{juju}?

This is the point of the exec.LookPath above. See:
http://golang.org/pkg/os/exec/#LookPath

https://codereview.appspot.com/55880043/

Revision history for this message
Michael Nelson (michael.nelson) wrote :

On Thu, Jan 23, 2014 at 12:02 PM, <email address hidden> wrote:
>
> This is the point of the exec.LookPath above. See:
> http://golang.org/pkg/os/exec/#LookPath
>

Excellent, thanks.

Revision history for this message
Andrew Wilkins (axwalk) wrote :
Revision history for this message
Tim Penhey (thumper) wrote :

LGTM - looks awesome!

https://codereview.appspot.com/55880043/diff/20001/provider/local/export_test.go
File provider/local/export_test.go (right):

https://codereview.appspot.com/55880043/diff/20001/provider/local/export_test.go#newcode14
provider/local/export_test.go:14: FinishBootstrap = &finishBootstrap
nice...

https://codereview.appspot.com/55880043/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'agent/agent.go'
2--- agent/agent.go 2013-11-22 01:52:18 +0000
3+++ agent/agent.go 2014-01-23 22:11:39 +0000
4@@ -31,6 +31,7 @@
5 SharedStorageAddr = "SHARED_STORAGE_ADDR"
6 AgentServiceName = "AGENT_SERVICE_NAME"
7 MongoServiceName = "MONGO_SERVICE_NAME"
8+ RsyslogConfPath = "RSYSLOG_CONF_PATH"
9 )
10
11 // The Config interface is the sole way that the agent gets access to the
12
13=== modified file 'cmd/jujud/machine.go'
14--- cmd/jujud/machine.go 2014-01-22 15:40:18 +0000
15+++ cmd/jujud/machine.go 2014-01-23 22:11:39 +0000
16@@ -18,6 +18,7 @@
17 "launchpad.net/juju-core/cmd"
18 "launchpad.net/juju-core/container/kvm"
19 "launchpad.net/juju-core/instance"
20+ "launchpad.net/juju-core/log/syslog"
21 "launchpad.net/juju-core/names"
22 "launchpad.net/juju-core/provider"
23 "launchpad.net/juju-core/state"
24@@ -378,6 +379,15 @@
25 errors = append(errors, fmt.Errorf("cannot remove service %q: %v", agentServiceName, err))
26 }
27 }
28+ // Remove the rsyslog conf file and restart rsyslogd.
29+ if rsyslogConfPath := a.Conf.config.Value(agent.RsyslogConfPath); rsyslogConfPath != "" {
30+ if err := os.Remove(rsyslogConfPath); err != nil {
31+ errors = append(errors, err)
32+ }
33+ if err := syslog.Restart(); err != nil {
34+ errors = append(errors, err)
35+ }
36+ }
37 // Remove the juju-run symlink.
38 if err := os.Remove(jujuRun); err != nil && !os.IsNotExist(err) {
39 errors = append(errors, err)
40
41=== modified file 'environs/cloudinit.go'
42--- environs/cloudinit.go 2014-01-21 03:26:37 +0000
43+++ environs/cloudinit.go 2014-01-23 22:11:39 +0000
44@@ -11,16 +11,29 @@
45 "launchpad.net/juju-core/constraints"
46 "launchpad.net/juju-core/environs/cloudinit"
47 "launchpad.net/juju-core/environs/config"
48+ "launchpad.net/juju-core/names"
49 "launchpad.net/juju-core/state"
50 "launchpad.net/juju-core/state/api"
51 "launchpad.net/juju-core/utils"
52 )
53
54-// Default data directory.
55+// DataDir is the default data directory.
56 // Tests can override this where needed, so they don't need to mess with global
57 // system state.
58 var DataDir = "/var/lib/juju"
59
60+// LogDir is the default log file path.
61+const LogDir = "/var/log/juju"
62+
63+// CloudInitOutputLog is the default cloud-init-output.log file path.
64+const CloudInitOutputLog = "/var/log/cloud-init-output.log"
65+
66+// RsyslogConfPath is the default rsyslogd conf file path.
67+const RsyslogConfPath = "/etc/rsyslog.d/25-juju.conf"
68+
69+// MongoServiceName is the default Upstart service name for Mongo.
70+const MongoServiceName = "juju-db"
71+
72 // NewMachineConfig sets up a basic machine configuration, for a non-bootstrap
73 // node. You'll still need to supply more information, but this takes care of
74 // the fixed entries and the ones that are always needed.
75@@ -28,7 +41,12 @@
76 stateInfo *state.Info, apiInfo *api.Info) *cloudinit.MachineConfig {
77 return &cloudinit.MachineConfig{
78 // Fixed entries.
79- DataDir: DataDir,
80+ DataDir: DataDir,
81+ LogDir: LogDir,
82+ CloudInitOutputLog: CloudInitOutputLog,
83+ RsyslogConfPath: RsyslogConfPath,
84+ MachineAgentServiceName: "jujud-" + names.MachineTag(machineID),
85+ MongoServiceName: MongoServiceName,
86
87 // Parameter entries.
88 MachineId: machineID,
89
90=== modified file 'environs/cloudinit/cloudinit.go'
91--- environs/cloudinit/cloudinit.go 2014-01-22 04:59:41 +0000
92+++ environs/cloudinit/cloudinit.go 2014-01-23 22:11:39 +0000
93@@ -91,6 +91,17 @@
94 // machine.
95 DataDir string
96
97+ // LogDir holds the directory that juju logs will be written to.
98+ LogDir string
99+
100+ // RsyslogConfPath is the path to the rsyslogd conf file written
101+ // for configuring distributed logging.
102+ RsyslogConfPath string
103+
104+ // CloudInitOutputLog specifies the path to the output log for cloud-init.
105+ // The directory containing the log file must already exist.
106+ CloudInitOutputLog string
107+
108 // MachineId identifies the new machine.
109 MachineId string
110
111@@ -131,6 +142,16 @@
112 // StateServer (member above) is set to true.
113 SystemPrivateSSHKey string
114
115+ // DisablePackageCommands is a flag that specifies whether to suppress
116+ // the addition of package management commands.
117+ DisablePackageCommands bool
118+
119+ // MachineAgentServiceName is the Upstart service name for the Juju machine agent.
120+ MachineAgentServiceName string
121+
122+ // MongoServiceName is the Upstart service name for the Mongo database.
123+ MongoServiceName string
124+
125 // ProxySettings define normal http, https and ftp proxies.
126 ProxySettings osenv.ProxySettings
127
128@@ -157,8 +178,6 @@
129 return ConfigureJuju(cfg, c)
130 }
131
132-const cloudInitOutputLog = "/var/log/cloud-init-output.log"
133-
134 // NonceFile is written by cloud-init as the last thing it does.
135 // The file will contain the machine's nonce. The filename is
136 // relative to the Juju data-dir.
137@@ -181,7 +200,7 @@
138 "set -xe", // ensure we run all the scripts or abort.
139 )
140 c.AddSSHAuthorizedKeys(cfg.AuthorizedKeys)
141- c.SetOutput(cloudinit.OutAll, "| tee -a "+cloudInitOutputLog, "")
142+ c.SetOutput(cloudinit.OutAll, "| tee -a "+cfg.CloudInitOutputLog, "")
143 // Create a file in a well-defined location containing the machine's
144 // nonce. The presence and contents of this file will be verified
145 // during bootstrap.
146@@ -215,45 +234,50 @@
147 // have been set. We don't want to show the log to the user, so simply
148 // append to the log file rather than teeing.
149 if stdout, _ := c.Output(cloudinit.OutAll); stdout == "" {
150- c.SetOutput(cloudinit.OutAll, ">> "+cloudInitOutputLog, "")
151+ c.SetOutput(cloudinit.OutAll, ">> "+cfg.CloudInitOutputLog, "")
152 c.AddBootCmd(initProgressCmd)
153- c.AddBootCmd(cloudinit.LogProgressCmd("Logging to %s on remote host", cloudInitOutputLog))
154- }
155-
156- // Write out the apt proxy settings
157- if (cfg.AptProxySettings != osenv.ProxySettings{}) {
158- filename := "/etc/apt/apt.conf.d/42-juju-proxy-settings"
159- c.AddBootCmd(fmt.Sprintf(
160- `[ -f %s ] || (printf '%%s\n' %s > %s)`,
161- filename,
162- shquote(utils.AptProxyContent(cfg.AptProxySettings)),
163- filename))
164- }
165-
166- // Bring packages up-to-date.
167- c.SetAptUpdate(true)
168- c.SetAptUpgrade(true)
169-
170- // juju requires git for managing charm directories.
171- c.AddPackage("git")
172- c.AddPackage("cpu-checker")
173+ c.AddBootCmd(cloudinit.LogProgressCmd("Logging to %s on remote host", cfg.CloudInitOutputLog))
174+ }
175+
176+ if !cfg.DisablePackageCommands {
177+ // Bring packages up-to-date.
178+ c.SetAptUpdate(true)
179+ c.SetAptUpgrade(true)
180+
181+ // juju requires git for managing charm directories.
182+ c.AddPackage("git")
183+ c.AddPackage("cpu-checker")
184+
185+ // Write out the apt proxy settings
186+ if (cfg.AptProxySettings != osenv.ProxySettings{}) {
187+ filename := "/etc/apt/apt.conf.d/42-juju-proxy-settings"
188+ c.AddBootCmd(fmt.Sprintf(
189+ `[ -f %s ] || (printf '%%s\n' %s > %s)`,
190+ filename,
191+ shquote(utils.AptProxyContent(cfg.AptProxySettings)),
192+ filename))
193+ }
194+ }
195
196 // Write out the normal proxy settings so that the settings are
197 // sourced by bash, and ssh through that.
198 c.AddScripts(
199 // We look to see if the proxy line is there already as
200- // the manual provider may have had it aleady.
201- `grep -q '.juju-proxy' /home/ubuntu/.profile || printf '\n# Added by juju\n[ -f "$HOME/.juju-proxy" ] && . "$HOME/.juju-proxy"\n' >> /home/ubuntu/.profile`)
202+ // the manual provider may have had it aleady. The ubuntu
203+ // user may not exist (local provider only).
204+ `([ ! -e /home/ubuntu/.profile ] || grep -q '.juju-proxy' /home/ubuntu/.profile) || ` +
205+ `printf '\n# Added by juju\n[ -f "$HOME/.juju-proxy" ] && . "$HOME/.juju-proxy"\n' >> /home/ubuntu/.profile`)
206 if (cfg.ProxySettings != osenv.ProxySettings{}) {
207 c.AddScripts(
208 fmt.Sprintf(
209- `printf '%%s\n' %s > /home/ubuntu/.juju-proxy && chown ubuntu:ubuntu /home/ubuntu/.juju-proxy`,
210+ `[ -e /home/ubuntu ] && (printf '%%s\n' %s > /home/ubuntu/.juju-proxy && chown ubuntu:ubuntu /home/ubuntu/.juju-proxy)`,
211 shquote(cfg.ProxySettings.AsEnvironmentValues())))
212 }
213
214 c.AddScripts(
215 fmt.Sprintf("mkdir -p %s", cfg.DataDir),
216- "mkdir -p /var/log/juju")
217+ fmt.Sprintf("mkdir -p %s", cfg.LogDir),
218+ )
219
220 // Make a directory for the tools to live in, then fetch the
221 // tools and unarchive them into it.
222@@ -303,23 +327,27 @@
223 // Add the cloud archive cloud-tools pocket to apt sources
224 // for series that need it. This gives us up-to-date LXC,
225 // MongoDB, and other infrastructure.
226- cfg.MaybeAddCloudArchiveCloudTools(c)
227+ if !cfg.DisablePackageCommands {
228+ cfg.MaybeAddCloudArchiveCloudTools(c)
229+ }
230
231 if cfg.StateServer {
232 identityFile := cfg.dataFile(SystemIdentity)
233 c.AddFile(identityFile, cfg.SystemPrivateSSHKey, 0600)
234- // Disable the default mongodb installed by the mongodb-server package.
235- // Only do this if the file doesn't exist already, so users can run
236- // their own mongodb server if they wish to.
237- c.AddBootCmd(
238- `[ -f /etc/default/mongodb ] ||
239+ if !cfg.DisablePackageCommands {
240+ // Disable the default mongodb installed by the mongodb-server package.
241+ // Only do this if the file doesn't exist already, so users can run
242+ // their own mongodb server if they wish to.
243+ c.AddBootCmd(
244+ `[ -f /etc/default/mongodb ] ||
245 (echo ENABLE_MONGODB="no" > /etc/default/mongodb)`)
246
247- if cfg.NeedMongoPPA() {
248- const key = "" // key is loaded from PPA
249- c.AddAptSource("ppa:juju/stable", key)
250+ if cfg.NeedMongoPPA() {
251+ const key = "" // key is loaded from PPA
252+ c.AddAptSource("ppa:juju/stable", key)
253+ }
254+ c.AddPackage("mongodb-server")
255 }
256- c.AddPackage("mongodb-server")
257 certKey := string(cfg.StateServerCert) + string(cfg.StateServerKey)
258 c.AddFile(cfg.dataFile("server.pem"), certKey, 0600)
259 if err := cfg.addMongoToBoot(c); err != nil {
260@@ -358,7 +386,7 @@
261
262 func (cfg *MachineConfig) addLogging(c *cloudinit.Config) error {
263 namespace := cfg.AgentEnvironment[agent.Namespace]
264- var configRenderer syslog.SyslogConfigRenderer
265+ var configRenderer *syslog.SyslogConfig
266 if cfg.StateServer {
267 configRenderer = syslog.NewAccumulateConfig(
268 names.MachineTag(cfg.MachineId), cfg.SyslogPort, namespace)
269@@ -366,11 +394,12 @@
270 configRenderer = syslog.NewForwardConfig(
271 names.MachineTag(cfg.MachineId), cfg.SyslogPort, namespace, cfg.stateHostAddrs())
272 }
273+ configRenderer.LogDir = cfg.LogDir
274 content, err := configRenderer.Render()
275 if err != nil {
276 return err
277 }
278- c.AddFile("/etc/rsyslog.d/25-juju.conf", string(content), 0600)
279+ c.AddFile(cfg.RsyslogConfPath, string(content), 0600)
280 c.AddRunCmd("restart rsyslog")
281 return nil
282 }
283@@ -418,9 +447,10 @@
284 if err != nil {
285 return nil, err
286 }
287- acfg.SetValue(agent.AgentServiceName, machineAgentServiceName(tag))
288+ acfg.SetValue(agent.RsyslogConfPath, cfg.RsyslogConfPath)
289+ acfg.SetValue(agent.AgentServiceName, cfg.MachineAgentServiceName)
290 if cfg.StateServer {
291- acfg.SetValue(agent.MongoServiceName, mongoServiceName)
292+ acfg.SetValue(agent.MongoServiceName, cfg.MongoServiceName)
293 }
294 cmds, err := acfg.WriteCommands()
295 if err != nil {
296@@ -430,12 +460,6 @@
297 return acfg, nil
298 }
299
300-const mongoServiceName = "juju-db"
301-
302-func machineAgentServiceName(tag string) string {
303- return "jujud-" + tag
304-}
305-
306 func (cfg *MachineConfig) addMachineAgentToBoot(c *cloudinit.Config, tag, machineId string) error {
307 // Make the agent run via a symbolic link to the actual tools
308 // directory, so it can upgrade itself without needing to change
309@@ -444,8 +468,8 @@
310 // TODO(dfc) ln -nfs, so it doesn't fail if for some reason that the target already exists
311 c.AddScripts(fmt.Sprintf("ln -s %v %s", cfg.Tools.Version, shquote(toolsDir)))
312
313- name := machineAgentServiceName(tag)
314- conf := upstart.MachineAgentUpstartService(name, toolsDir, cfg.DataDir, "/var/log/juju/", tag, machineId, nil)
315+ name := cfg.MachineAgentServiceName
316+ conf := upstart.MachineAgentUpstartService(name, toolsDir, cfg.DataDir, cfg.LogDir, tag, machineId, nil)
317 cmds, err := conf.InstallCommands()
318 if err != nil {
319 return fmt.Errorf("cannot make cloud-init upstart script for the %s agent: %v", tag, err)
320@@ -466,7 +490,7 @@
321 "dd bs=1M count=1 if=/dev/zero of="+dbDir+"/journal/prealloc.2",
322 )
323
324- name := mongoServiceName
325+ name := cfg.MongoServiceName
326 conf := upstart.MongoUpstartService(name, cfg.DataDir, dbDir, cfg.StatePort)
327 cmds, err := conf.InstallCommands()
328 if err != nil {
329@@ -602,6 +626,15 @@
330 if cfg.DataDir == "" {
331 return fmt.Errorf("missing var directory")
332 }
333+ if cfg.LogDir == "" {
334+ return fmt.Errorf("missing log directory")
335+ }
336+ if cfg.CloudInitOutputLog == "" {
337+ return fmt.Errorf("missing cloud-init output log path")
338+ }
339+ if cfg.RsyslogConfPath == "" {
340+ return fmt.Errorf("missing rsyslog.d conf path")
341+ }
342 if cfg.Tools == nil {
343 return fmt.Errorf("missing tools")
344 }
345@@ -623,7 +656,13 @@
346 if len(cfg.APIInfo.CACert) == 0 {
347 return fmt.Errorf("missing API CA certificate")
348 }
349+ if cfg.MachineAgentServiceName == "" {
350+ return fmt.Errorf("missing machine agent service name")
351+ }
352 if cfg.StateServer {
353+ if cfg.MongoServiceName == "" {
354+ return fmt.Errorf("missing mongo service name")
355+ }
356 if cfg.Config == nil {
357 return fmt.Errorf("missing environment configuration")
358 }
359
360=== modified file 'environs/cloudinit/cloudinit_test.go'
361--- environs/cloudinit/cloudinit_test.go 2014-01-22 04:59:41 +0000
362+++ environs/cloudinit/cloudinit_test.go 2014-01-23 22:11:39 +0000
363@@ -82,10 +82,15 @@
364 Password: "bletch",
365 CACert: []byte("CA CERT\n" + testing.CACert),
366 },
367- Constraints: envConstraints,
368- DataDir: environs.DataDir,
369- StateInfoURL: "some-url",
370- SystemPrivateSSHKey: "private rsa key",
371+ Constraints: envConstraints,
372+ DataDir: environs.DataDir,
373+ LogDir: environs.LogDir,
374+ CloudInitOutputLog: environs.CloudInitOutputLog,
375+ RsyslogConfPath: environs.RsyslogConfPath,
376+ StateInfoURL: "some-url",
377+ SystemPrivateSSHKey: "private rsa key",
378+ MachineAgentServiceName: "jujud-machine-0",
379+ MongoServiceName: "juju-db",
380 },
381 setEnvConfig: true,
382 expectScripts: `
383@@ -94,7 +99,7 @@
384 install -D -m 644 /dev/null '/var/lib/juju/nonce.txt'
385 printf '%s\\n' 'FAKE_NONCE' > '/var/lib/juju/nonce.txt'
386 test -e /proc/self/fd/9 \|\| exec 9>&2
387-grep -q '.juju-proxy' /home/ubuntu/.profile \|\| printf .* >> /home/ubuntu/.profile
388+\(\[ ! -e /home/ubuntu/.profile \] \|\| grep -q '.juju-proxy' /home/ubuntu/.profile\) \|\| printf .* >> /home/ubuntu/.profile
389 mkdir -p /var/lib/juju
390 mkdir -p /var/log/juju
391 echo 'Fetching tools.*
392@@ -163,10 +168,15 @@
393 Password: "bletch",
394 CACert: []byte("CA CERT\n" + testing.CACert),
395 },
396- Constraints: envConstraints,
397- DataDir: environs.DataDir,
398- StateInfoURL: "some-url",
399- SystemPrivateSSHKey: "private rsa key",
400+ Constraints: envConstraints,
401+ DataDir: environs.DataDir,
402+ LogDir: environs.LogDir,
403+ CloudInitOutputLog: environs.CloudInitOutputLog,
404+ RsyslogConfPath: environs.RsyslogConfPath,
405+ StateInfoURL: "some-url",
406+ SystemPrivateSSHKey: "private rsa key",
407+ MachineAgentServiceName: "jujud-machine-0",
408+ MongoServiceName: "juju-db",
409 },
410 setEnvConfig: true,
411 inexactMatch: true,
412@@ -184,13 +194,16 @@
413 }, {
414 // non state server.
415 cfg: cloudinit.MachineConfig{
416- MachineId: "99",
417- AuthorizedKeys: "sshkey1",
418- AgentEnvironment: map[string]string{agent.ProviderType: "dummy"},
419- DataDir: environs.DataDir,
420- StateServer: false,
421- Tools: newSimpleTools("1.2.3-linux-amd64"),
422- MachineNonce: "FAKE_NONCE",
423+ MachineId: "99",
424+ AuthorizedKeys: "sshkey1",
425+ AgentEnvironment: map[string]string{agent.ProviderType: "dummy"},
426+ DataDir: environs.DataDir,
427+ LogDir: environs.LogDir,
428+ CloudInitOutputLog: environs.CloudInitOutputLog,
429+ RsyslogConfPath: environs.RsyslogConfPath,
430+ StateServer: false,
431+ Tools: newSimpleTools("1.2.3-linux-amd64"),
432+ MachineNonce: "FAKE_NONCE",
433 StateInfo: &state.Info{
434 Addrs: []string{"state-addr.testing.invalid:12345"},
435 Tag: "machine-99",
436@@ -203,14 +216,15 @@
437 Password: "bletch",
438 CACert: []byte("CA CERT\n" + testing.CACert),
439 },
440- SyslogPort: 514,
441+ SyslogPort: 514,
442+ MachineAgentServiceName: "jujud-machine-99",
443 },
444 expectScripts: `
445 set -xe
446 install -D -m 644 /dev/null '/var/lib/juju/nonce.txt'
447 printf '%s\\n' 'FAKE_NONCE' > '/var/lib/juju/nonce.txt'
448 test -e /proc/self/fd/9 \|\| exec 9>&2
449-grep -q '.juju-proxy' /home/ubuntu/.profile \|\| printf .* >> /home/ubuntu/.profile
450+\(\[ ! -e /home/ubuntu/\.profile \] \|\| grep -q '.juju-proxy' /home/ubuntu/.profile\) \|\| printf .* >> /home/ubuntu/.profile
451 mkdir -p /var/lib/juju
452 mkdir -p /var/log/juju
453 echo 'Fetching tools.*
454@@ -243,6 +257,9 @@
455 AuthorizedKeys: "sshkey1",
456 AgentEnvironment: map[string]string{agent.ProviderType: "dummy"},
457 DataDir: environs.DataDir,
458+ LogDir: environs.LogDir,
459+ CloudInitOutputLog: environs.CloudInitOutputLog,
460+ RsyslogConfPath: environs.RsyslogConfPath,
461 StateServer: false,
462 Tools: newSimpleTools("1.2.3-linux-amd64"),
463 MachineNonce: "FAKE_NONCE",
464@@ -258,7 +275,8 @@
465 Password: "bletch",
466 CACert: []byte("CA CERT\n" + testing.CACert),
467 },
468- SyslogPort: 514,
469+ SyslogPort: 514,
470+ MachineAgentServiceName: "jujud-machine-2-lxc-1",
471 },
472 inexactMatch: true,
473 expectScripts: `
474@@ -276,13 +294,16 @@
475 }, {
476 // hostname verification disabled.
477 cfg: cloudinit.MachineConfig{
478- MachineId: "99",
479- AuthorizedKeys: "sshkey1",
480- AgentEnvironment: map[string]string{agent.ProviderType: "dummy"},
481- DataDir: environs.DataDir,
482- StateServer: false,
483- Tools: newSimpleTools("1.2.3-linux-amd64"),
484- MachineNonce: "FAKE_NONCE",
485+ MachineId: "99",
486+ AuthorizedKeys: "sshkey1",
487+ AgentEnvironment: map[string]string{agent.ProviderType: "dummy"},
488+ DataDir: environs.DataDir,
489+ LogDir: environs.LogDir,
490+ CloudInitOutputLog: environs.CloudInitOutputLog,
491+ RsyslogConfPath: environs.RsyslogConfPath,
492+ StateServer: false,
493+ Tools: newSimpleTools("1.2.3-linux-amd64"),
494+ MachineNonce: "FAKE_NONCE",
495 StateInfo: &state.Info{
496 Addrs: []string{"state-addr.testing.invalid:12345"},
497 Tag: "machine-99",
498@@ -297,6 +318,7 @@
499 },
500 SyslogPort: 514,
501 DisableSSLHostnameVerification: true,
502+ MachineAgentServiceName: "jujud-machine-99",
503 },
504 inexactMatch: true,
505 expectScripts: `
506@@ -325,9 +347,14 @@
507 Password: "bletch",
508 CACert: []byte("CA CERT\n" + testing.CACert),
509 },
510- DataDir: environs.DataDir,
511- StateInfoURL: "some-url",
512- SystemPrivateSSHKey: "private rsa key",
513+ DataDir: environs.DataDir,
514+ LogDir: environs.LogDir,
515+ CloudInitOutputLog: environs.CloudInitOutputLog,
516+ RsyslogConfPath: environs.RsyslogConfPath,
517+ StateInfoURL: "some-url",
518+ SystemPrivateSSHKey: "private rsa key",
519+ MachineAgentServiceName: "jujud-machine-0",
520+ MongoServiceName: "juju-db",
521 },
522 setEnvConfig: true,
523 inexactMatch: true,
524@@ -653,6 +680,15 @@
525 {"missing var directory", func(cfg *cloudinit.MachineConfig) {
526 cfg.DataDir = ""
527 }},
528+ {"missing log directory", func(cfg *cloudinit.MachineConfig) {
529+ cfg.LogDir = ""
530+ }},
531+ {"missing cloud-init output log path", func(cfg *cloudinit.MachineConfig) {
532+ cfg.CloudInitOutputLog = ""
533+ }},
534+ {"missing rsyslog.d conf path", func(cfg *cloudinit.MachineConfig) {
535+ cfg.RsyslogConfPath = ""
536+ }},
537 {"missing tools", func(cfg *cloudinit.MachineConfig) {
538 cfg.Tools = nil
539 }},
540@@ -702,6 +738,12 @@
541 {"missing machine nonce", func(cfg *cloudinit.MachineConfig) {
542 cfg.MachineNonce = ""
543 }},
544+ {"missing machine agent service name", func(cfg *cloudinit.MachineConfig) {
545+ cfg.MachineAgentServiceName = ""
546+ }},
547+ {"missing mongo service name", func(cfg *cloudinit.MachineConfig) {
548+ cfg.MongoServiceName = ""
549+ }},
550 }
551
552 // TestCloudInitVerify checks that required fields are appropriately
553@@ -727,10 +769,15 @@
554 Addrs: []string{"host:9999"},
555 CACert: []byte(testing.CACert),
556 },
557- Config: minimalConfig(c),
558- DataDir: environs.DataDir,
559- MachineNonce: "FAKE_NONCE",
560- SystemPrivateSSHKey: "private rsa key",
561+ Config: minimalConfig(c),
562+ DataDir: environs.DataDir,
563+ LogDir: environs.LogDir,
564+ CloudInitOutputLog: environs.CloudInitOutputLog,
565+ RsyslogConfPath: environs.RsyslogConfPath,
566+ MachineNonce: "FAKE_NONCE",
567+ SystemPrivateSSHKey: "private rsa key",
568+ MachineAgentServiceName: "jujud-machine-99",
569+ MongoServiceName: "juju-db",
570 }
571 // check that the base configuration does not give an error
572 ci := coreCloudinit.New()
573@@ -800,9 +847,9 @@
574 c.Assert(err, gc.IsNil)
575
576 cmds := cloudcfg.RunCmds()
577- first := `grep -q '.juju-proxy' /home/ubuntu/.profile || printf '\n# Added by juju\n[ -f "$HOME/.juju-proxy" ] && . "$HOME/.juju-proxy"\n' >> /home/ubuntu/.profile`
578- second := `printf '%s\n' 'export http_proxy=http://user@10.0.0.1
579-export HTTP_PROXY=http://user@10.0.0.1' > /home/ubuntu/.juju-proxy && chown ubuntu:ubuntu /home/ubuntu/.juju-proxy`
580+ first := `([ ! -e /home/ubuntu/.profile ] || grep -q '.juju-proxy' /home/ubuntu/.profile) || printf '\n# Added by juju\n[ -f "$HOME/.juju-proxy" ] && . "$HOME/.juju-proxy"\n' >> /home/ubuntu/.profile`
581+ second := `[ -e /home/ubuntu ] && (printf '%s\n' 'export http_proxy=http://user@10.0.0.1
582+export HTTP_PROXY=http://user@10.0.0.1' > /home/ubuntu/.juju-proxy && chown ubuntu:ubuntu /home/ubuntu/.juju-proxy)`
583 found := false
584 for i, cmd := range cmds {
585 if cmd == first {
586
587=== modified file 'environs/cloudinit_test.go'
588--- environs/cloudinit_test.go 2014-01-22 23:36:28 +0000
589+++ environs/cloudinit_test.go 2014-01-23 22:11:39 +0000
590@@ -175,14 +175,18 @@
591 CACert: []byte("CA CERT\n" + testing.CACert),
592 Tag: "machine-10",
593 },
594- DataDir: environs.DataDir,
595- Config: envConfig,
596- StatePort: envConfig.StatePort(),
597- APIPort: envConfig.APIPort(),
598- SyslogPort: envConfig.SyslogPort(),
599- StateServer: stateServer,
600- AgentEnvironment: map[string]string{agent.ProviderType: "dummy"},
601- AuthorizedKeys: "wheredidileavemykeys",
602+ DataDir: environs.DataDir,
603+ LogDir: environs.LogDir,
604+ CloudInitOutputLog: environs.CloudInitOutputLog,
605+ RsyslogConfPath: environs.RsyslogConfPath,
606+ Config: envConfig,
607+ StatePort: envConfig.StatePort(),
608+ APIPort: envConfig.APIPort(),
609+ SyslogPort: envConfig.SyslogPort(),
610+ StateServer: stateServer,
611+ AgentEnvironment: map[string]string{agent.ProviderType: "dummy"},
612+ AuthorizedKeys: "wheredidileavemykeys",
613+ MachineAgentServiceName: "jujud-machine-10",
614 }
615 script1 := "script1"
616 script2 := "script2"
617
618=== modified file 'provider/azure/customdata_test.go'
619--- provider/azure/customdata_test.go 2013-11-20 04:29:46 +0000
620+++ provider/azure/customdata_test.go 2014-01-23 22:11:39 +0000
621@@ -26,13 +26,15 @@
622
623 // makeMachineConfig produces a valid cloudinit machine config.
624 func makeMachineConfig(c *gc.C) *cloudinit.MachineConfig {
625- dir := c.MkDir()
626 machineID := "0"
627 return &cloudinit.MachineConfig{
628- MachineId: machineID,
629- MachineNonce: "gxshasqlnng",
630- DataDir: dir,
631- Tools: &tools.Tools{URL: "file://" + dir},
632+ MachineId: machineID,
633+ MachineNonce: "gxshasqlnng",
634+ DataDir: environs.DataDir,
635+ LogDir: environs.LogDir,
636+ CloudInitOutputLog: environs.CloudInitOutputLog,
637+ RsyslogConfPath: environs.RsyslogConfPath,
638+ Tools: &tools.Tools{URL: "file://" + c.MkDir()},
639 StateInfo: &state.Info{
640 CACert: []byte(testing.CACert),
641 Addrs: []string{"127.0.0.1:123"},
642@@ -44,7 +46,8 @@
643 Addrs: []string{"127.0.0.1:123"},
644 Tag: names.MachineTag(machineID),
645 },
646- SyslogPort: 2345,
647+ SyslogPort: 2345,
648+ MachineAgentServiceName: "jujud-machine-0",
649 }
650 }
651
652
653=== modified file 'provider/local/config.go'
654--- provider/local/config.go 2013-12-02 21:39:03 +0000
655+++ provider/local/config.go 2014-01-23 22:11:39 +0000
656@@ -11,7 +11,6 @@
657 "launchpad.net/juju-core/environs/config"
658 "launchpad.net/juju-core/instance"
659 "launchpad.net/juju-core/schema"
660- "launchpad.net/juju-core/utils"
661 )
662
663 var checkIfRoot = func() bool {
664@@ -26,6 +25,7 @@
665 "container": schema.String(),
666 "storage-port": schema.ForceInt(),
667 "shared-storage-port": schema.ForceInt(),
668+ "namespace": schema.String(),
669 }
670 // The port defaults below are not entirely arbitrary. Local user web
671 // frameworks often use 8000 or 8080, so I didn't want to use either of
672@@ -38,30 +38,20 @@
673 "bootstrap-ip": schema.Omit,
674 "storage-port": 8040,
675 "shared-storage-port": 8041,
676+ // namespace has a default of "", for backwards compatibility.
677+ "namespace": "",
678 }
679 )
680
681 type environConfig struct {
682 *config.Config
683- user string
684- attrs map[string]interface{}
685- runningAsRoot bool
686+ attrs map[string]interface{}
687 }
688
689 func newEnvironConfig(config *config.Config, attrs map[string]interface{}) *environConfig {
690- user := os.Getenv("USER")
691- root := checkIfRoot()
692- if root {
693- sudo_user := os.Getenv("SUDO_USER")
694- if sudo_user != "" {
695- user = sudo_user
696- }
697- }
698 return &environConfig{
699- Config: config,
700- user: user,
701- attrs: attrs,
702- runningAsRoot: root,
703+ Config: config,
704+ attrs: attrs,
705 }
706 }
707
708@@ -69,7 +59,7 @@
709 // have the same local provider name, we need to have a simple way to
710 // namespace the file locations, but more importantly the containers.
711 func (c *environConfig) namespace() string {
712- return fmt.Sprintf("%s-%s", c.user, c.Name())
713+ return c.attrs["namespace"].(string)
714 }
715
716 func (c *environConfig) rootDir() string {
717@@ -146,24 +136,5 @@
718 return err
719 }
720 }
721- if c.runningAsRoot {
722- // If we have SUDO_UID and SUDO_GID, start with rootDir(), and
723- // change ownership of the directories.
724- uid, gid, err := utils.SudoCallerIds()
725- if err != nil {
726- return err
727- }
728- if uid != 0 || gid != 0 {
729- filepath.Walk(c.rootDir(),
730- func(path string, info os.FileInfo, err error) error {
731- if info != nil && info.IsDir() {
732- if err := os.Chown(path, uid, gid); err != nil {
733- return err
734- }
735- }
736- return nil
737- })
738- }
739- }
740 return nil
741 }
742
743=== modified file 'provider/local/config_test.go'
744--- provider/local/config_test.go 2013-10-03 03:03:09 +0000
745+++ provider/local/config_test.go 2014-01-23 22:11:39 +0000
746@@ -26,7 +26,6 @@
747
748 func (s *configSuite) SetUpTest(c *gc.C) {
749 s.baseProviderSuite.SetUpTest(c)
750- s.PatchEnvironment("USER", "tester")
751 }
752
753 func minimalConfigValues() map[string]interface{} {
754@@ -108,28 +107,15 @@
755
756 func (s *configSuite) TestNamespace(c *gc.C) {
757 testConfig := minimalConfig(c)
758- c.Assert(local.ConfigNamespace(testConfig), gc.Equals, "tester-test")
759-}
760-
761-func (s *configSuite) TestNamespaceRootNoSudo(c *gc.C) {
762- restore := local.SetRootCheckFunction(func() bool { return true })
763- defer restore()
764- err := os.Setenv("USER", "root")
765- c.Assert(err, gc.IsNil)
766- testConfig := minimalConfig(c)
767- c.Assert(local.ConfigNamespace(testConfig), gc.Equals, "root-test")
768-}
769-
770-func (s *configSuite) TestNamespaceRootWithSudo(c *gc.C) {
771- restore := local.SetRootCheckFunction(func() bool { return true })
772- defer restore()
773- err := os.Setenv("USER", "root")
774- c.Assert(err, gc.IsNil)
775- err = os.Setenv("SUDO_USER", "tester")
776- c.Assert(err, gc.IsNil)
777- defer os.Setenv("SUDO_USER", "")
778- testConfig := minimalConfig(c)
779- c.Assert(local.ConfigNamespace(testConfig), gc.Equals, "tester-test")
780+ s.PatchEnvironment("USER", "tester")
781+ c.Assert(local.ConfigNamespace(testConfig), gc.Equals, "tester-test")
782+}
783+
784+func (s *configSuite) TestBootstrapAsRoot(c *gc.C) {
785+ restore := local.SetRootCheckFunction(func() bool { return true })
786+ defer restore()
787+ _, err := local.Provider.Prepare(minimalConfig(c))
788+ c.Assert(err, gc.ErrorMatches, "bootstrapping a local environment must not be done as root")
789 }
790
791 type configRootSuite struct {
792
793=== modified file 'provider/local/environ.go'
794--- provider/local/environ.go 2014-01-21 23:04:24 +0000
795+++ provider/local/environ.go 2014-01-23 22:11:39 +0000
796@@ -5,16 +5,16 @@
797
798 import (
799 "fmt"
800- "io/ioutil"
801 "net"
802- "net/url"
803 "os"
804+ "os/exec"
805 "path/filepath"
806+ "strings"
807 "sync"
808- "time"
809
810 "launchpad.net/juju-core/agent"
811- agenttools "launchpad.net/juju-core/agent/tools"
812+ coreCloudinit "launchpad.net/juju-core/cloudinit"
813+ "launchpad.net/juju-core/cloudinit/sshinit"
814 "launchpad.net/juju-core/constraints"
815 "launchpad.net/juju-core/container"
816 "launchpad.net/juju-core/container/factory"
817@@ -29,29 +29,18 @@
818 envtools "launchpad.net/juju-core/environs/tools"
819 "launchpad.net/juju-core/instance"
820 "launchpad.net/juju-core/juju/osenv"
821- "launchpad.net/juju-core/log/syslog"
822- "launchpad.net/juju-core/names"
823 "launchpad.net/juju-core/provider/common"
824 "launchpad.net/juju-core/state"
825 "launchpad.net/juju-core/state/api"
826 "launchpad.net/juju-core/tools"
827- "launchpad.net/juju-core/upstart"
828- "launchpad.net/juju-core/utils"
829 "launchpad.net/juju-core/version"
830+ "launchpad.net/juju-core/worker/terminationworker"
831 )
832
833 // boostrapInstanceId is just the name we give to the bootstrap machine.
834 // Using "localhost" because it is, and it makes sense.
835 const bootstrapInstanceId instance.Id = "localhost"
836
837-// upstartScriptLocation and syslogConfigDir are parameterised purely for
838-// testing purposes as we don't really want to be installing and starting
839-// scripts as root for testing.
840-var (
841- upstartScriptLocation = "/etc/init"
842- syslogConfigDir = "/etc/rsyslog.d"
843-)
844-
845 // localEnviron implements Environ.
846 var _ environs.Environ = (*localEnviron)(nil)
847
848@@ -87,8 +76,8 @@
849 return "juju-agent-" + env.config.namespace()
850 }
851
852-func (env *localEnviron) syslogFilename() string {
853- return fmt.Sprintf("25-juju-%s.conf", env.config.namespace())
854+func (env *localEnviron) rsyslogConfPath() string {
855+ return fmt.Sprintf("/etc/rsyslog.d/25-juju-%s.conf", env.config.namespace())
856 }
857
858 // PrecheckInstance is specified in the environs.Prechecker interface.
859@@ -103,29 +92,29 @@
860 return environs.NewContainersUnsupported("local provider does not support nested containers")
861 }
862
863+func ensureNotRoot() error {
864+ if checkIfRoot() {
865+ return fmt.Errorf("bootstrapping a local environment must not be done as root")
866+ }
867+ return nil
868+}
869+
870 // Bootstrap is specified in the Environ interface.
871 func (env *localEnviron) Bootstrap(ctx environs.BootstrapContext, cons constraints.Value) error {
872- if !env.config.runningAsRoot {
873- return fmt.Errorf("bootstrapping a local environment must be done as root")
874- }
875- if err := env.config.createDirs(); err != nil {
876- logger.Errorf("failed to create necessary directories: %v", err)
877+ if err := ensureNotRoot(); err != nil {
878 return err
879 }
880-
881- // TODO(thumper): check that the constraints don't include "container=lxc" for now.
882 privateKey, err := common.GenerateSystemSSHKey(env)
883 if err != nil {
884 return err
885 }
886
887- cert, key, err := env.setupLocalMongoService()
888- if err != nil {
889- return err
890- }
891-
892 // Before we write the agent config file, we need to make sure the
893 // instance is saved in the StateInfo.
894+ stateFileURL, err := bootstrap.CreateStateFile(env.Storage())
895+ if err != nil {
896+ return err
897+ }
898 if err := bootstrap.SaveState(env.Storage(), &bootstrap.BootstrapState{
899 StateInstances: []instance.Id{bootstrapInstanceId},
900 }); err != nil {
901@@ -139,23 +128,49 @@
902 return err
903 }
904
905- if err := env.configureLocalSyslog(); err != nil {
906- return err
907- }
908+ mcfg := environs.NewBootstrapMachineConfig(stateFileURL, privateKey)
909+ mcfg.Tools = selectedTools[0]
910+ mcfg.DataDir = env.config.rootDir()
911+ mcfg.LogDir = env.config.logDir()
912+ mcfg.RsyslogConfPath = env.rsyslogConfPath()
913+ mcfg.CloudInitOutputLog = filepath.Join(mcfg.LogDir, "cloud-init-output.log")
914+ mcfg.DisablePackageCommands = true
915+ mcfg.MachineAgentServiceName = env.machineAgentServiceName()
916+ mcfg.MongoServiceName = env.mongoServiceName()
917+ mcfg.AgentEnvironment = map[string]string{
918+ agent.Namespace: env.config.namespace(),
919+ agent.StorageDir: env.config.storageDir(),
920+ agent.StorageAddr: env.config.storageAddr(),
921+ agent.SharedStorageDir: env.config.sharedStorageDir(),
922+ agent.SharedStorageAddr: env.config.sharedStorageAddr(),
923+ }
924+ if err := environs.FinishMachineConfig(mcfg, env.Config(), cons); err != nil {
925+ return err
926+ }
927+ // don't write proxy settings for local machine
928+ mcfg.AptProxySettings = osenv.ProxySettings{}
929+ mcfg.ProxySettings = osenv.ProxySettings{}
930+ cloudcfg := coreCloudinit.New()
931+ if err := cloudinit.ConfigureJuju(mcfg, cloudcfg); err != nil {
932+ return err
933+ }
934+ return finishBootstrap(mcfg, cloudcfg, ctx)
935+}
936
937- // Need to write out the agent file for machine-0 before initializing
938- // state, as as part of that process, it will reset the password in the
939- // agent file.
940- agentConfig, err := env.writeBootstrapAgentConfFile(env.config.AdminSecret(), cert, key)
941+// finishBootstrap converts the machine config to cloud-config,
942+// converts that to a script, and then executes it locally.
943+//
944+// mcfg is supplied for testing purposes.
945+var finishBootstrap = func(mcfg *cloudinit.MachineConfig, cloudcfg *coreCloudinit.Config, ctx environs.BootstrapContext) error {
946+ script, err := sshinit.ConfigureScript(cloudcfg)
947 if err != nil {
948- return err
949- }
950-
951- if err := env.initializeState(agentConfig, cons); err != nil {
952- return err
953- }
954-
955- return env.setupLocalMachineAgent(cons, selectedTools, privateKey)
956+ return nil
957+ }
958+ cmd := exec.Command("sudo", "/bin/bash", "-s")
959+ cmd.Stdin = strings.NewReader(script)
960+ cmd.Stdout = ctx.Stdout()
961+ cmd.Stderr = ctx.Stderr()
962+ return cmd.Run()
963 }
964
965 // StateInfo is specified in the Environ interface.
966@@ -212,6 +227,9 @@
967 if ecfg.bootstrapped() {
968 return nil
969 }
970+ if err := ensureNotRoot(); err != nil {
971+ return err
972+ }
973 return env.bootstrapAddressAndStorage(cfg)
974 }
975
976@@ -233,7 +251,7 @@
977 return err
978 }
979 networkBridge := config.networkBridge()
980- bridgeAddress, err := env.findBridgeAddress(networkBridge)
981+ bridgeAddress, err := getAddressForInterface(networkBridge)
982 if err != nil {
983 logger.Infof("configure a different bridge using 'network-bridge' in the config file")
984 return fmt.Errorf("cannot find address of network-bridge: %q", networkBridge)
985@@ -380,46 +398,34 @@
986
987 // Destroy is specified in the Environ interface.
988 func (env *localEnviron) Destroy() error {
989- if !env.config.runningAsRoot {
990- return fmt.Errorf("destroying a local environment must be done as root")
991- }
992- // Kill all running instances.
993- containers, err := env.containerManager.ListContainers()
994- if err != nil {
995- return err
996- }
997- for _, inst := range containers {
998- if err := env.containerManager.StopContainer(inst); err != nil {
999- return err
1000- }
1001- }
1002-
1003- logger.Infof("removing service %s", env.machineAgentServiceName())
1004- machineAgent := upstart.NewService(env.machineAgentServiceName())
1005- machineAgent.InitDir = upstartScriptLocation
1006- if err := machineAgent.StopAndRemove(); err != nil {
1007- logger.Errorf("could not remove machine agent service: %v", err)
1008- return err
1009- }
1010-
1011- logger.Infof("removing service %s", env.mongoServiceName())
1012- mongo := upstart.NewService(env.mongoServiceName())
1013- mongo.InitDir = upstartScriptLocation
1014- if err := mongo.StopAndRemove(); err != nil {
1015- logger.Errorf("could not remove mongo service: %v", err)
1016- return err
1017- }
1018-
1019- env.removeLocalSyslog()
1020-
1021- // Remove the rootdir.
1022- logger.Infof("removing state dir %s", env.config.rootDir())
1023- if err := os.RemoveAll(env.config.rootDir()); err != nil {
1024- logger.Errorf("could not remove local state dir: %v", err)
1025- return err
1026- }
1027-
1028- return nil
1029+ // Kill all running instances. This must be done as
1030+ // root, or listing/stopping containers will fail.
1031+ if !checkIfRoot() {
1032+ juju, err := exec.LookPath(os.Args[0])
1033+ if err != nil {
1034+ return err
1035+ }
1036+ cmd := exec.Command("sudo", append([]string{juju}, os.Args[1:]...)...)
1037+ cmd.Stdout = os.Stdout
1038+ cmd.Stderr = os.Stderr
1039+ return cmd.Run()
1040+ } else {
1041+ containers, err := env.containerManager.ListContainers()
1042+ if err != nil {
1043+ return err
1044+ }
1045+ for _, inst := range containers {
1046+ if err := env.containerManager.StopContainer(inst); err != nil {
1047+ return err
1048+ }
1049+ }
1050+ cmd := exec.Command(
1051+ "pkill",
1052+ fmt.Sprintf("-%d", terminationworker.TerminationSignal),
1053+ "jujud",
1054+ )
1055+ return cmd.Run()
1056+ }
1057 }
1058
1059 // OpenPorts is specified in the Environ interface.
1060@@ -441,218 +447,3 @@
1061 func (env *localEnviron) Provider() environs.EnvironProvider {
1062 return providerInstance
1063 }
1064-
1065-// setupLocalMongoService returns the cert and key if there was no error.
1066-func (env *localEnviron) setupLocalMongoService() ([]byte, []byte, error) {
1067- journalDir := filepath.Join(env.config.mongoDir(), "journal")
1068- logger.Debugf("create mongo journal dir: %v", journalDir)
1069- if err := os.MkdirAll(journalDir, 0755); err != nil {
1070- logger.Errorf("failed to make mongo journal dir %s: %v", journalDir, err)
1071- return nil, nil, err
1072- }
1073-
1074- logger.Debugf("generate server cert")
1075- cert, key, err := env.config.GenerateStateServerCertAndKey()
1076- if err != nil {
1077- logger.Errorf("failed to generate server cert: %v", err)
1078- return nil, nil, err
1079- }
1080- if err := ioutil.WriteFile(
1081- env.config.configFile("server.pem"),
1082- append(cert, key...),
1083- 0600); err != nil {
1084- logger.Errorf("failed to write server.pem: %v", err)
1085- return nil, nil, err
1086- }
1087-
1088- mongo := upstart.MongoUpstartService(
1089- env.mongoServiceName(),
1090- env.config.rootDir(),
1091- env.config.mongoDir(),
1092- env.config.StatePort())
1093- mongo.InitDir = upstartScriptLocation
1094- logger.Infof("installing service %s to %s", env.mongoServiceName(), mongo.InitDir)
1095- if err := mongo.Install(); err != nil {
1096- logger.Errorf("could not install mongo service: %v", err)
1097- return nil, nil, err
1098- }
1099- return cert, key, nil
1100-}
1101-
1102-func (env *localEnviron) setupLocalMachineAgent(cons constraints.Value, possibleTools tools.List, privateKey string) error {
1103- dataDir := env.config.rootDir()
1104- // unpack the first tools into the agent dir.
1105- agentTools := possibleTools[0]
1106- logger.Debugf("tools: %#v", agentTools)
1107- // save the system identity file
1108- systemIdentityFilename := filepath.Join(dataDir, cloudinit.SystemIdentity)
1109- logger.Debugf("writing system identity to %s", systemIdentityFilename)
1110- if err := ioutil.WriteFile(systemIdentityFilename, []byte(privateKey), 0600); err != nil {
1111- return fmt.Errorf("failed to write system identity: %v", err)
1112- }
1113-
1114- // brutally abuse our knowledge of storage to directly open the file
1115- toolsUrl, err := url.Parse(agentTools.URL)
1116- if err != nil {
1117- return err
1118- }
1119- toolsLocation := filepath.Join(env.config.storageDir(), toolsUrl.Path)
1120- logger.Infof("tools location: %v", toolsLocation)
1121- toolsFile, err := os.Open(toolsLocation)
1122- defer toolsFile.Close()
1123- // Again, brutally abuse our knowledge here.
1124-
1125- // The tools that possible bootstrap tools are based on the
1126- // default series in the config. However we are running potentially on a
1127- // different series. When the machine agent is started, it will be
1128- // looking based on the current series, so we need to override the series
1129- // returned in the tools to be the current series.
1130- agentTools.Version.Series = version.Current.Series
1131- err = agenttools.UnpackTools(dataDir, agentTools, toolsFile)
1132-
1133- machineId := "0" // Always machine 0
1134- tag := names.MachineTag(machineId)
1135-
1136- // make sure we create the symlink so we have it for the upstart config to use
1137- if _, err := agenttools.ChangeAgentTools(dataDir, tag, agentTools.Version); err != nil {
1138- logger.Errorf("could not create tools directory symlink: %v", err)
1139- return err
1140- }
1141-
1142- toolsDir := agenttools.ToolsDir(dataDir, tag)
1143-
1144- logDir := env.config.logDir()
1145- machineEnvironment := map[string]string{
1146- "USER": env.config.user,
1147- "HOME": osenv.Home(),
1148- }
1149- agentService := upstart.MachineAgentUpstartService(
1150- env.machineAgentServiceName(),
1151- toolsDir, dataDir, logDir, tag, machineId, machineEnvironment)
1152-
1153- agentService.InitDir = upstartScriptLocation
1154- logger.Infof("installing service %s to %s", env.machineAgentServiceName(), agentService.InitDir)
1155- if err := agentService.Install(); err != nil {
1156- logger.Errorf("could not install machine agent service: %v", err)
1157- return err
1158- }
1159- return nil
1160-}
1161-
1162-func (env *localEnviron) findBridgeAddress(networkBridge string) (string, error) {
1163- return getAddressForInterface(networkBridge)
1164-}
1165-
1166-func (env *localEnviron) writeBootstrapAgentConfFile(secret string, cert, key []byte) (agent.Config, error) {
1167- tag := names.MachineTag("0")
1168- passwordHash := utils.UserPasswordHash(secret, utils.CompatSalt)
1169- // We don't check the existance of the CACert here as if it wasn't set, we
1170- // wouldn't get this far.
1171- cfg := env.config.Config
1172- caCert, _ := cfg.CACert()
1173- agentValues := map[string]string{
1174- agent.ProviderType: env.config.Type(),
1175- agent.Namespace: env.config.namespace(),
1176- agent.StorageDir: env.config.storageDir(),
1177- agent.StorageAddr: env.config.storageAddr(),
1178- agent.SharedStorageDir: env.config.sharedStorageDir(),
1179- agent.SharedStorageAddr: env.config.sharedStorageAddr(),
1180- agent.AgentServiceName: env.machineAgentServiceName(),
1181- agent.MongoServiceName: env.mongoServiceName(),
1182- }
1183- // NOTE: the state address HAS to be localhost, otherwise the mongo
1184- // initialization fails. There is some magic code somewhere in the mongo
1185- // connection code that treats connections from localhost as special, and
1186- // will raise unauthorized errors during the initialization if the caller
1187- // is not connected from localhost.
1188- stateAddress := fmt.Sprintf("localhost:%d", cfg.StatePort())
1189- apiAddress := fmt.Sprintf("localhost:%d", cfg.APIPort())
1190- config, err := agent.NewStateMachineConfig(
1191- agent.StateMachineConfigParams{
1192- AgentConfigParams: agent.AgentConfigParams{
1193- DataDir: env.config.rootDir(),
1194- Tag: tag,
1195- Password: passwordHash,
1196- Nonce: state.BootstrapNonce,
1197- StateAddresses: []string{stateAddress},
1198- APIAddresses: []string{apiAddress},
1199- CACert: caCert,
1200- Values: agentValues,
1201- },
1202- StateServerCert: cert,
1203- StateServerKey: key,
1204- StatePort: cfg.StatePort(),
1205- APIPort: cfg.APIPort(),
1206- })
1207- if err != nil {
1208- return nil, err
1209- }
1210- if err := config.Write(); err != nil {
1211- logger.Errorf("failed to write bootstrap agent file: %v", err)
1212- return nil, err
1213- }
1214- return config, nil
1215-}
1216-
1217-func (env *localEnviron) initializeState(agentConfig agent.Config, cons constraints.Value) error {
1218- bootstrapCfg, err := environs.BootstrapConfig(env.config.Config)
1219- if err != nil {
1220- return err
1221- }
1222- st, m, err := agentConfig.InitializeState(bootstrapCfg, agent.BootstrapMachineConfig{
1223- Constraints: cons,
1224- Jobs: []state.MachineJob{
1225- state.JobManageEnviron,
1226- state.JobManageState,
1227- },
1228- InstanceId: bootstrapInstanceId,
1229- }, state.DialOpts{
1230- Timeout: 60 * time.Second,
1231- })
1232- if err != nil {
1233- return err
1234- }
1235- defer st.Close()
1236- addr, err := env.findBridgeAddress(env.config.networkBridge())
1237- if err != nil {
1238- return fmt.Errorf("failed to get bridge address: %v", err)
1239- }
1240- err = m.SetAddresses([]instance.Address{{
1241- NetworkScope: instance.NetworkPublic,
1242- Type: instance.HostName,
1243- Value: "localhost",
1244- }, {
1245- NetworkScope: instance.NetworkCloudLocal,
1246- Type: instance.Ipv4Address,
1247- Value: addr,
1248- }})
1249- if err != nil {
1250- return fmt.Errorf("cannot set addresses on bootstrap instance: %v", err)
1251- }
1252- return nil
1253-}
1254-
1255-func (env *localEnviron) configureLocalSyslog() error {
1256- tag := names.MachineTag("0")
1257- syslogConfigRenderer := syslog.NewAccumulateConfig(tag, env.config.SyslogPort(), env.config.namespace())
1258- syslogConfigRenderer.ConfigDir = syslogConfigDir
1259- syslogConfigRenderer.ConfigFileName = env.syslogFilename()
1260- syslogConfigRenderer.LogDir = env.config.logDir()
1261- if err := syslogConfigRenderer.Write(); err != nil {
1262- return err
1263- }
1264- if err := syslog.Restart(); err != nil {
1265- logger.Warningf("cannot restart syslog daemon: %v", err)
1266- }
1267- return nil
1268-}
1269-
1270-func (env *localEnviron) removeLocalSyslog() {
1271- // Don't fail if we have issues, but warn the user.
1272- if err := os.Remove(filepath.Join(syslogConfigDir, env.syslogFilename())); err != nil {
1273- logger.Warningf("could not remove local syslog config: %v", err)
1274- }
1275- if err := syslog.Restart(); err != nil {
1276- logger.Warningf("cannot restart syslog daemon: %v", err)
1277- }
1278-}
1279
1280=== modified file 'provider/local/environ_test.go'
1281--- provider/local/environ_test.go 2013-10-24 00:20:59 +0000
1282+++ provider/local/environ_test.go 2014-01-23 22:11:39 +0000
1283@@ -11,14 +11,17 @@
1284
1285 gc "launchpad.net/gocheck"
1286
1287+ coreCloudinit "launchpad.net/juju-core/cloudinit"
1288 "launchpad.net/juju-core/constraints"
1289 "launchpad.net/juju-core/environs"
1290+ "launchpad.net/juju-core/environs/cloudinit"
1291 "launchpad.net/juju-core/environs/config"
1292 "launchpad.net/juju-core/environs/jujutest"
1293 envtesting "launchpad.net/juju-core/environs/testing"
1294 "launchpad.net/juju-core/environs/tools"
1295 "launchpad.net/juju-core/instance"
1296 "launchpad.net/juju-core/provider/local"
1297+ coretesting "launchpad.net/juju-core/testing"
1298 jc "launchpad.net/juju-core/testing/checkers"
1299 )
1300
1301@@ -96,14 +99,13 @@
1302 // Construct the directories first.
1303 err := local.CreateDirs(c, minimalConfig(c))
1304 c.Assert(err, gc.IsNil)
1305- s.oldUpstartLocation = local.SetUpstartScriptLocation(c.MkDir())
1306 s.oldPath = os.Getenv("PATH")
1307 s.testPath = c.MkDir()
1308 os.Setenv("PATH", s.testPath+":"+s.oldPath)
1309
1310 // Add in an admin secret
1311 s.Tests.TestConfig["admin-secret"] = "sekrit"
1312- s.restoreRootCheck = local.SetRootCheckFunction(func() bool { return true })
1313+ s.restoreRootCheck = local.SetRootCheckFunction(func() bool { return false })
1314 s.Tests.SetUpTest(c)
1315
1316 cfg, err := config.New(config.NoDefaults, s.TestConfig)
1317@@ -115,7 +117,6 @@
1318 s.Tests.TearDownTest(c)
1319 os.Setenv("PATH", s.oldPath)
1320 s.restoreRootCheck()
1321- local.SetUpstartScriptLocation(s.oldUpstartLocation)
1322 s.baseProviderSuite.TearDownTest(c)
1323 }
1324
1325@@ -141,7 +142,20 @@
1326 })
1327
1328 func (s *localJujuTestSuite) TestBootstrap(c *gc.C) {
1329- c.Skip("Cannot test bootstrap at this stage.")
1330+ s.PatchValue(local.FinishBootstrap, func(mcfg *cloudinit.MachineConfig, cloudcfg *coreCloudinit.Config, ctx environs.BootstrapContext) error {
1331+ c.Assert(cloudcfg.AptUpdate(), jc.IsFalse)
1332+ c.Assert(cloudcfg.AptUpgrade(), jc.IsFalse)
1333+ c.Assert(cloudcfg.Packages(), gc.HasLen, 0)
1334+ return nil
1335+ })
1336+ testConfig := minimalConfig(c)
1337+ environ, err := local.Provider.Prepare(testConfig)
1338+ c.Assert(err, gc.IsNil)
1339+ envtesting.UploadFakeTools(c, environ.Storage())
1340+ defer environ.Storage().RemoveAll()
1341+ ctx := envtesting.NewBootstrapContext(coretesting.Context(c))
1342+ err = environ.Bootstrap(ctx, constraints.Value{})
1343+ c.Assert(err, gc.IsNil)
1344 }
1345
1346 func (s *localJujuTestSuite) TestStartStop(c *gc.C) {
1347
1348=== modified file 'provider/local/environprovider.go'
1349--- provider/local/environprovider.go 2014-01-22 22:48:54 +0000
1350+++ provider/local/environprovider.go 2014-01-23 22:11:39 +0000
1351@@ -6,6 +6,7 @@
1352 import (
1353 "fmt"
1354 "net"
1355+ "os"
1356 "syscall"
1357
1358 "launchpad.net/loggo"
1359@@ -43,6 +44,17 @@
1360 }
1361 cfg = newCfg
1362 }
1363+ // Set the "namespace" attribute. We do this here, and not in Prepare,
1364+ // for backwards compatibility: older versions did not store the namespace
1365+ // in config.
1366+ if namespace := cfg.UnknownAttrs()["namespace"]; namespace == "" {
1367+ var err error
1368+ namespace = fmt.Sprintf("%s-%s", os.Getenv("USER"), cfg.Name())
1369+ cfg, err = cfg.Apply(map[string]interface{}{"namespace": namespace})
1370+ if err != nil {
1371+ return nil, fmt.Errorf("failed to create namespace: %v", err)
1372+ }
1373+ }
1374 // Do the initial validation on the config.
1375 localConfig, err := providerInstance.newConfig(cfg)
1376 if err != nil {
1377
1378=== modified file 'provider/local/export_test.go'
1379--- provider/local/export_test.go 2013-10-14 14:47:52 +0000
1380+++ provider/local/export_test.go 2014-01-23 22:11:39 +0000
1381@@ -10,7 +10,8 @@
1382 )
1383
1384 var (
1385- Provider = providerInstance
1386+ Provider = providerInstance
1387+ FinishBootstrap = &finishBootstrap
1388 )
1389
1390 // SetRootCheckFunction allows tests to override the check for a root user.
1391@@ -21,25 +22,18 @@
1392 return func() { checkIfRoot = old }
1393 }
1394
1395-// SetUpstartScriptLocation allows tests to override the directory where the
1396-// provider writes the upstart scripts.
1397-func SetUpstartScriptLocation(location string) (old string) {
1398- old, upstartScriptLocation = upstartScriptLocation, location
1399- return
1400-}
1401-
1402 // ConfigNamespace returns the result of the namespace call on the
1403 // localConfig.
1404 func ConfigNamespace(cfg *config.Config) string {
1405- localConfig, _ := providerInstance.newConfig(cfg)
1406- return localConfig.namespace()
1407+ env, _ := providerInstance.Open(cfg)
1408+ return env.(*localEnviron).config.namespace()
1409 }
1410
1411 // CreateDirs calls createDirs on the localEnviron.
1412 func CreateDirs(c *gc.C, cfg *config.Config) error {
1413- localConfig, err := providerInstance.newConfig(cfg)
1414+ env, err := providerInstance.Open(cfg)
1415 c.Assert(err, gc.IsNil)
1416- return localConfig.createDirs()
1417+ return env.(*localEnviron).config.createDirs()
1418 }
1419
1420 // CheckDirs returns the list of directories to check for permissions in the test.
1421
1422=== modified file 'provider/local/instance.go'
1423--- provider/local/instance.go 2013-12-16 09:53:26 +0000
1424+++ provider/local/instance.go 2014-01-23 22:11:39 +0000
1425@@ -33,19 +33,25 @@
1426 }
1427
1428 func (inst *localInstance) Addresses() ([]instance.Address, error) {
1429+ if inst.id == bootstrapInstanceId {
1430+ addrs := []instance.Address{{
1431+ NetworkScope: instance.NetworkPublic,
1432+ Type: instance.HostName,
1433+ Value: "localhost",
1434+ }, {
1435+ NetworkScope: instance.NetworkCloudLocal,
1436+ Type: instance.Ipv4Address,
1437+ Value: inst.env.config.bootstrapIPAddress(),
1438+ }}
1439+ return addrs, nil
1440+ }
1441 return nil, errors.NewNotImplementedError("localInstance.Addresses")
1442 }
1443
1444 // DNSName implements instance.Instance.DNSName.
1445 func (inst *localInstance) DNSName() (string, error) {
1446- if string(inst.id) == "localhost" {
1447- // get the bridge address from the environment
1448- addr, err := inst.env.findBridgeAddress(inst.env.config.networkBridge())
1449- if err != nil {
1450- logger.Errorf("failed to get bridge address: %v", err)
1451- return "", instance.ErrNoDNSName
1452- }
1453- return addr, nil
1454+ if inst.id == bootstrapInstanceId {
1455+ return inst.env.config.bootstrapIPAddress(), nil
1456 }
1457 // Get the IPv4 address from eth0
1458 return getAddressForInterface("eth0")

Subscribers

People subscribed via source and target branches

to status/vote changes: