Merge lp:~natefinch/juju-core/026-EnsureMongoServer into lp:~go-bot/juju-core/trunk
- 026-EnsureMongoServer
- Merge into trunk
Status: | Work in progress |
---|---|
Proposed branch: | lp:~natefinch/juju-core/026-EnsureMongoServer |
Merge into: | lp:~go-bot/juju-core/trunk |
Diff against target: |
756 lines (+448/-26) (has conflicts) 10 files modified
agent/agent.go (+10/-0) cmd/jujud/machine.go (+136/-14) cmd/jujud/machine_test.go (+9/-0) environs/cloudinit/cloudinit.go (+16/-3) provider/local/config.go (+0/-5) provider/local/environ.go (+210/-0) replicaset/replicaset.go (+4/-0) testing/mockupstart.go (+42/-0) upstart/service.go (+13/-2) upstart/upstart.go (+8/-2) Text conflict in cmd/jujud/machine.go Text conflict in cmd/jujud/machine_test.go Text conflict in environs/cloudinit/cloudinit.go Text conflict in provider/local/environ.go Text conflict in replicaset/replicaset.go |
To merge this branch: | bzr merge lp:~natefinch/juju-core/026-EnsureMongoServer |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju Engineering | Pending | ||
Review via email: mp+201961@code.launchpad.net |
Commit message
Description of the change
EnsureMongoServer function for MachineAgent
Implement a method to allow the machine agent to create & run the upstart script for mongo.
Nate Finch (natefinch) wrote : | # |
Reviewers: mp+201961_
Message:
Some more clean up, and now add the "ensure" part of ensure mongo
server. Also consolidated the code so we only do this in machine agent.
Also, just ignore the stuff in replicaset.go, it's not really part of
this change and I'll remove it.
Description:
EnxureMongoServer function for MachineAgent
IMplement a method to allow the machine agent to create & run the
upstart script for mongo.
https:/
(do not edit description out of merge proposal)
Please review this at https:/
Affected files (+130, -58 lines):
A [revision details]
M agent/agent.go
cmd/
M environs/
M provider/
M provider/
M replicaset/
M replicaset/
M upstart/service.go
Nate Finch (natefinch) wrote : | # |
Please take a look.
William Reade (fwereade) wrote : | # |
couple of quick comments, I think I may have misunderstood the StatePort
bits, can we chat when you're free?
https:/
File agent/agent.go (right):
https:/
agent/agent.go:99: StatePort() int
I'm a bit bothered about storing this in agent config. And actually
about storing *any* of this stuff in agent config -- we have manual
bootstrap now, and I don't see why this is needed any more.
https:/
File provider/
https:/
provider/
filepath.
"db", "journal"?
https:/
File upstart/service.go (right):
https:/
upstart/
port int) *Conf {
Hmm, I'm really not sure this is the right package for this file. Isn't
this stuff all specific to instance config?
- 2185. By Nate Finch
-
Add code to mock out upstart so we can successfully run the new machine agent code during tests. Also added code to only run EnsureMongoServer when the machine agent has the ManageEnviron job
- 2186. By Nate Finch
-
fix a couple build errors
- 2187. By Nate Finch
-
make sure the bootstrap node doesn't try to connect to the API to get its state
- 2188. By Nate Finch
-
I should learn to hit save before commiting
Unmerged revisions
- 2188. By Nate Finch
-
I should learn to hit save before commiting
- 2187. By Nate Finch
-
make sure the bootstrap node doesn't try to connect to the API to get its state
- 2186. By Nate Finch
-
fix a couple build errors
- 2185. By Nate Finch
-
Add code to mock out upstart so we can successfully run the new machine agent code during tests. Also added code to only run EnsureMongoServer when the machine agent has the ManageEnviron job
- 2184. By Nate Finch
-
add the ensure part of ensuremongoserver, also remove the old upstart stuff so we always run it from machine agent
- 2183. By Nate Finch
-
twiddle the comment to help kick rietveld
- 2182. By Nate Finch
-
line got deleted somehow
- 2181. By Nate Finch
-
create ensuremongoserver function to create/rewrite the mongo upstart script from the machine agent
- 2180. By Nate Finch
-
Let's spell replSet correctly, eh?
- 2179. By Nate Finch
-
uncomment some tests, change mongo upstart to always set mongo in replicaset mode
Preview Diff
1 | === modified file 'agent/agent.go' |
2 | --- agent/agent.go 2014-01-23 05:21:21 +0000 |
3 | +++ agent/agent.go 2014-01-28 15:13:44 +0000 |
4 | @@ -95,6 +95,10 @@ |
5 | // SetValue updates the value for the specified key. |
6 | SetValue(key, value string) |
7 | |
8 | + // StatePort specifies the TCP port that will be used |
9 | + // by the MongoDB server if one is running on this machine. |
10 | + StatePort() int |
11 | + |
12 | StateInitializer |
13 | } |
14 | |
15 | @@ -133,6 +137,7 @@ |
16 | stateServerKey []byte |
17 | apiPort int |
18 | values map[string]string |
19 | + statePort int |
20 | } |
21 | |
22 | type AgentConfigParams struct { |
23 | @@ -215,6 +220,7 @@ |
24 | config.stateServerCert = params.StateServerCert |
25 | config.stateServerKey = params.StateServerKey |
26 | config.apiPort = params.APIPort |
27 | + config.statePort = params.StatePort |
28 | return config, nil |
29 | } |
30 | |
31 | @@ -321,6 +327,10 @@ |
32 | return Dir(c.dataDir, c.tag) |
33 | } |
34 | |
35 | +func (c *configInternal) StatePort() int { |
36 | + return c.statePort |
37 | +} |
38 | + |
39 | func (c *configInternal) check() error { |
40 | if c.stateDetails == nil && c.apiDetails == nil { |
41 | return requiredError("state or API addresses") |
42 | |
43 | === modified file 'cmd/jujud/machine.go' |
44 | --- cmd/jujud/machine.go 2014-01-24 14:52:58 +0000 |
45 | +++ cmd/jujud/machine.go 2014-01-28 15:13:44 +0000 |
46 | @@ -5,6 +5,7 @@ |
47 | |
48 | import ( |
49 | "fmt" |
50 | + "io/ioutil" |
51 | "os" |
52 | "path/filepath" |
53 | "time" |
54 | @@ -45,15 +46,23 @@ |
55 | "launchpad.net/juju-core/worker/upgrader" |
56 | ) |
57 | |
58 | -var logger = loggo.GetLogger("juju.cmd.jujud") |
59 | - |
60 | -var newRunner = func(isFatal func(error) bool, moreImportant func(e0, e1 error) bool) worker.Runner { |
61 | - return worker.NewRunner(isFatal, moreImportant) |
62 | -} |
63 | - |
64 | -const bootstrapMachineId = "0" |
65 | - |
66 | -var retryDelay = 3 * time.Second |
67 | +const ( |
68 | + mongoSvcFmt = "juju-db-v%d" |
69 | + bootstrapMachineId = "0" |
70 | + oldMongoServiceName = "juju-db" |
71 | +) |
72 | + |
73 | +var ( |
74 | + logger = loggo.GetLogger("juju.cmd.jujud") |
75 | + |
76 | + newRunner = func(isFatal func(error) bool, moreImportant func(e0, e1 error) bool) worker.Runner { |
77 | + return worker.NewRunner(isFatal, moreImportant) |
78 | + } |
79 | + |
80 | + mongoServiceName = fmt.Sprintf(mongoSvcFmt, upstart.MongoScriptVersion) |
81 | + |
82 | + retryDelay = 3 * time.Second |
83 | +) |
84 | |
85 | var jujuRun = "/usr/local/bin/juju-run" |
86 | |
87 | @@ -119,6 +128,16 @@ |
88 | return err |
89 | } |
90 | |
91 | + mgr, err := a.isEnvironManager() |
92 | + if err != nil { |
93 | + return err |
94 | + } |
95 | + if mgr { |
96 | + if err := a.ensureMongoServer(); err != nil { |
97 | + return err |
98 | + } |
99 | + } |
100 | + |
101 | // ensureStateWorker ensures that there is a worker that |
102 | // connects to the state that runs within itself all the workers |
103 | // that need a state connection. Unless we're bootstrapping, we |
104 | @@ -144,10 +163,14 @@ |
105 | a.runner.StartWorker("api", func() (worker.Worker, error) { |
106 | return a.APIWorker(ensureStateWorker) |
107 | }) |
108 | +<<<<<<< TREE |
109 | a.runner.StartWorker("termination", func() (worker.Worker, error) { |
110 | return terminationworker.NewWorker(), nil |
111 | }) |
112 | err := a.runner.Wait() |
113 | +======= |
114 | + err = a.runner.Wait() |
115 | +>>>>>>> MERGE-SOURCE |
116 | if err == worker.ErrTerminateAgent { |
117 | err = a.uninstallAgent() |
118 | } |
119 | @@ -361,6 +384,7 @@ |
120 | return names.MachineTag(a.MachineId) |
121 | } |
122 | |
123 | +<<<<<<< TREE |
124 | func (a *MachineAgent) initAgent() error { |
125 | if err := os.Remove(jujuRun); err != nil && !os.IsNotExist(err) { |
126 | return err |
127 | @@ -369,6 +393,29 @@ |
128 | return os.Symlink(jujud, jujuRun) |
129 | } |
130 | |
131 | +======= |
132 | +// isEnvironManager returns if the current agent has the ManageEnviron job. |
133 | +func (a *MachineAgent) isEnvironManager() (bool, error) { |
134 | + if a.MachineId == bootstrapMachineId { |
135 | + return true, nil |
136 | + } |
137 | + |
138 | + st, entity, err := openAPIState(a.Conf.config, a) |
139 | + if err != nil { |
140 | + return false, err |
141 | + } |
142 | + defer st.Close() |
143 | + reportOpenedAPI(st) |
144 | + |
145 | + for _, job := range entity.Jobs() { |
146 | + if job == params.JobManageEnviron { |
147 | + return true, nil |
148 | + } |
149 | + } |
150 | + return false, nil |
151 | +} |
152 | + |
153 | +>>>>>>> MERGE-SOURCE |
154 | func (a *MachineAgent) uninstallAgent() error { |
155 | var errors []error |
156 | agentServiceName := a.Conf.config.Value(agent.AgentServiceName) |
157 | @@ -381,6 +428,7 @@ |
158 | errors = append(errors, fmt.Errorf("cannot remove service %q: %v", agentServiceName, err)) |
159 | } |
160 | } |
161 | +<<<<<<< TREE |
162 | // Remove the rsyslog conf file and restart rsyslogd. |
163 | if rsyslogConfPath := a.Conf.config.Value(agent.RsyslogConfPath); rsyslogConfPath != "" { |
164 | if err := os.Remove(rsyslogConfPath); err != nil { |
165 | @@ -394,15 +442,16 @@ |
166 | if err := os.Remove(jujuRun); err != nil && !os.IsNotExist(err) { |
167 | errors = append(errors, err) |
168 | } |
169 | +======= |
170 | + |
171 | +>>>>>>> MERGE-SOURCE |
172 | // The machine agent may terminate without knowing its jobs, |
173 | // for example if the machine's entry in state was removed. |
174 | // Thus, we do not rely on jobs here, and instead just check |
175 | // if the upstart config exists. |
176 | - mongoServiceName := a.Conf.config.Value(agent.MongoServiceName) |
177 | - if mongoServiceName != "" { |
178 | - if err := upstart.NewService(mongoServiceName).StopAndRemove(); err != nil { |
179 | - errors = append(errors, fmt.Errorf("cannot stop/remove service %q: %v", mongoServiceName, err)) |
180 | - } |
181 | + mongo := a.mongoService() |
182 | + if err := mongo.StopAndRemove(); err != nil { |
183 | + errors = append(errors, fmt.Errorf("cannot stop/remove service %q: %v", mongo.Name, err)) |
184 | } |
185 | if err := os.RemoveAll(a.Conf.dataDir); err != nil { |
186 | errors = append(errors, err) |
187 | @@ -413,6 +462,79 @@ |
188 | return fmt.Errorf("uninstall failed: %v", errors) |
189 | } |
190 | |
191 | +// ensureMongoServer ensures that the correct mongo upstart script is installed |
192 | +// and running. |
193 | +// |
194 | +// This method will remove old versions of the mongo upstart script as necessary |
195 | +// before installing the new version. |
196 | +func (a *MachineAgent) ensureMongoServer() error { |
197 | + service := a.mongoService() |
198 | + if service.Installed() { |
199 | + return nil |
200 | + } |
201 | + |
202 | + if err := removeOldMongoServices(); err != nil { |
203 | + return err |
204 | + } |
205 | + |
206 | + journalDir := filepath.Join(a.mongoDir(), "journal") |
207 | + |
208 | + if err := os.MkdirAll(journalDir, 0700); err != nil { |
209 | + logger.Errorf("failed to make mongo journal dir %s: %v", journalDir, err) |
210 | + return err |
211 | + } |
212 | + |
213 | + // manually create the prealloc files, since otherwise they get created as 100M files. |
214 | + zeroes := make([]byte, 1024*1024) |
215 | + for x := 0; x < 3; x++ { |
216 | + name := fmt.Sprintf("prealloc.%d", x) |
217 | + filename := filepath.Join(journalDir, name) |
218 | + if err := ioutil.WriteFile(filename, zeroes, 700); err != nil { |
219 | + logger.Errorf("failed to make write mongo prealloc file: %v", journalDir, err) |
220 | + return err |
221 | + } |
222 | + } |
223 | + |
224 | + if err := service.Install(); err != nil { |
225 | + logger.Errorf("Failed to install mongo service %q: %v", service.Name, err) |
226 | + return err |
227 | + } |
228 | + return service.Start() |
229 | +} |
230 | + |
231 | +// mongoDir returns the directory that mongo should use to store its data. |
232 | +func (a *MachineAgent) mongoDir() string { |
233 | + return a.Conf.dataDir |
234 | +} |
235 | + |
236 | +// mongoSvcConfig returns the upstart configuration object for mongo. |
237 | +func (a *MachineAgent) mongoService() *upstart.Conf { |
238 | + return upstart.MongoUpstartService( |
239 | + mongoServiceName, |
240 | + a.mongoDir(), |
241 | + a.Conf.config.StatePort()) |
242 | +} |
243 | + |
244 | +// removeOldMongoServices looks for any old juju mongo upstart scripts and |
245 | +// removes them. |
246 | +func removeOldMongoServices() error { |
247 | + old := upstart.NewService(oldMongoServiceName) |
248 | + if err := old.StopAndRemove(); err != nil { |
249 | + logger.Errorf("Failed to remove old mongo upstart service %q: %v", old.Name, err) |
250 | + return err |
251 | + } |
252 | + |
253 | + // the new formatting for the script name started at version 2 |
254 | + for x := 2; x < upstart.MongoScriptVersion; x++ { |
255 | + old := upstart.NewService(fmt.Sprintf(mongoSvcFmt, x)) |
256 | + if err := old.StopAndRemove(); err != nil { |
257 | + logger.Errorf("Failed to remove old mongo upstart service %q: %v", old.Name, err) |
258 | + return err |
259 | + } |
260 | + } |
261 | + return nil |
262 | +} |
263 | + |
264 | // Below pieces are used for testing,to give us access to the *State opened |
265 | // by the agent, and allow us to trigger syncs without waiting 5s for them |
266 | // to happen automatically. |
267 | |
268 | === modified file 'cmd/jujud/machine_test.go' |
269 | --- cmd/jujud/machine_test.go 2014-01-24 14:52:58 +0000 |
270 | +++ cmd/jujud/machine_test.go 2014-01-28 15:13:44 +0000 |
271 | @@ -29,6 +29,7 @@ |
272 | charmtesting "launchpad.net/juju-core/state/apiserver/charmrevisionupdater/testing" |
273 | statetesting "launchpad.net/juju-core/state/testing" |
274 | "launchpad.net/juju-core/state/watcher" |
275 | + "launchpad.net/juju-core/testing" |
276 | coretesting "launchpad.net/juju-core/testing" |
277 | jc "launchpad.net/juju-core/testing/checkers" |
278 | "launchpad.net/juju-core/testing/testbase" |
279 | @@ -44,6 +45,7 @@ |
280 | type MachineSuite struct { |
281 | agentSuite |
282 | lxctesting.TestSuite |
283 | + testing.MockUpstartSuite |
284 | } |
285 | |
286 | var _ = gc.Suite(&MachineSuite{}) |
287 | @@ -51,6 +53,7 @@ |
288 | func (s *MachineSuite) SetUpSuite(c *gc.C) { |
289 | s.agentSuite.SetUpSuite(c) |
290 | s.TestSuite.SetUpSuite(c) |
291 | + s.MockUpstartSuite.SetUpSuite(c) |
292 | restore := testbase.PatchValue(&charm.CacheDir, c.MkDir()) |
293 | s.AddSuiteCleanup(func(*gc.C) { restore() }) |
294 | } |
295 | @@ -58,21 +61,27 @@ |
296 | func (s *MachineSuite) TearDownSuite(c *gc.C) { |
297 | s.TestSuite.TearDownSuite(c) |
298 | s.agentSuite.TearDownSuite(c) |
299 | + s.MockUpstartSuite.TearDownSuite(c) |
300 | } |
301 | |
302 | func (s *MachineSuite) SetUpTest(c *gc.C) { |
303 | s.agentSuite.SetUpTest(c) |
304 | s.TestSuite.SetUpTest(c) |
305 | +<<<<<<< TREE |
306 | os.Remove(jujuRun) // ignore error; may not exist |
307 | // Fake $HOME, and ssh user to avoid touching ~ubuntu/.ssh/authorized_keys. |
308 | fakeHome := coretesting.MakeEmptyFakeHomeWithoutJuju(c) |
309 | s.AddCleanup(func(*gc.C) { fakeHome.Restore() }) |
310 | s.PatchValue(&authenticationworker.SSHUser, "") |
311 | +======= |
312 | + s.MockUpstartSuite.SetUpTest(c) |
313 | +>>>>>>> MERGE-SOURCE |
314 | } |
315 | |
316 | func (s *MachineSuite) TearDownTest(c *gc.C) { |
317 | s.TestSuite.TearDownTest(c) |
318 | s.agentSuite.TearDownTest(c) |
319 | + s.MockUpstartSuite.TearDownTest(c) |
320 | } |
321 | |
322 | const initialMachinePassword = "machine-password-1234567890" |
323 | |
324 | === modified file 'environs/cloudinit/cloudinit.go' |
325 | --- environs/cloudinit/cloudinit.go 2014-01-26 23:28:43 +0000 |
326 | +++ environs/cloudinit/cloudinit.go 2014-01-28 15:13:44 +0000 |
327 | @@ -352,9 +352,7 @@ |
328 | } |
329 | certKey := string(cfg.StateServerCert) + string(cfg.StateServerKey) |
330 | c.AddFile(cfg.dataFile("server.pem"), certKey, 0600) |
331 | - if err := cfg.addMongoToBoot(c); err != nil { |
332 | - return err |
333 | - } |
334 | + |
335 | // We temporarily give bootstrap-state a directory |
336 | // of its own so that it can get the state info via the |
337 | // same mechanism as other jujud commands. |
338 | @@ -449,11 +447,16 @@ |
339 | if err != nil { |
340 | return nil, err |
341 | } |
342 | +<<<<<<< TREE |
343 | acfg.SetValue(agent.RsyslogConfPath, cfg.RsyslogConfPath) |
344 | acfg.SetValue(agent.AgentServiceName, cfg.MachineAgentServiceName) |
345 | if cfg.StateServer { |
346 | acfg.SetValue(agent.MongoServiceName, cfg.MongoServiceName) |
347 | } |
348 | +======= |
349 | + acfg.SetValue(agent.AgentServiceName, machineAgentServiceName(tag)) |
350 | + |
351 | +>>>>>>> MERGE-SOURCE |
352 | cmds, err := acfg.WriteCommands() |
353 | if err != nil { |
354 | return nil, err |
355 | @@ -462,6 +465,13 @@ |
356 | return acfg, nil |
357 | } |
358 | |
359 | +<<<<<<< TREE |
360 | +======= |
361 | +func machineAgentServiceName(tag string) string { |
362 | + return "jujud-" + tag |
363 | +} |
364 | + |
365 | +>>>>>>> MERGE-SOURCE |
366 | func (cfg *MachineConfig) addMachineAgentToBoot(c *cloudinit.Config, tag, machineId string) error { |
367 | // Make the agent run via a symbolic link to the actual tools |
368 | // directory, so it can upgrade itself without needing to change |
369 | @@ -481,6 +491,7 @@ |
370 | return nil |
371 | } |
372 | |
373 | +<<<<<<< TREE |
374 | func (cfg *MachineConfig) addMongoToBoot(c *cloudinit.Config) error { |
375 | dbDir := path.Join(cfg.DataDir, "db") |
376 | c.AddScripts( |
377 | @@ -503,6 +514,8 @@ |
378 | return nil |
379 | } |
380 | |
381 | +======= |
382 | +>>>>>>> MERGE-SOURCE |
383 | // versionDir converts a tools URL into a name |
384 | // to use as a directory for storing the tools executables in |
385 | // by using the last element stripped of its extension. |
386 | |
387 | === modified file 'provider/local/config.go' |
388 | --- provider/local/config.go 2014-01-24 01:50:07 +0000 |
389 | +++ provider/local/config.go 2014-01-28 15:13:44 +0000 |
390 | @@ -81,10 +81,6 @@ |
391 | return filepath.Join(c.rootDir(), "storage") |
392 | } |
393 | |
394 | -func (c *environConfig) mongoDir() string { |
395 | - return filepath.Join(c.rootDir(), "db") |
396 | -} |
397 | - |
398 | func (c *environConfig) logDir() string { |
399 | return filepath.Join(c.rootDir(), "log") |
400 | } |
401 | @@ -127,7 +123,6 @@ |
402 | for _, dirname := range []string{ |
403 | c.sharedStorageDir(), |
404 | c.storageDir(), |
405 | - c.mongoDir(), |
406 | c.logDir(), |
407 | } { |
408 | logger.Tracef("creating directory %s", dirname) |
409 | |
410 | === modified file 'provider/local/environ.go' |
411 | --- provider/local/environ.go 2014-01-24 14:52:58 +0000 |
412 | +++ provider/local/environ.go 2014-01-28 15:13:44 +0000 |
413 | @@ -449,3 +449,213 @@ |
414 | func (env *localEnviron) Provider() environs.EnvironProvider { |
415 | return providerInstance |
416 | } |
417 | +<<<<<<< TREE |
418 | +======= |
419 | + |
420 | +// setupLocalMongoService returns the cert and key if there was no error. |
421 | +func (env *localEnviron) setupLocalMongoService() ([]byte, []byte, error) { |
422 | + journalDir := filepath.Join(env.config.rootDir(), "db/journal") |
423 | + logger.Debugf("create mongo journal dir: %v", journalDir) |
424 | + if err := os.MkdirAll(journalDir, 0755); err != nil { |
425 | + logger.Errorf("failed to make mongo journal dir %s: %v", journalDir, err) |
426 | + return nil, nil, err |
427 | + } |
428 | + |
429 | + logger.Debugf("generate server cert") |
430 | + cert, key, err := env.config.GenerateStateServerCertAndKey() |
431 | + if err != nil { |
432 | + logger.Errorf("failed to generate server cert: %v", err) |
433 | + return nil, nil, err |
434 | + } |
435 | + if err := ioutil.WriteFile( |
436 | + env.config.configFile("server.pem"), |
437 | + append(cert, key...), |
438 | + 0600); err != nil { |
439 | + logger.Errorf("failed to write server.pem: %v", err) |
440 | + return nil, nil, err |
441 | + } |
442 | + |
443 | + mongo := upstart.MongoUpstartService( |
444 | + env.mongoServiceName(), |
445 | + env.config.rootDir(), |
446 | + env.config.StatePort()) |
447 | + mongo.InitDir = upstartScriptLocation |
448 | + logger.Infof("installing service %s to %s", env.mongoServiceName(), mongo.InitDir) |
449 | + if err := mongo.Install(); err != nil { |
450 | + logger.Errorf("could not install mongo service: %v", err) |
451 | + return nil, nil, err |
452 | + } |
453 | + return cert, key, nil |
454 | +} |
455 | + |
456 | +func (env *localEnviron) setupLocalMachineAgent(cons constraints.Value, possibleTools tools.List) error { |
457 | + dataDir := env.config.rootDir() |
458 | + // unpack the first tools into the agent dir. |
459 | + agentTools := possibleTools[0] |
460 | + logger.Debugf("tools: %#v", agentTools) |
461 | + // brutally abuse our knowledge of storage to directly open the file |
462 | + toolsUrl, err := url.Parse(agentTools.URL) |
463 | + if err != nil { |
464 | + return err |
465 | + } |
466 | + toolsLocation := filepath.Join(env.config.storageDir(), toolsUrl.Path) |
467 | + logger.Infof("tools location: %v", toolsLocation) |
468 | + toolsFile, err := os.Open(toolsLocation) |
469 | + defer toolsFile.Close() |
470 | + // Again, brutally abuse our knowledge here. |
471 | + |
472 | + // The tools that possible bootstrap tools are based on the |
473 | + // default series in the config. However we are running potentially on a |
474 | + // different series. When the machine agent is started, it will be |
475 | + // looking based on the current series, so we need to override the series |
476 | + // returned in the tools to be the current series. |
477 | + agentTools.Version.Series = version.Current.Series |
478 | + err = agenttools.UnpackTools(dataDir, agentTools, toolsFile) |
479 | + |
480 | + machineId := "0" // Always machine 0 |
481 | + tag := names.MachineTag(machineId) |
482 | + |
483 | + // make sure we create the symlink so we have it for the upstart config to use |
484 | + if _, err := agenttools.ChangeAgentTools(dataDir, tag, agentTools.Version); err != nil { |
485 | + logger.Errorf("could not create tools directory symlink: %v", err) |
486 | + return err |
487 | + } |
488 | + |
489 | + toolsDir := agenttools.ToolsDir(dataDir, tag) |
490 | + |
491 | + logDir := env.config.logDir() |
492 | + machineEnvironment := map[string]string{ |
493 | + "USER": env.config.user, |
494 | + "HOME": osenv.Home(), |
495 | + } |
496 | + agentService := upstart.MachineAgentUpstartService( |
497 | + env.machineAgentServiceName(), |
498 | + toolsDir, dataDir, logDir, tag, machineId, machineEnvironment) |
499 | + |
500 | + agentService.InitDir = upstartScriptLocation |
501 | + logger.Infof("installing service %s to %s", env.machineAgentServiceName(), agentService.InitDir) |
502 | + if err := agentService.Install(); err != nil { |
503 | + logger.Errorf("could not install machine agent service: %v", err) |
504 | + return err |
505 | + } |
506 | + return nil |
507 | +} |
508 | + |
509 | +func (env *localEnviron) findBridgeAddress(networkBridge string) (string, error) { |
510 | + return getAddressForInterface(networkBridge) |
511 | +} |
512 | + |
513 | +func (env *localEnviron) writeBootstrapAgentConfFile(secret string, cert, key []byte) (agent.Config, error) { |
514 | + tag := names.MachineTag("0") |
515 | + passwordHash := utils.UserPasswordHash(secret, utils.CompatSalt) |
516 | + // We don't check the existance of the CACert here as if it wasn't set, we |
517 | + // wouldn't get this far. |
518 | + cfg := env.config.Config |
519 | + caCert, _ := cfg.CACert() |
520 | + agentValues := map[string]string{ |
521 | + agent.ProviderType: env.config.Type(), |
522 | + agent.Namespace: env.config.namespace(), |
523 | + agent.StorageDir: env.config.storageDir(), |
524 | + agent.StorageAddr: env.config.storageAddr(), |
525 | + agent.SharedStorageDir: env.config.sharedStorageDir(), |
526 | + agent.SharedStorageAddr: env.config.sharedStorageAddr(), |
527 | + agent.AgentServiceName: env.machineAgentServiceName(), |
528 | + agent.MongoServiceName: env.mongoServiceName(), |
529 | + } |
530 | + // NOTE: the state address HAS to be localhost, otherwise the mongo |
531 | + // initialization fails. There is some magic code somewhere in the mongo |
532 | + // connection code that treats connections from localhost as special, and |
533 | + // will raise unauthorized errors during the initialization if the caller |
534 | + // is not connected from localhost. |
535 | + stateAddress := fmt.Sprintf("localhost:%d", cfg.StatePort()) |
536 | + apiAddress := fmt.Sprintf("localhost:%d", cfg.APIPort()) |
537 | + config, err := agent.NewStateMachineConfig( |
538 | + agent.StateMachineConfigParams{ |
539 | + AgentConfigParams: agent.AgentConfigParams{ |
540 | + DataDir: env.config.rootDir(), |
541 | + Tag: tag, |
542 | + Password: passwordHash, |
543 | + Nonce: state.BootstrapNonce, |
544 | + StateAddresses: []string{stateAddress}, |
545 | + APIAddresses: []string{apiAddress}, |
546 | + CACert: caCert, |
547 | + Values: agentValues, |
548 | + }, |
549 | + StateServerCert: cert, |
550 | + StateServerKey: key, |
551 | + StatePort: cfg.StatePort(), |
552 | + APIPort: cfg.APIPort(), |
553 | + }) |
554 | + if err != nil { |
555 | + return nil, err |
556 | + } |
557 | + if err := config.Write(); err != nil { |
558 | + logger.Errorf("failed to write bootstrap agent file: %v", err) |
559 | + return nil, err |
560 | + } |
561 | + return config, nil |
562 | +} |
563 | + |
564 | +func (env *localEnviron) initializeState(agentConfig agent.Config, cons constraints.Value) error { |
565 | + bootstrapCfg, err := environs.BootstrapConfig(env.config.Config) |
566 | + if err != nil { |
567 | + return err |
568 | + } |
569 | + st, m, err := agentConfig.InitializeState(bootstrapCfg, agent.BootstrapMachineConfig{ |
570 | + Constraints: cons, |
571 | + Jobs: []state.MachineJob{ |
572 | + state.JobManageEnviron, |
573 | + state.JobManageState, |
574 | + }, |
575 | + InstanceId: bootstrapInstanceId, |
576 | + }, state.DialOpts{ |
577 | + Timeout: 60 * time.Second, |
578 | + }) |
579 | + if err != nil { |
580 | + return err |
581 | + } |
582 | + defer st.Close() |
583 | + addr, err := env.findBridgeAddress(env.config.networkBridge()) |
584 | + if err != nil { |
585 | + return fmt.Errorf("failed to get bridge address: %v", err) |
586 | + } |
587 | + err = m.SetAddresses([]instance.Address{{ |
588 | + NetworkScope: instance.NetworkPublic, |
589 | + Type: instance.HostName, |
590 | + Value: "localhost", |
591 | + }, { |
592 | + NetworkScope: instance.NetworkCloudLocal, |
593 | + Type: instance.Ipv4Address, |
594 | + Value: addr, |
595 | + }}) |
596 | + if err != nil { |
597 | + return fmt.Errorf("cannot set addresses on bootstrap instance: %v", err) |
598 | + } |
599 | + return nil |
600 | +} |
601 | + |
602 | +func (env *localEnviron) configureLocalSyslog() error { |
603 | + tag := names.MachineTag("0") |
604 | + syslogConfigRenderer := syslog.NewAccumulateConfig(tag, env.config.SyslogPort(), env.config.namespace()) |
605 | + syslogConfigRenderer.ConfigDir = syslogConfigDir |
606 | + syslogConfigRenderer.ConfigFileName = env.syslogFilename() |
607 | + syslogConfigRenderer.LogDir = env.config.logDir() |
608 | + if err := syslogConfigRenderer.Write(); err != nil { |
609 | + return err |
610 | + } |
611 | + if err := syslog.Restart(); err != nil { |
612 | + logger.Warningf("cannot restart syslog daemon: %v", err) |
613 | + } |
614 | + return nil |
615 | +} |
616 | + |
617 | +func (env *localEnviron) removeLocalSyslog() { |
618 | + // Don't fail if we have issues, but warn the user. |
619 | + if err := os.Remove(filepath.Join(syslogConfigDir, env.syslogFilename())); err != nil { |
620 | + logger.Warningf("could not remove local syslog config: %v", err) |
621 | + } |
622 | + if err := syslog.Restart(); err != nil { |
623 | + logger.Warningf("cannot restart syslog daemon: %v", err) |
624 | + } |
625 | +} |
626 | +>>>>>>> MERGE-SOURCE |
627 | |
628 | === modified file 'replicaset/replicaset.go' |
629 | --- replicaset/replicaset.go 2014-01-21 19:26:14 +0000 |
630 | +++ replicaset/replicaset.go 2014-01-28 15:13:44 +0000 |
631 | @@ -275,7 +275,11 @@ |
632 | Healthy bool `bson:"health"` |
633 | |
634 | // State describes the current state of the member. |
635 | +<<<<<<< TREE |
636 | State MemberState `bson:"state"` |
637 | +======= |
638 | + State MemberState `bson:"State"` |
639 | +>>>>>>> MERGE-SOURCE |
640 | |
641 | // Uptime describes how long the member has been online. |
642 | Uptime time.Duration `bson:"uptime"` |
643 | |
644 | === modified file 'replicaset/replicaset_test.go' |
645 | === added file 'testing/mockupstart.go' |
646 | --- testing/mockupstart.go 1970-01-01 00:00:00 +0000 |
647 | +++ testing/mockupstart.go 2014-01-28 15:13:44 +0000 |
648 | @@ -0,0 +1,42 @@ |
649 | +package testing |
650 | + |
651 | +import ( |
652 | + "fmt" |
653 | + "io/ioutil" |
654 | + "os" |
655 | + "path/filepath" |
656 | + |
657 | + gc "launchpad.net/gocheck" |
658 | + |
659 | + "launchpad.net/juju-core/testing/testbase" |
660 | + "launchpad.net/juju-core/upstart" |
661 | +) |
662 | + |
663 | +// MockUpstartSuite is a testing suite that you can embed to easily mock out |
664 | +// upstart. |
665 | +type MockUpstartSuite struct { |
666 | + testbase.CleanupSuite |
667 | + binDir string |
668 | +} |
669 | + |
670 | +var _ = gc.Suite(&MockUpstartSuite{}) |
671 | + |
672 | +func (s *MockUpstartSuite) SetUpSuite(c *gc.C) { |
673 | + s.CleanupSuite.SetUpSuite(c) |
674 | + s.binDir = c.MkDir() |
675 | + s.PatchEnvironment("PATH", fmt.Sprintf("%v%v%v", s.binDir, filepath.ListSeparator, os.Getenv("PATH"))) |
676 | + s.PatchValue(&upstart.InitDir, c.MkDir()) |
677 | + s.makeBin(c, "status", `echo "blah stop/waiting"`) |
678 | + s.makeBin(c, "stopped-status", `echo "blah stop/waiting"`) |
679 | + s.makeBin(c, "started-status", `echo "blah start/running, process 666"`) |
680 | + s.makeBin(c, "start", "cp $(which started-status) $(which status)") |
681 | + s.makeBin(c, "stop", "cp $(which stopped-status) $(which status)") |
682 | +} |
683 | + |
684 | +// makeBin makes a binary with the given name and contents in the binDir. This |
685 | +// is an easy way to mock the behavior of built-in applications. |
686 | +func (s *MockUpstartSuite) makeBin(c *gc.C, name, script string) { |
687 | + path := filepath.Join(s.binDir, name) |
688 | + err := ioutil.WriteFile(path, []byte("#!/bin/bash --norc\n"+script), 0755) |
689 | + c.Assert(err, gc.IsNil) |
690 | +} |
691 | |
692 | === modified file 'upstart/service.go' |
693 | --- upstart/service.go 2013-11-19 08:53:40 +0000 |
694 | +++ upstart/service.go 2014-01-28 15:13:44 +0000 |
695 | @@ -13,12 +13,22 @@ |
696 | const ( |
697 | maxMongoFiles = 65000 |
698 | maxAgentFiles = 20000 |
699 | + |
700 | + // MongoScriptVersion keeps track of changes to the mongo upstart script. |
701 | + // Update this version when you update the script that gets installed from |
702 | + // MongoUpstartService. |
703 | + MongoScriptVersion = 2 |
704 | ) |
705 | |
706 | // MongoUpstartService returns the upstart config for the mongo state service. |
707 | -func MongoUpstartService(name, dataDir, dbDir string, port int) *Conf { |
708 | +// |
709 | +// This method assumes there is a server.pem keyfile in dataDir. |
710 | +func MongoUpstartService(name, dataDir string, port int) *Conf { |
711 | keyFile := path.Join(dataDir, "server.pem") |
712 | svc := NewService(name) |
713 | + |
714 | + dbDir := path.Join(dataDir, "db") |
715 | + |
716 | return &Conf{ |
717 | Service: *svc, |
718 | Desc: "juju state database", |
719 | @@ -36,7 +46,8 @@ |
720 | " --port " + fmt.Sprint(port) + |
721 | " --noprealloc" + |
722 | " --syslog" + |
723 | - " --smallfiles", |
724 | + " --smallfiles" + |
725 | + " --replSet juju", |
726 | } |
727 | } |
728 | |
729 | |
730 | === modified file 'upstart/upstart.go' |
731 | --- upstart/upstart.go 2013-10-02 03:44:40 +0000 |
732 | +++ upstart/upstart.go 2014-01-28 15:13:44 +0000 |
733 | @@ -18,7 +18,13 @@ |
734 | "launchpad.net/juju-core/utils" |
735 | ) |
736 | |
737 | -var startedRE = regexp.MustCompile(`^.* start/running, process (\d+)\n$`) |
738 | +var ( |
739 | + startedRE = regexp.MustCompile(`^.* start/running, process (\d+)\n$`) |
740 | + |
741 | + // InitDir is the directory where upstart scripts will be created by |
742 | + // default. |
743 | + InitDir = "/etc/init" |
744 | +) |
745 | |
746 | var InstallStartRetryAttempts = utils.AttemptStrategy{ |
747 | Total: 1 * time.Second, |
748 | @@ -32,7 +38,7 @@ |
749 | } |
750 | |
751 | func NewService(name string) *Service { |
752 | - return &Service{Name: name, InitDir: "/etc/init"} |
753 | + return &Service{Name: name, InitDir: InitDir} |
754 | } |
755 | |
756 | // confPath returns the path to the service's configuration file. |
Looks good so far, with a couple of suggestions/queries below.
https:/ /codereview. appspot. com/53220043/ diff/20001/ agent/agent. go
File agent/agent.go (right):
https:/ /codereview. appspot. com/53220043/ diff/20001/ agent/agent. go#newcode232
agent/agent.go:232: func MongoDbDir(dir string) string {
or this could be a method on Config.
at the least, dir should be named "dataDir"
https:/ /codereview. appspot. com/53220043/ diff/20001/ cmd/jujud/ machine. go machine. go (right):
File cmd/jujud/
https:/ /codereview. appspot. com/53220043/ diff/20001/ cmd/jujud/ machine. go#newcode386 machine. go:386: if err := conf.Install(); err != nil {
cmd/jujud/
Do we really want to do this every time the machine agent bounces?
If possible I'd prefer to avoid bouncing mongod in that case.
https:/ /codereview. appspot. com/53220043/ diff/20001/ replicaset/ replicaset. go replicaset. go (right):
File replicaset/
https:/ /codereview. appspot. com/53220043/ diff/20001/ replicaset/ replicaset. go#newcode257 replicaset. go:257: State MemberState `bson:"State"`
replicaset/
"state", surely?
https:/ /codereview. appspot. com/53220043/