Merge lp:~rogpeppe/juju-core/mfoord-wrapsingletonworkers into lp:~go-bot/juju-core/trunk
- mfoord-wrapsingletonworkers
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Roger Peppe |
Approved revision: | no longer in the source branch. |
Merged at revision: | 2608 |
Proposed branch: | lp:~rogpeppe/juju-core/mfoord-wrapsingletonworkers |
Merge into: | lp:~go-bot/juju-core/trunk |
Diff against target: |
346 lines (+144/-17) 5 files modified
agent/mongo/mongo.go (+8/-0) cmd/jujud/machine.go (+66/-15) cmd/jujud/machine_test.go (+49/-0) replicaset/replicaset.go (+8/-2) replicaset/replicaset_test.go (+13/-0) |
To merge this branch: | bzr merge lp:~rogpeppe/juju-core/mfoord-wrapsingletonworkers |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju Engineering | Pending | ||
Review via email: mp+215035@code.launchpad.net |
Commit message
cmd/jujud: wrap singular workers
We start the various workers that require only a single
instance to be running via the runner returned by singular.New.
This is awkward to test - we just do a basic sanity check
that the right workers are started through the singular
runner.
Description of the change
cmd/jujud: wrap singular workers
We start the various workers that require only a single
instance to be running via the runner returned by singular.New.
This is awkward to test - we just do a basic sanity check
that the right workers are started through the singular
runner.
Roger Peppe (rogpeppe) wrote : | # |
Roger Peppe (rogpeppe) wrote : | # |
Please take a look.
Nate Finch (natefinch) wrote : | # |
LGTM with just a couple minor suggestions.
https:/
File cmd/jujud/
https:/
cmd/jujud/
could this code get moved to the loop over entity.jobs below? It
diesn't seem to neec to be here, and it puts the declaration far from
the use of the variable.
https:/
File cmd/jujud/
https:/
cmd/jujud/
jc.DeepEquals, []string{
This checks order of the slice, too, right? Does the order matter? If
not, we may want to use jc.SameContents here, so we don't fail the test
if the order changes.
Roger Peppe (rogpeppe) wrote : | # |
https:/
File cmd/jujud/
https:/
cmd/jujud/
On 2014/04/10 11:14:38, nate.finch wrote:
> could this code get moved to the loop over entity.jobs below? It
diesn't seem
> to neec to be here, and it puts the declaration far from the use of
the
> variable.
i don't we can do this, because newSingularRunner can fail, and if that
happens we don't want to leave the other jobs running.
https:/
File cmd/jujud/
https:/
cmd/jujud/
jc.DeepEquals, []string{
On 2014/04/10 11:14:38, nate.finch wrote:
> This checks order of the slice, too, right? Does the order matter?
If not, we
> may want to use jc.SameContents here, so we don't fail the test if the
order
> changes.
the slice is sorted (see the implementation of the started method)
Go Bot (go-bot) wrote : | # |
The attempt to merge lp:~rogpeppe/juju-core/mfoord-wrapsingletonworkers into lp:juju-core failed. Below is the output from the failed tests.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
? launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
Go Bot (go-bot) wrote : | # |
The attempt to merge lp:~rogpeppe/juju-core/mfoord-wrapsingletonworkers into lp:juju-core failed. Below is the output from the failed tests.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
? launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
Preview Diff
1 | === modified file 'agent/mongo/mongo.go' |
2 | --- agent/mongo/mongo.go 2014-04-08 20:10:22 +0000 |
3 | +++ agent/mongo/mongo.go 2014-04-10 14:24:10 +0000 |
4 | @@ -51,6 +51,14 @@ |
5 | addrs := obj.Addresses() |
6 | |
7 | masterHostPort, err := replicaset.MasterHostPort(session) |
8 | + |
9 | + // If the replica set has not been configured, then we |
10 | + // can have only one master and the caller must |
11 | + // be that master. |
12 | + if err == replicaset.ErrMasterNotConfigured { |
13 | + return true, nil |
14 | + } |
15 | + |
16 | if err != nil { |
17 | return false, err |
18 | } |
19 | |
20 | === modified file 'cmd/jujud/machine.go' |
21 | --- cmd/jujud/machine.go 2014-04-08 20:44:59 +0000 |
22 | +++ cmd/jujud/machine.go 2014-04-10 14:24:10 +0000 |
23 | @@ -11,6 +11,7 @@ |
24 | "time" |
25 | |
26 | "github.com/juju/loggo" |
27 | + "labix.org/v2/mgo" |
28 | "launchpad.net/gnuflag" |
29 | "launchpad.net/tomb" |
30 | |
31 | @@ -50,6 +51,7 @@ |
32 | "launchpad.net/juju-core/worker/provisioner" |
33 | "launchpad.net/juju-core/worker/resumer" |
34 | "launchpad.net/juju-core/worker/rsyslog" |
35 | + "launchpad.net/juju-core/worker/singular" |
36 | "launchpad.net/juju-core/worker/terminationworker" |
37 | "launchpad.net/juju-core/worker/upgrader" |
38 | ) |
39 | @@ -64,11 +66,15 @@ |
40 | type eitherState interface{} |
41 | |
42 | var ( |
43 | - retryDelay = 3 * time.Second |
44 | - jujuRun = "/usr/local/bin/juju-run" |
45 | - useMultipleCPUs = utils.UseMultipleCPUs |
46 | + retryDelay = 3 * time.Second |
47 | + jujuRun = "/usr/local/bin/juju-run" |
48 | + useMultipleCPUs = utils.UseMultipleCPUs |
49 | + |
50 | + // The following are defined as variables to |
51 | + // allow the tests to intercept calls to the functions. |
52 | ensureMongoServer = mongo.EnsureMongoServer |
53 | maybeInitiateMongoServer = mongo.MaybeInitiateMongoServer |
54 | + newSingularRunner = singular.New |
55 | |
56 | // reportOpenedAPI is exposed for tests to know when |
57 | // the State has been successfully opened. |
58 | @@ -239,15 +245,22 @@ |
59 | } |
60 | |
61 | rsyslogMode := rsyslog.RsyslogModeForwarding |
62 | + runner := newRunner(connectionIsFatal(st), moreImportant) |
63 | + var singularRunner worker.Runner |
64 | for _, job := range entity.Jobs() { |
65 | if job == params.JobManageEnviron { |
66 | rsyslogMode = rsyslog.RsyslogModeAccumulate |
67 | + conn := singularAPIConn{st, st.Agent()} |
68 | + singularRunner, err = newSingularRunner(runner, conn) |
69 | + if err != nil { |
70 | + return nil, fmt.Errorf("cannot make singular API Runner: %v", err) |
71 | + } |
72 | break |
73 | } |
74 | } |
75 | - runner := newRunner(connectionIsFatal(st), moreImportant) |
76 | |
77 | - // Run the upgrader and the upgrade-steps worker without waiting for the upgrade steps to complete. |
78 | + // Run the upgrader and the upgrade-steps worker without waiting for |
79 | + // the upgrade steps to complete. |
80 | runner.StartWorker("upgrader", func() (worker.Worker, error) { |
81 | return upgrader.NewUpgrader(st.Upgrader(), agentConfig), nil |
82 | }) |
83 | @@ -255,7 +268,8 @@ |
84 | return a.upgradeWorker(st, entity.Jobs(), agentConfig), nil |
85 | }) |
86 | |
87 | - // All other workers must wait for the upgrade steps to complete before starting. |
88 | + // All other workers must wait for the upgrade steps to complete |
89 | + // before starting. |
90 | a.startWorkerAfterUpgrade(runner, "machiner", func() (worker.Worker, error) { |
91 | return machiner.NewMachiner(st.Machiner(), agentConfig), nil |
92 | }) |
93 | @@ -272,7 +286,8 @@ |
94 | return newRsyslogConfigWorker(st.Rsyslog(), agentConfig, rsyslogMode) |
95 | }) |
96 | |
97 | - // If not a local provider bootstrap machine, start the worker to manage SSH keys. |
98 | + // If not a local provider bootstrap machine, start the worker to |
99 | + // manage SSH keys. |
100 | providerType := agentConfig.Value(agent.ProviderType) |
101 | if providerType != provider.Local || a.MachineId != bootstrapMachineId { |
102 | a.startWorkerAfterUpgrade(runner, "authenticationworker", func() (worker.Worker, error) { |
103 | @@ -293,16 +308,17 @@ |
104 | return deployer.NewDeployer(apiDeployer, context), nil |
105 | }) |
106 | case params.JobManageEnviron: |
107 | - a.startWorkerAfterUpgrade(runner, "environ-provisioner", func() (worker.Worker, error) { |
108 | + a.startWorkerAfterUpgrade(singularRunner, "environ-provisioner", func() (worker.Worker, error) { |
109 | return provisioner.NewEnvironProvisioner(st.Provisioner(), agentConfig), nil |
110 | }) |
111 | // TODO(axw) 2013-09-24 bug #1229506 |
112 | - // Make another job to enable the firewaller. Not all environments |
113 | - // are capable of managing ports centrally. |
114 | - a.startWorkerAfterUpgrade(runner, "firewaller", func() (worker.Worker, error) { |
115 | + // Make another job to enable the firewaller. Not all |
116 | + // environments are capable of managing ports |
117 | + // centrally. |
118 | + a.startWorkerAfterUpgrade(singularRunner, "firewaller", func() (worker.Worker, error) { |
119 | return firewaller.NewFirewaller(st.Firewaller()) |
120 | }) |
121 | - a.startWorkerAfterUpgrade(runner, "charm-revision-updater", func() (worker.Worker, error) { |
122 | + a.startWorkerAfterUpgrade(singularRunner, "charm-revision-updater", func() (worker.Worker, error) { |
123 | return charmrevisionworker.NewRevisionUpdateWorker(st.CharmRevisionUpdater()), nil |
124 | }) |
125 | case params.JobManageStateDeprecated: |
126 | @@ -399,7 +415,12 @@ |
127 | } |
128 | reportOpenedState(st) |
129 | |
130 | + singularStateConn := singularStateConn{st.MongoSession(), m} |
131 | runner := newRunner(connectionIsFatal(st), moreImportant) |
132 | + singularRunner, err := newSingularRunner(runner, singularStateConn) |
133 | + if err != nil { |
134 | + return nil, fmt.Errorf("cannot make singular State Runner: %v", err) |
135 | + } |
136 | |
137 | // Take advantage of special knowledge here in that we will only ever want |
138 | // the storage provider on one machine, and that is the "bootstrap" node. |
139 | @@ -444,16 +465,16 @@ |
140 | return apiserver.NewServer( |
141 | st, fmt.Sprintf(":%d", port), cert, key, dataDir, logDir) |
142 | }) |
143 | - a.startWorkerAfterUpgrade(runner, "cleaner", func() (worker.Worker, error) { |
144 | + a.startWorkerAfterUpgrade(singularRunner, "cleaner", func() (worker.Worker, error) { |
145 | return cleaner.NewCleaner(st), nil |
146 | }) |
147 | - a.startWorkerAfterUpgrade(runner, "resumer", func() (worker.Worker, error) { |
148 | + a.startWorkerAfterUpgrade(singularRunner, "resumer", func() (worker.Worker, error) { |
149 | // The action of resumer is so subtle that it is not tested, |
150 | // because we can't figure out how to do so without brutalising |
151 | // the transaction log. |
152 | return resumer.NewResumer(st), nil |
153 | }) |
154 | - a.startWorkerAfterUpgrade(runner, "minunitsworker", func() (worker.Worker, error) { |
155 | + a.startWorkerAfterUpgrade(singularRunner, "minunitsworker", func() (worker.Worker, error) { |
156 | return minunitsworker.NewMinUnitsWorker(st), nil |
157 | }) |
158 | case state.JobManageStateDeprecated: |
159 | @@ -670,3 +691,33 @@ |
160 | } |
161 | return fmt.Errorf("uninstall failed: %v", errors) |
162 | } |
163 | + |
164 | +// singularAPIConn implements singular.Conn on |
165 | +// top of an API connection. |
166 | +type singularAPIConn struct { |
167 | + apiState *api.State |
168 | + agentState *apiagent.State |
169 | +} |
170 | + |
171 | +func (c singularAPIConn) IsMaster() (bool, error) { |
172 | + return c.agentState.IsMaster() |
173 | +} |
174 | + |
175 | +func (c singularAPIConn) Ping() error { |
176 | + return c.apiState.Ping() |
177 | +} |
178 | + |
179 | +// singularStateConn implements singular.Conn on |
180 | +// top of a State connection. |
181 | +type singularStateConn struct { |
182 | + session *mgo.Session |
183 | + machine *state.Machine |
184 | +} |
185 | + |
186 | +func (c singularStateConn) IsMaster() (bool, error) { |
187 | + return mongo.IsMaster(c.session, c.machine) |
188 | +} |
189 | + |
190 | +func (c singularStateConn) Ping() error { |
191 | + return c.session.Ping() |
192 | +} |
193 | |
194 | === modified file 'cmd/jujud/machine_test.go' |
195 | --- cmd/jujud/machine_test.go 2014-04-08 20:42:04 +0000 |
196 | +++ cmd/jujud/machine_test.go 2014-04-10 14:24:10 +0000 |
197 | @@ -9,6 +9,7 @@ |
198 | "path/filepath" |
199 | "reflect" |
200 | "strings" |
201 | + "sync" |
202 | "time" |
203 | |
204 | "github.com/juju/testing" |
205 | @@ -39,6 +40,7 @@ |
206 | "launchpad.net/juju-core/tools" |
207 | "launchpad.net/juju-core/upstart" |
208 | "launchpad.net/juju-core/utils" |
209 | + "launchpad.net/juju-core/utils/set" |
210 | "launchpad.net/juju-core/utils/ssh" |
211 | sshtesting "launchpad.net/juju-core/utils/ssh/testing" |
212 | "launchpad.net/juju-core/version" |
213 | @@ -48,11 +50,13 @@ |
214 | "launchpad.net/juju-core/worker/instancepoller" |
215 | "launchpad.net/juju-core/worker/machineenvironmentworker" |
216 | "launchpad.net/juju-core/worker/rsyslog" |
217 | + "launchpad.net/juju-core/worker/singular" |
218 | "launchpad.net/juju-core/worker/upgrader" |
219 | ) |
220 | |
221 | type commonMachineSuite struct { |
222 | agentSuite |
223 | + singularRecord *singularRunnerRecord |
224 | lxctesting.TestSuite |
225 | } |
226 | |
227 | @@ -85,6 +89,9 @@ |
228 | fakeCmd(filepath.Join(testpath, "stop")) |
229 | |
230 | s.PatchValue(&upstart.InitDir, c.MkDir()) |
231 | + |
232 | + s.singularRecord = &singularRunnerRecord{} |
233 | + testing.PatchValue(&newSingularRunner, s.singularRecord.newSingularRunner) |
234 | } |
235 | |
236 | func fakeCmd(path string) { |
237 | @@ -373,6 +380,15 @@ |
238 | case <-time.After(5 * time.Second): |
239 | c.Fatalf("timed out waiting for agent to terminate") |
240 | } |
241 | + |
242 | + c.Assert(s.singularRecord.started(), jc.DeepEquals, []string{ |
243 | + "charm-revision-updater", |
244 | + "cleaner", |
245 | + "environ-provisioner", |
246 | + "firewaller", |
247 | + "minunitsworker", |
248 | + "resumer", |
249 | + }) |
250 | } |
251 | |
252 | func (s *MachineSuite) TestManageEnvironRunsInstancePoller(c *gc.C) { |
253 | @@ -948,3 +964,36 @@ |
254 | } |
255 | c.Assert(success, gc.Equals, true) |
256 | } |
257 | + |
258 | +type singularRunnerRecord struct { |
259 | + mu sync.Mutex |
260 | + startedWorkers set.Strings |
261 | +} |
262 | + |
263 | +func (r *singularRunnerRecord) newSingularRunner(runner worker.Runner, conn singular.Conn) (worker.Runner, error) { |
264 | + sr, err := singular.New(runner, conn) |
265 | + if err != nil { |
266 | + return nil, err |
267 | + } |
268 | + return &fakeSingularRunner{ |
269 | + Runner: sr, |
270 | + record: r, |
271 | + }, nil |
272 | +} |
273 | + |
274 | +// started returns the names of all singular-started workers. |
275 | +func (r *singularRunnerRecord) started() []string { |
276 | + return r.startedWorkers.SortedValues() |
277 | +} |
278 | + |
279 | +type fakeSingularRunner struct { |
280 | + worker.Runner |
281 | + record *singularRunnerRecord |
282 | +} |
283 | + |
284 | +func (r *fakeSingularRunner) StartWorker(name string, start func() (worker.Worker, error)) error { |
285 | + r.record.mu.Lock() |
286 | + defer r.record.mu.Unlock() |
287 | + r.record.startedWorkers.Add(name) |
288 | + return r.Runner.StartWorker(name, start) |
289 | +} |
290 | |
291 | === modified file 'replicaset/replicaset.go' |
292 | --- replicaset/replicaset.go 2014-04-09 16:45:35 +0000 |
293 | +++ replicaset/replicaset.go 2014-04-10 14:24:10 +0000 |
294 | @@ -241,13 +241,19 @@ |
295 | return results, nil |
296 | } |
297 | |
298 | -// MasterHostPort returns the "address:port" string for the |
299 | -// primary mongo server in the replicaset. |
300 | +var ErrMasterNotConfigured = fmt.Errorf("mongo master not configured") |
301 | + |
302 | +// MasterHostPort returns the "address:port" string for the primary |
303 | +// mongo server in the replicaset. It returns ErrMasterNotConfigured if |
304 | +// the replica set has not yet been initiated. |
305 | func MasterHostPort(session *mgo.Session) (string, error) { |
306 | results, err := IsMaster(session) |
307 | if err != nil { |
308 | return "", err |
309 | } |
310 | + if results.PrimaryAddress == "" { |
311 | + return "", ErrMasterNotConfigured |
312 | + } |
313 | return results.PrimaryAddress, nil |
314 | } |
315 | |
316 | |
317 | === modified file 'replicaset/replicaset_test.go' |
318 | --- replicaset/replicaset_test.go 2014-04-09 16:38:25 +0000 |
319 | +++ replicaset/replicaset_test.go 2014-04-10 14:24:10 +0000 |
320 | @@ -274,6 +274,8 @@ |
321 | break |
322 | } |
323 | c.Logf("attempting Set got error: %v", err) |
324 | + c.Logf("current session mode: %v", session.Mode()) |
325 | + session.Refresh() |
326 | } |
327 | c.Logf("Set() %d attempts in %s", attemptCount, time.Since(start)) |
328 | c.Assert(err, gc.IsNil) |
329 | @@ -354,6 +356,17 @@ |
330 | c.Assert(result, gc.Equals, expected) |
331 | } |
332 | |
333 | +func (s *MongoSuite) TestMasterHostPortOnUnconfiguredReplicaSet(c *gc.C) { |
334 | + inst := &coretesting.MgoInstance{} |
335 | + err := inst.Start(true) |
336 | + c.Assert(err, gc.IsNil) |
337 | + defer inst.Destroy() |
338 | + session := inst.MustDial() |
339 | + hp, err := MasterHostPort(session) |
340 | + c.Assert(err, gc.Equals, ErrMasterNotConfigured) |
341 | + c.Assert(hp, gc.Equals, "") |
342 | +} |
343 | + |
344 | func (s *MongoSuite) TestCurrentStatus(c *gc.C) { |
345 | session := root.MustDial() |
346 | defer session.Close() |
Reviewers: mp+215035_ code.launchpad. net,
Message:
Please take a look.
Description:
cmd/jujud: wrap singular workers
We start the various workers that require only a single
instance to be running via the runner returned by singular.New.
This is awkward to test - we just do a basic sanity check
that the right workers are started through the singular
runner.
https:/ /code.launchpad .net/~rogpeppe/ juju-core/ mfoord- wrapsingletonwo rkers/+ merge/215035
(do not edit description out of merge proposal)
Please review this at https:/ /codereview. appspot. com/86200043/
Affected files (+113, -15 lines): machine. go machine_ test.go
A [revision details]
M cmd/jujud/
M cmd/jujud/