Merge lp:~dimitern/juju-core/002-bug-1101140-ignore-unknown-relations into lp:~juju/juju-core/trunk
- 002-bug-1101140-ignore-unknown-relations
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 927 |
Proposed branch: | lp:~dimitern/juju-core/002-bug-1101140-ignore-unknown-relations |
Merge into: | lp:~juju/juju-core/trunk |
Prerequisite: | lp:~dimitern/juju-core/001-upgrade-charm-filter-restart-watchers |
Diff against target: |
355 lines (+111/-30) 6 files modified
worker/uniter/filter.go (+7/-7) worker/uniter/modes.go (+5/-0) worker/uniter/relationer.go (+5/-3) worker/uniter/relationer_test.go (+19/-10) worker/uniter/uniter.go (+12/-0) worker/uniter/uniter_test.go (+63/-10) |
To merge this branch: | bzr merge lp:~dimitern/juju-core/002-bug-1101140-ignore-unknown-relations |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
The Go Language Gophers | Pending | ||
Review via email: mp+149921@code.launchpad.net |
Commit message
Description of the change
uniter: skip unknown rel. endpoints
Ignore relations with unknown (not implemented by
the charm) when handling upgrades.
The uniter is now requesting all relation events
on upgrade, so they won't be missed.
This also fixes LP bug #1101140 (uniter does not
skip unimplemented relations).
William Reade (fwereade) wrote : | # |
Dimiter Naydenov (dimitern) wrote : | # |
Reviewers: mp+149921_
Message:
Please take a look.
Description:
uniter: ignore unknown endpoints
Ignore relations with unknown (not implemented by
the charm) when handling upgrades.
The uniter is now requesting all relation events
on upgrade, so they won't be missed.
Partial fix for bug #1101140.
(do not edit description out of merge proposal)
Please review this at https:/
Affected files:
A [revision details]
M worker/
M worker/
M worker/
M worker/
M worker/
M worker/
William Reade (fwereade) wrote : | # |
Thanks -- LGTM assuming the below.
https:/
File worker/
https:/
worker/
Now the upgrade is complete, we'll need to check all relations again:
some might previously have been skipped (if they involved endpoints only
implemented in the new charm).
https:/
File worker/
https:/
worker/
Nope, this is just a straight-up error, I think. We don't have any way
of resolving it, or reporting it, it's true; but panicking won't improve
matters so... ;).
https:/
worker/
implemented relation endpoint %q", ep)
"skipping relation with unknown endpoint"?
https:/
File worker/
https:/
worker/
*context) {
(c, ctx, path) feels neater to me, but YMMV
https:/
worker/
addRelation (?!)
This test does an add-relation as quickly as possible after an
upgrade-charm, in the hope that the scheduler will deliver the events in
the wrong order. The observed behaviour should be the same in either
case.
https:/
worker/
db2:0"},
May as well stick those steps together.
Dimiter Naydenov (dimitern) wrote : | # |
Please take a look.
https:/
File worker/
https:/
worker/
On 2013/02/25 10:31:57, fwereade wrote:
> Now the upgrade is complete, we'll need to check all relations again:
some might
> previously have been skipped (if they involved endpoints only
implemented in the
> new charm).
Done.
https:/
File worker/
https:/
worker/
On 2013/02/25 10:31:57, fwereade wrote:
> Nope, this is just a straight-up error, I think. We don't have any way
of
> resolving it, or reporting it, it's true; but panicking won't improve
matters
> so... ;).
Done.
https:/
worker/
On 2013/02/25 10:31:57, fwereade wrote:
> Nope, this is just a straight-up error, I think. We don't have any way
of
> resolving it, or reporting it, it's true; but panicking won't improve
matters
> so... ;).
Done.
https:/
worker/
implemented relation endpoint %q", ep)
On 2013/02/25 10:31:57, fwereade wrote:
> "skipping relation with unknown endpoint"?
Done.
https:/
File worker/
https:/
worker/
*context) {
On 2013/02/25 10:31:57, fwereade wrote:
> (c, ctx, path) feels neater to me, but YMMV
Done.
https:/
worker/
addRelation (?!)
On 2013/02/25 10:31:57, fwereade wrote:
> This test does an add-relation as quickly as possible after an
upgrade-charm, in
> the hope that the scheduler will deliver the events in the wrong
order. The
> observed behaviour should be the same in either case.
Done.
https:/
worker/
db2:0"},
On 2013/02/25 10:31:57, fwereade wrote:
> May as well stick those steps together.
Done.
Dimiter Naydenov (dimitern) wrote : | # |
*** Submitted:
uniter: skip unknown rel. endpoints
Ignore relations with unknown (not implemented by
the charm) when handling upgrades.
The uniter is now requesting all relation events
on upgrade, so they won't be missed.
This also fixes LP bug #1101140 (uniter does not
skip unimplemented relations).
R=fwereade, TheMue
CC=
https:/
https:/
File worker/
https:/
worker/
On 2013/02/25 11:20:47, TheMue wrote:
> IMHO not needed comment, only relates to removed code part.
Done.
https:/
worker/
it's there
On 2013/02/25 11:20:47, TheMue wrote:
> Punctuation. ;)
Done.
https:/
File worker/
https:/
worker/
dir
On 2013/02/25 11:20:47, TheMue wrote:
> Punctuation.
Done.
https:/
worker/
Join, so we must do it here
On 2013/02/25 11:20:47, TheMue wrote:
> Needless comment, it's related to a change.
Not exactly, the change breaks this unless dir.Ensure() is run. Updated
the comment to reflect this.
https:/
worker/
won't be created until necessary
On 2013/02/25 11:20:47, TheMue wrote:
> Ditto.
Actually, this is now part of the Join's behavior and it's worth the
comment, even more so, because it changes the test logic.
https:/
File worker/
https:/
worker/
disk
On 2013/02/25 11:20:47, TheMue wrote:
> // TODO(who): ..., and punctuation.
Done.
https:/
File worker/
https:/
worker/
that the scheduler will
On 2013/02/25 11:20:47, TheMue wrote:
> "in the hope"? Sounds a bit weak to me. Any way to force the delivery
in a wrong
> order?
No ways I can think of to tweak the scheduling at run-time.
Preview Diff
1 | === modified file 'worker/uniter/filter.go' |
2 | --- worker/uniter/filter.go 2013-02-22 14:11:34 +0000 |
3 | +++ worker/uniter/filter.go 2013-02-25 11:02:25 +0000 |
4 | @@ -78,7 +78,7 @@ |
5 | go func() { |
6 | defer f.tomb.Done() |
7 | err := f.loop(unitName) |
8 | - log.Printf("worker/uniter: filter error: %v", err) |
9 | + log.Printf("worker/uniter/filter: error: %v", err) |
10 | f.tomb.Kill(err) |
11 | }() |
12 | return f, nil |
13 | @@ -229,7 +229,7 @@ |
14 | f.outConfig = f.outConfigOn |
15 | discardConfig = f.discardConfig |
16 | case ids, ok := <-relationsw.Changes(): |
17 | - log.Debugf("filter: got relations change") |
18 | + log.Debugf("worker/uniter/filter: got relations change") |
19 | if !ok { |
20 | return watcher.MustErr(relationsw) |
21 | } |
22 | @@ -244,10 +244,10 @@ |
23 | log.Debugf("worker/uniter/filter: sent resolved event") |
24 | f.outResolved = nil |
25 | case f.outConfig <- nothing: |
26 | - log.Debugf("filter: sent config event") |
27 | + log.Debugf("worker/uniter/filter: sent config event") |
28 | f.outConfig = nil |
29 | case f.outRelations <- f.relations: |
30 | - log.Debugf("filter: sent relations event") |
31 | + log.Debugf("worker/uniter/filter: sent relations event") |
32 | f.outRelations = nil |
33 | f.relations = nil |
34 | |
35 | @@ -269,7 +269,7 @@ |
36 | watcher.Stop(relationsw, &f.tomb) |
37 | relationsw = f.service.WatchRelations() |
38 | case <-discardConfig: |
39 | - log.Debugf("filter: discarded config event") |
40 | + log.Debugf("worker/uniter/filter: discarded config event") |
41 | f.outConfig = nil |
42 | } |
43 | } |
44 | @@ -287,11 +287,11 @@ |
45 | if f.life != f.unit.Life() { |
46 | switch f.life = f.unit.Life(); f.life { |
47 | case state.Dying: |
48 | - log.Printf("worker/uniter: unit is dying") |
49 | + log.Printf("worker/uniter/filter: unit is dying") |
50 | close(f.outUnitDying) |
51 | f.outUpgrade = nil |
52 | case state.Dead: |
53 | - log.Printf("worker/uniter: unit is dead") |
54 | + log.Printf("worker/uniter/filter: unit is dead") |
55 | return worker.ErrDead |
56 | } |
57 | } |
58 | |
59 | === modified file 'worker/uniter/modes.go' |
60 | --- worker/uniter/modes.go 2013-02-15 10:09:00 +0000 |
61 | +++ worker/uniter/modes.go 2013-02-25 11:02:25 +0000 |
62 | @@ -141,6 +141,11 @@ |
63 | } else if err != nil { |
64 | return nil, err |
65 | } |
66 | + // Now the upgrade is complete, we'll need to check all |
67 | + // relations again: some might previously have been skipped |
68 | + // (if they involved endpoints only implemented in the new |
69 | + // charm). |
70 | + u.f.WantAllRelationsEvents() |
71 | return ModeContinue, nil |
72 | } |
73 | } |
74 | |
75 | === modified file 'worker/uniter/relationer.go' |
76 | --- worker/uniter/relationer.go 2013-02-15 10:09:00 +0000 |
77 | +++ worker/uniter/relationer.go 2013-02-25 11:02:25 +0000 |
78 | @@ -51,9 +51,7 @@ |
79 | if !ok { |
80 | return fmt.Errorf("cannot enter scope: private-address not set") |
81 | } |
82 | - if err := r.dir.Ensure(); err != nil { |
83 | - return err |
84 | - } |
85 | + // No need to check the r.dir exists yet |
86 | return r.ru.EnterScope(map[string]interface{}{"private-address": address}) |
87 | } |
88 | |
89 | @@ -123,6 +121,10 @@ |
90 | if err = r.dir.State().Validate(hi); err != nil { |
91 | return |
92 | } |
93 | + // We are about to use the dir, ensure it's there |
94 | + if err = r.dir.Ensure(); err != nil { |
95 | + return |
96 | + } |
97 | r.ctx.UpdateMembers(hi.Members) |
98 | if hi.Kind == hooks.RelationDeparted { |
99 | r.ctx.DeleteMember(hi.RemoteUnit) |
100 | |
101 | === modified file 'worker/uniter/relationer_test.go' |
102 | --- worker/uniter/relationer_test.go 2013-02-15 10:09:00 +0000 |
103 | +++ worker/uniter/relationer_test.go 2013-02-25 11:02:25 +0000 |
104 | @@ -17,11 +17,12 @@ |
105 | |
106 | type RelationerSuite struct { |
107 | testing.JujuConnSuite |
108 | - hooks chan hook.Info |
109 | - svc *state.Service |
110 | - rel *state.Relation |
111 | - ru *state.RelationUnit |
112 | - dir *relation.StateDir |
113 | + hooks chan hook.Info |
114 | + svc *state.Service |
115 | + rel *state.Relation |
116 | + ru *state.RelationUnit |
117 | + dir *relation.StateDir |
118 | + dirPath string |
119 | } |
120 | |
121 | var _ = Suite(&RelationerSuite{}) |
122 | @@ -36,7 +37,8 @@ |
123 | c.Assert(rels, HasLen, 1) |
124 | s.rel = rels[0] |
125 | s.ru = s.AddRelationUnit(c, "u/0") |
126 | - s.dir, err = relation.ReadStateDir(c.MkDir(), s.rel.Id()) |
127 | + s.dirPath = c.MkDir() |
128 | + s.dir, err = relation.ReadStateDir(s.dirPath, s.rel.Id()) |
129 | c.Assert(err, IsNil) |
130 | s.hooks = make(chan hook.Info) |
131 | } |
132 | @@ -96,6 +98,12 @@ |
133 | hi := hook.Info{Kind: hooks.RelationBroken} |
134 | _, err = r.PrepareHook(hi) |
135 | c.Assert(err, IsNil) |
136 | + |
137 | + // Verify PrepareHook created the dir |
138 | + fi, err := os.Stat(filepath.Join(s.dirPath, strconv.Itoa(s.rel.Id()))) |
139 | + c.Assert(err, IsNil) |
140 | + c.Assert(fi.IsDir(), Equals, true) |
141 | + |
142 | err = r.CommitHook(hi) |
143 | c.Assert(err, IsNil) |
144 | s.State.StartSync() |
145 | @@ -344,6 +352,8 @@ |
146 | |
147 | func (s *RelationerSuite) assertHook(c *C, expect hook.Info) { |
148 | s.State.StartSync() |
149 | + // We're no longer doing this in Join, so we must do it here |
150 | + c.Assert(s.dir.Ensure(), IsNil) |
151 | select { |
152 | case hi, ok := <-s.hooks: |
153 | c.Assert(ok, Equals, true) |
154 | @@ -396,12 +406,11 @@ |
155 | r := uniter.NewRelationer(ru, dir, hooks) |
156 | c.Assert(r.IsImplicit(), Equals, true) |
157 | |
158 | - // Join the relationer; check the dir was created and scope was entered. |
159 | + // Join the relationer; the dir won't be created until necessary |
160 | err = r.Join() |
161 | c.Assert(err, IsNil) |
162 | - fi, err := os.Stat(filepath.Join(relsDir, strconv.Itoa(rel.Id()))) |
163 | - c.Assert(err, IsNil) |
164 | - c.Assert(fi.IsDir(), Equals, true) |
165 | + _, err = os.Stat(filepath.Join(relsDir, strconv.Itoa(rel.Id()))) |
166 | + c.Assert(err, NotNil) |
167 | sub, err := logging.Unit("logging/0") |
168 | c.Assert(err, IsNil) |
169 | err = sub.SetPrivateAddress("blah") |
170 | |
171 | === modified file 'worker/uniter/uniter.go' |
172 | --- worker/uniter/uniter.go 2013-02-15 10:09:00 +0000 |
173 | +++ worker/uniter/uniter.go 2013-02-25 11:02:25 +0000 |
174 | @@ -315,6 +315,7 @@ |
175 | // restoreRelations reconciles the supplied relation state dirs with the |
176 | // remote state of the corresponding relations. |
177 | func (u *Uniter) restoreRelations() error { |
178 | + // TODO: Get these from state, not from disk |
179 | dirs, err := relation.ReadAllStateDirs(u.relationsDir) |
180 | if err != nil { |
181 | return err |
182 | @@ -376,6 +377,17 @@ |
183 | if rel.Life() != state.Alive { |
184 | continue |
185 | } |
186 | + // Make sure we ignore relations not implemented by the unit's charm |
187 | + ch, err := corecharm.ReadDir(u.charm.Path()) |
188 | + if err != nil { |
189 | + return nil, err |
190 | + } |
191 | + if ep, err := rel.Endpoint(u.unit.ServiceName()); err != nil { |
192 | + return nil, err |
193 | + } else if !ep.ImplementedBy(ch) { |
194 | + log.Printf("worker/uniter: skipping relation with unknown endpoint %q", ep) |
195 | + continue |
196 | + } |
197 | dir, err := relation.ReadStateDir(u.relationsDir, id) |
198 | if err != nil { |
199 | return nil, err |
200 | |
201 | === modified file 'worker/uniter/uniter_test.go' |
202 | --- worker/uniter/uniter_test.go 2013-02-18 13:18:29 +0000 |
203 | +++ worker/uniter/uniter_test.go 2013-02-25 11:02:25 +0000 |
204 | @@ -143,13 +143,13 @@ |
205 | func (ctx *context) matchLogHooks(c *C) (match bool, overshoot bool) { |
206 | // hookPattern matches juju-log calls as generated by writeHook. |
207 | hookPattern := fmt.Sprintf(`^.* JUJU `+ |
208 | - `u/0(| [a-z-]+:[0-9]+)`+ // juju-log badge; group matches relation id |
209 | + `u/0(| [a-z0-9-]+:[0-9]+)`+ // juju-log badge; group matches relation id |
210 | `: UniterSuite-%d`+ // test badge; prevents cross-pollution |
211 | ` ([0-9a-z-/ ]+)$`, // foo-relation-joined bar/123 |
212 | ctx.id, |
213 | ) |
214 | // donePattern matches uniter logging that indicates a hook has run. |
215 | - donePattern := `^.* JUJU worker/uniter: (ran "[a-z-]+" hook|hook failed)` |
216 | + donePattern := `^.* JUJU worker/uniter: (ran "[a-z0-9-]+" hook|hook failed)` |
217 | hookRegexp := regexp.MustCompile(hookPattern) |
218 | doneRegexp := regexp.MustCompile(donePattern) |
219 | |
220 | @@ -319,7 +319,7 @@ |
221 | ut( |
222 | "steady state config change with config-get verification", |
223 | createCharm{ |
224 | - customize: func(c *C, path string) { |
225 | + customize: func(c *C, ctx *context, path string) { |
226 | appendHook(c, path, "config-changed", "config-get --format yaml --output config.out") |
227 | }, |
228 | }, |
229 | @@ -520,7 +520,7 @@ |
230 | ), ut( |
231 | `upgrade: conflicting directories`, |
232 | createCharm{ |
233 | - customize: func(c *C, path string) { |
234 | + customize: func(c *C, ctx *context, path string) { |
235 | err := os.Mkdir(filepath.Join(path, "data"), 0755) |
236 | c.Assert(err, IsNil) |
237 | appendHook(c, path, "start", "echo DATA > data/newfile") |
238 | @@ -536,7 +536,7 @@ |
239 | |
240 | createCharm{ |
241 | revision: 1, |
242 | - customize: func(c *C, path string) { |
243 | + customize: func(c *C, ctx *context, path string) { |
244 | data := filepath.Join(path, "data") |
245 | err := ioutil.WriteFile(data, []byte("<nelson>ha ha</nelson>"), 0644) |
246 | c.Assert(err, IsNil) |
247 | @@ -563,7 +563,7 @@ |
248 | startUpgradeError{}, |
249 | createCharm{ |
250 | revision: 2, |
251 | - customize: func(c *C, path string) { |
252 | + customize: func(c *C, ctx *context, path string) { |
253 | otherdata := filepath.Join(path, "otherdata") |
254 | err := ioutil.WriteFile(otherdata, []byte("blah"), 0644) |
255 | c.Assert(err, IsNil) |
256 | @@ -614,6 +614,27 @@ |
257 | unitDead, |
258 | waitUniterDead{}, |
259 | waitHooks{}, |
260 | + ), ut( |
261 | + // This test does and add-relation as quickly as possible |
262 | + // after an upgrade-charm, in the hope that the scheduler will |
263 | + // deliver the events in the wrong order. The observed |
264 | + // behaviour should be the same in either case. |
265 | + "ignore unknown relations until upgrade is done", |
266 | + quickStart{}, |
267 | + createCharm{ |
268 | + revision: 2, |
269 | + customize: func(c *C, ctx *context, path string) { |
270 | + renameRelation(c, path, "db", "db2") |
271 | + hpath := filepath.Join(path, "hooks", "db2-relation-joined") |
272 | + ctx.writeHook(c, hpath, true) |
273 | + }, |
274 | + }, |
275 | + serveCharm{}, |
276 | + upgradeCharm{revision: 2}, |
277 | + addRelation{}, |
278 | + addRelationUnit{}, |
279 | + waitHooks{"upgrade-charm", "config-changed", "db2-relation-joined mysql/0 db2:0"}, |
280 | + verifyCharm{revision: 2}, |
281 | ), |
282 | |
283 | // Relations. |
284 | @@ -783,7 +804,7 @@ |
285 | type createCharm struct { |
286 | revision int |
287 | badHooks []string |
288 | - customize func(*C, string) |
289 | + customize func(*C, *context, string) |
290 | } |
291 | |
292 | var charmHooks = []string{ |
293 | @@ -805,7 +826,7 @@ |
294 | ctx.writeHook(c, path, good) |
295 | } |
296 | if s.customize != nil { |
297 | - s.customize(c, base) |
298 | + s.customize(c, ctx, base) |
299 | } |
300 | dir, err := charm.ReadDir(base) |
301 | c.Assert(err, IsNil) |
302 | @@ -1199,7 +1220,7 @@ |
303 | func (s startUpgradeError) step(c *C, ctx *context) { |
304 | steps := []stepper{ |
305 | createCharm{ |
306 | - customize: func(c *C, path string) { |
307 | + customize: func(c *C, ctx *context, path string) { |
308 | appendHook(c, path, "start", "echo STARTDATA > data") |
309 | }, |
310 | }, |
311 | @@ -1213,7 +1234,7 @@ |
312 | |
313 | createCharm{ |
314 | revision: 1, |
315 | - customize: func(c *C, path string) { |
316 | + customize: func(c *C, ctx *context, path string) { |
317 | data := filepath.Join(path, "data") |
318 | err := ioutil.WriteFile(data, []byte("<nelson>ha ha</nelson>"), 0644) |
319 | c.Assert(err, IsNil) |
320 | @@ -1472,3 +1493,35 @@ |
321 | _, err = f.Write([]byte(data)) |
322 | c.Assert(err, IsNil) |
323 | } |
324 | + |
325 | +func renameRelation(c *C, charmPath, oldName, newName string) { |
326 | + path := filepath.Join(charmPath, "metadata.yaml") |
327 | + f, err := os.Open(path) |
328 | + c.Assert(err, IsNil) |
329 | + defer f.Close() |
330 | + meta, err := charm.ReadMeta(f) |
331 | + c.Assert(err, IsNil) |
332 | + |
333 | + replace := func(what map[string]charm.Relation) bool { |
334 | + for relName, relation := range what { |
335 | + if relName == oldName { |
336 | + what[newName] = relation |
337 | + delete(what, oldName) |
338 | + return true |
339 | + } |
340 | + } |
341 | + return false |
342 | + } |
343 | + replaced := replace(meta.Provides) || replace(meta.Requires) || replace(meta.Peers) |
344 | + c.Assert(replaced, Equals, true, Commentf("charm %q does not implement relation %q", charmPath, oldName)) |
345 | + |
346 | + newmeta, err := goyaml.Marshal(meta) |
347 | + c.Assert(err, IsNil) |
348 | + ioutil.WriteFile(path, newmeta, 0644) |
349 | + |
350 | + f, err = os.Open(path) |
351 | + c.Assert(err, IsNil) |
352 | + defer f.Close() |
353 | + meta, err = charm.ReadMeta(f) |
354 | + c.Assert(err, IsNil) |
355 | +} |
Tests! But look into the prereq first anyway :).
https:/ /codereview. appspot. com/7385049/ diff/1/ worker/ uniter/ uniter. go uniter/ uniter. go (right):
File worker/
https:/ /codereview. appspot. com/7385049/ diff/1/ worker/ uniter/ uniter. go#newcode441 uniter/ uniter. go:441: sch, _, err := svc.Charm()
worker/
This will never catch problems. We know for *sure* that this charm
implements the endpoint... but we need to check the charm that is
actually deployed right here ;). I think you want something a bit more
like:
ch, err := corecharm. ReadDir( u.charm. Path())
...but I would also be potentially interested in smarter approaches
(like storing the *charm.Dir on the uniter, if you can think of a way to
do so cleanly and clearly... I don't *expect* you to do this, just a
suggestion).
https:/ /codereview. appspot. com/7385049/