Merge lp:~gz/juju-core/057-api-machiner-watch into lp:~go-bot/juju-core/trunk
- 057-api-machiner-watch
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | John A Meinel |
Approved revision: | no longer in the source branch. |
Merged at revision: | 1326 |
Proposed branch: | lp:~gz/juju-core/057-api-machiner-watch |
Merge into: | lp:~go-bot/juju-core/trunk |
Diff against target: |
290 lines (+123/-25) 7 files modified
state/api/params/params.go (+11/-0) state/apiserver/client.go (+1/-1) state/apiserver/common/interfaces.go (+17/-0) state/apiserver/machine/machiner.go (+28/-8) state/apiserver/machine/machiner_test.go (+55/-2) state/apiserver/resource.go (+6/-10) state/apiserver/root.go (+5/-4) |
To merge this branch: | bzr merge lp:~gz/juju-core/057-api-machiner-watch |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju Engineering | Pending | ||
Review via email: mp+170586@code.launchpad.net |
Commit message
state/apiserver: Implement Machiner.Watch
Server-side implementation of the final API call needed by the machiner.
R=fwereade, rogpeppe, wallyworld
Description of the change
state/apiserver: Implement Machiner.Watch
Adopting Dimiter's branch, with merge conflicts on trunk resolved.
See <https:/
As I understand it, William would like a couple more tests, but all
other comments were addressed.
Martin Packman (gz) wrote : | # |
Roger Peppe (rogpeppe) wrote : | # |
LGTM with a couple of trivial suggestions.
https:/
File state/apiserver
https:/
state/apiserver
common.
s/resourceRegis
https:/
File state/apiserver
https:/
state/apiserver
fakeResourceReg
s/resourcesRegi
Roger Peppe (rogpeppe) wrote : | # |
one thought:
https:/
File state/api/
https:/
state/api/
i think this should probably be EntityWatchResults, as anything that
returns an entity watcher will want to use the same type.
Martin Packman (gz) wrote : | # |
Please take a look.
https:/
File state/api/
https:/
state/api/
On 2013/06/20 15:29:34, rog wrote:
> i think this should probably be EntityWatchResults, as anything that
returns an
> entity watcher will want to use the same type.
Done, have left MachinesWatchRe
crazy about the construction of the array being done in the apiserver
module having to know about the array/etc from params to construct one
of a suitable length, but this is all old territory and let's not go
there.
https:/
File state/apiserver
https:/
state/apiserver
common.
On 2013/06/20 13:37:26, rog wrote:
> s/resourceRegis
Done.
https:/
File state/apiserver
https:/
state/apiserver
fakeResourceReg
On 2013/06/20 13:37:26, rog wrote:
> s/resourcesRegi
Done.
Roger Peppe (rogpeppe) wrote : | # |
https:/
File state/api/
https:/
state/api/
On 2013/06/20 16:01:30, gz wrote:
> On 2013/06/20 15:29:34, rog wrote:
> > i think this should probably be EntityWatchResults, as anything that
returns
> an
> > entity watcher will want to use the same type.
> Done, have left MachinesWatchRe
crazy
> about the construction of the array being done in the apiserver module
having to
> know about the array/etc from params to construct one of a suitable
length, but
> this is all old territory and let's not go there.
I think that EntityWatchResults is probably right actually.
Any bulk call that watches a set of entities will be able to use it, and
that seems reasonable to me.
I'm not sure what you mean about the array construction. Surely we can't
hide the fact that there's a slice here?
Martin Packman (gz) wrote : | # |
On 2013/06/20 16:13:25, rog wrote:
> I think that EntityWatchResults is probably right actually.
> Any bulk call that watches a set of entities will be able to use it,
and that
> seems reasonable to me.
> I'm not sure what you mean about the array construction. Surely we
can't hide
> the fact that there's a slice here?
I think it depends how you view that outer struct. Per the comment, I
was envisioning it as a "thing you pass to a particular api call to get
results", in which case you want one per api call, even if there happens
to be overlap. Also, you then perhaps don't want the caller having to
know all the details of the struct in order to create one and pass it
in. Maybe that's the wrong way of looking at it?
Ian Booth (wallyworld) wrote : | # |
LGTM. I have no real view on the array question as I'm not fully across
the nuances of it.
Roger Peppe (rogpeppe) wrote : | # |
On 20 June 2013 17:30, <email address hidden> wrote:
> On 2013/06/20 16:13:25, rog wrote:
>
>> I think that EntityWatchResults is probably right actually.
>> Any bulk call that watches a set of entities will be able to use it,
>
> and that
>>
>> seems reasonable to me.
>
>
>> I'm not sure what you mean about the array construction. Surely we
>
> can't hide
>>
>> the fact that there's a slice here?
>
>
> I think it depends how you view that outer struct. Per the comment, I
> was envisioning it as a "thing you pass to a particular api call to get
> results", in which case you want one per api call, even if there happens
> to be overlap. Also, you then perhaps don't want the caller having to
> know all the details of the struct in order to create one and pass it
> in. Maybe that's the wrong way of looking at it?
It's really just the return value of an API call. It needs a separate type
if the API call is returning something that no other API call returns,
but otherwise it's fine to share.
If we later need to add some other stuff to the return value for
a particular API call that shares a return type with another,
it's easy to change the return type - it's not part of the wire
contract and the return type can be changed, and a the other
stuff added, with no more compatibility
issues than adding a field to a non-shared return type.
In a way I think it's nice to share when possible as it's less lines
of code and easier to see the commonality (also easier to
write generic helper functions).
BTW the caller doesn't have to know the details of the struct
to pass it in. They do of course have to know the details of
the struct in order to deal with the results though.
William Reade (fwereade) wrote : | # |
On 2013/06/21 06:31:34, rog wrote:
> It's really just the return value of an API call. It needs a separate
type
> if the API call is returning something that no other API call returns,
> but otherwise it's fine to share.
+1
William Reade (fwereade) wrote : | # |
Commented where I think we need better testing. Otherwise LGTM.
https:/
File state/apiserver
https:/
state/apiserver
Can we perhaps call Next with the valid watcher's id and see what we
get?
Martin Packman (gz) wrote : | # |
Please take a look.
https:/
File state/apiserver
https:/
state/apiserver
On 2013/06/21 10:22:48, fwereade wrote:
> Can we perhaps call Next with the valid watcher's id and see what we
get?
I've had a crack at doing something with the watcher, if you can give me
more of a hint as to what we want I can improve the check.
William Reade (fwereade) wrote : | # |
On 2013/06/24 12:56:02, gz wrote:
> Please take a look.
https:/
> File state/apiserver
https:/
> state/apiserver
> On 2013/06/21 10:22:48, fwereade wrote:
> > Can we perhaps call Next with the valid watcher's id and see what we
get?
> I've had a crack at doing something with the watcher, if you can give
me more of
> a hint as to what we want I can improve the check.
LGTM, as discussed, but please read the first event off Changes() before
returning from Watch(); then do some basic verification of the watcher's
state with something like NotifyWatcherC.
Martin Packman (gz) wrote : | # |
Please take a look.
Roger Peppe (rogpeppe) wrote : | # |
LGTM
https:/
File state/apiserver
https:/
state/apiserver
s/=/:=
https:/
state/apiserver
state/watcher_
we've got loads of places that read from a channel without a timeout. if
we generalise the code, we'll look for and change lots of them, so i
don't think this comment is really necessary.
https:/
state/apiserver
this isn't actually checking that the watcher is watching the correct
machine, but i'm ok if we leave that for the client-side test.
https:/
state/apiserver
time.Millisecond):
2-5 seconds would be better. this is the worst case, and we want to be
resilient to slow machines.
Go Bot (go-bot) wrote : | # |
No approved revision specified.
Go Bot (go-bot) wrote : | # |
The attempt to merge lp:~gz/juju-core/057-api-machiner-watch into lp:juju-core failed. Below is the output from the failed tests.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
? launchpad.
ok launchpad.
-------
PANIC: agent.go:0: MachineSuite.
... Panic: Cannot drop MongoDB database juju: local error: bad record MAC (PC=0x41176C)
/usr/lib/
in panic
/home/tarmac/
in MgoReset
/home/tarmac/
in environState.
/home/tarmac/
in Reset
/home/tarmac/
in JujuConnSuite.
/home/tarmac/
in JujuConnSuite.
-------
PANIC: machine_
[LOG] 82.19123 INFO juju environs/testing: uploading FAKE tools 1.11.1-
[LOG] 82.19131 INFO juju environs: reading tools with major version 1
[LOG] 82.19133 DEBUG juju environs/tools: reading v1.* tools
[LOG] 82.19135 INFO juju environs: falling back to public bucket
[LOG] 82.19137 DEBUG juju environs/tools: reading v1.* tools
[LOG] 82.19141 DEBUG juju environs/tools: found 1.11.1-
[LOG] 82.19143 INFO juju environs: filtering tools by series: precise
[LOG] 82.19145 INFO juju environs: filtering tools by version: 1.11.1
[LOG] 82.19148 INFO juju environs/dummy: would pick tools from 1.11.1-
[LOG] 82.21977 INFO juju state: opening state; mongo addresses: ["localhost:
[LOG] 82.22346 INFO juju state: connection established
[LOG] 82.28876 INFO juju state: initializing environment
[LOG] 82.31655 INFO juju state/api: listening on "localhost:0"
[LOG] 82.34608 INFO juju state: opening state; mongo addresses: ["localhost:
[LOG] 82.34955 INFO juju state: connection established
[LOG] 82.35024 INFO juju juju: authorization error while connecting to state server; retrying
[LOG] 82.35030 INFO juju state: opening state; mongo addresses: ["localhost:
[LOG] 82.35341 INFO juju state: connection established
[LOG] 82.40812 INFO juju state/api: dialing "wss://
[LOG] 82.41266 INFO juju state/api: connection established
[LOG] 82.41286 DEBUG juju rpc/jsoncodec: <- {"RequestId"
[LOG] 82.41335 DEBUG juju rpc/jsoncodec: -> {"RequestId"
[LOG] 82.41971 DEBUG juju rpc/jsoncodec: <- {"RequestId"
Preview Diff
1 | === modified file 'state/api/params/params.go' |
2 | --- state/api/params/params.go 2013-06-20 16:59:05 +0000 |
3 | +++ state/api/params/params.go 2013-06-25 11:18:26 +0000 |
4 | @@ -60,6 +60,17 @@ |
5 | Machines []MachineSetStatus |
6 | } |
7 | |
8 | +// EntityWatchResult holds an EntityWatcher id and an error (if any). |
9 | +type EntityWatchResult struct { |
10 | + EntityWatcherId string |
11 | + Error *Error |
12 | +} |
13 | + |
14 | +// MachinesWatchResults holds the parameters for making a Machiner.Watch call. |
15 | +type MachinesWatchResults struct { |
16 | + Results []EntityWatchResult |
17 | +} |
18 | + |
19 | // AddRelation holds the parameters for making the AddRelation call. |
20 | // The endpoints specified are unordered. |
21 | type AddRelation struct { |
22 | |
23 | === modified file 'state/apiserver/client.go' |
24 | --- state/apiserver/client.go 2013-06-12 18:19:17 +0000 |
25 | +++ state/apiserver/client.go 2013-06-25 11:18:26 +0000 |
26 | @@ -37,7 +37,7 @@ |
27 | func (c *srvClient) WatchAll() (params.AllWatcherId, error) { |
28 | w := c.root.srv.state.Watch() |
29 | return params.AllWatcherId{ |
30 | - AllWatcherId: c.root.resources.register(w).id, |
31 | + AllWatcherId: c.root.resources.Register(w), |
32 | }, nil |
33 | } |
34 | |
35 | |
36 | === modified file 'state/apiserver/common/interfaces.go' |
37 | --- state/apiserver/common/interfaces.go 2013-06-22 08:49:38 +0000 |
38 | +++ state/apiserver/common/interfaces.go 2013-06-25 11:18:26 +0000 |
39 | @@ -27,3 +27,20 @@ |
40 | // a machine running the environment manager job. |
41 | AuthEnvironManager() bool |
42 | } |
43 | + |
44 | +// Resource represents any resource that should be cleaned up when an |
45 | +// API connection terminates. The Stop method will be called when |
46 | +// that happens. |
47 | +type Resource interface { |
48 | + Stop() error |
49 | +} |
50 | + |
51 | +// ResourceRegistry is an interface that allows the registration of |
52 | +// resources that will be cleaned up when an API connection |
53 | +// terminates. It is typically implemented by an API server. |
54 | +type ResourceRegistry interface { |
55 | + // Register registers the given resource. It returns a unique |
56 | + // identifier for the resource which can then be used in |
57 | + // subsequent API requests to refer to the resource. |
58 | + Register(resource Resource) string |
59 | +} |
60 | |
61 | === modified file 'state/apiserver/machine/machiner.go' |
62 | --- state/apiserver/machine/machiner.go 2013-06-22 08:49:38 +0000 |
63 | +++ state/apiserver/machine/machiner.go 2013-06-25 11:18:26 +0000 |
64 | @@ -9,18 +9,19 @@ |
65 | "launchpad.net/juju-core/state/apiserver/common" |
66 | ) |
67 | |
68 | -// Machiner implements the API used by the machiner worker. |
69 | +// MachinerAPI implements the API used by the machiner worker. |
70 | type MachinerAPI struct { |
71 | - st *state.State |
72 | - auth common.Authorizer |
73 | + st *state.State |
74 | + resources common.ResourceRegistry |
75 | + auth common.Authorizer |
76 | } |
77 | |
78 | // NewMachinerAPI creates a new instance of the Machiner API. |
79 | -func NewMachinerAPI(st *state.State, authorizer common.Authorizer) (*MachinerAPI, error) { |
80 | +func NewMachinerAPI(st *state.State, resources common.ResourceRegistry, authorizer common.Authorizer) (*MachinerAPI, error) { |
81 | if !authorizer.AuthMachineAgent() { |
82 | return nil, common.ErrPerm |
83 | } |
84 | - return &MachinerAPI{st, authorizer}, nil |
85 | + return &MachinerAPI{st, resources, authorizer}, nil |
86 | } |
87 | |
88 | // SetStatus sets the status of each given machine. |
89 | @@ -47,9 +48,28 @@ |
90 | } |
91 | |
92 | // Watch starts an EntityWatcher for each given machine. |
93 | -//func (m *MachinerAPI) Watch(args params.Machines) (params.MachinerWatchResults, error) { |
94 | -// TODO (dimitern) implement this once the watchers can handle bulk ops |
95 | -//} |
96 | +func (m *MachinerAPI) Watch(args params.Machines) (params.MachinesWatchResults, error) { |
97 | + result := params.MachinesWatchResults{ |
98 | + Results: make([]params.EntityWatchResult, len(args.Ids)), |
99 | + } |
100 | + if len(args.Ids) == 0 { |
101 | + return result, nil |
102 | + } |
103 | + for i, id := range args.Ids { |
104 | + machine, err := m.st.Machine(id) |
105 | + if err == nil { |
106 | + // Allow only for the owner agent. |
107 | + if !m.auth.AuthOwner(machine.Tag()) { |
108 | + err = common.ErrPerm |
109 | + } else { |
110 | + watcher := machine.Watch() |
111 | + result.Results[i].EntityWatcherId = m.resources.Register(watcher) |
112 | + } |
113 | + } |
114 | + result.Results[i].Error = common.ServerError(err) |
115 | + } |
116 | + return result, nil |
117 | +} |
118 | |
119 | // Life returns the lifecycle state of each given machine. |
120 | func (m *MachinerAPI) Life(args params.Machines) (params.MachinesLifeResults, error) { |
121 | |
122 | === modified file 'state/apiserver/machine/machiner_test.go' |
123 | --- state/apiserver/machine/machiner_test.go 2013-06-22 08:49:38 +0000 |
124 | +++ state/apiserver/machine/machiner_test.go 2013-06-25 11:18:26 +0000 |
125 | @@ -8,22 +8,41 @@ |
126 | "launchpad.net/juju-core/state" |
127 | "launchpad.net/juju-core/state/api" |
128 | "launchpad.net/juju-core/state/api/params" |
129 | + "launchpad.net/juju-core/state/apiserver/common" |
130 | "launchpad.net/juju-core/state/apiserver/machine" |
131 | + "strconv" |
132 | + "time" |
133 | ) |
134 | |
135 | type machinerSuite struct { |
136 | commonSuite |
137 | - machiner *machine.MachinerAPI |
138 | + |
139 | + resources fakeResourceRegistry |
140 | + machiner *machine.MachinerAPI |
141 | } |
142 | |
143 | var _ = Suite(&machinerSuite{}) |
144 | |
145 | +// fakeResourceRegistry implements the common.ResourceRegistry interface. |
146 | +type fakeResourceRegistry map[string]common.Resource |
147 | + |
148 | +func (registry fakeResourceRegistry) Register(resource common.Resource) string { |
149 | + id := strconv.Itoa(len(registry)) |
150 | + registry[id] = resource |
151 | + return id |
152 | +} |
153 | + |
154 | func (s *machinerSuite) SetUpTest(c *C) { |
155 | s.commonSuite.SetUpTest(c) |
156 | |
157 | + // Create the resource registry separately to track invocations to |
158 | + // Register. |
159 | + s.resources = make(fakeResourceRegistry) |
160 | + |
161 | // Create a machiner API for machine 1. |
162 | machiner, err := machine.NewMachinerAPI( |
163 | s.State, |
164 | + s.resources, |
165 | s.authorizer, |
166 | ) |
167 | c.Assert(err, IsNil) |
168 | @@ -39,7 +58,7 @@ |
169 | func (s *machinerSuite) TestMachinerFailsWithNonMachineAgentUser(c *C) { |
170 | anAuthorizer := s.authorizer |
171 | anAuthorizer.machineAgent = false |
172 | - aMachiner, err := machine.NewMachinerAPI(s.State, anAuthorizer) |
173 | + aMachiner, err := machine.NewMachinerAPI(s.State, s.resources, anAuthorizer) |
174 | c.Assert(err, NotNil) |
175 | c.Assert(aMachiner, IsNil) |
176 | c.Assert(err, ErrorMatches, "permission denied") |
177 | @@ -130,3 +149,37 @@ |
178 | c.Assert(err, IsNil) |
179 | c.Assert(s.machine1.Life(), Equals, state.Dead) |
180 | } |
181 | + |
182 | +func (s *machinerSuite) TestWatch(c *C) { |
183 | + c.Assert(s.resources, HasLen, 0) |
184 | + |
185 | + args := params.Machines{ |
186 | + Ids: []string{"1", "0", "42"}, |
187 | + } |
188 | + result, err := s.machiner.Watch(args) |
189 | + c.Assert(err, IsNil) |
190 | + c.Assert(result.Results, HasLen, 3) |
191 | + c.Assert(result.Results[0].Error, IsNil) |
192 | + s.assertError(c, result.Results[1].Error, api.CodeUnauthorized, "permission denied") |
193 | + s.assertError(c, result.Results[2].Error, api.CodeNotFound, "machine 42 not found") |
194 | + |
195 | + // Verify the resource was registered and stop when done |
196 | + c.Assert(s.resources, HasLen, 1) |
197 | + c.Assert(result.Results[0].EntityWatcherId, Equals, "0") |
198 | + resource := s.resources["0"] |
199 | + defer func() { |
200 | + err := resource.Stop() |
201 | + c.Assert(err, IsNil) |
202 | + }() |
203 | + |
204 | + // Check that the watcher returns an initial event |
205 | + channel := resource.(*state.EntityWatcher).Changes() |
206 | + // Should use helpers from state/watcher_test.go when generalised |
207 | + select { |
208 | + case _, ok := <-channel: |
209 | + c.Assert(ok, Equals, true) |
210 | + // The value is just an empty struct currently |
211 | + case <-time.After(50 * time.Millisecond): |
212 | + c.Fatal("timeout waiting for entity watcher") |
213 | + } |
214 | +} |
215 | |
216 | === modified file 'state/apiserver/resource.go' |
217 | --- state/apiserver/resource.go 2013-05-24 19:03:39 +0000 |
218 | +++ state/apiserver/resource.go 2013-06-25 11:18:26 +0000 |
219 | @@ -5,15 +5,11 @@ |
220 | |
221 | import ( |
222 | "launchpad.net/juju-core/log" |
223 | + "launchpad.net/juju-core/state/apiserver/common" |
224 | "strconv" |
225 | "sync" |
226 | ) |
227 | |
228 | -// resource represents the interface provided by state watchers and pingers. |
229 | -type resource interface { |
230 | - Stop() error |
231 | -} |
232 | - |
233 | // resources holds all the resources for a connection. |
234 | type resources struct { |
235 | mu sync.Mutex |
236 | @@ -25,7 +21,7 @@ |
237 | // Stop RPC method for all resources. |
238 | type srvResource struct { |
239 | rs *resources |
240 | - resource resource |
241 | + resource common.Resource |
242 | id string |
243 | } |
244 | |
245 | @@ -55,9 +51,9 @@ |
246 | return rs.rs[id] |
247 | } |
248 | |
249 | -// register records the given watcher and returns |
250 | -// a srvResource instance for it. |
251 | -func (rs *resources) register(r resource) *srvResource { |
252 | +// Register records the given watcher and returns |
253 | +// its id. |
254 | +func (rs *resources) Register(r common.Resource) string { |
255 | rs.mu.Lock() |
256 | defer rs.mu.Unlock() |
257 | rs.maxId++ |
258 | @@ -67,7 +63,7 @@ |
259 | resource: r, |
260 | } |
261 | rs.rs[sr.id] = sr |
262 | - return sr |
263 | + return sr.id |
264 | } |
265 | |
266 | func (rs *resources) stopAll() { |
267 | |
268 | === modified file 'state/apiserver/root.go' |
269 | --- state/apiserver/root.go 2013-06-22 08:49:38 +0000 |
270 | +++ state/apiserver/root.go 2013-06-25 11:18:26 +0000 |
271 | @@ -62,14 +62,15 @@ |
272 | return nil |
273 | } |
274 | |
275 | -// Machiner returns an object that provides access to the Machiner API. |
276 | -// The id argument is reserved for future use and must currently |
277 | -// be empty. |
278 | +// Machiner returns an object that provides access to the Machiner API |
279 | +// facade. The id argument is reserved for future use and currently |
280 | +// needs to be empty. |
281 | func (r *srvRoot) Machiner(id string) (*machine.MachinerAPI, error) { |
282 | if id != "" { |
283 | + // Safeguard id for possible future use. |
284 | return nil, common.ErrBadId |
285 | } |
286 | - return machine.NewMachinerAPI(r.srv.state, r) |
287 | + return machine.NewMachinerAPI(r.srv.state, r.resources, r) |
288 | } |
289 | |
290 | // MachineAgent returns an object that provides access to the machine |
Reviewers: mp+170586_ code.launchpad. net,
Message:
Please take a look.
Description:
state/apiserver: Implement Machiner.Watch
Adopting Dimiter's branch, with merge conflicts on trunk resolved.
See <https:/ /codereview. appspot. com/9937045/> for earlier discussion.
As I understand it, William would like a couple more tests, but all
other comments were addressed.
https:/ /code.launchpad .net/~gz/ juju-core/ 057-api- machiner- watch/+ merge/170586
(do not edit description out of merge proposal)
Please review this at https:/ /codereview. appspot. com/10439043/
Affected files: params/ params. go /client. go /common/ interfaces. go /machine/ machiner. go /machine/ machiner_ test.go /resource. go /root.go
A [revision details]
M state/api/
M state/apiserver
M state/apiserver
M state/apiserver
M state/apiserver
M state/apiserver
M state/apiserver