Merge lp:~thumper/juju-core/api-juju-run into lp:~go-bot/juju-core/trunk
- api-juju-run
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Tim Penhey |
Approved revision: | no longer in the source branch. |
Merged at revision: | 2192 |
Proposed branch: | lp:~thumper/juju-core/api-juju-run |
Merge into: | lp:~go-bot/juju-core/trunk |
Prerequisite: | lp:~thumper/juju-core/ssh-run-on-remote |
Diff against target: |
706 lines (+545/-12) 12 files modified
cmd/jujud/machine.go (+2/-1) provider/dummy/environs.go (+1/-1) state/api/client.go (+18/-0) state/api/params/internal.go (+29/-0) state/apiserver/apiserver.go (+9/-7) state/apiserver/client/client.go (+3/-1) state/apiserver/client/export_test.go (+2/-0) state/apiserver/client/run.go (+179/-0) state/apiserver/client/run_test.go (+299/-0) state/apiserver/login_test.go (+1/-0) state/apiserver/root.go (+1/-1) state/apiserver/server_test.go (+1/-1) |
To merge this branch: | bzr merge lp:~thumper/juju-core/api-juju-run |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju Engineering | Pending | ||
Review via email: mp+201103@code.launchpad.net |
Commit message
Provide api end points for juju-run
Adds two methods the the Client api end-point. One to run
commands against all machines, and another to run against
a specified collection of machines.
In order to have the api server able to ssh to the other
machines, it needed to know the location of the system
identity file, which is stored in the datadir, which means
that the agent config is needed by the api server. The
dummy provider is updated to pass a fake(ish) agent config
through to the apiserver.
Description of the change
Provide api end points for juju-run
Adds two methods the the Client api end-point. One to run
commands against all machines, and another to run against
a specified collection of machines.
In order to have the api server able to ssh to the other
machines, it needed to know the location of the system
identity file, which is stored in the datadir, which means
that the agent config is needed by the api server. The
dummy provider is updated to pass a fake(ish) agent config
through to the apiserver.
Tim Penhey (thumper) wrote : | # |
Ian Booth (wallyworld) wrote : | # |
I have an initial concern about the need for apiConfig - see comments in
code.
https:/
File cmd/jujud/
https:/
cmd/jujud/
This is clumsy - port, cert, key all are derived from agentConfig, aka
a.Conf.config
So we should include dataDir with the result of APIServerDetails and
pass that in
https:/
File provider/
https:/
provider/
apiserver.
[]byte(
As per my other comment, just pass in dataDir
https:/
File state/api/client.go (right):
https:/
state/api/
The above structs should be in params/internal.go right?
https:/
state/api/
Ditto
https:/
File state/apiserver
https:/
state/apiserver
Just dataDir would be better I think
https:/
File state/apiserver
https:/
state/apiserver
Should there be some sort of timeout? Will this hang if an individual
command hangs?
Tim Penhey (thumper) wrote : | # |
Please take a look.
Tim Penhey (thumper) wrote : | # |
https:/
File cmd/jujud/
https:/
cmd/jujud/
On 2014/01/10 03:19:03, wallyworld wrote:
> This is clumsy - port, cert, key all are derived from agentConfig, aka
> a.Conf.config
> So we should include dataDir with the result of APIServerDetails and
pass that
> in
Done.
https:/
File provider/
https:/
provider/
apiserver.
[]byte(
On 2014/01/10 03:19:03, wallyworld wrote:
> As per my other comment, just pass in dataDir
Done.
https:/
File state/api/client.go (right):
https:/
state/api/
On 2014/01/10 03:19:03, wallyworld wrote:
> The above structs should be in params/internal.go right?
Is there a standard? This needs to be accessed by both the api caller
and the apiserver. If internal is the place to put them, I'll move it,
but it wasn't clear when writing that they should go there.
https:/
File state/apiserver
https:/
state/apiserver
On 2014/01/10 03:19:03, wallyworld wrote:
> Just dataDir would be better I think
Done.
https:/
File state/apiserver
https:/
state/apiserver
On 2014/01/10 03:19:03, wallyworld wrote:
> Should there be some sort of timeout? Will this hang if an individual
command
> hangs?
It will wait based on the timeout passed through to
ExecuteCommandO
Ian Booth (wallyworld) wrote : | # |
LGTM with a couple of tweaks
https:/
File state/api/client.go (right):
https:/
state/api/
On 2014/01/10 04:24:39, thumper wrote:
> On 2014/01/10 03:19:03, wallyworld wrote:
> > The above structs should be in params/internal.go right?
> Is there a standard? This needs to be accessed by both the api caller
and the
> apiserver. If internal is the place to put them, I'll move it, but it
wasn't
> clear when writing that they should go there.
I'm not sure TBH. The main point of the comment was that we seem to put
params structs for most everything else in either params.go or
internal.go. Perhaps check with Dimiter or Roger before landing on the
best place to put them.
https:/
File state/apiserver
https:/
state/apiserver
*gc.C, cmd string) {
Perhaps a brief comment to explain how this method fits into the big
scheme of things - I figured it out but I needed to exercise some brain
cells.
https:/
state/apiserver
Agreed, this is unfortunate :-(
Go Bot (go-bot) wrote : | # |
The attempt to merge lp:~thumper/juju-core/api-juju-run into lp:juju-core failed. Below is the output from the failed tests.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
? launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
Go Bot (go-bot) wrote : | # |
The attempt to merge lp:~thumper/juju-core/api-juju-run into lp:juju-core failed. Below is the output from the failed tests.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
? launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
Go Bot (go-bot) wrote : | # |
The attempt to merge lp:~thumper/juju-core/api-juju-run into lp:juju-core failed. Below is the output from the failed tests.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
? launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
Go Bot (go-bot) wrote : | # |
The attempt to merge lp:~thumper/juju-core/api-juju-run into lp:juju-core failed. Below is the output from the failed tests.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
? launchpad.
? launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
? launchpad.
ok launchpad.
ok launchpad.
ok launchpad.
? launchpad.
? launchpad.
Preview Diff
1 | === modified file 'cmd/jujud/machine.go' |
2 | --- cmd/jujud/machine.go 2014-01-10 03:48:20 +0000 |
3 | +++ cmd/jujud/machine.go 2014-01-12 20:08:17 +0000 |
4 | @@ -310,7 +310,8 @@ |
5 | if len(cert) == 0 || len(key) == 0 { |
6 | return nil, &fatalError{"configuration does not have state server cert/key"} |
7 | } |
8 | - return apiserver.NewServer(st, fmt.Sprintf(":%d", port), cert, key) |
9 | + dataDir := a.Conf.config.DataDir() |
10 | + return apiserver.NewServer(st, fmt.Sprintf(":%d", port), cert, key, dataDir) |
11 | }) |
12 | runner.StartWorker("cleaner", func() (worker.Worker, error) { |
13 | return cleaner.NewCleaner(st), nil |
14 | |
15 | === modified file 'provider/dummy/environs.go' |
16 | --- provider/dummy/environs.go 2013-12-20 11:48:03 +0000 |
17 | +++ provider/dummy/environs.go 2014-01-12 20:08:17 +0000 |
18 | @@ -586,7 +586,7 @@ |
19 | if err != nil { |
20 | panic(err) |
21 | } |
22 | - estate.apiServer, err = apiserver.NewServer(st, "localhost:0", []byte(testing.ServerCert), []byte(testing.ServerKey)) |
23 | + estate.apiServer, err = apiserver.NewServer(st, "localhost:0", []byte(testing.ServerCert), []byte(testing.ServerKey), "") |
24 | if err != nil { |
25 | panic(err) |
26 | } |
27 | |
28 | === modified file 'state/api/client.go' |
29 | --- state/api/client.go 2014-01-07 07:35:12 +0000 |
30 | +++ state/api/client.go 2014-01-12 20:08:17 +0000 |
31 | @@ -9,6 +9,7 @@ |
32 | "io/ioutil" |
33 | "net/http" |
34 | "os" |
35 | + "time" |
36 | |
37 | "launchpad.net/juju-core/charm" |
38 | "launchpad.net/juju-core/constraints" |
39 | @@ -398,6 +399,23 @@ |
40 | return c.st.Call("Client", "", "SetEnvironAgentVersion", args, nil) |
41 | } |
42 | |
43 | +// RunOnAllMachines runs the command on all the machines with the specified |
44 | +// timeout. |
45 | +func (c *Client) RunOnAllMachines(commands string, timeout time.Duration) ([]params.RunResult, error) { |
46 | + var results params.RunResults |
47 | + args := params.RunParams{Commands: commands, Timeout: timeout} |
48 | + err := c.st.Call("Client", "", "RunOnAllMachines", args, &results) |
49 | + return results.Results, err |
50 | +} |
51 | + |
52 | +// Run the Commands specified on the machines identified through the ids |
53 | +// provided in the machines, services and units slices. |
54 | +func (c *Client) Run(run params.RunParams) ([]params.RunResult, error) { |
55 | + var results params.RunResults |
56 | + err := c.st.Call("Client", "", "Run", run, &results) |
57 | + return results.Results, err |
58 | +} |
59 | + |
60 | // DestroyEnvironment puts the environment into a "dying" state, |
61 | // and removes all non-manager machine instances. DestroyEnvironment |
62 | // will fail if there are any manually-provisioned non-manager machines |
63 | |
64 | === modified file 'state/api/params/internal.go' |
65 | --- state/api/params/internal.go 2013-12-17 09:49:11 +0000 |
66 | +++ state/api/params/internal.go 2014-01-12 20:08:17 +0000 |
67 | @@ -4,6 +4,9 @@ |
68 | package params |
69 | |
70 | import ( |
71 | + "time" |
72 | + |
73 | + "launchpad.net/juju-core/cmd" |
74 | "launchpad.net/juju-core/constraints" |
75 | "launchpad.net/juju-core/instance" |
76 | "launchpad.net/juju-core/tools" |
77 | @@ -509,3 +512,29 @@ |
78 | Error string `json:",omitempty"` |
79 | CharmURL string `json:",omitempty"` |
80 | } |
81 | + |
82 | +// RunParams is used to provide the parameters to the Run method. |
83 | +// Commands and Timeout are expected to have values, and one or more |
84 | +// values should be in the Machines, Services, or Units slices. |
85 | +type RunParams struct { |
86 | + Commands string |
87 | + Timeout time.Duration |
88 | + Machines []string |
89 | + Services []string |
90 | + Units []string |
91 | +} |
92 | + |
93 | +// RunResult contains the result from an individual run call on a machine. |
94 | +// UnitId is populated if the command was run inside the unit context. |
95 | +type RunResult struct { |
96 | + cmd.RemoteResponse |
97 | + MachineId string |
98 | + UnitId string |
99 | + Error string |
100 | +} |
101 | + |
102 | +// RunResults is used to return the slice of results. Api server side calls |
103 | +// need to return single structure values. |
104 | +type RunResults struct { |
105 | + Results []RunResult |
106 | +} |
107 | |
108 | === modified file 'state/apiserver/apiserver.go' |
109 | --- state/apiserver/apiserver.go 2013-12-11 15:19:25 +0000 |
110 | +++ state/apiserver/apiserver.go 2014-01-12 20:08:17 +0000 |
111 | @@ -25,16 +25,17 @@ |
112 | |
113 | // Server holds the server side of the API. |
114 | type Server struct { |
115 | - tomb tomb.Tomb |
116 | - wg sync.WaitGroup |
117 | - state *state.State |
118 | - addr net.Addr |
119 | + tomb tomb.Tomb |
120 | + wg sync.WaitGroup |
121 | + state *state.State |
122 | + addr net.Addr |
123 | + dataDir string |
124 | } |
125 | |
126 | // Serve serves the given state by accepting requests on the given |
127 | // listener, using the given certificate and key (in PEM format) for |
128 | // authentication. |
129 | -func NewServer(s *state.State, addr string, cert, key []byte) (*Server, error) { |
130 | +func NewServer(s *state.State, addr string, cert, key []byte, datadir string) (*Server, error) { |
131 | lis, err := net.Listen("tcp", addr) |
132 | if err != nil { |
133 | return nil, err |
134 | @@ -45,8 +46,9 @@ |
135 | return nil, err |
136 | } |
137 | srv := &Server{ |
138 | - state: s, |
139 | - addr: lis.Addr(), |
140 | + state: s, |
141 | + addr: lis.Addr(), |
142 | + dataDir: datadir, |
143 | } |
144 | // TODO(rog) check that *srvRoot is a valid type for using |
145 | // as an RPC server. |
146 | |
147 | === modified file 'state/apiserver/client/client.go' |
148 | --- state/apiserver/client/client.go 2014-01-07 07:35:12 +0000 |
149 | +++ state/apiserver/client/client.go 2014-01-12 20:08:17 +0000 |
150 | @@ -35,6 +35,7 @@ |
151 | auth common.Authorizer |
152 | resources *common.Resources |
153 | client *Client |
154 | + dataDir string |
155 | } |
156 | |
157 | // Client serves client-specific API methods. |
158 | @@ -43,11 +44,12 @@ |
159 | } |
160 | |
161 | // NewAPI creates a new instance of the Client API. |
162 | -func NewAPI(st *state.State, resources *common.Resources, authorizer common.Authorizer) *API { |
163 | +func NewAPI(st *state.State, resources *common.Resources, authorizer common.Authorizer, datadir string) *API { |
164 | r := &API{ |
165 | state: st, |
166 | auth: authorizer, |
167 | resources: resources, |
168 | + dataDir: datadir, |
169 | } |
170 | r.client = &Client{ |
171 | api: r, |
172 | |
173 | === modified file 'state/apiserver/client/export_test.go' |
174 | --- state/apiserver/client/export_test.go 2013-09-11 08:56:44 +0000 |
175 | +++ state/apiserver/client/export_test.go 2014-01-12 20:08:17 +0000 |
176 | @@ -4,3 +4,5 @@ |
177 | package client |
178 | |
179 | var ParseSettingsCompatible = parseSettingsCompatible |
180 | +var RemoteParamsForMachine = remoteParamsForMachine |
181 | +var GetAllUnitNames = getAllUnitNames |
182 | |
183 | === added file 'state/apiserver/client/run.go' |
184 | --- state/apiserver/client/run.go 1970-01-01 00:00:00 +0000 |
185 | +++ state/apiserver/client/run.go 2014-01-12 20:08:17 +0000 |
186 | @@ -0,0 +1,179 @@ |
187 | +// Copyright 2013 Canonical Ltd. |
188 | +// Licensed under the AGPLv3, see LICENCE file for details. |
189 | + |
190 | +package client |
191 | + |
192 | +import ( |
193 | + "fmt" |
194 | + "launchpad.net/juju-core/state" |
195 | + "launchpad.net/juju-core/utils" |
196 | + "path/filepath" |
197 | + "sort" |
198 | + "sync" |
199 | + "time" |
200 | + |
201 | + "launchpad.net/juju-core/environs/cloudinit" |
202 | + "launchpad.net/juju-core/instance" |
203 | + "launchpad.net/juju-core/state/api/params" |
204 | + "launchpad.net/juju-core/utils/set" |
205 | + "launchpad.net/juju-core/utils/ssh" |
206 | +) |
207 | + |
208 | +// remoteParamsForMachine returns a filled in RemoteExec instance |
209 | +// based on the machine, command and timeout params. If the machine |
210 | +// does not have an internal address, the Host is empty. This is caught |
211 | +// by the function that actually tries to execute the command. |
212 | +func remoteParamsForMachine(machine *state.Machine, command string, timeout time.Duration) *RemoteExec { |
213 | + // magic boolean parameters are bad :-( |
214 | + address := instance.SelectInternalAddress(machine.Addresses(), false) |
215 | + execParams := &RemoteExec{ |
216 | + ExecParams: ssh.ExecParams{ |
217 | + Command: command, |
218 | + Timeout: timeout, |
219 | + }, |
220 | + MachineId: machine.Id(), |
221 | + } |
222 | + if address != "" { |
223 | + execParams.Host = fmt.Sprintf("ubuntu@%s", address) |
224 | + } |
225 | + return execParams |
226 | +} |
227 | + |
228 | +// getAllUnitNames returns a sequence of valid Unit objects from state. If any |
229 | +// of the service names or unit names are not found, an error is returned. |
230 | +func getAllUnitNames(st *state.State, units, services []string) (result []*state.Unit, err error) { |
231 | + unitsSet := set.NewStrings(units...) |
232 | + for _, name := range services { |
233 | + service, err := st.Service(name) |
234 | + if err != nil { |
235 | + return nil, err |
236 | + } |
237 | + units, err := service.AllUnits() |
238 | + if err != nil { |
239 | + return nil, err |
240 | + } |
241 | + for _, unit := range units { |
242 | + unitsSet.Add(unit.Name()) |
243 | + } |
244 | + } |
245 | + for _, unitName := range unitsSet.Values() { |
246 | + unit, err := st.Unit(unitName) |
247 | + if err != nil { |
248 | + return nil, err |
249 | + } |
250 | + // We only operate on principal units, and only thise that have an |
251 | + // assigned machines. |
252 | + if unit.IsPrincipal() { |
253 | + if _, err := unit.AssignedMachineId(); err != nil { |
254 | + return nil, err |
255 | + } |
256 | + } else { |
257 | + return nil, fmt.Errorf("%s is not a principal unit", unit) |
258 | + } |
259 | + result = append(result, unit) |
260 | + } |
261 | + return result, nil |
262 | +} |
263 | + |
264 | +// Run the commands specified on the machines identified through the |
265 | +// list of machines, units and services. |
266 | +func (c *Client) Run(run params.RunParams) (results params.RunResults, err error) { |
267 | + units, err := getAllUnitNames(c.api.state, run.Units, run.Services) |
268 | + if err != nil { |
269 | + return results, err |
270 | + } |
271 | + // We want to create a RemoteExec for each unit and each machine. |
272 | + // If we have both a unit and a machine request, we run it twice, |
273 | + // once for the unit inside the exec context using juju-run, and |
274 | + // the other outside the context just using bash. |
275 | + var params []*RemoteExec |
276 | + var quotedCommands = utils.ShQuote(run.Commands) |
277 | + for _, unit := range units { |
278 | + // We know that the unit is both a principal unit, and that it has an |
279 | + // assigned machine. |
280 | + machineId, _ := unit.AssignedMachineId() |
281 | + machine, err := c.api.state.Machine(machineId) |
282 | + if err != nil { |
283 | + return results, err |
284 | + } |
285 | + command := fmt.Sprintf("juju-run %s %s", unit.Name(), quotedCommands) |
286 | + execParam := remoteParamsForMachine(machine, command, run.Timeout) |
287 | + execParam.UnitId = unit.Name() |
288 | + params = append(params, execParam) |
289 | + } |
290 | + for _, machineId := range run.Machines { |
291 | + machine, err := c.api.state.Machine(machineId) |
292 | + if err != nil { |
293 | + return results, err |
294 | + } |
295 | + execParam := remoteParamsForMachine(machine, run.Commands, run.Timeout) |
296 | + params = append(params, execParam) |
297 | + } |
298 | + return ParallelExecute(c.api.dataDir, params), nil |
299 | +} |
300 | + |
301 | +// RunOnAllMachines attempts to run the specified command on all the machines. |
302 | +func (c *Client) RunOnAllMachines(run params.RunParams) (params.RunResults, error) { |
303 | + machines, err := c.api.state.AllMachines() |
304 | + if err != nil { |
305 | + return params.RunResults{}, err |
306 | + } |
307 | + var params []*RemoteExec |
308 | + for _, machine := range machines { |
309 | + params = append(params, remoteParamsForMachine(machine, run.Commands, run.Timeout)) |
310 | + } |
311 | + return ParallelExecute(c.api.dataDir, params), nil |
312 | +} |
313 | + |
314 | +// RemoteExec extends the standard ssh.ExecParams by providing the machine and |
315 | +// perhaps the unit ids. These are then returned in the params.RunResult return |
316 | +// values. |
317 | +type RemoteExec struct { |
318 | + ssh.ExecParams |
319 | + MachineId string |
320 | + UnitId string |
321 | +} |
322 | + |
323 | +// ParallelExecute executes all of the requests defined in the params, |
324 | +// using the system identity stored in the dataDir. |
325 | +func ParallelExecute(dataDir string, runParams []*RemoteExec) params.RunResults { |
326 | + logger.Debugf("exec %#v", runParams) |
327 | + var outstanding sync.WaitGroup |
328 | + var lock sync.Mutex |
329 | + var result []params.RunResult |
330 | + identity := filepath.Join(dataDir, cloudinit.SystemIdentity) |
331 | + for _, param := range runParams { |
332 | + outstanding.Add(1) |
333 | + logger.Debugf("exec on %s: %#v", param.MachineId, *param) |
334 | + param.IdentityFile = identity |
335 | + go func(param *RemoteExec) { |
336 | + response, err := ssh.ExecuteCommandOnMachine(param.ExecParams) |
337 | + logger.Debugf("reponse from %s: %v (err:%v)", param.MachineId, response, err) |
338 | + execResponse := params.RunResult{ |
339 | + RemoteResponse: response, |
340 | + MachineId: param.MachineId, |
341 | + UnitId: param.UnitId, |
342 | + } |
343 | + if err != nil { |
344 | + execResponse.Error = fmt.Sprint(err) |
345 | + } |
346 | + |
347 | + lock.Lock() |
348 | + defer lock.Unlock() |
349 | + result = append(result, execResponse) |
350 | + outstanding.Done() |
351 | + }(param) |
352 | + } |
353 | + |
354 | + outstanding.Wait() |
355 | + sort.Sort(MachineOrder(result)) |
356 | + return params.RunResults{result} |
357 | +} |
358 | + |
359 | +// MachineOrder is used to provide the api to sort the results by the machine |
360 | +// id. |
361 | +type MachineOrder []params.RunResult |
362 | + |
363 | +func (a MachineOrder) Len() int { return len(a) } |
364 | +func (a MachineOrder) Swap(i, j int) { a[i], a[j] = a[j], a[i] } |
365 | +func (a MachineOrder) Less(i, j int) bool { return a[i].MachineId < a[j].MachineId } |
366 | |
367 | === added file 'state/apiserver/client/run_test.go' |
368 | --- state/apiserver/client/run_test.go 1970-01-01 00:00:00 +0000 |
369 | +++ state/apiserver/client/run_test.go 2014-01-12 20:08:17 +0000 |
370 | @@ -0,0 +1,299 @@ |
371 | +// Copyright 2013 Canonical Ltd. |
372 | +// Licensed under the AGPLv3, see LICENCE file for details. |
373 | + |
374 | +package client_test |
375 | + |
376 | +import ( |
377 | + "fmt" |
378 | + "io/ioutil" |
379 | + "os" |
380 | + "path/filepath" |
381 | + "time" |
382 | + |
383 | + gc "launchpad.net/gocheck" |
384 | + |
385 | + "launchpad.net/juju-core/cmd" |
386 | + "launchpad.net/juju-core/instance" |
387 | + "launchpad.net/juju-core/state" |
388 | + "launchpad.net/juju-core/state/api/params" |
389 | + "launchpad.net/juju-core/state/apiserver/client" |
390 | + "launchpad.net/juju-core/testing" |
391 | + jc "launchpad.net/juju-core/testing/checkers" |
392 | + "launchpad.net/juju-core/utils/ssh" |
393 | +) |
394 | + |
395 | +type runSuite struct { |
396 | + baseSuite |
397 | +} |
398 | + |
399 | +var _ = gc.Suite(&runSuite{}) |
400 | + |
401 | +func (s *runSuite) addMachine(c *gc.C) *state.Machine { |
402 | + machine, err := s.State.AddMachine("quantal", state.JobHostUnits) |
403 | + c.Assert(err, gc.IsNil) |
404 | + return machine |
405 | +} |
406 | + |
407 | +func (s *runSuite) addMachineWithAddress(c *gc.C, address string) *state.Machine { |
408 | + machine := s.addMachine(c) |
409 | + machine.SetAddresses([]instance.Address{instance.NewAddress(address)}) |
410 | + return machine |
411 | +} |
412 | + |
413 | +func (s *runSuite) TestRemoteParamsForMachinePopulates(c *gc.C) { |
414 | + machine := s.addMachine(c) |
415 | + result := client.RemoteParamsForMachine(machine, "command", time.Minute) |
416 | + c.Assert(result.Command, gc.Equals, "command") |
417 | + c.Assert(result.Timeout, gc.Equals, time.Minute) |
418 | + c.Assert(result.MachineId, gc.Equals, machine.Id()) |
419 | + // Now an empty host isn't particularly useful, but the machine doesn't |
420 | + // have an address to use. |
421 | + c.Assert(machine.Addresses(), gc.HasLen, 0) |
422 | + c.Assert(result.Host, gc.Equals, "") |
423 | +} |
424 | + |
425 | +func (s *runSuite) TestRemoteParamsForMachinePopulatesWithAddress(c *gc.C) { |
426 | + machine := s.addMachineWithAddress(c, "10.3.2.1") |
427 | + |
428 | + result := client.RemoteParamsForMachine(machine, "command", time.Minute) |
429 | + c.Assert(result.Command, gc.Equals, "command") |
430 | + c.Assert(result.Timeout, gc.Equals, time.Minute) |
431 | + c.Assert(result.MachineId, gc.Equals, machine.Id()) |
432 | + c.Assert(result.Host, gc.Equals, "ubuntu@10.3.2.1") |
433 | +} |
434 | + |
435 | +func (s *runSuite) addUnit(c *gc.C, service *state.Service) *state.Unit { |
436 | + unit, err := service.AddUnit() |
437 | + c.Assert(err, gc.IsNil) |
438 | + err = unit.AssignToNewMachine() |
439 | + c.Assert(err, gc.IsNil) |
440 | + mId, err := unit.AssignedMachineId() |
441 | + c.Assert(err, gc.IsNil) |
442 | + machine, err := s.State.Machine(mId) |
443 | + c.Assert(err, gc.IsNil) |
444 | + machine.SetAddresses([]instance.Address{instance.NewAddress("10.3.2.1")}) |
445 | + return unit |
446 | +} |
447 | + |
448 | +func (s *runSuite) TestGetAllUnitNames(c *gc.C) { |
449 | + charm := s.AddTestingCharm(c, "dummy") |
450 | + magic, err := s.State.AddService("magic", "user-admin", charm) |
451 | + s.addUnit(c, magic) |
452 | + s.addUnit(c, magic) |
453 | + |
454 | + notAssigned, err := s.State.AddService("not-assigned", "user-admin", charm) |
455 | + c.Assert(err, gc.IsNil) |
456 | + _, err = notAssigned.AddUnit() |
457 | + c.Assert(err, gc.IsNil) |
458 | + |
459 | + _, err = s.State.AddService("no-units", "user-admin", charm) |
460 | + c.Assert(err, gc.IsNil) |
461 | + |
462 | + for i, test := range []struct { |
463 | + message string |
464 | + expected []string |
465 | + units []string |
466 | + services []string |
467 | + error string |
468 | + }{{ |
469 | + message: "no units, expected nil slice", |
470 | + }, { |
471 | + message: "asking for a unit that isn't there", |
472 | + units: []string{"foo/0"}, |
473 | + error: `unit "foo/0" not found`, |
474 | + }, { |
475 | + message: "asking for a service that isn't there", |
476 | + services: []string{"foo"}, |
477 | + error: `service "foo" not found`, |
478 | + }, { |
479 | + message: "service with no units is not really an error", |
480 | + services: []string{"no-units"}, |
481 | + }, { |
482 | + message: "A service with units not assigned is an error", |
483 | + services: []string{"not-assigned"}, |
484 | + error: `unit "not-assigned/0" is not assigned to a machine`, |
485 | + }, { |
486 | + message: "A service with units", |
487 | + services: []string{"magic"}, |
488 | + expected: []string{"magic/0", "magic/1"}, |
489 | + }, { |
490 | + message: "Asking for just a unit", |
491 | + units: []string{"magic/0"}, |
492 | + expected: []string{"magic/0"}, |
493 | + }, { |
494 | + message: "Asking for a unit, and the service", |
495 | + services: []string{"magic"}, |
496 | + units: []string{"magic/0"}, |
497 | + expected: []string{"magic/0", "magic/1"}, |
498 | + }} { |
499 | + c.Log(fmt.Sprintf("%v: %s", i, test.message)) |
500 | + result, err := client.GetAllUnitNames(s.State, test.units, test.services) |
501 | + if test.error == "" { |
502 | + c.Check(err, gc.IsNil) |
503 | + var units []string |
504 | + for _, unit := range result { |
505 | + units = append(units, unit.Name()) |
506 | + } |
507 | + c.Check(units, jc.SameContents, test.expected) |
508 | + } else { |
509 | + c.Check(err, gc.ErrorMatches, test.error) |
510 | + } |
511 | + } |
512 | +} |
513 | + |
514 | +func (s *runSuite) mockSSH(c *gc.C, cmd string) { |
515 | + testbin := c.MkDir() |
516 | + fakessh := filepath.Join(testbin, "ssh") |
517 | + newPath := testbin + ":" + os.Getenv("PATH") |
518 | + s.PatchEnvironment("PATH", newPath) |
519 | + err := ioutil.WriteFile(fakessh, []byte(cmd), 0755) |
520 | + c.Assert(err, gc.IsNil) |
521 | +} |
522 | + |
523 | +func (s *runSuite) TestParallelExecuteErrorsOnBlankHost(c *gc.C) { |
524 | + s.mockSSH(c, echoInputShowArgs) |
525 | + |
526 | + params := []*client.RemoteExec{ |
527 | + &client.RemoteExec{ |
528 | + ExecParams: ssh.ExecParams{ |
529 | + Command: "foo", |
530 | + Timeout: testing.ShortWait, |
531 | + }, |
532 | + }, |
533 | + } |
534 | + |
535 | + runResults := client.ParallelExecute("/some/dir", params) |
536 | + c.Assert(runResults.Results, gc.HasLen, 1) |
537 | + result := runResults.Results[0] |
538 | + c.Assert(result.Error, gc.Equals, "missing host address") |
539 | +} |
540 | + |
541 | +func (s *runSuite) TestParallelExecuteAddsIdentity(c *gc.C) { |
542 | + s.mockSSH(c, echoInputShowArgs) |
543 | + |
544 | + params := []*client.RemoteExec{ |
545 | + &client.RemoteExec{ |
546 | + ExecParams: ssh.ExecParams{ |
547 | + Host: "localhost", |
548 | + Command: "foo", |
549 | + Timeout: testing.ShortWait, |
550 | + }, |
551 | + }, |
552 | + } |
553 | + |
554 | + runResults := client.ParallelExecute("/some/dir", params) |
555 | + c.Assert(runResults.Results, gc.HasLen, 1) |
556 | + result := runResults.Results[0] |
557 | + c.Assert(result.Error, gc.Equals, "") |
558 | + c.Assert(string(result.Stderr), jc.Contains, "-i /some/dir/system-identity") |
559 | +} |
560 | + |
561 | +func (s *runSuite) TestParallelExecuteCopiesAcrossMachineAndUnit(c *gc.C) { |
562 | + s.mockSSH(c, echoInputShowArgs) |
563 | + |
564 | + params := []*client.RemoteExec{ |
565 | + &client.RemoteExec{ |
566 | + ExecParams: ssh.ExecParams{ |
567 | + Host: "localhost", |
568 | + Command: "foo", |
569 | + Timeout: testing.ShortWait, |
570 | + }, |
571 | + MachineId: "machine-id", |
572 | + UnitId: "unit-id", |
573 | + }, |
574 | + } |
575 | + |
576 | + runResults := client.ParallelExecute("/some/dir", params) |
577 | + c.Assert(runResults.Results, gc.HasLen, 1) |
578 | + result := runResults.Results[0] |
579 | + c.Assert(result.Error, gc.Equals, "") |
580 | + c.Assert(result.MachineId, gc.Equals, "machine-id") |
581 | + c.Assert(result.UnitId, gc.Equals, "unit-id") |
582 | +} |
583 | + |
584 | +func (s *runSuite) TestRunOnAllMachines(c *gc.C) { |
585 | + // Make three machines. |
586 | + s.addMachineWithAddress(c, "10.3.2.1") |
587 | + s.addMachineWithAddress(c, "10.3.2.2") |
588 | + s.addMachineWithAddress(c, "10.3.2.3") |
589 | + |
590 | + s.mockSSH(c, echoInput) |
591 | + |
592 | + // hmm... this seems to be going through the api client, and from there |
593 | + // through to the apiserver implementation. Not ideal, but it is how the |
594 | + // other client tests are written. |
595 | + client := s.APIState.Client() |
596 | + results, err := client.RunOnAllMachines("hostname", testing.ShortWait) |
597 | + c.Assert(err, gc.IsNil) |
598 | + c.Assert(results, gc.HasLen, 3) |
599 | + var expectedResults []params.RunResult |
600 | + for i := 0; i < 3; i++ { |
601 | + expectedResults = append(expectedResults, |
602 | + params.RunResult{ |
603 | + RemoteResponse: cmd.RemoteResponse{Stdout: []byte("hostname\n")}, |
604 | + MachineId: fmt.Sprint(i), |
605 | + }) |
606 | + } |
607 | + |
608 | + c.Assert(results, jc.DeepEquals, expectedResults) |
609 | +} |
610 | + |
611 | +func (s *runSuite) TestRunMachineAndService(c *gc.C) { |
612 | + // Make three machines. |
613 | + s.addMachineWithAddress(c, "10.3.2.1") |
614 | + |
615 | + charm := s.AddTestingCharm(c, "dummy") |
616 | + magic, err := s.State.AddService("magic", "user-admin", charm) |
617 | + s.addUnit(c, magic) |
618 | + s.addUnit(c, magic) |
619 | + |
620 | + s.mockSSH(c, echoInput) |
621 | + |
622 | + // hmm... this seems to be going through the api client, and from there |
623 | + // through to the apiserver implementation. Not ideal, but it is how the |
624 | + // other client tests are written. |
625 | + client := s.APIState.Client() |
626 | + results, err := client.Run( |
627 | + params.RunParams{ |
628 | + Commands: "hostname", |
629 | + Timeout: testing.ShortWait, |
630 | + Machines: []string{"0"}, |
631 | + Services: []string{"magic"}, |
632 | + }) |
633 | + c.Assert(err, gc.IsNil) |
634 | + c.Assert(results, gc.HasLen, 3) |
635 | + expectedResults := []params.RunResult{ |
636 | + params.RunResult{ |
637 | + RemoteResponse: cmd.RemoteResponse{Stdout: []byte("hostname\n")}, |
638 | + MachineId: "0", |
639 | + }, |
640 | + params.RunResult{ |
641 | + RemoteResponse: cmd.RemoteResponse{Stdout: []byte("juju-run magic/0 'hostname'\n")}, |
642 | + MachineId: "1", |
643 | + UnitId: "magic/0", |
644 | + }, |
645 | + params.RunResult{ |
646 | + RemoteResponse: cmd.RemoteResponse{Stdout: []byte("juju-run magic/1 'hostname'\n")}, |
647 | + MachineId: "2", |
648 | + UnitId: "magic/1", |
649 | + }, |
650 | + } |
651 | + |
652 | + c.Assert(results, jc.DeepEquals, expectedResults) |
653 | +} |
654 | + |
655 | +var echoInputShowArgs = `#!/bin/bash |
656 | +# Write the args to stderr |
657 | +echo "$*" >&2 |
658 | +# And echo stdin to stdout |
659 | +while read line |
660 | +do echo $line |
661 | +done <&0 |
662 | +` |
663 | + |
664 | +var echoInput = `#!/bin/bash |
665 | +# And echo stdin to stdout |
666 | +while read line |
667 | +do echo $line |
668 | +done <&0 |
669 | +` |
670 | |
671 | === modified file 'state/apiserver/login_test.go' |
672 | --- state/apiserver/login_test.go 2013-12-12 07:58:43 +0000 |
673 | +++ state/apiserver/login_test.go 2014-01-12 20:08:17 +0000 |
674 | @@ -50,6 +50,7 @@ |
675 | "localhost:0", |
676 | []byte(coretesting.ServerCert), |
677 | []byte(coretesting.ServerKey), |
678 | + "", |
679 | ) |
680 | c.Assert(err, gc.IsNil) |
681 | info := &api.Info{ |
682 | |
683 | === modified file 'state/apiserver/root.go' |
684 | --- state/apiserver/root.go 2013-12-11 07:40:14 +0000 |
685 | +++ state/apiserver/root.go 2014-01-12 20:08:17 +0000 |
686 | @@ -66,7 +66,7 @@ |
687 | logger.Errorf("error closing the RPC connection: %v", err) |
688 | } |
689 | } |
690 | - r.clientAPI.API = client.NewAPI(r.srv.state, r.resources, r) |
691 | + r.clientAPI.API = client.NewAPI(r.srv.state, r.resources, r, r.srv.dataDir) |
692 | r.pingTimeout = newPingTimeout(action, maxPingInterval) |
693 | return r |
694 | } |
695 | |
696 | === modified file 'state/apiserver/server_test.go' |
697 | --- state/apiserver/server_test.go 2013-12-12 17:23:45 +0000 |
698 | +++ state/apiserver/server_test.go 2014-01-12 20:08:17 +0000 |
699 | @@ -36,7 +36,7 @@ |
700 | func (s *serverSuite) TestStop(c *gc.C) { |
701 | // Start our own instance of the server so we have |
702 | // a handle on it to stop it. |
703 | - srv, err := apiserver.NewServer(s.State, "localhost:0", []byte(coretesting.ServerCert), []byte(coretesting.ServerKey)) |
704 | + srv, err := apiserver.NewServer(s.State, "localhost:0", []byte(coretesting.ServerCert), []byte(coretesting.ServerKey), "") |
705 | c.Assert(err, gc.IsNil) |
706 | defer srv.Stop() |
707 |
Reviewers: mp+201103_ code.launchpad. net,
Message:
Please take a look.
Description:
Provide api end points for juju-run
Adds two methods the the Client api end-point. One to run
commands against all machines, and another to run against
a specified collection of machines.
In order to have the api server able to ssh to the other
machines, it needed to know the location of the system
identity file, which is stored in the datadir, which means
that the agent config is needed by the api server. The
dummy provider is updated to pass a fake(ish) agent config
through to the apiserver.
https:/ /code.launchpad .net/~thumper/ juju-core/ api-juju- run/+merge/ 201103
Requires: /code.launchpad .net/~thumper/ juju-core/ ssh-run- on-remote/ +merge/ 201095
https:/
(do not edit description out of merge proposal)
Please review this at https:/ /codereview. appspot. com/49960045/
Affected files (+565, -19 lines): machine. go dummy/environs. go /apiserver. go /client/ client. go /client/ export_ test.go /client/ run.go /client/ run_test. go /login_ test.go /root.go /server_ test.go
A [revision details]
M cmd/jujud/
M provider/
M state/api/client.go
M state/apiserver
M state/apiserver
M state/apiserver
A state/apiserver
A state/apiserver
M state/apiserver
M state/apiserver
M state/apiserver