Merge lp:~frankban/juju-quickstart/new-bootstrap-strategy into lp:juju-quickstart
- new-bootstrap-strategy
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 105 |
Proposed branch: | lp:~frankban/juju-quickstart/new-bootstrap-strategy |
Merge into: | lp:juju-quickstart |
Diff against target: |
1663 lines (+770/-511) 11 files modified
HACKING.rst (+2/-1) quickstart/app.py (+80/-44) quickstart/manage.py (+38/-17) quickstart/netutils.py (+99/-0) quickstart/tests/helpers.py (+19/-5) quickstart/tests/test_app.py (+195/-147) quickstart/tests/test_manage.py (+134/-122) quickstart/tests/test_netutils.py (+201/-0) quickstart/tests/test_utils.py (+0/-119) quickstart/utils.py (+0/-54) quickstart/watchers.py (+2/-2) |
To merge this branch: | bzr merge lp:~frankban/juju-quickstart/new-bootstrap-strategy |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju GUI Hackers | Pending | ||
Review via email: mp+241553@code.launchpad.net |
Commit message
Description of the change
New bootstrap strategy.
This is a massive branch: my apologies.
But:
- a new module has been created (netutils), so there
are license headers and moreover some existing
functions (with their tests) have been just moved
from utils.py and must not be re-reviewed.
The only new function there is check_listening.
- most of the code are tests: we reached 700 unit
tests yay!
This branch changes the way quickstart is run on an
existing environment: instead of always trying to
bootstrap, it looks for the jenv file for the current
environment, and, if present, it retrieves the API URL
from there. As a consequence, quickstart uses Juju
in a less expensive way, and it's also faster when
invoked on a bootstrapped environment.
Split the app.bootstrap function to two new
functions: app.bootstrap and app.status. The intent
is to make them more reusable and more easy to test.
Fix a subtle bug never reported and not easy to hit:
the environment type of an existing environment is
now retrieved from the jenv, rather than relying on
what's stored in the environments.yaml file.
Also reorganized the tests for manage.run: over
time they ended up failing to achieve their goal
of describing how the application is run, and
become not really easy to update and change.
Now this situation should be improved.
The little change to the HACKING file is to make
the rst to render correctly on sublime text.
Tests: `make check`.
QA: run quickstart as usual on local and ec2.
Run quickstart again on an already bootstrapped
environment (local and ec2). You should no longer
see the "bootstrapping environment" message.
Instead, a more correct "reusing the already
bootstrapped..." message is displayed.
Also this should feel quicker, especially on ec2.
Thank you!
Francesco Banconi (frankban) wrote : | # |
Brad Crittenden (bac) wrote : | # |
Code LGTM with minor comments. Will QA now.
https:/
File quickstart/app.py (right):
https:/
quickstart/
required at this point.
This is no different from the calling perspective than the 'not already
bootstrapped' situation, right?
If so, perhaps that should be noted in the docstring for this function.
https:/
File quickstart/
https:/
quickstart/
What is the scenario for api_url being None but it is already
bootstrapped? Someone deleted the jenv file out from under juju?
https:/
File quickstart/
https:/
quickstart/
Make timeout an arg with default = 3?
Richard Harding (rharding) wrote : | # |
LGTM thank you for the updates.
https:/
File quickstart/app.py (right):
https:/
quickstart/
'state-servers')
do you know how this list gets updated from Juju? In an HA situation
does juju rewrite the jenv file?
https:/
File quickstart/
https:/
quickstart/
On 2014/11/12 14:48:25, bac wrote:
> What is the scenario for api_url being None but it is already
bootstrapped?
> Someone deleted the jenv file out from under juju?
That has happened and there's been bugs/work around the scenario. It's a
good one to watch for.
Brad Crittenden (bac) wrote : | # |
Francesco Banconi (frankban) wrote : | # |
Thanks for the reviews, really good stuff!
https:/
File quickstart/app.py (right):
https:/
quickstart/
'state-servers')
On 2014/11/12 15:30:44, rharding wrote:
> do you know how this list gets updated from Juju? In an HA situation
does juju
> rewrite the jenv file?
I don't know that for sure: it could be the case when you set HA from
the CLI. I would be surprised if the jenv is updated when you will set
HA using the API. In case of stale addresses, we just fall back to the
old bootstrap strategy.
https:/
quickstart/
required at this point.
On 2014/11/12 14:48:25, bac wrote:
> This is no different from the calling perspective than the 'not
already
> bootstrapped' situation, right?
> If so, perhaps that should be noted in the docstring for this
function.
Done.
https:/
File quickstart/
https:/
quickstart/
On 2014/11/12 14:48:25, bac wrote:
> Make timeout an arg with default = 3?
Done.
- 111. By Francesco Banconi
-
Changes as per review.
Francesco Banconi (frankban) wrote : | # |
*** Submitted:
New bootstrap strategy.
This is a massive branch: my apologies.
But:
- a new module has been created (netutils), so there
are license headers and moreover some existing
functions (with their tests) have been just moved
from utils.py and must not be re-reviewed.
The only new function there is check_listening.
- most of the code are tests: we reached 700 unit
tests yay!
This branch changes the way quickstart is run on an
existing environment: instead of always trying to
bootstrap, it looks for the jenv file for the current
environment, and, if present, it retrieves the API URL
from there. As a consequence, quickstart uses Juju
in a less expensive way, and it's also faster when
invoked on a bootstrapped environment.
Split the app.bootstrap function to two new
functions: app.bootstrap and app.status. The intent
is to make them more reusable and more easy to test.
Fix a subtle bug never reported and not easy to hit:
the environment type of an existing environment is
now retrieved from the jenv, rather than relying on
what's stored in the environments.yaml file.
Also reorganized the tests for manage.run: over
time they ended up failing to achieve their goal
of describing how the application is run, and
become not really easy to update and change.
Now this situation should be improved.
The little change to the HACKING file is to make
the rst to render correctly on sublime text.
Tests: `make check`.
QA: run quickstart as usual on local and ec2.
Run quickstart again on an already bootstrapped
environment (local and ec2). You should no longer
see the "bootstrapping environment" message.
Instead, a more correct "reusing the already
bootstrapped..." message is displayed.
Also this should feel quicker, especially on ec2.
Thank you!
R=bac, rharding
CC=
https:/
Preview Diff
1 | === modified file 'HACKING.rst' |
2 | --- HACKING.rst 2014-11-11 14:15:47 +0000 |
3 | +++ HACKING.rst 2014-11-12 16:41:04 +0000 |
4 | @@ -152,7 +152,8 @@ |
5 | |
6 | * Verify an environment that has already been bootstrapped is recogized and |
7 | the GUI is deployed. This test also shows that a remote bundle is properly |
8 | - deployed:: |
9 | + deployed |
10 | +:: |
11 | |
12 | juju bootstrap -e local |
13 | juju quickstart -e local bundle:mediawiki/single |
14 | |
15 | === modified file 'quickstart/app.py' |
16 | --- quickstart/app.py 2014-11-11 13:21:57 +0000 |
17 | +++ quickstart/app.py 2014-11-12 16:41:04 +0000 |
18 | @@ -30,6 +30,7 @@ |
19 | |
20 | from quickstart import ( |
21 | juju, |
22 | + netutils, |
23 | platform_support, |
24 | settings, |
25 | ssh, |
26 | @@ -175,28 +176,55 @@ |
27 | raise ProgramExit(bytes(err)) |
28 | |
29 | |
30 | +def check_bootstrapped(env_name): |
31 | + """Check if the environment named env_name is already bootstrapped. |
32 | + |
33 | + If so, return the environment API URL to be used to connect to the Juju API |
34 | + server. If not already bootstrapped, or if the API URL cannot be retrieved, |
35 | + return None. |
36 | + """ |
37 | + if not jenv.exists(env_name): |
38 | + return None |
39 | + # Retrieve the Juju API addresses from the jenv file. |
40 | + try: |
41 | + candidates = jenv.get_value(env_name, 'state-servers') |
42 | + except ValueError as err: |
43 | + logging.warn(b'cannot retrieve the Juju API URL: {}'.format(err)) |
44 | + return None |
45 | + # Look for a reachable API URL. |
46 | + if not candidates: |
47 | + logging.warn('cannot retrieve the Juju API URL: no addresses found') |
48 | + return None |
49 | + for candidate in candidates: |
50 | + error = netutils.check_listening(candidate) |
51 | + if error is None: |
52 | + # Juju API URL found. |
53 | + return 'wss://{}'.format(candidate) |
54 | + logging.debug(error) |
55 | + logging.warn( |
56 | + 'cannot retrieve the Juju API URL: cannot connect to any of the ' |
57 | + 'following addresses: {}'.format(', '.join(candidates))) |
58 | + return None |
59 | + |
60 | + |
61 | def bootstrap( |
62 | env_name, juju_command, debug=False, upload_tools=False, |
63 | upload_series=None, constraints=None): |
64 | """Bootstrap the Juju environment with the given name. |
65 | |
66 | + Return a flag indicating whether the environment was already bootstrapped. |
67 | + |
68 | Do not bootstrap the environment if already bootstrapped. |
69 | - |
70 | - Return a tuple (already_bootstrapped, series) in which: |
71 | - - already_bootstrapped indicates whether the environment was already |
72 | - bootstrapped; |
73 | - - series is the bootstrap node Ubuntu series. |
74 | - |
75 | - The is_local argument indicates whether the environment is configured to |
76 | - use the local provider. If so, sudo privileges are requested in order to |
77 | - bootstrap the environment. |
78 | - |
79 | - If debug is True and the environment not bootstrapped, execute the |
80 | - bootstrap command passing the --debug flag. |
81 | + If the environment is not bootstrapped, execute the bootstrap command with |
82 | + the given juju_command, debug, upload_tools, upload_series and constraints |
83 | + arguments. |
84 | + |
85 | + When the function exists the Juju environment is bootstrapped, but we don't |
86 | + know if it is ready yet. For this reason, a call to status() is usually |
87 | + required at that point. |
88 | |
89 | Raise a ProgramExit if any error occurs in the bootstrap process. |
90 | """ |
91 | - already_bootstrapped = False |
92 | cmd = [juju_command, 'bootstrap', '-e', env_name] |
93 | if debug: |
94 | cmd.append('--debug') |
95 | @@ -207,34 +235,28 @@ |
96 | if constraints is not None: |
97 | cmd.extend(['--constraints', constraints]) |
98 | retcode, _, error = utils.call(*cmd) |
99 | - if retcode: |
100 | - # XXX frankban 2013-11-13 bug 1252322: the check below is weak. We are |
101 | - # relying on an error message in order to decide if the environment is |
102 | - # already bootstrapped. Other possibilities include checking if the |
103 | - # jenv file is present (in ~/.juju/environments/) and, if so, check the |
104 | - # juju status. Unfortunately this is also prone to errors, because a |
105 | - # jenv file can be there but the environment not really bootstrapped or |
106 | - # functional (e.g. sync-tools was used, or a previous bootstrap failed, |
107 | - # or the user terminated machines from the ec2 panel, etc.). Moreover |
108 | - # jenv files seems to be an internal juju-core detail. Definitely we |
109 | - # need to find a better way, but for now the "asking forgiveness" |
110 | - # approach feels like the best compromise we have. Also note that, |
111 | - # rather than comparing the expected error with the obtained one, we |
112 | - # search in the error in order to support bootstrap --debug. |
113 | - if 'environment is already bootstrapped' not in error: |
114 | - # We exit if the error is not "already bootstrapped". |
115 | - raise ProgramExit(error) |
116 | - # Juju is bootstrapped, but we don't know if it is ready yet. Fall |
117 | - # through to the next block for that check. |
118 | - already_bootstrapped = True |
119 | - print('reusing the already bootstrapped {} environment'.format( |
120 | - env_name)) |
121 | - # Call "juju status" multiple times until the bootstrap node is ready. |
122 | - # Exit with an error if the agent is not ready after ten minutes. |
123 | - # Note: when using the local provider, calling "juju status" is very fast, |
124 | - # but e.g. on ec2 the first call (right after "bootstrap") can take |
125 | - # several minutes, and subsequent calls are relatively fast (seconds). |
126 | - print('retrieving the environment status') |
127 | + if not retcode: |
128 | + return False |
129 | + # XXX frankban 2013-11-13 bug 1252322: the check below is weak. We are |
130 | + # relying on an error message in order to decide if the environment is |
131 | + # already bootstrapped. Also note that, rather than comparing the expected |
132 | + # error with the obtained one, we search in the error in order to support |
133 | + # bootstrap --debug. |
134 | + if 'environment is already bootstrapped' not in error: |
135 | + # We exit if the error is not "already bootstrapped". |
136 | + raise ProgramExit(error) |
137 | + # Juju is already bootstrapped. |
138 | + return True |
139 | + |
140 | + |
141 | +def status(env_name, juju_command): |
142 | + """Call "juju status" multiple times until the bootstrap node is ready. |
143 | + |
144 | + Return the bootstrap node series of the Juju environment. |
145 | + |
146 | + Raise a ProgramExit if the agent is not ready after ten minutes or if the |
147 | + agent is in an error state. |
148 | + """ |
149 | timeout = time.time() + (60*10) |
150 | while time.time() < timeout: |
151 | retcode, output, error = utils.call( |
152 | @@ -247,8 +269,7 @@ |
153 | except ValueError: |
154 | continue |
155 | if agent_state == 'started': |
156 | - series = utils.get_bootstrap_node_series(output) |
157 | - return already_bootstrapped, series |
158 | + return utils.get_bootstrap_node_series(output) |
159 | # If the agent is in an error state, there is nothing we can do, and |
160 | # it's not useful to keep trying. |
161 | if agent_state == 'error': |
162 | @@ -257,6 +278,21 @@ |
163 | raise ProgramExit('the state server is not ready:\n{}'.format(details)) |
164 | |
165 | |
166 | +def get_env_type(env_name): |
167 | + """Return the Juju environment type for the given environment name. |
168 | + |
169 | + Since the environment type is retrieved by parsing the jenv file, the |
170 | + environment must be already bootstrapped. |
171 | + |
172 | + Raise a ProgramExit if the environment type cannot be retrieved. |
173 | + """ |
174 | + try: |
175 | + return jenv.get_value(env_name, 'bootstrap-config', 'type') |
176 | + except ValueError as err: |
177 | + msg = b'cannot retrieve environment type: {}'.format(err) |
178 | + raise ProgramExit(msg) |
179 | + |
180 | + |
181 | def get_admin_secret(env_name): |
182 | """Return the Juju admin secret for the given environment name. |
183 | |
184 | @@ -385,7 +421,7 @@ |
185 | series = settings.JUJU_GUI_SUPPORTED_SERIES[-1] |
186 | try: |
187 | # Try to get the charm URL from charmworld. |
188 | - charm_url = utils.get_charm_url(series) |
189 | + charm_url = netutils.get_charm_url(series) |
190 | except (IOError, ValueError) as err: |
191 | # Fall back to the default URL for the current series. |
192 | msg = 'unable to retrieve the {} charm URL from the API: {}' |
193 | |
194 | === modified file 'quickstart/manage.py' |
195 | --- quickstart/manage.py 2014-11-11 11:10:33 +0000 |
196 | +++ quickstart/manage.py 2014-11-12 16:41:04 +0000 |
197 | @@ -32,6 +32,7 @@ |
198 | import quickstart |
199 | from quickstart import ( |
200 | app, |
201 | + netutils, |
202 | packaging, |
203 | platform_support, |
204 | settings, |
205 | @@ -110,7 +111,7 @@ |
206 | if bundle.startswith('http://') or bundle.startswith('https://'): |
207 | # Load the bundle from a remote URL. |
208 | try: |
209 | - bundle_yaml = utils.urlread(bundle) |
210 | + bundle_yaml = netutils.urlread(bundle) |
211 | except IOError as err: |
212 | return parser.error('unable to open bundle URL: {}'.format(err)) |
213 | else: |
214 | @@ -504,32 +505,52 @@ |
215 | logging.debug('ensuring SSH keys are available') |
216 | app.ensure_ssh_keys() |
217 | |
218 | - print('bootstrapping the {} environment (type: {})'.format( |
219 | - options.env_name, options.env_type)) |
220 | - if options.env_type == 'local': |
221 | - # If this is a local environment, notify the user that "sudo" will be |
222 | - # required by Juju to bootstrap the application. |
223 | - print('sudo privileges will be required to bootstrap the environment') |
224 | - |
225 | - already_bootstrapped, bootstrap_node_series = app.bootstrap( |
226 | - options.env_name, juju_command, |
227 | - debug=options.debug, |
228 | - upload_tools=options.upload_tools, |
229 | - upload_series=options.upload_series, |
230 | - constraints=options.constraints) |
231 | + # Bootstrap the Juju environment or reuse an already bootstrapped one. |
232 | + already_bootstrapped = True |
233 | + env_type = options.env_type |
234 | + api_url = app.check_bootstrapped(options.env_name) |
235 | + if api_url is None: |
236 | + print('bootstrapping the {} environment (type: {})'.format( |
237 | + options.env_name, env_type)) |
238 | + if env_type == 'local': |
239 | + # If this is a local environment, notify the user that "sudo" will |
240 | + # be required by Juju to bootstrap the environment. |
241 | + print('sudo privileges will be required to bootstrap the ' |
242 | + 'environment') |
243 | + already_bootstrapped = app.bootstrap( |
244 | + options.env_name, juju_command, |
245 | + debug=options.debug, |
246 | + upload_tools=options.upload_tools, |
247 | + upload_series=options.upload_series, |
248 | + constraints=options.constraints) |
249 | + if already_bootstrapped: |
250 | + # Retrieve the environment type from the jenv file: it may be different |
251 | + # from the one declared on the environments.yaml file. |
252 | + env_type = app.get_env_type(options.env_name) |
253 | + print('reusing the already bootstrapped {} environment ' |
254 | + '(type: {})'.format(options.env_name, env_type)) |
255 | + |
256 | + # Retrieve the environment status, ensure it is in a ready state and |
257 | + # contextually fetch the bootstrap node series. |
258 | + print('retrieving the environment status') |
259 | + bootstrap_node_series = app.status(options.env_name, juju_command) |
260 | + |
261 | + # If the environment was not already bootstrapped, we need to retrieve |
262 | + # the API address. |
263 | + if api_url is None: |
264 | + print('retrieving the Juju API address') |
265 | + api_url = app.get_api_url(options.env_name, juju_command) |
266 | |
267 | # Retrieve the admin-secret for the current environment. |
268 | admin_secret = app.get_admin_secret(options.env_name) |
269 | |
270 | - print('retrieving the Juju API address') |
271 | - api_url = app.get_api_url(options.env_name, juju_command) |
272 | print('connecting to {}'.format(api_url)) |
273 | env = app.connect(api_url, admin_secret) |
274 | |
275 | # Inspect the environment and deploy the charm if required. |
276 | charm_url, machine, service_data, unit_data = app.check_environment( |
277 | env, settings.JUJU_GUI_SERVICE_NAME, options.charm_url, |
278 | - options.env_type, bootstrap_node_series, already_bootstrapped) |
279 | + env_type, bootstrap_node_series, already_bootstrapped) |
280 | unit_name = app.deploy_gui( |
281 | env, settings.JUJU_GUI_SERVICE_NAME, charm_url, machine, |
282 | service_data, unit_data) |
283 | |
284 | === added file 'quickstart/netutils.py' |
285 | --- quickstart/netutils.py 1970-01-01 00:00:00 +0000 |
286 | +++ quickstart/netutils.py 2014-11-12 16:41:04 +0000 |
287 | @@ -0,0 +1,99 @@ |
288 | +# This file is part of the Juju Quickstart Plugin, which lets users set up a |
289 | +# Juju environment in very few steps (https://launchpad.net/juju-quickstart). |
290 | +# Copyright (C) 2014 Canonical Ltd. |
291 | +# |
292 | +# This program is free software: you can redistribute it and/or modify it under |
293 | +# the terms of the GNU Affero General Public License version 3, as published by |
294 | +# the Free Software Foundation. |
295 | +# |
296 | +# This program is distributed in the hope that it will be useful, but WITHOUT |
297 | +# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY, |
298 | +# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE. See the GNU |
299 | +# Affero General Public License for more details. |
300 | +# |
301 | +# You should have received a copy of the GNU Affero General Public License |
302 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
303 | + |
304 | +"""Juju Quickstart network utility functions.""" |
305 | + |
306 | +from __future__ import unicode_literals |
307 | + |
308 | +import json |
309 | +import httplib |
310 | +import logging |
311 | +import socket |
312 | +import urllib2 |
313 | + |
314 | +from quickstart import settings |
315 | + |
316 | + |
317 | +def check_resolvable(hostname): |
318 | + """Check that the hostname can be resolved to a numeric IP address. |
319 | + |
320 | + Return an error message if the address cannot be resolved. |
321 | + """ |
322 | + try: |
323 | + address = socket.gethostbyname(hostname) |
324 | + except socket.error as err: |
325 | + return bytes(err).decode('utf-8') |
326 | + logging.debug('{} resolved to {}'.format( |
327 | + hostname, address.decode('utf-8'))) |
328 | + return None |
329 | + |
330 | + |
331 | +def check_listening(address, timeout=3): |
332 | + """Check that the given address is listening and accepts connections. |
333 | + |
334 | + The address must be specified as a "host:port" string. |
335 | + Use the given socket timeout in seconds. |
336 | + |
337 | + Return an error message if connecting to the address fails. |
338 | + """ |
339 | + try: |
340 | + host, port = address.split(":") |
341 | + sock = socket.create_connection((host, int(port)), timeout) |
342 | + except (socket.error, TypeError, ValueError) as err: |
343 | + return 'cannot connect to {}: {}'.format( |
344 | + address, bytes(err).decode('utf-8')) |
345 | + # Ignore all possible connection close exceptions. |
346 | + try: |
347 | + sock.close() |
348 | + except: |
349 | + pass |
350 | + return None |
351 | + |
352 | + |
353 | +def get_charm_url(series): |
354 | + """Return the charm URL of the latest Juju GUI charm revision. |
355 | + |
356 | + Raise an IOError if any problems occur connecting to the API endpoint. |
357 | + Raise a ValueError if the API returns invalid data. |
358 | + """ |
359 | + url = settings.CHARMWORLD_API.format( |
360 | + series=series, charm=settings.JUJU_GUI_CHARM_NAME) |
361 | + charm_info = json.loads(urlread(url)) |
362 | + charm_url = charm_info.get('charm', {}).get('url') |
363 | + if charm_url is None: |
364 | + raise ValueError(b'unable to find the charm URL') |
365 | + return charm_url |
366 | + |
367 | + |
368 | +def urlread(url): |
369 | + """Open the given URL and return the page contents. |
370 | + |
371 | + Raise an IOError if any problems occur. |
372 | + """ |
373 | + try: |
374 | + response = urllib2.urlopen(url) |
375 | + except urllib2.URLError as err: |
376 | + raise IOError(err.reason) |
377 | + except (httplib.HTTPException, socket.error, urllib2.HTTPError) as err: |
378 | + raise IOError(bytes(err)) |
379 | + contents = response.read() |
380 | + content_type = response.headers['content-type'] |
381 | + charset = 'utf-8' |
382 | + if 'charset=' in content_type: |
383 | + sent_charset = content_type.split('charset=')[-1].strip() |
384 | + if sent_charset: |
385 | + charset = sent_charset |
386 | + return contents.decode(charset, 'ignore') |
387 | |
388 | === modified file 'quickstart/tests/helpers.py' |
389 | --- quickstart/tests/helpers.py 2014-11-11 16:39:13 +0000 |
390 | +++ quickstart/tests/helpers.py 2014-11-12 16:41:04 +0000 |
391 | @@ -21,6 +21,7 @@ |
392 | from contextlib import contextmanager |
393 | import os |
394 | import shutil |
395 | +import socket |
396 | import tempfile |
397 | |
398 | import mock |
399 | @@ -145,6 +146,7 @@ |
400 | 'bootstrap-config': { |
401 | 'admin-secret': 'Secret!', |
402 | 'api-port': 17070, |
403 | + 'type': 'ec2', |
404 | }, |
405 | 'life': {'universe': {'everything': 42}}, |
406 | } |
407 | @@ -231,22 +233,34 @@ |
408 | mock_print = mock.patch('__builtin__.print') |
409 | |
410 | |
411 | +def patch_socket_create_connection(error=None): |
412 | + """Patch the socket.create_connection function. |
413 | + |
414 | + If error is not None, the mock object raises a socket.error with the given |
415 | + message. |
416 | + """ |
417 | + mock_create_connection = mock.Mock() |
418 | + if error is not None: |
419 | + mock_create_connection.side_effect = socket.error(error) |
420 | + return mock.patch('socket.create_connection', mock_create_connection) |
421 | + |
422 | + |
423 | def patch_check_resolvable(error=None): |
424 | - """Patch the utils.check_resolvable function to return the given error. |
425 | + """Patch the netutils.check_resolvable function to return the given error. |
426 | |
427 | This is done so that tests do not try to resolve hostname addresses. |
428 | """ |
429 | return mock.patch( |
430 | - 'quickstart.utils.check_resolvable', |
431 | + 'quickstart.netutils.check_resolvable', |
432 | lambda hostname: error, |
433 | ) |
434 | |
435 | |
436 | class UrlReadTestsMixin(object): |
437 | - """Expose a method to mock the quickstart.utils.urlread helper function.""" |
438 | + """Helpers to mock the quickstart.netutils.urlread helper function.""" |
439 | |
440 | def patch_urlread(self, contents=None, error=False): |
441 | - """Patch the quickstart.utils.urlread helper function. |
442 | + """Patch the quickstart.netutils.urlread helper function. |
443 | |
444 | If contents is not None, urlread() will return the provided contents. |
445 | If error is set to True, an IOError will be simulated. |
446 | @@ -256,7 +270,7 @@ |
447 | mock_urlread.return_value = contents |
448 | if error: |
449 | mock_urlread.side_effect = IOError('bad wolf') |
450 | - return mock.patch('quickstart.utils.urlread', mock_urlread) |
451 | + return mock.patch('quickstart.netutils.urlread', mock_urlread) |
452 | |
453 | |
454 | class ValueErrorTestsMixin(object): |
455 | |
456 | === modified file 'quickstart/tests/test_app.py' |
457 | --- quickstart/tests/test_app.py 2014-11-11 13:21:57 +0000 |
458 | +++ quickstart/tests/test_app.py 2014-11-12 16:41:04 +0000 |
459 | @@ -454,141 +454,174 @@ |
460 | self.assertTrue(mock_create_keys.called) |
461 | |
462 | |
463 | -@helpers.mock_print |
464 | +class TestCheckBootstrapped(helpers.JenvFileTestsMixin, unittest.TestCase): |
465 | + |
466 | + def test_no_jenv_file(self): |
467 | + # A None API URL is returned if the jenv file is not present. |
468 | + with self.make_jenv('ec2', ''): |
469 | + with helpers.assert_logs([], level='warn'): |
470 | + api_url = app.check_bootstrapped('hp') |
471 | + self.assertIsNone(api_url) |
472 | + |
473 | + def test_invalid_jenv_file(self): |
474 | + # A None API URL is returned if the list of API addresses cannot be |
475 | + # retrieved from the jenv file. |
476 | + with self.make_jenv('ec2', '') as path: |
477 | + logs = [ |
478 | + 'cannot retrieve the Juju API URL: ' |
479 | + 'invalid YAML contents in {}: ' |
480 | + 'state-servers key not found in the root section'.format(path) |
481 | + ] |
482 | + with helpers.assert_logs(logs, level='warn'): |
483 | + api_url = app.check_bootstrapped('ec2') |
484 | + self.assertIsNone(api_url) |
485 | + |
486 | + def test_no_api_addresses(self): |
487 | + # A None API URL is returned if the list of API addresses is empty. |
488 | + jenv_data = {'state-servers': []} |
489 | + logs = ['cannot retrieve the Juju API URL: no addresses found'] |
490 | + with self.make_jenv('local', yaml.safe_dump(jenv_data)): |
491 | + with helpers.assert_logs(logs, level='warn'): |
492 | + api_url = app.check_bootstrapped('local') |
493 | + self.assertIsNone(api_url) |
494 | + |
495 | + def test_api_address_not_listening(self): |
496 | + # A None API URL is returned if there is no reachable API address. |
497 | + logs = [ |
498 | + 'cannot retrieve the Juju API URL: ' |
499 | + 'cannot connect to any of the following addresses: ' |
500 | + 'localhost:17070, 10.0.3.1:17070' |
501 | + ] |
502 | + with self.make_jenv('local', yaml.safe_dump(self.jenv_data)): |
503 | + with helpers.assert_logs(logs, level='warn'): |
504 | + with helpers.patch_socket_create_connection('bad wolf'): |
505 | + api_url = app.check_bootstrapped('local') |
506 | + self.assertIsNone(api_url) |
507 | + |
508 | + def test_bootstrapped(self): |
509 | + # The first listening API URL is returned if the environment is already |
510 | + # bootstrapped. |
511 | + with self.make_jenv('hp', yaml.safe_dump(self.jenv_data)): |
512 | + with helpers.assert_logs([], level='warn'): |
513 | + with helpers.patch_socket_create_connection(): |
514 | + api_url = app.check_bootstrapped('hp') |
515 | + # The first API address is returned. |
516 | + self.assertEqual('wss://localhost:17070', api_url) |
517 | + |
518 | + |
519 | class TestBootstrap( |
520 | helpers.CallTestsMixin, ProgramExitTestsMixin, unittest.TestCase): |
521 | |
522 | - env_name = 'my-juju-env' |
523 | - status_message = 'retrieving the environment status' |
524 | - juju_command = settings.JUJU_CMD_PATHS['default'] |
525 | - |
526 | - def make_status_output(self, agent_state, series='hoary'): |
527 | + def test_environment_not_bootstrapped(self): |
528 | + # The environment is successfully bootstrapped and False is returned. |
529 | + with self.patch_call(0) as mock_call: |
530 | + already_bootstrapped = app.bootstrap('ec2', '/usr/bin/juju') |
531 | + self.assertFalse(already_bootstrapped) |
532 | + mock_call.assert_called_once_with( |
533 | + '/usr/bin/juju', 'bootstrap', '-e', 'ec2') |
534 | + |
535 | + def test_environment_already_bootstrapped(self): |
536 | + # The function succeeds and returns True if the environment is already |
537 | + # bootstrapped. |
538 | + error = '***environment is already bootstrapped***' |
539 | + with self.patch_call(1, error=error) as mock_call: |
540 | + already_bootstrapped = app.bootstrap('hp', '/bin/juju') |
541 | + self.assertTrue(already_bootstrapped) |
542 | + mock_call.assert_called_once_with('/bin/juju', 'bootstrap', '-e', 'hp') |
543 | + |
544 | + def test_bootstrap_failure(self): |
545 | + # A ProgramExit is raised if an error occurs while bootstrapping. |
546 | + with self.patch_call(1, error='bad wolf') as mock_call: |
547 | + with self.assert_program_exit('bad wolf'): |
548 | + app.bootstrap('local', 'juju') |
549 | + mock_call.assert_called_once_with('juju', 'bootstrap', '-e', 'local') |
550 | + |
551 | + def test_debug(self): |
552 | + # The environment is bootstrapped in debug mode. |
553 | + with self.patch_call(0) as mock_call: |
554 | + app.bootstrap('ec2', '/usr/bin/juju', debug=True) |
555 | + mock_call.assert_called_once_with( |
556 | + '/usr/bin/juju', 'bootstrap', '-e', 'ec2', '--debug') |
557 | + |
558 | + def test_upload_tools(self): |
559 | + # The environment is bootstrapped with local tools |
560 | + with self.patch_call(0) as mock_call: |
561 | + app.bootstrap('local', '/usr/bin/juju', upload_tools=True) |
562 | + mock_call.assert_called_once_with( |
563 | + '/usr/bin/juju', 'bootstrap', '-e', 'local', '--upload-tools') |
564 | + |
565 | + def test_upload_series(self): |
566 | + # The environment is bootstrapped with tools for specific series. |
567 | + with self.patch_call(0) as mock_call: |
568 | + app.bootstrap('hp', '/usr/bin/juju', upload_series='trusty,utopic') |
569 | + mock_call.assert_called_once_with( |
570 | + '/usr/bin/juju', 'bootstrap', '-e', 'hp', |
571 | + '--upload-series', 'trusty,utopic') |
572 | + |
573 | + def test_constraints(self): |
574 | + # The environment is bootstrapped with the specified constraints. |
575 | + with self.patch_call(0) as mock_call: |
576 | + app.bootstrap('maas', '/usr/bin/juju', constraints='mem=7G') |
577 | + mock_call.assert_called_once_with( |
578 | + '/usr/bin/juju', 'bootstrap', '-e', 'maas', |
579 | + '--constraints', 'mem=7G') |
580 | + |
581 | + def test_all_options(self): |
582 | + # The environment is bootstrapped with all the options. |
583 | + with self.patch_call(0) as mock_call: |
584 | + app.bootstrap( |
585 | + 'local', '/usr/bin/juju', debug=True, upload_tools=True, |
586 | + upload_series='vivid', constraints='mem=8G') |
587 | + mock_call.assert_called_once_with( |
588 | + '/usr/bin/juju', 'bootstrap', '-e', 'local', |
589 | + '--debug', '--upload-tools', '--upload-series', 'vivid', |
590 | + '--constraints', 'mem=8G') |
591 | + |
592 | + |
593 | +class TestStatus( |
594 | + helpers.CallTestsMixin, ProgramExitTestsMixin, unittest.TestCase): |
595 | + |
596 | + def make_status_output(self, agent_state, series='utopic'): |
597 | """Create and return a YAML status output.""" |
598 | return yaml.safe_dump({ |
599 | - 'machines': {'0': {'agent-state': agent_state, |
600 | - 'series': series}}, |
601 | + 'machines': { |
602 | + '0': {'agent-state': agent_state, 'series': series}, |
603 | + }, |
604 | }) |
605 | |
606 | - def make_status_calls(self, number): |
607 | + def make_status_calls(self, env_name, juju_command, number): |
608 | """Return a list containing the given number of status calls.""" |
609 | call = mock.call( |
610 | - self.juju_command, 'status', '-e', self.env_name, |
611 | - '--format', 'yaml') |
612 | + juju_command, 'status', '-e', env_name, '--format', 'yaml') |
613 | return [call for _ in range(number)] |
614 | |
615 | - def make_side_effects(self): |
616 | - """Return the minimum number of side effects for a successful call.""" |
617 | - return [ |
618 | - (0, '', ''), # Add a bootstrap call. |
619 | - (0, self.make_status_output('started'), ''), # Add a status call. |
620 | - ] |
621 | - |
622 | - def assert_status_retried(self, side_effects): |
623 | + def assert_status_retried(self, env_name, juju_command, side_effects): |
624 | """Ensure the "juju status" command is retried several times. |
625 | |
626 | Receive the list of side effects the mock status call will return. |
627 | """ |
628 | - with self.patch_multiple_calls(side_effects) as mock_call: |
629 | - app.bootstrap(self.env_name, self.juju_command) |
630 | - mock_call.assert_has_calls([ |
631 | - mock.call(self.juju_command, 'bootstrap', '-e', self.env_name), |
632 | - ] + self.make_status_calls(5)) |
633 | - |
634 | - def test_success(self, mock_print): |
635 | - # The environment is successfully bootstrapped. |
636 | - with self.patch_multiple_calls(self.make_side_effects()) as mock_call: |
637 | - already_bootstrapped, series = app.bootstrap( |
638 | - self.env_name, self.juju_command) |
639 | - self.assertFalse(already_bootstrapped) |
640 | - self.assertEqual(series, 'hoary') |
641 | - mock_call.assert_has_calls([ |
642 | - mock.call(self.juju_command, 'bootstrap', '-e', self.env_name), |
643 | - ] + self.make_status_calls(1)) |
644 | - mock_print.assert_called_once_with(self.status_message) |
645 | - |
646 | - def test_success_debug(self, mock_print): |
647 | - # The environment is successfully bootstrapped in debug mode. |
648 | - with self.patch_multiple_calls(self.make_side_effects()) as mock_call: |
649 | - already_bootstrapped, series = app.bootstrap( |
650 | - self.env_name, self.juju_command, debug=True) |
651 | - self.assertFalse(already_bootstrapped) |
652 | - self.assertEqual(series, 'hoary') |
653 | - mock_call.assert_has_calls([ |
654 | - mock.call( |
655 | - self.juju_command, 'bootstrap', '-e', self.env_name, |
656 | - '--debug'), |
657 | - ] + self.make_status_calls(1)) |
658 | - |
659 | - def test_success_upload_tools(self, mock_print): |
660 | - # The environment is bootstrapped with local tools. |
661 | - with self.patch_multiple_calls(self.make_side_effects()) as mock_call: |
662 | - already_bootstrapped, series = app.bootstrap( |
663 | - self.env_name, self.juju_command, upload_tools=True) |
664 | - self.assertFalse(already_bootstrapped) |
665 | - mock_call.assert_has_calls([ |
666 | - mock.call( |
667 | - self.juju_command, 'bootstrap', '-e', self.env_name, |
668 | - '--upload-tools'), |
669 | - ] + self.make_status_calls(1)) |
670 | - |
671 | - def test_success_upload_series(self, mock_print): |
672 | - # The environment is bootstrapped with tools for specific series. |
673 | - with self.patch_multiple_calls(self.make_side_effects()) as mock_call: |
674 | - already_bootstrapped, series = app.bootstrap( |
675 | - self.env_name, self.juju_command, upload_series='hoary') |
676 | - self.assertFalse(already_bootstrapped) |
677 | - mock_call.assert_has_calls([ |
678 | - mock.call( |
679 | - self.juju_command, 'bootstrap', '-e', self.env_name, |
680 | - '--upload-series', 'hoary'), |
681 | - ] + self.make_status_calls(1)) |
682 | - |
683 | - def test_success_constraints(self, mock_print): |
684 | - # The environment is bootstrapped with given constraints. |
685 | - with self.patch_multiple_calls(self.make_side_effects()) as mock_call: |
686 | - already_bootstrapped, series = app.bootstrap( |
687 | - self.env_name, self.juju_command, constraints='mem=7G') |
688 | - self.assertFalse(already_bootstrapped) |
689 | - mock_call.assert_has_calls([ |
690 | - mock.call( |
691 | - self.juju_command, 'bootstrap', '-e', self.env_name, |
692 | - '--constraints', 'mem=7G'), |
693 | - ] + self.make_status_calls(1)) |
694 | - |
695 | - def test_already_bootstrapped(self, mock_print): |
696 | - # The function succeeds and returns True if the environment is already |
697 | - # bootstrapped. |
698 | - side_effects = [ |
699 | - (1, '', '***environment is already bootstrapped**'), |
700 | - (0, self.make_status_output('started', 'precise'), ''), |
701 | - ] |
702 | - with self.patch_multiple_calls(side_effects) as mock_call: |
703 | - already_bootstrapped, series = app.bootstrap( |
704 | - self.env_name, self.juju_command) |
705 | - self.assertTrue(already_bootstrapped) |
706 | - self.assertEqual(series, 'precise') |
707 | - mock_call.assert_has_calls([ |
708 | - mock.call(self.juju_command, 'bootstrap', '-e', self.env_name), |
709 | - ] + self.make_status_calls(1)) |
710 | - existing_message = 'reusing the already bootstrapped {} environment' |
711 | - mock_print.assert_has_calls([ |
712 | - mock.call(existing_message.format(self.env_name)), |
713 | - mock.call(self.status_message), |
714 | - ]) |
715 | - |
716 | - def test_bootstrap_failure(self, mock_print): |
717 | - # A ProgramExit is raised if an error occurs while bootstrapping. |
718 | - with self.patch_call(retcode=1, error='bad wolf') as mock_call: |
719 | - with self.assert_program_exit('bad wolf'): |
720 | - app.bootstrap(self.env_name, self.juju_command) |
721 | - mock_call.assert_called_once_with( |
722 | - self.juju_command, 'bootstrap', '-e', self.env_name), |
723 | - |
724 | - def test_status_retry_error(self, mock_print): |
725 | + count = len(side_effects) |
726 | + with self.patch_multiple_calls(side_effects) as mock_call: |
727 | + app.status(env_name, juju_command) |
728 | + self.assertEqual(count, mock_call.call_count) |
729 | + expected_calls = self.make_status_calls(env_name, juju_command, count) |
730 | + mock_call.assert_has_calls(expected_calls) |
731 | + |
732 | + def test_success(self): |
733 | + # The status command is correctly called and the bootstrap node series |
734 | + # returned. |
735 | + output = self.make_status_output('started') |
736 | + with self.patch_call(0, output=output) as mock_call: |
737 | + bootstrap_node_series = app.status('ec2', '/usr/bin/juju') |
738 | + self.assertEqual('utopic', bootstrap_node_series) |
739 | + self.assertEqual(1, mock_call.call_count) |
740 | + expected_calls = self.make_status_calls('ec2', '/usr/bin/juju', 1) |
741 | + mock_call.assert_has_calls(expected_calls) |
742 | + |
743 | + def test_status_retry_error(self): |
744 | # Before raising a ProgramExit, the functions tries to call |
745 | # "juju status" multiple times if it exits with an error. |
746 | side_effects = [ |
747 | - (0, '', ''), # Add the bootstrap call. |
748 | # Add four status calls with a non-zero exit code. |
749 | (1, '', 'these'), |
750 | (2, '', 'are'), |
751 | @@ -597,29 +630,28 @@ |
752 | # Add a final valid status call. |
753 | (0, self.make_status_output('started'), ''), |
754 | ] |
755 | - self.assert_status_retried(side_effects) |
756 | + self.assert_status_retried('local', 'juju', side_effects) |
757 | |
758 | - def test_status_retry_invalid_output(self, mock_print): |
759 | + def test_status_retry_invalid_output(self): |
760 | # Before raising a ProgramExit, the functions tries to call |
761 | # "juju status" multiple times if its output is not well formed or if |
762 | # the agent is not started. |
763 | side_effects = [ |
764 | - (0, '', ''), # Add the bootstrap call. |
765 | (0, '', ''), # Add the first status call: no output. |
766 | (0, ':', ''), # Add the second status call: not YAML. |
767 | (0, 'just-a-string', ''), # Add the third status call: bad YAML. |
768 | - # Add the fourth status call: the agent is still pending. |
769 | + # Add two other status calls: the agent is still pending. |
770 | + (0, self.make_status_output('pending'), ''), |
771 | (0, self.make_status_output('pending'), ''), |
772 | # Add a final valid status call. |
773 | (0, self.make_status_output('started'), ''), |
774 | ] |
775 | - self.assert_status_retried(side_effects) |
776 | + self.assert_status_retried('hp', '/usr/bin/juju', side_effects) |
777 | |
778 | - def test_status_retry_both(self, mock_print): |
779 | + def test_status_retry_both(self): |
780 | # Before raising a ProgramExit, the functions tries to call |
781 | # "juju status" multiple times in any case. |
782 | side_effects = [ |
783 | - (0, '', ''), # Add the bootstrap call. |
784 | (1, '', 'error'), # Add the first status call: error. |
785 | (2, '', 'another error'), # Add the second status call: error. |
786 | # Add the third status call: the agent is still pending. |
787 | @@ -628,28 +660,23 @@ |
788 | # Add a final valid status call. |
789 | (0, self.make_status_output('started'), ''), |
790 | ] |
791 | - self.assert_status_retried(side_effects) |
792 | + self.assert_status_retried('local', '/usr/bin/juju', side_effects) |
793 | |
794 | - def test_agent_error(self, mock_print): |
795 | + def test_agent_error(self): |
796 | # A ProgramExit is raised immediately if the Juju agent in the |
797 | # bootstrap node is in an error state. |
798 | - status_output = self.make_status_output('error') |
799 | - side_effects = [ |
800 | - (0, '', ''), # Add the bootstrap call. |
801 | - (0, status_output, ''), # Add the status call: agent error. |
802 | - ] |
803 | - expected = 'state server failure:\n{}'.format(status_output) |
804 | - with self.patch_multiple_calls(side_effects) as mock_call: |
805 | - with self.assert_program_exit(expected): |
806 | - app.bootstrap(self.env_name, self.juju_command) |
807 | - mock_call.assert_has_calls([ |
808 | - mock.call(self.juju_command, 'bootstrap', '-e', self.env_name), |
809 | - ] + self.make_status_calls(1)) |
810 | + output = self.make_status_output('error') |
811 | + expected_error = 'state server failure:\n{}'.format(output) |
812 | + with self.patch_call(0, output=output) as mock_call: |
813 | + with self.assert_program_exit(expected_error): |
814 | + app.status('ec2', '/usr/bin/juju') |
815 | + self.assertEqual(1, mock_call.call_count) |
816 | + expected_calls = self.make_status_calls('ec2', '/usr/bin/juju', 1) |
817 | + mock_call.assert_has_calls(expected_calls) |
818 | |
819 | - def test_status_failure(self, mock_print): |
820 | + def test_status_failure(self): |
821 | # A ProgramExit is raised if "juju status" keeps failing. |
822 | call_side_effects = [ |
823 | - (0, '', ''), # Add the bootstrap call. |
824 | (1, 'output1', 'error1'), # Add the first status call: retried. |
825 | (1, 'output2', 'error2'), # Add the second status call: error. |
826 | ] |
827 | @@ -660,17 +687,37 @@ |
828 | 1000, # Third call after the timeout expiration. |
829 | ] |
830 | mock_time = mock.Mock(side_effect=time_side_effects) |
831 | - expected = 'the state server is not ready:\noutput2error2' |
832 | + expected_error = 'the state server is not ready:\noutput2error2' |
833 | with self.patch_multiple_calls(call_side_effects) as mock_call: |
834 | # Simulate the timeout expired: the first time call is used to |
835 | # calculate the timeout, the second one for the first status check, |
836 | # the third for the second status check, the fourth should fail. |
837 | with mock.patch('time.time', mock_time): |
838 | - with self.assert_program_exit(expected): |
839 | - app.bootstrap(self.env_name, self.juju_command) |
840 | - mock_call.assert_has_calls([ |
841 | - mock.call(self.juju_command, 'bootstrap', '-e', self.env_name), |
842 | - ] + self.make_status_calls(2)) |
843 | + with self.assert_program_exit(expected_error): |
844 | + app.status('local', '/usr/bin/juju') |
845 | + self.assertEqual(2, mock_call.call_count) |
846 | + expected_calls = self.make_status_calls('local', '/usr/bin/juju', 2) |
847 | + mock_call.assert_has_calls(expected_calls) |
848 | + |
849 | + |
850 | +class TestGetEnvType( |
851 | + helpers.JenvFileTestsMixin, ProgramExitTestsMixin, unittest.TestCase): |
852 | + |
853 | + def test_success(self): |
854 | + # The environment type is successfully retrieved. |
855 | + with self.make_jenv('ec2', yaml.safe_dump(self.jenv_data)): |
856 | + env_type = app.get_env_type('ec2') |
857 | + self.assertEqual('ec2', env_type) |
858 | + |
859 | + def test_error(self): |
860 | + # A ProgramExit is raised if the environment type cannot be retrieved. |
861 | + with self.make_jenv('aws', '') as path: |
862 | + expected_error = ( |
863 | + 'cannot retrieve environment type: invalid YAML ' |
864 | + 'contents in {}: bootstrap-config key not found in the root ' |
865 | + 'section'.format(path)) |
866 | + with self.assert_program_exit(expected_error): |
867 | + app.get_env_type('aws') |
868 | |
869 | |
870 | class TestGetAdminSecret( |
871 | @@ -850,7 +897,8 @@ |
872 | """Patch the get_charm_url helper function.""" |
873 | mock_get_charm_url = mock.Mock( |
874 | return_value=return_value, side_effect=side_effect) |
875 | - return mock.patch('quickstart.utils.get_charm_url', mock_get_charm_url) |
876 | + return mock.patch( |
877 | + 'quickstart.netutils.get_charm_url', mock_get_charm_url) |
878 | |
879 | def test_environment_just_bootstrapped(self, mock_print): |
880 | # The function correctly retrieves the charm URL and machine, and |
881 | |
882 | === modified file 'quickstart/tests/test_manage.py' |
883 | --- quickstart/tests/test_manage.py 2014-11-11 13:21:57 +0000 |
884 | +++ quickstart/tests/test_manage.py 2014-11-12 16:41:04 +0000 |
885 | @@ -38,7 +38,6 @@ |
886 | from quickstart.cli import views |
887 | from quickstart.models import envs |
888 | from quickstart.tests import helpers |
889 | -from quickstart import app |
890 | |
891 | |
892 | class TestDescriptionAction(unittest.TestCase): |
893 | @@ -763,164 +762,177 @@ |
894 | options.update(kwargs) |
895 | return mock.Mock(**options) |
896 | |
897 | - def test_no_bundle(self, mock_app, mock_open): |
898 | + def configure_app(self, mock_app, **kwargs): |
899 | + """Configure the given mock_app with the given kwargs. |
900 | + |
901 | + Each key/value pair in kwargs represents a mock_app attribute and |
902 | + the associated return value. Those values are used to override |
903 | + defaults. |
904 | + |
905 | + Return the mock Juju environment object. |
906 | + """ |
907 | + env = mock.Mock() |
908 | + defaults = { |
909 | + # Dependencies are installed. |
910 | + 'ensure_dependencies': (1, 18, 0), |
911 | + # Ensure the current Juju version is supported. |
912 | + 'check_juju_supported': None, |
913 | + # Ensure the SSH keys are properly configured. |
914 | + 'ensure_ssh_keys': None, |
915 | + # The environment is not already bootstrapped. |
916 | + 'check_bootstrapped': None, |
917 | + # This is also confirmed by the bootstrap function. |
918 | + 'bootstrap': False, |
919 | + # Status is then called, returning the bootstrap node series. |
920 | + 'status': 'trusty', |
921 | + # The API URL must be retrieved (the environment was not ready). |
922 | + 'get_api_url': 'wss://1.2.3.4:17070', |
923 | + # Retrieve the admin secret. |
924 | + 'get_admin_secret': 'Secret!', |
925 | + # Connect to the Juju Environment API endpoint. |
926 | + 'connect': env, |
927 | + # The environment is then checked. |
928 | + 'check_environment': ( |
929 | + 'cs:trusty/juju-gui-42', |
930 | + '0', |
931 | + {'Name': 'juju-gui'}, |
932 | + {'Name': 'juju-gui/0'} |
933 | + ), |
934 | + # Deploy the Juju GUI charm. |
935 | + 'deploy_gui': 'juju-gui/0', |
936 | + # Watch the deployment progress and return the unit address. |
937 | + 'watch': '1.2.3.4', |
938 | + # Create the login token for the Juju GUI. |
939 | + 'create_auth_token': 'TOKEN', |
940 | + } |
941 | + defaults.update(kwargs) |
942 | + for attr, return_value in defaults.items(): |
943 | + getattr(mock_app, attr).return_value = return_value |
944 | + return env |
945 | + |
946 | + def patch_get_juju_command(self): |
947 | + """Patch the platform_support.get_juju_command function.""" |
948 | + path = 'quickstart.manage.platform_support.get_juju_command' |
949 | + return mock.patch(path, return_value=(self.juju_command, False)) |
950 | + |
951 | + def test_run(self, mock_app, mock_open): |
952 | # The application runs correctly if no bundle is provided. |
953 | - mock_app.ensure_dependencies.return_value = (1, 18, 0) |
954 | - # Make mock_app.bootstrap return the already_bootstrapped flag and the |
955 | - # bootstrap node series. |
956 | - mock_app.bootstrap.return_value = (True, 'trusty') |
957 | - # Make mock_app.check_environment return the charm URL, the machine |
958 | - # where to deploy the charm, the service and unit data. |
959 | - service_data = {'Name': 'juju-gui'} |
960 | - unit_data = {'Name': 'juju-gui/0'} |
961 | - mock_app.check_environment.return_value = ( |
962 | - 'cs:trusty/juju-gui-42', '0', service_data, unit_data) |
963 | - # Make mock_app.get_admin_secret return the admin secret |
964 | - mock_app.get_admin_secret.return_value = 'Secret!' |
965 | - # Make mock_app.watch return the Juju GUI unit address. |
966 | - mock_app.watch.return_value = '1.2.3.4' |
967 | - # Make mock_app.create_auth_token return a fake authentication token. |
968 | - mock_app.create_auth_token.return_value = 'AUTHTOKEN' |
969 | + env = self.configure_app(mock_app) |
970 | + # Run the application. |
971 | options = self.make_options() |
972 | - with mock.patch('quickstart.manage.platform_support.get_juju_command', |
973 | - side_effect=[(self.juju_command, False)]): |
974 | + with self.patch_get_juju_command(): |
975 | manage.run(options) |
976 | - mock_app.ensure_dependencies.assert_called() |
977 | - mock_app.ensure_ssh_keys.assert_called() |
978 | + # Ensure the functions have been used correctly. |
979 | + mock_app.ensure_dependencies.assert_called_once_with( |
980 | + options.distro_only, options.platform, self.juju_command) |
981 | + mock_app.check_juju_supported.assert_called_once_with((1, 18, 0)) |
982 | + mock_app.ensure_ssh_keys.assert_called_once_with() |
983 | + mock_app.check_bootstrapped.assert_called_once_with(options.env_name) |
984 | mock_app.bootstrap.assert_called_once_with( |
985 | options.env_name, self.juju_command, |
986 | debug=options.debug, |
987 | upload_tools=options.upload_tools, |
988 | upload_series=options.upload_series, |
989 | constraints=options.constraints) |
990 | + mock_app.status.assert_called_once_with( |
991 | + options.env_name, self.juju_command) |
992 | mock_app.get_api_url.assert_called_once_with( |
993 | options.env_name, self.juju_command) |
994 | + mock_app.get_admin_secret.assert_called_once_with(options.env_name) |
995 | mock_app.connect.assert_has_calls([ |
996 | - mock.call(mock_app.get_api_url(), 'Secret!'), |
997 | + mock.call('wss://1.2.3.4:17070', 'Secret!'), |
998 | mock.call().close(), |
999 | mock.call('wss://1.2.3.4:443/ws', 'Secret!'), |
1000 | mock.call().close(), |
1001 | ]) |
1002 | mock_app.check_environment.assert_called_once_with( |
1003 | - mock_app.connect(), settings.JUJU_GUI_SERVICE_NAME, |
1004 | - options.charm_url, options.env_type, mock_app.bootstrap()[1], |
1005 | - mock_app.bootstrap()[0]) |
1006 | + env, settings.JUJU_GUI_SERVICE_NAME, options.charm_url, |
1007 | + options.env_type, 'trusty', False) |
1008 | mock_app.deploy_gui.assert_called_once_with( |
1009 | - mock_app.connect(), settings.JUJU_GUI_SERVICE_NAME, |
1010 | - 'cs:trusty/juju-gui-42', '0', service_data, unit_data) |
1011 | - mock_app.watch.assert_called_once_with( |
1012 | - mock_app.connect(), mock_app.deploy_gui()) |
1013 | - mock_app.create_auth_token.assert_called_once_with(mock_app.connect()) |
1014 | - mock_open.assert_called_once_with( |
1015 | - 'https://{}/?authtoken={}'.format(mock_app.watch(), 'AUTHTOKEN')) |
1016 | + env, settings.JUJU_GUI_SERVICE_NAME, 'cs:trusty/juju-gui-42', |
1017 | + '0', {'Name': 'juju-gui'}, {'Name': 'juju-gui/0'}) |
1018 | + mock_app.watch.assert_called_once_with(env, 'juju-gui/0') |
1019 | + mock_app.create_auth_token.assert_called_once_with(env) |
1020 | + mock_open.assert_called_once_with('https://1.2.3.4/?authtoken=TOKEN') |
1021 | + # Ensure some of the app function have not been called. |
1022 | + self.assertFalse(mock_app.get_env_type.called) |
1023 | self.assertFalse(mock_app.deploy_bundle.called) |
1024 | |
1025 | + def test_already_bootstrapped(self, mock_app, mock_open): |
1026 | + # The application correctly reuses an already bootstrapped environment. |
1027 | + self.configure_app(mock_app, check_bootstrapped='wss://example.com') |
1028 | + # Run the application. |
1029 | + options = self.make_options() |
1030 | + with self.patch_get_juju_command(): |
1031 | + manage.run(options) |
1032 | + # The environment type is retrieved from the jenv. |
1033 | + mock_app.get_env_type.assert_called_once_with(options.env_name) |
1034 | + # No reason to call bootstrap or get_api_url functions. |
1035 | + self.assertFalse(mock_app.bootstrap.called) |
1036 | + self.assertFalse(mock_app.get_api_url.called) |
1037 | + |
1038 | + def test_already_bootstrapped_race(self, mock_app, mock_open): |
1039 | + # The application correctly reuses an already bootstrapped environment. |
1040 | + # In this case, the environment seems not bootstrapped at first, but |
1041 | + # it ended up being up and running later. |
1042 | + self.configure_app(mock_app, bootstrap=True) |
1043 | + # Run the application. |
1044 | + options = self.make_options() |
1045 | + with self.patch_get_juju_command(): |
1046 | + manage.run(options) |
1047 | + # The bootstrap and get_api_url functions are still called, but this |
1048 | + # time also get_env_type is required. |
1049 | + # The environment type is retrieved from the jenv. |
1050 | + mock_app.bootstrap.assert_called_once_with( |
1051 | + options.env_name, self.juju_command, |
1052 | + debug=options.debug, |
1053 | + upload_tools=options.upload_tools, |
1054 | + upload_series=options.upload_series, |
1055 | + constraints=options.constraints) |
1056 | + mock_app.get_api_url.assert_called_once_with( |
1057 | + options.env_name, self.juju_command) |
1058 | + mock_app.get_env_type.assert_called_once_with(options.env_name) |
1059 | + |
1060 | def test_no_token(self, mock_app, mock_open): |
1061 | # The process continues even if the authentication token cannot be |
1062 | # retrieved. |
1063 | - mock_app.create_auth_token.return_value = None |
1064 | - # Make mock_app.bootstrap return the already_bootstrapped flag and the |
1065 | - # bootstrap node series. |
1066 | - mock_app.bootstrap.return_value = (True, 'precise') |
1067 | - # Make mock_app.check_environment return the charm URL, the machine |
1068 | - # where to deploy the charm, the service and unit data. |
1069 | - mock_app.check_environment.return_value = ( |
1070 | - 'cs:precise/juju-gui-42', '0', None, None) |
1071 | + env = self.configure_app(mock_app, create_auth_token=None) |
1072 | + # Run the application. |
1073 | options = self.make_options() |
1074 | - manage.run(options) |
1075 | - mock_app.create_auth_token.assert_called_once_with(mock_app.connect()) |
1076 | - mock_open.assert_called_once_with( |
1077 | - 'https://{}'.format(mock_app.watch())) |
1078 | + with self.patch_get_juju_command(): |
1079 | + manage.run(options) |
1080 | + # Ensure the browser is still open without an auth token. |
1081 | + mock_app.create_auth_token.assert_called_once_with(env) |
1082 | + mock_open.assert_called_once_with('https://1.2.3.4') |
1083 | |
1084 | def test_bundle(self, mock_app, mock_open): |
1085 | # A bundle is correctly deployed by the application. |
1086 | + env = self.configure_app(mock_app, create_auth_token=None) |
1087 | + # Run the application. |
1088 | options = self.make_options( |
1089 | bundle='/my/bundle/file.yaml', bundle_yaml='mybundle: contents', |
1090 | bundle_name='mybundle', bundle_services=['service1', 'service2']) |
1091 | - # Make mock_app.bootstrap return the already_bootstrapped flag and the |
1092 | - # bootstrap node series. |
1093 | - mock_app.bootstrap.return_value = (True, 'trusty') |
1094 | - # Make mock_app.check_environment return the charm URL, the machine |
1095 | - # where to deploy the charm, the service and unit data. |
1096 | - mock_app.check_environment.return_value = ( |
1097 | - 'cs:trusty/juju-gui-42', '0', None, None) |
1098 | - # Make mock_app.watch return the Juju GUI unit address. |
1099 | - mock_app.watch.return_value = 'gui.example.com' |
1100 | - manage.run(options) |
1101 | + with self.patch_get_juju_command(): |
1102 | + manage.run(options) |
1103 | + # Ensure the bundle is correctly deployed. |
1104 | mock_app.deploy_bundle.assert_called_once_with( |
1105 | - mock_app.connect(), 'mybundle: contents', 'mybundle', None) |
1106 | + env, 'mybundle: contents', 'mybundle', None) |
1107 | |
1108 | - def test_local_provider_no_sudo(self, mock_app, mock_open): |
1109 | + def test_local_provider(self, mock_app, mock_open): |
1110 | # The application correctly handles working with local providers with |
1111 | # new Juju versions not requiring "sudo" to bootstrap the environment. |
1112 | - # Sudo privileges are not required if the Juju version is >= 1.17.2. |
1113 | + self.configure_app(mock_app, create_auth_token=None) |
1114 | + # Run the application. |
1115 | options = self.make_options(env_type='local') |
1116 | - versions = [ |
1117 | - (1, 17, 2), (1, 17, 10), (1, 18, 0), (1, 18, 2), (2, 16, 1)] |
1118 | - # Make mock_app.bootstrap return the already_bootstrapped flag and the |
1119 | - # bootstrap node series. |
1120 | - mock_app.bootstrap.return_value = (True, 'precise') |
1121 | - # Make mock_app.check_environment return the charm URL, the machine |
1122 | - # where to deploy the charm, the service and unit data. |
1123 | - mock_app.check_environment.return_value = ( |
1124 | - 'cs:precise/juju-gui-42', '0', None, None) |
1125 | - for version in versions: |
1126 | - mock_app.ensure_dependencies.return_value = version |
1127 | - with mock.patch( |
1128 | - 'quickstart.manage.platform_support.get_juju_command', |
1129 | - side_effect=[(self.juju_command, False)]): |
1130 | - manage.run(options) |
1131 | - mock_app.bootstrap.assert_called_once_with( |
1132 | - options.env_name, self.juju_command, |
1133 | - debug=options.debug, |
1134 | - upload_tools=options.upload_tools, |
1135 | - upload_series=options.upload_series, |
1136 | - constraints=options.constraints) |
1137 | - mock_app.bootstrap.reset_mock() |
1138 | + with self.patch_get_juju_command(): |
1139 | + manage.run(options) |
1140 | |
1141 | def test_no_browser(self, mock_app, mock_open): |
1142 | # It is possible to avoid opening the GUI in the browser. |
1143 | - # Make mock_app.bootstrap return the already_bootstrapped flag and the |
1144 | - # bootstrap node series. |
1145 | - mock_app.bootstrap.return_value = (True, 'trusty') |
1146 | - # Make mock_app.check_environment return the charm URL, the machine |
1147 | - # where to deploy the charm, the service and unit data. |
1148 | - mock_app.check_environment.return_value = ( |
1149 | - 'cs:trusty/juju-gui-42', '0', None, None) |
1150 | + self.configure_app(mock_app, create_auth_token=None) |
1151 | + # Run the application. |
1152 | options = self.make_options(open_browser=False) |
1153 | - manage.run(options) |
1154 | + with self.patch_get_juju_command(): |
1155 | + manage.run(options) |
1156 | + # The browser is not opened. |
1157 | self.assertFalse(mock_open.called) |
1158 | - |
1159 | - def test_no_admin_secret_found(self, mock_app, mock_open): |
1160 | - # If admin-secret cannot be found a ProgramExit is called. |
1161 | - mock_app.get_admin_secret.side_effect = app.ProgramExit('bad wolf') |
1162 | - # Make mock_app.bootstrap return the already_bootstrapped flag and the |
1163 | - # bootstrap node series. |
1164 | - mock_app.bootstrap.return_value = (True, 'precise') |
1165 | - # Make mock_app.check_environment return the charm URL, the machine |
1166 | - # where to deploy the charm, the service and unit data. |
1167 | - mock_app.check_environment.return_value = ( |
1168 | - 'cs:precise/juju-gui-42', '0', None, None) |
1169 | - options = self.make_options( |
1170 | - env_name='local', |
1171 | - env_file='environments.yaml') |
1172 | - with self.assertRaises(app.ProgramExit) as context: |
1173 | - manage.run(options) |
1174 | - self.assertEqual('bad wolf', context.exception.message) |
1175 | - |
1176 | - def test_juju_environ_var_set(self, mock_app, mock_open): |
1177 | - mock_app.bootstrap.return_value = (True, 'precise') |
1178 | - mock_app.check_environment.return_value = ( |
1179 | - 'cs:precise/juju-gui-42', '0', None, None) |
1180 | - options = self.make_options(env_type='aws') |
1181 | - juju_command = 'some/devel/path/juju' |
1182 | - with mock.patch('os.environ', {'JUJU': juju_command}): |
1183 | - manage.run(options) |
1184 | - mock_app.bootstrap.assert_called_once_with( |
1185 | - options.env_name, juju_command, |
1186 | - debug=options.debug, |
1187 | - upload_tools=options.upload_tools, |
1188 | - upload_series=options.upload_series, |
1189 | - constraints=options.constraints) |
1190 | - mock_app.get_api_url.assert_called_once_with( |
1191 | - options.env_name, juju_command) |
1192 | |
1193 | === added file 'quickstart/tests/test_netutils.py' |
1194 | --- quickstart/tests/test_netutils.py 1970-01-01 00:00:00 +0000 |
1195 | +++ quickstart/tests/test_netutils.py 2014-11-12 16:41:04 +0000 |
1196 | @@ -0,0 +1,201 @@ |
1197 | +# This file is part of the Juju Quickstart Plugin, which lets users set up a |
1198 | +# Juju environment in very few steps (https://launchpad.net/juju-quickstart). |
1199 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1200 | +# |
1201 | +# This program is free software: you can redistribute it and/or modify it under |
1202 | +# the terms of the GNU Affero General Public License version 3, as published by |
1203 | +# the Free Software Foundation. |
1204 | +# |
1205 | +# This program is distributed in the hope that it will be useful, but WITHOUT |
1206 | +# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY, |
1207 | +# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE. See the GNU |
1208 | +# Affero General Public License for more details. |
1209 | +# |
1210 | +# You should have received a copy of the GNU Affero General Public License |
1211 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
1212 | + |
1213 | +"""Tests for the Juju Quickstart network utility functions.""" |
1214 | + |
1215 | +from __future__ import unicode_literals |
1216 | + |
1217 | +import httplib |
1218 | +import json |
1219 | +import socket |
1220 | +import unittest |
1221 | +import urllib2 |
1222 | + |
1223 | +import mock |
1224 | + |
1225 | +from quickstart import netutils |
1226 | +from quickstart.tests import helpers |
1227 | + |
1228 | + |
1229 | +class TestCheckResolvable(unittest.TestCase): |
1230 | + |
1231 | + def test_resolvable(self): |
1232 | + # None is returned if the hostname can be resolved. |
1233 | + expected_log = 'example.com resolved to 1.2.3.4' |
1234 | + with helpers.assert_logs([expected_log], level='debug'): |
1235 | + with mock.patch('socket.gethostbyname', return_value='1.2.3.4'): |
1236 | + error = netutils.check_resolvable('example.com') |
1237 | + self.assertIsNone(error) |
1238 | + |
1239 | + def test_not_resolvable(self): |
1240 | + # An error message is returned if the hostname cannot be resolved. |
1241 | + exception = socket.gaierror('bad wolf') |
1242 | + with mock.patch('socket.gethostbyname', side_effect=exception): |
1243 | + error = netutils.check_resolvable('example.com') |
1244 | + self.assertEqual('bad wolf', error) |
1245 | + |
1246 | + |
1247 | +class TestCheckListening(unittest.TestCase): |
1248 | + |
1249 | + def test_listening(self): |
1250 | + # None is returned if the address can be connected to. |
1251 | + with helpers.patch_socket_create_connection() as mock_connection: |
1252 | + error = netutils.check_listening('1.2.3.4:17070') |
1253 | + self.assertIsNone(error) |
1254 | + mock_connection.assert_called_once_with(('1.2.3.4', 17070), 3) |
1255 | + |
1256 | + def test_timeout(self): |
1257 | + # A customized timeout can be passed. |
1258 | + with helpers.patch_socket_create_connection() as mock_connection: |
1259 | + error = netutils.check_listening('1.2.3.4:17070', timeout=42) |
1260 | + self.assertIsNone(error) |
1261 | + mock_connection.assert_called_once_with(('1.2.3.4', 17070), 42) |
1262 | + |
1263 | + def test_address_not_valid(self): |
1264 | + # An error message is returned if the address is not valid. |
1265 | + with helpers.patch_socket_create_connection() as mock_connection: |
1266 | + error = netutils.check_listening('1.2.3.4') |
1267 | + self.assertEqual( |
1268 | + 'cannot connect to 1.2.3.4: need more than 1 value to unpack', |
1269 | + error) |
1270 | + self.assertFalse(mock_connection.called) |
1271 | + |
1272 | + def test_port_not_valid(self): |
1273 | + # An error message is returned if the address port is not valid. |
1274 | + with helpers.patch_socket_create_connection() as mock_connection: |
1275 | + error = netutils.check_listening('1.2.3.4:bad-port') |
1276 | + self.assertEqual( |
1277 | + 'cannot connect to 1.2.3.4:bad-port: ' |
1278 | + "invalid literal for int() with base 10: 'bad-port'", |
1279 | + error) |
1280 | + self.assertFalse(mock_connection.called) |
1281 | + |
1282 | + def test_not_listening(self): |
1283 | + # An error message is returned if the address is not reachable. |
1284 | + with helpers.patch_socket_create_connection('boo!') as mock_connection: |
1285 | + error = netutils.check_listening('1.2.3.4:17070') |
1286 | + self.assertEqual('cannot connect to 1.2.3.4:17070: boo!', error) |
1287 | + mock_connection.assert_called_once_with(('1.2.3.4', 17070), 3) |
1288 | + |
1289 | + def test_closing(self): |
1290 | + # The socket connection is properly closed. |
1291 | + with helpers.patch_socket_create_connection() as mock_connection: |
1292 | + netutils.check_listening('1.2.3.4:17070') |
1293 | + mock_connection().close.assert_called_once_with() |
1294 | + |
1295 | + def test_error_closing(self): |
1296 | + # Errors closing the socket connection are ignored. |
1297 | + with helpers.patch_socket_create_connection() as mock_connection: |
1298 | + mock_connection().close.side_effect = socket.error('bad wolf') |
1299 | + netutils.check_listening('1.2.3.4:17070') |
1300 | + |
1301 | + |
1302 | +class TestGetCharmUrl(helpers.UrlReadTestsMixin, unittest.TestCase): |
1303 | + |
1304 | + def test_charm_url(self): |
1305 | + # The Juju GUI charm URL is correctly returned. |
1306 | + contents = json.dumps({'charm': {'url': 'cs:trusty/juju-gui-42'}}) |
1307 | + with self.patch_urlread(contents=contents) as mock_urlread: |
1308 | + charm_url = netutils.get_charm_url('trusty') |
1309 | + self.assertEqual('cs:trusty/juju-gui-42', charm_url) |
1310 | + mock_urlread.assert_called_once_with( |
1311 | + 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui') |
1312 | + |
1313 | + def test_io_error(self): |
1314 | + # IOErrors are properly propagated. |
1315 | + with self.patch_urlread(error=True) as mock_urlread: |
1316 | + with self.assertRaises(IOError) as context_manager: |
1317 | + netutils.get_charm_url('precise') |
1318 | + mock_urlread.assert_called_once_with( |
1319 | + 'http://manage.jujucharms.com/api/3/charm/precise/juju-gui') |
1320 | + self.assertEqual('bad wolf', bytes(context_manager.exception)) |
1321 | + |
1322 | + def test_value_error(self): |
1323 | + # A ValueError is raised if the API response is not valid. |
1324 | + contents = json.dumps({'charm': {}}) |
1325 | + with self.patch_urlread(contents=contents) as mock_urlread: |
1326 | + with self.assertRaises(ValueError) as context_manager: |
1327 | + netutils.get_charm_url('trusty') |
1328 | + mock_urlread.assert_called_once_with( |
1329 | + 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui') |
1330 | + self.assertEqual( |
1331 | + 'unable to find the charm URL', bytes(context_manager.exception)) |
1332 | + |
1333 | + |
1334 | +class TestUrlread(unittest.TestCase): |
1335 | + |
1336 | + def patch_urlopen(self, contents=None, error=None, content_type=None): |
1337 | + """Patch the urllib2.urlopen function. |
1338 | + |
1339 | + If contents is not None, the read() method of the returned mock object |
1340 | + returns the given contents. |
1341 | + If content_type is provided, the response includes the content type. |
1342 | + If an error is provided, the call raises the error. |
1343 | + """ |
1344 | + mock_urlopen = mock.MagicMock() |
1345 | + if contents is not None: |
1346 | + mock_urlopen().read.return_value = contents |
1347 | + if content_type is not None: |
1348 | + mock_urlopen().headers = {'content-type': content_type} |
1349 | + if error is not None: |
1350 | + mock_urlopen.side_effect = error |
1351 | + mock_urlopen.reset_mock() |
1352 | + return mock.patch('urllib2.urlopen', mock_urlopen) |
1353 | + |
1354 | + def test_contents(self): |
1355 | + # The URL contents are correctly returned. |
1356 | + with self.patch_urlopen(contents=b'URL contents') as mock_urlopen: |
1357 | + contents = netutils.urlread('http://example.com/path/') |
1358 | + self.assertEqual('URL contents', contents) |
1359 | + self.assertIsInstance(contents, unicode) |
1360 | + mock_urlopen.assert_called_once_with('http://example.com/path/') |
1361 | + |
1362 | + def test_content_type(self): |
1363 | + # The URL contents are decoded using the site charset. |
1364 | + patch_urlopen = self.patch_urlopen( |
1365 | + contents=b'URL contents: \xf8', # This is not a UTF-8 byte string. |
1366 | + content_type='text/html; charset=ISO-8859-1') |
1367 | + with patch_urlopen as mock_urlopen: |
1368 | + contents = netutils.urlread('http://example.com/path/') |
1369 | + self.assertEqual('URL contents: \xf8', contents) |
1370 | + self.assertIsInstance(contents, unicode) |
1371 | + mock_urlopen.assert_called_once_with('http://example.com/path/') |
1372 | + |
1373 | + def test_no_content_type(self): |
1374 | + # The URL contents are decoded with UTF-8 by default. |
1375 | + patch_urlopen = self.patch_urlopen( |
1376 | + contents=b'URL contents: \xf8', # This is not a UTF-8 byte string. |
1377 | + content_type='text/html') |
1378 | + with patch_urlopen as mock_urlopen: |
1379 | + contents = netutils.urlread('http://example.com/path/') |
1380 | + self.assertEqual('URL contents: ', contents) |
1381 | + self.assertIsInstance(contents, unicode) |
1382 | + mock_urlopen.assert_called_once_with('http://example.com/path/') |
1383 | + |
1384 | + def test_errors(self): |
1385 | + # An IOError is raised if an error occurs connecting to the API. |
1386 | + errors = { |
1387 | + 'httplib HTTPException': httplib.HTTPException, |
1388 | + 'socket error': socket.error, |
1389 | + 'urllib2 URLError': urllib2.URLError, |
1390 | + } |
1391 | + for message, exception_class in errors.items(): |
1392 | + exception = exception_class(message) |
1393 | + with self.patch_urlopen(error=exception) as mock_urlopen: |
1394 | + with self.assertRaises(IOError) as context_manager: |
1395 | + netutils.urlread('http://example.com/path/') |
1396 | + mock_urlopen.assert_called_once_with('http://example.com/path/') |
1397 | + self.assertEqual(message, bytes(context_manager.exception)) |
1398 | |
1399 | === modified file 'quickstart/tests/test_utils.py' |
1400 | --- quickstart/tests/test_utils.py 2014-11-10 09:08:44 +0000 |
1401 | +++ quickstart/tests/test_utils.py 2014-11-12 16:41:04 +0000 |
1402 | @@ -19,14 +19,11 @@ |
1403 | from __future__ import unicode_literals |
1404 | |
1405 | import datetime |
1406 | -import httplib |
1407 | import json |
1408 | import os |
1409 | import shutil |
1410 | -import socket |
1411 | import tempfile |
1412 | import unittest |
1413 | -import urllib2 |
1414 | |
1415 | import mock |
1416 | import yaml |
1417 | @@ -156,24 +153,6 @@ |
1418 | utils.call('echo', 'we are the borg!') |
1419 | |
1420 | |
1421 | -class TestCheckResolvable(unittest.TestCase): |
1422 | - |
1423 | - def test_resolvable(self): |
1424 | - # None is returned if the hostname can be resolved. |
1425 | - expected_log = 'example.com resolved to 1.2.3.4' |
1426 | - with helpers.assert_logs([expected_log], level='debug'): |
1427 | - with mock.patch('socket.gethostbyname', return_value='1.2.3.4'): |
1428 | - error = utils.check_resolvable('example.com') |
1429 | - self.assertIsNone(error) |
1430 | - |
1431 | - def test_not_resolvable(self): |
1432 | - # An error message is returned if the hostname cannot be resolved. |
1433 | - exception = socket.gaierror('bad wolf') |
1434 | - with mock.patch('socket.gethostbyname', side_effect=exception): |
1435 | - error = utils.check_resolvable('example.com') |
1436 | - self.assertEqual('bad wolf', error) |
1437 | - |
1438 | - |
1439 | @mock.patch('__builtin__.print', mock.Mock()) |
1440 | class TestParseGuiCharmUrl(unittest.TestCase): |
1441 | |
1442 | @@ -323,38 +302,6 @@ |
1443 | utils.convert_bundle_url(url) |
1444 | |
1445 | |
1446 | -class TestGetCharmUrl(helpers.UrlReadTestsMixin, unittest.TestCase): |
1447 | - |
1448 | - def test_charm_url(self): |
1449 | - # The Juju GUI charm URL is correctly returned. |
1450 | - contents = json.dumps({'charm': {'url': 'cs:trusty/juju-gui-42'}}) |
1451 | - with self.patch_urlread(contents=contents) as mock_urlread: |
1452 | - charm_url = utils.get_charm_url('trusty') |
1453 | - self.assertEqual('cs:trusty/juju-gui-42', charm_url) |
1454 | - mock_urlread.assert_called_once_with( |
1455 | - 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui') |
1456 | - |
1457 | - def test_io_error(self): |
1458 | - # IOErrors are properly propagated. |
1459 | - with self.patch_urlread(error=True) as mock_urlread: |
1460 | - with self.assertRaises(IOError) as context_manager: |
1461 | - utils.get_charm_url('precise') |
1462 | - mock_urlread.assert_called_once_with( |
1463 | - 'http://manage.jujucharms.com/api/3/charm/precise/juju-gui') |
1464 | - self.assertEqual('bad wolf', bytes(context_manager.exception)) |
1465 | - |
1466 | - def test_value_error(self): |
1467 | - # A ValueError is raised if the API response is not valid. |
1468 | - contents = json.dumps({'charm': {}}) |
1469 | - with self.patch_urlread(contents=contents) as mock_urlread: |
1470 | - with self.assertRaises(ValueError) as context_manager: |
1471 | - utils.get_charm_url('trusty') |
1472 | - mock_urlread.assert_called_once_with( |
1473 | - 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui') |
1474 | - self.assertEqual( |
1475 | - 'unable to find the charm URL', bytes(context_manager.exception)) |
1476 | - |
1477 | - |
1478 | class TestGetQuickstartBanner(unittest.TestCase): |
1479 | |
1480 | def patch_datetime(self): |
1481 | @@ -704,72 +651,6 @@ |
1482 | self.assertEqual(list.append.__doc__, self.func.__doc__) |
1483 | |
1484 | |
1485 | -class TestUrlread(unittest.TestCase): |
1486 | - |
1487 | - def patch_urlopen(self, contents=None, error=None, content_type=None): |
1488 | - """Patch the urllib2.urlopen function. |
1489 | - |
1490 | - If contents is not None, the read() method of the returned mock object |
1491 | - returns the given contents. |
1492 | - If content_type is provided, the response includes the content type. |
1493 | - If an error is provided, the call raises the error. |
1494 | - """ |
1495 | - mock_urlopen = mock.MagicMock() |
1496 | - if contents is not None: |
1497 | - mock_urlopen().read.return_value = contents |
1498 | - if content_type is not None: |
1499 | - mock_urlopen().headers = {'content-type': content_type} |
1500 | - if error is not None: |
1501 | - mock_urlopen.side_effect = error |
1502 | - mock_urlopen.reset_mock() |
1503 | - return mock.patch('urllib2.urlopen', mock_urlopen) |
1504 | - |
1505 | - def test_contents(self): |
1506 | - # The URL contents are correctly returned. |
1507 | - with self.patch_urlopen(contents=b'URL contents') as mock_urlopen: |
1508 | - contents = utils.urlread('http://example.com/path/') |
1509 | - self.assertEqual('URL contents', contents) |
1510 | - self.assertIsInstance(contents, unicode) |
1511 | - mock_urlopen.assert_called_once_with('http://example.com/path/') |
1512 | - |
1513 | - def test_content_type(self): |
1514 | - # The URL contents are decoded using the site charset. |
1515 | - patch_urlopen = self.patch_urlopen( |
1516 | - contents=b'URL contents: \xf8', # This is not a UTF-8 byte string. |
1517 | - content_type='text/html; charset=ISO-8859-1') |
1518 | - with patch_urlopen as mock_urlopen: |
1519 | - contents = utils.urlread('http://example.com/path/') |
1520 | - self.assertEqual('URL contents: \xf8', contents) |
1521 | - self.assertIsInstance(contents, unicode) |
1522 | - mock_urlopen.assert_called_once_with('http://example.com/path/') |
1523 | - |
1524 | - def test_no_content_type(self): |
1525 | - # The URL contents are decoded with UTF-8 by default. |
1526 | - patch_urlopen = self.patch_urlopen( |
1527 | - contents=b'URL contents: \xf8', # This is not a UTF-8 byte string. |
1528 | - content_type='text/html') |
1529 | - with patch_urlopen as mock_urlopen: |
1530 | - contents = utils.urlread('http://example.com/path/') |
1531 | - self.assertEqual('URL contents: ', contents) |
1532 | - self.assertIsInstance(contents, unicode) |
1533 | - mock_urlopen.assert_called_once_with('http://example.com/path/') |
1534 | - |
1535 | - def test_errors(self): |
1536 | - # An IOError is raised if an error occurs connecting to the API. |
1537 | - errors = { |
1538 | - 'httplib HTTPException': httplib.HTTPException, |
1539 | - 'socket error': socket.error, |
1540 | - 'urllib2 URLError': urllib2.URLError, |
1541 | - } |
1542 | - for message, exception_class in errors.items(): |
1543 | - exception = exception_class(message) |
1544 | - with self.patch_urlopen(error=exception) as mock_urlopen: |
1545 | - with self.assertRaises(IOError) as context_manager: |
1546 | - utils.urlread('http://example.com/path/') |
1547 | - mock_urlopen.assert_called_once_with('http://example.com/path/') |
1548 | - self.assertEqual(message, bytes(context_manager.exception)) |
1549 | - |
1550 | - |
1551 | class TestGetJujuVersion( |
1552 | helpers.CallTestsMixin, helpers.ValueErrorTestsMixin, |
1553 | unittest.TestCase): |
1554 | |
1555 | === modified file 'quickstart/utils.py' |
1556 | --- quickstart/utils.py 2014-11-10 09:08:44 +0000 |
1557 | +++ quickstart/utils.py 2014-11-12 16:41:04 +0000 |
1558 | @@ -25,15 +25,11 @@ |
1559 | import datetime |
1560 | import errno |
1561 | import functools |
1562 | -import httplib |
1563 | -import json |
1564 | import logging |
1565 | import os |
1566 | import pipes |
1567 | import re |
1568 | -import socket |
1569 | import subprocess |
1570 | -import urllib2 |
1571 | |
1572 | import quickstart |
1573 | from quickstart import ( |
1574 | @@ -105,20 +101,6 @@ |
1575 | return retcode, output.decode('utf-8'), error.decode('utf-8') |
1576 | |
1577 | |
1578 | -def check_resolvable(hostname): |
1579 | - """Check that the hostname can be resolved to a numeric IP address. |
1580 | - |
1581 | - Return an error message if the address cannot be resolved. |
1582 | - """ |
1583 | - try: |
1584 | - address = socket.gethostbyname(hostname) |
1585 | - except socket.error as err: |
1586 | - return bytes(err).decode('utf-8') |
1587 | - logging.debug('{} resolved to {}'.format( |
1588 | - hostname, address.decode('utf-8'))) |
1589 | - return None |
1590 | - |
1591 | - |
1592 | def parse_gui_charm_url(charm_url): |
1593 | """Parse the given charm URL. |
1594 | |
1595 | @@ -164,21 +146,6 @@ |
1596 | bundle_id) |
1597 | |
1598 | |
1599 | -def get_charm_url(series): |
1600 | - """Return the charm URL of the latest Juju GUI charm revision. |
1601 | - |
1602 | - Raise an IOError if any problems occur connecting to the API endpoint. |
1603 | - Raise a ValueError if the API returns invalid data. |
1604 | - """ |
1605 | - url = settings.CHARMWORLD_API.format( |
1606 | - series=series, charm=settings.JUJU_GUI_CHARM_NAME) |
1607 | - charm_info = json.loads(urlread(url)) |
1608 | - charm_url = charm_info.get('charm', {}).get('url') |
1609 | - if charm_url is None: |
1610 | - raise ValueError(b'unable to find the charm URL') |
1611 | - return charm_url |
1612 | - |
1613 | - |
1614 | def get_quickstart_banner(): |
1615 | """Return a quickstart banner suitable for being included in files. |
1616 | |
1617 | @@ -390,24 +357,3 @@ |
1618 | return function(*args, **kwargs) |
1619 | decorated.called = False |
1620 | return decorated |
1621 | - |
1622 | - |
1623 | -def urlread(url): |
1624 | - """Open the given URL and return the page contents. |
1625 | - |
1626 | - Raise an IOError if any problems occur. |
1627 | - """ |
1628 | - try: |
1629 | - response = urllib2.urlopen(url) |
1630 | - except urllib2.URLError as err: |
1631 | - raise IOError(err.reason) |
1632 | - except (httplib.HTTPException, socket.error, urllib2.HTTPError) as err: |
1633 | - raise IOError(bytes(err)) |
1634 | - contents = response.read() |
1635 | - content_type = response.headers['content-type'] |
1636 | - charset = 'utf-8' |
1637 | - if 'charset=' in content_type: |
1638 | - sent_charset = content_type.split('charset=')[-1].strip() |
1639 | - if sent_charset: |
1640 | - charset = sent_charset |
1641 | - return contents.decode(charset, 'ignore') |
1642 | |
1643 | === modified file 'quickstart/watchers.py' |
1644 | --- quickstart/watchers.py 2014-11-10 10:28:19 +0000 |
1645 | +++ quickstart/watchers.py 2014-11-12 16:41:04 +0000 |
1646 | @@ -23,7 +23,7 @@ |
1647 | |
1648 | import logging |
1649 | |
1650 | -from quickstart import utils |
1651 | +from quickstart import netutils |
1652 | |
1653 | |
1654 | IPV4_ADDRESS = 'ipv4' |
1655 | @@ -150,7 +150,7 @@ |
1656 | if not address: |
1657 | addresses = data.get('Addresses', []) |
1658 | public_address = retrieve_public_adddress( |
1659 | - addresses, utils.check_resolvable) |
1660 | + addresses, netutils.check_resolvable) |
1661 | if public_address is not None: |
1662 | address = public_address |
1663 | print('unit placed on {}'.format(address)) |
Reviewers: mp+241553_ code.launchpad. net,
Message:
Please take a look.
Description:
New bootstrap strategy.
This is a massive branch: my apologies.
But:
- a new module has been created (netutils), so there
are license headers and moreover some existing
functions (with their tests) have been just moved
from utils.py and must not be re-reviewed.
The only new function there is check_listening.
- most of the code are tests: we reached 700 unit
tests yay!
This branch changes the way quickstart is run on an
existing environment: instead of always trying to
bootstrap, it looks for the jenv file for the current
environment, and, if present, it retrieves the API URL
from there. As a consequence, quickstart uses Juju
in a less expensive way, and it's also faster when
invoked on a bootstrapped environment.
Split the app.bootstrap function to two new
functions: app.bootstrap and app.status. The intent
is to make them more reusable and more easy to test.
Fix a subtle bug never reported and not easy to hit:
the environment type of an existing environment is
now retrieved from the jenv, rather than relying on
what's stored in the environments.yaml file.
Also reorganized the tests for manage.run: over
time they ended up failing to achieve their goal
of describing how the application is run, and
become not really easy to update and change.
Now this situation should be improved.
The little change to the HACKING file is to make
the rst to render correctly on sublime text.
Tests: `make check`.
QA: run quickstart as usual on local and ec2.
Run quickstart again on an already bootstrapped
environment (local and ec2). You should no longer
see the "bootstrapping environment" message.
Instead, a more correct "reusing the already
bootstrapped..." message is displayed.
Also this should feel quicker, especially on ec2.
Thank you!
https:/ /code.launchpad .net/~frankban/ juju-quickstart /new-bootstrap- strategy/ +merge/ 241553
(do not edit description out of merge proposal)
Please review this at https:/ /codereview. appspot. com/172380043/
Affected files (+761, -511 lines): manage. py netutils. py tests/helpers. py tests/test_ app.py tests/test_ manage. py tests/test_ netutils. py tests/test_ utils.py watchers. py
M HACKING.rst
A [revision details]
M quickstart/app.py
M quickstart/
A quickstart/
M quickstart/
M quickstart/
M quickstart/
A quickstart/
M quickstart/
M quickstart/utils.py
M quickstart/