Merge lp:~frankban/juju-quickstart/new-bootstrap-strategy into lp:juju-quickstart

Proposed by Francesco Banconi
Status: Merged
Merged at revision: 105
Proposed branch: lp:~frankban/juju-quickstart/new-bootstrap-strategy
Merge into: lp:juju-quickstart
Diff against target: 1663 lines (+770/-511)
11 files modified
HACKING.rst (+2/-1)
quickstart/app.py (+80/-44)
quickstart/manage.py (+38/-17)
quickstart/netutils.py (+99/-0)
quickstart/tests/helpers.py (+19/-5)
quickstart/tests/test_app.py (+195/-147)
quickstart/tests/test_manage.py (+134/-122)
quickstart/tests/test_netutils.py (+201/-0)
quickstart/tests/test_utils.py (+0/-119)
quickstart/utils.py (+0/-54)
quickstart/watchers.py (+2/-2)
To merge this branch: bzr merge lp:~frankban/juju-quickstart/new-bootstrap-strategy
Reviewer Review Type Date Requested Status
Juju GUI Hackers Pending
Review via email: mp+241553@code.launchpad.net

Description of the change

New bootstrap strategy.

This is a massive branch: my apologies.
But:
    - a new module has been created (netutils), so there
      are license headers and moreover some existing
      functions (with their tests) have been just moved
      from utils.py and must not be re-reviewed.
      The only new function there is check_listening.
    - most of the code are tests: we reached 700 unit
      tests yay!

This branch changes the way quickstart is run on an
existing environment: instead of always trying to
bootstrap, it looks for the jenv file for the current
environment, and, if present, it retrieves the API URL
from there. As a consequence, quickstart uses Juju
in a less expensive way, and it's also faster when
invoked on a bootstrapped environment.

Split the app.bootstrap function to two new
functions: app.bootstrap and app.status. The intent
is to make them more reusable and more easy to test.

Fix a subtle bug never reported and not easy to hit:
the environment type of an existing environment is
now retrieved from the jenv, rather than relying on
what's stored in the environments.yaml file.

Also reorganized the tests for manage.run: over
time they ended up failing to achieve their goal
of describing how the application is run, and
become not really easy to update and change.
Now this situation should be improved.

The little change to the HACKING file is to make
the rst to render correctly on sublime text.

Tests: `make check`.

QA: run quickstart as usual on local and ec2.
Run quickstart again on an already bootstrapped
environment (local and ec2). You should no longer
see the "bootstrapping environment" message.
Instead, a more correct "reusing the already
bootstrapped..." message is displayed.
Also this should feel quicker, especially on ec2.

Thank you!

https://codereview.appspot.com/172380043/

To post a comment you must log in.
Revision history for this message
Francesco Banconi (frankban) wrote :

Reviewers: mp+241553_code.launchpad.net,

Message:
Please take a look.

Description:
New bootstrap strategy.

This is a massive branch: my apologies.
But:
     - a new module has been created (netutils), so there
       are license headers and moreover some existing
       functions (with their tests) have been just moved
       from utils.py and must not be re-reviewed.
       The only new function there is check_listening.
     - most of the code are tests: we reached 700 unit
       tests yay!

This branch changes the way quickstart is run on an
existing environment: instead of always trying to
bootstrap, it looks for the jenv file for the current
environment, and, if present, it retrieves the API URL
from there. As a consequence, quickstart uses Juju
in a less expensive way, and it's also faster when
invoked on a bootstrapped environment.

Split the app.bootstrap function to two new
functions: app.bootstrap and app.status. The intent
is to make them more reusable and more easy to test.

Fix a subtle bug never reported and not easy to hit:
the environment type of an existing environment is
now retrieved from the jenv, rather than relying on
what's stored in the environments.yaml file.

Also reorganized the tests for manage.run: over
time they ended up failing to achieve their goal
of describing how the application is run, and
become not really easy to update and change.
Now this situation should be improved.

The little change to the HACKING file is to make
the rst to render correctly on sublime text.

Tests: `make check`.

QA: run quickstart as usual on local and ec2.
Run quickstart again on an already bootstrapped
environment (local and ec2). You should no longer
see the "bootstrapping environment" message.
Instead, a more correct "reusing the already
bootstrapped..." message is displayed.
Also this should feel quicker, especially on ec2.

Thank you!

https://code.launchpad.net/~frankban/juju-quickstart/new-bootstrap-strategy/+merge/241553

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/172380043/

Affected files (+761, -511 lines):
   M HACKING.rst
   A [revision details]
   M quickstart/app.py
   M quickstart/manage.py
   A quickstart/netutils.py
   M quickstart/tests/helpers.py
   M quickstart/tests/test_app.py
   M quickstart/tests/test_manage.py
   A quickstart/tests/test_netutils.py
   M quickstart/tests/test_utils.py
   M quickstart/utils.py
   M quickstart/watchers.py

Revision history for this message
Brad Crittenden (bac) wrote :

Code LGTM with minor comments. Will QA now.

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py
File quickstart/app.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py#newcode245
quickstart/app.py:245: # For this reason, a call to status() is usually
required at this point.
This is no different from the calling perspective than the 'not already
bootstrapped' situation, right?

If so, perhaps that should be noted in the docstring for this function.

https://codereview.appspot.com/172380043/diff/1/quickstart/manage.py
File quickstart/manage.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/manage.py#newcode520
quickstart/manage.py:520: already_bootstrapped = app.bootstrap(
What is the scenario for api_url being None but it is already
bootstrapped? Someone deleted the jenv file out from under juju?

https://codereview.appspot.com/172380043/diff/1/quickstart/netutils.py
File quickstart/netutils.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/netutils.py#newcode50
quickstart/netutils.py:50: timeout = 3
Make timeout an arg with default = 3?

https://codereview.appspot.com/172380043/

Revision history for this message
Richard Harding (rharding) wrote :

LGTM thank you for the updates.

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py
File quickstart/app.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py#newcode190
quickstart/app.py:190: candidates = jenv.get_value(env_name,
'state-servers')
do you know how this list gets updated from Juju? In an HA situation
does juju rewrite the jenv file?

https://codereview.appspot.com/172380043/diff/1/quickstart/manage.py
File quickstart/manage.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/manage.py#newcode520
quickstart/manage.py:520: already_bootstrapped = app.bootstrap(
On 2014/11/12 14:48:25, bac wrote:
> What is the scenario for api_url being None but it is already
bootstrapped?
> Someone deleted the jenv file out from under juju?

That has happened and there's been bugs/work around the scenario. It's a
good one to watch for.

https://codereview.appspot.com/172380043/

Revision history for this message
Brad Crittenden (bac) wrote :
Revision history for this message
Francesco Banconi (frankban) wrote :

Thanks for the reviews, really good stuff!

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py
File quickstart/app.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py#newcode190
quickstart/app.py:190: candidates = jenv.get_value(env_name,
'state-servers')
On 2014/11/12 15:30:44, rharding wrote:
> do you know how this list gets updated from Juju? In an HA situation
does juju
> rewrite the jenv file?

I don't know that for sure: it could be the case when you set HA from
the CLI. I would be surprised if the jenv is updated when you will set
HA using the API. In case of stale addresses, we just fall back to the
old bootstrap strategy.

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py#newcode245
quickstart/app.py:245: # For this reason, a call to status() is usually
required at this point.
On 2014/11/12 14:48:25, bac wrote:
> This is no different from the calling perspective than the 'not
already
> bootstrapped' situation, right?

> If so, perhaps that should be noted in the docstring for this
function.

Done.

https://codereview.appspot.com/172380043/diff/1/quickstart/netutils.py
File quickstart/netutils.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/netutils.py#newcode50
quickstart/netutils.py:50: timeout = 3
On 2014/11/12 14:48:25, bac wrote:
> Make timeout an arg with default = 3?

Done.

https://codereview.appspot.com/172380043/

111. By Francesco Banconi

Changes as per review.

Revision history for this message
Francesco Banconi (frankban) wrote :

*** Submitted:

New bootstrap strategy.

This is a massive branch: my apologies.
But:
     - a new module has been created (netutils), so there
       are license headers and moreover some existing
       functions (with their tests) have been just moved
       from utils.py and must not be re-reviewed.
       The only new function there is check_listening.
     - most of the code are tests: we reached 700 unit
       tests yay!

This branch changes the way quickstart is run on an
existing environment: instead of always trying to
bootstrap, it looks for the jenv file for the current
environment, and, if present, it retrieves the API URL
from there. As a consequence, quickstart uses Juju
in a less expensive way, and it's also faster when
invoked on a bootstrapped environment.

Split the app.bootstrap function to two new
functions: app.bootstrap and app.status. The intent
is to make them more reusable and more easy to test.

Fix a subtle bug never reported and not easy to hit:
the environment type of an existing environment is
now retrieved from the jenv, rather than relying on
what's stored in the environments.yaml file.

Also reorganized the tests for manage.run: over
time they ended up failing to achieve their goal
of describing how the application is run, and
become not really easy to update and change.
Now this situation should be improved.

The little change to the HACKING file is to make
the rst to render correctly on sublime text.

Tests: `make check`.

QA: run quickstart as usual on local and ec2.
Run quickstart again on an already bootstrapped
environment (local and ec2). You should no longer
see the "bootstrapping environment" message.
Instead, a more correct "reusing the already
bootstrapped..." message is displayed.
Also this should feel quicker, especially on ec2.

Thank you!

R=bac, rharding
CC=
https://codereview.appspot.com/172380043

https://codereview.appspot.com/172380043/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'HACKING.rst'
--- HACKING.rst 2014-11-11 14:15:47 +0000
+++ HACKING.rst 2014-11-12 16:41:04 +0000
@@ -152,7 +152,8 @@
152152
153* Verify an environment that has already been bootstrapped is recogized and153* Verify an environment that has already been bootstrapped is recogized and
154 the GUI is deployed. This test also shows that a remote bundle is properly154 the GUI is deployed. This test also shows that a remote bundle is properly
155 deployed::155 deployed
156::
156157
157 juju bootstrap -e local158 juju bootstrap -e local
158 juju quickstart -e local bundle:mediawiki/single159 juju quickstart -e local bundle:mediawiki/single
159160
=== modified file 'quickstart/app.py'
--- quickstart/app.py 2014-11-11 13:21:57 +0000
+++ quickstart/app.py 2014-11-12 16:41:04 +0000
@@ -30,6 +30,7 @@
3030
31from quickstart import (31from quickstart import (
32 juju,32 juju,
33 netutils,
33 platform_support,34 platform_support,
34 settings,35 settings,
35 ssh,36 ssh,
@@ -175,28 +176,55 @@
175 raise ProgramExit(bytes(err))176 raise ProgramExit(bytes(err))
176177
177178
179def check_bootstrapped(env_name):
180 """Check if the environment named env_name is already bootstrapped.
181
182 If so, return the environment API URL to be used to connect to the Juju API
183 server. If not already bootstrapped, or if the API URL cannot be retrieved,
184 return None.
185 """
186 if not jenv.exists(env_name):
187 return None
188 # Retrieve the Juju API addresses from the jenv file.
189 try:
190 candidates = jenv.get_value(env_name, 'state-servers')
191 except ValueError as err:
192 logging.warn(b'cannot retrieve the Juju API URL: {}'.format(err))
193 return None
194 # Look for a reachable API URL.
195 if not candidates:
196 logging.warn('cannot retrieve the Juju API URL: no addresses found')
197 return None
198 for candidate in candidates:
199 error = netutils.check_listening(candidate)
200 if error is None:
201 # Juju API URL found.
202 return 'wss://{}'.format(candidate)
203 logging.debug(error)
204 logging.warn(
205 'cannot retrieve the Juju API URL: cannot connect to any of the '
206 'following addresses: {}'.format(', '.join(candidates)))
207 return None
208
209
178def bootstrap(210def bootstrap(
179 env_name, juju_command, debug=False, upload_tools=False,211 env_name, juju_command, debug=False, upload_tools=False,
180 upload_series=None, constraints=None):212 upload_series=None, constraints=None):
181 """Bootstrap the Juju environment with the given name.213 """Bootstrap the Juju environment with the given name.
182214
215 Return a flag indicating whether the environment was already bootstrapped.
216
183 Do not bootstrap the environment if already bootstrapped.217 Do not bootstrap the environment if already bootstrapped.
184218 If the environment is not bootstrapped, execute the bootstrap command with
185 Return a tuple (already_bootstrapped, series) in which:219 the given juju_command, debug, upload_tools, upload_series and constraints
186 - already_bootstrapped indicates whether the environment was already220 arguments.
187 bootstrapped;221
188 - series is the bootstrap node Ubuntu series.222 When the function exists the Juju environment is bootstrapped, but we don't
189223 know if it is ready yet. For this reason, a call to status() is usually
190 The is_local argument indicates whether the environment is configured to224 required at that point.
191 use the local provider. If so, sudo privileges are requested in order to
192 bootstrap the environment.
193
194 If debug is True and the environment not bootstrapped, execute the
195 bootstrap command passing the --debug flag.
196225
197 Raise a ProgramExit if any error occurs in the bootstrap process.226 Raise a ProgramExit if any error occurs in the bootstrap process.
198 """227 """
199 already_bootstrapped = False
200 cmd = [juju_command, 'bootstrap', '-e', env_name]228 cmd = [juju_command, 'bootstrap', '-e', env_name]
201 if debug:229 if debug:
202 cmd.append('--debug')230 cmd.append('--debug')
@@ -207,34 +235,28 @@
207 if constraints is not None:235 if constraints is not None:
208 cmd.extend(['--constraints', constraints])236 cmd.extend(['--constraints', constraints])
209 retcode, _, error = utils.call(*cmd)237 retcode, _, error = utils.call(*cmd)
210 if retcode:238 if not retcode:
211 # XXX frankban 2013-11-13 bug 1252322: the check below is weak. We are239 return False
212 # relying on an error message in order to decide if the environment is240 # XXX frankban 2013-11-13 bug 1252322: the check below is weak. We are
213 # already bootstrapped. Other possibilities include checking if the241 # relying on an error message in order to decide if the environment is
214 # jenv file is present (in ~/.juju/environments/) and, if so, check the242 # already bootstrapped. Also note that, rather than comparing the expected
215 # juju status. Unfortunately this is also prone to errors, because a243 # error with the obtained one, we search in the error in order to support
216 # jenv file can be there but the environment not really bootstrapped or244 # bootstrap --debug.
217 # functional (e.g. sync-tools was used, or a previous bootstrap failed,245 if 'environment is already bootstrapped' not in error:
218 # or the user terminated machines from the ec2 panel, etc.). Moreover246 # We exit if the error is not "already bootstrapped".
219 # jenv files seems to be an internal juju-core detail. Definitely we247 raise ProgramExit(error)
220 # need to find a better way, but for now the "asking forgiveness"248 # Juju is already bootstrapped.
221 # approach feels like the best compromise we have. Also note that,249 return True
222 # rather than comparing the expected error with the obtained one, we250
223 # search in the error in order to support bootstrap --debug.251
224 if 'environment is already bootstrapped' not in error:252def status(env_name, juju_command):
225 # We exit if the error is not "already bootstrapped".253 """Call "juju status" multiple times until the bootstrap node is ready.
226 raise ProgramExit(error)254
227 # Juju is bootstrapped, but we don't know if it is ready yet. Fall255 Return the bootstrap node series of the Juju environment.
228 # through to the next block for that check.256
229 already_bootstrapped = True257 Raise a ProgramExit if the agent is not ready after ten minutes or if the
230 print('reusing the already bootstrapped {} environment'.format(258 agent is in an error state.
231 env_name))259 """
232 # Call "juju status" multiple times until the bootstrap node is ready.
233 # Exit with an error if the agent is not ready after ten minutes.
234 # Note: when using the local provider, calling "juju status" is very fast,
235 # but e.g. on ec2 the first call (right after "bootstrap") can take
236 # several minutes, and subsequent calls are relatively fast (seconds).
237 print('retrieving the environment status')
238 timeout = time.time() + (60*10)260 timeout = time.time() + (60*10)
239 while time.time() < timeout:261 while time.time() < timeout:
240 retcode, output, error = utils.call(262 retcode, output, error = utils.call(
@@ -247,8 +269,7 @@
247 except ValueError:269 except ValueError:
248 continue270 continue
249 if agent_state == 'started':271 if agent_state == 'started':
250 series = utils.get_bootstrap_node_series(output)272 return utils.get_bootstrap_node_series(output)
251 return already_bootstrapped, series
252 # If the agent is in an error state, there is nothing we can do, and273 # If the agent is in an error state, there is nothing we can do, and
253 # it's not useful to keep trying.274 # it's not useful to keep trying.
254 if agent_state == 'error':275 if agent_state == 'error':
@@ -257,6 +278,21 @@
257 raise ProgramExit('the state server is not ready:\n{}'.format(details))278 raise ProgramExit('the state server is not ready:\n{}'.format(details))
258279
259280
281def get_env_type(env_name):
282 """Return the Juju environment type for the given environment name.
283
284 Since the environment type is retrieved by parsing the jenv file, the
285 environment must be already bootstrapped.
286
287 Raise a ProgramExit if the environment type cannot be retrieved.
288 """
289 try:
290 return jenv.get_value(env_name, 'bootstrap-config', 'type')
291 except ValueError as err:
292 msg = b'cannot retrieve environment type: {}'.format(err)
293 raise ProgramExit(msg)
294
295
260def get_admin_secret(env_name):296def get_admin_secret(env_name):
261 """Return the Juju admin secret for the given environment name.297 """Return the Juju admin secret for the given environment name.
262298
@@ -385,7 +421,7 @@
385 series = settings.JUJU_GUI_SUPPORTED_SERIES[-1]421 series = settings.JUJU_GUI_SUPPORTED_SERIES[-1]
386 try:422 try:
387 # Try to get the charm URL from charmworld.423 # Try to get the charm URL from charmworld.
388 charm_url = utils.get_charm_url(series)424 charm_url = netutils.get_charm_url(series)
389 except (IOError, ValueError) as err:425 except (IOError, ValueError) as err:
390 # Fall back to the default URL for the current series.426 # Fall back to the default URL for the current series.
391 msg = 'unable to retrieve the {} charm URL from the API: {}'427 msg = 'unable to retrieve the {} charm URL from the API: {}'
392428
=== modified file 'quickstart/manage.py'
--- quickstart/manage.py 2014-11-11 11:10:33 +0000
+++ quickstart/manage.py 2014-11-12 16:41:04 +0000
@@ -32,6 +32,7 @@
32import quickstart32import quickstart
33from quickstart import (33from quickstart import (
34 app,34 app,
35 netutils,
35 packaging,36 packaging,
36 platform_support,37 platform_support,
37 settings,38 settings,
@@ -110,7 +111,7 @@
110 if bundle.startswith('http://') or bundle.startswith('https://'):111 if bundle.startswith('http://') or bundle.startswith('https://'):
111 # Load the bundle from a remote URL.112 # Load the bundle from a remote URL.
112 try:113 try:
113 bundle_yaml = utils.urlread(bundle)114 bundle_yaml = netutils.urlread(bundle)
114 except IOError as err:115 except IOError as err:
115 return parser.error('unable to open bundle URL: {}'.format(err))116 return parser.error('unable to open bundle URL: {}'.format(err))
116 else:117 else:
@@ -504,32 +505,52 @@
504 logging.debug('ensuring SSH keys are available')505 logging.debug('ensuring SSH keys are available')
505 app.ensure_ssh_keys()506 app.ensure_ssh_keys()
506507
507 print('bootstrapping the {} environment (type: {})'.format(508 # Bootstrap the Juju environment or reuse an already bootstrapped one.
508 options.env_name, options.env_type))509 already_bootstrapped = True
509 if options.env_type == 'local':510 env_type = options.env_type
510 # If this is a local environment, notify the user that "sudo" will be511 api_url = app.check_bootstrapped(options.env_name)
511 # required by Juju to bootstrap the application.512 if api_url is None:
512 print('sudo privileges will be required to bootstrap the environment')513 print('bootstrapping the {} environment (type: {})'.format(
513514 options.env_name, env_type))
514 already_bootstrapped, bootstrap_node_series = app.bootstrap(515 if env_type == 'local':
515 options.env_name, juju_command,516 # If this is a local environment, notify the user that "sudo" will
516 debug=options.debug,517 # be required by Juju to bootstrap the environment.
517 upload_tools=options.upload_tools,518 print('sudo privileges will be required to bootstrap the '
518 upload_series=options.upload_series,519 'environment')
519 constraints=options.constraints)520 already_bootstrapped = app.bootstrap(
521 options.env_name, juju_command,
522 debug=options.debug,
523 upload_tools=options.upload_tools,
524 upload_series=options.upload_series,
525 constraints=options.constraints)
526 if already_bootstrapped:
527 # Retrieve the environment type from the jenv file: it may be different
528 # from the one declared on the environments.yaml file.
529 env_type = app.get_env_type(options.env_name)
530 print('reusing the already bootstrapped {} environment '
531 '(type: {})'.format(options.env_name, env_type))
532
533 # Retrieve the environment status, ensure it is in a ready state and
534 # contextually fetch the bootstrap node series.
535 print('retrieving the environment status')
536 bootstrap_node_series = app.status(options.env_name, juju_command)
537
538 # If the environment was not already bootstrapped, we need to retrieve
539 # the API address.
540 if api_url is None:
541 print('retrieving the Juju API address')
542 api_url = app.get_api_url(options.env_name, juju_command)
520543
521 # Retrieve the admin-secret for the current environment.544 # Retrieve the admin-secret for the current environment.
522 admin_secret = app.get_admin_secret(options.env_name)545 admin_secret = app.get_admin_secret(options.env_name)
523546
524 print('retrieving the Juju API address')
525 api_url = app.get_api_url(options.env_name, juju_command)
526 print('connecting to {}'.format(api_url))547 print('connecting to {}'.format(api_url))
527 env = app.connect(api_url, admin_secret)548 env = app.connect(api_url, admin_secret)
528549
529 # Inspect the environment and deploy the charm if required.550 # Inspect the environment and deploy the charm if required.
530 charm_url, machine, service_data, unit_data = app.check_environment(551 charm_url, machine, service_data, unit_data = app.check_environment(
531 env, settings.JUJU_GUI_SERVICE_NAME, options.charm_url,552 env, settings.JUJU_GUI_SERVICE_NAME, options.charm_url,
532 options.env_type, bootstrap_node_series, already_bootstrapped)553 env_type, bootstrap_node_series, already_bootstrapped)
533 unit_name = app.deploy_gui(554 unit_name = app.deploy_gui(
534 env, settings.JUJU_GUI_SERVICE_NAME, charm_url, machine,555 env, settings.JUJU_GUI_SERVICE_NAME, charm_url, machine,
535 service_data, unit_data)556 service_data, unit_data)
536557
=== added file 'quickstart/netutils.py'
--- quickstart/netutils.py 1970-01-01 00:00:00 +0000
+++ quickstart/netutils.py 2014-11-12 16:41:04 +0000
@@ -0,0 +1,99 @@
1# This file is part of the Juju Quickstart Plugin, which lets users set up a
2# Juju environment in very few steps (https://launchpad.net/juju-quickstart).
3# Copyright (C) 2014 Canonical Ltd.
4#
5# This program is free software: you can redistribute it and/or modify it under
6# the terms of the GNU Affero General Public License version 3, as published by
7# the Free Software Foundation.
8#
9# This program is distributed in the hope that it will be useful, but WITHOUT
10# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
11# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
12# Affero General Public License for more details.
13#
14# You should have received a copy of the GNU Affero General Public License
15# along with this program. If not, see <http://www.gnu.org/licenses/>.
16
17"""Juju Quickstart network utility functions."""
18
19from __future__ import unicode_literals
20
21import json
22import httplib
23import logging
24import socket
25import urllib2
26
27from quickstart import settings
28
29
30def check_resolvable(hostname):
31 """Check that the hostname can be resolved to a numeric IP address.
32
33 Return an error message if the address cannot be resolved.
34 """
35 try:
36 address = socket.gethostbyname(hostname)
37 except socket.error as err:
38 return bytes(err).decode('utf-8')
39 logging.debug('{} resolved to {}'.format(
40 hostname, address.decode('utf-8')))
41 return None
42
43
44def check_listening(address, timeout=3):
45 """Check that the given address is listening and accepts connections.
46
47 The address must be specified as a "host:port" string.
48 Use the given socket timeout in seconds.
49
50 Return an error message if connecting to the address fails.
51 """
52 try:
53 host, port = address.split(":")
54 sock = socket.create_connection((host, int(port)), timeout)
55 except (socket.error, TypeError, ValueError) as err:
56 return 'cannot connect to {}: {}'.format(
57 address, bytes(err).decode('utf-8'))
58 # Ignore all possible connection close exceptions.
59 try:
60 sock.close()
61 except:
62 pass
63 return None
64
65
66def get_charm_url(series):
67 """Return the charm URL of the latest Juju GUI charm revision.
68
69 Raise an IOError if any problems occur connecting to the API endpoint.
70 Raise a ValueError if the API returns invalid data.
71 """
72 url = settings.CHARMWORLD_API.format(
73 series=series, charm=settings.JUJU_GUI_CHARM_NAME)
74 charm_info = json.loads(urlread(url))
75 charm_url = charm_info.get('charm', {}).get('url')
76 if charm_url is None:
77 raise ValueError(b'unable to find the charm URL')
78 return charm_url
79
80
81def urlread(url):
82 """Open the given URL and return the page contents.
83
84 Raise an IOError if any problems occur.
85 """
86 try:
87 response = urllib2.urlopen(url)
88 except urllib2.URLError as err:
89 raise IOError(err.reason)
90 except (httplib.HTTPException, socket.error, urllib2.HTTPError) as err:
91 raise IOError(bytes(err))
92 contents = response.read()
93 content_type = response.headers['content-type']
94 charset = 'utf-8'
95 if 'charset=' in content_type:
96 sent_charset = content_type.split('charset=')[-1].strip()
97 if sent_charset:
98 charset = sent_charset
99 return contents.decode(charset, 'ignore')
0100
=== modified file 'quickstart/tests/helpers.py'
--- quickstart/tests/helpers.py 2014-11-11 16:39:13 +0000
+++ quickstart/tests/helpers.py 2014-11-12 16:41:04 +0000
@@ -21,6 +21,7 @@
21from contextlib import contextmanager21from contextlib import contextmanager
22import os22import os
23import shutil23import shutil
24import socket
24import tempfile25import tempfile
2526
26import mock27import mock
@@ -145,6 +146,7 @@
145 'bootstrap-config': {146 'bootstrap-config': {
146 'admin-secret': 'Secret!',147 'admin-secret': 'Secret!',
147 'api-port': 17070,148 'api-port': 17070,
149 'type': 'ec2',
148 },150 },
149 'life': {'universe': {'everything': 42}},151 'life': {'universe': {'everything': 42}},
150 }152 }
@@ -231,22 +233,34 @@
231mock_print = mock.patch('__builtin__.print')233mock_print = mock.patch('__builtin__.print')
232234
233235
236def patch_socket_create_connection(error=None):
237 """Patch the socket.create_connection function.
238
239 If error is not None, the mock object raises a socket.error with the given
240 message.
241 """
242 mock_create_connection = mock.Mock()
243 if error is not None:
244 mock_create_connection.side_effect = socket.error(error)
245 return mock.patch('socket.create_connection', mock_create_connection)
246
247
234def patch_check_resolvable(error=None):248def patch_check_resolvable(error=None):
235 """Patch the utils.check_resolvable function to return the given error.249 """Patch the netutils.check_resolvable function to return the given error.
236250
237 This is done so that tests do not try to resolve hostname addresses.251 This is done so that tests do not try to resolve hostname addresses.
238 """252 """
239 return mock.patch(253 return mock.patch(
240 'quickstart.utils.check_resolvable',254 'quickstart.netutils.check_resolvable',
241 lambda hostname: error,255 lambda hostname: error,
242 )256 )
243257
244258
245class UrlReadTestsMixin(object):259class UrlReadTestsMixin(object):
246 """Expose a method to mock the quickstart.utils.urlread helper function."""260 """Helpers to mock the quickstart.netutils.urlread helper function."""
247261
248 def patch_urlread(self, contents=None, error=False):262 def patch_urlread(self, contents=None, error=False):
249 """Patch the quickstart.utils.urlread helper function.263 """Patch the quickstart.netutils.urlread helper function.
250264
251 If contents is not None, urlread() will return the provided contents.265 If contents is not None, urlread() will return the provided contents.
252 If error is set to True, an IOError will be simulated.266 If error is set to True, an IOError will be simulated.
@@ -256,7 +270,7 @@
256 mock_urlread.return_value = contents270 mock_urlread.return_value = contents
257 if error:271 if error:
258 mock_urlread.side_effect = IOError('bad wolf')272 mock_urlread.side_effect = IOError('bad wolf')
259 return mock.patch('quickstart.utils.urlread', mock_urlread)273 return mock.patch('quickstart.netutils.urlread', mock_urlread)
260274
261275
262class ValueErrorTestsMixin(object):276class ValueErrorTestsMixin(object):
263277
=== modified file 'quickstart/tests/test_app.py'
--- quickstart/tests/test_app.py 2014-11-11 13:21:57 +0000
+++ quickstart/tests/test_app.py 2014-11-12 16:41:04 +0000
@@ -454,141 +454,174 @@
454 self.assertTrue(mock_create_keys.called)454 self.assertTrue(mock_create_keys.called)
455455
456456
457@helpers.mock_print457class TestCheckBootstrapped(helpers.JenvFileTestsMixin, unittest.TestCase):
458
459 def test_no_jenv_file(self):
460 # A None API URL is returned if the jenv file is not present.
461 with self.make_jenv('ec2', ''):
462 with helpers.assert_logs([], level='warn'):
463 api_url = app.check_bootstrapped('hp')
464 self.assertIsNone(api_url)
465
466 def test_invalid_jenv_file(self):
467 # A None API URL is returned if the list of API addresses cannot be
468 # retrieved from the jenv file.
469 with self.make_jenv('ec2', '') as path:
470 logs = [
471 'cannot retrieve the Juju API URL: '
472 'invalid YAML contents in {}: '
473 'state-servers key not found in the root section'.format(path)
474 ]
475 with helpers.assert_logs(logs, level='warn'):
476 api_url = app.check_bootstrapped('ec2')
477 self.assertIsNone(api_url)
478
479 def test_no_api_addresses(self):
480 # A None API URL is returned if the list of API addresses is empty.
481 jenv_data = {'state-servers': []}
482 logs = ['cannot retrieve the Juju API URL: no addresses found']
483 with self.make_jenv('local', yaml.safe_dump(jenv_data)):
484 with helpers.assert_logs(logs, level='warn'):
485 api_url = app.check_bootstrapped('local')
486 self.assertIsNone(api_url)
487
488 def test_api_address_not_listening(self):
489 # A None API URL is returned if there is no reachable API address.
490 logs = [
491 'cannot retrieve the Juju API URL: '
492 'cannot connect to any of the following addresses: '
493 'localhost:17070, 10.0.3.1:17070'
494 ]
495 with self.make_jenv('local', yaml.safe_dump(self.jenv_data)):
496 with helpers.assert_logs(logs, level='warn'):
497 with helpers.patch_socket_create_connection('bad wolf'):
498 api_url = app.check_bootstrapped('local')
499 self.assertIsNone(api_url)
500
501 def test_bootstrapped(self):
502 # The first listening API URL is returned if the environment is already
503 # bootstrapped.
504 with self.make_jenv('hp', yaml.safe_dump(self.jenv_data)):
505 with helpers.assert_logs([], level='warn'):
506 with helpers.patch_socket_create_connection():
507 api_url = app.check_bootstrapped('hp')
508 # The first API address is returned.
509 self.assertEqual('wss://localhost:17070', api_url)
510
511
458class TestBootstrap(512class TestBootstrap(
459 helpers.CallTestsMixin, ProgramExitTestsMixin, unittest.TestCase):513 helpers.CallTestsMixin, ProgramExitTestsMixin, unittest.TestCase):
460514
461 env_name = 'my-juju-env'515 def test_environment_not_bootstrapped(self):
462 status_message = 'retrieving the environment status'516 # The environment is successfully bootstrapped and False is returned.
463 juju_command = settings.JUJU_CMD_PATHS['default']517 with self.patch_call(0) as mock_call:
464518 already_bootstrapped = app.bootstrap('ec2', '/usr/bin/juju')
465 def make_status_output(self, agent_state, series='hoary'):519 self.assertFalse(already_bootstrapped)
520 mock_call.assert_called_once_with(
521 '/usr/bin/juju', 'bootstrap', '-e', 'ec2')
522
523 def test_environment_already_bootstrapped(self):
524 # The function succeeds and returns True if the environment is already
525 # bootstrapped.
526 error = '***environment is already bootstrapped***'
527 with self.patch_call(1, error=error) as mock_call:
528 already_bootstrapped = app.bootstrap('hp', '/bin/juju')
529 self.assertTrue(already_bootstrapped)
530 mock_call.assert_called_once_with('/bin/juju', 'bootstrap', '-e', 'hp')
531
532 def test_bootstrap_failure(self):
533 # A ProgramExit is raised if an error occurs while bootstrapping.
534 with self.patch_call(1, error='bad wolf') as mock_call:
535 with self.assert_program_exit('bad wolf'):
536 app.bootstrap('local', 'juju')
537 mock_call.assert_called_once_with('juju', 'bootstrap', '-e', 'local')
538
539 def test_debug(self):
540 # The environment is bootstrapped in debug mode.
541 with self.patch_call(0) as mock_call:
542 app.bootstrap('ec2', '/usr/bin/juju', debug=True)
543 mock_call.assert_called_once_with(
544 '/usr/bin/juju', 'bootstrap', '-e', 'ec2', '--debug')
545
546 def test_upload_tools(self):
547 # The environment is bootstrapped with local tools
548 with self.patch_call(0) as mock_call:
549 app.bootstrap('local', '/usr/bin/juju', upload_tools=True)
550 mock_call.assert_called_once_with(
551 '/usr/bin/juju', 'bootstrap', '-e', 'local', '--upload-tools')
552
553 def test_upload_series(self):
554 # The environment is bootstrapped with tools for specific series.
555 with self.patch_call(0) as mock_call:
556 app.bootstrap('hp', '/usr/bin/juju', upload_series='trusty,utopic')
557 mock_call.assert_called_once_with(
558 '/usr/bin/juju', 'bootstrap', '-e', 'hp',
559 '--upload-series', 'trusty,utopic')
560
561 def test_constraints(self):
562 # The environment is bootstrapped with the specified constraints.
563 with self.patch_call(0) as mock_call:
564 app.bootstrap('maas', '/usr/bin/juju', constraints='mem=7G')
565 mock_call.assert_called_once_with(
566 '/usr/bin/juju', 'bootstrap', '-e', 'maas',
567 '--constraints', 'mem=7G')
568
569 def test_all_options(self):
570 # The environment is bootstrapped with all the options.
571 with self.patch_call(0) as mock_call:
572 app.bootstrap(
573 'local', '/usr/bin/juju', debug=True, upload_tools=True,
574 upload_series='vivid', constraints='mem=8G')
575 mock_call.assert_called_once_with(
576 '/usr/bin/juju', 'bootstrap', '-e', 'local',
577 '--debug', '--upload-tools', '--upload-series', 'vivid',
578 '--constraints', 'mem=8G')
579
580
581class TestStatus(
582 helpers.CallTestsMixin, ProgramExitTestsMixin, unittest.TestCase):
583
584 def make_status_output(self, agent_state, series='utopic'):
466 """Create and return a YAML status output."""585 """Create and return a YAML status output."""
467 return yaml.safe_dump({586 return yaml.safe_dump({
468 'machines': {'0': {'agent-state': agent_state,587 'machines': {
469 'series': series}},588 '0': {'agent-state': agent_state, 'series': series},
589 },
470 })590 })
471591
472 def make_status_calls(self, number):592 def make_status_calls(self, env_name, juju_command, number):
473 """Return a list containing the given number of status calls."""593 """Return a list containing the given number of status calls."""
474 call = mock.call(594 call = mock.call(
475 self.juju_command, 'status', '-e', self.env_name,595 juju_command, 'status', '-e', env_name, '--format', 'yaml')
476 '--format', 'yaml')
477 return [call for _ in range(number)]596 return [call for _ in range(number)]
478597
479 def make_side_effects(self):598 def assert_status_retried(self, env_name, juju_command, side_effects):
480 """Return the minimum number of side effects for a successful call."""
481 return [
482 (0, '', ''), # Add a bootstrap call.
483 (0, self.make_status_output('started'), ''), # Add a status call.
484 ]
485
486 def assert_status_retried(self, side_effects):
487 """Ensure the "juju status" command is retried several times.599 """Ensure the "juju status" command is retried several times.
488600
489 Receive the list of side effects the mock status call will return.601 Receive the list of side effects the mock status call will return.
490 """602 """
491 with self.patch_multiple_calls(side_effects) as mock_call:603 count = len(side_effects)
492 app.bootstrap(self.env_name, self.juju_command)604 with self.patch_multiple_calls(side_effects) as mock_call:
493 mock_call.assert_has_calls([605 app.status(env_name, juju_command)
494 mock.call(self.juju_command, 'bootstrap', '-e', self.env_name),606 self.assertEqual(count, mock_call.call_count)
495 ] + self.make_status_calls(5))607 expected_calls = self.make_status_calls(env_name, juju_command, count)
496608 mock_call.assert_has_calls(expected_calls)
497 def test_success(self, mock_print):609
498 # The environment is successfully bootstrapped.610 def test_success(self):
499 with self.patch_multiple_calls(self.make_side_effects()) as mock_call:611 # The status command is correctly called and the bootstrap node series
500 already_bootstrapped, series = app.bootstrap(612 # returned.
501 self.env_name, self.juju_command)613 output = self.make_status_output('started')
502 self.assertFalse(already_bootstrapped)614 with self.patch_call(0, output=output) as mock_call:
503 self.assertEqual(series, 'hoary')615 bootstrap_node_series = app.status('ec2', '/usr/bin/juju')
504 mock_call.assert_has_calls([616 self.assertEqual('utopic', bootstrap_node_series)
505 mock.call(self.juju_command, 'bootstrap', '-e', self.env_name),617 self.assertEqual(1, mock_call.call_count)
506 ] + self.make_status_calls(1))618 expected_calls = self.make_status_calls('ec2', '/usr/bin/juju', 1)
507 mock_print.assert_called_once_with(self.status_message)619 mock_call.assert_has_calls(expected_calls)
508620
509 def test_success_debug(self, mock_print):621 def test_status_retry_error(self):
510 # The environment is successfully bootstrapped in debug mode.
511 with self.patch_multiple_calls(self.make_side_effects()) as mock_call:
512 already_bootstrapped, series = app.bootstrap(
513 self.env_name, self.juju_command, debug=True)
514 self.assertFalse(already_bootstrapped)
515 self.assertEqual(series, 'hoary')
516 mock_call.assert_has_calls([
517 mock.call(
518 self.juju_command, 'bootstrap', '-e', self.env_name,
519 '--debug'),
520 ] + self.make_status_calls(1))
521
522 def test_success_upload_tools(self, mock_print):
523 # The environment is bootstrapped with local tools.
524 with self.patch_multiple_calls(self.make_side_effects()) as mock_call:
525 already_bootstrapped, series = app.bootstrap(
526 self.env_name, self.juju_command, upload_tools=True)
527 self.assertFalse(already_bootstrapped)
528 mock_call.assert_has_calls([
529 mock.call(
530 self.juju_command, 'bootstrap', '-e', self.env_name,
531 '--upload-tools'),
532 ] + self.make_status_calls(1))
533
534 def test_success_upload_series(self, mock_print):
535 # The environment is bootstrapped with tools for specific series.
536 with self.patch_multiple_calls(self.make_side_effects()) as mock_call:
537 already_bootstrapped, series = app.bootstrap(
538 self.env_name, self.juju_command, upload_series='hoary')
539 self.assertFalse(already_bootstrapped)
540 mock_call.assert_has_calls([
541 mock.call(
542 self.juju_command, 'bootstrap', '-e', self.env_name,
543 '--upload-series', 'hoary'),
544 ] + self.make_status_calls(1))
545
546 def test_success_constraints(self, mock_print):
547 # The environment is bootstrapped with given constraints.
548 with self.patch_multiple_calls(self.make_side_effects()) as mock_call:
549 already_bootstrapped, series = app.bootstrap(
550 self.env_name, self.juju_command, constraints='mem=7G')
551 self.assertFalse(already_bootstrapped)
552 mock_call.assert_has_calls([
553 mock.call(
554 self.juju_command, 'bootstrap', '-e', self.env_name,
555 '--constraints', 'mem=7G'),
556 ] + self.make_status_calls(1))
557
558 def test_already_bootstrapped(self, mock_print):
559 # The function succeeds and returns True if the environment is already
560 # bootstrapped.
561 side_effects = [
562 (1, '', '***environment is already bootstrapped**'),
563 (0, self.make_status_output('started', 'precise'), ''),
564 ]
565 with self.patch_multiple_calls(side_effects) as mock_call:
566 already_bootstrapped, series = app.bootstrap(
567 self.env_name, self.juju_command)
568 self.assertTrue(already_bootstrapped)
569 self.assertEqual(series, 'precise')
570 mock_call.assert_has_calls([
571 mock.call(self.juju_command, 'bootstrap', '-e', self.env_name),
572 ] + self.make_status_calls(1))
573 existing_message = 'reusing the already bootstrapped {} environment'
574 mock_print.assert_has_calls([
575 mock.call(existing_message.format(self.env_name)),
576 mock.call(self.status_message),
577 ])
578
579 def test_bootstrap_failure(self, mock_print):
580 # A ProgramExit is raised if an error occurs while bootstrapping.
581 with self.patch_call(retcode=1, error='bad wolf') as mock_call:
582 with self.assert_program_exit('bad wolf'):
583 app.bootstrap(self.env_name, self.juju_command)
584 mock_call.assert_called_once_with(
585 self.juju_command, 'bootstrap', '-e', self.env_name),
586
587 def test_status_retry_error(self, mock_print):
588 # Before raising a ProgramExit, the functions tries to call622 # Before raising a ProgramExit, the functions tries to call
589 # "juju status" multiple times if it exits with an error.623 # "juju status" multiple times if it exits with an error.
590 side_effects = [624 side_effects = [
591 (0, '', ''), # Add the bootstrap call.
592 # Add four status calls with a non-zero exit code.625 # Add four status calls with a non-zero exit code.
593 (1, '', 'these'),626 (1, '', 'these'),
594 (2, '', 'are'),627 (2, '', 'are'),
@@ -597,29 +630,28 @@
597 # Add a final valid status call.630 # Add a final valid status call.
598 (0, self.make_status_output('started'), ''),631 (0, self.make_status_output('started'), ''),
599 ]632 ]
600 self.assert_status_retried(side_effects)633 self.assert_status_retried('local', 'juju', side_effects)
601634
602 def test_status_retry_invalid_output(self, mock_print):635 def test_status_retry_invalid_output(self):
603 # Before raising a ProgramExit, the functions tries to call636 # Before raising a ProgramExit, the functions tries to call
604 # "juju status" multiple times if its output is not well formed or if637 # "juju status" multiple times if its output is not well formed or if
605 # the agent is not started.638 # the agent is not started.
606 side_effects = [639 side_effects = [
607 (0, '', ''), # Add the bootstrap call.
608 (0, '', ''), # Add the first status call: no output.640 (0, '', ''), # Add the first status call: no output.
609 (0, ':', ''), # Add the second status call: not YAML.641 (0, ':', ''), # Add the second status call: not YAML.
610 (0, 'just-a-string', ''), # Add the third status call: bad YAML.642 (0, 'just-a-string', ''), # Add the third status call: bad YAML.
611 # Add the fourth status call: the agent is still pending.643 # Add two other status calls: the agent is still pending.
644 (0, self.make_status_output('pending'), ''),
612 (0, self.make_status_output('pending'), ''),645 (0, self.make_status_output('pending'), ''),
613 # Add a final valid status call.646 # Add a final valid status call.
614 (0, self.make_status_output('started'), ''),647 (0, self.make_status_output('started'), ''),
615 ]648 ]
616 self.assert_status_retried(side_effects)649 self.assert_status_retried('hp', '/usr/bin/juju', side_effects)
617650
618 def test_status_retry_both(self, mock_print):651 def test_status_retry_both(self):
619 # Before raising a ProgramExit, the functions tries to call652 # Before raising a ProgramExit, the functions tries to call
620 # "juju status" multiple times in any case.653 # "juju status" multiple times in any case.
621 side_effects = [654 side_effects = [
622 (0, '', ''), # Add the bootstrap call.
623 (1, '', 'error'), # Add the first status call: error.655 (1, '', 'error'), # Add the first status call: error.
624 (2, '', 'another error'), # Add the second status call: error.656 (2, '', 'another error'), # Add the second status call: error.
625 # Add the third status call: the agent is still pending.657 # Add the third status call: the agent is still pending.
@@ -628,28 +660,23 @@
628 # Add a final valid status call.660 # Add a final valid status call.
629 (0, self.make_status_output('started'), ''),661 (0, self.make_status_output('started'), ''),
630 ]662 ]
631 self.assert_status_retried(side_effects)663 self.assert_status_retried('local', '/usr/bin/juju', side_effects)
632664
633 def test_agent_error(self, mock_print):665 def test_agent_error(self):
634 # A ProgramExit is raised immediately if the Juju agent in the666 # A ProgramExit is raised immediately if the Juju agent in the
635 # bootstrap node is in an error state.667 # bootstrap node is in an error state.
636 status_output = self.make_status_output('error')668 output = self.make_status_output('error')
637 side_effects = [669 expected_error = 'state server failure:\n{}'.format(output)
638 (0, '', ''), # Add the bootstrap call.670 with self.patch_call(0, output=output) as mock_call:
639 (0, status_output, ''), # Add the status call: agent error.671 with self.assert_program_exit(expected_error):
640 ]672 app.status('ec2', '/usr/bin/juju')
641 expected = 'state server failure:\n{}'.format(status_output)673 self.assertEqual(1, mock_call.call_count)
642 with self.patch_multiple_calls(side_effects) as mock_call:674 expected_calls = self.make_status_calls('ec2', '/usr/bin/juju', 1)
643 with self.assert_program_exit(expected):675 mock_call.assert_has_calls(expected_calls)
644 app.bootstrap(self.env_name, self.juju_command)
645 mock_call.assert_has_calls([
646 mock.call(self.juju_command, 'bootstrap', '-e', self.env_name),
647 ] + self.make_status_calls(1))
648676
649 def test_status_failure(self, mock_print):677 def test_status_failure(self):
650 # A ProgramExit is raised if "juju status" keeps failing.678 # A ProgramExit is raised if "juju status" keeps failing.
651 call_side_effects = [679 call_side_effects = [
652 (0, '', ''), # Add the bootstrap call.
653 (1, 'output1', 'error1'), # Add the first status call: retried.680 (1, 'output1', 'error1'), # Add the first status call: retried.
654 (1, 'output2', 'error2'), # Add the second status call: error.681 (1, 'output2', 'error2'), # Add the second status call: error.
655 ]682 ]
@@ -660,17 +687,37 @@
660 1000, # Third call after the timeout expiration.687 1000, # Third call after the timeout expiration.
661 ]688 ]
662 mock_time = mock.Mock(side_effect=time_side_effects)689 mock_time = mock.Mock(side_effect=time_side_effects)
663 expected = 'the state server is not ready:\noutput2error2'690 expected_error = 'the state server is not ready:\noutput2error2'
664 with self.patch_multiple_calls(call_side_effects) as mock_call:691 with self.patch_multiple_calls(call_side_effects) as mock_call:
665 # Simulate the timeout expired: the first time call is used to692 # Simulate the timeout expired: the first time call is used to
666 # calculate the timeout, the second one for the first status check,693 # calculate the timeout, the second one for the first status check,
667 # the third for the second status check, the fourth should fail.694 # the third for the second status check, the fourth should fail.
668 with mock.patch('time.time', mock_time):695 with mock.patch('time.time', mock_time):
669 with self.assert_program_exit(expected):696 with self.assert_program_exit(expected_error):
670 app.bootstrap(self.env_name, self.juju_command)697 app.status('local', '/usr/bin/juju')
671 mock_call.assert_has_calls([698 self.assertEqual(2, mock_call.call_count)
672 mock.call(self.juju_command, 'bootstrap', '-e', self.env_name),699 expected_calls = self.make_status_calls('local', '/usr/bin/juju', 2)
673 ] + self.make_status_calls(2))700 mock_call.assert_has_calls(expected_calls)
701
702
703class TestGetEnvType(
704 helpers.JenvFileTestsMixin, ProgramExitTestsMixin, unittest.TestCase):
705
706 def test_success(self):
707 # The environment type is successfully retrieved.
708 with self.make_jenv('ec2', yaml.safe_dump(self.jenv_data)):
709 env_type = app.get_env_type('ec2')
710 self.assertEqual('ec2', env_type)
711
712 def test_error(self):
713 # A ProgramExit is raised if the environment type cannot be retrieved.
714 with self.make_jenv('aws', '') as path:
715 expected_error = (
716 'cannot retrieve environment type: invalid YAML '
717 'contents in {}: bootstrap-config key not found in the root '
718 'section'.format(path))
719 with self.assert_program_exit(expected_error):
720 app.get_env_type('aws')
674721
675722
676class TestGetAdminSecret(723class TestGetAdminSecret(
@@ -850,7 +897,8 @@
850 """Patch the get_charm_url helper function."""897 """Patch the get_charm_url helper function."""
851 mock_get_charm_url = mock.Mock(898 mock_get_charm_url = mock.Mock(
852 return_value=return_value, side_effect=side_effect)899 return_value=return_value, side_effect=side_effect)
853 return mock.patch('quickstart.utils.get_charm_url', mock_get_charm_url)900 return mock.patch(
901 'quickstart.netutils.get_charm_url', mock_get_charm_url)
854902
855 def test_environment_just_bootstrapped(self, mock_print):903 def test_environment_just_bootstrapped(self, mock_print):
856 # The function correctly retrieves the charm URL and machine, and904 # The function correctly retrieves the charm URL and machine, and
857905
=== modified file 'quickstart/tests/test_manage.py'
--- quickstart/tests/test_manage.py 2014-11-11 13:21:57 +0000
+++ quickstart/tests/test_manage.py 2014-11-12 16:41:04 +0000
@@ -38,7 +38,6 @@
38from quickstart.cli import views38from quickstart.cli import views
39from quickstart.models import envs39from quickstart.models import envs
40from quickstart.tests import helpers40from quickstart.tests import helpers
41from quickstart import app
4241
4342
44class TestDescriptionAction(unittest.TestCase):43class TestDescriptionAction(unittest.TestCase):
@@ -763,164 +762,177 @@
763 options.update(kwargs)762 options.update(kwargs)
764 return mock.Mock(**options)763 return mock.Mock(**options)
765764
766 def test_no_bundle(self, mock_app, mock_open):765 def configure_app(self, mock_app, **kwargs):
766 """Configure the given mock_app with the given kwargs.
767
768 Each key/value pair in kwargs represents a mock_app attribute and
769 the associated return value. Those values are used to override
770 defaults.
771
772 Return the mock Juju environment object.
773 """
774 env = mock.Mock()
775 defaults = {
776 # Dependencies are installed.
777 'ensure_dependencies': (1, 18, 0),
778 # Ensure the current Juju version is supported.
779 'check_juju_supported': None,
780 # Ensure the SSH keys are properly configured.
781 'ensure_ssh_keys': None,
782 # The environment is not already bootstrapped.
783 'check_bootstrapped': None,
784 # This is also confirmed by the bootstrap function.
785 'bootstrap': False,
786 # Status is then called, returning the bootstrap node series.
787 'status': 'trusty',
788 # The API URL must be retrieved (the environment was not ready).
789 'get_api_url': 'wss://1.2.3.4:17070',
790 # Retrieve the admin secret.
791 'get_admin_secret': 'Secret!',
792 # Connect to the Juju Environment API endpoint.
793 'connect': env,
794 # The environment is then checked.
795 'check_environment': (
796 'cs:trusty/juju-gui-42',
797 '0',
798 {'Name': 'juju-gui'},
799 {'Name': 'juju-gui/0'}
800 ),
801 # Deploy the Juju GUI charm.
802 'deploy_gui': 'juju-gui/0',
803 # Watch the deployment progress and return the unit address.
804 'watch': '1.2.3.4',
805 # Create the login token for the Juju GUI.
806 'create_auth_token': 'TOKEN',
807 }
808 defaults.update(kwargs)
809 for attr, return_value in defaults.items():
810 getattr(mock_app, attr).return_value = return_value
811 return env
812
813 def patch_get_juju_command(self):
814 """Patch the platform_support.get_juju_command function."""
815 path = 'quickstart.manage.platform_support.get_juju_command'
816 return mock.patch(path, return_value=(self.juju_command, False))
817
818 def test_run(self, mock_app, mock_open):
767 # The application runs correctly if no bundle is provided.819 # The application runs correctly if no bundle is provided.
768 mock_app.ensure_dependencies.return_value = (1, 18, 0)820 env = self.configure_app(mock_app)
769 # Make mock_app.bootstrap return the already_bootstrapped flag and the821 # Run the application.
770 # bootstrap node series.
771 mock_app.bootstrap.return_value = (True, 'trusty')
772 # Make mock_app.check_environment return the charm URL, the machine
773 # where to deploy the charm, the service and unit data.
774 service_data = {'Name': 'juju-gui'}
775 unit_data = {'Name': 'juju-gui/0'}
776 mock_app.check_environment.return_value = (
777 'cs:trusty/juju-gui-42', '0', service_data, unit_data)
778 # Make mock_app.get_admin_secret return the admin secret
779 mock_app.get_admin_secret.return_value = 'Secret!'
780 # Make mock_app.watch return the Juju GUI unit address.
781 mock_app.watch.return_value = '1.2.3.4'
782 # Make mock_app.create_auth_token return a fake authentication token.
783 mock_app.create_auth_token.return_value = 'AUTHTOKEN'
784 options = self.make_options()822 options = self.make_options()
785 with mock.patch('quickstart.manage.platform_support.get_juju_command',823 with self.patch_get_juju_command():
786 side_effect=[(self.juju_command, False)]):
787 manage.run(options)824 manage.run(options)
788 mock_app.ensure_dependencies.assert_called()825 # Ensure the functions have been used correctly.
789 mock_app.ensure_ssh_keys.assert_called()826 mock_app.ensure_dependencies.assert_called_once_with(
827 options.distro_only, options.platform, self.juju_command)
828 mock_app.check_juju_supported.assert_called_once_with((1, 18, 0))
829 mock_app.ensure_ssh_keys.assert_called_once_with()
830 mock_app.check_bootstrapped.assert_called_once_with(options.env_name)
790 mock_app.bootstrap.assert_called_once_with(831 mock_app.bootstrap.assert_called_once_with(
791 options.env_name, self.juju_command,832 options.env_name, self.juju_command,
792 debug=options.debug,833 debug=options.debug,
793 upload_tools=options.upload_tools,834 upload_tools=options.upload_tools,
794 upload_series=options.upload_series,835 upload_series=options.upload_series,
795 constraints=options.constraints)836 constraints=options.constraints)
837 mock_app.status.assert_called_once_with(
838 options.env_name, self.juju_command)
796 mock_app.get_api_url.assert_called_once_with(839 mock_app.get_api_url.assert_called_once_with(
797 options.env_name, self.juju_command)840 options.env_name, self.juju_command)
841 mock_app.get_admin_secret.assert_called_once_with(options.env_name)
798 mock_app.connect.assert_has_calls([842 mock_app.connect.assert_has_calls([
799 mock.call(mock_app.get_api_url(), 'Secret!'),843 mock.call('wss://1.2.3.4:17070', 'Secret!'),
800 mock.call().close(),844 mock.call().close(),
801 mock.call('wss://1.2.3.4:443/ws', 'Secret!'),845 mock.call('wss://1.2.3.4:443/ws', 'Secret!'),
802 mock.call().close(),846 mock.call().close(),
803 ])847 ])
804 mock_app.check_environment.assert_called_once_with(848 mock_app.check_environment.assert_called_once_with(
805 mock_app.connect(), settings.JUJU_GUI_SERVICE_NAME,849 env, settings.JUJU_GUI_SERVICE_NAME, options.charm_url,
806 options.charm_url, options.env_type, mock_app.bootstrap()[1],850 options.env_type, 'trusty', False)
807 mock_app.bootstrap()[0])
808 mock_app.deploy_gui.assert_called_once_with(851 mock_app.deploy_gui.assert_called_once_with(
809 mock_app.connect(), settings.JUJU_GUI_SERVICE_NAME,852 env, settings.JUJU_GUI_SERVICE_NAME, 'cs:trusty/juju-gui-42',
810 'cs:trusty/juju-gui-42', '0', service_data, unit_data)853 '0', {'Name': 'juju-gui'}, {'Name': 'juju-gui/0'})
811 mock_app.watch.assert_called_once_with(854 mock_app.watch.assert_called_once_with(env, 'juju-gui/0')
812 mock_app.connect(), mock_app.deploy_gui())855 mock_app.create_auth_token.assert_called_once_with(env)
813 mock_app.create_auth_token.assert_called_once_with(mock_app.connect())856 mock_open.assert_called_once_with('https://1.2.3.4/?authtoken=TOKEN')
814 mock_open.assert_called_once_with(857 # Ensure some of the app function have not been called.
815 'https://{}/?authtoken={}'.format(mock_app.watch(), 'AUTHTOKEN'))858 self.assertFalse(mock_app.get_env_type.called)
816 self.assertFalse(mock_app.deploy_bundle.called)859 self.assertFalse(mock_app.deploy_bundle.called)
817860
861 def test_already_bootstrapped(self, mock_app, mock_open):
862 # The application correctly reuses an already bootstrapped environment.
863 self.configure_app(mock_app, check_bootstrapped='wss://example.com')
864 # Run the application.
865 options = self.make_options()
866 with self.patch_get_juju_command():
867 manage.run(options)
868 # The environment type is retrieved from the jenv.
869 mock_app.get_env_type.assert_called_once_with(options.env_name)
870 # No reason to call bootstrap or get_api_url functions.
871 self.assertFalse(mock_app.bootstrap.called)
872 self.assertFalse(mock_app.get_api_url.called)
873
874 def test_already_bootstrapped_race(self, mock_app, mock_open):
875 # The application correctly reuses an already bootstrapped environment.
876 # In this case, the environment seems not bootstrapped at first, but
877 # it ended up being up and running later.
878 self.configure_app(mock_app, bootstrap=True)
879 # Run the application.
880 options = self.make_options()
881 with self.patch_get_juju_command():
882 manage.run(options)
883 # The bootstrap and get_api_url functions are still called, but this
884 # time also get_env_type is required.
885 # The environment type is retrieved from the jenv.
886 mock_app.bootstrap.assert_called_once_with(
887 options.env_name, self.juju_command,
888 debug=options.debug,
889 upload_tools=options.upload_tools,
890 upload_series=options.upload_series,
891 constraints=options.constraints)
892 mock_app.get_api_url.assert_called_once_with(
893 options.env_name, self.juju_command)
894 mock_app.get_env_type.assert_called_once_with(options.env_name)
895
818 def test_no_token(self, mock_app, mock_open):896 def test_no_token(self, mock_app, mock_open):
819 # The process continues even if the authentication token cannot be897 # The process continues even if the authentication token cannot be
820 # retrieved.898 # retrieved.
821 mock_app.create_auth_token.return_value = None899 env = self.configure_app(mock_app, create_auth_token=None)
822 # Make mock_app.bootstrap return the already_bootstrapped flag and the900 # Run the application.
823 # bootstrap node series.
824 mock_app.bootstrap.return_value = (True, 'precise')
825 # Make mock_app.check_environment return the charm URL, the machine
826 # where to deploy the charm, the service and unit data.
827 mock_app.check_environment.return_value = (
828 'cs:precise/juju-gui-42', '0', None, None)
829 options = self.make_options()901 options = self.make_options()
830 manage.run(options)902 with self.patch_get_juju_command():
831 mock_app.create_auth_token.assert_called_once_with(mock_app.connect())903 manage.run(options)
832 mock_open.assert_called_once_with(904 # Ensure the browser is still open without an auth token.
833 'https://{}'.format(mock_app.watch()))905 mock_app.create_auth_token.assert_called_once_with(env)
906 mock_open.assert_called_once_with('https://1.2.3.4')
834907
835 def test_bundle(self, mock_app, mock_open):908 def test_bundle(self, mock_app, mock_open):
836 # A bundle is correctly deployed by the application.909 # A bundle is correctly deployed by the application.
910 env = self.configure_app(mock_app, create_auth_token=None)
911 # Run the application.
837 options = self.make_options(912 options = self.make_options(
838 bundle='/my/bundle/file.yaml', bundle_yaml='mybundle: contents',913 bundle='/my/bundle/file.yaml', bundle_yaml='mybundle: contents',
839 bundle_name='mybundle', bundle_services=['service1', 'service2'])914 bundle_name='mybundle', bundle_services=['service1', 'service2'])
840 # Make mock_app.bootstrap return the already_bootstrapped flag and the915 with self.patch_get_juju_command():
841 # bootstrap node series.916 manage.run(options)
842 mock_app.bootstrap.return_value = (True, 'trusty')917 # Ensure the bundle is correctly deployed.
843 # Make mock_app.check_environment return the charm URL, the machine
844 # where to deploy the charm, the service and unit data.
845 mock_app.check_environment.return_value = (
846 'cs:trusty/juju-gui-42', '0', None, None)
847 # Make mock_app.watch return the Juju GUI unit address.
848 mock_app.watch.return_value = 'gui.example.com'
849 manage.run(options)
850 mock_app.deploy_bundle.assert_called_once_with(918 mock_app.deploy_bundle.assert_called_once_with(
851 mock_app.connect(), 'mybundle: contents', 'mybundle', None)919 env, 'mybundle: contents', 'mybundle', None)
852920
853 def test_local_provider_no_sudo(self, mock_app, mock_open):921 def test_local_provider(self, mock_app, mock_open):
854 # The application correctly handles working with local providers with922 # The application correctly handles working with local providers with
855 # new Juju versions not requiring "sudo" to bootstrap the environment.923 # new Juju versions not requiring "sudo" to bootstrap the environment.
856 # Sudo privileges are not required if the Juju version is >= 1.17.2.924 self.configure_app(mock_app, create_auth_token=None)
925 # Run the application.
857 options = self.make_options(env_type='local')926 options = self.make_options(env_type='local')
858 versions = [927 with self.patch_get_juju_command():
859 (1, 17, 2), (1, 17, 10), (1, 18, 0), (1, 18, 2), (2, 16, 1)]928 manage.run(options)
860 # Make mock_app.bootstrap return the already_bootstrapped flag and the
861 # bootstrap node series.
862 mock_app.bootstrap.return_value = (True, 'precise')
863 # Make mock_app.check_environment return the charm URL, the machine
864 # where to deploy the charm, the service and unit data.
865 mock_app.check_environment.return_value = (
866 'cs:precise/juju-gui-42', '0', None, None)
867 for version in versions:
868 mock_app.ensure_dependencies.return_value = version
869 with mock.patch(
870 'quickstart.manage.platform_support.get_juju_command',
871 side_effect=[(self.juju_command, False)]):
872 manage.run(options)
873 mock_app.bootstrap.assert_called_once_with(
874 options.env_name, self.juju_command,
875 debug=options.debug,
876 upload_tools=options.upload_tools,
877 upload_series=options.upload_series,
878 constraints=options.constraints)
879 mock_app.bootstrap.reset_mock()
880929
881 def test_no_browser(self, mock_app, mock_open):930 def test_no_browser(self, mock_app, mock_open):
882 # It is possible to avoid opening the GUI in the browser.931 # It is possible to avoid opening the GUI in the browser.
883 # Make mock_app.bootstrap return the already_bootstrapped flag and the932 self.configure_app(mock_app, create_auth_token=None)
884 # bootstrap node series.933 # Run the application.
885 mock_app.bootstrap.return_value = (True, 'trusty')
886 # Make mock_app.check_environment return the charm URL, the machine
887 # where to deploy the charm, the service and unit data.
888 mock_app.check_environment.return_value = (
889 'cs:trusty/juju-gui-42', '0', None, None)
890 options = self.make_options(open_browser=False)934 options = self.make_options(open_browser=False)
891 manage.run(options)935 with self.patch_get_juju_command():
936 manage.run(options)
937 # The browser is not opened.
892 self.assertFalse(mock_open.called)938 self.assertFalse(mock_open.called)
893
894 def test_no_admin_secret_found(self, mock_app, mock_open):
895 # If admin-secret cannot be found a ProgramExit is called.
896 mock_app.get_admin_secret.side_effect = app.ProgramExit('bad wolf')
897 # Make mock_app.bootstrap return the already_bootstrapped flag and the
898 # bootstrap node series.
899 mock_app.bootstrap.return_value = (True, 'precise')
900 # Make mock_app.check_environment return the charm URL, the machine
901 # where to deploy the charm, the service and unit data.
902 mock_app.check_environment.return_value = (
903 'cs:precise/juju-gui-42', '0', None, None)
904 options = self.make_options(
905 env_name='local',
906 env_file='environments.yaml')
907 with self.assertRaises(app.ProgramExit) as context:
908 manage.run(options)
909 self.assertEqual('bad wolf', context.exception.message)
910
911 def test_juju_environ_var_set(self, mock_app, mock_open):
912 mock_app.bootstrap.return_value = (True, 'precise')
913 mock_app.check_environment.return_value = (
914 'cs:precise/juju-gui-42', '0', None, None)
915 options = self.make_options(env_type='aws')
916 juju_command = 'some/devel/path/juju'
917 with mock.patch('os.environ', {'JUJU': juju_command}):
918 manage.run(options)
919 mock_app.bootstrap.assert_called_once_with(
920 options.env_name, juju_command,
921 debug=options.debug,
922 upload_tools=options.upload_tools,
923 upload_series=options.upload_series,
924 constraints=options.constraints)
925 mock_app.get_api_url.assert_called_once_with(
926 options.env_name, juju_command)
927939
=== added file 'quickstart/tests/test_netutils.py'
--- quickstart/tests/test_netutils.py 1970-01-01 00:00:00 +0000
+++ quickstart/tests/test_netutils.py 2014-11-12 16:41:04 +0000
@@ -0,0 +1,201 @@
1# This file is part of the Juju Quickstart Plugin, which lets users set up a
2# Juju environment in very few steps (https://launchpad.net/juju-quickstart).
3# Copyright (C) 2013-2014 Canonical Ltd.
4#
5# This program is free software: you can redistribute it and/or modify it under
6# the terms of the GNU Affero General Public License version 3, as published by
7# the Free Software Foundation.
8#
9# This program is distributed in the hope that it will be useful, but WITHOUT
10# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
11# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
12# Affero General Public License for more details.
13#
14# You should have received a copy of the GNU Affero General Public License
15# along with this program. If not, see <http://www.gnu.org/licenses/>.
16
17"""Tests for the Juju Quickstart network utility functions."""
18
19from __future__ import unicode_literals
20
21import httplib
22import json
23import socket
24import unittest
25import urllib2
26
27import mock
28
29from quickstart import netutils
30from quickstart.tests import helpers
31
32
33class TestCheckResolvable(unittest.TestCase):
34
35 def test_resolvable(self):
36 # None is returned if the hostname can be resolved.
37 expected_log = 'example.com resolved to 1.2.3.4'
38 with helpers.assert_logs([expected_log], level='debug'):
39 with mock.patch('socket.gethostbyname', return_value='1.2.3.4'):
40 error = netutils.check_resolvable('example.com')
41 self.assertIsNone(error)
42
43 def test_not_resolvable(self):
44 # An error message is returned if the hostname cannot be resolved.
45 exception = socket.gaierror('bad wolf')
46 with mock.patch('socket.gethostbyname', side_effect=exception):
47 error = netutils.check_resolvable('example.com')
48 self.assertEqual('bad wolf', error)
49
50
51class TestCheckListening(unittest.TestCase):
52
53 def test_listening(self):
54 # None is returned if the address can be connected to.
55 with helpers.patch_socket_create_connection() as mock_connection:
56 error = netutils.check_listening('1.2.3.4:17070')
57 self.assertIsNone(error)
58 mock_connection.assert_called_once_with(('1.2.3.4', 17070), 3)
59
60 def test_timeout(self):
61 # A customized timeout can be passed.
62 with helpers.patch_socket_create_connection() as mock_connection:
63 error = netutils.check_listening('1.2.3.4:17070', timeout=42)
64 self.assertIsNone(error)
65 mock_connection.assert_called_once_with(('1.2.3.4', 17070), 42)
66
67 def test_address_not_valid(self):
68 # An error message is returned if the address is not valid.
69 with helpers.patch_socket_create_connection() as mock_connection:
70 error = netutils.check_listening('1.2.3.4')
71 self.assertEqual(
72 'cannot connect to 1.2.3.4: need more than 1 value to unpack',
73 error)
74 self.assertFalse(mock_connection.called)
75
76 def test_port_not_valid(self):
77 # An error message is returned if the address port is not valid.
78 with helpers.patch_socket_create_connection() as mock_connection:
79 error = netutils.check_listening('1.2.3.4:bad-port')
80 self.assertEqual(
81 'cannot connect to 1.2.3.4:bad-port: '
82 "invalid literal for int() with base 10: 'bad-port'",
83 error)
84 self.assertFalse(mock_connection.called)
85
86 def test_not_listening(self):
87 # An error message is returned if the address is not reachable.
88 with helpers.patch_socket_create_connection('boo!') as mock_connection:
89 error = netutils.check_listening('1.2.3.4:17070')
90 self.assertEqual('cannot connect to 1.2.3.4:17070: boo!', error)
91 mock_connection.assert_called_once_with(('1.2.3.4', 17070), 3)
92
93 def test_closing(self):
94 # The socket connection is properly closed.
95 with helpers.patch_socket_create_connection() as mock_connection:
96 netutils.check_listening('1.2.3.4:17070')
97 mock_connection().close.assert_called_once_with()
98
99 def test_error_closing(self):
100 # Errors closing the socket connection are ignored.
101 with helpers.patch_socket_create_connection() as mock_connection:
102 mock_connection().close.side_effect = socket.error('bad wolf')
103 netutils.check_listening('1.2.3.4:17070')
104
105
106class TestGetCharmUrl(helpers.UrlReadTestsMixin, unittest.TestCase):
107
108 def test_charm_url(self):
109 # The Juju GUI charm URL is correctly returned.
110 contents = json.dumps({'charm': {'url': 'cs:trusty/juju-gui-42'}})
111 with self.patch_urlread(contents=contents) as mock_urlread:
112 charm_url = netutils.get_charm_url('trusty')
113 self.assertEqual('cs:trusty/juju-gui-42', charm_url)
114 mock_urlread.assert_called_once_with(
115 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui')
116
117 def test_io_error(self):
118 # IOErrors are properly propagated.
119 with self.patch_urlread(error=True) as mock_urlread:
120 with self.assertRaises(IOError) as context_manager:
121 netutils.get_charm_url('precise')
122 mock_urlread.assert_called_once_with(
123 'http://manage.jujucharms.com/api/3/charm/precise/juju-gui')
124 self.assertEqual('bad wolf', bytes(context_manager.exception))
125
126 def test_value_error(self):
127 # A ValueError is raised if the API response is not valid.
128 contents = json.dumps({'charm': {}})
129 with self.patch_urlread(contents=contents) as mock_urlread:
130 with self.assertRaises(ValueError) as context_manager:
131 netutils.get_charm_url('trusty')
132 mock_urlread.assert_called_once_with(
133 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui')
134 self.assertEqual(
135 'unable to find the charm URL', bytes(context_manager.exception))
136
137
138class TestUrlread(unittest.TestCase):
139
140 def patch_urlopen(self, contents=None, error=None, content_type=None):
141 """Patch the urllib2.urlopen function.
142
143 If contents is not None, the read() method of the returned mock object
144 returns the given contents.
145 If content_type is provided, the response includes the content type.
146 If an error is provided, the call raises the error.
147 """
148 mock_urlopen = mock.MagicMock()
149 if contents is not None:
150 mock_urlopen().read.return_value = contents
151 if content_type is not None:
152 mock_urlopen().headers = {'content-type': content_type}
153 if error is not None:
154 mock_urlopen.side_effect = error
155 mock_urlopen.reset_mock()
156 return mock.patch('urllib2.urlopen', mock_urlopen)
157
158 def test_contents(self):
159 # The URL contents are correctly returned.
160 with self.patch_urlopen(contents=b'URL contents') as mock_urlopen:
161 contents = netutils.urlread('http://example.com/path/')
162 self.assertEqual('URL contents', contents)
163 self.assertIsInstance(contents, unicode)
164 mock_urlopen.assert_called_once_with('http://example.com/path/')
165
166 def test_content_type(self):
167 # The URL contents are decoded using the site charset.
168 patch_urlopen = self.patch_urlopen(
169 contents=b'URL contents: \xf8', # This is not a UTF-8 byte string.
170 content_type='text/html; charset=ISO-8859-1')
171 with patch_urlopen as mock_urlopen:
172 contents = netutils.urlread('http://example.com/path/')
173 self.assertEqual('URL contents: \xf8', contents)
174 self.assertIsInstance(contents, unicode)
175 mock_urlopen.assert_called_once_with('http://example.com/path/')
176
177 def test_no_content_type(self):
178 # The URL contents are decoded with UTF-8 by default.
179 patch_urlopen = self.patch_urlopen(
180 contents=b'URL contents: \xf8', # This is not a UTF-8 byte string.
181 content_type='text/html')
182 with patch_urlopen as mock_urlopen:
183 contents = netutils.urlread('http://example.com/path/')
184 self.assertEqual('URL contents: ', contents)
185 self.assertIsInstance(contents, unicode)
186 mock_urlopen.assert_called_once_with('http://example.com/path/')
187
188 def test_errors(self):
189 # An IOError is raised if an error occurs connecting to the API.
190 errors = {
191 'httplib HTTPException': httplib.HTTPException,
192 'socket error': socket.error,
193 'urllib2 URLError': urllib2.URLError,
194 }
195 for message, exception_class in errors.items():
196 exception = exception_class(message)
197 with self.patch_urlopen(error=exception) as mock_urlopen:
198 with self.assertRaises(IOError) as context_manager:
199 netutils.urlread('http://example.com/path/')
200 mock_urlopen.assert_called_once_with('http://example.com/path/')
201 self.assertEqual(message, bytes(context_manager.exception))
0202
=== modified file 'quickstart/tests/test_utils.py'
--- quickstart/tests/test_utils.py 2014-11-10 09:08:44 +0000
+++ quickstart/tests/test_utils.py 2014-11-12 16:41:04 +0000
@@ -19,14 +19,11 @@
19from __future__ import unicode_literals19from __future__ import unicode_literals
2020
21import datetime21import datetime
22import httplib
23import json22import json
24import os23import os
25import shutil24import shutil
26import socket
27import tempfile25import tempfile
28import unittest26import unittest
29import urllib2
3027
31import mock28import mock
32import yaml29import yaml
@@ -156,24 +153,6 @@
156 utils.call('echo', 'we are the borg!')153 utils.call('echo', 'we are the borg!')
157154
158155
159class TestCheckResolvable(unittest.TestCase):
160
161 def test_resolvable(self):
162 # None is returned if the hostname can be resolved.
163 expected_log = 'example.com resolved to 1.2.3.4'
164 with helpers.assert_logs([expected_log], level='debug'):
165 with mock.patch('socket.gethostbyname', return_value='1.2.3.4'):
166 error = utils.check_resolvable('example.com')
167 self.assertIsNone(error)
168
169 def test_not_resolvable(self):
170 # An error message is returned if the hostname cannot be resolved.
171 exception = socket.gaierror('bad wolf')
172 with mock.patch('socket.gethostbyname', side_effect=exception):
173 error = utils.check_resolvable('example.com')
174 self.assertEqual('bad wolf', error)
175
176
177@mock.patch('__builtin__.print', mock.Mock())156@mock.patch('__builtin__.print', mock.Mock())
178class TestParseGuiCharmUrl(unittest.TestCase):157class TestParseGuiCharmUrl(unittest.TestCase):
179158
@@ -323,38 +302,6 @@
323 utils.convert_bundle_url(url)302 utils.convert_bundle_url(url)
324303
325304
326class TestGetCharmUrl(helpers.UrlReadTestsMixin, unittest.TestCase):
327
328 def test_charm_url(self):
329 # The Juju GUI charm URL is correctly returned.
330 contents = json.dumps({'charm': {'url': 'cs:trusty/juju-gui-42'}})
331 with self.patch_urlread(contents=contents) as mock_urlread:
332 charm_url = utils.get_charm_url('trusty')
333 self.assertEqual('cs:trusty/juju-gui-42', charm_url)
334 mock_urlread.assert_called_once_with(
335 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui')
336
337 def test_io_error(self):
338 # IOErrors are properly propagated.
339 with self.patch_urlread(error=True) as mock_urlread:
340 with self.assertRaises(IOError) as context_manager:
341 utils.get_charm_url('precise')
342 mock_urlread.assert_called_once_with(
343 'http://manage.jujucharms.com/api/3/charm/precise/juju-gui')
344 self.assertEqual('bad wolf', bytes(context_manager.exception))
345
346 def test_value_error(self):
347 # A ValueError is raised if the API response is not valid.
348 contents = json.dumps({'charm': {}})
349 with self.patch_urlread(contents=contents) as mock_urlread:
350 with self.assertRaises(ValueError) as context_manager:
351 utils.get_charm_url('trusty')
352 mock_urlread.assert_called_once_with(
353 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui')
354 self.assertEqual(
355 'unable to find the charm URL', bytes(context_manager.exception))
356
357
358class TestGetQuickstartBanner(unittest.TestCase):305class TestGetQuickstartBanner(unittest.TestCase):
359306
360 def patch_datetime(self):307 def patch_datetime(self):
@@ -704,72 +651,6 @@
704 self.assertEqual(list.append.__doc__, self.func.__doc__)651 self.assertEqual(list.append.__doc__, self.func.__doc__)
705652
706653
707class TestUrlread(unittest.TestCase):
708
709 def patch_urlopen(self, contents=None, error=None, content_type=None):
710 """Patch the urllib2.urlopen function.
711
712 If contents is not None, the read() method of the returned mock object
713 returns the given contents.
714 If content_type is provided, the response includes the content type.
715 If an error is provided, the call raises the error.
716 """
717 mock_urlopen = mock.MagicMock()
718 if contents is not None:
719 mock_urlopen().read.return_value = contents
720 if content_type is not None:
721 mock_urlopen().headers = {'content-type': content_type}
722 if error is not None:
723 mock_urlopen.side_effect = error
724 mock_urlopen.reset_mock()
725 return mock.patch('urllib2.urlopen', mock_urlopen)
726
727 def test_contents(self):
728 # The URL contents are correctly returned.
729 with self.patch_urlopen(contents=b'URL contents') as mock_urlopen:
730 contents = utils.urlread('http://example.com/path/')
731 self.assertEqual('URL contents', contents)
732 self.assertIsInstance(contents, unicode)
733 mock_urlopen.assert_called_once_with('http://example.com/path/')
734
735 def test_content_type(self):
736 # The URL contents are decoded using the site charset.
737 patch_urlopen = self.patch_urlopen(
738 contents=b'URL contents: \xf8', # This is not a UTF-8 byte string.
739 content_type='text/html; charset=ISO-8859-1')
740 with patch_urlopen as mock_urlopen:
741 contents = utils.urlread('http://example.com/path/')
742 self.assertEqual('URL contents: \xf8', contents)
743 self.assertIsInstance(contents, unicode)
744 mock_urlopen.assert_called_once_with('http://example.com/path/')
745
746 def test_no_content_type(self):
747 # The URL contents are decoded with UTF-8 by default.
748 patch_urlopen = self.patch_urlopen(
749 contents=b'URL contents: \xf8', # This is not a UTF-8 byte string.
750 content_type='text/html')
751 with patch_urlopen as mock_urlopen:
752 contents = utils.urlread('http://example.com/path/')
753 self.assertEqual('URL contents: ', contents)
754 self.assertIsInstance(contents, unicode)
755 mock_urlopen.assert_called_once_with('http://example.com/path/')
756
757 def test_errors(self):
758 # An IOError is raised if an error occurs connecting to the API.
759 errors = {
760 'httplib HTTPException': httplib.HTTPException,
761 'socket error': socket.error,
762 'urllib2 URLError': urllib2.URLError,
763 }
764 for message, exception_class in errors.items():
765 exception = exception_class(message)
766 with self.patch_urlopen(error=exception) as mock_urlopen:
767 with self.assertRaises(IOError) as context_manager:
768 utils.urlread('http://example.com/path/')
769 mock_urlopen.assert_called_once_with('http://example.com/path/')
770 self.assertEqual(message, bytes(context_manager.exception))
771
772
773class TestGetJujuVersion(654class TestGetJujuVersion(
774 helpers.CallTestsMixin, helpers.ValueErrorTestsMixin,655 helpers.CallTestsMixin, helpers.ValueErrorTestsMixin,
775 unittest.TestCase):656 unittest.TestCase):
776657
=== modified file 'quickstart/utils.py'
--- quickstart/utils.py 2014-11-10 09:08:44 +0000
+++ quickstart/utils.py 2014-11-12 16:41:04 +0000
@@ -25,15 +25,11 @@
25import datetime25import datetime
26import errno26import errno
27import functools27import functools
28import httplib
29import json
30import logging28import logging
31import os29import os
32import pipes30import pipes
33import re31import re
34import socket
35import subprocess32import subprocess
36import urllib2
3733
38import quickstart34import quickstart
39from quickstart import (35from quickstart import (
@@ -105,20 +101,6 @@
105 return retcode, output.decode('utf-8'), error.decode('utf-8')101 return retcode, output.decode('utf-8'), error.decode('utf-8')
106102
107103
108def check_resolvable(hostname):
109 """Check that the hostname can be resolved to a numeric IP address.
110
111 Return an error message if the address cannot be resolved.
112 """
113 try:
114 address = socket.gethostbyname(hostname)
115 except socket.error as err:
116 return bytes(err).decode('utf-8')
117 logging.debug('{} resolved to {}'.format(
118 hostname, address.decode('utf-8')))
119 return None
120
121
122def parse_gui_charm_url(charm_url):104def parse_gui_charm_url(charm_url):
123 """Parse the given charm URL.105 """Parse the given charm URL.
124106
@@ -164,21 +146,6 @@
164 bundle_id)146 bundle_id)
165147
166148
167def get_charm_url(series):
168 """Return the charm URL of the latest Juju GUI charm revision.
169
170 Raise an IOError if any problems occur connecting to the API endpoint.
171 Raise a ValueError if the API returns invalid data.
172 """
173 url = settings.CHARMWORLD_API.format(
174 series=series, charm=settings.JUJU_GUI_CHARM_NAME)
175 charm_info = json.loads(urlread(url))
176 charm_url = charm_info.get('charm', {}).get('url')
177 if charm_url is None:
178 raise ValueError(b'unable to find the charm URL')
179 return charm_url
180
181
182def get_quickstart_banner():149def get_quickstart_banner():
183 """Return a quickstart banner suitable for being included in files.150 """Return a quickstart banner suitable for being included in files.
184151
@@ -390,24 +357,3 @@
390 return function(*args, **kwargs)357 return function(*args, **kwargs)
391 decorated.called = False358 decorated.called = False
392 return decorated359 return decorated
393
394
395def urlread(url):
396 """Open the given URL and return the page contents.
397
398 Raise an IOError if any problems occur.
399 """
400 try:
401 response = urllib2.urlopen(url)
402 except urllib2.URLError as err:
403 raise IOError(err.reason)
404 except (httplib.HTTPException, socket.error, urllib2.HTTPError) as err:
405 raise IOError(bytes(err))
406 contents = response.read()
407 content_type = response.headers['content-type']
408 charset = 'utf-8'
409 if 'charset=' in content_type:
410 sent_charset = content_type.split('charset=')[-1].strip()
411 if sent_charset:
412 charset = sent_charset
413 return contents.decode(charset, 'ignore')
414360
=== modified file 'quickstart/watchers.py'
--- quickstart/watchers.py 2014-11-10 10:28:19 +0000
+++ quickstart/watchers.py 2014-11-12 16:41:04 +0000
@@ -23,7 +23,7 @@
2323
24import logging24import logging
2525
26from quickstart import utils26from quickstart import netutils
2727
2828
29IPV4_ADDRESS = 'ipv4'29IPV4_ADDRESS = 'ipv4'
@@ -150,7 +150,7 @@
150 if not address:150 if not address:
151 addresses = data.get('Addresses', [])151 addresses = data.get('Addresses', [])
152 public_address = retrieve_public_adddress(152 public_address = retrieve_public_adddress(
153 addresses, utils.check_resolvable)153 addresses, netutils.check_resolvable)
154 if public_address is not None:154 if public_address is not None:
155 address = public_address155 address = public_address
156 print('unit placed on {}'.format(address))156 print('unit placed on {}'.format(address))

Subscribers

People subscribed via source and target branches