Merge lp:~frankban/juju-quickstart/new-bootstrap-strategy into lp:juju-quickstart

Proposed by Francesco Banconi
Status: Merged
Merged at revision: 105
Proposed branch: lp:~frankban/juju-quickstart/new-bootstrap-strategy
Merge into: lp:juju-quickstart
Diff against target: 1663 lines (+770/-511)
11 files modified
HACKING.rst (+2/-1)
quickstart/app.py (+80/-44)
quickstart/manage.py (+38/-17)
quickstart/netutils.py (+99/-0)
quickstart/tests/helpers.py (+19/-5)
quickstart/tests/test_app.py (+195/-147)
quickstart/tests/test_manage.py (+134/-122)
quickstart/tests/test_netutils.py (+201/-0)
quickstart/tests/test_utils.py (+0/-119)
quickstart/utils.py (+0/-54)
quickstart/watchers.py (+2/-2)
To merge this branch: bzr merge lp:~frankban/juju-quickstart/new-bootstrap-strategy
Reviewer Review Type Date Requested Status
Juju GUI Hackers Pending
Review via email: mp+241553@code.launchpad.net

Description of the change

New bootstrap strategy.

This is a massive branch: my apologies.
But:
    - a new module has been created (netutils), so there
      are license headers and moreover some existing
      functions (with their tests) have been just moved
      from utils.py and must not be re-reviewed.
      The only new function there is check_listening.
    - most of the code are tests: we reached 700 unit
      tests yay!

This branch changes the way quickstart is run on an
existing environment: instead of always trying to
bootstrap, it looks for the jenv file for the current
environment, and, if present, it retrieves the API URL
from there. As a consequence, quickstart uses Juju
in a less expensive way, and it's also faster when
invoked on a bootstrapped environment.

Split the app.bootstrap function to two new
functions: app.bootstrap and app.status. The intent
is to make them more reusable and more easy to test.

Fix a subtle bug never reported and not easy to hit:
the environment type of an existing environment is
now retrieved from the jenv, rather than relying on
what's stored in the environments.yaml file.

Also reorganized the tests for manage.run: over
time they ended up failing to achieve their goal
of describing how the application is run, and
become not really easy to update and change.
Now this situation should be improved.

The little change to the HACKING file is to make
the rst to render correctly on sublime text.

Tests: `make check`.

QA: run quickstart as usual on local and ec2.
Run quickstart again on an already bootstrapped
environment (local and ec2). You should no longer
see the "bootstrapping environment" message.
Instead, a more correct "reusing the already
bootstrapped..." message is displayed.
Also this should feel quicker, especially on ec2.

Thank you!

https://codereview.appspot.com/172380043/

To post a comment you must log in.
Revision history for this message
Francesco Banconi (frankban) wrote :

Reviewers: mp+241553_code.launchpad.net,

Message:
Please take a look.

Description:
New bootstrap strategy.

This is a massive branch: my apologies.
But:
     - a new module has been created (netutils), so there
       are license headers and moreover some existing
       functions (with their tests) have been just moved
       from utils.py and must not be re-reviewed.
       The only new function there is check_listening.
     - most of the code are tests: we reached 700 unit
       tests yay!

This branch changes the way quickstart is run on an
existing environment: instead of always trying to
bootstrap, it looks for the jenv file for the current
environment, and, if present, it retrieves the API URL
from there. As a consequence, quickstart uses Juju
in a less expensive way, and it's also faster when
invoked on a bootstrapped environment.

Split the app.bootstrap function to two new
functions: app.bootstrap and app.status. The intent
is to make them more reusable and more easy to test.

Fix a subtle bug never reported and not easy to hit:
the environment type of an existing environment is
now retrieved from the jenv, rather than relying on
what's stored in the environments.yaml file.

Also reorganized the tests for manage.run: over
time they ended up failing to achieve their goal
of describing how the application is run, and
become not really easy to update and change.
Now this situation should be improved.

The little change to the HACKING file is to make
the rst to render correctly on sublime text.

Tests: `make check`.

QA: run quickstart as usual on local and ec2.
Run quickstart again on an already bootstrapped
environment (local and ec2). You should no longer
see the "bootstrapping environment" message.
Instead, a more correct "reusing the already
bootstrapped..." message is displayed.
Also this should feel quicker, especially on ec2.

Thank you!

https://code.launchpad.net/~frankban/juju-quickstart/new-bootstrap-strategy/+merge/241553

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/172380043/

Affected files (+761, -511 lines):
   M HACKING.rst
   A [revision details]
   M quickstart/app.py
   M quickstart/manage.py
   A quickstart/netutils.py
   M quickstart/tests/helpers.py
   M quickstart/tests/test_app.py
   M quickstart/tests/test_manage.py
   A quickstart/tests/test_netutils.py
   M quickstart/tests/test_utils.py
   M quickstart/utils.py
   M quickstart/watchers.py

Revision history for this message
Brad Crittenden (bac) wrote :

Code LGTM with minor comments. Will QA now.

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py
File quickstart/app.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py#newcode245
quickstart/app.py:245: # For this reason, a call to status() is usually
required at this point.
This is no different from the calling perspective than the 'not already
bootstrapped' situation, right?

If so, perhaps that should be noted in the docstring for this function.

https://codereview.appspot.com/172380043/diff/1/quickstart/manage.py
File quickstart/manage.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/manage.py#newcode520
quickstart/manage.py:520: already_bootstrapped = app.bootstrap(
What is the scenario for api_url being None but it is already
bootstrapped? Someone deleted the jenv file out from under juju?

https://codereview.appspot.com/172380043/diff/1/quickstart/netutils.py
File quickstart/netutils.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/netutils.py#newcode50
quickstart/netutils.py:50: timeout = 3
Make timeout an arg with default = 3?

https://codereview.appspot.com/172380043/

Revision history for this message
Richard Harding (rharding) wrote :

LGTM thank you for the updates.

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py
File quickstart/app.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py#newcode190
quickstart/app.py:190: candidates = jenv.get_value(env_name,
'state-servers')
do you know how this list gets updated from Juju? In an HA situation
does juju rewrite the jenv file?

https://codereview.appspot.com/172380043/diff/1/quickstart/manage.py
File quickstart/manage.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/manage.py#newcode520
quickstart/manage.py:520: already_bootstrapped = app.bootstrap(
On 2014/11/12 14:48:25, bac wrote:
> What is the scenario for api_url being None but it is already
bootstrapped?
> Someone deleted the jenv file out from under juju?

That has happened and there's been bugs/work around the scenario. It's a
good one to watch for.

https://codereview.appspot.com/172380043/

Revision history for this message
Brad Crittenden (bac) wrote :
Revision history for this message
Francesco Banconi (frankban) wrote :

Thanks for the reviews, really good stuff!

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py
File quickstart/app.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py#newcode190
quickstart/app.py:190: candidates = jenv.get_value(env_name,
'state-servers')
On 2014/11/12 15:30:44, rharding wrote:
> do you know how this list gets updated from Juju? In an HA situation
does juju
> rewrite the jenv file?

I don't know that for sure: it could be the case when you set HA from
the CLI. I would be surprised if the jenv is updated when you will set
HA using the API. In case of stale addresses, we just fall back to the
old bootstrap strategy.

https://codereview.appspot.com/172380043/diff/1/quickstart/app.py#newcode245
quickstart/app.py:245: # For this reason, a call to status() is usually
required at this point.
On 2014/11/12 14:48:25, bac wrote:
> This is no different from the calling perspective than the 'not
already
> bootstrapped' situation, right?

> If so, perhaps that should be noted in the docstring for this
function.

Done.

https://codereview.appspot.com/172380043/diff/1/quickstart/netutils.py
File quickstart/netutils.py (right):

https://codereview.appspot.com/172380043/diff/1/quickstart/netutils.py#newcode50
quickstart/netutils.py:50: timeout = 3
On 2014/11/12 14:48:25, bac wrote:
> Make timeout an arg with default = 3?

Done.

https://codereview.appspot.com/172380043/

111. By Francesco Banconi

Changes as per review.

Revision history for this message
Francesco Banconi (frankban) wrote :

*** Submitted:

New bootstrap strategy.

This is a massive branch: my apologies.
But:
     - a new module has been created (netutils), so there
       are license headers and moreover some existing
       functions (with their tests) have been just moved
       from utils.py and must not be re-reviewed.
       The only new function there is check_listening.
     - most of the code are tests: we reached 700 unit
       tests yay!

This branch changes the way quickstart is run on an
existing environment: instead of always trying to
bootstrap, it looks for the jenv file for the current
environment, and, if present, it retrieves the API URL
from there. As a consequence, quickstart uses Juju
in a less expensive way, and it's also faster when
invoked on a bootstrapped environment.

Split the app.bootstrap function to two new
functions: app.bootstrap and app.status. The intent
is to make them more reusable and more easy to test.

Fix a subtle bug never reported and not easy to hit:
the environment type of an existing environment is
now retrieved from the jenv, rather than relying on
what's stored in the environments.yaml file.

Also reorganized the tests for manage.run: over
time they ended up failing to achieve their goal
of describing how the application is run, and
become not really easy to update and change.
Now this situation should be improved.

The little change to the HACKING file is to make
the rst to render correctly on sublime text.

Tests: `make check`.

QA: run quickstart as usual on local and ec2.
Run quickstart again on an already bootstrapped
environment (local and ec2). You should no longer
see the "bootstrapping environment" message.
Instead, a more correct "reusing the already
bootstrapped..." message is displayed.
Also this should feel quicker, especially on ec2.

Thank you!

R=bac, rharding
CC=
https://codereview.appspot.com/172380043

https://codereview.appspot.com/172380043/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'HACKING.rst'
2--- HACKING.rst 2014-11-11 14:15:47 +0000
3+++ HACKING.rst 2014-11-12 16:41:04 +0000
4@@ -152,7 +152,8 @@
5
6 * Verify an environment that has already been bootstrapped is recogized and
7 the GUI is deployed. This test also shows that a remote bundle is properly
8- deployed::
9+ deployed
10+::
11
12 juju bootstrap -e local
13 juju quickstart -e local bundle:mediawiki/single
14
15=== modified file 'quickstart/app.py'
16--- quickstart/app.py 2014-11-11 13:21:57 +0000
17+++ quickstart/app.py 2014-11-12 16:41:04 +0000
18@@ -30,6 +30,7 @@
19
20 from quickstart import (
21 juju,
22+ netutils,
23 platform_support,
24 settings,
25 ssh,
26@@ -175,28 +176,55 @@
27 raise ProgramExit(bytes(err))
28
29
30+def check_bootstrapped(env_name):
31+ """Check if the environment named env_name is already bootstrapped.
32+
33+ If so, return the environment API URL to be used to connect to the Juju API
34+ server. If not already bootstrapped, or if the API URL cannot be retrieved,
35+ return None.
36+ """
37+ if not jenv.exists(env_name):
38+ return None
39+ # Retrieve the Juju API addresses from the jenv file.
40+ try:
41+ candidates = jenv.get_value(env_name, 'state-servers')
42+ except ValueError as err:
43+ logging.warn(b'cannot retrieve the Juju API URL: {}'.format(err))
44+ return None
45+ # Look for a reachable API URL.
46+ if not candidates:
47+ logging.warn('cannot retrieve the Juju API URL: no addresses found')
48+ return None
49+ for candidate in candidates:
50+ error = netutils.check_listening(candidate)
51+ if error is None:
52+ # Juju API URL found.
53+ return 'wss://{}'.format(candidate)
54+ logging.debug(error)
55+ logging.warn(
56+ 'cannot retrieve the Juju API URL: cannot connect to any of the '
57+ 'following addresses: {}'.format(', '.join(candidates)))
58+ return None
59+
60+
61 def bootstrap(
62 env_name, juju_command, debug=False, upload_tools=False,
63 upload_series=None, constraints=None):
64 """Bootstrap the Juju environment with the given name.
65
66+ Return a flag indicating whether the environment was already bootstrapped.
67+
68 Do not bootstrap the environment if already bootstrapped.
69-
70- Return a tuple (already_bootstrapped, series) in which:
71- - already_bootstrapped indicates whether the environment was already
72- bootstrapped;
73- - series is the bootstrap node Ubuntu series.
74-
75- The is_local argument indicates whether the environment is configured to
76- use the local provider. If so, sudo privileges are requested in order to
77- bootstrap the environment.
78-
79- If debug is True and the environment not bootstrapped, execute the
80- bootstrap command passing the --debug flag.
81+ If the environment is not bootstrapped, execute the bootstrap command with
82+ the given juju_command, debug, upload_tools, upload_series and constraints
83+ arguments.
84+
85+ When the function exists the Juju environment is bootstrapped, but we don't
86+ know if it is ready yet. For this reason, a call to status() is usually
87+ required at that point.
88
89 Raise a ProgramExit if any error occurs in the bootstrap process.
90 """
91- already_bootstrapped = False
92 cmd = [juju_command, 'bootstrap', '-e', env_name]
93 if debug:
94 cmd.append('--debug')
95@@ -207,34 +235,28 @@
96 if constraints is not None:
97 cmd.extend(['--constraints', constraints])
98 retcode, _, error = utils.call(*cmd)
99- if retcode:
100- # XXX frankban 2013-11-13 bug 1252322: the check below is weak. We are
101- # relying on an error message in order to decide if the environment is
102- # already bootstrapped. Other possibilities include checking if the
103- # jenv file is present (in ~/.juju/environments/) and, if so, check the
104- # juju status. Unfortunately this is also prone to errors, because a
105- # jenv file can be there but the environment not really bootstrapped or
106- # functional (e.g. sync-tools was used, or a previous bootstrap failed,
107- # or the user terminated machines from the ec2 panel, etc.). Moreover
108- # jenv files seems to be an internal juju-core detail. Definitely we
109- # need to find a better way, but for now the "asking forgiveness"
110- # approach feels like the best compromise we have. Also note that,
111- # rather than comparing the expected error with the obtained one, we
112- # search in the error in order to support bootstrap --debug.
113- if 'environment is already bootstrapped' not in error:
114- # We exit if the error is not "already bootstrapped".
115- raise ProgramExit(error)
116- # Juju is bootstrapped, but we don't know if it is ready yet. Fall
117- # through to the next block for that check.
118- already_bootstrapped = True
119- print('reusing the already bootstrapped {} environment'.format(
120- env_name))
121- # Call "juju status" multiple times until the bootstrap node is ready.
122- # Exit with an error if the agent is not ready after ten minutes.
123- # Note: when using the local provider, calling "juju status" is very fast,
124- # but e.g. on ec2 the first call (right after "bootstrap") can take
125- # several minutes, and subsequent calls are relatively fast (seconds).
126- print('retrieving the environment status')
127+ if not retcode:
128+ return False
129+ # XXX frankban 2013-11-13 bug 1252322: the check below is weak. We are
130+ # relying on an error message in order to decide if the environment is
131+ # already bootstrapped. Also note that, rather than comparing the expected
132+ # error with the obtained one, we search in the error in order to support
133+ # bootstrap --debug.
134+ if 'environment is already bootstrapped' not in error:
135+ # We exit if the error is not "already bootstrapped".
136+ raise ProgramExit(error)
137+ # Juju is already bootstrapped.
138+ return True
139+
140+
141+def status(env_name, juju_command):
142+ """Call "juju status" multiple times until the bootstrap node is ready.
143+
144+ Return the bootstrap node series of the Juju environment.
145+
146+ Raise a ProgramExit if the agent is not ready after ten minutes or if the
147+ agent is in an error state.
148+ """
149 timeout = time.time() + (60*10)
150 while time.time() < timeout:
151 retcode, output, error = utils.call(
152@@ -247,8 +269,7 @@
153 except ValueError:
154 continue
155 if agent_state == 'started':
156- series = utils.get_bootstrap_node_series(output)
157- return already_bootstrapped, series
158+ return utils.get_bootstrap_node_series(output)
159 # If the agent is in an error state, there is nothing we can do, and
160 # it's not useful to keep trying.
161 if agent_state == 'error':
162@@ -257,6 +278,21 @@
163 raise ProgramExit('the state server is not ready:\n{}'.format(details))
164
165
166+def get_env_type(env_name):
167+ """Return the Juju environment type for the given environment name.
168+
169+ Since the environment type is retrieved by parsing the jenv file, the
170+ environment must be already bootstrapped.
171+
172+ Raise a ProgramExit if the environment type cannot be retrieved.
173+ """
174+ try:
175+ return jenv.get_value(env_name, 'bootstrap-config', 'type')
176+ except ValueError as err:
177+ msg = b'cannot retrieve environment type: {}'.format(err)
178+ raise ProgramExit(msg)
179+
180+
181 def get_admin_secret(env_name):
182 """Return the Juju admin secret for the given environment name.
183
184@@ -385,7 +421,7 @@
185 series = settings.JUJU_GUI_SUPPORTED_SERIES[-1]
186 try:
187 # Try to get the charm URL from charmworld.
188- charm_url = utils.get_charm_url(series)
189+ charm_url = netutils.get_charm_url(series)
190 except (IOError, ValueError) as err:
191 # Fall back to the default URL for the current series.
192 msg = 'unable to retrieve the {} charm URL from the API: {}'
193
194=== modified file 'quickstart/manage.py'
195--- quickstart/manage.py 2014-11-11 11:10:33 +0000
196+++ quickstart/manage.py 2014-11-12 16:41:04 +0000
197@@ -32,6 +32,7 @@
198 import quickstart
199 from quickstart import (
200 app,
201+ netutils,
202 packaging,
203 platform_support,
204 settings,
205@@ -110,7 +111,7 @@
206 if bundle.startswith('http://') or bundle.startswith('https://'):
207 # Load the bundle from a remote URL.
208 try:
209- bundle_yaml = utils.urlread(bundle)
210+ bundle_yaml = netutils.urlread(bundle)
211 except IOError as err:
212 return parser.error('unable to open bundle URL: {}'.format(err))
213 else:
214@@ -504,32 +505,52 @@
215 logging.debug('ensuring SSH keys are available')
216 app.ensure_ssh_keys()
217
218- print('bootstrapping the {} environment (type: {})'.format(
219- options.env_name, options.env_type))
220- if options.env_type == 'local':
221- # If this is a local environment, notify the user that "sudo" will be
222- # required by Juju to bootstrap the application.
223- print('sudo privileges will be required to bootstrap the environment')
224-
225- already_bootstrapped, bootstrap_node_series = app.bootstrap(
226- options.env_name, juju_command,
227- debug=options.debug,
228- upload_tools=options.upload_tools,
229- upload_series=options.upload_series,
230- constraints=options.constraints)
231+ # Bootstrap the Juju environment or reuse an already bootstrapped one.
232+ already_bootstrapped = True
233+ env_type = options.env_type
234+ api_url = app.check_bootstrapped(options.env_name)
235+ if api_url is None:
236+ print('bootstrapping the {} environment (type: {})'.format(
237+ options.env_name, env_type))
238+ if env_type == 'local':
239+ # If this is a local environment, notify the user that "sudo" will
240+ # be required by Juju to bootstrap the environment.
241+ print('sudo privileges will be required to bootstrap the '
242+ 'environment')
243+ already_bootstrapped = app.bootstrap(
244+ options.env_name, juju_command,
245+ debug=options.debug,
246+ upload_tools=options.upload_tools,
247+ upload_series=options.upload_series,
248+ constraints=options.constraints)
249+ if already_bootstrapped:
250+ # Retrieve the environment type from the jenv file: it may be different
251+ # from the one declared on the environments.yaml file.
252+ env_type = app.get_env_type(options.env_name)
253+ print('reusing the already bootstrapped {} environment '
254+ '(type: {})'.format(options.env_name, env_type))
255+
256+ # Retrieve the environment status, ensure it is in a ready state and
257+ # contextually fetch the bootstrap node series.
258+ print('retrieving the environment status')
259+ bootstrap_node_series = app.status(options.env_name, juju_command)
260+
261+ # If the environment was not already bootstrapped, we need to retrieve
262+ # the API address.
263+ if api_url is None:
264+ print('retrieving the Juju API address')
265+ api_url = app.get_api_url(options.env_name, juju_command)
266
267 # Retrieve the admin-secret for the current environment.
268 admin_secret = app.get_admin_secret(options.env_name)
269
270- print('retrieving the Juju API address')
271- api_url = app.get_api_url(options.env_name, juju_command)
272 print('connecting to {}'.format(api_url))
273 env = app.connect(api_url, admin_secret)
274
275 # Inspect the environment and deploy the charm if required.
276 charm_url, machine, service_data, unit_data = app.check_environment(
277 env, settings.JUJU_GUI_SERVICE_NAME, options.charm_url,
278- options.env_type, bootstrap_node_series, already_bootstrapped)
279+ env_type, bootstrap_node_series, already_bootstrapped)
280 unit_name = app.deploy_gui(
281 env, settings.JUJU_GUI_SERVICE_NAME, charm_url, machine,
282 service_data, unit_data)
283
284=== added file 'quickstart/netutils.py'
285--- quickstart/netutils.py 1970-01-01 00:00:00 +0000
286+++ quickstart/netutils.py 2014-11-12 16:41:04 +0000
287@@ -0,0 +1,99 @@
288+# This file is part of the Juju Quickstart Plugin, which lets users set up a
289+# Juju environment in very few steps (https://launchpad.net/juju-quickstart).
290+# Copyright (C) 2014 Canonical Ltd.
291+#
292+# This program is free software: you can redistribute it and/or modify it under
293+# the terms of the GNU Affero General Public License version 3, as published by
294+# the Free Software Foundation.
295+#
296+# This program is distributed in the hope that it will be useful, but WITHOUT
297+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
298+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
299+# Affero General Public License for more details.
300+#
301+# You should have received a copy of the GNU Affero General Public License
302+# along with this program. If not, see <http://www.gnu.org/licenses/>.
303+
304+"""Juju Quickstart network utility functions."""
305+
306+from __future__ import unicode_literals
307+
308+import json
309+import httplib
310+import logging
311+import socket
312+import urllib2
313+
314+from quickstart import settings
315+
316+
317+def check_resolvable(hostname):
318+ """Check that the hostname can be resolved to a numeric IP address.
319+
320+ Return an error message if the address cannot be resolved.
321+ """
322+ try:
323+ address = socket.gethostbyname(hostname)
324+ except socket.error as err:
325+ return bytes(err).decode('utf-8')
326+ logging.debug('{} resolved to {}'.format(
327+ hostname, address.decode('utf-8')))
328+ return None
329+
330+
331+def check_listening(address, timeout=3):
332+ """Check that the given address is listening and accepts connections.
333+
334+ The address must be specified as a "host:port" string.
335+ Use the given socket timeout in seconds.
336+
337+ Return an error message if connecting to the address fails.
338+ """
339+ try:
340+ host, port = address.split(":")
341+ sock = socket.create_connection((host, int(port)), timeout)
342+ except (socket.error, TypeError, ValueError) as err:
343+ return 'cannot connect to {}: {}'.format(
344+ address, bytes(err).decode('utf-8'))
345+ # Ignore all possible connection close exceptions.
346+ try:
347+ sock.close()
348+ except:
349+ pass
350+ return None
351+
352+
353+def get_charm_url(series):
354+ """Return the charm URL of the latest Juju GUI charm revision.
355+
356+ Raise an IOError if any problems occur connecting to the API endpoint.
357+ Raise a ValueError if the API returns invalid data.
358+ """
359+ url = settings.CHARMWORLD_API.format(
360+ series=series, charm=settings.JUJU_GUI_CHARM_NAME)
361+ charm_info = json.loads(urlread(url))
362+ charm_url = charm_info.get('charm', {}).get('url')
363+ if charm_url is None:
364+ raise ValueError(b'unable to find the charm URL')
365+ return charm_url
366+
367+
368+def urlread(url):
369+ """Open the given URL and return the page contents.
370+
371+ Raise an IOError if any problems occur.
372+ """
373+ try:
374+ response = urllib2.urlopen(url)
375+ except urllib2.URLError as err:
376+ raise IOError(err.reason)
377+ except (httplib.HTTPException, socket.error, urllib2.HTTPError) as err:
378+ raise IOError(bytes(err))
379+ contents = response.read()
380+ content_type = response.headers['content-type']
381+ charset = 'utf-8'
382+ if 'charset=' in content_type:
383+ sent_charset = content_type.split('charset=')[-1].strip()
384+ if sent_charset:
385+ charset = sent_charset
386+ return contents.decode(charset, 'ignore')
387
388=== modified file 'quickstart/tests/helpers.py'
389--- quickstart/tests/helpers.py 2014-11-11 16:39:13 +0000
390+++ quickstart/tests/helpers.py 2014-11-12 16:41:04 +0000
391@@ -21,6 +21,7 @@
392 from contextlib import contextmanager
393 import os
394 import shutil
395+import socket
396 import tempfile
397
398 import mock
399@@ -145,6 +146,7 @@
400 'bootstrap-config': {
401 'admin-secret': 'Secret!',
402 'api-port': 17070,
403+ 'type': 'ec2',
404 },
405 'life': {'universe': {'everything': 42}},
406 }
407@@ -231,22 +233,34 @@
408 mock_print = mock.patch('__builtin__.print')
409
410
411+def patch_socket_create_connection(error=None):
412+ """Patch the socket.create_connection function.
413+
414+ If error is not None, the mock object raises a socket.error with the given
415+ message.
416+ """
417+ mock_create_connection = mock.Mock()
418+ if error is not None:
419+ mock_create_connection.side_effect = socket.error(error)
420+ return mock.patch('socket.create_connection', mock_create_connection)
421+
422+
423 def patch_check_resolvable(error=None):
424- """Patch the utils.check_resolvable function to return the given error.
425+ """Patch the netutils.check_resolvable function to return the given error.
426
427 This is done so that tests do not try to resolve hostname addresses.
428 """
429 return mock.patch(
430- 'quickstart.utils.check_resolvable',
431+ 'quickstart.netutils.check_resolvable',
432 lambda hostname: error,
433 )
434
435
436 class UrlReadTestsMixin(object):
437- """Expose a method to mock the quickstart.utils.urlread helper function."""
438+ """Helpers to mock the quickstart.netutils.urlread helper function."""
439
440 def patch_urlread(self, contents=None, error=False):
441- """Patch the quickstart.utils.urlread helper function.
442+ """Patch the quickstart.netutils.urlread helper function.
443
444 If contents is not None, urlread() will return the provided contents.
445 If error is set to True, an IOError will be simulated.
446@@ -256,7 +270,7 @@
447 mock_urlread.return_value = contents
448 if error:
449 mock_urlread.side_effect = IOError('bad wolf')
450- return mock.patch('quickstart.utils.urlread', mock_urlread)
451+ return mock.patch('quickstart.netutils.urlread', mock_urlread)
452
453
454 class ValueErrorTestsMixin(object):
455
456=== modified file 'quickstart/tests/test_app.py'
457--- quickstart/tests/test_app.py 2014-11-11 13:21:57 +0000
458+++ quickstart/tests/test_app.py 2014-11-12 16:41:04 +0000
459@@ -454,141 +454,174 @@
460 self.assertTrue(mock_create_keys.called)
461
462
463-@helpers.mock_print
464+class TestCheckBootstrapped(helpers.JenvFileTestsMixin, unittest.TestCase):
465+
466+ def test_no_jenv_file(self):
467+ # A None API URL is returned if the jenv file is not present.
468+ with self.make_jenv('ec2', ''):
469+ with helpers.assert_logs([], level='warn'):
470+ api_url = app.check_bootstrapped('hp')
471+ self.assertIsNone(api_url)
472+
473+ def test_invalid_jenv_file(self):
474+ # A None API URL is returned if the list of API addresses cannot be
475+ # retrieved from the jenv file.
476+ with self.make_jenv('ec2', '') as path:
477+ logs = [
478+ 'cannot retrieve the Juju API URL: '
479+ 'invalid YAML contents in {}: '
480+ 'state-servers key not found in the root section'.format(path)
481+ ]
482+ with helpers.assert_logs(logs, level='warn'):
483+ api_url = app.check_bootstrapped('ec2')
484+ self.assertIsNone(api_url)
485+
486+ def test_no_api_addresses(self):
487+ # A None API URL is returned if the list of API addresses is empty.
488+ jenv_data = {'state-servers': []}
489+ logs = ['cannot retrieve the Juju API URL: no addresses found']
490+ with self.make_jenv('local', yaml.safe_dump(jenv_data)):
491+ with helpers.assert_logs(logs, level='warn'):
492+ api_url = app.check_bootstrapped('local')
493+ self.assertIsNone(api_url)
494+
495+ def test_api_address_not_listening(self):
496+ # A None API URL is returned if there is no reachable API address.
497+ logs = [
498+ 'cannot retrieve the Juju API URL: '
499+ 'cannot connect to any of the following addresses: '
500+ 'localhost:17070, 10.0.3.1:17070'
501+ ]
502+ with self.make_jenv('local', yaml.safe_dump(self.jenv_data)):
503+ with helpers.assert_logs(logs, level='warn'):
504+ with helpers.patch_socket_create_connection('bad wolf'):
505+ api_url = app.check_bootstrapped('local')
506+ self.assertIsNone(api_url)
507+
508+ def test_bootstrapped(self):
509+ # The first listening API URL is returned if the environment is already
510+ # bootstrapped.
511+ with self.make_jenv('hp', yaml.safe_dump(self.jenv_data)):
512+ with helpers.assert_logs([], level='warn'):
513+ with helpers.patch_socket_create_connection():
514+ api_url = app.check_bootstrapped('hp')
515+ # The first API address is returned.
516+ self.assertEqual('wss://localhost:17070', api_url)
517+
518+
519 class TestBootstrap(
520 helpers.CallTestsMixin, ProgramExitTestsMixin, unittest.TestCase):
521
522- env_name = 'my-juju-env'
523- status_message = 'retrieving the environment status'
524- juju_command = settings.JUJU_CMD_PATHS['default']
525-
526- def make_status_output(self, agent_state, series='hoary'):
527+ def test_environment_not_bootstrapped(self):
528+ # The environment is successfully bootstrapped and False is returned.
529+ with self.patch_call(0) as mock_call:
530+ already_bootstrapped = app.bootstrap('ec2', '/usr/bin/juju')
531+ self.assertFalse(already_bootstrapped)
532+ mock_call.assert_called_once_with(
533+ '/usr/bin/juju', 'bootstrap', '-e', 'ec2')
534+
535+ def test_environment_already_bootstrapped(self):
536+ # The function succeeds and returns True if the environment is already
537+ # bootstrapped.
538+ error = '***environment is already bootstrapped***'
539+ with self.patch_call(1, error=error) as mock_call:
540+ already_bootstrapped = app.bootstrap('hp', '/bin/juju')
541+ self.assertTrue(already_bootstrapped)
542+ mock_call.assert_called_once_with('/bin/juju', 'bootstrap', '-e', 'hp')
543+
544+ def test_bootstrap_failure(self):
545+ # A ProgramExit is raised if an error occurs while bootstrapping.
546+ with self.patch_call(1, error='bad wolf') as mock_call:
547+ with self.assert_program_exit('bad wolf'):
548+ app.bootstrap('local', 'juju')
549+ mock_call.assert_called_once_with('juju', 'bootstrap', '-e', 'local')
550+
551+ def test_debug(self):
552+ # The environment is bootstrapped in debug mode.
553+ with self.patch_call(0) as mock_call:
554+ app.bootstrap('ec2', '/usr/bin/juju', debug=True)
555+ mock_call.assert_called_once_with(
556+ '/usr/bin/juju', 'bootstrap', '-e', 'ec2', '--debug')
557+
558+ def test_upload_tools(self):
559+ # The environment is bootstrapped with local tools
560+ with self.patch_call(0) as mock_call:
561+ app.bootstrap('local', '/usr/bin/juju', upload_tools=True)
562+ mock_call.assert_called_once_with(
563+ '/usr/bin/juju', 'bootstrap', '-e', 'local', '--upload-tools')
564+
565+ def test_upload_series(self):
566+ # The environment is bootstrapped with tools for specific series.
567+ with self.patch_call(0) as mock_call:
568+ app.bootstrap('hp', '/usr/bin/juju', upload_series='trusty,utopic')
569+ mock_call.assert_called_once_with(
570+ '/usr/bin/juju', 'bootstrap', '-e', 'hp',
571+ '--upload-series', 'trusty,utopic')
572+
573+ def test_constraints(self):
574+ # The environment is bootstrapped with the specified constraints.
575+ with self.patch_call(0) as mock_call:
576+ app.bootstrap('maas', '/usr/bin/juju', constraints='mem=7G')
577+ mock_call.assert_called_once_with(
578+ '/usr/bin/juju', 'bootstrap', '-e', 'maas',
579+ '--constraints', 'mem=7G')
580+
581+ def test_all_options(self):
582+ # The environment is bootstrapped with all the options.
583+ with self.patch_call(0) as mock_call:
584+ app.bootstrap(
585+ 'local', '/usr/bin/juju', debug=True, upload_tools=True,
586+ upload_series='vivid', constraints='mem=8G')
587+ mock_call.assert_called_once_with(
588+ '/usr/bin/juju', 'bootstrap', '-e', 'local',
589+ '--debug', '--upload-tools', '--upload-series', 'vivid',
590+ '--constraints', 'mem=8G')
591+
592+
593+class TestStatus(
594+ helpers.CallTestsMixin, ProgramExitTestsMixin, unittest.TestCase):
595+
596+ def make_status_output(self, agent_state, series='utopic'):
597 """Create and return a YAML status output."""
598 return yaml.safe_dump({
599- 'machines': {'0': {'agent-state': agent_state,
600- 'series': series}},
601+ 'machines': {
602+ '0': {'agent-state': agent_state, 'series': series},
603+ },
604 })
605
606- def make_status_calls(self, number):
607+ def make_status_calls(self, env_name, juju_command, number):
608 """Return a list containing the given number of status calls."""
609 call = mock.call(
610- self.juju_command, 'status', '-e', self.env_name,
611- '--format', 'yaml')
612+ juju_command, 'status', '-e', env_name, '--format', 'yaml')
613 return [call for _ in range(number)]
614
615- def make_side_effects(self):
616- """Return the minimum number of side effects for a successful call."""
617- return [
618- (0, '', ''), # Add a bootstrap call.
619- (0, self.make_status_output('started'), ''), # Add a status call.
620- ]
621-
622- def assert_status_retried(self, side_effects):
623+ def assert_status_retried(self, env_name, juju_command, side_effects):
624 """Ensure the "juju status" command is retried several times.
625
626 Receive the list of side effects the mock status call will return.
627 """
628- with self.patch_multiple_calls(side_effects) as mock_call:
629- app.bootstrap(self.env_name, self.juju_command)
630- mock_call.assert_has_calls([
631- mock.call(self.juju_command, 'bootstrap', '-e', self.env_name),
632- ] + self.make_status_calls(5))
633-
634- def test_success(self, mock_print):
635- # The environment is successfully bootstrapped.
636- with self.patch_multiple_calls(self.make_side_effects()) as mock_call:
637- already_bootstrapped, series = app.bootstrap(
638- self.env_name, self.juju_command)
639- self.assertFalse(already_bootstrapped)
640- self.assertEqual(series, 'hoary')
641- mock_call.assert_has_calls([
642- mock.call(self.juju_command, 'bootstrap', '-e', self.env_name),
643- ] + self.make_status_calls(1))
644- mock_print.assert_called_once_with(self.status_message)
645-
646- def test_success_debug(self, mock_print):
647- # The environment is successfully bootstrapped in debug mode.
648- with self.patch_multiple_calls(self.make_side_effects()) as mock_call:
649- already_bootstrapped, series = app.bootstrap(
650- self.env_name, self.juju_command, debug=True)
651- self.assertFalse(already_bootstrapped)
652- self.assertEqual(series, 'hoary')
653- mock_call.assert_has_calls([
654- mock.call(
655- self.juju_command, 'bootstrap', '-e', self.env_name,
656- '--debug'),
657- ] + self.make_status_calls(1))
658-
659- def test_success_upload_tools(self, mock_print):
660- # The environment is bootstrapped with local tools.
661- with self.patch_multiple_calls(self.make_side_effects()) as mock_call:
662- already_bootstrapped, series = app.bootstrap(
663- self.env_name, self.juju_command, upload_tools=True)
664- self.assertFalse(already_bootstrapped)
665- mock_call.assert_has_calls([
666- mock.call(
667- self.juju_command, 'bootstrap', '-e', self.env_name,
668- '--upload-tools'),
669- ] + self.make_status_calls(1))
670-
671- def test_success_upload_series(self, mock_print):
672- # The environment is bootstrapped with tools for specific series.
673- with self.patch_multiple_calls(self.make_side_effects()) as mock_call:
674- already_bootstrapped, series = app.bootstrap(
675- self.env_name, self.juju_command, upload_series='hoary')
676- self.assertFalse(already_bootstrapped)
677- mock_call.assert_has_calls([
678- mock.call(
679- self.juju_command, 'bootstrap', '-e', self.env_name,
680- '--upload-series', 'hoary'),
681- ] + self.make_status_calls(1))
682-
683- def test_success_constraints(self, mock_print):
684- # The environment is bootstrapped with given constraints.
685- with self.patch_multiple_calls(self.make_side_effects()) as mock_call:
686- already_bootstrapped, series = app.bootstrap(
687- self.env_name, self.juju_command, constraints='mem=7G')
688- self.assertFalse(already_bootstrapped)
689- mock_call.assert_has_calls([
690- mock.call(
691- self.juju_command, 'bootstrap', '-e', self.env_name,
692- '--constraints', 'mem=7G'),
693- ] + self.make_status_calls(1))
694-
695- def test_already_bootstrapped(self, mock_print):
696- # The function succeeds and returns True if the environment is already
697- # bootstrapped.
698- side_effects = [
699- (1, '', '***environment is already bootstrapped**'),
700- (0, self.make_status_output('started', 'precise'), ''),
701- ]
702- with self.patch_multiple_calls(side_effects) as mock_call:
703- already_bootstrapped, series = app.bootstrap(
704- self.env_name, self.juju_command)
705- self.assertTrue(already_bootstrapped)
706- self.assertEqual(series, 'precise')
707- mock_call.assert_has_calls([
708- mock.call(self.juju_command, 'bootstrap', '-e', self.env_name),
709- ] + self.make_status_calls(1))
710- existing_message = 'reusing the already bootstrapped {} environment'
711- mock_print.assert_has_calls([
712- mock.call(existing_message.format(self.env_name)),
713- mock.call(self.status_message),
714- ])
715-
716- def test_bootstrap_failure(self, mock_print):
717- # A ProgramExit is raised if an error occurs while bootstrapping.
718- with self.patch_call(retcode=1, error='bad wolf') as mock_call:
719- with self.assert_program_exit('bad wolf'):
720- app.bootstrap(self.env_name, self.juju_command)
721- mock_call.assert_called_once_with(
722- self.juju_command, 'bootstrap', '-e', self.env_name),
723-
724- def test_status_retry_error(self, mock_print):
725+ count = len(side_effects)
726+ with self.patch_multiple_calls(side_effects) as mock_call:
727+ app.status(env_name, juju_command)
728+ self.assertEqual(count, mock_call.call_count)
729+ expected_calls = self.make_status_calls(env_name, juju_command, count)
730+ mock_call.assert_has_calls(expected_calls)
731+
732+ def test_success(self):
733+ # The status command is correctly called and the bootstrap node series
734+ # returned.
735+ output = self.make_status_output('started')
736+ with self.patch_call(0, output=output) as mock_call:
737+ bootstrap_node_series = app.status('ec2', '/usr/bin/juju')
738+ self.assertEqual('utopic', bootstrap_node_series)
739+ self.assertEqual(1, mock_call.call_count)
740+ expected_calls = self.make_status_calls('ec2', '/usr/bin/juju', 1)
741+ mock_call.assert_has_calls(expected_calls)
742+
743+ def test_status_retry_error(self):
744 # Before raising a ProgramExit, the functions tries to call
745 # "juju status" multiple times if it exits with an error.
746 side_effects = [
747- (0, '', ''), # Add the bootstrap call.
748 # Add four status calls with a non-zero exit code.
749 (1, '', 'these'),
750 (2, '', 'are'),
751@@ -597,29 +630,28 @@
752 # Add a final valid status call.
753 (0, self.make_status_output('started'), ''),
754 ]
755- self.assert_status_retried(side_effects)
756+ self.assert_status_retried('local', 'juju', side_effects)
757
758- def test_status_retry_invalid_output(self, mock_print):
759+ def test_status_retry_invalid_output(self):
760 # Before raising a ProgramExit, the functions tries to call
761 # "juju status" multiple times if its output is not well formed or if
762 # the agent is not started.
763 side_effects = [
764- (0, '', ''), # Add the bootstrap call.
765 (0, '', ''), # Add the first status call: no output.
766 (0, ':', ''), # Add the second status call: not YAML.
767 (0, 'just-a-string', ''), # Add the third status call: bad YAML.
768- # Add the fourth status call: the agent is still pending.
769+ # Add two other status calls: the agent is still pending.
770+ (0, self.make_status_output('pending'), ''),
771 (0, self.make_status_output('pending'), ''),
772 # Add a final valid status call.
773 (0, self.make_status_output('started'), ''),
774 ]
775- self.assert_status_retried(side_effects)
776+ self.assert_status_retried('hp', '/usr/bin/juju', side_effects)
777
778- def test_status_retry_both(self, mock_print):
779+ def test_status_retry_both(self):
780 # Before raising a ProgramExit, the functions tries to call
781 # "juju status" multiple times in any case.
782 side_effects = [
783- (0, '', ''), # Add the bootstrap call.
784 (1, '', 'error'), # Add the first status call: error.
785 (2, '', 'another error'), # Add the second status call: error.
786 # Add the third status call: the agent is still pending.
787@@ -628,28 +660,23 @@
788 # Add a final valid status call.
789 (0, self.make_status_output('started'), ''),
790 ]
791- self.assert_status_retried(side_effects)
792+ self.assert_status_retried('local', '/usr/bin/juju', side_effects)
793
794- def test_agent_error(self, mock_print):
795+ def test_agent_error(self):
796 # A ProgramExit is raised immediately if the Juju agent in the
797 # bootstrap node is in an error state.
798- status_output = self.make_status_output('error')
799- side_effects = [
800- (0, '', ''), # Add the bootstrap call.
801- (0, status_output, ''), # Add the status call: agent error.
802- ]
803- expected = 'state server failure:\n{}'.format(status_output)
804- with self.patch_multiple_calls(side_effects) as mock_call:
805- with self.assert_program_exit(expected):
806- app.bootstrap(self.env_name, self.juju_command)
807- mock_call.assert_has_calls([
808- mock.call(self.juju_command, 'bootstrap', '-e', self.env_name),
809- ] + self.make_status_calls(1))
810+ output = self.make_status_output('error')
811+ expected_error = 'state server failure:\n{}'.format(output)
812+ with self.patch_call(0, output=output) as mock_call:
813+ with self.assert_program_exit(expected_error):
814+ app.status('ec2', '/usr/bin/juju')
815+ self.assertEqual(1, mock_call.call_count)
816+ expected_calls = self.make_status_calls('ec2', '/usr/bin/juju', 1)
817+ mock_call.assert_has_calls(expected_calls)
818
819- def test_status_failure(self, mock_print):
820+ def test_status_failure(self):
821 # A ProgramExit is raised if "juju status" keeps failing.
822 call_side_effects = [
823- (0, '', ''), # Add the bootstrap call.
824 (1, 'output1', 'error1'), # Add the first status call: retried.
825 (1, 'output2', 'error2'), # Add the second status call: error.
826 ]
827@@ -660,17 +687,37 @@
828 1000, # Third call after the timeout expiration.
829 ]
830 mock_time = mock.Mock(side_effect=time_side_effects)
831- expected = 'the state server is not ready:\noutput2error2'
832+ expected_error = 'the state server is not ready:\noutput2error2'
833 with self.patch_multiple_calls(call_side_effects) as mock_call:
834 # Simulate the timeout expired: the first time call is used to
835 # calculate the timeout, the second one for the first status check,
836 # the third for the second status check, the fourth should fail.
837 with mock.patch('time.time', mock_time):
838- with self.assert_program_exit(expected):
839- app.bootstrap(self.env_name, self.juju_command)
840- mock_call.assert_has_calls([
841- mock.call(self.juju_command, 'bootstrap', '-e', self.env_name),
842- ] + self.make_status_calls(2))
843+ with self.assert_program_exit(expected_error):
844+ app.status('local', '/usr/bin/juju')
845+ self.assertEqual(2, mock_call.call_count)
846+ expected_calls = self.make_status_calls('local', '/usr/bin/juju', 2)
847+ mock_call.assert_has_calls(expected_calls)
848+
849+
850+class TestGetEnvType(
851+ helpers.JenvFileTestsMixin, ProgramExitTestsMixin, unittest.TestCase):
852+
853+ def test_success(self):
854+ # The environment type is successfully retrieved.
855+ with self.make_jenv('ec2', yaml.safe_dump(self.jenv_data)):
856+ env_type = app.get_env_type('ec2')
857+ self.assertEqual('ec2', env_type)
858+
859+ def test_error(self):
860+ # A ProgramExit is raised if the environment type cannot be retrieved.
861+ with self.make_jenv('aws', '') as path:
862+ expected_error = (
863+ 'cannot retrieve environment type: invalid YAML '
864+ 'contents in {}: bootstrap-config key not found in the root '
865+ 'section'.format(path))
866+ with self.assert_program_exit(expected_error):
867+ app.get_env_type('aws')
868
869
870 class TestGetAdminSecret(
871@@ -850,7 +897,8 @@
872 """Patch the get_charm_url helper function."""
873 mock_get_charm_url = mock.Mock(
874 return_value=return_value, side_effect=side_effect)
875- return mock.patch('quickstart.utils.get_charm_url', mock_get_charm_url)
876+ return mock.patch(
877+ 'quickstart.netutils.get_charm_url', mock_get_charm_url)
878
879 def test_environment_just_bootstrapped(self, mock_print):
880 # The function correctly retrieves the charm URL and machine, and
881
882=== modified file 'quickstart/tests/test_manage.py'
883--- quickstart/tests/test_manage.py 2014-11-11 13:21:57 +0000
884+++ quickstart/tests/test_manage.py 2014-11-12 16:41:04 +0000
885@@ -38,7 +38,6 @@
886 from quickstart.cli import views
887 from quickstart.models import envs
888 from quickstart.tests import helpers
889-from quickstart import app
890
891
892 class TestDescriptionAction(unittest.TestCase):
893@@ -763,164 +762,177 @@
894 options.update(kwargs)
895 return mock.Mock(**options)
896
897- def test_no_bundle(self, mock_app, mock_open):
898+ def configure_app(self, mock_app, **kwargs):
899+ """Configure the given mock_app with the given kwargs.
900+
901+ Each key/value pair in kwargs represents a mock_app attribute and
902+ the associated return value. Those values are used to override
903+ defaults.
904+
905+ Return the mock Juju environment object.
906+ """
907+ env = mock.Mock()
908+ defaults = {
909+ # Dependencies are installed.
910+ 'ensure_dependencies': (1, 18, 0),
911+ # Ensure the current Juju version is supported.
912+ 'check_juju_supported': None,
913+ # Ensure the SSH keys are properly configured.
914+ 'ensure_ssh_keys': None,
915+ # The environment is not already bootstrapped.
916+ 'check_bootstrapped': None,
917+ # This is also confirmed by the bootstrap function.
918+ 'bootstrap': False,
919+ # Status is then called, returning the bootstrap node series.
920+ 'status': 'trusty',
921+ # The API URL must be retrieved (the environment was not ready).
922+ 'get_api_url': 'wss://1.2.3.4:17070',
923+ # Retrieve the admin secret.
924+ 'get_admin_secret': 'Secret!',
925+ # Connect to the Juju Environment API endpoint.
926+ 'connect': env,
927+ # The environment is then checked.
928+ 'check_environment': (
929+ 'cs:trusty/juju-gui-42',
930+ '0',
931+ {'Name': 'juju-gui'},
932+ {'Name': 'juju-gui/0'}
933+ ),
934+ # Deploy the Juju GUI charm.
935+ 'deploy_gui': 'juju-gui/0',
936+ # Watch the deployment progress and return the unit address.
937+ 'watch': '1.2.3.4',
938+ # Create the login token for the Juju GUI.
939+ 'create_auth_token': 'TOKEN',
940+ }
941+ defaults.update(kwargs)
942+ for attr, return_value in defaults.items():
943+ getattr(mock_app, attr).return_value = return_value
944+ return env
945+
946+ def patch_get_juju_command(self):
947+ """Patch the platform_support.get_juju_command function."""
948+ path = 'quickstart.manage.platform_support.get_juju_command'
949+ return mock.patch(path, return_value=(self.juju_command, False))
950+
951+ def test_run(self, mock_app, mock_open):
952 # The application runs correctly if no bundle is provided.
953- mock_app.ensure_dependencies.return_value = (1, 18, 0)
954- # Make mock_app.bootstrap return the already_bootstrapped flag and the
955- # bootstrap node series.
956- mock_app.bootstrap.return_value = (True, 'trusty')
957- # Make mock_app.check_environment return the charm URL, the machine
958- # where to deploy the charm, the service and unit data.
959- service_data = {'Name': 'juju-gui'}
960- unit_data = {'Name': 'juju-gui/0'}
961- mock_app.check_environment.return_value = (
962- 'cs:trusty/juju-gui-42', '0', service_data, unit_data)
963- # Make mock_app.get_admin_secret return the admin secret
964- mock_app.get_admin_secret.return_value = 'Secret!'
965- # Make mock_app.watch return the Juju GUI unit address.
966- mock_app.watch.return_value = '1.2.3.4'
967- # Make mock_app.create_auth_token return a fake authentication token.
968- mock_app.create_auth_token.return_value = 'AUTHTOKEN'
969+ env = self.configure_app(mock_app)
970+ # Run the application.
971 options = self.make_options()
972- with mock.patch('quickstart.manage.platform_support.get_juju_command',
973- side_effect=[(self.juju_command, False)]):
974+ with self.patch_get_juju_command():
975 manage.run(options)
976- mock_app.ensure_dependencies.assert_called()
977- mock_app.ensure_ssh_keys.assert_called()
978+ # Ensure the functions have been used correctly.
979+ mock_app.ensure_dependencies.assert_called_once_with(
980+ options.distro_only, options.platform, self.juju_command)
981+ mock_app.check_juju_supported.assert_called_once_with((1, 18, 0))
982+ mock_app.ensure_ssh_keys.assert_called_once_with()
983+ mock_app.check_bootstrapped.assert_called_once_with(options.env_name)
984 mock_app.bootstrap.assert_called_once_with(
985 options.env_name, self.juju_command,
986 debug=options.debug,
987 upload_tools=options.upload_tools,
988 upload_series=options.upload_series,
989 constraints=options.constraints)
990+ mock_app.status.assert_called_once_with(
991+ options.env_name, self.juju_command)
992 mock_app.get_api_url.assert_called_once_with(
993 options.env_name, self.juju_command)
994+ mock_app.get_admin_secret.assert_called_once_with(options.env_name)
995 mock_app.connect.assert_has_calls([
996- mock.call(mock_app.get_api_url(), 'Secret!'),
997+ mock.call('wss://1.2.3.4:17070', 'Secret!'),
998 mock.call().close(),
999 mock.call('wss://1.2.3.4:443/ws', 'Secret!'),
1000 mock.call().close(),
1001 ])
1002 mock_app.check_environment.assert_called_once_with(
1003- mock_app.connect(), settings.JUJU_GUI_SERVICE_NAME,
1004- options.charm_url, options.env_type, mock_app.bootstrap()[1],
1005- mock_app.bootstrap()[0])
1006+ env, settings.JUJU_GUI_SERVICE_NAME, options.charm_url,
1007+ options.env_type, 'trusty', False)
1008 mock_app.deploy_gui.assert_called_once_with(
1009- mock_app.connect(), settings.JUJU_GUI_SERVICE_NAME,
1010- 'cs:trusty/juju-gui-42', '0', service_data, unit_data)
1011- mock_app.watch.assert_called_once_with(
1012- mock_app.connect(), mock_app.deploy_gui())
1013- mock_app.create_auth_token.assert_called_once_with(mock_app.connect())
1014- mock_open.assert_called_once_with(
1015- 'https://{}/?authtoken={}'.format(mock_app.watch(), 'AUTHTOKEN'))
1016+ env, settings.JUJU_GUI_SERVICE_NAME, 'cs:trusty/juju-gui-42',
1017+ '0', {'Name': 'juju-gui'}, {'Name': 'juju-gui/0'})
1018+ mock_app.watch.assert_called_once_with(env, 'juju-gui/0')
1019+ mock_app.create_auth_token.assert_called_once_with(env)
1020+ mock_open.assert_called_once_with('https://1.2.3.4/?authtoken=TOKEN')
1021+ # Ensure some of the app function have not been called.
1022+ self.assertFalse(mock_app.get_env_type.called)
1023 self.assertFalse(mock_app.deploy_bundle.called)
1024
1025+ def test_already_bootstrapped(self, mock_app, mock_open):
1026+ # The application correctly reuses an already bootstrapped environment.
1027+ self.configure_app(mock_app, check_bootstrapped='wss://example.com')
1028+ # Run the application.
1029+ options = self.make_options()
1030+ with self.patch_get_juju_command():
1031+ manage.run(options)
1032+ # The environment type is retrieved from the jenv.
1033+ mock_app.get_env_type.assert_called_once_with(options.env_name)
1034+ # No reason to call bootstrap or get_api_url functions.
1035+ self.assertFalse(mock_app.bootstrap.called)
1036+ self.assertFalse(mock_app.get_api_url.called)
1037+
1038+ def test_already_bootstrapped_race(self, mock_app, mock_open):
1039+ # The application correctly reuses an already bootstrapped environment.
1040+ # In this case, the environment seems not bootstrapped at first, but
1041+ # it ended up being up and running later.
1042+ self.configure_app(mock_app, bootstrap=True)
1043+ # Run the application.
1044+ options = self.make_options()
1045+ with self.patch_get_juju_command():
1046+ manage.run(options)
1047+ # The bootstrap and get_api_url functions are still called, but this
1048+ # time also get_env_type is required.
1049+ # The environment type is retrieved from the jenv.
1050+ mock_app.bootstrap.assert_called_once_with(
1051+ options.env_name, self.juju_command,
1052+ debug=options.debug,
1053+ upload_tools=options.upload_tools,
1054+ upload_series=options.upload_series,
1055+ constraints=options.constraints)
1056+ mock_app.get_api_url.assert_called_once_with(
1057+ options.env_name, self.juju_command)
1058+ mock_app.get_env_type.assert_called_once_with(options.env_name)
1059+
1060 def test_no_token(self, mock_app, mock_open):
1061 # The process continues even if the authentication token cannot be
1062 # retrieved.
1063- mock_app.create_auth_token.return_value = None
1064- # Make mock_app.bootstrap return the already_bootstrapped flag and the
1065- # bootstrap node series.
1066- mock_app.bootstrap.return_value = (True, 'precise')
1067- # Make mock_app.check_environment return the charm URL, the machine
1068- # where to deploy the charm, the service and unit data.
1069- mock_app.check_environment.return_value = (
1070- 'cs:precise/juju-gui-42', '0', None, None)
1071+ env = self.configure_app(mock_app, create_auth_token=None)
1072+ # Run the application.
1073 options = self.make_options()
1074- manage.run(options)
1075- mock_app.create_auth_token.assert_called_once_with(mock_app.connect())
1076- mock_open.assert_called_once_with(
1077- 'https://{}'.format(mock_app.watch()))
1078+ with self.patch_get_juju_command():
1079+ manage.run(options)
1080+ # Ensure the browser is still open without an auth token.
1081+ mock_app.create_auth_token.assert_called_once_with(env)
1082+ mock_open.assert_called_once_with('https://1.2.3.4')
1083
1084 def test_bundle(self, mock_app, mock_open):
1085 # A bundle is correctly deployed by the application.
1086+ env = self.configure_app(mock_app, create_auth_token=None)
1087+ # Run the application.
1088 options = self.make_options(
1089 bundle='/my/bundle/file.yaml', bundle_yaml='mybundle: contents',
1090 bundle_name='mybundle', bundle_services=['service1', 'service2'])
1091- # Make mock_app.bootstrap return the already_bootstrapped flag and the
1092- # bootstrap node series.
1093- mock_app.bootstrap.return_value = (True, 'trusty')
1094- # Make mock_app.check_environment return the charm URL, the machine
1095- # where to deploy the charm, the service and unit data.
1096- mock_app.check_environment.return_value = (
1097- 'cs:trusty/juju-gui-42', '0', None, None)
1098- # Make mock_app.watch return the Juju GUI unit address.
1099- mock_app.watch.return_value = 'gui.example.com'
1100- manage.run(options)
1101+ with self.patch_get_juju_command():
1102+ manage.run(options)
1103+ # Ensure the bundle is correctly deployed.
1104 mock_app.deploy_bundle.assert_called_once_with(
1105- mock_app.connect(), 'mybundle: contents', 'mybundle', None)
1106+ env, 'mybundle: contents', 'mybundle', None)
1107
1108- def test_local_provider_no_sudo(self, mock_app, mock_open):
1109+ def test_local_provider(self, mock_app, mock_open):
1110 # The application correctly handles working with local providers with
1111 # new Juju versions not requiring "sudo" to bootstrap the environment.
1112- # Sudo privileges are not required if the Juju version is >= 1.17.2.
1113+ self.configure_app(mock_app, create_auth_token=None)
1114+ # Run the application.
1115 options = self.make_options(env_type='local')
1116- versions = [
1117- (1, 17, 2), (1, 17, 10), (1, 18, 0), (1, 18, 2), (2, 16, 1)]
1118- # Make mock_app.bootstrap return the already_bootstrapped flag and the
1119- # bootstrap node series.
1120- mock_app.bootstrap.return_value = (True, 'precise')
1121- # Make mock_app.check_environment return the charm URL, the machine
1122- # where to deploy the charm, the service and unit data.
1123- mock_app.check_environment.return_value = (
1124- 'cs:precise/juju-gui-42', '0', None, None)
1125- for version in versions:
1126- mock_app.ensure_dependencies.return_value = version
1127- with mock.patch(
1128- 'quickstart.manage.platform_support.get_juju_command',
1129- side_effect=[(self.juju_command, False)]):
1130- manage.run(options)
1131- mock_app.bootstrap.assert_called_once_with(
1132- options.env_name, self.juju_command,
1133- debug=options.debug,
1134- upload_tools=options.upload_tools,
1135- upload_series=options.upload_series,
1136- constraints=options.constraints)
1137- mock_app.bootstrap.reset_mock()
1138+ with self.patch_get_juju_command():
1139+ manage.run(options)
1140
1141 def test_no_browser(self, mock_app, mock_open):
1142 # It is possible to avoid opening the GUI in the browser.
1143- # Make mock_app.bootstrap return the already_bootstrapped flag and the
1144- # bootstrap node series.
1145- mock_app.bootstrap.return_value = (True, 'trusty')
1146- # Make mock_app.check_environment return the charm URL, the machine
1147- # where to deploy the charm, the service and unit data.
1148- mock_app.check_environment.return_value = (
1149- 'cs:trusty/juju-gui-42', '0', None, None)
1150+ self.configure_app(mock_app, create_auth_token=None)
1151+ # Run the application.
1152 options = self.make_options(open_browser=False)
1153- manage.run(options)
1154+ with self.patch_get_juju_command():
1155+ manage.run(options)
1156+ # The browser is not opened.
1157 self.assertFalse(mock_open.called)
1158-
1159- def test_no_admin_secret_found(self, mock_app, mock_open):
1160- # If admin-secret cannot be found a ProgramExit is called.
1161- mock_app.get_admin_secret.side_effect = app.ProgramExit('bad wolf')
1162- # Make mock_app.bootstrap return the already_bootstrapped flag and the
1163- # bootstrap node series.
1164- mock_app.bootstrap.return_value = (True, 'precise')
1165- # Make mock_app.check_environment return the charm URL, the machine
1166- # where to deploy the charm, the service and unit data.
1167- mock_app.check_environment.return_value = (
1168- 'cs:precise/juju-gui-42', '0', None, None)
1169- options = self.make_options(
1170- env_name='local',
1171- env_file='environments.yaml')
1172- with self.assertRaises(app.ProgramExit) as context:
1173- manage.run(options)
1174- self.assertEqual('bad wolf', context.exception.message)
1175-
1176- def test_juju_environ_var_set(self, mock_app, mock_open):
1177- mock_app.bootstrap.return_value = (True, 'precise')
1178- mock_app.check_environment.return_value = (
1179- 'cs:precise/juju-gui-42', '0', None, None)
1180- options = self.make_options(env_type='aws')
1181- juju_command = 'some/devel/path/juju'
1182- with mock.patch('os.environ', {'JUJU': juju_command}):
1183- manage.run(options)
1184- mock_app.bootstrap.assert_called_once_with(
1185- options.env_name, juju_command,
1186- debug=options.debug,
1187- upload_tools=options.upload_tools,
1188- upload_series=options.upload_series,
1189- constraints=options.constraints)
1190- mock_app.get_api_url.assert_called_once_with(
1191- options.env_name, juju_command)
1192
1193=== added file 'quickstart/tests/test_netutils.py'
1194--- quickstart/tests/test_netutils.py 1970-01-01 00:00:00 +0000
1195+++ quickstart/tests/test_netutils.py 2014-11-12 16:41:04 +0000
1196@@ -0,0 +1,201 @@
1197+# This file is part of the Juju Quickstart Plugin, which lets users set up a
1198+# Juju environment in very few steps (https://launchpad.net/juju-quickstart).
1199+# Copyright (C) 2013-2014 Canonical Ltd.
1200+#
1201+# This program is free software: you can redistribute it and/or modify it under
1202+# the terms of the GNU Affero General Public License version 3, as published by
1203+# the Free Software Foundation.
1204+#
1205+# This program is distributed in the hope that it will be useful, but WITHOUT
1206+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
1207+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
1208+# Affero General Public License for more details.
1209+#
1210+# You should have received a copy of the GNU Affero General Public License
1211+# along with this program. If not, see <http://www.gnu.org/licenses/>.
1212+
1213+"""Tests for the Juju Quickstart network utility functions."""
1214+
1215+from __future__ import unicode_literals
1216+
1217+import httplib
1218+import json
1219+import socket
1220+import unittest
1221+import urllib2
1222+
1223+import mock
1224+
1225+from quickstart import netutils
1226+from quickstart.tests import helpers
1227+
1228+
1229+class TestCheckResolvable(unittest.TestCase):
1230+
1231+ def test_resolvable(self):
1232+ # None is returned if the hostname can be resolved.
1233+ expected_log = 'example.com resolved to 1.2.3.4'
1234+ with helpers.assert_logs([expected_log], level='debug'):
1235+ with mock.patch('socket.gethostbyname', return_value='1.2.3.4'):
1236+ error = netutils.check_resolvable('example.com')
1237+ self.assertIsNone(error)
1238+
1239+ def test_not_resolvable(self):
1240+ # An error message is returned if the hostname cannot be resolved.
1241+ exception = socket.gaierror('bad wolf')
1242+ with mock.patch('socket.gethostbyname', side_effect=exception):
1243+ error = netutils.check_resolvable('example.com')
1244+ self.assertEqual('bad wolf', error)
1245+
1246+
1247+class TestCheckListening(unittest.TestCase):
1248+
1249+ def test_listening(self):
1250+ # None is returned if the address can be connected to.
1251+ with helpers.patch_socket_create_connection() as mock_connection:
1252+ error = netutils.check_listening('1.2.3.4:17070')
1253+ self.assertIsNone(error)
1254+ mock_connection.assert_called_once_with(('1.2.3.4', 17070), 3)
1255+
1256+ def test_timeout(self):
1257+ # A customized timeout can be passed.
1258+ with helpers.patch_socket_create_connection() as mock_connection:
1259+ error = netutils.check_listening('1.2.3.4:17070', timeout=42)
1260+ self.assertIsNone(error)
1261+ mock_connection.assert_called_once_with(('1.2.3.4', 17070), 42)
1262+
1263+ def test_address_not_valid(self):
1264+ # An error message is returned if the address is not valid.
1265+ with helpers.patch_socket_create_connection() as mock_connection:
1266+ error = netutils.check_listening('1.2.3.4')
1267+ self.assertEqual(
1268+ 'cannot connect to 1.2.3.4: need more than 1 value to unpack',
1269+ error)
1270+ self.assertFalse(mock_connection.called)
1271+
1272+ def test_port_not_valid(self):
1273+ # An error message is returned if the address port is not valid.
1274+ with helpers.patch_socket_create_connection() as mock_connection:
1275+ error = netutils.check_listening('1.2.3.4:bad-port')
1276+ self.assertEqual(
1277+ 'cannot connect to 1.2.3.4:bad-port: '
1278+ "invalid literal for int() with base 10: 'bad-port'",
1279+ error)
1280+ self.assertFalse(mock_connection.called)
1281+
1282+ def test_not_listening(self):
1283+ # An error message is returned if the address is not reachable.
1284+ with helpers.patch_socket_create_connection('boo!') as mock_connection:
1285+ error = netutils.check_listening('1.2.3.4:17070')
1286+ self.assertEqual('cannot connect to 1.2.3.4:17070: boo!', error)
1287+ mock_connection.assert_called_once_with(('1.2.3.4', 17070), 3)
1288+
1289+ def test_closing(self):
1290+ # The socket connection is properly closed.
1291+ with helpers.patch_socket_create_connection() as mock_connection:
1292+ netutils.check_listening('1.2.3.4:17070')
1293+ mock_connection().close.assert_called_once_with()
1294+
1295+ def test_error_closing(self):
1296+ # Errors closing the socket connection are ignored.
1297+ with helpers.patch_socket_create_connection() as mock_connection:
1298+ mock_connection().close.side_effect = socket.error('bad wolf')
1299+ netutils.check_listening('1.2.3.4:17070')
1300+
1301+
1302+class TestGetCharmUrl(helpers.UrlReadTestsMixin, unittest.TestCase):
1303+
1304+ def test_charm_url(self):
1305+ # The Juju GUI charm URL is correctly returned.
1306+ contents = json.dumps({'charm': {'url': 'cs:trusty/juju-gui-42'}})
1307+ with self.patch_urlread(contents=contents) as mock_urlread:
1308+ charm_url = netutils.get_charm_url('trusty')
1309+ self.assertEqual('cs:trusty/juju-gui-42', charm_url)
1310+ mock_urlread.assert_called_once_with(
1311+ 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui')
1312+
1313+ def test_io_error(self):
1314+ # IOErrors are properly propagated.
1315+ with self.patch_urlread(error=True) as mock_urlread:
1316+ with self.assertRaises(IOError) as context_manager:
1317+ netutils.get_charm_url('precise')
1318+ mock_urlread.assert_called_once_with(
1319+ 'http://manage.jujucharms.com/api/3/charm/precise/juju-gui')
1320+ self.assertEqual('bad wolf', bytes(context_manager.exception))
1321+
1322+ def test_value_error(self):
1323+ # A ValueError is raised if the API response is not valid.
1324+ contents = json.dumps({'charm': {}})
1325+ with self.patch_urlread(contents=contents) as mock_urlread:
1326+ with self.assertRaises(ValueError) as context_manager:
1327+ netutils.get_charm_url('trusty')
1328+ mock_urlread.assert_called_once_with(
1329+ 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui')
1330+ self.assertEqual(
1331+ 'unable to find the charm URL', bytes(context_manager.exception))
1332+
1333+
1334+class TestUrlread(unittest.TestCase):
1335+
1336+ def patch_urlopen(self, contents=None, error=None, content_type=None):
1337+ """Patch the urllib2.urlopen function.
1338+
1339+ If contents is not None, the read() method of the returned mock object
1340+ returns the given contents.
1341+ If content_type is provided, the response includes the content type.
1342+ If an error is provided, the call raises the error.
1343+ """
1344+ mock_urlopen = mock.MagicMock()
1345+ if contents is not None:
1346+ mock_urlopen().read.return_value = contents
1347+ if content_type is not None:
1348+ mock_urlopen().headers = {'content-type': content_type}
1349+ if error is not None:
1350+ mock_urlopen.side_effect = error
1351+ mock_urlopen.reset_mock()
1352+ return mock.patch('urllib2.urlopen', mock_urlopen)
1353+
1354+ def test_contents(self):
1355+ # The URL contents are correctly returned.
1356+ with self.patch_urlopen(contents=b'URL contents') as mock_urlopen:
1357+ contents = netutils.urlread('http://example.com/path/')
1358+ self.assertEqual('URL contents', contents)
1359+ self.assertIsInstance(contents, unicode)
1360+ mock_urlopen.assert_called_once_with('http://example.com/path/')
1361+
1362+ def test_content_type(self):
1363+ # The URL contents are decoded using the site charset.
1364+ patch_urlopen = self.patch_urlopen(
1365+ contents=b'URL contents: \xf8', # This is not a UTF-8 byte string.
1366+ content_type='text/html; charset=ISO-8859-1')
1367+ with patch_urlopen as mock_urlopen:
1368+ contents = netutils.urlread('http://example.com/path/')
1369+ self.assertEqual('URL contents: \xf8', contents)
1370+ self.assertIsInstance(contents, unicode)
1371+ mock_urlopen.assert_called_once_with('http://example.com/path/')
1372+
1373+ def test_no_content_type(self):
1374+ # The URL contents are decoded with UTF-8 by default.
1375+ patch_urlopen = self.patch_urlopen(
1376+ contents=b'URL contents: \xf8', # This is not a UTF-8 byte string.
1377+ content_type='text/html')
1378+ with patch_urlopen as mock_urlopen:
1379+ contents = netutils.urlread('http://example.com/path/')
1380+ self.assertEqual('URL contents: ', contents)
1381+ self.assertIsInstance(contents, unicode)
1382+ mock_urlopen.assert_called_once_with('http://example.com/path/')
1383+
1384+ def test_errors(self):
1385+ # An IOError is raised if an error occurs connecting to the API.
1386+ errors = {
1387+ 'httplib HTTPException': httplib.HTTPException,
1388+ 'socket error': socket.error,
1389+ 'urllib2 URLError': urllib2.URLError,
1390+ }
1391+ for message, exception_class in errors.items():
1392+ exception = exception_class(message)
1393+ with self.patch_urlopen(error=exception) as mock_urlopen:
1394+ with self.assertRaises(IOError) as context_manager:
1395+ netutils.urlread('http://example.com/path/')
1396+ mock_urlopen.assert_called_once_with('http://example.com/path/')
1397+ self.assertEqual(message, bytes(context_manager.exception))
1398
1399=== modified file 'quickstart/tests/test_utils.py'
1400--- quickstart/tests/test_utils.py 2014-11-10 09:08:44 +0000
1401+++ quickstart/tests/test_utils.py 2014-11-12 16:41:04 +0000
1402@@ -19,14 +19,11 @@
1403 from __future__ import unicode_literals
1404
1405 import datetime
1406-import httplib
1407 import json
1408 import os
1409 import shutil
1410-import socket
1411 import tempfile
1412 import unittest
1413-import urllib2
1414
1415 import mock
1416 import yaml
1417@@ -156,24 +153,6 @@
1418 utils.call('echo', 'we are the borg!')
1419
1420
1421-class TestCheckResolvable(unittest.TestCase):
1422-
1423- def test_resolvable(self):
1424- # None is returned if the hostname can be resolved.
1425- expected_log = 'example.com resolved to 1.2.3.4'
1426- with helpers.assert_logs([expected_log], level='debug'):
1427- with mock.patch('socket.gethostbyname', return_value='1.2.3.4'):
1428- error = utils.check_resolvable('example.com')
1429- self.assertIsNone(error)
1430-
1431- def test_not_resolvable(self):
1432- # An error message is returned if the hostname cannot be resolved.
1433- exception = socket.gaierror('bad wolf')
1434- with mock.patch('socket.gethostbyname', side_effect=exception):
1435- error = utils.check_resolvable('example.com')
1436- self.assertEqual('bad wolf', error)
1437-
1438-
1439 @mock.patch('__builtin__.print', mock.Mock())
1440 class TestParseGuiCharmUrl(unittest.TestCase):
1441
1442@@ -323,38 +302,6 @@
1443 utils.convert_bundle_url(url)
1444
1445
1446-class TestGetCharmUrl(helpers.UrlReadTestsMixin, unittest.TestCase):
1447-
1448- def test_charm_url(self):
1449- # The Juju GUI charm URL is correctly returned.
1450- contents = json.dumps({'charm': {'url': 'cs:trusty/juju-gui-42'}})
1451- with self.patch_urlread(contents=contents) as mock_urlread:
1452- charm_url = utils.get_charm_url('trusty')
1453- self.assertEqual('cs:trusty/juju-gui-42', charm_url)
1454- mock_urlread.assert_called_once_with(
1455- 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui')
1456-
1457- def test_io_error(self):
1458- # IOErrors are properly propagated.
1459- with self.patch_urlread(error=True) as mock_urlread:
1460- with self.assertRaises(IOError) as context_manager:
1461- utils.get_charm_url('precise')
1462- mock_urlread.assert_called_once_with(
1463- 'http://manage.jujucharms.com/api/3/charm/precise/juju-gui')
1464- self.assertEqual('bad wolf', bytes(context_manager.exception))
1465-
1466- def test_value_error(self):
1467- # A ValueError is raised if the API response is not valid.
1468- contents = json.dumps({'charm': {}})
1469- with self.patch_urlread(contents=contents) as mock_urlread:
1470- with self.assertRaises(ValueError) as context_manager:
1471- utils.get_charm_url('trusty')
1472- mock_urlread.assert_called_once_with(
1473- 'http://manage.jujucharms.com/api/3/charm/trusty/juju-gui')
1474- self.assertEqual(
1475- 'unable to find the charm URL', bytes(context_manager.exception))
1476-
1477-
1478 class TestGetQuickstartBanner(unittest.TestCase):
1479
1480 def patch_datetime(self):
1481@@ -704,72 +651,6 @@
1482 self.assertEqual(list.append.__doc__, self.func.__doc__)
1483
1484
1485-class TestUrlread(unittest.TestCase):
1486-
1487- def patch_urlopen(self, contents=None, error=None, content_type=None):
1488- """Patch the urllib2.urlopen function.
1489-
1490- If contents is not None, the read() method of the returned mock object
1491- returns the given contents.
1492- If content_type is provided, the response includes the content type.
1493- If an error is provided, the call raises the error.
1494- """
1495- mock_urlopen = mock.MagicMock()
1496- if contents is not None:
1497- mock_urlopen().read.return_value = contents
1498- if content_type is not None:
1499- mock_urlopen().headers = {'content-type': content_type}
1500- if error is not None:
1501- mock_urlopen.side_effect = error
1502- mock_urlopen.reset_mock()
1503- return mock.patch('urllib2.urlopen', mock_urlopen)
1504-
1505- def test_contents(self):
1506- # The URL contents are correctly returned.
1507- with self.patch_urlopen(contents=b'URL contents') as mock_urlopen:
1508- contents = utils.urlread('http://example.com/path/')
1509- self.assertEqual('URL contents', contents)
1510- self.assertIsInstance(contents, unicode)
1511- mock_urlopen.assert_called_once_with('http://example.com/path/')
1512-
1513- def test_content_type(self):
1514- # The URL contents are decoded using the site charset.
1515- patch_urlopen = self.patch_urlopen(
1516- contents=b'URL contents: \xf8', # This is not a UTF-8 byte string.
1517- content_type='text/html; charset=ISO-8859-1')
1518- with patch_urlopen as mock_urlopen:
1519- contents = utils.urlread('http://example.com/path/')
1520- self.assertEqual('URL contents: \xf8', contents)
1521- self.assertIsInstance(contents, unicode)
1522- mock_urlopen.assert_called_once_with('http://example.com/path/')
1523-
1524- def test_no_content_type(self):
1525- # The URL contents are decoded with UTF-8 by default.
1526- patch_urlopen = self.patch_urlopen(
1527- contents=b'URL contents: \xf8', # This is not a UTF-8 byte string.
1528- content_type='text/html')
1529- with patch_urlopen as mock_urlopen:
1530- contents = utils.urlread('http://example.com/path/')
1531- self.assertEqual('URL contents: ', contents)
1532- self.assertIsInstance(contents, unicode)
1533- mock_urlopen.assert_called_once_with('http://example.com/path/')
1534-
1535- def test_errors(self):
1536- # An IOError is raised if an error occurs connecting to the API.
1537- errors = {
1538- 'httplib HTTPException': httplib.HTTPException,
1539- 'socket error': socket.error,
1540- 'urllib2 URLError': urllib2.URLError,
1541- }
1542- for message, exception_class in errors.items():
1543- exception = exception_class(message)
1544- with self.patch_urlopen(error=exception) as mock_urlopen:
1545- with self.assertRaises(IOError) as context_manager:
1546- utils.urlread('http://example.com/path/')
1547- mock_urlopen.assert_called_once_with('http://example.com/path/')
1548- self.assertEqual(message, bytes(context_manager.exception))
1549-
1550-
1551 class TestGetJujuVersion(
1552 helpers.CallTestsMixin, helpers.ValueErrorTestsMixin,
1553 unittest.TestCase):
1554
1555=== modified file 'quickstart/utils.py'
1556--- quickstart/utils.py 2014-11-10 09:08:44 +0000
1557+++ quickstart/utils.py 2014-11-12 16:41:04 +0000
1558@@ -25,15 +25,11 @@
1559 import datetime
1560 import errno
1561 import functools
1562-import httplib
1563-import json
1564 import logging
1565 import os
1566 import pipes
1567 import re
1568-import socket
1569 import subprocess
1570-import urllib2
1571
1572 import quickstart
1573 from quickstart import (
1574@@ -105,20 +101,6 @@
1575 return retcode, output.decode('utf-8'), error.decode('utf-8')
1576
1577
1578-def check_resolvable(hostname):
1579- """Check that the hostname can be resolved to a numeric IP address.
1580-
1581- Return an error message if the address cannot be resolved.
1582- """
1583- try:
1584- address = socket.gethostbyname(hostname)
1585- except socket.error as err:
1586- return bytes(err).decode('utf-8')
1587- logging.debug('{} resolved to {}'.format(
1588- hostname, address.decode('utf-8')))
1589- return None
1590-
1591-
1592 def parse_gui_charm_url(charm_url):
1593 """Parse the given charm URL.
1594
1595@@ -164,21 +146,6 @@
1596 bundle_id)
1597
1598
1599-def get_charm_url(series):
1600- """Return the charm URL of the latest Juju GUI charm revision.
1601-
1602- Raise an IOError if any problems occur connecting to the API endpoint.
1603- Raise a ValueError if the API returns invalid data.
1604- """
1605- url = settings.CHARMWORLD_API.format(
1606- series=series, charm=settings.JUJU_GUI_CHARM_NAME)
1607- charm_info = json.loads(urlread(url))
1608- charm_url = charm_info.get('charm', {}).get('url')
1609- if charm_url is None:
1610- raise ValueError(b'unable to find the charm URL')
1611- return charm_url
1612-
1613-
1614 def get_quickstart_banner():
1615 """Return a quickstart banner suitable for being included in files.
1616
1617@@ -390,24 +357,3 @@
1618 return function(*args, **kwargs)
1619 decorated.called = False
1620 return decorated
1621-
1622-
1623-def urlread(url):
1624- """Open the given URL and return the page contents.
1625-
1626- Raise an IOError if any problems occur.
1627- """
1628- try:
1629- response = urllib2.urlopen(url)
1630- except urllib2.URLError as err:
1631- raise IOError(err.reason)
1632- except (httplib.HTTPException, socket.error, urllib2.HTTPError) as err:
1633- raise IOError(bytes(err))
1634- contents = response.read()
1635- content_type = response.headers['content-type']
1636- charset = 'utf-8'
1637- if 'charset=' in content_type:
1638- sent_charset = content_type.split('charset=')[-1].strip()
1639- if sent_charset:
1640- charset = sent_charset
1641- return contents.decode(charset, 'ignore')
1642
1643=== modified file 'quickstart/watchers.py'
1644--- quickstart/watchers.py 2014-11-10 10:28:19 +0000
1645+++ quickstart/watchers.py 2014-11-12 16:41:04 +0000
1646@@ -23,7 +23,7 @@
1647
1648 import logging
1649
1650-from quickstart import utils
1651+from quickstart import netutils
1652
1653
1654 IPV4_ADDRESS = 'ipv4'
1655@@ -150,7 +150,7 @@
1656 if not address:
1657 addresses = data.get('Addresses', [])
1658 public_address = retrieve_public_adddress(
1659- addresses, utils.check_resolvable)
1660+ addresses, netutils.check_resolvable)
1661 if public_address is not None:
1662 address = public_address
1663 print('unit placed on {}'.format(address))

Subscribers

People subscribed via source and target branches