Merge lp:~frankban/juju-quickstart/code-reorg into lp:juju-quickstart
- code-reorg
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 104 |
Proposed branch: | lp:~frankban/juju-quickstart/code-reorg |
Merge into: | lp:juju-quickstart |
Diff against target: |
849 lines (+370/-222) 12 files modified
HACKING.rst (+55/-1) quickstart/app.py (+10/-21) quickstart/manage.py (+3/-15) quickstart/models/envs.py (+1/-30) quickstart/models/jenv.py (+71/-0) quickstart/serializers.py (+20/-0) quickstart/tests/helpers.py (+36/-0) quickstart/tests/models/test_envs.py (+0/-76) quickstart/tests/models/test_jenv.py (+110/-0) quickstart/tests/test_app.py (+16/-18) quickstart/tests/test_manage.py (+7/-61) quickstart/tests/test_serializers.py (+41/-0) |
To merge this branch: | bzr merge lp:~frankban/juju-quickstart/code-reorg |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju GUI Hackers | Pending | ||
Review via email: mp+241416@code.launchpad.net |
Commit message
Description of the change
Code reorganization + docs.
Reorganize the jenv related code in preparation of the
bootstrap strategy change.
Separate jenv handling from the environments.yaml file
management in models.
As a consequence of only supporting juju >= 1.18 we
can now safely always retrieve the admin-secret from
the jenv file.
Also updated the HACKING docs to include a brief
description of the project structure.
Tests: `make check`.
QA: use quickstart as usual.
Francesco Banconi (frankban) wrote : | # |
Brad Crittenden (bac) wrote : | # |
LGTM. Will QA now.
https:/
File HACKING.rst (right):
https:/
HACKING.rst:78:
A very nice introduction to the code layout. Thanks for including it.
https:/
File quickstart/
https:/
quickstart/
Why did you chose to use *args here instead of a tuple of key names?
https:/
File quickstart/
https:/
quickstart/
JUJU_HOME is set to the ancestor
typo: contextx
Richard Harding (rharding) wrote : | # |
Thanks for the update. I just have one forward looking question that
doesn't directly effect this branch but I'm curious about while looking
at this branch.
LGTM, no qa
https:/
File HACKING.rst (right):
https:/
HACKING.rst:128: the code: go have a look!
nice ty
https:/
File quickstart/app.py (right):
https:/
quickstart/
do you know if we can use the password field in the jenv for this? Next
week we hope to start updating the gui for the new username/password
work in juju 1.21 and I know we'll need to update quickstart to pull the
username/password from the jenv file.
https:/
File quickstart/
https:/
quickstart/
section=
was this unused?
Brad Crittenden (bac) wrote : | # |
Had some QA oddities but it looks OK.
Francesco Banconi (frankban) wrote : | # |
Thanks for the reviews!
https:/
File quickstart/app.py (right):
https:/
quickstart/
On 2014/11/11 15:52:36, rharding wrote:
> do you know if we can use the password field in the jenv for this?
Next week we
> hope to start updating the gui for the new username/password work in
juju 1.21
> and I know we'll need to update quickstart to pull the
username/password from
> the jenv file.
With this change it should be easy to grab what we want from the jenv
with jenv.get_value(). We will want to use some fallback logic to
support backward compatibility.
If the user is != user-admin, we will need to update the one-show
automatic authentication with token as well in the guiserver and maybe
in the GUI itself.
https:/
File quickstart/
https:/
quickstart/
section=
On 2014/11/11 15:52:36, rharding wrote:
> was this unused?
It was used, it is no longer required now.
https:/
File quickstart/
https:/
quickstart/
On 2014/11/11 15:37:55, bac wrote:
> Why did you chose to use *args here instead of a tuple of key names?
No real reason actually, I started with just one single key as the
argument and then, after deciding to allow nested keys, it just felt
natural.
https:/
File quickstart/
https:/
quickstart/
JUJU_HOME is set to the ancestor
On 2014/11/11 15:37:55, bac wrote:
> typo: contextx
Done.
Francesco Banconi (frankban) wrote : | # |
*** Submitted:
Code reorganization + docs.
Reorganize the jenv related code in preparation of the
bootstrap strategy change.
Separate jenv handling from the environments.yaml file
management in models.
As a consequence of only supporting juju >= 1.18 we
can now safely always retrieve the admin-secret from
the jenv file.
Also updated the HACKING docs to include a brief
description of the project structure.
Tests: `make check`.
QA: use quickstart as usual.
R=bac, rharding
CC=
https:/
Preview Diff
1 | === modified file 'HACKING.rst' |
2 | --- HACKING.rst 2014-09-01 14:49:47 +0000 |
3 | +++ HACKING.rst 2014-11-11 14:26:05 +0000 |
4 | @@ -73,6 +73,60 @@ |
5 | |
6 | $ .venv/bin/python juju-quickstart --help |
7 | |
8 | +Project structure |
9 | +~~~~~~~~~~~~~~~~~ |
10 | + |
11 | +Juju Quickstart is a Python project. Source files can be found in the |
12 | +``quickstart`` Python package included in this distribution. |
13 | + |
14 | +A brief description of the project structure follows, including the goals of |
15 | +each module in the ``quickstart`` package. |
16 | + |
17 | +* ``manage.py`` includes the main entry points for the ``juju-quickstart`` |
18 | + command. Specifically: |
19 | + * ``manage.setup`` is used to set up the command line parser, the logging |
20 | + infrastructure and the interactive session (if required); |
21 | + * ``manage.run`` executes the application with the given options. |
22 | + |
23 | +* ``app.py`` defines the main application functions, like bootstrapping an |
24 | + environment, deploying the Juju GUI or watching the deployment progress. |
25 | + The ``app.ProgramExit`` error can only be raised by functions in this module, |
26 | + and it causes the immediate exit (with an error) from the application. |
27 | + |
28 | +The ``manage.py`` and ``app.py`` modules must be considered ``juju-quickstart`` |
29 | +execution specific: for this reason those modules are unlikely to be used as |
30 | +libraries. All the other packages/modules in the application (except for views, |
31 | +see below), should only include library-like code that can be reused in |
32 | +different places by either Juju Quickstart or external programs. |
33 | + |
34 | +* ``setting.py`` includes the application settings. Those must be considered as |
35 | + constant values to be reused throughout the application. The settings module |
36 | + should not import other ``quickstart`` modules. |
37 | + |
38 | +* ``utils.py`` defines general purpose helper functions: when writing such |
39 | + utility objects it is likely that this is the right place where they should |
40 | + live. Separate modules are created when a set of utilities are related each |
41 | + other: this is the case, for instance, of ``serializers.py`` (YAML/JSON |
42 | + serialization utilities), ``ssh.py`` (SSH protocol related functions), |
43 | + ``platform_support.py`` etc. |
44 | + |
45 | +* ``juju.py`` defines the WebSocket API client used by Juju Quickstart to |
46 | + connect to the Juju API server. |
47 | + |
48 | +* the ``models`` package includes a module for each application model. Models |
49 | + are abstractions representing the data and information managed by Juju |
50 | + Quickstart, e.g. environment files, jenv files or charms. In the |
51 | + implementation of models, an effort has been made to use simple data |
52 | + structures in order to represent entities/objects, and composable functions |
53 | + to manipulate this information. |
54 | + |
55 | +* the ``cli`` package contains the command line interactive session handling. |
56 | + Juju Quickstart uses Urwid to implement an ncurses-like interactive session: |
57 | + Urwid code must only live in the ``cli`` package. |
58 | + |
59 | +That said, module docstrings often describe the goals and usage of each part of |
60 | +the code: go have a look! |
61 | + |
62 | Pre-release QA |
63 | ~~~~~~~~~~~~~~ |
64 | |
65 | @@ -98,7 +152,7 @@ |
66 | |
67 | * Verify an environment that has already been bootstrapped is recogized and |
68 | the GUI is deployed. This test also shows that a remote bundle is properly |
69 | - deployed.:: |
70 | + deployed:: |
71 | |
72 | juju bootstrap -e local |
73 | juju quickstart -e local bundle:mediawiki/single |
74 | |
75 | === modified file 'quickstart/app.py' |
76 | --- quickstart/app.py 2014-11-10 11:39:40 +0000 |
77 | +++ quickstart/app.py 2014-11-11 14:26:05 +0000 |
78 | @@ -23,7 +23,6 @@ |
79 | |
80 | import json |
81 | import logging |
82 | -import os |
83 | import sys |
84 | import time |
85 | |
86 | @@ -37,7 +36,7 @@ |
87 | utils, |
88 | watchers, |
89 | ) |
90 | -from quickstart.models import envs |
91 | +from quickstart.models import jenv |
92 | |
93 | |
94 | class ProgramExit(Exception): |
95 | @@ -258,27 +257,17 @@ |
96 | raise ProgramExit('the state server is not ready:\n{}'.format(details)) |
97 | |
98 | |
99 | -def get_value_from_jenv(env_name, juju_home, key): |
100 | - """Read the key from the generated environment file. |
101 | - |
102 | - At bootstrap, juju (v1.16 and later) writes a generated file |
103 | - located in JUJU_HOME. Return the value for the key. |
104 | - |
105 | - Raise a ValueError if: |
106 | - - the environment file is not found; |
107 | - - the environment file contents are not parsable by YAML; |
108 | - - the YAML contents are not properly structured; |
109 | - - the key is not found. |
110 | +def get_admin_secret(env_name): |
111 | + """Return the Juju admin secret for the given environment name. |
112 | + |
113 | + Parse the jenv file to retrieve the admin secret. |
114 | + Raise a ProgramExit if the admin secret cannot be retrieved. |
115 | """ |
116 | - filename = '{}.jenv'.format(env_name) |
117 | - juju_env_file = os.path.expanduser( |
118 | - os.path.join(juju_home, 'environments', filename)) |
119 | - jenv_db = envs.load_generated(juju_env_file) |
120 | try: |
121 | - return jenv_db[key] |
122 | - except KeyError: |
123 | - msg = '{} not found in {}'.format(key, juju_env_file) |
124 | - raise ValueError(msg.encode('utf-8')) |
125 | + return jenv.get_value(env_name, 'bootstrap-config', 'admin-secret') |
126 | + except ValueError as err: |
127 | + msg = b'cannot retrieve environment admin-secret: {}'.format(err) |
128 | + raise ProgramExit(msg) |
129 | |
130 | |
131 | def get_api_url(env_name, juju_command): |
132 | |
133 | === modified file 'quickstart/manage.py' |
134 | --- quickstart/manage.py 2014-11-10 11:31:02 +0000 |
135 | +++ quickstart/manage.py 2014-11-11 14:26:05 +0000 |
136 | @@ -293,7 +293,6 @@ |
137 | 'this host platform does not support local environments' |
138 | ) |
139 | # Update the options namespace with the new values. |
140 | - options.admin_secret = env_data.get('admin-secret') |
141 | options.env_file = env_file |
142 | options.env_name = env_data['name'] |
143 | options.default_series = env_data.get('default-series') |
144 | @@ -343,10 +342,9 @@ |
145 | """Set up the application options and logger. |
146 | |
147 | Return the options as a namespace containing the following attributes: |
148 | - - admin_secret: the password to use to access the Juju API or None if |
149 | - no admin-secret is present in the $JUJU_HOME/environment.yaml file; |
150 | - bundle: the optional bundle (path or URL) to be deployed; |
151 | - charm_url: the Juju GUI charm URL or None if not specified; |
152 | + - constraints: the environment constrains or None if not set; |
153 | - debug: whether debug mode is activated; |
154 | - distro_only: install Juju only using the distribution packages; |
155 | - env_file: the absolute path of the Juju environments.yaml file; |
156 | @@ -357,8 +355,7 @@ |
157 | - platform: The host platform; |
158 | - upload_tools: whether to upload local version of tools; |
159 | - upload_series: the comma-separated series list for which tools will |
160 | - be uploaded, or None if not set; |
161 | - - constraints: the environment constrains or None if not set. |
162 | + be uploaded, or None if not set. |
163 | |
164 | The following attributes will also be included in the namespace if a bundle |
165 | deployment is requested: |
166 | @@ -522,16 +519,7 @@ |
167 | constraints=options.constraints) |
168 | |
169 | # Retrieve the admin-secret for the current environment. |
170 | - try: |
171 | - admin_secret = app.get_value_from_jenv( |
172 | - options.env_name, settings.JUJU_HOME, 'admin-secret') |
173 | - except ValueError as err: |
174 | - admin_secret = options.admin_secret |
175 | - if admin_secret is None: |
176 | - # The admin-secret cannot be found in the jenv file and is not |
177 | - # explicitly specified in the environments.yaml file. |
178 | - msg = b'{} or {}'.format(err, options.env_file.encode('utf-8')) |
179 | - raise app.ProgramExit(msg) |
180 | + admin_secret = app.get_admin_secret(options.env_name) |
181 | |
182 | print('retrieving the Juju API address') |
183 | api_url = app.get_api_url(options.env_name, juju_command) |
184 | |
185 | === modified file 'quickstart/models/envs.py' |
186 | --- quickstart/models/envs.py 2014-10-09 13:14:30 +0000 |
187 | +++ quickstart/models/envs.py 2014-11-11 14:26:05 +0000 |
188 | @@ -134,23 +134,6 @@ |
189 | return {'environments': {}} |
190 | |
191 | |
192 | -def _load_file(env_file): |
193 | - """Load the given file and return the YAML-parsed contents.""" |
194 | - # Load the Juju environments file. |
195 | - try: |
196 | - environments_file = open(env_file.encode('utf-8')) |
197 | - except IOError as err: |
198 | - msg = b'unable to open environments file: {}'.format(err) |
199 | - raise ValueError(msg) |
200 | - # Parse the Juju environments file. |
201 | - try: |
202 | - contents = serializers.yaml_load(environments_file) |
203 | - except Exception as err: |
204 | - msg = b'unable to parse environments file {}: {}' |
205 | - raise ValueError(msg.format(env_file.encode('utf-8'), err)) |
206 | - return contents |
207 | - |
208 | - |
209 | def load(env_file): |
210 | """Load and parse the provided Juju environments.yaml file. |
211 | |
212 | @@ -174,7 +157,7 @@ |
213 | - the environment file contents are not parsable by YAML; |
214 | - the YAML contents are not properly structured. |
215 | """ |
216 | - contents = _load_file(env_file) |
217 | + contents = serializers.yaml_load_from_path(env_file) |
218 | if contents is None: |
219 | return create_empty_env_db() |
220 | # Retrieve the environment list. |
221 | @@ -199,18 +182,6 @@ |
222 | return env_db |
223 | |
224 | |
225 | -def load_generated(env_file, section='bootstrap-config'): |
226 | - """Given the path to a YAML file, load the file and return the section.""" |
227 | - contents = _load_file(env_file) |
228 | - try: |
229 | - section_contents = contents[section] |
230 | - except (KeyError, TypeError): |
231 | - msg = 'invalid YAML contents in {}: {}'.format(env_file, contents) |
232 | - raise ValueError(msg.encode('utf-8')) |
233 | - |
234 | - return section_contents |
235 | - |
236 | - |
237 | def save(env_file, env_db, backup_function=None): |
238 | """Save the given env_db to the provided environments.yaml file. |
239 | |
240 | |
241 | === added file 'quickstart/models/jenv.py' |
242 | --- quickstart/models/jenv.py 1970-01-01 00:00:00 +0000 |
243 | +++ quickstart/models/jenv.py 2014-11-11 14:26:05 +0000 |
244 | @@ -0,0 +1,71 @@ |
245 | +# This file is part of the Juju Quickstart Plugin, which lets users set up a |
246 | +# Juju environment in very few steps (https://launchpad.net/juju-quickstart). |
247 | +# Copyright (C) 2014 Canonical Ltd. |
248 | +# |
249 | +# This program is free software: you can redistribute it and/or modify it under |
250 | +# the terms of the GNU Affero General Public License version 3, as published by |
251 | +# the Free Software Foundation. |
252 | +# |
253 | +# This program is distributed in the hope that it will be useful, but WITHOUT |
254 | +# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY, |
255 | +# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE. See the GNU |
256 | +# Affero General Public License for more details. |
257 | +# |
258 | +# You should have received a copy of the GNU Affero General Public License |
259 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
260 | + |
261 | +"""Juju Quickstart jenv generated files handling. |
262 | + |
263 | +At bootstrap, Juju writes a generated file (jenv) located in JUJU_HOME. |
264 | +This module includes functions to load and parse those jenv file contents. |
265 | +""" |
266 | + |
267 | +from __future__ import unicode_literals |
268 | + |
269 | +import os |
270 | + |
271 | +from quickstart import ( |
272 | + serializers, |
273 | + settings, |
274 | +) |
275 | + |
276 | + |
277 | +def exists(env_name): |
278 | + """Report whether the jenv generated file exists for the given env_name.""" |
279 | + jenv_path = _get_jenv_path(env_name) |
280 | + return os.path.isfile(jenv_path) |
281 | + |
282 | + |
283 | +def get_value(env_name, *args): |
284 | + """Read a value from the Juju generated environment file (jenv). |
285 | + |
286 | + Return the value corresponding to the section specified in args. |
287 | + For instance, calling get_value('ec2', 'bootstrap-config', 'admin-secret') |
288 | + returns the value associated with the "admin-secret" key included on the |
289 | + "bootstrap-config" section of the jenv file. |
290 | + |
291 | + Raise a ValueError if: |
292 | + - the environment file is not found; |
293 | + - the environment file contents are not parsable by YAML; |
294 | + - the YAML contents are not properly structured; |
295 | + - one the keys in args is not found. |
296 | + """ |
297 | + jenv_path = _get_jenv_path(env_name) |
298 | + data = serializers.yaml_load_from_path(jenv_path) |
299 | + section = 'root' |
300 | + for key in args: |
301 | + try: |
302 | + data = data[key] |
303 | + except (KeyError, TypeError): |
304 | + msg = ('invalid YAML contents in {}: ''{} key not found in the {} ' |
305 | + 'section'.format(jenv_path, key, section)) |
306 | + raise ValueError(msg.encode('utf-8')) |
307 | + section = key |
308 | + return data |
309 | + |
310 | + |
311 | +def _get_jenv_path(env_name): |
312 | + """Return the path to the generated jenv file for the given env_name.""" |
313 | + filename = '{}.jenv'.format(env_name) |
314 | + path = os.path.join(settings.JUJU_HOME, 'environments', filename) |
315 | + return os.path.expanduser(path) |
316 | |
317 | === modified file 'quickstart/serializers.py' |
318 | --- quickstart/serializers.py 2013-12-06 17:17:19 +0000 |
319 | +++ quickstart/serializers.py 2014-11-11 14:26:05 +0000 |
320 | @@ -45,3 +45,23 @@ |
321 | Always serialize collections in the block style. |
322 | """ |
323 | return yaml.safe_dump(data, stream, default_flow_style=False) |
324 | + |
325 | + |
326 | +def yaml_load_from_path(path): |
327 | + """Load the given file path and return the YAML-parsed contents. |
328 | + |
329 | + Raise a ValueError if the file cannot be opened or parsed. |
330 | + """ |
331 | + filename = path.encode('utf-8') |
332 | + try: |
333 | + yaml_file = open(filename) |
334 | + except IOError as err: |
335 | + msg = b'unable to open file {}: {}'.format(filename, err) |
336 | + raise ValueError(msg) |
337 | + # Parse the Juju environments file. |
338 | + try: |
339 | + contents = yaml_load(yaml_file) |
340 | + except Exception as err: |
341 | + msg = b'unable to parse file {}: {}'.format(filename, err) |
342 | + raise ValueError(msg) |
343 | + return contents |
344 | |
345 | === modified file 'quickstart/tests/helpers.py' |
346 | --- quickstart/tests/helpers.py 2014-11-10 09:08:44 +0000 |
347 | +++ quickstart/tests/helpers.py 2014-11-11 14:26:05 +0000 |
348 | @@ -136,6 +136,42 @@ |
349 | return env_file.name |
350 | |
351 | |
352 | +class JenvFileTestsMixin(object): |
353 | + """Shared methods for testing Juju generated environment files (jenv).""" |
354 | + |
355 | + jenv_data = { |
356 | + 'user': 'admin', |
357 | + 'state-servers': ['localhost:17070', '10.0.3.1:17070'], |
358 | + 'bootstrap-config': { |
359 | + 'admin-secret': 'Secret!', |
360 | + 'api-port': 17070, |
361 | + }, |
362 | + 'life': {'universe': {'everything': 42}}, |
363 | + } |
364 | + |
365 | + @contextmanager |
366 | + def make_jenv(self, env_name, contents): |
367 | + """Create a temporary jenv file with the given env_name and contents. |
368 | + |
369 | + In the contentx manager block, the JUJU_HOME is set to the ancestor |
370 | + of the generated temporary file. |
371 | + |
372 | + Return the jenv file path. |
373 | + """ |
374 | + # Create a temporary JUJU_HOME. |
375 | + playground = tempfile.mkdtemp() |
376 | + self.addCleanup(shutil.rmtree, playground) |
377 | + environments_dir = os.path.join(playground, 'environments') |
378 | + os.mkdir(environments_dir) |
379 | + # Create the jenv file inside the temporary JUJU_HOME. |
380 | + jenv_path = os.path.join(environments_dir, '{}.jenv'.format(env_name)) |
381 | + with open(jenv_path, 'w') as jenv_file: |
382 | + jenv_file.write(contents) |
383 | + # Patch the JUJU_HOME and return the jenv file path. |
384 | + with mock.patch('quickstart.settings.JUJU_HOME', playground): |
385 | + yield jenv_path |
386 | + |
387 | + |
388 | def make_env_db(default=None, exclude_invalid=False): |
389 | """Create and return an env_db. |
390 | |
391 | |
392 | === modified file 'quickstart/tests/models/test_envs.py' |
393 | --- quickstart/tests/models/test_envs.py 2014-10-09 15:29:46 +0000 |
394 | +++ quickstart/tests/models/test_envs.py 2014-11-11 14:26:05 +0000 |
395 | @@ -99,28 +99,6 @@ |
396 | self.assertEqual({'environments': {}}, env_db) |
397 | |
398 | |
399 | -class TestLoadFile( |
400 | - helpers.EnvFileTestsMixin, helpers.ValueErrorTestsMixin, |
401 | - unittest.TestCase): |
402 | - |
403 | - def test_no_file(self): |
404 | - # A ValueError is raised if the environments file is not found. |
405 | - expected = ( |
406 | - "unable to open environments file: " |
407 | - "[Errno 2] No such file or directory: '/no/such/file.yaml'" |
408 | - ) |
409 | - with self.assert_value_error(expected): |
410 | - envs._load_file('/no/such/file.yaml') |
411 | - |
412 | - def test_invalid_yaml(self): |
413 | - # A ValueError is raised if the environments file is not a valid YAML. |
414 | - env_file = self.make_env_file(':') |
415 | - with self.assertRaises(ValueError) as context_manager: |
416 | - envs._load_file(env_file) |
417 | - expected = 'unable to parse environments file {}'.format(env_file) |
418 | - self.assertIn(expected, bytes(context_manager.exception)) |
419 | - |
420 | - |
421 | class TestLoad( |
422 | helpers.EnvFileTestsMixin, helpers.ValueErrorTestsMixin, |
423 | unittest.TestCase): |
424 | @@ -205,60 +183,6 @@ |
425 | self.assertEqual(expected, env_db) |
426 | |
427 | |
428 | -class TestLoadGenerated( |
429 | - helpers.EnvFileTestsMixin, helpers.ValueErrorTestsMixin, |
430 | - unittest.TestCase): |
431 | - |
432 | - def test_empty_file(self): |
433 | - # A ValueError is raised if the environments file is empty. |
434 | - env_file = self.make_env_file('') |
435 | - expected = 'invalid YAML contents in {}: None'.format(env_file) |
436 | - with self.assert_value_error(expected): |
437 | - envs.load_generated(env_file) |
438 | - |
439 | - def test_invalid_yaml_contents(self): |
440 | - # A ValueError is raised if the environments file is not well formed. |
441 | - env_file = self.make_env_file('a-string') |
442 | - expected = 'invalid YAML contents in {}: a-string'.format(env_file) |
443 | - with self.assert_value_error(expected): |
444 | - envs.load_generated(env_file) |
445 | - |
446 | - def test_section_not_found(self): |
447 | - expected = { |
448 | - 'shoehorn-config': { |
449 | - 'admin-secret': 'Secret!', |
450 | - 'type': 'ec2'}, |
451 | - } |
452 | - yaml_contents = expected.copy() |
453 | - env_file = self.make_env_file(yaml.safe_dump(yaml_contents)) |
454 | - expected = 'invalid YAML contents in {}: {}'.format( |
455 | - env_file, yaml_contents) |
456 | - with self.assert_value_error(expected): |
457 | - envs.load_generated(env_file) |
458 | - |
459 | - def test_successful_default_section(self): |
460 | - expected = { |
461 | - 'bootstrap-config': { |
462 | - 'admin-secret': 'Secret!', |
463 | - 'type': 'ec2'}, |
464 | - } |
465 | - yaml_contents = expected.copy() |
466 | - env_file = self.make_env_file(yaml.safe_dump(yaml_contents)) |
467 | - env_db = envs.load_generated(env_file) |
468 | - self.assertEqual(expected['bootstrap-config'], env_db) |
469 | - |
470 | - def test_successful_specified_section(self): |
471 | - expected = { |
472 | - 'my-config': { |
473 | - 'admin-secret': 'Secret!', |
474 | - 'type': 'ec2'}, |
475 | - } |
476 | - yaml_contents = expected.copy() |
477 | - env_file = self.make_env_file(yaml.safe_dump(yaml_contents)) |
478 | - env_db = envs.load_generated(env_file, section='my-config') |
479 | - self.assertEqual(expected['my-config'], env_db) |
480 | - |
481 | - |
482 | class TestSave(helpers.EnvFileTestsMixin, unittest.TestCase): |
483 | |
484 | def setUp(self): |
485 | |
486 | === added file 'quickstart/tests/models/test_jenv.py' |
487 | --- quickstart/tests/models/test_jenv.py 1970-01-01 00:00:00 +0000 |
488 | +++ quickstart/tests/models/test_jenv.py 2014-11-11 14:26:05 +0000 |
489 | @@ -0,0 +1,110 @@ |
490 | +# This file is part of the Juju Quickstart Plugin, which lets users set up a |
491 | +# Juju environment in very few steps (https://launchpad.net/juju-quickstart). |
492 | +# Copyright (C) 2014 Canonical Ltd. |
493 | +# |
494 | +# This program is free software: you can redistribute it and/or modify it under |
495 | +# the terms of the GNU Affero General Public License version 3, as published by |
496 | +# the Free Software Foundation. |
497 | +# |
498 | +# This program is distributed in the hope that it will be useful, but WITHOUT |
499 | +# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY, |
500 | +# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE. See the GNU |
501 | +# Affero General Public License for more details. |
502 | +# |
503 | +# You should have received a copy of the GNU Affero General Public License |
504 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
505 | + |
506 | +"""Tests for the Juju Quickstart jenv generated files handling.""" |
507 | + |
508 | +from __future__ import unicode_literals |
509 | + |
510 | +import unittest |
511 | + |
512 | +import yaml |
513 | + |
514 | +from quickstart.models import jenv |
515 | +from quickstart.tests import helpers |
516 | + |
517 | + |
518 | +class TestExists(helpers.JenvFileTestsMixin, unittest.TestCase): |
519 | + |
520 | + def test_found(self): |
521 | + # True is returned if the jenv file exists. |
522 | + with self.make_jenv('ec2', ''): |
523 | + exists = jenv.exists('ec2') |
524 | + self.assertTrue(exists) |
525 | + |
526 | + def test_not_found(self): |
527 | + # False is returned if the jenv file does not exist. |
528 | + with self.make_jenv('ec2', ''): |
529 | + exists = jenv.exists('local') |
530 | + self.assertFalse(exists) |
531 | + |
532 | + |
533 | +class TestGetValue( |
534 | + helpers.JenvFileTestsMixin, helpers.ValueErrorTestsMixin, |
535 | + unittest.TestCase): |
536 | + |
537 | + def test_whole_content(self): |
538 | + # The function correctly returns the whole jenv content. |
539 | + with self.make_jenv('local', yaml.safe_dump(self.jenv_data)): |
540 | + data = jenv.get_value('local') |
541 | + self.assertEqual(self.jenv_data, data) |
542 | + |
543 | + def test_section(self): |
544 | + # The function correctly returns a whole section. |
545 | + with self.make_jenv('ec2', yaml.safe_dump(self.jenv_data)): |
546 | + data = jenv.get_value('ec2', 'bootstrap-config') |
547 | + self.assertEqual(self.jenv_data['bootstrap-config'], data) |
548 | + |
549 | + def test_value(self): |
550 | + # The function correctly returns a value in the root node. |
551 | + with self.make_jenv('ec2', yaml.safe_dump(self.jenv_data)): |
552 | + value = jenv.get_value('ec2', 'user') |
553 | + self.assertEqual('admin', value) |
554 | + |
555 | + def test_section_value(self): |
556 | + # The function correctly returns a section value. |
557 | + with self.make_jenv('ec2', yaml.safe_dump(self.jenv_data)): |
558 | + value = jenv.get_value('ec2', 'bootstrap-config', 'admin-secret') |
559 | + self.assertEqual('Secret!', value) |
560 | + |
561 | + def test_nested_section(self): |
562 | + # The function correctly returns nested section's values. |
563 | + with self.make_jenv('hp', yaml.safe_dump(self.jenv_data)): |
564 | + value = jenv.get_value('hp', 'life', 'universe', 'everything') |
565 | + self.assertEqual(42, value) |
566 | + |
567 | + def test_file_not_found(self): |
568 | + # A ValueError is raised if the jenv file cannot be found. |
569 | + expected_error = 'unable to open file' |
570 | + with self.make_jenv('hp', yaml.safe_dump(self.jenv_data)): |
571 | + with self.assertRaises(ValueError) as context_manager: |
572 | + jenv.get_value('local') |
573 | + self.assertIn(expected_error, bytes(context_manager.exception)) |
574 | + |
575 | + def test_section_not_found(self): |
576 | + # A ValueError is raised if the specified section cannot be found. |
577 | + with self.make_jenv('local', yaml.safe_dump(self.jenv_data)) as path: |
578 | + expected_error = ( |
579 | + 'invalid YAML contents in {}: no-such key not found in the ' |
580 | + 'root section'.format(path)) |
581 | + with self.assert_value_error(expected_error): |
582 | + jenv.get_value('local', 'no-such') |
583 | + |
584 | + def test_subsection_not_found(self): |
585 | + # A ValueError is raised if the specified subsection cannot be found. |
586 | + with self.make_jenv('local', yaml.safe_dump(self.jenv_data)) as path: |
587 | + expected_error = ( |
588 | + 'invalid YAML contents in {}: series key not found in the ' |
589 | + 'bootstrap-config section'.format(path)) |
590 | + with self.assert_value_error(expected_error): |
591 | + jenv.get_value('local', 'bootstrap-config', 'series') |
592 | + |
593 | + def test_invalid_yaml_contents(self): |
594 | + # A ValueError is raised if the jenv file is not well formed. |
595 | + expected_error = 'unable to parse file' |
596 | + with self.make_jenv('ec2', ':'): |
597 | + with self.assertRaises(ValueError) as context_manager: |
598 | + jenv.get_value('ec2') |
599 | + self.assertIn(expected_error, bytes(context_manager.exception)) |
600 | |
601 | === modified file 'quickstart/tests/test_app.py' |
602 | --- quickstart/tests/test_app.py 2014-11-10 11:39:40 +0000 |
603 | +++ quickstart/tests/test_app.py 2014-11-11 14:26:05 +0000 |
604 | @@ -673,26 +673,24 @@ |
605 | ] + self.make_status_calls(2)) |
606 | |
607 | |
608 | -class TestGetValueFromJenv(unittest.TestCase): |
609 | - |
610 | - def test_no_key(self): |
611 | - with mock.patch('quickstart.manage.envs.load_generated', |
612 | - lambda x: {}): |
613 | - with self.assertRaises(ValueError) as exc: |
614 | - app.get_value_from_jenv( |
615 | - 'local', '/home/bac/.juju', 'my-key') |
616 | - expected = ( |
617 | - u'my-key not found in ' |
618 | - '/home/bac/.juju/environments/local.jenv') |
619 | - self.assertIn(expected, bytes(exc.exception)) |
620 | +class TestGetAdminSecret( |
621 | + helpers.JenvFileTestsMixin, ProgramExitTestsMixin, unittest.TestCase): |
622 | |
623 | def test_success(self): |
624 | - expected = 'superchunk' |
625 | - with mock.patch('quickstart.manage.envs.load_generated', |
626 | - lambda x: {'my-key': expected}): |
627 | - secret = app.get_value_from_jenv( |
628 | - 'local', '~bac/.juju', 'my-key') |
629 | - self.assertEqual(expected, secret) |
630 | + # The admin secret is successfully retrieved. |
631 | + with self.make_jenv('ec2', yaml.safe_dump(self.jenv_data)): |
632 | + admin_secret = app.get_admin_secret('ec2') |
633 | + self.assertEqual('Secret!', admin_secret) |
634 | + |
635 | + def test_error(self): |
636 | + # A ProgramExit is raised if the admin secret cannot be retrieved. |
637 | + with self.make_jenv('ec2', '') as path: |
638 | + expected_error = ( |
639 | + 'cannot retrieve environment admin-secret: invalid YAML ' |
640 | + 'contents in {}: bootstrap-config key not found in the root ' |
641 | + 'section'.format(path)) |
642 | + with self.assert_program_exit(expected_error): |
643 | + app.get_admin_secret('ec2') |
644 | |
645 | |
646 | class TestGetApiUrl( |
647 | |
648 | === modified file 'quickstart/tests/test_manage.py' |
649 | --- quickstart/tests/test_manage.py 2014-11-10 11:27:04 +0000 |
650 | +++ quickstart/tests/test_manage.py 2014-11-11 14:26:05 +0000 |
651 | @@ -509,7 +509,6 @@ |
652 | options = self.make_options( |
653 | env_file, env_name='aws', interactive=False) |
654 | manage._setup_env(options, self.parser) |
655 | - self.assertEqual('Secret!', options.admin_secret) |
656 | self.assertEqual(env_file, options.env_file) |
657 | self.assertEqual('aws', options.env_name) |
658 | self.assertEqual('ec2', options.env_type) |
659 | @@ -550,7 +549,6 @@ |
660 | options = self.make_options( |
661 | env_file, env_name='lxc', interactive=False) |
662 | manage._setup_env(options, self.parser) |
663 | - self.assertEqual('Secret!', options.admin_secret) |
664 | self.assertEqual(env_file, options.env_file) |
665 | self.assertEqual('lxc', options.env_name) |
666 | self.assertEqual('local', options.env_type) |
667 | @@ -593,7 +591,6 @@ |
668 | mock_interactive.assert_called_once_with( |
669 | self.parser, mock_get_env_type_db(), env_db, env_file) |
670 | # The options is updated with data from the selected environment. |
671 | - self.assertEqual('Secret!', options.admin_secret) |
672 | self.assertEqual(env_file, options.env_file) |
673 | self.assertEqual('aws', options.env_name) |
674 | self.assertEqual('ec2', options.env_type) |
675 | @@ -754,7 +751,6 @@ |
676 | def make_options(self, **kwargs): |
677 | """Set up the options to be passed to the run function.""" |
678 | options = { |
679 | - 'admin_secret': 'Secretz!', |
680 | 'bundle': None, |
681 | 'bundle_id': None, |
682 | 'charm_url': None, |
683 | @@ -767,17 +763,6 @@ |
684 | options.update(kwargs) |
685 | return mock.Mock(**options) |
686 | |
687 | - @staticmethod |
688 | - def mock_get_value_from_jenv_success(name, home, key): |
689 | - return 'from jenv' |
690 | - |
691 | - @staticmethod |
692 | - def mock_get_value_from_jenv_error(name, home, key): |
693 | - fn = '{}.jenv'.format(name) |
694 | - path = os.path.join(home, 'environments', fn) |
695 | - msg = '{} not found in {}'.format(key, path) |
696 | - raise ValueError(msg.encode('utf-8')) |
697 | - |
698 | def test_no_bundle(self, mock_app, mock_open): |
699 | # The application runs correctly if no bundle is provided. |
700 | mock_app.ensure_dependencies.return_value = (1, 18, 0) |
701 | @@ -790,7 +775,8 @@ |
702 | unit_data = {'Name': 'juju-gui/0'} |
703 | mock_app.check_environment.return_value = ( |
704 | 'cs:trusty/juju-gui-42', '0', service_data, unit_data) |
705 | - mock_app.get_value_from_jenv = self.mock_get_value_from_jenv_error |
706 | + # Make mock_app.get_admin_secret return the admin secret |
707 | + mock_app.get_admin_secret.return_value = 'Secret!' |
708 | # Make mock_app.watch return the Juju GUI unit address. |
709 | mock_app.watch.return_value = '1.2.3.4' |
710 | # Make mock_app.create_auth_token return a fake authentication token. |
711 | @@ -810,10 +796,9 @@ |
712 | mock_app.get_api_url.assert_called_once_with( |
713 | options.env_name, self.juju_command) |
714 | mock_app.connect.assert_has_calls([ |
715 | - mock.call( |
716 | - mock_app.get_api_url(), options.admin_secret), |
717 | + mock.call(mock_app.get_api_url(), 'Secret!'), |
718 | mock.call().close(), |
719 | - mock.call('wss://1.2.3.4:443/ws', options.admin_secret), |
720 | + mock.call('wss://1.2.3.4:443/ws', 'Secret!'), |
721 | mock.call().close(), |
722 | ]) |
723 | mock_app.check_environment.assert_called_once_with( |
724 | @@ -906,44 +891,9 @@ |
725 | manage.run(options) |
726 | self.assertFalse(mock_open.called) |
727 | |
728 | - def test_admin_secret_fetched(self, mock_app, mock_open): |
729 | - # If an admin secret is fetched from jenv it is used, even if one is |
730 | - # found in environments.yaml, as set in options.admin_secret. |
731 | - mock_app.get_value_from_jenv = self.mock_get_value_from_jenv_success |
732 | - # Make mock_app.bootstrap return the already_bootstrapped flag and the |
733 | - # bootstrap node series. |
734 | - mock_app.bootstrap.return_value = (True, 'precise') |
735 | - # Make mock_app.check_environment return the charm URL, the machine |
736 | - # where to deploy the charm, the service and unit data. |
737 | - mock_app.check_environment.return_value = ( |
738 | - 'cs:precise/juju-gui-42', '0', None, None) |
739 | - options = self.make_options(admin_secret='secret in environments.yaml') |
740 | - manage.run(options) |
741 | - mock_app.connect.assert_has_calls([ |
742 | - mock.call(mock_app.get_api_url(), 'from jenv'), |
743 | - ]) |
744 | - |
745 | - def test_admin_secret_from_environments_yaml(self, mock_app, mock_open): |
746 | - # If an admin secret is not fetched from jenv, then the one from |
747 | - # environments.yaml is used, as found in options.admin_secret. |
748 | - mock_app.get_value_from_jenv = self.mock_get_value_from_jenv_error |
749 | - # Make mock_app.bootstrap return the already_bootstrapped flag and the |
750 | - # bootstrap node series. |
751 | - mock_app.bootstrap.return_value = (True, 'precise') |
752 | - # Make mock_app.check_environment return the charm URL, the machine |
753 | - # where to deploy the charm, the service and unit data. |
754 | - mock_app.check_environment.return_value = ( |
755 | - 'cs:precise/juju-gui-42', '0', None, None) |
756 | - options = self.make_options(admin_secret='secret in environments.yaml') |
757 | - manage.run(options) |
758 | - mock_app.connect.assert_has_calls([ |
759 | - mock.call(mock_app.get_api_url(), 'secret in environments.yaml'), |
760 | - ]) |
761 | - |
762 | def test_no_admin_secret_found(self, mock_app, mock_open): |
763 | - # If admin-secret cannot be found anywhere a ProgramExit is called. |
764 | - mock_app.ProgramExit = app.ProgramExit |
765 | - mock_app.get_value_from_jenv = self.mock_get_value_from_jenv_error |
766 | + # If admin-secret cannot be found a ProgramExit is called. |
767 | + mock_app.get_admin_secret.side_effect = app.ProgramExit('bad wolf') |
768 | # Make mock_app.bootstrap return the already_bootstrapped flag and the |
769 | # bootstrap node series. |
770 | mock_app.bootstrap.return_value = (True, 'precise') |
771 | @@ -952,15 +902,11 @@ |
772 | mock_app.check_environment.return_value = ( |
773 | 'cs:precise/juju-gui-42', '0', None, None) |
774 | options = self.make_options( |
775 | - admin_secret=None, |
776 | env_name='local', |
777 | env_file='environments.yaml') |
778 | with self.assertRaises(app.ProgramExit) as context: |
779 | manage.run(options) |
780 | - expected = ( |
781 | - u'admin-secret not found in {}/environments/local.jenv ' |
782 | - 'or environments.yaml'.format(settings.JUJU_HOME)) |
783 | - self.assertEqual(expected, context.exception.message) |
784 | + self.assertEqual('bad wolf', context.exception.message) |
785 | |
786 | def test_juju_environ_var_set(self, mock_app, mock_open): |
787 | mock_app.bootstrap.return_value = (True, 'precise') |
788 | |
789 | === modified file 'quickstart/tests/test_serializers.py' |
790 | --- quickstart/tests/test_serializers.py 2013-12-06 18:16:02 +0000 |
791 | +++ quickstart/tests/test_serializers.py 2014-11-11 14:26:05 +0000 |
792 | @@ -18,12 +18,15 @@ |
793 | |
794 | from __future__ import unicode_literals |
795 | |
796 | +import os |
797 | +import tempfile |
798 | import unittest |
799 | |
800 | import mock |
801 | import yaml |
802 | |
803 | from quickstart import serializers |
804 | +from quickstart.tests import helpers |
805 | |
806 | |
807 | class TestYamlLoad(unittest.TestCase): |
808 | @@ -56,3 +59,41 @@ |
809 | # Collections are serialized in the block style. |
810 | contents = serializers.yaml_dump(self.data) |
811 | self.assertEqual('myint: 42\nmystring: foo\n', contents) |
812 | + |
813 | + |
814 | +class TestYamlLoadFromPath(helpers.ValueErrorTestsMixin, unittest.TestCase): |
815 | + |
816 | + def make_file(self, contents): |
817 | + """Create a temporary file with the given contents. |
818 | + |
819 | + Return the file path. |
820 | + """ |
821 | + yaml_file = tempfile.NamedTemporaryFile(delete=False) |
822 | + self.addCleanup(os.remove, yaml_file.name) |
823 | + yaml_file.write(contents) |
824 | + yaml_file.close() |
825 | + return yaml_file.name |
826 | + |
827 | + def test_resulting_contents(self): |
828 | + # The YAML deserialized contents are correctly returned. |
829 | + expected_data = {'myint': 42, 'mystring': 'foo'} |
830 | + path = self.make_file(yaml.safe_dump(expected_data)) |
831 | + obtained_data = serializers.yaml_load_from_path(path) |
832 | + self.assertEqual(expected_data, obtained_data) |
833 | + |
834 | + def test_no_file(self): |
835 | + # A ValueError is raised if the given file is not found. |
836 | + expected_error = ( |
837 | + "unable to open file /no/such/file.yaml: " |
838 | + "[Errno 2] No such file or directory: '/no/such/file.yaml'" |
839 | + ) |
840 | + with self.assert_value_error(expected_error): |
841 | + serializers.yaml_load_from_path('/no/such/file.yaml') |
842 | + |
843 | + def test_invalid_yaml(self): |
844 | + # A ValueError is raised if the given file is not a valid YAML. |
845 | + path = self.make_file(':') |
846 | + with self.assertRaises(ValueError) as context_manager: |
847 | + serializers.yaml_load_from_path(path) |
848 | + expected = 'unable to parse file {}'.format(path) |
849 | + self.assertIn(expected, bytes(context_manager.exception)) |
Reviewers: mp+241416_ code.launchpad. net,
Message:
Please take a look.
Description:
Code reorganization + docs.
Reorganize the jenv related code in preparation of the
bootstrap strategy change.
Separate jenv handling from the environments.yaml file
management in models.
As a consequence of only supporting juju >= 1.18 we
can now safely always retrieve the admin-secret from
the jenv file.
Also updated the HACKING docs to include a brief
description of the project structure.
Tests: `make check`.
QA: use quickstart as usual.
https:/ /code.launchpad .net/~frankban/ juju-quickstart /code-reorg/ +merge/ 241416
(do not edit description out of merge proposal)
Please review this at https:/ /codereview. appspot. com/169380043/
Affected files (+372, -222 lines): manage. py models/ envs.py models/ jenv.py serializers. py tests/helpers. py tests/models/ test_envs. py tests/models/ test_jenv. py tests/test_ app.py tests/test_ manage. py tests/test_ serializers. py
M HACKING.rst
A [revision details]
M quickstart/app.py
M quickstart/
M quickstart/
A quickstart/
M quickstart/
M quickstart/
M quickstart/
A quickstart/
M quickstart/
M quickstart/
M quickstart/