Merge lp:~hazmat/pyjuju/fast-yaml into lp:pyjuju
- fast-yaml
- Merge into trunk
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Jim Baker | ||||
Approved revision: | 576 | ||||
Merged at revision: | 578 | ||||
Proposed branch: | lp:~hazmat/pyjuju/fast-yaml | ||||
Merge into: | lp:pyjuju | ||||
Diff against target: |
3890 lines (+454/-553) 84 files modified
juju/agents/tests/test_unit.py (+2/-2) juju/charm/config.py (+4/-3) juju/charm/metadata.py (+5/-2) juju/charm/tests/test_base.py (+6/-4) juju/charm/tests/test_bundle.py (+5/-5) juju/charm/tests/test_config.py (+10/-8) juju/charm/tests/test_directory.py (+3/-3) juju/charm/tests/test_metadata.py (+4/-3) juju/charm/tests/test_publisher.py (+3/-3) juju/control/config_set.py (+2/-2) juju/control/constraints_get.py (+3/-2) juju/control/deploy.py (+4/-3) juju/control/initialize.py (+2/-2) juju/control/status.py (+4/-2) juju/control/tests/test_add_unit.py (+2/-3) juju/control/tests/test_bootstrap.py (+2/-2) juju/control/tests/test_config_get.py (+3/-5) juju/control/tests/test_config_set.py (+9/-10) juju/control/tests/test_constraints_get.py (+7/-8) juju/control/tests/test_deploy.py (+10/-11) juju/control/tests/test_destroy_environment.py (+1/-1) juju/control/tests/test_expose.py (+2/-2) juju/control/tests/test_initialize.py (+3/-3) juju/control/tests/test_open_tunnel.py (+3/-2) juju/control/tests/test_remove_relation.py (+2/-2) juju/control/tests/test_scp.py (+0/-1) juju/control/tests/test_status.py (+6/-6) juju/control/tests/test_unexpose.py (+2/-2) juju/control/tests/test_upgrade_charm.py (+3/-2) juju/control/tests/test_utils.py (+1/-1) juju/environment/config.py (+3/-2) juju/environment/tests/test_config.py (+63/-62) juju/hooks/protocol.py (+4/-3) juju/hooks/scheduler.py (+4/-3) juju/hooks/tests/test_invoker.py (+5/-5) juju/hooks/tests/test_scheduler.py (+19/-20) juju/lib/format.py (+3/-3) juju/lib/serializer.py (+21/-0) juju/providers/common/cloudinit.py (+3/-2) juju/providers/common/state.py (+4/-3) juju/providers/common/tests/test_cloudinit.py (+3/-3) juju/providers/common/tests/test_findzookeepers.py (+2/-2) juju/providers/common/tests/test_state.py (+4/-3) juju/providers/common/tests/test_utils.py (+2/-2) juju/providers/common/utils.py (+3/-2) juju/providers/ec2/tests/common.py (+2/-2) juju/providers/ec2/tests/test_bootstrap.py (+4/-5) juju/providers/ec2/tests/test_findzookeeper.py (+1/-2) juju/providers/ec2/tests/test_launch.py (+3/-4) juju/providers/ec2/tests/test_provider.py (+1/-0) juju/providers/ec2/tests/test_state.py (+6/-6) juju/providers/local/files.py (+3/-4) juju/providers/local/tests/test_agent.py (+1/-2) juju/providers/local/tests/test_container.py (+1/-0) juju/providers/local/tests/test_files.py (+3/-2) juju/providers/openstack/tests/test_bootstrap.py (+4/-8) juju/providers/openstack/tests/test_launch.py (+2/-2) juju/providers/openstack/tests/test_state.py (+4/-5) juju/providers/orchestra/tests/common.py (+2/-1) juju/providers/orchestra/tests/test_bootstrap.py (+1/-1) juju/providers/orchestra/tests/test_findzookeepers.py (+1/-2) juju/providers/orchestra/tests/test_state.py (+1/-1) juju/state/charm.py (+4/-4) juju/state/environment.py (+7/-6) juju/state/hook.py (+1/-2) juju/state/machine.py (+2/-2) juju/state/relation.py (+8/-8) juju/state/security.py (+6/-5) juju/state/service.py (+24/-21) juju/state/tests/test_charm.py (+3/-2) juju/state/tests/test_environment.py (+9/-7) juju/state/tests/test_hook.py (+19/-20) juju/state/tests/test_machine.py (+5/-6) juju/state/tests/test_relation.py (+12/-12) juju/state/tests/test_security.py (+4/-4) juju/state/tests/test_service.py (+12/-12) juju/state/tests/test_topology.py (+3/-63) juju/state/tests/test_utils.py (+10/-8) juju/state/topology.py (+5/-82) juju/state/utils.py (+5/-4) juju/unit/lifecycle.py (+3/-3) juju/unit/tests/test_lifecycle.py (+3/-2) juju/unit/tests/test_workflow.py (+5/-5) juju/unit/workflow.py (+8/-8) |
||||
To merge this branch: | bzr merge lp:~hazmat/pyjuju/fast-yaml | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju Engineering | Pending | ||
Review via email: mp+123470@code.launchpad.net |
Commit message
Description of the change
ensure use of py yaml c extension for speed.
Backwards compatible. Drops test time in half. Should improve speed
across the board, things like status will see marked improvements.
Kapil Thangavelu (hazmat) wrote : | # |
Martin Packman (gz) wrote : | # |
Looks like a good improvement, I like that this also sorts out the
confusion over whether to use dump/load or safe_dump/safe_load at the
same time.
I would have been tempted to name the wrapper module juju.lib.yaml to
avoid touching most lines, but this way means you did look at all the
callers at least.
There are several callsites that still use the yaml module functions and
pass in a Loader or Dumper, apart from the one noted below, shouldn't
they all be changed to using juju.lib for consistency?
https:/
File juju/charm/
https:/
juju/charm/
Loader=
Why not use dump from juju.lib here rather than specifying the loader?
Will need the yaml import for the Mark below anyway, but seems best to
consistently use the same loader.
https:/
File juju/control/
https:/
juju/control/
Dumper=
I'd dump using the juju.lib function and write to stout separately
afterwards here.
https:/
File juju/control/
https:/
juju/control/
Dumper=
This looks like the only place that really wants extra args to dump,
leaving this as a special case seems fine.
https:/
File juju/lib/
https:/
juju/lib/
I'd be tempted to make this just take a dict rather than passing through
args and kwargs.
https:/
juju/lib/
Likewise, would make this just take a string.
- 576. By Kapil Thangavelu
-
explicit yaml dump/load in serializer, and yaml_mark_with_path utility method, switch out juju/charm & juju/control to use
Kapil Thangavelu (hazmat) wrote : | # |
Please take a look.
Jim Baker (jimbaker) wrote : | # |
+1, LGTM, nice speedup and a clean transition to the C extensions. Just
need to take care of the two trivials where CSafeLoader/
used instead of the equivalent juju.lib.serializer wrappers.
https:/
File juju/environmen
https:/
juju/environmen
Loader=
Use juju.lib.
https:/
juju/environmen
Dumper=
Use juju.lib.
- 577. By Kapil Thangavelu
-
switch out juju/env pkg to use serializer for yaml dump/load
- 578. By Kapil Thangavelu
-
merge trunk and resolve conflicts
- 579. By Kapil Thangavelu
-
address some comments from martin's review (sys.stdout.write, use contextmanager for files)
- 580. By Kapil Thangavelu
-
kill useless topo migration, fix some laggards wrt to dump/load future format change isomorphism
Preview Diff
1 | === modified file 'juju/agents/tests/test_unit.py' |
2 | --- juju/agents/tests/test_unit.py 2012-03-28 18:57:58 +0000 |
3 | +++ juju/agents/tests/test_unit.py 2012-09-15 19:26:18 +0000 |
4 | @@ -1,7 +1,6 @@ |
5 | import argparse |
6 | import logging |
7 | import os |
8 | -import yaml |
9 | |
10 | from twisted.internet.defer import inlineCallbacks, returnValue |
11 | |
12 | @@ -10,6 +9,7 @@ |
13 | from juju.charm import get_charm_from_path |
14 | from juju.charm.url import CharmURL |
15 | from juju.errors import JujuError |
16 | +from juju.lib import serializer |
17 | from juju.state.environment import GlobalSettingsStateManager |
18 | from juju.state.errors import ServiceStateNotFound |
19 | from juju.state.service import NO_HOOKS, RETRY_HOOKS |
20 | @@ -375,7 +375,7 @@ |
21 | self.assertEqual( |
22 | {"greeting": "hello", "planet": "earth", |
23 | "private-address": "mysql-0.example.com"}, |
24 | - yaml.load(contents)) |
25 | + serializer.load(contents)) |
26 | |
27 | self.failUnlessIn("wordpress/0", output) |
28 | |
29 | |
30 | === modified file 'juju/charm/config.py' |
31 | --- juju/charm/config.py 2012-08-30 03:06:56 +0000 |
32 | +++ juju/charm/config.py 2012-09-15 19:26:18 +0000 |
33 | @@ -1,9 +1,9 @@ |
34 | import copy |
35 | import os |
36 | import sys |
37 | - |
38 | import yaml |
39 | |
40 | +from juju.lib import serializer |
41 | from juju.lib.format import YAMLFormat |
42 | from juju.lib.schema import (SchemaError, KeyDict, Dict, String, |
43 | Constant, OneOf, Int, Float) |
44 | @@ -78,11 +78,12 @@ |
45 | """ |
46 | if isinstance(data, basestring): |
47 | try: |
48 | - raw_data = yaml.load(data) |
49 | + raw_data = serializer.yaml_load(data) |
50 | except yaml.MarkedYAMLError, e: |
51 | # Capture the path name on the error if present. |
52 | if pathname is not None: |
53 | - e.problem_mark.name = pathname |
54 | + e.problem_mark = serializer.yaml_mark_with_path( |
55 | + pathname, e.problem_mark) |
56 | raise |
57 | elif isinstance(data, dict): |
58 | raw_data = data |
59 | |
60 | === modified file 'juju/charm/metadata.py' |
61 | --- juju/charm/metadata.py 2012-06-14 17:51:48 +0000 |
62 | +++ juju/charm/metadata.py 2012-09-15 19:26:18 +0000 |
63 | @@ -5,6 +5,7 @@ |
64 | |
65 | from juju.charm.errors import MetaDataError |
66 | from juju.errors import FileNotFound |
67 | +from juju.lib import serializer |
68 | from juju.lib.format import is_valid_charm_format |
69 | from juju.lib.schema import ( |
70 | SchemaError, Bool, Constant, Dict, Int, |
71 | @@ -200,11 +201,13 @@ |
72 | @raise MetaDataError: When errors are found in the info data. |
73 | """ |
74 | try: |
75 | - self.parse_serialization_data(yaml.load(content), path) |
76 | + self.parse_serialization_data( |
77 | + serializer.yaml_load(content), path) |
78 | except yaml.MarkedYAMLError, e: |
79 | # Capture the path name on the error if present. |
80 | if path is not None: |
81 | - e.problem_mark.name = path |
82 | + e.problem_mark = serializer.yaml_mark_with_path( |
83 | + path, e.problem_mark) |
84 | raise |
85 | |
86 | if "revision" in self._data and path: |
87 | |
88 | === modified file 'juju/charm/tests/test_base.py' |
89 | --- juju/charm/tests/test_base.py 2011-10-05 19:12:34 +0000 |
90 | +++ juju/charm/tests/test_base.py 2012-09-15 19:26:18 +0000 |
91 | @@ -1,8 +1,7 @@ |
92 | -import yaml |
93 | - |
94 | from juju.charm.base import CharmBase, get_revision |
95 | from juju.charm.metadata import MetaData |
96 | from juju.errors import CharmError |
97 | +from juju.lib import serializer |
98 | from juju.lib.testing import TestCase |
99 | |
100 | |
101 | @@ -73,6 +72,9 @@ |
102 | def test_metadata_fallback(self): |
103 | metadata = MetaData() |
104 | self.assertEquals(get_revision(None, metadata, None), None) |
105 | - metadata.parse(yaml.dump({ |
106 | - "name": "x", "summary": "y", "description": "z","revision": 33})) |
107 | + metadata.parse( |
108 | + serializer.yaml_dump( |
109 | + {"name": "x", "summary": "y", "description": "z","revision": 33}, |
110 | + )) |
111 | + |
112 | self.assertEquals(get_revision(None, metadata, None), 33) |
113 | |
114 | === modified file 'juju/charm/tests/test_bundle.py' |
115 | --- juju/charm/tests/test_bundle.py 2012-04-04 11:26:08 +0000 |
116 | +++ juju/charm/tests/test_bundle.py 2012-09-15 19:26:18 +0000 |
117 | @@ -3,9 +3,9 @@ |
118 | import inspect |
119 | import shutil |
120 | import stat |
121 | -import yaml |
122 | import zipfile |
123 | |
124 | +from juju.lib import serializer |
125 | from juju.lib.testing import TestCase |
126 | from juju.lib.filehash import compute_file_hash |
127 | from juju.charm.metadata import MetaData |
128 | @@ -84,9 +84,9 @@ |
129 | continue |
130 | content = zf_src.read(name) |
131 | if name == "metadata.yaml": |
132 | - data = yaml.load(content) |
133 | + data = serializer.yaml_load(content) |
134 | data["revision"] = 303 |
135 | - content = yaml.dump(data) |
136 | + content = serializer.yaml_dump(data) |
137 | zf_dst.writestr(name, content) |
138 | zf_src.close() |
139 | zf_dst.close() |
140 | @@ -97,9 +97,9 @@ |
141 | def test_competing_revisions(self): |
142 | zf = zipfile.ZipFile(self.filename, "a") |
143 | zf.writestr("revision", "999") |
144 | - data = yaml.load(zf.read("metadata.yaml")) |
145 | + data = serializer.yaml_load(zf.read("metadata.yaml")) |
146 | data["revision"] = 303 |
147 | - zf.writestr("metadata.yaml", yaml.dump(data)) |
148 | + zf.writestr("metadata.yaml", serializer.yaml_dump(data)) |
149 | zf.close() |
150 | |
151 | charm = CharmBundle(self.filename) |
152 | |
153 | === modified file 'juju/charm/tests/test_config.py' |
154 | --- juju/charm/tests/test_config.py 2012-07-02 08:48:19 +0000 |
155 | +++ juju/charm/tests/test_config.py 2012-09-15 19:26:18 +0000 |
156 | @@ -3,6 +3,7 @@ |
157 | |
158 | import yaml |
159 | |
160 | +from juju.lib import serializer |
161 | from juju.lib.testing import TestCase |
162 | from juju.charm.config import ConfigOptions |
163 | from juju.charm.errors import ServiceConfigError, ServiceConfigValueError |
164 | @@ -25,7 +26,7 @@ |
165 | type: int |
166 | """ |
167 | |
168 | -sample_yaml_data = yaml.load(sample_configuration) |
169 | +sample_yaml_data = serializer.yaml_load(sample_configuration) |
170 | |
171 | sample_config_defaults = {"title": "My Title", |
172 | "username": "admin001"} |
173 | @@ -82,7 +83,7 @@ |
174 | e = self.assertRaises( |
175 | ServiceConfigValueError, |
176 | self.config.parse, |
177 | - yaml.dump( |
178 | + serializer.yaml_dump( |
179 | {"options": { |
180 | "foobar": { |
181 | "description": "beyond what?", |
182 | @@ -98,8 +99,9 @@ |
183 | |
184 | # Verify dictionary serialization |
185 | schema_dict = self.config.as_dict() |
186 | - self.assertEqual(schema_dict, |
187 | - yaml.load(sample_configuration)["options"]) |
188 | + self.assertEqual( |
189 | + schema_dict, |
190 | + serializer.yaml_load(sample_configuration)["options"]) |
191 | |
192 | # Verify the dictionary is a copy |
193 | # Poke at embedded objects |
194 | @@ -143,7 +145,7 @@ |
195 | ServiceConfigValueError, config.validate, sample_input) |
196 | |
197 | def test_validate_float(self): |
198 | - self.config.parse(yaml.dump( |
199 | + self.config.parse(serializer.yaml_dump( |
200 | {"options": { |
201 | "score": { |
202 | "description": "A number indicating score.", |
203 | @@ -166,7 +168,7 @@ |
204 | self.assertEqual(data, {"title": u"Good"}) |
205 | |
206 | def test_validate_boolean(self): |
207 | - self.config.parse(yaml.dump( |
208 | + self.config.parse(serializer.yaml_dump( |
209 | {"options": { |
210 | "active": { |
211 | "description": "A boolean indicating activity.", |
212 | @@ -200,9 +202,9 @@ |
213 | 'string' now). Remove support for it after a while, and take |
214 | this test with it. |
215 | """ |
216 | - config = yaml.load(sample_configuration) |
217 | + config = serializer.yaml_load(sample_configuration) |
218 | config["options"]["title"]["type"] = "str" |
219 | - obsolete_config = yaml.dump(config) |
220 | + obsolete_config = serializer.yaml_dump(config) |
221 | |
222 | sio = StringIO() |
223 | self.patch(sys, "stderr", sio) |
224 | |
225 | === modified file 'juju/charm/tests/test_directory.py' |
226 | --- juju/charm/tests/test_directory.py 2012-03-08 02:50:43 +0000 |
227 | +++ juju/charm/tests/test_directory.py 2012-09-15 19:26:18 +0000 |
228 | @@ -4,7 +4,6 @@ |
229 | import inspect |
230 | import shutil |
231 | import tempfile |
232 | -import yaml |
233 | import zipfile |
234 | |
235 | from juju.errors import CharmError, FileNotFound |
236 | @@ -12,6 +11,7 @@ |
237 | from juju.charm.metadata import MetaData |
238 | from juju.charm.directory import CharmDirectory |
239 | from juju.charm.bundle import CharmBundle |
240 | +from juju.lib import serializer |
241 | from juju.lib.filehash import compute_file_hash |
242 | from juju.charm import tests |
243 | |
244 | @@ -46,10 +46,10 @@ |
245 | def set_metadata_revision(self, dir_, revision): |
246 | metadata_path = os.path.join(dir_, "metadata.yaml") |
247 | with open(metadata_path) as f: |
248 | - data = yaml.load(f.read()) |
249 | + data = serializer.yaml_load(f.read()) |
250 | data["revision"] = 999 |
251 | with open(metadata_path, "w") as f: |
252 | - f.write(yaml.dump(data)) |
253 | + f.write(serializer.yaml_dump(data)) |
254 | |
255 | def test_metadata_is_required(self): |
256 | directory = self.makeDir() |
257 | |
258 | === modified file 'juju/charm/tests/test_metadata.py' |
259 | --- juju/charm/tests/test_metadata.py 2012-05-25 22:43:52 +0000 |
260 | +++ juju/charm/tests/test_metadata.py 2012-09-15 19:26:18 +0000 |
261 | @@ -9,6 +9,7 @@ |
262 | MetaData, MetaDataError, InterfaceExpander, SchemaError) |
263 | from juju.errors import FileNotFound |
264 | from juju.lib.testing import TestCase |
265 | +from juju.lib import serializer |
266 | |
267 | test_repository_path = os.path.join( |
268 | os.path.dirname(inspect.getabsfile(tests)), |
269 | @@ -39,11 +40,11 @@ |
270 | class HackManager(object): |
271 | |
272 | def __enter__(mgr): |
273 | - mgr.data = yaml.load(self.sample) |
274 | + mgr.data = serializer.yaml_load(self.sample) |
275 | return mgr.data |
276 | |
277 | def __exit__(mgr, exc_type, exc_val, exc_tb): |
278 | - self.sample = yaml.dump(mgr.data) |
279 | + self.sample = serializer.yaml_dump(mgr.data) |
280 | return False |
281 | return HackManager() |
282 | |
283 | @@ -145,7 +146,7 @@ |
284 | others. |
285 | """ |
286 | serialization_data = {"Hi": "there!"} |
287 | - yaml_data = yaml.dump(serialization_data) |
288 | + yaml_data = serializer.yaml_dump(serialization_data) |
289 | path = self.makeFile(yaml_data) |
290 | mock = self.mocker.patch(self.metadata) |
291 | mock.parse(yaml_data, path) |
292 | |
293 | === modified file 'juju/charm/tests/test_publisher.py' |
294 | --- juju/charm/tests/test_publisher.py 2012-04-06 16:35:31 +0000 |
295 | +++ juju/charm/tests/test_publisher.py 2012-09-15 19:26:18 +0000 |
296 | @@ -1,6 +1,5 @@ |
297 | import fcntl |
298 | import os |
299 | -import yaml |
300 | import zookeeper |
301 | |
302 | from twisted.internet.defer import inlineCallbacks, fail |
303 | @@ -12,7 +11,7 @@ |
304 | from juju.charm.directory import CharmDirectory |
305 | from juju.charm.publisher import CharmPublisher |
306 | from juju.charm.tests import local_charm_id |
307 | -from juju.lib import under |
308 | +from juju.lib import under, serializer |
309 | from juju.providers.dummy import FileStorage |
310 | from juju.state.charm import CharmStateManager |
311 | from juju.state.errors import StateChanged |
312 | @@ -140,7 +139,8 @@ |
313 | |
314 | # assert the checksum matches the initially published checksum |
315 | self.assertEqual( |
316 | - yaml.load(content)["sha256"], self.charm.get_sha256()) |
317 | + serializer.yaml_load(content)["sha256"], |
318 | + self.charm.get_sha256()) |
319 | |
320 | store_path = os.path.join(self.storage_dir, self.charm_storage_key) |
321 | self.assertTrue(os.path.exists(store_path)) |
322 | |
323 | === modified file 'juju/control/config_set.py' |
324 | --- juju/control/config_set.py 2012-07-02 23:55:27 +0000 |
325 | +++ juju/control/config_set.py 2012-09-15 19:26:18 +0000 |
326 | @@ -1,11 +1,11 @@ |
327 | import argparse |
328 | - |
329 | import yaml |
330 | |
331 | from twisted.internet.defer import inlineCallbacks |
332 | |
333 | from juju.charm.errors import ServiceConfigValueError |
334 | from juju.control.utils import get_environment |
335 | +from juju.lib import serializer |
336 | from juju.lib.format import get_charm_formatter |
337 | from juju.state.service import ServiceStateManager |
338 | |
339 | @@ -62,7 +62,7 @@ |
340 | |
341 | yaml_data = options.config.read() |
342 | try: |
343 | - data = yaml.safe_load(yaml_data) |
344 | + data = serializer.yaml_load(yaml_data) |
345 | except yaml.YAMLError: |
346 | raise ServiceConfigValueError( |
347 | "Config file %r not valid YAML" % options.config.name) |
348 | |
349 | === modified file 'juju/control/constraints_get.py' |
350 | --- juju/control/constraints_get.py 2012-03-30 09:41:54 +0000 |
351 | +++ juju/control/constraints_get.py 2012-09-15 19:26:18 +0000 |
352 | @@ -1,10 +1,10 @@ |
353 | import argparse |
354 | import sys |
355 | -import yaml |
356 | |
357 | from twisted.internet.defer import inlineCallbacks |
358 | |
359 | from juju.control.utils import get_environment, sync_environment_state |
360 | +from juju.lib import serializer |
361 | from juju.state.environment import EnvironmentStateManager |
362 | from juju.state.machine import MachineStateManager |
363 | from juju.state.service import ServiceStateManager |
364 | @@ -70,6 +70,7 @@ |
365 | log.info("Fetching constraints for environment") |
366 | constraints = yield esm.get_constraints() |
367 | result = dict(constraints) |
368 | - yaml.safe_dump(result, sys.stdout) |
369 | + contents = serializer.yaml_dump(result) |
370 | + sys.stdout.write(contents) |
371 | finally: |
372 | yield client.close() |
373 | |
374 | === modified file 'juju/control/deploy.py' |
375 | --- juju/control/deploy.py 2012-04-01 00:48:01 +0000 |
376 | +++ juju/control/deploy.py 2012-09-15 19:26:18 +0000 |
377 | @@ -1,7 +1,5 @@ |
378 | import os |
379 | |
380 | -import yaml |
381 | - |
382 | from twisted.internet.defer import inlineCallbacks |
383 | |
384 | from juju.control import legacy |
385 | @@ -12,6 +10,7 @@ |
386 | from juju.charm.publisher import CharmPublisher |
387 | from juju.charm.repository import resolve |
388 | from juju.errors import CharmError |
389 | +from juju.lib import serializer |
390 | from juju.state.endpoint import RelationEndpoint |
391 | from juju.state.placement import place_unit |
392 | from juju.state.relation import RelationStateManager |
393 | @@ -85,7 +84,9 @@ |
394 | raise ServiceConfigValueError( |
395 | "Config file %r not accessible." % config_file) |
396 | |
397 | - options = yaml.load(open(config_file, "r").read()) |
398 | + with open(config_file) as fh: |
399 | + options = serializer.yaml_load(fh.read()) |
400 | + |
401 | if not options or not isinstance(options, dict) or \ |
402 | service_name not in options: |
403 | raise ServiceConfigValueError( |
404 | |
405 | === modified file 'juju/control/initialize.py' |
406 | --- juju/control/initialize.py 2012-03-29 01:37:57 +0000 |
407 | +++ juju/control/initialize.py 2012-09-15 19:26:18 +0000 |
408 | @@ -1,11 +1,11 @@ |
409 | from base64 import b64decode |
410 | import os |
411 | -import yaml |
412 | |
413 | from twisted.internet.defer import inlineCallbacks |
414 | |
415 | from txzookeeper import ZookeeperClient |
416 | |
417 | +from juju.lib import serializer |
418 | from juju.state.initialize import StateHierarchy |
419 | |
420 | |
421 | @@ -34,7 +34,7 @@ |
422 | zk_address = os.environ.get("ZOOKEEPER_ADDRESS", "127.0.0.1:2181") |
423 | client = yield ZookeeperClient(zk_address).connect() |
424 | try: |
425 | - constraints_data = yaml.load(b64decode(options.constraints_data)) |
426 | + constraints_data = serializer.load(b64decode(options.constraints_data)) |
427 | hierarchy = StateHierarchy( |
428 | client, |
429 | options.admin_identity, |
430 | |
431 | === modified file 'juju/control/status.py' |
432 | --- juju/control/status.py 2012-08-05 00:18:28 +0000 |
433 | +++ juju/control/status.py 2012-09-15 19:26:18 +0000 |
434 | @@ -3,12 +3,13 @@ |
435 | import functools |
436 | import json |
437 | import sys |
438 | +import yaml |
439 | |
440 | from twisted.internet.defer import inlineCallbacks, returnValue |
441 | -import yaml |
442 | |
443 | from juju.control.utils import get_environment |
444 | from juju.errors import ProviderError |
445 | + |
446 | from juju.state.errors import UnitRelationStateNotFound |
447 | from juju.state.charm import CharmStateManager |
448 | from juju.state.machine import MachineStateManager |
449 | @@ -576,7 +577,8 @@ |
450 | |
451 | def render_yaml(data, filelike, environment): |
452 | # remove the root nodes empty name |
453 | - yaml.safe_dump(data, filelike, default_flow_style=False) |
454 | + yaml.dump( |
455 | + data, filelike, default_flow_style=False, Dumper=yaml.CSafeDumper) |
456 | |
457 | renderers["yaml"] = render_yaml |
458 | |
459 | |
460 | === modified file 'juju/control/tests/test_add_unit.py' |
461 | --- juju/control/tests/test_add_unit.py 2012-05-04 22:43:40 +0000 |
462 | +++ juju/control/tests/test_add_unit.py 2012-09-15 19:26:18 +0000 |
463 | @@ -1,8 +1,7 @@ |
464 | -from yaml import dump |
465 | - |
466 | from twisted.internet.defer import inlineCallbacks |
467 | |
468 | from juju.control import main |
469 | +from juju.lib.serializer import yaml_dump |
470 | from juju.state.environment import EnvironmentStateManager |
471 | |
472 | from .common import MachineControlToolTest |
473 | @@ -184,7 +183,7 @@ |
474 | "environments": {"firstenv": { |
475 | "some-legacy-key": "blah", |
476 | "type": "dummy"}}} |
477 | - self.write_config(dump(local_config)) |
478 | + self.write_config(yaml_dump(local_config)) |
479 | self.config.load() |
480 | |
481 | finished = self.setup_cli_reactor() |
482 | |
483 | === modified file 'juju/control/tests/test_bootstrap.py' |
484 | --- juju/control/tests/test_bootstrap.py 2012-05-04 22:43:40 +0000 |
485 | +++ juju/control/tests/test_bootstrap.py 2012-09-15 19:26:18 +0000 |
486 | @@ -1,9 +1,9 @@ |
487 | |
488 | from twisted.internet.defer import inlineCallbacks, succeed |
489 | -from yaml import dump |
490 | |
491 | +from juju.control import main |
492 | +from juju.lib.serializer import yaml_dump as dump |
493 | from juju.providers.dummy import MachineProvider |
494 | -from juju.control import main |
495 | |
496 | from .common import ControlToolTest |
497 | |
498 | |
499 | === modified file 'juju/control/tests/test_config_get.py' |
500 | --- juju/control/tests/test_config_get.py 2012-09-13 18:31:19 +0000 |
501 | +++ juju/control/tests/test_config_get.py 2012-09-15 19:26:18 +0000 |
502 | @@ -1,10 +1,8 @@ |
503 | -import yaml |
504 | - |
505 | from twisted.internet.defer import inlineCallbacks |
506 | |
507 | from juju.control import main |
508 | - |
509 | from juju.charm.tests import local_charm_id |
510 | +from juju.lib import serializer |
511 | from .common import MachineControlToolTest |
512 | |
513 | |
514 | @@ -32,7 +30,7 @@ |
515 | main(["get", "wordpress"]) |
516 | |
517 | yield finished |
518 | - data = yaml.load(output.getvalue()) |
519 | + data = serializer.yaml_load(output.getvalue()) |
520 | self.assertEqual( |
521 | {"service": "wordpress", |
522 | "charm": "local:series/wordpress-3", |
523 | @@ -59,7 +57,7 @@ |
524 | main(["get", "dummy"]) |
525 | |
526 | yield finished |
527 | - data = yaml.load(output.getvalue()) |
528 | + data = serializer.yaml_load(output.getvalue()) |
529 | |
530 | self.assertEqual( |
531 | {"service": "dummy", |
532 | |
533 | === modified file 'juju/control/tests/test_config_set.py' |
534 | --- juju/control/tests/test_config_set.py 2012-08-30 03:06:56 +0000 |
535 | +++ juju/control/tests/test_config_set.py 2012-09-15 19:26:18 +0000 |
536 | @@ -1,9 +1,8 @@ |
537 | -from yaml import dump |
538 | - |
539 | from twisted.internet.defer import inlineCallbacks |
540 | |
541 | from juju.control import main |
542 | from juju.control.config_set import config_set |
543 | +from juju.lib import serializer |
544 | from .common import MachineControlToolTest |
545 | |
546 | |
547 | @@ -38,8 +37,8 @@ |
548 | finished = self.setup_cli_reactor() |
549 | self.setup_exit(0) |
550 | self.mocker.replay() |
551 | - config_file = self.makeFile(dump(dict( |
552 | - wordpress={"blog-title": "Hello World"}))) |
553 | + config_file = self.makeFile(serializer.yaml_dump( |
554 | + dict(wordpress={"blog-title": "Hello World"}))) |
555 | |
556 | main(["set", "wordpress", |
557 | "--config=%s" % config_file]) |
558 | @@ -54,13 +53,13 @@ |
559 | finished = self.setup_cli_reactor() |
560 | self.setup_exit(0) |
561 | self.mocker.replay() |
562 | + |
563 | # missing the service_name dict (will do nothing to values) |
564 | - config_file = self.makeFile(dump({"blog-title": "Hello World"})) |
565 | - |
566 | - main(["set", "wordpress", |
567 | - "--config=%s" % config_file]) |
568 | - |
569 | + config_file = self.makeFile( |
570 | + serializer.yaml_dump({"blog-title": "Hello World"})) |
571 | + main(["set", "wordpress", "--config=%s" % config_file]) |
572 | yield finished |
573 | + |
574 | state = yield self.service_state.get_config() |
575 | self.assertEqual(state, {'blog-title': 'My Title'}) |
576 | |
577 | @@ -90,7 +89,7 @@ |
578 | self.mocker.replay() |
579 | |
580 | # valid file, but incorrect cli usage |
581 | - config_file = self.makeFile(dump(dict( |
582 | + config_file = self.makeFile(serializer.yaml_dump(dict( |
583 | wordpress={"blog-title": "Hello World"}))) |
584 | |
585 | main(["-v", "set", "wordpress", |
586 | |
587 | === modified file 'juju/control/tests/test_constraints_get.py' |
588 | --- juju/control/tests/test_constraints_get.py 2012-03-30 09:41:54 +0000 |
589 | +++ juju/control/tests/test_constraints_get.py 2012-09-15 19:26:18 +0000 |
590 | @@ -1,8 +1,7 @@ |
591 | -import yaml |
592 | - |
593 | from twisted.internet.defer import inlineCallbacks |
594 | |
595 | from juju.control import main |
596 | +from juju.lib import serializer |
597 | from juju.machine.tests.test_constraints import dummy_cs |
598 | from juju.state.environment import EnvironmentStateManager |
599 | |
600 | @@ -65,7 +64,7 @@ |
601 | def test_env(self): |
602 | main(["get-constraints"]) |
603 | yield self.finished |
604 | - result = yaml.load(self.stdout.getvalue()) |
605 | + result = serializer.load(self.stdout.getvalue()) |
606 | self.assertEquals(result, self.expect_env) |
607 | self.assert_messages(env_log) |
608 | |
609 | @@ -73,7 +72,7 @@ |
610 | def test_service(self): |
611 | main(["get-constraints", "mysql"]) |
612 | yield self.finished |
613 | - result = yaml.load(self.stdout.getvalue()) |
614 | + result = serializer.load(self.stdout.getvalue()) |
615 | self.assertEquals(result, {"mysql": self.expect_service}) |
616 | self.assert_messages(service_log) |
617 | |
618 | @@ -81,7 +80,7 @@ |
619 | def test_unit(self): |
620 | main(["get-constraints", "mysql/0"]) |
621 | yield self.finished |
622 | - result = yaml.load(self.stdout.getvalue()) |
623 | + result = serializer.load(self.stdout.getvalue()) |
624 | self.assertEquals(result, {"mysql/0": self.expect_unit}) |
625 | self.assert_messages(unit_log) |
626 | |
627 | @@ -89,7 +88,7 @@ |
628 | def test_machine(self): |
629 | main(["get-constraints", "1"]) |
630 | yield self.finished |
631 | - result = yaml.load(self.stdout.getvalue()) |
632 | + result = serializer.load(self.stdout.getvalue()) |
633 | self.assertEquals(result, {"1": self.expect_machine}) |
634 | self.assert_messages(machine_log) |
635 | |
636 | @@ -97,7 +96,7 @@ |
637 | def test_all(self): |
638 | main(["get-constraints", "mysql", "mysql/0", "1"]) |
639 | yield self.finished |
640 | - result = yaml.load(self.stdout.getvalue()) |
641 | + result = serializer.load(self.stdout.getvalue()) |
642 | expect = {"mysql": self.expect_service, |
643 | "mysql/0": self.expect_unit, |
644 | "1": self.expect_machine} |
645 | @@ -111,6 +110,6 @@ |
646 | yield self.client.delete("/environment") |
647 | main(["get-constraints", "mysql/0"]) |
648 | yield self.finished |
649 | - result = yaml.load(self.stdout.getvalue()) |
650 | + result = serializer.load(self.stdout.getvalue()) |
651 | self.assertEquals(result, {"mysql/0": self.expect_unit}) |
652 | self.assert_messages(unit_log) |
653 | |
654 | === modified file 'juju/control/tests/test_deploy.py' |
655 | --- juju/control/tests/test_deploy.py 2012-05-04 22:43:40 +0000 |
656 | +++ juju/control/tests/test_deploy.py 2012-09-15 19:26:18 +0000 |
657 | @@ -1,7 +1,5 @@ |
658 | import logging |
659 | import os |
660 | -import yaml |
661 | - |
662 | |
663 | from twisted.internet.defer import inlineCallbacks, succeed |
664 | |
665 | @@ -15,6 +13,7 @@ |
666 | from juju.charm.repository import RemoteCharmRepository |
667 | from juju.charm.url import CharmURL |
668 | from juju.charm.errors import ServiceConfigValueError |
669 | +from juju.lib import serializer |
670 | |
671 | from juju.state.charm import CharmStateManager |
672 | from juju.state.environment import EnvironmentStateManager |
673 | @@ -58,7 +57,7 @@ |
674 | "secondenv": { |
675 | "type": "dummy", "admin-secret": "marge"}}} |
676 | |
677 | - self.write_config(yaml.dump(config)) |
678 | + self.write_config(serializer.dump(config)) |
679 | stderr = self.capture_logging() |
680 | main(["deploy", "--repository", self.unbundled_repo_path, "mysql"]) |
681 | self.assertIn("There are multiple environments", stderr.getvalue()) |
682 | @@ -167,7 +166,7 @@ |
683 | "secondenv": { |
684 | "type": "dummy", "admin-secret": "marge"}}} |
685 | |
686 | - self.write_config(yaml.dump(config)) |
687 | + self.write_config(serializer.dump(config)) |
688 | |
689 | stderr = self.capture_logging() |
690 | main(["deploy", "--environment", "roman-candle", |
691 | @@ -186,7 +185,7 @@ |
692 | "secondenv": { |
693 | "type": "dummy", "admin-secret": "marge"}}} |
694 | |
695 | - self.write_config(yaml.dump(config)) |
696 | + self.write_config(serializer.dump(config)) |
697 | |
698 | def match_config(config): |
699 | return isinstance(config, EnvironmentsConfig) |
700 | @@ -353,8 +352,8 @@ |
701 | env_state_manager = EnvironmentStateManager(self.client) |
702 | env_config = yield env_state_manager.get_config() |
703 | |
704 | - self.assertEquals(yaml.load(env_config.serialize("firstenv")), |
705 | - yaml.load(self.config.serialize("firstenv"))) |
706 | + self.assertEquals(serializer.load(env_config.serialize("firstenv")), |
707 | + serializer.load(self.config.serialize("firstenv"))) |
708 | |
709 | @inlineCallbacks |
710 | def test_deploy_reuses_machines(self): |
711 | @@ -394,7 +393,7 @@ |
712 | """Valid config options should be available to the deployed |
713 | service.""" |
714 | config_file = self.makeFile( |
715 | - yaml.dump(dict(otherservice=dict(application_file="foo")))) |
716 | + serializer.dump(dict(otherservice=dict(application_file="foo")))) |
717 | environment = self.config.get("firstenv") |
718 | |
719 | failure = deploy.deploy( |
720 | @@ -408,7 +407,7 @@ |
721 | def test_deploy_with_invalid_config(self): |
722 | """Can't deploy with config that doesn't pass charm validation.""" |
723 | config_file = self.makeFile( |
724 | - yaml.dump(dict(myblog=dict(application_file="foo")))) |
725 | + serializer.dump(dict(myblog=dict(application_file="foo")))) |
726 | environment = self.config.get("firstenv") |
727 | |
728 | failure = deploy.deploy( |
729 | @@ -426,7 +425,7 @@ |
730 | def test_deploy_with_config(self): |
731 | """Valid config options should be available to the deployed |
732 | service.""" |
733 | - config_file = self.makeFile(yaml.dump(dict( |
734 | + config_file = self.makeFile(serializer.dump(dict( |
735 | myblog=dict(outlook="sunny", |
736 | username="tester01")))) |
737 | environment = self.config.get("firstenv") |
738 | @@ -555,7 +554,7 @@ |
739 | "type": "dummy", |
740 | "some-legacy-key": "blah", |
741 | "default-series": "series"}}} |
742 | - self.write_config(yaml.dump(local_config)) |
743 | + self.write_config(serializer.dump(local_config)) |
744 | self.config.load() |
745 | finished = self.setup_cli_reactor() |
746 | self.setup_exit(0) |
747 | |
748 | === modified file 'juju/control/tests/test_destroy_environment.py' |
749 | --- juju/control/tests/test_destroy_environment.py 2012-05-04 22:43:40 +0000 |
750 | +++ juju/control/tests/test_destroy_environment.py 2012-09-15 19:26:18 +0000 |
751 | @@ -1,6 +1,6 @@ |
752 | from twisted.internet.defer import succeed, inlineCallbacks |
753 | -from yaml import dump |
754 | |
755 | +from juju.lib.serializer import dump |
756 | from juju.lib.mocker import MATCH |
757 | from juju.providers.dummy import MachineProvider |
758 | from juju.control import main |
759 | |
760 | === modified file 'juju/control/tests/test_expose.py' |
761 | --- juju/control/tests/test_expose.py 2012-05-04 22:43:40 +0000 |
762 | +++ juju/control/tests/test_expose.py 2012-09-15 19:26:18 +0000 |
763 | @@ -1,8 +1,8 @@ |
764 | -import yaml |
765 | from twisted.internet.defer import inlineCallbacks |
766 | |
767 | from juju.control import main |
768 | from juju.control.tests.common import ControlToolTest |
769 | +from juju.lib import serializer |
770 | from juju.state.tests.test_service import ServiceStateManagerTestBase |
771 | |
772 | |
773 | @@ -15,7 +15,7 @@ |
774 | config = { |
775 | "environments": {"firstenv": {"type": "dummy"}}} |
776 | |
777 | - self.write_config(yaml.dump(config)) |
778 | + self.write_config(serializer.dump(config)) |
779 | self.config.load() |
780 | self.service_state = yield self.add_service_from_charm("wordpress") |
781 | self.output = self.capture_logging() |
782 | |
783 | === modified file 'juju/control/tests/test_initialize.py' |
784 | --- juju/control/tests/test_initialize.py 2012-03-30 09:12:11 +0000 |
785 | +++ juju/control/tests/test_initialize.py 2012-09-15 19:26:18 +0000 |
786 | @@ -1,11 +1,11 @@ |
787 | from base64 import b64encode |
788 | -from yaml import safe_dump |
789 | + |
790 | |
791 | from twisted.internet.defer import succeed |
792 | |
793 | from txzookeeper import ZookeeperClient |
794 | from juju.state.initialize import StateHierarchy |
795 | - |
796 | +from juju.lib.serializer import dump |
797 | from juju.control import admin |
798 | from .common import ControlToolTest |
799 | |
800 | @@ -29,7 +29,7 @@ |
801 | self.setup_exit(0) |
802 | self.mocker.replay() |
803 | |
804 | - constraints_data = b64encode(safe_dump({ |
805 | + constraints_data = b64encode(dump({ |
806 | "ubuntu-series": "foo", "provider-type": "bar"})) |
807 | |
808 | admin(["initialize", |
809 | |
810 | === modified file 'juju/control/tests/test_open_tunnel.py' |
811 | --- juju/control/tests/test_open_tunnel.py 2011-09-15 19:24:47 +0000 |
812 | +++ juju/control/tests/test_open_tunnel.py 2012-09-15 19:26:18 +0000 |
813 | @@ -1,7 +1,8 @@ |
814 | -from yaml import dump |
815 | |
816 | +from juju.control import main, open_tunnel |
817 | +from juju.lib.serializer import dump |
818 | from juju.providers.dummy import MachineProvider |
819 | -from juju.control import main, open_tunnel |
820 | + |
821 | |
822 | from .common import ControlToolTest |
823 | |
824 | |
825 | === modified file 'juju/control/tests/test_remove_relation.py' |
826 | --- juju/control/tests/test_remove_relation.py 2012-05-04 22:43:40 +0000 |
827 | +++ juju/control/tests/test_remove_relation.py 2012-09-15 19:26:18 +0000 |
828 | @@ -1,12 +1,12 @@ |
829 | import logging |
830 | |
831 | -import yaml |
832 | |
833 | from twisted.internet.defer import inlineCallbacks, returnValue |
834 | |
835 | from juju.charm.tests.test_repository import RepositoryTestBase |
836 | from juju.control import main, remove_relation |
837 | from juju.control.tests.common import ControlToolTest |
838 | +from juju.lib import serializer |
839 | from juju.machine.tests.test_constraints import dummy_constraints |
840 | from juju.state.errors import ServiceStateNotFound |
841 | from juju.state.tests.test_service import ServiceStateManagerTestBase |
842 | @@ -22,7 +22,7 @@ |
843 | "environments": { |
844 | "firstenv": { |
845 | "type": "dummy", "admin-secret": "homer"}}} |
846 | - self.write_config(yaml.dump(config)) |
847 | + self.write_config(serializer.dump(config)) |
848 | self.config.load() |
849 | self.output = self.capture_logging() |
850 | self.stderr = self.capture_stream("stderr") |
851 | |
852 | === modified file 'juju/control/tests/test_scp.py' |
853 | --- juju/control/tests/test_scp.py 2012-03-27 23:56:09 +0000 |
854 | +++ juju/control/tests/test_scp.py 2012-09-15 19:26:18 +0000 |
855 | @@ -1,7 +1,6 @@ |
856 | import logging |
857 | import os |
858 | |
859 | -from yaml import dump |
860 | |
861 | from twisted.internet.defer import inlineCallbacks |
862 | |
863 | |
864 | === modified file 'juju/control/tests/test_status.py' |
865 | --- juju/control/tests/test_status.py 2012-08-09 09:53:55 +0000 |
866 | +++ juju/control/tests/test_status.py 2012-09-15 19:26:18 +0000 |
867 | @@ -4,7 +4,7 @@ |
868 | import logging |
869 | import os |
870 | from StringIO import StringIO |
871 | -import yaml |
872 | + |
873 | |
874 | from twisted.internet.defer import inlineCallbacks, returnValue |
875 | |
876 | @@ -13,7 +13,7 @@ |
877 | from juju.environment.environment import Environment |
878 | from juju.control import status |
879 | from juju.control import tests |
880 | -from juju.lib.mocker import ANY |
881 | +from juju.lib import serializer |
882 | from juju.state.endpoint import RelationEndpoint |
883 | from juju.state.environment import GlobalSettingsStateManager |
884 | from juju.state.tests.test_service import ServiceStateManagerTestBase |
885 | @@ -24,7 +24,7 @@ |
886 | |
887 | tests_path = os.path.dirname(inspect.getabsfile(tests)) |
888 | sample_path = os.path.join(tests_path, "sample_cluster.yaml") |
889 | -sample_cluster = yaml.load(open(sample_path, "r")) |
890 | +sample_cluster = serializer.load(open(sample_path, "r")) |
891 | |
892 | |
893 | def dump_stringio(stringio, filename): |
894 | @@ -643,7 +643,7 @@ |
895 | |
896 | yield status.status(self.environment, [], |
897 | status.render_yaml, self.output, None) |
898 | - state = yaml.load(self.output.getvalue()) |
899 | + state = serializer.yaml_load(self.output.getvalue()) |
900 | |
901 | self.assertEqual(set(state["machines"].keys()), |
902 | set([0, 1, 2, 3, 4, 5, 6])) |
903 | @@ -863,7 +863,7 @@ |
904 | yield status.status(self.environment, [], |
905 | status.render_yaml, self.output, None) |
906 | |
907 | - state = yaml.load(self.output.getvalue()) |
908 | + state = serializer.load(self.output.getvalue()) |
909 | |
910 | # verify our changes |
911 | log_state = state["services"]["logging"] |
912 | @@ -920,6 +920,6 @@ |
913 | yield status.status(self.environment, [], |
914 | status.render_yaml, self.output, None) |
915 | |
916 | - output = yaml.load(self.output.getvalue()) |
917 | + output = serializer.load(self.output.getvalue()) |
918 | self.assertNotIn(mu1.unit_name, output["services"]["mysql"]["units"]) |
919 | self.assertIn(mu2.unit_name, output["services"]["mysql"]["units"]) |
920 | |
921 | === modified file 'juju/control/tests/test_unexpose.py' |
922 | --- juju/control/tests/test_unexpose.py 2012-05-04 22:43:40 +0000 |
923 | +++ juju/control/tests/test_unexpose.py 2012-09-15 19:26:18 +0000 |
924 | @@ -1,8 +1,8 @@ |
925 | -import yaml |
926 | from twisted.internet.defer import inlineCallbacks |
927 | |
928 | from juju.control import main |
929 | from juju.control.tests.common import ControlToolTest |
930 | +from juju.lib import serializer |
931 | from juju.state.tests.test_service import ServiceStateManagerTestBase |
932 | |
933 | |
934 | @@ -15,7 +15,7 @@ |
935 | config = { |
936 | "environments": {"firstenv": {"type": "dummy"}}} |
937 | |
938 | - self.write_config(yaml.dump(config)) |
939 | + self.write_config(serializer.dump(config)) |
940 | self.config.load() |
941 | self.service_state = yield self.add_service_from_charm("wordpress") |
942 | self.output = self.capture_logging() |
943 | |
944 | === modified file 'juju/control/tests/test_upgrade_charm.py' |
945 | --- juju/control/tests/test_upgrade_charm.py 2012-08-06 19:29:30 +0000 |
946 | +++ juju/control/tests/test_upgrade_charm.py 2012-09-15 19:26:18 +0000 |
947 | @@ -1,6 +1,5 @@ |
948 | import json |
949 | import os |
950 | -from yaml import dump |
951 | |
952 | from twisted.internet.defer import inlineCallbacks, succeed |
953 | |
954 | @@ -11,8 +10,10 @@ |
955 | from juju.control import main |
956 | from juju.errors import FileNotFound |
957 | from juju.environment.environment import Environment |
958 | +from juju.lib.mocker import ANY |
959 | +from juju.lib.serializer import dump |
960 | from juju.unit.workflow import UnitWorkflowState |
961 | -from juju.lib.mocker import ANY |
962 | + |
963 | |
964 | from .common import MachineControlToolTest |
965 | |
966 | |
967 | === modified file 'juju/control/tests/test_utils.py' |
968 | --- juju/control/tests/test_utils.py 2012-03-27 23:56:09 +0000 |
969 | +++ juju/control/tests/test_utils.py 2012-09-15 19:26:18 +0000 |
970 | @@ -1,7 +1,6 @@ |
971 | import os |
972 | |
973 | from twisted.internet.defer import inlineCallbacks, returnValue |
974 | -from yaml import dump |
975 | |
976 | from juju.environment.tests.test_config import EnvironmentsConfigTestBase |
977 | from juju.control.tests.common import ControlToolTest |
978 | @@ -10,6 +9,7 @@ |
979 | expand_path, parse_passthrough_args, ParseError) |
980 | from juju.environment.config import EnvironmentsConfig |
981 | from juju.environment.errors import EnvironmentsConfigError |
982 | +from juju.lib.serializer import yaml_dump as dump |
983 | from juju.lib.testing import TestCase |
984 | from juju.state.errors import ServiceUnitStateMachineNotAssigned |
985 | from juju.state.tests.test_service import ServiceStateManagerTestBase |
986 | |
987 | === modified file 'juju/environment/config.py' |
988 | --- juju/environment/config.py 2012-08-31 15:03:31 +0000 |
989 | +++ juju/environment/config.py 2012-09-15 19:26:18 +0000 |
990 | @@ -5,6 +5,7 @@ |
991 | from juju.environment.environment import Environment |
992 | from juju.environment.errors import EnvironmentsConfigError |
993 | from juju.errors import FileAlreadyExists, FileNotFound |
994 | +from juju.lib import serializer |
995 | from juju.lib.schema import ( |
996 | Constant, Dict, Int, KeyDict, OAuthString, OneOf, SchemaError, SelectDict, |
997 | String, Bool) |
998 | @@ -174,7 +175,7 @@ |
999 | self._fail("Configuration must be a string", path, repr(content)) |
1000 | |
1001 | try: |
1002 | - config = yaml.load(content) |
1003 | + config = serializer.yaml_load(content) |
1004 | except yaml.YAMLError, error: |
1005 | self._fail(error, path=path, content=content) |
1006 | |
1007 | @@ -309,4 +310,4 @@ |
1008 | assert data.keys() == [name] |
1009 | config["environments"].update(data) |
1010 | |
1011 | - return yaml.safe_dump(config) |
1012 | + return serializer.dump(config) |
1013 | |
1014 | === modified file 'juju/environment/tests/test_config.py' |
1015 | --- juju/environment/tests/test_config.py 2012-08-31 15:17:53 +0000 |
1016 | +++ juju/environment/tests/test_config.py 2012-09-15 19:26:18 +0000 |
1017 | @@ -1,5 +1,4 @@ |
1018 | import os |
1019 | -import yaml |
1020 | |
1021 | from twisted.internet.defer import inlineCallbacks |
1022 | |
1023 | @@ -8,8 +7,10 @@ |
1024 | from juju.environment.environment import Environment |
1025 | from juju.environment.errors import EnvironmentsConfigError |
1026 | from juju.errors import FileNotFound, FileAlreadyExists |
1027 | +from juju.lib import serializer |
1028 | from juju.state.environment import EnvironmentStateManager |
1029 | |
1030 | + |
1031 | from juju.lib.testing import TestCase |
1032 | |
1033 | |
1034 | @@ -102,7 +103,7 @@ |
1035 | # self.client. |
1036 | |
1037 | def push_config(self, name, config): |
1038 | - self.write_config(yaml.dump(config)) |
1039 | + self.write_config(serializer.yaml_dump(config)) |
1040 | self.config.load() |
1041 | esm = EnvironmentStateManager(self.client) |
1042 | return esm.set_config_state(self.config, name) |
1043 | @@ -129,8 +130,8 @@ |
1044 | self.assertEquals(self.config.get_default_path(), self.default_path) |
1045 | |
1046 | def compare_config(self, config1, sample_config2): |
1047 | - config1 = yaml.load(config1) |
1048 | - config2 = yaml.load( |
1049 | + config1 = serializer.yaml_load(config1) |
1050 | + config2 = serializer.yaml_load( |
1051 | sample_config2 % config1["environments"]["sample"]) |
1052 | self.assertEqual(config1, config2) |
1053 | |
1054 | @@ -193,7 +194,7 @@ |
1055 | self.assertTrue(os.path.isfile(self.default_path)) |
1056 | |
1057 | with open(self.default_path) as file: |
1058 | - config = yaml.load(file.read()) |
1059 | + config = serializer.yaml_load(file.read()) |
1060 | self.assertEqual( |
1061 | config["environments"]["sample"]["type"], "ec2") |
1062 | self.assertEqual( |
1063 | @@ -373,9 +374,9 @@ |
1064 | get_default() must return the one defined environment, when it's |
1065 | indeed a single one. |
1066 | """ |
1067 | - config = yaml.load(SAMPLE_ENV) |
1068 | + config = serializer.yaml_load(SAMPLE_ENV) |
1069 | del config["environments"]["mysecondenv"] |
1070 | - self.write_config(yaml.dump(config)) |
1071 | + self.write_config(serializer.yaml_dump(config)) |
1072 | self.config.load() |
1073 | env = self.config.get_default() |
1074 | self.assertEquals(env.name, "myfirstenv") |
1075 | @@ -385,9 +386,9 @@ |
1076 | get_default() must otherwise return the environment named |
1077 | through the "default:" option. |
1078 | """ |
1079 | - config = yaml.load(SAMPLE_ENV) |
1080 | + config = serializer.yaml_load(SAMPLE_ENV) |
1081 | config["default"] = "mysecondenv" |
1082 | - self.write_config(yaml.dump(config)) |
1083 | + self.write_config(serializer.yaml_dump(config)) |
1084 | self.config.load() |
1085 | env = self.config.get_default() |
1086 | self.assertEquals(env.name, "mysecondenv") |
1087 | @@ -396,9 +397,9 @@ |
1088 | """ |
1089 | The schema should mention the "default:" option as a string. |
1090 | """ |
1091 | - config = yaml.load(SAMPLE_ENV) |
1092 | + config = serializer.yaml_load(SAMPLE_ENV) |
1093 | config["default"] = 1 |
1094 | - self.write_config(yaml.dump(config)) |
1095 | + self.write_config(serializer.yaml_dump(config)) |
1096 | error = self.assertRaises(EnvironmentsConfigError, self.config.load) |
1097 | self.assertEquals( |
1098 | str(error), |
1099 | @@ -410,10 +411,10 @@ |
1100 | get_default() must raise an error if the environment named through |
1101 | the "default:" option isn't found. |
1102 | """ |
1103 | - config = yaml.load(SAMPLE_ENV) |
1104 | + config = serializer.yaml_load(SAMPLE_ENV) |
1105 | config["default"] = "non-existent" |
1106 | # Use a different path to ensure the error message is right. |
1107 | - self.write_config(yaml.dump(config), other_path=True) |
1108 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1109 | self.config.load(self.other_path) |
1110 | try: |
1111 | self.config.get_default() |
1112 | @@ -447,10 +448,10 @@ |
1113 | contains a machine provider configuration without any type |
1114 | information. |
1115 | """ |
1116 | - config = yaml.load(SAMPLE_ENV) |
1117 | + config = serializer.yaml_load(SAMPLE_ENV) |
1118 | # Delete the type. |
1119 | del config["environments"]["myfirstenv"]["type"] |
1120 | - self.write_config(yaml.dump(config), other_path=True) |
1121 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1122 | |
1123 | try: |
1124 | self.config.load(self.other_path) |
1125 | @@ -468,12 +469,12 @@ |
1126 | self.write_config(SAMPLE_ENV) |
1127 | self.config.load() |
1128 | config = self.config.serialize() |
1129 | - serialized = yaml.load(SAMPLE_ENV) |
1130 | + serialized = serializer.yaml_load(SAMPLE_ENV) |
1131 | |
1132 | for d in serialized["environments"].values(): |
1133 | d["dynamicduck"] = "magic" |
1134 | |
1135 | - self.assertEqual(yaml.load(config), serialized) |
1136 | + self.assertEqual(serializer.yaml_load(config), serialized) |
1137 | |
1138 | def test_serialize_environment(self): |
1139 | """ |
1140 | @@ -484,12 +485,12 @@ |
1141 | self.write_config(SAMPLE_ENV) |
1142 | self.config.load() |
1143 | |
1144 | - data = yaml.load(SAMPLE_ENV) |
1145 | + data = serializer.yaml_load(SAMPLE_ENV) |
1146 | del data["environments"]["mysecondenv"] |
1147 | data["environments"]["myfirstenv"]["dynamicduck"] = "magic" |
1148 | |
1149 | self.assertEqual( |
1150 | - yaml.load(self.config.serialize("myfirstenv")), |
1151 | + serializer.yaml_load(self.config.serialize("myfirstenv")), |
1152 | data) |
1153 | |
1154 | def test_load_serialized_environment(self): |
1155 | @@ -518,9 +519,9 @@ |
1156 | |
1157 | def test_serialize_custom_variables_outside_environment(self): |
1158 | """Serializing captures custom variables out of the environment.""" |
1159 | - data = yaml.load(SAMPLE_ENV) |
1160 | + data = serializer.yaml_load(SAMPLE_ENV) |
1161 | data["default"] = "myfirstenv" |
1162 | - self.write_config(yaml.dump(data)) |
1163 | + self.write_config(serializer.yaml_dump(data)) |
1164 | self.config.load() |
1165 | serialized = self.config.serialize() |
1166 | |
1167 | @@ -543,17 +544,17 @@ |
1168 | self.write_config("\0") |
1169 | error = self.assertRaises(EnvironmentsConfigError, self.config.load) |
1170 | self.assertIn( |
1171 | - "special characters are not allowed", str(error)) |
1172 | + "control characters are not allowed", str(error)) |
1173 | |
1174 | def test_ec2_verifies_region(self): |
1175 | # sample doesn't include credentials |
1176 | self.setup_ec2_credentials() |
1177 | self.config.write_sample() |
1178 | with open(self.default_path) as file: |
1179 | - config = yaml.load(file.read()) |
1180 | + config = serializer.yaml_load(file.read()) |
1181 | config["environments"]["sample"]["region"] = "ap-southeast-2" |
1182 | |
1183 | - self.write_config(yaml.dump(config), other_path=True) |
1184 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1185 | |
1186 | e = self.assertRaises(EnvironmentsConfigError, |
1187 | self.config.load, |
1188 | @@ -562,11 +563,11 @@ |
1189 | str(e)) |
1190 | |
1191 | with open(self.default_path) as file: |
1192 | - config = yaml.load(file.read()) |
1193 | + config = serializer.yaml_load(file.read()) |
1194 | # Authorized keys are required for environment serialization. |
1195 | config["environments"]["sample"]["authorized-keys"] = "mickey" |
1196 | config["environments"]["sample"]["region"] = "ap-southeast-1" |
1197 | - self.write_config(yaml.dump(config), other_path=True) |
1198 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1199 | |
1200 | self.config.load(self.other_path) |
1201 | data = self.config.get_default().get_serialization_data() |
1202 | @@ -575,9 +576,9 @@ |
1203 | def assert_ec2_sample_config(self, delete_key): |
1204 | self.config.write_sample() |
1205 | with open(self.default_path) as file: |
1206 | - config = yaml.load(file.read()) |
1207 | + config = serializer.yaml_load(file.read()) |
1208 | del config["environments"]["sample"][delete_key] |
1209 | - self.write_config(yaml.dump(config), other_path=True) |
1210 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1211 | |
1212 | try: |
1213 | self.config.load(self.other_path) |
1214 | @@ -604,9 +605,9 @@ |
1215 | self.setup_ec2_credentials() |
1216 | self.config.write_sample() |
1217 | with open(self.default_path) as file: |
1218 | - config = yaml.load(file.read()) |
1219 | + config = serializer.yaml_load(file.read()) |
1220 | config["environments"]["sample"]["placement"] = "random" |
1221 | - self.write_config(yaml.dump(config), other_path=True) |
1222 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1223 | |
1224 | e = self.assertRaises(EnvironmentsConfigError, |
1225 | self.config.load, |
1226 | @@ -615,11 +616,11 @@ |
1227 | str(e)) |
1228 | |
1229 | with open(self.default_path) as file: |
1230 | - config = yaml.load(file.read()) |
1231 | + config = serializer.yaml_load(file.read()) |
1232 | # Authorized keys are required for environment serialization. |
1233 | config["environments"]["sample"]["authorized-keys"] = "mickey" |
1234 | config["environments"]["sample"]["placement"] = "local" |
1235 | - self.write_config(yaml.dump(config), other_path=True) |
1236 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1237 | |
1238 | self.config.load(self.other_path) |
1239 | data = self.config.get_default().get_serialization_data() |
1240 | @@ -630,9 +631,9 @@ |
1241 | self.setup_ec2_credentials() |
1242 | self.config.write_sample() |
1243 | with open(self.default_path) as f: |
1244 | - config = yaml.load(f.read()) |
1245 | + config = serializer.yaml_load(f.read()) |
1246 | config["environments"]["sample"]["default-series"] = "astounding" |
1247 | - self.write_config(yaml.dump(config), other_path=True) |
1248 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1249 | |
1250 | self.config.load(self.other_path) |
1251 | |
1252 | @@ -643,9 +644,9 @@ |
1253 | self.setup_ec2_credentials() |
1254 | self.config.write_sample() |
1255 | with open(self.default_path) as f: |
1256 | - config = yaml.load(f.read()) |
1257 | + config = serializer.yaml_load(f.read()) |
1258 | config["environments"]["sample"]["ssl-hostname-verification"] = True |
1259 | - self.write_config(yaml.dump(config), other_path=True) |
1260 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1261 | |
1262 | self.config.load(self.other_path) |
1263 | |
1264 | @@ -658,9 +659,9 @@ |
1265 | "admin-secret acquired-mgmt-class available-mgmt-class " |
1266 | "default-series").split() |
1267 | for require in requires: |
1268 | - config = yaml.load(SAMPLE_ORCHESTRA) |
1269 | + config = serializer.yaml_load(SAMPLE_ORCHESTRA) |
1270 | del config["environments"]["sample"][require] |
1271 | - self.write_config(yaml.dump(config), other_path=True) |
1272 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1273 | |
1274 | try: |
1275 | self.config.load(self.other_path) |
1276 | @@ -675,25 +676,25 @@ |
1277 | % require) |
1278 | |
1279 | def test_orchestra_respects_default_series(self): |
1280 | - config = yaml.load(SAMPLE_ORCHESTRA) |
1281 | + config = serializer.yaml_load(SAMPLE_ORCHESTRA) |
1282 | config["environments"]["sample"]["default-series"] = "magnificent" |
1283 | - self.write_config(yaml.dump(config), other_path=True) |
1284 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1285 | self.config.load(self.other_path) |
1286 | |
1287 | provider = self.config.get_default().get_machine_provider() |
1288 | self.assertEqual(provider.config["default-series"], "magnificent") |
1289 | |
1290 | def test_orchestra_verifies_placement(self): |
1291 | - config = yaml.load(SAMPLE_ORCHESTRA) |
1292 | + config = serializer.yaml_load(SAMPLE_ORCHESTRA) |
1293 | config["environments"]["sample"]["placement"] = "random" |
1294 | - self.write_config(yaml.dump(config), other_path=True) |
1295 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1296 | e = self.assertRaises( |
1297 | EnvironmentsConfigError, self.config.load, self.other_path) |
1298 | self.assertIn("expected 'unassigned', got 'random'", |
1299 | str(e)) |
1300 | |
1301 | config["environments"]["sample"]["placement"] = "local" |
1302 | - self.write_config(yaml.dump(config), other_path=True) |
1303 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1304 | self.config.load(self.other_path) |
1305 | |
1306 | data = self.config.get_default().placement |
1307 | @@ -702,9 +703,9 @@ |
1308 | def test_maas_schema_requires(self): |
1309 | requires = "maas-server maas-oauth admin-secret default-series".split() |
1310 | for require in requires: |
1311 | - config = yaml.load(SAMPLE_MAAS) |
1312 | + config = serializer.yaml_load(SAMPLE_MAAS) |
1313 | del config["environments"]["sample"][require] |
1314 | - self.write_config(yaml.dump(config), other_path=True) |
1315 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1316 | |
1317 | try: |
1318 | self.config.load(self.other_path) |
1319 | @@ -719,9 +720,9 @@ |
1320 | % require) |
1321 | |
1322 | def test_maas_default_series(self): |
1323 | - config = yaml.load(SAMPLE_MAAS) |
1324 | + config = serializer.yaml_load(SAMPLE_MAAS) |
1325 | config["environments"]["sample"]["default-series"] = "magnificent" |
1326 | - self.write_config(yaml.dump(config), other_path=True) |
1327 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1328 | e = self.assertRaises( |
1329 | EnvironmentsConfigError, self.config.load, self.other_path) |
1330 | self.assertIn( |
1331 | @@ -730,16 +731,16 @@ |
1332 | str(e)) |
1333 | |
1334 | def test_maas_verifies_placement(self): |
1335 | - config = yaml.load(SAMPLE_MAAS) |
1336 | + config = serializer.yaml_load(SAMPLE_MAAS) |
1337 | config["environments"]["sample"]["placement"] = "random" |
1338 | - self.write_config(yaml.dump(config), other_path=True) |
1339 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1340 | e = self.assertRaises( |
1341 | EnvironmentsConfigError, self.config.load, self.other_path) |
1342 | self.assertIn("expected 'unassigned', got 'random'", |
1343 | str(e)) |
1344 | |
1345 | config["environments"]["sample"]["placement"] = "local" |
1346 | - self.write_config(yaml.dump(config), other_path=True) |
1347 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1348 | self.config.load(self.other_path) |
1349 | |
1350 | data = self.config.get_default().placement |
1351 | @@ -747,51 +748,51 @@ |
1352 | |
1353 | def test_lxc_requires_data_dir(self): |
1354 | """lxc dev only supports local placement.""" |
1355 | - config = yaml.load(SAMPLE_LOCAL) |
1356 | - self.write_config(yaml.dump(config), other_path=True) |
1357 | + config = serializer.yaml_load(SAMPLE_LOCAL) |
1358 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1359 | error = self.assertRaises( |
1360 | EnvironmentsConfigError, self.config.load, self.other_path) |
1361 | self.assertIn("data-dir: required value not found", str(error)) |
1362 | |
1363 | def test_lxc_verifies_placement(self): |
1364 | """lxc dev only supports local placement.""" |
1365 | - config = yaml.load(SAMPLE_LOCAL) |
1366 | + config = serializer.yaml_load(SAMPLE_LOCAL) |
1367 | config["environments"]["sample"]["placement"] = "unassigned" |
1368 | - self.write_config(yaml.dump(config), other_path=True) |
1369 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1370 | error = self.assertRaises( |
1371 | EnvironmentsConfigError, self.config.load, self.other_path) |
1372 | self.assertIn("expected 'local', got 'unassigned'", str(error)) |
1373 | |
1374 | def test_openstack_requires_default_image_id(self): |
1375 | """A VM image must be supplied for openstack provider.""" |
1376 | - config = yaml.load(SAMPLE_OPENSTACK) |
1377 | + config = serializer.yaml_load(SAMPLE_OPENSTACK) |
1378 | del config["environments"]["sample"]["default-image-id"] |
1379 | - self.write_config(yaml.dump(config), other_path=True) |
1380 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1381 | error = self.assertRaises( |
1382 | EnvironmentsConfigError, self.config.load, self.other_path) |
1383 | self.assertIn("default-image-id: required value not found", str(error)) |
1384 | |
1385 | def test_openstack_ignores_placement(self): |
1386 | """The placement config is not verified for openstack provider.""" |
1387 | - config = yaml.load(SAMPLE_OPENSTACK) |
1388 | + config = serializer.yaml_load(SAMPLE_OPENSTACK) |
1389 | config["environments"]["sample"]["placement"] = "whatever" |
1390 | - self.write_config(yaml.dump(config), other_path=True) |
1391 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1392 | self.config.load(self.other_path) |
1393 | |
1394 | def test_openstack_s3_requires_default_image_id(self): |
1395 | """A VM image must be supplied for openstack_s3 provider.""" |
1396 | - config = yaml.load(SAMPLE_OPENSTACK) |
1397 | + config = serializer.yaml_load(SAMPLE_OPENSTACK) |
1398 | config["environments"]["sample"]["type"] = "openstack_s3" |
1399 | del config["environments"]["sample"]["default-image-id"] |
1400 | - self.write_config(yaml.dump(config), other_path=True) |
1401 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1402 | error = self.assertRaises( |
1403 | EnvironmentsConfigError, self.config.load, self.other_path) |
1404 | self.assertIn("default-image-id: required value not found", str(error)) |
1405 | |
1406 | def test_openstack_s3_ignores_placement(self): |
1407 | """The placement config is not verified for openstack_s3 provider.""" |
1408 | - config = yaml.load(SAMPLE_OPENSTACK) |
1409 | + config = serializer.yaml_load(SAMPLE_OPENSTACK) |
1410 | config["environments"]["sample"]["type"] = "openstack_s3" |
1411 | config["environments"]["sample"]["placement"] = "whatever" |
1412 | - self.write_config(yaml.dump(config), other_path=True) |
1413 | + self.write_config(serializer.yaml_dump(config), other_path=True) |
1414 | self.config.load(self.other_path) |
1415 | |
1416 | === modified file 'juju/hooks/protocol.py' |
1417 | --- juju/hooks/protocol.py 2012-08-30 03:25:04 +0000 |
1418 | +++ juju/hooks/protocol.py 2012-09-15 19:26:18 +0000 |
1419 | @@ -48,9 +48,10 @@ |
1420 | from twisted.internet import defer |
1421 | from twisted.internet import protocol |
1422 | from twisted.protocols import amp |
1423 | -import yaml |
1424 | + |
1425 | |
1426 | from juju.errors import JujuError |
1427 | +from juju.lib import serializer |
1428 | from juju.lib.format import get_charm_formatter, get_charm_formatter_from_env |
1429 | from juju.state.errors import UnitRelationStateNotFound |
1430 | from juju.state.hook import RelationHookContext |
1431 | @@ -310,7 +311,7 @@ |
1432 | |
1433 | # NOTE: no need to consider charm format for blob here, this |
1434 | # blob has always been in YAML format |
1435 | - defer.returnValue(dict(data=yaml.safe_dump(options))) |
1436 | + defer.returnValue(dict(data=serializer.dump(options))) |
1437 | |
1438 | @OpenPortCommand.responder |
1439 | @defer.inlineCallbacks |
1440 | @@ -457,7 +458,7 @@ |
1441 | client_id=client_id, |
1442 | option_name=option_name) |
1443 | # Unbundle and deserialize |
1444 | - result = yaml.safe_load(result["data"]) |
1445 | + result = serializer.load(result["data"]) |
1446 | defer.returnValue(result) |
1447 | |
1448 | @defer.inlineCallbacks |
1449 | |
1450 | === modified file 'juju/hooks/scheduler.py' |
1451 | --- juju/hooks/scheduler.py 2012-03-30 05:17:34 +0000 |
1452 | +++ juju/hooks/scheduler.py 2012-09-15 19:26:18 +0000 |
1453 | @@ -1,10 +1,11 @@ |
1454 | import logging |
1455 | import os |
1456 | -import yaml |
1457 | + |
1458 | |
1459 | from twisted.internet.defer import ( |
1460 | DeferredQueue, inlineCallbacks, succeed, Deferred, |
1461 | QueueUnderflow, QueueOverflow) |
1462 | +from juju.lib import serializer |
1463 | from juju.state.hook import RelationHookContext, RelationChange |
1464 | |
1465 | |
1466 | @@ -202,7 +203,7 @@ |
1467 | |
1468 | def _load_state(self): |
1469 | with open(self._state_path) as f: |
1470 | - state = yaml.load(f.read()) |
1471 | + state = serializer.load(f.read()) |
1472 | if not state: |
1473 | return self._create_state() |
1474 | self._context_members = set(state["context_members"]) |
1475 | @@ -211,7 +212,7 @@ |
1476 | self._run_queue.pending = state["change_queue"] |
1477 | |
1478 | def _save_state(self): |
1479 | - state = yaml.dump({ |
1480 | + state = serializer.dump({ |
1481 | "context_members": sorted(self._context_members), |
1482 | "member_versions": self._member_versions, |
1483 | "change_queue": self._run_queue.pending}) |
1484 | |
1485 | === modified file 'juju/hooks/tests/test_invoker.py' |
1486 | --- juju/hooks/tests/test_invoker.py 2012-09-01 05:22:05 +0000 |
1487 | +++ juju/hooks/tests/test_invoker.py 2012-09-15 19:26:18 +0000 |
1488 | @@ -7,7 +7,6 @@ |
1489 | import os |
1490 | import stat |
1491 | import sys |
1492 | -import yaml |
1493 | |
1494 | from twisted.internet import defer |
1495 | from twisted.internet.process import Process |
1496 | @@ -20,6 +19,7 @@ |
1497 | from juju.hooks import invoker |
1498 | from juju.hooks import commands |
1499 | from juju.hooks.protocol import UnitSettingsFactory |
1500 | +from juju.lib import serializer |
1501 | from juju.lib.mocker import MATCH |
1502 | from juju.lib.twistutils import get_module_directory |
1503 | from juju.state import hook |
1504 | @@ -728,7 +728,7 @@ |
1505 | zk_data = yield self.relation.get_data() |
1506 | self.assertEqual( |
1507 | {"a": "b", "c": "d", "private-address": "mysql-0.example.com"}, |
1508 | - yaml.load(zk_data)) |
1509 | + serializer.load(zk_data)) |
1510 | yield exe.ended |
1511 | self.assertIn( |
1512 | "Flushed values for hook %r on 'database:42'\n" |
1513 | @@ -771,7 +771,7 @@ |
1514 | self.assertEqual( |
1515 | {"new-value": "2", "changed": "abc", "changed2": "xyz", |
1516 | "private-address": "mysql-0.example.com"}, |
1517 | - yaml.load(zk_data)) |
1518 | + serializer.load(zk_data)) |
1519 | |
1520 | # Verify that unicode/strings longer than 100 characters in |
1521 | # representation (including quotes and the u marker) are cut |
1522 | @@ -816,7 +816,7 @@ |
1523 | zk_data = yield self.relation.get_data() |
1524 | self.assertEqual({"no-change": "42", "untouched": "xyz", |
1525 | "private-address": "mysql-0.example.com"}, |
1526 | - yaml.load(zk_data)) |
1527 | + serializer.load(zk_data)) |
1528 | self.assertNotIn( |
1529 | "Flushed values for hook 'set-does-nothing'", |
1530 | output.getvalue()) |
1531 | @@ -960,7 +960,7 @@ |
1532 | internal_unit_id = (yield context.get_local_unit_state()).internal_id |
1533 | path = yield context.get_settings_path(internal_unit_id) |
1534 | data, stat = yield self.client.get(path) |
1535 | - self.assertEqual(yaml.load(data), expected) |
1536 | + self.assertEqual(serializer.load(data), expected) |
1537 | |
1538 | @defer.inlineCallbacks |
1539 | def test_implied_relation_hook_context(self): |
1540 | |
1541 | === modified file 'juju/hooks/tests/test_scheduler.py' |
1542 | --- juju/hooks/tests/test_scheduler.py 2012-03-28 02:55:12 +0000 |
1543 | +++ juju/hooks/tests/test_scheduler.py 2012-09-15 19:26:18 +0000 |
1544 | @@ -1,10 +1,9 @@ |
1545 | import logging |
1546 | import os |
1547 | -import yaml |
1548 | |
1549 | from twisted.internet.defer import ( |
1550 | inlineCallbacks, fail, succeed, Deferred, returnValue) |
1551 | - |
1552 | +from juju.lib import serializer |
1553 | from juju.hooks.scheduler import HookScheduler |
1554 | from juju.state.tests.test_service import ServiceStateManagerTestBase |
1555 | |
1556 | @@ -44,7 +43,7 @@ |
1557 | |
1558 | def write_single_unit_state(self): |
1559 | with open(self.state_file, "w") as f: |
1560 | - f.write(yaml.dump({ |
1561 | + f.write(serializer.dump({ |
1562 | "context_members": ["u-1"], |
1563 | "member_versions": {"u-1": 0}, |
1564 | "unit_ops": {}, |
1565 | @@ -82,7 +81,7 @@ |
1566 | yield sched_done |
1567 | self.assertFalse(self.scheduler.running) |
1568 | with open(self.state_file) as f: |
1569 | - self.assertEquals(yaml.load(f.read()), { |
1570 | + self.assertEquals(serializer.load(f.read()), { |
1571 | "context_members": ['u-1'], |
1572 | "member_versions": {"u-1": 0}, |
1573 | "change_queue": [ |
1574 | @@ -118,7 +117,7 @@ |
1575 | self.assertFalse(collected) |
1576 | |
1577 | with open(self.state_file) as f: |
1578 | - self.assertEquals(yaml.load(f.read()), { |
1579 | + self.assertEquals(serializer.load(f.read()), { |
1580 | "context_members": [], |
1581 | "member_versions": {}, |
1582 | "change_queue": [ |
1583 | @@ -163,7 +162,7 @@ |
1584 | self.scheduler.cb_change_members(["u-1"], ["u-2"]) |
1585 | |
1586 | with open(self.state_file) as f: |
1587 | - self.assertEquals(yaml.load(f.read()), { |
1588 | + self.assertEquals(serializer.load(f.read()), { |
1589 | "context_members": ['u-2'], |
1590 | "member_versions": {"u-2": 0}, |
1591 | "change_queue": [ |
1592 | @@ -200,7 +199,7 @@ |
1593 | self.assertFalse(self.scheduler.running) |
1594 | |
1595 | with open(self.state_file) as f: |
1596 | - self.assertEquals(yaml.load(f.read()), { |
1597 | + self.assertEquals(serializer.load(f.read()), { |
1598 | "context_members": ['u-2'], |
1599 | "member_versions": {"u-2": 0}, |
1600 | "change_queue": [ |
1601 | @@ -318,7 +317,7 @@ |
1602 | ("u-1", "modified", ["u-1"])]) |
1603 | |
1604 | with open(self.state_file) as f: |
1605 | - self.assertEquals(yaml.load(f.read()), { |
1606 | + self.assertEquals(serializer.load(f.read()), { |
1607 | "context_members": ['u-1'], |
1608 | "member_versions": {"u-1": 0}, |
1609 | "change_queue": []}) |
1610 | @@ -468,7 +467,7 @@ |
1611 | |
1612 | def test_empty_state(self): |
1613 | with open(self.state_file, "w") as f: |
1614 | - f.write(yaml.dump({})) |
1615 | + f.write(serializer.dump({})) |
1616 | |
1617 | # Induce lazy creation to verify it can still survive |
1618 | self.scheduler |
1619 | @@ -527,7 +526,7 @@ |
1620 | it hasn't previously been given a notify of before. |
1621 | """ |
1622 | with open(self.state_file, "w") as f: |
1623 | - f.write(yaml.dump({ |
1624 | + f.write(serializer.dump({ |
1625 | "context_members": ["u-1", "u-2"], |
1626 | "member_versions": {"u-1": 0, "u-2": 0}, |
1627 | "change_queue": []})) |
1628 | @@ -559,7 +558,7 @@ |
1629 | self.assertEqual(members, ["u-2", "u-3", "u-4"]) |
1630 | |
1631 | with open(self.state_file) as f: |
1632 | - state = yaml.load(f.read()) |
1633 | + state = serializer.load(f.read()) |
1634 | self.assertEquals(state, { |
1635 | "change_queue": [], |
1636 | "context_members": ["u-2", "u-3", "u-4"], |
1637 | @@ -568,7 +567,7 @@ |
1638 | @inlineCallbacks |
1639 | def test_state_is_loaded(self): |
1640 | with open(self.state_file, "w") as f: |
1641 | - f.write(yaml.dump({ |
1642 | + f.write(serializer.dump({ |
1643 | "context_members": ["u-1", "u-2", "u-3"], |
1644 | "member_versions": {"u-1": 5, "u-2": 2, "u-3": 0}, |
1645 | "change_queue": [ |
1646 | @@ -594,7 +593,7 @@ |
1647 | self.assertEqual(members, ["u-1", "u-2", "u-3"]) |
1648 | |
1649 | with open(self.state_file) as f: |
1650 | - state = yaml.load(f.read()) |
1651 | + state = serializer.load(f.read()) |
1652 | self.assertEquals(state, { |
1653 | "context_members": ["u-1", "u-2", "u-3"], |
1654 | "member_versions": {"u-1": 5, "u-2": 2, "u-3": 0}, |
1655 | @@ -602,7 +601,7 @@ |
1656 | |
1657 | def test_state_is_stored(self): |
1658 | with open(self.state_file, "w") as f: |
1659 | - f.write(yaml.dump({ |
1660 | + f.write(serializer.dump({ |
1661 | "context_members": ["u-1", "u-2"], |
1662 | "member_versions": {"u-1": 0, "u-2": 2}, |
1663 | "change_queue": []})) |
1664 | @@ -614,7 +613,7 @@ |
1665 | self.scheduler.stop() |
1666 | |
1667 | with open(self.state_file) as f: |
1668 | - state = yaml.load(f.read()) |
1669 | + state = serializer.load(f.read()) |
1670 | self.assertEquals(state, { |
1671 | "context_members": ["u-2", "u-3"], |
1672 | "member_versions": {"u-2": 3, "u-3": 0}, |
1673 | @@ -641,7 +640,7 @@ |
1674 | self.executor = execute |
1675 | |
1676 | with open(self.state_file, "w") as f: |
1677 | - f.write(yaml.dump({ |
1678 | + f.write(serializer.dump({ |
1679 | "context_members": ["u-1", "u-2"], |
1680 | "member_versions": {"u-1": 1, "u-2": 0, "u-3": 0}, |
1681 | "change_queue": [ |
1682 | @@ -655,7 +654,7 @@ |
1683 | yield self.poke_zk() |
1684 | yield self.assertFailure(d, SomeError) |
1685 | with open(self.state_file) as f: |
1686 | - self.assertEquals(yaml.load(f.read()), { |
1687 | + self.assertEquals(serializer.load(f.read()), { |
1688 | "context_members": ["u-1", "u-2"], |
1689 | "member_versions": {"u-1": 1, "u-2": 0, "u-3": 0}, |
1690 | "change_queue": [ |
1691 | @@ -681,14 +680,14 @@ |
1692 | "members":["u-1", "u-2", "u-3"]}, |
1693 | ]} |
1694 | with open(self.state_file, "w") as f: |
1695 | - f.write(yaml.dump(initial_state)) |
1696 | + f.write(serializer.dump(initial_state)) |
1697 | |
1698 | d = self.scheduler.run() |
1699 | while not self.execute_called: |
1700 | yield self.poke_zk() |
1701 | yield self.assertFailure(d, SomeError) |
1702 | with open(self.state_file) as f: |
1703 | - self.assertEquals(yaml.load(f.read()), initial_state) |
1704 | + self.assertEquals(serializer.load(f.read()), initial_state) |
1705 | |
1706 | def test_ignore_equal_settings_version(self): |
1707 | """ |
1708 | @@ -718,7 +717,7 @@ |
1709 | on old_units. |
1710 | """ |
1711 | with open(self.state_file, "w") as f: |
1712 | - f.write(yaml.dump({ |
1713 | + f.write(serializer.dump({ |
1714 | "context_members": ["u-1", "u-2"], |
1715 | "member_versions": {"u-1": 0, "u-2": 0}, |
1716 | "change_queue": []})) |
1717 | |
1718 | === modified file 'juju/lib/format.py' |
1719 | --- juju/lib/format.py 2012-09-01 05:22:05 +0000 |
1720 | +++ juju/lib/format.py 2012-09-15 19:26:18 +0000 |
1721 | @@ -90,9 +90,9 @@ |
1722 | # output format, False |
1723 | if data is None: |
1724 | return "" |
1725 | - serialized = yaml.safe_dump( |
1726 | + serialized = yaml.dump( |
1727 | data, indent=4, default_flow_style=False, width=80, |
1728 | - allow_unicode=True) |
1729 | + allow_unicode=True, Dumper=yaml.CSafeDumper) |
1730 | if serialized.endswith("\n...\n"): |
1731 | # Remove explicit doc end sentinel, still valid yaml |
1732 | serialized = serialized[0:-5] |
1733 | @@ -111,7 +111,7 @@ |
1734 | |
1735 | def load(self, data): |
1736 | """Loads data safely, ensuring no Python specific type info leaks""" |
1737 | - return yaml.safe_load(data) |
1738 | + return yaml.load(data, Loader=yaml.CSafeLoader) |
1739 | |
1740 | |
1741 | def is_valid_charm_format(charm_format): |
1742 | |
1743 | === added file 'juju/lib/serializer.py' |
1744 | --- juju/lib/serializer.py 1970-01-01 00:00:00 +0000 |
1745 | +++ juju/lib/serializer.py 2012-09-15 19:26:18 +0000 |
1746 | @@ -0,0 +1,21 @@ |
1747 | +from yaml import CSafeLoader, CSafeDumper, Mark |
1748 | +from yaml import dump as _dump |
1749 | +from yaml import load as _load |
1750 | + |
1751 | +def dump(value): |
1752 | + return _dump(value, Dumper=CSafeDumper) |
1753 | + |
1754 | +yaml_dump = dump |
1755 | + |
1756 | +def load(value): |
1757 | + return _load(value, Loader=CSafeLoader) |
1758 | + |
1759 | +yaml_load = load |
1760 | + |
1761 | +def yaml_mark_with_path(path, mark): |
1762 | + # yaml c ext, cant be modded, convert to capture path |
1763 | + return Mark( |
1764 | + path, mark.index, |
1765 | + mark.line, mark.column, |
1766 | + mark.buffer, mark.pointer) |
1767 | + |
1768 | |
1769 | === modified file 'juju/providers/common/cloudinit.py' |
1770 | --- juju/providers/common/cloudinit.py 2012-05-29 15:23:08 +0000 |
1771 | +++ juju/providers/common/cloudinit.py 2012-09-15 19:26:18 +0000 |
1772 | @@ -1,9 +1,10 @@ |
1773 | from base64 import b64encode |
1774 | from subprocess import Popen, PIPE |
1775 | -from yaml import safe_dump |
1776 | + |
1777 | |
1778 | from juju.errors import CloudInitError |
1779 | from juju.lib.upstart import UpstartService |
1780 | +from juju.lib import serializer |
1781 | from juju.providers.common.utils import format_cloud_init |
1782 | from juju.state.auth import make_identity |
1783 | import juju |
1784 | @@ -41,7 +42,7 @@ |
1785 | " --constraints-data=%s" |
1786 | " --provider-type=%s" |
1787 | % (instance_id, make_identity("admin:%s" % secret), |
1788 | - b64encode(safe_dump(constraints.data)), provider_type)] |
1789 | + b64encode(serializer.dump(constraints.data)), provider_type)] |
1790 | |
1791 | |
1792 | def _machine_scripts(machine_id, zookeeper_hosts): |
1793 | |
1794 | === modified file 'juju/providers/common/state.py' |
1795 | --- juju/providers/common/state.py 2011-09-15 18:50:23 +0000 |
1796 | +++ juju/providers/common/state.py 2012-09-15 19:26:18 +0000 |
1797 | @@ -1,5 +1,6 @@ |
1798 | from cStringIO import StringIO |
1799 | -from yaml import safe_dump, load |
1800 | +from juju.lib import serializer |
1801 | + |
1802 | |
1803 | from juju.errors import FileNotFound |
1804 | |
1805 | @@ -28,7 +29,7 @@ |
1806 | return d |
1807 | |
1808 | def _deserialize(self, data): |
1809 | - return load(data.read()) or False |
1810 | + return serializer.load(data.read()) or False |
1811 | |
1812 | def _no_data(self, failure): |
1813 | failure.trap(FileNotFound) |
1814 | @@ -51,5 +52,5 @@ |
1815 | :param dict state: state to save. |
1816 | """ |
1817 | storage = self._provider.get_file_storage() |
1818 | - data = safe_dump(state) |
1819 | + data = serializer.dump(state) |
1820 | return storage.put(_STATE_FILE, StringIO(data)) |
1821 | |
1822 | === modified file 'juju/providers/common/tests/test_cloudinit.py' |
1823 | --- juju/providers/common/tests/test_cloudinit.py 2012-05-22 22:08:15 +0000 |
1824 | +++ juju/providers/common/tests/test_cloudinit.py 2012-09-15 19:26:18 +0000 |
1825 | @@ -1,9 +1,9 @@ |
1826 | import os |
1827 | import stat |
1828 | |
1829 | -import yaml |
1830 | |
1831 | from juju.errors import CloudInitError |
1832 | +from juju.lib import serializer |
1833 | from juju.lib.testing import TestCase |
1834 | from juju.machine.tests.test_constraints import dummy_cs |
1835 | from juju.providers.common.cloudinit import ( |
1836 | @@ -46,10 +46,10 @@ |
1837 | |
1838 | def assert_render(self, cloud_init, name): |
1839 | with open(os.path.join(DATA_DIR, name)) as f: |
1840 | - expected = yaml.load(f.read()) |
1841 | + expected = serializer.load(f.read()) |
1842 | rendered = cloud_init.render() |
1843 | self.assertTrue(rendered.startswith("#cloud-config")) |
1844 | - self.assertEquals(yaml.load(rendered), expected) |
1845 | + self.assertEquals(serializer.load(rendered), expected) |
1846 | |
1847 | def test_render_validate_normal(self): |
1848 | cloud_init = CloudInit() |
1849 | |
1850 | === modified file 'juju/providers/common/tests/test_findzookeepers.py' |
1851 | --- juju/providers/common/tests/test_findzookeepers.py 2012-07-18 19:46:53 +0000 |
1852 | +++ juju/providers/common/tests/test_findzookeepers.py 2012-09-15 19:26:18 +0000 |
1853 | @@ -1,9 +1,9 @@ |
1854 | from cStringIO import StringIO |
1855 | -from yaml import dump |
1856 | |
1857 | from twisted.internet.defer import fail, succeed |
1858 | |
1859 | from juju.errors import EnvironmentNotFound, MachinesNotFound |
1860 | +from juju.lib import serializer |
1861 | from juju.lib.testing import TestCase |
1862 | from juju.providers.common.base import MachineProviderBase |
1863 | |
1864 | @@ -21,7 +21,7 @@ |
1865 | |
1866 | def get(self, path): |
1867 | test.assertEquals(path, "provider-state") |
1868 | - return succeed(StringIO(dump(state))) |
1869 | + return succeed(StringIO(serializer.dump(state))) |
1870 | |
1871 | class DummyProvider(MachineProviderBase): |
1872 | |
1873 | |
1874 | === modified file 'juju/providers/common/tests/test_state.py' |
1875 | --- juju/providers/common/tests/test_state.py 2011-09-15 18:50:23 +0000 |
1876 | +++ juju/providers/common/tests/test_state.py 2012-09-15 19:26:18 +0000 |
1877 | @@ -1,10 +1,10 @@ |
1878 | from cStringIO import StringIO |
1879 | -from yaml import dump |
1880 | |
1881 | from twisted.internet.defer import fail, succeed |
1882 | |
1883 | from juju.errors import FileNotFound |
1884 | from juju.lib.mocker import MATCH |
1885 | +from juju.lib import serializer |
1886 | from juju.lib.testing import TestCase |
1887 | from juju.providers.common.base import MachineProviderBase |
1888 | from juju.providers.common.state import LoadState, SaveState |
1889 | @@ -39,7 +39,8 @@ |
1890 | return load_state.run() |
1891 | |
1892 | def test_load_state(self): |
1893 | - d = self.mock_get(succeed(StringIO(dump({"some": "thing"})))) |
1894 | + d = self.mock_get(succeed(StringIO( |
1895 | + serializer.dump({"some": "thing"})))) |
1896 | |
1897 | def verify(result): |
1898 | self.assertEquals(result, {"some": "thing"}) |
1899 | @@ -74,7 +75,7 @@ |
1900 | class SaveStateTest(TestCase): |
1901 | |
1902 | def is_expected_yaml(self, result): |
1903 | - return result.read() == dump({"some": "thing"}) |
1904 | + return result.read() == serializer.dump({"some": "thing"}) |
1905 | |
1906 | def test_save(self): |
1907 | put = self.mocker.mock() |
1908 | |
1909 | === modified file 'juju/providers/common/tests/test_utils.py' |
1910 | --- juju/providers/common/tests/test_utils.py 2012-08-23 16:14:42 +0000 |
1911 | +++ juju/providers/common/tests/test_utils.py 2012-09-15 19:26:18 +0000 |
1912 | @@ -1,10 +1,10 @@ |
1913 | import os |
1914 | -from yaml import load |
1915 | |
1916 | from twisted.python.failure import Failure |
1917 | |
1918 | from juju.environment.tests.test_config import EnvironmentsConfigTestBase |
1919 | from juju.errors import ProviderInteractionError, JujuError |
1920 | +from juju.lib import serializer |
1921 | from juju.lib.testing import TestCase |
1922 | from juju.providers.common.utils import ( |
1923 | convert_unknown_error, format_cloud_init, get_user_authorized_keys) |
1924 | @@ -107,7 +107,7 @@ |
1925 | |
1926 | lines = output.split("\n") |
1927 | self.assertEqual(lines.pop(0), "#cloud-config") |
1928 | - config = load("\n".join(lines)) |
1929 | + config = serializer.load("\n".join(lines)) |
1930 | self.assertEqual(config["ssh_authorized_keys"], ["zebra"]) |
1931 | self.assertTrue(config["apt_update"]) |
1932 | self.assertTrue(config["apt_upgrade"]) |
1933 | |
1934 | === modified file 'juju/providers/common/utils.py' |
1935 | --- juju/providers/common/utils.py 2012-08-23 16:14:42 +0000 |
1936 | +++ juju/providers/common/utils.py 2012-09-15 19:26:18 +0000 |
1937 | @@ -1,9 +1,10 @@ |
1938 | import logging |
1939 | import os |
1940 | -from yaml import safe_dump |
1941 | + |
1942 | |
1943 | from twisted.python.failure import Failure |
1944 | |
1945 | +from juju.lib import serializer |
1946 | from juju.errors import JujuError, ProviderInteractionError |
1947 | |
1948 | log = logging.getLogger("juju.common") |
1949 | @@ -136,6 +137,6 @@ |
1950 | if scripts: |
1951 | cloud_config["runcmd"] = scripts |
1952 | |
1953 | - output = safe_dump(cloud_config) |
1954 | + output = serializer.dump(cloud_config) |
1955 | output = "#cloud-config\n%s" % (output) |
1956 | return output |
1957 | |
1958 | === modified file 'juju/providers/ec2/tests/common.py' |
1959 | --- juju/providers/ec2/tests/common.py 2012-07-05 21:49:12 +0000 |
1960 | +++ juju/providers/ec2/tests/common.py 2012-09-15 19:26:18 +0000 |
1961 | @@ -1,4 +1,3 @@ |
1962 | -from yaml import dump |
1963 | |
1964 | from twisted.internet.defer import fail, succeed, inlineCallbacks, returnValue |
1965 | |
1966 | @@ -8,6 +7,7 @@ |
1967 | from txaws.ec2.exception import EC2Error |
1968 | from txaws.ec2.model import Instance, Reservation, SecurityGroup |
1969 | |
1970 | +from juju.lib import serializer |
1971 | from juju.lib.mocker import KWARGS, MATCH |
1972 | from juju.providers.ec2 import MachineProvider |
1973 | from juju.providers.ec2.machine import EC2ProviderMachine |
1974 | @@ -156,7 +156,7 @@ |
1975 | self.mocker.result(fail(error)) |
1976 | return |
1977 | |
1978 | - state = dump({ |
1979 | + state = serializer.dump({ |
1980 | "zookeeper-instances": |
1981 | [i.instance_id for i in hosts]}) |
1982 | |
1983 | |
1984 | === modified file 'juju/providers/ec2/tests/test_bootstrap.py' |
1985 | --- juju/providers/ec2/tests/test_bootstrap.py 2012-03-29 01:37:57 +0000 |
1986 | +++ juju/providers/ec2/tests/test_bootstrap.py 2012-09-15 19:26:18 +0000 |
1987 | @@ -1,12 +1,11 @@ |
1988 | import logging |
1989 | import os |
1990 | |
1991 | -import yaml |
1992 | - |
1993 | from twisted.internet.defer import succeed, inlineCallbacks |
1994 | |
1995 | from txaws.ec2.model import SecurityGroup |
1996 | |
1997 | +from juju.lib import serializer |
1998 | from juju.lib.mocker import MATCH |
1999 | from juju.lib.testing import TestCase |
2000 | from juju.providers.ec2.machine import EC2ProviderMachine |
2001 | @@ -40,8 +39,8 @@ |
2002 | def verify_user_data(data): |
2003 | expect_path = os.path.join(DATA_DIR, "bootstrap_cloud_init") |
2004 | with open(expect_path) as f: |
2005 | - expect_cloud_init = yaml.load(f.read()) |
2006 | - self.assertEquals(yaml.load(data), expect_cloud_init) |
2007 | + expect_cloud_init = serializer.load(f.read()) |
2008 | + self.assertEquals(serializer.load(data), expect_cloud_init) |
2009 | return True |
2010 | |
2011 | self.ec2.run_instances( |
2012 | @@ -109,7 +108,7 @@ |
2013 | If the provider bootstrap is run when there is already a running |
2014 | bootstrap instance, it will just return the existing machine. |
2015 | """ |
2016 | - state = yaml.dump({"zookeeper-instances": ["i-foobar"]}) |
2017 | + state = serializer.dump({"zookeeper-instances": ["i-foobar"]}) |
2018 | self.s3.get_object(self.env_name, "provider-state") |
2019 | self.mocker.result(succeed(state)) |
2020 | self.ec2.describe_instances("i-foobar") |
2021 | |
2022 | === modified file 'juju/providers/ec2/tests/test_findzookeeper.py' |
2023 | --- juju/providers/ec2/tests/test_findzookeeper.py 2011-09-15 18:50:23 +0000 |
2024 | +++ juju/providers/ec2/tests/test_findzookeeper.py 2012-09-15 19:26:18 +0000 |
2025 | @@ -3,9 +3,8 @@ |
2026 | from txaws.ec2.exception import EC2Error |
2027 | from txaws.s3.exception import S3Error |
2028 | |
2029 | -from yaml import dump |
2030 | - |
2031 | from juju.errors import EnvironmentNotFound |
2032 | +from juju.lib.serializer import dump |
2033 | from juju.lib.testing import TestCase |
2034 | from juju.providers.ec2.machine import EC2ProviderMachine |
2035 | from juju.providers.ec2.tests.common import EC2TestMixin |
2036 | |
2037 | === modified file 'juju/providers/ec2/tests/test_launch.py' |
2038 | --- juju/providers/ec2/tests/test_launch.py 2012-07-18 20:04:16 +0000 |
2039 | +++ juju/providers/ec2/tests/test_launch.py 2012-09-15 19:26:18 +0000 |
2040 | @@ -1,7 +1,5 @@ |
2041 | import os |
2042 | |
2043 | -import yaml |
2044 | - |
2045 | from twisted.internet.defer import inlineCallbacks, succeed |
2046 | |
2047 | from txaws.ec2.model import Instance, SecurityGroup |
2048 | @@ -9,6 +7,7 @@ |
2049 | from juju.errors import EnvironmentNotFound, ProviderError |
2050 | from juju.providers.ec2.machine import EC2ProviderMachine |
2051 | |
2052 | +from juju.lib import serializer |
2053 | from juju.lib.testing import TestCase |
2054 | from juju.lib.mocker import MATCH |
2055 | |
2056 | @@ -37,8 +36,8 @@ |
2057 | def verify_user_data(data): |
2058 | expect_path = os.path.join(DATA_DIR, cloud_init) |
2059 | with open(expect_path) as f: |
2060 | - expect_cloud_init = yaml.load(f.read()) |
2061 | - self.assertEquals(yaml.load(data), expect_cloud_init) |
2062 | + expect_cloud_init = serializer.load(f.read()) |
2063 | + self.assertEquals(serializer.load(data), expect_cloud_init) |
2064 | return True |
2065 | |
2066 | self.ec2.run_instances( |
2067 | |
2068 | === modified file 'juju/providers/ec2/tests/test_provider.py' |
2069 | --- juju/providers/ec2/tests/test_provider.py 2012-08-29 16:36:54 +0000 |
2070 | +++ juju/providers/ec2/tests/test_provider.py 2012-09-15 19:26:18 +0000 |
2071 | @@ -7,6 +7,7 @@ |
2072 | from juju.providers import ec2 as _mod_provider |
2073 | from juju.providers.ec2 import MachineProvider |
2074 | from juju.providers.ec2.files import FileStorage |
2075 | + |
2076 | import logging |
2077 | |
2078 | from .common import EC2TestMixin |
2079 | |
2080 | === modified file 'juju/providers/ec2/tests/test_state.py' |
2081 | --- juju/providers/ec2/tests/test_state.py 2011-09-15 18:50:23 +0000 |
2082 | +++ juju/providers/ec2/tests/test_state.py 2012-09-15 19:26:18 +0000 |
2083 | @@ -1,9 +1,8 @@ |
2084 | -from yaml import dump |
2085 | - |
2086 | from twisted.internet.defer import succeed, fail |
2087 | |
2088 | from txaws.s3.exception import S3Error |
2089 | |
2090 | +from juju.lib import serializer |
2091 | from juju.lib.testing import TestCase |
2092 | from juju.providers.ec2.tests.common import EC2TestMixin |
2093 | |
2094 | @@ -20,7 +19,7 @@ |
2095 | the machine, it will serialize the data to an s3 bucket. |
2096 | """ |
2097 | instances = [self.get_instance("i-foobar", dns_name="x1.example.com")] |
2098 | - state = dump( |
2099 | + state = serializer.dump( |
2100 | {"zookeeper-instances": |
2101 | [[i.instance_id, i.dns_name] for i in instances]}) |
2102 | self.s3.put_object( |
2103 | @@ -46,7 +45,7 @@ |
2104 | already exist. |
2105 | """ |
2106 | instances = [self.get_instance("i-foobar", dns_name="x1.example.com")] |
2107 | - state = dump( |
2108 | + state = serializer.dump( |
2109 | {"zookeeper-instances": |
2110 | [[i.instance_id, i.dns_name] for i in instances]}) |
2111 | self.s3.put_object( |
2112 | @@ -78,7 +77,8 @@ |
2113 | s3. |
2114 | """ |
2115 | self.s3.get_object(self.env_name, "provider-state") |
2116 | - self.mocker.result(succeed(dump({"zookeeper-instances": []}))) |
2117 | + self.mocker.result( |
2118 | + succeed(serializer.dump({"zookeeper-instances": []}))) |
2119 | self.mocker.replay() |
2120 | |
2121 | provider = self.get_provider() |
2122 | @@ -116,7 +116,7 @@ |
2123 | handles the scenario where there is no saved state. |
2124 | """ |
2125 | self.s3.get_object(self.env_name, "provider-state") |
2126 | - self.mocker.result(succeed(dump([]))) |
2127 | + self.mocker.result(succeed(serializer.dump([]))) |
2128 | self.mocker.replay() |
2129 | |
2130 | provider = self.get_provider() |
2131 | |
2132 | === modified file 'juju/providers/local/files.py' |
2133 | --- juju/providers/local/files.py 2012-01-13 00:15:07 +0000 |
2134 | +++ juju/providers/local/files.py 2012-09-15 19:26:18 +0000 |
2135 | @@ -1,13 +1,12 @@ |
2136 | -from getpass import getuser |
2137 | import os |
2138 | from StringIO import StringIO |
2139 | -import yaml |
2140 | |
2141 | from twisted.internet.defer import inlineCallbacks, returnValue |
2142 | from twisted.internet.error import ConnectionRefusedError |
2143 | from twisted.web.client import getPage |
2144 | |
2145 | from juju.errors import ProviderError, FileNotFound |
2146 | +from juju.lib import serializer |
2147 | from juju.lib.upstart import UpstartService |
2148 | from juju.providers.common.files import FileStorage |
2149 | |
2150 | @@ -79,7 +78,7 @@ |
2151 | storage = LocalStorage(self._storage_dir) |
2152 | yield storage.put( |
2153 | SERVER_URL_KEY, |
2154 | - StringIO(yaml.safe_dump( |
2155 | + StringIO(serializer.dump( |
2156 | {"storage-url": "http://%s:%s/" % (self._host, self._port)}))) |
2157 | |
2158 | self._service.set_command(" ".join(self._service_args)) |
2159 | @@ -112,5 +111,5 @@ |
2160 | |
2161 | if not storage_data or not "storage-url" in storage_data: |
2162 | raise ProviderError("Storage not initialized") |
2163 | - url = yaml.load(storage_data)["storage-url"] |
2164 | + url = serializer.load(storage_data)["storage-url"] |
2165 | returnValue(url + key) |
2166 | |
2167 | === modified file 'juju/providers/local/tests/test_agent.py' |
2168 | --- juju/providers/local/tests/test_agent.py 2012-08-03 12:28:29 +0000 |
2169 | +++ juju/providers/local/tests/test_agent.py 2012-09-15 19:26:18 +0000 |
2170 | @@ -1,9 +1,8 @@ |
2171 | import os |
2172 | import tempfile |
2173 | import subprocess |
2174 | -import sys |
2175 | |
2176 | -from twisted.internet.defer import inlineCallbacks, succeed |
2177 | +from twisted.internet.defer import inlineCallbacks |
2178 | |
2179 | from juju.lib.lxc.tests.test_lxc import uses_sudo |
2180 | from juju.lib.testing import TestCase |
2181 | |
2182 | === modified file 'juju/providers/local/tests/test_container.py' |
2183 | --- juju/providers/local/tests/test_container.py 2011-09-30 03:20:11 +0000 |
2184 | +++ juju/providers/local/tests/test_container.py 2012-09-15 19:26:18 +0000 |
2185 | @@ -3,6 +3,7 @@ |
2186 | from juju.lib.testing import TestCase |
2187 | from juju.lib.lxc import get_containers |
2188 | from juju.lib import lxc |
2189 | + |
2190 | lxc_output_sample = "\ |
2191 | calendarserver\ncaprica\ngemini\nreconnoiter\nvirgo\ncalendarserver\n" |
2192 | |
2193 | |
2194 | === modified file 'juju/providers/local/tests/test_files.py' |
2195 | --- juju/providers/local/tests/test_files.py 2012-01-27 17:02:48 +0000 |
2196 | +++ juju/providers/local/tests/test_files.py 2012-09-15 19:26:18 +0000 |
2197 | @@ -2,12 +2,12 @@ |
2198 | import signal |
2199 | from StringIO import StringIO |
2200 | import subprocess |
2201 | -import yaml |
2202 | |
2203 | from twisted.internet.defer import inlineCallbacks, succeed |
2204 | from twisted.web.client import getPage |
2205 | |
2206 | from juju.errors import ProviderError, ServiceError |
2207 | +from juju.lib import serializer |
2208 | from juju.lib.lxc.tests.test_lxc import uses_sudo |
2209 | from juju.lib.testing import TestCase |
2210 | from juju.lib.upstart import UpstartService |
2211 | @@ -177,7 +177,8 @@ |
2212 | yield self.assertFailure(self._storage.get_url("abc"), ProviderError) |
2213 | self._storage.put( |
2214 | SERVER_URL_KEY, |
2215 | - StringIO(yaml.dump({"storage-url": "http://localhost/"}))) |
2216 | + StringIO( |
2217 | + serializer.dump({"storage-url": "http://localhost/"}))) |
2218 | |
2219 | self.assertEqual((yield self._storage.get_url("abc")), |
2220 | "http://localhost/abc") |
2221 | |
2222 | === modified file 'juju/providers/openstack/tests/test_bootstrap.py' |
2223 | --- juju/providers/openstack/tests/test_bootstrap.py 2012-07-20 15:45:56 +0000 |
2224 | +++ juju/providers/openstack/tests/test_bootstrap.py 2012-09-15 19:26:18 +0000 |
2225 | @@ -6,12 +6,8 @@ |
2226 | """ |
2227 | |
2228 | import logging |
2229 | -import yaml |
2230 | |
2231 | -from juju.lib import ( |
2232 | - mocker, |
2233 | - testing, |
2234 | - ) |
2235 | +from juju.lib import mocker, testing, serializer |
2236 | |
2237 | from juju.providers.openstack.machine import NovaProviderMachine |
2238 | from juju.providers.openstack.tests import OpenStackTestMixin |
2239 | @@ -104,7 +100,7 @@ |
2240 | self.expect_nova_post("servers/1000/action", |
2241 | {"addFloatingIp": {"address": "8.8.8.8"}}, code=202) |
2242 | self.expect_swift_put("testing/provider-state", |
2243 | - yaml.dump({'zookeeper-instances': ['1000']})) |
2244 | + serializer.dump({'zookeeper-instances': ['1000']})) |
2245 | |
2246 | def _check_machine(self, machine_list): |
2247 | [machine] = machine_list |
2248 | @@ -165,7 +161,7 @@ |
2249 | def test_existing_machine(self): |
2250 | """A preexisting zookeeper instance is returned if present""" |
2251 | self.expect_swift_get("testing/provider-state", |
2252 | - response=yaml.dump({'zookeeper-instances': ['1000']})) |
2253 | + response=serializer.dump({'zookeeper-instances': ['1000']})) |
2254 | self.expect_nova_get("servers/1000", |
2255 | response={'server': { |
2256 | 'id': '1000', |
2257 | @@ -187,7 +183,7 @@ |
2258 | def test_existing_machine_missing(self): |
2259 | """Bootstrap overwrites existing zookeeper if instance is present""" |
2260 | self.expect_swift_get("testing/provider-state", |
2261 | - response=yaml.dump({'zookeeper-instances': [3000]})) |
2262 | + response=serializer.dump({'zookeeper-instances': [3000]})) |
2263 | self.expect_nova_get("servers/3000", code=404, |
2264 | response={'itemNotFound': |
2265 | {'message': "The resource could not be found.", 'code': 404} |
2266 | |
2267 | === modified file 'juju/providers/openstack/tests/test_launch.py' |
2268 | --- juju/providers/openstack/tests/test_launch.py 2012-08-24 14:21:57 +0000 |
2269 | +++ juju/providers/openstack/tests/test_launch.py 2012-09-15 19:26:18 +0000 |
2270 | @@ -1,10 +1,10 @@ |
2271 | """Tests for launching a new server customised for juju using Nova""" |
2272 | |
2273 | -import yaml |
2274 | |
2275 | from twisted.internet.defer import succeed |
2276 | |
2277 | from juju import errors |
2278 | +from juju.lib import serializer |
2279 | from juju.lib.mocker import MATCH |
2280 | from juju.lib.testing import TestCase |
2281 | |
2282 | @@ -68,7 +68,7 @@ |
2283 | def match(self, user_data): |
2284 | """Check contents of user_data but always match if assertions pass""" |
2285 | self._case.assertEqual("#cloud-config", user_data.split("\n", 1)[0]) |
2286 | - cc = yaml.safe_load(user_data) |
2287 | + cc = serializer.load(user_data) |
2288 | if self.is_master: |
2289 | zookeeper_hosts = "localhost:2181" |
2290 | else: |
2291 | |
2292 | === modified file 'juju/providers/openstack/tests/test_state.py' |
2293 | --- juju/providers/openstack/tests/test_state.py 2012-06-18 15:03:50 +0000 |
2294 | +++ juju/providers/openstack/tests/test_state.py 2012-09-15 19:26:18 +0000 |
2295 | @@ -4,8 +4,7 @@ |
2296 | state handling is a pretty thin layer over the file storage. |
2297 | """ |
2298 | |
2299 | -import yaml |
2300 | - |
2301 | +from juju.lib import serializer |
2302 | from juju.lib.testing import TestCase |
2303 | from juju.providers.openstack.tests import OpenStackTestMixin |
2304 | |
2305 | @@ -17,7 +16,7 @@ |
2306 | state = {"zookeeper-instances": [ |
2307 | [1000, "x1.example.com"], |
2308 | ]} |
2309 | - self.expect_swift_put("testing/provider-state", yaml.dump(state)) |
2310 | + self.expect_swift_put("testing/provider-state", serializer.dump(state)) |
2311 | self.mocker.replay() |
2312 | return self.get_provider().save_state(state) |
2313 | |
2314 | @@ -26,7 +25,7 @@ |
2315 | state = {"zookeeper-instances": [ |
2316 | [1000, "x1.example.com"], |
2317 | ]} |
2318 | - state_bytes = yaml.dump(state) |
2319 | + state_bytes = serializer.dump(state) |
2320 | self.expect_swift_put("testing/provider-state", state_bytes, code=404) |
2321 | self.expect_swift_put_container("testing") |
2322 | self.expect_swift_put("testing/provider-state", state_bytes) |
2323 | @@ -37,7 +36,7 @@ |
2324 | """Loading deserializes yaml from provider-state to a python dict""" |
2325 | state = {"zookeeper-instances": []} |
2326 | self.expect_swift_get("testing/provider-state", |
2327 | - response=yaml.dump(state)) |
2328 | + response=serializer.dump(state)) |
2329 | self.mocker.replay() |
2330 | deferred = self.get_provider().load_state() |
2331 | return deferred.addCallback(self.assertEqual, state) |
2332 | |
2333 | === modified file 'juju/providers/orchestra/tests/common.py' |
2334 | --- juju/providers/orchestra/tests/common.py 2012-04-12 01:01:57 +0000 |
2335 | +++ juju/providers/orchestra/tests/common.py 2012-09-15 19:26:18 +0000 |
2336 | @@ -1,12 +1,13 @@ |
2337 | from base64 import b64decode |
2338 | import os |
2339 | from xmlrpclib import Fault |
2340 | -from yaml import dump, load |
2341 | + |
2342 | |
2343 | from twisted.internet.defer import fail, succeed |
2344 | from twisted.web.error import Error |
2345 | from twisted.web.xmlrpc import Proxy |
2346 | |
2347 | +from juju.lib.serializer import dump, load |
2348 | from juju.lib.mocker import ANY, MATCH |
2349 | from juju.providers.orchestra import MachineProvider |
2350 | |
2351 | |
2352 | === modified file 'juju/providers/orchestra/tests/test_bootstrap.py' |
2353 | --- juju/providers/orchestra/tests/test_bootstrap.py 2012-04-12 01:01:57 +0000 |
2354 | +++ juju/providers/orchestra/tests/test_bootstrap.py 2012-09-15 19:26:18 +0000 |
2355 | @@ -1,9 +1,9 @@ |
2356 | from xmlrpclib import Fault |
2357 | -from yaml import dump |
2358 | |
2359 | from twisted.internet.defer import succeed, inlineCallbacks |
2360 | |
2361 | from juju.errors import ProviderError |
2362 | +from juju.lib.serializer import dump |
2363 | from juju.lib.testing import TestCase |
2364 | from juju.providers.orchestra.machine import OrchestraMachine |
2365 | from juju.providers.orchestra.tests.common import OrchestraTestMixin |
2366 | |
2367 | === modified file 'juju/providers/orchestra/tests/test_findzookeepers.py' |
2368 | --- juju/providers/orchestra/tests/test_findzookeepers.py 2011-10-17 13:49:47 +0000 |
2369 | +++ juju/providers/orchestra/tests/test_findzookeepers.py 2012-09-15 19:26:18 +0000 |
2370 | @@ -1,8 +1,7 @@ |
2371 | -from yaml import dump |
2372 | - |
2373 | from twisted.internet.defer import succeed |
2374 | |
2375 | from juju.errors import EnvironmentNotFound |
2376 | +from juju.lib.serializer import dump |
2377 | from juju.lib.testing import TestCase |
2378 | from juju.providers.orchestra import MachineProvider |
2379 | from juju.providers.orchestra.machine import OrchestraMachine |
2380 | |
2381 | === modified file 'juju/providers/orchestra/tests/test_state.py' |
2382 | --- juju/providers/orchestra/tests/test_state.py 2011-10-10 07:18:11 +0000 |
2383 | +++ juju/providers/orchestra/tests/test_state.py 2012-09-15 19:26:18 +0000 |
2384 | @@ -1,5 +1,5 @@ |
2385 | -from yaml import dump |
2386 | |
2387 | +from juju.lib.serializer import dump |
2388 | from juju.lib.testing import TestCase |
2389 | from juju.providers.orchestra import MachineProvider |
2390 | |
2391 | |
2392 | === modified file 'juju/state/charm.py' |
2393 | --- juju/state/charm.py 2012-04-01 03:53:48 +0000 |
2394 | +++ juju/state/charm.py 2012-09-15 19:26:18 +0000 |
2395 | @@ -1,4 +1,4 @@ |
2396 | -import yaml |
2397 | + |
2398 | |
2399 | from twisted.internet.defer import ( |
2400 | inlineCallbacks, returnValue, succeed) |
2401 | @@ -8,7 +8,7 @@ |
2402 | from juju.charm.config import ConfigOptions |
2403 | from juju.charm.metadata import MetaData |
2404 | from juju.charm.url import CharmURL |
2405 | -from juju.lib import under |
2406 | +from juju.lib import under, serializer |
2407 | from juju.state.base import StateBase |
2408 | from juju.state.errors import CharmStateNotFound |
2409 | |
2410 | @@ -41,7 +41,7 @@ |
2411 | # replacements here. For now this will do, and will |
2412 | # explode reliably in case of conflicts. |
2413 | yield self._client.create( |
2414 | - _charm_path(charm_id), yaml.safe_dump(charm_data)) |
2415 | + _charm_path(charm_id), serializer.dump(charm_data)) |
2416 | charm_state = CharmState(self._client, charm_id, charm_data) |
2417 | returnValue(charm_state) |
2418 | |
2419 | @@ -52,7 +52,7 @@ |
2420 | content, stat = yield self._client.get(_charm_path(charm_id)) |
2421 | except NoNodeException: |
2422 | raise CharmStateNotFound(charm_id) |
2423 | - charm_data = yaml.load(content) |
2424 | + charm_data = serializer.load(content) |
2425 | charm_state = CharmState(self._client, charm_id, charm_data) |
2426 | returnValue(charm_state) |
2427 | |
2428 | |
2429 | === modified file 'juju/state/environment.py' |
2430 | --- juju/state/environment.py 2012-03-30 08:26:21 +0000 |
2431 | +++ juju/state/environment.py 2012-09-15 19:26:18 +0000 |
2432 | @@ -1,11 +1,12 @@ |
2433 | import zookeeper |
2434 | -import yaml |
2435 | + |
2436 | |
2437 | from twisted.internet.defer import inlineCallbacks, returnValue |
2438 | |
2439 | from txzookeeper.utils import retry_change |
2440 | |
2441 | from juju.environment.config import EnvironmentsConfig |
2442 | +from juju.lib import serializer |
2443 | from juju.state.errors import EnvironmentStateNotFound |
2444 | from juju.state.base import StateBase |
2445 | |
2446 | @@ -45,7 +46,7 @@ |
2447 | """ |
2448 | data = constraints.data |
2449 | data.pop("ubuntu-series", None) |
2450 | - return self._force("/constraints", yaml.safe_dump(data)) |
2451 | + return self._force("/constraints", serializer.dump(data)) |
2452 | |
2453 | @inlineCallbacks |
2454 | def get_constraint_set(self): |
2455 | @@ -66,7 +67,7 @@ |
2456 | constraint_set = yield self.get_constraint_set() |
2457 | try: |
2458 | content, stat = yield self._client.get("/constraints") |
2459 | - data = yaml.load(content) |
2460 | + data = serializer.load(content) |
2461 | except zookeeper.NoNodeException: |
2462 | data = {} |
2463 | returnValue(constraint_set.load(data)) |
2464 | @@ -107,7 +108,7 @@ |
2465 | content, stat = yield self._client.get(SETTINGS_PATH) |
2466 | except zookeeper.NoNodeException: |
2467 | returnValue(default) |
2468 | - data = yaml.load(content) |
2469 | + data = serializer.load(content) |
2470 | returnValue(data.get(key, default)) |
2471 | |
2472 | def _set_value(self, key, value): |
2473 | @@ -116,9 +117,9 @@ |
2474 | if not old_content: |
2475 | data = {} |
2476 | else: |
2477 | - data = yaml.load(old_content) |
2478 | + data = serializer.load(old_content) |
2479 | data[key] = value |
2480 | - return yaml.safe_dump(data) |
2481 | + return serializer.dump(data) |
2482 | |
2483 | return retry_change(self._client, SETTINGS_PATH, set_value) |
2484 | |
2485 | |
2486 | === modified file 'juju/state/hook.py' |
2487 | --- juju/state/hook.py 2012-04-09 19:40:57 +0000 |
2488 | +++ juju/state/hook.py 2012-09-15 19:26:18 +0000 |
2489 | @@ -6,8 +6,7 @@ |
2490 | from juju.state.errors import ( |
2491 | UnitRelationStateNotFound, StateNotFound, RelationBrokenContextError, |
2492 | RelationStateNotFound, InvalidRelationIdentity, StateChanged) |
2493 | -from juju.state.relation import ( |
2494 | - RelationStateManager, ServiceRelationState, UnitRelationState) |
2495 | +from juju.state.relation import ServiceRelationState, UnitRelationState |
2496 | from juju.state.service import ServiceStateManager, parse_service_name |
2497 | from juju.state.utils import YAMLState |
2498 | |
2499 | |
2500 | === modified file 'juju/state/machine.py' |
2501 | --- juju/state/machine.py 2012-04-02 23:33:39 +0000 |
2502 | +++ juju/state/machine.py 2012-09-15 19:26:18 +0000 |
2503 | @@ -1,9 +1,9 @@ |
2504 | -import yaml |
2505 | import zookeeper |
2506 | |
2507 | from twisted.internet.defer import inlineCallbacks, returnValue |
2508 | |
2509 | from juju.errors import ConstraintError |
2510 | +from juju.lib import serializer |
2511 | from juju.state.agent import AgentStateMixin |
2512 | from juju.state.environment import EnvironmentStateManager |
2513 | from juju.state.errors import MachineStateNotFound, MachineStateInUse |
2514 | @@ -25,7 +25,7 @@ |
2515 | "Unprovisionable machine: incomplete constraints") |
2516 | machine_data = {"constraints": constraints.data} |
2517 | path = yield self._client.create( |
2518 | - "/machines/machine-", yaml.dump(machine_data), |
2519 | + "/machines/machine-", serializer.dump(machine_data), |
2520 | flags=zookeeper.SEQUENCE) |
2521 | _, internal_id = path.rsplit("/", 1) |
2522 | |
2523 | |
2524 | === modified file 'juju/state/relation.py' |
2525 | --- juju/state/relation.py 2012-05-15 19:07:47 +0000 |
2526 | +++ juju/state/relation.py 2012-09-15 19:26:18 +0000 |
2527 | @@ -1,7 +1,6 @@ |
2528 | import logging |
2529 | from os.path import basename, dirname |
2530 | |
2531 | -import yaml |
2532 | import zookeeper |
2533 | |
2534 | from twisted.internet.defer import ( |
2535 | @@ -9,6 +8,7 @@ |
2536 | |
2537 | from txzookeeper.utils import retry_change |
2538 | |
2539 | +from juju.lib import serializer |
2540 | from juju.state.base import StateBase |
2541 | from juju.state.errors import ( |
2542 | DuplicateEndpoints, IncompatibleEndpoints, RelationAlreadyExists, |
2543 | @@ -310,7 +310,7 @@ |
2544 | except zookeeper.NodeExistsException: |
2545 | pass |
2546 | |
2547 | - node_data = yaml.safe_dump(relation_data) |
2548 | + node_data = serializer.dump(relation_data) |
2549 | role_path = "%s/%s" % (scope_path, relation_data["role"]) |
2550 | |
2551 | yield retry_change( |
2552 | @@ -346,18 +346,18 @@ |
2553 | def update_address(content, stat): |
2554 | unit_map = None |
2555 | if content: |
2556 | - unit_map = yaml.load(content) |
2557 | + unit_map = serializer.load(content) |
2558 | if not unit_map: |
2559 | unit_map = {} |
2560 | unit_map["private-address"] = private_address |
2561 | - return yaml.safe_dump(unit_map) |
2562 | + return serializer.dump(unit_map) |
2563 | |
2564 | yield retry_change(self._client, settings_path, update_address) |
2565 | |
2566 | # Update the unit name -> id mapping on the relation node |
2567 | def update_unit_mapping(content, stat): |
2568 | if content: |
2569 | - unit_map = yaml.load(content) |
2570 | + unit_map = serializer.load(content) |
2571 | else: |
2572 | unit_map = {} |
2573 | # If it's already present, we're done, just return the |
2574 | @@ -366,7 +366,7 @@ |
2575 | if unit_state.internal_id in unit_map: |
2576 | return content |
2577 | unit_map[unit_state.internal_id] = unit_state.unit_name |
2578 | - return yaml.dump(unit_map) |
2579 | + return serializer.dump(unit_map) |
2580 | |
2581 | yield retry_change(self._client, |
2582 | "/relations/%s" % self._relation_id, |
2583 | @@ -481,7 +481,7 @@ |
2584 | """ |
2585 | path = yield self.get_settings_path() |
2586 | # encode as a YAML string |
2587 | - data = yaml.safe_dump(data) |
2588 | + data = serializer.dump(data) |
2589 | |
2590 | yield retry_change( |
2591 | self._client, path, lambda content, stat: data) |
2592 | @@ -724,7 +724,7 @@ |
2593 | if self._relation_scope == "container": |
2594 | relation_path = dirname(relation_path) |
2595 | content, stat = yield self._client.get(relation_path) |
2596 | - self._unit_name_map = yaml.load(content) |
2597 | + self._unit_name_map = serializer.load(content) |
2598 | results = [] |
2599 | for unit_id_list in unit_ids: |
2600 | names = [] |
2601 | |
2602 | === modified file 'juju/state/security.py' |
2603 | --- juju/state/security.py 2011-09-15 18:50:23 +0000 |
2604 | +++ juju/state/security.py 2012-09-15 19:26:18 +0000 |
2605 | @@ -1,7 +1,7 @@ |
2606 | import base64 |
2607 | import random |
2608 | import string |
2609 | -import yaml |
2610 | + |
2611 | |
2612 | from zookeeper import ( |
2613 | BadArgumentsException, BadVersionException, |
2614 | @@ -10,6 +10,7 @@ |
2615 | from twisted.internet.defer import inlineCallbacks, returnValue |
2616 | from txzookeeper.client import ZOO_OPEN_ACL_UNSAFE, ZookeeperClient |
2617 | |
2618 | +from juju.lib import serializer |
2619 | from juju.state.auth import make_identity, make_ace |
2620 | from juju.state.errors import ( |
2621 | StateNotFound, PrincipalNotFound) |
2622 | @@ -74,7 +75,7 @@ |
2623 | except NoNodeException: |
2624 | raise StateNotFound("Group does not exist at %r" % self._path) |
2625 | |
2626 | - credentials = yaml.load(data) |
2627 | + credentials = serializer.load(data) |
2628 | self._name = credentials["name"] |
2629 | self._password = credentials["password"] |
2630 | |
2631 | @@ -108,7 +109,7 @@ |
2632 | try: |
2633 | yield self._client.create( |
2634 | self._path, |
2635 | - yaml.safe_dump(dict(name=name, password=password))) |
2636 | + serializer.dump(dict(name=name, password=password))) |
2637 | except NodeExistsException: |
2638 | raise RuntimeError("Group already exists at %r" % self._path) |
2639 | self._name = name |
2640 | @@ -211,7 +212,7 @@ |
2641 | yield client.add_auth("digest", credentials) |
2642 | # Load the otp principal data |
2643 | data, stat = yield client.get(path) |
2644 | - principal_data = yaml.load(data) |
2645 | + principal_data = serializer.load(data) |
2646 | # Consume the otp node |
2647 | yield client.delete(path) |
2648 | returnValue((principal_data["name"], principal_data["password"])) |
2649 | @@ -237,7 +238,7 @@ |
2650 | |
2651 | self._path = yield self._client.create( |
2652 | self._path, |
2653 | - yaml.safe_dump(dict(name=name, password=password)), |
2654 | + serializer.dump(dict(name=name, password=password)), |
2655 | acls=acl, |
2656 | flags=SEQUENCE) |
2657 | |
2658 | |
2659 | === modified file 'juju/state/service.py' |
2660 | --- juju/state/service.py 2012-04-05 22:27:11 +0000 |
2661 | +++ juju/state/service.py 2012-09-15 19:26:18 +0000 |
2662 | @@ -1,4 +1,3 @@ |
2663 | -import yaml |
2664 | import zookeeper |
2665 | |
2666 | from twisted.internet.defer import ( |
2667 | @@ -7,6 +6,7 @@ |
2668 | from txzookeeper.utils import retry_change |
2669 | |
2670 | from juju.charm.url import CharmURL |
2671 | +from juju.lib import serializer |
2672 | from juju.state.agent import AgentStateMixin |
2673 | from juju.state.base import log, StateBase |
2674 | from juju.state.endpoint import RelationEndpoint |
2675 | @@ -55,13 +55,14 @@ |
2676 | "charm": charm_id, "constraints": constraints.data} |
2677 | # charm metadata is always decoded into unicode, ensure any |
2678 | # serialized state references strings to avoid tying to py runtime. |
2679 | - node_data = yaml.safe_dump(service_details) |
2680 | + node_data = serializer.dump(service_details) |
2681 | path = yield self._client.create("/services/service-", node_data, |
2682 | flags=zookeeper.SEQUENCE) |
2683 | internal_id = path.rsplit("/", 1)[1] |
2684 | |
2685 | # create a child node for configuration options |
2686 | - yield self._client.create("%s/config" % path, yaml.dump({})) |
2687 | + yield self._client.create( |
2688 | + "%s/config" % path, serializer.dump({})) |
2689 | |
2690 | def add_service(topology): |
2691 | if topology.find_service_with_name(service_name): |
2692 | @@ -416,7 +417,9 @@ |
2693 | charm_id = yield self.get_charm_id() |
2694 | unit_data = {"charm": charm_id, "constraints": constraints.data} |
2695 | path = yield self._client.create( |
2696 | - "/units/unit-", yaml.dump(unit_data), flags=zookeeper.SEQUENCE) |
2697 | + "/units/unit-", |
2698 | + serializer.dump(unit_data), |
2699 | + flags=zookeeper.SEQUENCE) |
2700 | |
2701 | internal_unit_id = path.rsplit("/", 1)[1] |
2702 | |
2703 | @@ -1094,7 +1097,7 @@ |
2704 | try: |
2705 | yield self._client.create( |
2706 | self._hook_debug_path, |
2707 | - yaml.safe_dump({"debug_hooks": hook_names}), |
2708 | + serializer.dump({"debug_hooks": hook_names}), |
2709 | flags=zookeeper.EPHEMERAL) |
2710 | except zookeeper.NodeExistsException: |
2711 | raise ServiceUnitDebugAlreadyEnabled(self.unit_name) |
2712 | @@ -1125,7 +1128,7 @@ |
2713 | except zookeeper.NoNodeException: |
2714 | # We get to the same end state. |
2715 | returnValue(None) |
2716 | - returnValue(yaml.load(content)) |
2717 | + returnValue(serializer.load(content)) |
2718 | |
2719 | @inlineCallbacks |
2720 | def watch_hook_debug(self, callback, permanent=True): |
2721 | @@ -1179,12 +1182,12 @@ |
2722 | def update(content, stat): |
2723 | if not content: |
2724 | flags = dict(force=force) |
2725 | - return yaml.dump(flags) |
2726 | + return serializer.dump(flags) |
2727 | |
2728 | - flags = yaml.load(content) |
2729 | + flags = serializer.load(content) |
2730 | if not isinstance(flags, dict): |
2731 | flags = dict(force=force) |
2732 | - return yaml.dump(flags) |
2733 | + return serializer.dump(flags) |
2734 | |
2735 | if flags['force'] != force: |
2736 | raise ServiceUnitUpgradeAlreadyEnabled(self.unit_name) |
2737 | @@ -1205,7 +1208,7 @@ |
2738 | returnValue(False) |
2739 | except zookeeper.NoNodeException: |
2740 | returnValue(False) |
2741 | - returnValue(yaml.load(content)) |
2742 | + returnValue(serializer.load(content)) |
2743 | |
2744 | @inlineCallbacks |
2745 | def clear_upgrade_flag(self): |
2746 | @@ -1282,7 +1285,7 @@ |
2747 | |
2748 | try: |
2749 | yield self._client.create( |
2750 | - self._unit_resolved_path, yaml.safe_dump({"retry": retry})) |
2751 | + self._unit_resolved_path, serializer.dump({"retry": retry})) |
2752 | except zookeeper.NodeExistsException: |
2753 | raise ServiceUnitResolvedAlreadyEnabled(self.unit_name) |
2754 | |
2755 | @@ -1299,7 +1302,7 @@ |
2756 | except zookeeper.NoNodeException: |
2757 | # Return a default value. |
2758 | returnValue(None) |
2759 | - returnValue(yaml.load(content)) |
2760 | + returnValue(serializer.load(content)) |
2761 | |
2762 | @inlineCallbacks |
2763 | def clear_resolved(self): |
2764 | @@ -1392,10 +1395,10 @@ |
2765 | |
2766 | def update_relation_resolved(content, stat): |
2767 | if not content: |
2768 | - return yaml.safe_dump(relation_map) |
2769 | + return serializer.dump(relation_map) |
2770 | |
2771 | - content = yaml.safe_dump( |
2772 | - dict_merge(yaml.load(content), relation_map)) |
2773 | + content = serializer.dump( |
2774 | + dict_merge(serializer.load(content), relation_map)) |
2775 | return content |
2776 | |
2777 | try: |
2778 | @@ -1416,7 +1419,7 @@ |
2779 | self._relation_resolved_path) |
2780 | except zookeeper.NoNodeException: |
2781 | returnValue(None) |
2782 | - returnValue(yaml.load(content)) |
2783 | + returnValue(serializer.load(content)) |
2784 | |
2785 | @inlineCallbacks |
2786 | def clear_relation_resolved(self): |
2787 | @@ -1477,14 +1480,14 @@ |
2788 | if content is None: |
2789 | data = {} |
2790 | else: |
2791 | - data = yaml.load(content) |
2792 | + data = serializer.load(content) |
2793 | if data is None: |
2794 | data = {} |
2795 | open_ports = data.setdefault("open", []) |
2796 | port_proto = dict(port=port, proto=proto) |
2797 | if port_proto not in open_ports: |
2798 | open_ports.append(port_proto) |
2799 | - return yaml.safe_dump(data) |
2800 | + return serializer.dump(data) |
2801 | |
2802 | yield retry_change(self._client, self._ports_path, zk_open_port) |
2803 | |
2804 | @@ -1499,14 +1502,14 @@ |
2805 | if content is None: |
2806 | data = {} |
2807 | else: |
2808 | - data = yaml.load(content) |
2809 | + data = serializer.load(content) |
2810 | if data is None: |
2811 | data = {} |
2812 | open_ports = data.setdefault("open", []) |
2813 | port_proto = dict(port=port, proto=proto) |
2814 | if port_proto in open_ports: |
2815 | open_ports.remove(port_proto) |
2816 | - return yaml.dump(data) |
2817 | + return serializer.dump(data) |
2818 | |
2819 | yield retry_change( |
2820 | self._client, self._ports_path, zk_close_port) |
2821 | @@ -1524,7 +1527,7 @@ |
2822 | content, stat = yield self._client.get(self._ports_path) |
2823 | except zookeeper.NoNodeException: |
2824 | returnValue([]) |
2825 | - data = yaml.load(content) |
2826 | + data = serializer.load(content) |
2827 | if data is None: |
2828 | returnValue(()) |
2829 | returnValue(data.get("open", ())) |
2830 | |
2831 | === modified file 'juju/state/tests/test_charm.py' |
2832 | --- juju/state/tests/test_charm.py 2012-03-29 08:26:36 +0000 |
2833 | +++ juju/state/tests/test_charm.py 2012-09-15 19:26:18 +0000 |
2834 | @@ -1,7 +1,6 @@ |
2835 | import os |
2836 | import shutil |
2837 | |
2838 | -import yaml |
2839 | |
2840 | from twisted.internet.defer import inlineCallbacks |
2841 | |
2842 | @@ -10,6 +9,8 @@ |
2843 | from juju.charm.tests.test_directory import sample_directory |
2844 | from juju.charm.tests.test_repository import unbundled_repository |
2845 | |
2846 | +from juju.lib import serializer |
2847 | + |
2848 | from juju.state.charm import CharmStateManager |
2849 | from juju.state.errors import CharmStateNotFound |
2850 | from juju.state.tests.common import StateTestBase |
2851 | @@ -44,7 +45,7 @@ |
2852 | content, stat = yield self.client.get( |
2853 | "/charms/local_3a_series_2f_dummy-1") |
2854 | |
2855 | - charm_data = yaml.load(content) |
2856 | + charm_data = serializer.load(content) |
2857 | self.assertEquals(charm_data, { |
2858 | "metadata": self.charm.metadata.get_serialization_data(), |
2859 | "config": self.charm.config.get_serialization_data(), |
2860 | |
2861 | === modified file 'juju/state/tests/test_environment.py' |
2862 | --- juju/state/tests/test_environment.py 2012-03-29 01:37:57 +0000 |
2863 | +++ juju/state/tests/test_environment.py 2012-09-15 19:26:18 +0000 |
2864 | @@ -1,10 +1,9 @@ |
2865 | -import yaml |
2866 | - |
2867 | from twisted.internet.defer import inlineCallbacks, Deferred |
2868 | |
2869 | from juju.environment.tests.test_config import ( |
2870 | EnvironmentsConfigTestBase, SAMPLE_ENV) |
2871 | from juju.machine.tests.test_constraints import dummy_cs |
2872 | +from juju.lib import serializer |
2873 | from juju.state.errors import EnvironmentStateNotFound |
2874 | from juju.state.environment import ( |
2875 | EnvironmentStateManager, GlobalSettingsStateManager, SETTINGS_PATH) |
2876 | @@ -38,7 +37,8 @@ |
2877 | |
2878 | serialized = self.config.serialize("myfirstenv") |
2879 | content, stat = yield self.client.get("/environment") |
2880 | - self.assertEquals(yaml.load(content), yaml.load(serialized)) |
2881 | + self.assertEquals(serializer.load(content), |
2882 | + serializer.load(serialized)) |
2883 | |
2884 | @inlineCallbacks |
2885 | def test_set_config_state_replaces_environment(self): |
2886 | @@ -53,7 +53,8 @@ |
2887 | |
2888 | serialized = self.config.serialize("myfirstenv") |
2889 | content, stat = yield self.client.get("/environment") |
2890 | - self.assertEquals(yaml.load(content), yaml.load(serialized)) |
2891 | + self.assertEquals(serializer.load(content), |
2892 | + serializer.load(serialized)) |
2893 | |
2894 | @inlineCallbacks |
2895 | def test_get_config(self): |
2896 | @@ -65,7 +66,8 @@ |
2897 | config = yield manager.get_config() |
2898 | serialized1 = self.config.serialize("myfirstenv") |
2899 | serialized2 = config.serialize("myfirstenv") |
2900 | - self.assertEquals(yaml.load(serialized1), yaml.load(serialized2)) |
2901 | + self.assertEquals(serializer.load(serialized1), |
2902 | + serializer.load(serialized2)) |
2903 | |
2904 | def test_get_config_when_missing(self): |
2905 | """ |
2906 | @@ -140,7 +142,7 @@ |
2907 | yield self.manager.set_provider_type("ec2") |
2908 | self.assertEqual((yield self.manager.get_provider_type()), "ec2") |
2909 | content, stat = yield self.client.get("/settings") |
2910 | - self.assertEqual(yaml.load(content), |
2911 | + self.assertEqual(serializer.load(content), |
2912 | {"provider-type": "ec2"}) |
2913 | |
2914 | @inlineCallbacks |
2915 | @@ -211,7 +213,7 @@ |
2916 | |
2917 | data = dict(x=1, y=2, z=3, moose=u"moon") |
2918 | yield self.client.set( |
2919 | - SETTINGS_PATH, yaml.safe_dump(data)) |
2920 | + SETTINGS_PATH, serializer.dump(data)) |
2921 | value = yield callbacks[1] |
2922 | self.assertEqual(value.type_name, "changed") |
2923 | |
2924 | |
2925 | === modified file 'juju/state/tests/test_hook.py' |
2926 | --- juju/state/tests/test_hook.py 2012-04-09 19:40:57 +0000 |
2927 | +++ juju/state/tests/test_hook.py 2012-09-15 19:26:18 +0000 |
2928 | @@ -1,7 +1,6 @@ |
2929 | -import yaml |
2930 | - |
2931 | from twisted.internet.defer import inlineCallbacks, returnValue |
2932 | from juju.lib.pick import pick_attr |
2933 | +from juju.lib import serializer |
2934 | from juju.lib.testing import TestCase |
2935 | |
2936 | from juju.state.endpoint import RelationEndpoint |
2937 | @@ -218,7 +217,7 @@ |
2938 | self.assertNotEqual(current_data, data) |
2939 | |
2940 | self.client.set(self.get_unit_settings_path(self.mysql_states), |
2941 | - yaml.dump({"hello": "world"})) |
2942 | + serializer.dump({"hello": "world"})) |
2943 | data = yield self.context.get("mysql/0") |
2944 | data["abc"] = 1 |
2945 | |
2946 | @@ -286,7 +285,7 @@ |
2947 | content, stat = yield self.client.get( |
2948 | self.get_unit_settings_path(self.wordpress_states)) |
2949 | data["private-address"] = "wordpress-0.example.com" |
2950 | - self.assertEqual(yaml.load(content), data) |
2951 | + self.assertEqual(serializer.load(content), data) |
2952 | self.assertEqual(changes, |
2953 | [AddedItem("abc", 12), AddedItem("bar", "21")]) |
2954 | |
2955 | @@ -297,7 +296,7 @@ |
2956 | yield self.context.set_value("donkey", u"abc") |
2957 | data, stat = yield self.client.get( |
2958 | self.get_unit_settings_path(self.wordpress_states)) |
2959 | - self.assertEqual(yaml.load(data), |
2960 | + self.assertEqual(serializer.load(data), |
2961 | {"private-address": "wordpress-0.example.com"}) |
2962 | |
2963 | changes = yield self.context.flush() |
2964 | @@ -305,7 +304,7 @@ |
2965 | self.get_unit_settings_path(self.wordpress_states)) |
2966 | |
2967 | self.assertEqual( |
2968 | - yaml.load(data), |
2969 | + serializer.load(data), |
2970 | {"zebra": 12, "donkey": "abc", |
2971 | "private-address": "wordpress-0.example.com"}) |
2972 | |
2973 | @@ -320,14 +319,14 @@ |
2974 | yield self.client.set( |
2975 | self.get_unit_settings_path( |
2976 | self.wordpress_states), |
2977 | - yaml.dump({"key": "secret"})) |
2978 | + serializer.dump({"key": "secret"})) |
2979 | |
2980 | yield self.context.delete_value("key") |
2981 | changes = yield self.context.flush() |
2982 | data, stat = yield self.client.get( |
2983 | self.get_unit_settings_path(self.wordpress_states)) |
2984 | |
2985 | - self.assertNotIn("key", yaml.load(data)) |
2986 | + self.assertNotIn("key", serializer.load(data)) |
2987 | self.assertEqual(changes, [DeletedItem("key", "secret")]) |
2988 | |
2989 | @inlineCallbacks |
2990 | @@ -336,14 +335,14 @@ |
2991 | """ |
2992 | yield self.client.set( |
2993 | self.get_unit_settings_path(self.wordpress_states), |
2994 | - yaml.dump({"lantern": "green"})) |
2995 | + serializer.dump({"lantern": "green"})) |
2996 | |
2997 | yield self.context.delete_value("key") |
2998 | changes = yield self.context.flush() |
2999 | data, stat = yield self.client.get( |
3000 | self.get_unit_settings_path(self.wordpress_states)) |
3001 | |
3002 | - self.assertEqual(yaml.load(data), {"lantern": "green"}) |
3003 | + self.assertEqual(serializer.load(data), {"lantern": "green"}) |
3004 | self.assertEqual(changes, []) |
3005 | |
3006 | @inlineCallbacks |
3007 | @@ -351,12 +350,12 @@ |
3008 | """Flushing a context which has no writes is a noop.""" |
3009 | yield self.client.set( |
3010 | self.get_unit_settings_path(self.wordpress_states), |
3011 | - yaml.dump({"key": "secret"})) |
3012 | + serializer.dump({"key": "secret"})) |
3013 | changes = yield self.context.flush() |
3014 | data, stat = yield self.client.get( |
3015 | self.get_unit_settings_path(self.wordpress_states)) |
3016 | |
3017 | - self.assertEqual(yaml.load(data), |
3018 | + self.assertEqual(serializer.load(data), |
3019 | {"key": "secret"}) |
3020 | self.assertEqual(changes, []) |
3021 | |
3022 | @@ -378,7 +377,7 @@ |
3023 | |
3024 | yield self.client.set( |
3025 | self.get_unit_settings_path(self.wordpress_states), |
3026 | - yaml.dump(data)) |
3027 | + serializer.dump(data)) |
3028 | |
3029 | # On the context: |
3030 | # - add a new key |
3031 | @@ -406,7 +405,7 @@ |
3032 | |
3033 | yield self.client.set( |
3034 | self.get_unit_settings_path(self.wordpress_states), |
3035 | - yaml.dump(data)) |
3036 | + serializer.dump(data)) |
3037 | |
3038 | changes = yield self.context.flush() |
3039 | |
3040 | @@ -414,7 +413,7 @@ |
3041 | self.get_unit_settings_path(self.wordpress_states)) |
3042 | |
3043 | self.assertEqual( |
3044 | - yaml.load(data), |
3045 | + serializer.load(data), |
3046 | {"port": 22, "host": "xe2.example.com", "db": 21, "home": "good", |
3047 | "tower": "rock", "zoo": "keeper"}) |
3048 | self.assertEqual( |
3049 | @@ -430,7 +429,7 @@ |
3050 | """We can set a value even if we have existing saved settings.""" |
3051 | yield self.client.set( |
3052 | self.get_unit_settings_path(self.wordpress_states), |
3053 | - yaml.dump({"key": "secret"})) |
3054 | + serializer.dump({"key": "secret"})) |
3055 | yield self.context.set_value("magic", "room") |
3056 | value = yield self.context.get_value("wordpress/0", "key") |
3057 | self.assertEqual(value, "secret") |
3058 | @@ -675,7 +674,7 @@ |
3059 | def test_get_self(self): |
3060 | """Own settings can be retrieved.""" |
3061 | self.client.set(self.get_unit_settings_path(self.wordpress_states), |
3062 | - yaml.dump({"hello": "world"})) |
3063 | + serializer.dump({"hello": "world"})) |
3064 | data = yield self.context.get(None) |
3065 | self.assertEquals(data, {"hello": "world"}) |
3066 | |
3067 | @@ -683,7 +682,7 @@ |
3068 | def test_get_self_by_name(self): |
3069 | """Own settings can be retrieved by name.""" |
3070 | self.client.set(self.get_unit_settings_path(self.wordpress_states), |
3071 | - yaml.dump({"hello": "world"})) |
3072 | + serializer.dump({"hello": "world"})) |
3073 | data = yield self.context.get("wordpress/0") |
3074 | self.assertEquals(data, {"hello": "world"}) |
3075 | |
3076 | @@ -699,7 +698,7 @@ |
3077 | def test_get_value_self(self): |
3078 | """Own settings can be retrieved.""" |
3079 | self.client.set(self.get_unit_settings_path(self.wordpress_states), |
3080 | - yaml.dump({"hello": "world"})) |
3081 | + serializer.dump({"hello": "world"})) |
3082 | self.assertEquals( |
3083 | (yield self.context.get_value("wordpress/0", "hello")), "world") |
3084 | self.assertEquals( |
3085 | @@ -847,7 +846,7 @@ |
3086 | self.unit, |
3087 | self.container_unit) |
3088 | self.client.set(settings_path, |
3089 | - yaml.dump({"hello": "world"})) |
3090 | + serializer.dump({"hello": "world"})) |
3091 | data = yield self.context.get(None) |
3092 | self.assertEquals(data, {"hello": "world"}) |
3093 | |
3094 | |
3095 | === modified file 'juju/state/tests/test_machine.py' |
3096 | --- juju/state/tests/test_machine.py 2012-03-28 00:38:16 +0000 |
3097 | +++ juju/state/tests/test_machine.py 2012-09-15 19:26:18 +0000 |
3098 | @@ -1,11 +1,10 @@ |
3099 | import functools |
3100 | |
3101 | -import yaml |
3102 | - |
3103 | from twisted.internet.defer import inlineCallbacks, Deferred, returnValue |
3104 | |
3105 | from juju.charm.tests import local_charm_id |
3106 | from juju.errors import ConstraintError |
3107 | +from juju.lib import serializer |
3108 | from juju.machine.tests.test_constraints import ( |
3109 | dummy_constraints, series_constraints) |
3110 | from juju.state.charm import CharmStateManager |
3111 | @@ -237,7 +236,7 @@ |
3112 | |
3113 | content, stat = yield self.client.get("/machines/machine-0000000000") |
3114 | self.assertEquals( |
3115 | - yaml.load(content)["provider-machine-id"], "custom-id") |
3116 | + serializer.load(content)["provider-machine-id"], "custom-id") |
3117 | |
3118 | @inlineCallbacks |
3119 | def test_set_instance_id_preserves_existing_data(self): |
3120 | @@ -246,11 +245,11 @@ |
3121 | """ |
3122 | machine_state = yield self.add_machine_state() |
3123 | yield self.client.set("/machines/machine-0000000000", |
3124 | - yaml.dump({"foo": "bar"})) |
3125 | + serializer.dump({"foo": "bar"})) |
3126 | yield machine_state.set_instance_id("custom-id") |
3127 | |
3128 | content, stat = yield self.client.get("/machines/machine-0000000000") |
3129 | - self.assertEquals(yaml.load(content), |
3130 | + self.assertEquals(serializer.load(content), |
3131 | {"provider-machine-id": "custom-id", |
3132 | "foo": "bar"}) |
3133 | |
3134 | @@ -299,7 +298,7 @@ |
3135 | """ |
3136 | machine_state = yield self.add_machine_state() |
3137 | yield self.client.set("/machines/machine-0000000000", |
3138 | - yaml.dump({"foo": "bar"})) |
3139 | + serializer.dump({"foo": "bar"})) |
3140 | |
3141 | instance_id = yield machine_state.get_instance_id() |
3142 | self.assertEquals(instance_id, None) |
3143 | |
3144 | === modified file 'juju/state/tests/test_relation.py' |
3145 | --- juju/state/tests/test_relation.py 2012-06-01 22:50:44 +0000 |
3146 | +++ juju/state/tests/test_relation.py 2012-09-15 19:26:18 +0000 |
3147 | @@ -2,15 +2,14 @@ |
3148 | import os |
3149 | import time |
3150 | |
3151 | -import yaml |
3152 | import zookeeper |
3153 | |
3154 | - |
3155 | from twisted.internet.defer import ( |
3156 | inlineCallbacks, returnValue, Deferred, fail, succeed) |
3157 | |
3158 | from juju.charm.directory import CharmDirectory |
3159 | from juju.charm.tests import local_charm_id |
3160 | +from juju.lib import serializer |
3161 | from juju.machine.tests.test_constraints import dummy_constraints |
3162 | from juju.charm.tests.test_metadata import test_repository_path |
3163 | from juju.charm.tests.test_repository import unbundled_repository |
3164 | @@ -803,14 +802,14 @@ |
3165 | # Verify the unit mapping |
3166 | unit_map_data, stat = yield self.client.get("/relations/%s" % ( |
3167 | self.relation_state.internal_id)) |
3168 | - unit_map = yaml.load(unit_map_data) |
3169 | + unit_map = serializer.load(unit_map_data) |
3170 | self.assertEqual( |
3171 | unit_map, |
3172 | {unit_state.internal_id: unit_state.unit_name}) |
3173 | |
3174 | content, stat = yield self.client.get(settings_path) |
3175 | self.assertEqual( |
3176 | - yaml.load(content), {"private-address": "foobar.local"}) |
3177 | + serializer.load(content), {"private-address": "foobar.local"}) |
3178 | |
3179 | @inlineCallbacks |
3180 | def test_add_unit_state_scope_container_relation(self): |
3181 | @@ -930,7 +929,7 @@ |
3182 | unit_state.internal_id) |
3183 | |
3184 | data = {"hello": "world", "private-address": "foobar.local"} |
3185 | - yield self.client.create(settings_path, yaml.dump(data)) |
3186 | + yield self.client.create(settings_path, serializer.dump(data)) |
3187 | |
3188 | yield unit_state.set_private_address("northwest.local") |
3189 | yield self.service1_relation.add_unit_state(unit_state) |
3190 | @@ -939,11 +938,11 @@ |
3191 | |
3192 | # The unit address has been updated to current |
3193 | data["private-address"] = "northwest.local" |
3194 | - self.assertEqual(node_data, yaml.dump(data)) |
3195 | + self.assertEqual(node_data, serializer.dump(data)) |
3196 | |
3197 | data, stat = yield self.client.get( |
3198 | "/relations/%s" % self.relation_state.internal_id) |
3199 | - unit_map = yaml.load(data) |
3200 | + unit_map = serializer.load(data) |
3201 | self.assertEqual(unit_map, |
3202 | {unit_state.internal_id: unit_state.unit_name}) |
3203 | |
3204 | @@ -1094,13 +1093,14 @@ |
3205 | unit_relation = states["unit_relation"] |
3206 | |
3207 | data = yield unit_relation.get_data() |
3208 | - self.assertEqual(yaml.load(data), {"private-address": None}) |
3209 | + self.assertEqual(serializer.load(data), {"private-address": None}) |
3210 | |
3211 | unit_relation_path = self.get_unit_settings_path(states) |
3212 | - self.client.set(unit_relation_path, yaml.dump(dict(hello="world"))) |
3213 | + self.client.set( |
3214 | + unit_relation_path, serializer.dump(dict(hello="world"))) |
3215 | |
3216 | data = yield unit_relation.get_data() |
3217 | - self.assertEqual(data, yaml.dump(dict(hello="world"))) |
3218 | + self.assertEqual(data, serializer.dump(dict(hello="world"))) |
3219 | |
3220 | @inlineCallbacks |
3221 | def test_set_data(self): |
3222 | @@ -1109,7 +1109,7 @@ |
3223 | unit_relation_path = self.get_unit_settings_path(states) |
3224 | yield unit_relation.set_data(dict(hello="world")) |
3225 | data, stat = yield self.client.get(unit_relation_path) |
3226 | - self.assertEqual(data, yaml.dump(dict(hello="world"))) |
3227 | + self.assertEqual(data, serializer.dump(dict(hello="world"))) |
3228 | |
3229 | @inlineCallbacks |
3230 | def test_get_relation_role(self): |
3231 | @@ -1137,7 +1137,7 @@ |
3232 | yield self.service_manager.remove_service_state(states["service"]) |
3233 | yield self.assertFailure( |
3234 | states["unit_relation"].get_relation_role(), StateChanged) |
3235 | - |
3236 | + |
3237 | @inlineCallbacks |
3238 | def test_get_related_unit_container(self): |
3239 | """Retrieve the container path of the related units.""" |
3240 | |
3241 | === modified file 'juju/state/tests/test_security.py' |
3242 | --- juju/state/tests/test_security.py 2012-02-22 09:23:16 +0000 |
3243 | +++ juju/state/tests/test_security.py 2012-09-15 19:26:18 +0000 |
3244 | @@ -1,5 +1,4 @@ |
3245 | import base64 |
3246 | -import yaml |
3247 | import zookeeper |
3248 | |
3249 | from twisted.internet.defer import inlineCallbacks, succeed |
3250 | @@ -11,6 +10,7 @@ |
3251 | ACL, Principal, GroupPrincipal, OTPPrincipal, TokenDatabase, |
3252 | SecurityPolicy, SecurityPolicyConnection) |
3253 | |
3254 | +from juju.lib import serializer |
3255 | from juju.lib.testing import TestCase |
3256 | from juju.tests.common import get_test_zookeeper_address |
3257 | |
3258 | @@ -230,7 +230,7 @@ |
3259 | |
3260 | data, stat = yield self.client.get(otp_path) |
3261 | |
3262 | - credentials = yaml.load(data) |
3263 | + credentials = serializer.load(data) |
3264 | self.assertEqual(credentials["name"], "foobar") |
3265 | self.assertEqual(credentials["password"], "secret") |
3266 | |
3267 | @@ -291,7 +291,7 @@ |
3268 | principal = Principal("zebra", "zoo") |
3269 | yield self.db.add(principal) |
3270 | content, stat = yield self.client.get("/token-test") |
3271 | - data = yaml.load(content) |
3272 | + data = serializer.load(content) |
3273 | self.assertEqual(data, {"zebra": principal.get_token()}) |
3274 | |
3275 | @inlineCallbacks |
3276 | @@ -300,7 +300,7 @@ |
3277 | yield self.db.add(principal) |
3278 | yield self.db.remove(principal) |
3279 | content, stat = yield self.client.get("/token-test") |
3280 | - data = yaml.load(content) |
3281 | + data = serializer.load(content) |
3282 | self.assertEqual(data, {"zebra": principal.get_token()}) |
3283 | |
3284 | @inlineCallbacks |
3285 | |
3286 | === modified file 'juju/state/tests/test_service.py' |
3287 | --- juju/state/tests/test_service.py 2012-07-03 07:30:53 +0000 |
3288 | +++ juju/state/tests/test_service.py 2012-09-15 19:26:18 +0000 |
3289 | @@ -1,7 +1,6 @@ |
3290 | import os |
3291 | import shutil |
3292 | |
3293 | -import yaml |
3294 | import zookeeper |
3295 | |
3296 | from twisted.internet.defer import inlineCallbacks, Deferred, returnValue |
3297 | @@ -10,6 +9,7 @@ |
3298 | from juju.charm.directory import CharmDirectory |
3299 | from juju.charm.tests import local_charm_id |
3300 | from juju.charm.tests.test_metadata import test_repository_path |
3301 | +from juju.lib import serializer |
3302 | from juju.machine.tests.test_constraints import ( |
3303 | dummy_constraints, dummy_cs, series_constraints) |
3304 | from juju.lib.pick import pick_attr |
3305 | @@ -178,7 +178,7 @@ |
3306 | "service-0000000000", "service-0000000001"]) |
3307 | |
3308 | content, stat = yield self.client.get("/services/service-0000000000") |
3309 | - details = yaml.load(content) |
3310 | + details = serializer.load(content) |
3311 | self.assertTrue(details) |
3312 | self.assertEquals(details.get("charm"), "local:series/dummy-1") |
3313 | self.assertFalse(isinstance(details.get("charm"), unicode)) |
3314 | @@ -513,7 +513,7 @@ |
3315 | # role path |
3316 | content, stat = yield self.client.get(role_path) |
3317 | self.assertTrue(stat) |
3318 | - node_info = yaml.load(content) |
3319 | + node_info = serializer.load(content) |
3320 | |
3321 | self.assertEqual( |
3322 | node_info["name"], |
3323 | @@ -527,7 +527,7 @@ |
3324 | unit_state.internal_id |
3325 | content, stat = yield self.client.get(settings_path) |
3326 | self.assertTrue(stat) |
3327 | - settings_info = yaml.load(content) |
3328 | + settings_info = serializer.load(content) |
3329 | |
3330 | # Verify that private address was set |
3331 | # we verify the content elsewhere |
3332 | @@ -1383,7 +1383,7 @@ |
3333 | self.assertIdentical(enabled, True) |
3334 | content, stat = yield self.client.get( |
3335 | "/units/%s/debug" % unit_state.internal_id) |
3336 | - data = yaml.load(content) |
3337 | + data = serializer.load(content) |
3338 | self.assertEqual(data, {"debug_hooks": ["*"]}) |
3339 | |
3340 | @inlineCallbacks |
3341 | @@ -1396,7 +1396,7 @@ |
3342 | self.assertIdentical(enabled, True) |
3343 | content, stat = yield self.client.get( |
3344 | "/units/%s/debug" % unit_state.internal_id) |
3345 | - data = yaml.load(content) |
3346 | + data = serializer.load(content) |
3347 | self.assertEqual(data, {"debug_hooks": |
3348 | ["db-relation-broken", "db-relation-changed"]}) |
3349 | |
3350 | @@ -1434,7 +1434,7 @@ |
3351 | self.assertIdentical(enabled, True) |
3352 | content, stat = yield self.client.get( |
3353 | "/units/%s/debug" % unit_state.internal_id) |
3354 | - data = yaml.load(content) |
3355 | + data = serializer.load(content) |
3356 | self.assertEqual(data, {"debug_hooks": ["db-relation-changed"]}) |
3357 | |
3358 | @inlineCallbacks |
3359 | @@ -2495,7 +2495,7 @@ |
3360 | # The default was never written to storage. |
3361 | data, stat = yield self.client.get( |
3362 | "/services/%s/config" % wordpress.internal_id) |
3363 | - self.assertEqual(yaml.load(data), {"foo": "bar"}) |
3364 | + self.assertEqual(serializer.load(data), {"foo": "bar"}) |
3365 | |
3366 | @inlineCallbacks |
3367 | def test_get_charm_state(self): |
3368 | @@ -2761,20 +2761,20 @@ |
3369 | yield unit_state.open_port(80, "tcp") |
3370 | content, stat = yield self.client.get(ports_path) |
3371 | self.assertEquals( |
3372 | - yaml.load(content), |
3373 | + serializer.load(content), |
3374 | {"open": [{"port": 80, "proto": "tcp"}]}) |
3375 | |
3376 | yield unit_state.open_port(53, "udp") |
3377 | content, stat = yield self.client.get(ports_path) |
3378 | self.assertEquals( |
3379 | - yaml.load(content), |
3380 | + serializer.load(content), |
3381 | {"open": [{"port": 80, "proto": "tcp"}, |
3382 | {"port": 53, "proto": "udp"}]}) |
3383 | |
3384 | yield unit_state.open_port(443, "tcp") |
3385 | content, stat = yield self.client.get(ports_path) |
3386 | self.assertEquals( |
3387 | - yaml.load(content), |
3388 | + serializer.load(content), |
3389 | {"open": [{"port": 80, "proto": "tcp"}, |
3390 | {"port": 53, "proto": "udp"}, |
3391 | {"port": 443, "proto": "tcp"}]}) |
3392 | @@ -2782,7 +2782,7 @@ |
3393 | yield unit_state.close_port(80, "tcp") |
3394 | content, stat = yield self.client.get(ports_path) |
3395 | self.assertEquals( |
3396 | - yaml.load(content), |
3397 | + serializer.load(content), |
3398 | {"open": [{"port": 53, "proto": "udp"}, |
3399 | {"port": 443, "proto": "tcp"}]}) |
3400 | |
3401 | |
3402 | === modified file 'juju/state/tests/test_topology.py' |
3403 | --- juju/state/tests/test_topology.py 2012-04-12 18:39:20 +0000 |
3404 | +++ juju/state/tests/test_topology.py 2012-09-15 19:26:18 +0000 |
3405 | @@ -1,6 +1,5 @@ |
3406 | -import yaml |
3407 | - |
3408 | from juju.errors import IncompatibleVersion |
3409 | +from juju.lib import serializer |
3410 | from juju.lib.testing import TestCase |
3411 | from juju.state.endpoint import RelationEndpoint |
3412 | from juju.state.topology import ( |
3413 | @@ -891,7 +890,7 @@ |
3414 | loaded back. |
3415 | """ |
3416 | empty_data = self.topology.dump() |
3417 | - self.assertEquals(yaml.load(empty_data), {"version": VERSION}) |
3418 | + self.assertEquals(serializer.load(empty_data), {"version": VERSION}) |
3419 | self.topology.add_machine("m-0") |
3420 | machine_data = self.topology.dump() |
3421 | self.topology.parse(empty_data) |
3422 | @@ -902,7 +901,7 @@ |
3423 | def test_incompatible_version(self): |
3424 | """Verify `IncompatibleVersion` raised if using old topology.""" |
3425 | empty_data = self.topology.dump() |
3426 | - self.assertEquals(yaml.load(empty_data), {"version": VERSION}) |
3427 | + self.assertEquals(serializer.load(empty_data), {"version": VERSION}) |
3428 | self.topology.add_machine("m-0") |
3429 | machine_data = self.topology.dump() |
3430 | self.topology.parse(machine_data) |
3431 | @@ -1265,62 +1264,3 @@ |
3432 | "r-0", "s-0", "riak", "peer") |
3433 | self.assertEqual(self.topology.get_relation_between_endpoints( |
3434 | [riak_ep]), "r-0") |
3435 | - |
3436 | - |
3437 | -# Topology migration tests use these. defined at global scope to ease |
3438 | -# string formatting |
3439 | - |
3440 | -TOPOLOGY_V1 = """ |
3441 | -relations: |
3442 | - relation-0000000000: |
3443 | - - mysql |
3444 | - - service-0000000000: {name: db, role: client} |
3445 | - service-0000000001: {name: server, role: server} |
3446 | -version: 1 |
3447 | -""" |
3448 | - |
3449 | -TOPOLOGY_V2_EXPECTED = """ |
3450 | -relations: |
3451 | - interface: mysql |
3452 | - relation-0000000000: {} |
3453 | - scope: global |
3454 | - services: |
3455 | - service-0000000000: {name: db, role: client} |
3456 | - service-0000000001: {name: server, role: server} |
3457 | -version: 2 |
3458 | -""" |
3459 | - |
3460 | - |
3461 | -class TestMigrations(TestCase): |
3462 | - |
3463 | - # DISABLED: We don't do transparent data migrations, till we |
3464 | - # have explicit juju core code upgrades, else older agents |
3465 | - # will die on new topology formats. |
3466 | - def xtest_migration_v1_to_v2(self): |
3467 | - """Parse a fragment of a version 1 topology |
3468 | - |
3469 | - Ensure that a version 2 topology is emitted. |
3470 | - """ |
3471 | - topology = InternalTopology() |
3472 | - topology.parse(TOPOLOGY_V1) |
3473 | - self.assertEqual(topology.get_version(), 2) |
3474 | - self.assertEqual(topology.dump().strip(), |
3475 | - TOPOLOGY_V2_EXPECTED.strip()) |
3476 | - |
3477 | - def test_migration_v1_to_unknown(self): |
3478 | - """Parse a fragment of a version 1 topology |
3479 | - |
3480 | - Ensure that a version 2 topology is emitted. |
3481 | - """ |
3482 | - topology = InternalTopology() |
3483 | - |
3484 | - actual_version = VERSION |
3485 | - import juju |
3486 | - self.patch(juju.state.topology, "VERSION", actual_version + 1) |
3487 | - |
3488 | - ex = self.assertRaises(IncompatibleVersion, |
3489 | - topology.parse, TOPOLOGY_V1) |
3490 | - self.assertEqual( |
3491 | - str(ex), |
3492 | - "Incompatible juju protocol versions (found 1, want %d)" % ( |
3493 | - juju.state.topology.VERSION)) |
3494 | |
3495 | === modified file 'juju/state/tests/test_utils.py' |
3496 | --- juju/state/tests/test_utils.py 2012-04-06 14:42:06 +0000 |
3497 | +++ juju/state/tests/test_utils.py 2012-09-15 19:26:18 +0000 |
3498 | @@ -8,8 +8,8 @@ |
3499 | from twisted.internet import reactor |
3500 | from twisted.internet.defer import inlineCallbacks |
3501 | from twisted.internet.threads import deferToThread |
3502 | -import yaml |
3503 | |
3504 | +from juju.lib import serializer |
3505 | from juju.lib.testing import TestCase |
3506 | from juju.state.base import StateBase |
3507 | from juju.state.errors import StateChanged, StateNotFound |
3508 | @@ -382,7 +382,7 @@ |
3509 | |
3510 | # and a direct look at zk should work as well |
3511 | zk_data, stat = yield self.client.get(self.path) |
3512 | - zk_data = yaml.load(zk_data) |
3513 | + zk_data = serializer.load(zk_data) |
3514 | self.assertEqual(zk_data, options) |
3515 | |
3516 | @inlineCallbacks |
3517 | @@ -465,7 +465,7 @@ |
3518 | |
3519 | # and a direct look at zk should work as well |
3520 | zk_data, stat = yield self.client.get(self.path) |
3521 | - zk_data = yaml.load(zk_data) |
3522 | + zk_data = serializer.load(zk_data) |
3523 | self.assertEqual(zk_data, options) |
3524 | |
3525 | @inlineCallbacks |
3526 | @@ -678,13 +678,14 @@ |
3527 | |
3528 | @inlineCallbacks |
3529 | def test_get_missing(self): |
3530 | - yield self.client.create(self.path, yaml.dump({"foo": "bar"})) |
3531 | + yield self.client.create(self.path, serializer.dump({"foo": "bar"})) |
3532 | mixee = TestMixee(self.client, self.path) |
3533 | self.assertEquals((yield mixee.get()), None) |
3534 | |
3535 | @inlineCallbacks |
3536 | def test_get_exists(self): |
3537 | - yield self.client.create(self.path, yaml.dump({"key": "butterfly"})) |
3538 | + yield self.client.create( |
3539 | + self.path, serializer.dump({"key": "butterfly"})) |
3540 | mixee = TestMixee(self.client, self.path) |
3541 | self.assertEquals((yield mixee.get()), "butterfly") |
3542 | |
3543 | @@ -694,13 +695,14 @@ |
3544 | mixee = TestMixee(self.client, self.path) |
3545 | yield mixee.set("caterpillar") |
3546 | content, _ = yield self.client.get(self.path) |
3547 | - self.assertEquals(yaml.load(content), {"key": "caterpillar"}) |
3548 | + self.assertEquals(serializer.load(content), {"key": "caterpillar"}) |
3549 | |
3550 | @inlineCallbacks |
3551 | def test_set_safely(self): |
3552 | - yield self.client.create(self.path, yaml.dump({"foo": "bar"})) |
3553 | + yield self.client.create(self.path, serializer.dump({"foo": "bar"})) |
3554 | mixee = TestMixee(self.client, self.path) |
3555 | yield mixee.set("cocoon") |
3556 | content, _ = yield self.client.get(self.path) |
3557 | - self.assertEquals(yaml.load(content), {"foo": "bar", "key": "cocoon"}) |
3558 | + self.assertEquals( |
3559 | + serializer.load(content), {"foo": "bar", "key": "cocoon"}) |
3560 | |
3561 | |
3562 | === modified file 'juju/state/topology.py' |
3563 | --- juju/state/topology.py 2012-04-12 18:39:20 +0000 |
3564 | +++ juju/state/topology.py 2012-09-15 19:26:18 +0000 |
3565 | @@ -1,7 +1,7 @@ |
3566 | -import yaml |
3567 | + |
3568 | |
3569 | from juju.errors import IncompatibleVersion |
3570 | - |
3571 | +from juju.lib import serializer |
3572 | |
3573 | # The protocol version, which is stored in the /topology node under |
3574 | # the "version" key. The protocol version should *only* be updated |
3575 | @@ -59,7 +59,7 @@ |
3576 | This string may be provided to the :method:`parse` to |
3577 | reestablish the same topology state back. |
3578 | """ |
3579 | - return yaml.safe_dump(self._state) |
3580 | + return serializer.dump(self._state) |
3581 | |
3582 | def parse(self, data): |
3583 | """Parse the dumped data provided and restore internal state. |
3584 | @@ -67,12 +67,11 @@ |
3585 | The provided data must necessarily have been retrieved by |
3586 | calling the :method:`dump`. |
3587 | """ |
3588 | - parsed = yaml.load(data) |
3589 | + parsed = serializer.load(data) |
3590 | self._state = parsed |
3591 | version = self.get_version() |
3592 | if version != VERSION: |
3593 | - # This will raise if it cannot update the topology. |
3594 | - migrate_topology(self) |
3595 | + raise IncompatibleVersion(version, VERSION) |
3596 | |
3597 | def get_version(self): |
3598 | return self._state.get("version", 0) |
3599 | @@ -573,79 +572,3 @@ |
3600 | else: |
3601 | return relation_id |
3602 | return None |
3603 | - |
3604 | - |
3605 | -def _migrate_version_1(topology): |
3606 | - """Migrate topology version 1 to version 2. |
3607 | - |
3608 | - This change includes the transition from:: |
3609 | - |
3610 | - relations: |
3611 | - relation-0000000000: |
3612 | - - mysql |
3613 | - - service-0000000000: {name: db, role: client} |
3614 | - service-0000000001: {name: server, role: server} |
3615 | - |
3616 | - to:: |
3617 | - |
3618 | - relations |
3619 | - relation-00000001: |
3620 | - interface: name |
3621 | - scope: name |
3622 | - services: |
3623 | - service-00000001: {name: name, role: role} |
3624 | - service-00000002: {name: name, role: role} |
3625 | - |
3626 | - for all relations. |
3627 | - |
3628 | - """ |
3629 | - version = topology.get_version() |
3630 | - if version > 1: |
3631 | - return topology |
3632 | - elif version != 1: |
3633 | - raise IncompatibleVersion(version, VERSION) |
3634 | - |
3635 | - relations = topology._state.get("relations") |
3636 | - if relations: |
3637 | - new_relations = {} |
3638 | - for relation, relation_data in relations.items(): |
3639 | - new_relations[relation] = {} |
3640 | - relation_type, relation_services = relation_data |
3641 | - new_relations["interface"] = relation_type |
3642 | - new_relations["scope"] = "global" |
3643 | - new_relations["services"] = relation_services |
3644 | - |
3645 | - topology._state["relations"] = new_relations |
3646 | - |
3647 | - topology._state["version"] = 2 |
3648 | - return topology |
3649 | - |
3650 | - |
3651 | -# A dict of version migration plans for VERSION n (where n is the key) |
3652 | -# to version n + 1. migrate_version will be called until the topology |
3653 | -# version is equal to VERSION. If no migration plan exists FAIL |
3654 | -_VERSION_MIGRATION = { |
3655 | - |
3656 | -# DISABLED: We can't migrate the topology till we can migrate older |
3657 | -# code currently running in the env that depends on a previous format. |
3658 | -# 1: _migrate_version_1 |
3659 | -} |
3660 | - |
3661 | - |
3662 | -def migrate_topology(topology): |
3663 | - """Migrate topology version to current. |
3664 | - |
3665 | - Does an in-place (destructive) stepwise modification of topology |
3666 | - state from its current version to the VERSION represented in this |
3667 | - module. |
3668 | - """ |
3669 | - version = topology.get_version() |
3670 | - if version == VERSION: |
3671 | - return topology |
3672 | - |
3673 | - current_version = version |
3674 | - while current_version < VERSION: |
3675 | - if current_version not in _VERSION_MIGRATION: |
3676 | - raise IncompatibleVersion(version, VERSION) |
3677 | - _VERSION_MIGRATION[current_version](topology) |
3678 | - current_version = topology.get_version() |
3679 | |
3680 | === modified file 'juju/state/utils.py' |
3681 | --- juju/state/utils.py 2012-03-09 13:24:58 +0000 |
3682 | +++ juju/state/utils.py 2012-09-15 19:26:18 +0000 |
3683 | @@ -7,9 +7,10 @@ |
3684 | from twisted.internet.defer import inlineCallbacks, returnValue |
3685 | from twisted.internet.threads import deferToThread |
3686 | from txzookeeper.utils import retry_change |
3687 | -import yaml |
3688 | + |
3689 | import zookeeper |
3690 | |
3691 | +from juju.lib import serializer |
3692 | from juju.state.errors import StateChanged |
3693 | from juju.state.errors import StateNotFound |
3694 | |
3695 | @@ -177,7 +178,7 @@ |
3696 | self._cache = {} |
3697 | try: |
3698 | data, stat = yield self._client.get(self._path) |
3699 | - data = yaml.load(data) |
3700 | + data = serializer.load(data) |
3701 | if data: |
3702 | self._pristine_cache = data |
3703 | self._cache = data.copy() |
3704 | @@ -228,7 +229,7 @@ |
3705 | def apply_changes(content, stat): |
3706 | """Apply the local state to the Zookeeper node state.""" |
3707 | del changes[:] |
3708 | - current = yaml.load(content) if content else {} |
3709 | + current = serializer.load(content) if content else {} |
3710 | missing = object() |
3711 | for key in set(pristine_cache).union(cache): |
3712 | old_value = pristine_cache.get(key, missing) |
3713 | @@ -244,7 +245,7 @@ |
3714 | elif key in current: |
3715 | del current[key] |
3716 | changes.append(DeletedItem(key, old_value)) |
3717 | - return yaml.safe_dump(current) |
3718 | + return serializer.dump(current) |
3719 | |
3720 | # Apply the change till it takes. |
3721 | yield retry_change(self._client, self._path, apply_changes) |
3722 | |
3723 | === modified file 'juju/unit/lifecycle.py' |
3724 | --- juju/unit/lifecycle.py 2012-05-01 00:26:25 +0000 |
3725 | +++ juju/unit/lifecycle.py 2012-09-15 19:26:18 +0000 |
3726 | @@ -2,7 +2,6 @@ |
3727 | import logging |
3728 | import shutil |
3729 | import tempfile |
3730 | -import yaml |
3731 | |
3732 | from twisted.internet.defer import ( |
3733 | inlineCallbacks, DeferredLock, DeferredList, returnValue) |
3734 | @@ -10,6 +9,7 @@ |
3735 | from juju.errors import CharmUpgradeError |
3736 | from juju.hooks.invoker import Invoker |
3737 | from juju.hooks.scheduler import HookScheduler |
3738 | +from juju.lib import serializer |
3739 | from juju.state.hook import ( |
3740 | DepartedRelationHookContext, HookContext, RelationChange) |
3741 | from juju.state.errors import StopWatcher, UnitRelationStateNotFound |
3742 | @@ -493,7 +493,7 @@ |
3743 | state_dict = {} |
3744 | for relation_wf in self._relations.itervalues(): |
3745 | state_dict.update(relation_wf.get_relation_info()) |
3746 | - state = yaml.dump(state_dict) |
3747 | + state = serializer.dump(state_dict) |
3748 | temp_path = self._known_relations_path + "~" |
3749 | |
3750 | with open(temp_path, "w") as f: |
3751 | @@ -519,7 +519,7 @@ |
3752 | relations_by_id = dict((r.internal_relation_id, r) for r in relations) |
3753 | |
3754 | with open(self._known_relations_path) as f: |
3755 | - known_relations = yaml.load(f.read()) |
3756 | + known_relations = serializer.load(f.read()) |
3757 | |
3758 | for relation_id, relation_info in known_relations.items(): |
3759 | if relation_id in relations_by_id: |
3760 | |
3761 | === modified file 'juju/unit/tests/test_lifecycle.py' |
3762 | --- juju/unit/tests/test_lifecycle.py 2012-05-04 03:39:27 +0000 |
3763 | +++ juju/unit/tests/test_lifecycle.py 2012-09-15 19:26:18 +0000 |
3764 | @@ -6,7 +6,6 @@ |
3765 | import stat |
3766 | import sys |
3767 | |
3768 | -import yaml |
3769 | import zookeeper |
3770 | |
3771 | from twisted.internet.defer import (inlineCallbacks, Deferred, |
3772 | @@ -20,6 +19,7 @@ |
3773 | from juju.errors import CharmInvocationError, CharmError, CharmUpgradeError |
3774 | from juju.hooks.invoker import Invoker |
3775 | from juju.hooks.executor import HookExecutor |
3776 | +from juju.lib import serializer |
3777 | from juju.machine.tests.test_constraints import series_constraints |
3778 | |
3779 | from juju.state.endpoint import RelationEndpoint |
3780 | @@ -1083,7 +1083,8 @@ |
3781 | self.assertTrue(self.lifecycle.watching) |
3782 | self.assertFalse(self.lifecycle.executing) |
3783 | |
3784 | - rel_states["unit_relation"].set_data(yaml.dump(dict(hello="world"))) |
3785 | + rel_states["unit_relation"].set_data( |
3786 | + serializer.dump(dict(hello="world"))) |
3787 | # Sleep to give an error a chance. |
3788 | yield self.sleep(0.1) |
3789 | self.assertFalse(changed_executed.called) |
3790 | |
3791 | === modified file 'juju/unit/tests/test_workflow.py' |
3792 | --- juju/unit/tests/test_workflow.py 2012-04-01 00:48:01 +0000 |
3793 | +++ juju/unit/tests/test_workflow.py 2012-09-15 19:26:18 +0000 |
3794 | @@ -2,7 +2,6 @@ |
3795 | import itertools |
3796 | import logging |
3797 | import os |
3798 | -import yaml |
3799 | |
3800 | from twisted.internet.defer import inlineCallbacks, returnValue |
3801 | |
3802 | @@ -12,6 +11,7 @@ |
3803 | |
3804 | from juju.charm.directory import CharmDirectory |
3805 | from juju.charm.url import CharmURL |
3806 | +from juju.lib import serializer |
3807 | from juju.lib.statemachine import WorkflowState |
3808 | from juju.unit.lifecycle import UnitLifecycle, UnitRelationLifecycle |
3809 | from juju.unit.workflow import ( |
3810 | @@ -40,10 +40,10 @@ |
3811 | |
3812 | state = open(workflow.state_file_path).read() |
3813 | history = open(workflow.state_history_path) |
3814 | - zk_state = yaml.load(data)["workflow_state"] |
3815 | - returnValue((yaml.load(state), |
3816 | - [yaml.load(r[0]) for r in csv.reader(history)], |
3817 | - yaml.load(zk_state[history_id]))) |
3818 | + zk_state = serializer.load(data)["workflow_state"] |
3819 | + returnValue((serializer.load(state), |
3820 | + [serializer.load(r[0]) for r in csv.reader(history)], |
3821 | + serializer.load(zk_state[history_id]))) |
3822 | |
3823 | @inlineCallbacks |
3824 | def assert_history(self, expected, **kwargs): |
3825 | |
3826 | === modified file 'juju/unit/workflow.py' |
3827 | --- juju/unit/workflow.py 2012-03-29 07:52:41 +0000 |
3828 | +++ juju/unit/workflow.py 2012-09-15 19:26:18 +0000 |
3829 | @@ -1,4 +1,3 @@ |
3830 | -import yaml |
3831 | import csv |
3832 | import os |
3833 | import logging |
3834 | @@ -9,6 +8,7 @@ |
3835 | from txzookeeper.utils import retry_change |
3836 | |
3837 | from juju.errors import CharmError, FileNotFound |
3838 | +from juju.lib import serializer |
3839 | from juju.lib.statemachine import ( |
3840 | WorkflowState, Workflow, Transition, TransitionError) |
3841 | |
3842 | @@ -223,16 +223,16 @@ |
3843 | @inlineCallbacks |
3844 | def _store(self, state_dict): |
3845 | """Store the workflow state dictionary in zookeeper.""" |
3846 | - state_serialized = yaml.safe_dump(state_dict) |
3847 | + state_serialized = serializer.dump(state_dict) |
3848 | |
3849 | def update_state(content, stat): |
3850 | - unit_data = yaml.load(content) |
3851 | + unit_data = serializer.load(content) |
3852 | if not unit_data: |
3853 | unit_data = {} |
3854 | |
3855 | persistent_workflow = unit_data.setdefault("workflow_state", {}) |
3856 | persistent_workflow[self.zk_state_id] = state_serialized |
3857 | - return yaml.dump(unit_data) |
3858 | + return serializer.dump(unit_data) |
3859 | |
3860 | yield retry_change(self._client, self.zk_state_path, update_state) |
3861 | yield super(ZookeeperWorkflowState, self)._store( |
3862 | @@ -245,8 +245,8 @@ |
3863 | data, stat = yield self._client.get(self.zk_state_path) |
3864 | except NoNodeException: |
3865 | returnValue({"state": None}) |
3866 | - unit_data = yaml.load(data) |
3867 | - data = yaml.load(unit_data.get("workflow_state", {}).get( |
3868 | + unit_data = serializer.load(data) |
3869 | + data = serializer.load(unit_data.get("workflow_state", {}).get( |
3870 | self.zk_state_id, "")) |
3871 | returnValue(data) |
3872 | |
3873 | @@ -273,7 +273,7 @@ |
3874 | Internally the history file is stored a csv, with a new |
3875 | row per entry with CSV escaping. |
3876 | """ |
3877 | - state_serialized = yaml.safe_dump(state_dict) |
3878 | + state_serialized = serializer.dump(state_dict) |
3879 | # State File |
3880 | with open(self.state_file_path, "w") as handle: |
3881 | handle.write(state_serialized) |
3882 | @@ -295,7 +295,7 @@ |
3883 | content = handle.read() |
3884 | |
3885 | # TODO load ZK state and overwrite with disk state if different? |
3886 | - return yaml.load(content) |
3887 | + return serializer.load(content) |
3888 | |
3889 | |
3890 | class WorkflowStateClient(ZookeeperWorkflowState): |
Reviewers: mp+123470_ code.launchpad. net,
Message:
Please take a look.
Description:
ensure use of py yaml c extension for speed.
Backwards compatible. Drops test time in half. Should improve speed
across the board, things like status will see marked improvements.
https:/ /code.launchpad .net/~hazmat/ juju/fast- yaml/+merge/ 123470
(do not edit description out of merge proposal)
Please review this at https:/ /codereview. appspot. com/6493100/
Affected files: tests/test_ unit.py config. py metadata. py tests/test_ base.py tests/test_ bundle. py tests/test_ config. py tests/test_ directory. py tests/test_ metadata. py tests/test_ publisher. py config_ set.py constraints_ get.py deploy. py initialize. py status. py tests/test_ add_unit. py tests/test_ bootstrap. py tests/test_ config_ get.py tests/test_ config_ set.py tests/test_ constraints_ get.py tests/test_ deploy. py tests/test_ destroy_ environment. py tests/test_ expose. py tests/test_ initialize. py tests/test_ open_tunnel. py tests/test_ remove_ relation. py tests/test_ scp.py tests/test_ status. py tests/test_ unexpose. py tests/test_ upgrade_ charm.py tests/test_ utils.py t/config. py t/tests/ test_config. py protocol. py scheduler. py tests/test_ cli.py tests/test_ invoker. py tests/test_ scheduler. py serializer. py common/ cloudinit. py common/ state.py common/ tests/test_ cloudinit. py common/ tests/test_ findzookeepers. py common/ tests/test_ state.py common/ tests/test_ utils.py common/ utils.py ec2/tests/ common. py ec2/tests/ test_bootstrap. py ec2/tests/ test_findzookee per.py ec2/tests/ test_launch. py ec2/tests/ test_provider. py ec2/tests/ test_state. py local/files. py local/tests/ test_agent. py local/tests/ test_container. py local/tests/ test_files. py openstack/ tests/test_ bootstrap. py openstack/ tests/test_ launch. py openstack/ tests/test_ state.py orchestra/ tests/common. py orchestra/ tests/test_ bootstrap. py orchestra/ tests/test_ findzookeepers. py orchestra/ tests/test_ state.py environment. py machine. py relation. ...
A [revision details]
M juju/agents/
M juju/charm/
M juju/charm/
M juju/charm/
M juju/charm/
M juju/charm/
M juju/charm/
M juju/charm/
M juju/charm/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/control/
M juju/environmen
M juju/environmen
M juju/hooks/
M juju/hooks/
M juju/hooks/
M juju/hooks/
M juju/hooks/
M juju/lib/format.py
A juju/lib/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/providers/
M juju/state/charm.py
M juju/state/
M juju/state/hook.py
M juju/state/
M juju/state/