Merge lp:~fwereade/pyjuju/provider-base into lp:pyjuju

Proposed by William Reade
Status: Merged
Approved by: Kapil Thangavelu
Approved revision: 325
Merged at revision: 319
Proposed branch: lp:~fwereade/pyjuju/provider-base
Merge into: lp:pyjuju
Prerequisite: lp:~fwereade/pyjuju/spike-catchup
Diff against target: 1364 lines (+566/-318)
16 files modified
ensemble/control/shutdown.py (+1/-1)
ensemble/control/tests/test_shutdown.py (+1/-1)
ensemble/providers/common/base.py (+173/-0)
ensemble/providers/common/bootstrap.py (+4/-5)
ensemble/providers/common/launch.py (+13/-10)
ensemble/providers/common/tests/test_base.py (+135/-0)
ensemble/providers/common/tests/test_findzookeepers.py (+41/-31)
ensemble/providers/common/tests/test_launch.py (+31/-34)
ensemble/providers/common/tests/test_state.py (+2/-1)
ensemble/providers/common/tests/test_utils.py (+1/-1)
ensemble/providers/dummy.py (+15/-10)
ensemble/providers/ec2/__init__.py (+24/-84)
ensemble/providers/ec2/tests/test_provider.py (+4/-2)
ensemble/providers/ec2/tests/test_shutdown.py (+103/-14)
ensemble/providers/orchestra/__init__.py (+14/-120)
ensemble/providers/tests/test_dummy.py (+4/-4)
To merge this branch: bzr merge lp:~fwereade/pyjuju/provider-base
Reviewer Review Type Date Requested Status
Kapil Thangavelu (community) Approve
Gustavo Niemeyer Approve
Review via email: mp+71272@code.launchpad.net

Description of the change

The ec2 and orchestra MachineProviders have been getting much more similar, and it's time to create a common base class; as part of this change, I've altered the interface slightly such that the base class provides .shutdown_machine and .destroy_environment (renamed from .shutdown); subclasses are expected to implement the new .shutdown_machines method.

I've left the DummyProvider separate (besides fixing the interface), because *many* tests use it in such a way as to assume it's already bootstrapped (or doesn't need to be bootstrapped, from an alternative perspective), and it seems more sensible to keep it optimized for the common case than to (1) rearrange its code and (2) demand that all the clients change to accommodate things they don't really care about.

To post a comment you must log in.
Revision history for this message
William Reade (fwereade) wrote :

(er, didn't directly test the base class itself... marked WIP again before someone does it for me)

lp:~fwereade/pyjuju/provider-base updated
319. By William Reade

unsaved file

Revision history for this message
William Reade (fwereade) wrote :

> (er, didn't directly test the base class itself... marked WIP again before
> someone does it for me)

OK, I think the base class is now thoroughly tested; the parts that aren't hit in test_base.py are hit via MachineProviderBase subclasses in the other tests, which only mock out/override the parts of the interface which would otherwise raise NotImplemented. One exception: the LaunchMachine test stubs out get_zookeeper_machines separately, to avoid adding to the already-excessive complexity, but I think we have that covered from all angles already.

This branch now makes me think that find_zookeepers, SaveState, and LoadState (at least) should lose their independent existence and just become plain MachineProviderBase methods, but that feels like a job for another branch.

lp:~fwereade/pyjuju/provider-base updated
320. By William Reade

merge parent

321. By William Reade

merge parent

322. By William Reade

merge parent

323. By William Reade

merge parent

Revision history for this message
Gustavo Niemeyer (niemeyer) wrote :

Very good branch, thanks!

+1, with [1] sorted.

[1]

+ # XXX something's a bit inelegant here; also see Bootstrap operation
+ launch = self.launch_machine_class(self)
+ return launch.run(machine_data=machine_data)

Indeed.. it doesn't look necessary. launch_machine_class can go away
entirely and the subclass can simply define its own start_machie method,
with the base raising NotImplemented. I bet it's going to be less code
in the end.

+ super(MachineProvider, self).__init__(
+ environment_name, config, EC2LaunchMachine)

Yeah, this really sounds wrong. Why EC2LaunchMachine and not
everything else? Let's drop it.

[2]

+ data = super(MachineProvider, self).get_serialization_data()
+ data.setdefault("access-key", os.environ.get("AWS_ACCESS_KEY_ID"))
+ data.setdefault("secret-key", os.environ.get("AWS_SECRET_ACCESS_KEY"))

Nice clean up!

[3]

+ """Retrieve an S3-backed C{FileStorage}."""
+ return FileStorage(self.s3, self.config["control-bucket"])

Good stuff too.

review: Approve
lp:~fwereade/pyjuju/provider-base updated
324. By William Reade

merge trunk

325. By William Reade

merge lp:~fwereade/ensemble/provider-base-launch-machine

Revision history for this message
William Reade (fwereade) wrote :

[1] addressed in https://code.launchpad.net/~fwereade/ensemble/provider-base-launch-machine/+merge/71850 (now merged); includes a general rename of "bootstrap" -- in the narrow context of launching a machine which will run a zookeeper and a provisioning agent, as opposed to the broader start-ensemble-itself context -- to "master".

Revision history for this message
Kapil Thangavelu (hazmat) wrote :

looks very nice +1

review: Approve
lp:~fwereade/pyjuju/provider-base updated
326. By William Reade

merge trunk

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'ensemble/control/shutdown.py'
2--- ensemble/control/shutdown.py 2011-06-09 15:13:03 +0000
3+++ ensemble/control/shutdown.py 2011-08-17 16:02:43 +0000
4@@ -29,4 +29,4 @@
5 returnValue(None)
6 options.log.info("Shutting down environment %r (type: %s)..." % (
7 environment.name, environment.type))
8- yield provider.shutdown()
9+ yield provider.destroy_environment()
10
11=== modified file 'ensemble/control/tests/test_shutdown.py'
12--- ensemble/control/tests/test_shutdown.py 2011-06-09 15:13:03 +0000
13+++ ensemble/control/tests/test_shutdown.py 2011-08-17 16:02:43 +0000
14@@ -89,7 +89,7 @@
15 return succeed(True)
16
17 provider = self.mocker.patch(MachineProvider)
18- provider.shutdown()
19+ provider.destroy_environment()
20 self.mocker.call(track_shutdown_call, with_object=True)
21
22 self.setup_exit(0)
23
24=== added file 'ensemble/providers/common/base.py'
25--- ensemble/providers/common/base.py 1970-01-01 00:00:00 +0000
26+++ ensemble/providers/common/base.py 2011-08-17 16:02:43 +0000
27@@ -0,0 +1,173 @@
28+import copy
29+from operator import itemgetter
30+
31+from ensemble.environment.errors import EnvironmentsConfigError
32+from ensemble.providers.common.bootstrap import Bootstrap
33+from ensemble.providers.common.findzookeepers import find_zookeepers
34+from ensemble.providers.common.state import SaveState, LoadState
35+from ensemble.providers.common.utils import get_user_authorized_keys
36+
37+
38+class MachineProviderBase(object):
39+ """Base class supplying common functionality for MachineProviders.
40+
41+ To write a working subclass, you will need to override the following
42+ methods:
43+
44+ * connect
45+ * get_file_storage
46+ * start_machine
47+ * get_machines
48+ * shutdown_machines
49+ * open_port
50+ * close_port
51+ * get_opened_ports
52+
53+ You may want to override the following methods, but you should be careful
54+ to call MachineProviderBase's implementation (or be very sure you don't
55+ need to:
56+
57+ * __init__
58+ * get_serialization_data
59+
60+ You probably shouldn't override anything else.
61+ """
62+
63+ def __init__(self, environment_name, config):
64+ if ("authorized-keys-path" in config and
65+ "authorized-keys" in config):
66+ raise EnvironmentsConfigError(
67+ "Environment config cannot define both authorized-keys "
68+ "and authorized-keys-path. Pick one!")
69+
70+ self.environment_name = environment_name
71+ self.config = config
72+
73+ def get_serialization_data(self):
74+ """Get provider configuration suitable for serialization."""
75+ data = copy.deepcopy(self.config)
76+ data["authorized-keys"] = get_user_authorized_keys(data)
77+ # Not relevant, on a remote system.
78+ data.pop("authorized-keys-path", None)
79+ return data
80+
81+ #================================================================
82+ # Subclasses need to implement their own versions of everything
83+ # in the following block
84+
85+ def connect(self, share=False):
86+ """Connect to the zookeeper ensemble running in the machine provider.
87+
88+ @param share: Requests sharing of the connection with other clients
89+ attempting to connect to the same provider, if that's feasible.
90+
91+ returns an open C{txzookeeper.client.ZookeeperClient} and a
92+ C{ensemble.storage.connection.TunnelProtocol}
93+ """
94+ raise NotImplementedError()
95+
96+ def get_file_storage(self):
97+ """Retrieve the provider C{FileStorage} abstraction."""
98+ raise NotImplementedError()
99+
100+ def start_machine(self, machine_data, master=False):
101+ """Start a machine in the provider.
102+
103+ @param machine_data: a dictionary of data to pass along to the newly
104+ launched machine.
105+
106+ @param master: if True, machine will initialize the ensemble admin
107+ and run a provisioning agent.
108+ """
109+ raise NotImplementedError()
110+
111+ def get_machines(self, instance_ids=()):
112+ """List machines running in the provider.
113+
114+ @param instance_ids: ids of instances you want to get. Leave empty
115+ to list all machines owned by this provider.
116+
117+ @return: a list of provider-specific ProviderMachines.
118+
119+ @raise: MachinesNotFound
120+ """
121+ raise NotImplementedError()
122+
123+ def shutdown_machines(self, requested_machines=()):
124+ """Terminate machines associated with this provider.
125+
126+ @param requested_machines: list of machines to shut down; leave
127+ empty to shut down all associated machines.
128+ """
129+ raise NotImplementedError()
130+
131+ def open_port(self, machine, machine_id, port, protocol="tcp"):
132+ """Authorizes `port` using `protocol` for `machine`."""
133+ raise NotImplementedError()
134+
135+ def close_port(self, machine, machine_id, port, protocol="tcp"):
136+ """Revokes `port` using `protocol` for `machine`."""
137+ raise NotImplementedError()
138+
139+ def get_opened_ports(self, machine, machine_id):
140+ """Returns a set of open (port, protocol) pairs for `machine`."""
141+ raise NotImplementedError()
142+
143+ #================================================================
144+ # Subclasses will not generally need to override the methods in
145+ # this block
146+
147+ def get_zookeeper_machines(self):
148+ """Find running zookeeper instances.
149+
150+ @return: the first valid instance found as a single element list.
151+
152+ @raise: EnvironmentNotFound
153+ """
154+ return find_zookeepers(self)
155+
156+ def bootstrap(self):
157+ """Bootstrap an ensemble server in the provider."""
158+ return Bootstrap(self).run()
159+
160+ def get_machine(self, instance_id):
161+ """Retrieve a provider machine by instance id.
162+
163+ @param instance_id: instance_id of the Machine you want to get.
164+
165+ @raise: MachinesNotFound
166+ """
167+ d = self.get_machines([instance_id])
168+ d.addCallback(itemgetter(0))
169+ return d
170+
171+ def shutdown_machine(self, machine):
172+ """Terminate one machine associated with this provider.
173+
174+ @param machine: machine to shut down.
175+
176+ @raise: MachinesNotFound
177+ """
178+ d = self.shutdown_machines([machine])
179+ d.addCallback(itemgetter(0))
180+ return d
181+
182+ def destroy_environment(self):
183+ """Clear ensemble state and terminate all associated machines"""
184+ return self.shutdown_machines()
185+
186+ def save_state(self, state):
187+ """Save state to the provider.
188+
189+ @param state
190+ @type dict
191+ """
192+ return SaveState(self).run(state)
193+
194+ def load_state(self):
195+ """Load state from the provider.
196+
197+ @return: a dictionary.
198+ """
199+ return LoadState(self).run()
200+
201
202=== modified file 'ensemble/providers/common/bootstrap.py'
203--- ensemble/providers/common/bootstrap.py 2011-08-04 09:50:57 +0000
204+++ ensemble/providers/common/bootstrap.py 2011-08-17 16:02:43 +0000
205@@ -41,11 +41,10 @@
206 return storage.put(_VERIFY_PATH, StringIO("storage is writable"))
207
208 def _cannot_write(self, failure):
209- raise ProviderError("Bootstrap aborted: file storage not writable (%s)"
210- % str(failure.value))
211+ raise ProviderError(
212+ "Bootstrap aborted because file storage is not writable: %s"
213+ % str(failure.value))
214
215 def _launch_machine(self, unused):
216 log.debug("Launching Ensemble bootstrap instance.")
217- launch_class = self._provider.launch_machine_class
218- launch = launch_class(self._provider, bootstrap=True)
219- return launch.run({"machine-id": "0"})
220+ return self._provider.start_machine({"machine-id": "0"}, master=True)
221
222=== modified file 'ensemble/providers/common/launch.py'
223--- ensemble/providers/common/launch.py 2011-08-11 18:43:06 +0000
224+++ ensemble/providers/common/launch.py 2011-08-17 16:02:43 +0000
225@@ -56,7 +56,7 @@
226 class LaunchMachine(object):
227 """Abstract class with generic instance-launching logic.
228
229- Constructing with bootstrap=True will cause the run method to
230+ Constructing with master=True will cause the run method to
231 construct a machine which is also running a zookeeper for the
232 cluster, and a provisioning agent, as well as the usual machine
233 agent.
234@@ -66,9 +66,9 @@
235 convenient to override C{get_machine_variables} as well.
236 """
237
238- def __init__(self, provider, bootstrap=False):
239+ def __init__(self, provider, master=False):
240 self._provider = provider
241- self._bootstrap = bootstrap
242+ self._master = master
243
244 @inlineCallbacks
245 def run(self, machine_data):
246@@ -86,8 +86,8 @@
247 machine_variables = yield self.get_machine_variables(machine_data)
248 provider_machines = yield self.start_machine(
249 machine_variables, machine_data["machine-id"])
250- if self._bootstrap:
251- yield self._on_bootstrap_launched(provider_machines)
252+ if self._master:
253+ yield self._on_master_launched(provider_machines)
254 returnValue(provider_machines)
255
256 def start_machine(self, machine_variables, machine_id):
257@@ -128,7 +128,7 @@
258
259 @inlineCallbacks
260 def _get_zookeeper_hosts(self):
261- if self._bootstrap:
262+ if self._master:
263 returnValue("localhost:2181")
264 machines = yield self._provider.get_zookeeper_machines()
265 hosts = [m.private_dns_name for m in machines]
266@@ -137,7 +137,7 @@
267
268 def _get_packages(self):
269 result = list(DEFAULT_PACKAGES)
270- if self._bootstrap:
271+ if self._master:
272 result.extend(BOOTSTRAP_PACKAGES)
273 return result
274
275@@ -150,7 +150,7 @@
276 "sudo mkdir -p /var/lib/ensemble",
277 "sudo mkdir -p /var/log/ensemble"])
278
279- if self._bootstrap:
280+ if self._master:
281 launch_scripts.append(self._get_initialize_script())
282
283 # Every machine has its own agent.
284@@ -162,7 +162,7 @@
285 "--pidfile=/var/run/ensemble/machine-agent.pid")
286 launch_scripts.append(machine_agent_script_template % machine_data)
287
288- if self._bootstrap:
289+ if self._master:
290 launch_scripts.append(_get_provision_agent_script(machine_data))
291
292 return launch_scripts
293@@ -185,7 +185,10 @@
294 return _get_initialize_script(
295 self.get_instance_id_command(), admin_secret)
296
297- def _on_bootstrap_launched(self, machines):
298+ def _on_master_launched(self, machines):
299+ # TODO this should be part of Bootstrap (and should really extend,
300+ # rather than effectively replace, the result of
301+ # self._provider.get_zookeeper_machines)
302 instance_ids = [m.instance_id for m in machines]
303 d = self._provider.save_state({"zookeeper-instances": instance_ids})
304 d.addCallback(lambda _: machines)
305
306=== added file 'ensemble/providers/common/tests/test_base.py'
307--- ensemble/providers/common/tests/test_base.py 1970-01-01 00:00:00 +0000
308+++ ensemble/providers/common/tests/test_base.py 2011-08-17 16:02:43 +0000
309@@ -0,0 +1,135 @@
310+from twisted.internet.defer import fail, succeed
311+
312+from ensemble.environment.errors import EnvironmentsConfigError
313+from ensemble.lib.testing import TestCase
314+from ensemble.machine import ProviderMachine
315+from ensemble.providers.common.base import MachineProviderBase
316+
317+
318+class SomeError(Exception):
319+ pass
320+
321+
322+class DummyLaunchMachine(object):
323+
324+ def __init__(self, master=False):
325+ self._master = master
326+
327+ def run(self, machine_data):
328+ return succeed([ProviderMachine(machine_data["machine-id"])])
329+
330+
331+class DummyProvider(MachineProviderBase):
332+
333+ def __init__(self, config=None):
334+ super(DummyProvider, self).__init__("venus", config or {})
335+
336+
337+class MachineProviderBaseTest(TestCase):
338+
339+ def test_init(self):
340+ provider = DummyProvider({"some": "config"})
341+ self.assertEquals(provider.environment_name, "venus")
342+ self.assertEquals(provider.config, {"some": "config"})
343+
344+ def test_bad_config(self):
345+ try:
346+ DummyProvider({"authorized-keys": "foo",
347+ "authorized-keys-path": "bar"})
348+ except EnvironmentsConfigError as error:
349+ expect = ("Environment config cannot define both authorized-keys "
350+ "and authorized-keys-path. Pick one!")
351+ self.assertEquals(str(error), expect)
352+ else:
353+ self.fail("Failed to detect bad config")
354+
355+ def test_get_serialization_data(self):
356+ keys_path = self.makeFile("some-key")
357+ provider = DummyProvider({"foo": {"bar": "baz"},
358+ "authorized-keys-path": keys_path})
359+ data = provider.get_serialization_data()
360+ self.assertEquals(data, {"foo": {"bar": "baz"},
361+ "authorized-keys": "some-key"})
362+ data["foo"]["bar"] = "qux"
363+ self.assertEquals(provider.config, {"foo": {"bar": "baz"},
364+ "authorized-keys-path": keys_path})
365+
366+ def test_get_machine_error(self):
367+ provider = DummyProvider()
368+ provider.get_machines = self.mocker.mock()
369+ provider.get_machines(["piffle"])
370+ self.mocker.result(fail(SomeError()))
371+ self.mocker.replay()
372+
373+ d = provider.get_machine("piffle")
374+ self.assertFailure(d, SomeError)
375+ return d
376+
377+ def test_get_machine_success(self):
378+ provider = DummyProvider()
379+ provider.get_machines = self.mocker.mock()
380+ provider.get_machines(["piffle"])
381+ machine = object()
382+ self.mocker.result(succeed([machine]))
383+ self.mocker.replay()
384+
385+ d = provider.get_machine("piffle")
386+
387+ def verify(result):
388+ self.assertEquals(result, machine)
389+ d.addCallback(verify)
390+ return d
391+
392+ def test_shutdown_machine_error(self):
393+ provider = DummyProvider()
394+ provider.shutdown_machines = self.mocker.mock()
395+ machine = object()
396+ provider.shutdown_machines([machine])
397+ self.mocker.result(fail(SomeError()))
398+ self.mocker.replay()
399+
400+ d = provider.shutdown_machine(machine)
401+ self.assertFailure(d, SomeError)
402+ return d
403+
404+ def test_shutdown_machine_success(self):
405+ provider = DummyProvider()
406+ provider.shutdown_machines = self.mocker.mock()
407+ machine = object()
408+ provider.shutdown_machines([machine])
409+ probably_the_same_machine = object()
410+ self.mocker.result(succeed([probably_the_same_machine]))
411+ self.mocker.replay()
412+
413+ d = provider.shutdown_machine(machine)
414+
415+ def verify(result):
416+ self.assertEquals(result, probably_the_same_machine)
417+ d.addCallback(verify)
418+ return d
419+
420+ def test_destroy_environment_error(self):
421+ provider = DummyProvider()
422+ provider.shutdown_machines = self.mocker.mock()
423+ provider.shutdown_machines()
424+ self.mocker.result(fail(SomeError()))
425+ self.mocker.replay()
426+
427+ d = provider.destroy_environment()
428+ self.assertFailure(d, SomeError)
429+ return d
430+
431+ def test_destroy_environment_success(self):
432+ provider = DummyProvider()
433+ provider.shutdown_machines = self.mocker.mock()
434+ provider.shutdown_machines()
435+ machines = object()
436+ self.mocker.result(succeed(machines))
437+ self.mocker.replay()
438+
439+ d = provider.destroy_environment()
440+
441+ def verify(result):
442+ self.assertEquals(result, machines)
443+ d.addCallback(verify)
444+ return d
445
446=== modified file 'ensemble/providers/common/tests/test_findzookeepers.py'
447--- ensemble/providers/common/tests/test_findzookeepers.py 2011-08-16 13:24:41 +0000
448+++ ensemble/providers/common/tests/test_findzookeepers.py 2011-08-17 16:02:43 +0000
449@@ -1,69 +1,79 @@
450+from cStringIO import StringIO
451+from yaml import dump
452+
453 from twisted.internet.defer import fail, succeed
454
455 from ensemble.errors import EnvironmentNotFound, MachinesNotFound
456 from ensemble.lib.testing import TestCase
457-from ensemble.providers.common.findzookeepers import find_zookeepers
458-
459-
460-class DummyProvider(object):
461-
462- def __init__(self, state, get_machine):
463- self.state = state
464- self.get_machine = get_machine
465-
466- def load_state(self):
467- return succeed(self.state)
468+from ensemble.providers.common.base import MachineProviderBase
469+
470+
471+class SomeError(Exception):
472+ pass
473
474
475 class FindZookeepersTest(TestCase):
476
477 def get_provider(self, state):
478- return DummyProvider(state, self.mocker.mock())
479+ test = self
480+
481+ class DummyStorage(object):
482+
483+ def get(self, path):
484+ test.assertEquals(path, "provider-state")
485+ return succeed(StringIO(dump(state)))
486+
487+ class DummyProvider(MachineProviderBase):
488+
489+ def __init__(self):
490+ self.get_machines = test.mocker.mock()
491+
492+ def get_file_storage(self):
493+ return DummyStorage()
494+
495+ return DummyProvider()
496
497 def test_no_state(self):
498 provider = self.get_provider(False)
499- d = find_zookeepers(provider)
500+ d = provider.get_zookeeper_machines()
501 self.assertFailure(d, EnvironmentNotFound)
502 return d
503
504 def test_empty_state(self):
505 provider = self.get_provider({})
506- d = find_zookeepers(provider)
507+ d = provider.get_zookeeper_machines()
508 self.assertFailure(d, EnvironmentNotFound)
509 return d
510
511 def test_get_machine_error_aborts(self):
512 provider = self.get_provider(
513 {"zookeeper-instances": ["porter", "carter"]})
514- provider.get_machine("porter")
515-
516- class SomeError(Exception):
517- pass
518+ provider.get_machines(["porter"])
519 self.mocker.result(fail(SomeError()))
520 self.mocker.replay()
521
522- d = find_zookeepers(provider)
523+ d = provider.get_zookeeper_machines()
524 self.assertFailure(d, SomeError)
525 return d
526
527 def test_bad_machine(self):
528 provider = self.get_provider({"zookeeper-instances": ["porter"]})
529- provider.get_machine("porter")
530+ provider.get_machines(["porter"])
531 self.mocker.result(fail(MachinesNotFound(["porter"])))
532 self.mocker.replay()
533
534- d = find_zookeepers(provider)
535+ d = provider.get_zookeeper_machines()
536 self.assertFailure(d, EnvironmentNotFound)
537 return d
538
539 def test_good_machine(self):
540 provider = self.get_provider({"zookeeper-instances": ["porter"]})
541- provider.get_machine("porter")
542+ provider.get_machines(["porter"])
543 machine = object()
544- self.mocker.result(succeed(machine))
545+ self.mocker.result(succeed([machine]))
546 self.mocker.replay()
547
548- d = find_zookeepers(provider)
549+ d = provider.get_zookeeper_machines()
550
551 def verify_machine(result):
552 self.assertEquals(result, [machine])
553@@ -73,19 +83,19 @@
554 def test_gets_all_good_machines(self):
555 provider = self.get_provider(
556 {"zookeeper-instances": ["porter", "carter", "miller", "baker"]})
557- provider.get_machine("porter")
558+ provider.get_machines(["porter"])
559 self.mocker.result(fail(MachinesNotFound(["porter"])))
560- provider.get_machine("carter")
561+ provider.get_machines(["carter"])
562 carter = object()
563- self.mocker.result(succeed(carter))
564- provider.get_machine("miller")
565+ self.mocker.result(succeed([carter]))
566+ provider.get_machines(["miller"])
567 self.mocker.result(fail(MachinesNotFound(["miller"])))
568- provider.get_machine("baker")
569+ provider.get_machines(["baker"])
570 baker = object()
571- self.mocker.result(succeed(baker))
572+ self.mocker.result(succeed([baker]))
573 self.mocker.replay()
574
575- d = find_zookeepers(provider)
576+ d = provider.get_zookeeper_machines()
577
578 def verify_machine(result):
579 self.assertEquals(result, [carter, baker])
580
581=== modified file 'ensemble/providers/common/tests/test_launch.py'
582--- ensemble/providers/common/tests/test_launch.py 2011-08-10 18:16:51 +0000
583+++ ensemble/providers/common/tests/test_launch.py 2011-08-17 16:02:43 +0000
584@@ -1,11 +1,11 @@
585 import logging
586 import tempfile
587
588-from twisted.internet.defer import fail, succeed
589+from twisted.internet.defer import fail, inlineCallbacks, succeed
590
591 from ensemble.errors import EnvironmentNotFound, ProviderError
592 from ensemble.lib.testing import TestCase
593-from ensemble.providers.common.bootstrap import Bootstrap
594+from ensemble.providers.common.base import MachineProviderBase
595 from ensemble.providers.common.launch import (
596 BOOTSTRAP_PACKAGES, DEFAULT_PACKAGES, DEFAULT_REPOSITORIES, LaunchMachine)
597 from ensemble.providers.dummy import DummyMachine, FileStorage
598@@ -18,7 +18,7 @@
599 class DummyLaunchMachine(LaunchMachine):
600
601 def start_machine(self, variables, data):
602- if self._bootstrap:
603+ if self._master:
604 name = "bootstrapped-instance-id"
605 else:
606 name = "some-instance-id"
607@@ -42,11 +42,15 @@
608
609 def get_provider(launch_class=DummyLaunchMachine, config=None, zookeeper=True,
610 file_storage_class=WorkingFileStorage):
611- class DummyProvider(object):
612- config = {"authorized-keys": "abc"}
613- launch_machine_class = launch_class
614+
615+ class DummyProvider(MachineProviderBase):
616+
617+ def __init__(self, config):
618+ super(DummyProvider, self).__init__("venus", config)
619+ self._file_storage = file_storage_class()
620
621 def get_zookeeper_machines(self):
622+ # this is mocked out to avoid insane complexity
623 if isinstance(zookeeper, Exception):
624 return fail(zookeeper)
625 return succeed(
626@@ -54,21 +58,21 @@
627 private_dns_name="zookeeper.internal")])
628
629 def get_file_storage(self):
630- return file_storage_class()
631-
632- def save_state(self, state):
633- self.saved_state = state
634- return succeed(None)
635-
636+ return self._file_storage
637+
638+ def start_machine(self, machine_data, master=False):
639+ return launch_class(self, master).run(machine_data)
640+
641+ create_config = {"authorized-keys": "abc"}
642 if config is not None:
643- DummyProvider.config.update(config)
644- return DummyProvider()
645+ create_config.update(config)
646+ return DummyProvider(create_config)
647
648
649 def get_launch(launch_class=DummyLaunchMachine, config=None,
650- zookeeper=True, bootstrap=False):
651+ zookeeper=True, master=False):
652 provider = get_provider(launch_class, config, zookeeper)
653- return launch_class(provider, bootstrap=bootstrap)
654+ return launch_class(provider, master=master)
655
656
657 class LaunchMachineTest(TestCase):
658@@ -90,17 +94,14 @@
659 d.addCallback(self._verify_machines)
660 return d
661
662+ @inlineCallbacks
663 def test_bootstrap_run(self):
664 provider = get_provider(config={"admin-secret": "whatever"})
665- launch = DummyLaunchMachine(provider, bootstrap=True)
666- d = launch.run({"machine-id": "machine-32767"})
667-
668- def verify_state_saved(_):
669- self.assertEquals(
670- provider.saved_state,
671- {"zookeeper-instances": ["bootstrapped-instance-id"]})
672- d.addCallback(verify_state_saved)
673- return d
674+ launch = DummyLaunchMachine(provider, master=True)
675+ yield launch.run({"machine-id": "machine-32767"})
676+ saved_state = yield provider.load_state()
677+ self.assertEquals(
678+ saved_state, {"zookeeper-instances": ["bootstrapped-instance-id"]})
679
680 def test_get_machine_variables_normal(self):
681 launch = get_launch()
682@@ -152,8 +153,7 @@
683 return d
684
685 def test_get_machine_variables_bootstrap(self):
686- launch = get_launch(config={"admin-secret": "SEEKRIT"},
687- bootstrap=True)
688+ launch = get_launch(config={"admin-secret": "SEEKRIT"}, master=True)
689 d = launch.get_machine_variables({"machine-id": "machine-757"})
690
691 def verify_vars(vars):
692@@ -199,23 +199,21 @@
693 class BBQError(Exception):
694 pass
695 provider = get_provider(zookeeper=BBQError())
696- bootstrap = Bootstrap(provider)
697- d = bootstrap.run()
698+ d = provider.bootstrap()
699 self.assertFailure(d, BBQError)
700 return d
701
702 def test_bootstrap_unwritable_storage(self):
703 provider = get_provider(zookeeper=EnvironmentNotFound(),
704 file_storage_class=UnwritableFileStorage)
705- d = Bootstrap(provider).run()
706+ d = provider.bootstrap()
707 self.assertFailure(d, ProviderError)
708 return d
709
710 def test_bootstrap_no_launch(self):
711 log = self.capture_logging("ensemble.common", level=logging.DEBUG)
712 provider = get_provider()
713- bootstrap = Bootstrap(provider)
714- d = bootstrap.run()
715+ d = provider.bootstrap()
716
717 def verify_machines(machines):
718 (machine,) = machines
719@@ -232,8 +230,7 @@
720 log = self.capture_logging("ensemble.common", level=logging.DEBUG)
721 provider = get_provider(config={"admin-secret": "SEEKRIT"},
722 zookeeper=EnvironmentNotFound())
723- bootstrap = Bootstrap(provider)
724- d = bootstrap.run()
725+ d = provider.bootstrap()
726
727 def verify_machines(machines):
728 (machine,) = machines
729
730=== modified file 'ensemble/providers/common/tests/test_state.py'
731--- ensemble/providers/common/tests/test_state.py 2011-07-26 10:45:08 +0000
732+++ ensemble/providers/common/tests/test_state.py 2011-08-17 16:02:43 +0000
733@@ -6,10 +6,11 @@
734 from ensemble.errors import FileNotFound
735 from ensemble.lib.mocker import MATCH
736 from ensemble.lib.testing import TestCase
737+from ensemble.providers.common.base import MachineProviderBase
738 from ensemble.providers.common.state import LoadState, SaveState
739
740
741-class DummyProvider(object):
742+class DummyProvider(MachineProviderBase):
743
744 def __init__(self, get=None, put=None):
745 self._get = get
746
747=== modified file 'ensemble/providers/common/tests/test_utils.py'
748--- ensemble/providers/common/tests/test_utils.py 2011-08-03 15:53:38 +0000
749+++ ensemble/providers/common/tests/test_utils.py 2011-08-17 16:02:43 +0000
750@@ -91,7 +91,7 @@
751 "OSError('Bad',)")
752
753
754-class FormatCloudIniTest(TestCase):
755+class FormatCloudInitTest(TestCase):
756
757 def test_format_cloud_init_with_data(self):
758 """The format cloud init creates a user-data cloud-init config file.
759
760=== modified file 'ensemble/providers/dummy.py'
761--- ensemble/providers/dummy.py 2011-08-12 19:28:37 +0000
762+++ ensemble/providers/dummy.py 2011-08-17 16:02:43 +0000
763@@ -2,17 +2,16 @@
764 import os
765 import tempfile
766
767+from twisted.internet.defer import inlineCallbacks, returnValue, succeed, fail
768+
769 from txzookeeper import ZookeeperClient
770-from twisted.internet.defer import succeed, fail
771-
772-
773-log = logging.getLogger("ensemble.providers")
774-
775
776 from ensemble.errors import (
777 EnvironmentNotFound, FileNotFound, MachinesNotFound, ProviderError)
778 from ensemble.machine import ProviderMachine
779
780+log = logging.getLogger("ensemble.providers")
781+
782
783 class DummyMachine(ProviderMachine):
784 """Provider machine implementation specific to the dummy provider."""
785@@ -59,7 +58,7 @@
786 return fail(MachinesNotFound(missing_instance_ids))
787 return succeed(machines)
788
789- def start_machine(self, machine_data):
790+ def start_machine(self, machine_data, master=False):
791 """Start a machine in the provider."""
792 if not "machine-id" in machine_data:
793 return fail(ProviderError(
794@@ -85,13 +84,16 @@
795 return succeed(self._machines[:1])
796 return self.start_machine({"machine-id": 0})
797
798- def shutdown(self):
799+ @inlineCallbacks
800+ def shutdown_machines(self, requested_machines=()):
801 """
802 Terminate any machine resources associated to the provider.
803 """
804- machines = self._machines
805- self._machines = []
806- return succeed(machines)
807+ instance_ids = [m.instance_id for m in requested_machines]
808+ machines = yield self.get_machines(instance_ids)
809+ for machine in machines:
810+ self._machines.remove(machine)
811+ returnValue(machines)
812
813 def shutdown_machine(self, machine):
814 """Terminate the given machine"""
815@@ -103,6 +105,9 @@
816 return
817 return fail(ProviderError("Machine not found %r" % machine))
818
819+ def destroy_environment(self):
820+ return self.shutdown_machines()
821+
822 def save_state(self, state):
823 """Save the state to the provider."""
824 self._state = state
825
826=== modified file 'ensemble/providers/ec2/__init__.py'
827--- ensemble/providers/ec2/__init__.py 2011-08-17 08:58:51 +0000
828+++ ensemble/providers/ec2/__init__.py 2011-08-17 16:02:43 +0000
829@@ -1,5 +1,3 @@
830-import copy
831-from operator import itemgetter
832 import os
833 import re
834
835@@ -8,14 +6,10 @@
836 from txaws.ec2.exception import EC2Error
837 from txaws.service import AWSServiceRegion
838
839-from ensemble.environment.errors import EnvironmentsConfigError
840 from ensemble.errors import (
841 MachinesNotFound, ProviderError, ProviderInteractionError)
842-from ensemble.providers.common.bootstrap import Bootstrap
843-from ensemble.providers.common.findzookeepers import find_zookeepers
844-from ensemble.providers.common.state import SaveState, LoadState
845-from ensemble.providers.common.utils import (
846- convert_unknown_error, get_user_authorized_keys)
847+from ensemble.providers.common.base import MachineProviderBase
848+from ensemble.providers.common.utils import convert_unknown_error
849
850 from .connect import EC2Connect
851 from .files import FileStorage
852@@ -26,13 +20,10 @@
853 from .utils import get_region_uri
854
855
856-class MachineProvider(object):
857-
858- launch_machine_class = EC2LaunchMachine
859+class MachineProvider(MachineProviderBase):
860
861 def __init__(self, environment_name, config):
862- self.environment_name = environment_name
863- self.config = config
864+ super(MachineProvider, self).__init__(environment_name, config)
865
866 if not config.get("ec2-uri"):
867 ec2_uri = get_region_uri(config.get("region", "us-east-1"))
868@@ -47,27 +38,14 @@
869 self.s3 = self._service.get_s3_client()
870 self.ec2 = self._service.get_ec2_client()
871
872- if ("authorized-keys-path" in config and
873- "authorized-keys" in config):
874- raise EnvironmentsConfigError(
875- "Environment config cannot define both authorized-keys "
876- "and authorized-keys-path. Pick one!")
877-
878 def get_serialization_data(self):
879- """Return a dictionary serialization of the provider configuration.
880+ """Get provider configuration suitable for serialization.
881
882- Additionally this extracts crednetial information from the environment.
883+ Also extracts credential information from the environment.
884 """
885- data = copy.deepcopy(self.config)
886- data["secret-key"] = self.config.get(
887- "secret-key", os.environ.get("AWS_SECRET_ACCESS_KEY"))
888- data["access-key"] = self.config.get(
889- "access-key", os.environ.get("AWS_ACCESS_KEY_ID"))
890- data["authorized-keys"] = get_user_authorized_keys(data)
891-
892- # Not relevant, on a remote system.
893- data.pop("authorized-keys-path", None)
894-
895+ data = super(MachineProvider, self).get_serialization_data()
896+ data.setdefault("access-key", os.environ.get("AWS_ACCESS_KEY_ID"))
897+ data.setdefault("secret-key", os.environ.get("AWS_SECRET_ACCESS_KEY"))
898 return data
899
900 def _run_operation(self, operation, *args, **kw):
901@@ -88,9 +66,19 @@
902 return self._run_operation(connect, share=share)
903
904 def get_file_storage(self):
905- """Retrieve the provider C{FileStorage} abstraction."""
906- file_storage = FileStorage(self.s3, self.config["control-bucket"])
907- return file_storage
908+ """Retrieve an S3-backed C{FileStorage}."""
909+ return FileStorage(self.s3, self.config["control-bucket"])
910+
911+ def start_machine(self, machine_data, master=False):
912+ """Start a machine in the provider.
913+
914+ @param machine_data: a dictionary of data to pass along to the newly
915+ launched machine.
916+
917+ @param master: if True, machine will initialize the ensemble admin
918+ and run a provisioning agent.
919+ """
920+ return EC2LaunchMachine(self, master).run(machine_data)
921
922 @inlineCallbacks
923 def get_machines(self, instance_ids=()):
924@@ -132,33 +120,8 @@
925 raise MachinesNotFound(missing)
926 returnValue(machines)
927
928- def get_machine(self, instance_id):
929- """Retrieve a provider machine by instance id."""
930- d = self.get_machines([instance_id])
931- d.addCallback(itemgetter(0))
932- return d
933-
934- def start_machine(self, machine_data):
935- """Start a machine in the provider.
936-
937- @param machine_data a dictionary of data to pass along to the newly
938- launched machine.
939- @type dict
940- """
941- launch = EC2LaunchMachine(self)
942- return launch.run(machine_data=machine_data)
943-
944- def bootstrap(self):
945- """Bootstrap an ensemble server in the provider."""
946- bootstrap = Bootstrap(self)
947- return bootstrap.run()
948-
949- def shutdown_machine(self, machine):
950- """Stop a machine in the provider."""
951- return self.shutdown([machine])
952-
953 @inlineCallbacks
954- def shutdown(self, requested_machines=()):
955+ def shutdown_machines(self, requested_machines=()):
956 """Terminate machine resources associated with this provider."""
957 for machine in requested_machines:
958 if not isinstance(machine, EC2ProviderMachine):
959@@ -170,30 +133,7 @@
960 if killable_machines:
961 killable_ids = [m.instance_id for m in killable_machines]
962 yield self.ec2.terminate_instances(*killable_ids)
963-
964- def save_state(self, state):
965- """Save state to the provider.
966-
967- @param state
968- @type dict
969- """
970- return SaveState(self).run(state)
971-
972- def load_state(self):
973- """Load state from the provider.
974-
975- @return: a dictionary.
976- """
977- return LoadState(self).run()
978-
979- def get_zookeeper_machines(self):
980- """Find running zookeeper instances.
981-
982- @return: the first valid instance found as a single element list.
983-
984- @raise: EnvironmentNotFound
985- """
986- return find_zookeepers(self)
987+ returnValue(killable_machines)
988
989 def open_port(self, machine, machine_id, port, protocol="tcp"):
990 """Authorizes `port` using `protocol` on EC2 for `machine`."""
991
992=== modified file 'ensemble/providers/ec2/tests/test_provider.py'
993--- ensemble/providers/ec2/tests/test_provider.py 2011-05-03 08:54:13 +0000
994+++ ensemble/providers/ec2/tests/test_provider.py 2011-08-17 16:02:43 +0000
995@@ -115,14 +115,16 @@
996 serialized.pop("authorized-keys", None)
997 self.assertEqual(config, serialized)
998
999+class FailCreateTest(TestCase):
1000+
1001 def test_conflicting_authorized_keys_options(self):
1002 """
1003 We can't handle two different authorized keys options, so deny
1004 constructing an environment that way.
1005 """
1006- config = self.get_config()
1007+ config = {}
1008 config["authorized-keys"] = "File content"
1009 config["authorized-keys-path"] = "File path"
1010 error = self.assertRaises(EnvironmentsConfigError,
1011- MachineProvider, self.env_name, config)
1012+ MachineProvider, "some-env-name", config)
1013 self.assertIn("authorized-keys", str(error))
1014
1015=== modified file 'ensemble/providers/ec2/tests/test_shutdown.py'
1016--- ensemble/providers/ec2/tests/test_shutdown.py 2011-08-12 21:01:27 +0000
1017+++ ensemble/providers/ec2/tests/test_shutdown.py 2011-08-17 16:02:43 +0000
1018@@ -11,7 +11,7 @@
1019
1020 class EC2ShutdownMachineTest(EC2TestMixin, TestCase):
1021
1022- def test_shutdown(self):
1023+ def test_shutdown_machine(self):
1024 instance = self.get_instance("i-foobar")
1025 self.ec2.describe_instances("i-foobar")
1026 self.mocker.result(succeed([instance]))
1027@@ -21,9 +21,15 @@
1028
1029 machine = EC2ProviderMachine("i-foobar")
1030 provider = self.get_provider()
1031- return provider.shutdown_machine(machine)
1032-
1033- def test_shutdown_invalid_group(self):
1034+ d = provider.shutdown_machine(machine)
1035+
1036+ def verify(machine):
1037+ self.assertTrue(isinstance(machine, EC2ProviderMachine))
1038+ self.assertEquals(machine.instance_id, "i-foobar")
1039+ d.addCallback(verify)
1040+ return d
1041+
1042+ def test_shutdown_machine_invalid_group(self):
1043 """
1044 Attempting to shutdown a machine that does not belong to this
1045 provider instance raises an exception.
1046@@ -43,7 +49,7 @@
1047 d.addCallback(verify)
1048 return d
1049
1050- def test_shutdown_invalid_machine(self):
1051+ def test_shutdown_machine_invalid_machine(self):
1052 """
1053 Attempting to shutdown a machine that from a different provider
1054 type will raise a syntaxerror.
1055@@ -61,12 +67,81 @@
1056 d.addCallback(check_error)
1057 return d
1058
1059-
1060-class EC2ShutdownTest(EC2TestMixin, TestCase):
1061-
1062- def test_shutdown(self):
1063+ def test_shutdown_machines_all(self):
1064+ self.ec2.describe_instances()
1065+ self.mocker.result(succeed([
1066+ self.get_instance("i-amkillable"),
1067+ self.get_instance("i-amdead", "shutting-down"),
1068+ self.get_instance("i-amalien", groups=["other"]),
1069+ self.get_instance("i-amkillabletoo")]))
1070+ self.ec2.terminate_instances("i-amkillable", "i-amkillabletoo")
1071+ self.mocker.result(succeed([
1072+ ("i-amkillable", "running", "shutting-down"),
1073+ ("i-amkillabletoo", "running", "shutting-down")]))
1074+ self.mocker.replay()
1075+
1076+ provider = self.get_provider()
1077+ d = provider.shutdown_machines()
1078+
1079+ def verify(result):
1080+ (machine_1, machine_2) = result
1081+ self.assertTrue(isinstance(machine_1, EC2ProviderMachine))
1082+ self.assertEquals(machine_1.instance_id, "i-amkillable")
1083+ self.assertTrue(isinstance(machine_2, EC2ProviderMachine))
1084+ self.assertEquals(machine_2.instance_id, "i-amkillabletoo")
1085+ d.addCallback(verify)
1086+ return d
1087+
1088+ def test_shutdown_machines_some_invalid(self):
1089+ self.ec2.describe_instances("i-amkillable", "i-amdead")
1090+ self.mocker.result(succeed([
1091+ self.get_instance("i-amkillable"),
1092+ self.get_instance("i-amdead", "shutting-down")]))
1093+ self.mocker.replay()
1094+
1095+ provider = self.get_provider()
1096+ d = provider.shutdown_machines([
1097+ EC2ProviderMachine("i-amkillable"),
1098+ EC2ProviderMachine("i-amdead")])
1099+ self.failUnlessFailure(d, MachinesNotFound)
1100+
1101+ def verify(error):
1102+ self.assertEquals(str(error),
1103+ "Cannot find machine: i-amdead")
1104+ d.addCallback(verify)
1105+ return d
1106+
1107+ def test_shutdown_machines_some_success(self):
1108+ self.ec2.describe_instances("i-amkillable", "i-amkillabletoo")
1109+ self.mocker.result(succeed([
1110+ self.get_instance("i-amkillable"),
1111+ self.get_instance("i-amkillabletoo")]))
1112+ self.ec2.terminate_instances("i-amkillable", "i-amkillabletoo")
1113+ self.mocker.result(succeed([
1114+ ("i-amkillable", "running", "shutting-down"),
1115+ ("i-amkillabletoo", "running", "shutting-down")]))
1116+ self.mocker.replay()
1117+
1118+ provider = self.get_provider()
1119+ d = provider.shutdown_machines([
1120+ EC2ProviderMachine("i-amkillable"),
1121+ EC2ProviderMachine("i-amkillabletoo")])
1122+
1123+ def verify(result):
1124+ (machine_1, machine_2) = result
1125+ self.assertTrue(isinstance(machine_1, EC2ProviderMachine))
1126+ self.assertEquals(machine_1.instance_id, "i-amkillable")
1127+ self.assertTrue(isinstance(machine_2, EC2ProviderMachine))
1128+ self.assertEquals(machine_2.instance_id, "i-amkillabletoo")
1129+ d.addCallback(verify)
1130+ return d
1131+
1132+
1133+class EC2DestroyTest(EC2TestMixin, TestCase):
1134+
1135+ def test_destroy_environment(self):
1136 """
1137- The shutdown operation terminates all running and pending
1138+ The destroy_environment operation terminates all running and pending
1139 instances associated to the C{MachineProvider} instance.
1140 """
1141 instances = [self.get_instance("i-canbekilled"),
1142@@ -83,16 +158,30 @@
1143 self.mocker.replay()
1144
1145 provider = self.get_provider()
1146- return provider.shutdown()
1147+ d = provider.destroy_environment()
1148+
1149+ def verify(result):
1150+ (machine_1, machine_2) = result
1151+ self.assertTrue(isinstance(machine_1, EC2ProviderMachine))
1152+ self.assertEquals(machine_1.instance_id, "i-canbekilled")
1153+ self.assertTrue(isinstance(machine_2, EC2ProviderMachine))
1154+ self.assertEquals(machine_2.instance_id, "i-canbekilledtoo")
1155+ d.addCallback(verify)
1156+ return d
1157
1158 def test_shutdown_no_instances(self):
1159 """
1160- If there are no instances to shutdown, running the shutdown
1161- operation returns None.
1162+ If there are no instances to shutdown, running the destroy_environment
1163+ operation does nothing.
1164 """
1165 self.ec2.describe_instances()
1166 self.mocker.result(succeed([]))
1167 self.mocker.replay()
1168
1169 provider = self.get_provider()
1170- return provider.shutdown()
1171+ d = provider.destroy_environment()
1172+
1173+ def verify(result):
1174+ self.assertEquals(result, None)
1175+ d.addCallback(verify)
1176+ return d
1177
1178=== modified file 'ensemble/providers/orchestra/__init__.py'
1179--- ensemble/providers/orchestra/__init__.py 2011-08-12 20:31:16 +0000
1180+++ ensemble/providers/orchestra/__init__.py 2011-08-17 16:02:43 +0000
1181@@ -1,13 +1,6 @@
1182-import copy
1183-from operator import itemgetter
1184-
1185 from twisted.internet.defer import inlineCallbacks, returnValue
1186
1187-from ensemble.environment.errors import EnvironmentsConfigError
1188-from ensemble.providers.common.bootstrap import Bootstrap
1189-from ensemble.providers.common.findzookeepers import find_zookeepers
1190-from ensemble.providers.common.state import SaveState, LoadState
1191-from ensemble.providers.common.utils import get_user_authorized_keys
1192+from ensemble.providers.common.base import MachineProviderBase
1193
1194 from .cobbler import CobblerClient
1195 from .files import FileStorage
1196@@ -15,43 +8,12 @@
1197 from .machine import machine_from_dict
1198
1199
1200-class MachineProvider(object):
1201-
1202- launch_machine_class = OrchestraLaunchMachine
1203+class MachineProvider(MachineProviderBase):
1204
1205 def __init__(self, environment_name, config):
1206- self.environment_name = environment_name
1207- self.config = config
1208+ super(MachineProvider, self).__init__(environment_name, config)
1209 self.cobbler = CobblerClient(config)
1210
1211- if ("authorized-keys-path" in config and
1212- "authorized-keys" in config):
1213- raise EnvironmentsConfigError(
1214- "Environment config cannot define both authorized-keys "
1215- "and authorized-keys-path. Pick one!")
1216-
1217- def get_serialization_data(self):
1218- """Return a dictionary serialization of the provider configuration.
1219-
1220- Additionally this extracts credential information from the environment.
1221- """
1222- data = copy.deepcopy(self.config)
1223- data["authorized-keys"] = get_user_authorized_keys(data)
1224- data.pop("authorized-keys-path", None)
1225- return data
1226-
1227- def connect(self, share=False):
1228- """
1229- Connect to the zookeeper ensemble running in the machine provider.
1230-
1231- @param share: Requests sharing of the connection with other clients
1232- attempting to connect to the same provider, if that's feasible.
1233-
1234- returns an open C{txzookeeper.client.ZookeeperClient} and a
1235- C{ensemble.storage.connection.TunnelProtocol}
1236- """
1237- raise NotImplementedError()
1238-
1239 def get_file_storage(self):
1240 """Retrieve the provider C{FileStorage} abstraction."""
1241 if "storage-url" not in self.config:
1242@@ -59,6 +21,17 @@
1243 "http://%(orchestra-server)s/webdav" % self.config)
1244 return FileStorage(self.config["storage-url"])
1245
1246+ def start_machine(self, machine_data, master=False):
1247+ """Start a machine in the provider.
1248+
1249+ @param machine_data: a dictionary of data to pass along to the newly
1250+ launched machine.
1251+
1252+ @param master: if True, machine will initialize the ensemble admin
1253+ and run a provisioning agent.
1254+ """
1255+ return OrchestraLaunchMachine(self, master).run(machine_data)
1256+
1257 @inlineCallbacks
1258 def get_machines(self, instance_ids=()):
1259 """List machines running in the provider.
1260@@ -72,82 +45,3 @@
1261 """
1262 instances = yield self.cobbler.describe_systems(*instance_ids)
1263 returnValue([machine_from_dict(i) for i in instances])
1264-
1265- def get_machine(self, instance_id):
1266- """Retrieve a provider machine by instance id. """
1267- d = self.get_machines([instance_id])
1268- d.addCallback(itemgetter(0))
1269- return d
1270-
1271- def start_machine(self, machine_data):
1272- """
1273- Start a machine in the provider.
1274-
1275- @param machine_data a dictionary of data to pass along to the newly
1276- launched machine.
1277- @type dict
1278- """
1279- launch = OrchestraLaunchMachine(self)
1280- return launch.run(machine_data)
1281-
1282- def bootstrap(self):
1283- """
1284- Bootstrap an ensemble server in the provider.
1285- """
1286- bootstrap = Bootstrap(self)
1287- return bootstrap.run()
1288-
1289- def shutdown_machine(self):
1290- """
1291- Stop a machine in the provider.
1292- """
1293- raise NotImplementedError()
1294-
1295- def shutdown(self, requested_machines=()):
1296- """Terminate machine resources associated with this provider."""
1297- raise NotImplementedError()
1298-
1299- def save_state(self, state):
1300- """Save state to the provider.
1301-
1302- @param state
1303- @type dict
1304- """
1305- return SaveState(self).run(state)
1306-
1307- def load_state(self):
1308- """Load state from the provider.
1309-
1310- @return: a dictionary.
1311- """
1312- return LoadState(self).run()
1313-
1314- def get_zookeeper_machines(self):
1315- """Find running zookeeper instances.
1316-
1317- @return: the first valid instance found as a single element list.
1318-
1319- @raise: EnvironmentNotFound
1320- """
1321- return find_zookeepers(self)
1322-
1323- def open_port(self, machine, port, protocol="tcp"):
1324- """Expose port to the environment.
1325-
1326- Approximate equivalent of ec2-authorize-group
1327- """
1328- raise NotImplementedError()
1329-
1330- def close_port(self, machine, port, protocol="tcp"):
1331- """Close opened port
1332-
1333- Approximate equivalent of ec2-revoke-group
1334- """
1335- raise NotImplementedError()
1336-
1337- def get_opened_ports(self, machine):
1338- """List ports currently exposed to the environment.
1339-
1340- Approximate equivalent of ec2-describe-group
1341- """
1342- raise NotImplementedError()
1343
1344=== modified file 'ensemble/providers/tests/test_dummy.py'
1345--- ensemble/providers/tests/test_dummy.py 2011-08-11 05:59:30 +0000
1346+++ ensemble/providers/tests/test_dummy.py 2011-08-17 16:02:43 +0000
1347@@ -67,14 +67,14 @@
1348 return self.assertFailure(d, ProviderError)
1349
1350 @inlineCallbacks
1351- def test_shutdown(self):
1352- result = yield self.provider.shutdown()
1353+ def test_destroy_environment(self):
1354+ result = yield self.provider.destroy_environment()
1355 self.assertEqual(result, [])
1356
1357 @inlineCallbacks
1358- def test_shutdown_returns_machines(self):
1359+ def test_destroy_environment_returns_machines(self):
1360 yield self.provider.bootstrap()
1361- result = yield self.provider.shutdown()
1362+ result = yield self.provider.destroy_environment()
1363 self.assertEqual(len(result), 1)
1364 self.assertTrue(isinstance(result[0], ProviderMachine))
1365

Subscribers

People subscribed via source and target branches

to status/vote changes: