Merge lp:~hazmat/pyjuju/local-provider into lp:pyjuju

[Kapil Thangavelu]

Local/lxc for juju development/deployment on a single machine

Implements a local machine provider using lxc.

Resubmit with pre-requisite

[lp:~hazmat/pyjuju/local-provider]

370. By Kapil Thangavelu on 2011-09-21

ensure wait for agent start, state save, and log file present

371. By Kapil Thangavelu on 2011-09-21

setup log file for the agent

372. By Kapil Thangavelu on 2011-09-21

fully initialize the managed agent with all arges on bootstrap

373. By Kapil Thangavelu on 2011-09-21

additional logging

374. By Kapil Thangavelu on 2011-09-21

correct pid file usage

[Gustavo Niemeyer (niemeyer)]

Thanks Kapil. Good stuff in record time.

[1]

Tests fail due to missing dependencies in the environment.

===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1020, in _inlineCallbacks
    result = g.send(result)
  File "/home/niemeyer/src/juju/hazmat/local-provider/juju/providers/lxc/__init__.py", line 53, in bootstrap
    ", ".join(sorted(list(missing)))))
juju.errors.ProviderError: Missing packages apt-cacher-ng, libvirt-bin, lxc, zookeeperd

Not sure if we should just say "install the packages" as the answer for
that one, though.

[2]

+ # Install required packages

It's checking for, rather than installing. (which is fine, message just
needs fixing)

[3]

Another one:

  File "/home/niemeyer/src/juju/hazmat/local-provider/juju/providers/lxc/__init__.py", line 64, in bootstrap
    net_attributes = yield net.get_attributes()
  File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 207, in _worker
    result = context.call(ctx, function, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 59, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 37, in callWithContext
    return func(*args,**kw)
  File "/home/niemeyer/src/juju/hazmat/local-provider/juju/providers/lxc/network.py", line 135, in get_network_attributes
    output = subprocess.check_output(["virsh", "net-dumpxml", name])
  File "/usr/lib/python2.7/subprocess.py", line 537, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['virsh', 'net-dumpxml', 'default']' returned non-zero exit status 1

juju.providers.lxc.tests.test_provider.LocalProviderTest.test_bootstrap

Apparently virsh is being called for real in the system for tests to pass.

[4]

+ log.debug("Starting zookeeper...")

These debug messages feel like good info for the user. I suggest making them
visible through log.info instead.

[5]

+ # Default to running zookeeper as the user, unless the user is root.
+ zookeeper_user = os.geteuid() == 0 and "zookeeper" or None

This shorthand version is actually not that short and makes things
significantly less straightforward to parse when reading.

Compare it to:

    zookeeper_user = None
    if os.getuid() == 0:
        zookeeper_user = "zookeeper"

Please also tweak the comment if possible (or remove since the version above
is clear). The comment is confusing since it's running as the zookeeper user
if the user is root, and it's easy to read it as stating the exact opposite.

[6]

+ log.info("Destroying unit containers..")
(...)
+ log.debug("Stopping zookeeper..")
(...)
+ log.info("Environment bootstrapped")

There's some inconsisteny in the logging. Would be nice to have
the messages ending consistently in "..." or ".".

[7]

+ "local") # "lxc")

The comment looks like a brainstorm left over.

[8]

+ """
+ Shutdown the m...

review: Approve

[lp:~hazmat/pyjuju/local-provider]

375. By Kapil Thangavelu on 2011-09-22

address review comments, local provider tests don't require pkg install

[Kapil Thangavelu (hazmat)]

Excerpts from Gustavo Niemeyer's message of Thu Sep 22 16:03:25 UTC 2011:
> Review: Approve
>
> Thanks Kapil. Good stuff in record time.
>

Thanks for the review!

>
> [1]
>
> Tests fail due to missing dependencies in the environment.
>
> ===============================================================================
> [ERROR]
> Traceback (most recent call last):
> File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1020, in _inlineCallbacks
> result = g.send(result)
> File "/home/niemeyer/src/juju/hazmat/local-provider/juju/providers/lxc/__init__.py", line 53, in bootstrap
> ", ".join(sorted(list(missing)))))
> juju.errors.ProviderError: Missing packages apt-cacher-ng, libvirt-bin, lxc, zookeeperd
>
> Not sure if we should just say "install the packages" as the answer for
> that one, though.
>

I'll move this to a package level global and patch it for testing.

>
> [2]
>
> + # Install required packages
>
> It's checking for, rather than installing. (which is fine, message just
> needs fixing)
>
>
> [3]
>
> Another one:
>
> File "/home/niemeyer/src/juju/hazmat/local-provider/juju/providers/lxc/__init__.py", line 64, in bootstrap
> net_attributes = yield net.get_attributes()
> File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 207, in _worker
> result = context.call(ctx, function, *args, **kwargs)
> File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 59, in callWithContext
> return self.currentContext().callWithContext(ctx, func, *args, **kw)
> File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 37, in callWithContext
> return func(*args,**kw)
> File "/home/niemeyer/src/juju/hazmat/local-provider/juju/providers/lxc/network.py", line 135, in get_network_attributes
> output = subprocess.check_output(["virsh", "net-dumpxml", name])
> File "/usr/lib/python2.7/subprocess.py", line 537, in check_output
> raise CalledProcessError(retcode, cmd, output=output)
> subprocess.CalledProcessError: Command '['virsh', 'net-dumpxml', 'default']' returned non-zero exit status 1
>
> juju.providers.lxc.tests.test_provider.LocalProviderTest.test_bootstrap
>
> Apparently virsh is being called for real in the system for tests to pass.
>

fixed, nice catch, i should do some runs without the required packages.

>
> [4]
>
> + log.debug("Starting zookeeper...")
>
> These debug messages feel like good info for the user. I suggest making them
> visible through log.info instead.
>

Done.

> [5]
>
> + # Default to running zookeeper as the user, unless the user is root.
> + zookeeper_user = os.geteuid() == 0 and "zookeeper" or None
>
> This shorthand version is actually not that short and makes things
> significantly less straightforward to parse when reading.
>
> Compare it to:
>
> zookeeper_user = None
> if os.getuid() == 0:
> zookeeper_user = "zookeeper"
>
> Please also tweak the comment if possible (or remove since the version above
> is clear). The comment is confusing since it's running as the zookeeper user
> if the user is root, and it's easy to read it as stating the...

=== modified file 'juju/lib/zk.py'
--- juju/lib/zk.py	2011-09-16 00:36:31 +0000
+++ juju/lib/zk.py	2011-09-22 19:36:25 +0000
@@ -49,7 +49,7 @@
 
 class Zookeeper(object):
 
-    def __init__(self, run_dir, port, host=None,
+    def __init__(self, run_dir, port=None, host=None,
                  zk_location="system", user=None, group=None,
                  use_deferred=True):
         """
@@ -75,6 +75,8 @@
         self._use_deferred = use_deferred
 
     def start(self):
+        assert self._port is not None
+
         if self._use_deferred:
             return deferToThread(self._start)
         return self._start()

=== modified file 'juju/providers/common/connect.py'
--- juju/providers/common/connect.py	2011-09-15 18:50:23 +0000
+++ juju/providers/common/connect.py	2011-09-22 19:36:25 +0000
@@ -31,7 +31,7 @@
         candidates = yield self._provider.get_zookeeper_machines()
         chosen = yield self._pick_machine(candidates)
         client = yield self._connect_to_machine(chosen, share)
-        yield self._wait_for_initialization(client)
+        yield self.wait_for_initialization(client)
         returnValue(client)
 
     def _pick_machine(self, machines):
@@ -55,7 +55,7 @@
             % (machine.instance_id, str(failure.value)))
 
     @inlineCallbacks
-    def _wait_for_initialization(self, client):
+    def wait_for_initialization(self, client):
         exists_d, watch_d = client.exists_and_watch("/initialized")
         exists = yield exists_d
         if not exists:

=== modified file 'juju/providers/lxc/__init__.py'
--- juju/providers/lxc/__init__.py	2011-08-30 21:10:46 +0000
+++ juju/providers/lxc/__init__.py	2011-09-22 19:36:25 +0000
@@ -1,1 +1,210 @@
-#
+import os
+import logging
+
+from twisted.internet.defer import succeed, fail, inlineCallbacks, returnValue
+
+from txzookeeper import ZookeeperClient
+
+from juju.errors import ProviderError, EnvironmentNotFound
+from juju.lib.lxc import LXCContainer
+from juju.lib.zk import Zookeeper
+from juju.providers.common.base import MachineProviderBase
+from juju.providers.common.files import FileStorage
+from juju.providers.common.connect import ZookeeperConnect
+from juju.providers.lxc.agent import ManagedMachineAgent
+from juju.providers.lxc.container import get_containers
+from juju.providers.lxc.machine import LocalMachine
+from juju.providers.lxc.network import Network
+from juju.providers.lxc.pkg import check_packages
+from juju.state.auth import make_identity
+from juju.state.initialize import StateHierarchy
+from juju.state.utils import get_open_port
+
+log = logging.getLogger("juju.local-dev")
+
+REQUIRED_PACKAGES = ["zookeeperd", "libvirt-bin", "lxc", "apt-cacher-ng"]
+
+
+class MachineProvider(MachineProviderBase):
+    """LXC/Ubuntu local provider.
+
+    Only the host machine is utilized.
+    """
+    def __init__(self, environment_name, config):
+        super(MachineProvider, self).__init__(environment_name, config)
+        self._qualified_name = self._get_qualified_name()
+        self._directory = os.path.join(
+            self.config.get("data-dir", "/var/lib/juju/"),
+            self._qualified_name)
+
+    @inlineCallbacks
+    def bootstrap(self):
+        """Bootstrap a local development environment.
+        """
+        # Check for existing environment
+        state = yield self.load_state()
+        if state is not False:
+            raise ProviderError("Environment already bootstrapped")
+
+        # Check for required packages
+        log.info("Checking for required packages...")
+        missing = check_packages(*REQUIRED_PACKAGES)
+        if missing:
+            raise ProviderError("Missing packages %s" % (
+                ", ".join(sorted(list(missing)))))
+
+        # Get/create directory for zookeeper and files
+        zookeeper_dir = os.path.join(self._directory, "zookeeper")
+        if not os.path.exists(zookeeper_dir):
+            os.makedirs(zookeeper_dir)
+
+        # Start networking, and get an open port.
+        log.info("Starting networking...")
+        net = Network("default", subnet=122)
+        # Start is a noop if its already started, which it is by default,
+        # per libvirt-bin package installation
+        yield net.start()
+        net_attributes = yield net.get_attributes()
+        port = get_open_port(net_attributes["ip"]["address"])
+
+        # Start zookeeper
+        log.info("Starting zookeeper...")
+        # Run zookeeper as the current user, unless we're being run as root
+        # in which case run zookeeper as the 'zookeeper' user.
+        zookeeper_user = None
+        if os.geteuid() == 0:
+            zookeeper_user = "zookeeper"
+        zookeeper = Zookeeper(zookeeper_dir,
+                              port=port,
+                              host=net_attributes["ip"]["address"],
+                              user=zookeeper_user, group=zookeeper_user)
+
+        yield zookeeper.start()
+
+        # Save the zookeeper start to provider storage.
+        yield self.save_state({"zookeeper-instances": ["local"],
+                               "zookeeper-address": zookeeper.address})
+
+        # Initialize the zookeeper state
+        log.debug("Initializing state...")
+        admin_identity = make_identity(
+            "admin:%s" % self.config["admin-secret"])
+        client = ZookeeperClient(zookeeper.address)
+        yield client.connect()
+        hierarchy = StateHierarchy(client, admin_identity, "local")
+        yield hierarchy.initialize()
+
+        # Startup the machine agent
+        log.info("Starting machine agent...")
+        pid_file = os.path.join(self._directory, "machine-agent.pid")
+        log_file = os.path.join(self._directory, "machine-agent.log")
+        agent = ManagedMachineAgent(pid_file,
+                                    zookeeper_hosts=zookeeper.address,
+                                    machine_id="0",
+                                    juju_directory=self._directory,
+                                    log_file=log_file)
+        yield agent.start()
+
+        log.info("Environment bootstrapped")
+
+    @inlineCallbacks
+    def destroy_environment(self):
+        """Shutdown the machine environment.
+        """
+        # Stop all the containers
+        log.info("Destroying unit containers...")
+        yield self._destroy_containers()
+
+        # Stop the machine agent
+        log.debug("Stopping machine agent...")
+        pid_file = os.path.join(self._directory, "machine-agent.pid")
+        agent = ManagedMachineAgent(pid_file)
+        yield agent.stop()
+
+        # Stop zookeeper
+        log.debug("Stopping zookeeper...")
+        zookeeper_dir = os.path.join(self._directory, "zookeeper")
+        zookeeper = Zookeeper(zookeeper_dir, None)
+        yield zookeeper.stop()
+
+        # Clean out local state
+        yield self.save_state(False)
+
+        # Don't stop the network since we're using the default libvirt
+        log.debug("Environment destroyed.")
+
+    @inlineCallbacks
+    def _destroy_containers(self):
+        container_map = yield get_containers(self._qualified_name)
+        for container_name in container_map:
+            container = LXCContainer(container_name)
+            if container_map[container.container_name]:
+                yield container.stop()
+            yield container.destroy()
+
+    @inlineCallbacks
+    def connect(self, share=False):
+        """Connect to juju's zookeeper.
+        """
+        state = yield self.load_state()
+        if not state:
+            raise EnvironmentNotFound()
+        client = yield ZookeeperClient(state["zookeeper-address"]).connect()
+        yield ZookeeperConnect(self).wait_for_initialization(client)
+        returnValue(client)
+
+    def get_file_storage(self):
+        """Retrieve the provider C{FileStorage} abstraction.
+        """
+        storage_path = self.config.get(
+            "storage-dir", os.path.join(self._directory, "files"))
+        if not os.path.exists(storage_path):
+            try:
+                os.makedirs(storage_path)
+            except OSError:
+                raise ProviderError(
+                    "Unable to create file storage for environment")
+        return FileStorage(storage_path)
+
+    def start_machine(self, machine_data, master=False):
+        """Start a machine in the provider.
+
+        @param machine_data: a dictionary of data to pass along to the newly
+           launched machine.
+
+        @param master: if True, machine will initialize the juju admin
+            and run a provisioning agent.
+        """
+        return fail(ProviderError("Only local machine available"))
+
+    def shutdown_machine(self, machine_id):
+        return fail(ProviderError(
+            "Not enabled for local dev, use remove-unit"))
+
+    def get_machines(self, instance_ids=()):
+        """List machines running in the provider.
+
+        @param instance_ids: ids of instances you want to get. Leave blank
+            to list all machines owned by juju.
+
+        @return: a list of LocalMachine, always contains one item.
+
+        @raise: MachinesNotFound
+        """
+        if instance_ids and instance_ids != [LocalMachine.instance_id]:
+            raise ProviderError("Only local machine available")
+        return succeed([LocalMachine()])
+
+    def _get_qualified_name(self):
+        """Get a qualified environment name for use by local dev resources
+        """
+        # Ensure we namespace resources by user.
+        user = os.environ.get("USER")
+
+        # We need sudo access for lxc (till user namespaces), use the actual
+        # user.
+        if user == "root":
+            sudo_user = os.environ.get("SUDO_USER")
+            if sudo_user:
+                user = sudo_user
+        return "%s-%s" % (user, self.environment_name)

=== modified file 'juju/providers/lxc/agent.py'
--- juju/providers/lxc/agent.py	2011-09-17 00:07:16 +0000
+++ juju/providers/lxc/agent.py	2011-09-22 19:36:25 +0000
@@ -22,16 +22,16 @@
         :param juju_directory: The directory to use for all state and logs.
         """
         self._pid_file = pid_file
-        self._log_file = log_file
         self._machine_id = machine_id
         self._zookeeper_hosts = zookeeper_hosts
         self._juju_directory = juju_directory
+        self._log_file = log_file
 
     @inlineCallbacks
     def start(self):
         """Start the machine agent.
         """
-        assert self._zookeeper_hosts
+        assert self._zookeeper_hosts and self._log_file
 
         if (yield self.is_running()):
             return
@@ -44,7 +44,8 @@
                 "JUJU_HOME=%s" % self._juju_directory,
                 "PYTHONPATH=%s" % ":".join(sys.path),
                 sys.executable, "-m", self.agent_module,
-                "-n", "--pidfile", self._pid_file, "--logfile", self._log_file]
+                "-n", "--pidfile", self._pid_file,
+                "--logfile", self._log_file]
 
         yield deferToThread(subprocess.check_call, args)
 

=== added file 'juju/providers/lxc/container.py'
--- juju/providers/lxc/container.py	1970-01-01 00:00:00 +0000
+++ juju/providers/lxc/container.py	2011-09-22 19:36:25 +0000
@@ -0,0 +1,29 @@
+from twisted.internet.threads import deferToThread
+
+import subprocess
+
+
+def get_containers(prefix=None):
+    """Return a dictionary of containers key names to runtime boolean value.
+
+    :param prefix: Optionally specify a prefix that the container should
+          match any returned containers.
+    """
+    return deferToThread(_list_containers, prefix)
+
+
+def _list_containers(prefix):
+    output = subprocess.check_output(["lxc-ls"])
+
+    containers = {}
+    for i in filter(None, output.split("\n")):
+        if i in containers:
+            containers[i] = True
+        else:
+            containers[i] = False
+
+    if prefix:
+        remove = [k for k in containers.keys() if not k.startswith(prefix)]
+        map(containers.pop, remove)
+
+    return containers

=== added file 'juju/providers/lxc/pkg.py'
--- juju/providers/lxc/pkg.py	1970-01-01 00:00:00 +0000
+++ juju/providers/lxc/pkg.py	2011-09-22 19:36:25 +0000
@@ -0,0 +1,18 @@
+import apt
+
+
+def check_packages(*packages):
+    """Given a list of packages, return the packages which are not installed.
+    """
+    cache = apt.Cache()
+    missing = set()
+    for pkg_name in packages:
+        try:
+            pkg = cache[pkg_name]
+        except KeyError:
+            missing.add(pkg_name)
+            continue
+        if pkg.is_installed:
+            continue
+        missing.add(pkg_name)
+    return missing

=== added file 'juju/providers/lxc/tests/test_container.py'
--- juju/providers/lxc/tests/test_container.py	1970-01-01 00:00:00 +0000
+++ juju/providers/lxc/tests/test_container.py	2011-09-22 19:36:25 +0000
@@ -0,0 +1,38 @@
+import subprocess
+
+from twisted.internet.defer import inlineCallbacks
+
+from juju.lib.testing import TestCase
+from juju.providers.lxc.container import get_containers
+
+lxc_output_sample = "\
+calendarserver\ncaprica\ngemini\nreconnoiter\nvirgo\ncalendarserver\n"
+
+
+class GetContainersTest(TestCase):
+
+    @inlineCallbacks
+    def test_get_containers(self):
+        lxc_ls_mock = self.mocker.mock()
+        self.patch(subprocess, "check_output", lxc_ls_mock)
+        lxc_ls_mock(["lxc-ls"])
+        self.mocker.result(lxc_output_sample)
+        self.mocker.replay()
+
+        container_map = yield get_containers()
+        self.assertEqual(
+            dict(caprica=False, gemini=False, reconnoiter=False, virgo=False,
+                 calendarserver=True),
+            container_map)
+
+    @inlineCallbacks
+    def test_get_containers_with_prefix(self):
+        lxc_ls_mock = self.mocker.mock()
+        self.patch(subprocess, "check_output", lxc_ls_mock)
+        lxc_ls_mock(["lxc-ls"])
+        self.mocker.result(lxc_output_sample)
+        self.mocker.replay()
+
+        container_map = yield get_containers("ca")
+        self.assertEqual(
+            dict(calendarserver=True, caprica=False), container_map)

=== added file 'juju/providers/lxc/tests/test_pkg.py'
--- juju/providers/lxc/tests/test_pkg.py	1970-01-01 00:00:00 +0000
+++ juju/providers/lxc/tests/test_pkg.py	2011-09-22 19:36:25 +0000
@@ -0,0 +1,27 @@
+import apt
+from juju.lib.testing import TestCase
+from juju.providers.lxc.pkg import check_packages
+
+
+class PackageInstallTest(TestCase):
+
+    def test_package_reports_missing(self):
+        pkg_name = "tryton-modules-sale-opportunity"
+        missing_pkg = self.mocker.mock()
+        mock_cache = self.mocker.patch(apt.Cache)
+        mock_cache[pkg_name]
+        self.mocker.result(missing_pkg)
+        missing_pkg.is_installed
+        self.mocker.result(False)
+        self.mocker.replay()
+        self.assertEqual(
+            check_packages(pkg_name), set([pkg_name]))
+
+    def test_package_reports_installed(self):
+        self.assertEqual(
+            check_packages("python-apt"), set())
+
+    def test_package_handles_unknown(self):
+        self.assertEqual(
+            check_packages("global-gook"),
+            set(["global-gook"]))

=== added file 'juju/providers/lxc/tests/test_provider.py'
--- juju/providers/lxc/tests/test_provider.py	1970-01-01 00:00:00 +0000
+++ juju/providers/lxc/tests/test_provider.py	2011-09-22 19:36:25 +0000
@@ -0,0 +1,177 @@
+import logging
+import os
+import pwd
+import subprocess
+from StringIO import StringIO
+import zookeeper
+
+from twisted.internet.defer import succeed, inlineCallbacks
+
+from txzookeeper.tests.utils import deleteTree
+
+from juju.errors import ProviderError, EnvironmentNotFound
+from juju.lib.lxc import LXCContainer
+from juju.lib.testing import TestCase
+from juju.tests.common import get_test_zookeeper_address
+from juju.lib.zk import Zookeeper
+from juju.providers.lxc import MachineProvider
+from juju.providers import lxc
+from juju.providers.lxc.agent import ManagedMachineAgent
+from juju.providers.lxc.network import Network
+
+
+class LocalProviderTest(TestCase):
+
+    @inlineCallbacks
+    def setUp(self):
+        self.provider = MachineProvider(
+            "local-test", {
+                "admin-secret": "admin:abc", "data-dir": self.makeDir()})
+        self.output = self.capture_logging(
+            "juju.local-dev", level=logging.DEBUG)
+        zookeeper.set_debug_level(0)
+        self.client = yield self.get_zookeeper_client().connect()
+
+    def tearDown(self):
+        deleteTree("/", self.client.handle)
+        self.client.close()
+
+    @property
+    def qualified_name(self):
+        user_name = pwd.getpwuid(os.getuid()).pw_name
+        return "%s-%s" % (user_name, self.provider.environment_name)
+
+    def assertDir(self, *path_parts):
+        path = os.path.join(*path_parts)
+        self.assertTrue(os.path.isdir(path))
+
+    def bootstrap_mock(self):
+        self.patch(lxc, "REQUIRED_PACKAGES", [])
+        mock_network = self.mocker.patch(Network)
+        mock_network.start()
+        self.mocker.result(succeed(True))
+        mock_network.get_attributes()
+        self.mocker.result({"ip": {"address": "127.0.0.1"}})
+
+        mock_zookeeper = self.mocker.patch(Zookeeper)
+        mock_zookeeper.start()
+        self.mocker.result(succeed(True))
+
+        mock_zookeeper.address
+        self.mocker.result(get_test_zookeeper_address())
+        self.mocker.count(3)
+
+        mock_agent = self.mocker.patch(ManagedMachineAgent)
+        mock_agent.start()
+        self.mocker.result(succeed(True))
+
+    @inlineCallbacks
+    def test_bootstrap(self):
+        self.bootstrap_mock()
+        self.mocker.replay()
+
+        yield self.provider.bootstrap()
+
+        children = yield self.client.get_children("/")
+        self.assertEqual(
+            sorted(['services', 'charms', 'relations', 'zookeeper',
+                    'initialized', 'topology', 'machines', 'units']),
+            sorted(children))
+        output = self.output.getvalue()
+        self.assertIn("Starting networking...", output)
+        self.assertIn("Starting zookeeper...", output)
+        self.assertIn("Initializing state...", output)
+        self.assertIn("Starting machine agent...", output)
+        self.assertIn("Environment bootstrapped", output)
+
+        self.assertDir(
+            self.provider.config["data-dir"], self.qualified_name, "files")
+        self.assertDir(
+            self.provider.config["data-dir"], self.qualified_name, "zookeeper")
+
+        self.assertEqual((yield self.provider.load_state()),
+                         {"zookeeper-address": get_test_zookeeper_address(),
+                          "zookeeper-instances": ["local"]})
+
+    @inlineCallbacks
+    def test_boostrap_previously_bootstrapped(self):
+        """Any local state is a sign that we had a previous bootstrap.
+        """
+        yield self.provider.save_state({"xyz": 1})
+        error = yield self.assertFailure(
+            self.provider.bootstrap(), ProviderError)
+        self.assertEqual(str(error), "Environment already bootstrapped")
+
+    @inlineCallbacks
+    def test_destroy_environment(self):
+        """Destroying a local environment kills units, zk, and machine agent.
+        """
+        user_name = pwd.getpwuid(os.getuid()).pw_name
+        # Mock container destruction, including stopping running contianers.
+        lxc_ls_mock = self.mocker.mock()
+        self.patch(subprocess, "check_output", lxc_ls_mock)
+        lxc_ls_mock(["lxc-ls"])
+        self.mocker.result(
+            "%(name)s-local-test-unit-1\n%(name)s-local-test-unit-2\n"
+            "%(name)s-local-test-unit-3\n%(name)s-local-test-unit-1\n"
+            "%(name)s-local-test-unit-2\n" % dict(name=user_name))
+
+        mock_container = self.mocker.patch(LXCContainer)
+        mock_container.stop()
+        self.mocker.count(2)
+        mock_container.destroy()
+        self.mocker.count(3)
+
+        mock_agent = self.mocker.patch(ManagedMachineAgent)
+        mock_agent.stop()
+
+        mock_zookeeper = self.mocker.patch(Zookeeper)
+        mock_zookeeper.stop()
+        yield self.provider.save_state({"i-exist": 1})
+        self.mocker.replay()
+
+        yield self.provider.destroy_environment()
+
+        output = self.output.getvalue()
+
+        self.assertIn("Destroying unit containers..", output)
+        self.assertIn("Stopping machine agent..", output)
+        self.assertIn("Stopping zookeeper..", output)
+        self.assertIn("Environment destroyed", output)
+
+        self.assertEqual((yield self.provider.load_state()), False)
+
+    @inlineCallbacks
+    def test_connect(self):
+        self.bootstrap_mock()
+        self.mocker.replay()
+        yield self.provider.bootstrap()
+        client = yield self.provider.connect()
+        self.assertTrue((yield client.exists("/initialized")))
+
+    def test_connect_sans_environment(self):
+        return self.assertFailure(self.provider.connect(), EnvironmentNotFound)
+
+    def test_shutdown_machine(self):
+        return self.assertFailure(
+            self.provider.shutdown_machine("local"), ProviderError)
+
+    def test_start_machine(self):
+        return self.assertFailure(
+            self.provider.start_machine({}), ProviderError)
+
+    @inlineCallbacks
+    def test_get_machines(self):
+        machines = yield self.provider.get_machines()
+        self.assertEqual(len(machines), 1)
+        self.assertEqual(machines[0].instance_id, "local")
+        self.assertEqual(machines[0].dns_name, "localhost")
+
+    @inlineCallbacks
+    def test_get_file_storage(self):
+        storage = self.provider.get_file_storage()
+        content = StringIO("put")
+        yield storage.put("abc", content)
+        data_dir = self.provider.config["data-dir"]
+        self.assertTrue("abc" in os.listdir(
+            os.path.join(data_dir, self.qualified_name, "files")))

=== modified file 'juju/state/tests/test_utils.py'
--- juju/state/tests/test_utils.py	2011-09-15 18:50:23 +0000
+++ juju/state/tests/test_utils.py	2011-09-22 19:36:25 +0000
@@ -244,6 +244,19 @@
         sock.close()
         del sock
 
+    def test_get_open_port_with_host(self):
+        port = get_open_port("localhost")
+        self.assertTrue(isinstance(port, int))
+
+        sock = socket.socket(
+            socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP)
+
+        # would raise an error if we got it wrong.
+        sock.bind(("127.0.0.1", port))
+        sock.listen(1)
+        sock.close()
+        del sock
+
 
 class ChangeItemsTest(TestCase):
     """Tests the formatting of change items.

=== modified file 'juju/state/utils.py'
--- juju/state/utils.py	2011-09-15 18:50:23 +0000
+++ juju/state/utils.py	2011-09-22 19:36:25 +0000
@@ -83,11 +83,12 @@
     yield client.delete(path)
 
 
-def get_open_port():
+def get_open_port(host=""):
     """Get an open port on the machine.
     """
     temp_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-    temp_sock.bind(("", 0))
+    temp_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+    temp_sock.bind((host, 0))
     port = temp_sock.getsockname()[1]
     temp_sock.close()
     del temp_sock