Merge lp:~allenap/postgresfixture/it-begins into lp:~lazr-developers/postgresfixture/trunk

[Gavin Panella]

It starts now.

This is taken from an unlanded branch for the MAAS project, lp:~allenap/maas/database-run. I've then split it up and projectified it. I started using buildout until I just couldn't bear it any longer - it would not Just Work - so I switched to virtualenv+pip and have been much happier since. It's less powerful, but equally I have spent *much* less time toying with it.

[Robert Collins (lifeless)]

How is this better than van.pg which we already use? Looks like
little/no difference. Did you try using van.pg?

[Gavin Panella (allenap)]

> How is this better than van.pg which we already use? Looks like
> little/no difference. Did you try using van.pg?

I did try using van.pg and started writing wrappers around it to get
the behaviour I wanted, until it became apparent that van.pg made too
many assumptions that were not suitable.

Additionally:

- van.pg has 2 tests. This has 69.

- IMO, this models a cluster much better than van.pg. The model for a
  pre-existing cluster and one that this code creates is the same;
  van.pg has RunningCluster and Cluster, the former supporting a
  subset of the operations of the latter.

- This has the concept of creating a cluster but preserving it instead
  of destroying it. This makes subsequent runs quicker, and also helps
  when using the cluster for local development.

- This has a command-line tool for working with the cluster. With this
  it's easy to run a cluster, open a shell, stop, or destroy it.

- This has the concept of multiple users of the cluster. Only the last
  user will stop or destroy the cluster. Again, this is useful for
  local development, where a cluster is used both for the development
  service and running tests.

- I believe that this is a better platform for future development; I
  think it's higher quality.

- van.pg contains unavoidable code that seems very specific to someone
  else's problem, in createdb() for example:

        assert template is None or template.startswith('test_db'), template
        assert template is None or int(template[7:]) <= self._db_counter, (template, self._db_counter)

Things van.pg has that this does not:

- Can be used as a test resource. However, so can postgresfixture via
  FixtureResource.

- Integration with the transaction module when using the test resource
  class, DatabaseManager (not sure why it's limited in this way). I
  developed postgresfixture for Django in MAAS, which has its own way
  of sorting out the database for tests, but it would be fairly simple
  to add something like this if there was a need. Also,

- When not using the test resource, van.pg has ConnWrapper to try and
  detect dirtied databases. It's a leaky abstraction though, and I
  think there are better ways of doing this.

- Background creation of new databases; while one test runs a pristine
  database is prepared. I wonder if this actually buys much.

[Richard Harding (rharding)]

Phew, so I've not used this or van.pg, but from a pure code review stand point. Thanks for the lesson that assertions will return items. I hadn't seen things like the .code on the assertRaises before.

- The dropdb test: does it provide no feedback that you're dropping a db that doesn't exist? Since the test asserts nothing, is it basically just that it runs?

- Just a suggestion to move the testtools requirement to a tests_require and try to wrap the test script into a test_suite entry in the setup.py so that the normal setup.py test can work and it'll fetch the extra dependency.

- As a new user, a little more in the readme, at least a hint as to the cmd line client name so I can hit up --help ootb would be useful. It links to the LP project page, but there's no additional usage/etc there I can see.

- There's a TODO, should this follow our XXX style in LP more? I'm not sure on how this works for outside projects.

review: Approve (code)

[Gavin Panella (allenap)]

> Phew, so I've not used this or van.pg, but from a pure code review
> stand point. Thanks for the lesson that assertions will return
> items. I hadn't seen things like the .code on the assertRaises
> before.
>
> - The dropdb test: does it provide no feedback that you're dropping
> a db that doesn't exist? Since the test asserts nothing, is it
> basically just that it runs?

Yes. I've added a comment to make that clear.

>
> - Just a suggestion to move the testtools requirement to a
> tests_require and try to wrap the test script into a test_suite
> entry in the setup.py so that the normal setup.py test can work and
> it'll fetch the extra dependency.

Oh, that's perfect, great suggestion, thanks.

>
> - As a new user, a little more in the readme, at least a hint as to the cmd
> line client name so I can hit up --help ootb would be useful. It links to the
> LP project page, but there's no additional usage/etc there I can see.

Yes, I'll work on this before landing.

>
> - There's a TODO, should this follow our XXX style in LP more? I'm not sure on
> how this works for outside projects.

Oops, that's done, so I've removed it.

Thanks for the review!

[lp:~allenap/postgresfixture/it-begins]

25. By Gavin Panella on 2012-05-21

Comment about the dropdb test.

26. By Gavin Panella on 2012-05-21

Move test stuff into setup.py.

27. By Gavin Panella on 2012-05-21

Remove test script; use setup.py instead (via Makefile).

28. By Gavin Panella on 2012-05-21

Remove TODO; it's done.

29. By Gavin Panella on 2012-05-21

testtools is needed for running as well as tests.

30. By Gavin Panella on 2012-05-21

More docs.

=== added file '.bzrignore'
--- .bzrignore	1970-01-01 00:00:00 +0000
+++ .bzrignore	2012-05-21 11:22:18 +0000
@@ -0,0 +1,10 @@
+*.egg
+./bin
+./build
+./dist
+./include
+./lib
+./local
+./postgresfixture.egg-info
+./TAGS
+./tags

=== added file 'Makefile'
--- Makefile	1970-01-01 00:00:00 +0000
+++ Makefile	2012-05-21 11:22:18 +0000
@@ -0,0 +1,25 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+PYTHON := python
+
+build: bin/python bin/pip requirements.txt
+	bin/pip install --requirement=requirements.txt
+
+dist: setup.py
+	bin/python setup.py egg_info -r sdist
+
+test: bin/python
+	bin/python setup.py test
+
+clean:
+	$(RM) -r bin build dist include lib local TAGS tags
+	find . -name '*.py[co]' -print0 | xargs -r0 $(RM) -r
+	find . -name '*.egg' -print0 | xargs -r0 $(RM) -r
+	find . -name '*.egg-info' -print0 | xargs -r0 $(RM) -r
+	find . -name '*~' -print0 | xargs -r0 $(RM)
+
+bin/python bin/pip:
+	virtualenv --python=$(PYTHON) $(PWD)
+
+.PHONY: build dist clean

=== added file 'README.txt'
--- README.txt	1970-01-01 00:00:00 +0000
+++ README.txt	2012-05-21 11:22:18 +0000
@@ -0,0 +1,43 @@
+.. -*- mode: rst -*-
+
+***************
+postgresfixture
+***************
+
+A Python fixture for creating PostgreSQL clusters and databases, and
+tearing them down again, intended for use during development and
+testing.
+
+For more information see the `Launchpad project page`_.
+
+.. _Launchpad project page: https://launchpad.net/postgresfixture
+
+
+Getting started
+===============
+
+Use like any other fixture::
+
+  from contextlib import closing
+  from postgresfixture import ClusterFixture
+
+  def test_something(self):
+      cluster = self.useFixture(ClusterFixture("db"))
+      cluster.createdb("example")
+      with closing(cluster.connect("example")) as conn:
+          ...
+      cluster.dropbdb("example")  # Optional.
+
+This will create a new cluster, create a database called "example",
+then tear it all down at the end; nothing will remain on disk. If you
+want the cluster and its databases to remain on disk, pass
+``preserve=True`` to the ``ClusterFixture`` constructor.
+
+
+From the command line
+=====================
+
+Once this package is installed, you'll have a ``postgresfixture``
+script. Alternatively you can use ``python -m postgresfixture`` to
+achieve the same thing. Use ``--help`` to discover the options
+available to you.

=== added directory 'postgresfixture'
=== added file 'postgresfixture/__init__.py'
--- postgresfixture/__init__.py	1970-01-01 00:00:00 +0000
+++ postgresfixture/__init__.py	2012-05-21 11:22:18 +0000
@@ -0,0 +1,17 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""PostgreSQL cluster fixture."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+__metaclass__ = type
+__all__ = [
+    "ClusterFixture",
+    ]
+
+from postgresfixture.clusterfixture import ClusterFixture

=== added file 'postgresfixture/__main__.py'
--- postgresfixture/__main__.py	1970-01-01 00:00:00 +0000
+++ postgresfixture/__main__.py	2012-05-21 11:22:18 +0000
@@ -0,0 +1,17 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Entry point for `postgresfixture` on the command-line."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+__metaclass__ = type
+__all__ = []
+
+from postgresfixture.main import main
+
+main()

=== added file 'postgresfixture/cluster.py'
--- postgresfixture/cluster.py	1970-01-01 00:00:00 +0000
+++ postgresfixture/cluster.py	2012-05-21 11:22:18 +0000
@@ -0,0 +1,167 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Manage a PostgreSQL cluster."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+__metaclass__ = type
+__all__ = [
+    "Cluster",
+    ]
+
+from contextlib import closing
+from os import (
+    devnull,
+    environ,
+    makedirs,
+    path,
+    )
+import pipes
+from shutil import rmtree
+from subprocess import (
+    CalledProcessError,
+    check_call,
+    )
+
+import psycopg2
+
+
+PG_VERSION = "9.1"
+PG_BIN = "/usr/lib/postgresql/%s/bin" % PG_VERSION
+
+
+def path_with_pg_bin(exe_path):
+    """Return `exe_path` with `PG_BIN` added."""
+    exe_path = [
+        elem for elem in exe_path.split(path.pathsep)
+        if len(elem) != 0 and not elem.isspace()
+        ]
+    if PG_BIN not in exe_path:
+        exe_path.insert(0, PG_BIN)
+    return path.pathsep.join(exe_path)
+
+
+class Cluster:
+    """Represents a PostgreSQL cluster, running or not."""
+
+    def __init__(self, datadir):
+        self.datadir = path.abspath(datadir)
+
+    def execute(self, *command, **options):
+        """Execute a command with an environment suitable for this cluster."""
+        env = options.pop("env", environ).copy()
+        env["PATH"] = path_with_pg_bin(env.get("PATH", ""))
+        env["PGDATA"] = env["PGHOST"] = self.datadir
+        check_call(command, env=env, **options)
+
+    @property
+    def exists(self):
+        """Whether or not this cluster exists on disk."""
+        version_file = path.join(self.datadir, "PG_VERSION")
+        return path.exists(version_file)
+
+    @property
+    def pidfile(self):
+        """The (expected) pidfile for a running cluster.
+
+        Does *not* guarantee that the pidfile exists.
+        """
+        return path.join(self.datadir, "postmaster.pid")
+
+    @property
+    def logfile(self):
+        """The log file used (or will be used) by this cluster."""
+        return path.join(self.datadir, "backend.log")
+
+    @property
+    def running(self):
+        """Whether this cluster is running or not."""
+        with open(devnull, "rb") as null:
+            try:
+                self.execute("pg_ctl", "status", stdout=null)
+            except CalledProcessError, error:
+                if error.returncode == 1:
+                    return False
+                else:
+                    raise
+            else:
+                return True
+
+    def create(self):
+        """Create this cluster, if it does not exist."""
+        if not self.exists:
+            if not path.isdir(self.datadir):
+                makedirs(self.datadir)
+            self.execute("pg_ctl", "init", "-s", "-o", "-E utf8 -A trust")
+
+    def start(self):
+        """Start this cluster, if it's not already started."""
+        if not self.running:
+            self.create()
+            # pg_ctl options:
+            #  -l <file> -- log file.
+            #  -s -- no informational messages.
+            #  -w -- wait until startup is complete.
+            # postgres options:
+            #  -h <arg> -- host name; empty arg means Unix socket only.
+            #  -F -- don't bother fsync'ing.
+            #  -k -- socket directory.
+            self.execute(
+                "pg_ctl", "start", "-l", self.logfile, "-s", "-w",
+                "-o", "-h '' -F -k %s" % pipes.quote(self.datadir))
+
+    def connect(self, database="template1", autocommit=True):
+        """Connect to this cluster.
+
+        Starts the cluster if necessary.
+        """
+        self.start()
+        connection = psycopg2.connect(
+            database=database, host=self.datadir)
+        connection.autocommit = autocommit
+        return connection
+
+    def shell(self, database="template1"):
+        self.execute("psql", "--quiet", "--", database)
+
+    @property
+    def databases(self):
+        """The names of databases in this cluster."""
+        with closing(self.connect("postgres")) as conn:
+            with closing(conn.cursor()) as cur:
+                cur.execute("SELECT datname FROM pg_catalog.pg_database")
+                return {name for (name,) in cur.fetchall()}
+
+    def createdb(self, name):
+        """Create the named database."""
+        with closing(self.connect()) as conn:
+            with closing(conn.cursor()) as cur:
+                cur.execute("CREATE DATABASE %s" % name)
+
+    def dropdb(self, name):
+        """Drop the named database."""
+        with closing(self.connect()) as conn:
+            with closing(conn.cursor()) as cur:
+                cur.execute("DROP DATABASE %s" % name)
+
+    def stop(self):
+        """Stop this cluster, if started."""
+        if self.running:
+            # pg_ctl options:
+            #  -w -- wait for shutdown to complete.
+            #  -m <mode> -- shutdown mode.
+            self.execute("pg_ctl", "stop", "-s", "-w", "-m", "fast")
+
+    def destroy(self):
+        """Destroy this cluster, if it exists.
+
+        The cluster will be stopped if it's started.
+        """
+        if self.exists:
+            self.stop()
+            rmtree(self.datadir)

=== added file 'postgresfixture/clusterfixture.py'
--- postgresfixture/clusterfixture.py	1970-01-01 00:00:00 +0000
+++ postgresfixture/clusterfixture.py	2012-05-21 11:22:18 +0000
@@ -0,0 +1,142 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Manage a PostgreSQL cluster."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+__metaclass__ = type
+__all__ = [
+    "ClusterFixture",
+    ]
+
+from errno import (
+    EEXIST,
+    ENOENT,
+    ENOTEMPTY,
+    )
+from os import (
+    getpid,
+    listdir,
+    makedirs,
+    path,
+    rmdir,
+    unlink,
+    )
+
+from fixtures import Fixture
+from postgresfixture.cluster import Cluster
+
+
+class ProcessSemaphore:
+    """A sort-of-semaphore where it is considered locked if a directory cannot
+    be removed.
+
+    The locks are taken out one per-process, so this is a way of keeping a
+    reference to a shared resource between processes.
+    """
+
+    def __init__(self, lockdir):
+        super(ProcessSemaphore, self).__init__()
+        self.lockdir = lockdir
+        self.lockfile = path.join(
+            self.lockdir, "%d" % getpid())
+
+    def acquire(self):
+        try:
+            makedirs(self.lockdir)
+        except OSError, error:
+            if error.errno != EEXIST:
+                raise
+        open(self.lockfile, "w").close()
+
+    def release(self):
+        try:
+            unlink(self.lockfile)
+        except OSError, error:
+            if error.errno != ENOENT:
+                raise
+
+    @property
+    def locked(self):
+        try:
+            rmdir(self.lockdir)
+        except OSError, error:
+            if error.errno == ENOTEMPTY:
+                return True
+            elif error.errno == ENOENT:
+                return False
+            else:
+                raise
+        else:
+            return False
+
+    @property
+    def locked_by(self):
+        try:
+            return [
+                int(name) if name.isdigit() else name
+                for name in listdir(self.lockdir)
+                ]
+        except OSError, error:
+            if error.errno == ENOENT:
+                return []
+            else:
+                raise
+
+
+class ClusterFixture(Cluster, Fixture):
+    """A fixture for a `Cluster`."""
+
+    def __init__(self, datadir, preserve=False):
+        """
+        @param preserve: Leave the cluster and its databases behind, even if
+            this fixture creates them.
+        """
+        super(ClusterFixture, self).__init__(datadir)
+        self.preserve = preserve
+        self.lock = ProcessSemaphore(
+            path.join(self.datadir, "locks"))
+
+    def setUp(self):
+        super(ClusterFixture, self).setUp()
+        # Only destroy the cluster if we create it...
+        if not self.exists:
+            # ... unless we've been asked to preserve it.
+            if not self.preserve:
+                self.addCleanup(self.destroy)
+            self.create()
+        self.addCleanup(self.stop)
+        self.start()
+        self.addCleanup(self.lock.release)
+        self.lock.acquire()
+
+    def createdb(self, name):
+        """Create the named database if it does not exist already.
+
+        Arranges to drop the named database during clean-up, unless `preserve`
+        has been specified.
+        """
+        if name not in self.databases:
+            super(ClusterFixture, self).createdb(name)
+            if not self.preserve:
+                self.addCleanup(self.dropdb, name)
+
+    def dropdb(self, name):
+        """Drop the named database if it exists."""
+        if name in self.databases:
+            super(ClusterFixture, self).dropdb(name)
+
+    def stop(self):
+        """Stop the cluster, but only if there are no other users."""
+        if not self.lock.locked:
+            super(ClusterFixture, self).stop()
+
+    def destroy(self):
+        """Destroy the cluster, but only if there are no other users."""
+        if not self.lock.locked:
+            super(ClusterFixture, self).destroy()

=== added file 'postgresfixture/main.py'
--- postgresfixture/main.py	1970-01-01 00:00:00 +0000
+++ postgresfixture/main.py	2012-05-21 11:22:18 +0000
@@ -0,0 +1,185 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Manage a PostgreSQL cluster."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+__metaclass__ = type
+__all__ = [
+    "main",
+    ]
+
+import argparse
+from itertools import imap
+from os import (
+    environ,
+    fdopen,
+    )
+import pipes
+import signal
+from subprocess import CalledProcessError
+import sys
+from time import sleep
+
+from postgresfixture.clusterfixture import ClusterFixture
+
+
+def setup():
+    # Ensure stdout and stderr are line-bufferred.
+    sys.stdout = fdopen(sys.stdout.fileno(), "ab", 1)
+    sys.stderr = fdopen(sys.stderr.fileno(), "ab", 1)
+    # Run the SIGINT handler on SIGTERM; `svc -d` sends SIGTERM.
+    signal.signal(signal.SIGTERM, signal.default_int_handler)
+
+
+def repr_pid(pid):
+    if isinstance(pid, int) or pid.isdigit():
+        try:
+            with open("/proc/%s/cmdline" % pid, "rb") as fd:
+                cmdline = fd.read().rstrip("\0").split("\0")
+        except IOError:
+            return "%s (*unknown*)" % pid
+        else:
+            return "%s (%s)" % (
+                pid, " ".join(imap(pipes.quote, cmdline)))
+    else:
+        return pipes.quote(pid)
+
+
+def locked_by_description(lock):
+    pids = sorted(lock.locked_by)
+    return "locked by:\n* %s" % (
+        "\n* ".join(imap(repr_pid, pids)))
+
+
+def error(*args, **kwargs):
+    kwargs.setdefault("file", sys.stderr)
+    return print(*args, **kwargs)
+
+
+def get_database_name(default="data"):
+    """Return the desired database name, used by some commands.
+
+    Obtained from the ``PGDATABASE`` environment variable.
+    """
+    return environ.get("PGDATABASE", default)
+
+
+def action_destroy(cluster):
+    """Destroy a cluster."""
+    action_stop(cluster)
+    cluster.destroy()
+    if cluster.exists:
+        if cluster.lock.locked:
+            message = "%s: cluster is %s" % (
+                cluster.datadir, locked_by_description(cluster.lock))
+        else:
+            message = "%s: cluster could not be removed." % cluster.datadir
+        error(message)
+        raise SystemExit(2)
+
+
+def action_run(cluster):
+    """Create and run a cluster.
+
+    If specified in the ``PGDATABASE`` environment variable, a database will
+    also be created within the cluster.
+    """
+    database_name = get_database_name(default=None)
+    with cluster:
+        if database_name is not None:
+            cluster.createdb(database_name)
+        while cluster.running:
+            sleep(5.0)
+
+
+def action_shell(cluster):
+    """Spawn a ``psql`` shell for a database in the cluster.
+
+    The database name can be specified in the ``PGDATABASE`` environment
+    variable.
+    """
+    database_name = get_database_name()
+    with cluster:
+        cluster.createdb(database_name)
+        cluster.shell(database_name)
+
+
+def action_status(cluster):
+    """Display a message about the state of the cluster.
+
+    The return code is also set:
+
+    - 0: cluster is running.
+    - 1: cluster exists, but is not running.
+    - 2: cluster does not exist.
+
+    """
+    if cluster.exists:
+        if cluster.running:
+            print("%s: running" % cluster.datadir)
+            raise SystemExit(0)
+        else:
+            print("%s: not running" % cluster.datadir)
+            raise SystemExit(1)
+    else:
+        print("%s: not created" % cluster.datadir)
+        raise SystemExit(2)
+
+
+def action_stop(cluster):
+    """Stop a cluster."""
+    cluster.stop()
+    if cluster.running:
+        if cluster.lock.locked:
+            message = "%s: cluster is %s" % (
+                cluster.datadir, locked_by_description(cluster.lock))
+        else:
+            message = "%s: cluster is still running." % cluster.datadir
+        error(message)
+        raise SystemExit(2)
+
+
+actions = {
+    "destroy": action_destroy,
+    "run": action_run,
+    "shell": action_shell,
+    "status": action_status,
+    "stop": action_stop,
+    }
+
+
+argument_parser = argparse.ArgumentParser(description=__doc__)
+argument_parser.add_argument(
+    "action", choices=sorted(actions), nargs="?", default="shell",
+    help="the action to perform (default: %(default)s)")
+argument_parser.add_argument(
+    "-D", "--datadir", dest="datadir", action="store_true",
+    default="db", help=(
+        "the directory in which to place, or find, the cluster "
+        "(default: %(default)s)"))
+argument_parser.add_argument(
+    "--preserve", dest="preserve", action="store_true",
+    default=False, help=(
+        "preserve the cluster and its databases when exiting, "
+        "even if it was necessary to create and start it "
+        "(default: %(default)s)"))
+
+
+def main(args=None):
+    args = argument_parser.parse_args(args)
+    try:
+        setup()
+        action = actions[args.action]
+        cluster = ClusterFixture(
+            datadir=args.datadir, preserve=args.preserve)
+        action(cluster)
+    except CalledProcessError, error:
+        raise SystemExit(error.returncode)
+    except KeyboardInterrupt:
+        pass

=== added directory 'postgresfixture/testing'
=== added file 'postgresfixture/testing/__init__.py'
--- postgresfixture/testing/__init__.py	1970-01-01 00:00:00 +0000
+++ postgresfixture/testing/__init__.py	2012-05-21 11:22:18 +0000
@@ -0,0 +1,25 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Testing resources for `postgresfixture`."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+__metaclass__ = type
+__all__ = [
+    "TestCase",
+    ]
+
+from fixtures import TempDir
+import testtools
+
+
+class TestCase(testtools.TestCase):
+    """Convenience subclass."""
+
+    def make_dir(self):
+        return self.useFixture(TempDir()).path

=== added directory 'postgresfixture/tests'
=== added file 'postgresfixture/tests/__init__.py'
=== added file 'postgresfixture/tests/test_cluster.py'
--- postgresfixture/tests/test_cluster.py	1970-01-01 00:00:00 +0000
+++ postgresfixture/tests/test_cluster.py	2012-05-21 11:22:18 +0000
@@ -0,0 +1,186 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for `postgresfixture.cluster`."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+__metaclass__ = type
+__all__ = []
+
+from contextlib import closing
+from os import (
+    getpid,
+    path,
+    )
+from subprocess import CalledProcessError
+
+import postgresfixture.cluster
+from postgresfixture.cluster import (
+    Cluster,
+    path_with_pg_bin,
+    PG_BIN,
+    )
+from postgresfixture.main import repr_pid
+from postgresfixture.testing import TestCase
+from testtools.matchers import (
+    DirExists,
+    FileExists,
+    Not,
+    StartsWith,
+    )
+
+
+class TestFunctions(TestCase):
+
+    def test_path_with_pg_bin(self):
+        self.assertEqual(PG_BIN, path_with_pg_bin(""))
+        self.assertEqual(
+            PG_BIN + path.pathsep + "/bin:/usr/bin",
+            path_with_pg_bin("/bin:/usr/bin"))
+
+    def test_repr_pid_not_a_number(self):
+        self.assertEqual("alice", repr_pid("alice"))
+        self.assertEqual("'alice and bob'", repr_pid("alice and bob"))
+
+    def test_repr_pid_not_a_process(self):
+        self.assertEqual("0 (*unknown*)", repr_pid(0))
+
+    def test_repr_pid_this_process(self):
+        pid = getpid()
+        self.assertThat(repr_pid(pid), StartsWith("%d (" % pid))
+
+
+class TestCluster(TestCase):
+
+    make = Cluster
+
+    def test_init(self):
+        # The datadir passed into the Cluster constructor is resolved to an
+        # absolute path.
+        datadir = path.join(self.make_dir(), "locks")
+        cluster = self.make(path.relpath(datadir))
+        self.assertEqual(datadir, cluster.datadir)
+
+    def patch_check_call(self, returncode=0):
+        calls = []
+
+        def check_call(command, **options):
+            calls.append((command, options))
+            if returncode != 0:
+                raise CalledProcessError(returncode, command)
+
+        self.patch(postgresfixture.cluster, "check_call", check_call)
+        return calls
+
+    def test_execute(self):
+        calls = self.patch_check_call()
+        cluster = self.make(self.make_dir())
+        cluster.execute("true")
+        [(command, options)] = calls
+        self.assertEqual(("true",), command)
+        self.assertIn("env", options)
+        env = options["env"]
+        self.assertEqual(cluster.datadir, env.get("PGDATA"))
+        self.assertEqual(cluster.datadir, env.get("PGHOST"))
+        self.assertThat(
+            env.get("PATH", ""),
+            StartsWith(PG_BIN + path.pathsep))
+
+    def test_exists(self):
+        cluster = self.make(self.make_dir())
+        # The PG_VERSION file is used as a marker of existence.
+        version_file = path.join(cluster.datadir, "PG_VERSION")
+        self.assertThat(version_file, Not(FileExists()))
+        self.assertFalse(cluster.exists)
+        open(version_file, "wb").close()
+        self.assertTrue(cluster.exists)
+
+    def test_pidfile(self):
+        self.assertEqual(
+            "/some/where/postmaster.pid",
+            self.make("/some/where").pidfile)
+
+    def test_logfile(self):
+        self.assertEqual(
+            "/some/where/backend.log",
+            self.make("/some/where").logfile)
+
+    def test_running(self):
+        calls = self.patch_check_call(returncode=0)
+        cluster = self.make("/some/where")
+        self.assertTrue(cluster.running)
+        [(command, options)] = calls
+        self.assertEqual(("pg_ctl", "status"), command)
+
+    def test_running_not(self):
+        self.patch_check_call(returncode=1)
+        cluster = self.make("/some/where")
+        self.assertFalse(cluster.running)
+
+    def test_running_error(self):
+        self.patch_check_call(returncode=2)
+        cluster = self.make("/some/where")
+        self.assertRaises(
+            CalledProcessError, getattr, cluster, "running")
+
+    def test_create(self):
+        cluster = self.make(self.make_dir())
+        cluster.create()
+        self.assertTrue(cluster.exists)
+        self.assertFalse(cluster.running)
+
+    def test_start_and_stop(self):
+        cluster = self.make(self.make_dir())
+        cluster.create()
+        try:
+            cluster.start()
+            self.assertTrue(cluster.running)
+        finally:
+            cluster.stop()
+            self.assertFalse(cluster.running)
+
+    def test_connect(self):
+        cluster = self.make(self.make_dir())
+        cluster.create()
+        self.addCleanup(cluster.stop)
+        cluster.start()
+        with closing(cluster.connect()) as conn:
+            with closing(conn.cursor()) as cur:
+                cur.execute("SELECT 1")
+                self.assertEqual([(1,)], cur.fetchall())
+
+    def test_databases(self):
+        cluster = self.make(self.make_dir())
+        cluster.create()
+        self.addCleanup(cluster.stop)
+        cluster.start()
+        self.assertEqual(
+            {"postgres", "template0", "template1"},
+            cluster.databases)
+
+    def test_createdb_and_dropdb(self):
+        cluster = self.make(self.make_dir())
+        cluster.create()
+        self.addCleanup(cluster.stop)
+        cluster.start()
+        cluster.createdb("setherial")
+        self.assertEqual(
+            {"postgres", "template0", "template1", "setherial"},
+            cluster.databases)
+        cluster.dropdb("setherial")
+        self.assertEqual(
+            {"postgres", "template0", "template1"},
+            cluster.databases)
+
+    def test_destroy(self):
+        cluster = self.make(self.make_dir())
+        cluster.create()
+        cluster.destroy()
+        self.assertFalse(cluster.exists)
+        self.assertFalse(cluster.running)
+        self.assertThat(cluster.datadir, Not(DirExists()))

=== added file 'postgresfixture/tests/test_clusterfixture.py'
--- postgresfixture/tests/test_clusterfixture.py	1970-01-01 00:00:00 +0000
+++ postgresfixture/tests/test_clusterfixture.py	2012-05-21 11:22:18 +0000
@@ -0,0 +1,168 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for `postgresfixture.clusterfixture`."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+__metaclass__ = type
+__all__ = []
+
+from os import (
+    getpid,
+    path,
+    )
+
+from postgresfixture.clusterfixture import (
+    ClusterFixture,
+    ProcessSemaphore,
+    )
+from postgresfixture.testing import TestCase
+from postgresfixture.tests.test_cluster import TestCluster
+from testtools.matchers import (
+    FileExists,
+    Not,
+    )
+
+
+class TestProcessSemaphore(TestCase):
+
+    def test_init(self):
+        lockdir = self.make_dir()
+        psem = ProcessSemaphore(lockdir)
+        self.assertEqual(lockdir, psem.lockdir)
+        self.assertEqual(
+            path.join(lockdir, "%s" % getpid()),
+            psem.lockfile)
+
+    def test_acquire(self):
+        psem = ProcessSemaphore(
+            path.join(self.make_dir(), "locks"))
+        psem.acquire()
+        self.assertThat(psem.lockfile, FileExists())
+        self.assertTrue(psem.locked)
+        self.assertEqual([getpid()], psem.locked_by)
+
+    def test_release(self):
+        psem = ProcessSemaphore(
+            path.join(self.make_dir(), "locks"))
+        psem.acquire()
+        psem.release()
+        self.assertThat(psem.lockfile, Not(FileExists()))
+        self.assertFalse(psem.locked)
+        self.assertEqual([], psem.locked_by)
+
+
+class TestClusterFixture(TestCluster):
+
+    def make(self, *args, **kwargs):
+        fixture = ClusterFixture(*args, **kwargs)
+        # Run the basic fixture set-up so that clean-ups can be added.
+        super(ClusterFixture, fixture).setUp()
+        return fixture
+
+    def test_init_fixture(self):
+        fixture = self.make("/some/where")
+        self.assertEqual(False, fixture.preserve)
+        self.assertIsInstance(fixture.lock, ProcessSemaphore)
+        self.assertEqual(
+            path.join(fixture.datadir, "locks"),
+            fixture.lock.lockdir)
+
+    def test_createdb_no_preserve(self):
+        fixture = self.make(self.make_dir(), preserve=False)
+        self.addCleanup(fixture.stop)
+        fixture.start()
+        fixture.createdb("danzig")
+        self.assertIn("danzig", fixture.databases)
+        # The database is only created if it does not already exist.
+        fixture.createdb("danzig")
+        # Creating a database arranges for it to be dropped when stopping the
+        # fixture.
+        fixture.cleanUp()
+        self.assertNotIn("danzig", fixture.databases)
+
+    def test_createdb_preserve(self):
+        fixture = self.make(self.make_dir(), preserve=True)
+        self.addCleanup(fixture.stop)
+        fixture.start()
+        fixture.createdb("emperor")
+        self.assertIn("emperor", fixture.databases)
+        # The database is only created if it does not already exist.
+        fixture.createdb("emperor")
+        # Creating a database arranges for it to be dropped when stopping the
+        # fixture.
+        fixture.cleanUp()
+        self.assertIn("emperor", fixture.databases)
+
+    def test_dropdb(self):
+        fixture = self.make(self.make_dir())
+        self.addCleanup(fixture.stop)
+        fixture.start()
+        # The database is only dropped if it exists.
+        fixture.dropdb("diekrupps")
+        fixture.dropdb("diekrupps")
+        # The test is that we arrive here without error.
+
+    def test_stop_locked(self):
+        # The cluster is not stopped if a lock is held.
+        fixture = self.make(self.make_dir())
+        self.addCleanup(fixture.stop)
+        fixture.start()
+        fixture.lock.acquire()
+        fixture.stop()
+        self.assertTrue(fixture.running)
+        fixture.lock.release()
+        fixture.stop()
+        self.assertFalse(fixture.running)
+
+    def test_destroy_locked(self):
+        # The cluster is not destroyed if a lock is held.
+        fixture = self.make(self.make_dir())
+        fixture.create()
+        fixture.lock.acquire()
+        fixture.destroy()
+        self.assertTrue(fixture.exists)
+        fixture.lock.release()
+        fixture.destroy()
+        self.assertFalse(fixture.exists)
+
+    def test_use_no_preserve(self):
+        # The cluster is stopped and destroyed when preserve=False.
+        with self.make(self.make_dir(), preserve=False) as fixture:
+            self.assertTrue(fixture.exists)
+            self.assertTrue(fixture.running)
+        self.assertFalse(fixture.exists)
+        self.assertFalse(fixture.running)
+
+    def test_use_no_preserve_cluster_already_exists(self):
+        # The cluster is stopped but *not* destroyed when preserve=False if it
+        # existed before the fixture was put into use.
+        fixture = self.make(self.make_dir(), preserve=False)
+        fixture.create()
+        with fixture:
+            self.assertTrue(fixture.exists)
+            self.assertTrue(fixture.running)
+        self.assertTrue(fixture.exists)
+        self.assertFalse(fixture.running)
+
+    def test_use_preserve(self):
+        # The cluster is not stopped and destroyed when preserve=True.
+        with self.make(self.make_dir(), preserve=True) as fixture:
+            self.assertTrue(fixture.exists)
+            self.assertTrue(fixture.running)
+            fixture.createdb("gallhammer")
+        self.assertTrue(fixture.exists)
+        self.assertFalse(fixture.running)
+        self.addCleanup(fixture.stop)
+        fixture.start()
+        self.assertIn("gallhammer", fixture.databases)
+
+    def test_namespace(self):
+        # ClusterFixture is in the postgresfixture namespace.
+        import postgresfixture
+        self.assertIs(postgresfixture.ClusterFixture, ClusterFixture)

=== added file 'postgresfixture/tests/test_main.py'
--- postgresfixture/tests/test_main.py	1970-01-01 00:00:00 +0000
+++ postgresfixture/tests/test_main.py	2012-05-21 11:22:18 +0000
@@ -0,0 +1,163 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for `postgresfixture.main`."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+__metaclass__ = type
+__all__ = []
+
+from base64 import b16encode
+from os import urandom
+from StringIO import StringIO
+import sys
+
+from fixtures import EnvironmentVariableFixture
+from postgresfixture import main
+from postgresfixture.clusterfixture import ClusterFixture
+from postgresfixture.testing import TestCase
+from testtools.matchers import StartsWith
+
+
+class TestActions(TestCase):
+
+    class Finished(Exception):
+        """A marker exception used for breaking out."""
+
+    def get_random_database_name(self):
+        return "db%s" % b16encode(urandom(8)).lower()
+
+    def test_run(self):
+        cluster = ClusterFixture(self.make_dir())
+        self.addCleanup(cluster.stop)
+
+        database_name = self.get_random_database_name()
+        self.useFixture(
+            EnvironmentVariableFixture("PGDATABASE", database_name))
+
+        # Instead of sleeping, check the cluster is running, then break out.
+        def sleep_patch(time):
+            self.assertTrue(cluster.running)
+            self.assertIn(database_name, cluster.databases)
+            raise self.Finished
+
+        self.patch(main, "sleep", sleep_patch)
+        self.assertRaises(self.Finished, main.action_run, cluster)
+
+    def test_run_without_database(self):
+        # A database is not created if it's not specified in the PGDATABASE
+        # environment variable.
+        cluster = ClusterFixture(self.make_dir())
+        self.addCleanup(cluster.stop)
+
+        # Erase the PGDATABASE environment variable, if it's set.
+        self.useFixture(
+            EnvironmentVariableFixture("PGDATABASE", None))
+
+        # Instead of sleeping, check the cluster is running, then break out.
+        def sleep_patch(time):
+            self.assertTrue(cluster.running)
+            self.assertEqual(
+                {"template0", "template1", "postgres"},
+                cluster.databases)
+            raise self.Finished
+
+        self.patch(main, "sleep", sleep_patch)
+        self.assertRaises(self.Finished, main.action_run, cluster)
+
+    def test_shell(self):
+        cluster = ClusterFixture(self.make_dir())
+        self.addCleanup(cluster.stop)
+
+        database_name = self.get_random_database_name()
+        self.useFixture(
+            EnvironmentVariableFixture("PGDATABASE", database_name))
+
+        def shell_patch(database):
+            self.assertEqual(database_name, database)
+            raise self.Finished
+
+        self.patch(cluster, "shell", shell_patch)
+        self.assertRaises(self.Finished, main.action_shell, cluster)
+
+    def test_status_running(self):
+        cluster = ClusterFixture(self.make_dir())
+        self.addCleanup(cluster.stop)
+        cluster.start()
+        self.patch(sys, "stdout", StringIO())
+        code = self.assertRaises(
+            SystemExit, main.action_status, cluster).code
+        self.assertEqual(0, code)
+        self.assertEqual(
+            "%s: running\n" % cluster.datadir,
+            sys.stdout.getvalue())
+
+    def test_status_not_running(self):
+        cluster = ClusterFixture(self.make_dir())
+        cluster.create()
+        self.patch(sys, "stdout", StringIO())
+        code = self.assertRaises(
+            SystemExit, main.action_status, cluster).code
+        self.assertEqual(1, code)
+        self.assertEqual(
+            "%s: not running\n" % cluster.datadir,
+            sys.stdout.getvalue())
+
+    def test_status_not_created(self):
+        cluster = ClusterFixture(self.make_dir())
+        self.patch(sys, "stdout", StringIO())
+        code = self.assertRaises(
+            SystemExit, main.action_status, cluster).code
+        self.assertEqual(2, code)
+        self.assertEqual(
+            "%s: not created\n" % cluster.datadir,
+            sys.stdout.getvalue())
+
+    def test_stop(self):
+        cluster = ClusterFixture(self.make_dir())
+        self.addCleanup(cluster.stop)
+        cluster.start()
+        main.action_stop(cluster)
+        self.assertFalse(cluster.running)
+        self.assertTrue(cluster.exists)
+
+    def test_stop_when_locked(self):
+        cluster = ClusterFixture(self.make_dir())
+        self.addCleanup(cluster.stop)
+        cluster.start()
+        self.addCleanup(cluster.lock.release)
+        cluster.lock.acquire()
+        self.patch(sys, "stderr", StringIO())
+        error = self.assertRaises(
+            SystemExit, main.action_stop, cluster)
+        self.assertEqual(2, error.code)
+        self.assertThat(
+            sys.stderr.getvalue(), StartsWith(
+                "%s: cluster is locked by:" % cluster.datadir))
+        self.assertTrue(cluster.running)
+
+    def test_destroy(self):
+        cluster = ClusterFixture(self.make_dir())
+        self.addCleanup(cluster.stop)
+        cluster.start()
+        main.action_destroy(cluster)
+        self.assertFalse(cluster.running)
+        self.assertFalse(cluster.exists)
+
+    def test_destroy_when_locked(self):
+        cluster = ClusterFixture(self.make_dir())
+        cluster.create()
+        cluster.lock.acquire()
+        self.patch(sys, "stderr", StringIO())
+        error = self.assertRaises(
+            SystemExit, main.action_destroy, cluster)
+        self.assertEqual(2, error.code)
+        self.assertThat(
+            sys.stderr.getvalue(), StartsWith(
+                "%s: cluster is locked by:" % cluster.datadir))
+        self.assertTrue(cluster.exists)

=== added file 'requirements.txt'
--- requirements.txt	1970-01-01 00:00:00 +0000
+++ requirements.txt	2012-05-21 11:22:18 +0000
@@ -0,0 +1,3 @@
+fixtures >= 0.3.8
+psycopg2 >= 2.4.4
+testtools >= 0.9.14

=== added file 'setup.py'
--- setup.py	1970-01-01 00:00:00 +0000
+++ setup.py	2012-05-21 11:22:18 +0000
@@ -0,0 +1,41 @@
+#!/usr/bin/env python
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Distutils installer for postgresfixture."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+__metaclass__ = type
+
+from setuptools import setup
+
+
+with open("requirements.txt", "rb") as fd:
+    requirements = {line.strip() for line in fd}
+
+
+setup(
+    name='postgresfixture',
+    version="0.1",
+    packages={b'postgresfixture'},
+    package_dir={'postgresfixture': 'postgresfixture'},
+    install_requires=requirements,
+    tests_require={"testtools >= 0.9.14"},
+    test_suite="postgresfixture.tests",
+    include_package_data=True,
+    zip_safe=False,
+    description=(
+        "A fixture for creating PostgreSQL clusters and databases, and "
+        "tearing them down again, intended for use during development "
+        "and testing."),
+    entry_points={
+        "console_scripts": [
+            "postgresfixture = postgresfixture.main:main",
+            ],
+        },
+    )